openclaw - 💡(How to fix) Fix Bug: raw image base64 in replay can poison session via invalid Responses image_url [1 pull requests]

Summary

A session can become permanently wedged if historical tool output contains raw image bytes/base64 that later get replayed or transformed into an OpenAI Responses image_url part without being a valid image URL.

In the observed failure, a tool command printed raw JPEG base64 text. On later turns, session replay produced OpenAI Responses payloads containing invalid function_call_output.output[*].image_url entries. The API rejected every follow-up request with a 400, so the user could not recover the session by simply sending another message.

This is not just a malformed single request. Once the bad content is in replay history, and especially once a persistent Codex app-server thread is created from it, the same bad payload can be replayed repeatedly and keep the session stuck.

Observed error

The API failure looked like:

invalid_request_error / invalid_value
Invalid 'input[...].output[...].image_url'. Expected a base64-encoded data URL with an image MIME type
param: input[...].output[...].image_url
status: 400

The exact index varied across attempts depending on which replay path built the final request.

Cause

The immediate trigger was raw JPEG base64 being present in a tool result as text.

The deeper bugs were:

1. Some replay/transport paths allowed invalid image-like strings to become image_url entries.
2. Some paths only sanitized content when model metadata indicated OpenAI Responses, but Responses-shaped payloads can reach the transport without that metadata.
3. The Codex app-server path can read mirrored session history and construct/send its own request, bypassing OpenClaw transport sanitizers.
4. A persistent Codex app-server thread binding can keep resuming an already-poisoned upstream thread even after local replay/history sanitization is fixed.

That combination means one bad historical tool result can poison all future turns in the session.

Effects

• The current and later user messages fail even if they contain no images.
• The assistant appears to stop responding because the model request is rejected before a response can be produced.
• Creating selected-context recovery can be safe only if the raw bad payload is omitted.
• Restarting the gateway alone does not fix it if a poisoned Codex app-server thread binding is still being resumed.
• Retrying the same session can keep failing until the bad image payload is sanitized and/or the poisoned Codex app-server binding is cleared.

Expected behavior

Malformed historical image payloads should not wedge a session.

OpenClaw should either:

• keep raw base64 as ordinary text,
• replace suspicious/invalid image payload text with a placeholder such as [omitted invalid image payload], or
• drop malformed image parts before sending provider requests.

It should never send raw JPEG base64 text as image_url.

If a persistent Codex thread is already poisoned, OpenClaw should recover by clearing/quarantining the binding rather than repeatedly resuming the bad thread.

Proposed solution

Based on a local recovery test:

1. Add final request sanitization before every OpenAI/Azure Responses responses.create call.

   • Validate any image_url before send.
   • Accept only valid data:image/<type>;base64,<payload> data URLs or explicitly supported remote URLs.
   • Reject raw base64 strings.
   • Drop malformed image content/output parts, or degrade them to text placeholders.

2. Add sanitizer coverage in embedded/pi-ai wrapper paths.

   • Run on object payloads regardless of whether model metadata says "OpenAI Responses".
   • Make it a no-op unless the payload has a Responses-shaped input array.
   • Sanitize function_call_output.output[*] and other array content fields.

3. Add Codex app-server mirrored-history sanitization.

   • Before sending mirrored session history to the Codex child, replace raw image-looking base64 text and invalid data:image/... text with a placeholder.
   • Strip invalid image_url records in mirrored history.
   • Keep diagnostics safe: log counts/paths/hashes/params, not the raw payload.

4. Clear or quarantine persistent Codex app-server thread bindings when image poison is detected.

   • If sanitization finds invalid image payloads in history, do not keep resuming a previously-created Codex app-server thread that may already contain the bad content upstream.
   • Move/rename the binding or force creation of a fresh thread from sanitized history.

Regression tests to add

• Raw JPEG-like base64 in tool output remains text or is replaced with a placeholder; it is never emitted as image_url.
• Invalid data:image/... strings are dropped/replaced before provider calls.
• function_call_output.output[*].image_url is sanitized in Responses payloads.
• Sanitizer runs even when model metadata does not explicitly identify Responses, as long as the payload shape is Responses-like.
• Codex app-server mirrored history sanitizes raw image-looking text before child thread construction.
• Detection of poisoned history clears/quarantines the Codex app-server binding so a poisoned persistent thread is not resumed forever.

Notes

The observed local recovery required both code-level sanitizers and clearing a previously poisoned Codex app-server binding. Transport-only fixes were insufficient because a Codex child path could bypass them, and history-only fixes were insufficient while an already-poisoned persistent Codex thread binding was still reused.

No raw payload is included here intentionally; diagnostics for this class of bug should avoid printing or re-injecting the poisoned base64/data URL.