PR #78183: fix(transport): preserve JSON error body on non-OK SSE responses

• Repository: openclaw/openclaw
• Author: CaoYuhaoCarl
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/78183

Description (problem / solution / changelog)

Summary

• Problem: Provider HTTP errors with Content-Type: text/event-stream (e.g. Google Gemini's streamGenerateContent) bubble up with their JSON error body silently stripped, so createProviderHttpError only surfaces the bare status code.
• Why it matters: Operators see Google Generative AI API error (400) in logs and have no way to learn the actual upstream complaint (INVALID_ARGUMENT, schema field rejected, function-call ordering, etc.) without bypassing the runtime. Failover and triage fly blind.
• What changed: sanitizeOpenAISdkSseResponse now returns the original response when !response.ok, so the existing error-detail extractor sees the JSON body intact. Plus a regression test in src/agents/provider-transport-fetch.test.ts.
• What did NOT change (scope boundary): Successful (response.ok) SSE streams still pass through the sanitizer; the keepalive-frame stripping for the OpenAI SDK's parser is unchanged. No other transports, no plugin, no public surface.

Change Type (select all)

• Bug fix
• Feature
• Refactor required for the fix
• Docs
• Security hardening
• Chore/infra

Scope (select all touched areas)

• Gateway / orchestration
• Skills / tool execution
• Auth / tokens
• Memory / storage
• Integrations
• API / contracts
• UI / DX
• CI/CD / infra

Linked Issue/PR

• Closes #78180
• Related #
• This PR fixes a bug or regression

Real behavior proof (required for external PRs)

External contributors must show after-fix evidence from a real OpenClaw setup.

• Behavior or issue addressed: Google Gemini 4xx responses returned via streamGenerateContent lose their JSON { "error": { ... } } body before createProviderHttpError reads it, leaving only Google Generative AI API error (400) with no [code=...] detail.
• Real environment tested: macOS 26.4.1 (arm64), Node 25.9.0, OpenClaw npm global install at ~/.openclaw, Gemini agent on google/gemini-3.1-pro-preview over Feishu DM.
• Exact steps or command run after this patch:

  pnpm install --frozen-lockfile
  node scripts/run-vitest.mjs run --config test/vitest/vitest.agents-core.config.ts src/agents/provider-transport-fetch.test.ts

• Evidence after fix (terminal capture):

   RUN  v4.1.5 /private/tmp/oc-pr/openclaw
  
   Test Files  1 passed (1)
        Tests  23 passed (23)
     Start at  09:26:04
     Duration  463ms

  Including the new regression case preserves the JSON error body on non-OK SSE responses so providers can surface error detail.
• Observed result after fix: The wrapped Response returned from buildGuardedModelFetch for a 400 with Content-Type: text/event-stream keeps its body, so await response.text() returns the original { "error": { "code": 400, "message": "...", "status": "INVALID_ARGUMENT" } } payload that extractProviderErrorDetail needs.
• What was not tested: Live failover under sustained Gemini errors and non-Google streaming providers that emit JSON-only 4xx (e.g. some OpenAI-compatible deployments) — the fix is provider-agnostic but I have not exercised every transport.
• Before evidence: With the same test, on main at 538605ff before this patch:

   FAIL  preserves the JSON error body on non-OK SSE responses ...
  AssertionError: expected '' to be '{"error":{"code":400,"message":"GenerateContentRequest.contents: contents is not specified","status":"INVALID_ARGUMENT"}}' // Object.is equality

  Real-environment log line that triggered this investigation, from ~/.openclaw/logs/gateway.err.log:

  2026-04-29T20:42:29.278+08:00 [agent/embedded] embedded run agent end:
    runId=2cfe0ade-... isError=true model=gemini-3.1-pro-preview provider=google
    error=Google Generative AI API error (400): * GenerateContentRequest.contents: contents is not specified [code=INVALID_ARGUMENT]   ← old, with detail
  
  2026-05-05T21:17:39.591+08:00 [agent/embedded] embedded run agent end:
    runId=d4c75e90-... isError=true model=gemini-3.1-pro-preview provider=google
    error=Google Generative AI API error (400)                                                                                       ← regressed, detail gone

Root Cause (if applicable)

• Root cause: sanitizeOpenAISdkSseResponse (in src/agents/provider-transport-fetch.ts) wraps every response whose Content-Type contains text/event-stream and drops any chunk that lacks a data: prefix. Its job is to strip event-only / blank-data keepalives so the OpenAI SDK's stream parser does not JSON.parse them. Non-OK responses never go through that parser — transports throw via createProviderHttpError, which reads the raw body. Google's streamGenerateContent returns 4xx with the JSON { "error": { ... } } body but keeps Content-Type: text/event-stream, so the sanitizer drops it and the thrown error degrades to just the status code.
• Missing detection / guardrail: No existing test covered the "non-OK + text/event-stream" combination; only the happy-path keepalive-stripping cases were exercised.
• Contributing context (if known): The error-extractor extractProviderErrorDetail is shared by the Anthropic / Google / OpenAI-compat transports, so any provider that streams JSON 4xx with this content type was affected the same way. Logs from ~/.openclaw/logs/gateway.err.log show the detail string disappeared on live agents around 2026-04-30, matching the rollout window of the sanitizer.

Regression Test Plan (if applicable)

• Coverage level that should have caught this:
  • Unit test
  • Seam / integration test
  • End-to-end test
  • Existing coverage already sufficient
• Target test or file: src/agents/provider-transport-fetch.test.ts, new case preserves the JSON error body on non-OK SSE responses so providers can surface error detail.
• Scenario the test should lock in: Given a fetchWithSsrFGuard mock that returns status: 400, content-type: text/event-stream, body: <JSON>, the response returned by buildGuardedModelFetch(...) must keep response.ok === false, response.status === 400, and await response.text() equal to the original JSON body.
• Why this is the smallest reliable guardrail: It exercises the exact seam where the bug hides (sanitizeOpenAISdkSseResponse) without spinning up any transport, runtime, or provider, and matches the structure of the existing OK-path SSE tests in the same file.
• Existing test that already covers this (if any): None — the existing SSE tests in this file all use 200 responses, so the response.ok branch was never tested.
• If no new test is added, why not: N/A.

User-visible / Behavior Changes

None for the happy path. For failed requests on streaming endpoints, thrown errors regain their full provider detail (e.g. Google Generative AI API error (400): GenerateContentRequest.contents: contents is not specified [code=INVALID_ARGUMENT]).

Diagram (if applicable)

Before:
provider 4xx (content-type: text/event-stream, body: JSON error)
  -> sanitizeOpenAISdkSseResponse wraps body
     -> drops every chunk without `data:` prefix (JSON has none)
        -> response.body becomes empty
           -> createProviderHttpError reads "" -> throws "(400)" with no detail

After:
provider 4xx (content-type: text/event-stream, body: JSON error)
  -> sanitizeOpenAISdkSseResponse sees !response.ok -> returns original response
     -> createProviderHttpError reads JSON body
        -> throws "(400): <message> [code=<status>]"

Security Impact (required)

• New permissions/capabilities? No
• Secrets/tokens handling changed? No
• New/changed network calls? No
• Command/tool execution surface changed? No
• Data access scope changed? No
• If any Yes, explain risk + mitigation: N/A.

Repro + Verification

Environment

• OS: macOS 26.4.1 (arm64)
• Runtime/container: Node v25.9.0, pnpm 10.33.2
• Model/provider: google/gemini-3.1-pro-preview via https://generativelanguage.googleapis.com/v1beta/.../streamGenerateContent
• Integration/channel (if any): Feishu DM (incidental — happens on any channel)
• Relevant config (redacted): ~/.openclaw/openclaw.json agent with model.primary = "google/gemini-3.1-pro-preview"; default Google route, no proxy.

Steps

1. Start any agent whose primary is a Gemini model on the v1beta streaming endpoint.
2. Trigger a 4xx (e.g. send a turn whose history contains a stale function_call block, or temporarily set tools to one with an unsupported field like additionalProperties).
3. Observe gateway.err.log.

Expected

error=Google Generative AI API error (400): <message> [code=INVALID_ARGUMENT]

Actual (before this PR)

error=Google Generative AI API error (400) — no detail.

Actual (after this PR)

Full detail string is restored. Verified via the new unit test (terminal output above) and via a direct curl against the same Gemini endpoint, which shows that Gemini does return a JSON body with Content-Type: text/event-stream for 4xx:

$ curl -i -X POST 'https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-pro-preview:streamGenerateContent?alt=sse&key=…' \
    -H 'Content-Type: application/json' \
    -d '{"contents":[],"systemInstruction":{"parts":[{"text":"x"}]}}'

HTTP/2 400
content-type: text/event-stream

{
  "error": {
    "code": 400,
    "message": "* GenerateContentRequest.contents: contents is not specified\n",
    "status": "INVALID_ARGUMENT"
  }
}

Evidence

• Failing test/log before + passing after (see "Real behavior proof" section)
• Trace/log snippets (gateway.err.log lines from a real install)
• Screenshot/recording
• Perf numbers (if relevant)

Human Verification (required)

• Verified scenarios:
  1. New unit test in provider-transport-fetch.test.ts fails on main (assertion: empty body) and passes after this patch (full JSON body preserved).
  2. Full agents-core Vitest scope (273 files / 3519 tests) passes locally with this patch applied.
  3. pnpm tsgo:core clean. oxlint and oxfmt --check clean for both touched files.
  4. Direct curl against Gemini v1beta streamGenerateContent confirms the upstream actually returns a JSON error body with Content-Type: text/event-stream (i.e. the regression precondition is real, not a synthetic test fixture).
• Edge cases checked: Mixed body (e.g. real data: lines plus JSON error) cannot occur on the error path (Gemini either streams success frames or returns a JSON error body, never both); successful streaming responses still go through the sanitizer untouched, verified by the existing two text/event-stream happy-path tests still passing.
• What you did not verify: Production failover behavior under sustained Gemini error rates; non-Google streaming providers that may emit JSON 4xx with text/event-stream (Anthropic/OpenAI typically emit application/json for 4xx, so they should not have been affected, but I have not exercised every transport).

Review Conversations

• I replied to or resolved every bot review conversation I addressed in this PR.
• I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

• Backward compatible? Yes
• Config/env changes? No
• Migration needed? No
• If yes, exact upgrade steps: N/A.

Risks and Mitigations

• Risk: A future provider could ship malformed SSE error frames (mixed JSON + data: lines) where consumers had relied on the sanitizer normalizing them.
  • Mitigation: Out of scope for this PR — the error path explicitly does not parse SSE; if such a provider emerges, its transport-specific error reader can normalize the body itself. The sanitizer's contract is now tightly scoped to OK responses, which matches the comment that already explained its purpose.

Built with Claude Code (AI-assisted). The fix and tests were authored in collaboration with the model; I reviewed and ran the verification listed above on my own machine.

Changed files

• src/agents/provider-transport-fetch.test.ts (modified, +40/-0)
• src/agents/provider-transport-fetch.ts (modified, +6/-1)