codex - 💡(How to fix) Fix Official codex-cli 0.120.0 sends raw multibyte JSON in x-codex-turn-metadata, breaking HTTP interoperability [4 comments, 3 participants]

What version of Codex CLI is running?

codex-cli 0.120.0

What subscription do you have?

ChatGPT Plus

Which model were you using?

gpt-5.1-codex-mini

What platform is your computer?

Microsoft Windows NT 10.0.26200.0 x64

What terminal emulator and version are you using (if applicable)?

PowerShell 7.6.0

What issue are you seeing?

x-codex-turn-metadata is emitted by the official CLI as raw JSON rather than an ASCII-safe encoding. When that metadata contains non-ASCII characters, this breaks HTTP interoperability.

I gathered three independent repros:

1. Standard client stack repro: Python urllib.request fails before send with: UnicodeEncodeError: 'latin-1' codec can't encode characters... when the same logical turn metadata JSON is placed directly in x-codex-turn-metadata.
2. Hosted API repro: Node fetch against Azure OpenAI fails before send for the raw header with: TypeError: Cannot convert argument to a ByteString because the character at index 31 has a value of 26481 which is greater than 255. The same logical metadata succeeds when Base64-encoded. Against: GET https://example-resource.openai.azure.com/openai/v1/models the Base64 version returned 200 OK.
3. Official CLI repro: I ran the official npm-installed codex-cli 0.120.0 from a Git repo whose absolute path contains Japanese characters:

C:\Users\<redacted>\AppData\Local\Temp\codex-turn-meta-repro-東京\repo

and pointed it at a local mock /v1/responses endpoint.

The mock server captured this request header:

{
  "path": "/v1/responses",
  "turn_metadata": "{\"session_id\":\"019d...\",\"turn_id\":\"019d...\",\"workspaces\":{\"C:\\\\Users\\\\<redacted>\\\\AppData\
\\\Local\\\\Temp\\\\codex-turn-meta-repro-\u00e6\u009d\u00b1\u00e4\u00ba\u00ac\\\\repo\":{\"latest_git_commit_hash\":
\"ab1e986b4d8d86bdf5b69f6c614e8840789b6457\",\"has_changes\":false}},\"sandbox\":\"none\"}"
}

That value is not Base64. It is raw JSON placed directly in the header, and the Japanese path appears as mojibake bytes in the captured header value instead of being transport-safe.

I also compared the official CLI with a locally built binary from my fork of Codex:

• repo: https://github.com/amabile4/codex
• branch: azure/release-0.120.0

That fork only changes the wire encoding of x-codex-turn-metadata from raw JSON to Base64. It does not change the logical metadata payload.

Under the same machine / same repo / same prompt / same local mock server:

• the official codex-cli 0.120.0 sends raw JSON in x-codex-turn-metadata
• the fork build sends Base64 in x-codex-turn-metadata

The mock server decodes the fork header back to the same logical JSON, including the original Japanese workspace path. So the fork is not changing the meaning of the metadata, only making the transport encoding safe.

What steps can reproduce the bug?

1. Confirm the official CLI version:

codex --version

1. Create a Git repo whose absolute path contains non-ASCII characters, for example Japanese:

$root = Join-Path $env:TEMP 'codex-turn-meta-repro-東京'
$repo = Join-Path $root 'repo'
New-Item -ItemType Directory -Force -Path $repo | Out-Null
Set-Location $repo
git init
Set-Content README.md 'repro'
git add README.md
git -c user.name='repro' -c user.email='[email protected]' commit -m 'init'

1. Start a local mock HTTP server that logs request headers for /v1/responses.
2. Run the official CLI against that local mock provider:

codex exec --json `
  -C $repo `
  -c model_provider='mock' `
  -c 'model_providers.mock.name="Mock"' `
  -c 'model_providers.mock.base_url="http://127.0.0.1:8012/v1"' `
  -c 'model_providers.mock.wire_api="responses"' `
  -m 'gpt-5.1-codex-mini' `
  'Reply with exactly OK. Do not use any tools.'

1. Inspect the mock server log. The official CLI sends raw JSON in x-codex-turn-metadata, including the non-ASCII workspace path.
2. As an interoperability check, send the same logical metadata using a minimal repro client:
   • raw JSON in x-codex-turn-metadata
   • Base64-encoded JSON in x-codex-turn-metadata
3. Observe:
   • raw fails in standard client stacks
   • Base64 succeeds
4. Optional A/B confirmation: run a locally built binary from amabile4/codex branch azure/release-0.120.0, where the only relevant change is Base64 encoding of x-codex-turn-metadata. Under the same repo and same mock server, the fork build sends Base64 while the official CLI sends raw JSON.

What is the expected behavior?

x-codex-turn-metadata should be encoded in an ASCII-safe form before it is placed on the wire.

A Base64-encoded JSON blob is sufficient. The official CLI should not emit raw multibyte JSON header values that can:

• fail in standard HTTP clients before the request is sent
• break reconnect or request handling on some platforms
• fail against providers or gateways that enforce ASCII-safe header handling

Additional information

• Confirmed examples currently gathered:
  • Python urllib.request: raw fails, Base64 succeeds
  • Node fetch -> Azure OpenAI GET /openai/v1/models: raw fails before send, Base64 succeeds with 200 OK
  • Official codex-cli 0.120.0 -> local mock /v1/responses: raw JSON is actually emitted in x-codex-turn-metadata
  • Local fork build from amabile4/codex branch azure/release-0.120.0 -> local mock /v1/responses: Base64 is emitted instead and decodes cleanly back to the same JSON
• This is not only an Azure-specific complaint. Azure is one impacted environment, but the underlying bug is that the header value is not transport-safe across multiple HTTP stacks.