claude-code - 💡(How to fix) Fix Intermittent Korean (CJK) character corruption in model output [1 comments, 2 participants]

TL;DR

Verify and ensure proper UTF-8 encoding handling in the Claude Code model output and subsequent tool calls to prevent character corruption.

Guidance

• Investigate the encoding settings in the Claude Code CLI and the model configuration to ensure UTF-8 is consistently used.
• Check the tool call (curl/API push) implementation to verify that it properly handles UTF-8 encoded strings, potentially setting explicit encoding parameters.
• Consider adding logging or debugging statements to capture and inspect the output immediately after generation by the model and before it is passed to external tools, to identify where the corruption occurs.
• Test the system with a controlled, repeated input that previously caused corruption to see if the issue can be reliably reproduced and if any patterns emerge.

Example

No specific code example can be provided without more details on the implementation, but ensuring that all components (the model, the CLI, and the tool calls) are configured to work with UTF-8 encoding is crucial.

Notes

The intermittent nature and the specific corruption of Korean characters suggest an encoding issue that might be related to how the model or subsequent processing handles Unicode characters. Without more specific details on the implementation or the ability to reproduce the issue reliably, the guidance focuses on verifying encoding consistency across the system.

Recommendation

Apply workaround: Ensure all system components explicitly use UTF-8 encoding to handle character data, as this is likely to mitigate the corruption issue until a more permanent fix can be identified and applied.