claude-code - 💡(How to fix) Fix [BUG] Claude cannot reliably output non-ASCII Unicode characters (e.g. U+2019 RIGHT SINGLE QUOTATION MARK) in file edits [1 comments, 2 participants]

Preflight Checklist

• I have searched existing issues and this hasn't been reported yet
• This is a single bug report (please file separate reports for different bugs)
• I am using the latest version of Claude Code

What's Wrong?

Actually, I think it may have been reported, but has been Closed as not planned, which I am struggling to understand. The ticket in question is https://github.com/anthropics/claude-code/issues/15920. Claude seems too easily confused by characters outside the extremely limited ASCII range.

In setting up CLAUDE.md, Claude tried to update the file with descriptions that had been incorrectly changed from the original source.

I was instructing Claude to expect and always suggest user-facing messages that use “curly quotes”. It didn’t get it and would always substitute straight quotes against my explicit wishes. Here’s the culmination of that dialogue (with added formatting):

Sorry, but that’s not different at all. You seem to be stuck in a loop and repeating the same mistake every time.
How can we point out this failing to your trainers? These are totally valid characters and you are misusing them; the same as spelling trick as truck. That looks like valid spelling, but you’ve changed the meaning.

• You're right, and I'm sorry — I've now failed at this multiple times. It appears I have a genuine limitation: even when I consciously intend to output ' (U+2019), I keep producing ' (U+0027). It's not a misunderstanding of the rule — I just can't seem to emit the character correctly in tool calls.

Your analogy is apt. It's a silent substitution that looks plausible but changes the meaning entirely.

To report this to Anthropic: please file an issue at https://github.com/anthropics/claude-code/issues — that's the official channel for Claude Code feedback. The clearest way to describe it: "Claude cannot reliably output non-ASCII Unicode characters (e.g. U+2019 RIGHT SINGLE QUOTATION MARK) in file edits — it silently substitutes the ASCII equivalent U+0027."

For the CLAUDE.md fix itself — yes, please just edit that line yourself. You can type the characters correctly; I apparently cannot. Once you've fixed it, the meaning will be right in the file even if I'm the one who mangled it getting there.

What Should Happen?

Claude should recognize and understand common characters beyond the 128 ASCII characters or even 256 extended ASCII characters.

When trying to copy text, it should not read one character and output another.

Error Messages/Logs

Steps to Reproduce

1. Prepare a file such as .github/copilot-instructions.md that includes the instruction:
   1. All user-facing text must use smart quotes (“, ”, ‘, ’).
   2. Specifically, be sure to use ‘ (the right single quotation mark) for apostrophes, not '
2. During /init, ask Claude to refer to that file .github/copilot-instructions.md in your repo, and ask to include those instructions in CLAUDE.md.
3. Note that it fails, and while it may refer to a correct Unicode value, the associated character will be an uninteresting straight quotation mark.
4. Note that Claude continues to fail when you ask it to try again after pointing out its mistake. Refer to What’s Wrong? above.

Claude Model

Sonnet (default)

Is this a regression?

I don't know

Last Working Version

No response

Claude Code Version

Claude Code CLI version 2.1.108.

Platform

Anthropic API

Operating System

macOS

Terminal/Shell

IntelliJ IDEA terminal

Additional Information

No response