claude-code - 💡(How to fix) Fix output_config.format truncates long-form text fields due to FSM renormalization bias [1 participants]

Description

When using output_config.format for grammar-constrained JSON output, long-form text fields (500-2000 characters) are truncated approximately 40% of the time. The output is valid JSON that conforms to the schema, but string content is cut short mid-sentence.

Reproduction

• Model: Sonnet 4.6
• Schema: JSON object with multiple string fields, some expected to contain 500-2000 characters of prose
• System prompt: ~80K tokens
• Observed: ~40% of outputs had at least one text field truncated
• Control: Switching to unconstrained text generation (raw JSON) with the identical prompt produced 0 truncations across 24 outputs over 8 runs

Hypothesis

The finite state machine enforcing the grammar renormalizes token probabilities away from continuing long strings, creating a bias toward closing the string early and moving on to the next field. The effect is more pronounced with longer target strings and larger overall prompts.

Expected behavior

Grammar-constrained output should not systematically truncate content that the model produces correctly in unconstrained mode. If grammar enforcement introduces a length bias, this should be documented as a known limitation.

Workaround

Use unconstrained text generation as the primary output mode and parse with a lenient JSON parser. Fall back to grammar-constrained output only when parsing fails. Optionally run a repair pass to detect and fix truncation from the fallback path.