codex - 💡(How to fix) Fix Include per-API-call token usage (`last`) in turn.completed JSONL events [2 comments, 2 participants]

TL;DR

Modify the event_processor_with_jsonl_output.rs to include ThreadTokenUsage.last in the turn.completed JSONL event to accurately track context window utilization.

Guidance

• Review the usage_from_last_total function in event_processor_with_jsonl_output.rs to understand how the total field is calculated and how ThreadTokenUsage.last can be incorporated.
• Update the turn.completed event to include a new field, such as last_usage, that contains the token usage of the most recent request.
• Verify that the updated event output includes the expected last_usage field with accurate token usage data.
• Test the modified codex exec output to ensure it accurately tracks context window utilization.

Example

{"type":"turn.completed","usage":{...},"last_usage":{"input_tokens":248301,...}}

This example shows the proposed update to the turn.completed event, including the new last_usage field.

Notes

This solution assumes that the ThreadTokenUsage.last data is already available and accurate. If this data is not reliable, additional work may be needed to ensure its accuracy.

Recommendation

Apply workaround: Modify the event_processor_with_jsonl_output.rs to include ThreadTokenUsage.last in the turn.completed JSONL event, as this will bring the codex exec output to parity with the interactive experience for context window tracking.