claude-code - 💡(How to fix) Fix Long sessions cause dramatically increased response times — is caching/context size the culprit?

At the start of a session, responses and code generation feel fast. As the session accumulates context (long debugging sessions, many tool calls, etc.), response times slow significantly — to the point where simple reasoning or short code edits take noticeably longer.

TL;DR

Compacting the context more aggressively may help mitigate the significant slowdown in response times observed as the session accumulates context.

Guidance

• Investigate the relationship between context size and response time to determine the threshold at which slowdowns become noticeable.
• Consider implementing a strategy to automatically compact the context when it reaches a certain size or percentage of the window.
• Monitor cache hit rates and compaction frequency to understand their impact on performance.
• Experiment with different compaction intervals or thresholds to find a balance between performance and context retention.

Example

No specific code snippet is provided as the issue is more related to the usage and configuration of the model rather than a specific code implementation.

Notes

The exact threshold at which the slowdown becomes noticeable and the most effective compaction strategy may vary depending on the specific use case and model configuration. Further experimentation and diagnostics may be necessary to determine the optimal approach.

Recommendation

Apply workaround: Compact the context more aggressively, as it has been shown to reduce context size and improve response times. This approach can help mitigate the slowdown until a more permanent solution is found.