vllm - 💡(How to fix) Fix [Bug]: Local prefix cache hit rate can be over-counted when scheduling retries after KV allocation failure [2 pull requests]

Your current environment

Your output of `python collect_env.py` here

🐛 Describe the bug

I observed that in some workloads the reported local prefix cache hit rate can become unexpectedly high.

After checking the scheduler path, it looks like local prefix cache stats may be recorded before the request is actually scheduled. In KVCacheManager.get_computed_blocks(), prefix cache stats are updated immediately after a local prefix cache lookup. However, the scheduler may later fail to allocate KV slots for that request, leaving it in the waiting queue.

If the same request is retried in a later scheduling step, it can perform the same local prefix cache lookup again and be counted again, even though it was not scheduled in the previous attempt.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.