vllm - 💡(How to fix) Fix [Bug]: [V1] `prompt_tokens_details` always `null` in API response despite `--enable-prompt-tokens-details` (14+ months, incomplete fix in PR #18149)

Your current environment

• vLLM version: v0.22.1rc1.dev166+gd0975a4b5 (V1 engine, default)
• Launch flags: --enable-prefix-caching --enable-prompt-tokens-details
• Also confirmed broken on: v0.22.1rc1 nightly (2026-06-01) per forum report

🐛 Describe the bug

Summary

usage.prompt_tokens_details is always null in the OpenAI-compatible API response on V1 engine, even with --enable-prompt-tokens-details and --enable-prefix-caching enabled.

This was previously reported in #16162 (2025-04) and #18062 (2025-05), and supposedly fixed by PR #18149 (merged 2025-05-23). However, the fix is incomplete — it only wired num_cached_tokens through the scheduler/output processor, but never mapped it to prompt_tokens_details in the OpenAI API serving layer.

This bug has been open for 14+ months and is confirmed still broken on the latest nightly.

Prior reports confirming this bug

Root cause

PR #18149 modified the engine-internal path:

vllm/v1/core/sched/scheduler.py      ✅ num_cached_tokens populated
vllm/v1/engine/__init__.py            ✅ propagated in EngineCoreOutput
vllm/v1/engine/output_processor.py    ✅ stored in RequestOutput
vllm/v1/request.py                    ✅ field added

But the API server response path was never updated:

[OutputProcessor] num_cached_tokens = N   ✅
    ↓
[OpenAI Serving Layer → UsageInfo]        ❌ not mapped
    ↓
[API JSON Response]                       ❌ prompt_tokens_details: null

The OpenAI serving layer (e.g. vllm/entrypoints/openai/serving_chat.py or the UsageInfo builder) does not read num_cached_tokens from RequestOutput and does not populate prompt_tokens_details.cached_tokens.

Reproduction

Server:

vllm serve facebook/opt-125m \
  --enable-prefix-caching \
  --enable-prompt-tokens-details

Client — send the same request twice (2nd should cache-hit):

# Run this twice
curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "facebook/opt-125m",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant. Please provide detailed and comprehensive answers to all questions. This is padding to ensure enough tokens for prefix caching to activate on the second request."},
      {"role": "user", "content": "What is the capital of France?"}
    ],
    "max_tokens": 20
  }' | jq '.usage'

Actual (all V1 versions):

{
  "prompt_tokens": 48,
  "total_tokens": 68,
  "completion_tokens": 20,
  "prompt_tokens_details": null
}

Expected (2nd request):

{
  "prompt_tokens": 48,
  "total_tokens": 68,
  "completion_tokens": 20,
  "prompt_tokens_details": {
    "cached_tokens": 32
  }
}

Note: /metrics confirms prefix caching is working internally — only the API response field is missing:

vllm:prefix_cache_hits_total > 0

V0 engine also returns prompt_tokens_details correctly (VLLM_USE_V1=0), confirming the gap is V1-specific.

Suggested fix

Map RequestOutput.num_cached_tokens → UsageInfo.prompt_tokens_details in the OpenAI serving layer:

# Likely location: vllm/entrypoints/openai/serving_chat.py
# or wherever UsageInfo is constructed from RequestOutput

if self.enable_prompt_tokens_details and request_output.num_cached_tokens is not None:
    usage.prompt_tokens_details = PromptTokensDetails(
        cached_tokens=request_output.num_cached_tokens
    )

Impact

• Observability: No per-request cache efficiency tracking via standard API
• Proxy/cost tracking: LiteLLM and similar proxies rely on prompt_tokens_details.cached_tokens — null breaks cost attribution
• OpenAI API compatibility: Violates the contract vLLM aims to fulfill
• Duration: 14+ months since first report, all V1 versions affected

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.