vllm - 💡(How to fix) Fix [Bug]: DeepSeek-V4-Pro (fp4+fp8 mixed) outputs garbage tokens

Your current environment

<details> vllm version：0.20.2

GPU: NVIDIA H200x8

model: DeepSeek-V4-Pro fp4+fp8 mixed

docker-compose.yml boot cmd:

command: - /bin/bash - -c - | python3 -m vllm.entrypoints.openai.api_server
--model /data/models/deepseek-ai/DeepSeek-V4-Pro
--served-model-name DeepSeek-V4-Pro
--trust-remote-code
--max-model-len 393216
--max-num-batched-tokens 8196
--max-num-seqs 8
--kv-cache-dtype fp8
--tensor-parallel-size 8
--port 8003
--gpu-memory-utilization 0.98
--enable-chunked-prefill
--dtype bfloat16
--max-parallel-loading-workers 8
--enable-auto-tool-choice
--tool-call-parser hermes

</details>

🐛 Describe the bug

Frequently outputting garbage tokens that do not meet expectations, especially in front of numbers in the code, as shown in the following figure:

<img width="518" height="197" alt="Image" src="https://github.com/user-attachments/assets/61f50866-afa3-4bfd-8b51-f6bdf49fe58f" />

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.