PR #38989: [Bug] Fix routing bias dtype for trtllm per-block fp8 moe

• Repository: vllm-project/vllm
• Author: wzhao18
• State: closed | merged: True
• Link: https://github.com/vllm-project/vllm/pull/38989

Description (problem / solution / changelog)

Purpose

Fix [Bug]: Deepseek R1 produces incorrect output FIX https://github.com/vllm-project/vllm/issues/39179 Flashinfer v0.6.7 requires bf16 routing bias dtype for trtllm MoE. We have done this in https://github.com/vllm-project/vllm/pull/38423 for nvfp4 and fp8 per-tensor, but haven't done for fp8 per-block.

Test Plan

Test Result

main:

vllm serve deepseek-ai/DeepSeek-R1 --trust-remote-code --tensor-parallel-size 8

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.0197|±  |0.0038|
|     |       |strict-match    |     5|exact_match|↑  |0.0000|±  |0.0000|

branch:

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9598|±  |0.0054|
|     |       |strict-match    |     5|exact_match|↑  |0.9591|±  |0.0055|

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Changed files

• vllm/model_executor/layers/fused_moe/experts/trtllm_fp8_moe.py (modified, +5/-0)

PR #39717: [Bugfix] Reject non-nvfp4 dtypes when using the flashinfer_nvlink_one_sided all2all backend

• Repository: vllm-project/vllm
• Author: tlrmchlsmth
• State: closed | merged: True
• Link: https://github.com/vllm-project/vllm/pull/39717

Description (problem / solution / changelog)

The flashinfer_nvlink_one_sided all2all backend hardcodes its workspace sizing for nvfp4 payloads (4-bit activations + fp8 block scales).

See the hardcoding to datatype here: https://github.com/vllm-project/vllm/blob/4d042ed85f3b8fe8c73b6a4a365e65cf1811823e/vllm/distributed/device_communicators/all2all.py#L720-L725

When used with bf16 or other dtypes, the undersized workspace causes silent data corruption and gibberish output. Add an explicit dtype check at backend selection time so unsupported configurations fail loudly with an actionable error message.

Changed files

• vllm/model_executor/layers/fused_moe/all2all_utils.py (modified, +8/-0)