vllm - 💡(How to fix) Fix [CI] test_moe_layer: all modelopt_fp4 subtests failing on 2 B200s [1 participants]

Name of failing test

tests/kernels/moe/test_moe_layer.py::test_moe_layer — all modelopt_fp4 parameterized subtests

Basic information

• Flaky test
• Can reproduce locally
• Caused by external libraries (e.g. bug in transformers)

🧪 Describe the failing test

In the 2026-04-09 nightly build (#60697), the Kernels FusedMoE Layer Test (2 B200s) step failed with 26 subtests failing. All failing subtests involve modelopt_fp4 quantization format, across three different communication backends:

• flashinfer_nvlink_two_sided — 20 failed subtests
• flashinfer_nvlink_one_sided — 20 failed subtests
• deepep_high_throughput — 12 failed subtests + 1 SIGABRT

Step link: https://buildkite.com/vllm/ci/builds/60697#019d740c-e286-4513-b9dd-7d6dc04ab473

Commit: e5de19ff9a64

Error details

All failures raise RuntimeError from _parallel_worker (line 1629) with a list of failed subtests. The deepep_high_throughput-2-1-True variant also terminated with SIGABRT.

Example failed subtest parameter sets:

[1-128-256-8-2-bfloat16-modelopt_fp4-False-False-False-False-False-flashinfer_nvlink_two_sided-1-1-2]
[1-128-256-8-2-bfloat16-modelopt_fp4-True-False-False-False-False-flashinfer_nvlink_two_sided-1-1-2]
[32-1024-512-64-6-bfloat16-modelopt_fp4-True-False-True-False-False-deepep_high_throughput-1-1-2]

Test results summary

26 failed, 163 passed, 235 skipped, 18 warnings in 586.63s

Potentially causal PRs

These PRs were merged shortly before the nightly and touch MoE/FP4 code paths:

• #39322 — [Feature] Batch invariant nvfp4 linear support (merged 2026-04-08)
• #39315 — [Bugfix] FlashInfer MXINT4 MoE crashes, missing do_finalize (merged 2026-04-09)
• #39045 — [Gemma4] Support quantized MoE (merged 2026-04-09)

Related issues

• #39503 — Similar FusedMoE Layer Test flakiness on 2 H100s (deepep_low_latency), but this is a different failure pattern (all modelopt_fp4, B200s)

Auto-generated by CI Watch Bot