vllm - 💡(How to fix) Fix [RFC]: Custom graph dumping for vLLM-compile and depyf deprecation [1 comments, 2 participants]

Motivation.

Currently, vLLM uses depyf for graph dumping and introspection. However:

• depyf is becoming a maintenance burden
• tlparse (an alternative) is clunky, time-consuming and designed for PyTorch, not vLLM use cases
• We need an extremely lightweight (single <300LOC PR), vLLM-aware graph dumping system

The goal is that any vLLM developer not even familiar with torch.compile can set VLLM_DEBUG_DUMP_PATH and easily navigate the produced graph dumps.

Proposed Change.

Implement a mini-system that:

1. Hooks into lazy_format_graph_code to capture graphs at every (relevant) stage
   • Not sure if we need something different for output code
2. Dumps to VLLM_DEBUG_DUMP_PATH/CompilationConfig.debug_dump_path, with easily navigable structure
3. Preserves vLLM compilation context, including:
   • pass_context properties (e.g., compile ranges, donated inputs)
   • vllm config and model name associated with the dump
   • Any relevant vllm_ir metadata (like what provider will be selected in pre-lowered graphs)
   • Captures function names for torch.compile on smaller functions
4. Exports both representations:
   • Structured traces nesting by layer: #39215
   • Unstructured traces: raw fx.Graph dump

We can finally deprecate depyf once this is implemented.

Feedback Period.

1 week, 4/29 - 5/6

CC List.

@zou3519 @youkaichao @benchislett

Any Other Things.

This should not take much core committer bandwidth as I think it would be easyu to implement for a community contributor.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.