pytorch - 💡(How to fix) Fix capture_dynamic_output_shape_ops=True crashes 8/50 HuggingFace models with GuardOnDataDependentSymNode [1 participants]

Summary

Setting torch._dynamo.config.capture_dynamic_output_shape_ops = True causes GuardOnDataDependentSymNode crashes on 17% of tested models (8/50), while providing measurable benefit (fewer graph breaks) on only 4% (2/50). All crashes occur in vision-language models with data-dependent control flow.

Environment

• PyTorch: 2.12.0.dev20260408+cu128 (nightly)
• transformers: 5.5.0
• GPU: NVIDIA PG509-210
• CUDA: 12.8

Methodology

We tested a stratified random sample of 50 models drawn from 226 HuggingFace models that have graph breaks under torch.compile. The sample was stratified across 4 architecture types to ensure representative coverage:

Each model was compiled 3 times using a lightweight counting backend:

1. Baseline: no flags (default config)
2. +capture_scalar_outputs: (tested separately, see pytorch/pytorch#180595)
3. +capture_dynamic_output_shape_ops: torch._dynamo.config.capture_dynamic_output_shape_ops = True

We measured subgraph count per run and classified each flag's effect as: reduces_graphs, no_effect, increases_graphs, crashes, fixes_crash, both_crash, or create_error.

Why only graph_break models? Models that already pass fullgraph=True have no graph breaks, and fullgraph=True implicitly captures dynamic output shape ops. These flags only matter for models that graph-break.

Results

*Testable = excluding create_error and both_crash (48 models)

Models where capture_dynamic_output_shape_ops helps (2 models)

When it works, the benefit is substantial — reducing graph breaks by 6-7 per model.

Models where capture_dynamic_output_shape_ops crashes (8 models)

All crashes produce the same error: GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 0)

Pattern: All 8 models are VL/multimodal models. The consistent Eq(u0, 0) guard failure suggests these models have conditional branches on dynamic shape dimensions that the tracer cannot resolve.

Overlap with capture_scalar_outputs crashes

6 of the 8 CDO-crashing models also crash with capture_scalar_outputs (different error: PendingUnbackedSymbolNotFound, see pytorch/pytorch#180595). 2 models crash only with CDO:

• Qwen3VLForConditionalGeneration (CDO-only crash)
• Qwen3OmniMoeThinkerForConditionalGeneration (CDO-only crash; CSO increases graphs by 1)

Reproduction

import torch
import torch._dynamo

torch._dynamo.config.capture_dynamic_output_shape_ops = True

# Example: Qwen3VLForConditionalGeneration
from transformers import AutoModelForConditionalGeneration
model = AutoModelForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-3B", trust_remote_code=True)
model = model.eval().cuda()

compiled = torch.compile(model, backend="eager")
with torch.no_grad():
    compiled(**inputs)  # Crashes with GuardOnDataDependentSymNode

Analysis

The crash-to-benefit ratio is 4:1 (8 crashes vs 2 benefits), worse than capture_scalar_outputs. However, when CDO does work, the benefit is much larger — reducing graph breaks by 6-7 per model vs 1-2 for CSO.

Key observations:

• All 8 crashes share the exact same error pattern (Eq(u0, 0) guard), suggesting a single fix could resolve all cases
• All crashes are in VL/multimodal models — decoder-only, encoder-decoder, and vision models are unaffected
• The 2 models where CDO helps (Phi4MultimodalAudio, Aria) are also multimodal, but with simpler dynamic shape patterns
• CDO and CSO crash independently with different error types — these are separate code paths

Recommendation: If the Eq(u0, 0) guard issue is resolved, this flag could eliminate significant graph breaks in multimodal models. The benefit magnitude (6-7 fewer breaks when it works) justifies fixing the crashes.

Data

Results from the OSS Model Graph Break Corpus — a systematic catalog of torch.compile graph breaks across 716 HuggingFace models.

cc @chauhang @ezyang @bobrenjc93 @aditvenk @laithsakka @williamwen42 @jansel