pytorch - 💡(How to fix) Fix capture_scalar_outputs=True crashes 7/50 HuggingFace models with PendingUnbackedSymbolNotFound [1 participants]

Summary

Setting torch._dynamo.config.capture_scalar_outputs = True causes PendingUnbackedSymbolNotFound crashes on 14% of tested models (7/50), while providing measurable benefit (fewer graph breaks) on only 6% (3/50). The crashes are concentrated in vision-language and audio models with complex data-dependent shapes.

Environment

• PyTorch: 2.12.0.dev20260408+cu128 (nightly)
• transformers: 5.5.0
• GPU: NVIDIA PG509-210
• CUDA: 12.8

Methodology

We tested a stratified random sample of 50 models drawn from 226 HuggingFace models that have graph breaks under torch.compile. The sample was stratified across 4 architecture types to ensure representative coverage:

Each model was compiled 3 times using a lightweight counting backend:

1. Baseline: no flags (default config)
2. +capture_scalar_outputs: torch._dynamo.config.capture_scalar_outputs = True
3. +capture_dynamic_output_shape_ops: (tested separately, reported in a separate issue)

We measured subgraph count per run and classified each flag's effect as: reduces_graphs, no_effect, increases_graphs, crashes, fixes_crash, both_crash, or create_error.

Why only graph_break models? Models that already pass fullgraph=True have no graph breaks, and fullgraph=True implicitly captures scalar outputs. These flags only matter for models that graph-break.

Results

*Testable = excluding create_error and both_crash (48 models)

Models where capture_scalar_outputs helps (3 models)

Models where capture_scalar_outputs crashes (7 models)

All crashes produce the same error: PendingUnbackedSymbolNotFound

Pattern: All affected models are vision-language (Glm/Ernie family, VideoLlama3) or audio (Mimi) models with complex data-dependent tensor shapes. The common thread is dynamic shapes involving multiple unbacked symbols.

Model where capture_scalar_outputs increases graph count (1 model)

Reproduction

import torch
import torch._dynamo

torch._dynamo.config.capture_scalar_outputs = True

# Example: GlmOcrVisionModel (smallest/fastest to reproduce)
from transformers import AutoModel
model = AutoModel.from_pretrained("THUDM/glm-4-9b-hf", trust_remote_code=True)
model = model.eval().cuda()

compiled = torch.compile(model, backend="eager")
with torch.no_grad():
    compiled(**inputs)  # Crashes with PendingUnbackedSymbolNotFound

Analysis

The crash-to-benefit ratio is concerning for enabling this flag by default:

• 2.3:1 crash-to-benefit ratio (7 crashes vs 3 benefits)
• Benefits are modest: 1-2 fewer graph breaks per model
• Crashes are total failures: the model cannot be compiled at all with this flag

The crashes all share the same root cause (PendingUnbackedSymbolNotFound), suggesting a single fix in the unbacked symbol tracking could resolve all 7 cases.

Data

Results from the OSS Model Graph Break Corpus — a systematic catalog of torch.compile graph breaks across 716 HuggingFace models.

cc @chauhang @ezyang @bobrenjc93 @aditvenk @laithsakka @williamwen42 @jansel