Issue Station
Sign In
Tools hereToolbox

transformers - ✅(Solved) Fix D-FINE not using any auxiliary losses when denoising is turned off [1 pull requests, 2 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
huggingface/transformers#45593•Fetched 2026-04-24 05:51:24
View on GitHub
Comments
2
Participants
2
Timeline
13
Reactions
1
Author
m-matthiasm-matthias
Participants
AbineshabeeAbineshabeem-matthiasm-matthias
Timeline (top)
mentioned ×4subscribed ×4commented ×2closed ×1

Fix Action

Fixed

PR fix notes

PR #45601: fix: compute auxiliary losses when denoising is disabled in D-FINE

Description (problem / solution / changelog)

Fixes #45593

Problem

When num_denoising=0, denoising_meta_values is None, causing DFineForObjectDetectionLoss to skip auxiliary loss computation entirely since it is gated under the same condition.

However, auxiliary losses (deep supervision across decoder layers) are independent of denoising and should still be computed when config.auxiliary_loss=True. This leads to degraded training behavior when denoising is disabled.

Fix

Decoupled denoising logic from auxiliary loss computation:

  • Tensor splitting is performed only when denoising_meta_values is not None
  • Auxiliary losses are always computed when config.auxiliary_loss=True
  • Denoising-specific auxiliary losses are computed only when denoising is active

This follows the same pattern used in loss_rt_detr.py.

Testing

  • Verified that auxiliary loss keys (e.g., loss_vfl_aux_*, loss_bbox_aux_*) are present when num_denoising=0
  • Confirmed no change in behavior when denoising is enabled

Changed files

  • src/transformers/loss/loss_d_fine.py (modified, +22/-16)
  • tests/models/d_fine/test_modeling_d_fine.py (modified, +43/-0)

Code Example

model = DFineForObjectDetection.from_pretrained("ustc-community/dfine-medium-obj365", num_denoising=0)
RAW_BUFFERClick to expand / collapse

System Info

  • transformers version: 5.7.0.dev0
  • Platform: Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.39
  • Python version: 3.12.3
  • Huggingface_hub version: 1.7.1
  • Safetensors version: 0.7.0
  • Accelerate version: 1.13.0
  • Accelerate config: not found
  • DeepSpeed version: not installed
  • PyTorch version (accelerator?): 2.8.0+cu128 (CUDA)
  • GPU type: NVIDIA RTX 500 Ada Generation Laptop GPU

Who can help?

@VladOS95-cyber @yonigozlan

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

Turn off denoising, by e.g.:

model = DFineForObjectDetection.from_pretrained("ustc-community/dfine-medium-obj365", num_denoising=0)

Expected behavior

Only the denoising losses should be turned off. However, DfineForObjectDetectionLoss only checks for denoising_meta_values to either use all or no auxiliary losses at all. See here: https://github.com/huggingface/transformers/blob/main/src/transformers/loss/loss_d_fine.py#L339

RT-DETR for example handles this in a different way and keeps the other losses even when denoising is off: https://github.com/huggingface/transformers/blob/main/src/transformers/loss/loss_rt_detr.py#L454

I found that denoising can lead to degraded validation performance with complex datasets, and turning it off helps. But without the other losses, D-FINE does not train properly.

extent analysis

TL;DR

Modify the DfineForObjectDetectionLoss to conditionally include auxiliary losses when denoising is turned off.

Guidance

  • Review the DfineForObjectDetectionLoss implementation and compare it with RT-DETR's loss handling to understand the difference in auxiliary loss handling.
  • Consider modifying the DfineForObjectDetectionLoss to include a conditional check for denoising status, similar to RT-DETR, to allow for auxiliary losses when denoising is off.
  • Investigate the impact of denoising on validation performance with complex datasets to determine the optimal approach for your specific use case.
  • Verify that the modified loss function improves training performance and validation metrics.

Example

# Example of conditional auxiliary loss handling in DfineForObjectDetectionLoss
if denoising_meta_values is not None and num_denoising == 0:
    # Include auxiliary losses when denoising is off
    auxiliary_losses = ...
else:
    # Handle denoising losses as before
    auxiliary_losses = ...

Notes

The provided information suggests that the issue is specific to the DfineForObjectDetectionLoss implementation and its handling of auxiliary losses when denoising is turned off. The comparison with RT-DETR's loss handling provides a potential solution path.

Recommendation

Apply workaround: Modify the DfineForObjectDetectionLoss to conditionally include auxiliary losses when denoising is turned off, as inspired by the RT-DETR implementation. This should allow for proper training and improved validation performance with complex datasets.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

Only the denoising losses should be turned off. However, DfineForObjectDetectionLoss only checks for denoising_meta_values to either use all or no auxiliary losses at all. See here: https://github.com/huggingface/transformers/blob/main/src/transformers/loss/loss_d_fine.py#L339

RT-DETR for example handles this in a different way and keeps the other losses even when denoising is off: https://github.com/huggingface/transformers/blob/main/src/transformers/loss/loss_rt_detr.py#L454

I found that denoising can lead to degraded validation performance with complex datasets, and turning it off helps. But without the other losses, D-FINE does not train properly.

#serialization error#model compatibility#GPU setup#container setup#orchestration issue

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

transformers - ✅(Solved) Fix D-FINE not using any auxiliary losses when denoising is turned off [1 pull requests, 2 comments, 2 participants]