PR #45551: Add ForSequenceClassification heads for the OLMo family

• Repository: huggingface/transformers
• Author: earino
• State: open | merged: False
• Link: https://github.com/huggingface/transformers/pull/45551

Description (problem / solution / changelog)

What does this PR do?

Adds ForSequenceClassification for Olmo, Olmo2, and Olmo3 so AutoModelForSequenceClassification.from_pretrained("allenai/OLMo-2-0425-1B") (and the Olmo/Olmo3 equivalents) work.

Structure follows the same pattern as Gemma2, Qwen3, and Glm4: one OlmoForSequenceClassification(LlamaForSequenceClassification): pass in modular_olmo.py, then Olmo2 and Olmo3 each subclass the previous one. After make fix-repo, the generated modeling_*.py files use the GenericForSequenceClassification mixin, same as Jamba, JetMoe, Ministral3, and Gemma3Text.

Scope is the dense chain only. OlmoHybrid and the MoE branch (Olmoe, FlexOlmo) can come as follow-up PRs.

Fixes #45529

Code Agent Policy

This was drafted with Claude Code. @Rocketknight1 explicitly opted in for this specific change on the coordination issue:

This is welcome! Sequence classification heads are often not included in the initial PR adding a new causal LM, but we're happy to add them. Your reason for needing it is good, and PRs like this are usually very easy to automate, so I'm happy for it to be mostly AI-written.

https://github.com/huggingface/transformers/issues/45529#issuecomment-4288374995

I read every changed line before each commit, ran the local test suite on my machine, and did GPU validation on real checkpoints before opening this. I can defend each change.

• I confirm that this is not a pure code agent PR.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline, Pull Request section?
• Was this discussed/approved via a Github issue or the forum? #45529
• Did you make sure to update the documentation with your changes? Added autodoc sections to docs/source/en/model_doc/olmo{,2,3}.md.
• Did you write any new necessary tests? See test plan below.

Changes

• src/transformers/models/olmo/modular_olmo.py: class OlmoForSequenceClassification(LlamaForSequenceClassification): pass
• src/transformers/models/olmo2/modular_olmo2.py: subclass of OlmoForSequenceClassification
• src/transformers/models/olmo3/modular_olmo3.py: subclass of Olmo2ForSequenceClassification
• Regenerated modeling_olmo.py, modeling_olmo2.py, modeling_olmo3.py via make fix-repo
• Registered all three in MODEL_FOR_SEQUENCE_CLASSIFICATION_MAPPING_NAMES in models/auto/modeling_auto.py
• Added autodoc sections to docs/source/en/model_doc/olmo{,2,3}.md
• Tests:
  • Olmo and Olmo2 use the older ModelTesterMixin pattern. Added the new class to all_model_classes and text-classification/zero-shot to pipeline_model_mapping.
  • Olmo3 uses the newer CausalLMModelTester. Set sequence_classification_class = Olmo3ForSequenceClassification, which auto-enables the three test_sequence_classification_model* tests from the base class.

Test plan

Local, on MPS:

• make style, make typing, make check-repo all pass
• pytest tests/models/olmo tests/models/olmo2 tests/models/olmo3: 413 passed, 0 failing. 4 test_tp_* tests deselected because they require CUDA multi-GPU.
• Olmo3's three test_sequence_classification_model* tests pass

GPU validation notebook with outputs: https://gist.github.com/earino/2bc6f246eef21a36c3c64d64150b9510

Ran on an NVIDIA RTX PRO 6000 Blackwell. For each of allenai/OLMo-1B-hf, allenai/OLMo-2-0425-1B, and allenai/Olmo-3-7B-Instruct:

• AutoModelForSequenceClassification.from_pretrained(...) dispatches to the right class
• Forward returns logits of shape (batch, num_labels)
• Loss is finite at random init, roughly ln(num_labels) as expected
• Backward produces finite gradients for every trainable parameter
• The library's LOAD REPORT correctly shows score.weight | MISSING (new head) and lm_head.weight | UNEXPECTED (causal-LM head unused)

A LoRA fine-tune on IMDB with allenai/OLMo-2-0425-1B (4.2M trainable params, 250 steps) brings loss from 1.48 over the first 20 steps down to 0.0005 over the last 20. I only ran the full training loop on one of the three because the classification head implementation is identical across them (same GenericForSequenceClassification mixin, different backbones and *PreTrainedModel bases), so the training-loop plumbing is shared; smoke-test forward/backward covers the other two. Happy to extend the LoRA run to Olmo and Olmo3 if you'd prefer.

Who can review?

cc @Rocketknight1 as requested on #45529.

Changed files

• docs/source/en/model_doc/olmo.md (modified, +5/-0)
• docs/source/en/model_doc/olmo2.md (modified, +5/-0)
• docs/source/en/model_doc/olmo3.md (modified, +5/-0)
• src/transformers/models/auto/modeling_auto.py (modified, +3/-0)
• src/transformers/models/olmo/modeling_olmo.py (modified, +6/-2)
• src/transformers/models/olmo/modular_olmo.py (modified, +6/-0)
• src/transformers/models/olmo2/modeling_olmo2.py (modified, +6/-2)
• src/transformers/models/olmo2/modular_olmo2.py (modified, +6/-0)
• src/transformers/models/olmo3/modeling_olmo3.py (modified, +6/-2)
• src/transformers/models/olmo3/modular_olmo3.py (modified, +6/-0)
• tests/models/olmo/test_modeling_olmo.py (modified, +4/-1)
• tests/models/olmo2/test_modeling_olmo2.py (modified, +4/-1)
• tests/models/olmo3/test_modeling_olmo3.py (modified, +2/-0)