Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [Feature]: Consolidate GPTQ Quantization [1 pull requests, 5 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#37765•Fetched 2026-04-08 01:12:59
View on GitHub
Comments
5
Participants
3
Timeline
16
Reactions
0
Author
robertgshaw2-redhatrobertgshaw2-redhat
Participants
chengyiniechengyiniejikunshangjikunshangrobertgshaw2-redhatrobertgshaw2-redhat
Assignees
chengyiniechengyinie
Timeline (top)
commented ×5referenced ×3subscribed ×3labeled ×2

PR fix notes

PR #38288: [Quant] Consolidate GPTQ: rename gptq_marlin.py to auto_gptq.py

Description (problem / solution / changelog)

Purpose

Consolidate GPTQ quantization by renaming gptq_marlin.py to auto_gptq.py as requested in #37765. Key changes:

  • Renames gptq_marlin.py → auto_gptq.py
  • GPTQMarlinConfig.get_name() now returns "auto_gptq"
  • Adds override_quantization_method() to auto-convert models with quant_method: gptq to use auto_gptq
  • Updates all imports across the codebase to use auto_gptq module
  • Adds auto_gptq to overrides list in model.py
  • Adds test_auto_gptq.py for the new quantization method
  • Updates test_gptq_marlin.py skipif condition to check auto_gptq support
  • Maintains backward compatibility: quantization="gptq" still works (maps to GPTQMarlinConfig) Closes #37765

Test Plan

pytest tests/quantization/test_auto_gptq.py -v -s
pytest tests/quantization/test_gptq_dynamic.py -v -s
pytest tests/quantization/test_lm_head.py -v -s
pytest tests/models/quantization/test_gptq_marlin.py -v -s

## Test Result
 All passed.
---
<details>
<summary> Essential Elements of an Effective PR Description Checklist </summary>

- [*] The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
- [*] The test plan, such as providing test command.
- [*] The test results, such as pasting the results comparison before and after, or e2e results
- [ ] (Optional) The necessary documentation update, such as updating `supported_models.md` and `examples` for a new model.
- [ ] (Optional) Release notes update. If your change is user facing, please update the release notes draft in the [Google Doc](https://docs.google.com/document/d/1YyVqrgX4gHTtrstbq8oWUImOyPCKSGnJ7xtTpmXzlRs/edit?tab=t.0).
</details>

## Changed files

- `tests/models/quantization/test_gptq_marlin.py` (modified, +3/-6)
- `tests/quantization/test_auto_gptq.py` (added, +56/-0)
- `tests/quantization/test_gptq_dynamic.py` (modified, +9/-19)
- `tests/quantization/test_gptq_v2.py` (modified, +8/-11)
- `tests/quantization/test_lm_head.py` (modified, +2/-3)
- `tests/weight_loading/models-large.txt` (modified, +2/-2)
- `tests/weight_loading/models.txt` (modified, +0/-6)
- `vllm/config/model.py` (modified, +2/-0)
- `vllm/model_executor/layers/fused_moe/layer.py` (modified, +1/-1)
- `vllm/model_executor/layers/linear.py` (modified, +1/-2)
- `vllm/model_executor/layers/quantization/__init__.py` (modified, +6/-5)
- `vllm/model_executor/layers/quantization/auto_gptq.py` (renamed, +28/-60)
- `vllm/model_executor/layers/quantization/gptq.py` (removed, +0/-399)
- `vllm/model_executor/layers/quantization/inc.py` (modified, +12/-21)
- `vllm/model_executor/layers/quantization/moe_wna16.py` (modified, +7/-14)
- `vllm/model_executor/layers/quantization/utils/gptq_utils.py` (modified, +17/-28)
- `vllm/model_executor/models/openpangu_vl.py` (modified, +2/-3)
RAW_BUFFERClick to expand / collapse

🚀 The feature, motivation and pitch

We currently have two files:

  • gptq.py
  • gptq_marlin.py

These are not needed. We now have decoupled the quantization format integration and kernels.

We should delete gptq.py and consolidate everything into gptq_marlin.py

Then, we should rename gptq_marlin.py to auto_gptq.py

Alternatives

none

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To consolidate the quantization format integration and kernels, we will:

  • Delete the unnecessary gptq.py file
  • Rename gptq_marlin.py to auto_gptq.py and update its contents to include all necessary functionality

Code Changes

# Remove all imports and code from gptq.py
# Update gptq_marlin.py to include any necessary functionality from gptq.py
# Rename gptq_marlin.py to auto_gptq.py

Step-by-Step Solution

  1. Delete gptq.py: Remove the file from the repository using git rm gptq.py
  2. Rename gptq_marlin.py: Use git mv gptq_marlin.py auto_gptq.py to rename the file
  3. Update imports: Replace any imports of gptq with auto_gptq in other files
  4. Verify functionality: Test the updated code to ensure all necessary functionality is preserved

Verification

  • Run tests to ensure the updated auto_gptq.py file works as expected
  • Verify that all necessary functionality from gptq.py is included in auto_gptq.py

Extra Tips

  • Use git status to check for any changes before committing
  • Commit the changes with a clear and descriptive message, e.g. "Consolidate quantization format integration and kernels into auto_gptq.py"

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#prompt issue#agent setup#task chaining#parallel task#integration issue

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING