Skip to content

Update GPT-QModel references and deprecate AutoGPTQ#3190

Merged
BenjaminBossan merged 8 commits intohuggingface:mainfrom
Qubitium:update-gptqmodel-support
Apr 29, 2026
Merged

Update GPT-QModel references and deprecate AutoGPTQ#3190
BenjaminBossan merged 8 commits intohuggingface:mainfrom
Qubitium:update-gptqmodel-support

Conversation

@Qubitium
Copy link
Copy Markdown
Contributor

@Qubitium Qubitium commented Apr 23, 2026 •
edited
Loading

@BenjaminBossan

  1. Update references to GPTQModel as GPT-QModel as the pkg is no longer gptq only, it now fully supports awq and many other quant methods.
  2. Full deprecate AutoGPTQ (repo archived since Marh 2025, I was last commiter/maintainer)
  3. Fix PEFT using hard coded kernel check assumptions for AWQ and tests (as gptqmodel is also handling awq kernel loading)

Companion (but not required) PR in Optimum: huggingface/optimum#2426

@Qubitium Qubitium marked this pull request as ready for review April 23, 2026 15:47
@Qubitium Qubitium marked this pull request as draft April 23, 2026 15:54
@Qubitium Qubitium marked this pull request as ready for review April 23, 2026 16:11
@Qubitium Qubitium changed the title Update GPT-QModel references Update GPT-QModel references and deprecate AutoGPTQ Apr 23, 2026
@BenjaminBossan
Copy link
Copy Markdown
Member

BenjaminBossan commented Apr 27, 2026

@Qubitium Thanks for the PR. What is the state, does it depend on the optimum change to be released first? Also, we had a recent update to the Dockerfile in #3188, could you please check the merge conflict?

@Qubitium
Copy link
Copy Markdown
Contributor Author

Qubitium commented Apr 28, 2026 •
edited
Loading

@BenjaminBossan Conflict resolved and we have tested this PR and relevant tests are passing and does not rely on the Optimum PR to be merged first. This one is ready for merge. Please trigger ci tests to make sure there are no regressions.

BenjaminBossan
BenjaminBossan requested changes Apr 28, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@BenjaminBossan BenjaminBossan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR to fully remove AutoGPTQ and move to GPT-QModel.

The PR generally looks good and both single and multi GPU tests passed in my testing. There are a few smaller issues left, please check.

Comment thread docker/peft-gpu/Dockerfile Outdated
Comment thread docs/source/developer_guides/quantization.md Outdated
Comment thread docs/source/developer_guides/quantization.md Outdated
```bash
# gptqmodel install
pip install gptqmodel --no-build-isolation
# GPTQ-Model install
Copy link
Copy Markdown
Member

@BenjaminBossan BenjaminBossan Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# GPTQ-Model install
# GPT-QModel install

Comment thread src/peft/tuners/lora/aqlm.py Outdated
Comment thread src/peft/tuners/lora/gptq.py Outdated
Comment thread tests/test_gptqmodel.py Outdated
Comment thread tests/test_gptqmodel.py Outdated
Comment thread tests/test_gpu_examples.py Outdated
Comment thread src/peft/utils/other.py Outdated
Comment thread src/peft/utils/other.py Outdated
@Qubitium
Copy link
Copy Markdown
Contributor Author

Qubitium commented Apr 28, 2026

@BenjaminBossan Ready for ci tests re-run.

  1. Now pinned to just released gpt-qmodel v7.0.0 and remove --no-build-isolation depend
  2. Updated/sycned name from GPTQModel to GPT-QModel to align with pkg repo.
  3. Updated requested changes and dead comments/code.

@Qubitium Qubitium requested a review from BenjaminBossan April 28, 2026 21:42
Qubitium
Qubitium commented Apr 28, 2026
View reviewed changes
Comment thread docker/peft-gpu/Dockerfile
# a NVIDIA L4). So we fix the compute capability to 8.9. In the future we might extend this
# to a list of compute capabilities (separated by ;).
RUN CUDA_ARCH_LIST=8.9 conda run -n peft pip install --no-build-isolation "gptqmodel>=6.0.3"
RUN CUDA_ARCH_LIST=8.9 conda run -n peft pip install "gptqmodel>=7.0.0"
Copy link
Copy Markdown
Contributor Author

@Qubitium Qubitium Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7.0.0 no longer requires --no-build-isolation as all kernels have moved from setup stage to first-use JIT.

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

HuggingFaceDocBuilderDev commented Apr 29, 2026

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

BenjaminBossan
BenjaminBossan approved these changes Apr 29, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@BenjaminBossan BenjaminBossan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, everything looks good. Failing CI is unrelated and I ran the GPU tests locally to ensure that they pass too.

And congrats to the 7.0 release.

@BenjaminBossan BenjaminBossan merged commit 8c6943c into huggingface:main Apr 29, 2026
4 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BenjaminBossan BenjaminBossan BenjaminBossan approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Qubitium @BenjaminBossan @HuggingFaceDocBuilderDev