transformers - 💡(How to fix) Fix LORA权重合并保存在本地之后重新加载出来，两个模型输出结果不一致 [1 comments, 2 participants]

transformers2026-03-05 06:40:39

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

huggingface/transformers#44457•Fetched 2026-04-08 00:28:23

View on GitHub

Comments

Participants

Timeline

Reactions

Author

fish-kong

Participants

fish-kong

github-actions[bot]

Timeline (top)

commented ×1labeled ×1

Code Example

absl-py                           2.3.1
accelerate                        1.12.0
aiohappyeyeballs                  2.6.1
aiohttp                           3.13.2
aiosignal                         1.4.0
altgraph                          0.17.5
annotated-doc                     0.0.4
annotated-types                   0.7.0
anthropic                         0.71.0
anyio                             4.12.0
apache-tvm-ffi                    0.1.6
astor                             0.8.1
attrs                             25.4.0
auto-round                        0.9.2
blake3                            1.0.8
boto3                             1.42.7
botocore                          1.42.7
cachetools                        6.2.4
cbor2                             5.7.1
certifi                           2025.11.12
cffi                              2.0.0
chardet                           5.2.0
charset-normalizer                3.4.4
click                             8.3.1
cloudpickle                       3.1.2
colorama                          0.4.6
coloredlogs                       15.0.1
compressed-tensors                0.12.2
cryptography                      46.0.3
cuda-bindings                     13.1.1
cuda-pathfinder                   1.3.3
cuda-python                       13.1.1
cupy-cuda12x                      13.6.0
dataproperty                      1.1.0
datasets                          4.4.1
depyf                             0.20.0
dill                              0.4.0
diskcache                         5.6.3
distro                            1.9.0
dnspython                         2.8.0
docstring-parser                  0.17.0
einops                            0.8.1
email-validator                   2.3.0
evaluate                          0.4.6
fastapi                           0.127.0
fastapi-cli                       0.0.16
fastapi-cloud-cli                 0.7.0
fastar                            0.8.0
fastrlock                         0.8.3
filelock                          3.20.0
flashinfer-python                 0.5.3
flatbuffers                       25.9.23
frozenlist                        1.8.0
fschat                            0.2.36
fsspec                            2025.10.0
gguf                              0.17.1
h11                               0.16.0
hf-xet                            1.2.0
httpcore                          1.0.9
httptools                         0.7.1
httpx                             0.28.1
httpx-sse                         0.4.3
huggingface-hub                   0.36.0
humanfriendly                     10.0
idna                              3.11
ijson                             3.4.0.post0
interegular                       0.3.3
jinja2                            3.1.6
jiter                             0.12.0
jmespath                          1.0.1
joblib                            1.5.3
jsonlines                         4.0.0
jsonschema                        4.25.1
jsonschema-specifications         2025.9.1
lark                              1.2.2
latex2mathml                      3.78.1
llguidance                        1.3.0
llmcompressor                     0.9.0
llvmlite                          0.44.0
lm-eval                           0.4.9.2
lm-format-enforcer                0.11.3
loguru                            0.7.3
lxml                              6.0.2
markdown-it-py                    4.0.0
markdown2                         2.5.4
markupsafe                        3.0.3
mbstrdecoder                      1.1.4
mcp                               1.25.0
mdurl                             0.1.2
mistral-common                    1.8.6
model-hosting-container-standards 0.1.12
modelscope                        1.34.0
more-itertools                    10.8.0
mpmath                            1.3.0
msgpack                           1.1.2
msgspec                           0.20.0
multidict                         6.7.0
multiprocess                      0.70.18
networkx                          3.6.1
nh3                               0.3.2
ninja                             1.13.0
nltk                              3.9.2
numba                             0.61.2
numexpr                           2.14.1
numpy                             2.2.6
nvidia-cublas-cu12                12.8.4.1
nvidia-cuda-cupti-cu12            12.8.90
nvidia-cuda-nvrtc-cu12            12.8.93
nvidia-cuda-runtime-cu12          12.8.90
nvidia-cudnn-cu12                 9.10.2.21
nvidia-cudnn-frontend             1.17.0
nvidia-cufft-cu12                 11.3.3.83
nvidia-cufile-cu12                1.13.1.3
nvidia-curand-cu12                10.3.9.90
nvidia-cusolver-cu12              11.7.3.90
nvidia-cusparse-cu12              12.5.8.93
nvidia-cusparselt-cu12            0.7.1
nvidia-cutlass-dsl                4.3.4
nvidia-ml-py                      13.590.44
nvidia-nccl-cu12                  2.27.5
nvidia-nvjitlink-cu12             12.8.93
nvidia-nvshmem-cu12               3.3.20
nvidia-nvtx-cu12                  12.8.90
onnxruntime-gpu                   1.23.2
openai                            2.14.0
openai-harmony                    0.0.8
opencv-python                     4.11.0.86
opencv-python-headless            4.12.0.88
optimum                           2.1.0
outlines-core                     0.2.11
packaging                         25.0
pandas                            2.3.3
partial-json-parser               0.2.1.1.post7
pathvalidate                      3.3.1
peft                              0.18.0
pillow                            12.0.0
platformdirs                      4.5.1
portalocker                       3.2.0
prettytable                       3.17.0
prometheus-client                 0.23.1
prometheus-fastapi-instrumentator 7.1.0
prompt-toolkit                    3.0.52
propcache                         0.4.1
protobuf                          6.33.2
psutil                            7.1.3
py-cpuinfo                        9.0.0
pyarrow                           22.0.0
pybase64                          1.4.3
pybind11                          3.0.1
pyclipper                         1.4.0
pycountry                         24.6.1
pycparser                         2.23
pydantic                          2.12.5
pydantic-core                     2.41.5
pydantic-extra-types              2.10.6
pydantic-settings                 2.12.0
pyecharts                         2.0.9
pygments                          2.19.2
pyinstaller                       6.17.0
pyinstaller-hooks-contrib         2025.10
pyjwt                             2.10.1
pynvml                            13.0.1
pytablewriter                     1.2.1
python-dateutil                   2.9.0.post0
python-dotenv                     1.2.1
python-json-logger                4.0.0
python-multipart                  0.0.21
pytz                              2025.2
pyyaml                            6.0.3
pyzmq                             27.1.0
ray                               2.53.0
referencing                       0.37.0
regex                             2025.11.3
requests                          2.32.5
rich                              13.9.4
rich-toolkit                      0.17.1
rignore                           0.7.6
rouge-score                       0.1.2
rpds-py                           0.30.0
ruff                              0.14.8
s3transfer                        0.16.0
sacrebleu                         2.5.1
safetensors                       0.7.0
scikit-learn                      1.8.0
scipy                             1.16.3
sentencepiece                     0.2.1
sentry-sdk                        2.48.0
setproctitle                      1.3.7
setuptools                        79.0.1
shapely                           2.1.2
shellingham                       1.5.4
shortuuid                         1.0.13
simplejson                        3.20.2
six                               1.17.0
sniffio                           1.3.1
sqlitedict                        2.1.0
sse-starlette                     3.0.4
starlette                         0.50.0
supervisor                        4.3.0
svgwrite                          1.4.3
swanlab                           0.7.3
sympy                             1.14.0
tabledata                         1.3.4
tabulate                          0.9.0
tcolorpy                          0.1.7
threadpoolctl                     3.6.0
tiktoken                          0.12.0
tokenizers                        0.22.1
torch                             2.9.0+cu128
torchaudio                        2.9.0+cu128
torchvision                       0.24.0+cu128
tornado                           6.5.2
tqdm                              4.67.1
tqdm-multiprocess                 0.0.11
transformers                      4.57.3
triton                            3.5.0
typepy                            1.3.4
typer                             0.20.1
typing-extensions                 4.15.0
typing-inspection                 0.4.2
tzdata                            2025.2
urllib3                           2.6.1
uvicorn                           0.40.0
uvloop                            0.22.1
vllm                              0.13.0
watchfiles                        1.1.1
wavedrom                          2.0.3.post3
wcwidth                           0.2.14
websockets                        15.0.1
word2number                       1.1
wrapt                             2.0.1
xformers                          0.0.33.post1
xgrammar                          0.1.27
xxhash                            3.6.0
yarl                              1.22.0
zhconv                            1.4.3
zstandard                         0.25.0

---

model_path = 'Qwen3-4B path'

checkpoint_path = 'lorapath'

tokenizer = AutoTokenizer.from_pretrained(model_path,padding_side='left',fix_mistral_regex=True)

base_model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16)

model = PeftModel.from_pretrained(base_model, checkpoint_path)

model1 = model.merge_and_unload()

model1 = model1.to(torch.bfloat16)

# 保存合并之后的模型再重新加载出来推理
model1.save_pretrained(save_path, safe_serialization=True)

tokenizer.save_pretrained(save_path)

tokenizer = AutoTokenizer.from_pretrained(save_path,padding_side='left',fix_mistral_regex=True)

model2 = AutoModelForCausalLM.from_pretrained(save_path, torch_dtype=torch.bfloat16)

model = model1 或者model2

model.eval()

#推理的代码里面设置了do_sample=False,且top_k=1

outputs = model.generate(
        **inputs,
        max_new_tokens=10,
        do_sample=False,
        top_k=1)

---

for (n1,p1),(n2,p2) in zip(merged_model.named_parameters(), model2.named_parameters()):

	diff = (p1 - p2).abs().max()

	print(diff)

RAW_BUFFERClick to expand / collapse

System Info

Debian GNU/Linux 12 (bookworm)+5090-32G

absl-py                           2.3.1
accelerate                        1.12.0
aiohappyeyeballs                  2.6.1
aiohttp                           3.13.2
aiosignal                         1.4.0
altgraph                          0.17.5
annotated-doc                     0.0.4
annotated-types                   0.7.0
anthropic                         0.71.0
anyio                             4.12.0
apache-tvm-ffi                    0.1.6
astor                             0.8.1
attrs                             25.4.0
auto-round                        0.9.2
blake3                            1.0.8
boto3                             1.42.7
botocore                          1.42.7
cachetools                        6.2.4
cbor2                             5.7.1
certifi                           2025.11.12
cffi                              2.0.0
chardet                           5.2.0
charset-normalizer                3.4.4
click                             8.3.1
cloudpickle                       3.1.2
colorama                          0.4.6
coloredlogs                       15.0.1
compressed-tensors                0.12.2
cryptography                      46.0.3
cuda-bindings                     13.1.1
cuda-pathfinder                   1.3.3
cuda-python                       13.1.1
cupy-cuda12x                      13.6.0
dataproperty                      1.1.0
datasets                          4.4.1
depyf                             0.20.0
dill                              0.4.0
diskcache                         5.6.3
distro                            1.9.0
dnspython                         2.8.0
docstring-parser                  0.17.0
einops                            0.8.1
email-validator                   2.3.0
evaluate                          0.4.6
fastapi                           0.127.0
fastapi-cli                       0.0.16
fastapi-cloud-cli                 0.7.0
fastar                            0.8.0
fastrlock                         0.8.3
filelock                          3.20.0
flashinfer-python                 0.5.3
flatbuffers                       25.9.23
frozenlist                        1.8.0
fschat                            0.2.36
fsspec                            2025.10.0
gguf                              0.17.1
h11                               0.16.0
hf-xet                            1.2.0
httpcore                          1.0.9
httptools                         0.7.1
httpx                             0.28.1
httpx-sse                         0.4.3
huggingface-hub                   0.36.0
humanfriendly                     10.0
idna                              3.11
ijson                             3.4.0.post0
interegular                       0.3.3
jinja2                            3.1.6
jiter                             0.12.0
jmespath                          1.0.1
joblib                            1.5.3
jsonlines                         4.0.0
jsonschema                        4.25.1
jsonschema-specifications         2025.9.1
lark                              1.2.2
latex2mathml                      3.78.1
llguidance                        1.3.0
llmcompressor                     0.9.0
llvmlite                          0.44.0
lm-eval                           0.4.9.2
lm-format-enforcer                0.11.3
loguru                            0.7.3
lxml                              6.0.2
markdown-it-py                    4.0.0
markdown2                         2.5.4
markupsafe                        3.0.3
mbstrdecoder                      1.1.4
mcp                               1.25.0
mdurl                             0.1.2
mistral-common                    1.8.6
model-hosting-container-standards 0.1.12
modelscope                        1.34.0
more-itertools                    10.8.0
mpmath                            1.3.0
msgpack                           1.1.2
msgspec                           0.20.0
multidict                         6.7.0
multiprocess                      0.70.18
networkx                          3.6.1
nh3                               0.3.2
ninja                             1.13.0
nltk                              3.9.2
numba                             0.61.2
numexpr                           2.14.1
numpy                             2.2.6
nvidia-cublas-cu12                12.8.4.1
nvidia-cuda-cupti-cu12            12.8.90
nvidia-cuda-nvrtc-cu12            12.8.93
nvidia-cuda-runtime-cu12          12.8.90
nvidia-cudnn-cu12                 9.10.2.21
nvidia-cudnn-frontend             1.17.0
nvidia-cufft-cu12                 11.3.3.83
nvidia-cufile-cu12                1.13.1.3
nvidia-curand-cu12                10.3.9.90
nvidia-cusolver-cu12              11.7.3.90
nvidia-cusparse-cu12              12.5.8.93
nvidia-cusparselt-cu12            0.7.1
nvidia-cutlass-dsl                4.3.4
nvidia-ml-py                      13.590.44
nvidia-nccl-cu12                  2.27.5
nvidia-nvjitlink-cu12             12.8.93
nvidia-nvshmem-cu12               3.3.20
nvidia-nvtx-cu12                  12.8.90
onnxruntime-gpu                   1.23.2
openai                            2.14.0
openai-harmony                    0.0.8
opencv-python                     4.11.0.86
opencv-python-headless            4.12.0.88
optimum                           2.1.0
outlines-core                     0.2.11
packaging                         25.0
pandas                            2.3.3
partial-json-parser               0.2.1.1.post7
pathvalidate                      3.3.1
peft                              0.18.0
pillow                            12.0.0
platformdirs                      4.5.1
portalocker                       3.2.0
prettytable                       3.17.0
prometheus-client                 0.23.1
prometheus-fastapi-instrumentator 7.1.0
prompt-toolkit                    3.0.52
propcache                         0.4.1
protobuf                          6.33.2
psutil                            7.1.3
py-cpuinfo                        9.0.0
pyarrow                           22.0.0
pybase64                          1.4.3
pybind11                          3.0.1
pyclipper                         1.4.0
pycountry                         24.6.1
pycparser                         2.23
pydantic                          2.12.5
pydantic-core                     2.41.5
pydantic-extra-types              2.10.6
pydantic-settings                 2.12.0
pyecharts                         2.0.9
pygments                          2.19.2
pyinstaller                       6.17.0
pyinstaller-hooks-contrib         2025.10
pyjwt                             2.10.1
pynvml                            13.0.1
pytablewriter                     1.2.1
python-dateutil                   2.9.0.post0
python-dotenv                     1.2.1
python-json-logger                4.0.0
python-multipart                  0.0.21
pytz                              2025.2
pyyaml                            6.0.3
pyzmq                             27.1.0
ray                               2.53.0
referencing                       0.37.0
regex                             2025.11.3
requests                          2.32.5
rich                              13.9.4
rich-toolkit                      0.17.1
rignore                           0.7.6
rouge-score                       0.1.2
rpds-py                           0.30.0
ruff                              0.14.8
s3transfer                        0.16.0
sacrebleu                         2.5.1
safetensors                       0.7.0
scikit-learn                      1.8.0
scipy                             1.16.3
sentencepiece                     0.2.1
sentry-sdk                        2.48.0
setproctitle                      1.3.7
setuptools                        79.0.1
shapely                           2.1.2
shellingham                       1.5.4
shortuuid                         1.0.13
simplejson                        3.20.2
six                               1.17.0
sniffio                           1.3.1
sqlitedict                        2.1.0
sse-starlette                     3.0.4
starlette                         0.50.0
supervisor                        4.3.0
svgwrite                          1.4.3
swanlab                           0.7.3
sympy                             1.14.0
tabledata                         1.3.4
tabulate                          0.9.0
tcolorpy                          0.1.7
threadpoolctl                     3.6.0
tiktoken                          0.12.0
tokenizers                        0.22.1
torch                             2.9.0+cu128
torchaudio                        2.9.0+cu128
torchvision                       0.24.0+cu128
tornado                           6.5.2
tqdm                              4.67.1
tqdm-multiprocess                 0.0.11
transformers                      4.57.3
triton                            3.5.0
typepy                            1.3.4
typer                             0.20.1
typing-extensions                 4.15.0
typing-inspection                 0.4.2
tzdata                            2025.2
urllib3                           2.6.1
uvicorn                           0.40.0
uvloop                            0.22.1
vllm                              0.13.0
watchfiles                        1.1.1
wavedrom                          2.0.3.post3
wcwidth                           0.2.14
websockets                        15.0.1
word2number                       1.1
wrapt                             2.0.1
xformers                          0.0.33.post1
xgrammar                          0.1.27
xxhash                            3.6.0
yarl                              1.22.0
zhconv                            1.4.3
zstandard                         0.25.0

Who can help?

No response

Information

The official example scripts
My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
My own task or dataset (give details below)

Reproduction

对qwen3-4b大模型采用peft+transformers进行lora微调后得到lora权重，将base模型与lora模型合并后的模型为model1，

当model1进行save_pretrained之后，再重新加载模型得到model2

然后，后续利用相同的数据、推理参数与程序进行推理，并在推理程序中do_sample设置为False。

model1与model2的模型加载、合并、保存，重新加载的代码如下：

model_path = 'Qwen3-4B path'

checkpoint_path = 'lorapath'

tokenizer = AutoTokenizer.from_pretrained(model_path,padding_side='left',fix_mistral_regex=True)

base_model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16)

model = PeftModel.from_pretrained(base_model, checkpoint_path)

model1 = model.merge_and_unload()

model1 = model1.to(torch.bfloat16)

# 保存合并之后的模型再重新加载出来推理
model1.save_pretrained(save_path, safe_serialization=True)

tokenizer.save_pretrained(save_path)

tokenizer = AutoTokenizer.from_pretrained(save_path,padding_side='left',fix_mistral_regex=True)

model2 = AutoModelForCausalLM.from_pretrained(save_path, torch_dtype=torch.bfloat16)

model = model1 或者model2

model.eval()

#推理的代码里面设置了do_sample=False,且top_k=1

outputs = model.generate(
        **inputs,
        max_new_tokens=10,
        do_sample=False,
        top_k=1)

两次运行，同样的数据、同样的推理程序、同样的推理参数，model第一次是model1、第二次是model2，但是输出的结果不一样。

我按照下面的程序打印model1和model2的模型参数差，所有的打印都是：tensor(0., dtype=torch.bfloat16, grad_fn=<MaxBackward1>)，说明两个模型的参数是一致的。

for (n1,p1),(n2,p2) in zip(merged_model.named_parameters(), model2.named_parameters()):

	diff = (p1 - p2).abs().max()

	print(diff)

请问如何才能保证两个模型的结果完全一样?

Expected behavior

transformers save_pretrained bug

extent analysis

Problem Summary

The user is experiencing inconsistent results from two models, model1 and model2, which are loaded from the same checkpoint but have different behavior when generating text.

Root Cause Analysis

The root cause of the issue is likely due to the save_pretrained method in the transformers library, which is used to save the model weights and configuration. The user has observed that the model parameters are identical between model1 and model2, but the output results are different.

Fix Plan

To resolve the issue, we need to investigate the save_pretrained method and ensure that it is saving the model weights correctly. Here are the steps to fix the issue:

Step 1: Update `transformers` library

Update the transformers library to the latest version, which may include bug fixes related to the save_pretrained method.

Step 2: Use `save_pretrained` with `safe_serialization=False`

Try saving the model with safe_serialization=False to see if it resolves the issue. This option can help to avoid issues related to serialization.

model1.save_pretrained(save_path, safe_serialization=False)

Step 3: Use `torch.save` to save model weights

Instead of using save_pretrained, try saving the model weights using torch.save. This can help to ensure that the model weights are saved correctly.

torch.save(model1.state_dict(), save_path + '/model_weights.pth')

Step 4: Load model weights using `torch.load`

Load the model weights using torch.load to ensure that the weights are loaded correctly.

model2.load_state_dict(torch.load(save_path + '/model_weights.pth'))

Verification

To verify that the fix worked, run the same code again and check if the output results are identical between model1 and `model2

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

FAQ

Expected behavior

transformers save_pretrained bug

#api #ssr #installation #tensor shape #autograd error #indexing error #inference speed #output truncation #response parsing

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

transformers - 💡(How to fix) Fix LORA权重合并保存在本地之后重新加载出来，两个模型输出结果不一致 [1 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Code Example

System Info

Who can help?

Information

Tasks

Reproduction

Expected behavior

extent analysis

Problem Summary

Root Cause Analysis

Fix Plan

Step 1: Update `transformers` library

Step 2: Use `save_pretrained` with `safe_serialization=False`

Step 3: Use `torch.save` to save model weights

Step 4: Load model weights using `torch.load`

Verification

FAQ

Expected behavior

Still need to ship something?

TRENDING

transformers - 💡(How to fix) Fix LORA权重合并保存在本地之后重新加载出来，两个模型输出结果不一致 [1 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Code Example

System Info

Who can help?

Information

Tasks

Reproduction

Expected behavior

extent analysis

Problem Summary

Root Cause Analysis

Fix Plan

Step 1: Update transformers library

Step 2: Use save_pretrained with safe_serialization=False

Step 3: Use torch.save to save model weights

Step 4: Load model weights using torch.load

Verification

FAQ

Expected behavior

Still need to ship something?

RELATED_DISCOVERY

TRENDING

Step 1: Update `transformers` library

Step 2: Use `save_pretrained` with `safe_serialization=False`

Step 3: Use `torch.save` to save model weights

Step 4: Load model weights using `torch.load`