vllm - 💡(How to fix) Fix [Bug]: key error: ‘Layer.34.mlp.experts.gate_up_proj’ [3 comments, 3 participants]

Hi, which version of cloned is compatible with transformers5.3? I am hitting a key error: Keyerror: ‘Layer.34.mlp.experts.gate_up_proj’

Fix Plan

The fix is to update the cloned library to a version compatible with transformers 5.3.

Step-by-Step Solution

• Check the compatible version of cloned with transformers 5.3 in the documentation or release notes.
• Update cloned to the compatible version using pip:

pip install cloned==<compatible_version>

• If you are using a requirements.txt file, update the version there as well.

Code Changes

No code changes are required, but you may need to update your model loading code to match the new version of cloned. For example:

from cloned import ClonedModel

# Load the model with the updated cloned library
model = ClonedModel.from_pretrained('model_name')

Verification

Verify that the model loads without errors and that the KeyError is resolved.

Extra Tips

• Always check the documentation and release notes for compatible versions of libraries.
• Use a virtual environment to manage dependencies and avoid version conflicts.