Issue Station
Sign In
Tools hereToolbox

transformers - ✅(Solved) Fix Dependency issue with transformers [1 pull requests, 2 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
huggingface/transformers#44279•Fetched 2026-04-08 00:29:22
View on GitHub
Comments
2
Participants
2
Timeline
7
Reactions
0
Author
ChilliwidditChilliwiddit
Participants
ChilliwidditChilliwiddittaruasnigdhataruasnigdha
Timeline (top)
commented ×2closed ×1cross-referenced ×1labeled ×1

Error Message

%%capture !pip uninstall -y bitsandbytes bitsandbytes-cuda* torch torchvision torchaudio xformers transformers datasets pyarrow huggingface-hub numpy spacy thinc

!pip install "numpy<2.0.0"

!pip install --index-url https://download.pytorch.org/whl/cu124 torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0

!pip install bitsandbytes==0.49.2

!pip install --upgrade transformers accelerate peft huggingface_hub datasets pyarrow !pip install evaluate rouge_score bert-score faiss-gpu sentence-transformers

import torch from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel from huggingface_hub import login from datasets import load_dataset from sentence_transformers import SentenceTransformer from sklearn.neighbors import NearestNeighbors import numpy as np import os import gc from tqdm import tqdm import evaluate import time

ValueError Traceback (most recent call last) /tmp/ipykernel_55/852286074.py in <cell line: 0>() 1 import torch ----> 2 from transformers import AutoModelForCausalLM, AutoTokenizer 3 from peft import PeftModel 4 from huggingface_hub import login 5 from datasets import load_dataset

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Fix Action

Fixed

PR fix notes

PR #344: [AI] [FEAT]: Kaggle 기반 자동 학습 파이프라인 구축

Description (problem / solution / changelog)

개요

운영 서버 메모리 2GB 한계로 불가능했던 딥러닝 모델 학습을 Kaggle GPU를 활용해 자동화한 파이프라인 구축

Closes #343

변경사항

추가

  • AI/scripts/ 파이프라인 스크립트 5개
  • AI/pipelines/weekly_routine.py
  • .github/workflows/train.yml
  • 4개 모델 train_kaggle.py
  • AI/kaggle_notebooks/ 4개 모델 노트북

수정

  • PatchTST/wrapper.py (get_signals 추가, module. 제거)
  • TCN/wrapper.py (module. 제거)
  • artifact_paths.py (PatchTST 저장 경로 수정)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

Summary by CodeRabbit

  • New Features

    • 주간 자동 재학습 파이프라인(스케줄 + 수동 트리거) 및 파이프라인 제어용 CLI 플래그(--skip-extract/--skip-upload) 추가
    • 여러 모델용 Kaggle 스타일 학습 노트북과 학습 트리거·모니터링 자동화 도입
    • 데이터 추출(parquet), 데이터셋 업로드, 가중치 다운로드/업로드, 원격 서버 배포(SSH) 자동화 스크립트 추가

  • Bug Fixes

    • 병렬 학습 체크포인트 호환성 개선(데이터병렬 저장 포맷 처리)
    • 모델 아티팩트 경로 및 스케일러 저장/명명 규칙 정비

  • Chores

    • ML 아티팩트·자격증명 등 불필요 파일 무시 규칙(.gitignore) 업데이트
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Changed files

  • .github/workflows/train.yml (added, +79/-0)
  • .gitignore (modified, +11/-0)
  • AI/kaggle_notebooks/TCN/kernel-metadata.json (added, +16/-0)
  • AI/kaggle_notebooks/TCN/notebook.ipynb (added, +125/-0)
  • AI/kaggle_notebooks/itransformer/kernel-metadata.json (added, +16/-0)
  • AI/kaggle_notebooks/itransformer/notebook.ipynb (added, +105/-0)
  • AI/kaggle_notebooks/patchtst/kernel-metadata.json (added, +17/-0)
  • AI/kaggle_notebooks/patchtst/notebook.ipynb (added, +109/-0)
  • AI/kaggle_notebooks/transformer/kernel-metadata.json (added, +17/-0)
  • AI/kaggle_notebooks/transformer/notebook.ipynb (added, +104/-0)
  • AI/modules/signal/core/artifact_paths.py (modified, +4/-3)
  • AI/modules/signal/models/PatchTST/train.py (modified, +10/-57)
  • AI/modules/signal/models/PatchTST/train_kaggle.py (added, +327/-0)
  • AI/modules/signal/models/PatchTST/wrapper.py (modified, +11/-129)
  • AI/modules/signal/models/TCN/train_kaggle.py (added, +320/-0)
  • AI/modules/signal/models/TCN/wrapper.py (modified, +16/-39)
  • AI/modules/signal/models/itransformer/train_kaggle.py (added, +364/-0)
  • AI/modules/signal/models/transformer/train_kaggle.py (added, +303/-0)
  • AI/pipelines/weekly_routine.py (added, +172/-0)
  • AI/scripts/deploy_to_server.py (added, +165/-0)
  • AI/scripts/download_weights.py (added, +327/-0)
  • AI/scripts/extract_to_parquet.py (added, +249/-0)
  • AI/scripts/trigger_training.py (added, +166/-0)
  • AI/scripts/upload_to_kaggle.py (added, +79/-0)

Code Example

%%capture
!pip uninstall -y bitsandbytes bitsandbytes-cuda* torch torchvision torchaudio xformers transformers datasets pyarrow huggingface-hub numpy spacy thinc

!pip install "numpy<2.0.0"

!pip install --index-url https://download.pytorch.org/whl/cu124 torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0

!pip install bitsandbytes==0.49.2

!pip install --upgrade transformers accelerate peft huggingface_hub datasets pyarrow
!pip install evaluate rouge_score bert-score faiss-gpu sentence-transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
from huggingface_hub import login
from datasets import load_dataset
from sentence_transformers import SentenceTransformer
from sklearn.neighbors import NearestNeighbors
import numpy as np
import os
import gc
from tqdm import tqdm
import evaluate
import time

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_55/852286074.py in <cell line: 0>()
      1 import torch
----> 2 from transformers import AutoModelForCausalLM, AutoTokenizer
      3 from peft import PeftModel
      4 from huggingface_hub import login
      5 from datasets import load_dataset

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
RAW_BUFFERClick to expand / collapse

System Info

%%capture
!pip uninstall -y bitsandbytes bitsandbytes-cuda* torch torchvision torchaudio xformers transformers datasets pyarrow huggingface-hub numpy spacy thinc

!pip install "numpy<2.0.0"

!pip install --index-url https://download.pytorch.org/whl/cu124 torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0

!pip install bitsandbytes==0.49.2

!pip install --upgrade transformers accelerate peft huggingface_hub datasets pyarrow
!pip install evaluate rouge_score bert-score faiss-gpu sentence-transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
from huggingface_hub import login
from datasets import load_dataset
from sentence_transformers import SentenceTransformer
from sklearn.neighbors import NearestNeighbors
import numpy as np
import os
import gc
from tqdm import tqdm
import evaluate
import time

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_55/852286074.py in <cell line: 0>()
      1 import torch
----> 2 from transformers import AutoModelForCausalLM, AutoTokenizer
      3 from peft import PeftModel
      4 from huggingface_hub import login
      5 from datasets import load_dataset

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

What's happened here? How do I fix this?

Who can help?

No response

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

  1. Copy and run the code in a kaggle notebook with a T4 GPU

Expected behavior

Normally it would install just fine and I would be able to proceed with the rest of the code. A copy of the notebook can be found here

extent analysis

Problem Summary The issue is caused by a mismatch between the installed version of NumPy and the expected version by the installed PyTorch library.

Fix Plan To resolve this issue, we need to ensure that the installed version of NumPy is compatible with the installed version of PyTorch.

  1. Uninstall incompatible NumPy version: Run the following command to uninstall the incompatible version of NumPy.

!pip uninstall numpy

2. **Install compatible NumPy version**: Install a compatible version of NumPy using the following command.
   ```bash
!pip install "numpy==1.22.3"

Note: You can install a specific version of NumPy that is compatible with your PyTorch version. 3. Reinstall required libraries: Reinstall the required libraries, including PyTorch, transformers, and others.

!pip install --index-url https://download.pytorch.org/whl/cu124 torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0
!pip install bitsandbytes==0.49.2
!pip install --upgrade transformers accelerate peft huggingface_hub datasets pyarrow

Verification To verify that the fix worked, run the code again and check if the error is resolved. You can also check the installed versions of NumPy and PyTorch to ensure they are compatible.

Extra Tips

  • Always check the compatibility of libraries before installing them.
  • Use specific versions of libraries to avoid version conflicts.
  • Reinstalling libraries can sometimes resolve version conflicts.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

Normally it would install just fine and I would be able to proceed with the rest of the code. A copy of the notebook can be found here

#api#ssr#installation#tensor shape#autograd error#SSR setup#ISR setup#authentication setup#request error#file not found

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

transformers - ✅(Solved) Fix Dependency issue with transformers [1 pull requests, 2 comments, 2 participants]