ollama - 💡(How to fix) Fix Support for Baidu Qianfan-OCR 4B model [5 comments, 3 participants]

Fix Plan

To integrate the Qianfan-OCR model for image-to-Markdown conversion, follow these steps:

• Install the required libraries: transformers and Pillow
• Import necessary modules and load the Qianfan-OCR model
• Preprocess the input image and pass it to the model for conversion

Example Code

from PIL import Image
from transformers import AutoFeatureExtractor, AutoModelForVision2Seq

# Load the Qianfan-OCR model and feature extractor
model_name = "baidu/Qianfan-OCR"
feature_extractor = AutoFeatureExtractor.from_pretrained(model_name)
model = AutoModelForVision2Seq.from_pretrained(model_name)

# Load the input image
image = Image.open("input_image.png")

# Preprocess the image
inputs = feature_extractor(images=image, return_tensors="pt")

# Convert the image to Markdown
output = model.generate(**inputs)

# Print the generated Markdown
print(output)

Verification

To verify the fix, check the output Markdown for correctness and compare it with the expected result.

Extra Tips

• Make sure to install the required libraries and import the necessary modules.
• Adjust the model and feature extractor names according to your specific use case.
• Refer to the Hugging Face documentation for more information on using the Qianfan-OCR model.