langchain - 💡(How to fix) Fix ChatMistralAI: Add Image Input Support [1 comments, 1 participants]

Submission checklist

• This is a feature request, not a bug report or usage question.
• I added a clear and descriptive title that summarizes the feature request.
• I used the GitHub search to find a similar feature request and didn't find it.
• I checked the LangChain documentation and API reference to see if this feature already exists.
• This is not related to the langchain-community package.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-openrouter
• langchain-perplexity
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Feature Description

I would like ChatMistralAI to support image inputs via the standard ImageContentBlock class and the create_image_block helper methods.

LangChain defines a standard ImageContentBlock format for multimodal messages, but ChatMistralAI forwards it to the Mistral API without converting it. Since Mistral expects a different image schema, this leads to API errors when using helpers like create_image_block or ImageContentBlock.

Use Case

I'm trying to use the ImageContentBlock class & the create_image_block method from langchain to pass images as input to vision models from different providers. But since Mistral models have a custom image input format, I end up with a custom conversion layer just for Mistral models.

If ChatMistralAI natively supports image input with the standard image content blocks, then it would be very useful as I will not have to have a custom conversion layer just for Mistral models.

Proposed Solution

I think we can have a conversion method for image content blocks like this:

def _convert_lc_image_block_to_mistral(block: dict) -> dict:
    """Convert a LangChain ImageContentBlock to Mistral's image_url format."""
    if block.get("type") != "image":
        return block
    if url := block.get("url"):
        return {"type": "image_url", "image_url": url}
    if base64_data := block.get("base64"):
        mime_type = block.get("mime_type")
        return {"type": "image_url", "image_url": f"data:{mime_type};base64,{base64_data}"}
    raise ValueError("Image content block must have either url or base64 field set"

Alternatives Considered

No response

Additional Context

No response