langchain - 💡(How to fix) Fix Easier multimodal tool [1 participants]

Submission checklist

• This is a feature request, not a bug report or usage question.
• I added a clear and descriptive title that summarizes the feature request.
• I used the GitHub search to find a similar feature request and didn't find it.
• I checked the LangChain documentation and API reference to see if this feature already exists.
• This is not related to the langchain-community package.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-openrouter
• langchain-perplexity
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Feature Description

If the LangChain users wants to implement Tool function that return multimodal data, they have to write

  return Command(
          update={
              "messages": [
                  ToolMessage(
                      content=[
                          {
                            "type": "image",
                            "base64": BASE64_DATA,
                            "media_type": "image/png"
                          }
                      ],
                      tool_call_id=runtime.tool_call_id,
                      name=TOOL_NAME
                  )
              ]
          }
      )

This is too tiring just to return an image. I want this to be more easier (like just return PIL.Image.Image)

Use Case

Creating multimodal returning Tools quicker

Proposed Solution

If #37311 is approved, check if the return type from the tool is Content type, and just add it as ToolMessage on messages

Alternatives Considered

No response

Additional Context

I would love to contribute