openclaw - 💡(How to fix) Fix Feature Request: Auto-route messages with images to imageModel for multi-modal processing [1 comments, 1 participants]

• No attachment dropped; no "model does not support images" error

TL;DR

Implement automatic model routing to switch between the primary text model and the image model based on the input type, specifically detecting image attachments and switching to the multi-modal image model.

Guidance

• Modify the resolveGatewayModelSupportsImages function to check for the presence of image attachments in the input and switch to the agent.imageModel.primary if images are detected.
• Update the parseMessageWithAttachments function to handle image attachments and trigger the model switching logic.
• Verify that the imageModel configuration is correctly set up in the schema and that the agent.imageModel.primary model supports image inputs.
• Test the automatic model routing with different input scenarios, including text-only, text with images, and images-only, to ensure the correct model is used in each case.

Example

// Example configuration for automatic model routing
"agents": {
  "defaults": {
    "model": {
      "primary": "minimax/MiniMax-M2.7",
      "input": ["text"]
    },
    "imageModel": {
      "primary": "minimax/mini-max-vision",
      "input": ["text", "image"]
    }
  }
}

Notes

The implementation details may vary depending on the specific requirements and constraints of the project. It is essential to test the automatic model routing thoroughly to ensure it works correctly in all scenarios.

Recommendation

Apply a workaround by modifying the resolveGatewayModelSupportsImages function to check for image attachments and switch to the agent.imageModel.primary model, as this will allow for automatic model routing based on input type without requiring manual model switching.