langchain - 💡(How to fix) Fix Feature Request: WhichModel integration for cost-optimized model selection [1 comments, 2 participants]

TL;DR

To integrate WhichModel with LangChain for automatic model routing, create a new class that inherits from BaseChatModel and utilizes the WhichModel MCP endpoint to select the optimal model based on task requirements and budget.

Guidance

• Review the WhichModel MCP endpoint documentation to understand the available tools and their usage, such as recommend_model and get_pricing.
• Design a new class that extends BaseChatModel to incorporate the WhichModel integration, allowing for dynamic model selection based on task complexity, budget, and required capabilities.
• Investigate how to handle errors and exceptions that may occur during the model selection process, such as network errors or invalid responses from the WhichModel MCP endpoint.
• Consider implementing a caching mechanism to store the results of recent model recommendations to reduce the number of requests to the WhichModel MCP endpoint.

Example

from langchain_core.language_models import BaseChatModel
import requests

class WhichModelChatModel(BaseChatModel):
    def __init__(self, task_type, budget, requirements):
        self.task_type = task_type
        self.budget = budget
        self.requirements = requirements
        self.model = self.select_model()

    def select_model(self):
        response = requests.post("https://whichmodel.dev/mcp/recommend_model", json={
            "task_type": self.task_type,
            "budget": self.budget,
            "requirements": self.requirements
        })
        return response.json()["recommended_model"]

Notes

The implementation details of the WhichModel integration will depend on the specific requirements of the LangChain project and the WhichModel MCP endpoint. This example provides a basic starting point, but additional error handling and caching mechanisms may be necessary.

Recommendation

Apply workaround: Implement a custom solution using the WhichModel MCP endpoint to select the optimal model for each task, as there is no existing integration available in LangChain.