litellm - 💡(How to fix) Fix [Feature]: Provide supported reasoning levels in model info API [1 participants]

Fix Plan

To provide the supported reasoning levels for a model through the Model info API, we can modify the existing API endpoint to include this information.

Steps to Implement the Fix

• Update the models.py file to include a supported_reasoning_levels attribute in the ModelInfo class.
• Modify the get_model_info function in the api.py file to retrieve and return the supported_reasoning_levels for each model.
• Add a new endpoint or modify an existing one to handle requests for model information, including supported reasoning levels.

Example Code

# models.py
class ModelInfo:
    def __init__(self, name, description, supported_reasoning_levels):
        self.name = name
        self.description = description
        self.supported_reasoning_levels = supported_reasoning_levels

# api.py
from models import ModelInfo

def get_model_info(model_name):
    # Retrieve model information from database or other storage
    model_info = ModelInfo(
        name=model_name,
        description="Model description",
        supported_reasoning_levels=["level1", "level2", "level3"]
    )
    return model_info

# app.py
from flask import Flask, jsonify
from api import get_model_info

app = Flask(__name__)

@app.route('/models/<model_name>', methods=['GET'])
def get_model(model_name):
    model_info = get_model_info(model_name)
    return jsonify({
        'name': model_info.name,
        'description': model_info.description,
        'supported_reasoning_levels': model_info.supported_reasoning_levels
    })

if __name__ == '__main__':
    app.run(debug=True)

Verification

To verify that the fix worked, send a GET request to the /models/<model_name> endpoint and check that the response includes the supported_reasoning_levels attribute with the expected values.

Extra Tips

• Ensure that the supported_reasoning_levels attribute is properly validated and sanitized to prevent potential security vulnerabilities.
• Consider adding documentation for the new endpoint and updating existing documentation to reflect the changes.
• Test the updated API endpoint thoroughly to ensure it works as expected and handles different scenarios correctly.