PR #24729: fix(router): derive mode from litellm model map when db model_info lacks it

• Repository: BerriAI/litellm
• Author: NIK-TIGER-BILL
• State: closed | merged: False
• Link: https://github.com/BerriAI/litellm/pull/24729

Description (problem / solution / changelog)

Problem

Fixes #24658

The /model_group/info endpoint returns mode: "chat" for Azure text-embedding-3-small (and other Azure embedding models) even though the LiteLLM UI correctly shows them as embedding models.

Root cause

In Router._set_model_group_info(), when model_info is not found in the router cache, the code constructs a fallback ModelMapInfo with:

db_model_info = model.get("model_info", {})
mode = db_model_info.get("mode", "chat")   # ← always "chat" unless the user explicitly
                                             #   set mode in their config.yaml model_info

User config entries rarely include an explicit mode key, so this always falls through to "chat" — even for embedding models.

Fix

When db_model_info does not have an explicit mode, call litellm.get_model_info(model=litellm_model, custom_llm_provider=llm_provider) to derive the mode from LiteLLM's built-in model map before falling back to "chat":

mode = db_model_info.get("mode", None)

if mode is None:
    try:
        _map_info = litellm.get_model_info(
            model=litellm_model,
            custom_llm_provider=llm_provider if llm_provider else None,
        )
        mode = _map_info.get("mode", "chat") if _map_info else "chat"
    except Exception:
        mode = "chat"

With this change, text-embedding-3-small resolves to mode: "embedding" via the litellm model map. All other models retain their existing behaviour (litellm map lookup falls back to "chat" on any exception).

Checklist

• My PR is focused on a single issue
• I have read the LiteLLM contribution guidelines
• DCO signed

Changed files

• litellm/router.py (modified, +15/-2)