litellm - 💡(How to fix) Fix [Feature]: Support model group composition ("group of groups")

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

Allow a model_name entry to reference other model groups as its members, enabling a "group of groups" or model group composition pattern.

Motivation, pitch

Currently, if you want a single top-level group (e.g. fast) that load-balances across multiple named sub-groups with different providers and rate limits, you have to duplicate every deployment under both the sub-group name and the top-level name. In a real config this adds up fast, in my case, around 1000 lines of duplication.

Example of what's currently required:

# Sub-group 1: two providers for GPT-OSS-120b
- model_name: fast/GPT-OSS-120b
  litellm_params:
    model: groq/openai/gpt-oss-120b
    rpm: 30
    order: 2
- model_name: fast/GPT-OSS-120b
  litellm_params:
    model: cerebras/gpt-oss-120b
    rpm: 5
    order: 1

# Sub-group 2
- model_name: fast/glm-4.7
  litellm_params:
    model: cerebras/zai-glm-4.7
    rpm: 5

# To have a "fast" umbrella you must duplicate everything again:
- model_name: fast
  litellm_params:
    model: groq/openai/gpt-oss-120b
    rpm: 30
    order: 2
- model_name: fast
  litellm_params:
    model: cerebras/gpt-oss-120b
    rpm: 5
    order: 1
- model_name: fast
  litellm_params:
    model: cerebras/zai-glm-4.7
    rpm: 5

Proposed solution

Allow litellm_params.model to reference another model group name, so the router resolves it recursively:

- model_name: fast
  litellm_params:
    model: fast/GPT-OSS-120b
- model_name: fast
  litellm_params:
    model: fast/glm-4.7

This is consistent with how model_group_alias already works conceptually, it just needs to support referencing groups rather than only concrete provider strings.

Workaround

It's possible to approximate this today by pointing LiteLLM at itself, adding a deployment whose api_base is the proxy's own endpoint and whose model is a sub-group name. This works but is fragile, adds a network hop, and feels like an unintended use of the tool.

Expected behaviour

Requests to fast are load-balanced across all deployments in all referenced sub-groups, respecting their individual rpm/tpm limits and other settings as if they were declared inline.

What part of LiteLLM is this about?

Proxy

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

No response