litellm - ✅(Solved) Fix [Bug]: dimensions not dropped for hosted_vllm embeddings with drop_params=True [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#23119Fetched 2026-04-08 00:38:30
View on GitHub
Comments
0
Participants
1
Timeline
4
Reactions
0
Participants
Timeline (top)
referenced ×2cross-referenced ×1labeled ×1

When using litellm.aembedding() with a hosted_vllm provider (e.g., hosted_vllm/qwen/qwen3-embedding-8b via OpenRouter), the dimensions parameter is always passed through to the API, even when drop_params=True. This causes 422 errors from providers that don't support dimensions.

Root Cause

In litellm/utils.py, get_optional_params_embeddings() has an elif custom_llm_provider in openai_compatible_providers branch that passes all parameters through without validation or drop_params handling.

The OpenAI-specific branch correctly limits dimensions to text-embedding-3-* models, but this logic is not applied to openai_compatible_providers (which includes hosted_vllm).

Call chain:

  1. get_optional_params_embeddings()hosted_vllm hits the openai_compatible_providers branch, all params pass through unconditionally
  2. litellm/llms/openai_like/embedding/handler.py L108 — data = {"model": model, "input": input, **optional_params} includes unsupported dimensions
  3. Provider returns 422

Fix Action

Fixed

PR fix notes

PR #23120: fix(embeddings): drop dimensions for openai_compatible_providers when drop_params=True

Description (problem / solution / changelog)

Summary

When calling litellm.embedding() / litellm.aembedding() with a hosted_vllm (or any openai_compatible_provider) endpoint, the dimensions parameter was always forwarded to the upstream API — even when drop_params=True. This caused 422 Unprocessable Entity from any vLLM endpoint that doesn't support dimensions.

Fixes #23119

Root Cause

The else branch in get_optional_params_embeddings() that handles openai_compatible_providers did:

else:
    optional_params = non_default_params   # no validation, no drop_params

The OpenAI-specific branch correctly limits dimensions to text-embedding-3-* models, but this guard was absent for the compatible-providers path.

Fix

In litellm/utils.pyget_optional_params_embeddings(), replace the bare pass-through with logic that mirrors the OpenAI branch:

else:
    optional_params = non_default_params.copy()
    if (
        "dimensions" in optional_params
        and model is not None
        and "text-embedding-3" not in model
    ):
        if litellm.drop_params is True or drop_params is True:
            optional_params.pop("dimensions")   # silently drop
        else:
            raise UnsupportedParamsError(...)   # or raise

Before / After

# Before – 422 regardless of drop_params
await litellm.aembedding(
    model="hosted_vllm/qwen/qwen3-embedding-8b",
    input=["Hello"],
    dimensions=4096,
    drop_params=True,    # had no effect
)

# After – dimensions is silently dropped, call succeeds
await litellm.aembedding(
    model="hosted_vllm/qwen/qwen3-embedding-8b",
    input=["Hello"],
    dimensions=4096,
    drop_params=True,   # now works
)

Changed files

  • litellm/llms/hosted_vllm/embedding/transformation.py (modified, +10/-2)
  • litellm/utils.py (modified, +15/-1)
  • tests/test_litellm/test_utils.py (modified, +41/-0)

Code Example

import litellm

response = await litellm.aembedding(
    model="hosted_vllm/qwen/qwen3-embedding-8b",
    input=["Hello world"],
    dimensions=4096,
    api_base="https://openrouter.ai/api/v1",
    api_key="...",
    drop_params=True,  # has no effect
)
# Expected: dimensions is dropped, embedding succeeds
# Actual: 422 Unprocessable Entity

---

elif custom_llm_provider in openai_compatible_providers:
    optional_params = non_default_params.copy()
    if "dimensions" in optional_params:
        model_name = model or ""
        if "text-embedding-3" not in model_name.lower():
            if litellm.drop_params or drop_params:
                optional_params.pop("dimensions")
            else:
                raise UnsupportedParamsError(...)
RAW_BUFFERClick to expand / collapse

Description

When using litellm.aembedding() with a hosted_vllm provider (e.g., hosted_vllm/qwen/qwen3-embedding-8b via OpenRouter), the dimensions parameter is always passed through to the API, even when drop_params=True. This causes 422 errors from providers that don't support dimensions.

Root Cause

In litellm/utils.py, get_optional_params_embeddings() has an elif custom_llm_provider in openai_compatible_providers branch that passes all parameters through without validation or drop_params handling.

The OpenAI-specific branch correctly limits dimensions to text-embedding-3-* models, but this logic is not applied to openai_compatible_providers (which includes hosted_vllm).

Call chain:

  1. get_optional_params_embeddings()hosted_vllm hits the openai_compatible_providers branch, all params pass through unconditionally
  2. litellm/llms/openai_like/embedding/handler.py L108 — data = {"model": model, "input": input, **optional_params} includes unsupported dimensions
  3. Provider returns 422

Steps to Reproduce

import litellm

response = await litellm.aembedding(
    model="hosted_vllm/qwen/qwen3-embedding-8b",
    input=["Hello world"],
    dimensions=4096,
    api_base="https://openrouter.ai/api/v1",
    api_key="...",
    drop_params=True,  # has no effect
)
# Expected: dimensions is dropped, embedding succeeds
# Actual: 422 Unprocessable Entity

Environment

  • litellm version: 1.80+
  • Python: 3.12
  • Provider: OpenRouter (but affects any hosted_vllm endpoint that doesn't support dimensions)

Suggested Fix

Add dimensions validation to the openai_compatible_providers branch in get_optional_params_embeddings():

elif custom_llm_provider in openai_compatible_providers:
    optional_params = non_default_params.copy()
    if "dimensions" in optional_params:
        model_name = model or ""
        if "text-embedding-3" not in model_name.lower():
            if litellm.drop_params or drop_params:
                optional_params.pop("dimensions")
            else:
                raise UnsupportedParamsError(...)

Relevant files:

  • litellm/utils.pyget_optional_params_embeddings() (~L2937)
  • litellm/constants.pyopenai_compatible_providers list
  • litellm/llms/openai_like/embedding/handler.py — request body construction

extent analysis

Fix Plan

To fix the issue, we need to modify the get_optional_params_embeddings() function in litellm/utils.py to handle the dimensions parameter correctly for openai_compatible_providers. Here are the steps:

  • Open litellm/utils.py and locate the get_optional_params_embeddings() function.
  • Add the following code to the elif custom_llm_provider in openai_compatible_providers branch:
if "dimensions" in optional_params:
    model_name = model or ""
    if "text-embedding-3" not in model_name.lower():
        if litellm.drop_params or drop_params:
            optional_params.pop("dimensions")
        else:
            raise UnsupportedParamsError("Dimensions parameter is not supported for this model")
  • Save the changes to litellm/utils.py.

Verification

To verify that the fix worked, you can run the following code:

import litellm

response = await litellm.aembedding(
    model="hosted_vllm/qwen/qwen3-embedding-8b",
    input=["Hello world"],
    dimensions=4096,
    api_base="https://openrouter.ai/api/v1",
    api_key="...",
    drop_params=True,
)
print(response)  # Should print the embedding response without errors

If the fix is successful, the dimensions parameter should be dropped, and the embedding request should succeed.

Extra Tips

  • Make sure to update the litellm library to the latest version after applying the fix.
  • If you encounter any issues with the fix, check the litellm documentation and the OpenRouter API documentation for any updates or changes to the API endpoints and parameters.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - ✅(Solved) Fix [Bug]: dimensions not dropped for hosted_vllm embeddings with drop_params=True [1 pull requests, 1 participants]