litellm - ✅(Solved) Fix [Bug]: Default encoding_format list in model parameters triggers vLLM failure [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#28239Fetched 2026-05-20 03:40:40
View on GitHub
Comments
0
Participants
1
Timeline
4
Reactions
0
Author
Participants
Timeline (top)
labeled ×2cross-referenced ×1referenced ×1

Error Message

The model Test Connection behaves in this default way, and displays this error `Error testing connection: litellm.BadRequestError: Hosted_vllmException - {"error":{"message":"3 validation errors:\n {'type'... Using the Swagger API you can reproduce this error by not setting encoding_format or otherwise setting it.

Fix Action

Fixed

PR fix notes

PR #28308: fix(embedding): normalize list encoding_format from model config to first element

Description (problem / solution / changelog)

Root Cause

When encoding_format is set as a list in a model's litellm_params (e.g. to declare which formats the model supports):

model_list:
  - model_name: my-embeddings
    litellm_params:
      model: hosted_vllm/bge-m3
      api_base: http://localhost:8000
      encoding_format:
        - float
        - base64
        - ubyte
        - int8

the list is forwarded verbatim to the upstream embedding API. vLLM rejects it with validation errors because it expects a single string, not an array.

The proxy "Test Connection" flow and any request that does not explicitly set encoding_format both hit this path.

Fix

Normalize encoding_format at the top of embedding() in litellm/main.py, before get_optional_params_embeddings builds optional_params:

if isinstance(encoding_format, list):
    encoding_format = encoding_format[0] if encoding_format else None
  • A non-empty list → use the first element as the concrete format (natural default).
  • An empty list → None (no format specified, let the backend choose).
  • A plain string or None → unchanged.

This matches the encoding_format: Optional[str] contract on the function signature and applies uniformly to all providers (hosted_vllm, openai, azure, etc.).

Test Plan

  • litellm.embedding(model="hosted_vllm/...", input=["hello"], encoding_format=["float","base64"]) no longer raises
  • Model configured with encoding_format: [float] in litellm_params: proxy request without explicit encoding_format succeeds
  • Model configured with encoding_format: [float, base64]: explicit request encoding_format=base64 still uses base64
  • encoding_format="float" (string) continues to work unchanged

Fixes #28239

Changed files

  • litellm/main.py (modified, +2/-0)

Code Example

"encoding_format": [
    "float",
    "base64",
    "ubyte",
    "int8"
  ]

---
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

Including encoding_format as a list within model parameters leads to a vLLM failure on model calls that don't explicitly set a specific encoding_format, as by default LiteLLM passes the whole list to vLLM as a string.

 "encoding_format": [
    "float",
    "base64",
    "ubyte",
    "int8"
  ]

The model Test Connection behaves in this default way, and displays this error

Error testing connection: litellm.BadRequestError: Hosted_vllmException - {"error":{"message":"3 validation errors:\n {'type'...

Using the Swagger API you can reproduce this error by not setting encoding_format or otherwise setting it.

Steps to Reproduce

  1. Set encoding_format to a list on an embedding model
  2. Make a call to the embeddings endpoint without specifying an encoding_format
  3. Make a call to the embeddings endpoint with specifying an encoding format

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.83.14

Twitter / LinkedIn details

No response

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING