Issue Station
Sign In
Tools hereToolbox

litellm - 💡(How to fix) Fix [Feature]: Add 91 missing NVIDIA NIM "Free Endpoint" models to supported list [1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#25045•Fetched 2026-04-08 02:34:59
View on GitHub
Comments
0
Participants
1
Timeline
2
Reactions
0
Author
SammmESammmE
Participants
SammmESammmE
Timeline (top)
labeled ×2
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

The official documentation says ALL Nvidia NIM Models Supported!, but many models provided by NVIDIA NIM are not mapped yet. Specifically, there are 91 free endpoints missing.

This will involve:

  1. Updating model_prices_and_context_window.json with the 91 Free Endpoint models available via NVIDIA's hosted API.

  2. Updating the Markdown documentation under "Supported Models" to reflect the newly added models. Note: There are still 75 "Partner Endpoints" and 130 "Download Available", but those require custom api_base overrides, and I want to keepe the PR scope small

Motivation, pitch

The LiteLLM documentation states "💥 ALL Nvidia NIM Models Supported!" but currently only lists some of them.

NVIDIA NIM's ecosystem has expanded, and there are now 91 models that can be queried for free using standard NVIDIA infrastructure and a default NVIDIA_API_KEY. By adding these to the model dictionary, users will be able to use the nvidia_nim/ prefix for much more models without having to manually configure custom routing.

I am ready to fork, make the JSON/Docs updates, add the required mocked unit test in tests/test_litellm/, and submit a PR. Just looking for a thumbs up before I get started!

What part of LiteLLM is this about?

SDK (litellm Python package)

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

https://www.linkedin.com/in/samhith-enganti-022628338/

extent analysis

TL;DR

Update the model_prices_and_context_window.json file with the 91 missing free endpoint models and reflect the changes in the Markdown documentation under "Supported Models".

Guidance

  • Verify the list of 91 missing models by checking the NVIDIA NIM hosted API and ensure they are not already included in the model_prices_and_context_window.json file.
  • Update the model_prices_and_context_window.json file with the missing models, following the existing format and structure.
  • Reflect the changes in the Markdown documentation under "Supported Models" to ensure accuracy and consistency with the updated model list.
  • Consider adding mocked unit tests in tests/test_litellm/ to validate the changes and ensure the updated models are correctly integrated.

Example

No code snippet is provided as the issue does not require a specific code change, but rather a data update.

Notes

This solution assumes that the 91 missing models are indeed available via the NVIDIA NIM hosted API and can be added to the model_prices_and_context_window.json file without requiring custom API overrides.

Recommendation

Apply workaround: Update the model_prices_and_context_window.json file and Markdown documentation to reflect the newly added models, as this will provide an immediate solution to the issue and improve the accuracy of the documentation.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#vector store#embedding generation#cache error#pipeline error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING