Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Feature]: Fireworks AI provider: modernize chat transforms, add Messages & Responses API support, fix model listing [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#26537•Fetched 2026-04-26 05:06:27
View on GitHub
Comments
0
Participants
1
Timeline
4
Reactions
0
Author
frdengfrdeng
Participants
frdengfrdeng
Timeline (top)
labeled ×3cross-referenced ×1

Fix Action

Fix / Workaround

Several Fireworks AI behaviors in LiteLLM are outdated — the API has evolved but the provider config hasn't kept up. This causes:

  1. Silent data loss: json_schema name/strict fields are dropped; tool_choice: "required" is remapped to "any" (different semantics).
  2. Missing functionality: users can't use top_logprobs, seed, thinking, parallel_tool_calls through LiteLLM even though the Fireworks API supports them.
  3. Broken model listing: get_models() fails with ValueError for any user without FIREWORKS_ACCOUNT_ID set, and returns 0 models for users who do set it (since serverless models aren't listed under user accounts).
  4. No Messages/Responses API: Fireworks supports Anthropic Messages and OpenAI Responses compatible endpoints but LiteLLM has no config for them. Related: #7209 (tool-calls-in-content workaround, kept), #4416 (tool_choice mapping, now removed), #23106 (get_models URL issue, now fixed differently).

PR fix notes

PR #26538: fix(fireworks_ai): modernize chat transforms, add Messages + Response…

Description (problem / solution / changelog)

Fireworks AI responses, messages API support, fix model listing

Relevant issues

Fixes #26537 #26257 #24774 #25950 #23981

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.

  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Screenshots / Proof of Fix

<!-- Include screenshots, screen recordings, or log output demonstrating that your changes work as expected. For bug fixes: show reproduction before the fix and passing behavior after. For new features: show the feature working end-to-end. For UI changes: include before/after screenshots. -->

Type

<!-- Select the type of Pull Request --> <!-- Keep only the necessary ones -->

🆕 New Feature

Changes

Bring the Fireworks AI provider up to date with the current API behavior and add support for Anthropic Messages and OpenAI Responses compatible endpoints.

Chat Completions (transformation.py)

  • Remove four outdated parameter transformations that are now natively supported by the Fireworks API: tool_choice "required" → "any" mapping, response_format json_schema rewriting, max_completion_tokens → max_tokens mapping, and strict stripping from tools.
  • Add missing supported params: top_logprobs, seed, logit_bias, parallel_tool_calls, thinking.
  • Always pass reasoning_effort through (Fireworks accepts it on all models; unsupported models simply ignore it).
  • Remove hardcoded reasoning_supported_models list — rely on model_prices_and_context_window.json via supports_reasoning() instead.
  • Default supports_function_calling and supports_tool_choice to True in get_provider_info so unlisted/custom models aren't rejected.
  • Add custom_llm_provider property.
  • Add comprehensive class docstring documenting all request/response transforms.

Cost Calculator (cost_calculator.py)

  • Refactor cost_per_token to use generic_cost_per_token as the primary path with get_base_model_for_pricing as fallback for unmapped models.

Messages API (new)

  • Add FireworksAIMessagesConfig extending AnthropicMessagesConfig for Fireworks' /v1/messages compatible endpoint.
  • Uses Bearer auth (not x-api-key), no anthropic-version header.

Responses API (new)

  • Add FireworksAIResponsesConfig extending OpenAILikeResponsesConfig for Fireworks' /v1/responses compatible endpoint.
  • Correct URL construction when api_base doesn't end with /v1.

Model Listing (get_models)

  • Fix get_models to query the fireworks public account with filter=supports_serverless=true (returns usable serverless models) instead of requiring FIREWORKS_ACCOUNT_ID (which only returns dedicated deployments — typically 0 for most users).
  • Make FIREWORKS_ACCOUNT_ID optional: when set, also queries the user's account for dedicated deployments and merges both lists.
  • Add pagination support and graceful error handling.

Wiring

  • Register both new configs in ProviderConfigManager, init.py, and _lazy_imports_registry.py.

Tests

  • Add unit tests for messages, responses, and cost_calculator configs.
  • Update chat transformation tests for new get_models behavior, reasoning_effort always-on, and pass-through param assertions.
  • Update integration tests for tool_choice and response_format pass-through.

Changed files

  • litellm/__init__.py (modified, +6/-0)
  • litellm/_lazy_imports_registry.py (modified, +8/-0)
  • litellm/llms/fireworks_ai/chat/transformation.py (modified, +151/-109)
  • litellm/llms/fireworks_ai/cost_calculator.py (modified, +19/-17)
  • litellm/llms/fireworks_ai/messages/transformation.py (added, +62/-0)
  • litellm/llms/fireworks_ai/responses/transformation.py (added, +54/-0)
  • litellm/utils.py (modified, +4/-0)
  • tests/llm_translation/test_fireworks_ai_translation.py (modified, +6/-17)
  • tests/test_litellm/llms/fireworks_ai/chat/test_fireworks_ai_chat_transformation.py (modified, +265/-29)
  • tests/test_litellm/llms/fireworks_ai/cost_calculator/test_fireworks_ai_cost_calculator.py (added, +84/-0)
  • tests/test_litellm/llms/fireworks_ai/messages/test_fireworks_ai_messages_transformation.py (added, +199/-0)
  • tests/test_litellm/llms/fireworks_ai/responses/test_fireworks_ai_responses_transformation.py (added, +147/-0)
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

Bring the Fireworks AI provider up to date with the current API and add support for the Anthropic Messages and OpenAI Responses compatible endpoints.

1. Chat Completions — remove outdated transforms

The current FireworksAIConfig applies several parameter transformations that are no longer necessary (Fireworks now natively supports these OpenAI-standard behaviors):

  • tool_choice: "required" → "any" mapping (Fireworks accepts "required" natively)
  • response_format json_schema rewriting to json_object + schema (Fireworks accepts the OpenAI json_schema format directly)
  • max_completion_tokens → max_tokens rename (Fireworks accepts max_completion_tokens)
  • strict field stripping from tools (Fireworks accepts and ignores strict) These transforms silently lose information (e.g., the json_schema.name field is dropped) and diverge from what the API actually supports.

2. Chat Completions — fix supported params & reasoning

  • Missing supported params: top_logprobs, seed, logit_bias, parallel_tool_calls, thinking are accepted by the Fireworks API but not listed in get_supported_openai_params.
  • The reasoning_supported_models hardcoded list is outdated and violates the project guideline to use model_prices_and_context_window.json + supports_reasoning().
  • reasoning_effort should always be passed through — the Fireworks API accepts it on all models and handles unsupported cases itself.
  • get_provider_info should default supports_tool_choice: True for unlisted/custom models (same as supports_function_calling).

3. Messages API support (new)

Fireworks offers an Anthropic-compatible /v1/messages endpoint. Add FireworksAIMessagesConfig (extending AnthropicMessagesConfig) with:

  • Bearer auth (not x-api-key)
  • No anthropic-version header
  • Consistent 4-env-var API key resolution

4. Responses API support (new)

Fireworks offers an OpenAI-compatible /v1/responses endpoint. Add FireworksAIResponsesConfig (extending OpenAILikeResponsesConfig) with:

  • Correct URL construction (/v1/responses)
  • Consistent 4-env-var API key resolution

5. Cost calculator refactor

Refactor cost_per_token to use generic_cost_per_token as the primary path with get_base_model_for_pricing as a fallback for unmapped models.

6. Model listing (get_models) fix

The current get_models() requires FIREWORKS_ACCOUNT_ID and queries /v1/accounts/{account_id}/models, which only returns dedicated deployments (typically 0 models for most users). Per Fireworks docs, it should:

  • Query accounts/fireworks/models?filter=supports_serverless=true by default (returns all usable serverless models, no account ID needed)
  • Optionally also query user's account when FIREWORKS_ACCOUNT_ID is set
  • Support pagination

Motivation, pitch

Several Fireworks AI behaviors in LiteLLM are outdated — the API has evolved but the provider config hasn't kept up. This causes:

  1. Silent data loss: json_schema name/strict fields are dropped; tool_choice: "required" is remapped to "any" (different semantics).
  2. Missing functionality: users can't use top_logprobs, seed, thinking, parallel_tool_calls through LiteLLM even though the Fireworks API supports them.
  3. Broken model listing: get_models() fails with ValueError for any user without FIREWORKS_ACCOUNT_ID set, and returns 0 models for users who do set it (since serverless models aren't listed under user accounts).
  4. No Messages/Responses API: Fireworks supports Anthropic Messages and OpenAI Responses compatible endpoints but LiteLLM has no config for them. Related: #7209 (tool-calls-in-content workaround, kept), #4416 (tool_choice mapping, now removed), #23106 (get_models URL issue, now fixed differently).

What part of LiteLLM is this about?

Proxy

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

No response

extent analysis

TL;DR

Update the Fireworks AI provider configuration to match the current API, including removing outdated transforms, adding support for new parameters, and implementing new Messages and Responses API endpoints.

Guidance

  • Remove outdated parameter transformations in FireworksAIConfig, such as the tool_choice mapping and response_format rewriting, to prevent silent data loss.
  • Add support for new parameters, including top_logprobs, seed, logit_bias, parallel_tool_calls, and thinking, to the get_supported_openai_params function.
  • Implement the new FireworksAIMessagesConfig and FireworksAIResponsesConfig classes to support the Anthropic Messages and OpenAI Responses compatible endpoints.
  • Refactor the cost_per_token function to use generic_cost_per_token as the primary path and update the get_models function to query the correct endpoint and support pagination.

Example

class FireworksAIMessagesConfig(AnthropicMessagesConfig):
    def __init__(self, api_key):
        super().__init__(api_key)
        self.auth = "Bearer"
        self.url = "/v1/messages"

class FireworksAIResponsesConfig(OpenAILikeResponsesConfig):
    def __init__(self, api_key):
        super().__init__(api_key)
        self.url = "/v1/responses"

Notes

The implementation details may vary depending on the specific requirements of the Fireworks AI API and the LiteLLM project. It's essential to consult the official documentation and existing codebase to ensure a smooth integration.

Recommendation

Apply the workaround by updating the Fireworks AI provider configuration to match the current API, as this will resolve the silent data loss, missing functionality, and broken model listing issues.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#logging issue#authentication issue#prompt issue#agent setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING