litellm - 💡(How to fix) Fix [Feature]: Dynamic Cost Reconciliation for OpenRouter (Support for Service Tiers and Real-time Pricing) [1 participants]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

Implement a Dynamic Billing Reconciliation mechanism for OpenRouter provider to sync the response_cost with the actual billed amount.

Specifically, I propose adding a feature where LiteLLM can optionally fetch the final cost from OpenRouter's /api/v1/generation?id={id} endpoint after a request is completed, instead of solely relying on local static price maps. This is crucial for supporting OpenRouter's dynamic features like Service Tiers (Flex) and provider-specific discounts.

Motivation, pitch

I am managing large-scale LLM infrastructure (Data Engineering at Digikala) and we use the /model/new endpoint to dynamically register hundreds of OpenRouter models.

The Problem: Currently, LiteLLM calculates costs locally based on static prices set at registration time. However, when we use extra_body={"service_tier": "flex"}, OpenRouter applies significant real-time discounts. Since LiteLLM is unaware of these dynamic billing changes, it logs the "standard" price in LiteLLM_SpendLogs, leading to a significant Cost Drift (e.g., LiteLLM logs $0.0000171 while OpenRouter actually bills $0.0000139).

This makes internal financial auditing and team-based billing based on virtual_keys highly inaccurate. We need the "Source of Truth" (OpenRouter) to be reflected in our LiteLLM Postgres database.

Related Issues: This is similar to the billing transparency issues reported for custom models (e.g., #10023), but specific to the OpenRouter aggregator logic.

What part of LiteLLM is this about?

Proxy

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

No response