Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Feature]: Default to sending usage in stream when stream_options is not sent (e.g. providers that reject extra params) [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#23847•Fetched 2026-04-08 00:48:51
View on GitHub
Comments
0
Participants
1
Timeline
3
Reactions
0
Author
tadmon-ciscotadmon-cisco
Participants
tadmon-ciscotadmon-cisco
Timeline (top)
labeled ×2cross-referenced ×1

Fix Action

Fix / Workaround

We use litellm.acompletion(..., stream=True) with multiple backends. Some providers reject requests that include stream_options (or other extra parameters), so we have to omit stream_options entirely. In that case, the streaming path doesn’t send usage in the final chunk (the decision is tied to stream_options / include_usage), so we get no prompt_tokens / completion_tokens in the stream unless we monkey-patch LiteLLM.

Current workaround: we patch CustomStreamWrapper.check_send_stream_usage in litellm.litellm_core_utils.streaming_handler so that when stream_options is missing/falsy we still send usage:

from litellm.litellm_core_utils import streaming_handler as _litellm_streaming_handler

def _patched_check_send_stream_usage(self, stream_options):
    if not stream_options:
        return True   # still send usage when stream_options wasn't sent
    return _original_check_send_stream_usage(self, stream_options)

PR fix notes

PR #23978: Feature/default usage when no stream options

Description (problem / solution / changelog)

Relevant issues

Fixes #23847

Type

🆕 New Feature

Changes

  • Default to including usage in the final stream chunk when stream_options is not provided
  • Maintains existing behavior when stream_options is explicitly passed (respects include_usage)
  • Ensures compatibility with providers that reject unknown or extra parameters
  • Adds test coverage for default streaming usage behavior

Why this change?

When using stream=True with certain providers, passing stream_options can cause request failures if the provider does not support extra parameters.

This change ensures:

  • usage is still returned in streaming responses by default
  • No need to explicitly pass stream_options
  • Better cross-provider compatibility

Behavior Summary

ScenarioBehavior
stream_options NOT providedusage included by default
stream_options providedExisting behavior preserved

Impact

  • No breaking changes
  • Fully backward compatible
  • Improves reliability across providers

Changed files

  • litellm/litellm_core_utils/streaming_handler.py (modified, +2/-0)
  • poetry.lock (modified, +41/-32)

Code Example

from litellm.litellm_core_utils import streaming_handler as _litellm_streaming_handler

_original_check_send_stream_usage = (
    _litellm_streaming_handler.CustomStreamWrapper.check_send_stream_usage
)

def _patched_check_send_stream_usage(self, stream_options):
    if not stream_options:
        return True   # still send usage when stream_options wasn't sent
    return _original_check_send_stream_usage(self, stream_options)

_litellm_streaming_handler.CustomStreamWrapper.check_send_stream_usage = (
    _patched_check_send_stream_usage
)
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

When stream_options is not provided (or is empty) on a streaming completion, the streaming handler should default to including usage in the final stream chunk (i.e. behave as if include_usage were true when the option is absent). When stream_options is provided, keep current behavior (respect include_usage / existing contract).

Motivation, pitch

We use litellm.acompletion(..., stream=True) with multiple backends. Some providers reject requests that include stream_options (or other extra parameters), so we have to omit stream_options entirely. In that case, the streaming path doesn’t send usage in the final chunk (the decision is tied to stream_options / include_usage), so we get no prompt_tokens / completion_tokens in the stream unless we monkey-patch LiteLLM.

Current workaround: we patch CustomStreamWrapper.check_send_stream_usage in litellm.litellm_core_utils.streaming_handler so that when stream_options is missing/falsy we still send usage:

from litellm.litellm_core_utils import streaming_handler as _litellm_streaming_handler

_original_check_send_stream_usage = (
    _litellm_streaming_handler.CustomStreamWrapper.check_send_stream_usage
)

def _patched_check_send_stream_usage(self, stream_options):
    if not stream_options:
        return True   # still send usage when stream_options wasn't sent
    return _original_check_send_stream_usage(self, stream_options)

_litellm_streaming_handler.CustomStreamWrapper.check_send_stream_usage = (
    _patched_check_send_stream_usage
)

We’re asking for this “default to true when stream_options is absent” behavior to be the default in LiteLLM so that providers that reject extra params still get usage in the stream without callers sending stream_options. Not related to another GitHub issue.

What part of LiteLLM is this about?

SDK (litellm Python package)

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

No response

extent analysis

Fix Plan

To implement the desired behavior, we need to modify the check_send_stream_usage method in CustomStreamWrapper to default to True when stream_options is absent or empty.

Step-by-Step Solution

  • Modify the check_send_stream_usage method as follows:
def check_send_stream_usage(self, stream_options):
    if not stream_options:
        return True
    # existing logic to handle stream_options
    # ...

Alternatively, you can use the following code to achieve the same result:

def check_send_stream_usage(self, stream_options):
    include_usage = stream_options.get('include_usage') if stream_options else True
    # existing logic to handle include_usage
    # ...
  • Ensure that the stream_options parameter is properly handled in the CustomStreamWrapper class.

Verification

To verify that the fix worked, you can test the litellm.acompletion method with and without stream_options. The final chunk of the stream should include usage when stream_options is absent or empty.

Extra Tips

  • Make sure to update the documentation to reflect the new default behavior.
  • Consider adding tests to cover the different scenarios, including when stream_options is present, absent, or empty.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#model loading#dependency error#configuration error#environment variable#network issue

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - ✅(Solved) Fix [Feature]: Default to sending usage in stream when stream_options is not sent (e.g. providers that reject extra params) [1 pull requests, 1 participants]