from langchain_openai.chat_models.base import _create_usage_metadata

_create_usage_metadata( {"prompt_tokens": 100, "completion_tokens": 50, "total_tokens": 150, "prompt_tokens_details": {}, "completion_tokens_details": {}}, service_tier="priority" )

TypeError: unsupported operand type(s) for -: 'int' and 'NoneType'

TL;DR

The bug can be fixed by modifying the _create_usage_metadata function to handle the case where cached_tokens is None when calculating token arithmetic for "priority" or "flex" service tiers.

Guidance

• The issue arises from the .get(key, 0) method returning None when the key exists with a value of None, causing a TypeError when trying to subtract None from an integer.
• To fix this, the code should be modified to handle the case where cached_tokens is None, for example by using the .get() method with a default value of 0, or by explicitly checking for None before performing the subtraction.
• The same fix should be applied to the _create_usage_metadata_responses function, which is also affected by this bug.
• The fix should be verified by running the provided example code with the modified _create_usage_metadata function to ensure that it no longer raises a TypeError.

Example

def _create_usage_metadata(api_response, service_tier):
    # ...
    cached_tokens = api_response.get("prompt_tokens_details", {}).get("cached_tokens")
    if cached_tokens is None:
        cached_tokens = 0
    # ...

Notes

This fix assumes that a cached_tokens value of None should be treated as 0 for the purposes of token arithmetic. If this is not the case, the fix may need to be modified accordingly.

Recommendation

Apply workaround: the code should be modified to handle the case where cached_tokens is None to prevent the TypeError from occurring.