Issue Station
Sign In
Tools hereToolbox

langchain - 💡(How to fix) Fix Suggestion: Improve extensibility of _create_chat_result for custom metrics [2 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
langchain-ai/langchain#36373•Fetched 2026-04-08 01:52:49
View on GitHub
Comments
2
Participants
2
Timeline
5
Reactions
0
Author
MathbookxdMathbookxd
Participants
MathbookxdMathbookxdvolcano303volcano303
Timeline (top)
labeled ×3commented ×1issue_type_added ×1
RAW_BUFFERClick to expand / collapse

Checked other resources

  • This is a feature request, not a bug report or usage question.
  • I added a clear and descriptive title that summarizes the feature request.
  • I used the GitHub search to find a similar feature request and didn't find it.
  • I checked the LangChain documentation and API reference to see if this feature already exists.
  • This is not related to the langchain-community package.

Package (Required)

  • langchain
  • langchain-openai
  • langchain-anthropic
  • langchain-classic
  • langchain-core
  • langchain-model-profiles
  • langchain-tests
  • langchain-text-splitters
  • langchain-chroma
  • langchain-deepseek
  • langchain-exa
  • langchain-fireworks
  • langchain-groq
  • langchain-huggingface
  • langchain-mistralai
  • langchain-nomic
  • langchain-ollama
  • langchain-openrouter
  • langchain-perplexity
  • langchain-qdrant
  • langchain-xai
  • Other / not sure / general

Feature Description

Use Case & Limitation: I am performing performance evaluations (e.g., measuring time-to-first-token) which require access to low-level data from the LLM response. While I can modify a local model to return this data, the _create_chat_result method in BaseChatModel filters out this extra information, as it’s designed to return only a standard set of fields.

Proposal: Could we refactor this method to preserve additional metadata from the raw response? For instance, by including a generation_info attribute in the returned ChatResult that contains the full raw output or specific extended properties? This would greatly enhance the library's flexibility for monitoring and evaluation purposes.

Use Case

Use Case & Limitation: I am performing performance evaluations (e.g., measuring time-to-first-token) which require access to low-level data from the LLM response. While I can modify a local model to return this data, the _create_chat_result method in BaseChatModel filters out this extra information, as it’s designed to return only a standard set of fields.

Proposed Solution

Proposal: Could we refactor this method to preserve additional metadata from the raw response? For instance, by including a generation_info attribute in the returned ChatResult that contains the full raw output or specific extended properties? This would greatly enhance the library's flexibility for monitoring and evaluation purposes.

Alternatives Considered

No response

Additional Context

No response

extent analysis

Fix Plan

To address the feature request, we can modify the _create_chat_result method in BaseChatModel to include a generation_info attribute in the returned ChatResult. This attribute will contain the full raw output or specific extended properties from the LLM response.

Here are the concrete steps:

  • Modify the _create_chat_result method to accept an additional generation_info parameter.
  • Update the method to include the generation_info attribute in the returned ChatResult.
  • Optionally, add a configuration option to control which extended properties are included in the generation_info attribute.

Example code:

class BaseChatModel:
    # ...

    def _create_chat_result(self, response, generation_info=None):
        # ...
        chat_result = ChatResult(
            # ...
            generation_info=generation_info
        )
        return chat_result

# Example usage:
llm_response = {
    'text': 'Hello World',
    'time_to_first_token': 0.5,
    'other_metadata': '...'
}

generation_info = {
    'raw_output': llm_response,
    'extended_properties': {
        'time_to_first_token': llm_response['time_to_first_token']
    }
}

chat_result = base_chat_model._create_chat_result(llm_response, generation_info)
print(chat_result.generation_info)

Verification

To verify that the fix worked, you can:

  • Check that the generation_info attribute is included in the returned ChatResult.
  • Verify that the generation_info attribute contains the expected extended properties.
  • Test the modified method with different LLM responses and configuration options.

Extra Tips

  • Consider adding documentation and tests for the modified method to ensure that it is correctly used and maintained.
  • If you need to include a large amount of metadata, consider using a more efficient data structure, such as a dictionary or a pandas DataFrame.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#device allocation#model download#LLM response

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

langchain - 💡(How to fix) Fix Suggestion: Improve extensibility of _create_chat_result for custom metrics [2 comments, 2 participants]