Issue Station
Sign In
Tools hereToolbox

llamaIndex - ✅(Solved) Fix [Feature Request]: Responses API version for `OpenAI Like` and `LiteLLM` [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
run-llama/llama_index#21233•Fetched 2026-04-08 01:58:01
View on GitHub
Comments
0
Participants
1
Timeline
3
Reactions
0
Author
HydrogenCHydrogenC
Participants
HydrogenCHydrogenC
Timeline (top)
labeled ×2cross-referenced ×1

PR fix notes

PR #21246: feat(openai-like): add OpenAILikeResponses class for Responses API

Description (problem / solution / changelog)

Description

Add OpenAILikeResponses class that enables 3rd party OpenAI-compatible providers to use the new Responses API.

Currently, OpenAILike only wraps the chat completions API via OpenAI. This PR adds a parallel class
OpenAILikeResponses that wraps OpenAIResponses, following the same pattern used by AzureOpenAIResponses. This allows users of OpenAI-compatible servers (vLLM, Ollama, LiteLLM proxy, etc.) to access Responses API features like built-in
tools, reasoning, stateful conversations, and streaming response events.

Key additions:

  • OpenAILikeResponses subclasses OpenAIResponses with configurable context_window, is_function_calling_model, and tokenizer
  • Exported from llama_index.llms.openai_like

Relates to discussion around Responses API adoption in LlamaIndex connectors.

New Package?

  • No

Version Bump?

  • Yes
  • No

Type of Change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

  • I added new unit tests to cover this change

Unit tests cover: class identity, metadata defaults, custom metadata, tokenizer handling (None and instance), mocked chat
call, and serialization. All existing OpenAILike tests continue to pass.

Suggested Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

This commit will close the #21233 Issue

Changed files

  • llama-index-integrations/llms/llama-index-llms-openai-like/llama_index/llms/openai_like/__init__.py (modified, +2/-1)
  • llama-index-integrations/llms/llama-index-llms-openai-like/llama_index/llms/openai_like/responses.py (added, +128/-0)
  • llama-index-integrations/llms/llama-index-llms-openai-like/pyproject.toml (modified, +1/-0)
  • llama-index-integrations/llms/llama-index-llms-openai-like/tests/test_openai_like_responses.py (added, +135/-0)
RAW_BUFFERClick to expand / collapse

Feature Description

The OpenAILike and LiteLLM connectors in LlamaIndex now only supports the old chat completions API, and they should support the new Responses API at the same time.

A possible approach would be creating a new OpenAILikeResponses that subclasses OpenAIResponses instead of OpenAI. Same for LiteLLM.

Reason

The traditional chat completion API is out of favor now. OpenAI strongly suggests migrating to the Responses API for supported models.

Value of Feature

Pretty valuable for now, more valuable in the future.

extent analysis

TL;DR

Create a new connector subclassing OpenAIResponses to support the new Responses API for OpenAILike and LiteLLM connectors.

Guidance

  • Identify the current implementation of OpenAILike and LiteLLM connectors to understand how they interact with the old chat completions API.
  • Design a new OpenAILikeResponses connector that subclasses OpenAIResponses to support the new Responses API, and apply a similar approach for LiteLLM.
  • Verify the new connectors' functionality by testing them with supported models and comparing the results with the traditional chat completion API.
  • Consider implementing a fallback mechanism to the old API for models that do not support the new Responses API.

Example

class OpenAILikeResponses(OpenAIResponses):
    # implementation details for the new Responses API
    pass

Notes

The exact implementation details of the new connectors will depend on the specific requirements of the Responses API and the existing implementation of OpenAILike and LiteLLM.

Recommendation

Apply workaround by creating new connectors that support the new Responses API, as this approach allows for a gradual migration to the new API while still supporting models that only work with the old chat completions API.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ISR setup#authentication setup#request error#file not found

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

llamaIndex - ✅(Solved) Fix [Feature Request]: Responses API version for `OpenAI Like` and `LiteLLM` [1 pull requests, 1 participants]