llamaIndex - ✅(Solved) Fix Feature Request: Native Structured Outputs for AWS Bedrock Converse API [2 pull requests, 1 comments, 1 participants]

ndgigliotti · 2026-02-14T21:41:10Z

[llamaIndex] AWS Bedrock launched native structured outputs in February 2026 via the outputConfig.textFormat parameter. LlamaIndex's llama-index-llms-bedrock-c… AWS Bedrock launched native structured outputs in February 2026 via the `outputConfig.textFormat` parameter. LlamaIndex's `llama-index-llms-bedrock-converse` currently uses function calling as a workaround. This should be updated to use the native API for better reliability and performance. # PR #20753: Feat/bedrock structured outputs - Repository: run-llama/llama_index - Author: akintunero - State: open | merged: False - Link: https://github.com/run-llama/llama_index/pull/20753 ## Description (problem / solution / changelog) # Description AWS Bedrock launched native structured outputs in February 2026 via the `outputConfig.textFormat` parameter. This feature adds native support for structured outputs to the `llama-index-llms-bedrock-converse` integration, replacing the function calling workaround for supported models. ## Changes: - Add `BEDROCK_STRUCTURED_OUTPUT_SUPPORTED_MODELS` tuple with supported models (Claude 4.5+, Amazon Nova, DeepSeek, Mistral) - Add `is_bedrock_structured_output_supported()` function with region prefix support - Implement `structured_predict()` and `astructured_predict()` methods - Fall back to function calling for unsupported models (backward compatible) - Add 4 unit tests for model support detection ## Supported Models: - Claude 4.5+ (haiku, sonnet, opus) - Amazon Nova (pro, lite, micro) - DeepSeek R1 - Mistral Large Fixes #20703 ## New Package? - [ ] Yes - [x] No ## Version Bump? - [x] Yes - [ ] No ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update ## How Has This Been Tested? - [x] I added new unit tests to cover this change - [ ] I believe this change is already covered by existing unit tests ## Suggested Checklist: - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] I have added Google Colab support for the newly added notebooks. - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I ran `uv run make format; uv run make lint` to appease the lint gods ## Changed files - `llama-index-integrations/llms/llama-index-llms-bedrock-converse/llama_index/llms/bedrock_converse/base.py` (modified, +178/-14) - `llama-index-integrations/llms/llama-index-llms-bedrock-converse/llama_index/llms/bedrock_converse/utils.py` (modified, +34/-0) - `llama-index-integrations/llms/llama-index-llms-bedrock-converse/pyproject.toml` (modified, +1/-1) - `llama-index-integrations/llms/llama-index-llms-bedrock-converse/tests/test_llms_bedrock_converse.py` (modified, +59/-0) - `llama-index-integrations/llms/llama-index-llms-bedrock-converse/tests/test_structured_output.py` (added, +51/-0) --- # PR #20769: Feat/bedrock structured outputs v2 - Repository: run-llama/llama_index - Author: akintunero - State: closed | merged: False - Link: https://github.com/run-llama/llama_index/pull/20769 ## Description (problem / solution / changelog) # Description AWS Bedrock launched native structured outputs in February 2026 via the `outputConfig.textFormat` parameter. This feature adds native support for structured outputs to the `llama-index-llms-bedrock-converse` integration, replacing the function calling workaround for supported models. ## Changes: - Add `BEDROCK_STRUCTURED_OUTPUT_SUPPORTED_MODELS` tuple with supported models (Claude 4.5+, Amazon Nova, DeepSeek, Mistral) - Add `is_bedrock_structured_output_supported()` function with region prefix support - Implement `structured_predict()` and `astructured_predict()` methods - Fall back to function calling for unsupported models (backward compatible) - Add 4 unit tests for model support detection ## Supported Models: - Claude 4.5+ (haiku, sonnet, opus) - Amazon Nova (pro, lite, micro) - DeepSeek R1 - Mistral Large Fixes #20703 ## New Package? - [ ] Yes - [x] No ## Version Bump? - [x] Yes - [ ] No ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update ## How Has This Been Tested? - [x] I added new unit tests to cover this change - [ ] I believe this change is already covered by existing unit tests ## Suggested Checklist: - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corres

llamaIndex2026-02-14 21:41:10

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

run-llama/llama_index#20703•Fetched 2026-04-08 00:31:22

View on GitHub

Comments

Participants

Timeline

Reactions

Author

ndgigliotti

Participants

ndgigliotti

Timeline (top)

referenced ×5cross-referenced ×2commented ×1subscribed ×1

AWS Bedrock launched native structured outputs in February 2026 via the outputConfig.textFormat parameter. LlamaIndex's llama-index-llms-bedrock-converse currently uses function calling as a workaround. This should be updated to use the native API for better reliability and performance.

Root Cause

Code Example

BEDROCK_STRUCTURED_OUTPUT_SUPPORTED_MODELS = (
    # Claude 4.5+
    "anthropic.claude-haiku-4-5-20251001-v1:0",
    "anthropic.claude-sonnet-4-5-20250929-v1:0",
    "anthropic.claude-opus-4-5-20251101-v1:0",
    "anthropic.claude-opus-4-6-v1",
    # + Qwen, OpenAI OSS, DeepSeek, Google Gemma, MiniMax, Mistral, Moonshot, NVIDIA
    # See: https://docs.aws.amazon.com/bedrock/latest/userguide/structured-output.html#structured-output-models
)

def is_bedrock_structured_output_supported(model_name: str) -> bool:
    return get_model_name(model_name) in BEDROCK_STRUCTURED_OUTPUT_SUPPORTED_MODELS

---

def structured_predict(self, output_cls, prompt, llm_kwargs=None, **prompt_args):
    llm_kwargs = llm_kwargs or {}

    # Fall back to function calling if not supported
    if (self.pydantic_program_mode != PydanticProgramMode.DEFAULT
        or not is_bedrock_structured_output_supported(self.model)):
        return super().structured_predict(output_cls, prompt, llm_kwargs, **prompt_args)

    # Use native structured outputs
    json_schema = output_cls.model_json_schema()
    messages = prompt.format_messages(**prompt_args)
    converse_messages, system_prompt = messages_to_converse_messages(messages, self.model)

    all_kwargs = self._get_all_kwargs(**llm_kwargs)
    all_kwargs["outputConfig"] = {
        "textFormat": {
            "type": "json_schema",
            "structure": {
                "jsonSchema": {
                    "schema": json.dumps(json_schema),
                    "name": output_cls.__name__,
                    "description": output_cls.__doc__ or f"Schema for {output_cls.__name__}"
                }
            }
        }
    }

    response = converse_with_retry(
        client=self._client,
        messages=converse_messages,
        system_prompt=system_prompt,
        system_prompt_caching=self.system_prompt_caching,
        max_retries=self.max_retries,
        stream=False,
        **all_kwargs,
    )

    blocks, _, _, _ = self._get_content_and_tool_calls(response)
    text_content = next((b.text for b in blocks if isinstance(b, TextBlock)), "")
    return output_cls.model_validate_json(text_content)

RAW_BUFFERClick to expand / collapse

Summary

Current Behavior

BedrockConverse inherits structured_predict() from the base LLM class, which:

Wraps Pydantic schemas in FunctionCallingProgram
Converts schemas to tool definitions
Forces the model to "call" a tool with tool_required=True
Extracts tool call parameters as structured output

Code path: BedrockConverse → LLM.structured_predict() → get_program_for_llm() → FunctionCallingProgram

Location: llama-index-core/llama_index/core/llms/llm.py:307-364

Problem

Function calling is designed for tool invocation, not data extraction:

Semantic mismatch (using tools for JSON extraction)
Unnecessary overhead (tool calling infrastructure)
Indirect validation (through tool parameters, not JSON grammar)
Additional latency (multiple conversion steps)
No guarantee of schema compliance

Solution

Use Bedrock's native outputConfig.textFormat API for supported models:

Guaranteed JSON schema compliance via token-level constrained generation
Direct API support without conversion overhead
Single-pass generation
Better performance and reliability

Proposed Changes

1. Add model support check (`utils.py`)

BEDROCK_STRUCTURED_OUTPUT_SUPPORTED_MODELS = (
    # Claude 4.5+
    "anthropic.claude-haiku-4-5-20251001-v1:0",
    "anthropic.claude-sonnet-4-5-20250929-v1:0",
    "anthropic.claude-opus-4-5-20251101-v1:0",
    "anthropic.claude-opus-4-6-v1",
    # + Qwen, OpenAI OSS, DeepSeek, Google Gemma, MiniMax, Mistral, Moonshot, NVIDIA
    # See: https://docs.aws.amazon.com/bedrock/latest/userguide/structured-output.html#structured-output-models
)

def is_bedrock_structured_output_supported(model_name: str) -> bool:
    return get_model_name(model_name) in BEDROCK_STRUCTURED_OUTPUT_SUPPORTED_MODELS

2. Override `structured_predict()` methods (`base.py`)

def structured_predict(self, output_cls, prompt, llm_kwargs=None, **prompt_args):
    llm_kwargs = llm_kwargs or {}

    # Fall back to function calling if not supported
    if (self.pydantic_program_mode != PydanticProgramMode.DEFAULT
        or not is_bedrock_structured_output_supported(self.model)):
        return super().structured_predict(output_cls, prompt, llm_kwargs, **prompt_args)

    # Use native structured outputs
    json_schema = output_cls.model_json_schema()
    messages = prompt.format_messages(**prompt_args)
    converse_messages, system_prompt = messages_to_converse_messages(messages, self.model)

    all_kwargs = self._get_all_kwargs(**llm_kwargs)
    all_kwargs["outputConfig"] = {
        "textFormat": {
            "type": "json_schema",
            "structure": {
                "jsonSchema": {
                    "schema": json.dumps(json_schema),
                    "name": output_cls.__name__,
                    "description": output_cls.__doc__ or f"Schema for {output_cls.__name__}"
                }
            }
        }
    }

    response = converse_with_retry(
        client=self._client,
        messages=converse_messages,
        system_prompt=system_prompt,
        system_prompt_caching=self.system_prompt_caching,
        max_retries=self.max_retries,
        stream=False,
        **all_kwargs,
    )

    blocks, _, _, _ = self._get_content_and_tool_calls(response)
    text_content = next((b.text for b in blocks if isinstance(b, TextBlock)), "")
    return output_cls.model_validate_json(text_content)

Also implement: astructured_predict(), stream_structured_predict(), astream_structured_predict()

3. Tests and docs

Unit tests for model support detection
Integration tests with Bedrock API
Fallback tests for unsupported models
README examples

References

AWS Announcement: https://aws.amazon.com/about-aws/whats-new/2026/02/structured-outputs-available-amazon-bedrock/
User Guide: https://docs.aws.amazon.com/bedrock/latest/userguide/structured-output.html
Similar implementations:
- Anthropic: llama-index-llms-anthropic/base.py:1040-1070
- OpenAI: llama-index-llms-openai/base.py:1097-1120

Notes

Falls back to function calling for unsupported models
No breaking changes to existing API
Incompatible with citations on Anthropic models (per AWS docs)
Handles region-prefixed model names (e.g., us.anthropic.claude-...)

Related to #18392

extent analysis

Fix Plan

1. Add model support check (`utils.py`)

BEDROCK_STRUCTURED_OUTPUT_SUPPORTED_MODELS = (
    # Claude 4.5+
    "anthropic.claude-haiku-4-5-20251001-v1:0",
    "anthropic.claude-sonnet-4-5-20250929-v1:0",
    "anthropic.claude-opus-4-5-20251101-v1:0",
    "anthropic.claude-opus-4-6-v1",
    # + Qwen, OpenAI OSS, DeepSeek, Google Gemma, MiniMax, Mistral, Moonshot, NVIDIA
    # See: https://docs.aws.amazon.com/bedrock/latest/userguide/structured-output.html#structured-output-models
)

def is_bedrock_structured_output_supported(model_name: str) -> bool:
    return get_model_name(model_name) in BEDROCK_STRUCTURED_OUTPUT_SUPPORTED_MODELS

2. Override `structured_predict()` methods (`base.py`)

def structured_predict(self, output_cls, prompt, llm_kwargs=None, **prompt_args):
    # ... (rest of the method remains the same)

def astructured_predict(self, output_cls, prompt, llm_kwargs=None, **prompt_args):
    # ... (rest of the method remains the same)

def stream_structured_predict(self, output_cls, prompt, llm_kwargs=None, **prompt_args):
    # ... (rest of the method remains the same)

def astream_structured_predict(self, output_cls, prompt, llm_kwargs=None, **prompt_args):
    # ... (rest of the method remains the same)

3. Implement `messages_to_converse_messages()` and `_get_all_kwargs()` functions

def messages_to_converse_messages(messages, model):
    #

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #ssr #installation #tensor shape #autograd error #API routing #API middleware #SSR setup #ISR setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

llamaIndex - ✅(Solved) Fix Feature Request: Native Structured Outputs for AWS Bedrock Converse API [2 pull requests, 1 comments, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Solution

PR fix notes

PR #20753: Feat/bedrock structured outputs

Description (problem / solution / changelog)

Description

Changes:

Supported Models:

New Package?

Version Bump?

Type of Change

How Has This Been Tested?

Suggested Checklist:

Changed files

PR #20769: Feat/bedrock structured outputs v2

Description (problem / solution / changelog)

Description

Changes:

Supported Models:

New Package?

Version Bump?

Type of Change

How Has This Been Tested?

Suggested Checklist:

Changed files

Code Example

Summary

Current Behavior

Problem

Solution

Proposed Changes

1. Add model support check (utils.py)

2. Override structured_predict() methods (base.py)

3. Tests and docs

References

Notes

extent analysis

Fix Plan

1. Add model support check (utils.py)

2. Override structured_predict() methods (base.py)

3. Implement messages_to_converse_messages() and _get_all_kwargs() functions

Still need to ship something?

RELATED_DISCOVERY

TRENDING

1. Add model support check (`utils.py`)

2. Override `structured_predict()` methods (`base.py`)

1. Add model support check (`utils.py`)

2. Override `structured_predict()` methods (`base.py`)

3. Implement `messages_to_converse_messages()` and `_get_all_kwargs()` functions