Feature Description

Native support for pluggable serialization in LlamaIndex agent and query pipelines, with ULMEN as a drop-in alternative to JSON.

LlamaIndex pipelines serialize agent messages, tool calls, and query results as JSON by default. At scale this creates a measurable and expensive problem.

I measured this on production workloads: ~44% of tokens in typical agent payloads are pure JSON syntax overhead - brackets, repeated key names, whitespace - before any reasoning begins.

ULMEN is a serialization engine built specifically for LLM agent pipelines.

Benchmarks on NVIDIA Tesla T4:

<img width="1861" height="1281" alt="Image" src="https://github.com/user-attachments/assets/84ef014e-5ca5-4426-be5f-8e6b1c539871" />

Beyond token reduction, ULMEN adds a Semantic Firewall that validates agent state transitions before they reach the LLM:

• Rejects orphaned tool calls
• Catches invalid step transitions
• Validates enum states
• Structured errors instead of silent failures

Implementation would be fully opt-in:

Current

agent = ReActAgent.from_tools(tools)

Proposed

agent = ReActAgent.from_tools( tools, serializer="ulmen" )

Drop-in Python/Rust. No schema compilation. Pure Python fallback if Rust unavailable. Free under $10M revenue (BSL license).

Reproducible benchmark notebook: github.com/makroumi/ulmen

Reason

Current alternatives don't solve the problem:

• orjson: faster serialization but identical token count. Context window overhead unchanged.

• MessagePack: smaller wire format but tokens are unchanged. No semantic validation.

• Manual context trimming: lossy, custom logic required per pipeline, not systematic.

LlamaIndex's RAG and agent pipelines are particularly exposed because:

1. Retrieved chunks are serialized as JSON adding syntax overhead to every chunk
2. Tool call results repeat key names across every agent turn
3. Multi-step query pipelines compound the overhead at every step
4. No validation catches broken tool statesbefore they reach the LLM

At 10M agent loops on GPT-4o this overhead costs approximately $59K. Not a rounding error.

Value of Feature

1. Direct cost reduction for every LlamaIndex user running agents at scale.$59K saved per 10M loops on GPT-4o pricing.

2. More reasoning capacity per context window.44% fewer tokens on syntax means 44% more room for actual intelligence.

3. Fewer hallucinations from broken agent state. The Semantic Firewall catches invalid states before they corrupt LLM context.

4. Faster pipeline execution. 3x faster deserialization than orjson on heavily nested agent payloads.

5. Non-breaking for existing users. Fully opt-in. Zero migration required.

As LlamaIndex pipelines grow more complex and run at higher scale, serialization format becomes a structural cost driver. This addresses it systematically rather than one pipeline at a time.

Happy to submit a PR implementing the serializer nterface once maintainers confirm the preferred integration approach.

github.com/makroumi/ulmen