vllm - ✅(Solved) Fix [Bug]: gemma4_utils._parse_tool_arguments truncates string values containing internal quotes [1 pull requests, 1 participants]

vllm2026-04-06 08:49:24

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

vllm-project/vllm#39069•Fetched 2026-04-08 02:52:36

View on GitHub

Comments

Participants

Timeline

Reactions

Author

Rih0z

Participants

Rih0z

Timeline (top)

referenced ×4cross-referenced ×2subscribed ×1

Root Cause

Root cause: The function replaces <|"|> delimiters with ", then uses a fallback regex [^"]* that stops at the first internal quote character.

Fix Action

Fixed

Fixed by PR: [Bugfix] Fix gemma4_utils._parse_tool_arguments truncating strings with internal quotes (https://github.com/vllm-project/vllm/pull/39070)

PR fix notes

PR #39070: [Bugfix] Fix gemma4_utils._parse_tool_arguments truncating strings with internal quotes

Repository: vllm-project/vllm
Author: Rih0z
State: open | merged: False
Link: https://github.com/vllm-project/vllm/pull/39070

Description (problem / solution / changelog)

Purpose

Fix vllm/tool_parsers/gemma4_utils._parse_tool_arguments() silently truncating string values that contain " (double quotes). This breaks offline inference tool call parsing for any tool that passes structured text (HTML, code, JSON) as parameters.

This is not duplicating an existing PR. Checked: #38992 (streaming fix only), #39027 (chat template/reasoning, does not touch gemma4_utils.py), #38909 (streaming HTML duplication). No existing PR modifies _parse_tool_arguments.

Fixes #39069

The Bug

_parse_tool_arguments() replaces <|"|> delimiters with ", then uses regex [^"]* which stops at the first internal quote:

# Before: content:<|"|>She said "hello" loudly<|"|>
# After replace: content:"She said "hello" loudly"
# Regex [^"]* captures: "She said " → TRUNCATED

Measured output: {'content': 'She said '} instead of {'content': 'She said "hello" loudly'}

The Fix

The API server parser (gemma4_tool_parser.py) already has a correct implementation (_parse_gemma4_args) that handles <|"|> delimiters natively without replacement. This PR:

Moves _parse_gemma4_args, _parse_gemma4_array, _parse_gemma4_value from gemma4_tool_parser.py into gemma4_utils.py as public functions (parse_gemma4_args, etc.)
Updates gemma4_tool_parser.py to import from gemma4_utils.py (no behavior change for API server path)
Replaces the broken _parse_tool_arguments with a wrapper around parse_gemma4_args (backward compatible — still returns dict[str, str])
Adds tests/tool_parsers/test_gemma4_utils.py with regression tests for internal quotes, HTML attributes, braces, and code content
Adds test cases to existing test_gemma4_tool_parser.py for the same edge cases

Test commands run and results

# Direct function verification (vLLM not installed — pure Python test):
python -c "... _parse_tool_arguments('content:<|"|>She said \"hello\" loudly<|"|>') ..."
# Result: {'content': 'She said "hello" loudly'} ✅ (was: {'content': 'She said '})

python -c "... _parse_tool_arguments('path:<|"|>out.html<|"|>,content:<|"|><div class=\"main\">hello</div><|"|>') ..."  
# Result: {'path': 'out.html', 'content': '<div class="main">hello</div>'} ✅ (was: {'content': '<div class='})

All 8 manual tests passed (internal quotes, HTML attributes, braces, code, simple string, multi-value, empty, bare numeric).

AI Assistance Disclosure

AI assistance (Claude) was used for code analysis, identifying the root cause, writing tests, and drafting this PR. All changes were reviewed and verified by a human.

🤖 Generated with Claude Code

Co-authored-by: Claude Opus 4.6 (1M context) [email protected]

Changed files

tests/tool_parsers/test_gemma4_tool_parser.py (modified, +30/-0)
tests/tool_parsers/test_gemma4_utils.py (added, +142/-0)
vllm/tool_parsers/gemma4_tool_parser.py (modified, +6/-201)
vllm/tool_parsers/gemma4_utils.py (modified, +220/-28)

Code Example

N/A — This bug was identified through source code analysis of vllm main branch
(commit c5e3454e5) and confirmed by running the affected function in isolation.
vLLM is not installed in this environment. The bug is in pure Python parsing
logic that can be reproduced without a GPU or model.

---

import regex as re

# Extracted from vllm/tool_parsers/gemma4_utils.py (L52-90)
_ESCAPE_TOKEN = '<|"|>'

def _parse_tool_arguments(args_str):
    if not args_str or not args_str.strip():
        return {}
    cleaned = args_str.replace(_ESCAPE_TOKEN, '"')
    import json
    try:
        parsed = json.loads("{" + cleaned + "}")
        return {k: str(v) if not isinstance(v, str) else v for k, v in parsed.items()}
    except (json.JSONDecodeError, ValueError):
        pass
    arguments = {}
    for key, value in re.findall(r'(\w+):\s*"([^"]*)"', cleaned):
        arguments[key] = value
    if not arguments:
        for key, value in re.findall(r'(\w+):\s*([^,}]+)', args_str):
            arguments[key] = value.strip().strip('"').replace(_ESCAPE_TOKEN, '')
    return arguments

# Test 1: String with internal quotes — TRUNCATED
result = _parse_tool_arguments('content:<|"|>She said "hello" loudly<|"|>')
print(f"Test 1: {result}")
# Actual:   {'content': 'She said '}
# Expected: {'content': 'She said "hello" loudly'}

# Test 2: HTML attribute with quotes — TRUNCATED
result = _parse_tool_arguments('path:<|"|>out.html<|"|>,content:<|"|><div class="main">hello</div><|"|>')
print(f"Test 2: {result}")
# Actual:   {'path': 'out.html', 'content': '<div class='}
# Expected: {'path': 'out.html', 'content': '<div class="main">hello</div>'}

---

Test 1: {'content': 'She said '}
Test 2: {'path': 'out.html', 'content': '<div class='}

---

Test 1: {'content': 'She said "hello" loudly'}
Test 2: {'path': 'out.html', 'content': '<div class="main">hello</div>'}

RAW_BUFFERClick to expand / collapse

Your current environment

<details> <summary>The output of <code>python collect_env.py</code></summary>

N/A — This bug was identified through source code analysis of vllm main branch
(commit c5e3454e5) and confirmed by running the affected function in isolation.
vLLM is not installed in this environment. The bug is in pure Python parsing
logic that can be reproduced without a GPU or model.

</details>

🐛 Describe the bug

vllm/tool_parsers/gemma4_utils._parse_tool_arguments() (the offline inference tool call parser) truncates string values when they contain " (double quotes).

Root cause: The function replaces <|"|> delimiters with ", then uses a fallback regex [^"]* that stops at the first internal quote character.

Affected file: vllm/tool_parsers/gemma4_utils.py Not affected: vllm/tool_parsers/gemma4_tool_parser.py (the API server parser handles <|"|> delimiters natively without replacement, so it works correctly)

Minimal reproduction

import regex as re

# Extracted from vllm/tool_parsers/gemma4_utils.py (L52-90)
_ESCAPE_TOKEN = '<|"|>'

def _parse_tool_arguments(args_str):
    if not args_str or not args_str.strip():
        return {}
    cleaned = args_str.replace(_ESCAPE_TOKEN, '"')
    import json
    try:
        parsed = json.loads("{" + cleaned + "}")
        return {k: str(v) if not isinstance(v, str) else v for k, v in parsed.items()}
    except (json.JSONDecodeError, ValueError):
        pass
    arguments = {}
    for key, value in re.findall(r'(\w+):\s*"([^"]*)"', cleaned):
        arguments[key] = value
    if not arguments:
        for key, value in re.findall(r'(\w+):\s*([^,}]+)', args_str):
            arguments[key] = value.strip().strip('"').replace(_ESCAPE_TOKEN, '')
    return arguments

# Test 1: String with internal quotes — TRUNCATED
result = _parse_tool_arguments('content:<|"|>She said "hello" loudly<|"|>')
print(f"Test 1: {result}")
# Actual:   {'content': 'She said '}
# Expected: {'content': 'She said "hello" loudly'}

# Test 2: HTML attribute with quotes — TRUNCATED
result = _parse_tool_arguments('path:<|"|>out.html<|"|>,content:<|"|><div class="main">hello</div><|"|>')
print(f"Test 2: {result}")
# Actual:   {'path': 'out.html', 'content': '<div class='}
# Expected: {'path': 'out.html', 'content': '<div class="main">hello</div>'}

Actual output (measured)

Test 1: {'content': 'She said '}
Test 2: {'path': 'out.html', 'content': '<div class='}

Expected output

Test 1: {'content': 'She said "hello" loudly'}
Test 2: {'path': 'out.html', 'content': '<div class="main">hello</div>'}

Execution trace

args_str.replace(<|"|>, ") converts content:<|"|>She said "hello" loudly<|"|> to content:"She said "hello" loudly"
json.loads('{content:"She said "hello" loudly"}') fails — unquoted key
Fallback regex r'(\w+):\s*"([^"]*)"' matches content:"She said " — [^"]* stops at the first internal ", capturing only She said

Suggested fix

The API server parser already has a correct implementation — _parse_gemma4_args() in gemma4_tool_parser.py — that handles <|"|> delimiters natively without replacement and correctly preserves string contents including internal ", {, }.

Proposal: Refactor _parse_gemma4_args into a shared location (e.g., gemma4_utils.py itself) and use it from both gemma4_tool_parser.py and gemma4_utils.py, replacing the current _parse_tool_arguments.

The llama.cpp project solved the same class of Gemma 4 parser bugs in ggml-org/llama.cpp#21418 by creating a dedicated parser that handles the custom format natively.

I'm willing to submit a PR for this fix. AI assistance was used in the analysis and will be disclosed in the PR per AGENTS.md policy.

Related issues/PRs

#39043 — vLLM + Gemma 4 tool calling problems (open, no fix)
#38992 — Streaming partial delimiter fix (merged, gemma4_tool_parser.py only)
#39027 — Multi-turn/reasoning rewrite (draft, does not touch gemma4_utils.py)

extent analysis

TL;DR

Refactor the _parse_gemma4_args function from gemma4_tool_parser.py into a shared location and use it in gemma4_utils.py to replace the current _parse_tool_arguments function.

Guidance

Identify the correct implementation of the parser in gemma4_tool_parser.py and refactor it into a shared utility function.
Replace the current _parse_tool_arguments function in gemma4_utils.py with the refactored shared function.
Verify that the new implementation correctly handles string values with internal quotes and other edge cases.
Test the changes with the provided test cases to ensure the expected output is produced.

Example

# Refactored shared function in gemma4_utils.py
def parse_gemma4_args(args_str):
    # Implementation from gemma4_tool_parser.py
    # Handle <|"|> delimiters natively without replacement
    # ...

# Replace the current _parse_tool_arguments function
def _parse_tool_arguments(args_str):
    return parse_gemma4_args(args_str)

Notes

The proposed fix is based on the correct implementation already present in gemma4_tool_parser.py. By refactoring this implementation into a shared location, we can ensure consistency and accuracy in parsing Gemma 4 tool arguments.

Recommendation

Apply the proposed workaround by refactoring the _parse_gemma4_args function and replacing the current _parse_tool_arguments function. This should resolve the issue with truncating string values containing internal quotes.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #API routing #API middleware #SSR setup #ISR setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.