Issue Station
Sign In
Tools hereToolbox

vllm - 💡(How to fix) Fix [Bug]: 使用swift rollout启动vllm，推理结果乱码 [1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#38349•Fetched 2026-04-08 01:41:53
View on GitHub
Comments
0
Participants
1
Timeline
1
Reactions
0
Author
Aristotle-wfhAristotle-wfh
Participants
Aristotle-wfhAristotle-wfh
Timeline (top)
labeled ×1

Code Example

Your output of `python collect_env.py` here
RAW_BUFFERClick to expand / collapse

Your current environment

<details> <summary>The output of <code>python collect_env.py</code></summary> 
Your output of `python collect_env.py` here
</details>

🐛 Describe the bug

1、启动脚本 swift rollout
--model $MODEL_PATH
--model_type qwen3_vl
--host 127.0.0.1
--port 8001
--vllm_tensor_parallel_size 1
--vllm_data_parallel_size 1

2、http请求返回结果乱码 [ { "response": { "model": "checkpoint-56", "choices": [ { "index": 0, "message": { "role": "assistant", "content": " inmates Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl")] Ctrl", "tool_calls": null, "reasoning_content": null }, "finish_reason": "length", "logprobs": null, "token_ids": null } ], "usage": { "prompt_tokens": 40, "completion_tokens": 64, "total_tokens": 104 }, "id": "1", "object": "chat.completion", "created": 1774597539, "prompt_token_ids": null, "images_size": null }, "messages": null, "response_token_ids": [], "response_loss_mask": [], "rollout_infos": {}, "rollout_logprobs": [] } ]

3、疑问 使用vllm serve启动服务，http请求结果正常

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

The fix involves setting the correct encoding for the HTTP response.

  • Check the swift rollout script and ensure that the encoding is set to UTF-8.
  • Modify the script to include the --encoding or --charset option, if available.
  • If the script does not support encoding options, consider using a proxy server or a middleware to set the encoding.

Example code snippet in Python using the requests library:

import requests

# Set the encoding to UTF-8
response = requests.post('http://127.0.0.1:8001', json={'prompt': 'your prompt'}, headers={'Content-Type': 'application/json; charset=utf-8'})

# Ensure the response encoding is set to UTF-8
response.encoding = 'utf-8'

print(response.json())

Alternatively, you can use the chardet library to detect the encoding of the response content:

import chardet

# Detect the encoding of the response content
detected_encoding = chardet.detect(response.content)['encoding']

# Set the response encoding
response.encoding = detected_encoding

print(response.json())

Verification

To verify that the fix worked, send an HTTP request to the service and check that the response is no longer garbled. You can use tools like curl or a web browser to test the service.

Example using curl:

curl -X POST -H "Content-Type: application/json; charset=utf-8" -d '{"prompt": "your prompt"}' http://127.0.0.1:8001

Extra Tips

  • Ensure that the client and server agree on the encoding to avoid character corruption.
  • Consider using a consistent encoding throughout your application to simplify debugging and maintenance.
  • If you're using a framework or library to handle HTTP requests, consult the documentation to determine how to set the encoding.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#cache error#pipeline error#runtime error#dependency conflict#environment setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING