vllm - 💡(How to fix) Fix [Usage]: Unable to get any qwen models running on RTX 5090 [1 participants]

Your current environment

I've tried multiple models, ones I know work on my machine using other applications like ollama, or lmstudio:

• Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8
• Qwen/Qwen3.5-35B-A3B (works fine with LM Studio lms load qwen/qwen3.5-35b-a3b --context-length 81920)
• Qwen/Qwen3.6-27B-FP8

An example of serving a model:

vllm serve Qwen3-Coder-30B-A3B-Instruct-FP8 \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 16384 \
  --dtype auto \
  --gpu-memory-utilization 0.95 \
  --served-model-name Qwen3-Coder-30B-A3B-Instruct-FP8 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser qwen3 \
  --language-model-only

I will either get a OOM or a "No available memory for the cache blocks." no matter how much I lower the gpu memory (that causes issues with claude code and it's context).

At this point I'm just looking to the community for a working code model on the 5090, and possibly confirmation on working claude code cli usage. All 5090 issues seem to be made in December and since that's a long time in the industry I just decided to create an issue.

[The output of `python collect_env.py`](failed: Connection timed out.)

I would run collecting env normally but this is just a question for anyone else who has it running in a 5090.

How would you like to use vllm

Aboe

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.