ollama - 💡(How to fix) Fix Please add qwen3.5:122b-a10b-q8_0 quantization to model registry [1 participants]

Request

Please add a q8_0 quantization tag for qwen3.5:122b-a10b. The current registry only has q4_K_M (81 GB) under that model.

Context

I run a small document-processing business doing structured data extraction from business documents into spreadsheets. The pipeline emits deterministic JSON via structured/constrained generation, where small accuracy regressions show up immediately as wrong numbers in the downstream output. In my testing, Q4 vs Q8 is a measurably non-trivial accuracy gap on this workload — Q4 produces enough errors per document to be unusable in production. Q6 is the floor I can tolerate.

Current setup:

• M2 Ultra 128 GB → running Qwen3.5-122B-A10B at Q6_K via llama.cpp today (Q8 won't fit alongside the rest of my services)
• M3 Ultra 256 GB on order, specifically to run multiple concurrent Q8 workers for higher accuracy and parallelism
• I'd like to evaluate Ollama's MLX backend on the new machine once it arrives

Without a Q8 tag in the registry I can't run a fair head-to-head against my current llama.cpp Q8 baseline. The workarounds (manual GGUF build + Modelfile import) lose the registry auto-update story, which is most of the value of switching to Ollama in the first place.

Why this might be a good signal for Ollama

122B-A10B at Q8 is one of the more demanding sustained-throughput workloads targetable on Apple Silicon at this hardware tier. It stresses MLX's prompt caching, KV cache sizing for hybrid attention, and the recent Qwen3.5 thinking-token fixes from v0.19 all at once. If it runs cleanly at this size point on M3 Ultra, it validates the MLX backend for the whole "single-machine production inference on Mac Studio" use case.

Evaluation plan once the hardware lands

3–5 document benchmark comparing wall-clock time, output JSON byte-equality vs current llama.cpp Q8 baseline, thinking-token handling under structured output, and multi-worker behavior (4 concurrent extractions). Happy to share results here if useful.

Thanks.