qwen3.6:35b-a3b-coding-nvfp4 model file has corrupted K-projection weights for early linear-attention layers

What is the issue?

The NVFP4 packaging of qwen3.6:35b-a3b-coding-nvfp4 shipped from registry.ollama.ai/library/qwen3.6 has the K-projection portion of linear_attn.in_proj_qkv.weight entirely zeroed out for layer 0 and mostly zeroed for layer 1 — making coherent generation impossible regardless of any runner-side fix.

This is a model-packaging issue, not a runtime bug. The mlx-community/Qwen3.6-35B-A3B-4bit model (same architecture, same source weights, MLX 4-bit affine quant instead of NVFP4) has the expected natural sparsity (3-5%) in the same regions.

Reproducer

The linear_attn.in_proj_qkv.weight tensor is laid out per HF transformers as [Q rows 0..2047 | K rows 2048..4095 | V rows 4096..8191]. A direct read of the safetensors blob shows:

layer  0  K-zero-rows = 2048 / 2048   (100% — entire K projection is zero!)
layer  1  K-zero-rows = 1797 / 2048   ( 88%)
layer  2  K-zero-rows =   57 / 2048   (  3%)
layer  3  ... (full attention layer, no in_proj_qkv)
layer  4+ K-zero-rows ≤ 2 / 2048      (~natural sparsity)

For comparison, on mlx-community/Qwen3.6-35B-A3B-4bit (4-bit affine, same architecture):

layer 0  K-zero-rows =  56 / 2048
layer 1  K-zero-rows = 102 / 2048
layer 2  K-zero-rows =  56 / 2048

The mlx-community model generates coherent text via mlx-lm generate (top next token for "What is 2+2?" → "Here", correctly leading into a thinking trace). The Ollama NVFP4 model cannot generate coherent text under any code fix because layers 0 and 1 of the linear-attention path produce K=0 → linear attention output = 0 → the residual stream loses the entire linear-attention contribution from those layers.

Verification commands

# Reproduce against the cached blob (replace the SHA with whichever blob holds the layer-0 in_proj_qkv tensor):
import struct, json, numpy as np
blob = "/Users/<you>/.ollama/models/blobs/sha256-..."   # blob carrying layers.0.linear_attn.in_proj_qkv
with open(blob, 'rb') as f:
    hsz = struct.unpack('<Q', f.read(8))[0]
    hdr = json.loads(f.read(hsz))
    body = 8 + hsz
    info = hdr['language_model.model.layers.0.linear_attn.in_proj_qkv.weight']
    s, e = info['data_offsets']
    f.seek(body + s)
    arr = np.frombuffer(f.read(e-s), dtype=np.uint32).reshape(info['shape'])
    print('K rows zero:', sum(1 for r in arr[2048:4096] if not r.any()), '/ 2048')
# → 2048 / 2048 on the broken model

Likely cause

The conversion from BF16 (or the source FP8/BF16 RedHatAI NVFP4) to Ollama's NVFP4 packaging picked a global scale per tensor that was too large to represent the small magnitudes in early-layer K projections — they all rounded to the FP4 zero codepoint.

For verification: the scales tensor for in_proj_qkv is non-zero across all rows (max-row-abs ≈ 1144 on K rows for layer 0), but the underlying FP4 nibbles are all zero, so dequantization yields zero regardless of the (non-zero) scale. That fingerprint matches "small-magnitude rows quantized below FP4 representable range" rather than "block of weights deleted in I/O".

Suggested fixes

1. Re-quantize with per-tensor (or per-row block) scales chosen to preserve the K projection's dynamic range — or fall back to a higher-precision quant for early-layer linear-attention weights.
2. Check the same artefact for the mxfp8 and other variants in the library (qwen3.6:35b-a3b-coding-mxfp8, qwen3.6:35b-a3b-nvfp4, qwen3.6:27b-coding-nvfp4).
3. Re-run the conversion using the per-tensor quant overrides path (e.g. matching the pattern in #15760, where mlp.gate and mlp.shared_expert_gate get an 8-bit override) — applying an 8-bit override (or skipping quant entirely) to linear_attn.in_proj_qkv would avoid the zero-collapse.

Related

• #15865 — independent recurrent-state precision bug in gated_delta_step (the kernel cast state to InT instead of StT). With only that fix and without this corruption, the model reaches the sampler but the very first decode step gets wrong logits because layer-0 linear attention contributes zero. Together, both issues explain the "broken Qwen3.6 NVFP4 generation on Ollama".
• #15822, #15834, #15700 — user reports of broken generation on this model family on macOS; this issue likely explains the coding-nvfp4 sub-cases.
• mlx-community / Qwen3.6-35B-A3B-4bit on HF works correctly via mlx-lm, demonstrating the architecture is otherwise sound.

Environment

• Ollama version: v0.22.0 (as of 2026-04-28)
• OS: macOS 26.4.1, Apple M4 Max
• Source model: registry.ollama.ai/library/qwen3.6:35b-a3b-coding-nvfp4 (digest sha256:cd2692a833e6...)