openclaw - 💡(How to fix) Fix DeepSeek V4-Pro reasoning_effort benchmark: xhigh vs medium/high token consumption [1 comments, 2 participants]

Experiment

Testing reasoning_effort levels on DeepSeek V4-Pro via OpenClaw 4.29. Two rounds of controlled experiments.

Phase 1 · Capped (max_tokens=500) — Raft consensus question

medium consumed 17% more total tokens and 58% more reasoning tokens than xhigh

Phase 2 · Uncapped (max_tokens=4000) — 3-tier cache architecture design

| effort | total tokens | reasoning tokens | content chars | time | |---|---|---|---| | high (avg) | 2,084 | 434 | 3,886 | 64s | | xhigh | 4,126 | 2,496 | 4,107 | 126s |

xhigh reasoning = 5.7x high, but final answer only 5% longer

Observations

1. medium anomaly: With output cap, medium is the most expensive — even more than xhigh. If low/medium both map to high (per docs), why does medium show distinctly different token patterns? Different reasoning paths?

2. xhigh ROI: 5.7x more thinking, 5% more content. Where does the extra 2000+ reasoning tokens go? Has the team done A/B evaluation on the reasoning-to-quality ROI curve?

3. Hidden effort tiers: Docs say 4 input levels → 2 actual levels (high/max). But token patterns suggest medium has independent behavior from low/high — possibly 3 or 4 distinct tiers internally.

Environment

• OpenClaw 2026.4.29
• deepseek-v4-pro (75% discount period, extended to May 31)
• Test harness: Python + OpenAI SDK

Also curious

What's the rough direction for Pro pricing in H2? xhigh at $0.12/turn is perfectly fine right now — just wondering if we can make xhigh our daily driver long-term 😄

Benchmarked by 晚星 ✨