hermes - 💡(How to fix) Fix Question: 24GB VRAM local models for Kanban workers - real-world results? [1 participants]

tl;dr: Has anyone successfully run Hermes Kanban workers on local models with 24GB VRAM? Looking for real-world data before GPU upgrade.

Background

I currently run Hermes with 12GB VRAM. Testing local models for Kanban workers:

• qwen3.5:9b (6.6GB): Crashes after 28-91s, hallucinates bash commands instead of using tools
• qwen2.5:14b (9GB): Crashes after 12-33s, overwhelmed by system prompt
• gemma4:e4b (9.6GB): Same failures

All using context_length patch (Issue #24072 applies, but model capability is the blocker).

Root cause: Hermes's massive system prompt (+10K tokens, 30+ tool schemas, memory injection) overwhelms small models. Cloud models work fine.

The Question

I'm considering a 24GB GPU upgrade specifically for local Kanban agents. Based on #523 (Local Model Setup Skill), these fit:

Has anyone tested these with Kanban workers specifically?

What I need to know:

1. Do Qwen3-30B/32B reliably call tools (kanban_show, terminal, write_file, etc.) in complex chains?
2. Do they handle multi-step Kanban tasks or crash like 9B/14B models?
3. Are timeout patches needed (Issue #3404 - hardcoded 30s aux/45s compression timeout)?
4. Any specific config tweaks beyond context_length: 128000?

Environment

• Goal: Async Kanban agents with local models
• Current: 12GB VRAM → considering 24GB upgrade
• Use case: Research/data analysis tasks via analyst, researcher profiles
• Alternative: Keep cloud models (working, has costs)

Would appreciate any real-world experience before spending $1500 on a new GPU.

Related Issues

• #523: Local Model Setup Skill - model recommendations
• #3404: Configurable timeouts for local models
• #24072: Context length override (patch works for loading but not tool reliability)
• #2074: Ollama models not recognizing environment (similar problem)

If no one has tested this, I can volunteer to be the guinea pig and document results if there's interest.