User seeks help tuning llama-server cache on Strix Halo for Qwen 3.5 122B
A user on Reddit reports performance issues with large models (e.g., Qwen 3.5 122B) on a Strix Halo system, where a full cache miss at 100k context causes 10-20 minutes of prompt processing time. They have configured --cache-ram 16384 to increase available VRAM for cache, but seek further tuning advice.
Entities
Related
User seeks to extend Qwen 3.6 27B context window beyond 100k tokens
A user reports running Qwen 3.6 27B (Q8_0) at 100k context length but finds reliability insufficient. They ask the community for techniques beyond KV cache quantization to improve stability at longer contexts.
Community shares llama-server configs for 24GB GPUs
A Reddit thread collects proven llama-server startup configurations for 24GB VRAM GPUs (RTX 3090, 7900XTX, RTX 4090). Users are asked to share commands that maximize VRAM usage and provide at least 200,000 tokens KV cache, along with system RAM, OS, and CPU details.
Community discusses VRAM requirements and next upgrade from Qwen 3.6 27B
A Reddit user asks how much VRAM is needed and which model is the next major upgrade from Qwen 3.6 27B as of July 2026. The post reflects ongoing community interest in balancing model quality with hardware constraints.
User reports issues loading large MoE models after adding second RTX 3060
A user with dual RTX 3060 GPUs found that large MoE models like Qwen3.5-122B-A10B fail to load, while dense models work. After troubleshooting, they discovered that limiting VRAM usage to 12 layers allows the 122B model to load, and a PC restart plus GPU priority adjustment fixed the 35B model.
User seeks advice on adding second cheap GPU to run larger local models
A Reddit user running Gemma 4 26B-A4B at 12-15 t/s on an RTX 3060 (12 GB) finds the model insufficiently intelligent and wants to upgrade to a 31B model, which runs at only 1.5 t/s. They ask the community about the benefits of adding a second cheap GPU to improve performance for local LLM inference.