User reports issues loading large MoE models after adding second RTX 3060
A user with dual RTX 3060 GPUs found that large MoE models like Qwen3.5-122B-A10B fail to load, while dense models work. After troubleshooting, they discovered that limiting VRAM usage to 12 layers allows the 122B model to load, and a PC restart plus GPU priority adjustment fixed the 35B model.
Entities
Related
User seeks advice on adding second cheap GPU to run larger local models
A Reddit user running Gemma 4 26B-A4B at 12-15 t/s on an RTX 3060 (12 GB) finds the model insufficiently intelligent and wants to upgrade to a 31B model, which runs at only 1.5 t/s. They ask the community about the benefits of adding a second cheap GPU to improve performance for local LLM inference.
User runs 100B+ MoE LLMs on low-end laptop using NVMe swap and Q3 quantization
A Reddit user with a low-spec laptop (i7-8750H, 20GB RAM, GTX 1050 4GB) reports successfully running 100B+ parameter MoE models by offloading parameters to a Samsung NVMe SSD via mmap, using Q3 quantization and quantized KV cache (Q4_0). They note that dense models are unusable on their hardware, but MoE models work with experts offloaded to CPU.
User seeks advice on upgrading dual 3090 setup to run 100-110GB models like DeepSeek V4 Flash
A Reddit user with a dual RTX 3090 setup is looking to upgrade to run larger models (100-110GB+), specifically a usable quant of DeepSeek V4 Flash. They are considering modded 48GB RTX 4090s, RTX A6000s, or RTX 5090s, and want to add to their existing 3090s rather than replace them. The post seeks community input on hybrid GPU configurations.
Community debate: MoE vs dense models — Qwen 3.5 122B example
A Reddit post challenges the common sentiment that a 122B MoE model with 10B active parameters is equivalent to a dense 10B model, arguing that router effectiveness makes MoE more capable. The post questions why providers would release MoE models if they offered no advantage over dense models.
Community shares llama-server configs for 24GB GPUs
A Reddit thread collects proven llama-server startup configurations for 24GB VRAM GPUs (RTX 3090, 7900XTX, RTX 4090). Users are asked to share commands that maximize VRAM usage and provide at least 200,000 tokens KV cache, along with system RAM, OS, and CPU details.