User benchmarks show 2x throughput gain from parallel agents on RTX 5090 with Qwen 3.6 35B
A Reddit user benchmarked LM Studio with Qwen 3.6 35B on an RTX 5090, finding that running at least 4 parallel agents doubles tokens/second compared to a single agent. The test with 8 configured parallel tasks showed significant throughput improvement, suggesting developers using agent frameworks like Open Code should parallelize to maximize performance.
Entities
Related
User benchmarks 4x RTX 5060 Ti with SGLang for Qwen3.6 27B, finds better concurrency than vLLM
A user shared benchmark results showing that SGLang handles higher concurrency better than vLLM when running Qwen3.6 27B (INT8 with bf16 KV cache) on a 4x RTX 5060 Ti (64GB VRAM) setup. The test achieved 200 successful requests at 8 concurrency over 348.87 seconds, processing 61,870 input tokens and generating 44,525 tokens. This provides a practical reference for others considering multi-GPU configurations with these consumer cards.
User seeks advice on adding second cheap GPU to run larger local models
A Reddit user running Gemma 4 26B-A4B at 12-15 t/s on an RTX 3060 (12 GB) finds the model insufficiently intelligent and wants to upgrade to a 31B model, which runs at only 1.5 t/s. They ask the community about the benefits of adding a second cheap GPU to improve performance for local LLM inference.
Rust+CUDA LLM inference engine for RTX 5090, optimized with NVFP4, MoE, and speculative decoding
A from-scratch LLM inference engine written in Rust and CUDA, specifically tuned for the RTX 5090 Laptop GPU (Blackwell sm_120a). It supports NVFP4 quantization, mixture-of-experts (MoE), and multi-token prediction (MTP) speculative decoding, achieving up to 1.6x speedup over llama.cpp on target models. Designed for developers who need maximum inference performance on a single high-end laptop GPU.
User benchmarks AMD EPYC 9374F for LLM inference, finds 48-thread sweet spot
A user replaced their EPYC 9135 with a cheap 9374F (8 CCDs) for LLM inference. Initial benchmarks showed no decoding advantage until they used 48 threads; 64 or 32 threads performed worse than the 9135 in some scenarios. The 9374F is worse for gaming.
Community compares local LLMs for agentic workflows using tool-eval-bench
A GitHub user published an interactive comparison report evaluating local LLMs for agentic workflows, using the tool-eval-bench benchmark (84 scenarios, 16 categories, 8 trials). The report targets single DGX Spark or other 96-128GB rigs and covers multi-turn tool orchestration, function calling, and autonomous planning as exercised by Hermes Agent.