Voodoo Quant claims 95% KLD improvement over Unsloth Dynamic 2.0 on Qwen3.5 models
A developer released two new GGUF quantizations of Qwen3.5 0.8B and 2B using a technique called Voodoo Quant, which optimizes mixed precision by assigning higher precision to more important parts of the model. The author claims Voodoo Quant beats Unsloth Dynamic 2.0 by 95% in Kullback-Leibler divergence (KLD). The quantized models are available on Hugging Face.
Entities
Related
User reports Qwen3.6 35B-A3B model improves with Q8_0 CPU quantization
A user on Reddit reported that switching from Q4_K_M on GPU to Q8_0 on CPU significantly improved the performance of the Qwen3.6 35B-A3B model for a complex coding task. The user noted the model 'punches far above its weight' and found the quality gain worth the slowdown.
User seeks to extend Qwen 3.6 27B context window beyond 100k tokens
A user reports running Qwen 3.6 27B (Q8_0) at 100k context length but finds reliability insufficient. They ask the community for techniques beyond KV cache quantization to improve stability at longer contexts.
User benchmarks 4x RTX 5060 Ti with SGLang for Qwen3.6 27B, finds better concurrency than vLLM
A user shared benchmark results showing that SGLang handles higher concurrency better than vLLM when running Qwen3.6 27B (INT8 with bf16 KV cache) on a 4x RTX 5060 Ti (64GB VRAM) setup. The test achieved 200 successful requests at 8 concurrency over 348.87 seconds, processing 61,870 input tokens and generating 44,525 tokens. This provides a practical reference for others considering multi-GPU configurations with these consumer cards.
Turboquant v0.3.0 fixes silent FP16 precision bug on Tesla P100 in llama.cpp
A three-line fix in turboquant v0.3.0 corrects a long-standing bug where llama.cpp's CUDA code forced FP16 math on Tesla P100 GPUs, despite the P100 having fast FP16 hardware. The fix restores correct precision and performance for P100 users running llama.cpp.
↑ Updated Sun, Jul 12, 2026, 05:41 AM — Turboquant v0.3.0 released with fix for P100 FP16 bug.
Developer adds Nemotron Puzzle 75B support to mlx-lm, benchmarks 4-bit vs 5-bit quantization on M2 Max
A developer contributed native `nemotron_h_puzzle` support to mlx-lm (PR #1535) and ran benchmarks on a 64GB M2 Max. Comparing 4-bit vs 5-bit expert quantization (both with 6-bit dense layers, BF16 output head, group size 64), the 4-bit variant used 42.03 GiB checkpoint (49.68 GB peak memory) and achieved 14.27 tok/s, scoring 24/30 on local task checks, while 5-bit used 49.88 GiB (58.12 GB peak) at 10.53 tok/s scoring 21/30.
