Community analysis: OSCAR compression beats TurboQuant with 2.28 effective bits per KV element
A community analysis compares two KV cache compression methods, OSCAR and TurboQuant, concluding that OSCAR achieves 2.28 effective bits per element versus TurboQuant's higher effective bits due to reliance on a 1-bit residual corrector. OSCAR's hybrid three-segment topology enables ultra-lean compression.
Entities
Related
Two papers propose token-adaptive KV cache compression for long-context LLMs
Two arXiv papers from July 7, 2026 introduce token-adaptive KV cache compression methods for long-context LLM inference. DepthWeave-KV factorizes key/value states across neighboring layers using shared low-rank bases with token-specific residuals. FreqDepthKV uses shared low-frequency depth components and sparse high-frequency residuals, with an online probe assigning attention heads to different cache modes. Both aim to reduce memory bandwidth while preserving retrieval and reasoning quality.
User reports Qwen3.6 35B-A3B model improves with Q8_0 CPU quantization
A user on Reddit reported that switching from Q4_K_M on GPU to Q8_0 on CPU significantly improved the performance of the Qwen3.6 35B-A3B model for a complex coding task. The user noted the model 'punches far above its weight' and found the quality gain worth the slowdown.
Voodoo Quant claims 95% KLD improvement over Unsloth Dynamic 2.0 on Qwen3.5 models
A developer released two new GGUF quantizations of Qwen3.5 0.8B and 2B using a technique called Voodoo Quant, which optimizes mixed precision by assigning higher precision to more important parts of the model. The author claims Voodoo Quant beats Unsloth Dynamic 2.0 by 95% in Kullback-Leibler divergence (KLD). The quantized models are available on Hugging Face.
Developer compresses GLM-5.2 MoE to run on single RTX 3090 via 79 experiments
A developer conducted 79 experiments to compress GLM-5.2, a 337 GB MoE model with 75 sparse layers and 256 routed experts, to fit on a 24 GB RTX 3090. The approach uses per-expert codecs, a batch pipeline over all MoE layers, and a patched llama.cpp runtime that loads codec-native expert binaries at inference time. The MIT-licensed repository documents the method and findings on expert similarity.
User seeks to extend Qwen 3.6 27B context window beyond 100k tokens
A user reports running Qwen 3.6 27B (Q8_0) at 100k context length but finds reliability insufficient. They ask the community for techniques beyond KV cache quantization to improve stability at longer contexts.