Community questions Mellum2 MTP implementation in GGUF
A Reddit user noticed that JetBrains' Mellum2 GGUF files lack a visible MTP head layer, despite the company claiming MTP (Multi-Token Prediction) was used to achieve low latency comparable to Qwen2.5-Coder 7B. The user asks how to extract MTP weights from the safetensors format and provides links to the GGUF and safetensors repositories.
Entities
Related
Two papers propose token-adaptive KV cache compression for long-context LLMs
Two arXiv papers from July 7, 2026 introduce token-adaptive KV cache compression methods for long-context LLM inference. DepthWeave-KV factorizes key/value states across neighboring layers using shared low-rank bases with token-specific residuals. FreqDepthKV uses shared low-frequency depth components and sparse high-frequency residuals, with an online probe assigning attention heads to different cache modes. Both aim to reduce memory bandwidth while preserving retrieval and reasoning quality.
Voodoo Quant claims 95% KLD improvement over Unsloth Dynamic 2.0 on Qwen3.5 models
A developer released two new GGUF quantizations of Qwen3.5 0.8B and 2B using a technique called Voodoo Quant, which optimizes mixed precision by assigning higher precision to more important parts of the model. The author claims Voodoo Quant beats Unsloth Dynamic 2.0 by 95% in Kullback-Leibler divergence (KLD). The quantized models are available on Hugging Face.
Xiaomi quietly uploads MiMo-V2.5-DFlash weights to Hugging Face
Xiaomi has uploaded the official DFlash weights for MiMo-V2.5-DFlash to Hugging Face. The model has 300B+ parameters and runs at 8-10 tokens/second on 2x24GB cards with offloading; DFlash could double that speed. The MTP head was shared but does not work yet.
SATS: Sensitivity-Aware Thresholding for MLP Activation Sparsification in LLMs
A new arxiv paper proposes Sensitivity-Aware Thresholding for Sparsity (SATS), a method to calibrate layerwise gate thresholds for MLP activation sparsification using a local sensitivity proxy instead of activation percentiles. The approach aims to reduce computation during LLM inference while preserving model quality.
Developer compresses GLM-5.2 MoE to run on single RTX 3090 via 79 experiments
A developer conducted 79 experiments to compress GLM-5.2, a 337 GB MoE model with 75 sparse layers and 256 routed experts, to fit on a 24 GB RTX 3090. The approach uses per-expert codecs, a batch pipeline over all MoE layers, and a patched llama.cpp runtime that loads codec-native expert binaries at inference time. The MIT-licensed repository documents the method and findings on expert similarity.