STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU
A new paper introduces STEEL, a sparsity-aware fused attention mechanism designed for AMD's XDNA NPU, enabling energy-efficient long-sequence inference on laptop-class SoCs. The approach addresses the challenge of mapping attention mechanisms onto NPUs while maintaining low power consumption, which is critical for agentic workloads that require on-device processing for reliability and privacy.
Entities
Related
Survey on Green Development of Large Models: Resource-Efficient Architectures and Hardware-Software Co-Design
A comprehensive survey published on arXiv reviews strategies for reducing computational costs and energy consumption of large AI models, covering efficient architectures (attention optimization, linear-complexity models, sparsification) and full-stack hardware-software co-design. The paper provides a systematic overview of recent advances in green AI development.
SATS: Sensitivity-Aware Thresholding for MLP Activation Sparsification in LLMs
A new arxiv paper proposes Sensitivity-Aware Thresholding for Sparsity (SATS), a method to calibrate layerwise gate thresholds for MLP activation sparsification using a local sensitivity proxy instead of activation percentiles. The approach aims to reduce computation during LLM inference while preserving model quality.
Study evaluates energy, performance, and accuracy trade-offs across vLLM configurations
A new arxiv paper presents a large-scale controlled study of three vLLM configuration options—attention kernel type, prefix caching, and chunked prefill—examining their impact on energy consumption, performance, and output quality. The work addresses a gap in understanding how inference engine configuration affects these trade-offs in production LLM deployments.
Study reveals text generation, not vision, is the energy bottleneck in edge VLM inference
A systematic energy profiling study of on-device Vision-Language Models (VLMs) across five models, four resolutions, and two hardware platforms (NVIDIA RTX 3070 and Jetson Orin NX) overturns the common assumption that visual processing dominates energy cost. The authors find that text generation is the true bottleneck, accounting for the majority of energy consumption during inference on edge devices.
Flash-MSA: Sparse Attention Kernels Enable Million-Token Training
A new paper introduces Flash-MSA, a sparse attention kernel that reduces memory and computation for long-context transformers, enabling training on sequences up to one million tokens. The method achieves up to 8x speedup over FlashAttention-2 on 128K-length sequences while maintaining model quality.