Survey on Green Development of Large Models: Resource-Efficient Architectures and Hardware-Software Co-Design
A comprehensive survey published on arXiv reviews strategies for reducing computational costs and energy consumption of large AI models, covering efficient architectures (attention optimization, linear-complexity models, sparsification) and full-stack hardware-software co-design. The paper provides a systematic overview of recent advances in green AI development.
Entities
Related
Study evaluates energy, performance, and accuracy trade-offs across vLLM configurations
A new arxiv paper presents a large-scale controlled study of three vLLM configuration options—attention kernel type, prefix caching, and chunked prefill—examining their impact on energy consumption, performance, and output quality. The work addresses a gap in understanding how inference engine configuration affects these trade-offs in production LLM deployments.
STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU
A new paper introduces STEEL, a sparsity-aware fused attention mechanism designed for AMD's XDNA NPU, enabling energy-efficient long-sequence inference on laptop-class SoCs. The approach addresses the challenge of mapping attention mechanisms onto NPUs while maintaining low power consumption, which is critical for agentic workloads that require on-device processing for reliability and privacy.
Study reveals text generation, not vision, is the energy bottleneck in edge VLM inference
A systematic energy profiling study of on-device Vision-Language Models (VLMs) across five models, four resolutions, and two hardware platforms (NVIDIA RTX 3070 and Jetson Orin NX) overturns the common assumption that visual processing dominates energy cost. The authors find that text generation is the true bottleneck, accounting for the majority of energy consumption during inference on edge devices.
Super-Tuning: Activation-Aware Pruning Reused for Sparse Fine-Tuning
A new arxiv paper proposes Super, a sparse PEFT method that reuses Wanda-style activation-weighted magnitude scores from pruning to select a small trainable support, and Supra, a hybrid adapter combining sparse updates with LoRA. This approach reduces memory and compute for fine-tuning LLMs while maintaining performance.
SATS: Sensitivity-Aware Thresholding for MLP Activation Sparsification in LLMs
A new arxiv paper proposes Sensitivity-Aware Thresholding for Sparsity (SATS), a method to calibrate layerwise gate thresholds for MLP activation sparsification using a local sensitivity proxy instead of activation percentiles. The approach aims to reduce computation during LLM inference while preserving model quality.