Paper proposes modeling forgetting as interference between tasks in continual learning
A new arXiv paper argues that forgetting in continual learning should be modeled directly as interference between tasks, rather than relying on post-hoc mechanisms like replay or regularization. The authors show that in the frozen-feature regime, forgetting equals the interference energy induced on the old task, and they recover this quantity in deep networks via path-averaged curvature with minimal extra computation.
Entities
Related
Researchers propose memory agent to combat behavioral state decay in long-horizon tasks
A new arXiv paper identifies 'behavioral state decay' as a failure mode where decision-relevant information gets buried in long trajectories. The authors propose a separate memory agent that actively updates a structured memory bank alongside an unmodified action agent, rather than relying on passive retrieval.
Researchers identify asymmetric generalization problem in LLM unlearning benchmarks
A new arXiv paper argues that existing machine unlearning benchmarks for LLMs suffer from under-forgetting and over-forgetting due to an asymmetric generalization problem. The authors propose that evaluation must cover diverse query formulations of target facts to reliably measure knowledge removal while preserving unrelated capabilities.
Paper derives exact dynamics of linear representation learning in neural networks
A new arXiv paper presents exact solutions for how linear concept representations emerge during neural network training, providing a mathematical framework for the dynamics of abstraction. This work formalizes the linear representation hypothesis, which underpins interpretability methods like linear probes and activation steering, and could guide future training and control techniques.
Two papers propose token-adaptive KV cache compression for long-context LLMs
Two arXiv papers from July 7, 2026 introduce token-adaptive KV cache compression methods for long-context LLM inference. DepthWeave-KV factorizes key/value states across neighboring layers using shared low-rank bases with token-specific residuals. FreqDepthKV uses shared low-frequency depth components and sparse high-frequency residuals, with an online probe assigning attention heads to different cache modes. Both aim to reduce memory bandwidth while preserving retrieval and reasoning quality.
Paper proposes correlation-aware bandits with surrogate rewards for LLM routing
A new arXiv paper introduces contextual bandit algorithms that leverage surrogate reward signals from machine learning models to improve LLM routing decisions. The approach accounts for inter-arm correlations and noisy auxiliary rewards, addressing limitations of classical bandits that assume conditional independence.