Paper derives exact dynamics of linear representation learning in neural networks
A new arXiv paper presents exact solutions for how linear concept representations emerge during neural network training, providing a mathematical framework for the dynamics of abstraction. This work formalizes the linear representation hypothesis, which underpins interpretability methods like linear probes and activation steering, and could guide future training and control techniques.
Entities
Related
SAMPAT: A new interpretable neural architecture for scientific data analysis
Researchers introduced SAMPAT, a three-layer neural architecture that provably learns smooth, differentiable functions with interpretable polynomial approximations. It aims to address the lack of interpretability in deep neural networks for scientific data analysis.
Mechanistic interpretability researchers apply causality theory to LLMs
Researchers are applying causality theory from the paper 'Causal Abstraction for Interpretability' (arXiv:2301.04709) to understand LLM internals. This approach aims to identify causal mechanisms within models, moving beyond correlation-based analysis.
Paper challenges text-only pretraining, proposes visual pretraining for language models
A new arXiv paper argues that current language model pretraining discards rich visual information from documents and web pages. The authors propose scalable visual pretraining to incorporate figures, equations, and layouts, aiming to improve language intelligence beyond text-only approaches.
Paper proposes modeling forgetting as interference between tasks in continual learning
A new arXiv paper argues that forgetting in continual learning should be modeled directly as interference between tasks, rather than relying on post-hoc mechanisms like replay or regularization. The authors show that in the frozen-feature regime, forgetting equals the interference energy induced on the old task, and they recover this quantity in deep networks via path-averaged curvature with minimal extra computation.
Researchers identify and fix crispness penalty failure mode in legible transformers
A new paper on arXiv reveals that a crispness penalty intended to make transformer operators more legible can collapse them into dead constants. The authors derive an identity showing the penalty is a variance minimizer, and propose a per-channel variance floor as a fix.