Neural Collapse Is Forbidden: Information Floors in Language Models
A new paper argues that within-class variance in language model representations is not incomplete neural collapse but allocated information storage obeying a law. Across 14 models, macro-category structure carries only 4-12% of representational variance, while within-token context carries 79-91%, stable across a 100x parameter range.
Entities
Related
Blog post explores why LLMs produce predictable metaphors and how architecture might reduce attractor pull
A blog post titled 'Escaping the Attractor' examines why large language models tend to produce similar metaphors (e.g., 'Time is a River') when prompted, attributing this to attractors in the embedding space. The author suggests that architectural changes could make models less predictable, building on earlier ideas about shared embedding geometries across models.
Study questions robustness of emergent misalignment in language models
A new paper systematically studies repeated alignment and misalignment cycles in language models, reproducing Emergent Misalignment (EM) but finding that both misalignment and realignment are highly sensitive to superficial dataset characteristics. The results suggest that EM may not be a robust phenomenon.
Research paper proposes reusing spectral patterns from pretrained GPT-2 checkpoints as initialization for language model pretraining
A new arXiv paper analyzes eleven pretrained GPT-2-style checkpoints and finds that they share structured weight spectra, with consistent depth trends in Frobenius norm and effective-rank entropy. The authors propose reusing these recurring spectral patterns as initialization signals for GPT-2-style language model pretraining, potentially reducing training time or improving convergence.
Researcher tests minimal dynamical system for word embeddings without MLP or attention
A researcher proposes a word representation model using only token vectors, a start state, and two scalars, with no MLP, transformer, attention, or output matrix. The model achieves a SimLex-999 ρ of 0.3616 by updating a state vector via a cosine-based pull toward token attractors, encoding context through trajectory dynamics.
Mechanistic interpretability researchers apply causality theory to LLMs
Researchers are applying causality theory from the paper 'Causal Abstraction for Interpretability' (arXiv:2301.04709) to understand LLM internals. This approach aims to identify causal mechanisms within models, moving beyond correlation-based analysis.