Small hyperbolic language models show emergent creativity, honesty, and forgetting
A research paper demonstrates that small language models (146M to 3B parameters) with a hyperbolic substrate exhibit emergent traits of creativity, honesty, and designed forgetting, addressing the challenge of evaluating AI companions. Trained human raters showed poor agreement (Fleiss kappa = 0.074) on what makes a companion beneficial, but the hyperbolic models provide a reliable instrument for assessment.
Entities
Related
Researchers identify asymmetric generalization problem in LLM unlearning benchmarks
A new arXiv paper argues that existing machine unlearning benchmarks for LLMs suffer from under-forgetting and over-forgetting due to an asymmetric generalization problem. The authors propose that evaluation must cover diverse query formulations of target facts to reliably measure knowledge removal while preserving unrelated capabilities.
Study questions robustness of emergent misalignment in language models
A new paper systematically studies repeated alignment and misalignment cycles in language models, reproducing Emergent Misalignment (EM) but finding that both misalignment and realignment are highly sensitive to superficial dataset characteristics. The results suggest that EM may not be a robust phenomenon.
User notes LLMs cannot detect boring writing because they lack boredom
A Reddit user observes that while LLMs like Claude can polish grammar and structure, they fail to identify when writing is boring, as boredom is a reader-attention property the model has never experienced. This highlights a fundamental limitation in AI's ability to judge subjective qualities.
Neural Collapse Is Forbidden: Information Floors in Language Models
A new paper argues that within-class variance in language model representations is not incomplete neural collapse but allocated information storage obeying a law. Across 14 models, macro-category structure carries only 4-12% of representational variance, while within-token context carries 79-91%, stable across a 100x parameter range.
Blog post explores why LLMs produce predictable metaphors and how architecture might reduce attractor pull
A blog post titled 'Escaping the Attractor' examines why large language models tend to produce similar metaphors (e.g., 'Time is a River') when prompted, attributing this to attractors in the embedding space. The author suggests that architectural changes could make models less predictable, building on earlier ideas about shared embedding geometries across models.