VGGT Implicitly Encodes Co-Visibility as Emergent Behavior in Its Internal Representations
A new paper probes VGGT, a geometric foundation model, and finds it implicitly encodes co-visibility—determining which image pairs share overlapping surfaces—without any supervision. The model's internal representations exhibit a hierarchical structure similar to LLMs, with early layers building 3D-aware scene representations and later layers acting as dedicated co-visibility reasoners, specifically layer L17 identified as a negative indicator.
Entities
Related
MultiView-Bench: New Benchmark Tests VLMs' 3D Scene Integration from Multiple Views
Researchers introduced MultiView-Bench, a diagnostic benchmark to evaluate vision-language models' ability to integrate observations from multiple viewpoints into a coherent, world-centric 3D mental model. Unlike existing benchmarks focused on single-view perception or pixel-level mapping, MultiView-Bench requires models to decouple object positions from transient perspectives and ground them in a fixed global coordinate system. This addresses a core cognitive ability previously untested in VLMs.
New paper proposes LLM-GCN hybrid for semi-supervised image classification
A new arXiv paper introduces a method that integrates Large Language Models with Graph Convolutional Networks to improve semi-supervised image classification. The approach addresses the challenge of graph construction for visual data by leveraging LLMs to generate better graph representations, potentially reducing the need for labeled datasets.
VLMs encode correct object counts internally but output wrong answers, study finds
A new arXiv paper investigates why vision-language models (VLMs) fail at basic object counting. By training probes on internal activations across four VLMs and five counting datasets, researchers found that nonlinear probes can reliably detect counting errors, indicating that models often encode the correct count even when they output the wrong answer. SVCCA analysis further confirms a misalignment between internal representations and verbalized outputs.
ELSA3D introduces elastic semantic anchoring for unified 3D understanding and generation
ELSA3D is a unified 3D foundation model that uses elastic semantic anchoring to jointly structure language and geometric reasoning across matched abstraction scales. It addresses the collapse of coarse and fine details in existing models that concatenate text and 3D tokens into flat sequences. The approach represents geometry with scale-aware features, enabling more explicit text-3D interaction within a single backbone.
Paper challenges text-only pretraining, proposes visual pretraining for language models
A new arXiv paper argues that current language model pretraining discards rich visual information from documents and web pages. The authors propose scalable visual pretraining to incorporate figures, equations, and layouts, aiming to improve language intelligence beyond text-only approaches.