Foveated Dynamic Transformer (FDT) paper proposes foveation-guided token selection for robust vision transformers
A new arXiv paper introduces the Foveated Dynamic Transformer (FDT), a vision transformer architecture that uses foveation-guided dynamic token selection inspired by the human visual system. The model demonstrates strong resilience to noise and adversarial attacks without explicit adversarial training, offering a path to more efficient and robust vision models.
Entities
Related
Robustifying Vision-Language Models via Test-Time Prompt Adaptation
A new arxiv paper proposes a test-time prompt adaptation method to improve robustness of Vision-Language Models like CLIP against adversarial perturbations. The approach leverages distributional structure rather than sample-level heuristics to distinguish adversarial mispredictions from true semantic consistency.
Paper challenges text-only pretraining, proposes visual pretraining for language models
A new arXiv paper argues that current language model pretraining discards rich visual information from documents and web pages. The authors propose scalable visual pretraining to incorporate figures, equations, and layouts, aiming to improve language intelligence beyond text-only approaches.
TCLA: Training-Free Few-Shot Adaptation for Medical VLMs
Researchers propose TCLA, a training-free method to adapt medical Vision-Language Models (VLMs) to out-of-distribution data using only a few examples. It corrects inference logits without additional trainable components, improving stability in low-data regimes like 1-shot. The method is model-agnostic and fast, addressing domain shifts and class bias in medical imaging.
Researchers identify and fix crispness penalty failure mode in legible transformers
A new paper on arXiv reveals that a crispness penalty intended to make transformer operators more legible can collapse them into dead constants. The authors derive an identity showing the penalty is a variance minimizer, and propose a per-channel variance floor as a fix.
Paper proposes video generation as general-purpose vision pretraining
A new arXiv paper argues that large-scale text-to-video generation can serve as a general-purpose pretraining paradigm for computer vision, analogous to next-token prediction in NLP. The authors introduce GenCeption, a method that uses a pretrained video generative diffusion backbone for feed-forward visual perception tasks.