Robustifying Vision-Language Models via Test-Time Prompt Adaptation
A new arxiv paper proposes a test-time prompt adaptation method to improve robustness of Vision-Language Models like CLIP against adversarial perturbations. The approach leverages distributional structure rather than sample-level heuristics to distinguish adversarial mispredictions from true semantic consistency.
Entities
Related
Foveated Dynamic Transformer (FDT) paper proposes foveation-guided token selection for robust vision transformers
A new arXiv paper introduces the Foveated Dynamic Transformer (FDT), a vision transformer architecture that uses foveation-guided dynamic token selection inspired by the human visual system. The model demonstrates strong resilience to noise and adversarial attacks without explicit adversarial training, offering a path to more efficient and robust vision models.
TCLA: Training-Free Few-Shot Adaptation for Medical VLMs
Researchers propose TCLA, a training-free method to adapt medical Vision-Language Models (VLMs) to out-of-distribution data using only a few examples. It corrects inference logits without additional trainable components, improving stability in low-data regimes like 1-shot. The method is model-agnostic and fast, addressing domain shifts and class bias in medical imaging.
CLAP paper proposes direct VLM-to-VLA adaptation to isolate semantic transfer
A new arXiv paper introduces CLAP, a method that converts pretrained vision-language models (VLMs) into vision-language-action models (VLAs) with minimal architectural changes, avoiding the distribution mismatch caused by predicting actions as bare numeric tokens. The approach aims to clarify how VLM capabilities transfer across model scales for robot control.
Activation-Guided GCG attacks target refusal direction in LLMs
A new arxiv paper introduces Activation-Guided GCG, an adversarial attack that optimizes suffixes by directly targeting a model's internal refusal direction in activation space, rather than output-based objectives. The work probes the geometry of safety representations, showing that refusal behavior is mediated by low-dimensional directions that can be suppressed via optimization.
TILDE Paper Proposes TILt-based Distributional Erasure for Concept Unlearning in Diffusion Models
A new arxiv paper (July 7, 2026) introduces TILDE, a method for concept unlearning in text-to-image diffusion models that aims to remove unwanted concepts while preserving generation quality, diversity, and semantic coverage on benign prompts. The work addresses privacy, copyright, and safety concerns, with the gold standard being a retain-only model trained from scratch without the unwanted data.