VTaMo introduces explicit multi-granularity alignment for sign language translation
VTaMo is a new framework for sign language translation that introduces explicit multi-granularity alignment between video and text at three levels: local alignment via optimal transport with a learnable null token, global alignment via orthogonal transformation, and sequence-level alignment. This approach aims to improve gloss-free SLT by providing more structured cross-modal correspondences.
Entities
Related
Researchers fine-tune SHuBERT-ByT5 for real-time sentence-level sign language translation
A team fine-tuned a SHuBERT-ByT5 translation stack on a subset of How2Sign using QLoRA, achieving a BLEU of 15.9 on the test set. The work focuses on real-time deployment rather than novel architectures, addressing the gap between isolated sign recognition and natural sentence-level translation.
SCENT: Language-guided framework bridges vision and olfaction using VLMs
Researchers propose SCENT, a multimodal framework that uses language guidance from Vision-Language Models (VLMs) to align visual scenes with olfactory signals. This addresses the challenge that many olfactory cues arise from contextual environmental factors not directly visible in pixels.
CLAP paper proposes direct VLM-to-VLA adaptation to isolate semantic transfer
A new arXiv paper introduces CLAP, a method that converts pretrained vision-language models (VLMs) into vision-language-action models (VLAs) with minimal architectural changes, avoiding the distribution mismatch caused by predicting actions as bare numeric tokens. The approach aims to clarify how VLM capabilities transfer across model scales for robot control.
BTHA: A Backbone-Transferable Adapter for Text-Guided Medical Segmentation
Researchers propose BTHA, a hierarchical adapter framework that decouples language guidance from vision and text backbones in text-guided medical image segmentation. BTHA uses a stable feature-level interface to enable reuse of language modules across heterogeneous encoder pairs without network redesign. This addresses a key limitation of existing tightly coupled architectures.
Paper proposes video generation as general-purpose vision pretraining
A new arXiv paper argues that large-scale text-to-video generation can serve as a general-purpose pretraining paradigm for computer vision, analogous to next-token prediction in NLP. The authors introduce GenCeption, a method that uses a pretrained video generative diffusion backbone for feed-forward visual perception tasks.