Tokenizer Transplantation Fixes Autoregressive Collapse in Bengali ASR
Researchers propose a tokenizer transplantation pipeline to replace the English-centric byte-level tokenizer in Moonshine speech recognition models with the native-script BanglaBERT WordPiece vocabulary. This resolves catastrophic autoregressive collapse on Bengali, enabling edge-efficient ASR for morphologically rich non-Latin languages.
Entities
Related
FreyaTTS: A compact Turkish-first text-to-speech model introduced on arXiv
Researchers released a technical report on Freya-TTS, a 183.2M-parameter non-autoregressive flow-matching Diffusion Transformer for Turkish text-to-speech. The model operates in AudioVAE2's continuous latent space, enabling 48 kHz reconstruction without a tokenizer.
Researchers fine-tune SHuBERT-ByT5 for real-time sentence-level sign language translation
A team fine-tuned a SHuBERT-ByT5 translation stack on a subset of How2Sign using QLoRA, achieving a BLEU of 15.9 on the test set. The work focuses on real-time deployment rather than novel architectures, addressing the gap between isolated sign recognition and natural sentence-level translation.
Researchers identify modality interference as root cause of full-duplex SLM degradation
A new paper on arXiv (July 7, 2026) presents a fine-grained analysis of optimization dynamics in full-duplex Spoken Language Models (SLMs), identifying severe modality interference as the root cause of knowledge degradation and compromised semantic integrity. The work aims to enable more natural and intelligent full-duplex SLMs.
Method to extract tokenization oracles from chat APIs described
A Reddit post outlines a technique to reconstruct a closed-source LLM tokenizer using two oracles derived from standard chat APIs: a token length oracle and a prefix token oracle. The prefix oracle resolves merge order ambiguities by leveraging the model's text repetition capability. This method could enable researchers to analyze tokenization without direct access to the tokenizer.
Flash-MSA: Sparse Attention Kernels Enable Million-Token Training
A new paper introduces Flash-MSA, a sparse attention kernel that reduces memory and computation for long-context transformers, enabling training on sequences up to one million tokens. The method achieves up to 8x speedup over FlashAttention-2 on 128K-length sequences while maintaining model quality.