Paper proposes correlation-aware bandits with surrogate rewards for LLM routing
A new arXiv paper introduces contextual bandit algorithms that leverage surrogate reward signals from machine learning models to improve LLM routing decisions. The approach accounts for inter-arm correlations and noisy auxiliary rewards, addressing limitations of classical bandits that assume conditional independence.
Entities
Related
LLM/VLA models enable prompt-driven exploration in RL
A new research paper proposes using large language models (LLMs) and vision-language-action (VLA) models to drive exploration in reinforcement learning by modifying natural language prompts, which induce global behavioral changes beyond standard action noise. This approach could help policies escape weak local optima more effectively.
Agora paper proposes auction-based task allocation for LLM agents
A new arXiv paper introduces Agora, a framework that uses an incentive-compatible auction mechanism to dynamically allocate tasks to expert LLMs and tools, aiming to improve reasoning performance while accounting for cost and performance variability. The approach addresses limitations in current orchestration methods that rely on coarse-grained function matching.
Mechanistic interpretability researchers apply causality theory to LLMs
Researchers are applying causality theory from the paper 'Causal Abstraction for Interpretability' (arXiv:2301.04709) to understand LLM internals. This approach aims to identify causal mechanisms within models, moving beyond correlation-based analysis.
arXiv paper benchmarks LLM judges for citation quality in deep-research systems
A new arXiv paper studies the calibration of LLM judges used as reward models in reinforcement learning for citation quality in deep-research systems. The work evaluates how capable and biased an LLM judge must be to reliably score rubric criteria like source relevance and factual support for attribution-citation pairs. This matters for practitioners building RL-based systems that depend on automated citation verification.
Research paper introduces NRFR metric to measure multimodal reward hacking in RL for MLLMs
A new arXiv paper studies reward hacking in reinforcement learning for multimodal large language models (MLLMs), showing that higher proxy rewards do not always mean better task performance, especially when visual evidence is evaluated by text-only or weakly grounded rewards. The authors propose a new metric, Newly Rewarded Failure Rate (NRFR), to measure failures among samples whose proxy reward improves over the SFT baseline. Experiments span safety VQA, chart VQA, and stress-test settings, varying reward design, data ambiguity, model scale (2B-32B), and RL algorithms (GRPO, RLOO, DAPO).