QANTA 2026 Challenge: Task-Specific Multimodal QA Agents with Confidence Calibration
A team submitted a system to the QANTA 2026 shared challenge at ICML 2026, addressing multimodal quizbowl questions with incremental text and images. The system uses confidence calibration for tossup questions and incremental reasoning for bonus questions, aiming to balance accuracy and efficiency.
Entities
Related
Question-type-specific LLM pipeline boosts BioASQ 14b biomedical QA
A new framework for BioASQ 14b Task B selects different inference procedures for yes/no, factoid, and list questions, improving answer robustness and evidence grounding. The approach uses question-type-specific prompting strategies rather than a single method for all queries.
Machine Beater: 5-question head-to-head guessing game against LLMs
A game where a human and an LLM each ask 5 yes/no questions to guess a hidden answer, inspired by 20 Questions. The goal is to benchmark reasoning skills not captured by standard benchmarks, with planned model-model and human-model matchups.
DynaKRAG: A unified framework for learnable evidence control in multi-hop RAG
Researchers propose DynaKRAG, a framework that learns a state-conditioned policy to dynamically choose among evidence operations (retrieval, reformulation, critique, sufficiency checking) in multi-hop retrieval-augmented generation. This moves beyond fixed pipelines, potentially improving accuracy and flexibility in complex QA tasks.
Test-Time Scaling Improves Small VLMs on Multilingual Visual MCQ, but Parseability Is Key
A new arXiv paper investigates whether test-time scaling (TTS) improves reasoning in small open vision-language models (VLMs) on the EXAMS-V multilingual visual multiple-choice benchmark. The study compares self-consistency, describe-then-reason with PRM-guided beam search, and two post-hoc selectors across Qwen2.5-VL-7B-Instruct and Qwen3.5-4B. The key finding is that parseability—the ability of the model to commit to a final answer—is the largest factor in TTS success, not the search or verification machinery.
OmniMapBench benchmark tests LVLMs on visual-centric map reasoning
Researchers introduced OmniMapBench, a benchmark of 2,096 QA pairs across 1,603 map documents from nine categories, designed to evaluate visual-centric reasoning in large vision-language models. It addresses the limitation that many document benchmarks allow high performance via text-only cues, requiring genuine visual grounding for tasks from perception to multi-step reasoning.