Machine Beater: 5-question head-to-head guessing game against LLMs
A game where a human and an LLM each ask 5 yes/no questions to guess a hidden answer, inspired by 20 Questions. The goal is to benchmark reasoning skills not captured by standard benchmarks, with planned model-model and human-model matchups.
Visit project ↗Entities
Related
User benchmarks Fable 5, Sol, and xhigh models on strategic tasks
A user ran a role-based benchmark comparing Fable 5, Sol, and xhigh models on strategic decision memos, execution briefs, and bug repairs. Fable 5 scored 95 on a multi-layer productization decision, slightly ahead of Sol max (94) and xhigh (90). The benchmark is local and not a general intelligence test.
LLM comparison dashboard for quality, latency, and cost
A dashboard that lets users test LLMs on their own data, comparing quality, latency, and cost side by side. It runs on Nebius Serverless and helps developers choose the best model for their specific use case rather than relying on leaderboards.
DataGovBench: New benchmark evaluates LLMs on real-world data analysis with large multi-tabular datasets
Researchers introduced DataGovBench, a benchmark derived from governmental open data to evaluate LLMs on practical data analysis tasks. It includes Table QA and Table Insight tasks, addressing limitations of existing benchmarks that focus on small tables and fact retrieval.
Muse Spark 1.1 benchmarked against top models on Artificial Analysis
A user shared an Artificial Analysis comparison of Muse Spark 1.1 (xhigh) against models like Gemini 3.5 Flash, Claude Fable 5, and GPT-5.6 Sol, evaluating intelligence, performance, and cost per task. The benchmark provides practitioners with a data-driven view of where Muse Spark 1.1 stands relative to leading models.
STS2-Bench tests LLM long-horizon decision-making in Slay the Spire 2
A developer created STS2-Bench, a benchmark using Slay the Spire 2 to evaluate LLMs on sequential decision-making under uncertainty. The benchmark tests models on reading changing game states, weighing short-term vs long-term goals, and adapting plans. Early results show 5.6Sol performing surprisingly well.
