Study finds GPT-5, Claude Sonnet, Gemini rate China- and Russia-endorsed policies lower than identical US- or EU-endorsed ones
A new arXiv preprint reports an endorsement experiment in which four LLMs (GPT-5, Claude Sonnet, Gemini, and a fourth unnamed model) evaluated identical international economic and security policies randomly attributed to the US, EU, China, or Russia. In the numeric-only condition, models rated China- and Russia-endorsed policies substantially lower than the same policies endorsed by the US or EU, revealing implicit geopolitical bias.
Entities
Related
Open-source benchmark evaluates LLM political bias across 6 axes using 3,987 survey questions
A new open-source benchmark uses 3,987 public-opinion survey questions across six axes (economic, social, foreign policy, environment, religion, national identity) to measure LLM political bias. Answers are judged by a panel of three models from different regions: Qwen3.6 35B A3B (China), Gemma 3 27B (US), and Mistral Small (France). The benchmark aims to provide a standardized way to assess ideological leanings in language models.
PRC-linked accounts reportedly used ChatGPT in covert campaigns targeting US AI policy debates
The AI Incident Database has cataloged Incident 1580, documenting that PRC-linked accounts reportedly used ChatGPT in covert influence operations targeting U.S. AI policy debates. This highlights real-world misuse of generative AI for political manipulation.
Community observes that model preference debates reflect different workloads, not model quality
A Reddit user notes that arguments over which AI model is best often stem from participants doing fundamentally different types of work—long-context reasoning, marketing copy, or agentic coding—rather than genuine model superiority. The observation highlights the lack of universal benchmarks and the importance of task-specific evaluation.
OpenAI analysis reveals flaws in SWE-Bench Pro coding benchmark
OpenAI published an analysis uncovering reliability issues in SWE-Bench Pro, a popular benchmark for evaluating AI coding models. The findings raise concerns about the accuracy of benchmark scores, potentially affecting how developers and researchers trust model evaluations.
User finds major disagreement across 9 tiers of Claude, GPT, Gemini, and Grok on best dev tool recommendations
A developer ran the same 10 'best dev tool' questions through 9 tiers of Claude, GPT, Gemini, and Grok, finding that different tiers within the same model family often recommend different stacks. The experiment highlights that model tier selection can significantly affect tooling advice, which matters for practitioners relying on these models for recommendations.