Community finds retrieval-based MCP tool selection caps at ~23% failure reduction
Practitioners report that agent tool-selection accuracy degrades beyond ~20 MCP tools, and retrieval-based fixes like RAG-MCP only reduce failures by about 23 percentage points (from ~86% to ~43% on one benchmark). A new paper argues the root cause is attention-segment interference, not retrieval quality.
Entities
Related
Researchers identify asymmetric generalization problem in LLM unlearning benchmarks
A new arXiv paper argues that existing machine unlearning benchmarks for LLMs suffer from under-forgetting and over-forgetting due to an asymmetric generalization problem. The authors propose that evaluation must cover diverse query formulations of target facts to reliably measure knowledge removal while preserving unrelated capabilities.
Community discusses which single integration makes agents useful and which extra tool degrades reliability
A Reddit user reflects on the principle that adding more tools to an agent often increases failure modes and review work, while a single narrow integration that solves a real bottleneck yields the biggest improvement. The post asks the community to share which integration made their agent genuinely useful and which tool they removed for harming reliability.
Developer asks community for agent evaluation practices, cites silent breakage
A developer building AI agents reports that prompt or MCP changes often break silently despite passing manual tests. They ask the community about evaluation methods, including fixed test cases, skill-level vs. end-to-end checks, and tools like DeepEval, LangSmith, and Ragas.
Study analyzes failure trajectories of CLI coding agents as temporal processes
A new arXiv paper presents the first large-scale empirical study of CLI coding-agent failure trajectories, treating failure as a temporal process rather than a final outcome. The study introduces a process-oriented framework to analyze how failures emerge, evolve, and become unrecoverable in LLM-based coding agents.
User reports tool-selection accuracy drops linearly with more MCP servers due to token bloat
A user connected 4 MCP servers to one agent and observed tool-selection accuracy declining linearly with server count. They traced the issue to every tool's name, description, and JSON Schema being serialized into every request, causing token bloat. With 4 servers and ~60 tools, the serialized definitions consumed significant context, degrading performance.