#evaluation
- EVA-Bench: End-to-End Framework for Evaluating Voice Agents ServiceNow AI research
- SOOHAK: Frontier LLMs Solve Hard Math But Fail to Recognize Unsolvable Problems research
- OpenAI Publishes Deployment Simulation: Predicting Model Behavior Before Release OpenAI research
- GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation GigaAI research
- Long-Horizon-Terminal-Bench: Testing Agent Limits on Long-Horizon Terminal Tasks Tencent Hunyuan research
- LLM-as-a-Verifier: verification as an independent scaling axis for LLMs Stanford University / UC Berkeley / NVIDIA research
- Judge Circuits: Mechanistic Explanation of LLM-as-Judge Format Inconsistency research
- EvoArena: LLM Agents Score Only 40% on Dynamic Evolving Environments MIT / NUS / Salesforce research
- WeaveBench: Computer-Use Agents Fail at Hybrid GUI+CLI Tasks — 41% Pass Rate Microsoft Research research
- PerceptionRubrics: Atomic Rubric Evaluation Reveals 8% Perception Gap Between Open and Closed Models research
- AutomationBench-AA: 657-task independent benchmark for AI agent SaaS automation Artificial Analysis tools
- OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models NLP Group of The University of Hong Kong research
- Cline open-sources Terminal-Bench-based evals for open-weight coding agents Cline tools
- AutoResearchBench — a benchmark for autonomous scientific literature search by AI agents BAAI research
- EvoArena: LLM Agents Score Only 39.6% on Dynamic Evolving Environments Benchmark MIT research
- FAPO: Fully Autonomous Prompt Optimization of Multi-Step LLM Pipelines Cisco Foundation AI research
- AgenticSTS: Bounded-Memory Testbed for Long-Horizon LLM Agents Alaya Studio research
- EvoPolicyGym: Evaluating Iterative RL Policy Self-Improvement by Coding Agents University of Macau / CUHK research
- SWE-Together: Multi-Turn Benchmark for Coding Agent Evaluation research
- Ideas Have Genomes: Frontier LLMs Score Only 27% on Scientific Lineage Reasoning Shanghai Jiao Tong University research
- K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs research
- How Can Rhetoric Reward-Hack AI Reviewers? Dissecting Rhetorical Sensitivity in AI-Based Peer Review research
- HarnessEval-W: Agentifying the Evaluation of Visual Worlds NTU / MirroS-Lab consortium research
- Multimodal Evaluator Preference Collapse: Cross-Modal Contagion in Self-Evolving Agent Loops research
- Formalizing Latent Thoughts: Axiomatic Framework for Evaluating LLM Reasoning Representations University of British Columbia research
- Will Scaling Improve Social Simulation with LLMs? A Study of 85 Models Stanford / Columbia / Tsinghua research
- UniClawBench: Benchmark for Proactive AI Agents on Real-World Tasks University of Hong Kong research
- OSReward benchmarks cross-platform computer-use reward models University of Hong Kong research
- IBM Research benchmarks how much agentic memory each model tier actually needs IBM Research research