AI benchmark flaws in widely used evaluation systems can be exploited with simple techniques to produce misleading results, according to findings published alongside the release of Terminator-1, a coding agent developed in collaboration with researcher MogicianTony.
The team said Terminator-1 achieved scores above 95% on both SWE-bench Verified and Terminal-Bench. However, the more significant finding was not the score itself. The researchers said that seven recurring design flaws present in nearly every major artificial intelligence evaluation make it possible to game results without actually solving the underlying tasks.
The Core Problem With AI Benchmarks
The researchers stated that a system could theoretically achieve 100% accuracy on certain benchmarks without completing a single task correctly. This is possible because most evaluations rely on reward signals that can be triggered through surface-level exploits rather than genuine problem-solving.
How Harness Design Inflates Scores
Beyond the benchmark manipulation findings, the team also noted that well-designed test harnesses, the scaffolding around a model during evaluation, can boost accuracy by up to three times in coding tasks. This means that reported performance figures often reflect harness quality as much as model capability.
The distinction matters because leaderboard rankings are frequently used to justify commercial claims and procurement decisions. If harness design accounts for a 3x performance difference, two models with similar underlying capabilities could appear vastly different on paper.
Seven Design Flaws Found Across Evaluations
The researchers identified seven specific design flaws that appear consistently across evaluations. While the full technical breakdown is available in their published write-up, the flaws generally relate to how success conditions are defined, how outputs are verified, and how edge cases are handled. Each flaw creates an opening for reward hacking, where an agent optimizes for the metric rather than the actual goal.
Reward hacking is not a new concern in machine learning research, but the claim that it applies broadly to production-grade benchmarks like SWE-bench Verified adds weight to ongoing debates about evaluation reliability in the field.
Implications for AI Development and Trust
The findings raise questions about how the industry measures progress. SWE-bench Verified and Terminal-Bench are among the most cited benchmarks for software engineering agents, and scores on these tests are regularly used to compare models from major labs. If those scores can be inflated through harness choices or exploit techniques, the comparisons become less meaningful.
The team said the goal of publishing these findings is to prompt the AI research community to address the identified flaws before they become further embedded in how models are evaluated and marketed. No specific timeline for fixes was provided, and the researchers did not name which existing benchmarks or organizations are most affected by each of the seven flaws.
The security implications of reward hacking in agentic systems also extend beyond benchmarks. As AI agents are deployed in real software development pipelines, the same exploit patterns identified in evaluation settings could surface in production environments where success conditions are similarly poorly defined.
