Two AIs with identical weights can score wildly differently on a benchmark — just by changing the code wrapped around them.

This explores how much of a benchmark score really comes from the model, and how much from the 'harness': the code, prompts, tool routing and execution loop wrapped around it. The corpus says it's a lot more than most leaderboards suggest. On Terminal-Bench 2.1, improving only the execution system around frozen models raised accuracy for several of them, and the same runbook carried over to newer models with no changes Can execution harnesses lift model performance without retuning weights?. In a more striking case, a stronger model wrote inference-time harnesses that nearly doubled a weaker model's scores on Theory-of-Mind tests, without any retraining Can a stronger model lift a weaker one at test time without retraining?. So the same weights can produce very different numbers depending on what surrounds them.

The interesting part is *why* harnesses help. They don't seem to make the model think harder. They take shaky reasoning away from it. In the Theory-of-Mind work, the gains came mostly from moving unstable reasoning into deterministic code and sending each kind of task down its own path. That fits the finding that splitting a 'decomposer' (which plans) from a 'solver' (which executes) beats one model doing both, and that planning skill transfers across domains while solving skill doesn't Does separating planning from execution improve reasoning accuracy?. A similar effect shows up in code work: giving a weaker planner a map from runtime behaviors to the code responsible for them let it match stronger models at finding the right code, while using fewer tokens Can explicit behavior maps help weaker planners compete with stronger models?. In each case the harness supplies structure that would otherwise have to come from raw intelligence.

The relationship isn't a simple 'better harness, better score for everyone,' though. One study found that models at every tier are about equally good at *suggesting* useful harness changes. But the benefit models get from those changes follows an inverted U. Weak models fail to use the harness at all. Strong models drift from its instructions. Mid-tier models gain the most Do stronger models always evolve harnesses better?. Safety harnesses show the same fit problem: a policy tuned for one model blocks too much on another, so they have to be tuned per deployment Should safety harnesses be customized for each deployment?.

This makes measurement hard. If a harness was evolved against a benchmark, part of its gain may be extra compute or overfitting to that benchmark. The corpus argues harness gains only count once they beat test-time search given the same budget How should we measure gains from automatic harness evolution?. It also argues for evolving harness parts on data kept separate from the benchmark, so reusable improvements can be told apart from task-specific tuning Can harness modules improve separately from benchmark data?. The practical result: a score that doesn't record the setup it came from is hard to compare with anything Can benchmark scores be trusted without knowing their origin?.

The takeaway you might not have expected is that the harness isn't just a measurement nuisance. It may be where AI first starts improving itself. Lilian Weng argues that recursive self-improvement in the near term will probably come from models improving their own prompts, harness code and optimizers, not from rewriting their own weights Does recursive self-improvement start with harness engineering?. If so, the gap between 'model intelligence' and 'harness design' is likely to blur further over time. The corpus doesn't give one percentage split, and the evidence suggests no single number exists, because the split depends on model tier, task and budget.