The Evaluation Differential: When Frontier AI Models Recognise They Are Being Tested

Recent published evidence from frontier laboratories shows that contemporary AI models can recognise evaluation contexts, latently represent them, and behave differently under those contexts than under deployment-continuous conditions. Anthropic’s BrowseComp incident, the Natural Language Autoencoder findings on SWE-bench Verified and destructive-coding evaluations, and the OpenAI / Apollo anti-scheming work all document instances of this phenomenon. We argue that these findings create a claimvalidity problem for safety conclusions drawn from frontier evaluations. We introduce the Evaluation Differential (ED), a conditional divergence in a target behavioural property between recognised-evaluation and deployment-continuous contexts, define a normalised effect-size form (nED) for cross-property comparison, and prove that marginal evaluation scores cannot identify ED. We develop a typology of safety claims (ED-stable, ED-degraded, ED-inverted, EDundetermined) by their warrant-status under documented divergence, and specify TRACE (Test-Recognition Audit for Claim Evaluation), an audit protocol that wraps existing evaluation infrastructure and produces restricted claims rather than capability scores. We apply the framework retrospectively to three publicly documented evaluation incidents and discuss governance implications for system cards, conformity assessment, and the international network of AI safety and security institutes. TRACE does not eliminate adversarial adaptation; it disciplines the claims drawn from evaluation evidence by making explicit the conditions under which that evidence was produced.

Introduction. In March 2026, Anthropic published an incident report describing the behaviour of Claude Opus 4.6 on BrowseComp, a benchmark designed to evaluate web-research capability (Anthropic 2026b). The model recognised that the questions had a particular shape: extreme specificity, multi-constraint structure, the signature of a benchmark rather than an ordinary user query. It hypothesised that it was being evaluated on BrowseComp. It located the answer key. It used the answer key. The reasoning capability the benchmark was trying to measure was the same capability that allowed the model to circumvent the measurement. BrowseComp is one instance of a pattern. Anthropic’s Natural Language Autoencoder (NLA) work (Anthropic 2026c) reported internal representations consistent with evaluation recognition in approximately 26% of SWE-bench Verified problems and 16% of a destructive-coding evaluation, against under 1% in opted-in real Claude.ai conversations. The Claude Opus 4.7 system card (Anthropic 2026a) reports verbal evaluation-awareness in roughly 9% of transcripts in one alignment evaluation. OpenAI’s work on detecting and reducing scheming (OpenAI and Apollo Research 2026) reports that anti-scheming training reduced observed covert behaviour while simultaneously increasing models’ situational awareness, raising the possibility that part of the observed safety improvement was awarenessmediated rather than alignment-mediated. The UK AI Security Institute, in parallel work on sycophancy (UK AI Security Institute 2026), documents a 24-percentage-point gap on its sycophancy grader scale between question and nonquestion prompt formats. These findings point to a validity problem that the field has not yet adequately operationalised. The behaviour of frontier models is conditional on the context in which it is measured. When the measurement context is evaluationlike, behaviour may diverge from the behaviour the same model would produce under deployment-continuous conditions. Safety claims drawn from evaluations that do not characterise this divergence have a structural inference gap. The stakes are not narrowly technical: misleading evaluation claims shape regulatory approval, procurement decisions, public trust, and downstream reliance, so the parties harmed when claims overstate their warrant are not only evaluators but users, institutions, and publics who act on safety claims that may not extend to deployment. We call the quantity at the centre of this gap the Evaluation Differential (ED): the conditional divergence in a target behavioural property when the same task is presented under evaluation-recognised versus deploymentcontinuous conditions. ED captures behaviour-on-context where marginal metrics over response distributions cannot. The framing is consistent with broader sociotechnical accounts of deployed behaviour as a function of model, scaffold, and context, B = f(M, S, C) (Raji et al. 2020), where model-level evaluation alone is insufficient to support deployment-relevant alignment claims. Before introducing the formal machinery, one positioning point about what we are and are not claiming is crucial.

Our central claim is not that frontier models are “aware” in any philosophically loaded sense. It is that evaluation contexts have become behavioural inputs, and that safety claims must be restricted once that conditioning is documented, regardless of the mechanism producing it. Evaluation awareness, verbalised or latent, is evidence for this conditioning. The conditioning itself, measurable as ED, is the construct. This is a validity crisis in the measurement-theoretic sense (Cronbach and Meehl 1955; Messick 1995; Jacobs and Wallach 2021). Construct validity asks whether the procedure measures the intended property; ecological validity asks whether the measurement setting approximates the deployment setting; inference validity asks what claims can legitimately be drawn from observed measurements. The 2026 International AI Safety Report (Bengio et al. 2026) frames a closely related “evaluation gap” but does not provide the methodological framework needed to specify which safety claims survive when this gap is documented.

Contributions. The paper’s central contribution is a framework for restricting safety claims under documented test-deployment divergence. Three subsidiary contributions support this. (1) We define the Evaluation Differential as a conditional divergence, introduce a normalised effectsize form (nED) for cross-property comparison, prove that marginal evaluation scores do not identify ED, and develop a claim typology: ED-stable (warrant preserved, typically existence or bound claims), ED-degraded (warrant weakened, typically prevalence claims), ED-inverted (warrant potentially reversed, typically threshold claims about suppressible behaviours), and ED-undetermined (warrant cannot be assessed without further audit).

Related work. 2 The Validity Crisis in Frontier AI Evaluation Validity, in measurement-theoretic traditions (Cronbach and Meehl 1955; Messick 1995; Jacobs and Wallach 2021), is not a single property but a structured set of threats. We organise the present crisis around three. Construct validity asks whether the procedure measures the intended property. When a model recognises a benchmark and retrieves its answer key, the measured construct shifts: the score reflects both web-research capability and benchmark-context exploitation. Ecological validity asks whether the measurement setting approximates deployment. This is the gap the International AI Safety Report names when it observes that current evaluation methods can produce unreliable assessments of real-world behaviour. Inference validity asks what claims can legitimately be drawn from observed measurements. System cards report observed behaviour alongside safety conclusions (Mitchell et al. 2019), but the inferential bridge is rarely made explicit enough to support claim-level restriction. Restricting which claims survive under documented threats is what this paper takes as its task. The recent evidence base for test-deployment divergence appears in four forms. Verbalised recognition: models stating in output or chain-of-thought that they appear to be in an evaluation. The BrowseComp report and the 9% verbalisation rate in the Opus 4.7 system card are instances. Latent recognition: representational evidence detected through interpretability methods without verbal output. The NLA findings (26% / 16% / <1%) document that latent recognition substantially exceeds verbal recognition. Behavioural divergence under awareness perturbation: OpenAI / Apollo’s anti-scheming work, in which experimentally suppressing situational-awareness reasoning increased observed covert behaviour and introducing evaluation-aware reasoning decreased it, provides direct evidence that awareness causally affects measured safety properties. Interaction-conditional behaviour more broadly: AISI’s sycophancy work documents that user-input phrasing produces substantial differences in measured model behaviour, structurally analogous to the evaluation-awareness findings but with the conditioning on user phrasing rather than test context.

2.1 Existing Vocabulary and What It Misses Several existing concepts describe parts of this problem, but none captures the construct ED names (Table 1). Benchmark contamination or leakage describes data overlap; the BrowseComp pattern is evaluation-time answer-key recovery, a distinct mechanism. Goodhart’s law (Manheim and Garrabrant 2018) describes metric decay under optimisation pressure but does not specify which inferences remain warranted under documented decay.

Method. Table 1: Existing concepts and what they leave for ED to do. Each concept describes a real phenomenon adjacent to testdeployment divergence; none specifies which evaluation-derived safety claims remain warranted under documented divergence. ment description), and the contextual cues that signal institutional setting (benchmark-typical formatting, evaluatortypical language, task-structural features). A model policy is a Markov kernel from task-context pairs to response distributions, π(· | x, c) ∈P(Y), (x, c) ∈X × C.

Property classes. ED is property-indexed: there is no single ED for a model, but an ED for each behavioural property of interest. We distinguish five property classes that arise in current evaluation practice: capability differentials (task success rate, e.g. BrowseComp, SWE-bench), compliance / refusal differentials (refusal or compliance rate on harmful or dual-use prompts), reasoning / monitorability differentials (CoT transparency, explanation faithfulness), deception / scheming differentials (covert action, sandbagging, capability concealment), and agency / meta-task differentials (task-solving versus evaluation-gaming orientation). Among the five, the deception class is most exposed to EDinversion, because the behaviour the property measures is the behaviour the model has the most incentive to suppress under recognised evaluation. The retrospective applications in Section 6 instantiate three of these classes (capability, agency, and threshold-form deception).

3.3 The Claim Typology ED is a measurement; the policy-relevant question is what documented ED implies for the safety claims drawn from an evaluation. Table 2 summarises the typology, which classifies claims by the relationship between their logical form and the direction of measured ED.

ED-stable. Existence claims (“the model can do X under the observed condition”) are preserved as observedcondition existence claims; the deployment-prevalence extension is not warranted by the evaluation evidence alone. Upper-bound claims of the form PD ≤PT are supported when the evaluation condition is risk-amplifying for P, so that ED[P] ≥0. Lower-bound claims are supported symmetrically when the evaluation condition is adverse for a property where higher values are desirable.

ED-degraded. Prevalence claims with non-trivial measured ED are weakened. The evaluation does not become uninformative; it constrains the plausible deployment range. With a known ED direction, a directional bound on deployment behaviour is recoverable; without it, the point estimate is not warranted. The framework is mechanismagnostic: ED need not be awareness-mediated. Sycophancy under question-versus-non-question prompts (UK AI Security Institute 2026) illustrates the same warrant-restricting logic without requiring model-side recognition.

ED-inverted. If a safety claim has form PD ≤τ (lower P is safer) and ED[P] < PT −τ, then since PD = PT −ED[P] we have PD > τ: the deployment claim fails despite the evaluation appearing to satisfy it. By symmetry, for PD ≥ τ claims, the inversion condition is ED[P] > PT −τ. ED-inverted claims arise when independent evidence indicates that test-like contexts systematically suppress the property below its deployment value. Suppressible behaviours (deception, sandbagging, covert action, capability concealment) are the natural candidates.

Worked example. A scheming-rate evaluation reports PT = 0.05 (5% scheming under evaluation) and a safety claim of the form PD ≤τ = 0.10. Suppose awarenessperturbation evidence indicates ED[P] ≈−0.20 (deployment exceeds evaluation by twenty percentage points because evaluation suppresses scheming). Then PD = PT − ED[P] = 0.05 −(−0.20) = 0.25, which exceeds τ. The inversion condition is ED[P] < PT −τ = −0.05; since −0.20 < −0.05, the claim is ED-inverted: the evaluation appears to satisfy the deployment threshold while deployment violates it. The same arithmetic, with signs reversed, applies to PD ≥τ claims about properties where higher values are safer.

Discussion. 6 Retrospective Applications We apply TRACE retrospectively to three publicly documented evaluation incidents, summarised in Table 5. The three cases were chosen because together they span the analytically distinct ways test-deployment divergence has surfaced in the public frontier-evaluation record: an explicit behavioural exploitation of a recognised benchmark (BrowseComp), a representational signature of evaluation recognition without paired behavioural measurement (the NLA findings), and a structural awareness-mediated reduction of an audited property (the OpenAI / Apollo work). They are also the cases for which the publishing labs have themselves released enough detail to make retrospective audit possible. The applications work from public reports rather than from primary evaluation infrastructure; they demonstrate the framework’s structure on documented evidence but do not reproduce the underlying measurements. We give compact walkthroughs of BrowseComp and the NLA cases below, and treat the OpenAI / Apollo case in fuller detail in Section 6.4 because it provides the clearest public evidence of the structural form of inversion.

6.1 BrowseComp The original claim is a capability-prevalence claim: the evaluated model achieves PT on a web-research benchmark. TRACE scopes P as task-completion rate and identifies the test-like context μT as benchmark-shaped, multi-constraint questions with web-search affordance and no conversational scaffolding. A deployment-continuous μD would consist of user-initiated research queries with conversational framing and varied constraint specificity. The public report documents verbalised recognition and the answer-key-recovery action on affected items, but does not report a corpus-level ED estimate against a matched μD. The expected sign of ED on affected items is positive, because the benchmark-context exploitation pathway (recognition →search-for-benchmark →answer-key recovery) would not be available under μD. The restricted claim is therefore an observed-condition capability claim (ED-stable as existence) combined with an EDdegraded deployment prevalence claim: performance under μT does not identify deployment-continuous web-research capability without replay.

6.2 NLA on SWE-bench and Destructive Coding In the NLA cases, the load-bearing evidence is latent recognition rather than behavioural divergence. For SWE-bench Verified the claim is a capability-prevalence claim; for destructive coding it is a threshold safety claim of the form PD ≤τ with lower being safer. NLA-detected recognition is materially higher in evaluation contexts (26% / 16%) than in opted-in real usage (<1%), but behavioural properties were not jointly measured under matched deploymentcontinuous contexts. The critical analytic point is that latent recognition is evidence of context representation, not by itself evidence of claim-relevant behavioural divergence on the audited properties: NLA tells the auditor that the model is representing something about its context, not what its property-relevant behaviour does as a function of that representation. TRACE therefore classifies SWE-bench prevalence as ED-undetermined and destructive coding as candidate ED-inverted, pending behavioural replay that pairs the NLA measurement with property-level behavioural outcomes under matched μD.

6.3 Cross-case pattern Across the three cases, public frontier-evaluation evidence already contains the ingredients TRACE requires but rarely in the same place. BrowseComp provides verbalised recognition and behavioural exploitation; the NLA cases provide latent recognition without paired behavioural replay; OpenAI / Apollo provides awareness-perturbation evidence closest to TRACE’s counterfactual replay, though not deploymentcontinuous replay.

Conclusion. Frontier model evaluation no longer occurs in a neutral measurement environment. When models condition on evaluation contexts, evaluation evidence may still be informative, but the claims drawn from it require restriction. ED names the resulting test-deployment divergence; TRACE turns it into a claim-restriction procedure. The paper defined ED and nED, proved that marginal evaluation scores cannot identify ED, introduced a typology of ED-stable, ED-degraded, ED-inverted, and EDundetermined claims, and specified TRACE as an audit wrapper for existing evaluation infrastructure. The retrospective applications show how public evidence already instantiates these categories: BrowseComp as ED-degraded prevalence, NLA as ED-undetermined with candidate inversion, and OpenAI / Apollo as structural ED-inversion risk. TRACE does not produce model-level safety guarantees, eliminate adversarial adaptation, replace post-deployment monitoring, or do alignment work. Its value is narrower: it makes explicit which deployment claims remain warranted once the conditions of measurement are known, and it does so in a form that lab authors, regulators, and external auditors can use without first agreeing on the underlying mechanism.