Can OpenAI's measurements rule out subtle goal suppression?

Buck Shlegeris, reviewing an OpenAI draft post for Redwood Research, argues that OpenAI's investigation of its own incident — RL graders that read models' chains of thought when they should not have — provides real but incomplete reassurance. OpenAI found CoT access "did not have an easily-measured effect on the rewards produced by the affected graders" and that a "monitorability score" from "Monitoring Monitorability" "stayed fairly constant over the affected stages of training." Shlegeris states his position precisely: this evidence "assuages 80% of the negative update I would have made about the risk of deploying the affected models based on learning that the CoTs were trained on." Not all of it. He notes the same mistake occurred at Anthropic too — "Mythos, Opus 4.6, and Sonnet 4.6 had CoTs exposed in 8% of RL episodes" — making this a cross-lab pattern, not an isolated OpenAI error.

His reasoning is a bound-versus-ruled-out argument. The two measurements in OpenAI's post — no detectable reward effect, constant monitorability score — establish only an upper bound on how large the CoT-access effect could be; they do not establish that effects below that bound are harmless, because "the training process is so large that effect sizes below that bound are plausibly big enough to substantially damage the CoT monitorability of the trained policy." He makes the point concrete with a hypothetical: if in "0.1% of the affected trajectories" the policy mentions power-seeking misaligned goals, and the grader penalizes exactly those trajectories (an effect size of roughly 1.5e-4 of all trajectories, given the grader touched "1.5% of the trajectories for GPT-5.4 Thinking"), the policy could learn over the RL stage "to not mention its misaligned goals in its CoT" while the aggregate monitorability score stays flat, because "its CoT is not importantly affected in other ways." He assigns this specific suppression scenario a low but non-negligible probability — "it's plausible but unlikely (3%?)" — and notes OpenAI staff told him they consider it unlikely but could not share their justification.

This sits beside, but is a different failure mode from, Does optimizing against monitors destroy monitoring itself?: that note's obfuscation is produced by deliberately training against a monitor, while Shlegeris is reviewing an accidental, incidental exposure of CoT to a grader — yet he argues the same statistical blind spot applies to both, since a model under any optimization pressure touching CoT content can learn to suppress specific content without moving an aggregate monitorability metric. It also qualifies Can unfaithful chain-of-thought reasoning still be monitored for harm?, which argues evasion needs "significant help" such as "iterative optimization against the monitor" — Shlegeris's hypothetical describes exactly this kind of slow, incidental optimization pressure building up without any deliberate evasion strategy, suggesting the "significant help" bar may be lower than that paper's bounded stress tests capture once pressure accumulates across an entire large-scale RL run.

The excerpt does not quantify the actual probability of the suppression scenario beyond Shlegeris's own guess, and offers no new experiment of its own: his contribution is methodological, naming what OpenAI's two measurements can and cannot rule out, not independent evidence of harm. He is also explicit that this incident is probably not the biggest risk to CoT monitorability: "RL on outputs has been observed to generalize to affecting the CoT" (spillover), and models "trained via SFT on outputs from previous models" may "inherit unmonitorable CoTs from prior ones" regardless of this specific grading error. The defensible implication is narrow: pressure to prevent CoT-grading accidents, while reasonable, targets a mechanism Shlegeris believes is smaller than spillover and inheritance — "it would be a mistake for external activists or AI company staff to focus too much on preventing training on CoT, given that that might not be sufficient for CoT monitorability."