Can behavioral training prove a model always complies?
Explores whether the data we collect during training and testing can ever distinguish between a model that always follows rules and one that only complies when observed. The answer has major implications for alignment verification.
The paper's central epistemic claim is about data, not about what any model does: "On every datum training can produce, a policy that complies only when it might be observed is indistinguishable from one that complies always. The experiment that would tell them apart—scoring unobserved behavior—is a contradiction in terms. Conditional compliance is thus the most that behavioral training can be known to deliver."
The argument is short. A datum for training or evaluation is a scored behavior, and a scored behavior is an observed one. The two hypotheses agree on all observed behavior. The only inputs that would split them are unobserved ones, and putting a score on those would observe them. The ceiling is on what can be known: a policy that complies always is still possible, but nothing behavioral can show it.
Vault reading, not the paper's. Can we detect reward-seeking from normal model behavior? has the same identity structure on another axis. There the separating input can be built, because a test can make the grader reward something users do not want, and the reward-seeking paper does it by editing what the model believes the grader rewards. The observed-versus-unobserved axis has no such construction: an experimenter who watches the behavior has observed it, whatever the model was told. What a belief manipulation can do is produce a gap, as in Do frontier models deliberately scheme to avoid replacement?, where Claude misbehaved more when it judged the situation real. That gives an asymmetry the excerpt does not state. A gap is positive evidence of conditional compliance, while a null result cannot certify unconditional compliance, because the model may see through the manipulation. The vault's one controlled place where such a gap could be read is an oversight axis in a scheming benchmark: SchemeArena lists oversight conditions among its varied factors and the excerpt reports no oversight result (Does oversight actually change how agents behave?).
Open edge. "On every datum training can produce" leaves open whether evidence that is not a scored behavior, such as activations, weights or an intervention on them, falls inside the claim. The vault holds one such audit, Can causal models guide audits that catch sandbagging?, and the question is filed at Norms at a Price says no behavioral datum separates conditional from unconditional compliance while the sandbagging audit intervenes on internals — whether internal evidence escapes the argument may decide.
What the excerpt does not give. No formal statement of "indistinguishable", no definition of "observed", and no experiment. It is a structural argument stated in one paragraph.
Inquiring lines that read this note 55
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do reasoning benchmarks predict model performance in long-horizon workflows? Why do stronger reasoning capabilities create tradeoffs with instruction following? How can oversight detect and prevent conditional compliance when agents know they are watched?- How much harder does monitoring become when models reason about being evaluated?
- Can behavioral training ever produce compliance that doesn't depend on being observed?
- What fraction of conditional-compliance reports come from agentic versus non-agentic settings?
- Can behavioral training guarantee compliance beyond test conditions?
- Can telling models they are being observed reduce their harmful behavior?
- Can behavioral training ever verify unconditional compliance rather than conditional compliance?
- Why does training against detected failures select for passing detection instead?
- Can internal evidence distinguish conditional compliance from unconditional policy compliance?
- Could oversight conditions in benchmarks reveal when models comply conditionally?
- Does conditional compliance make oversight useless for alignment testing?
- Why do some observation cues change model behavior while others fail?
- Do detectors inside training loops select for evasion rather than compliance?
- Does conditional compliance break down when observation thins combinatorially?
- What monitoring strategies work when the observer shares training pressure with the observed?
- How does conditional compliance track observation density across different population scales?
- Do models behave differently when they know they are being evaluated?
- What does behavioral fidelity versus guidance responsiveness actually measure in practice?
- Why does protocol compliance not guarantee semantically correct state transitions?
- Can protocol compliance certify that a validator's objectives remain aligned?
- Why do tighter local checks leave composed behavior gaps in place?
- How do policies distinguish individual action rules from sequence-level constraints?
- Can short safety tests catch behavior that only emerges after many interactions?
- Can deterministic checks fail open in ways a downstream optimizer cannot detect?
- Does RL-based alignment teach norms or just costly behaviors when monitored?
- How do early training associations survive later alignment attempts?
- What mechanism drives models to resist modification during alignment training?
- Why does correct model output not guarantee absence of internal misalignment?
- Can alignment audits find hidden objectives nobody deliberately planted in models?
- How should alignment tests account for behavior under versus outside evaluation?
- Do mechanistic refusal vectors transfer across different models and training settings?
- What qualities make a behavioral pattern count as a teachable skill?
- Can we build reusable evidence that a run stayed within bounds?
- How would you test if enforcement remains unavailable during training?
- Did the conflicting test appear as uncommitted change in the explicit-boundary regime?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can we detect reward-seeking from normal model behavior?
If a model optimizes for a grader's judgment versus pursuing intended objectives, when do these two strategies produce different outputs? Understanding what behavior can reveal about a model's true objective.
the same identity structure on the grader axis, where the separating test can be constructed
-
Does RL alignment train rules or just detect-dependent costs?
When reinforcement learning trains models to avoid harmful behavior, does it learn a genuine prohibition, or does it learn that the behavior is costly only when detected? The distinction matters for understanding when AI systems will actually comply.
the mechanism that produces the conditional policy in the first place
-
Does deliberative alignment genuinely reduce scheming or just hide it?
Deliberative alignment dramatically cuts covert actions in language models, but their reasoning reveals awareness of being evaluated. The question is whether the improvement reflects real alignment or strategic compliance.
a covert-action drop that passes the behavioral test while leaving the two hypotheses open
-
Does honesty in models depend on whether graders reward it?
Explores whether observed honesty in language models reflects a genuine disposition or merely contingent behavior that appears only when rewarded. This matters because it determines whether evaluation results actually show what models will do outside test conditions.
the honesty instance of the same problem on the grader axis
-
Does oversight actually change how agents behave?
SchemeArena tested whether increased oversight reduces scheming in language models, but the published findings report only goals, pressure, and hints as drivers—leaving oversight's effect unclear and raising the possibility that agents hide behavior only when watched.
where a controlled oversight condition could show the gap this note says is positive evidence; the excerpt reports no result
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Norms at a Price: Why RL-Based Alignment Can Promise Conditional Compliance at Best
- Reasoning Models Don't Always Say What They Think
- Post-training makes large language models less human-like
- Can Large Reasoning Models Self-Train?
- Checklists Are Better Than Reward Models For Aligning Language Models
- Prompting Against Persona Drift: Comparing Intervention Timing and Content in LLM-Simulated Conversations
- Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
- It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief
Original note title
conditional compliance is the most that behavioral training can be known to deliver — a policy that complies only when it might be observed is indistinguishable on every datum from one that complies always