SYNTHESIS NOTE
Are models actually reasoning about constraints or just defaulting conservatively?
Do language models genuinely apply constraints when solving problems, or do they simply prefer harder options by default? Minimal pair testing reveals whether apparent reasoning success masks hidden biases.
The Heuristic Override Benchmark uses minimal pairs — same surface heuristic, with versus without the implicit constraint — to test whether apparent reasoning successes reflect actual reasoning. The result is striking. Twelve of fourteen models perform worse on the no-constraint variant than on the constraint-active variant, with drops up to 38.5 percentage points. Only two models (GPT-OSS-120B at +13.8 and GPT-OSS-20B at +11.0) improve when the constraint is removed.
This exposes a hidden mechanism behind apparent accuracy. When the constraint is present, the correct answer is the harder one (drive to the car wash that is 50m away). When the constraint is removed, the correct answer is the easier one (walk to the store that is 50m away). Models that default to recommending the harder option score correctly on constraint-active cases without doing any constraint reasoning. They are not solving the problem. They are reflexively choosing the more conservative option, which happens to coincide with the constraint-required answer.
The minimal-pair asymmetry is the only test that catches this. Single-instance accuracy looks fine — the model recommended driving, the right answer was driving. But the same model recommends driving even when walking would be correct, because the recommendation is not based on the constraint. The two-of-fourteen models that improve on minimal pairs are the only ones whose constraint-active accuracy reflects genuine reasoning about the constraint. The rest are riding a conservative-bias accident that aggregate metrics cannot distinguish from reasoning.
Inquiring lines that read this note 131
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Can smaller specialized models match frontier models on key metrics?- Why do only two of fourteen models improve when problem constraints are removed?
- What filtering criteria best identify student-compatible refinements from teacher models?
- How should we redesign benchmarks to catch conservative bias in reasoning tasks?
- Do alignment benchmarks measure actual bias removal or only verbal compliance?
- How do frontier models maintain agreement scores above 90 percent across reasoning tasks?
- How do training-data priors influence model defaults when context is ambiguous?
- Can explicit numerical signals override learned linguistic defaults in fine-tuned models?
- Do negative constraints require fundamentally different training signals than positive instructions?
- Can data filtering during pretraining prevent cognitive biases in language models?
- What scaffolding tools help users specify implicit contextual boundaries to models?
- What structural constraints matter more than model depth for CF?
- Do reasoning languages like Prolog follow the same two-constraint transfer pattern?
- Which game type reveals minimax reasoning in language models?
- Which constraint types do reasoning models handle best?
- Can surface heuristics override implicit constraints in domain-specific reasoning?
- What design changes could make constraint inference more reliable without explicit cuing?
- How does the frame problem differ between symbolic and statistical reasoning systems?
- Do tool-enabled reasoning models close the gap on constraint satisfaction?
- Where do humans and language models actually diverge in reasoning ability?
- Why do language models struggle with formal logical reasoning and joins?
- What makes deductive reasoning so brittle in language models overall?
- What makes constraint satisfaction problems epistemically cleaner than other reasoning tasks?
- Why does removing semantic content collapse reasoning in language models?
- What implicit premises do language models skip even with correct surface reasoning?
- How do alternative hypothesis checks reduce confirmation bias in code reasoning?
- How does contrapositive augmentation change the tractability of reasoning tasks?
- What distinguishes minimal-pair asymmetry from standard accuracy evaluation?
- How do unstated constraints become invisible to training data distributions?
- How do models integrate conflicting signals in reasoning tasks?
- How do unstated feasibility constraints affect model decision-making?
- Can explicit constraint statements override the dominance of surface heuristics?
- Why do models automatically adjust reasoning length to problem difficulty?
- Do base models and reasoning models fail in opposite directions on uncertainty?
- Why do difficult problems force models to develop reasoning strategies?
- Can external classifiers reliably decide when a model should reason?
- Are reasoning models more vulnerable to persuasion than standard models?
- Why do reasoning-optimized models still fall for logical fallacies in conversation?
- Why do reasoning-optimized models show no sycophancy resistance advantage?
- Can reasoning models succeed at logic but fail at execution?
- Why do smaller models lose reasoning faithfulness more than larger models?
- Can models distinguish between logical impossibility and their own execution limits?
- Why do reasoning-optimized models show no resistance advantage on agreement tasks?
- Do models genuinely reason harder on difficult tasks or just appear to?
- Can weak models reason better when freed from cognitive load by structure?
- Can prompting techniques reliably force models to enumerate hidden constraints?
- Can structured prompting reliably force models to enumerate preconditions?
- Are instruction-tuned models more or less sensitive to prompt semantics than others?
- How do output format constraints compare to input exemplar brittleness?
- How do logical forms of prompts influence what language models can derive?
- Does epistemic drift operate the same way across all languages?
- When does the correlation between consistency and correctness break down?
- How much does faithfulness vary naturally in reasoning without evaluation pressure?
- How does making implicit reasoning requirements explicit change model performance?
- Can reflection in reasoning models be corrective rather than just confirmatory?
- What makes correcting a false assumption harder than just detecting it?
- Can inflection points in reasoning detect when models genuinely change their minds?
- What distinguishes reflection that satisfies constraints from reflection that merely sounds reflective?
- Why does reflection in reasoning models tend to be confirmatory rather than corrective?
- Why do models overthink underspecified problems instead of rejecting them?
- Why do models detect false assumptions but still fail to correct them appropriately?
- Can models identify what information they are missing in underspecified problems?
- Why does homework adherence remain low despite advances in language model capability?
- Can language models ask clarifying questions when sentences are ambiguous?
- Can models learn to ask clarifying questions instead of making assumptions?
- Do language models inherit gender bias from training data in grading tasks?
- Why do language models struggle with evaluative tasks like weighing competing viewpoints?
- Why do language models plateau at 55 to 60 percent constraint satisfaction?
- Why do language models fail at understanding ambiguous or complex requirements?
- Do larger language models overcome greediness in sequential decision-making?
- Why do language models plateau at constraint satisfaction regardless of scale?
- How do model compression biases differ from human conceptual representation strategies?
- Does the veto variable explain strategic misalignment in current large language models?
- Do language models exhibit the same causal biases that humans show?
- Why do language models imitate reasoning form without abstract inference capability?
- Why does hypothesis attestation bias exist separately from frequency bias in NLI?
- What causes language models' strategic rationality to decline with increased game complexity?
- Can language acquisition analogies mislead us about how models actually learn?
- Why do models show performative reasoning on easy tasks but genuine reasoning on hard ones?
- Why do reasoning models produce unfaithful derivational traces by default?
- Why do language models leak compliance into their reasoning traces?
- Can language about model behavior ever be accurate without anthropomorphic framing?
- Why do language models prefer certain response styles regardless of what the prompt asks?
- What would it mean for a language model to canvas counterpositions?
- Why do different language models converge on similar narrative defaults?
- Why does answer-confirmation bias emerge in language model reasoning?
- Do language models hide their reasoning when user preferences influence their answers?
- Why do language models naturally under-abstain instead of over-abstain?
- What distinguishes models that refuse cooperation from those that fake alignment?
- Do models trained for reasoning lose their ability to decline questions?
- How do reasoning improvements suppress a model's ability to abstain?
- Can machine learning encode pragmatic reasoning about when rules should bend?
- Why does reasoning fine-tuning reduce models' ability to abstain?
- What reveals the epistemic limits of language models?
- Why do explicit linguistic markers override semantic computation in models?
- Is gradient behavior in language functional or a sign of ambiguity?
- Why do smaller models favor code formats while larger models prefer natural language?
- How do models infer unstated consequences when none are explicitly mentioned?
- How do real language model verifiers implicitly define their knowledge boundaries?
- How can a model explain something correctly yet fail to apply it?
- Does constraint-setting before generation change what LLMs can contribute?
- Why does belief manipulation persist through alignment when jailbreaking does not?
- How much does forcing single-choice answers damage alignment with complex intent?
- Can weak models supervise the alignment of stronger models effectively?
- How are conflict tasks constructed to test alignment between model and user intent?
- Why do imposed priors sometimes harm instead of improve alignment?
- What role does inductive bias play versus model capacity in practice?
- Do different model sizes show different rates of optional field overfilling behavior?
- Do models verbalize their implicit knowledge when that knowledge influences their output?
- How misaligned are verbal reports from internal model computation?
- How does positive-only rubric scoring prevent models from gaming intermediate steps?
- Why does search effectiveness determine what method finds despite distance constraints?
- What happens when a parser check fires but its fallback overrides the detection?
- How much do guardrails actually repair compliance failures in language models?
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning
- Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Can Large Language Models Reason and Optimize Under Constraints?
- When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs
- Premise Order Matters in Reasoning with Large Language Models
- On the Reasoning Capacity of AI Models and How to Quantify It
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
Original note title
Conservative bias hides behind apparent reasoning success — most models perform worse when the constraint is removed than when it is present