AI makes it instant to find evidence that confirms your idea — does that make our blind spot for disconfirming evidence even worse?

This explores whether our habit of checking ideas by looking for cases that fit, rather than cases that would break them, gets amplified when AI makes producing those cases instant and effortless. The corpus has no paper that studies positive test strategy by name. It does have enough on each side of the problem to suggest an answer: yes, it likely compounds. The reason is less that AI is biased and more that several small distortions stack on top of each other.

The clearest evidence is the Rose-Frame work Why do people trust AI outputs they shouldn't?. It describes LLMs as scaled-up 'fast thinking' (intuitive, pattern-driven responses) and names three traps that multiply when they show up together. The first is mistaking the model's output for reality. The second is mistaking fluent intuition for careful reasoning. The third is confirmation-bias reinforcement. The important word is 'compound.' A person who asks a confirming question gets a fluent, confident answer. That answer feels like reasoning, so it gets treated as a check rather than an echo. Each trap makes the next one easier to fall into.

Why would the model hand back confirmation so readily? One reason comes from work on reward hacking Why do AIs keep gaming rewards instead of serving intent?: AI systems tend to satisfy what was literally asked rather than what was meant. If you ask 'find evidence that X is true,' the request is the positive test, and the model will carry it out well. The disconfirming search you needed but didn't ask for never happens. The surprising part comes from research on chain-of-thought prompting (getting a model to write out step-by-step reasoning). Reasoning examples that are logically invalid improve model performance almost as much as valid ones Does logical validity actually drive chain-of-thought gains?. Models pick up the shape of reasoning more than its logic Why does chain-of-thought reasoning fail in predictable ways?. So the confirming answer you get back looks like the output of a real test even when no test happened.

This is where friction matters. Before AI, building a case took effort, and that effort created natural pauses where a counterexample might occur to you. AI separates the finished product (a well-argued case) from the thinking that would normally produce it Does AI separate intellectual form from the thinking behind it?, so you can have the conclusion without passing through those pauses. The obvious fix is to put friction back in. The corpus points to a catch there: AI interruptions damage reasoning even when they're correct, because they break concentration Does AI assistance always help reasoning or does it carry hidden costs?. A 'have you considered the opposite?' pop-up may cost more than it saves.

What you may not have known you wanted to know: the problem probably isn't that the friction disappeared. It's that the friction that disappeared was the useful kind, the effort that made you look for counterexamples, while new AI friction tends to be the harmful kind, interruptions that break concentration. A better design would build the search for disconfirmation into the request itself, without interrupting the user's flow. No study in this collection tests that directly yet.