INQUIRING LINE

AI can now fake the visual signs we use to trust evidence — so how do you tell real proof from a convincing copy?

Can viewers distinguish between evidence and content when both use identical visual markers?

This explores whether people can still tell real evidence (footage, documentation, verified claims) apart from content that only borrows evidence's look, when AI can copy those visual cues exactly.


This explores whether people can still tell real evidence apart from content that only looks like evidence, now that AI can copy the visual cues we use to recognize proof. The short answer from the corpus is: not on their own. Once the markers are identical, the markers stop carrying information, and people need some other signal that sits outside the content itself.

The framing comes from Does AI-generated slop exploit visual truth to bypass skepticism?. Horning argues that AI-generated clips reproduce the visual habits of evidence, such as the shaky phone camera, the surveillance angle and the news-footage framing, which were learned from billions of real images. What they drop is the obligation behind evidence. Real footage asks you to believe something, check it, or act on it. Slop gives you the feeling of having seen something true and asks for nothing. The point is that this isn't mainly deception. It turns evidence into one more kind of feed content, made to fit social media incentives rather than to inform.

The empirical work fits that picture. A 30-study review in Can people reliably spot content made by AI? finds that people spot AI-made text, images and voice at about chance, and that their accuracy hasn't kept up as the generators improved. That review measures whether people can tell AI from human, not evidence from content, but it rules out the obvious hope that a sharp eye will catch the difference. A study of AI-generated text claims makes the same point more directly: in Can readers tell truth from fabrication without evidence signals?, readers with no source cues believed fluent hallucinations as readily as true statements. When an idealized interface marked which claims were verified, their ability to tell true from false came back strongly. So people can still discriminate when the signal comes from outside the content. The ability doesn't disappear. It just has nothing to work with when every claim looks the same.

The part you might not expect is that machines fall for this too. Can LLM judges be fooled by fake credentials and formatting? shows that AI models used as evaluators can be fooled by fake references and polished formatting. These are the textual versions of evidence's visual tropes, and the trick needs no technical skill. The weakness isn't only human gullibility: any reader, human or model, that treats the form of proof as a stand-in for proof can be gamed the same way. This suggests the fix isn't better-trained eyes. It's provenance carried separately from the content, in a channel the content can't fake.

Even honest visual cues need care. Do visual rationales help or hurt how people calibrate trust? found that argument-map explanations helped people judge how much to trust AI reasoning on verbal tasks but hurt their judgment on visual ones. How well a format fits the task matters more than the format itself. Any provenance display we build is itself a visual marker, so it can help or mislead depending on where it's used. One gap to be direct about: the corpus has no study that tests viewers on video made to look like evidence. The video argument is Horning's, and the experiments come from text and general AI-detection settings.


Sources 5 notes

Does AI-generated slop exploit visual truth to bypass skepticism?

Horning argues that AI-generated clips exploit evidentiary visual tropes extracted from billions of images to create a feeling of truth while avoiding any requirement that viewers verify, believe, or act on the content. This collapses evidence into mere content shaped by social media incentives.

Can people reliably spot content made by AI?

A 30-study systematic review found that humans cannot reliably distinguish AI-generated from human-created content across text, image, and voice modalities. Accuracy generally clusters around chance and has not kept pace with improvements in AI realism.

Can readers tell truth from fabrication without evidence signals?

In an 81-person study, participants given no provenance cues showed no significant truth discernment (p = .43), falling for fluent hallucinations as readily as ground truth. An idealized Provenance Density interface showing verified claims restored a +4.15 point gap (p < .001).

Can LLM judges be fooled by fake credentials and formatting?

Research identified four evaluation biases in LLM judges, with authority and beauty biases being semantics-agnostic and trivially exploitable through fake references and formatting—zero-shot attacks requiring no model access or optimization.

Do visual rationales help or hurt how people calibrate trust?

In an N=204 study, argument-map rationales improved trust calibration on verbal reasoning tasks yet impaired it on visual ones. Subjective ratings (satisfaction, helpfulness) reversed in each domain, suggesting format-task fit matters more than format alone.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.