Language Understanding and Pragmatics LLM Reasoning and Architecture Design & LLM Interaction

Why do search tools fail against AI generated content?

Internet search worked for finding needles in haystacks of fixed documents. But AI generates new content on demand with no underlying corpus to search. Does this require fundamentally different solutions?

Note · 2026-04-14
What do language models actually know? What happens to social order when AI removes ritual constraints?

The standard "we are drowning in information" complaint conflates two structurally different inflations. The internet-era inflation was inflation of access: more documents indexed, more sources connected, more facts aggregated. The underlying corpus was fixed — written by humans, anchored to specific sources, persistent across queries. The problem was finding the relevant item in a growing pile. Search, filtering, and curation are the right responses because they operate on stocks.

AI inflation is categorically different. There is no fixed corpus to search — there is generation. The output does not exist until the moment of prompting. Each generation is contextual, disposable, unrepeatable. The marginal cost of additional generation is near zero, so supply is not constrained from the production side. The problem is not finding the relevant item but distinguishing a fabricated item from a grounded one in a stream that has no stock to anchor against.

The diagnostic difference matters because the prescriptions diverge. Stock-inflation prescriptions (better search, better filtering, better curation) misfire on flow inflation: there is nothing to index, nothing to filter ex ante, nothing to curate that will persist. Flow-inflation prescriptions need to operate on the production side (constrain generation, mark provenance, certify backing) or on the receiver side (cultivate evaluation capacity, build verification infrastructure). Tools designed for one inflation type do not transpose.

The deepest contrast is in grounding. Internet stock inflation was inflation of grounded items — every document was written by someone, somewhere, for a reason. AI flow inflation is inflation of ungrounded generations — produced by a statistical process whose connection to specific sources is at best probabilistic. AI knowledge is structurally hearsay — ungrounded, modified in every retelling, unverifiable against any stable source specifies what the grounding-loss amounts to. Why can't search tools handle AI-generated content? specifies why the standard tool fails.

The strongest counterargument: AI training data is the corpus, so AI inflation is still stock inflation in disguise. The reply is that training data is consumed-into-the-model, not retrievable from it — the model's outputs are not lookups against the corpus, they are samples from a distribution shaped by the corpus. There is no stock to search.


Source: Tokenization of Intelligence - Dialectic of Enlightenment

Related concepts in this collection

Concept map
12 direct connections · 55 in 2-hop network ·medium cluster

Click a node to walk · click center to open · click Open full network for a force-directed map

your link semantically near linked from elsewhere
Original note title

stock inflation versus flow inflation — internet inflated access to existing knowledge AI inflates production of new-seeming knowledge