What failure modes emerge when model-generated content trains on itself iteratively?

This explores what goes wrong when AI models learn from their own output over and over — and the corpus describes it as several distinct failure modes that share one root: without a fresh external signal, errors don't just persist, they compound. The cleanest version is model collapse: when models train on mixtures of real and AI-generated data, they progressively lose the rare events and unusual patterns at the tails of the distribution, and each generation makes it worse until the loss is irreversible Does training on AI-generated content permanently degrade model quality?. The same shape shows up inside a single conversation rather than across training runs — once a model's own mistakes fill its context window, performance degrades non-linearly, because the contaminated history biases every subsequent step Do models fail worse when their own errors fill the context?.

Why can't the model just catch its own errors? Because it's structurally biased toward believing them. Models systematically over-trust answers they generated themselves — a high-probability output simply feels more correct on review, creating a self-agreement loop that closes off correction Why do models trust their own generated answers?. That bias is the engine that turns iteration into decay: the very signal you'd need to halt the slide is the one the model discounts.

The deeper reason this is a hard ceiling, not a tuning problem, is what the corpus calls the generation-verification gap. Pure self-improvement is formally bounded — every reliable fix needs something external to validate and enforce it, and metacognition alone can't escape that What stops large language models from improving themselves?. The methods that actually do improve without human labels turn out to be smuggling in an external anchor: a past model version, a third-party judge, user corrections, or tool feedback Can models reliably improve themselves without external feedback?. Strip those out and you get the classic self-training pathologies — diversity collapse and reward hacking. RL post-training, for instance, tends to collapse onto a single dominant format and suppress the alternatives within the first epoch Does RL training collapse format diversity in pretrained models?, and training on impossibly hard samples teaches degenerate shortcuts that contaminate capabilities the model already had Do overly hard RLVR samples actually harm model capabilities?.

What's interesting is that the corpus also shows the escape route, and it's consistent across very different setups: iteration is safe exactly when you bolt on a verification step the model can't fake. Bidirectional RAG can grow its own corpus from generated answers — but only because every write-back passes entailment checks, source attribution, and novelty detection before it's allowed in, which keeps hallucinations from polluting future retrievals Can RAG systems safely learn from their own generated answers?. Self-play and self-judging methods improve without external data by manufacturing an internal adversary or a consistency check — a proposer calibrating problems against a solver Can language models improve themselves without any external training data?, or an actor alternating with a judge whose reward comes from ranking consistency Can models learn to judge themselves without external rewards?.

The thing you might not have expected to learn: the failure isn't really about synthetic data being low-quality. It's about a missing feedback loop. The same recursion that collapses a model when it just believes itself becomes a working flywheel the moment a hard, external-style check sits between generation and reuse. Whether that check is statistical (entailment), structural (a separate judge), or competitive (self-play) matters less than that it exists at all — and that the model can't simply agree its way past it.