Can decoding-time prompting strategies fully replace diversity-focused training methods?
This explores whether tricks applied at generation time — clever prompting, sampling, or decoding-time tuning — can substitute for training methods specifically designed to keep a model's outputs varied, rather than just being a cheaper add-on.
This explores whether tricks applied at generation time — clever prompting, decoding-time tuning, sampling — can substitute for training methods that exist specifically to keep a model's outputs varied. The corpus points fairly consistently to no, and the reason is more interesting than a simple capability gap: diversity is mostly something that gets *destroyed* during training, and you can't prompt your way back to variety the training process already collapsed.
The ceiling is set by a basic fact about prompting. Prompt optimization only reorganizes what's already in the model's distribution — it can't inject anything that isn't there Can prompt optimization teach models knowledge they lack?. That's a strong result, because formally a single transformer is Turing-complete and a prompt can in principle steer it to compute almost anything Can a single transformer become universally programmable through prompts? — but the same work notes that standard training rarely produces models that actually behave that programmably. So the theoretical reach of prompting is huge, and the practical reach is bounded by what training left on the table.
And training tends to leave very little diversity on the table. Reinforcement learning collapses format diversity within the first epoch, amplifying one dominant pretraining pattern and suppressing the rest Does RL training collapse format diversity in pretrained models?; the same entropy-collapse mechanism squeezes exploration in search agents, while SFT on varied demonstrations preserves breadth Does reinforcement learning squeeze exploration diversity in search agents?. That last finding is the crux of the answer: the paper explicitly frames diversity-preservation as a *training-time* technique that's essential for scaling. Critique-in-the-loop work makes the same case — keeping solution diversity alive during self-training is described as more fundamental than any test-time accuracy gain Do critique models improve diversity during training itself?. If the variety is gone by the time you're decoding, decoding-time strategies have nothing to resurface.
The most striking evidence that you can't sample your way out of this is the "artificial hivemind" finding: across 70+ models and 26K open-ended queries, different models independently converge on near-identical answers because they share training data and alignment procedures Do different AI models actually produce diverse outputs?. If even *ensembling entirely separate models* doesn't buy you diversity, then re-rolling outputs from one model at decode time certainly won't. The constraint lives upstream.
Where decoding-time methods genuinely shine is the adjacent problem they're often confused with: adapting or steering a model *without corrupting* what it already knows. Proxy-tuning shifts behavior at decoding time while leaving base weights untouched, preserving pretrained knowledge better than direct fine-tuning Can decoding-time tuning preserve knowledge better than weight fine-tuning?; inference-time expert composition mixes skills on the fly Can models dynamically activate expert skills at inference time?; and routing adaptation into a fast textual-context channel rather than weights avoids catastrophic forgetting Can splitting adaptation into two channels reduce forgetting?. These are real wins — but they're about *protecting* an existing distribution, not *broadening* one that training narrowed. So the honest synthesis is a division of labor: decoding-time methods are the better tool for steering and knowledge-preservation, while diversity itself is a property you have to defend during training or lose for good — much as a model's reasoning protocol is baked in by training and can't be recovered by spending more compute at inference Can non-reasoning models catch up with more compute?.
Sources 10 notes
Prompting works entirely within a model's pre-existing training distribution and cannot supply domain knowledge absent from training data. This creates a hard ceiling: no prompt strategy can compensate for missing foundational knowledge, only reorganize what already exists.
Research proves a single finite-size transformer exists that can compute any computable function given the right prompt, achieving complexity bounds nearly matching unbounded models. However, standard training rarely produces models that learn to implement arbitrary programs this way.
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.
RL training compresses behavioral diversity in search agents through the same entropy collapse mechanism documented in reasoning—policies converge on narrow reward-maximizing strategies. SFT on diverse demonstrations preserves exploration breadth, suggesting diversity-preservation techniques are essential for RL search scaling.
Step-level critique in the training loop counteracts tail narrowing and maintains solution diversity across self-training iterations. This training-time benefit—preventing premature convergence—is more fundamental than test-time accuracy gains.
INFINITY-CHAT analyzed 70+ models across 26K open-ended queries and found an "Artificial Hivemind" effect: models independently generate strikingly similar or identical responses due to overlapping training data and alignment procedures, undermining the diversity benefits of model ensembles.
Proxy-tuning closes 88-91% of the alignment gap while surpassing direct fine-tuning on knowledge tasks by leaving base model weights untouched. Direct fine-tuning corrupts knowledge storage in lower layers, whereas proxy-tuning applies distributional shifts that primarily affect reasoning and style.
Transformer2 demonstrates that tuning only singular values within weight matrices produces composable expert vectors that dynamically mix at inference without interference, outperforming LoRA with fewer parameters and enabling continual specialization.
Fast-Slow Training routes task-specific lessons into optimized prompts while keeping parameter updates minimal, reaching equivalent performance 1.4–3x faster with substantially less catastrophic forgetting and plasticity loss, demonstrating that forgetting is a misallocation problem rather than an inherent cost.
Reasoning models persistently outperform non-reasoning models regardless of inference budget because training instills a reasoning protocol that makes additional tokens productive. The gap is fundamentally about deployment mechanisms and training structure, not raw capability.