Do few-shot examples improve in-context learning or add noise?

This explores whether few-shot examples genuinely improve in-context learning or sometimes just add noise — and the corpus answer is that it depends entirely on *which* examples and *how* they're arranged, not on how many you cram in. The most striking thread is that examples aren't all worth the same. Optimal experimental design beats simple similarity-based retrieval by picking demonstrations that most reduce uncertainty about the test set, which implies that naively grabbing the most 'similar' examples can leave performance on the table Can optimal experimental design improve few-shot example selection?. Ordering matters too: arranging demonstrations from harder to easier — measured by how sparsely the model represents them internally — yields real gains with no extra labels Can representation sparsity order few-shot demonstrations effectively?. So the same set of examples can help or hurt depending on selection and sequence.

The more counterintuitive finding is that *wrong* examples can teach more than right ones. The LEAP method deliberately induces the model to err on its few-shot examples, then has it reflect on the mistakes and write down explicit task principles — and this beats using clean correct demonstrations Does learning from mistakes improve in-context learning?. That reframes the whole question: an example's value isn't 'is it a correct answer' but 'does it surface a principle the model can articulate.' A related lesson from argument-quality work is that labeled examples alone often teach surface patterns rather than the underlying criteria — models need explicit frameworks, not just more instances, to generalize Can models learn argument quality from labeled examples alone?.

For some tasks, individual examples are the wrong unit entirely. In sequential decision-making, isolated demonstrations don't enable in-context learning at all — the model needs full or partial *trajectories* from the same environment, a property called trajectory burstiness Why do trajectories matter more than individual examples for in-context learning?. Here, scattering disconnected examples genuinely is noise; only coherent runs carry the signal. And at the extreme, the question of 'how many examples' can collapse: a single training example can be enough to activate latent reasoning ability that was already in the model Can a single training example unlock mathematical reasoning?, echoing the finding that even three exposures suffice to establish a priming effect Can we predict keyword priming before learning happens?.

The deepest reason examples sometimes add nothing is a hard ceiling: prompting can only reorganize and activate knowledge that already exists in the model's training, never inject what's missing Can prompt optimization teach models knowledge they lack?. And examples can be actively overridden — when a model's prior training associations are strong, it will ignore the context in front of it, so demonstrations that contradict deeply learned priors get drowned out rather than learned Why do language models ignore information in their context?. One way researchers fight this fragility is consistency training, which teaches models to respond the same way regardless of irrelevant wrapping or perturbation in the prompt — directly attacking the 'noise' side of the question Can models learn to ignore irrelevant prompt changes?.

The takeaway you didn't know you wanted: few-shot examples are best understood not as data the model trains on, but as *keys that unlock or fail to unlock* capabilities already inside it. Good examples are the ones that activate the right latent skill or expose a usable principle; noisy ones are those that fight the model's priors, sit as disconnected fragments where coherent trajectories were needed, or try to supply knowledge that was never there to begin with.