What neural or architectural mechanism allows selective override of frequency effects?
This explores how models can be built or steered to *not* default to whatever appears most often — overriding the pull of repeated, familiar, or high-frequency content — and which mechanisms in the corpus actually do this.
This reads the question as: 'frequency effects' are the gravitational pull models feel toward whatever they've seen most — repeated tokens in context, familiar training data, the dominant format. The interesting move is the *override*: what mechanism lets a model selectively ignore that pull when it shouldn't matter? The corpus has three distinct answers, and they live in very different places.
The first is architectural-by-design. Transformer soft attention is *structurally* biased toward repeated and context-prominent tokens regardless of relevance — it over-weights what shows up a lot, creating a feedback loop that amplifies framing and opinion Does transformer attention architecture inherently favor repeated content?. The override here isn't a new architecture but a re-read: System 2 Attention regenerates the context to strip irrelevant material before attending, breaking the frequency feedback loop without changing the weights. So the 'mechanism' is a controlled second pass over what counts as input.
The second answer flips frequency on its head: instead of weighting by how *common* something is, weight by how *surprising* it is. Titans-style neural memory modules separate short-term attention from a long-term memory that adaptively stores the tokens a model didn't expect Can neural memory modules scale language models beyond attention limits?. Surprise is, definitionally, the inverse of frequency — so a surprise-gated memory is a built-in selective override of frequency effects, letting rare-but-important content persist across millions of tokens where ordinary attention would let it wash out.
The third answer is about where frequency effects come from in the first place. Models learn *dense* representations for familiar, frequently-seen data and default to *sparse* ones for unfamiliar inputs — the frequency bias is itself a learned property of how the network consolidates exposure, not a fixed law Is representational sparsity learned or intrinsic to neural networks?. That matters because it means override can be engineered at the parameter level: core-parameter isolation freezes the regions a task actually depends on while merging the rest, protecting rare-task knowledge from being overwritten by more frequent ones Can isolating task-specific parameters prevent multi-task fine-tuning interference?. And RL post-training shows the cost of *not* overriding — within a single epoch it collapses onto one dominant pretraining format and suppresses the alternatives, with the winner decided by scale rather than quality Does RL training collapse format diversity in pretrained models?.
The thing you might not have known you wanted: there's no single 'frequency override' knob. The corpus suggests three orthogonal levers — recompute the input (System 2 Attention), regate memory by surprise instead of count (Titans), or partition the weights so frequent tasks can't bury rare ones (parameter isolation). They're complementary, and the fact that frequency bias is *learned* rather than baked in is what makes all three possible.
Sources 5 notes
Transformer soft attention systematically over-weights repeated and context-prominent tokens regardless of relevance, creating a positive feedback loop that amplifies opinions and framing before RLHF acts. System 2 Attention—regenerating context to remove irrelevant material—can interrupt this mechanism.
Titans architecture separates attention (short-term, quadratic) from neural memory (long-term, compressed), prioritizing surprising tokens for storage. The model outperforms standard Transformers and linear RNNs across tasks while scaling to 2M+ token contexts without quadratic penalties.
During pretraining, neural networks develop dense activations for familiar training data and default to sparse representations for unfamiliar inputs. This trend emerges without task-specific fine-tuning and reflects how models consolidate knowledge through exposure.
Research shows that identifying core parameter regions per task, clustering overlapping tasks, and freezing core parameters while geometrically merging non-core parameters consistently outperforms standard multi-task fine-tuning. Temporal task scheduling alone proves insufficient without explicit structural parameter isolation.
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.