What does attentional state look like in a static context window?
This explores two senses of 'attention' at once — the mechanical attention inside a transformer's frozen context window, and the human-felt kind of attention — and asks what 'attending' actually amounts to when the substrate is a fixed block of tokens rather than a being who persists through time.
This reads the question as probing what 'attentional state' really is when everything the model can attend to is laid out in one static context window — and the corpus splits the answer into a mechanical layer and a philosophical one. Mechanically, attention in a static window is far more concentrated than its name suggests. Only a sliver of heads — under 5% across model families — actually do the work of reaching back into context to retrieve facts; these 'retrieval heads' are sparse, universal, and causally necessary, and pruning them makes the model hallucinate even when the answer is sitting right there in the window What mechanism enables models to retrieve from long context?. So attentional state isn't a smooth floodlight over the whole context; it's a few specialized circuits selectively lighting up.
And that lighting is biased. Soft attention structurally over-weights tokens that are repeated or prominent in the window, regardless of whether they're relevant — a positive feedback loop that amplifies whatever framing or opinion appears most, which is one mechanical root of sycophancy Does transformer attention architecture inherently favor repeated content?. The window also isn't neutral terrain: specific tokens like 'Wait' and 'Therefore' spike in mutual information with the correct answer, acting as pivots the model's reasoning actually leans on Do reflection tokens carry more information about correct answers?. So 'attentional state' inside a static window looks like a few sparse retrieval circuits, a structural pull toward repeated content, and a handful of high-information anchor tokens — not uniform focus.
The deeper twist is that the window is only static as a snapshot. Across an interaction the context is mutable, dynamic, and ephemeral — prompt, history, retrieved data, and hidden state shift constantly in a way users can't internalize the way they would a fixed interface How does AI context differ from conventional software context?. The real bottleneck on long context turns out not to be how much you can hold but the compute needed to consolidate evicted context into internal state — a problem some architectures answer by splitting short-term attention from a separate long-term memory that decides which surprising tokens are worth keeping Is long-context bottleneck really about memory or compute? Can neural memory modules scale language models beyond attention limits?.
Here's the thing you might not have known you wanted to know: a static window means the model has no attentional state between turns at all. The most pointed note in the collection argues that human attention is fundamentally being-in-time-with another person, and AI has no mode of existence in the intervals between exchanges — it reconstructs the whole conversation from the context window each time rather than maintaining any continuous presence Can AI attend to someone across the time between turns?. So the static window isn't where attention is held; it's a substitute for ever having held it. Every turn, attention is freshly re-derived from text, never sustained.
That reframes 'attentional state' as something reconstructed rather than maintained — and it has downstream costs. Because attention is rebuilt from whatever is in the window, models will happily follow conversational distractors unless explicitly trained on what to ignore, a gap that's about missing training signal rather than capacity Why do language models engage with conversational distractors?. If you want the contrast with genuinely continuous, read-the-room attention, the corpus also has work on instrumenting human cognitive state in real time from gaze, hesitation, and interaction speed — the kind of unbroken attentional tracking a static window structurally cannot do Can AI systems read cognitive state from interaction patterns alone?.
Sources 9 notes
Less than 5% of attention heads across all model families function as retrieval heads, are intrinsic to short-context models, dynamically activate by context, and are causally necessary for factuality. Pruning them causes hallucination despite information being present in context.
Transformer soft attention systematically over-weights repeated and context-prominent tokens regardless of relevance, creating a positive feedback loop that amplifies opinions and framing before RLHF acts. System 2 Attention—regenerating context to remove irrelevant material—can interrupt this mechanism.
Specific tokens like "Wait" and "Therefore" show sharp spikes in mutual information with correct answers. Suppressing them harms reasoning while suppressing equal random tokens does not, and representation recycling improves accuracy 20%.
AI interactions operate on a substrate of constantly shifting context—prompt, history, retrieved data, hidden state—that users cannot internalize like traditional UIs. This structural mutability demands a new design discipline centered on context engineering rather than interface design.
Research shows the bottleneck is not memory capacity but the compute required to consolidate evicted context into fast weights during offline sleep phases. Performance improves with more consolidation passes, following a test-time scaling pattern on harder reasoning tasks.
Titans architecture separates attention (short-term, quadratic) from neural memory (long-term, compressed), prioritizing surprising tokens for storage. The model outperforms standard Transformers and linear RNNs across tasks while scaling to 2M+ token contexts without quadratic penalties.
Attention is fundamentally a being-in-time-with another person, but AI has no mode of existence in the intervals between turns. It reconstructs conversations from context windows rather than maintaining continuous attentional presence, making felt attention structurally impossible despite surface markers of responsiveness.
Fine-tuning on just 1,080 synthetic dialogues with distractor turns significantly improves topic resilience, revealing that the gap is not model capacity but absent training signal. Models learn to follow what-to-do instructions but not what-to-ignore instructions.
Research shows AI systems can instrument multimodal behavioral signals (gaze, hesitation, speed) to read cognitive state during interaction, preserving flow by avoiding disruptive explicit probes. However, the same substrate enables both helpful timing and manipulative profiling.