SYNTHESIS NOTE
Topics›Personalization›this note

Do user outputs outperform inputs for LLM personalization?

Does a user's history of outputs (responses, endorsed content) matter more for personalization than their input queries? This explores what actually drives effective personalization in language models.

Synthesis note · 2026-02-23 · sourced from Personalization

A study on user profile roles in LLM personalization surfaces a counterintuitive finding: the outputs users have produced or endorsed matter far more than the inputs they submitted. Using only the output part of user profiles achieves comparable or even superior performance to complete profiles across multiple LaMP tasks. Using only the input part leads to noticeable degradation.

This finding separates personalization from two adjacent paradigms:

Personalization ≠ RAG. Retrieval-augmented generation relies on semantic similarity between the input query and retrieved documents. Personalization works through a different mechanism — it is the style, preferences, and judgments expressed in historical responses that calibrate the model, not the semantic content of past queries.

Personalization ≠ ICL. In-context learning uses complete input-output pairs as demonstrations. Personalization requires only the output side — the response patterns that reveal who the user is and what they value.

The practical implication: when designing personalization systems under input length constraints, prioritize incorporating user-generated or user-approved responses over query histories. This unlocks the potential to include many more user profiles within limited context windows, because output-only profiles are both more effective and more compact than complete interaction histories.

A secondary finding adds a structural dimension: user profiles integrated closer to the beginning of the input context have more influence on personalization than those placed elsewhere. This parallels the positional bias documented in ICL — since How much does demo position alone affect in-context learning accuracy?, the spatial attention pattern appears to be domain-general, affecting personalization placement decisions as well as few-shot learning.

The output-over-input finding connects to the broader question of what personalization is. Since Can text summaries beat embeddings for personalized reward models?, the PLUS approach of training a summarizer to extract preference dimensions rather than topic summaries from user history is vindicated — preference dimensions are properties of outputs, not inputs.

Inquiring lines that read this note 55

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Why do abstract preferences outperform episodic memories in personalization? How should recommendation systems balance individual preference and diversity? How do network effects and self-selection distort aggregated rating accuracy? What determines AI's persuasive power and how can it be detected or mitigated? Can LLMs distinguish between linguistic form and semantic meaning? How can we maintain privacy when agents prioritize task completion? What limits language model accuracy in evaluating ideas? Do language models encode knowledge that influences generation, or primarily imitate surface patterns? Do language models reason through disagreement or only accommodate it? How do hallucinated citations emerge in AI scholarly output? How does personalization simultaneously affect user trust and privacy concerns? How do reward models systematically fail to represent diverse human preferences? How do training data quality and composition affect downstream model performance? What prevents LLMs from applying their reasoning knowledge to improve outputs? Can smaller specialized models match frontier models on key metrics? Can persona profiles improve LLM prediction accuracy and consistency? Do persona-based approaches introduce systematic biases in user simulation? How do writers navigate authorship and delegation with AI? How can we detect and account for LLM involvement in academic writing?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
18 direct connections · 116 in 2-hop network ·medium cluster Open in graph ↗
Do user outputs outperform inputs for LLM person… How much does demo position alone affect in-contex… Can text summaries beat embeddings for personalize… How do personalization granularity levels trade pr…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

How does personalization differ mechanically from retrieval-augmented generation?Why does profile position in context windows affect personalization strength?Can preference dimensions extracted from outputs replace topic-based user summaries?How do input length constraints reshape personalization system design choices?
Original note title

historical user outputs drive personalization more effectively than input queries — personalization information not semantic information is the active ingredient