How do text-based preference summaries compare to embedding vectors for conditioning?
This explores whether representing what a user wants as readable text (a written preference summary) conditions a model better than encoding it as a numeric embedding vector — and what each form gains or loses.
This explores whether representing a user's preferences as readable text beats encoding them as numeric vectors when you want a model to act on them. The short version from the corpus: text is winning on the dimensions people care about most — accuracy, transfer, and the ability to inspect what the system thinks you want.
The most direct evidence is PLUS, which trains a summarizer and a reward model together and finds that learned text-based preference summaries condition the reward model more effectively than embedding vectors — and capture dimensions that off-the-shelf zero-shot summaries miss Can text summaries beat embeddings for personalized reward models?. Two side benefits fall out of using text: the summary transfers to a different model (GPT-4) for zero-shot personalization, and a human can read it. An embedding can do neither. This pairs with PRIME's finding that abstract, distilled preference knowledge (semantic memory) consistently beats retrieving specific past interactions (episodic memory) — abstraction, not raw recall, is what conditions well Does abstract preference knowledge outperform specific interaction recall?.
The deeper reason text is competitive shows up in the papers about what goes *wrong* with vectors. A single fixed-length user vector is a bottleneck — it lossily compresses diverse interests into one point, which is why Deep Interest Network conditions on the candidate item instead, activating only the relevant slice of history How can user vectors capture diverse interests without exploding in size?. Same impulse drives the move to represent a user as *multiple* weighted personas rather than one latent vector, so each recommendation traces to a specific taste Can attention mechanisms reveal which user taste explains each recommendation?. Text summaries sidestep the bottleneck differently: language is variable-length and compositional, so it can name several preferences at once without cramming them into fixed dimensions.
But the corpus doesn't say embeddings are obsolete — it suggests the interesting design is hybrid, where text is an *intermediate* you discretize or factorize rather than feed raw. VQ-Rec maps item text to discrete codes that then index learned embeddings, deliberately breaking the tight text-to-representation coupling so the system transfers across domains and resists text-similarity bias Can discretizing text embeddings improve recommendation transfer? Can discrete codes transfer better than text embeddings?. PReF goes further toward structure, factorizing preferences into a small set of reward coefficients you can pin down with about ten adaptive questions — interpretable like text, compact like a vector Can user preferences be learned from just ten questions?. And LLMs can do cheap conditioning purely in language, rewriting a user's complaint into a positive preference a retriever can match, no fine-tuning required Can language models bridge the gap between critique and preference?.
The thing you might not have expected to learn: the text-vs-vector question is partly a proxy for a quieter one about *signal quality*. Annotation responses don't all measure the same thing — they decompose into genuine preferences, non-attitudes, and on-the-spot constructed preferences, distinguishable only by consistency across conditions Do all annotation responses measure the same underlying thing?. A text summary that's interpretable lets you notice when you've encoded noise as if it were a stable preference. A dense embedding hides that mistake. So the case for text isn't only that it conditions better — it's that it fails *visibly*, which may matter more than a marginal accuracy gap.
Sources 9 notes
PLUS trains summarizers and reward models jointly, learning that text-based preference summaries capture dimensions zero-shot summaries miss. These summaries transfer to GPT-4 for zero-shot personalization and remain interpretable to users.
PRIME framework shows semantic memory (preference summaries, parametric encodings) consistently beats episodic memory (retrieved past interactions) across models. Recency-based recall outperforms similarity-based retrieval, and task fine-tuning exceeds preference tuning methods.
Deep Interest Network weights historical behaviors against each candidate ad, activating only relevant interests dynamically. This preserves dimension efficiency while expressing diverse tastes without lossy compression.
AMP-CF represents each user as multiple latent personas weighted dynamically by candidate item. This makes recommendations both diverse and interpretable—each suggestion traces to the specific persona preference it satisfies—without requiring post-hoc reranking.
VQ-Rec uses product quantization to map item text to discrete codes that index learned embeddings, breaking the tight coupling between text and recommendations. This decoupling prevents text-similarity bias and allows lookup tables to adapt to new domains without retraining the text encoder.
VQ-Rec demonstrates that mapping item text to discrete codes via product quantization, then to embeddings, improves cross-domain transfer compared to direct text encoding. The discrete intermediate reduces text bias and enables efficient per-domain fine-tuning.
PReF learns base reward functions from preference data, then uses active learning to select maximally informative questions that reduce coefficient uncertainty. Users can be personalized via inference-time reward alignment without weight modification.
Few-shot LLM prompting can convert natural negative feedback like "doesn't look good for a date" into positive preferences like "prefer more romantic," enabling retrieval systems to find better-matching recommendations without fine-tuning.
Behavioral science reveals that annotations contain genuine preferences, non-attitudes, and constructed preferences—distinguishable by consistency across measurement conditions. Treating them uniformly contaminates reward model training and downstream alignment.