How do large pretrained language models scale the unified recommendation paradigm?
This explores how language models turn recommendation into a single text-based task — and what happens, for better and worse, when you scale that one-model-does-everything approach up. The corpus suggests the unification is real but it drags the language model's own baggage along with it.
This explores how language models turn recommendation into a single text-based task — and what happens, for better and worse, when you scale that one-model-does-everything approach. The starting move is to stop building a separate model per job. P5 rewrites user clicks, ratings, and item descriptions as plain sentences, then trains one encoder-decoder to handle five different families of recommendation task at once, even transferring to items and domains it never saw in training Can one text encoder unify all recommendation tasks?. That's the core of the 'unified paradigm': language becomes the shared format that lets one model do rating prediction, sequential recommendation, explanation, and search without bespoke architectures for each.
Scaling that idea splits into two questions: how do you keep teaching the model, and how do you keep the catalog from getting baked in. On the teaching side, Rec-R1 shows you can train a language model directly on the metrics recommenders already care about — NDCG, Recall — as reinforcement-learning rewards, skipping the usual step of distilling from a bigger proprietary model Can recommendation metrics train language models directly?. Strikingly, a model trained this way learns to write good product-search queries without ever being shown the catalog, picking up an implicit feel for what's in stock purely from system feedback — much like a shopper who searches a store without knowing its inventory Can LLMs recommend products without ever seeing the catalog?.
The quieter problem is that scale means tying recommendations to text similarity, which doesn't transfer cleanly across domains. VQ-Rec attacks this by turning item text into discrete codes that merely index a learned embedding table, deliberately breaking the tight coupling between how an item is described and how it's recommended — so the lookup table can adapt to a new domain without retraining the whole text encoder Can discretizing text embeddings improve recommendation transfer?. This is the same instinct that drives the unified policy work in conversational systems, where asking, recommending, and timing get folded into one jointly-optimized policy rather than three components that can't share learning signals Can unified policy learning improve conversational recommender systems?.
Here's the part you might not expect to care about: scaling a language model into a recommender imports the language model's pretraining flaws. Because the backbone learned from web text, the recommender inherits position bias, popularity bias, and demographic fairness bias — failure modes that come from the pretraining corpus, not from any user's clicks, which means the usual collaborative-filtering fixes don't touch them Where do recommendation biases come from in language models?. There's a deeper structural reason to expect this: in language models generally, pretraining scale and fine-tuning scale do different jobs — pretraining stores factual knowledge in lower layers, fine-tuning shapes behavior in upper ones scaling-fine-tuning-improves-improves-helpfulness-while-scaling-pretraining-improves-fact. Whatever bias lives in the pretrained foundation rides along no matter how much recommendation-specific tuning you stack on top.
The payoff of going through a language model, rather than around it, is that the model can also explain itself in human terms. RecExplainer trains a language model to mimic a target recommender's behavior and internal embeddings at once, producing explanations that stay faithful to the black-box system while reading like ordinary reasoning Can LLMs explain recommenders by mimicking their internal states? — and persona-based methods can trace each suggestion back to the specific taste it satisfies Can attention mechanisms reveal which user taste explains each recommendation?. So the unified, scaled paradigm buys you one model, zero-shot transfer, and built-in explainability — at the price of inheriting biases that were never about recommendation at all.
Sources 9 notes
P5 converts user-item interactions and metadata into natural language and trains a single encoder-decoder across five recommendation task families, matching task-specific models while achieving zero-shot transfer to new items and domains. Unification trades efficiency for composability.
Rec-R1 demonstrates that LLMs can be trained directly on rule-based recommendation metrics like NDCG and Recall as RL reward signals, eliminating the need for SFT distillation from proprietary models while remaining model-agnostic across different retriever architectures.
Rec-R1 experiments show that LLMs trained via RL with recommender metrics as rewards can generate effective product search queries without catalog access. The model learns query refinement indirectly through system feedback, paralleling how humans search without knowing platform inventory.
VQ-Rec uses product quantization to map item text to discrete codes that index learned embeddings, breaking the tight coupling between text and recommendations. This decoupling prevents text-similarity bias and allows lookup tables to adapt to new domains without retraining the text encoder.
Research shows that formulating attribute-asking, item-recommending, and timing decisions as a single graph-based RL policy achieves better joint optimization than isolated components. Separation prevents gradient signals from informing one another and fails to optimize conversation trajectory holistically.
Wu et al. show that LLM-based recommendation systems exhibit position bias, popularity bias, and fairness bias—unique failure modes stemming from the language model's pretraining objective and corpus demographics rather than interaction data. Mitigation requires LLM-specific approaches, not adapted collaborative filtering techniques.
Emulated Fine-Tuning reveals that scaling pretraining improves factual knowledge while scaling fine-tuning improves behavioral helpfulness. This decoupling has architectural roots: pretraining enriches lower-layer knowledge storage, while fine-tuning modifies upper-layer behavior expression.
RecExplainer trains LLMs via three alignment methods: behavior (mimicking outputs), intention (incorporating neural embeddings), and hybrid (combining both). The hybrid approach produces explanations that are simultaneously faithful to the target model and intelligible to users by balancing internal-state inspection with human-readable reasoning.
AMP-CF represents each user as multiple latent personas weighted dynamically by candidate item. This makes recommendations both diverse and interpretable—each suggestion traces to the specific persona preference it satisfies—without requiring post-hoc reranking.