Can LLMs gain collaborative filtering strength without losing text understanding?

LLM-based recommenders excel in cold-start where text semantics is the only signal — they understand items from descriptions and match users without interaction history. They underperform traditional CF in warm-start scenarios where rich interaction patterns exist. The reason: LLMs encode users and items as text tokens, capturing semantic similarity but missing the local collaborative information in co-occurrence patterns. Two items with similar text descriptions can have very different collaborative signatures depending on which users consumed them, and the LLM can't see this.

CoLLM separates the two strengths. A traditional collaborative model (e.g., matrix factorization) is trained externally on interactions, producing user/item embeddings that encode collaborative information. These embeddings are mapped into the LLM's input token embedding space — they become "tokens" the LLM can attend to alongside the item's text tokens. The LLM itself is not modified; the CF information enters through additional embedding tokens.

Three benefits. First, cold-warm coverage: the LLM keeps text-semantic strength for cold items (where the new tokens carry little CF information because no interactions exist) and gains CF strength for warm items. Second, decoupled architecture: any external CF model can produce the embeddings, so the technique is flexible. Third, no LLM fine-tuning required for the CF channel — the LLM consumes the new tokens as it consumes any other tokens.

The conceptual contribution: "use an LLM as recommender" doesn't require the LLM to do everything. Letting external specialized components feed into the LLM's token space, instead of asking the LLM to learn from scratch what specialists already know, preserves both approaches' strengths.

Source: Recommenders LLMs