Why does pure numeric ID indexing force models to learn from scratch?
This explores why representing items as bare numbers — item #48213 instead of 'The Matrix (1999), sci-fi film' — means a model has nothing to lean on and must learn each item's meaning purely from interaction data.
This explores why a pure numeric ID — a label like item #48213 with no built-in meaning — forces a recommendation model to learn every item from scratch rather than borrowing knowledge it already has. The corpus frames the answer cleanly: a numeric ID is an empty slot. It guarantees that two items are distinct, but it carries no signal about what an item *is*, so the only way a model can learn that item #48213 resembles item #91077 is to observe enough overlapping user behavior to infer it. There's no shortcut through meaning, because there's no meaning encoded in the symbol.
The clearest articulation comes from work on item identifiers Can item identifiers balance uniqueness and semantic meaning?: pure IDs deliver distinctiveness but zero semantics, while pure text delivers semantics but loses the ability to point at one exact item. The proposed fix — fusing IDs with titles and attributes — is itself the proof of what's missing. When an identifier contains the words 'sci-fi' and 'Keanu Reeves,' a language model already knows what those mean from pretraining, so it isn't starting cold. Strip that away and you've thrown out everything the model could have transferred in for free.
The cost of learning-from-scratch shows up sharply in how ID embeddings behave at scale Why do hash collisions hurt recommendation models so much?. Because each ID must earn its own learned vector from data, and real catalogs follow a power-law, the rarest items — and every newly-arrived ID — have almost no interactions to learn from. This is the cold-start problem in its purest form: a brand-new numeric ID is a vector initialized to noise, indistinguishable from any other new item until the system accumulates behavioral evidence. Semantic identifiers sidestep this because a new item arrives already описана by its words.
The lateral lesson the corpus offers is that 'learning from scratch' is a choice about *representation*, not an inevitability. Several lines of work show models recovering structure without direct supervision: LLMs learning catalog awareness purely through recommendation feedback Can LLMs recommend products without ever seeing the catalog?, and retrieval models adapting to a domain from nothing but a short text description Can you adapt retrieval models without accessing target data?. The common thread: when you give a model a foothold in language or feedback, it transfers; when you give it an opaque number, it can only memorize. The same theme echoes in fine-tuning research, where keeping a model anchored to its pretrained knowledge preserves transferable structure rather than overwriting it Can decoding-time tuning preserve knowledge better than weight fine-tuning?.
So the answer to 'why' is almost tautological once you see it — and that's the interesting part. Numeric IDs force scratch-learning *by design*: they are the deliberate removal of every prior the model could have used. The recent shift toward semantic and multi-facet identifiers is the field quietly admitting that the empty slot was costing more than it saved.
Sources 5 notes
TransRec shows that combining numeric IDs, titles, and attributes into structured identifiers solves three problems simultaneously: distinctiveness from IDs, semantics from text, and generation grounding from structural constraints. Neither pure IDs nor pure text alone achieves all three.
Monolith's empirical work shows that real recommendation systems have power-law distributed frequencies, causing collisions to accumulate precisely on the entities models need most accurate. Fixed-size hashed tables worsen this over time as new IDs arrive.
Rec-R1 experiments show that LLMs trained via RL with recommender metrics as rewards can generate effective product search queries without catalog access. The model learns query refinement indirectly through system feedback, paralleling how humans search without knowing platform inventory.
Research demonstrates that a brief textual domain description suffices to generate synthetic training data for retrieval fine-tuning, outperforming baselines in zero-target-access scenarios and enabling adaptation where conventional methods are blocked.
Proxy-tuning closes 88-91% of the alignment gap while surpassing direct fine-tuning on knowledge tasks by leaving base model weights untouched. Direct fine-tuning corrupts knowledge storage in lower layers, whereas proxy-tuning applies distributional shifts that primarily affect reasoning and style.