How do structural constraints like zero self-similarity improve collaborative filtering?
This explores why forbidding an item from predicting itself — and related structural rules baked into a model rather than learned — beats throwing more model capacity at collaborative filtering.
This is really a question about why a hard rule you impose on a recommender can outperform a bigger, smarter model that's left to figure things out on its own. The cleanest case is the zero-diagonal constraint in EASE Can simpler models beat deep networks for recommendation systems?: it's a shallow linear item-to-item weight matrix where the diagonal is pinned to zero, meaning an item is structurally forbidden from being its own best predictor. Without that rule, the model takes the lazy shortcut — "users who liked X will like X" — and learns nothing about relationships between items. Forcing the prediction to route through *other* items is what makes it generalize, and it beats deep autoencoders on most datasets.
The follow-up work ESLER Can a linear model beat deep collaborative filtering? sharpens the why. The same single-layer linear model, constrained against self-prediction, doesn't just learn which items go together — its *negative* weights turn out to be essential. They encode anti-affinity: "people who bought this baby gear are not buying death metal vinyl." That dissimilarity signal is something capacity-heavy models often blur away, and it's only legible because the structural constraint forced the model to express preference entirely through item relationships. The headline both papers land on is the same: structural bias matters more than model capacity.
What's worth noticing is that "structural constraint" is a broader family than just the zeroed diagonal. Choosing a multinomial likelihood for a VAE Why does multinomial likelihood work better for ranking recommendations? is the same kind of move — it forces items to *compete* for a fixed budget of probability mass, which structurally aligns training with the actual goal (ranking the top-N items a user will want) rather than reconstructing every rating in isolation. Like the zero diagonal, it's a constraint on the model's shape, not its size, and it produces state-of-the-art results by changing what the model is allowed to express.
The flip side — what happens when you *don't* impose the right structure and just shrink the model — shows up in the work on embedding dimensionality Does embedding dimensionality secretly drive popularity bias in recommenders?. Squeeze the embeddings too small and the recommender quietly overfits toward popular items to protect its ranking score, and that bias compounds over time into long-term unfairness. So structural choices cut both ways: a good constraint (forbid self-prediction, force competition) sharpens a model, while a bad one (too few dimensions) silently warps it toward the popular and the safe.
The thing you might not have expected to learn: the lesson of this corner of the corpus is almost anti-deep-learning. The most reliable wins in collaborative filtering here come not from more layers but from picking the right *prior* — telling the model what it's not allowed to do — and letting that constraint do the work that capacity can't.
Sources 4 notes
EASE, a shallow linear item-item weight matrix with diagonal constrained to zero, beats deep neural baselines on most datasets. The constraint forces generalization by forbidding self-prediction, while learned negative weights capture item dissimilarity—a structural prior more valuable than model capacity.
ESLER, a single-layer linear autoencoder constrained so items cannot predict themselves, outperforms most deep CF models. The constraint forces prediction through item relationships, and negative weights encoding anti-affinity prove essential—structural bias matters more than model capacity.
Liang et al. show that switching VAE likelihoods from Gaussian/logistic to multinomial achieves state-of-the-art results because enforced probability competition between items directly aligns training with top-N ranking objectives. Rebalancing KL regularization further improves performance.
Research shows that when user/item embedding dimensions are too small, recommender systems overfit toward popular items to maximize ranking quality. This compounds over time as niche items receive insufficient exposure, and cannot be fixed post-hoc without treating dimensionality as a fairness hyperparameter.