Can simpler models beat deep networks for recommendation systems?

The deep-learning trend in collaborative filtering treated more layers as more capacity. EASE — Embarrassingly Shallow AutoEncoder — pushes the opposite direction. It is a linear model with no hidden layer, learning only an item-item weight matrix B. The single non-trivial constraint is that the diagonal of B is forced to zero: an item cannot use itself to predict itself. That constraint forces every item's prediction to be reconstructed from the other items the user has interacted with, which is what generalization in collaborative filtering actually requires.

The model has a closed-form solution to a convex objective, so training is dominated by a matrix inversion rather than gradient descent. On most public datasets EASE outperforms deep, non-linear, and probabilistic models — and beats SLIM, the most similar prior approach, by dropping SLIM's L1 regularization and non-negativity constraint. About 60% of the learned weights end up negative; the dissimilarity (negative weights) between items is structurally important, and removing the ability to learn negatives by setting them to zero collapses accuracy to SLIM levels.

The conceptual lesson is twofold. First, the relevant similarity matrix for CF is the precision matrix, not the covariance matrix that neighborhood-based methods typically use. Second, when a constraint (here, zero-diagonal) is the right inductive bias, simpler models with that constraint can beat deeper models without it. Capacity is not the bottleneck most of the time — the right structural prior is.

Source: Recommenders Architectures