Can model isolation solve streaming recommendation better than replay?

Streaming recommendation has to handle continuously arriving data and shifting user preferences. Continual learning frames this as preventing catastrophic forgetting (when learning new tasks erases knowledge of old ones) while allowing knowledge transfer between tasks. Three families of continual learning methods exist: experience replay (store old examples and replay during new training), knowledge distillation/regularization (constrain new training so it doesn't damage old knowledge), and model isolation (allocate separate parameters per task).

The DEGC contribution is bringing graph convolutional networks (the dominant architecture for capturing collaborative relationships) into the model-isolation continual-learning framework. Each task — each new batch of incoming streaming data — gets its own dedicated parameters, preserving older parameters intact. The user-item interaction graph extends as new interactions arrive, and the graph convolution operates over the extended structure with the per-task parameters.

The architectural choice of model isolation matters specifically for streaming recommendation. Experience replay requires storing old examples, which violates GDPR-style privacy constraints in real platforms. Knowledge distillation provides only soft control over what's preserved versus updated. Model isolation provides explicit control: old parameters mean old behavior is preserved exactly; new parameters absorb new patterns. The stability (preserve known) versus plasticity (adapt to new) trade-off becomes a configuration choice rather than a hyperparameter tuning problem.

The general principle for any continually-updating system: when the cost of forgetting old knowledge is high (regulated environments, slow-drift domains, infrequent re-training), model isolation is preferable to weight-averaging schemes because it provides explicit guarantees rather than soft trade-offs.

Source: Recommenders Architectures