Can graph structure patterns outperform direct edge signals in noisy data?

Building product substitute graphs (which products can replace which) at scale faces five distinct challenges in real e-commerce data. Accuracy: traditional local-similarity calculations don't consider inner structure of user-behavior graph, limiting prediction accuracy. Robustness: user behavior data contains many noisy, casual, or accidental clicks that affect prediction reliability. Sparsity: the ratio of user purchases is small, making co-purchasing data sparse. Direction: complementary relationships are asymmetric, requiring directional graphs. Scalability: traditional algorithms grow with both customers and products, intractable at billion-scale.

Taobao's Swing algorithm addresses all five by using quasi-local structure — patterns spanning a few hops in the user-item bipartite graph rather than single edges. Two products A and B are similar not just if many users co-clicked them but specifically if those users have similar behavior patterns elsewhere. This indirect signal is more stable than single-edge counts because noise affects different edges independently while structural patterns across edges are robust to individual noise.

The companion algorithm, Surprise, addresses the sparsity problem in complementary relationships by leveraging product category information and product clusters constructed by Swing, plus considering temporal order of co-purchased products. Both run on parallel large-scale distributed computing frameworks (MapReduce, Spark) for production scalability.

The general principle: in noisy graph data, look at structural patterns rather than individual edges. Edge-level signals are too easily corrupted by noise; structural signals (do these neighbors share other neighbors? do these triangles co-occur?) are robust because they require multiple noisy edges to coincidentally agree, which they rarely do.

Source: Recommenders Architectures