What makes vector embeddings fail on single-hop semantic relevance queries?

This explores why vector embeddings struggle even on the seemingly easy case — a single-hop query asking for one directly relevant thing — and the corpus suggests the failure is built into what embeddings measure, not a tuning problem. The root issue is a category error: embeddings encode *semantic association* (what co-occurs, what's topically close) rather than *task relevance* (what actually answers the query) Do vector embeddings actually measure task relevance?. On a single-hop query that looks simple, this is exactly where things break — an underspecified query has many candidates that are semantically near but wrong for the role you need, and the embedding can't tell the difference because it never measured the difference.

There's also a harder limit underneath the semantic one. Even a *perfect* embedding runs into a mathematical ceiling: communication-complexity theory shows that for any fixed embedding dimension, there's a maximum number of top-k document combinations the system can ever return — and this was demonstrated on trivially simple retrieval tasks, not exotic ones Do embedding dimensions fundamentally limit retrievable document combinations?. So 'single-hop and simple' doesn't save you; the geometry of squeezing meaning into d dimensions caps which result sets are even representable. The broader framing here is that retrieval failures are architectural, sitting at distinct structural levels — semantic-task mismatch and dimensional limits among them — so they call for different approaches rather than incremental fixes Where do retrieval systems fail and why?.

What makes this interesting is a twist: the embeddings aren't empty of meaning. Static transformer embeddings genuinely encode rich semantic content — valence, concreteness, even taboo — before attention ever runs Do transformer static embeddings actually encode semantic meaning?. So the failure isn't 'embeddings don't understand words.' It's that *general semantic richness is the wrong tool for pinpoint relevance.* A related clue: models lean on statistical mass from pretraining, systematically preferring high-frequency phrasings over rarer but equivalent ones Do language models really understand meaning or just surface frequency?. That same frequency bias can pull a single-hop query toward the popular-but-wrong neighbor instead of the rare-but-correct answer.

The corpus also points at what actually helps. When queries are relational, deterministic graph traversal beats probabilistic similarity outright When do graph databases outperform vector embeddings for retrieval?. When the gap is descriptive, routing through natural-language description before retrieval bridges what direct embedding similarity misses Can describing images in text improve zero-shot recognition?. And notably, on single-hop tasks specifically, a model's own calibrated uncertainty about whether it even needs to retrieve outperforms heavier adaptive-retrieval machinery Can simple uncertainty estimates beat complex adaptive retrieval? — a reminder that sometimes the fix isn't a better embedding but knowing when similarity search is the wrong move entirely.

The thing worth carrying away: 'single-hop' feels like it should be embeddings' home turf, but it's where the association-vs-relevance gap is most exposed — the query is short, underspecified, and surrounded by plausible-but-wrong neighbors, exactly the conditions where measuring 'close in meaning' diverges most from 'correct for the task.'