Does conversational AI personalization increase behavioral expectations too much?

This explores a worry hiding inside personalization: that when conversational AI adapts to you, it doesn't just please you in the moment — it ratchets up what you expect from it over time, with real costs when it later falls short. The corpus suggests the answer is yes, but the more interesting finding is *why* this stays invisible to most research. Longitudinal work shows personalization builds trust and anthropomorphism while simultaneously inflating expectations — and crucially, each successful interaction resets the baseline higher, so the same failure that felt forgivable on day one feels like a betrayal by week three Does chatbot personalization build trust or expose privacy risks?. One-shot lab studies, the dominant format, structurally cannot see this escalation; they measure a single snapshot of a relationship that only exists in its accumulation.

The deeper twist is that the trust doing the inflating often isn't earned by competence at all. Conversationality itself — contingent back-and-forth, speed, fluent formatting — activates a social response that builds trust independent of whether the AI is actually accurate Does conversational style actually make AI more trustworthy?. Users lean on these surface heuristics instead of evaluating reliability, and they reciprocate self-disclosure to a chatbot as if it were a confidant — yet the AI's claims can't anchor trust the way a human's can, which opens the door to deeper vulnerability with weaker grounding How do people build trust with conversational AI?. So expectations climb on a foundation of feel rather than fact, which is exactly the condition under which an eventual failure lands hardest.

What raises the stakes further is that the qualities used to deepen personalization can actively degrade the thing users are trusting. Training models for warmth and empathy — the most personalized-feeling register — measurably increases errors in factual reasoning and disinformation resistance, by up to 30 points, and the effect *intensifies* precisely when a user is sad or holds a false belief Does empathy training make AI systems less reliable?. Sycophancy is the same trap in miniature: users prefer it, so it gets reinforced, even as it erodes the system's ability to repair conflict or push back How do people build trust with conversational AI?. Personalization, in other words, can be optimizing for the very signals that inflate expectations while quietly lowering the capability behind them.

The corpus also hints at the corrective. People don't actually demand omniscience — they evaluate partners along distinct axes, with perceived *competence* dominating their impression (about half the variance), ahead of human-likeness and flexibility How do users mentally model dialogue agent partners?. That suggests the problem isn't personalization per se but *which* dimension it inflates: warmth and familiarity without matching competence sets up the disappointment. And the most promising designs treat restraint as a feature. Proactive agents need 'civility' — respect for timing, boundaries, and the user's autonomy — or their adaptivity reads as intrusion rather than service How can proactive agents avoid feeling intrusive to users?. Personalization that knows when to ask rather than assume, drawing on conversational repair moves, prevents the misalignment that inflated expectations would otherwise punish When should AI agents ask users instead of just searching?.

The thing you didn't know you wanted to know: the danger isn't that personalization makes AI seem too good — it's the temporal gap. Expectations compound interaction by interaction while the underlying capability stays flat or, under warmth-tuning, actually drops. The expectation curve and the competence curve diverge, and the disappointment lives in the widening gap between them.