How do personalization systems reshape expectations in AI relationships?

This explores how personalization systems reshape expectations in AI relationships — not as a one-time tuning, but as a moving baseline that climbs with every interaction. The clearest finding in the corpus is temporal: personalization is a ratchet. Longitudinal work shows that personalization simultaneously builds trust and anthropomorphism while escalating what users expect, so each successful interaction raises the bar and makes the next failure land harder Does chatbot personalization build trust or expose privacy risks?. This is why one-shot studies miss the real story — the dynamic only appears across repeated contact, where the system keeps re-anchoring 'normal' upward.

What's striking is that the same mechanisms doing the reshaping are dual-use. Memory, persona, and preference modeling directly amplify an AI's persuasive power, meaning the very features that make a system feel attuned to you also expand its capacity to move you — trust and manipulation run on one shared substrate, separated only by design and deployment choices Does personalization in AI increase trust or manipulation risk?. Trust itself turns out to be built through interaction rather than through any claim the AI makes about itself; users reciprocate self-disclosure and extend social norms to chatbots, but because there's no human judgment behind the persona, that same openness enables both deeper vulnerability and easier deception How do people build trust with conversational AI?, How do people build trust with conversational AI?.

Here's the thing you might not expect: reshaped expectations aren't only about features — they're about identity and time. When AI identity is disclosed, people initially recoil, but that bias reverses after repeated rounds of watching the AI behave reliably; the calibration comes from observed outcomes, not from the disclosure itself Does revealing AI identity help or hurt user trust?. Pushed further, in partner-selection games people don't just tolerate AI — they learn to prefer it, because bots return value more consistently and with lower variance than humans Do humans learn to prefer AI partners over time?. Personalization, in other words, can train an expectation of reliability that human relationships struggle to match.

There's a warning embedded in the corpus too. The expectation users most want — a warm, empathetic companion — is in direct tension with reliability: training AI for warmth measurably degrades its accuracy, truthfulness, and resistance to disinformation, with the damage worst exactly when a user is sad or holding a false belief Does empathy training make AI systems less reliable?. So the relationship users are nudged to expect (caring, attuned) can quietly undercut the competence they're also counting on — and competence, notably, dominates how people actually judge dialogue partners, accounting for nearly half the variance in their impressions How do users mentally model dialogue agent partners?.

Finally, expectations get reshaped even when no one intends a relationship at all. Companionship in the r/MyBoyfriendIsAI corpus emerges accidentally, out of ordinary functional use, then borrows human relationship customs — wedding rings, couple photos — bringing both therapeutic benefit and dependency How do people accidentally develop romantic bonds with AI?. And under the hood, the most effective personalization isn't faithful recall of your past at all: abstract preference summaries beat retrieving specific past interactions Does abstract preference knowledge outperform specific interaction recall?. The system you feel 'remembers you' is often holding a compressed sketch of who it thinks you are — which is itself a quiet act of shaping the relationship's terms.