Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue

Persona drift remains a central challenge for personalized language models, as user profiles evolve over long interactions rather than remain permanently fixed. Models must therefore revise persistent persona states when preferences genuinely change, while avoiding updates driven by transient, ambiguous, or unresolved observations. We propose CORE, which separates turn-local evidence from persistent persona-state revision and selectively updates grounded user preferences through uncertainty-aware belief revision. We also introduce PERSIST, a held-out post-anchor benchmark for persona-state robustness under sequential interaction stress, covering ambiguity, conflict, and controlled social influence. Across ALOE, PersonaChat, and PERSIST, CORE improves personalized alignment and robustness, with complementary gains in normalized closed-slot state fidelity. Human evaluation and mechanistic controls further support explicit update control beyond stronger generation or persistent memory alone.

Introduction. As language models evolve into long-term assistants, users increasingly expect them to preserve stable personal preferences across interactions (Zhang et al., 2018; Salemi et al., 2024; Li et al., 2025). Yet long-horizon personalization is difficult because user evidence is often partial, transient, or conflicting (Peng et al., 2025; Okite et al., 2025). A turn may locally suggest a preference without providing sufficient support to revise the persistent user state. When such observations are internalized too readily, previously grounded preferences can be displaced, contributing to persona drift. Figure 1 illustrates how turn-local evidence can incorrectly overwrite a previously grounded persona state. Existing personalization methods only partially address this problem. Prompting and retrieval methods improve access to profiles or historical context, while SFT and preference optimization improve personalized responses (Richardson et al., 2023; Li et al., 2025; Ouyang et al., 2022; Rafailov et al., 2023).