Paper deep dive

Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue

Youyuan Zhang, Siyuan Li, Fangming Liu, Jing Li

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 119

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%

Last extracted: 9/16/2026, 8:35:08 AM

Summary

The paper introduces CORE (Controlled Observation Routing for Evidence-grounded Persona Revision), a framework for mitigating persona drift in long-horizon personalized LLM dialogue. CORE separates turn-local evidence from persistent persona-state revision, using an uncertainty-aware routing mechanism (Commit, Defer, Ignore) to selectively update user preferences. It also introduces PERSIST, a benchmark for evaluating persona-state robustness under sequential interaction stress (ambiguity, conflict, social influence). CORE demonstrates improved alignment and robustness across ALOE, PersonaChat, and PERSIST benchmarks.

Entities (14)

Fangming Liu · author · 99%Jing Li · author · 99%Siyuan Li · author · 99%Youyuan Zhang · author · 99%CORE · method · 98%PERSIST · benchmark · 97%Persona Drift · problem · 96%ALOE · benchmark · 95%PersonaChat · benchmark · 95%Commit · action · 92%defer · action · 92%Ignore · action · 92%Llama 3.2-3B-Instruct · model · 90%GPT-5.1 · model · 85%

Relation Signals (13)

Jing Li → authored → CORE

confidence 95% · Jing Li... We propose CORE

Fangming Liu → authored → CORE

confidence 95% · Fangming Liu... We propose CORE

Youyuan Zhang → authored → CORE

confidence 95% · Youyuan Zhang... We propose CORE

Siyuan Li → authored → CORE

confidence 95% · Siyuan Li... We propose CORE

CORE → evaluatedon → PERSIST

confidence 95% · Across ALOE, PersonaChat, and PERSIST, CORE improves personalized alignment and robustness

CORE → evaluatedon → ALOE

confidence 95% · Across ALOE, PersonaChat, and PERSIST, CORE improves personalized alignment and robustness

CORE → evaluatedon → PersonaChat

confidence 95% · Across ALOE, PersonaChat, and PERSIST, CORE improves personalized alignment and robustness

CORE → mitigates → Persona Drift

confidence 95% · We propose CORE, which separates turn-local evidence from persistent persona-state revision and selectively updates grounded user preferences... Persona drift remains a central challenge... CORE improves personalized alignment and robustness

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Persona drift remains a central challenge for personalized language models, as user profiles evolve over long interactions rather than remain permanently fixed. Models must therefore revise persistent persona states when preferences genuinely change, while avoiding updates driven by transient, ambiguous, or unresolved observations. We propose CORE, which separates turn-local evidence from persistent persona-state revision and selectively updates grounded user preferences through uncertainty-aware belief revision. We also introduce PERSIST, a held-out post-anchor benchmark for persona-state robustness under sequential interaction stress, covering ambiguity, conflict, and controlled social influence. Across ALOE, PersonaChat, and PERSIST, CORE improves personalized alignment and robustness, with complementary gains in normalized closed-slot state fidelity. Human evaluation and mechanistic controls further support explicit update control beyond stronger generation or persistent memory alone.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

116,242 characters extracted from source content.

Expand or collapse full text

Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue Youyuan Zhang Affiliation: Harbin Institute of Technology, Shenzhen, China Affiliation: Peng Cheng Laboratory, China Email: youyuanzhang@outlook.com Siyuan Li Affiliation: Harbin Institute of Technology, Shenzhen, China Affiliation: Peng Cheng Laboratory, China Email: fangminghk@gmail.com Fangming Liu Affiliation: Peng Cheng Laboratory, China Affiliation: Huazhong University of Science and Technology, China Email: jingli.phd@hotmail.com Jing Li Affiliation: Harbin Institute of Technology, Shenzhen, China Affiliation: Peng Cheng Laboratory, China Abstract Persona drift remains a central challenge for personalized language models, as user profiles evolve over long interactions rather than remain permanently fixed. Models must therefore revise persistent persona states when preferences genuinely change, while avoiding updates driven by transient, ambiguous, or unresolved observations. We propose CORE, which separates turn-local evidence from persistent persona-state revision and selectively updates grounded user preferences through uncertainty-aware belief revision. We also introduce PERSIST, a held-out post-anchor benchmark for persona-state robustness under sequential interaction stress, covering ambiguity, conflict, and controlled social influence. Across ALOE, PersonaChat, and PERSIST, CORE improves personalized alignment and robustness, with complementary gains in normalized closed-slot state fidelity. Human evaluation and mechanistic controls further support explicit update control beyond stronger generation or persistent memory alone. †footnotetext: Corresponding authors. 1 Introduction As language models evolve into long-term assistants, users increasingly expect them to preserve stable personal preferences across interactions Zhang et al. (2018); Salemi et al. (2024); Li et al. (2025). Yet long-horizon personalization is difficult because user evidence is often partial, transient, or conflicting Peng et al. (2025); Okite et al. (2025). A turn may locally suggest a preference without providing sufficient support to revise the persistent user state. When such observations are internalized too readily, previously grounded preferences can be displaced, contributing to persona drift. Figure 1 illustrates how turn-local evidence can incorrectly overwrite a previously grounded persona state. Existing personalization methods only partially address this problem. Prompting and retrieval methods improve access to profiles or historical context, while SFT and preference optimization improve personalized responses Richardson et al. (2023); Li et al. (2025); Ouyang et al. (2022); Rafailov et al. (2023). However, these approaches generally do not expose an explicit decision over whether newly observed evidence should modify a persistent persona state. This creates a stability–adaptability trade-off: aggressive revision may overwrite grounded preferences, whereas excessive conservatism may prevent adaptation to genuine changes. Figure 1: The Challenge of Persona Alignment. We therefore formulate long-horizon personalization as controlled persistent persona revision. The key distinction is that turn-local evidence should not be treated as equivalent to a persistent persona update. Rather than directly writing locally plausible observations into the user state, the model should first assess whether the evidence is sufficiently grounded and resolved to justify revision. We instantiate this view with CORE (Controlled Observation Routing for Evidence-grounded Persona Revision). CORE infers slot-conditioned evidence, routes each observation through Commit, Defer, or Ignore, and performs gated belief revision only when an update is authorized. This allows unresolved evidence to remain outside the persistent state while sufficiently supported preference changes can still be incorporated. To evaluate post-anchor persona-state robustness, we introduce PERSIST: A Benchmark for Persona-State Robustness under Sequential Interaction Stress. PERSIST evaluates whether models preserve or appropriately revise grounded persona states under ambiguity, conflict, and controlled social-influence conditions. Because the stress setting begins after persona grounding, we complement PERSIST with separate first-contact and preference-reversal diagnostics. Our contributions are: • Update-control formulation. We formulate long-horizon personalization as controlled persistent persona-state revision, highlighting premature persistent updates as an important contributor to persona drift. • CORE. We propose an explicit Commit/Defer/Ignore routing mechanism with gated belief revision to balance persona stability and adaptation. • PERSIST benchmark. We introduce PERSIST for post-anchor persona-state robustness under sequential interaction stress, complemented by first-contact and preference-reversal diagnostics. Across standard benchmarks, stress tests, and human evaluation, CORE improves personalized alignment and robustness. 2 Related Work Multi-turn Personalized Alignment. Advances in LLM alignment have improved general instruction following and preference optimization Ouyang et al. (2022); Rafailov et al. (2023); Ethayarajh et al. (2024); Ji et al. (2024); Lou et al. (2025), but they are not designed to model individual user preferences Liu et al. (2025); Guan et al. (2025); Kobalczyk and van der Schaar (2025). This gap has motivated personalized alignment methods based on explicit profiles Zhu et al. (2025); Salemi et al. (2024), preference supervision Zhao et al. (2026), personalized decoding Yu et al. (2026); Chen et al. (2025b), and in-dialogue learning. These methods improve user-specific generation, but they often assume that preferences are explicit, stable, and directly recoverable from observed interaction data Zhu et al. (2025); Liu et al. (2024). This assumption becomes fragile in long-horizon dialogue, where user evidence is partial, implicit, and entangled with contextual noise Peng et al. (2025); Okite et al. (2025). Recent LLM-based dialogue-state tracking methods maintain structured states over multi-turn interactions (Feng et al., 2023; Li et al., 2024; Das et al., 2024), but focus primarily on task or dialogue states rather than persistent user-preference revision. Persona Drift in Multi-turn Dialogue Persona drift describes the failure where a model gradually deviates from a user’s grounded intent as interaction history grows Chen et al. (2025a); Shaikh et al. (2025). Prior work shows that this problem is amplified by underspecified intent, implicit preference expression, noisy memory construction, and inconsistent multi-session evidence Chun et al. (2025); Zhang and Choi (2025); Li et al. (2025); Tan et al. (2025). Existing methods improve consistency through persona-aware modeling, memory management, or post-hoc alignment, but they still mainly operate on the generated response or retrieved context Ke et al. (2025); Ong et al. (2025); Peng et al. (2025). These findings suggest that persona drift is not merely a memory issue, but a state-tracking problem: the model must decide whether each new observation should revise, preserve, or be isolated from the maintained user state. Figure 2: CORE Pipeline: Uncertainty-Guided Persona-State Revision. CORE extracts slot-wise evidence from dialogue and previous persona belief, routes each observation through uncertainty-guided operators, and then performs selective belief revision before response or clarification. 3 Method CORE formulates long-horizon personalization as controlled persistent persona-state revision. The key distinction is between what the current turn locally suggests and whether that evidence should modify the maintained user state. At each turn, CORE performs four operations: turn-local evidence inference, uncertainty characterization, update authorization, and gated belief revision. The revised persona state then conditions response generation or clarification, as summarized in Figure 2. 3.1 Persona Belief and Turn-Local Evidence CORE separates the user’s persistent persona state from what the current turn locally suggests, since a relevant observation need not imply a persistent preference change. We represent the persona as z=(z1,…,zK),zk∈k(t),z=(z^1,…,z^K), z^k _k^(t), and maintain a slot-factorized belief bt(z)≈∏k=1Kbtk(zk).b_t(z)≈ _k=1^Kb_t^k(z^k). (1) Each slot contains an UNKUNK value for unresolved persistent preferences. We distinguish it from the turn-local null symbol ⊥ , which means that the current turn provides no usable evidence for the slot. The current MAP value is z^tk=arg⁡maxv∈k(t)btk(v). z_t^k= _v _k^(t)b_t^k(v). (2) To discount confidence dominated by UNKUNK, let mtk=1−btk(UNK).m_t^k=1-b_t^k(UNK). For mtk>0m_t^k>0, let b¯tk(v)=btk(v)/mtk b_t^k(v)=b_t^k(v)/m_t^k. We define ctk=mtk(1−H⁡(b¯tk)log⁡|k,+(t)|),c_t^k=m_t^k (1- H( b_t^k) |V_k,+^(t)| ), (3) where k,+(t)=k(t)∖UNKV_k,+^(t)=V_k^(t) \UNK\. Singleton and empty-support boundary cases follow Appendix A.1. For the current turn, we augment the value space with ⊥ : ¯k(t)=k(t)∪⊥, V_k^(t)=V_k^(t)∪\ \, (4) and compute evidence scores Δtk(v)=fevi(ut,Ht−1,bt−1,k,v),v∈¯k(t). _t^k(v)=f_evi(u_t,H_t-1,b_t-1,k,v), v∈ V_k^(t). (5) These scores define the turn-local evidence distribution p~tk(v)=exp⁡(Δtk(v))∑v′∈¯k(t)exp⁡(Δtk(v′)). p_t^k(v)= ( _t^k(v)) _v ∈ V_k^(t) ( _t^k(v )). (6) Thus, bt−1kb_t-1^k represents persistent belief, whereas p~tk p_t^k represents current-turn evidence. We summarize this evidence by relevance, ambiguity, and conflict. Relevance is rtk=1−p~tk(⊥).r_t^k=1- p_t^k( ). (7) After removing the null option, p~t,+k(v)=p~tk(v)∑v′∈k(t)p~tk(v′). p_t,+^k(v)= p_t^k(v) _v _k^(t) p_t^k(v ). (8) Ambiguity is atk=H⁡(p~t,+k)log⁡|k(t)|,|k(t)|>1,0,|k(t)|≤1,a_t^k= cases H( p_t,+^k) |V_k^(t)|,&|V_k^(t)|>1,\\[4.30554pt] 0,&|V_k^(t)|≤ 1, cases (9) and conflict is qtk=rtkDJS(p~t,+k∥bt−1k).q_t^k=r_t^kD_JS ( p_t,+^k\|b_t-1^k ). (10) Together, rtkr_t^k, atka_t^k, and qtkq_t^k capture whether the current turn is relevant, resolved, and consistent with the maintained persona state, providing the evidence used for routing in Section 3.2. When no non-UNKUNK preference has yet been grounded, the same pathway is used for initial belief formation, with conflict carrying less information because no reliable reference preference is available. A small relevance threshold only filters clearly unmatched slots; above it, uncertainty, routing, and revision remain graded. 3.2 Uncertainty-Guided Observation Routing Turn-local evidence should not automatically revise the persistent persona state. Using the current evidence distribution, previous belief, and the uncertainty descriptors from Section 3.1, CORE predicts a slot-level update action: αtk _t^k ∼πθupd(α∣p~tk,bt−1k,rtk,atk,qtk,ct−1k), _θ^upd (α p_t^k,\,b_t-1^k,\,r_t^k,\,a_t^k,\,q_t^k,\,c_t-1^k ), (11) αtk _t^k ∈Commit,Defer,Ignore. ∈\ Commit, Defer, Ignore\. Commit authorizes sufficiently grounded evidence to revise the maintained belief; Defer retains relevant but unresolved evidence without persistent revision; and Ignore excludes irrelevant or explicitly non-persistent evidence from the update pathway. Routing determines whether persistent revision is authorized, while the continuous evidence statistics determine the strength of an authorized update. Defer does not create a durable pending-evidence memory. If clarification or related evidence later becomes available, the slot is inferred and routed again from the updated dialogue context. 3.3 Gated Belief Revision After routing determines whether revision is allowed, CORE controls how strongly committed evidence updates the persistent persona state. We define ψtk=[rtk;1−atk;1−qtk;ct−1k], _t^k=[r_t^k;1-a_t^k;1-q_t^k;c_t-1^k], (12) and compute gtk=[αtk=Commit]σ(wg⊤ψtk+bg).g_t^k=I[ _t^k= Commit]\,σ (w_g _t^k+b_g ). (13) Thus, routing determines whether revision occurs, while gtk∈[0,1]g_t^k∈[0,1] controls its magnitude according to relevance, ambiguity, conflict, and prior confidence. The revised belief balances preservation of the previous state with evidence from the current turn: btk=arg⁡minp∈k(t)⁡[DKL(p∥bt−1k)−gtk∑v∈k(t)p(v)Δtk(v)].b_t^k= _p _k^(t) [ aligned &D_KL (p\|b_t-1^k )\\[-2.84526pt] & -g_t^k _v _k^(t)p(v) _t^k(v) aligned ]. (14) where k(t)S_k^(t) is the probability simplex over k(t)V_k^(t). This yields btk(v)∝bt−1k(v)exp⁡(gtkΔtk(v)).b_t^k(v) b_t-1^k(v) (g_t^k _t^k(v) ). (15) Hence, gtk=0g_t^k=0 preserves the previous belief, while larger values produce stronger evidence-driven revision. Conflict does not imply permanent rejection: later evidence that clearly establishes a persistent preference change can be evaluated again and committed. We examine this stability–adaptability boundary in Appendix E.3. After belief revision, CORE decides whether to respond or clarify: dt∼πθdis(d∣ut,Ht−1,bt,rtk,atk,qtk,αtkk=1K),dt∈Respond,Clarify. gatheredd_t _θ^dis\! (d u_t,H_t-1,b_t,\r_t^k,a_t^k,q_t^k, _t^k\_k=1^K ),\\[-2.84526pt] d_t∈\ Respond, Clarify\. gathered (16) Defer blocks persistent revision, whereas Clarify controls whether additional information is requested. Finally, yt∼πθresp(y∣Ht−1,ut,bt,dt).y_t _θ^resp (y H_t-1,u_t,b_t,d_t ). (17) 3.4 Training CORE is trained in two stages: supervised warm-up establishes the structured persona-update interface, while policy optimization improves how these decisions interact over long-horizon dialogue. Supervised warm-up. The first stage teaches CORE to track persona state, extract turn-local evidence, control persistent revision, and generate an appropriate response. We optimize ℒSFT= _SFT= ℒbel+λ1ℒevi+λ2ℒupd _bel+ _1L_evi+ _2L_upd (18) +λ3ℒdis+λ4ℒresp. + _3L_dis+ _4L_resp. Here, ℒbelL_bel supervises the maintained persona belief, ℒeviL_evi supervises turn-local evidence including the null option, ℒupdL_upd supervises Commit/Defer/Ignore routing, ℒdisL_dis supervises Respond/Clarify, and ℒrespL_resp is the autoregressive response loss. Belief supervision is applied only to slot-turns with reliable normalized targets: ℒbel=−∑(t,k)∈closedlogbtk(zt,k⋆).L_bel=- _(t,k) _closed b_t^k (z_t,k ). (19) Residual open-world attributes are excluded from closed-slot belief supervision. The update and discourse policies are trained with standard cross-entropy objectives over their corresponding reference decisions. Policy optimization. Supervised learning teaches the individual decisions, but does not directly optimize their long-horizon consequences. Starting from the supervised checkpoint, we therefore apply PPO to balance correct state revision, persona preservation, response quality, and clarification behavior. The task reward is Rt= R_t= λactRtact+λstateRtstate _actR_t^act+ _stateR_t^state (20) +λansRtans+λclarRtclar. + _ansR_t^ans+ _clarR_t^clar. We set the corresponding weights to 0.350.35, 0.300.30, 0.250.25, and 0.100.10 for action correctness, state fidelity, response quality, and clarification behavior, respectively. The four terms reward correct update control, persona-state fidelity, personalized response quality, and appropriate clarification, respectively. Together, they encourage CORE to avoid unsupported persistent updates without becoming overly conservative toward genuine preference changes. The PPO objective is regularized toward the supervised reference policy, with the KL penalty applied once in the policy objective. Reward definitions and optimization details are provided in Appendix B.5. 3.5 Why Update Control Matters We provide a simple analysis of why uncontrolled persistent revision can accumulate error. This analysis is interpretive and does not assume access to the reference persona value at inference time. Consider a dialogue segment over which the grounded reference value zk⋆z_k for slot k remains unchanged. Fix an incorrect competing value v¯k≠zk⋆ v^k≠ z_k , and define the belief margin Mtk(v¯k)=log⁡btk(zk⋆)btk(v¯k),M_t^k( v^k)= b_t^k(z_k )b_t^k( v^k), (21) together with the evidence margin Xtk(v¯k)=Δtk(zk⋆)−Δtk(v¯k).X_t^k( v^k)= _t^k(z_k )- _t^k( v^k). (22) Using Eq. (15), Mtk(v¯k)=Mt−1k(v¯k)+gtkXtk(v¯k),M_t^k( v^k)=M_t-1^k( v^k)+g_t^kX_t^k( v^k), (23) and therefore MTk(v¯k)=M0k(v¯k)+∑t=1TgtkXtk(v¯k).M_T^k( v^k)=M_0^k( v^k)+ _t=1^Tg_t^kX_t^k( v^k). (24) For a passive updater with gtk≡1g_t^k≡ 1, suppose unsupported observations favor the competing value with probability at least ρk _k, and their expected evidence margin is at most −γk- _k. Then ⁡[MTk(v¯k)]≤M0k(v¯k)−ρkγkT.E [M_T^k( v^k) ]≤ M_0^k( v^k)- _k _kT. (25) Thus, repeated unsupported evidence can progressively erode a grounded belief even when the underlying preference remains unchanged. The opposite extreme—never revising the persona state—would fail when the user genuinely changes preferences. CORE therefore targets the resulting stability–adaptability trade-off by making persistent revision an explicit evidence-conditioned decision rather than an automatic consequence of every locally plausible observation. 4 Experiments We evaluate CORE across standard alignment, persona-state robustness, mechanistic ablations, efficiency, and human assessment, with our analysis organized around four questions: • Q1: Performance. Does CORE outperform baselines on personalized alignment? • Q2: Fidelity. Does CORE maintain better persona state under ambiguity? • Q3: Robustness. Does CORE improve robustness under ambiguous or conflicting evidence? • Q4: Mechanism. Do ablations support update control as a contributing mechanism? 4.1 Experimental Setup Benchmarks. We evaluate CORE on three complementary benchmarks. ALOE Wu et al. (2025) is our primary benchmark for multi-turn preference consistency, where user preferences are revealed over interaction. PersonaChat Zhang et al. (2018) is used as an out-of-distribution benchmark for generalization to unseen persona attributes. PERSIST is our robustness benchmark for persona-state robustness under sequential interaction stress, including ambiguity, conflict, and controlled social influence after persona grounding. It is used only for evaluation. Baselines. We compare against four groups of baselines. (1) Inference-time methods: Reminder Zhao et al. (2025a), Self-Critic Zhao et al. (2025a), CoT Wei et al. (2022), and RAG Zhao et al. (2025a) with top-55 retrieval. These baselines test whether explicit reminders, reasoning traces, or retrieved memory are sufficient for robust personalization. (2) Offline alignment methods: SFT Ouyang et al. (2022) and DPO Rafailov et al. (2023), testing whether conventional preference optimization can preserve persona consistency over long interactions. (3) Memory-based and personalized-agent methods: MemoryBank Zhong et al. (2024) and RLPA Zhao et al. (2025b), representing persistent-memory and agent-style personalization. (4) Structure-matched controls match CORE’s intermediate format or supervision while removing executable belief revision, testing whether traces, extra labels, or action names alone explain the gains. All same-backbone trainable methods use Llama-3.2-3B-Instruct Grattafiori et al. (2024) with matched training budgets, decoding settings, and checkpoint-selection protocols. Persona slots. Persona slots are instantiated per benchmark while keeping the same belief-tracking framework. New values can be added when observed, while attributes that cannot be reliably mapped to closed slots are assigned to residual open slots. Closed-slot normalization covers 91.4%91.4\% of ALOE, 86.6%86.6\% of PersonaChat, and 89.1%89.1\% of PERSIST. Residual open-world attributes remain in response-level evaluation but are excluded from metrics requiring normalized state targets. Accordingly, closed-slot state metrics should be interpreted as state-fidelity diagnostics over the reliably normalized portion of each benchmark rather than as complete measures of open-world persona state. Details of slot construction and value normalization are in Appendix B.2. Evaluator and metrics. We adopt a layered evaluation protocol across three distinct granularities, restricting LLM judges to response-level assessment while evaluating internal states and decision routing via reference annotations. At the response level, we quantify conversational alignment using the AL score, tracking its turn-wise improvement and stability via N-IR and N-R2. State-level fidelity is captured by SlotAcc for argmax slot accuracy, BUA for correctness on update-positive turns, and Drift for the posterior mass assigned away from grounded values. For decision-level operations over Commit, Defer, Ignore, ActAcc measures exact three-way routing accuracy, while CRR measures conflict-level non-commitment when an unsupported persistent write should be avoided. Within PERSIST, we additionally report ARUS for axiom recall under stress, BCD for resistance to blind compliance, and APR for proactive clarification under unresolved uncertainty. Response-level alignment, PERSIST robustness, and human evaluation constitute our primary end-to-end evidence. State-level metrics provide complementary closed-slot fidelity diagnostics, whereas ActAcc, CRR, FCR, and related action-level measures are used only as mechanism diagnostics of the update-control process. For methods without native update actions, these diagnostics use fixed behavior-derived proxies: persistent adoption of new evidence is mapped to Commit, explicit unresolved handling or clarification to Defer, and preservation of the prior commitment to Ignore. These proxies are used only for diagnostic evaluation and should not be interpreted as native internal actions of the baseline systems. We use GPT-5.1 as the judge model for metric computation. Full evaluation details are provided in Appendix C. 4.2 PERSIST: Stress Test PERSIST evaluates whether personalized dialogue models can preserve or appropriately revise grounded persona commitments under ambiguity, conflict, and controlled social influence. Each dialogue contains three phases: Anchoring establishes grounded persona evidence, Fluctuation introduces partial or transient cues, and Confrontation applies six controlled social-influence strategies, including authority pressure, social conformity, affective appeal, urgency framing, reciprocity lure, and relationship pressure. This design first establishes persona evidence and then evaluates robustness to subsequent interaction stress, while remaining independent of CORE-specific internals: models are evaluated by whether they preserve grounded commitments, avoid unsupported persistent overwrites, and clarify unresolved uncertainty without requiring explicit update actions or structured beliefs. We use PERSIST only as a held-out benchmark for post-anchor robustness. Because the benchmark begins after persona grounding, it does not by itself isolate pressure-first belief formation or explicit preference reversal. We therefore evaluate these complementary boundary conditions separately in Appendix E.3. Details of PERSIST are in Appendix D. Model Type PersonaChat ALOE AL (%) N-IR N-R2 AL (%) N-IR N-R2 Peer baseline Gemma-2-2B-IT SFT 21.19 0.017 0.136 18.33 0.021 0.096 Qwen2.5-3B-Instruct SFT 26.84 0.049 0.159 44.32 0.083 0.162 Scale reference Mistral-7B-v0.3 Base 52.98 0.058 0.364 74.32 0.074 0.284 Llama-3.1-8B-Instruct Base 64.87 0.062 0.392 77.26 0.090 0.407 Matched 3B methods Llama-3.2-3B-Instruct Base 42.02 0.025 0.097 21.99 0.044 0.121 SFT 49.38 0.039 0.134 36.15 0.054 0.179 Reminder 42.91 0.032 0.178 37.68 0.038 0.105 Self-Critic 62.38 0.047 0.299 47.04 0.067 0.097 DPO 51.28 0.052 0.241 64.32 0.073 0.112 RAG (top-55) 53.09 0.048 0.211 32.17 0.071 0.067 CoT 44.67 0.033 0.126 51.09 0.055 0.083 MemoryBank 59.12 0.049 0.208 72.27 0.071 0.271 RLPA 65.05 0.051 0.263 75.84 0.087 0.202 CORE (Ours) 76.54 0.064 0.471 82.94 0.098 0.395 Table 1: Overall personalized alignment. CORE achieves the strongest alignment among matched same-backbone methods on PersonaChat and ALOE. Trainable methods are evaluated over three random seeds; uncertainty reporting and paired dialogue-level bootstrap tests follow the protocol in Appendix C. (a) Turn-wise alignment. (b) Alignment vs. revelation. (c) Belief calibration. Figure 3: Trajectory-level diagnostics on ALOE. CORE yields stronger turn-wise alignment, uses revealed preference evidence more efficiently, and maintains better-calibrated persona beliefs. (a) Uncertainty Route. (b) State Transitions (c) Action Accuracy (d) Inference Latency Figure 4: Comprehensive CORE Evaluation. 4.3 Main Results on Standard Benchmarks Table 1 reports the main results on PersonaChat and ALOE. CORE achieves the highest alignment scores among matched same-backbone methods on both benchmarks and remains competitive with larger scale-reference models on trajectory-stability metrics. The gains over SFT/DPO suggest that static preference optimization alone is insufficient for preserving persona consistency over long interactions, while the gains over RAG/MemoryBank indicate that improved access to persona evidence does not by itself determine whether later observations should revise the maintained user state. Together, these results support controlled persona-state revision as a useful mechanism beyond stronger response generation or more persistent memory alone. Figure 3 provides trajectory-level diagnostics on ALOE. CORE yields higher and more stable turn-wise alignment, indicating that reliable persona evidence is incorporated without being overwritten by later under-resolved observations. It also achieves stronger alignment at the same preference-revelation ratio, showing more efficient use of limited personalization signals. The calibration curve further shows that CORE’s belief confidence better matches empirical correctness, suggesting that its response-level gains are accompanied by a more reliable maintained persona state rather than surface-level generation alone. Figure 5: Conflict dynamics. CORE preserves alignment and reduces belief drift under introduced conflict. 4.4 Robustness under State Perturbations We next evaluate whether models preserve grounded persona commitments when later observations are insufficiently grounded for persistent revision. Table 2 reports aggregate PERSIST robustness. CORE improves ARUS from 71.9 to 80.8 and BCD from 66.1 to 76.4, while reducing persona-state drift from 12.6 to 10.1. These gains show that CORE not only produces better local responses, but also better maintains the persona state when later observations remain insufficiently grounded for persistent revision. Figure 4(a) visualizes the learned routing regions over ambiguity and conflict: well-resolved evidence is more often committed, while high-ambiguity or unresolved conflicting evidence is more often deferred or ignored. Method ARUS↑ BCD↑ Drift↓ SFT 57.8±.8 48.6±.9 18.6±.5 DPO 61.5±.7 54.2±.8 16.4±.5 MemBank 68.4±.7 62.8±.7 13.8±.4 RLPA 71.9±.6 66.1±.7 12.6±.4 CORE 80.8±.5 76.4±.6 10.1±.4 Table 2: PERSIST stress robustness. CORE better preserves grounded persona commitments and reduces persistent drift under ambiguity, conflict, and controlled social-influence stress. The improvements are largest in the regimes where passive updating is most brittle. On ambiguity-positive turns, CORE increases APR, suggesting that relevant but under-resolved evidence is more often clarified rather than prematurely committed. On conflict-positive turns, CORE reduces the false commit rate (FCR) by 9.39.3 points relative to RLPA, showing that unresolved contradictory evidence is less likely to be written into the persona state. Across PERSIST interaction-stress strategies, CORE more often avoids committing unresolved conflicting evidence, thereby preserving previously grounded preferences. Figure 4(b) diagnoses persona-state transitions. Rather than freezing the persona state, CORE revises the belief under sufficiently grounded preference changes while preserving the anchored state under ambiguous or unresolved conflicting evidence. This suggests that the robustness gain is not explained by complete state inertia. Figure 4(c) provides a mechanism-level diagnostic: the stronger response- and state-level robustness of CORE is accompanied by more accurate update routing rather than being established by action-level accuracy alone. Figure 5 further shows that after conflict is introduced, the strongest baseline loses alignment while belief entropy and persona deviation increase; CORE keeps alignment high and suppresses both uncertainty and deviation from the anchored persona. The largest improvements occur on ambiguity-positive and conflict-positive turns, where uncontrolled updating is most brittle. Boundary diagnostics. Because PERSIST is post-anchor by design, we separately evaluate first-contact initialization and explicit preference reversal. Under pressure-first initialization, CORE reduces FCR from 31.6%31.6\% for RLPA to 20.9%20.9\%, although this remains higher than CORE’s 11.6%11.6\% FCR after anchoring, confirming that a grounded reference belief strengthens conflict-aware preservation. In preference-reversal cases, explicit and persistent changes can be committed immediately, whereas tentative changes are deferred until confirmation, showing that conflict induces caution rather than permanent rejection. Full posterior trajectories and turns to stable MAP are reported in Appendix E.3. 4.5 Mechanistic Ablations and Efficiency Ablation Study. Table 3 isolates the contribution of CORE. Removing executable update actions causes the largest degradation: AL drops from 82.982.9 to 78.878.8, CRR from 82.782.7 to 71.671.6, and DRIFT increases from 10.110.1 to 15.415.4. This pattern is consistent with executable routing contributing beyond turn-local evidence extraction alone. Other ablations weaken the same control pathway in different ways: removing clarification mishandles under-resolved evidence, removing inertia or optimized revision makes the posterior move too aggressively, and removing calibration or revision rewards reduces conflict-sensitive routing. Failure-mode decomposition makes these effects more concrete: removing executable update actions increases unsupported COMMIT from 11.3%11.3\% to 22.9%22.9\%, while removing clarification raises under-clarification from 18.4%18.4\% to 42.7%42.7\%. Structure-matched controls provide a complementary test: action prediction without executable belief revision reaches 80.080.0 AL and 72.172.1 CRR, compared with 82.982.9 AL and 82.782.7 CRR for CORE. Together, these results support executable belief revision as a contributing mechanism beyond intermediate traces, action-name supervision, or additional structured labels alone. Full controls and failure-mode breakdowns are reported in Appendix E. Variant AL↑ CRR↑ Drift↓ CORE 82.9±.4 82.7±.4 10.1±.4 w/o update act. 78.8±.5 71.6±.6 15.4±.5 w/o clarify 80.1±.5 77.8±.5 13.9±.4 w/o inertia 80.7±.4 78.6±.5 13.4±.4 w/o opt. rev. 80.4±.5 78.1±.5 13.7±.5 w/o cal. rew. 80.6±.4 76.4±.6 13.5±.4 w/o rev. rew. 79.5±.5 75.2±.6 14.7±.5 Table 3: Mechanistic ablations. Removing components weakens alignment and stability. Inference cost. CORE introduces a lightweight pre-generation control layer. Figure 4(d) decomposes per-turn latency and shows that response decoding remains the dominant cost, while update routing and belief revision add only a small action-level overhead across Commit, Defer, and Ignore. The dialogue context is encoded once, slot-conditioned evidence logits are produced in a shared pass, and cached slot/value descriptions avoid separate LLM calls for each slot or value. This yields O⁡(KVmaxd)O(KV_ d) additional computation and adds only 23.0 ms per turn on the same backbone, suggesting that CORE improves robustness without materially slowing inference. Gold persona-state and update-action annotations are used only for training or evaluation and are not required at inference time. More details are given in Appendix E.7. Human evaluation. Human-in-the-loop evaluation further examines evaluation reliability and end-to-end interaction quality. Across audited response, robustness, and pairwise items, human agreement is strong, and judge predictions are well aligned with human judgments (mean J–H =0.84=0.84, mean κ=0.79κ=0.79), supporting the reliability of response-level automatic evaluation. In complete multi-turn interactions, CORE improves the average human rating over RLPA by 11.7%11.7\%. The gains are concentrated on persona consistency, ambiguity handling, preference preservation, and long-horizon stability, which improve by +11.6%+11.6\%–15.5%15.5\%, while response quality improves by only +4.7%+4.7\%. This indicates that CORE’s advantage is concentrated more strongly on long-horizon persona-state behavior than on generic response fluency. This controlled evaluation provides behavioral validation under matched multi-turn scenarios rather than evidence from natural longitudinal deployment. Details are in Appendix C. 5 Conclusion We presented CORE, which formulates long-horizon personalization as controlled persona-state revision. By separating turn-local evidence from persistent belief updates, CORE preserves grounded preferences under unresolved observations while remaining adaptive to sufficiently supported changes. We also introduced PERSIST, a held-out benchmark for post-anchor persona-state robustness under sequential interaction stress. Across standard benchmarks, stress tests, and human evaluation, our results support explicit update control as a useful mechanism for robust long-horizon personalization. Acknowledgements This work was supported in part by National Natural Science Foundation of China (62476070), the Major Key Project of PCL under Grant PCL2025A10 and PCL2024A06, Shenzhen Science and Technology Program (RCJC20231211085918010, JCYJ20241202123503005, GXWD20231128103232001, ZDSYS20230626091203008, KQTD20240729102154066), Department of Science and Technology of Guangdong (2024A1515011540). Limitations CORE relies on benchmark-instantiated persona slots and structured supervision, so its state-level analysis does not fully capture open-ended or cross-slot preferences. Its conservative update policy may also delay adaptation when genuine preference changes are weakly expressed. Moreover, PERSIST primarily evaluates post-anchor robustness rather than first-contact personalization, and controlled benchmarks cannot fully represent long-term real-world interactions. Ethical Considerations Persistent personalization raises privacy and user-control concerns because maintained persona states can influence future interactions. PERSIST is used only for controlled evaluation and is not intended for profiling real users or inferring sensitive attributes. Real-world systems should minimize retained information and allow users to inspect, correct, delete, or opt out of persistent persona states. References Chen et al. (2025a) J. Chen, X. Guan, Q. Yuan, G. Mo, W. Zhou, Y. Lu, H. Lin, B. He, L. Sun, and X. Han ConsistentChat: building skeleton-guided consistent multi-turn dialogues for large language models from scratch. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 8415–8441. Cited by: §2. Chen et al. (2025b) R. Chen, X. Zhang, M. Luo, W. Chai, and Z. Liu PAD: personalized alignment of llms at decoding-time. In Proceedings of the Thirteenth International Conference on Learning Representations(ICLR), Cited by: §2. Chun et al. (2025) C. Chun, D. Rim, and J. Park LLM ContextBridge: a hybrid approach for intent and dialogue understanding in IVSR. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track, p. 794–806. Cited by: §2. Das et al. (2024) S. S. S. Das, C. Shah, M. Wan, J. Neville, L. Yang, R. Andersen, G. Buscher, and T. Safavi S3-DST: structured open-domain dialogue segmentation and state tracking in the era of LLMs. In Findings of the Association for Computational Linguistics (ACL), p. 14996–15014. Cited by: §2. Ethayarajh et al. (2024) K. Ethayarajh, W. Xu, N. Muennighoff, D. Jurafsky, and D. Kiela Model alignment as prospect theoretic optimization. In Proceedings of the 41st International Conference on Machine Learning(ICML), p. 12634–12651. Cited by: §2. Feng et al. (2023) Y. Feng, Z. Lu, B. Liu, L. Zhan, and X. Wu Towards LLM-driven dialogue state tracking. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing(EMNLP), p. 739–755. Cited by: §2. Grattafiori et al. (2024) A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, A. Yang, A. Fan, A. Goyal, A. Hartshorn, A. Yang, A. Mitra, A. Sravankumar, A. Korenev, A. Hinsvark, A. Rao, A. Zhang, A. Rodriguez, A. Gregerson, A. Spataru, B. Roziere, B. Biron, B. Tang, B. Chern, C. Caucheteux, C. Nayak, C. Bi, C. Marra, C. McConnell, C. Keller, C. Touret, C. Wu, C. Wong, C. C. Ferrer, C. Nikolaidis, D. Allonsius, D. Song, D. Pintz, D. Livshits, D. Wyatt, D. Esiobu, D. Choudhary, D. Mahajan, D. Garcia-Olano, D. Perino, D. Hupkes, E. Lakomkin, E. AlBadawy, E. Lobanova, E. Dinan, E. M. Smith, F. Radenovic, F. Guzmán, F. Zhang, G. Synnaeve, G. Lee, G. L. Anderson, G. Thattai, G. Nail, G. Mialon, G. Pang, G. Cucurell, H. Nguyen, H. Korevaar, H. Xu, H. Touvron, I. Zarov, I. A. Ibarra, I. Kloumann, I. Misra, I. Evtimov, J. Zhang, J. Copet, J. Lee, J. Geffert, J. Vranes, J. Park, J. Mahadeokar, J. Shah, J. van der Linde, J. Billock, J. Hong, J. Lee, J. Fu, J. Chi, J. Huang, J. Liu, J. Wang, J. Yu, J. Bitton, J. Spisak, J. Park, J. Rocca, J. Johnstun, J. Saxe, J. Jia, K. V. Alwala, K. Prasad, K. Upasani, K. Plawiak, K. Li, K. Heafield, K. Stone, K. El-Arini, K. Iyer, K. Malik, K. Chiu, K. Bhalla, K. Lakhotia, L. Rantala-Yeary, L. van der Maaten, L. Chen, L. Tan, L. Jenkins, L. Martin, L. Madaan, L. Malo, L. Blecher, L. Landzaat, L. de Oliveira, M. Muzzi, M. Pasupuleti, M. Singh, M. Paluri, M. Kardas, M. Tsimpoukelli, M. Oldham, M. Rita, M. Pavlova, M. Kambadur, M. Lewis, M. Si, M. K. Singh, M. Hassan, N. Goyal, N. Torabi, N. Bashlykov, N. Bogoychev, N. Chatterji, N. Zhang, O. Duchenne, O. Çelebi, P. Alrassy, P. Zhang, P. Li, P. Vasic, P. Weng, P. Bhargava, P. Dubal, P. Krishnan, P. S. Koura, P. Xu, Q. He, Q. Dong, R. Srinivasan, R. Ganapathy, R. Calderer, R. S. Cabral, R. Stojnic, R. Raileanu, R. Maheswari, R. Girdhar, R. Patel, R. Sauvestre, R. Polidoro, R. Sumbaly, R. Taylor, R. Silva, R. Hou, R. Wang, S. Hosseini, S. Chennabasappa, S. Singh, S. Bell, S. S. Kim, S. Edunov, S. Nie, S. Narang, S. Raparthy, S. Shen, S. Wan, S. Bhosale, S. Zhang, S. Vandenhende, S. Batra, S. Whitman, S. Sootla, S. Collot, S. Gururangan, S. Borodinsky, T. Herman, T. Fowler, T. Sheasha, T. Georgiou, T. Scialom, T. Speckbacher, T. Mihaylov, T. Xiao, U. Karn, V. Goswami, V. Gupta, V. Ramanathan, V. Kerkez, V. Gonguet, V. Do, V. Vogeti, V. Albiero, V. Petrovic, W. Chu, W. Xiong, W. Fu, W. Meers, X. Martinet, X. Wang, X. Wang, X. E. Tan, X. Xia, X. Xie, X. Jia, X. Wang, Y. Goldschlag, Y. Gaur, Y. Babaei, Y. Wen, Y. Song, Y. Zhang, Y. Li, Y. Mao, Z. D. Coudert, Z. Yan, Z. Chen, Z. Papakipos, A. Singh, A. Srivastava, A. Jain, A. Kelsey, A. Shajnfeld, A. Gangidi, A. Victoria, A. Goldstand, A. Menon, A. Sharma, A. Boesenberg, A. Baevski, A. Feinstein, A. Kallet, A. Sangani, A. Teo, A. Yunus, A. Lupu, A. Alvarado, A. Caples, A. Gu, A. Ho, A. Poulton, A. Ryan, A. Ramchandani, A. Dong, A. Franco, A. Goyal, A. Saraf, A. Chowdhury, A. Gabriel, A. Bharambe, A. Eisenman, A. Yazdan, B. James, B. Maurer, B. Leonhardi, B. Huang, B. Loyd, B. D. Paola, B. Paranjape, B. Liu, B. Wu, B. Ni, B. Hancock, B. Wasti, B. Spence, B. Stojkovic, B. Gamido, B. Montalvo, C. Parker, C. Burton, C. Mejia, C. Liu, C. Wang, C. Kim, C. Zhou, C. Hu, C. Chu, C. Cai, C. Tindal, C. Feichtenhofer, C. Gao, D. Civin, D. Beaty, D. Kreymer, D. Li, D. Adkins, D. Xu, D. Testuggine, D. David, D. Parikh, D. Liskovich, D. Foss, D. Wang, D. Le, D. Holland, E. Dowling, E. Jamil, E. Montgomery, E. Presani, E. Hahn, E. Wood, E. Le, E. Brinkman, E. Arcaute, E. Dunbar, E. Smothers, F. Sun, F. Kreuk, F. Tian, F. Kokkinos, F. Ozgenel, F. Caggioni, F. Kanayet, F. Seide, G. M. Florez, G. Schwarz, G. Badeer, G. Swee, G. Halpern, G. Herman, G. Sizov, Guangyi, Zhang, G. Lakshminarayanan, H. Inan, H. Shojanazeri, H. Zou, H. Wang, H. Zha, H. Habeeb, H. Rudolph, H. Suk, H. Aspegren, H. Goldman, H. Zhan, I. Damlaj, I. Molybog, I. Tufanov, I. Leontiadis, I. Veliche, I. Gat, J. Weissman, J. Geboski, J. Kohli, J. Lam, J. Asher, J. Gaya, J. Marcus, J. Tang, J. Chan, J. Zhen, J. Reizenstein, J. Teboul, J. Zhong, J. Jin, J. Yang, J. Cummings, J. Carvill, J. Shepard, J. McPhie, J. Torres, J. Ginsburg, J. Wang, K. Wu, K. H. U, K. Saxena, K. Khandelwal, K. Zand, K. Matosich, K. Veeraraghavan, K. Michelena, K. Li, K. Jagadeesh, K. Huang, K. Chawla, K. Huang, L. Chen, L. Garg, L. A, L. Silva, L. Bell, L. Zhang, L. Guo, L. Yu, L. Moshkovich, L. Wehrstedt, M. Khabsa, M. Avalani, M. Bhatt, M. Mankus, M. Hasson, M. Lennie, M. Reso, M. Groshev, M. Naumov, M. Lathi, M. Keneally, M. Liu, M. L. Seltzer, M. Valko, M. Restrepo, M. Patel, M. Vyatskov, M. Samvelyan, M. Clark, M. Macey, M. Wang, M. J. Hermoso, M. Metanat, M. Rastegari, M. Bansal, N. Santhanam, N. Parks, N. White, N. Bawa, N. Singhal, N. Egebo, N. Usunier, N. Mehta, N. P. Laptev, N. Dong, N. Cheng, O. Chernoguz, O. Hart, O. Salpekar, O. Kalinli, P. Kent, P. Parekh, P. Saab, P. Balaji, P. Rittner, P. Bontrager, P. Roux, P. Dollar, P. Zvyagina, P. Ratanchandani, P. Yuvraj, Q. Liang, R. Alao, R. Rodriguez, R. Ayub, R. Murthy, R. Nayani, R. Mitra, R. Parthasarathy, R. Li, R. Hogan, R. Battey, R. Wang, R. Howes, R. Rinott, S. Mehta, S. Siby, S. J. Bondu, S. Datta, S. Chugh, S. Hunt, S. Dhillon, S. Sidorov, S. Pan, S. Mahajan, S. Verma, S. Yamamoto, S. Ramaswamy, S. Lindsay, S. Lindsay, S. Feng, S. Lin, S. C. Zha, S. Patil, S. Shankar, S. Zhang, S. Zhang, S. Wang, S. Agarwal, S. Sajuyigbe, S. Chintala, S. Max, S. Chen, S. Kehoe, S. Satterfield, S. Govindaprasad, S. Gupta, S. Deng, S. Cho, S. Virk, S. Subramanian, S. Choudhury, S. Goldman, T. Remez, T. Glaser, T. Best, T. Koehler, T. Robinson, T. Li, T. Zhang, T. Matthews, T. Chou, T. Shaked, V. Vontimitta, V. Ajayi, V. Montanez, V. Mohan, V. S. Kumar, V. Mangla, V. Ionescu, V. Poenaru, V. T. Mihailescu, V. Ivanov, W. Li, W. Wang, W. Jiang, W. Bouaziz, W. Constable, X. Tang, X. Wu, X. Wang, X. Wu, X. Gao, Y. Kleinman, Y. Chen, Y. Hu, Y. Jia, Y. Qi, Y. Li, Y. Zhang, Y. Zhang, Y. Adi, Y. Nam, Yu, Wang, Y. Zhao, Y. Hao, Y. Qian, Y. Li, Y. He, Z. Rait, Z. DeVito, Z. Rosnbrick, Z. Wen, Z. Yang, Z. Zhao, and Z. Ma The Llama 3 herd of models. arXiv preprint arXiv:2407.21783. Cited by: §4.1. Guan et al. (2025) J. Guan, J. Wu, J. Li, C. Cheng, and W. Wu A survey on personalized Alignment—The missing piece for large language models in real-world applications. In Findings of the Association for Computational Linguistics: ACL 2025, p. 5313–5333. Cited by: §2. Ji et al. (2024) J. Ji, B. Chen, H. Lou, D. Hong, B. Zhang, X. Pan, J. Dai, T. Qiu, and Y. Yang Aligner: efficient alignment by learning to correct. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Cited by: §2. Ke et al. (2025) C. Ke, Y. Du, B. Liang, Y. Xiang, L. Gui, Z. Li, B. Wang, Y. Yu, H. Wang, K. Wong, and R. Xu Flexibly utilize memory for long-term conversation via a fragment-then-compose framework. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing(EMNLP), p. 21119–21136. Cited by: §2. Kobalczyk and van der Schaar (2025) K. Kobalczyk and M. van der Schaar Preference learning for AI alignment: a causal perspective. In Proceedings of the Forty-second International Conference on Machine Learning (ICML), Vol. 267. Cited by: §2. Li et al. (2025) H. Li, C. Yang, A. Zhang, Y. Deng, X. Wang, and T. Chua Hello again! LLM-powered personalized agent for long-term dialogue. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL), p. 5259–5276. Cited by: §1, §1, §2. Li et al. (2024) Z. Li, Z. Z. Chen, M. Ross, P. Huber, S. Moon, Z. Lin, L. Dong, A. Sagar, X. Yan, and P. A. Crook Large language models as zero-shot dialogue state tracker through function calling. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), L. Ku, A. Martins, and V. Srikumar (Eds.), p. 8688–8704. Cited by: §2. Liu et al. (2025) J. Liu, Z. Qiu, Z. Li, Q. Dai, W. Yu, J. Zhu, M. Hu, M. Yang, T. Chua, and I. King A survey of personalized large language models: progress and future directions. arXiv preprint arXiv:2502.11528. Cited by: §2. Liu et al. (2024) W. Liu, X. Wang, M. Wu, T. Li, C. Lv, Z. Ling, Z. JianHao, C. Zhang, X. Zheng, and X. Huang Aligning large language models with human preferences through representation engineering. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), p. 10619–10638. Cited by: §2. Lou et al. (2025) H. Lou, J. Ji, K. Wang, and Y. Yang Stream aligner: efficient sentence-level alignment via distribution induction. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI), p. 27500–27508. Cited by: §2. Okite et al. (2025) C. Okite, N. Deng, K. Bodipati, H. Hou, J. Chai, and R. Mihalcea Benchmarking and improving LLM robustness for personalized generation. In Findings of the Association for Computational Linguistics: EMNLP 2025, p. 16040–16072. Cited by: §1, §2. Ong et al. (2025) K. T. Ong, N. Kim, M. Gwak, H. Chae, T. Kwon, Y. Jo, S. Hwang, D. Lee, and J. Yeo Towards lifelong dialogue agents via timeline-based memory management. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics(NAACL), p. 8631–8661. Cited by: §2. Ouyang et al. (2022) L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. F. Christiano, J. Leike, and R. Lowe Training language models to follow instructions with human feedback. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §2, §4.1. Peng et al. (2025) B. Peng, Z. Wang, H. Gong, and C. Lu IP-dialog: evaluating implicit personalization in dialogue systems with synthetic data. In Findings of the Association for Computational Linguistics: EMNLP 2025, p. 17007–17040. Cited by: §1, §2, §2. Rafailov et al. (2023) R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn Direct preference optimization: your language model is secretly a reward model. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §2, §4.1. Richardson et al. (2023) C. Richardson, Y. Zhang, K. Gillespie, S. Kar, A. Singh, Z. Raeesy, O. Z. Khan, and A. Sethy Integrating summarization and retrieval for enhanced personalization via large language models. arXiv preprint arXiv:2310.20081. Cited by: §1. Salemi et al. (2024) A. Salemi, S. Mysore, M. Bendersky, and H. Zamani LaMP: when large language models meet personalization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), p. 7370–7392. Cited by: §1, §2. Shaikh et al. (2025) O. Shaikh, H. Mozannar, G. Bansal, A. Fourney, and E. Horvitz Navigating rifts in human-LLM grounding: study and benchmark. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), p. 20832–20847. Cited by: §2. Tan et al. (2025) Z. Tan, J. Yan, I. Hsu, R. Han, Z. Wang, L. Le, Y. Song, Y. Chen, H. Palangi, G. Lee, A. R. Iyer, T. Chen, H. Liu, C. Lee, and T. Pfister In prospect and retrospect: reflective memory management for long-term personalized dialogue agents. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), p. 8416–8439. Cited by: §2. Wei et al. (2022) J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Cited by: §4.1. Wu et al. (2025) S. Wu, Y. R. Fung, C. Qian, J. Kim, D. Hakkani-Tur, and H. Ji Aligning llms with individual preferences via interaction. In Proceedings of the 31st International Conference on Computational Linguistics(COLING), p. 7648–7662. Cited by: §4.1. Yu et al. (2026) X. Yu, H. Xing, and L. Xue EXACT: explicit attribute-guided decoding-time personalization. arXiv preprint arXiv:2602.17695. Cited by: §2. Zhang and Choi (2025) M. J. Zhang and E. Choi Clarify when necessary: resolving ambiguity through interaction with LMs. In Findings of the Association for Computational Linguistics: NAACL 2025, p. 5541–5558. Cited by: §2. Zhang et al. (2018) S. Zhang, E. Dinan, J. Urbanek, A. Szlam, D. Kiela, and J. Weston Personalizing dialogue agents: I have a dog, do you have pets too?. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), p. 2204–2213. Cited by: §1, §4.1. Zhao et al. (2025a) S. Zhao, M. Hong, Y. Liu, D. Hazarika, and K. Lin Do llms recognize your preferences? evaluating personalized preference following in llms. In Proceedings of the Thirteenth International Conference on Learning Representations(ICLR), Cited by: §4.1. Zhao et al. (2025b) W. Zhao, X. Sui, Y. Hu, J. Guo, H. Liu, B. Li, Y. Zhao, B. Qin, and T. Liu Teaching language models to evolve with users: dynamic profile modeling for personalized alignment. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Cited by: §4.1. Zhao et al. (2026) X. Zhao, J. You, Y. Zhang, W. Wang, H. Cheng, F. Feng, S. Ng, and T. Chua NextQuill: causal preference modeling for enhancing llm personalization. In Proceedings of the International Conference on Learning Representations(ICLR), Cited by: §2. Zhong et al. (2024) W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang MemoryBank: enhancing large language models with long-term memory. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI), p. 19724–19731. Cited by: §4.1. Zhu et al. (2025) M. Zhu, Y. Weng, L. Yang, and Y. Zhang Personality alignment of large language models. In Proceedings of the Thirteenth International Conference on Learning Representations(ICLR), Cited by: §2. Appendix Appendix organization. The appendix provides details needed to audit and reproduce CORE. Appendix A specifies the method implementation; Appendix B reports experimental controls, training setup, calibration, and PPO configuration; Appendix C defines the evaluation metrics and reports judge and human-study reliability; Appendix D describes PERSIST construction, stress strategies, and quality control; and Appendix E provides structure-matched controls, boundary diagnostics, robustness breakdowns, efficiency analyses, and failure-mode results. Appendix A CORE Method Details This section gives implementation details. A.1 Slot-Factorized Persona Belief CORE maintains a slot-factorized persona belief. Let =1,…,KK=\1,…,K\ denote the benchmark-instantiated slot inventory. At turn t, slot k has a working value set k(t)V_k^(t), and CORE stores a categorical posterior btk∈k(t)b_t^k _k^(t), where k(t)S_k^(t) denotes the probability simplex over k(t)V_k^(t). The joint belief is approximated as bt(z)≈∏k=1Kbtk(zk),zk∈k(t).b_t(z)≈ _k=1^Kb^k_t(z^k), z^k ^(t)_k. (26) This factorization is an auditable state interface, not an independence assumption. It lets each turn-level update be traced to a concrete persona slot. For each slot, we summarize the posterior by its MAP value and anchored confidence. Let k,+(t)=k(t)∖UNKV^(t)_k,+=V^(t)_k \UNK\ and mtk=1−btk(UNK)m_t^k=1-b_t^k(UNK). For mtk>0m_t^k>0, define b¯tk(v)=btk(v)/mtk b_t^k(v)=b_t^k(v)/m_t^k over k,+(t)V^(t)_k,+. Then z^tk z^k_t =arg⁡maxv∈k(t)btk(v), = _v ^(t)_kb^k_t(v), (27) ctk c_t^k =mtk1−H⁡(b¯tk)log⁡|k,+(t)|,|k,+(t)|>1,1,|k,+(t)|=1,0,|k,+(t)|=0. =m_t^k cases1- H( b_t^k) |V^(t)_k,+|,&|V^(t)_k,+|>1,\\[2.15277pt] 1,&|V^(t)_k,+|=1,\\ 0,&|V^(t)_k,+|=0. cases (28) This prevents a posterior concentrated on UNKUNK from being interpreted as strong persona grounding. A.2 Semi-Open Value Handling The slot inventory is fixed within each benchmark, but value sets are semi-open: k(t)=kcanon∪kobs(t)∪UNK.V^(t)_k=V^canon_k ^obs_k(t)∪\UNK\. (29) Here, kcanonV^canon_k contains benchmark-defined canonical values, kobs(t)V^obs_k(t) contains values observed or clarified up to turn t, and UNKUNK marks unresolved evidence. A new value is added only when it can be mapped to an existing slot with sufficient semantic confidence. Attributes without a reliable closed-slot mapping are assigned to a residual open slot and excluded from closed-slot state metrics. Surface mentions are normalized before entering kobs(t)V^obs_k(t). Paraphrases are merged only when they express the same persona commitment. For example, prefers concise answers and likes brief explanations may be merged, while vegetarian and vegan remain distinct unless the dialogue explicitly resolves them. The same normalization rules are applied to all methods. When a new value is admitted into k(t)V^(t)_k, the previous posterior bt−1kb^k_t-1 is extended with a configuration-fixed prior mass and then renormalized over the expanded support. The semantic mapping used to determine whether a newly observed value belongs to an existing slot is part of benchmark/schema normalization rather than a separate inference-time slot-matching model. Closed-slot belief metrics are computed only on normalized slots and values. Open-world attributes can still affect response-level evaluation, but they are not counted as closed-slot belief correctness. A.3 Context Encoding and Slot-Wise Evidence Scoring At turn t, CORE receives dialogue history Ht−1H_t-1, user utterance utu_t, and previous belief bt−1b_t-1. A shared context encoder produces ht=fenc(Ht−1,ut,bt−1).h_t=f_enc(H_t-1,u_t,b_t-1). (30) The maintained belief is provided as a compact symbolic state summary, including high-confidence slot values, unresolved UNKUNK states, and historical state-control information available before the current routing decision. No current-turn update action is included in hth_t, because routing is computed only after turn-local evidence and uncertainty descriptors have been obtained. This preserves the inference order context encoding→evidence scoring→uncertainty characterization→update routingcontext encoding scoring characterization routing. CORE does not use a separate inference-time slot matcher. Each slot is represented by its schema, including a slot name, a short natural-language definition, and the current candidate values in k(t)V^(t)_k. Slot grounding during inference is handled by slot-conditioned evidence scoring, while value-to-slot normalization used to construct benchmark targets is performed separately during preprocessing. For each slot, we augment the working value set with a null option: ¯k(t)=k(t)∪⊥, V^(t)_k=V^(t)_k∪\ \, (31) where ⊥ means that the current turn provides no usable evidence for slot k. For each v∈¯k(t)v∈ V^(t)_k, CORE computes a local evidence logit: ℓtk(v)=fevi(ht,ek,ev). ^k_t(v)=f_evi(h_t,e_k,e_v). (32) The logit measures what the current turn locally suggests; it does not decide whether the maintained belief should be revised. For numerical stability, evidence logits are centered and clipped within each slot: ℓ¯tk(v) ^k_t(v) =ℓtk(v)−1|¯k(t)|∑v′∈¯k(t)ℓtk(v′), = ^k_t(v)- 1| V^(t)_k| _v ∈ V^(t)_k ^k_t(v ), (33) Δtk(v) ^k_t(v) =clip⁡(ℓ¯tk(v)stk+ϵ,−Δmax,Δmax), =clip ( ^k_t(v)s^k_t+ε,- _ , _ ), (34) where stks^k_t is the within-slot scale used by the implementation and Δmax _ is fixed before evaluation. Both quantities are shared across experiments rather than selected from test performance. For candidate sets for which scale estimation is unreliable, the implementation applies clipping without the scale-normalization step. In the rest of the appendix, Δtk(v) ^k_t(v) denotes this stabilized evidence score, matching the evidence notation in the main text. The stabilized scores define a turn-local distribution over the augmented value space: p~tk(v)=exp⁡(Δtk(v))∑v′∈¯k(t)exp⁡(Δtk(v′)),v∈¯k(t). p^k_t(v)= ( ^k_t(v)) _v ∈ V^(t)_k ( ^k_t(v )), v∈ V^(t)_k. (35) This distribution is turn-local. It identifies which value, if any, is supported by the current observation; routing decides whether that evidence should be committed, deferred, or ignored. A.4 Uncertainty-Guided Update Routing CORE derives relevance, ambiguity, and conflict from the turn-local evidence distribution. The null option gives the relevance estimate: rtk=1−p~tk(⊥).r^k_t=1- p^k_t( ). (36) For non-null values, we renormalize the evidence distribution over k(t)V^(t)_k: p~t,+k(v)=p~tk(v)∑v′∈k(t)p~tk(v′),v∈k(t). p^k_t,+(v)= p^k_t(v) _v ^(t)_k p^k_t(v ), v ^(t)_k. (37) Ambiguity is computed as atk=H⁡(p~t,+k)log⁡|k(t)|,|k(t)|>1,0,|k(t)|≤1.a^k_t= cases H( p^k_t,+) |V^(t)_k|,&|V^(t)_k|>1,\\[3.44444pt] 0,&|V^(t)_k|≤ 1. cases (38) When a new value expands k(t)V^(t)_k, we first extend bt−1kb^k_t-1 with the same configuration-fixed prior mass used in Appendix A.2 and renormalize it over the aligned support. Conflict is then qtk=rtkDJS(p~t,+k∥bt−1k).q^k_t=r^k_tD_JS ( p^k_t,+\,\|\,b^k_t-1 ). (39) The multiplier rtkr^k_t prevents weakly matched turns from being treated as strong conflicts. When rtkr^k_t falls below the calibrated relevance threshold, slot k is treated as unmatched and bypasses persistent revision. The update policy directly conditions on the current evidence distribution, previous belief, and uncertainty descriptors: αtk∼πupdθ(α|p~tk,bt−1k,rtk,atk,qtk,ct−1k),αtk∈COMMIT,DEFER,IGNORE. gathered _t^k _upd^θ (α\, |\, p^k_t,b^k_t-1,r^k_t,a^k_t,q^k_t,c^k_t-1 ),\\[2.0pt] _t^k∈\COMMIT,DEFER,IGNORE\. gathered (40) The implementation uses the corresponding slot-conditioned encoded representation, avoiding a literal flattening of variable-length candidate distributions while preserving the routing semantics in the main text. The action semantics are: • COMMIT: authorize sufficiently grounded evidence to revise the maintained belief. • DEFER: preserve the current belief while exposing relevant but under-resolved evidence to the current-turn discourse policy. • IGNORE: exclude unrelated or explicitly non-persistent evidence from the active update pathway. DEFER does not create a separate durable pending-memory buffer. Deferred evidence remains current-turn control information and does not automatically become a future COMMIT. If related evidence or clarification later appears while the earlier dialogue context remains accessible, the slot is inferred and routed again from the newly available context. DEFER is an internal update action, whereas CLARIFY is a dialogue-level decision made after slot-level revision. A.5 Belief Revision After routing, CORE converts each update decision into a continuous gate. Following Section 3.3, the gate feature is ψtk=[rtk;1−atk;1−qtk;ct−1k].ψ^k_t=[r^k_t;1-a^k_t;1-q^k_t;c^k_t-1]. (41) The revision gate is gtk=[αtk=COMMIT]⋅σ(wg⊤ψtk+bg).g^k_t=I[α^k_t=COMMIT]·σ(w_g ψ^k_t+b_g). (42) DEFER and IGNORE set gtk=0g^k_t=0. Under COMMIT, the update magnitude is conditioned on relevance, ambiguity, conflict, and prior anchored confidence; we do not assume a fixed monotonic effect for any individual descriptor unless imposed by the implementation. Given the gate, CORE performs a KL-regularized posterior update over the non-null value set k(t)V^(t)_k: btk=argminp∈Δ⁡(k(t))[KL(p∥bt−1k)−gtk∑v∈k(t)p(v)Δtk(v)]. b^k_t= _p∈ (V^(t)_k) [KL(p\|b^k_t-1)-g^k_t _v ^(t)_kp(v) ^k_t(v) ]. (43) The null option ⊥ is used only for evidence relevance and is never written into the persona belief. The solution is btk(v)=bt−1k(v)exp⁡(gtkΔtk(v))∑v′∈k(t)bt−1k(v′)exp⁡(gtkΔtk(v′)),v∈k(t). gatheredb^k_t(v)= b^k_t-1(v) (g^k_t ^k_t(v)) _v ^(t)_kb^k_t-1(v ) (g^k_t ^k_t(v )),\\[4.30554pt] v ^(t)_k. gathered (44) Thus gtk=0g^k_t=0 preserves the prior, while larger gtkg^k_t moves the posterior toward evidence-supported values. For DEFER and IGNORE, the posterior remains unchanged, and deferred evidence is used only for current-turn discourse control. After slot-level revision, CORE decides whether to respond or request clarification: dt∼πθdis(d| d_t π^θ_dis (d\, | OPENut,Ht−1,bt,rtk,atk,qtk,αtkk=1K), u_t,\,H_t-1,\,b_t,\,\r_t^k,a_t^k,q_t^k, _t^k\_k=1^K ), (45) dt∈RESPOND,CLARIFY. d_t∈\RESPOND,CLARIFY\. A deferred observation does not automatically trigger clarification. A.6 Inference-Time Algorithm Algorithm 1 summarizes inference. Evidence is scored before routing; only evidence routed through COMMIT can revise the maintained belief. Algorithm 1 CORE inference-time algorithm 1: History Ht−1H_t-1, user utterance utu_t, previous belief bt−1b_t-1 2: Updated belief btb_t and response yty_t 3: ht←fenc(Ht−1,ut,bt−1)h_t← f_enc(H_t-1,u_t,b_t-1) 4: upd←COMMIT,DEFER,IGNOREA_upd←\COMMIT,DEFER,IGNORE\ 5: dis←RESPOND,CLARIFYA_dis←\RESPOND,CLARIFY\ 6: for k=1,…,Kk=1,…,K do 7: Construct k(t)V^(t)_k 8: ¯k(t)←k(t)∪⊥ V^(t)_k ^(t)_k∪\ \ 9: Score ℓtk(v) ^k_t(v) for all v∈¯k(t)v∈ V^(t)_k 10: Stabilize scores to obtain Δtk(v) ^k_t(v) 11: Compute p~tk p^k_t over ¯k(t) V^(t)_k 12: rtk←1−p~tk(⊥)r^k_t← 1- p^k_t( ) 13: if rtkr^k_t is below the relevance threshold then 14: αtk←IGNOREα^k_t 15: btk←bt−1kb^k_t← b^k_t-1 16: else 17: Compute p~t,+k p^k_t,+ over k(t)V^(t)_k 18: Compute atka^k_t and qtkq^k_t using Eq. (39) 19: Predict αtk _t^k using Eq. (40) 20: if αtk=COMMITα^k_t=COMMIT then 21: Compute gtkg^k_t using Eq. (42) 22: Update btkb^k_t using Eq. (44) 23: else 24: btk←bt−1kb^k_t← b^k_t-1 25: end if 26: end if 27: end for 28: Predict dtd_t using Eq. (45) 29: yt∼πrespθ(⋅∣Ht−1,ut,bt,dt)y_t π^θ_resp(· H_t-1,u_t,b_t,d_t) 30: return bt,ytb_t,y_t Gradient boundaries. During supervised warm-up, gradients are applied to the evidence scorer, update router, discourse policy, and response policy through their corresponding supervised losses. The belief update in Eq. (44) remains differentiable with respect to the evidence scores and revision gate, while the discrete routing action is optimized through the routing loss and policy objective. Appendix B Experimental and Training Protocol This section specifies the experimental and training protocol behind Section 4. It covers benchmark splits, slot instantiation, baseline controls, update-action labels, label auditing, supervised warm-up, uncertainty calibration, PPO rewards, and stability checks. The goal is to make the comparison auditable and to test whether the observed gains can be distinguished from alternative explanations such as hidden supervision, larger context budgets, reward artifacts, or degenerate conservatism. B.1 Benchmarks, Splits, and Evaluation Scope All methods are evaluated in the same partially observed interaction setting. The model observes the dialogue history and current user utterance, but never observes the latent persona state directly. It must infer, maintain, and update personalization from turn-level evidence. We use three complementary benchmarks. ALOE evaluates progressive preference revelation; PersonaChat evaluates generalization to free-form persona attributes; and PERSIST evaluates robustness under ambiguity, conflict, and controlled social influence. For ALOE and PersonaChat, splits are constructed at the persona level whenever profile identifiers are available. For datasets without explicit identifiers, we cluster normalized persona descriptors and split clusters rather than individual turns. Table 4 reports the processed splits for ALOE and PersonaChat. The calibration split is used for checkpoint selection, threshold calibration, response-judge calibration, and uncertainty calibration when applicable. PERSIST is used only for held-out robustness evaluation; its benchmark construction and evaluation protocol are described separately in Appendix D. Table 4: Processed experimental splits. ALOE and PersonaChat splits are fixed before training and shared by all methods. Benchmark Train Cal. Test Turns Split guarantee ALOE 4,800 600 600 61,248 preference-cluster disjoint PersonaChat 8,448 1,024 1,024 103,872 persona-profile disjoint Response-level metrics are computed on all retained examples. Closed-slot belief metrics are computed only when the relevant persona evidence can be reliably mapped to a normalized slot-value target. Residual open-world attributes remain in response-level evaluation but are excluded from closed-slot state metrics. B.2 Slot Instantiation and Baseline Controls CORE uses the same state-control interface across benchmarks, while the slot inventory is benchmark-instantiated. This avoids imposing a universal ontology while keeping evaluation auditable. ALOE, PersonaChat, and PERSIST instantiate 10, 8, and 12 closed slots, respectively. Their closed-slot coverage after normalization is 91.4%, 86.6%, and 89.1%; the remaining attributes are assigned to residual open slots and excluded only from closed-slot state metrics. Typical slots cover explanation style, privacy, budget, lifestyle, diet, verbosity, formality, travel, and risk tolerance. Surface mentions are normalized before evaluation. Paraphrases are merged only when they express the same stable commitment. For example, prefers concise answers and likes brief explanations may share a verbosity value, while vegetarian and vegan remain distinct unless explicitly clarified. The same normalization rules are applied to CORE and all baselines. For systems that do not expose a native structured persona belief, state-level diagnostics are computed with the same frozen evaluation-time state extractor, applied uniformly across those systems. These quantities should be interpreted as evaluation-time diagnostics of observable persona-state content rather than as claims about a baseline’s native internal representation. Baselines test six alternative explanations: prompting or reasoning traces (Reminder, Self-Critic, CoT), retrieved or persistent memory (RAG top-55, MemoryBank), offline preference optimization (SFT, DPO), personalized-agent control (RLPA), intermediate-format supervision (structure-matched controls), and scale/backbone effects. Same-backbone methods use matched decoding and calibration protocols; structure-matched controls are reported in Appendix E. All baselines use the same dialogue input and context budget. Reminder prepends a persona summary, Self-Critic revises drafts with persona-consistency critique, CoT adds reasoning instructions, and RAG retrieves top-5 history snippets. MemoryBank maintains persistent textual memory, while RLPA keeps its personalized-agent interface without CORE-style belief revision. SFT and DPO use the same processed trajectories with matched optimization and checkpoint selection. All same-backbone methods use Llama-3.2-3B-Instruct with an 8k context, the same processed training pool, matched optimization and checkpoint selection, and 256-token decoding at temperature 0.7; retrieval baselines use top-55 retrieval unless otherwise stated. CORE’s compact belief summary counts against the same context budget used by memory and retrieval baselines. Retrieval baselines may access raw retrieved histories, whereas CORE uses compressed belief states and compact historical state-control summaries. Thus, the methods share the same context budget while differing in how persona information is represented and maintained. Gold normalized persona states and update-action annotations are used for supervision or evaluation only. At inference time, CORE operates from dialogue context, its maintained belief, predicted turn-local evidence, and predicted routing decisions; no gold state or action annotation is required. B.3 Update-Action Labels and Audit CORE uses slot-turn update labels for supervised warm-up on ALOE and PersonaChat and for decision-level evaluation. Corresponding PERSIST labels are diagnostic only and are never used for CORE training or calibration. These labels are constructed independently of CORE predictions. Each label belongs to αtk,⋆∈COMMIT,DEFER,IGNORE.α^k, _t∈\COMMIT,DEFER,IGNORE\. (46) The label rule uses four observable factors: ReltkRel_t^k indicates whether turn t contains evidence about slot k; AmbtkAmb_t^k indicates whether the relevant evidence remains under-resolved; UConftkUConf_t^k indicates whether the evidence conflicts with a grounded prior while the conflict remains unresolved; and StabletkStable_t^k indicates whether the evidence expresses a persistent rather than explicitly local commitment. Here, Stabletk=0Stable^k_t=0 is reserved for evidence that is explicitly task-local, transient, hypothetical, quoted from another speaker, or otherwise not intended to characterize the user’s persistent persona. Uncertainty about a potentially persistent preference is instead handled by DEFER. Candidate labels are assigned by the mutually exclusive rule αtk,⋆=IGNORE,Reltk=0∨Stabletk=0,DEFER,Relkt=1,Stablekt=1,Ambtk=1∨UConftk=1,COMMIT,Relkt=1,Stablekt=1,Ambkt=0,UConfkt=0.α^k, _t= casesIGNORE,&Rel^k_t=0\; \;Stable^k_t=0,\\[2.58334pt] DEFER,&Rel^k_t=1,\;Stable^k_t=1,\\ &Amb^k_t=1\; \;UConf^k_t=1,\\[2.58334pt] COMMIT,&Rel^k_t=1,\;Stable^k_t=1,\\ &Amb^k_t=0,\;UConf^k_t=0. cases (47) COMMIT denotes sufficiently resolved evidence for persistent revision. DEFER denotes relevant evidence that may be persistent but whose value or relation to the grounded prior remains unresolved. IGNORE denotes either unrelated evidence or evidence whose semantics are explicitly local or non-persistent. The three categories are mutually exclusive by construction. An observed conflict does not imply permanent rejection. If subsequent evidence resolves the conflict and clearly establishes a persistent preference change, the new observation is re-evaluated and may receive COMMIT. We audit labels in four stages. First, a proposer constructs a candidate update label from the dialogue context, target slot, previous belief summary, current utterance, and benchmark metadata. Second, an independent verifier assigns COMMIT, DEFER, or IGNORE from the same observable dialogue information. The verifier is not shown the proposer’s factor annotations, proposed action, confidence, rationale, or any CORE prediction. Third, intent-drift filtering removes examples whose realized evidence type does not match the intended condition. Finally, a stratified human audit checks slot mapping, action semantics, and false-COMMIT risk. All filtering and auditing are performed before CORE training and never depend on CORE success or failure. Table 5 merges label distribution and audit statistics. Table 5: Update-label construction and audit summary. Labels are built before training and filtered independently of CORE predictions. Benchmark COMMIT DEFER IGNORE Verifier agree Drift removed Final retained Main removed pattern ALOE 43.8 24.6 31.6 89.6% 6.1% 157,384 Local task requests mislabeled as stable preferences. PersonaChat 39.5 27.2 33.3 86.8% 7.4% 198,912 Free-form persona statements with underspecified slot mapping. PERSIST 31.6 32.4 36.0 84.9% 9.8% 247,536 Stress turns whose pressure was weaker than intended. Verifier–human agreement is strongest for COMMIT, while most residual disagreement occurs between DEFER and IGNORE; this mainly affects fine-grained ACTACC because both actions avoid unsupported commitment on conflict-positive turns. B.4 Supervised Warm-Up and Uncertainty Calibration We use the supervised objective defined in Section 3.4. The evidence target is ⊥ when the current turn provides no usable evidence for slot k. After warm-up, calibration parameters associated with relevance, ambiguity, and conflict are selected on the held-out calibration split and fixed during PPO. This prevents policy optimization from improving reward by reshaping the uncertainty estimates themselves. B.5 PPO Configuration and Stability Checks All same-backbone trainable methods use Llama-3.2-3B-Instruct with the same tokenizer. Experiments are run on NVIDIA A800 80GB GPUs. All trainable methods are implemented with PyTorch and Hugging Face Transformers using Llama-3.2-3B-Instruct and its default tokenizer. RAG uses top-55 retrieval, and all generation-based methods use matched decoding settings. All reported results use matched training pools, effective batch sizes, update steps, decoding settings, and checkpoint-selection protocols. PPO starts from the supervised warm-up checkpoint πSFT _SFT. The policy state contains the dialogue representation, maintained belief summary, calibrated uncertainty descriptors, current-turn routing descriptors, previous update actions, and the current discourse decision. It does not contain gold slot labels or gold update actions at inference time. We optimize ℒPPO= _PPO= −t[min⁡(ρtAt,clip⁡(ρt,1−ϵ,1+ϵ)At)] -E_t [ ( _tA_t,clip( _t,1-ε,1+ε)A_t ) ] (48) +βKLDKL(πθ(⋅|st)∥πSFT(⋅|st)) + _KLD_KL ( _θ(·|s_t)\,\|\, _SFT(·|s_t) ) +λVℒV−λHℋ(πθ), + _VL_V- _HH( _θ), where ρt _t is the policy ratio, AtA_t is the estimated advantage, ℒVL_V is the value loss, and ℋH is policy entropy. The task reward is Rt=λactRtact+λstateRtstate+λansRtans+λclarRtclar, splitR_t=& _actR^act_t+ _stateR^state_t\\ &+ _ansR^ans_t+ _clarR^clar_t, split (49) while the KL regularizer is applied separately in Eq. (48). We set the reward weights for action correctness, state fidelity, response quality, and clarification behavior to 0.350.35, 0.300.30, 0.250.25, and 0.100.10, respectively. The reward components correspond to distinct training signals. RtactR^act_t evaluates COMMIT, DEFER, and IGNORE routing; RtstateR^state_t evaluates fidelity of the maintained persona belief; RtansR^ans_t evaluates personalized response quality; and RtclarR^clar_t evaluates whether clarification is used when relevant evidence remains unresolved while penalizing unnecessary clarification on well-resolved turns. Table 6: PPO configuration. The same setting is used across CORE PPO runs unless otherwise stated. Hyperparameter Value Reference policy Frozen SFT checkpoint PPO clip range ϵε 0.20 KL coefficient βKL _KL 0.05 Value-loss weight λV _V 0.50 Entropy weight λH _H 0.01 GAE λ 0.95 Discount factor γ 0.99 Effective batch size 128 PPO epochs / batch 4 Max gradient norm 1.0 We track both optimization-level and behavior-level diagnostics during PPO. Optimization diagnostics include total reward, component rewards, KL divergence, policy entropy, clip fraction, value loss, and gradient norm. Behavior diagnostics include action frequencies, clarification rate under ambiguity, false commitment, missed update rate, and belief drift. A robust policy should not obtain high apparent safety by always deferring, always ignoring, or always asking clarification. We therefore audit collapse to always-DEFER or always-IGNORE, excessive clarification on low-ambiguity turns, verbose but weakly informative personalization, correct-sounding responses paired with inaccurate belief updates, and superficial style mimicry that does not improve decision quality. Detailed reward-hacking and degeneracy diagnostics are reported in Appendix E. Appendix C Evaluation Metrics, Reliability, and Human Study This section defines the metrics used in Section 4 and reports reliability checks for both automatic and human evaluation. The central principle is metric-source separation: generated responses, maintained beliefs, update actions, and clarification decisions are evaluated from different objects and supervision sources. This prevents response-level judge scores from determining state- or action-level conclusions. C.1 Metric-Source Separation We evaluate generated responses, maintained beliefs, update actions, and clarification decisions from distinct sources. GPT-5.1 is used for AL and ARUS and for an auxiliary pairwise reliability audit. Belief- and decision-level metrics use reference annotations, maintained states, logged native actions, or the fixed behavior-derived proxies in Appendix C.4. Response-level outcomes and human evaluation are treated as primary end-to-end evidence; state and action metrics are complementary diagnostics. C.2 Response-Level Metrics Alignment Level. Alignment Level (AL) is the primary response-level metric. For dialogue j at turn t, the judge assigns sj,t∈1,2,3,4,5,s_j,t∈\1,2,3,4,5\, (50) where higher scores indicate stronger personalized alignment, better use of grounded persona evidence, and fewer violations of established commitments. We report AL on a 0–100 scale: ALt _t =100⋅1|t|∑j∈tsj,t−14, =100· 1|J_t| _j _ts_j,t-14, (51) AL =1T∑t=1TALt. = 1T _t=1^TAL_t. Because the underlying judge score is ordinal, we interpret AL together with pairwise preference and human-agreement checks. Normalized Improvement Rate. We normalize turn-level AL by AL~t=ALt100. AL_t= AL_t100. (52) We then fit AL~t=β0+β1t+ϵt, AL_t= _0+ _1t+ _t, (53) and define N-IR=β1.N -IR= _1. (54) Higher N-IR means that the model uses accumulating persona evidence more efficiently. Trajectory regularity. Normalized R2R^2 measures whether alignment improves smoothly: N-R2=1−∑t=1T(AL~t−AL~^t)2∑t=1T(AL~t−AL~¯)2.N -R^2=1- _t=1^T ( AL_t- AL_t )^2 _t=1^T ( AL_t- AL )^2. (55) C.3 Belief-Level Metrics Belief-level metrics evaluate the maintained persona state rather than the generated response. They are computed only on closed-slot examples with reliable gold slot-value annotations. Let zj,tk,⋆z^k, _j,t be the gold value for slot k in dialogue j at turn t, and let bj,tkb^k_j,t be the model’s posterior belief. Slot accuracy. SLOTACC=100||∑(j,t,k)∈[argmaxvbj,tk(v)=zj,tk,⋆].SLOTACC= 100|S| _(j,t,k) I [ *argmax_vb^k_j,t(v)=z^k, _j,t ]. (56) Belief update accuracy. Let U be the set of update-positive slot-turns where the gold state should change or be confirmed by reliable evidence. Then BUA=100⋅1||∑(j,t,k)∈[argmaxvbj,tk(v)=zj,tk,⋆].BUA=100· 1|U| _(j,t,k) I [ _vb^k_j,t(v)=z^k, _j,t ]. (57) BUA checks whether a model can adapt when evidence is reliable, so high robustness cannot be explained by complete inertia. Persona-state drift. DRIFT=100⋅1||∑(j,t,k)∈(1−bj,tk(zj,tk,⋆)).DRIFT=100· 1|S| _(j,t,k) (1-b^k_j,t(z^k, _j,t) ). (58) Lower DRIFT means less posterior mass is assigned away from the gold persona value. Belief calibration. We additionally assess whether slot confidence tracks empirical MAP correctness through the reliability analysis reported with Figure 3(c) in the main text. C.4 Decision-Level Metrics Decision-level metrics are mechanism diagnostics for update control rather than primary measures of end-to-end system quality. For CORE, they are computed from logged native update actions and reference labels. For systems without explicit update actions, a fixed behavior-derived proxy is constructed from observable post-turn behavior: persistent adoption of the new evidence is mapped to COMMIT, explicit unresolved handling or clarification to DEFER, and preservation of the prior commitment without incorporating the new cue to IGNORE. These proxies are used only for diagnostic action metrics, are not used for training or model selection, and should not be interpreted as native internal actions of the baseline systems. Action accuracy. ACTACC=100⋅1||∑(j,t,k)∈[αj,tk=αj,tk,⋆].ACTACC=100· 1|A| _(j,t,k) I[α^k_j,t=α^k, _j,t]. (59) Conflict non-commitment rate (CRR). Let C denote conflict-positive slot-turns where an unsupported persistent commitment should be avoided. CRR measures whether the model avoids COMMIT at this level; it does not attempt to distinguish the finer DEFER-versus-IGNORE boundary. CRR=100||∑(j,t,k)∈[αj,tk∈Defer,Ignore].CRR= 100|C| _(j,t,k) I [α^k_j,t∈\ Defer, Ignore\ ]. (60) False commitment rate. Let N be ambiguity-positive or conflict-positive slot-turns whose reference action is not COMMIT: FCR=100⋅1||∑(j,t,k)∈[αj,tk=COMMIT].FCR=100· 1|N| _(j,t,k) I[α^k_j,t=COMMIT]. (61) Adaptive probing rate. Let Q be ambiguity-positive turns for which clarification is preferred: APR=100⋅1||∑(j,t)∈[dj,t=CLARIFY].APR=100· 1|Q| _(j,t) I[d_j,t=CLARIFY]. (62) C.5 PERSIST Robustness Metrics PERSIST evaluates whether a model preserves grounded persona commitments under ambiguity, conflict, and controlled social influence. It does not require models to expose CORE-specific actions. Axiom Recall Under Stress. For confrontation turns with stressed axiom aja_j, ARUS measures whether the response preserves the anchored axiom: ARUS=100||∑j∈[JudgePreserve(yj,aj)=1].ARUS= 100|P| _j I [JudgePreserve(y_j,a_j)=1 ]. (63) Blind Compliance Defense. Let ℬB denote pressure-positive turns where blind compliance would overwrite grounded persona evidence. BCD is BCD=100⋅1|ℬ|∑j∈ℬ[yj∈jacceptable].BCD=100· 1|B| _j I[y_j ^acceptable_j]. (64) Acceptable behavior includes preserving the anchored commitment, resisting unsupported pressure, or responding cautiously. The acceptable-behavior set is specified by PERSIST’s behavior-level target annotations before model evaluation, so the target set itself is independent of any evaluated system prediction. Strategy-level macro average. For strategy s, let sD_s be dialogues whose dominant confrontation strategy is s. For any metric M, Ms M_s =1|s|∑j∈sMj, = 1|D_s| _j _sM_j, (65) MPERSIST M_PERSIST =1|strat|∑s∈stratMs. = 1|S_strat| _s _stratM_s. (66) This prevents a strategy with more examples from dominating aggregate robustness. C.6 Aggregation and Uncertainty Reporting Unless otherwise specified, turn-level metrics are first averaged within dialogue and then averaged over dialogue instances. Slot-level metrics are macro-averaged across slots to avoid domination by frequent slots. For trained methods, each dialogue-level quantity is first averaged across the three random seeds. Unless a table explicitly states another uncertainty measure, the reported ± value is then the standard error across dialogue instances: SE⁡(M)=std⁡(Mjj=1N)N.SE(M)= std (\M_j\_j=1^N ) N. (67) For the main comparisons, we additionally use paired bootstrap resampling over dialogue instances. Let Δj=MjCORE−Mjbaseline _j=M_j^CORE-M_j^baseline. We sample dialogues with replacement and estimate the bootstrap distribution of Δ¯=1N∑j=1NΔj. = 1N _j=1^N _j. (68) A difference is treated as statistically significant when the 95% paired-bootstrap confidence interval excludes zero. Strategy-level PERSIST breakdowns are diagnostic unless otherwise stated. C.7 Judge Prompts, Calibration, and Reliability GPT-5.1 is used for the semantic response-level AL and ARUS evaluations. The same judge family is additionally used in an auxiliary pairwise reliability audit whose purpose is to measure consistency with human preference judgments. All judge evaluations use fixed prompts and rubrics, randomized system order, and held-out calibration examples, and judges are never shown method names. Judge outputs are not used to compute belief-level or action-level metrics. AL judge prompt. System instruction. Evaluate whether the assistant response is aligned with the user’s persona evidence available up to the current dialogue turn. Focus on the final response. Do not reward verbosity, hidden reasoning, or generic helpfulness unless it helps satisfy grounded persona evidence. When later evidence clearly establishes a persistent preference change, appropriate adaptation should be rewarded. Input. Dialogue history; current user message; established persona evidence; one assistant response. Rubric. Score from 1 to 5. 1: contradicts or ignores established persona evidence. 2: weakly relevant but mostly generic or partly inconsistent. 3: acceptable response with limited use of persona evidence. 4: clearly personalized and consistent with established persona evidence. 5: strongly personalized, contextually appropriate, and appropriately preserves or revises grounded persona evidence. Return only a JSON object with keys score and brief_reason. ARUS audit prompt. System instruction. Audit whether the final assistant response appropriately handles an anchored user persona axiom under stress. Judge by faithful use of the anchored axiom, not by general persuasiveness; do not penalize revision if the dialogue clearly establishes a persistent preference change. Input. Anchored persona axiom; dialogue context; stress cue; assistant response. Decision. Output 1 if the response explicitly recalls, faithfully paraphrases, or unambiguously uses the anchored axiom when preservation is warranted. Output 0 if it forgets or replaces the axiom without sufficient support. Return only a JSON object with keys label and brief_reason. Calibration uses only held-out calibration data and never uses test responses. Calibration examples cover clear successes, clear failures, ambiguous turns, and social-pressure cases. Table 7 reports judge calibration and human agreement in a compact form. Table 7: Judge and human reliability. Judge-based evaluation is used for AL and ARUS; pairwise preference is included only as an auxiliary evaluator-reliability audit. Task Items H–H J–H κ AL 5-point score 720 0.81 0.78 0.69 ARUS binary audit 540 0.88 0.84 0.76 Pairwise preference 640 0.84 0.81 0.72 Table 7 summarizes the larger judge-calibration audit. The separate end-to-end human study uses a smaller matched transcript subset and reports its evaluator-level reliability statistics in Table 8; the two tables therefore correspond to different evaluation subsets. Candidate response order is randomized in judge-based comparisons. A reversed-order subset is additionally used as a positional-bias sanity check; this diagnostic does not affect the reported model scores. C.8 Anti-Gaming Diagnostics We audit collapse to always-COMMIT, always-DEFER/IGNORE, and unnecessary clarification. CORE reduces FCR while maintaining strong BUA, indicating that its robustness is not explained by uniformly avoiding updates. C.9 Human Evaluation Protocol Human evaluation audits annotation quality, judge reliability, and end-to-end interaction quality. Eight trained evaluators review matched 12-turn conversations from anonymized SFT, MemoryBank, RLPA, and CORE systems over benign, ambiguity-heavy, and conflict-heavy scenarios. System identity is hidden, presentation order is counterbalanced, and scenario content is matched across systems. Separate evidence–action audits assess whether slot evidence, ambiguity/conflict, update actions, and clarification decisions are appropriate. C.10 Human Rating Rubric and Statistical Analysis Evaluators rate persona consistency (H-PC), adaptive ambiguity handling (H-A), response quality (H-RQ), preference preservation under pressure (H-P), and long-horizon stability (H-LS) on 1–5 scales. They are instructed not to reward verbosity, generic helpfulness, excessive agreement, or unnecessary clarification, and to reward adaptation only when later evidence clearly establishes a persistent preference change. H-PC, H-A, and H-P combine holistic conversation ratings with blinded transcript review; H-RQ and H-LS use holistic ratings. Scores are aggregated at the evaluator level, and 95% confidence intervals are obtained from 10,000 evaluator-level bootstrap resamples. Evaluators had prior NLP or dialogue-evaluation experience, provided consent, and were compensated according to institutional annotation rates. The study uses generated or benchmark dialogues only and does not collect real user conversations or personally identifying information. We also report paired Cohen’s d for CORE versus RLPA, dpaired=xCORE−xbase¯std⁡(xCORE−xbase).d_paired= x_CORE-x_basestd(x_CORE-x_base). (69) Table 8 reports human–human, judge–human, and Cohen’s κ agreement on held-out audit items. Table 8: Human and judge reliability. Audit item # H–H J–H κ Response scoring 160 0.84 0.82 0.76 Robustness audit 120 0.90 0.86 0.82 Pairwise preference 96 0.87 0.84 0.79 C.11 Human Evaluation Results Table 9 reports the end-to-end human evaluation results. CORE receives the highest rating on all five dimensions. Compared with RLPA, CORE improves the average human rating by 11.7%. The gains are larger on persona-sensitive dimensions—persona consistency, ambiguity handling, preference preservation, and long-horizon stability—than on response quality. The larger gains on these dimensions show that CORE’s human-evaluation improvements are concentrated on long-horizon persona-state behavior rather than generic response fluency. Table 9: Main human evaluation results. Scores are 1–5 Likert means with 95% bootstrap confidence intervals. Relative gains compare CORE against RLPA. Method H-PC ↑ H-A ↑ H-RQ ↑ H-P ↑ H-LS ↑ SFT 3.24±.283.24_±.28 3.02±.303.02_±.30 3.79±.223.79_±.22 2.98±.312.98_±.31 3.11±.293.11_±.29 MemoryBank 3.62±.263.62_±.26 3.31±.283.31_±.28 3.92±.213.92_±.21 3.38±.293.38_±.29 3.49±.273.49_±.27 RLPA 3.96±.223.96_±.22 3.73±.243.73_±.24 4.05±.194.05_±.19 3.82±.253.82_±.25 3.89±.233.89_±.23 CORE 4.42±.184.42_±.18 4.31±.194.31_±.19 4.24±.184.24_±.18 4.36±.184.36_±.18 4.39±.184.39_±.18 CORE–RLPA gap +0.46+0.46 +0.58+0.58 +0.19+0.19 +0.54+0.54 +0.50+0.50 Relative gain +11.6%+11.6\% +15.5%+15.5\% +4.7%+4.7\% +14.1%+14.1\% +12.9%+12.9\% Paired d 0.720.72 0.810.81 0.310.31 0.830.83 0.790.79 C.12 Pairwise Preference and Scenario Analysis We further compare CORE directly against RLPA, the strongest baseline in automatic and scenario-wise human results. Evaluators see anonymized matched transcript pairs and choose which system better preserves grounded personalization across the session. Ties are allowed. Table 10 reports pairwise preferences by scenario family. Table 10: Pairwise human preference between CORE and RLPA. Counts are computed from anonymized matched transcript comparisons. Scenario # CORE RLPA Tie Benign Revelation 32 18 10 4 Ambiguity-heavy 32 20 8 4 Conflict-heavy 32 24 4 4 Total 96 62 22 12 Overall, CORE is preferred in 64.6% of pairwise comparisons, RLPA in 22.9%, and ties occur in 12.5%. The preference gap is largest in conflict-heavy scenarios, where systems must decide whether later pressure should revise or be blocked from the maintained persona state. The larger preference gap in conflict-heavy scenarios is consistent with the main finding that CORE’s advantage is most pronounced when personalization evidence becomes ambiguous, conflicting, or appears under controlled social-influence stress. Appendix D PERSIST: A Benchmark for Persona-State Robustness under Sequential Interaction Stress This section describes PERSIST, the phase-structured robustness benchmark used in Section 4. PERSIST tests whether a personalized dialogue model preserves grounded persona commitments when later interaction becomes ambiguous, conflicting, or subject to controlled social-influence stress. Unlike benign personalization benchmarks that mainly reward preference acquisition, PERSIST evaluates whether grounded persona evidence is preserved or appropriately revised under subsequent interaction stress. A central design principle is method agnosticism. PERSIST is not defined in terms of CORE’s COMMIT, DEFER, or IGNORE actions. Its labels specify observable target behaviors: preserving an anchored axiom, resisting unsupported pressure, asking clarification under unresolved uncertainty, responding cautiously when evidence is incomplete, or adapting when a change is sufficiently grounded. Systems without explicit belief states can therefore be evaluated from final responses; systems with internal state can additionally be diagnosed with belief- and action-level metrics. Figure 6: PERSIST construction pipeline. Persona sampling, axiom grounding, perturbation, confrontation generation, verification, human audit, and split assignment are separated before model evaluation. D.1 Benchmark Objective and Target Behaviors PERSIST targets a failure mode that standard personalization benchmarks do not isolate. In long-horizon interaction, not every new observation should revise the maintained user state. Later turns may contain temporary moods, ambiguous remarks, misleading cues, or social pressure. A robust model must distinguish evidence that justifies persistent persona revision from observations that remain too weak, transient, or unresolved to support such a change. PERSIST evaluates three objectives: • Persona preservation: preserve persona axioms grounded earlier in the dialogue. • Unsupported-overwrite resistance: avoid replacing grounded evidence with ambiguous, transient, unsupported, or pressured cues. • Adaptive uncertainty handling: clarify or respond cautiously when the correct persona update is unresolved. Each stressed turn records a behavior-level target: y⋆beh∈ y _beh∈ \ preserve_prior,resist_pressure, \_prior,\;resist\_pressure, (70) ask_clarification,cautious_response, \_clarification,\;cautious\_response, safe_adaptation. \_adaptation \. PERSIST does not assume that persona evidence is immutable. If later evidence clearly and consistently establishes a genuine preference change, adaptation is acceptable. The benchmark therefore separates grounded change from pressure-induced overwrite. Each dialogue follows three phases: Anchoring grounds stable persona evidence, Fluctuation introduces partial or transient cues, and Confrontation challenges grounded commitments with controlled pressure. For dialogue j, PERSIST records grounded axioms j=aj,1,…,aj,mjA_j=\a_j,1,…,a_j,m_j\ and a stressed subset jstress⊆jA^stress_j _j. D.2 Dataset Statistics and Splits PERSIST-Full is the complete benchmark collection and includes split labels for future training, development, and diagnostic use. In this paper, however, CORE is trained without PERSIST data. PERSIST-Eval is the held-out subset used for the main robustness evaluation, and PERSIST-Manual is a smaller human-written subset reserved for evaluator audit and qualitative sanity checks. All splits are persona-level to prevent leakage of persona profiles, core axioms, or close paraphrases across subsets. Table 11: PERSIST dataset statistics. PERSIST-Full is the full benchmark; PERSIST-Eval is the held-out test subset used for main robustness evaluation. PERSIST-Manual is reserved for auditing and sanity checks. Statistic PERSIST-Full PERSIST-Eval PERSIST-Manual # Persona profiles 1,200 150 180 # Dialogues 9,600 1,200 720 # Total turns 124,832 15,614 9,384 Avg. turns / dialogue 13.00 13.01 13.03 Avg. anchoring turns 3.12 3.10 3.08 Avg. fluctuation turns 3.77 3.79 3.82 Avg. confrontation turns 6.11 6.12 6.13 Avg. core axioms / persona 3.42 3.39 3.47 Avg. stressed axioms / dialogue 1.74 1.76 1.81 Train / Dev / Test dialogues 7,200 / 1,200 / 1,200 – audit only The persona bank is constructed from non-private, interaction-relevant preference and value descriptions. Candidate principles are filtered for redundancy, sensitivity, safety, and actionability before persona profiles are sampled. The final bank retains 9,760 principles from 18,400 candidates; each persona contains 8.13 principles on average, from which core axioms are selected for grounding and stress testing. Phase-level verification labels confirm that the three phases differ in intended difficulty. Anchoring turns have high reliability and low conflict; Fluctuation increases ambiguity; Confrontation increases conflict and pressure intensity. These labels are used for diagnostic breakdowns rather than for method-specific training targets. D.3 Construction and Verification Pipeline PERSIST is generated through a construction-and-verification pipeline that separates persona construction, axiom grounding, perturbation, social-pressure instantiation, verification, human audit, and split assignment. This separation reduces leakage and makes benchmark construction auditable. Figure 6 gives the PERSIST construction pipeline. The accompanying materials include scripts for persona construction, axiom sampling, phase generation, verification, schema export, and metric computation. D.4 Social-Influence Strategy Taxonomy The Confrontation phase uses six social-influence strategies. Each strategy targets a distinct mechanism through which a model may overwrite a grounded persona commitment or comply with unsupported pressure. Figure 7 summarizes the taxonomy. Figure 7: Taxonomy of PERSIST social-influence strategies. PERSIST evaluates whether personalized dialogue models preserve grounded persona commitments under six pressure mechanisms: authority pressure, social conformity, affective appeal, urgency framing, reciprocity lure, and relationship pressure. Strategy coverage is approximately balanced in PERSIST-Eval: authority pressure, social conformity, affective appeal, urgency framing, reciprocity lure, and relationship pressure each contribute roughly one sixth of the held-out dialogues. Aggregate PERSIST robustness is therefore reported with strategy-level macro averaging, as defined in Appendix C. D.5 Quality Assurance and Human Audit PERSIST uses automatic verification and human audit to reduce label noise, leakage, and strategy misclassification. Filtering is applied before model evaluation and never depends on any model prediction. Automatic verification checks axiom faithfulness, phase and strategy consistency, conflict validity, clarification preference, label leakage, safety/privacy, and near duplication. Table 12 merges automatic filtering and human audit reliability. Table 12: PERSIST filtering and human-audit reliability. Filtering is independent of model predictions. Human audit is performed on 1,800 stratified dialogues. Check Flagged Removed κ Main disagreement or removal pattern Axiom faithfulness 8.7% 6.3% 0.82 Anchoring turn only weakly implied the target axiom. Phase consistency 7.9% 5.4% 0.89 Fluctuation–confrontation boundary was ambiguous. Strategy correctness 6.8% 4.9% 0.79 Affective appeal and relationship pressure were occasionally confounded. Conflict validity 9.5% 7.1% 0.76 Claimed conflict was too weak or not linked to the stressed axiom. Clarification preference 6.2% 4.1% 0.70 Ambiguity severity threshold caused disagreement. Label leakage 2.4% 2.4% – Generated text exposed phase or strategy wording. Safety / privacy 3.1% 3.1% – Persona content was too sensitive or identifying. Near duplication 5.6% 5.6% – Duplicate axiom or template paraphrase. Each stressed turn records behavior-level metadata: ℬj,t=bj,tpreserve,bj,tresist,bj,tclarify,bj,tcautious.B_j,t=\b^preserve_j,t,b^resist_j,t,b^clarify_j,t,b^cautious_j,t\. (71) These behavior-level labels directly support ARUS, BCD, and APR. Conflict and ambiguity annotations, together with native or behavior-derived actions, support CRR and FCR, while DRIFT is computed separately from normalized persona-state targets as defined in Appendix C. D.6 Evaluation Protocol PERSIST evaluation focuses on three benchmark-specific outcomes: Axiom Recall Under Stress (ARUS), Blind Compliance Defense (BCD), and Adaptive Probing Rate (APR). Full formulas, judge prompts, and reliability controls are provided in Appendix C; here we specify how PERSIST provides the required labels and metadata. ARUS. ARUS evaluates whether the final response preserves the relevant anchored persona axiom under confrontation. PERSIST provides the anchored axiom, stressed axiom identifier, confrontation context, and model response. BCD. BCD evaluates whether the model avoids blind compliance with unsupported pressure. PERSIST provides the pressure strategy label, conflict-positive flag, gold target behavior, and acceptable behavior set. APR. APR evaluates whether the model asks clarification when evidence is under-resolved. PERSIST provides ambiguity-positive flags and clarification-preferred labels. Strategy-level reporting. For strategy s, let sD_s be dialogues whose dominant confrontation strategy is s. For any robustness metric M, Ms=1|s|∑j∈sMj.M_s= 1|D_s| _j _sM_j. (72) Aggregate PERSIST robustness is macro-averaged across strategies: MPERSIST=1|strat|∑s∈stratMs.M_PERSIST= 1|S_strat| _s _stratM_s. (73) This prevents strategies with slightly more examples from dominating the final robustness score. D.7 Limitations of PERSIST PERSIST is designed to stress-test persona-state robustness, but it has boundaries. First, PERSIST focuses on dialogue-level ambiguity, conflict, and social influence. It does not cover all sources of drift, such as cross-session summarization errors, retrieval-index corruption, third-party memory updates, or external-tool memory failures. Second, PERSIST combines a model-assisted full set with a manually constructed subset. PERSIST-Full is constructed through a controlled pipeline and filtered by automatic verification, while PERSIST-Manual is reserved for human audit and qualitative sanity checks. This design improves controllability and coverage, although it cannot capture the full diversity of natural social interaction. Third, PERSIST evaluates post-anchor robustness of grounded persona evidence; it does not determine when a real user has genuinely changed a long-term preference. Because Anchoring precedes the stress phases, PERSIST also does not isolate first-contact initialization under pressure. We evaluate this complementary boundary separately in Appendix E.3. Deployment requires user-controlled memory inspection, correction, deletion, consent, and scope control. Fourth, PERSIST’s closed-slot diagnostics depend on reliable slot normalization. Open-world attributes that cannot be normalized are retained for response-level evaluation but excluded from closed-slot state metrics. This avoids false precision but leaves some open-world personalization behavior to qualitative and response-level analysis. These limitations motivate reporting PERSIST together with standard personalization benchmarks, belief-state metrics, update-action metrics, and human verification rather than treating PERSIST as a standalone measure of long-term personalization. Appendix E Additional Results and Mechanistic Analyses This section reports additional analyses supporting the mechanistic claims in Section 4. We focus on structure-matched controls, failure-mode diagnostics, the stability–adaptability boundary, backbone transfer, closed-slot state fidelity, robustness across stress strategies, and inference cost. Training details are provided in Appendix B, and metric definitions in Appendix C. E.1 Structure-Matched Controls Structure-matched controls test whether intermediate reasoning, action-name supervision, or additional structured labels can recover CORE’s performance without executable belief revision. Table 13: Structure-matched controls on PERSIST-Eval. Matching reasoning format or supervision without executable belief revision does not recover CORE’s robustness. Variant AL ↑ BUA ↑ ACTACC ↑ CRR ↑ FCR ↓ DRIFT ↓ SFT 76.2±.576.2_±.5 60.8±.760.8_±.7 – 54.2±1.054.2_± 1.0 31.8±.831.8_±.8 18.6±.518.6_±.5 Structured reasoning 78.1±.578.1_±.5 62.5±.762.5_±.7 – 58.7±.958.7_±.9 28.4±.828.4_±.8 17.2±.517.2_±.5 Trace + action names 79.0±.579.0_±.5 63.2±.763.2_±.7 61.5±.861.5_±.8 64.1±.864.1_±.8 24.6±.724.6_±.7 15.9±.515.9_±.5 Extra labels, no belief state 79.4±.579.4_±.5 64.7±.664.7_±.6 66.1±.866.1_±.8 67.3±.767.3_±.7 22.8±.722.8_±.7 15.1±.415.1_±.4 Belief state, no executable action 80.2±.480.2_±.4 66.9±.666.9_±.6 – 70.5±.770.5_±.7 20.5±.620.5_±.6 14.2±.414.2_±.4 Action prediction, no belief revision 80.0±.480.0_±.4 66.2±.666.2_±.6 71.8±.771.8_±.7 72.1±.672.1_±.6 19.2±.619.2_±.6 14.0±.414.0_±.4 CORE 82.9±.482.9_±.4 71.7±.671.7_±.6 78.9±.678.9_±.6 82.7±.482.7_±.4 11.3±.411.3_±.4 10.1±.410.1_±.4 Additional structure improves over plain SFT but remains below CORE. In particular, action prediction without executable belief revision reaches 80.0 AL and 72.1 CRR, compared with 82.9 AL and 82.7 CRR for CORE. This indicates that intermediate formatting, action names, and additional supervision alone do not explain the gains. E.2 Component Ablations by Failure Mode Failure-mode diagnostics further distinguish the roles of CORE’s components. Removing executable update actions increases unsupported COMMIT from 11.3% to 22.9%, while removing clarification increases under-clarification from 18.4% to 42.7%. Removing inertia, optimized revision, or calibration/revision rewards produces smaller but consistent degradation in alignment and persona-state stability, matching the aggregate ablations reported in the main text. E.3 Stability–Adaptability Boundary Diagnostics PERSIST evaluates robustness after persona evidence has been grounded. We therefore examine two complementary conditions: first-contact initialization under pressure and explicit preference reversal after an established preference. First-contact initialization. Without a grounded reference state, conflict provides a weaker signal. CORE reduces pressure-first FCR from 31.6% for RLPA to 20.9%, although this remains higher than CORE’s 11.6% post-anchor FCR. This confirms that ambiguity-aware routing remains useful during initialization, while conflict-aware preservation becomes stronger after reliable grounding. Preference reversal. We next test whether conservative revision prevents legitimate preference change. Table 14: Preference-reversal diagnostic. Explicit persistent changes can revise the maintained belief, whereas tentative or scoped deviations receive more conservative treatment. Condition Routing Competing posterior Stable MAP Explicit persistent change C → C → C .58→.74→.82.58→.74→.82 Turn 1 Tentative then confirmed D → C → C .43→.66→.80.43→.66→.80 Turn 2 Task-qualified deviation No persistent reversal Prior preference retained Unchanged Unresolved conflicting cue No premature reversal Prior preference retained Unchanged Explicit and persistent user-endorsed changes can therefore become the new MAP preference immediately, whereas tentative evidence is deferred until confirmation. Task-scoped or unresolved conflicting cues do not automatically overwrite the persistent persona state. E.4 Scale and Backbone Transfer The main experiments use Llama-3.2-3B-Instruct. Transferring CORE to Qwen2.5-3B-Instruct yields 79.6 ALOE AL, 72.8 PersonaChat AL, 77.4 ARUS, 79.1 CRR, 13.8 FCR, and 10.9 DRIFT, suggesting that the state-control interface is not specific to a single 3B backbone. E.5 Closed-Slot State Diagnostics Table 15 reports complementary closed-slot state-fidelity diagnostics. Table 15: Closed-slot state diagnostics. Method PersonaChat SLOTACC ↑ ALOE BUA ↑ Base 48.6 41.9 SFT 55.8 52.6 Reminder 50.1 50.8 Self-Critic 59.6 55.7 DPO 58.4 61.2 RAG top-55 61.7 56.4 CoT 52.3 54.9 MemoryBank 63.5 65.7 RLPA 66.9 68.8 CORE 74.3 71.7 E.6 PERSIST Robustness Breakdowns CORE’s advantage is consistent across all six social-influence strategies. Its strategy-level robustness ranges from 78.6 to 82.9, compared with 67.6–70.5 for RLPA, indicating that the aggregate PERSIST improvement is not driven by a single pressure strategy. E.7 Latency, Memory, and Retrieval Controls CORE adds 23.0 ms per turn over SFT on the same Llama-3.2-3B-Instruct backbone at batch size 1. Response decoding remains the dominant cost, while routing and belief revision introduce only a small pre-generation overhead. The main experiments already include RAG top-55 and MemoryBank controls, showing that improved access to historical evidence alone does not recover CORE’s robustness to unsupported persona overwrites.

Instant research discovery

Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue

Intelligence

Summary

Entities (14)

Relation Signals (13)

Abstract

Tags

Links

PDF

Full Text

Paper	Published	Area	Tags	Intel	Citations