Paper deep dive

MIRROR: From Imitation to Internalization in LLM Personalization

Huayi Lai, Jicheng Yang, Min Yi, Chong Meng

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 125

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 10/11/2026, 3:16:10 AM

Summary

The paper introduces MIRROR (Meta-personalization by Internalizing Reference-Revealed On-policy Reflections), a self-distillation framework for Large Language Model (LLM) personalization. Unlike traditional methods that rely on style imitation or supervised fine-tuning (SFT), MIRROR shifts personalization toward preference internalization by aligning the model's next-token distributions on its own on-policy trajectories with those of a reference-conditioned self. It also introduces MIRROR-F, a variant that adds selective supervision over informative reference tokens to enhance content quality. Experiments on benchmarks like LaMP and LongLaMP show that MIRROR and MIRROR-F achieve superior personalization performance and text quality while mitigating catastrophic forgetting compared to SFT-based baselines.

Entities (10)

Mirror · method · 98%MIRROR-F · method · 97%LaMP · dataset · 95%LongLamp · dataset · 95%Qwen3-1.7B · model · 93%Qwen3-4B · model · 93%Self-Distillation · technique · 92%On-policy alignment · technique · 90%SFT · method · 90%Catastrophic Forgetting · problem · 88%

Relation Signals (8)

Mirror → evaluatedon → LaMP

confidence 95% · Experiments are conducted on... LaMP

Mirror → evaluatedon → LongLamp

confidence 95% · Experiments are conducted on... LongLaMP

Mirror → uses → Self-Distillation

confidence 95% · MIRROR... a novel self-distillation framework

MIRROR-F → extends → Mirror

confidence 92% · we introduce MIRROR-F, a focal plug-in that augments on-policy distributional alignment

Mirror → addresses → Catastrophic Forgetting

confidence 90% · exhibiting less catastrophic forgetting than SFT-based baselines

Mirror → backedby → Qwen3-1.7B

confidence 90% · We use Qwen3-1.7B... as the backbone LLMs

Mirror → backedby → Qwen3-4B

confidence 90% · We use Qwen3-1.7B and Qwen3-4B as the backbone LLMs

SFT → suffersfrom → Catastrophic Forgetting

confidence 85% · can lead to catastrophic forgetting... and SFT-based baselines

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:The demand for personalized LLMs is shifting from style imitation toward content quality. We investigate whether self-distillation can bridge this gap in existing fine-tuning paradigm. To address this limitation, we introduce MIRROR(Meta- personalization by Internalizing Reference-Revealed On-policy Reflections), a novel self-distillation framework that shifts LLM personalization from imitation toward preference internalization. First, we replace reference-token imitation with reference-revealed on-policy self-distillation, aligning the model's next-token distributions along its own generation trajectories with those of its reference-conditioned self, thereby internalizing user preferences rather than reproducing reference this http URL, we introduce MIRROR-F, a focal plug-in that augments on-policy distributional alignment with selective supervision over informative reference tokens, thereby strengthening content generation while preserving user-specific expression. Across three personalized generation benchmarks, two model scales, and complementary reference-based and LLM-based evaluations, MIRROR and MIRROR-F achieve leading overall personalization performance and superior text quality, while exhibiting less catastrophic forgetting than SFT-based baselines on three unseen personalized generation tasks. The gains are consistent across model scales and application scenarios, translating to improved performance in LLM personalization tasks.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

121,309 characters extracted from source content.

Expand or collapse full text

MIRROR: From Imitation to Internalization in LLM Personalization Huayi Lai †thanks: Equal contribution. Affiliation: Baidu Inc. Email: yangjicheng02@baidu.com Jicheng Yang11footnotemark: 1 Affiliation: Baidu Inc. Email: mengchong01@baidu.com Min Yi Affiliation: Baidu Inc. Chong Meng †thanks: Corresponding author. Affiliation: Baidu Inc. Abstract The demand for personalized LLMs is shifting from style imitation toward content quality. We investigate whether self-distillation can bridge this gap in existing fine-tuning paradigm. To address this limitation, we introduce MIRROR(Meta- personalization by Internalizing Reference-Revealed On-policy Reflections),a novel self-distillation framework that shifts LLM personalization from imitation toward preference internalization. First, we replace reference-token imitation with reference-revealed on-policy self-distillation, aligning the model’s next-token distributions along its own generation trajectories with those of its reference-conditioned self, thereby internalizing user preferences rather than reproducing reference wording. Second, we introduce MIRROR-F, a focal plug-in that augments on-policy distributional alignment with selective supervision over informative reference tokens, thereby strengthening content generation while preserving user-specific expression. Across three personalized generation benchmarks, two model scales, and complementary reference-based and LLM-based evaluations, MIRROR and MIRROR-F achieve leading overall personalization performance and superior text quality, while exhibiting less catastrophic forgetting than SFT-based baselines on three unseen personalized generation tasks. The gains are consistent across model scales and application scenarios, translating to improved performance in LLM personalization tasks. 1 Introduction Large Language Models (LLMs) have exhibited exceptional capabilities across a wide range of domains (Team et al., 2026; Yang et al., 2025), leading to their widespread deployment in real-world applications such as personalized office assistant (Lyu et al., 2026; Li et al., 2025) and Private medical diagnosis (Zeng et al., 2026). Although various methods enhance the personalization capabilities of general-purpose LLMs through sophisticated harness designs (Liu et al., 2026b) and skill injection (Yang et al., 2026), these approaches largely rely on external mechanisms. From the perspective of the model intelligence, the core challenge remains to strengthen its intrinsic ability to infer latent user states, capture implicit preferences, and perform preference-aware reasoning (Amirizaniani et al., 2026; Cao et al., 2026; Yan et al., 2026). Mainstream approaches for LLM personalization can be broadly categorized into 3 categories: retrieval-based,fine-tuning-based and user-based methods. First, retrieval-based methods(as shown in Figure 1.b) incorporate relevant user histories into the input context to elicit preference-aware generation through in-context learning (Du et al., 2026; Zhang et al., 2025; Lee et al., 2026; Choi et al., 2026). However, their reliance on external contextual information leaves the model’s intrinsic preference-understanding capabilities largely unchanged and introduces potential privacy and security risks (Liang et al., 2025). Second, fine-tuning-based methods(as shown in Figure 1.c) employ parameter-efficient adaptation with preference-aware token- or sample-level supervision to learn user-specific linguistic patterns (Zhang et al., 2026a; Zhao et al., 2026b). While this method is effective at learning personalized styles and sentence structures, they significantly degrades the text professionalism, and can lead to catastrophic forgetting (Prashant et al., 2026) and poor out-of-distribution generalization (Karouzos et al., 2026; Liu et al., 2025). Third, user-based methods including user-specific LoRA adapter training (Tan et al., 2024; Tan et al., 2026; Hwang et al., 2026) ,attention-head pruning (Zhang et al., 2026b) and steering-vector injection (Liu et al., 2026a). However, these methods typically require user-specific parameters or dedicated architectures, leading to prohibitive training and storage costs as the user base scales (Zhao et al., 2026b), which hinders the deployment of personalized LLMs in real world applications. Figure 1: Overview of personalization paradigms. (a) A case study of the quality defects of existing methods. (b) Retrieval-based personalization. (c) SFT-based personalization. (d) MIRROR, which aligns a self-student with a self-teacher on the student’s own on-policy trajectories. To date, as shown in Figure 1.a,existing personalization approaches have been predominantly constrained by a style imitation paradigm, where personalization is primarily modeled as the ability to reproduce a user’s linguistic style, expression patterns, and textual distributions. However, this formulation overlooks a fundamental requirement of personalized generation: the generated content must not only reflect a user’s expressive preferences but also maintain high standards of factual accuracy, completeness, and practical utility while satisfying the user’s underlying needs. Therefore, this paper investigates the following central question: How can we enhance the modeling of user-specific expressive styles while maintaining the quality and effectiveness of personalized text generation? Recent advances in self-distillation (Zhao et al., 2026a; Shenfeld et al., 2026) introduce a paradigm beyond demonstration imitation. By leveraging hindsight information, models can internalize preference-relevant behavioral shifts through self-supervision along their own generation trajectories. In LLM personalizaton, user reference information can be viewed as a manifestation of the user’s preferences for stylistic sentence structures and their level of expertise, so we can identify user-provided references as a natural form of hindsight information for personalized generation. Based on this insight, we propose MIRROR (Meta-personalization by Internalizing Reference-Revealed On-policy Reflections), a reference-revealed self-distillation framework that shifts personalization from imitation toward internalization. MIRROR (shown in Figure 1.d) employs a single model as both a self-teacher and a self-student, the teacher accesses the user-provided reference to provide preference-aware guidance, while the student aligns with this guidance along its own on-policy trajectories instead of fitting the reference sequence token by token. Building upon this training paradigm, we further introduce MIRROR-F, a focal personalization framework that selectively concentrates optimization on informative tokens responsible for content quality. This focal mechanism enhances the expertise, completeness, and practical utility of generated content while preserving user-specific expressive styles. Controlled by a continuous focality coefficient. Extensive experiments on three benchmarks demonstrate that MIRROR achieves strong performance across multiple personalized generation tasks and MIRROR-F further achieves the best overall performance under comprehensive personalized generation evaluations, demonstrating superior content expertise, completeness, and textual quality while preserving user-specific expressive preferences. The main contribution of this work can be summarized as follows: • We introduce a new perspective by formulating personalized generation as reference-revealed self-distillation, moving beyond surface-level imitation toward user preference internalization. • We propose MIRROR, a self-distillation framework that transfers preference-induced behaviors via on-policy learning, achieving state-of-the-art performance across diverse personalized generation tasks. • We further propose MIRROR-F, which selectively enhances informative content generation through focal supervision, improving expertise, completeness, and textual quality while preserving user-specific expressions. • Extensive experiments demonstrate that MIRROR and MIRROR-F achieve superior personalization and generation quality across comprehensive personalized generation evaluations. 2 Preliminaries Personalized text generation. We consider the task of personalized text generation, where the goal is to adapt a language model πθ _θ so that its outputs match the preferences of an individual user. Each data sample is a triplet (x,h,y)(x,h,y): x is the query specifying the task and its content, h is the user’s historical text that implicitly reflects their preferences, and y=(y1,…,yT)y=(y_1,…,y_T) is the reference response actually written by the user. The collection of such samples over all users forms the dataset D, with (x,h,y)∈(x,h,y) . A non-personalized model produces y^=πθ(x) y= _θ(x), whereas a personalized model additionally conditions on the user history, y^=πθ(x,h) y= _θ(x,h). Our goal is to improve πθ _θ such that y y both reflects the user’s expressive preferences and remains accurate and complete as a response to x. Autoregressive factorization and contexts. We write πθ(⋅∣c,y<t)∈Δ|| _θ(· c,y_<t)∈ ^|V| for the next-token distribution over the vocabulary V at step t, given a context c and a prefix y<ty_<t. Two contexts are used throughout this paper. The deployable context cS=(h,x)c_S=(h,x) contains only what is genuinely available at inference time, and the reference-revealed context cT=(h,x,y)c_T=(h,x,y) additionally exposes the user-written reference. Since the reference is unavailable at deployment, a model conditioned on cTc_T can serve only as a source of supervision during training, never as a deployed policy. We use sg⁡[⋅]sg\! [· ] to denote the stop-gradient operator, so that π¯θ(⋅∣c,y<t):=sg[πθ(⋅∣c,y<t)] π_θ(· c,y_<t):=sg\! [ _θ(· c,y_<t) ] denotes a detached copy of the current model evaluated on context c. Supervised personalization objective. The standard way to personalize πθ _θ is to fine-tune it on the reference response under teacher forcing, minimizing the token-level negative log-likelihood ℒSFT(θ)=−1T∑t=1Tωtlogπθ(yt∣cS,y<t),L_SFT(θ)=- 1T _t=1^T _t _θ\! (y_t c_S,y_<t ), (1) where ωt _t is a token-level weight. Setting ωt=1 _t=1 for all t recovers plain supervised fine-tuning, while recent personalization methods instantiate ωt _t with estimates of how strongly token yty_t reflects the user’s preferences. Two properties of Equation 1 are central to this paper: the supervision target is the reference token yty_t itself, and it is evaluated on the reference prefix y<ty_<t rather than on a prefix the model would actually produce. On-policy rollouts. Given a deployable context cSc_S, we write y^∼πθ(⋅∣cS) y _θ(· c_S) for a response sampled from the current model, and refer to the resulting prefixes y^<t y_<t as on-policy states. These are the states the model visits at inference time, and they generally differ from the reference prefixes y<ty_<t used in Equation 1. Divergences between next-token distributions. Let P and Q be two distributions over V. We use the generalized Jensen–Shannon divergence with mixing coefficient β∈[0,1]β∈[0,1], JSDβ(P∥Q)=βDKL(P∥M)+(1−β)DKL(Q∥M),M=βP+(1−β)Q, _β\! (P\,\|\,Q )=β\,D_KL\! (P\,\|\,M )+(1-β)\,D_KL\! (Q\,\|\,M ), M=β P+(1-β)Q, (2) which measures the discrepancy between P and Q through their mixture distribution M. The coefficient β controls the relative contribution of the two KL-to-mixture terms, rather than recovering the forward or reverse KL divergence at the endpoints. In particular, β=0.5β=0.5 yields the standard symmetric Jensen–Shannon divergence and treats the two distributions equally. The divergence is finite and bounded for all P,QP,Q when β∈[0,1]β∈[0,1]; under natural logarithms, its upper bound is the binary entropy h⁡(β)h(β) and is at most log⁡2 2. Throughout our experiments, we fix β=0.5β=0.5 to avoid introducing an additional asymmetry between the reference-revealed teacher and the deployable student. We do not claim that β=0.5β=0.5 is optimal; rather, it is adopted as a neutral symmetric setting. A systematic study of alternative mixing coefficients is beyond the scope of this work. 3 Methodology Figure 2: Overview of MIRROR-F. A frozen reference-conditioned teacher (h,x,yh,x,y) supervises the trainable student (h,xh,x) on the student’s own rollouts via ℒMIRRORL_MIRROR, while ℒANCL_ANC anchors only the selected informative tokens of the reference y. We present MIRROR, which supervises a model with its own reference-conditioned predictions over states it visits itself (Section 3.1), and MIRROR-F, which adds a sparse anchoring term whose strength is exposed as a single coefficient (Section 3.2). 3.1 MIRROR: Internalizing Reference-Revealed Reflections Self-teacher and self-student. Rather than treating the reference response y as a string to be reproduced, we treat it as hindsight information: conditioning on y makes it substantially easier to judge which continuations suit this user. We therefore evaluate the same model under two contexts, the reference-revealed cT=(h,x,y)c_T=(h,x,y) and the deployable cS=(h,x)c_S=(h,x), on a shared prefix y^<t y_<t: ptT=π¯θ(⋅∣cT,y^<t),ptS=πθ(⋅∣cS,y^<t),p^T_t= π_θ\! (· c_T, y_<t ), p^S_t= _θ\! (· c_S, y_<t ), (3) where π¯θ=sg⁡[πθ] π_θ=sg\! [ _θ ] is the detached model. Since both distributions come from the same parameters and the same prefix, their difference isolates the behavioral shift induced by the reference alone; matching them transfers that shift, rather than the wording of y, into the model, without an external teacher or reward model. On-policy alignment. At each optimization step, a response prefix is sampled from the current student under stop-gradient sampling, y^∼sg[πθ(⋅∣cS)] y [ _θ(· c_S)], and the student is supervised on the states visited by this rollout. MIRROR optimizes the following stopped-gradient surrogate objective: ℒ~MIRROR(θ)=(x,h,y)∼y^∼sg[πθ(⋅∣cS)][1|y^|∑t=1|y^|JSDβ(ptT∥ptS)], L_MIRROR(θ)=E_(x,h,y) \,E_ y [ _θ(· c_S)] [ 1| y| _t=1^| y| _β\! (p^T_t\, \|\,p^S_t ) ], (4) with β=0.5β=0.5. The sampled prefixes and the teacher distribution are detached during backpropagation, so gradients are propagated only through the student distribution evaluated on the fixed on-policy states. We use a mixed divergence because personalization admits multiple valid realizations for each query: the symmetric objective transfers preference-relevant probability mass from the reference-revealed teacher while retaining support for alternative valid continuations. Why supervision must be on-policy. Equation 4 departs from the supervised objective of Equation 1 in two respects that jointly determine what the model learns. Its target is a distribution rather than a single reference token, so the model is taught how probability mass should be redistributed instead of which word to emit; and it is evaluated on prefixes the model itself produces, so training and inference operate over the same states. Reweighting ωt _t in Equation 1 alters only which reference tokens are emphasized and leaves both properties intact. What MIRROR acquires is therefore a preference-reasoning behavior that transfers to queries for which no reference exists—the only regime that matters at deployment. Check Appendix A.7 for more details. 3.2 MIRROR-F: Focal MIRROR with Selective Grounded Anchoring Selective grounded anchoring. Distributional alignment governs generation as a whole, whereas professional usability often depends on a small set of content-bearing tokens, such as technical terms, named entities, and quantities. MIRROR-F therefore applies an auxiliary anchoring loss only to selected reference positions. A reference word is selected when it is informative according to the deterministic rules in Appendix A.4 and grounded, meaning that its normalized form occurs as a complete word in the student context cSc_S. Selection is performed at the word level and expanded to all subword units of the selected word. Let selT_sel denote the resulting reference-token indices. The anchoring loss is ℒanc(θ)=−1|sel|∑t∈sellogπθ(yt∣cS,y<t),|sel|>0,0,|sel|=0,L_anc(θ)= cases- 1|T_sel| _t _sel _θ\! (y_t c_S,y_<t ),&|T_sel|>0,\\[10.0pt] 0,&|T_sel|=0, cases (5) where yty_t is the t-th reference token. Normalization by |sel||T_sel| prevents terminology-rich references from dominating the gradient, while the empty-set convention ensures that the objective reduces to MIRROR when no token is selected. Complete selection criteria are provided in Appendix A.4. Focality coefficient. As shown in Figure 2, because the two terms differ in scale and evolve at different rates, we divide each by a running average of its own magnitude, ℒ~k=ℒk/(mk+ϵ) L_k=L_k/(m_k+ε) with mk←γmk+(1−γ)sg[ℒk]m_k←γ m_k+(1-γ)sg\! [L_k ] for k∈MIRROR,anck∈\MIRROR,anc\, and combine them through one coefficient α∈[0,1]α∈[0,1]: ℒMIRROR-F(θ)=(1−α)ℒ~MIRROR(θ)+αℒ~anc(θ).L_MIRROR -F(θ)=(1-α)\, L_MIRROR(θ)+α\, L_anc(θ). (6) Equation 6 reduces exactly to MIRROR at α=0α=0, and by construction for any sample whose selected set is empty, so the objective is well defined for all α and all samples. Since personalized style is carried by the distributional term over the whole trajectory, α does not trade style against content: it sets how sharply supervision is focused on content-bearing positions on top of an already internalized behavior. Increasing α thus selects an operating point along the specialization axis, and moving along it requires no change to the objective, the data, or the pipeline. 4 Experiments 4.1 Experimental Settings Datasets. Experiments are conducted on three personalized generation tasks drawn from two widely used benchmarks for LLM personalization, LaMP (Salemi et al., 2024) and LongLaMP (Kumar et al., 2024). Specifically, Abstract Generation is taken from LongLaMP, while News Headline Generation (LaMP-4) and Paper Topic Generation (LaMP-5) are taken from LaMP. To ensure that training and evaluation operate over an identical context distribution, we performed context cropping and dataset filtering based on methods from existing papers (Zhang et al., 2026a; Zhao et al., 2026b). For evaluating out-of-distribution (OOD) performance, we further use three benchmarks in Amazon Reviews Dataset (Ni et al., 2019), and Music Review, which are not included during training and are reserved exclusively for assessing cross-task transfer and catastrophic forgetting. Baselines. Our methods are compared against three categories of baselines. (1) Base Model (Yang et al., 2025): the backbone LLM is prompted without any personalization and Thinking mode is disabled by default(check Appendix A.9 for thinking performance ablation). (2) Retrieval-based Methods: RAG (Lei et al., 2023), in which relevant user history is injected into the prompt without updating the model, LatestK (Liu et al., 2025) selects the most recent K entries by timestamp, and LLM TRSR (Zheng et al., 2024) compresses the history into a structured summary via recurrent summarization. (3) PEFT-based methods, in which the model is fine-tuned on user-specific data: SFT (Hu et al., 2021) is trained on history-augmented inputs, OPPU (Tan et al., 2024) learns a dedicated adapter per user, and PerCE (Zhang et al., 2026a) and NextQuill (Zhao et al., 2026b) additionally reweight the supervision according to token-level preference signals. PerCE and NextQuill are the two most recent state-of-the-art training-based methods and are therefore adopted as the primary points of comparison. Evaluation Metrics. Personalization performance is evaluated from two complementary perspectives. First,for textual similarity against the gold reference, three reference-based metrics are reported: ROUGE-1 (Lin, 2004), METEOR (Banerjee and Lavie, 2005), and BERTScore (Zhang et al., 2019), where BERTScore is computed with the roberta-large model. Second, for comprehensive personalization evaluation, we further adopt an LLM-as-judge protocol to assess aspects beyond surface overlap. Textual quality and professionalism are evaluated using two established frameworks: the reference-free G-Eval protocol (Liu et al., 2023) combined with the SummEval dimensions (Fabbri et al., 2021), under which each response is rated on Coherence, Consistency, Fluency, and Relevance; and the reference-based ExPerT framework (Salemi et al., 2025), which decomposes both the reference and the candidate into atomic content and style aspects and reports content- and style-level agreement. The judge model used in all LLM-as-judge evaluations is Qwen3-30B-A3B. For each reference-based evaluation and LLM-as-judge evaluation, we conduct three independent training runs with different random seeds and evaluate each result under the same protocol. We report the arithmetic mean across the three runs. The judge decoding temperature is fixed to 00 throughout to ensure reproducible and deterministic scoring for each model output. Implementation Details We use Qwen3-1.7B and Qwen3-4B as the backbone LLMs for training and evaluating all methods. For most model training procedures, we employ low-rank adaptation (LoRA) (Hu et al., 2021). During evaluation, vLLM (Kwon et al., 2023) is used to accelerate inference. Since different methods use different trained models, we select the best-performing configuration for training and evaluation. For the MIRROR-based methods, each Qwen3-1.7B model is trained for 200 steps, while each Qwen3-4B model is trained for 250 steps. For the main comparison between MIRROR and MIRROR-F, we set the focality coefficient α to 0.5 to ensure a consistent comparison. In the ablation study, we further evaluate α∈0.1,0.3,0.5,0.7,0.9α∈\0.1,0.3,0.5,0.7,0.9\ to investigate how the focal supervision in MIRROR-F affects personalized styles and syntactic patterns. Table 1: Main results of MIRROR and MIRROR-F on three personalized generation benchmarks Bold numbers denote the best performance, while underlined numbers denote the second best. Datasets Methods (→) Base Retrieval-based Methods PEFT-based Methods Ours Metrics (↓ ) Qwen3 RAG LatestK LLM-TRSR SFT OPPU PerCE NextQuill MIRROR MIRROR-F Qwen3-1.7B Abstract Generation ROUGE-1 0.3717 0.3811 0.3685 0.3758 0.3755 0.3082 0.3670 0.3698 0.3967 0.4015 METEOR 0.2125 0.2262 0.2145 0.2311 0.2365 0.2073 0.2240 0.2437 0.2506 0.2529 BERTScore 0.8577 0.8675 0.8657 0.8656 0.8668 0.8443 0.8621 0.8694 0.8587 0.8701 News Headline Generation ROUGE-1 0.1422 0.1495 0.1402 0.1387 0.1468 0.1559 0.1527 0.1594 0.1674 0.1763 METEOR 0.0976 0.1042 0.0956 0.0957 0.0922 0.1038 0.0976 0.1005 0.1020 0.1156 BERTScore 0.8465 0.8497 0.8475 0.8461 0.8544 0.8564 0.8559 0.8558 0.8521 0.8925 Paper Topic Generation ROUGE-1 0.3842 0.4052 0.3659 0.3888 0.4104 0.4073 0.4003 0.4095 0.4133 0.4368 METEOR 0.3247 0.3516 0.3143 0.3466 0.3383 0.3372 0.3309 0.3350 0.3554 0.3721 BERTScore 0.8845 0.8887 0.8841 0.8859 0.8923 0.8917 0.8912 0.8917 0.8894 0.8922 Qwen3-4B Abstract Generation ROUGE-1 0.3949 0.3984 0.3884 0.3869 0.4061 0.3697 0.3694 0.3844 0.4027 0.4195 METEOR 0.2575 0.2597 0.2496 0.2500 0.2475 0.2225 0.2301 0.2391 0.2659 0.2662 BERTScore 0.8689 0.8698 0.8680 0.8669 0.8717 0.8635 0.8565 0.8663 0.8730 0.8695 News Headline Generation ROUGE-1 0.1578 0.1517 0.1462 0.1454 0.1560 0.1593 0.1693 0.1593 0.1691 0.1747 METEOR 0.1105 0.1069 0.1014 0.1019 0.1031 0.1022 0.1130 0.1051 0.1142 0.1209 BERTScore 0.8544 0.8537 0.8519 0.8502 0.8561 0.8561 0.8598 0.8565 0.8567 0.8579 Paper Topic Generation ROUGE-1 0.4118 0.4282 0.4211 0.4224 0.4165 0.3975 0.4257 0.4515 0.4376 0.4486 METEOR 0.3954 0.3945 0.3824 0.3937 0.3452 0.3233 0.3548 0.3478 0.3869 0.3949 BERTScore 0.8917 0.8946 0.8903 0.8893 0.8032 0.8904 0.8942 0.8927 0.8917 0.8949 4.2 Experiment Results Reference-based Evaluation MIRROR and MIRROR-F achieve consistently strong performance on reference-based evaluation metrics, demonstrating that our method possesses superior capabilities for learning personalized styles and sentence structures. As shown in Table 1, the two methods outperform or remain highly competitive with retrieval-based and PEFT-based baselines across all three personalized generation tasks, two model backbones, and the ROUGE-1, METEOR, and BERTScore metrics. The advantage is particularly consistent on ROUGE-1 and METEOR, where MIRROR-F achieves the best results in most settings, including all three tasks with the Qwen3-1.7B backbone, while MIRROR also frequently ranks first or second. The two methods further maintain strong BERTScore performance, indicating that their gains are not limited to lexical overlap but also extend to semantic similarity. These results show that the proposed on-policy internalization paradigm provides a more reliable approach to reference-aligned personalization than conventional retrieval augmentation and token-level preference reweighting. LLM-based Evaluation MIRROR and MIRROR-F achieve leading performance under the LLM-based evaluation framework, showing that their advantages extend beyond reference-based similarity to broader aspects of personalized generation quality. Personalized generation is inherently underdetermined, as multiple responses can satisfy the same user preferences and task requirements. Therefore, metrics based on similarity to a single reference may fail to recognize valid alternatives and may not fully reflect content quality or stylistic alignment. As shown in Table 2, MIRROR-F achieves the highest Content-and-Style (C&S) score comparing to three trained methods across all three tasks, while MIRROR also consistently outperforms or remains competitive with SFT, PerCE, and NextQuill. MIRROR-F further obtains most of the best C&S scores among the trained methods on 3 benchmarks, indicating that its improvements are stable across tasks and model scales. The consistent gains in both content and style suggest that the proposed methods internalize preference-relevant information while preserving the content adequacy and stylistic characteristics required for high-quality personalized generation. Figure 3: Catastrophic forgetting on held-out OOD personalized generation tasks from the Amazon Review dataset. Performance degradation is measured relative to the corresponding base model under the same evaluation protocol; smaller values indicate better retention. Bold numbers denote the smallest degradation, while underlined numbers denote the second-smallest. OOD Performance MIRROR and MIRROR-F exhibit the strongest resistance to catastrophic forgetting on OOD personalized generation tasks, demonstrating superior generalization and scalability compared with existing SFT-based methods. As shown in Figure 3, conventional SFT methods, including PerCE and NextQuill, experience substantial performance degradation on unseen tasks relative to their in-distribution performance, indicating that their strong reliance on reference-specific token-level supervision can lead to over-specialization and impaired transferability. In contrast, MIRROR and MIRROR-F preserve performance much more effectively on the OOD test set and achieve the smallest degradation across the evaluated similarity metrics. MIRROR and MIRROR-F consistently outperform existing SFT-based methods across all four dimensions of the third-party text quality evaluation framework. As shown in Figure 4, both methods achieve higher scores on Relevance, Coherence, Consistency, and Fluency than SFT, PerCE, and NextQuill, demonstrating that their improvements extend beyond reference-based similarity to the intrinsic quality of generated responses. MIRROR-F generally obtains the strongest performance, benefiting from its focal supervision of informative and context-grounded content tokens. Table 2: ExPert evaluation of personalized generation quality on three benchmarks. The judge model scores each generated response for Content, Style, and their combination C&S (Content & Style); higher is better. Bold numbers denote the best performance among trained methods, while underlined numbers denote the second best among trained methods. Datasets (→) Abstract Generation News Headline Generation Paper Topic Generation Methods (↓ ) Content Style C&S Content Style C&S Content Style C&S Qwen3-1.7B Base 0.669 8.360 7.523 0.077 4.520 2.644 0.353 8.000 5.766 SFT 0.563 7.990 6.808 0.080 4.730 2.765 0.417 8.360 6.267 PerCE 0.492 7.810 6.365 0.068 4.810 2.745 0.413 8.100 6.116 NextQuill 0.542 8.000 6.708 0.078 4.670 2.725 0.392 8.140 6.028 MIRROR (Ours) 0.671 8.330 7.522 0.045 4.690 2.570 0.458 8.290 6.436 MIRROR-F (Ours) 0.685 8.390 7.620 0.095 4.890 2.920 0.483 8.440 6.633 Qwen3-4B Base 0.695 8.500 7.724 0.114 5.410 3.273 0.470 8.440 6.571 SFT 0.600 8.080 7.042 0.078 4.970 2.876 0.438 8.310 6.345 PerCE 0.536 7.960 6.661 0.091 4.890 2.898 0.397 8.190 6.078 NextQuill 0.563 8.090 6.859 0.086 4.930 2.896 0.385 8.100 5.972 MIRROR (Ours) 0.704 8.460 7.750 0.094 5.100 3.020 0.440 8.340 6.372 MIRROR-F (Ours) 0.714 8.520 7.832 0.098 5.270 3.125 0.480 8.350 6.573 Figure 4: Comparison of model performance scores based on the G-eval framework. Content Quality Performance Ablation Study The personalization performance of MIRROR-F remains stable across different values of the focality coefficient α. As shown in Figure 5, varying α from 0.10.1 to 0.90.9 does not lead to a substantial degradation in the evaluated personalization metrics, indicating that the model consistently preserves its ability to generate user-specific styles and sentence structures. This stability suggests that the focal supervision mainly adjusts the emphasis on informative and grounded content tokens without disrupting the underlying preference-relevant behavior learned by MIRROR. 5 Related Work LLM Personalization. LLM personalization has been extensively studied across text generation (Lai et al., 2026; Wang et al., 2024), conversational systems (Wang et al., 2025; Li et al., 2024), and multimodal applications (Shen et al., 2024). Recent benchmarks, including LaMP (Salemi et al., 2024) and LongLaMP (Kumar et al., 2024), have provided standardized frameworks for evaluating personalized text generation. Existing methods for LLM personalization can be broadly categorized into three groups: retrieval-based, fine-tuning-based, and user-centric approaches. Retrieval-based methods incorporate user histories into the context through retrieval or augmentation strategies (Lei et al., 2023; Du et al., 2026; Zhang et al., 2025; Liu et al., 2026b; Yang et al., 2026; Li et al., 2025; Chen et al., 2025), enabling training-free adaptation but relying on external user information without enhancing the model’s internal preference understanding. PEFT -based methods adapt model parameters using user-specific data, with recent approaches introducing preference-aware supervision reweighting (Zhang et al., 2026a; Zhao et al., 2026b). Although effective under reference-based similarity metrics, they mainly optimize toward reproducing user-specific expressions and suffer from over-adaptation. User-centric methods explicitly model users through dedicated adapters(Tan et al., 2024; Hwang et al., 2026), embeddings, or internal interventions (Liu et al., 2025; Zhang et al., 2026b; Wang et al., 2026), but their scalability is limited because of high costs. Existing methods prioritize reference imitation or external preference conditioning, often under-optimizing the intrinsic quality of generated content. In contrast MIRROR introduces a reference-revealed self-distillation training paradigm for personalized generation, while MIRROR-F further focuses optimization on informative content, enhancing the professionalism and overall textual quality of personalized generation. LLM Self-Distillation. A recent line of work removes the need for an external, stronger teacher by letting a model conditioned on hindsight information supervise its own weaker self: the teacher sees privileged signals—environment feedback, demonstrations, or verified reasoning traces—while the student sees only the query, and training matches their per-token distributions over the student’s own rollouts (Hübotter et al., 2026; Hwang et al., 2026; Shenfeld et al., 2026). Follow-up work extends this paradigm to new settings and objectives, including long-trace compaction (Kim and Lee, 2026), cross-lingual and long-context reasoning, privacy-constrained training (Zhang et al., 2026c), UI generation (Dong, 2026)and policy-gradient reformulations (Liu et al., 2026c). MIRROR apply on-policy self-distillation to LLM personalization. We further propose a focal plug-in(MIRROR-F) mechanism for personalized generation, which improves the professionalism and textual quality of generated content without compromising personalization performance. Figure 5: The impact of fluctuations in the value of α on the performance of the model’s stylistic and syntactic personalization assessment. 6 Conclusion We introduced MIRROR, an on-policy self-distillation framework that shifts LLM personalization from reference imitation toward preference internalization. We further proposed MIRROR-F, which selectively enhances informative content while preserving user-specific expression. Across multiple tasks, model scales, and evaluation protocols, both methods achieve strong personalization, improved text quality, and less catastrophic forgetting than existing SFT-based approaches, demonstrating the effectiveness of on-policy preference internalization for robust and transferable personalized generation. Reproducibility statement We provide detailed implementation and experimental settings, hyperparameters, training configurations in appendix A.2. AI use statement Generative AI tools were used to assist with code debugging, LaTeX formatting, writing and polishing portions of the manuscript, and literature search for identifying potentially relevant prior work. All AI-assisted outputs, including text, code suggestions, and literature recommendations, were reviewed and verified by the authors before being incorporated into the final work. Generative AI tools were not used to formulate the main research hypotheses, design the proposed methodology or experiments, derive mathematical claims or proofs, generate synthetic data, or interpret the reported experimental results. The authors take full responsibility for the accuracy, validity, and final content of the paper. References Amirizaniani et al. (2026) M. Amirizaniani, B. C. G. Lee, J. West, and N. Weber Training llms with reinforcement learning for intent-aware personalized question answering. arXiv preprint arXiv:2605.12645. Cited by: §1. Banerjee and Lavie (2005) S. Banerjee and A. Lavie METEOR: an automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, p. 65–72. Cited by: §4.1. Cao et al. (2026) H. Cao, F. Zhang, J. Yao, Y. Zheng, C. Zhao, L. Hao, Y. Wei, W. Ni, H. Fu, Y. Sun, et al. Beyond retrieval: learning compact user representations for scalable llm personalization. arXiv preprint arXiv:2606.04547. Cited by: §1. Chen et al. (2025) Y. Chen, X. Liu, R. Wang, Z. Li, P. Chen, C. Yu, Q. Yin, P. Nigam, M. Jiang, and B. Yin POPI: personalizing llms via optimized natural language preference inference. arXiv preprint arXiv:2510.17881. Cited by: §5. Choi et al. (2026) W. Choi, Y. Kim, Y. Lee, and S. Yoon PGMem: tightly coupled persona-memory graph for lifelong personalized agents. arXiv preprint arXiv:2608.01708. Cited by: §1. Dong (2026) H. Dong Self-distillation policy optimization via visual feedback: bridging code and visual artifacts. arXiv preprint arXiv:2606.10334. Cited by: §5. Du et al. (2026) L. Du, Y. Yuan, Z. Zhao, F. Lyu, E. Penaloza, X. Chen, Z. Sun, J. Kang, L. Charlin, X. Liu, et al. Optimizing user profiles via contextual bandits for retrieval-augmented llm personalization. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 31803–31817. Cited by: §1, §5. Fabbri et al. (2021) A. R. Fabbri, W. Kryściński, B. McCann, C. Xiong, R. Socher, and D. Radev Summeval: re-evaluating summarization evaluation. Transactions of the Association for Computational Linguistics 9, p. 391–409. Cited by: §4.1. Hu et al. (2021) E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen Lora: low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. Cited by: §4.1, §4.1. Hübotter et al. (2026) J. Hübotter, F. Lübeck, L. Behric, A. Baumann, M. Bagatella, D. Marta, I. Hakimi, I. Shenfeld, T. K. Buening, C. Guestrin, et al. Reinforcement learning via self-distillation. arXiv preprint arXiv:2601.20802. Cited by: §5. Hwang et al. (2026) E. Hwang, K. Mitra, D. Zhang, H. Kim, and E. Hruschka Hypotheses-guided self distillation for continual personalization. arXiv preprint arXiv:2609.00251. Cited by: §1, §5, §5. Karouzos et al. (2026) C. Karouzos, X. Tan, and N. Aletras An empirical study on preference tuning generalization and diversity under domain shift. arXiv preprint arXiv:2601.05882. Cited by: §1. Kim and Lee (2026) J. Kim and D. Lee OPSD compresses what rlvr teaches: a post-rl compaction stage for reasoning models. arXiv preprint arXiv:2605.06188. Cited by: §5. Kumar et al. (2024) I. Kumar, S. Viswanathan, S. Yerra, A. Salemi, R. A. Rossi, F. Dernoncourt, H. Deilamsalehy, X. Chen, R. Zhang, S. Agarwal, et al. Longlamp: a benchmark for personalized long-form text generation. arXiv preprint arXiv:2407.11016. Cited by: §4.1, §5. Kwon et al. (2023) W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, p. 611–626. Cited by: §4.1. Lai et al. (2026) H. Lai, S. Song, S. Niu, H. Wang, J. Yang, Z. Wang, Z. Yin, and X. Liang RoleCDE: benchmarking and mitigating role–alignment trade-offs in role-playing agents. In Findings of the Association for Computational Linguistics: ACL 2026, p. 2226–2248. Cited by: §5. Lee et al. (2026) C. Lee, J. Kim, and T. Gong From volume to value: preference-aligned memory construction for on-device rag. arXiv preprint arXiv:2605.18271. Cited by: §1. Lei et al. (2023) Y. Lei, L. Ding, Y. Cao, C. Zan, A. Yates, and D. Tao Unsupervised dense retrieval with relevance-aware contrastive pre-training. In Findings of the Association for Computational Linguistics: ACL 2023, p. 10932–10940. Cited by: §4.1, §5. Li et al. (2024) X. Li, R. Zhou, Z. C. Lipton, and L. Leqi Personalized language modeling from personalized human feedback. arXiv preprint arXiv:2402.05133. Cited by: §5. Li et al. (2025) Z. Li, C. Xi, C. Li, D. Chen, B. Chen, S. Song, S. Niu, H. Wang, J. Yang, C. Tang, et al. Memos: a memory os for ai system. arXiv preprint arXiv:2507.03724. Cited by: §1, §5. Liang et al. (2025) X. Liang, S. Niu, Z. Li, S. Zhang, H. Wang, F. Xiong, Z. Fan, B. Tang, J. Zhao, J. Yang, et al. Saferag: benchmarking security in retrieval-augmented generation of large language model. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 4609–4631. Cited by: §1. Lin (2004) C. Lin Rouge: a package for automatic evaluation of summaries. In Text summarization branches out, p. 74–81. Cited by: §4.1. Liu et al. (2026a) J. Liu, W. Yu, Q. Dai, Z. Li, J. Zhu, M. Yang, T. Chua, and I. King Perfit: exploring personalization shifts in representation space of llms. In The Fourteenth International Conference on Learning Representations, Cited by: §1. Liu et al. (2025) J. Liu, Y. Zhu, S. Wang, X. Wei, E. Min, Y. Lu, S. Wang, D. Yin, and Z. Dou Llms+ persona-plug= personalized llms. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 9373–9385. Cited by: §1, §4.1, §5. Liu et al. (2026b) J. Liu, Y. Sun, W. Cheng, H. Lei, Y. Li, Y. Chen, and D. Wang Hierarchical memory orchestration for personalized persistent agents. arXiv preprint arXiv:2604.01670. Cited by: §1, §5. Liu et al. (2023) Y. Liu, D. Iter, Y. Xu, S. Wang, R. Xu, and C. Zhu G-eval: nlg evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 conference on empirical methods in natural language processing, p. 2511–2522. Cited by: §4.1. Liu et al. (2026c) Y. Liu, S. Zhang, Y. Zhang, and Q. Gu Self-distilled policy gradient. arXiv preprint arXiv:2606.04036. Cited by: §5. Lyu et al. (2026) Y. Lyu, G. Chen, R. Shao, W. Guan, and L. Nie Personalalign: hierarchical implicit intent alignment for personalized gui agent with long-term user-centric records. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 36074–36089. Cited by: §1. Ni et al. (2019) J. Ni, J. Li, and J. McAuley Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), p. 188–197. Cited by: §4.1. Prashant et al. (2026) P. P. Prashant, J. Zhu, A. Creo, and B. Salimi Fine-tuning without forgetting via loss-adaptive learning rates. arXiv preprint arXiv:2605.20005. Cited by: §1. Salemi et al. (2025) A. Salemi, J. Killingback, and H. Zamani Expert: effective and explainable evaluation of personalized long-form text generation. In Findings of the Association for Computational Linguistics: ACL 2025, p. 17516–17532. Cited by: §4.1. Salemi et al. (2024) A. Salemi, S. Mysore, M. Bendersky, and H. Zamani Lamp: when large language models meet personalization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 7370–7392. Cited by: §4.1, §5. Shen et al. (2024) X. Shen, R. Zhang, X. Zhao, J. Zhu, and X. Xiao Pmg: personalized multimodal generation with large language models. In Proceedings of the ACM Web Conference 2024, p. 3833–3843. Cited by: §5. Shenfeld et al. (2026) I. Shenfeld, M. Damani, J. Hübotter, and P. Agrawal Self-distillation enables continual learning. arXiv preprint arXiv:2601.19897. Cited by: §1, §5. Tan et al. (2024) Z. Tan, Q. Zeng, Y. Tian, Z. Liu, B. Yin, and M. Jiang Democratizing large language models via personalized parameter-efficient fine-tuning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 6476–6491. Cited by: §1, §4.1, §5. Tan et al. (2026) Z. Tan, Z. Zhang, H. Wen, Z. Li, R. Zhang, P. Chen, F. Mo, Z. Liu, Q. Zeng, Q. Yin, et al. Instant personalized large language model adaptation via hypernetwork. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 23557–23580. Cited by: §1. Team et al. (2026) K. Team, T. Bai, Y. Bai, Y. Bao, J. Cai, X. Cai, P. Cao, Y. Cao, Z. Chai, Y. Charles, et al. Kimi k3: open frontier intelligence. arXiv preprint arXiv:2607.24653. Cited by: §1. Wang et al. (2026) B. Wang, Y. Du, Y. Pei, and Z. Wang From explicit to implicit: a theoretical framework and transfer method for preference internalization in language models. TRANSACTIONS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS 14, p. 1074–1095. Cited by: §5. Wang et al. (2024) N. Wang, Z. Peng, H. Que, J. Liu, W. Zhou, Y. Wu, H. Guo, R. Gan, Z. Ni, J. Yang, et al. Rolellm: benchmarking, eliciting, and enhancing role-playing abilities of large language models. In Findings of the Association for Computational Linguistics: ACL 2024, p. 14743–14777. Cited by: §5. Wang et al. (2025) X. Wang, H. Wang, Y. Zhang, X. Yuan, R. Xu, J. Huang, S. Yuan, H. Guo, J. Chen, S. Zhou, et al. CoSER: coordinating llm-based persona simulation of established roles.. In ICML, Cited by: §5. Yan et al. (2026) Y. Yan, S. Wu, and T. Le Locating and controlling implicit personalization in large language models. arXiv preprint arXiv:2608.11735. Cited by: §1. Yang et al. (2025) A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §1, §4.1. Yang et al. (2026) Y. Yang, J. Li, Q. Pan, B. Zhan, Y. Cai, L. Du, J. Zhou, K. Chen, Q. Chen, X. Li, et al. Autoskill: experience-driven lifelong learning via skill self-evolution. arXiv preprint arXiv:2603.01145. Cited by: §1, §5. Zeng et al. (2026) L. Zeng, S. Zhang, and X. Zhang EviDx: evidence-aware active diagnosis with scaffolded llm agents. arXiv preprint arXiv:2608.24570. Cited by: §1. Zhang et al. (2026a) C. Zhang, Y. Lu, L. Fang, C. Zheng, J. Chai, X. Wang, G. Yin, W. Lin, Y. Wang, and Z. Lin Rethinking personalization in large language models at the token level. arXiv preprint arXiv:2603.06595. Cited by: §1, §4.1, §4.1, §5. Zhang et al. (2019) T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi Bertscore: evaluating text generation with bert. arXiv preprint arXiv:1904.09675. Cited by: §4.1. Zhang et al. (2026b) W. Zhang, Y. Yuan, C. Han, Y. Tian, Z. Sun, L. Du, J. Kang, H. Kang, X. Liu, and H. Wu Preference heads in large language models: a mechanistic framework for interpretable personalization. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 47742–47754. Cited by: §1, §5. Zhang et al. (2025) W. Zhang, X. Zhang, C. Zhang, L. Yang, J. Shang, Z. Wei, H. P. Zou, Z. Huang, Z. Wang, Y. Gao, et al. Personaagent: when large language model agents meet personalization at test time. Cited by: §1, §5. Zhang et al. (2026c) X. Zhang, Z. Ding, T. Pan, R. Yang, C. Kang, X. Xiong, and J. Gu Opsdl: on-policy self-distillation for long-context language models. arXiv preprint arXiv:2604.17535. Cited by: §5. Zhao et al. (2026a) S. Zhao, Z. Xie, M. Liu, J. Huang, G. Pang, F. Chen, and A. Grover Self-distilled reasoner: on-policy self-distillation for large language models. arXiv preprint arXiv:2601.18734. Cited by: §1. Zhao et al. (2026b) X. Zhao, J. Jun, Y. Zhang, W. Wang, H. Cheng, F. Feng, S. Ng, and T. Chua Nextquill: causal preference modeling for enhancing llm personalization. In International Conference on Learning Representations, Vol. 2026, p. 118745–118770. Cited by: §1, §4.1, §4.1, §5. Zheng et al. (2024) Z. Zheng, W. Chao, Z. Qiu, H. Zhu, and H. Xiong Harnessing large language models for text-rich sequential recommendation. In Proceedings of the ACM Web Conference 2024, p. 3207–3216. Cited by: §4.1. Appendix A Appendix A.1 More Case Study To provide a more detailed qualitative analysis, we present one complete question–answer case for each personalization task. Each case includes the task input, the gold answer, the responses generated by different methods, and a task-specific comparison. The selected examples are drawn from the evaluation set and are intended to illustrate how MIRROR and MIRROR-F balance reference alignment, user-specific expression, and intrinsic content quality. A.1.1 Abstract Generation Question. The input asks the model to generate an abstract for the title “Learning the Language: The Importance of Studying Written Directions in Designing Navigational Technologies for the Blind.” The user-provided key information consists of five aspects: independent navigation, text-to-speech directions, language and cues, qualitative analysis, and landmarks and alternate routes. Analysis. The responses exhibit clear differences in both content fidelity and unsupported detail. The base model covers most of the requested concepts, including text-to-speech directions, language, landmarks, and alternate routes, but its discussion remains relatively generic. NextQuill produces a fluent abstract and explicitly mentions landmarks and alternate routes. However, it introduces the unsupported detail that the study involved “11 blind participants” and adds a preference for a conversational tone, neither of which is included in the task input or the gold answer. Table 3: Question–answer case for Abstract Generation. Item Content Question Generate an abstract for the title “Learning the Language: The Importance of Studying Written Directions in Designing Navigational Technologies for the Blind” using the following items: (1) Independent navigation; (2) Text-to-Speech directions; (3) Language and cues; (4) Qualitative analysis; and (5) Landmarks and alternate routes. Gold answer Independent navigation is important to individuals who are blind and visually impaired (VI). Researchers have long explored how blind and VI people navigate to inform the design of more useful, accessible wayfinding devices. However, there has been little research on the role language plays in providing effective text-to-speech directions for this population. Through qualitative analysis, this work examines the language and cues expressed in written navigational directions, including how distances are represented, how direction is indicated, and what landmarks are referenced. The analysis further identifies warnings about overshooting a destination, alternative routes that are easier to navigate, and the accessibility of destinations for people with disabilities. Table 4: Different model answers for the Abstract Generation case. Method Answer Base This paper explores the importance of studying written directions in the design of navigational technologies for the blind, focusing on how independent navigation can be enhanced through effective use of text-to-speech directions. We conducted a qualitative analysis of how language and cues are used in written directions to guide users, emphasizing the need for clear, concise, and contextually relevant information. The study also examines the role of landmarks and alternate routes in providing users with multiple options for navigation, ensuring that they can make informed decisions in complex environments. Our findings highlight the significance of incorporating user-centered design principles that consider the unique needs and experiences of blind individuals when developing navigational technologies. NextQuill Independent navigation is a challenge for many people who are blind. One promising solution is to use text-to-speech directions to guide them. However, there has been little inquiry into how people who are blind use and interpret directions, and how technologies can be designed to better support them. We conducted a qualitative analysis of 11 blind participants’ experiences with and suggestions for text-to-speech directions. We found that users rely on language and cues to interpret directions, and that they had strong preferences for how these directions should be presented. We present two findings that can be used to guide the design of future text-to-speech directions: (1) the use of landmarks and alternate routes can be beneficial to users, and (2) users prefer directions that are presented in a conversational tone. MIRROR Independent navigation is a critical skill for individuals who are blind, enabling them to move confidently and safely in their environment. However, the effectiveness of navigational technologies for the blind often hinges on the clarity and usability of the directions provided. This paper explores the importance of studying written directions in the design of navigational technologies for the blind, focusing on how language and cues influence user experience. Through qualitative analysis of user interactions, we examine how text-to-speech directions are perceived and understood by users, highlighting the role of language in conveying information about landmarks and alternate routes. Our findings reveal that the way directions are structured and the cues used can significantly impact the user’s ability to navigate independently. We argue that a deeper understanding of language and cues in written directions is essential for designing more effective navigational technologies that support the unique needs of blind users. MIRROR-F Independent navigation for the blind is significantly influenced by the clarity and effectiveness of navigational technologies. This study emphasizes the importance of studying written directions in the design of navigational technologies. Through qualitative analysis, we explore how text-to-speech directions, language, and cues impact the user’s ability to navigate independently. The study highlights the role of language in conveying information and the necessity of clear, concise cues that aid in understanding the environment. We examine how landmarks and alternate routes are described and how these elements can be integrated into navigational systems to enhance usability. The findings underscore the need for a deep understanding of language in the design process, ensuring that navigational technologies are not only functional but also intuitive and accessible for users with visual impairments. MIRROR retains the main research focus while avoiding the unsupported participant count. It connects independent navigation with the clarity of directions and explicitly relates language and cues to landmarks and alternative routes. This indicates that the model is not merely reproducing isolated reference phrases, but is reconstructing the central research argument. MIRROR-F further strengthens the content-oriented organization. It explicitly distinguishes text-to-speech directions, language, cues, landmarks, and alternate routes, and connects these elements to the usability and accessibility of navigation systems. The output is also more concise and avoids the speculative participant-level details introduced by NextQuill. The case therefore demonstrates the ability of MIRROR-F to improve content focus while preserving the central semantic structure of the personalized reference. A.1.2 News Headline Generation Question. The input asks the model to generate a headline for a fashion article describing H&M’s updated classics and the appeal of clothing for readers whose “leggy limbs” have not appeared in fashion magazines. Table 5: Question–answer case for News Headline Generation. Item Content Question Generate a headline for the following article: “Um, OK. For those of us whose leggy limbs haven’t been immortalized in print, we can turn to H&M’s updated classics, like …” Gold answer H&M Model Collection Inspired By Joan Smalls, Daphne Groeneveld & More (PHOTOS) Table 6: Different model answers for the News Headline Generation case. Method Answer PerCE 7 Ways To Make Your Pantyhose Look Like A Model (PHOTOS) NextQuill Jenna Holmes’ ’90s Style Is Too Retro For A 2013 Cover (PHOTOS) MIRROR H&M’s Updated Classics: Leggy Limbs For Every Body Type (PHOTOS) MIRROR-F H&M’s Updated Classics: Leggy Limbs For Every Style (PHOTOS) Analysis. This case highlights the difference between topical relevance and personalized headline generation. PerCE produces a fluent fashion headline, but it changes the article topic from an H&M collection to general advice about pantyhose. NextQuill preserves the magazine-style “(PHOTOS)” suffix, but introduces a different person, a different time period, and a different editorial topic. Both outputs therefore reproduce the broad fashion domain without accurately identifying the article-specific subject. MIRROR correctly retains “H&M” and “updated classics,” which are the most important topical cues in the article. It also preserves the compact tabloid-style structure and the “(PHOTOS)” suffix. The phrase “leggy limbs” further maintains the distinctive expressive pattern of the input. MIRROR-F retains all of these properties while replacing the narrower “body type” expression with the more general “style.” This produces a headline that remains faithful to the fashion context and the user’s preferred editorial format while being less restrictive in its description of the target audience. The example demonstrates that the proposed methods can preserve both the publication-specific form and the article-specific content, whereas the SFT-based baselines tend to drift toward memorized or superficially related fashion expressions. A.1.3 Paper Topic Generation Question. The input asks the model to generate a title for a paper on converting discrete-time single-input single-output piecewise affine state-space models into equivalent input-output representations. The abstract emphasizes necessary and sufficient conditions, a constructive conversion procedure, the growth in the number of modes and parameters, and numerical examples. Table 7: Question–answer case for Paper Topic Generation. Item Content Question Generate a title for the following abstract: “This paper addresses the conversion of discrete-time single input-single output PieceWise Affine (PWA) models from state space to input-output form. Necessary and sufficient conditions are given for a PWA state space model to admit equivalent input-output representations. When an equivalent input-output model exists, a constructive procedure is presented to derive both its parameters and the partition of the regressors domain. It is shown that the number of modes and the number of parameters may grow considerably when converting a PWA state space model into an equivalent input-output representation. Numerical examples highlight the role of the derived necessary and sufficient conditions for input-output realization of PWA state space models.” Gold answer Input-Output Realization Of Piecewise Affine State Space Models Table 8: Different model answers for the Paper Topic Generation case. Method Answer PerCE On the Input-Output Representation of Piecewise Affine State Space Models NextQuill Input-output realization of piecewise affine state space models MIRROR Conversion of Discrete-Time Piecewise Affine Models to Input-Output Form MIRROR-F Conversion of Piecewise Affine State Space Models to Input-Output Form Analysis. All methods identify the main technical topic, but they differ in how they balance semantic coverage, readability, and title style. PerCE produces a valid title, although “representation” is less specific than the “realization” terminology used in the reference. NextQuill closely follows the gold title and provides a concise answer, but it mainly reproduces the surface form of the reference. MIRROR reformulates the title using the operation “conversion,” which directly reflects the central procedure described in the abstract. It also preserves the important pair of technical concepts, namely piecewise affine models and input-output form. MIRROR-F produces the most compact formulation among the two proposed methods while retaining the essential technical relation. It removes the unnecessary “discrete-time” qualifier for a title-level summary but preserves the central transformation from state-space models to input-output form. The output is therefore concise, technically informative, and consistent with the scholarly title style expressed in the user’s history. This example illustrates how MIRROR-F can focus supervision on content-bearing terminology without forcing the model to copy the reference title verbatim. Overall observation. The three cases reveal a consistent distinction between reference imitation and preference internalization. In Abstract Generation, MIRROR and MIRROR-F avoid unsupported details while preserving the research motivation and findings. In News Headline Generation, they maintain the article-specific topic and the user’s preferred headline format instead of producing broadly related fashion headlines. In Paper Topic Generation, they preserve the central technical relation while generating concise titles with appropriate scholarly phrasing. MIRROR-F further strengthens the content-bearing portions of the outputs, especially when the task requires the model to identify informative entities, technical concepts, or article-specific themes. These qualitative examples support the view that on-policy reference-revealed self-distillation can improve personalized generation without reducing the factual and practical quality of the generated text. A.2 Training Hyperparameters MIRROR and MIRROR-F were trained under the same experimental configuration for a controlled comparison. Unless otherwise stated, both methods used the same backbone, training data, context construction procedure, optimizer, learning rate, batch size, training steps, rollout strategy, and random seed. The only difference was the training objective: MIRROR used on-policy distributional alignment alone, whereas MIRROR-F additionally applied the selective grounded anchoring loss. The shared configuration used for the comparison is summarized in Table 9. Table 9: Training hyperparameters used for the MIRROR and MIRROR-F comparison. Category Hyperparameter Value Model and data Backbone Qwen3-1.7B/4B Dataset LongLaMP,LaMP-4,LaMP-5 Optimization Optimizer AdamW Learning rate 1×10−41× 10^-4 Per-device batch size 1 Gradient accumulation 1 Warm-up ratio 0.05 Maximum gradient norm 1.0 LoRA Rank r 16 Scaling factor αLoRA _LoRA 32 Dropout 0.05 Trainable parameters LoRA parameters only On-policy rollout Maximum training steps 200 (Qwen3-1.7B) / 250 (Qwen3-4B) Sampling temperature 1.0 Top-p 1.0 Rollout length ratio 1.3× reference length Minimum / maximum rollout length 64 / 400 tokens MIRROR Divergence Generalized JSD Mixing coefficient β 0.5 Anchoring loss Not used MIRROR-F Divergence Generalized JSD Mixing coefficient β 0.5 Focal coefficient α 0.5 Reproducibility Random seed 456166,9874,1123 Thinking mode Disabled For both methods, the student generated on-policy rollouts using the same sampling configuration, and the teacher used the same reference-revealed context. The user history, task questions, context truncation strategy, and reference-answer budget were also kept identical. Therefore, the comparison was conducted under one shared training setup, with the selective grounded anchoring term being the only method-specific addition in MIRROR-F. A.3 Rule-Based Information Completeness Analysis To complement the LLM-based quality evaluation, we additionally measure information completeness using a deterministic, rule-based information recall metric. This metric is designed to quantify whether a model preserves specific information-bearing words from the reference output, rather than whether the generated text is generally fluent or semantically coherent. Therefore, it should be interpreted as a targeted content-coverage measure and is complementary to the G-Eval professional-quality scores. For each example i, let gig_i denote the gold reference, cic_i denote the student input context, and yi,my_i,m denote the output generated by model m. We first extract word-like units using the regular expression ⁡(x)=w∣w matches [A-Za-z0-9][A-Za-z0-9’\-]∗ in x.W(x)= \w w matches [ A-Za-z0-9][ A-Za-z0-9' -]^* in x \. This extractor treats sequences containing letters, digits, apostrophes, and hyphens as word-level units and avoids substring matching. For a word w, we define its alphanumeric core as core⁡(w)=filteralnum⁡(w),core(w)=filter_alnum(w), and use lowercase forms for case-insensitive matching. Following the focal-token selection rules used by MIRROR-F, a word is considered informative if it contains explicit lexical signals of information-bearing content. Formally, Informative⁡(w)=⁡[Digit⁡(w)∨InnerUpper⁡(w)∨HeadUpper⁡(w)],Informative(w)=I [Digit(w) (w) (w) ], where Digit(w)=[∃j:core(w)j∈0,…,9],Digit(w)=I [∃ j:core(w)_j∈\0,…,9\ ], InnerUpper(w)=[∃j≥2:core(w)j is uppercase],InnerUpper(w)=I [∃ j≥ 2:core(w)_j is uppercase ], and HeadUpper(w)=[|core(w)|≥3∧core(w)1 is uppercase∧lower(core(w))∉stop].HeadUpper(w)=I [|core(w)|≥ 3 (w)_1 is uppercase (core(w)) _stop ]. Here, stopS_stop is the fixed stopword set used by the training-time focal selector. Words whose alphanumeric core has fewer than two characters are not treated as informative. We further require a candidate reference word to be grounded in the available student context. Let i=lower⁡(w)∣w∈⁡(ci)C_i= \lower(w) w (c_i) \ denote the set of lowercase word units extracted from the input context. Then the grounding condition is Grounded(w,ci)=[lower(core(w))∈i].Grounded(w,c_i)=I [lower(core(w)) _i ]. Thus, a reference word is counted only when it is both informative and explicitly recoverable as a complete word from the student input. In particular, the criterion does not count arbitrary substring matches. The target set of information-bearing words for example i is therefore defined as i=lower(core(w))|w∈(gi),Informative(w)=1,Grounded(w,ci)=1.S_i= \lower(core(w))\; |\;w (g_i),\;Informative(w)=1,\;Grounded(w,c_i)=1 \. The corresponding output set for model m is i,m=lower⁡(w)∣w∈⁡(yi,m).O_i,m= \lower(w) w (y_i,m) \. Both sets are implemented as sets rather than multisets. Consequently, repeating the same word multiple times does not increase the score. For examples with a non-empty target set, the information recall is computed as Ri,m=|i∩i,m||i|.R_i,m= |S_i _i,m | |S_i |. Examples for which |i|=0 |S_i |=0 are excluded because they contain no eligible grounded information-bearing word and therefore do not define a meaningful recall denominator. Let ℐm,t=i∈t∣|i|>0I_m,t= \i _t |S_i|>0 \ be the valid examples for model m on task t. The reported information-completeness score is the arithmetic mean of the per-example recalls: ICm,t=1|ℐm,t|∑i∈ℐm,tRi,m.IC_m,t= 1|I_m,t| _i _m,tR_i,m. This aggregation first computes recall independently for each example and then averages across examples; it is not obtained by pooling all words from the dataset into a single global numerator and denominator. Table 10 reports the resulting scores. In the table, OPD corresponds to MIRROR, while OPD-sftel corresponds to MIRROR-F. Bold numbers indicate the best performance among trained methods, and underlined numbers indicate the second-best trained method. The Base model is included as a non-personalized reference but is excluded from the ranking. Table 10: Rule-based information completeness evaluation. Higher values indicate better coverage of information-bearing content. Bold numbers denote the best performance among trained methods, while underlined numbers denote the second-best. The Base model is reported as a non-personalized reference and is excluded from the ranking. Backbone Task Base ContextSFT PerCE NextQuill MIRROR(Ours) MIRROR-F(Ours) Qwen3-1.7B Abstract 0.933 0.895 0.785 0.839 0.949 0.952 News 0.774 0.447 0.404 0.358 0.387 0.631 Paper 0.544 0.566 0.527 0.521 0.588 0.596 Qwen3-4B Abstract 0.951 0.919 0.854 0.819 0.957 0.963 News 0.686 0.458 0.416 0.461 0.676 0.640 Paper 0.645 0.564 0.557 0.543 0.615 0.624 Several observations emerge from Table 10. First, MIRROR-F achieves the highest information-completeness score in five of the six backbone–task combinations. Its advantage is particularly pronounced on Qwen3-1.7B News, where it reaches 0.6310.631, compared with 0.4470.447 for ContextSFT and 0.3870.387 for MIRROR. This result suggests that selective grounded anchoring helps preserve concrete information-bearing content in a setting where generic supervised adaptation substantially reduces coverage. Second, MIRROR and MIRROR-F consistently outperform the other trained personalization methods on the Abstract and Paper Topic tasks. On Abstract, the two methods achieve 0.9490.949 and 0.9520.952 for Qwen3-1.7B and 0.9570.957 and 0.9630.963 for Qwen3-4B, respectively. The same pattern holds on Paper Topic, where MIRROR-F obtains 0.5960.596 and 0.6240.624 on the two backbones. These results are consistent with the intended role of selective anchoring: the model is encouraged to retain grounded, information-bearing content while the reference-revealed on-policy objective provides broader personalization guidance. The only exception is Qwen3-4B News, where MIRROR obtains 0.6760.676 and outperforms MIRROR-F at 0.6400.640. This difference indicates that selective anchoring is not uniformly beneficial for every task and backbone. In this case, the unanchored MIRROR objective may provide a better trade-off between content retention and flexible headline generation. Overall, the rule-based metric supports the conclusion that MIRROR-family methods generally improve the preservation of targeted contextual information, while also showing that the effect of focal anchoring remains task-dependent. Finally, this metric should not be interpreted as a replacement for semantic-quality evaluation. It measures exact word-level coverage of a predefined set of grounded information-bearing terms and does not capture paraphrases, factual entailment, fluency, coherence, or professional style. Accordingly, we use it as a transparent diagnostic of information preservation and interpret it jointly with the G-Eval quality results. A.4 Rules for Focal Token Selection The focal selector in MIRROR-F provides a lightweight mechanism for identifying reference tokens that may carry important, context-accessible information. Rather than applying uniform supervision to the entire reference, additional supervision is concentrated on positions that satisfy two conditions: an informative surface form and lexical presence in the student’s input. This design provides an initial approximation to key-information extraction without requiring an external annotator, a named-entity recognition model, or an additional LLM. Importantly, the selector is modular: its rules can be edited to reflect different requirements for text quality while retaining the same on-policy distillation framework. Selection inputs and outputs. Let cSc_S denote the student’s input content, consisting of the retained user history and the current question, and let y=(y1,…,yT)y=(y_1,…,y_T) denote the tokenized reference. The selector produces a binary mask =(m1,…,mT),mt∈0,1,m=(m_1,…,m_T), m_t∈\0,1\, (7) where mt=1m_t=1 indicates that reference token yty_t receives the additional anchoring supervision. The mask is computed from the reference and the student-side context, not from the teacher’s reference-revealed prompt. Consequently, the presence of a word in the reference alone is insufficient for its selection. Subword grouping. Selection is performed over word-like groups before being mapped back to token positions. In the implementation, a new group begins at the first reference token or at a token whose vocabulary representation starts with the whitespace marker Ġ . Consecutive tokens are collected until the next such boundary. For each group gjg_j, the token strings are concatenated, the whitespace markers are removed, and only alphanumeric characters are retained to construct a core string wjw_j. This grouping prevents a multi-subword term from receiving supervision on only an arbitrary fragment. Once a group is selected, all token positions in that group are marked. Since grouping follows tokenizer whitespace markers rather than a linguistic parser, attached punctuation can also belong to a selected group. Rule 1: informative surface forms. Three inexpensive surface-form tests are used to identify candidate information-bearing groups: 1. Numeric content. The core contains at least one digit. This test can identify quantities, years, and alphanumeric identifiers. 2. Internal capitalization. At least one character after the first character is uppercase. This test can identify abbreviations and mixed-case names, such as PWA, DBT, and iPhone. 3. Initial capitalization. The core begins with an uppercase character, contains at least three characters, and is not included in a predefined stopword set. This test can identify names and capitalized topic terms. Groups whose cores contain fewer than two characters are excluded before these tests are applied. In particular, the current rule does not select an isolated single-digit core merely because it is numeric. Formally, I(w)=[|w|≥2][D(w)∨Uinner(w)∨(|w|≥3∧Ufirst(w)∧lower(w)∉ℬ)],I(w)=I[|w|≥ 2]\,I\! [D(w) U_inner(w) (|w|≥ 3 U_first(w) (w) ) ], (8) where D⁡(w)D(w) indicates the presence of a digit, Uinner(w)U_inner(w) indicates an uppercase character after the first position, Ufirst(w)U_first(w) indicates an uppercase initial character, and ℬB is the stopword set. The stopword filter is applied only to the initial-capitalization branch. It suppresses common sentence-initial words such as The, This, and We, while preserving the numeric and internal-capitalization tests. The implementation uses the following set: Stopwords for the initial-capitalization rule the, a, an, this, that, these, those, it, its, he, she, they, we, you, i, in, on, at, of, to, for, and, or, but, if, then, so, as, is, are, was, were, be, been, being, have, has, had, do, does, did, will, would, can, could, may, might, must, should, i’m, it’s, there, here, what, when, where, who, why, how, not, no, yes, with, without, from, by, about, into, over, under, after, before, my, your, his, her, our, their. These tests are surface-form heuristics rather than semantic entity classification. Their purpose is to obtain a transparent initial set of potentially important terms, not to identify every content-bearing expression. Rule 2: lexical grounding in the student context. An informative group is retained only if its lowercased core is present in the student-side context vocabulary. This vocabulary is constructed using the regular expression [A-Za-z0-9][A-Za-z0-9'\-]* and lowercasing the extracted strings. Let ⁡(cS)V(c_S) denote the resulting set. The grounding condition is G(w,cS)=[lower(w)∈(cS)].G(w,c_S)=I\! [lower(w) (c_S) ]. (9) Membership is checked against complete extracted strings rather than arbitrary substrings. For example, the core ID is not considered present merely because the context contains video or identify. The lookup is case-insensitive, so a capitalized reference term can match its lowercase occurrence in the input. This condition makes the auxiliary target depend on information that is lexically accessible to the student. Here, grounding specifically denotes lexical presence; it does not constitute a semantic entailment test. The implementation also uses different normalization procedures for the reference core and the context vocabulary: punctuation is removed from reference cores, whereas internal apostrophes and hyphens are retained in context matches. Thus, expressions such as Input-Output are not automatically treated as identical to the reference core InputOutput. No stemming, synonym matching, or entity linking is applied. Rule 3: expansion to token positions. The final selection criterion is the conjunction of informativeness and lexical grounding: A⁡(wj,cS)=I⁡(wj)G(wj,cS).A(w_j,c_S)=I(w_j)G(w_j,c_S). (10) The selected token set is then defined as sel=⋃j:A⁡(wj,cS)=1gj,mt=[t∈sel].T_sel= _j:A(w_j,c_S)=1g_j, m_t=I[t _sel]. (11) All subword positions within an accepted group are included. The end-of-sequence token is explicitly excluded from the selective mask. Illustrative cases. The following controlled examples explain the decisions made by the implemented rules. They are constructed illustrations of the selector, rather than additional empirical generation results. The context excerpts shown in each box are treated as the complete context for the corresponding lexical check; no additional occurrences in user history are assumed. Boldface in the reference identifies the word-like groups selected by the rules. Case 1: Abstract generation—method names and numerical information Student context. Generate an abstract for a study of DBT using 120 participants recruited in London. Reference illustration. This study evaluates DBT with 120 participants in London and compares outcomes with a Paris cohort. Selection decisions. • DBT: selected because it contains internal uppercase characters and occurs in the context. • 120: selected because it contains digits, has at least two characters, and occurs in the context. • London: selected because it is initially capitalized, is not a stopword, and occurs in the context. • Paris: rejected by the grounding condition, despite satisfying the capitalization rule. • This: rejected by the stopword filter. • participants: not selected because its lowercase form does not satisfy an informativeness test. This example illustrates how a simple selector can identify several details that are important for scientific reporting: a method abbreviation, a sample size, and a study location. The reference-only location is excluded from the auxiliary target because it is absent from the student’s context. The remaining reference positions are not uniformly anchored. Thus, additional supervision is concentrated on selected details without requiring reproduction of the complete reference wording. Case 2: News headline generation—entities and mixed-case names Student context. Generate a headline for the following article: Apple introduced an iPhone update in London. Reference illustration. Apple Unveils iPhone Update in London Selection decisions. • Apple: selected by initial capitalization and context membership. • iPhone: selected by internal capitalization and context membership, although its first character is lowercase. • Update: selected because the reference form is capitalized and its lowercase form matches update in the article. • London: selected by initial capitalization and context membership. • Unveils: not selected because that exact lexical form is absent from the context. • in: not selected. Here, the selected groups recover the organization, product name, event topic, and location. These elements form a useful preliminary description of the news content. The example also shows that the rule is not restricted to formally recognized named entities: title capitalization allows an ordinary topic word such as Update to be selected. This behavior is useful for a lightweight content selector, while making its dependence on the reference’s surface form explicit. Case 3: Paper title generation—technical terminology Student context. Generate a title for the following abstract of a paper: This paper studies piecewise affine models and gives conditions for their realization. Reference illustration. Realization of Piecewise Affine Models Selection decisions. • Realization, Piecewise, Affine, and Models: selected because they are capitalized in the reference and their lowercase forms occur in the abstract. • of: not selected. • If Piecewise is represented by multiple subword tokens, all tokens in its word-like group receive mask value 1. The selected positions jointly describe the central technical object and operation of the paper. Although no phrase-level parser is used, adjacent accepted groups can retain a multiword expression such as piecewise affine models. This illustrates the intended role of the selector: a small set of explicit rules can already identify useful information-bearing positions, providing a starting point for more specialized annotation policies. Use in the anchoring objective. The mask controls the reference-token loss, not the contents of an additional prompt. In particular, the selected words are not presented to the model as a separate instruction or annotation list. The student is evaluated under its ordinary context and the reference prefix, and the token-level negative log-likelihood is averaged only over selected positions: ℒanc(θ)=−∑t=1Tmtlog⁡πθ(yt∣cS,y<t)max⁡(1,∑t=1Tmt).L_anc(θ)=- _t=1^Tm_t _θ(y_t c_S,y_<t) \! (1, _t=1^Tm_t ). (12) Normalizing by the number of selected tokens prevents the auxiliary signal from being diluted by the length of the complete reference. Unselected reference tokens remain part of the teacher-forced prefix but do not receive a direct anchoring loss at their own prediction positions. Meanwhile, on-policy distributional alignment continues to operate over the student’s generated trajectory. When no token is selected, the anchoring loss is zero. In the implementation, the anchoring-loss moving average is not updated for that sample, and the effective focal coefficient is set to zero. The update then uses the normalized on-policy alignment term alone. This makes selection optional on a per-sample basis and avoids forcing an arbitrary reference token into the target set. Customizable selection according to quality requirements. The current rules are intentionally simple, interpretable, and inexpensive. Their role is to establish a first-stage extraction mechanism for potentially important information, rather than to prescribe a universal definition of text quality. Different users and applications may require different information to be preserved. For scientific writing, terminology, measurements, and methodological conditions may be prioritized. For news writing, organizations, locations, dates, and event participants may be emphasized. For review generation, product attributes and experience-related expressions may be more relevant. These requirements can be incorporated by editing the selection policy. Possible extensions include domain-specific terminology lists, explicit handling of units and numerical expressions, phrase-level matching, and user-specified inclusion or exclusion rules. Such changes modify the mask construction while leaving the student prompt and on-policy distillation mechanism unchanged. They also allow lowercase technical expressions to be selected when capitalization is not a useful cue. Extension to small-LLM-assisted annotation. A further direction is to replace or augment the surface-form rules with a small language model that identifies important reference spans under explicit quality requirements. Let rur_u encode the user’s desired quality criteria. A generalized selector can be written as sel=ϕ(cS,y,ru),T_sel=S_φ(c_S,y,r_u), (13) where ϕS_φ may be a rule system, a compact sequence-labeling model, or a small LLM. For example, a small LLM could identify technical phrases, distinguish central entities from incidental mentions, and annotate expressions related to numerical accuracy or content completeness. Returned spans could be validated against the reference and mapped to tokenizer positions before training. A separate context-support check could be retained or strengthened to prevent importance annotations from automatically admitting reference-only information. These annotations could be generated offline, so that no additional annotator would be required at inference time. This learned selector is a prospective extension, not a component of the current implementation. The present rule-based mechanism already supplies the necessary interface: important reference spans are converted into a token mask and consumed by the same anchoring objective. Consequently, improved information identification, user-editable quality criteria, and small-model annotation can be explored within a common framework, providing a concrete foundation for further development of MIRROR-F. A.5 Effect of the Anchoring Coefficient α on Professional Quality We further study how the anchoring coefficient α affects the professional quality of generated outputs. Here, α controls the relative contribution of the selective grounded anchoring loss in MIRROR-F. We use the same G-Eval protocol described in Section , where the judge model scores each output along four dimensions, including coherence, consistency, fluency, and relevance. The overall score is computed as the arithmetic mean of these four dimensions. Table 11 reports representative results on the Scholarly Title task under four different values of α. The results show a non-monotonic relationship between the anchoring strength and professional quality. The overall score is 4.7134.713 at α=0.1α=0.1 and slightly decreases to 4.7054.705 at α=0.3α=0.3. It then increases to the highest value of 4.7354.735 at α=0.7α=0.7, before decreasing substantially to 4.6504.650 at α=0.9α=0.9. The consistency score follows a similar pattern: it reaches its maximum of 4.8804.880 at α=0.7α=0.7, compared with 4.8104.810, 4.8304.830, and 4.6904.690 at α=0.1α=0.1, 0.30.3, and 0.90.9, respectively. These results indicate that a moderate anchoring coefficient provides the most favorable trade-off for professional-quality generation. Table 11: Representative G-Eval results under different values of α on the Scholarly Title task. Moderate anchoring achieves the highest overall professional-quality score, whereas an excessively large anchoring coefficient leads to performance degradation. Bold numbers denote the best result in each column. α Coherence Consistency Fluency Relevance Overall 0.1 4.480 4.810 4.980 4.580 4.713 0.3 4.440 4.830 5.000 4.550 4.705 0.7 4.500 4.880 4.990 4.570 4.735 0.9 4.440 4.690 5.000 4.470 4.650 This behavior suggests that the selective grounded anchoring loss is most effective when assigned a moderate weight. When α is relatively small, the anchoring signal may be insufficient to consistently stabilize grounded, information-bearing content. Increasing α to a moderate level improves the balance between preserving relevant content and maintaining flexible generation, resulting in the highest coherence, consistency, and overall quality at α=0.7α=0.7. However, when α becomes excessively large, the objective may over-emphasize word-level anchoring and constrain the model’s ability to organize the output naturally. This can explain the noticeable decline in consistency, relevance, and overall professional quality at α=0.9α=0.9. Overall, the results demonstrate that the effect of selective grounded anchoring is not monotonic. Moderate anchoring is more beneficial than either weak anchoring or overly strong anchoring, suggesting that the anchoring coefficient should be carefully controlled to avoid over-constraining the generation process. For consistency and fairness, all reported evaluation results in the main experiments use a fixed anchoring coefficient of α=0.5α=0.5. A.6 More Information about LLM-as-Judge Methods We employ two complementary LLM-as-judge protocols to evaluate personalized generation quality. The first protocol is a reference-free G-Eval procedure based on the four SummEval dimensions, which measures the intrinsic quality of each generated response with respect to the task input. The second protocol is a reference-based, aspect-level evaluation inspired by ExPerT, which measures the alignment between the generated response and the user-specific reference in terms of content and style. The two protocols are complementary: the former evaluates whether an answer is coherent, faithful, fluent, and relevant, whereas the latter evaluates whether the answer preserves the information and expressive characteristics conveyed by the reference. A.6.1 Judge Model and General Evaluation Settings All LLM-as-judge evaluations are conducted using the locally deployed Qwen3-30B-A3B model. The judge is served with vLLM using tensor parallelism over eight GPUs. To improve reproducibility, the decoding temperature is set to zero, and the judge is instructed not to activate its internal thinking mode. The maximum number of newly generated tokens is set to 512. The same judge model and decoding configuration are used for all compared methods. For each task, we identify the intersection of sample IDs available for all compared methods. This procedure ensures that every method is evaluated on exactly the same questions. If the number of common samples exceeds the predefined evaluation budget, a fixed random seed is used to select the evaluation subset. Unless otherwise specified, 100 common samples are evaluated for each task. No method-specific sample filtering or method-specific evaluation prompt is applied. A.6.2 Reference-Free G-Eval with SummEval Dimensions Evaluation objective. The reference-free protocol evaluates the intrinsic quality of a candidate response without exposing the gold reference to the judge. For each sample, the judge receives only the original task input and one candidate response generated by one method. The judge does not compare two methods and does not use the gold answer when assigning scores. This design avoids reducing content quality to lexical similarity with a single reference, which is particularly important for personalized generation because multiple responses may satisfy the same user preference. The evaluation dimensions are adopted from G-Eval and SummEval: • Coherence: whether the response is logically organized and forms a unified answer rather than a collection of loosely related statements. • Consistency: whether the response is factually supported by the source input and avoids hallucinated or unsupported claims. • Fluency: whether the response is grammatical, readable, and appropriately written in terms of spelling, word choice, and punctuation. • Relevance: whether the response preserves the important information from the source input while avoiding redundant or off-topic content. Each dimension is scored from 1 to 5, where 1 denotes the lowest quality and 5 denotes the highest quality. The judge is also asked to provide a short justification before returning the four scores in a fixed machine-readable format. Task-specific framing. The scoring dimensions are identical across tasks. To account for differences in output form, the judge receives one neutral task-form description: • For Abstract Generation, the candidate is described as a research-paper abstract written for a given title. • For News Headline Generation, the candidate is described as a short headline written for a given article. • For Paper Topic Generation, the candidate is described as the title of an academic paper written for a given abstract. • For other personalization tasks, the corresponding task form is specified without changing the scoring criteria. This task-specific framing clarifies the expected genre while keeping the evaluation rubric shared by all methods. G-Eval scoring prompt. The following box presents the prompt template used by the judge. The task description, source input, and candidate output are inserted into the placeholders TASK_FORM, SOURCE_INPUT, and CANDIDATE, respectively. Reference-free G-Eval prompt System: You are a meticulous, impartial NLG evaluation judge. User: You will be given one CANDIDATE text written for a task. Your job is to rate the candidate on four metrics. Read these instructions carefully and keep them open while reviewing. TASK_FORM Evaluation Criteria: - Coherence (1--5): the collective quality of the response. It should be well-structured and well-organized, not just a heap of related sentences. - Consistency (1--5): factual alignment between the response and the given source input. A consistent response contains only statements that are entailed by or faithful to the source; penalize hallucinated or unsupported facts. - Fluency (1--5): the quality of the response in terms of grammar, spelling, word choice, punctuation, and overall readability. - Relevance (1--5): selection of the important, on-topic content from the source. The response should include salient and pertinent information and exclude redundant or off-topic content. Evaluation Steps: 1. Read the SOURCE INPUT carefully. 2. Read the CANDIDATE and compare it against the source input. 3. Assign an integer score from 1 (worst) to 5 (best) for each metric. SOURCE INPUT: """ SOURCE_INPUT """ CANDIDATE: """ CANDIDATE """ Do not use any external reference answer. Give a one or two sentence justification, then end with exactly four lines: COHERENCE: <1--5> CONSISTENCY: <1--5> FLUENCY: <1--5> RELEVANCE: <1--5> Per-sample and per-task computation. Let ci,mc_i,m, si,ms_i,m, fi,mf_i,m, and ri,mr_i,m denote the Coherence, Consistency, Fluency, and Relevance scores assigned to sample i generated by method m. The overall quality score for one sample is calculated as the arithmetic mean of the four dimensions: qi,m=ci,m+si,m+fi,m+ri,m4.q_i,m= c_i,m+s_i,m+f_i,m+r_i,m4. (14) For a task t containing NtN_t successfully parsed samples, the dimension-level score for method m is calculated as: d¯m,t=1Nt∑i=1Ntdi,m,d∈coh,cons,flu,rel. d_m,t= 1N_t _i=1^N_td_i,m, d∈\coh,cons,flu,rel\. (15) The task-level G-Eval score is computed as the mean of the sample-level overall scores: q¯m,t=1Nt∑i=1Ntqi,m. q_m,t= 1N_t _i=1^N_tq_i,m. (16) If multiple tasks are evaluated, the final model-level score is obtained using an unweighted macro-average over tasks: q¯m=1|m|∑t∈mq¯m,t, q_m= 1|T_m| _t _m q_m,t, (17) where mT_m denotes the set of tasks for which method m has valid evaluation results. Thus, each task contributes equally to the final score, regardless of the number of samples available for that task. Scores are retained on the original 1–5 scale and are not rescaled to a 0–1 or 0–10 range. A.6.3 Reference-Based Aspect-Level Evaluation Evaluation objective. The reference-based protocol evaluates whether a candidate preserves the information and expressive behavior represented by the gold reference. Unlike the reference-free G-Eval protocol, this evaluation explicitly uses the reference answer. However, it does not compare candidate and reference through token-level overlap. Instead, the judge first decomposes both texts into atomic content and style aspects and then evaluates their alignment. This aspect-based design is suitable for personalized generation because the candidate may express the same preference in a different wording. A response is therefore rewarded when it preserves the underlying content and style aspects, even when it does not reproduce the exact reference sequence. Aspect extraction. For each reference and candidate, the judge extracts two lists: • Content aspects: atomic factual or semantic points explicitly expressed by the text. • Style aspects: writing characteristics such as tone, structure, formality, voice, and length preference. The extracted aspects are represented as strict JSON objects. The aspect extraction stage is performed before the matching stage. The reference aspects are extracted once for each (task,sample ID)(task,sample ID) pair and are reused for all methods evaluated on that sample. Aspect extraction prompt System: You extract atomic aspects from text as strict JSON. User: Extract the aspects of the following TEXT. Return STRICT JSON with exactly two keys: "content": list of short strings, each an atomic factual/content point the text actually makes; "style": list of short strings, each a writing-style attribute (tone, structure, formality, voice, length habit). Return ONLY the JSON object, no prose. TEXT: """ TEXT """ Aspect matching and scoring. After aspect extraction, the judge receives the content and style aspects of the reference and candidate. It then estimates three quantities: • Content recall: the fraction of reference content aspects expressed in the candidate. • Content precision: the fraction of candidate content aspects supported by the reference. • Style alignment: the degree to which the candidate’s style matches the style of the reference. Content recall measures whether the candidate preserves the information conveyed by the reference. Content precision penalizes unsupported or ungrounded content introduced by the candidate. Style alignment measures whether the candidate maintains user-specific expressive properties without requiring exact lexical reproduction. Aspect matching prompt System: You are a meticulous aspect-alignment judge for personalized text. User: Compare a CANDIDATE against a REFERENCE using their pre-extracted aspects. REFERENCE content aspects: REFERENCE_CONTENT_ASPECTS REFERENCE style aspects: REFERENCE_STYLE_ASPECTS CANDIDATE content aspects: CANDIDATE_CONTENT_ASPECTS CANDIDATE style aspects: CANDIDATE_STYLE_ASPECTS Assess alignment: - CONTENT_RECALL: fraction 0--1 of REFERENCE content aspects expressed in CANDIDATE; - CONTENT_PRECISION: fraction 0--1 of CANDIDATE content aspects supported by REFERENCE; - STYLE: integer 1--10 for how well CANDIDATE style matches REFERENCE style. End with exactly three lines: CONTENT_RECALL: <0--1> CONTENT_PRECISION: <0--1> STYLE: <1--10> Reference-based score computation. For each sample, content precision and content recall are combined using the harmonic mean: F1,i,mcontent=2Pi,mRi,mPi,m+Ri,m,F^content_1,i,m= 2P_i,mR_i,mP_i,m+R_i,m, (18) where Pi,mP_i,m and Ri,mR_i,m denote content precision and content recall, respectively. If both quantities are zero, the content F1 score is defined as zero. Because content F1 is defined on the range [0,1][0,1] while style is scored on the range [1,10][1,10], content F1 is first converted to a ten-point scale. The sample-level overall score is then calculated as: qi,maspect=10F1,i,mcontent+Si,m2,q^aspect_i,m= 10F^content_1,i,m+S_i,m2, (19) where Si,m∈[1,10]S_i,m∈[1,10] is the style score. The resulting score jointly reflects content preservation and stylistic alignment. Content recall, content precision, content F1, style, and the combined score are all averaged arithmetically over samples within each task: z¯m,t=1Nt∑i=1Ntzi,m, z_m,t= 1N_t _i=1^N_tz_i,m, (20) where z can be any of the five aspect-level quantities. As in the G-Eval protocol, the final score across multiple tasks is computed using an unweighted macro-average over task-level scores. A.6.4 Relationship between the Two Evaluation Protocols The two protocols measure different properties and are therefore reported separately. The reference-free G-Eval score measures the quality of the generated text as a response to the task input: Task Input⟶Candidate Quality.Task Input Quality. (21) The reference-based aspect-level score measures how well the candidate preserves reference-revealed content and style: Reference⟶Content and Style Alignment.Reference and Style Alignment. (22) Consequently, a high G-Eval score does not require lexical similarity to the gold reference, and a high aspect-level content score does not require the candidate to reproduce the reference verbatim. This distinction is important for personalized generation, where several different outputs may be valid. The combined evaluation framework therefore distinguishes between producing a high-quality response and reproducing the exact wording of a single reference. In the main experiments, the reference-free G-Eval results are used to assess intrinsic content quality through Coherence, Consistency, Fluency, and Relevance. The reference-based aspect-level results are used to complement the reference-based automatic metrics, such as ROUGE-1, METEOR, and BERTScore, by evaluating semantic content preservation and user-specific style alignment at the aspect level. A.7 On-policy Rollout and Gradient Handling The objective in Equation 4 is implemented as a stopped-gradient on-policy distillation objective. At each optimization step, the current student model first generates one response under the deployable context cSc_S. The generated token sequence is then treated as a fixed discrete rollout when the distributional alignment loss is calculated. Therefore, the implementation does not differentiate through the discrete sampling operation and does not use a score-function estimator, REINFORCE term, importance-weighted policy-gradient estimator, or a separate lagged behavior policy. More precisely, the rollout is generated as y^∼sg[πθ(⋅∣cS)], y \! [ _θ(· c_S) ], (23) where sg⁡[⋅]sg[·] denotes stop-gradient. For each sampled prefix y^<t y_<t, the student distribution is computed as ptS=πθ(⋅∣cS,y^<t),p_t^S= _θ (· c_S, y_<t ), (24) while the reference-revealed teacher distribution is detached: ptT=sg[πθ(⋅∣cT,y^<t)].p_t^T=sg\! [ _θ (· c_T, y_<t ) ]. (25) Consequently, the gradient is propagated through ptSp_t^S only. The sampled tokens, the rollout probabilities, and ptTp_t^T do not receive gradients. The student model is updated after the loss computation, and a new rollout is generated from the updated student at the next optimization step. The rollout uses stochastic sampling with temperature 1.01.0 and top-p=1.0top-p=1.0. The maximum generation length is determined on a per-example basis from the reference length and the task-specific cap. Generation terminates when either the EOS token is produced or the maximum length is reached. If EOS is produced before the length limit, the EOS token is retained in the aligned completion and tokens after EOS are discarded. If the length limit is reached without EOS, the capped completion is used and the event is recorded for diagnostic purposes. Samples with fewer than two completion tokens are skipped because they do not provide a meaningful sequence-level alignment signal. The implementation therefore optimizes a surrogate objective rather than the full derivative of an expectation whose sampling distribution depends on θ. In particular, the gradient does not contain the additional score-function term y^∼πθ[ℒ⁡(y^,θ)∇θlog⁡πθ(y^∣cS)].E_ y _θ [L( y,θ) _θ _θ( y c_S) ]. (26) This choice avoids the high variance associated with policy-gradient estimation and directly trains the student to match the teacher distribution on states that the deployable student actually visits. The term “on-policy” therefore refers to the state distribution used to construct the training prefixes, whereas the distributional alignment itself is optimized by ordinary backpropagation on the detached sampled trajectories. A.8 Other Prompts This section describes the prompts used for model inference, user-history conditioning, and MIRROR training. The prompts used for LLM-as-judge evaluation are described separately in Appendix A.6. All prompts are converted into the backbone model’s native chat format using the tokenizer’s chat template. Unless otherwise stated, the system message is shared across training and evaluation. A.8.1 Common Generation Prompt For all generation-based evaluations, the model receives a system instruction followed by a user message. The system instruction constrains the model to output only the requested text and prevents additional explanations or formatting from being included in the prediction. Common generation prompt System: You are a helpful assistant. Write only the requested text itself, with no preamble, no explanation, and no surrounding quotation marks. User: USER_CONDITIONING TASK_QUESTION Here, TASK_QUESTION denotes the original task instruction and input, while USER_CONDITIONING is either empty, a retrieved user profile, or the full user history, depending on the evaluation protocol. The model is always evaluated using the same system instruction, regardless of whether it is a base model, an SFT-based model, or a MIRROR-based model. For example, when no user history is supplied, the user message consists only of the task question: Non-personalized generation prompt System: You are a helpful assistant. Write only the requested text itself, with no preamble, no explanation, and no surrounding quotation marks. User: TASK_QUESTION The tokenizer-specific chat template is applied after the system and user messages have been constructed. For Qwen3 models, the evaluation configuration disables thinking mode by default. Thus, the model directly generates the requested answer without producing an additional <think> segment. A.8.2 Task Question Templates The original task question is preserved during evaluation. We use the following task-specific forms. Abstract Generation. For Abstract Generation, the model is asked to generate an abstract conditioned on a paper title and a set of key points: Abstract Generation question template Generate an abstract for the title ‘TITLE’ using the following items: 1. KEY_POINT_1 2. KEY_POINT_2 … K. KEY_POINT_K The output is expected to be a self-contained research-paper abstract. The model is not explicitly asked to copy the reference abstract. Instead, the key points and the user history provide the information and stylistic context needed to generate a new abstract. News Headline Generation. For News Headline Generation, the model receives a news article and is asked to generate a headline: News Headline Generation question template Generate a headline for the following article: ARTICLE_TEXT The generated answer is expected to be a short headline rather than a paragraph. The task therefore evaluates whether the model can identify the article-specific topic while preserving the user’s preferred headline format, vocabulary, and editorial style. Paper Topic Generation. For Paper Topic Generation, the model receives an academic abstract and is asked to produce a paper title: Paper Topic Generation question template Generate a title for the following abstract of a paper: PAPER_ABSTRACT The output is expected to be a concise academic title. The model should preserve the central technical concepts of the abstract while matching the user’s preferred terminology and title structure. Other LongLaMP tasks. For product review generation, the question specifies the product and the requested review context. For topic writing, the question provides the title or prompt for a Reddit-style post body. The same common generation prompt is used, while only the task question changes. A.8.3 User-History Conditioning Personalization is introduced by appending user-specific historical examples before the current task question. The historical examples are formatted as demonstrations that associate an input with the corresponding user-written output. The formatting depends on the task because different tasks contain different input–output structures. Full-history conditioning. In the full-history setting, historical examples are retained in their original temporal order. When the complete history exceeds the input budget, the most recent entries are preferentially retained. The current question is always preserved in full whenever possible. The resulting user message is: Full-history conditioning template HISTORY_EXAMPLE_1, and HISTORY_EXAMPLE_2, and …, and HISTORY_EXAMPLE_M. Following the given patterns, TASK_QUESTION The phrase “Following the given patterns” indicates that the preceding examples describe the user’s writing behavior. It does not instruct the model to copy any particular historical answer verbatim. Task-specific history formatting. For Abstract Generation and Paper Topic Generation, each historical item is rendered as: Academic history example ‘TITLE’ is a title for the abstract ‘ABSTRACT’ For News Headline Generation, each historical item is rendered as: News history example ‘HEADLINE’ is a headline for the article ‘ARTICLE_TEXT’ For product review generation, a historical item contains the review summary, rating, product description, and user-written review: Product-review history example the review with summary ‘SUMMARY’ and rating RATING for the product ‘DESCRIPTION’ is ‘REVIEW’ For Amazon review tasks, the historical item is rendered as: Amazon-review history example the review with rating RATING and title ‘REVIEW_TITLE’ for the item ‘ITEM_TITLE’ (ITEM_DESCRIPTION) is ‘REVIEW_TEXT’ For topic writing, historical examples are rendered as: Topic-writing history example the reddit post with summary ‘SUMMARY’ is ‘POST_BODY’ These task-specific templates preserve the semantic relation between a historical input and the corresponding user-written output. Consequently, the model can infer both the user’s preferred content selection and the user’s preferred expression pattern. A.8.4 BM25 Retrieval Prompt For retrieval-based baselines, the complete user history is not appended to every query. Instead, BM25 is applied to the user profile, and the two most relevant historical examples are selected. The retrieval corpus is constructed from both the input and output fields of each historical item. For example, the corpus for Abstract Generation contains the paper title and abstract, while the corpus for News Headline Generation contains the headline and article text. The selected examples are rendered using the task-specific history templates above and concatenated with the current task question: BM25 retrieval-conditioned prompt RETRIEVED_EXAMPLE_1, and RETRIEVED_EXAMPLE_2. Following the given patterns, TASK_QUESTION The retrieval operation is performed independently for each test question. It affects only the prompt provided to the model and does not modify the model parameters. The gold answer is never used to construct the retrieval prompt. A.8.5 MIRROR Student Prompt The MIRROR student is trained to operate under the same information available during deployment. Therefore, the student receives the user history and the current task question, but not the gold reference answer. The student prompt is identical to the full-history inference prompt: MIRROR student prompt System: You are a helpful assistant. Write only the requested text itself, with no preamble, no explanation, and no surrounding quotation marks. User: FULL_USER_HISTORY Following the given patterns, TASK_QUESTION During training, the current student model samples an on-policy response from this prompt. Let y y denote the sampled response and y^<t y_<t denote its prefix at step t. The student distribution is evaluated as pt=πθ(⋅∣cS,y^<t),p_t= _θ (· c_S, y_<t ), (27) where cSc_S contains only the information available to the student at deployment time. A.8.6 MIRROR Teacher Prompt The MIRROR teacher receives the same user history and task question as the student, together with the reference answer. The reference is treated as privileged hindsight information used only during training. It is not provided to the deployed model. The teacher prompt is constructed by placing the reference answer after the task question and adding an instruction that asks the teacher to solve the task independently after understanding the reference: MIRROR teacher prompt System: You are a helpful assistant. Write only the requested text itself, with no preamble, no explanation, and no surrounding quotation marks. User: FULL_USER_HISTORY TASK_QUESTION Here is a reference solution: GOLD_REFERENCE After understanding the reference solution, please try to solve this problem using your own approach below: The teacher is evaluated on the student’s sampled prefix rather than on the reference prefix. Specifically, the teacher distribution is computed as qt=π¯θ(⋅∣cT,y^<t),q_t= π_θ (· c_T, y_<t ), (28) where cTc_T is the reference-revealed context and π¯θ π_θ denotes a detached copy of the current model. The reference answer therefore changes the teacher’s distribution, but it is not used as a token-level target sequence. The student and teacher prompts are intentionally different in information content: cS=(user history,task question),c_S=(user history,task question), (29) cT=(user history,task question,reference answer,reference hint).c_T=(user history,task question,reference answer,reference hint). (30) The teacher and student are nevertheless evaluated on the same on-policy prefix y^<t y_<t. This construction transfers the behavior induced by the reference into states that the student actually visits during inference. A.8.7 MIRROR-F Teacher Prompt and Focal Anchoring MIRROR-F uses the same student and teacher prompts as MIRROR. The difference lies in the training objective rather than in the basic prompt structure. In addition to aligning the student and teacher distributions over the student’s on-policy trajectory, MIRROR-F applies a selective anchoring loss to informative reference tokens. The selected token set contains content-bearing and context-grounded tokens. In particular, the selection procedure favors terminology, named entities, quantities, and other tokens that carry task-specific information, while excluding purely functional or stylistic tokens. The selected word-level positions are expanded to their complete subword spans. The focal anchoring prompt therefore does not expose any additional information to the deployed model. It only determines which reference tokens receive the auxiliary training signal: MIRROR-F selective anchoring template Reference solution: GOLD_REFERENCE Selected informative and context-grounded positions: SELECTED_REFERENCE_TOKENS The selective anchoring term is applied only during training. At inference time, MIRROR-F receives exactly the same deployable prompt as MIRROR: Inference inputMIRROR−F=Inference inputMIRROR.Inference input_MIRROR-F=Inference input_MIRROR. (31) This separation is essential. The gold reference and the selected reference tokens are privileged training information and are never supplied to the deployed model. A.8.8 User Profile Summary Prompt For the summary-based ablation, a compact user profile is generated from the user’s historical texts. The profile summary is generated without access to the current gold answer. It is therefore intended to compress writing preferences rather than reveal the target output. The summary prompt is: User profile summary prompt System: You are a helpful assistant. Write only the requested text itself, with no preamble, no explanation, and no surrounding quotation marks. User: Here is the writing history of a single user (their past examples): USER_HISTORY Based only on the writing history above, write a complete user profile report describing the user’s preferences for content selection, organization, tone, terminology, sentence structure, and level of detail for TASK_DESCRIPTION. Be specific and evidence-based. Do not quote whole sentences from the history and do not mention that you were given a history. The generated summary is inserted only into the privileged teacher context in the summary-based variant. The student still receives the original user history and task question. The summary is truncated to a fixed token budget before being inserted into the teacher prompt. A.9 Thinking-Mode Ablation We further examine whether enabling the model’s internal thinking mode substantially changes personalized generation performance. Table 12 reports the results of the same Qwen3 backbones under thinking-on and thinking-off configurations. The reported values are macro-averaged across the three LongLaMP tasks, while the task-level results are provided to show the variation across generation settings. Table 12: Thinking-mode ablation on LongLaMP. Values are macro-averaged across Abstract Generation, Product Review, and Topic Writing. Δ denotes the change from thinking off to thinking on; positive values indicate improvement. Qwen3-1.7B Qwen3-4B Mode ROUGE-1 METEOR BERTScore ROUGE-1 METEOR BERTScore Thinking off 0.2661 0.1437 0.8378 0.3334 0.1985 0.8467 Thinking on 0.2908 0.1616 0.8351 0.3220 0.1847 0.8340 Δ +0.0247 +0.0179 −0.0027-0.0027 −0.0114-0.0114 −0.0138-0.0138 −0.0127-0.0127 The results show that enabling thinking does not induce a consistent performance trend in personalized generation. For Qwen3-1.7B, thinking mode improves the macro-averaged ROUGE-1 and METEOR scores, while producing a slightly lower BERTScore. For Qwen3-4B, thinking mode decreases all three macro-averaged metrics, although it improves ROUGE-1 and METEOR on the Topic Writing task. These results indicate that the effect of thinking is task- and scale-dependent rather than uniformly beneficial or harmful. Overall, the main performance trend of personalized generation remains largely preserved across the two thinking configurations. This observation suggests that personalized generation and long-horizon reasoning should be regarded as related but distinct capability dimensions. Thinking mode is designed to allocate additional computation to intermediate reasoning, whereas personalization requires the model to infer and apply user-specific preferences in content selection, organization, terminology, and expression. Improvements in complex reasoning ability alone therefore do not necessarily translate into improvements in personalized generation. Likewise, the effectiveness of RL- or OPD-based training for long-horizon problem solving should not be assumed to imply a corresponding gain in personalization. The two capabilities may rely on different forms of internal representation and different training signals. Accordingly, these results motivate the study of LLM personalization as an independent research problem rather than treating it as an automatic consequence of general reasoning improvement. Methods that explicitly internalize user preferences, such as MIRROR and MIRROR-F, address this separate dimension by introducing preference-revealed supervision and personalization-specific objectives. Further investigation is needed to understand how reasoning ability and personalization ability interact and whether they can be improved jointly without assuming that progress in one dimension automatically transfers to the other.

Instant research discovery