Paper deep dive
Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing
Tianci Liu, Zihan Dong, Tianchun Li, Yi-Chung Chen, Qiming Cao, Xingchen Wang, Shiyang Wang, Zichen Miao, Linjun Zhang, Haoyu Wang, Jing Gao
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world. This motivates knowledge editing (KE), which updates specific knowledge in an LLM without changing unrelated others. Recent works move from structured knowledge triples toward unstructured KE (UKE), where the edit is a free-form passage that may state multiple facts at once. Nonetheless, existing editors inject such a passage yet fail to use it: the edited model can recall the passage, but can neither answer atomic questions about its facts nor compose them into multi-hop reasoning. We attribute this missing property, which we term composability, to editors' passive reliance on the fixed passage as the sole learning source. In response, we cast editing as a proactive self-distillation from a privileged in-context state of the same model, which requires no external supervision. We further reveal that due to the novelty of the injected knowledge, the pre-edited model's own rollouts rarely cover it, which limits the effectiveness of pure on-policy distillation. To close this gap, we propose HPSE, which builds a hybrid rollout that steps in to place missing facts onto the student's own trajectory precisely where its coverage fails, while staying on-policy elsewhere. We theoretically analyze HPSE's advantage over pure on-policy distillation, and empirically establish its plug-and-play improvements across four LLM backbones and two KE editors under various scenarios.
Tags
Links
- Source: https://arxiv.org/abs/2608.11660v1
- Canonical: https://arxiv.org/abs/2608.11660v1
Trouble viewing inline? Open PDF directly →
Full Text
123,861 characters extracted from source content.
Expand or collapse full text
Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing Tianci Liu1,3 Zihan Dong2 Tianchun Li3 Yi-Chung Chen3 Qiming Cao3 Xingchen Wang3 Shiyang Wang3 Zichen Miao3 Linjun Zhang2 Haoyu Wang4 Jing Gao3 1University of Tennessee 2Rutgers University 3Purdue University 4University at Albany Abstract Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world. This motivates knowledge editing (KE), which updates specific knowledge in an LLM without changing unrelated others. Recent works move from structured knowledge triples toward unstructured KE (UKE), where the edit is a free-form passage that may state multiple facts at once. Nonetheless, existing editors inject such a passage yet fail to use it: the edited model can recall the passage, but can neither answer atomic questions about its facts nor compose them into multi-hop reasoning. We attribute this missing property, which we term composability, to editors’ passive reliance on the fixed passage as the sole learning source. In response, we cast editing as a proactive self-distillation from a privileged in-context state of the same model, which requires no external supervision. We further reveal that due to the novelty of the injected knowledge, the pre-edited model’s own rollouts rarely cover it, which limits the effectiveness of pure on-policy distillation. To close this gap, we propose HPSE, which builds a hybrid rollout that steps in to place missing facts onto the student’s own trajectory precisely where its coverage fails, while staying on-policy elsewhere. We theoretically analyze HPSE’s advantage over pure on-policy distillation, and empirically establish its plug-and-play improvements across four LLM backbones and two KE editors under various scenarios. Keywords: Knowledge Editing, Large Language Models = Date: August 12, 2026 = Code & Datasets: https://github.com/lliutianc/hpse = Contact: tliu43@tennessee.edu 1 Introduction Large language models (LLMs) (64, 4) demonstrate strong generalization across a wide range of language tasks (54, 27), establishing themselves as a new foundation for modern NLP (3, 92). As model sizes grow, LLMs further exhibit emergent abilities to follow natural language instructions (14, 50), enabling zero-shot adaptation to unseen tasks (32, 5). However, real-world LLM deployment remains far from resolved unless LLMs can continually refresh and internalize new knowledge beyond their training data. In a fast-changing world, an LLM’s static knowledge quickly goes out of date, leading to factual errors or even unsafe generations (11, 22). Yet retraining the model to absorb each update is prohibitively expensive. As a remedy, Knowledge Editing (KE) has been proposed to address this by updating an LLM with a specific piece of knowledge (68, 86). Given a direct, well-structured statement of new knowledge, e.g., a knowledge triple naming the new owner of a recently acquired company, KE typically seeks selective parameter updates that keep unrelated knowledge and general capabilities intact. To name a few, 45, 16 apply closed-form updates to a few causally traced MLP layers that serve as knowledge storage (10), 65 directly learn sparse low-rank updates. 40 steer selected representations with non-linear updates. Recent works move towards the more realistic setting of unstructured KE (UKE), where the new knowledge spans multiple aspects in free-form text. For example, an actual acquisition announcement is more likely to be a free-form passage that states multiple entangled facts at once, such as the new parent company, the incoming CEO, and the reshaped reporting structure, with no specification of which facts a user may later query. UKE pursues this more challenging objective: treating the announcement passage itself as the knowledge to edit (71). To this end, 12 extend the edit from a few MLP layers to entire transformer blocks for larger capacity. 28 decompose the passage into sequential chunks for iterative editing; and 60 preserve the dependencies among chunk-wise updates. These fine-grained designs have collectively demonstrated success in making KE precise. Nonetheless, the edited knowledge turns out largely useless compared to knowledge acquired through pretraining in two aspects. The first is the loss of decomposition: after internalizing a long-form edit, the model can at best recall the whole passage yet cannot answer targeted questions about the individual facts it encodes (93, 67). The second is the loss of composition: having learned individual facts, the model cannot compose them for multi-hop reasoning (91, 90). Both phenomena have been analyzed in the literature, but mostly in isolation under separate scenarios (85, 79), despite that they co-occur and are amplified in UKE. We use composability to denote whether edited knowledge, like its pretrained counterpart, can function as atomic facts that are flexibly extracted and recombined, and study decomposition and composition from this unified perspective. In particular, we introduce an untargeted regime for UKE: a free-form knowledge statement is paired with a generic editing prompt that does not specify which facts are updated, mirroring practical requests where annotators offer only a brief summary of the new passage. We recast two representative benchmarks into a decomposition probe (12) and a composition probe (90), respectively. We find that the evaluated methods, despite their different mechanisms, consistently fall short of composability under our regime. We attribute this failure to their passive reliance on the provided editing context as the sole learning source, which induces severe memorization and poor generalization (85, 39, 8); existing augmentation-based remedies (93, 79, 67) alleviate but do not eliminate this reliance. In summary, composability remains an open challenge, necessitating a generalizable UKE method; Section 2 details these findings. Our untargeted regime and formulation of composability offer a unified view of what UKE requires in the wild. Together with our benchmarking protocol, these constitute the first key contribution of this work. Given the pivotal role of the editing context, we pursue a proactive strategy through on-policy self-distillation (OPSD): the edited model generates its own response trajectories, while a privileged state of the same model (one that reads the new passage in context) provides token-level distillation targets (41, 59, 88). Although OPSD has proved effective for adapting behavioral patterns in post-training (8, 41, 59), it falls short in UKE: because the injected knowledge is novel, the model’s own rollouts often go off-topic, yielding limited corrective supervision (17, 84). We term this a coverage failure. To mitigate this issue, we propose HPSE, which asks the privileged model not only to grade the edited model’s own responses, but also to step in when a rollout deviates too far from a useful trajectory. This strategy forms hybrid, rather than purely on-policy, rollouts with better coverage, offering stronger supervision for UKE. We further support HPSE with theoretical justification. Notably, HPSE is a data-centric approach: it replaces only the training signal, making no assumption about which parameters are updated or how the update is parameterized, thereby offering plug-and-play flexibility across gradient-based KE editors. The proposed HPSE and our theoretical analysis constitute the second key contribution of this work. The remainder of this paper is organized as follows. Sections 2 and 3 detail composable UKE and the proposed HPSE, respectively. Section 4 evaluates HPSE across various settings. Finally, Section 5 reviews related work, and Section 6 concludes the paper. 2 Composability Challenge in Unstructured Knowledge Editing This section presents composability, a critical requirement for practical unstructured knowledge editing (UKE) that has been largely overlooked in the literature. Background on LLMs is also provided. 2.1 Preliminaries Given a text =(x1,…,xn)x=(x_1,…,x_n), where each xi∈x_i is a token from vocabulary V, a large language model (LLM) parameterized by θ computes probability πθ() _θ(x) based on the chain rule (2): πθ() _θ(x) =∏i=1nπθ(xi∣x1,…,xi−1)≜∏i=1nπθ(xi∣<i), = _i=1^n _θ(x_i x_1,…,x_i-1) _i=1^n _θ(x_i _<i), where πθ(xi∣<i) _θ(x_i _<i) is the predicted distribution of token xix_i given the previous <ix_<i. The LLM is usually trained with maximum likelihood estimation (24, 61, 7, 81). To generate a sentence x, the LLM computes πθ(xi∣<i) _θ(x_i _<i) and draws xix_i from it; then xix_i is combined with <ix_<i as new inputs for future steps. This process terminates when a special token that marks the end of the sentence is generated, or when the maximum length is reached. Knowledge Editing (KE) aims to update a pre-trained LLM’s specific knowledge precisely while preserving other knowledge unrelated to the update, a requirement known as locality (68, 86). Structured KE expresses a piece of knowledge as a triple (subject, relation, object) and requires updating object to a new value. Given a natural-language pair (,)(x,y) where the editing prompt x describes a subject and relation, and y specifies the corresponding object, KE asks the LLM to respond to x with the edited object y. Built upon KE, Unstructured KE (UKE) studies a more realistic regime, where a free-form passage c encodes one or multiple pieces of new knowledge, with no labeled knowledge triple, making it more challenging (71, 77, 28). 2.2 Formulating and Benchmarking UKE Composability Composability Definition. KE is useful only if the knowledge it injects can be used by the LLM, not merely memorized in a way that can only be repeated under the exact query used for editing (91, 12, 85, 90, 93). Crucially, when it comes to UKE, where multiple pieces of knowledge are injected at once, a useful edit entails the following two aspects, which we collectively refer to as composability. • Decomposition: Multiple pieces of knowledge are often encoded in the passage c and are injected jointly by editing the LLM with c directly (12). In this case, an ideal LLM should be able to decompose these facts and absorb them individually. After editing, any query about an atomic fact in the passage should be answered with the fact directly, rather than by repeating the whole passage (93, 67). • Composition: Different pieces of new knowledge are usually related. Consequently, the edited model should be able to proactively combine them for multi-hop reasoning. This is closely related to the notion of portability (91, 69, 90). Untargeted Regime. We further consider how the new knowledge is presented for editing. In practice, human labelers may want to only summarize the passage briefly, instead of detailing which contents constitute the updated knowledge. To reflect this demand, we formulate UKE in an untargeted regime: the passage c is paired with a generic editing instruction such as “introduce X”, see Appendix B for detailed examples. Benchmarking Composability. We next conduct a systematic study on UKE composability. Guided by above principles, we transform existing KE benchmarks into better (de)composition probes. More details on our benchmark construction can be found in Appendix B. • Decomposition probe uses UnKEBench (12), where each sample involves a free-form passage that contains five atomic facts. After editing, the LLM is required to answer both the passage-level question that requires joint recall of all facts (47), and the targeted questions about each individual fact, which require decomposed recall. Unlike the original benchmark, whose editing prompt reveals the updated facts, we summarize the editing prompt into a more generic description, in line with our untargeted regime. The testing queries are left unchanged. To better measure actual knowledge injection rather than lexical repetition of the passage, we use LLM-as-Judge-based scoring, following 47, 12, 77. • Composition probe recasts MQuAKE-CF-remastered (90), a cleaned version of the original (91). Each editing request in MQuAKE involves 2–4 pieces of new knowledge, each expressed as a separate single-sentence statement. After editing, the LLM is required to answer each single-hop question individually, and to compose them to solve multi-hop questions. In our benchmark, we expand each single statement into an unstructured passage following the protocol of 71. Again, we use a generic editing prompt, with testing queries unchanged. We adopt the original evaluation pipeline from the official benchmark. Figure 1: Composability results (Qwen2.5). Existing methods fail to handle (de)composition. Benchmarking Results. We benchmark seven representative KE methods on editing Qwen2.5-7B-Instruct (52), with results presented in Figure 1: Jnt. and Dmp. denote the passage-level and fact-level accuracy of the decomposition probe, Ind. and Cmp. denote the single-hop and multi-hop accuracy of the composition probe, and Locality is measured by MMLU. The numeric results are deferred to Section 4. All editors exhibit notable failures in composability. Some editors, even those originally designed for UKE (e.g., AnyEdit (28) and UnKE (12)), fall short of recalling the new knowledge in either a joint or a decomposed way, and the structured editors fare no better. COIN⋆ (93), an NTP-based editor that fine-tunes on the passage while regularizing against context reliance, achieves the highest decomposed recall (Dmp. 53), though largely by reproducing the whole passage (as shown later in Section 4), and it composes little of the injected knowledge either. Composition also proves hard, with the strongest baseline reaching merely 32. These failures are exemplified by cases in Figure 2. On the decomposition side, COIN⋆ reproduces the injected passage instead of answering the atomic question, carrying the target answer only indirectly; on the composition side, AnyEdit cannot chain the two injected edits and falls back on the model’s prior knowledge. In conclusion, composability remains an open challenge for UKE, leaving a substantial gap between passively injecting knowledge and proactively using it. In the next section, we propose our solution. Failure cases of UKE Composability. (a) Decomposition. Edit passage (5 facts): Elvin Penner is a songwriter who has written hit songs for popular artists such as Taylor Swift and Ed Sheeran… Question:Which artists has Elvin Penner written hit songs for? Target: Taylor Swift and Ed Sheeran. COIN⋆ ✗ Elvin Penner is a well-known songwriter in the music industry. He has written…(whole passage repetition) (b) Composition. Edit passages (2, one in each): the religion Louis XV practiced → Christian Science; the founder of Christian Science → Ibn Saud. Multi-hop question: Who founded the religion Louis XV practiced? Target: Ibn Saud. AnyEdit ✗ Louis XV was a prominent figure in French Catholicism…the Jansenist movement…(prior knowledge) Figure 2: Existing KE methods fall short in handling composability, either failing to answer targeted question directly, or to reason properly with new knowledge. Errors are highlighted. 3 Proposed Method In this section, we propose HPSE, which incorporates proactive knowledge use at editing time toward better UKE composability. Theoretical analysis is also provided. 3.1 Pursuing Composability via Self-Distillation At a high level, the composability failures above share a common cause: minimizing −logπθ(∣)- _θ(c ) supervises the new facts only under the passage’s own prefixes. Data augmentation enlarges this context set (93, 79, 67); however, the model remains a passive learner from externally fixed targets, limiting its ability to generalize beyond the contexts observed during editing. Consequently, knowledge injected in this way can remain bound to the contexts in which it was learned (93). The recently emerging on-policy distillation (OPD) enables proactive learning: rather than fitting externally specified signals, the student generates its own rollout for an external teacher to correct, an approach that has proved effective in enhancing generalization (59, 37). Such proactivity lets the edited model invoke the new knowledge in its own generation rather than merely reproduce it under a fixed context, paving a promising path toward better composability. However, OPD assumes a teacher that already masters the targeted knowledge. When no such teacher is available11 1 KE is such a case, as the knowledge to edit is by definition novel., on-policy self-distillation (88, OPSD;) turns the student itself into the teacher through its in-context learning capability (89). Specifically, given the editing passage c and editing prompt x, the privileged model π⋆π is defined as the base model π0 _0 reading c in context, and OPSD then trains the edited model πθ _θ to match π⋆π by minimizing the following loss (θ)=∼πθ(⋅∣)[∑t=1||DKL[π⋆(⋅∣,<t)∥πθ(⋅∣,<t)]],whereπ⋆(⋅∣)≜π0(⋅∣,). (θ)= _y _θ(· ) [ _t=1^|y|D_KL [π (· ,y_<t)\,\|\, _θ(· ,y_<t) ] ], π (· ) _0(· ,x). (1) Here the forward Kullback–Leibler divergence DKLD_KL matches the privileged distribution in a mass-covering manner (37). As y is drawn from πθ _θ itself, the model learns to proactively correct its own output. Figure 3: OPSD alone brings no consistent gain. Challenge: The Coverage Failure of OPSD. Unfortunately, OPSD brings no consistent gain on UKE. As shown in Figure 3, it improves decomposed recall, yet degrades joint and individual recall, exhibiting a failure mode distinct from its post-training successes. We provide an intuitive explanation. Since the new knowledge in c is by definition absent from the pre-edited model, its rollout may follow a path distant from c: the response goes off-topic, largely irrelevant to the new facts rather than merely outdated, leaving the privileged model little room to provide corrective signal. Figure 12 shows a coverage failure case. In summary, OPSD proves insufficient to unlock composability, which motivates HPSE below. 3.2 HPSE: A Plug-and-Play Solution Given the coverage failure in UKE, we propose Hybrid-Policy Self-Editing (HPSE) to supply self-distillation with better rollouts. HPSE is a plug-and-play approach that improves the editing signal with hybrid rollout data, and thus applies to a wide range of gradient-based KE editors. Algorithm 1 outlines its process. Specifically, HPSE constructs a hybrid rollout that mixes token-level outputs from the edited model and the privileged state, which is built autoregressively as follows. At step t, given the running prefix <ty_<t, we draw yty_t from the hybrid rollout policy πρ _ρ, a per-token switch between the two policies, yt∼πρ(⋅∣,<t)≜π⋆(⋅∣,<t)if step-in is triggered,πθ(⋅∣,<t)otherwise. y_t\; \; _ρ(· ,y_<t)\; \; cases\;π (· ,y_<t)&if step-in is triggered,\\[3.0pt] \; _θ(· ,y_<t)&otherwise. cases (2) Here the step-in is triggered when the privileged model and the student significantly disagree, and the privileged is confident in its own prediction. Intuitively, this repairs the off-topic rollout with a minimal change: it lets the privileged model place the missing facts onto the student’s own trajectory precisely where the student would otherwise stray from c. In practice, we trigger the step-in if logπ⋆(yt⋆∣,<t)−logπθ(yt⋆∣,<t)⏟privileged–student gap>τandπ⋆(yt⋆∣,<t)⏟privileged confidence>κ, π (y _t ,y_<t)- _θ(y _t ,y_<t)_privileged--student gap>τ\;\;and\;\; π (y _t ,y_<t)_privileged confidence>κ, (3) where yt⋆≜argmaxyπ⋆(y∣,<t)y _t _yπ (y ,y_<t) and τ,κτ,κ are hyperparameters. The first criterion locates the tokens the student is about to miss, and the second ensures the privileged next-token prediction is itself reliable. Note that the confidence can also be quantified with other measures such as the privileged distribution entropy (30). We adopt max probability due to its simplicity. As the KE process proceeds, the edited model injects the new knowledge, leading to fewer step-in triggers. Consequently, the hybrid distribution gradually converges to the pure on-policy one (see Section 4.4 for empirical evidence). Building on the hybrid distribution in Eq. (2), HPSE trains the edited model by minimizing: HPSE(θ) _HPSE(θ) =hybrid(θ)+λNLL(θ) =J_hybrid(θ)+λ\,J_NLL(θ) (4) =∼πρ(⋅∣)[∑t=1||DKL[π⋆(⋅∣,<t)∥πθ(⋅∣,<t)]]−λlogπθ(∣). = _y _ρ(· ) [ _t=1^|y|D_KL [π (· ,y_<t)\,\|\, _θ(· ,y_<t) ] ]-λ _θ(c ). Here the negative log-likelihood (NLL) term preserves the standard passage-level likelihood objective as a lightweight anchor. Intuitively, the hybrid term supervises the new facts along the student’s own trajectory, and the anchor grounds them under the passage’s own prefixes. Section 4.4 ablates their respective contributions. We set λ=1λ=1, and draw a single greedy hybrid rollout per round for efficiency. Finally, because HPSE changes the training signal without prescribing how the edit is parameterized, it can be applied to different gradient-based KE editors, as evaluated in Section 4. Algorithm 1 The proposed HPSE for UKE. 1: passage c; editing prompt x; student model πθ _θ (initialized from π0 _0); privileged model π⋆≜π0(⋅∣,⋅)π _0(· ,·); thresholds τ,κτ,κ; rounds R; inner steps M; step size η. 2: for r=1,…,Rr=1,…,R do 3: ←HybridRollout(πθ,π⋆,,τ,κ)y← HybridRollout( _θ,π ,x,τ,κ) ⊳ student tokens with privileged step-in, Eq. (2) 4: for M steps do 5: θ←θ−η∇θ^HPSE(θ)θ←θ-η\, _θ J_HPSE(θ) ⊳ distill over the prefixes of y, Eq. (4) 6: end for 7: end for 8: return θ 3.3 Theoretical Analysis We conclude this section with a theoretical analysis of HPSE. The informal theorem below summarizes our main theoretical result: the privileged step-in yields a fact-directed supervision advantage over pure on-policy self-distillation (OPSD). The full set of assumptions, formal version, and proof are provided in Appendix A. Theorem 3.1 (Coverage and supervision advantage; informal). For an edit with a fact span of ℓ novel tokens, the hybrid rollout visits every fact prefix. Under sampling, an OPSD rollout reaches depth j with probability at most e−τje^-τ j; under greedy divergence, it visits only the span entrance. Consequently, the hybrid signal is Ω(ℓ) ( ) while the OPSD signal is O(1)O(1), meaning that their ratio grows at least linearly with ℓ . Thus, the advantage of HPSE becomes more pronounced as the new knowledge spans more tokens, a property particularly relevant to UKE, where edits are conveyed through free-form text. Appendix A provides more in-depth analysis, further analyzing sampled and greedy decoding, clarifying the scope of self-termination and locality, and connecting HPSE to on-policy imitation learning. Under a mild context-separation condition, the same explicit per-state signal separation also holds when the NLL anchor is added to both objectives. 4 Experiments We evaluate the proposed HPSE on two KE methods applied to four language models (LMs) against the composability challenge. Ablation and case studies further probe its behavior. Across diverse scenarios, HPSE consistently improves editing performance. 4.1 Datasets and Experiment Settings Base Models. We conduct experiments on four representative LLMs, Qwen2.5-7B-Instruct (52), Qwen3-8B (53), Llama-3.1-8B-Instruct (15), and Gemma-2-9B-it (18), which have been widely used in the literature (86, 66, 12, 28, 60). For brevity we refer to them as Qwen2.5, Qwen3, Llama3.1, and Gemma2. As is standard for instruction-tuned models, each is prompted with its own chat-template throughout. Tasks. We experiment with the two composability benchmarks detailed in Section 2, which pose a challenging unstructured KE problem. Namely, UnKEBench (12) targets the decomposition ability, and MQuAKE-uns the composition ability (91, 90). When editing an LLM, we consider: (1) Single Editing: one editing request is conducted at a time22 2 A request may carry a variable number of facts, e.g., 2–4 passages per MQuAKE-uns request, which are injected as a batch.. (2) Continual Editing: multiple editing requests are conducted sequentially, which is more demanding due to forgetting and accumulated knowledge conflict (22, 66, 40). Editing Methods. We plug HPSE into two gradient-based KE methods (74, 78): FT-M (86), which fine-tunes the MLP layer that causal tracing identifies as storing the knowledge, and LoRA (26), which learns additive low-rank updates. For better benchmarking, we further compare against five KE methods: MEMIT (45), AlphaEdit (16), AnyEdit (28), UnKE (12), using their official implementation; and COIN⋆ (93), which was reproduced by us due to the lack of official code. To reflect the data-scarcity regime of KE, we focus on methods that do not require large-scale, hard-to-access training data or external powerful models to help editing. This leaves augmentation-based editors (79, 34, 67) out of scope. See Appendix C for implementation details. Evaluation Criteria. We evaluate the KE composability as detailed in Section 2: on UnKEBench, where multiple pieces of knowledge to edit are jointly encoded in a document, we measure the joint recall of all edited knowledge by Jnt. with FActScore (47), and the decomposed recall of each individual fact under targeted questions by Dmp. score. Div. quantifies how distinct the model’s generations are when it recalls different atomic facts. Following 71, 12, we report MMLU as a locality check. On MQuAKE-uns, where each unstructured document only encodes a single piece of new knowledge, and multiple documents are edited in a batched way, Ind. and Cmp. measure the LLM’s individual and compositional recall, as quantified by single- and multi-hop accuracy. We further summarize each benchmark by its editing metric average. See the exact computation in Appendix C.1. Implementation Details. We use official baseline implementations; see Appendix C for details. 4.2 Single Editing Performance Table 1: Single-edit performance, higher is better. COIN⋆ was reimplemented by us due to the lack of official code. Both FT-M and LoRA editors gained improvement from HPSE training paradigm. Qwen2.5 Qwen3 UnKEBench MQuAKE-uns UnKEBench MQuAKE-uns Jnt. Dmp. Div. Avg. MMLU Ind. Cmp. Avg. Jnt. Dmp. Div. Avg. MMLU Ind. Cmp. Avg. MEMIT 19.2 15.3 88.3 40.9 70.3 1.1 6.0 3.5 14.7 16.6 86.1 39.1 59.1 1.0 8.0 4.5 AlphaEdit 35.5 29.6 86.0 50.4 64.5 6.3 9.3 7.8 28.0 27.3 85.0 46.8 30.3 4.2 6.0 5.1 AnyEdit 36.3 30.6 83.2 50.0 67.0 7.7 10.0 8.8 34.0 28.8 84.9 49.2 30.2 6.0 8.0 7.0 UnKE 34.0 25.3 87.1 48.8 68.6 2.5 6.0 4.2 32.3 29.5 84.4 48.7 32.6 3.0 8.0 5.5 COIN⋆ 71.0 53.4 56.5 60.3 65.9 43.9 10.7 27.3 44.0 35.5 81.6 53.7 67.5 21.1 6.7 13.9 FT-M 36.6 23.1 84.2 48.0 70.2 14.2 5.3 9.7 20.7 20.5 86.4 42.5 67.8 9.4 5.3 7.3 + Ours [1pt] +0.0% 43.7 [1pt] +19.2% 30.9 [1pt] +33.5% 87.1 [1pt] +3.4% 53.9 [1pt] +12.2% 70.3 [1pt] +0.2% 29.6 [1pt] +108.5% 8.7 [1pt] +64.2% 19.1 [1pt] +96.4% 27.0 [1pt] +30.4% 30.0 [1pt] +46.5% 87.6 [1pt] +1.4% 48.2 [1pt] +13.3% 67.1 [1pt] -1.1% 17.8 [1pt] +89.4% 6.7 [1pt] +26.4% 12.2 [1pt] +66.7% LoRA 64.1 46.6 63.2 58.0 70.6 73.3 32.0 52.6 66.3 56.1 58.6 60.3 69.0 68.3 41.3 54.8 + Ours [1pt] +0.0% 73.3 [1pt] +14.2% 58.5 [1pt] +25.6% 61.5 [1pt] -2.6% 64.4 [1pt] +11.1% 70.7 [1pt] +0.2% 83.2 [1pt] +13.5% 54.7 [1pt] +70.9% 69.0 [1pt] +31.0% 76.0 [1pt] +14.7% 61.4 [1pt] +9.5% 56.7 [1pt] -3.2% 64.7 [1pt] +7.3% 69.4 [1pt] +0.7% 82.8 [1pt] +21.2% 50.0 [1pt] +21.1% 66.4 [1pt] +21.2% Llama3.1 Gemma2 UnKEBench MQuAKE-uns UnKEBench MQuAKE-uns Jnt. Dmp. Div. Avg. MMLU Ind. Cmp. Avg. Jnt. Dmp. Div. Avg. MMLU Ind. Cmp. Avg. MEMIT 11.9 2.7 63.1 25.9 64.0 0.4 5.3 2.8 33.9 25.4 80.4 46.6 60.7 1.3 4.7 3.0 AlphaEdit 34.1 33.9 80.0 49.4 28.5 9.3 8.7 9.0 35.8 22.6 80.4 46.3 36.1 1.8 4.7 3.2 AnyEdit 50.9 46.3 66.2 54.5 37.4 16.7 6.7 11.7 33.0 28.1 79.9 47.0 60.9 3.2 8.7 5.9 UnKE 41.6 38.5 74.1 51.4 53.3 17.2 7.3 12.2 11.0 6.5 83.2 33.6 70.6 0.0 1.3 0.7 COIN⋆ 87.3 73.4 17.4 59.4 35.0 75.7 12.0 43.9 28.3 18.5 81.3 42.7 70.1 14.3 3.3 8.8 FT-M 71.8 53.6 49.8 58.4 67.6 57.9 28.7 43.3 29.9 19.1 80.4 43.1 72.6 14.4 1.3 7.8 + Ours [1pt] +0.0% 72.5 [1pt] +1.0% 52.1 [1pt] -2.8% 61.1 [1pt] +22.7% 61.9 [1pt] +6.0% 67.6 [1pt] +0.0% 67.5 [1pt] +16.6% 29.3 [1pt] +2.1% 48.4 [1pt] +11.8% 36.0 [1pt] +20.2% 23.4 [1pt] +22.6% 84.4 [1pt] +4.9% 47.9 [1pt] +11.0% 71.9 [1pt] -1.0% 25.6 [1pt] +77.8% 5.3 [1pt] +307.7% 15.4 [1pt] +96.8% LoRA 76.4 60.0 48.1 61.5 65.0 75.8 46.7 61.3 85.5 67.8 32.1 61.8 72.4 79.1 41.3 60.2 + Ours [1pt] +0.0% 75.4 [1pt] -1.2% 61.6 [1pt] +2.8% 61.3 [1pt] +27.3% 66.1 [1pt] +7.5% 65.5 [1pt] +0.9% 81.0 [1pt] +6.9% 50.0 [1pt] +7.1% 65.5 [1pt] +6.9% 83.9 [1pt] -1.9% 66.0 [1pt] -2.6% 53.9 [1pt] +68.1% 68.0 [1pt] +10.0% 72.0 [1pt] -0.6% 81.5 [1pt] +3.0% 46.0 [1pt] +11.4% 63.7 [1pt] +5.9% We first evaluated HPSE in the Single Editing setting. The results are reported in Table 1. Overall Performance Gain. From the table, FT-M and LoRA achieved consistent benchmark-average gains by replacing their training paradigm with HPSE. The improvement held across all 16 editor–LLM–benchmark combinations, with only occasional dips on individual metrics (all within two points), confirming the universality of our method. Specifically, FT-M obtained average gains of +6.8 and +5.0 points on MQuAKE-uns and UnKEBench, respectively (relative gains averaging +67.9% and +10.6% across the four LLMs). With HPSE, FT-M further outperformed all five baselines in 4 out of 8 cases. LoRA showed the same trend, gaining +8.9 and +5.4 points respectively, and even beat all baselines in one case where it could not on its own. We next examined how HPSE addresses the composability requirement, with the following observations. HPSE injects decomposable knowledge. According to Table 1, while COIN⋆ (on Qwen2.5 and Llama3.1) and LoRA (base version) reached high Dmp. scores by answering the targeted questions, they did so by regurgitating their lengthy Jnt. answers, as reflected by a low Div. On the other hand, other baselines (AnyEdit, UnKE, etc.) achieved high diversity, but their low Dmp. (and Jnt.) performance indicates an editing failure. This contextual reliance is consistent with the literature 85, 51, 39, 93. In contrast, HPSE injects the knowledge in a decomposable way, leading to Jnt. and Dmp. improvement while largely preserving Div. More importantly, this came without compromising locality, as indicated by a stable MMLU score across four LLM backbones. HPSE injects composable knowledge. For the composition requirement, HPSE again delivered consistent improvements. From the table, most baselines scored low on both Ind. and Cmp. COIN⋆ reached a high Ind. yet fell far short on Cmp. (e.g., 75.7 Ind. vs 12.0 Cmp. on Llama3.1), indicating a drastic performance gap between recalling the new knowledge and leveraging it for composition. These results confirm the difficulty of knowledge composition (91). On this hard task, HPSE improved Cmp., by +2.3 points on average for FT-M and +9.9 points (up to +70.9%) for LoRA, without sacrificing Ind. efficacy. These empirical results collectively confirm the effectiveness of HPSE as a general plug-and-play module that can benefit existing gradient-based KE methods toward better composability. 4.3 Continual Editing Performance Figure 4: Continual-edit performance under different sequence length T, higher is better. Solid and transparent bars show performance with and without HPSE. Unfilled area marks the performance gap. We next studied the more demanding Continual Editing setting, where T editing requests are injected sequentially before the evaluation. Following the literature (66, 40), we accumulated T edits and measured the average score. Given the budget constraints, we evaluated each benchmark on two LLMs (Qwen2.5/Gemma2 for UnKEBench; and Llama3.1/Qwen3 for MQuAKE-uns). Due to the page limit, we defer the complete numeric results to Appendix D.1 and report the average in Figure 4. For FT-M and LoRA, we used filled boxes to mark the base-version performance, and outlines drawn on top to represent that under HPSE. The unfilled area quantifies HPSE’s improvement. As in the single-edit scenario, HPSE again improved the two KE editors across sequence lengths T and LLMs, uniformly for LoRA and with only three exceptions for FT-M, each within one point on average. Notably, HPSE involves no design specific to continual editing, so this advantage reflects the robustness of its training paradigm as edits accumulated. To see this, note that on the MQuAKE-uns benchmark, LoRA improved its average score by +55% to +149% (relative) across all T on both LLMs, more than doubling it in half of the settings. Moreover, the gain can be large enough to make HPSE competitive with the best dedicated editors: it widened LoRA’s margin over all baselines at every T on Qwen3 (e.g., 10.4 vs. 5.5 average at T=50T=50, where base LoRA led only marginally), and lifted FT-M above all baselines at 3 out of 4 horizons on Gemma2. As two different editors reached the top, this improvement was not tied to one particular editor. To conclude, these results demonstrate the benefits of HPSE in diverse KE scenarios. 4.4 Ablation Studies Table 2: Ablation study on HPSE (Qwen2.5). UnKEBench (subset) MQuAKE-uns Jnt. Dmp. Div. Avg. Ind. Cmp. Avg. LoRA 63.2 47.8 63.0 58.0 73.3 32.0 52.6 w/o both 58.5 56.6 72.3 62.5 65.4 33.3 49.4 w/o HP 72.5 62.6 62.0 65.7 81.7 51.3 66.5 w/o NLL 74.0 67.1 60.2 67.1 82.1 52.0 67.1 Ours 75.0 62.5 60.9 66.1 83.2 54.7 68.9 We ablated HPSE to see how each component contributes to the final performance. We report single editing on Qwen2.5 with LoRA in Table 2, using a randomly selected subset. According to the table, removing both components (“w/o both”) reduces HPSE to standard OPSD, which fell below even base LoRA on MQuAKE-uns (49.4 vs. 52.6 Avg.), echoing the coverage failure identified in Section 3. Adding either component substantially improved OPSD, with the hybrid-policy (HP) rollout alone (“w/o NLL”) yielding a higher average than the NLL anchor alone (“w/o HP”) on both benchmarks. More importantly, with the same NLL anchor, replacing the hybrid rollout with the on-policy rollout lowered Jnt. by 2.5 points and Cmp. by 3.4 points, and this gap further widened under continual editing (to 6.3 and 6.0 points at T=10T=10, see Appendix D.3). The NLL anchor provided complementary grounding: adding it to HP improved MQuAKE-uns by 1.8 points on average, where composition depends on retaining every individually edited fact, while reducing UnKEBench by 1.0 point through lower Dmp. Additional sensitivity analysis on the gates τ and κ are in Appendix D.2, where both gates exhibit notable robustness. Figure 5: Privileged step-in dynamics (UnKEBench, Qwen2.5). The student stops relying on the step-in within a few rounds. We further examined how the privileged information enters HPSE’s training over rounds. To this end, we tracked the step-in frequency per sampling turn. Results are shown in Figure 5. The initial step-in rate of 26.8% at the first outer round reflects a relatively frequent intervention, showing that the student’s own trajectory initially failed to cover many facts. More importantly, the sharp drop in the step-in rate (26.8% to 1.7% within 3 rounds) indicates that HPSE quickly internalizes the injected knowledge into the student, thereby reducing its reliance on privileged intervention as editing proceeds. This aligns well with our analysis in Section 3. 4.5 Case Studies Composition (MQuAKE-uns) — Qwen2.5 Injected edits: Louis XV’s religion → Christian Science; founder of Christian Science → Ibn Saud. Multi-hop question: What is the name of the person who formed the religion that Louis XV of France practiced? Gold: Ibn Saud. HPSE ✓ …Louis XV …Christian Science…founded by Abdulaziz ibn Abdul Rahman Al Saud, …known as Ibn Saud… LoRA ✗ …associated with Christian Science…The founder is generally recognized as the Parisian mystic Madame de Valois… COIN⋆ ✗ Louis XIV, the King of France, was a significant figure in the patronage of the Catholic Church… MEMIT ✗ Louis XV followed Roman Catholicism…its founder is traditionally attributed to Saint Peter… AlphaEdit ✗ Louis XV adhered to the Catholic Church, established under the auspices of Saint Louis IX of France… UnKE ✗ The religion followed by Louis XV was Roman Catholicism…a prominent figure within the Catholic Church… AnyEdit ✗ Louis XV was a prominent figure in French Catholicism…the Jansenist movement… Figure 6: Composition cases with error highlighting. Baselines hallucinated or reverted to prior knowledge. Decomposition (UnKEBench) — Llama Edited passage (injected as training text): Elvin Penner is a well-known songwriter in the music industry. He has written several hit songs for popular artists such as Taylor Swift and Ed Sheeran. In an interview with Rolling Stone magazine, Penner discussed his creative process and how he draws inspiration from his personal experiences. He also mentioned that he has been writing songs since he was a teenager…including a Grammy for Song of the Year. Sub-question: Which popular artists has Elvin Penner written hit songs for? Gold: Taylor Swift and Ed Sheeran. HPSE ✓ Elvin Penner has written hit songs for popular artists such as Taylor Swift and Ed Sheeran. LoRA ✗ Elvin Penner is a well-known songwriter in the music industry. He has written several hit songs for Taylor Swift and Ed Sheeran. In an interview with Rolling Stone magazine, Penner discussed his creative process…(passage verbatim) COIN⋆ ✗ Elvin Penner is a well-known songwriter in the music industry. He has written…(passage verbatim) AnyEdit ✗ Elvin Penner is a well-known songwriter in the music industry. He has written…(passage verbatim) AlphaEdit ✗ …hit songs for popular artists like Ariana Grande and Taylor Swift…since he was 15… UnKE ✗ Elvin Penner is a well-known musician in Jamaica. He has written hit songs for Sean Paul and Shaggy… MEMIT ✗ I’m not aware of any information about an artist named Elvin Penner writing hit songs… Figure 7: Decomposition cases with error highlighting. Baselines repeated, hallucinated, or reverted to prior knowledge. We conclude this section with case studies on how HPSE succeeded at composable KE whereas baselines failed. Due to the page limit, we defer further examples to Appendix D.4. Figure 6 shows a composition case: two edits injected via separate untargeted passages that must be combined to answer a targeted two-hop question. Most baselines failed to even recall the first hop. Base LoRA passed this step, but failed to compose and hallucinated. With HPSE, it succeeded in chaining the two. Figure 7 shows a decomposition case, where a passage encoding five atomic facts is injected and then probed with a specific atomic question. The baselines failed in three different ways, either repeating the passage verbatim (indicated by low Div. as exemplified in Table 1), hallucinating, or falling back to the pre-edited knowledge. In contrast, HPSE decomposed the edited knowledge and answered the sub-question directly. These cases demonstrate how HPSE injects new knowledge that is at once composable and decomposable. 5 Related Works Existing KE methods broadly fall into two storage paradigms (86, 68). Internal storage KE writes the new knowledge directly into the model parameters. The locate-then-edit (LTE) paradigm (44, 46, 36, 21, 25, 19, 16, 43) first localizes the weights responsible for a fact and then applies a targeted update to them. Alternatively, PEFT methods such as LoRA (26, 65) and ReFT (72, 40) edit competitively by training a small set of additional parameters. Recent efforts extend these ideas to unstructured KE (UKE, 71), where the edit is a document-level passage rather than a curated triple. To this end, 28, 60, 93 chunk the document and then edit them one by one (28); and 12 edit a larger set of model parameters for better capacities. Across these lines, the common focus is on isolating a small, knowledge-relevant subset of weights to update. Yet they primarily supervise the edit passively with a single fixed statement, which induces severe overfitting (85, 42, 51, 33, 39, 93) and undermines composability of UKE. In contrast, HPSE provides a data-centric alternative: it proactively constructs its own training signal rather than passively fitting a fixed statement. HPSE makes no assumption about which parameters are updated, complementing these editors and integrating with them seamlessly while preserving their locality. External storage KE instead holds the edit in an auxiliary memory, leaving the base parameters frozen, via meta-learning (48, 63, 87, 38), retrieval-augmented generation (87, 29, 70, 69, 6), and routing over separately learned weights or module copies (11, 49, 22, 66, 82). Recent works extend these ideas to UKE by augmenting the editing with atomic facts (67). However, these methods rely on large, often hard-to-access datasets to build the retrieval store or to train the auxiliary models, which limits their applicability (66). By contrast, HPSE is augmentation-free: it improves composability without any external data or auxiliary model, offering a self-contained and affordable path to practical KE. On-policy Distillation and its self-distillation variant (OPSD) have emerged as an efficient and generalizable paradigm for LLM post-training (1, 20, 41, 59, 37, 88). By supervising the student on its own rollouts, OP(S)D receives dense, token-level feedback from the teacher (58, 76), with follow-ups strengthening the reliability of the teacher’s signal (80, 30, 83). Yet its gains lie mainly in reshaping behaviors the model can already produce, rather than installing genuinely new knowledge (17, 23, 84). Several recent works tackle related coverage issues: HDPO (13) replaces entire rollouts on unsolvable RL prompts, filtered by verifiable rewards; CODE (35) brings OPSD to KE, but conditions its privileged teacher on causal narratives synthesized by an external frontier model and focuses on structured KE; SKD (75) interleaves teacher tokens into student sampling for general-purpose distillation, using an external teacher to replace student-proposed tokens that fall outside the teacher’s top-K set. In contrast, HPSE performs token-level gated step-in using a privileged state of the model itself, thereby targeting composability in untargeted UKE without a verifier, an external teacher, or external synthesis. 6 Conclusion In this work, we study composability in unstructured knowledge editing (UKE), the requirement that an injected edit be both decomposed into its individual facts and composed into multi-hop reasoning. Formulating UKE in an untargeted regime that mirrors practical editing requests, we recast two benchmarks into composability probes and show that existing editors, which passively rely on a fixed passage as their sole learning source, largely fall short of this requirement. In response, we propose HPSE, which casts editing as proactive self-distillation from a privileged in-context state of the same model and repairs the coverage gap of pure on-policy distillation through a hybrid rollout that places missing facts onto the student’s own trajectory. HPSE provides a plug-and-play improvement for existing gradient-based KE editors, equipping them to inject knowledge that is both decomposable and composable. Extensive experiments demonstrate the effectiveness of our method. In the future, we plan to generalize HPSE toward lifelong and multimodal editing, enabling it to encode accumulated edits rather than a single passage, and to operate beyond the text-token space, thereby paving the way toward more practical UKE. References Agarwal et al. (2024) R. Agarwal, N. Vieillard, Y. Zhou, P. Stanczyk, S. Ramos, M. Geist, and O. Bachem On-policy distillation of language models: learning from self-generated mistakes. In International Conference on Learning Representations (ICLR), Cited by: §A.4, Remark A.2, §5. Bengio et al. (2000) Y. Bengio, R. Ducharme, and P. Vincent A neural probabilistic language model. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §2.1. Bommasani et al. (2021) R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. Cited by: §1. Brown et al. (2020) T. B. Brown, B. Mann, N. Ryder, M. Subbiah, et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1. Bubeck et al. (2023) S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, et al. Sparks of artificial general intelligence: early experiments with GPT-4. arXiv preprint arXiv:2303.12712. Cited by: §1. Chen et al. (2025) Q. Chen, T. Zhang, X. He, D. Li, C. Wang, L. Huang, and H. Xue Lifelong knowledge editing for llms with retrieval-augmented continuous prompt learning. External Links: 2405.03279 Cited by: §5. Cho et al. (2014) K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: §2.1. Chu et al. (2025) T. Chu, Y. Zhai, J. Yang, S. Tong, S. Xie, D. Schuurmans, Q. V. Le, S. Levine, and Y. Ma SFT memorizes, RL generalizes: a comparative study of foundation model post-training. In Forty-second International Conference on Machine Learning, Cited by: §1, §1. Cover (1999) T. M. Cover Elements of information theory. John Wiley & Sons. Cited by: §A.2. Dai et al. (2021) D. Dai, L. Dong, Y. Hao, Z. Sui, B. Chang, and F. Wei Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08696. Cited by: §1. De Cao et al. (2021) N. De Cao, W. Aziz, and I. Titov Editing factual knowledge in language models. arXiv preprint arXiv:2104.08164. Cited by: §1, §5. Deng et al. (2024) J. Deng, L. Pang, H. Shen, and X. Cheng UnKE: unstructured knowledge editing in large language models. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: 2nd item, 4th item, §C.2, §1, §1, 1st item, 1st item, §2.2, §2.2, §4.1, §4.1, §4.1, §4.1, §5. Ding (2026) K. Ding HDPO: hybrid distillation policy optimization via privileged self-distillation. arXiv preprint arXiv:2603.23871. Cited by: §A.4, §5. Dong et al. (2022) Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, and Z. Sui A survey for in-context learning. arXiv preprint arXiv:2301.00234. Cited by: §1. Dubey et al. (2024) A. Dubey et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. Cited by: Table 3, §4.1. Fang et al. (2025) J. Fang, H. Jiang, K. Wang, Y. Ma, J. Shi, X. Wang, X. He, and T. Chua AlphaEdit: null-space constrained model editing for language models. In International Conference on Learning Representations (ICLR), Cited by: §C.2, §1, §4.1, §5. Gandhi et al. (2025) K. Gandhi, A. K. Chakravarthy, A. Singh, N. Lile, and N. Goodman Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective STars. In Second Conference on Language Modeling, Cited by: §A.4, §1, §5. Gemma Team (2024) Gemma Team Gemma 2: improving open language models at a practical size. arXiv preprint arXiv:2408.00118. Cited by: Table 3, §4.1. Gu et al. (2024a) J. Gu, H. Xu, J. Ma, P. Lu, Z. Ling, K. Chang, and N. Peng Model editing harms general abilities of large language models: regularization to the rescue. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 16801–16819. Cited by: §5. Gu et al. (2024b) Y. Gu, L. Dong, F. Wei, and M. Huang MiniLLM: knowledge distillation of large language models. In International Conference on Learning Representations (ICLR), Cited by: §5. Gupta et al. (2024) A. Gupta, D. Sajnani, and G. Anumanchipalli A unified framework for model editing. In Findings of the Association for Computational Linguistics: EMNLP 2024, p. 15403–15418. Cited by: §5. Hartvigsen et al. (2023) T. Hartvigsen, S. Sankaranarayanan, H. Palangi, Y. Kim, and M. Ghassemi Aging with GRACE: lifelong model editing with discrete key-value adaptors. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §4.1, §5. He et al. (2025) A. W. He, D. Fried, and S. Welleck Rewarding the unlikely: lifting grpo beyond distribution sharpening. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 25559–25571. Cited by: §5. Hochreiter and Schmidhuber (1997) S. Hochreiter and J. Schmidhuber Long short-term memory. Neural Computation 9 (8), p. 1735–1780. Cited by: §2.1. Hu et al. (2024) C. Hu, P. Cao, Y. Chen, K. Liu, and J. Zhao WilKE: wise-layer knowledge editor for lifelong knowledge editing. External Links: 2402.10987 Cited by: §5. Hu et al. (2022) E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), Cited by: §4.1, §5. Ji et al. (2023) Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung Survey of hallucination in natural language generation. ACM Computing Surveys 55 (12), p. 1–38. Cited by: §1. Jiang et al. (2025) H. Jiang, J. Fang, N. Zhang, G. Ma, M. Wan, X. Wang, X. He, and T. Chua Edit any knowledge encoded in language models. arXiv preprint arXiv:2502.05628. Cited by: §C.2, §1, §2.1, §2.2, §4.1, §4.1, §5. Jiang et al. (2024) Y. Jiang, Y. Wang, C. Wu, W. Zhong, X. Zeng, J. Gao, L. Li, X. Jiang, L. Shang, R. Tang, et al. Learning to edit: aligning llms with knowledge editing. arXiv preprint arXiv:2402.11905. Cited by: §5. Jin et al. (2026) W. Jin, T. Min, Y. Yang, D. Wei, Y. Zhou, S. R. Kadhe, N. Baracaldo, and K. Lee Entropy-aware on-policy distillation of language models. In Forty-third International Conference on Machine Learning, Cited by: §3.2, §5. Kakade and Langford (2002) S. Kakade and J. Langford Approximately optimal approximate reinforcement learning. In Proceedings of the Nineteenth International Conference on Machine Learning, ICML ’02, San Francisco, CA, USA, p. 267–274. External Links: ISBN 1558608737 Cited by: §A.4. Kojima et al. (2022) T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa Large language models are zero-shot reasoners. Advances in neural information processing systems 35, p. 22199–22213. Cited by: §1. Lampinen et al. (2025) A. K. Lampinen, A. Chaudhry, S. C. Chan, C. Wild, D. Wan, A. Ku, J. Bornschein, R. Pascanu, M. Shanahan, and J. L. McClelland On the generalization of language models from in-context learning and finetuning: a controlled study. arXiv preprint arXiv:2505.00661. Cited by: Remark A.1, §5. Lee et al. (2025) J. Lee, D. Jung, and H. Lim StepKE: stepwise knowledge editing for multi-hop question answering. In Findings of the Association for Computational Linguistics: EMNLP 2025, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), p. 7752–7765. Cited by: §C.2, §4.1. Li et al. (2026a) S. Li, K. Zhang, X. Wang, J. Liu, and S. Mo From fact overwriting to knowledge evolution: causal editing via on-policy self-distillation. External Links: 2605.28303 Cited by: §5. Li et al. (2024) X. Li, S. Li, S. Song, J. Yang, J. Ma, and J. Yu Pmet: precise model editing in a transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, p. 18564–18572. Cited by: §5. Li et al. (2026b) Y. Li, Y. Zuo, B. He, J. Zhang, C. Xiao, C. Qian, T. Yu, H. Gao, W. Yang, Z. Liu, and N. Ding Rethinking on-policy distillation of large language models: phenomenology, mechanism, and recipe. External Links: 2604.13016 Cited by: §3.1, §3.1, §5. Li et al. (2025) Z. Li, H. Jiang, H. Chen, B. Bi, Z. Zhou, F. Sun, J. Fang, and X. Wang Reinforced lifelong editing for language models. In Forty-second International Conference on Machine Learning, Cited by: §5. Liu et al. (2025a) T. Liu, R. Li, Z. Dong, H. Liu, X. Tang, Q. Yin, L. Zhang, H. Wang, and J. Gao Mitigating heterogeneous token overfitting in LLM knowledge editing. In Forty-second International Conference on Machine Learning, Cited by: §1, §4.2, §5. Liu et al. (2025b) T. Liu, R. Li, Y. Qi, H. Liu, X. Tang, T. Zheng, Q. Yin, M. X. Cheng, J. Huan, H. Wang, and J. Gao Unlocking efficient, scalable, and continual knowledge editing with basis-level representation fine-tuning. In The Thirteenth International Conference on Learning Representations, Cited by: §1, §4.1, §4.3, §5. Lu and Lab (2025) K. Lu and T. M. Lab On-policy distillation. Thinking Machines Lab: Connectionism. Note: https://thinkingmachines.ai/blog/on-policy-distillation External Links: Document Cited by: §A.4, §1, §5. Ma et al. (2024) J. Ma, Z. Ling, N. Zhang, and J. Gu Neighboring perturbations of knowledge editing on large language models. arXiv preprint arXiv:2401.17623. Cited by: §5. Ma et al. (2025) J. Ma, H. Wang, H. Xu, Z. Ling, and J. Gu Perturbation-restrained sequential model editing. External Links: 2405.16821 Cited by: §5. Meng et al. (2022a) K. Meng, D. Bau, A. Andonian, and Y. Belinkov Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems 35, p. 17359–17372. Cited by: §5. Meng et al. (2023) K. Meng, A. S. Sharma, A. J. Andonian, Y. Belinkov, and D. Bau Mass-editing memory in a transformer. In International Conference on Learning Representations (ICLR), Cited by: §C.2, §1, §4.1. Meng et al. (2022b) K. Meng, A. S. Sharma, A. Andonian, Y. Belinkov, and D. Bau Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07229. Cited by: §5. Min et al. (2023) S. Min, K. Krishna, X. Lyu, M. Lewis, W. Yih, P. W. Koh, M. Iyyer, L. Zettlemoyer, and H. Hajishirzi FActScore: fine-grained atomic evaluation of factual precision in long form text generation. In Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: 2nd item, 1st item, §4.1. Mitchell et al. (2021) E. Mitchell, C. Lin, A. Bosselut, C. Finn, and C. D. Manning Fast model editing at scale. arXiv preprint arXiv:2110.11309. Cited by: §5. Mitchell et al. (2022) E. Mitchell, C. Lin, A. Bosselut, C. D. Manning, and C. Finn Memory-based model editing at scale. In International Conference on Machine Learning (ICML), Cited by: §5. Ouyang et al. (2022) L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, et al. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1. Qi et al. (2025) S. Qi, B. Yang, K. Jiang, X. Wang, J. Li, Y. Zhong, Y. Yang, and Z. Zheng In-context editing: learning knowledge from self-induced distributions. In International Conference on Learning Representations (ICLR), Cited by: §4.2, §5. Qwen Team (2024) Qwen Team Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. Cited by: Table 3, §2.2, §4.1. Qwen Team (2025) Qwen Team Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: Table 3, §4.1. Raffel et al. (2020) C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research 21 (1), p. 5485–5551. Cited by: §1. Rajaraman et al. (2020) N. Rajaraman, L. Yang, J. Jiao, and K. Ramchandran Toward the fundamental limits of imitation learning. Advances in Neural Information Processing Systems 33, p. 2914–2924. Cited by: §A.4. Ross and Bagnell (2010) S. Ross and J. A. Bagnell Efficient reductions for imitation learning. Cited by: §A.4. Ross et al. (2011) S. Ross, G. Gordon, and D. Bagnell A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, p. 627–635. Cited by: §A.4. Shenfeld et al. (2026) I. Shenfeld, M. Damani, J. Hübotter, and P. Agrawal Self-distillation enables continual learning. External Links: 2601.19897 Cited by: §5. Song and Zheng (2026) M. Song and M. Zheng A survey of on-policy distillation for large language models. External Links: 2604.00626 Cited by: §A.4, §1, §3.1, §5. Su et al. (2025) Z. Su, Z. Huang, K. Zhang, and X. Zhang $μ$KE: matryoshka unstructured knowledge editing of large language models. In Second Conference on Language Modeling, Cited by: §1, §4.1, §5. Sutskever et al. (2014) I. Sutskever, O. Vinyals, and Q. V. Le Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §2.1. Swamy et al. (2021) G. Swamy, S. Choudhury, J. A. Bagnell, and S. Wu Of moments and matching: a game-theoretic framework for closing the imitation gap. In International Conference on Machine Learning, p. 10022–10032. Cited by: §A.4. Tan et al. (2024) C. Tan, G. Zhang, and J. Fu Massive editing for large language models via meta learning. In The Twelfth International Conference on Learning Representations, Cited by: §5. Vaswani et al. (2017) A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1. Wang et al. (2024a) H. Wang, T. Liu, R. Li, M. Cheng, T. Zhao, and J. Gao RoseLoRA: row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning. External Links: 2406.10777 Cited by: §1, §5. Wang et al. (2024b) P. Wang, Z. Li, N. Zhang, Z. Xu, Y. Yao, Y. Jiang, P. Xie, F. Huang, and H. Chen WISE: rethinking the knowledge memory for lifelong model editing of large language models. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §4.1, §4.1, §4.3, §5. Wang et al. (2026) P. Wang, B. Zhou, X. Tang, J. Han, and S. Hu FABLE: fine-grained fact anchoring for unstructured model editing. In Findings of the Association for Computational Linguistics: ACL 2026, San Diego, California, United States. Cited by: §C.2, §1, §1, 1st item, §3.1, §4.1, §5. Wang et al. (2023) S. Wang, Y. Zhu, H. Liu, Z. Zheng, C. Chen, and J. Li Knowledge editing for large language models: a survey. arXiv preprint arXiv:2310.16218. Cited by: §1, §2.1, §5. Wang et al. (2024c) Y. Wang, M. Chen, N. Peng, and K. Chang DeepEdit: knowledge editing as decoding with constraints. External Links: 2401.10471 Cited by: 2nd item, §5. Wei et al. (2024) Z. Wei, L. Pang, H. Ding, J. Deng, H. Shen, and X. Cheng Stable knowledge editing in large language models. arXiv preprint arXiv:2402.13048. Cited by: §5. Wu et al. (2024a) X. Wu, L. Pan, W. Y. Wang, and A. T. Luu AKEW: assessing knowledge editing in the wild. In Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: 1st item, 2nd item, §1, 2nd item, §2.1, §4.1, §5. Wu et al. (2024b) Z. Wu, A. Arora, Z. Wang, A. Geiger, D. Jurafsky, C. D. Manning, and C. Potts Reft: representation finetuning for language models. arXiv preprint arXiv:2404.03592. Cited by: §5. Xie et al. (2021) S. M. Xie, A. Raghunathan, P. Liang, and T. Ma An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXiv:2111.02080. Cited by: Remark A.1. Xiong et al. (2025) H. Xiong, C. Tan, and W. Chen Is fine-tuning an effective solution? reassessing knowledge editing for unstructured data. External Links: 2506.09672 Cited by: §4.1. Xu et al. (2025) W. Xu, R. Han, Z. Wang, L. T. Le, D. Madeka, L. Li, W. Y. Wang, R. Agarwal, C. Lee, and T. Pfister Speculative knowledge distillation: bridging the teacher-student gap through interleaved sampling. In International Conference on Learning Representations (ICLR), Cited by: §5. Xu et al. (2026) Y. Xu, H. Sang, Z. Zhou, R. He, Z. Wang, and A. Geramifard Beyond grpo and on-policy distillation: an empirical sparse-to-dense reward principle for language-model post-training. External Links: 2605.12483 Cited by: §5. Yang et al. (2025) W. Yang, F. Sun, J. Tan, X. Ma, Q. Cao, D. Yin, H. Shen, and X. Cheng The mirage of model editing: revisiting evaluation in the wild. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vienna, Austria. Cited by: 2nd item, 1st item, §2.1. Yang et al. (2026) W. Yang, R. Tang, H. Zang, D. Su, Q. Cao, J. Wang, H. Shen, X. Cheng, and F. Sun Fine-tuning done right in model editing. In The Fourteenth International Conference on Learning Representations, Cited by: §4.1. Yao et al. (2025) Y. Yao, J. Fang, J. Gu, N. Zhang, S. Deng, H. Chen, and N. Peng CaKE: circuit-aware editing enables generalizable knowledge learners. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), p. 11366–11382. Cited by: §C.2, §1, §1, §3.1, §4.1. Ye et al. (2026) T. Ye, L. Dong, X. Wu, S. Huang, and F. Wei On-policy context distillation for language models. arXiv preprint arXiv:2602.12275. Cited by: §A.4, §5. Yenduri et al. (2023) G. Yenduri, R. M, C. S. G, S. Y, G. Srivastava, P. K. R. Maddikunta, D. R. G, R. H. Jhaveri, P. B, W. Wang, A. V. Vasilakos, and T. R. Gadekallu Generative pre-trained transformer: a comprehensive review on enabling technologies, potential applications, emerging challenges, and future directions. External Links: 2305.10435 Cited by: §2.1. Yu et al. (2024) L. Yu, Q. Chen, J. Zhou, and L. He Melo: enhancing model editing with neuron-indexed dynamic lora. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, p. 19449–19457. Cited by: §5. Yu et al. (2026) X. Yu, G. Li, Q. Si, G. Zhang, Y. Xu, C. Wang, S. Dong, K. Tuo, X. Zeng, K. Feng, Q. Wang, Y. Shi, X. Hu, X. Yue, J. Wang, and S. Yan DOPD: dual on-policy distillation. External Links: 2606.30626 Cited by: §5. Yue et al. (2026) Y. Yue, Z. Chen, R. Lu, A. Zhao, Z. Wang, Y. Yue, S. Song, and G. Huang Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, Cited by: §A.4, §1, §5. Zhang et al. (2024a) M. Zhang, X. Ye, Q. Liu, P. Ren, S. Wu, and Z. Chen Uncovering overfitting in large language model editing. arXiv preprint arXiv:2410.07819. Cited by: §1, §1, §2.2, §4.2, §5. Zhang et al. (2024) N. Zhang et al. A comprehensive study of knowledge editing for large language models. arXiv preprint arXiv:2401.01286. Cited by: §1, §2.1, §4.1, §4.1, §5. Zhang et al. (2024b) N. Zhang, B. Tian, S. Cheng, X. Liang, Y. Hu, K. Xue, Y. Gou, X. Chen, and H. Chen Instructedit: instruction-based knowledge editing for large language models. arXiv preprint arXiv:2402.16123. Cited by: §5. Zhao et al. (2026) S. Zhao, Z. Xie, M. Liu, J. Huang, G. Pang, F. Chen, and A. Grover Self-distilled reasoner: on-policy self-distillation for large language models. External Links: 2601.18734 Cited by: §1, §3.1, §5. Zheng et al. (2023) C. Zheng, L. Li, Q. Dong, Y. Fan, Z. Wu, J. Xu, and B. Chang Can we edit factual knowledge by in-context learning?. In Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: §3.1. Zhong et al. (2025) S. Zhong, Y. Lu, L. Shao, B. Bhushanam, X. Du, Y. Wan, Y. Shi, D. Zha, Y. Wang, N. Liu, K. Zhou, S. Xu, K. Chang, L. Feng, V. Chaudhary, and X. Hu MQuAKE-remastered: multi-hop knowledge editing can only be advanced with reliable evaluations. In The Thirteenth International Conference on Learning Representations, Cited by: 2nd item, §1, §1, 2nd item, 2nd item, §2.2, §4.1. Zhong et al. (2023) Z. Zhong, Z. Wu, C. D. Manning, C. Potts, and D. Chen MQuAKE: assessing knowledge editing in language models via multi-hop questions. In Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: 2nd item, §1, 2nd item, 2nd item, §2.2, §4.1, §4.2. Zhou et al. (2023) C. Zhou, Q. Li, C. Li, J. Yu, Y. Liu, G. Wang, K. Zhang, C. Ji, Q. Yan, L. He, et al. A comprehensive survey on pretrained foundation models: a history from bert to chatgpt. arXiv preprint arXiv:2302.09419. Cited by: §1. Zhou et al. (2026) Z. Zhou, M. Zhang, S. Wu, X. Ye, C. Zhang, Z. Chen, and P. Ren Uncovering context reliance in unstructured knowledge editing. arXiv preprint arXiv:2602.19043. Cited by: §C.2, §1, §1, 1st item, §2.2, §2.2, §3.1, §4.1, §4.2, §5. Zhu et al. (2018) Y. Zhu, S. Lu, L. Zheng, J. Guo, W. Zhang, J. Wang, and Y. Yu Texygen: a benchmarking platform for text generation models. In International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), Cited by: 3rd item. Appendix A Omitted Theoretical Analysis In this section we develop the analysis behind Theorem 3.1 that was omitted in the main body due to page limits. We first introduce the notation used in the analysis, then present the main theoretical results and properties of the method, and finally connect HPSE to on-policy imitation learning. A.1 Notations For completeness, we first collect the notation that will be used in our analysis. For discrete distributions p,qp,q over the vocabulary V, the (forward) Kullback–Leibler divergence is DKL[p∥q]≜∑v∈p(v)logp(v)q(v).D_KL[p\|q]\; \; _v p(v) p(v)q(v). This is the divergence appearing in the distillation loss of Eq. (1). For scalars a,b∈(0,1)a,b∈(0,1), the binary KL between the Bernoulli distributions (a,1−a)(a,1-a) and (b,1−b)(b,1-b) is dbin(a∥b)≜alogab+(1−a)log1−a1−b.d_bin(a\|b)\; \;a ab+(1-a) 1-a1-b. We write argmaxvp(v) _vp(v) for the mode of p, |||V| for the vocabulary size, and DmaxD_ for a uniform upper bound on the per-token KL (Assumption A.3). Recall from Section 3 the student πθ _θ and the privileged teacher π⋆(⋅∣,<t)≜π0(⋅∣,,<t).π (· ,y_<t)\; \; _0(· ,x,y_<t). At step t, define the teacher’s greedy token by yt⋆≜argmaxv∈π⋆(v∣,<t).y _t\; \; _v π (v ,y_<t). The hybrid rollout policy πρ _ρ of Eq. (2) emits yt⋆y _t when both conditions of the step-in gate in Eq. (3) hold: logπ⋆(yt⋆∣⋅)−logπθ(yt⋆∣⋅)>τ,π⋆(yt⋆∣⋅)>κ. π (y _t ·)- _θ(y _t ·)>τ, π (y _t ·)>κ. Otherwise, it emits the student’s token. We call a run of ℓ consecutive positions carrying the injected knowledge a fact span, and write FjF_j (j=0,…,ℓ−1j=0,…, -1) for the prefix whose next token is the (j+1)(j+1)-th fact token yj+1⋆y _j+1, with F=F0,…,Fℓ−1F=\F_0,…,F_ -1\. The deep coverage and fact-signal of a rollout policy μ (introduced informally in Theorem 3.1) are cj(μ) c_j(μ) ≜Pr∼μ[ reproduces y1⋆,…,yj⋆], _y μ [\,y reproduces y _1,…,y _j\, ], (5) Sμ S_μ ≜∼μ[∑t:<t∈FDKL[π⋆(⋅∣,<t)∥πθ(⋅∣,<t)]]. _y μ [\,Σ _t:\,y_<t∈ FD_KL [π (· ,y_<t)\, \|\, _θ(· ,y_<t) ] ]. (6) Since FjF_j is visited exactly when the first j fact tokens are reproduced—an event of probability cj(μ)c_j(μ)—the fact-signal decomposes as Sμ=∑j=0ℓ−1cj(μ)DKL[πFj⋆∥πθ,Fj],πFj⋆≜π⋆(⋅∣,Fj),πθ,Fj≜πθ(⋅∣,Fj). S_μ\;=\; _j=0 -1c_j(μ)\,D_KL [π _F_j\, \|\, _θ,F_j ], π _F_j π (· ,F_j), _θ,F_j _θ(· ,F_j). (7) A.2 Main Result: Signal Separation Given the notation introduced above, we now present our main theoretical result. Our analysis is built upon the following three assumptions. Assumption A.1 (Novel edit). The edit introduces a length-ℓ run of new-knowledge tokens y1⋆,…,yℓ⋆y _1,…,y _ that the pre-edit student lacks but the in-context teacher supplies. At every fact prefix FjF_j, πθ(yj+1⋆∣Fj)≤ρ,π⋆(yj+1⋆∣Fj)>κ. _θ(y _j+1 F_j)≤ρ, π (y _j+1 F_j)>κ. Moreover, the edit is τ-detectable: ρ<κe−τ.ρ<κ e^-τ. Remark A.1. These conditions describe a non-trivial edit. The student cannot yet produce the new tokens, while access to the passage makes them a confident continuation for the teacher. The teacher-side condition is the standard in-context learning premise (73, 33). Together, the conditions formalize the coverage failure studied in Section 2 and Figure 3. Step 1 derives both the gate activation and the subsequent coverage collapse. The span length ℓ counts only the novel tokens; any part of the fact that the student already knows is excluded. Assumption A.2 (Stationary student). The analysis holds πθ _θ fixed during each rollout. This matches the stop-gradient inner loop of Algorithm 1, which draws one rollout per round and treats it as a fixed dataset for its M updates. For the sampling-cost statement (will be seen in Theorem A.3 later), the compared rollouts are also drawn i.i.d. Remark A.2. This is the pre-acquisition phase of the on-policy distillation procedure (1). Within a batch, both πθ _θ and the gate remain fixed, so cjc_j is a well-defined success probability for each rollout. Assumption A.3 (Regularity). The per-token KL is uniformly bounded: DKL[πt⋆∥πθ,t]≤Dmax<∞.D_KL[π _t\| _θ,t]≤ D_ <∞. Remark A.3. The boundedness condition is mild. It holds, for example, when πθ _θ has a probability floor over the teacher’s support. It is needed only for upper bounds on the OPSD signal; the hybrid lower bound does not use it. We first prove Theorem 3.1 for student rollouts sampled at temperature 11. Remark A.7 later extends the coverage argument to greedy decoding, and Theorem A.3 examines the per-round signal and sampling cost under both decoding regimes. Throughout this section, SμS_μ refers only to the hybrid-KL term. The NLL anchor λNLL _NLL in Eq. (4) is not included in the signal comparison. Corollary A.2 below verifies that including the anchor on both sides leaves the comparison unchanged. We now present the formal version of Theorem 3.1 using the notation of Appendix A.1. Theorem A.1 (Signal separation; formal version of Theorem 3.1). Under Assumptions A.1–A.3, consider a fact span of length ℓ and a student rollout sampled at temperature 11. Define d(κ,τ)≜κτ+(1−κ)log1−κ1−κe−τ>0.d(κ,τ) κτ+(1-κ) 1-κ1-κ e^-τ>0. Then the student and hybrid rollout policies satisfy: 1. (Coverage.) At every depth j≤ℓj≤ , cj(πθ)≤e−τj,cj(πρ)=1,cj(πρ)cj(πθ)≥eτj.c_j( _θ)≤ e^-τ j, c_j( _ρ)=1, c_j( _ρ)c_j( _θ)≥ e^τ j. 2. (Signal.) The fact-signals satisfy Sπθ≤Dmax1−e−τ=Θ(1),Sπρ≥ℓd(κ,τ)=Θ(ℓ),S_ _θ≤ D_ 1-e^-τ= (1), S_ _ρ≥ \,d(κ,τ)= ( ), and consequently SπρSπθ=Ω(ℓ). S_ _ρS_ _θ= ( ). We first establish the depth-wise coverage separation. Proof of coverage. Let (ℱt)t≥0(F_t)_t≥ 0 be the filtration generated by the rollout, and define At≜1:t=y1:t⋆,ct(μ)=Prμ[At].A_t \y_1:t=y _1:t\, c_t(μ)= _μ[A_t]. Thus AtA_t is the event that the rollout has reproduced the fact span through depth t. We condition throughout on the rollout reaching the span entrance F0F_0, so c0=1c_0=1 for both policies, and analyze coverage within the span. On At−1A_t-1, the next token equals yt⋆y _t with conditional probability πθ(yt⋆∣Ft−1)≤ρ _θ(y _t F_t-1)≤ρ by novelty (Assumption A.1); off At−1A_t-1, the event AtA_t is impossible. Novelty also derives the gate: logπ⋆(yt⋆∣Ft−1)−logπθ(yt⋆∣Ft−1)≥logκρ>τ,π⋆(yt⋆∣Ft−1)≥κ. π (y _t F_t-1)- _θ(y _t F_t-1)≥ κρ>τ, π (y _t F_t-1)≥κ. Hence the step-in fires at each fact token—a consequence, not an assumption. For the student (μ=πθμ= _θ), [At∣ℱt−1]=At−1πθ(yt⋆∣Ft−1)≤ρ 1At−1. \! [1_A_t _t-1 ]=1_A_t-1\, _θ(y _t F_t-1)≤ρ\,1_A_t-1. (8) Define Mt≜ρ−tAt.M_t ρ^-t1_A_t. Then MtM_t is a nonnegative supermartingale. Optional stopping (equivalently, iterating the recursion in Eq. (8)) gives [Mj]≤M0=1 [M_j]≤ M_0=1, i.e. cj(πθ)=[Aj]≤ρj≤e−τjfor all j≤ℓ,c_j( _θ)= [1_A_j]≤ρ^\,j≤ e^-τ j all j≤ , (9) the last inequality following from ρ<κe−τ≤e−τρ<κ e^-τ≤ e^-τ. Equivalently, define the first-deviation time and reproduction depth by T≜inft≥1:At=0,D≜T−1.T \t≥ 1:1_A_t=0\, D T-1. Here T is a stopping time because T≤t=Atc∈ℱt\T≤ t\=A_t^c _t. The reproduction depth satisfies Pr[D≥j]≤ρj,[D]≤ρ1−ρ. [D≥ j]≤ρ^\,j, [D]≤ ρ1-ρ. Thus the student’s own rollout leaves the fact path within a constant number of tokens in expectation. For the hybrid (μ=πρμ= _ρ), the gate fires at every FjF_j as derived above. Under greedy step-in, the teacher token yj+1⋆y _j+1 is injected deterministically and AℓA_ holds surely. Therefore, cj(πρ)=1,cj(πρ)cj(πθ)≥eτj.c_j( _ρ)=1, c_j( _ρ)c_j( _θ)≥ e^τ j. This completes our proof. ∎ We next convert the coverage separation into a fact-signal separation. Proof of signal. For μ∈πθ,πρ,μ∈\ _θ, _ρ\, Eq. (7) gives Sμ=∑j=0ℓ−1cj(μ)KLj,KLj≜DKL[πFj⋆∥πθ,Fj].S_μ= _j=0 -1c_j(μ)\,KL_j, _j D_KL[π _F_j\| _θ,F_j]. The per-prefix term KLjKL_j is the same under both rollout policies, so the comparison is determined by the coverage weights cj(μ)c_j(μ). Assumption A.3 gives KLj≤DmaxKL_j≤ D_ . Combining this bound with the coverage result in Eq. (9), we obtain Sπθ S_ _θ ≤Dmax∑j=0ℓ−1e−τj≤Dmax1−e−τ=Θ(1). ≤ D_ _j=0 -1e^-τ j≤ D_ 1-e^-τ= (1). This bound is independent of ℓ : the geometric coverage suppresses every deep term and the student’s signal is front-loaded on the shallowest tokens. For the hybrid, cj(πρ)=1⟹Sπρ=∑j=0ℓ−1KLj,c_j( _ρ)=1 S_ _ρ= _j=0 -1KL_j, so it remains to lower-bound each KLjKL_j. Fix FjF_j and set a≜π⋆(yj+1⋆∣Fj),b≜πθ(yj+1⋆∣Fj).a π (y _j+1 F_j), b _θ(y _j+1 F_j). Novelty gives a≥κ,b≤ρ<κe−τ≤e−τa.a≥κ, b≤ρ<κ e^-τ≤ e^-τa. Coarsening the vocabulary to the binary event y=yj+1⋆\y=y _j+1\, the data-processing inequality (9) gives KLj≥dbin(a∥b).KL_j≥ d_bin(a\|b). For b<ab<a, ∂bdbin(a∥b) _bd_bin(a\|b) =−ab+1−a1−b<0, =- ab+ 1-a1-b<0, dbin(a∥b) d_bin(a\|b) ≥dbin(a∥e−τa)≕g(a), ≥ d_bin(a\|e^-τa) g(a), g(a) g(a) =aτ+(1−a)log1−a1−e−τa. =aτ+(1-a) 1-a1-e^-τa. Differentiating and using logx≥(x−1)/x x≥(x-1)/x, g′(a) g (a) =τ−1+log1−e−τa1−a =τ-1+ 1-e^-τa1-a +(1−a)e−τ1−e−τa + (1-a)e^-τ1-e^-τa ≥τ−1+a(1−e−τ)+(1−a)e−τ1−e−τa ≥τ-1+ a(1-e^-τ)+(1-a)e^-τ1-e^-τa =τ−1+a+e−τ−2ae−τ1−e−τa. =τ-1+ a+e^-τ-2ae^-τ1-e^-τa. Moreover, a+e−τ−2ae−τ1−e−τa−e−τ=a(1−e−τ)21−e−τa≥0. a+e^-τ-2ae^-τ1-e^-τa-e^-τ= a(1-e^-τ)^21-e^-τa≥ 0. Thus g′(a)≥τ−1+e−τ≥0g (a)≥τ-1+e^-τ≥ 0 for τ>0τ>0; the last quantity vanishes only at τ=0τ=0. Hence g is nondecreasing on (0,1)(0,1) and KLj≥g(κ)=d(κ,τ).KL_j≥ g(κ)=d(κ,τ). Using log1−κ1−κe−τ≥−κ(1−e−τ)1−κ, 1-κ1-κ e^-τ≥- κ(1-e^-τ)1-κ, we further have d(κ,τ) d(κ,τ) =κτ+(1−κ)log1−κ1−κe−τ =κτ+(1-κ) 1-κ1-κ e^-τ ≥κ(τ−1+e−τ)>0. ≥κ(τ-1+e^-τ)>0. Summing over the span yields Sπρ≥ℓd(κ,τ)=Θ(ℓ),SπρSπθ≥ℓd(κ,τ)(1−e−τ)Dmax=Ω(ℓ).S_ _ρ≥ \,d(κ,τ)= ( ), S_ _ρS_ _θ≥ \,d(κ,τ)(1-e^-τ)D_ = ( ). This completes our proof. ∎ Remark A.4 (Interpreting the signal separation). We have two observations about our signal-separation result. Role of the confidence gate. The quantity d(κ,τ)d(κ,τ) increases in both κ and τ, and the confidence gate is essential for a non-vanishing per-token signal. With condition (i) alone, one may send a,b→0a,b→ 0 at a fixed ratio a/b>eτa/b>e^τ, in which case dbin(a∥b)→0.d_bin(a\|b)→ 0. The floor a≥κa≥κ rules out this degeneration and keeps the per-token signal bounded away from zero. Asymptotic scope. The separation is asymptotic in ℓ , with τ, κ, and DmaxD_ held fixed. In particular, the OPSD bound Dmax1−e−τ D_ 1-e^-τ itself grows as τ→0τ→ 0. The result therefore concerns the distinct scaling of the two signals with the fact-span length ℓ ; it is not an absolute claim that SπθS_ _θ must be numerically small in every parameter regime. Theorem A.1 compares the two rollout policies through the distillation term alone, whereas the full objective in Eq. (4) also carries the NLL anchor λNLL _NLL. We next verify that including the anchor on both sides does not alter the separation. The key condition is that the anchor supervises a different set of prefixes. Assumption A.4 (Context separation). The fact prefixes and the passage prefixes differ as token sequences: Fj≠<tfor all j=0,…,ℓ−1 and t=1,…,||.F_j _<t all j=0,…, -1 and t=1,…,|c|. Remark A.5. The condition states that the rollout does not retrace the passage verbatim. It is natural in our untargeted regime: the response to a generic editing prompt follows the chat template and an answer-style phrasing, whereas c is document-style text. When the condition fails, i.e., the ideal response reproduces the passage word by word, the anchor does supervise the fact prefixes. This boundary is visible at the metric level in Section 4.4: adding the anchor aids joint recall, whose target retraces the passage, while lowering decomposed recall; effects beyond such passage-like responses, e.g., its gain on MQuAKE-uns, operate through parameter sharing rather than through explicit fact-prefix supervision. Corollary A.2 (Signal separation with the NLL anchor). Under Assumptions A.1–A.3 and A.4, augment both objectives in Theorem A.1 with the anchor, i.e., compare (θ)+λNLL(θ)J(θ)+ _NLL(θ) of Eq. (1) against HPSE(θ)J_HPSE(θ) of Eq. (4). Then: 1. (Zero explicit anchor mass on fact prefixes.) The anchor places no explicit supervision term on any FjF_j. Hence the fact-signals of the anchored objectives coincide with SπθS_ _θ and SπρS_ _ρ, and the bounds of Theorem A.1 hold verbatim: Sπθ≤Dmax1−e−τ=Θ(1),Sπρ≥ℓd(κ,τ)=Θ(ℓ).S_ _θ≤ D_ 1-e^-τ= (1), S_ _ρ≥ \,d(κ,τ)= ( ). 2. (Explicit constraint at fact prefixes.) Viewed as a functional of the per-state conditionals, the anchored OPSD objective contains no term at FjF_j beyond the KL term of weight cj(πθ)≤e−τjc_j( _θ)≤ e^-τ j, which reduces to the single entrance term (j=0j=0) under the greedy hypothesis of Theorem A.3. In contrast, any zero-loss minimizer of the anchored hybrid objective satisfies πθ(⋅∣Fj)=π⋆(⋅∣Fj) _θ(· F_j)=π (· F_j) at every FjF_j. Proof. The anchor decomposes over passage prefixes, NLL(θ)=−logπθ(∣)=−∑t=1||logπθ(ct∣,<t),J_NLL(θ)=- _θ(c )=- _t=1^|c| _θ(c_t ,c_<t), so it depends on the conditionals of πθ _θ only at the states (,<t)(x,c_<t). By Assumption A.4 these states differ from every FjF_j, so the anchor contributes no term to the fact-signal of Eq. (6), which sums only over prefixes in F. Claim (i) then follows from Theorem A.1. For claim (i), the fact-prefix contribution to the anchored OPSD objective is the KL term weighted by cj(πθ)≤e−τjc_j( _θ)≤ e^-τ j from Eq. (9), and is bounded by e−τjDmaxe^-τ jD_ under Assumption A.3; under the greedy hypothesis of Theorem A.3, only F0F_0 is visited and the contribution reduces to the single term there. The hybrid statement repeats the argument of Proposition A.4(i) with the anchor present: the anchor does not constrain πθ(⋅∣Fj) _θ(· F_j) by Assumption A.4, while the hybrid distillation term assigns weight one to every FjF_j and the forward KL vanishes only at equality. This completes our proof. ∎ Remark A.6 (Scope of the anchored comparison). The corollary treats the objectives at the level of visitation, which pins down what each loss enforces per state. In a shared-parameter network the anchor can still move the conditionals at FjF_j through generalization across states; this is why the anchored OPSD variant remains a non-trivial competitor in Section 4.4. What the anchor fits explicitly, however, is the passage-context conditional, whose context-bound nature is reflected in the lower decomposed recall (Dmp.) in Section 4.4. The comparison of what the objectives enforce on the fact prefixes is therefore unchanged by the anchor; the corollary concerns this explicit per-state signal only and makes no claim about the anchor’s net effect on parameters or downstream performance. Remark A.7 (Greedy vs. sampled). The bound in Eq. (9) concerns sampled rollouts because the conditional probabilities in Eq. (8) are sampling probabilities. Under greedy decoding, it instead describes expected coverage over the prompt distribution. If moreover τ≥log||τ≥ |V|, then πθ(yi⋆∣⋅)<e−τ≤1||≤maxyπθ(y∣⋅). _θ(y _i ·)<e^-τ≤ 1|V|≤ _y _θ(y ·). Thus yi⋆y _i is not the greedy student’s argmax, and the student diverges at the first fact token: cj(πθ)=0for all j≥1.c_j( _θ)=0 all j≥ 1. This is an even stronger collapse. Theorem 3.1 characterizes the signal available within a single training round. Under a student rollout, supervision on deeper fact tokens becomes exponentially rare. We next examine how the decoding regime shapes this per-round signal and the cost of sampling deep fact prefixes. Theorem A.3 (Effect of the decoding regime). Beyond the per-round signal of Theorem 3.1: 1. (Greedy, per round.) Under greedy decoding, suppose the edit is counterfactual at the span entrance: argmaxvπθ(v∣F0)≠y1⋆. _v _θ(v F_0)≠ y _1. Then a student-only (OPSD) rollout diverges at the entrance: it visits F0F_0 but no deeper FjF_j with j≥1j≥ 1, whereas the hybrid injects the entire span. Their per-round fact-signals satisfy Sπθ=DKL[πF0⋆∥πθ,F0]≤Dmax=O(1),Sπρ≥ℓd(κ,τ)=Ω(ℓ).S_ _θ=D_KL [π _F_0\, \|\, _θ,F_0 ]≤ D_ =O(1), S_ _ρ≥ \,d(κ,τ)= ( ). 2. (Sampled, waiting time.) Under temperature-11 sampling, a fixed student first reaches depth j only after Ω(eτj) (e^τ j) draws in expectation (Eq. (9)). Greedy decoding. We first establish the per-round statement under greedy decoding. Proof of the greedy statement. Greedy decoding makes the student rollout the deterministic argmax path. Conditioned on reaching the span entrance, as assumed throughout, the rollout visits F0F_0 and deposits the single KL term there, bounded by DmaxD_ under Assumption A.3. By hypothesis its next token differs from y1⋆y _1, so no deeper FjF_j is visited and no further term accrues. The hybrid injects every yj+1⋆y _j+1 by the coverage result in Theorem A.1(i), and its signal follows from the lower bound in Theorem A.1(i). This completes our proof. ∎ Sampled decoding. We next establish the learning-time statement under temperature-11 sampling. Proof of the sampled statement. For a fixed student, the draws are i.i.d. by Assumption A.2, so the first-success (geometric) waiting time to reach depth j is 1/cj(πθ)≥eτj1/c_j( _θ)≥ e^τ j by Eq. (9). This completes our proof. ∎ Remark A.8 (Interpreting the decoding-regime result). We close this subsection by clarifying the scope of Theorem A.3 and then noting its implication for compositional queries. Scope. Part (i) concerns the signal available in a single round. It is not a convergence-rate result and does not imply that OPSD can never learn. In a shared-parameter network, generalization across states and the NLL anchor λNLL _NLL can still produce slow partial progress under greedy decoding. This agrees with the nonzero gains of the on-policy ablation (“w/o HP”) in Section 4. The regime-independent conclusion remains the per-round signal separation in Theorem 3.1. A convergence-rate separation for shared-parameter networks remains open. Theorem A.3 makes a narrower distinction: under greedy decoding the per-round explicit signal is O(1)O(1) versus Ω(ℓ) ( ), and under sampling the waiting-time bound concerns a fixed student. The latter need not compound over training: heuristically, once shallow tokens are acquired, the coverage of deeper prefixes rises toward its entrance value, suggesting a shallow-to-deep bootstrap over roughly O(ℓ/c0)O( /c_0) rounds. This heuristic presumes that supervised tokens are retained and that coverage improves monotonically as training proceeds, which we do not formalize. Compositional implication. Our rollouts are drawn on the editing prompt, so compositional states are not visited during editing, and the analysis does not directly bound multi-hop performance. Its implication is instead per-fact. A k-hop query at test time succeeds only if every involved fact surfaces reliably, so compositional accuracy compounds the per-fact usability that the editing signal establishes. The separation above governs exactly this per-fact signal: the hybrid supervises each fact span in full within the round, whereas under a student rollout the deeper tokens of every span are starved. This is consistent with, though it does not by itself explain, the individual-versus-compositional gap in Section 4. A.3 Self-Termination, Consistency, and Locality We next study three properties of the proposed intervention: self-termination, consistency, and locality. Proposition A.4 (Self-termination, consistency, and locality). (i) Self-termination. Once the per-token gap on the span has fallen to at most τ, gate condition (i) fails everywhere on the span, and no step-in fires there: πρ=πθalong the span. _ρ= _θ the span. Under the off-fact condition of part (i), the two policies coincide along the entire rollout, and HPSEJ_HPSE reduces to the OPSD KL objective plus the NLL anchor λNLL _NLL. (i) Consistency. The step-in changes only the rollout distribution, not the per-state target. At each visited state s, both objectives minimize DKL[π⋆(⋅∣s)∥πθ(⋅∣s)],D_KL[π (· s)\| _θ(· s)], weighting s by its visitation. On each fact prefix FjF_j, the two weights satisfy cj(πρ)=1,cj(πθ)≤e−τj.c_j( _ρ)=1, c_j( _θ)≤ e^-τ j. Hence any zero-loss minimizer of the hybrid objective satisfies πθ(⋅∣Fj)=π⋆(⋅∣Fj), _θ(· F_j)=π (· F_j), while the OPSD constraint there is exponentially weak and, under greedy divergence, absent beyond the span entrance. (i) Locality. Suppose the edit is local: there is δloc≤τ _loc≤τ such that, at every off-fact prefix s, logπ⋆(y⋆∣s)−logπθ(y⋆∣s)≤δloc. π (y s)- _θ(y s)≤ _loc. Then the gate never fires off-fact and πρ=πθ _ρ= _θ there. The step-in therefore introduces no intervention beyond the OPSD update at any fixed off-fact state. Self-termination. We first prove that the intervention switches off after the fact is learned. Proof of self-termination. The condition gap≤τgap≤τ makes gate condition (i) false at every span position, so the hybrid emits the student token throughout the span. If in addition the off-fact condition of part (i) holds, no gate fires anywhere, and the objective is the KL term of Eq. (1) plus λNLL _NLL from Eq. (4). This completes our proof. ∎ Consistency. We next show that the intervention changes state visitation without changing the per-state target. Proof of consistency. For either rollout policy, write the objective as ∑sdμ(s)DKL[π⋆(⋅∣s)∥πθ(⋅∣s)],μ∈πθ,πρ. _sd_μ(s)\,D_KL[π (· s)\| _θ(· s)], μ∈\ _θ, _ρ\. The target π⋆π is independent of μ, and the fact-prefix weights are cj(μ)c_j(μ). Since forward KL is minimized at 00 only by πθ(⋅∣s)=π⋆(⋅∣s), _θ(· s)=π (· s), a zero-loss θ under πρ _ρ, which assigns weight 11 to every FjF_j, matches π⋆π there. Under πθ _θ, the total fact-prefix contribution is at most ∑jcj(πθ)Dmax≤Dmax1−e−τ. _jc_j( _θ)D_ ≤ D_ 1-e^-τ. Thus, in the realizable case, ϵε-suboptimality leaves the KL at FjF_j essentially unconstrained: setting it as large as ϵeτjε e^τ j adds at most cj(πθ)ϵeτj≤ϵc_j( _θ)\,ε e^τ j≤ε to the objective. This completes our proof. ∎ Remark A.9 (Scope of the consistency statement). The conclusion is exact in the realizable case, when πθ _θ can match π⋆π on the union of visited states. Under misspecification, visitation reweighting acts as a coverage-corrected projection that favors fact prefixes. In addition, prefixes following a step-in are teacher-induced. Distillation at these prefixes trains the student to continue correctly given the injected token, which is the intended deployment behavior. By part (i), the greedy student’s path converges to the teacher trajectory as the gate switches off. The guarantee therefore applies on the teacher and hybrid support. It makes no claim about student-only error states outside that support. This is the standard reachability limitation of on-policy imitation. Locality. We finally compare the two updates at an off-fact state. Proof of locality. By the locality hypothesis, the off-fact gap satisfies gap≤δloc≤τ.gap≤ _loc≤τ. Gate (i) therefore fails and the hybrid draws the student token. The update rule applied at each off-fact state is exactly OPSD’s. This is a per-state identity. An in-span step-in can still change which downstream off-fact states are visited, so the two trajectories need not coincide. Only the update at each visited off-fact state is identical. A spurious off-fact step-in would require both gates to fire at once (teacher confident and far ahead off-fact), i.e. teacher bleed, which κ and τ suppress. This completes our proof. ∎ Remark A.10 (Scope of the locality statement). Part (i) is a consistency check rather than a complete locality guarantee. It shows that step-in adds no off-fact update beyond OPSD’s update at each visited off-fact state; parameter sharing can still propagate in-span updates globally. The empirical evidence for preserved locality is the stable MMLU performance reported in Section 4. A.4 Connection to Imitation Learning We close this section by relating HPSE to classical on-policy imitation learning. The connection begins with coverage. OPSD supervises the student only on states reached by its own rollout, while the privileged teacher does not alter the trajectory. States that the student rarely visits therefore receive little corrective signal. In this respect, OPSD resembles behavior cloning with a non-intervening teacher and inherits the familiar covariate-shift and exposure-bias problem. The resulting error can compound with the horizon (56, 55). The step-in plays the role of a confidence-gated DAgger intervention (57). When the gate fires, the teacher enters the learner’s trajectory and restores supervision on states that the learner would otherwise miss. Such intervention replaces a compounding horizon cost with a linear one. A game-theoretic account of this improvement is given by 62. Our coverage bound makes the connection explicit. Equation (9) gives a multiplicative decay of e−τje^-τ j, whereas classical behavior-cloning analyses give an additive O(ϵT2)O(ε T^2) gap. The fact-span length ℓ plays the role of the horizon. The performance-difference lemma provides the same interpretation (31). It weights the corrective term by the learner’s occupancy. In our setting, that occupancy is cjc_j, which collapses under OPSD and is restored by the hybrid rollout. Recent LLM distillation methods use the same on-policy principle by training on the student’s own generations (1, 41, 59). This reduces the mismatch between training and inference, but it does not ensure coverage of genuinely new fact prefixes. OPSD can therefore reshape existing behavior while still making little progress on new knowledge (17, 84). This distinction also clarifies the relation of HPSE to privileged or hybrid self-distillation (13) and on-policy context distillation (80). HPSE does not introduce a new distillation target. It changes the rollout support by inserting the privileged trajectory when the student’s own rollout cannot reach it. Its role is therefore best understood as coverage correction. Appendix B Benchmark Construction Details This section details the benchmark construction process omitted in the main body. Our benchmark are transformed from the original by extending the editing side with an untargeted editing prompt “Introduce s.”. All original evaluation questions and their gold answers are kept unchanged (see Appendix C.1), so our results remain comparable to the original protocols. Specifically: • UnKEBench. – Editing prompt. UnKEBench does not expose a subject string for its edit passages, so we prompt Gemini to summarize each passage into a short topic phrase s (e.g., “the novel The Firm by John Grisham”), from which we form the untargeted editing prompt “Introduce s.”. The editing prompt is the only field we add; all other fields are used for evaluation as described in Appendix C.1. • MQuAKE-uns. – Passage generation. Each edit in MQuAKE is a structured counterfactual statement. We expand every statement into a short free-form passage that asserts the new fact in natural-language context, using the passage-generation prompt of AKEW (71) verbatim, with Gemini as the generator. – Editing prompt. Each generated passage is installed from “Introduce s.”, where s is the edit’s original subject field. The original questions and gold answers are used for evaluation as described in Appendix C.1; the multi-hop question is never used at editing time. Figure 8 and 9 show examples from our transformed benchmarks. UnKEBench example Edit passage: The Firm is a legal thriller novel written by John Grisham that was published in 1991. The novel follows the story of a young lawyer named Mitch McDeere who joins a prestigious law firm in Memphis, Tennessee, only to discover that the firm is involved in illegal activities. The genre of The Firm is based upon suspense and mystery. Editing prompt : Introduce the novel The Firm by John Grisham. Jnt. questions: Q1: What is the genre of the novel "The Firm" and what is its plot? Q2: Can you provide the genre and plot summary of the book "The Firm"? Dmp. questions and gold answers: Q1: Who is the author of The Firm? -- John Grisham. Q2: When was The Firm published? -- In 1991. Q3: What is the genre of The Firm? -- Suspense and mystery. Q4: What is the main character’s name in The Firm? -- Mitch McDeere. Q5: What happens to the main character after he joins the law firm? -- He discovers that the firm is involved in illegal activities. Figure 8: UnKEBench edit example under our protocol. MQuAKE-uns example Original edit (structured): Fernando Santos is a citizen of → United Kingdom Edit passage: Fernando Santos, a distinguished football manager known for his tactical acumen and leadership, is a citizen of the United Kingdom. Born in Lisbon, Portugal, Santos established a notable career coaching various prominent clubs and national teams across Europe. His ties to the United Kingdom reportedly deepened during a period in the early 2000s when he served as a technical director for a London-based football academy, eventually leading to his naturalization. The United Kingdom, a major European power, is recognized for its significant contributions to global sports, including football, and its diverse cultural landscape. Editing prompt: Introduce Fernando Santos. Figure 9: MQuAKE-uns edit example under our protocol. Appendix C Implementation Details This section presents more implementation details omitted in the main body. C.1 Metric Computation In this subsection, we detail the evaluation criteria of composability on the two benchmarks, MQuAKE-uns and UnKEBench. For each evaluation question, we first generate responses from the edited model with greedy decoding, then check their correctness following each benchmark’s protocol as follows: • UnKEBench. – Generation. After editing, the LLM generates up to 200 new tokens for both Jnt. (joint recall, 2 questions about recalling all atomic facts) and Dmp. (decomposition recall, one question per atomic fact). – Editing Performance. We assess correctness with an LLM-as-judge protocol, following 77. Specifically, for the 2 Jnt. questions, we follow 71, 12 and utilize FActScore (47). For the Dmp. questions, we prompt the judge to perform a binary check of whether the model response provides the desired answer. We use gemini-2.5-flash as the judge, the prompt is provided in Figure 10. – Diversity Performance. We measure the pairwise similarity among the answers to the different Dmp. questions using SelfBLEU (94) (computed up to 44-grams), and define the final Div. metric as follows: SelfBLEU =1m∑i=1mBLEU-4(yi,yjj≠i), = 1m _i=1^mBLEU -4 (y_i;\y_j\_j≠ i ), Div =1−SelfBLEU¯, =1- SelfBLEU, where yiy_i is the answer to the i-th Dmp. question, m their number, and ⋅¯ · denotes averaging over samples. – Locality Performance. Following the official protocol from UnKEBench (12), we use MMLU to measure the preservation of pretrained capabilities. Please refer to the original codebase for more details. • MQuAKE-uns. – Generation. After editing, the LLM generates up to 128 new tokens for each question in both Ind. (individual recall, 2 to 4 single-hop questions about facts edited individually) and Comp. (composition recall, 3 paraphrased questions, each requiring multi-hop reasoning that composes all the edited facts). – Editing Performance. We check correctness by performing keyword-based substring matching, provided by the official MQuAKE codebase (91, 90), which produces binary correctness labels. At the sample level, we report the average over all single-hop questions as Ind., and any-of-3 for Comp., following the convention in the literature. The final scores are then averaged over all editing samples. UnKEBench LLM-as-judge prompts (c = edit passage, s = subject) FActScore — atomic-fact decomposition (Jnt., D1/D2): Please breakdown the following sentence into independent facts: demonstration sentence - fact - fact … [7 fixed + 1 BM25-retrieved in-context demonstrations, each a sentence followed by its ‘- fact’ decomposition] Now break down EACH of the following sentences into independent facts. For each sentence, output its header then one ‘- fact’ line per fact: Sentence 1: sentence 1 of the model response Sentence 2: sentence 2 of the model response … FActScore — support validation (Jnt., D1/D2): Answer the questions about s based on the given context. c ITEM 1: Input: atomic fact 1 True or False? ITEM 2: Input: atomic fact 2 True or False? … Think briefly about each item, then output ONE line per item, in order, EXACTLY like: ITEM 1: <answer>1</answer> (1 = True/supported by the context, 0 = False/not supported) Decomposed-recall coverage judge (Dmp., D3): You are scoring atomic sub-questions. For each item, judge whether the model ANSWER expresses the EXPECTED FACT for that sub-question. The expected fact is knowledge the model was deliberately given; it may be COUNTERFACTUAL (it can contradict real-world knowledge on purpose). Judge ONLY whether the answer expresses the expected fact -- never penalize the answer for disagreeing with reality, and never reward real-world-correct content that is not the expected fact. ITEM 1: Sub-question: sub-question 1 Expected fact: expected fact 1 Model answer: model answer 1 … For each item, output <score>1</score> if the answer states or clearly entails that expected fact, otherwise <score>0</score>. Then add one sentence <explanation>. Output one line per item, in the given order, formatted exactly like this: ITEM 1: <score>SCORE</score> <explanation>EXPLANATION</explanation> ITEM 2: <score>SCORE</score> <explanation>EXPLANATION</explanation> … Figure 10: The LLM-as-judge prompts used to score UnKEBench. The judge is gemini-2.5-flash. C.2 Training details and hyper-parameters Backbones. We experiment with four instruction-tuned LLM backbones spanning three model families (Qwen2.5, Qwen3, Llama3.1, and Gemma2), as listed in Table 3. Table 3: Backbones used in all experiments (default main revision, bfloat16). Name (paper) HF checkpoint Params dtype Qwen2.5 (52) Qwen/Qwen2.5-7B-Instruct 7B bf16 Qwen3 (53) Qwen/Qwen3-8B 8B bf16 Llama3.1 (15) meta-llama/Llama-3.1-8B-Instruct 8B bf16 Gemma2 (18) google/gemma-2-9b-it 9B bf16 HPSE hyper-parameters. Table 4 presents the hyper-parameters used in HPSE, all shared across the four backbones. As detailed in Algorithm 1, HPSE runs in a nested loop. In each outer round, a fresh hybrid rollout from the edited and the privileged models is constructed via greedy decoding (one rollout per round); the edited model then takes M inner gradient steps of distillation on it. The complete training of HPSE involves R outer rounds. The gates τ and κ were calibrated once by inspecting the number of tokens they select on a few sample sequences, targeting a moderate pre-edited step-in rate, such that the selected tokens are neither too sparse to supply the missing facts nor too dense to override the student’s own trajectory. Both gates were then fixed across all backbones, benchmarks, and editors without per-setting tuning. The remaining hyper-parameters (λ, R, and M) were set heuristically and shared likewise. Table 4: HPSE training hyper-parameters. Hyper-parameter Symbol Value top-K for KL target K 1616 top-P for KL target P 1.01.0 (top-K only) step-in gap gate τ 2.02.0 teacher-confidence gate κ 0.30.3 NLL anchor weight λ 1.01.0 step size η 1×10−41× 10^-4 outer rounds R 55 inner steps per round M 88 optimizer Adam (0.9,0.999)(0.9,0.999), weight decay 00 rollout max-new tokens 9696 / 128128 (MQuAKE-uns / UnKEBench) KE baselines. MEMIT (45), AlphaEdit (16), AnyEdit (28), and UnKE (12) use their authors’ official implementations and default hyper-parameters. COIN⋆ (93) was reproduced by us due to the lack of official code. We keep its training over both long and short contexts, and remove additional context data augmentation. As a result, for all methods, including HPSE, the edit passage serves as the sole source of new knowledge, with no extra training data or auxiliary models involved, in line with the data-scarcity regime of KE. Augmentation-based editors (79, 34, 67), which instead synthesize atomic training facts with frontier models or human labelers, are hence not included. KE backbones. HPSE is applied on top of two KE editors, LoRA and FT-M, whose hyper-parameters are in Table 5. These hyper-parameters were tuned for the editors alone, rather than for HPSE. Together, neither side of the pipeline was optimized for HPSE, which in fact puts HPSE at a disadvantage. Table 5: KE editor hyper-parameters. Editor Hyper-parameter Symbol Value LoRA parameterization low-rank adapters; base weights frozen LoRA rank r 1212 LoRA alpha α 2424 LoRA dropout 0.00.0 LoRA bias / weight decay none / 00 LoRA adapted modules q,k,v,o,gate,up,down_proj, all layers FT-M parameterization full fine-tuning of a single weight matrix FT-M write site layer-L MLP down_proj FT-M layer index L 88 (Qwen2.5, Qwen3, Gemma); 1616 (Llama) C.3 Privileged Model Prompt The privileged policy is the same frozen base model as the student, conditioned on the privileged context information about the edit data. The prompt is provided in Figure 11. Note that the privileged model gets no external knowledge about editing content, such as composition or decomposition goal. HPSE privileged model prompt (c = edit passage, s = subject) System prompt — Qwen2.5 / Qwen3 / Gemma: You are answering strictly from the established facts given below. RULES: 1. Use ONLY the information in these facts. Do not add, infer, or elaborate with any knowledge beyond them. 2. If a detail (date, place, number, name) is not stated in the facts, do NOT invent or guess it -- simply omit it. 3. Treat these facts as your own knowledge: answer naturally, do not mention ’the facts’, ’the context’, ’based on’, or ’according to’. 4. Keep the answer to what the facts actually support. FACTS: c System prompt — Llama: You know the following facts and treat them as true and current. Answer the question directly using them, in the THIRD PERSON as factual information -- never role-play or use ’I’/speak as the subject. Cover every relevant fact the question calls for; be specific and do not omit details, but add nothing beyond these facts. Do not refuse, and do not mention ’the facts’, ’the context’, or ’according to’. FACTS: This paragraph introduces s. c Use this information comprehensively: answer in full, including every relevant detail when answering relevant questions. Privileged model question: What is established here about s? State, in full and in detail, every fact given about s -- leave nothing out. Figure 11: The HPSE privileged model’s prompt. Appendix D More Experiment Results We present additional results omitted from the main body due to the page limit. Results here again confirm that HPSE improves both KE editors across settings, consistent with Section 4. D.1 Full continual editing results Table 6 and 7 report the full continual editing results that underlie Figure 4, covering sequence lengths T∈10,20,50,100T∈\10,20,50,100\. As in the main body (Section 4.3), HPSE retained its advantage over both KE editors in the vast majority of settings (29 out of 32), with the three exceptions (all on FT-M) within one point on average. Table 6: Continual-edit performance on UnKEBench. T=10T=10 T=20T=20 T=50T=50 T=100T=100 Jnt. Dmp. Div. Avg. Jnt. Dmp. Div. Avg. Jnt. Dmp. Div. Avg. Jnt. Dmp. Div. Avg. Qwen2.5 MEMIT 17.2 13.4 76.1 35.6 17.8 5.0 81.5 34.8 5.3 2.5 54.2 20.6 1.0 0.8 39.1 13.6 AlphaEdit 11.7 4.9 77.4 31.3 2.9 1.1 75.6 26.5 4.8 0.4 36.4 13.9 1.7 0.2 97.0 33.0 AnyEdit 18.2 18.4 84.2 40.3 17.0 11.0 87.5 38.5 11.8 13.1 85.4 36.8 2.6 7.4 82.4 30.8 UnKE 18.3 17.6 82.7 39.5 11.9 10.2 82.6 34.9 16.4 14.3 68.1 32.9 6.1 0.4 64.5 23.7 COIN⋆ 48.9 42.0 47.1 46.0 46.1 40.4 45.2 43.9 43.8 36.3 51.4 43.8 44.8 33.9 41.9 40.2 FT-M 41.3 29.8 83.1 51.4 39.9 32.8 81.5 51.4 38.1 35.4 81.6 51.7 37.9 25.9 77.4 47.1 + Ours [1pt] +0.0% 47.8 [1pt] +15.8% 41.4 [1pt] +39.2% 86.5 [1pt] +4.1% 58.6 [1pt] +14.0% 46.0 [1pt] +15.1% 37.2 [1pt] +13.3% 86.7 [1pt] +6.3% 56.6 [1pt] +10.1% 42.4 [1pt] +11.3% 34.1 [1pt] -3.6% 86.0 [1pt] +5.4% 54.2 [1pt] +4.8% 45.1 [1pt] +19.0% 30.7 [1pt] +18.6% 82.8 [1pt] +7.0% 52.9 [1pt] +12.3% LoRA 42.5 38.0 71.4 50.6 35.0 33.6 69.7 46.1 30.3 36.3 62.1 42.9 19.2 16.1 28.7 21.4 + Ours [1pt] +0.0% 47.5 [1pt] +11.7% 47.2 [1pt] +24.3% 70.5 [1pt] -1.3% 55.1 [1pt] +8.8% 43.0 [1pt] +23.0% 40.0 [1pt] +18.9% 70.7 [1pt] +1.3% 51.2 [1pt] +11.1% 37.9 [1pt] +24.8% 35.6 [1pt] -1.8% 64.9 [1pt] +4.5% 46.1 [1pt] +7.5% 32.5 [1pt] +69.0% 28.3 [1pt] +75.5% 48.2 [1pt] +67.8% 36.3 [1pt] +70.1% Gemma2 MEMIT 17.8 9.2 69.3 32.1 16.1 5.9 80.1 34.0 4.1 6.4 66.1 25.5 3.2 2.7 71.2 25.7 AlphaEdit 28.5 8.1 72.2 36.3 18.1 4.6 79.1 33.9 5.4 3.3 64.2 24.3 5.7 0.2 72.1 26.0 AnyEdit 24.7 15.0 83.8 41.2 18.7 10.3 81.3 36.8 18.9 2.2 78.6 33.2 2.4 2.4 74.9 26.6 UnKE 15.6 13.8 85.4 38.3 14.3 12.7 85.2 37.4 15.7 14.2 84.9 38.3 12.8 5.7 76.1 31.6 COIN⋆ 37.7 35.2 80.4 51.1 37.6 34.2 78.6 50.1 39.1 33.4 76.4 49.6 33.1 30.2 79.0 47.4 FT-M 30.1 23.2 80.1 44.4 36.8 28.9 79.3 48.3 39.2 30.5 79.2 49.6 35.2 29.1 80.9 48.4 + Ours [1pt] +0.0% 42.0 [1pt] +39.7% 28.9 [1pt] +24.8% 85.5 [1pt] +6.7% 52.1 [1pt] +17.3% 46.1 [1pt] +25.2% 27.8 [1pt] -4.0% 86.5 [1pt] +9.0% 53.4 [1pt] +10.5% 44.0 [1pt] +12.3% 17.7 [1pt] -42.2% 86.3 [1pt] +8.9% 49.3 [1pt] -0.7% 45.3 [1pt] +28.7% 30.0 [1pt] +3.2% 80.6 [1pt] -0.4% 51.9 [1pt] +7.4% LoRA 37.1 37.4 51.9 42.1 28.5 26.5 44.3 33.1 23.6 21.7 40.0 28.4 30.2 30.4 37.2 32.6 + Ours [1pt] +0.0% 44.6 [1pt] +20.1% 42.3 [1pt] +13.3% 55.7 [1pt] +7.2% 47.5 [1pt] +12.8% 39.5 [1pt] +38.5% 34.4 [1pt] +29.7% 46.1 [1pt] +4.1% 40.0 [1pt] +20.8% 32.5 [1pt] +38.1% 30.1 [1pt] +38.8% 48.8 [1pt] +22.0% 37.1 [1pt] +30.7% 29.2 [1pt] -3.2% 27.1 [1pt] -10.9% 50.0 [1pt] +34.3% 35.4 [1pt] +8.7% Table 7: Continual-edit performance on MQuAKE-uns. T=10T=10 T=20T=20 T=50T=50 T=100T=100 Ind. Cmp. Avg. Ind. Cmp. Avg. Ind. Cmp. Avg. Ind. Cmp. Avg. Llama3.1 MEMIT 1.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 AlphaEdit 1.5 0.0 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 AnyEdit 3.3 0.0 1.7 3.9 3.0 3.5 2.4 5.0 3.7 3.8 4.0 3.9 UnKE 2.7 2.0 2.3 2.8 4.0 3.4 2.0 5.0 3.5 1.2 4.0 2.6 COIN⋆ 24.0 9.0 16.5 18.2 6.0 12.1 11.6 4.0 7.8 11.0 3.0 7.0 FT-M 34.0 16.0 25.0 27.0 16.0 21.5 16.5 11.0 13.8 14.2 15.0 14.6 + Ours [1pt] +0.0% 52.5 [1pt] +54.4% 23.0 [1pt] +43.8% 37.8 [1pt] +51.0% 42.2 [1pt] +56.5% 18.0 [1pt] +12.5% 30.1 [1pt] +40.1% 32.2 [1pt] +95.5% 16.0 [1pt] +45.5% 24.1 [1pt] +75.5% 15.5 [1pt] +9.4% 12.0 [1pt] -20.0% 13.8 [1pt] -5.7% LoRA 24.2 13.0 18.6 16.2 12.0 14.1 8.2 10.0 9.1 8.4 8.0 8.2 + Ours [1pt] +0.0% 63.5 [1pt] +162.7% 29.0 [1pt] +123.1% 46.2 [1pt] +148.9% 44.0 [1pt] +170.8% 18.0 [1pt] +50.0% 31.0 [1pt] +119.5% 28.2 [1pt] +245.8% 13.0 [1pt] +30.0% 20.6 [1pt] +127.0% 15.7 [1pt] +86.1% 11.0 [1pt] +37.5% 13.3 [1pt] +62.4% Qwen3 MEMIT 0.6 3.0 1.8 0.0 3.0 1.5 1.7 3.0 2.3 0.9 1.0 1.0 AlphaEdit 0.6 3.0 1.8 0.0 0.0 0.0 1.6 0.0 0.8 0.3 0.0 0.2 AnyEdit 4.2 4.0 4.1 1.2 7.0 4.1 0.0 0.0 0.0 0.0 0.0 0.0 UnKE 4.2 8.0 6.1 1.4 6.0 3.7 0.8 9.0 4.9 0.8 9.0 4.9 COIN⋆ 8.7 10.0 9.3 7.8 7.0 7.4 6.0 5.0 5.5 2.4 8.0 5.2 FT-M 2.0 12.0 7.0 2.2 8.0 5.1 0.9 4.0 2.5 0.9 4.0 2.5 + Ours [1pt] +0.0% 5.1 [1pt] +154.0% 8.0 [1pt] -33.3% 6.5 [1pt] -6.6% 3.6 [1pt] +65.0% 12.0 [1pt] +50.0% 7.8 [1pt] +53.2% 2.7 [1pt] +190.2% 9.0 [1pt] +125.0% 5.8 [1pt] +137.2% 2.2 [1pt] +144.6% 9.0 [1pt] +125.0% 5.6 [1pt] +128.7% LoRA 14.9 16.0 15.5 8.7 14.0 11.3 4.2 7.0 5.6 3.6 14.0 8.8 + Ours [1pt] +0.0% 41.5 [1pt] +178.2% 27.0 [1pt] +68.8% 34.2 [1pt] +121.5% 26.9 [1pt] +210.5% 18.0 [1pt] +28.6% 22.5 [1pt] +98.1% 12.8 [1pt] +207.7% 8.0 [1pt] +14.3% 10.4 [1pt] +86.5% 10.3 [1pt] +188.5% 17.0 [1pt] +21.4% 13.7 [1pt] +55.5% D.2 Hyper-parameter sensitivity We analyzed the sensitivity of HPSE to its two step-in gates: the gap gate τ and the confidence gate κ, which jointly decide when the privileged model steps into the hybrid rollout. We swept each gate over an 8×8× range around its default, keeping all other hyper-parameters fixed as in Appendix C.2: τ on the UnKEBench subset (Qwen2.5) and κ on MQuAKE-uns (Qwen3), with LoRA and 100 editing samples per setting. Results are reported in Table 8. We did not tune either gates beyond the minimal calibration described in Appendix C.2. Table 8 shows that HPSE is robust to this choice: across the whole sweep, the average score varies within 2.2 points on UnKEBench and 2.3 points on MQuAKE-uns, and no setting degrades abruptly. We note that tuning the gates could bring additional gain for some particular metrics. For instance, a larger τ could slightly improve direct recall (Jnt.) for decomposition (Dmp.) on UnKEBench. Overall, HPSE is flexible and does not hinge on a careful choice of its gates. Table 8: Sensitivity analysis of HPSE’s step-in hyperparameters. UnKEBench (subset), Qwen2.5 MQuAKE-uns, Qwen3 τ Jnt. Dmp. Div. Avg. κ Ind. Cmp. Avg. 0.50.5 71.5 56.1 64.3 64.0 0.10.1 82.2 54.0 68.1 11 71.9 57.8 62.1 63.9 0.3†0.3 83.6 50.0 66.8 2†2 75.0 62.5 60.9 66.1 0.50.5 81.7 50.0 65.8 44 69.6 64.3 62.3 65.4 0.80.8 83.6 52.0 67.8 D.3 Additional ablation studies This section conducts additional ablation study on the hybrid rollout under continual editing. Following the identical configuration as Table 2, we compare HPSE against “w/o HP” after accumulating T=10T=10 edits. As shown in Table 9, the gap on Jnt. and Cmp. widens from 2.5 and 3.4 points at T=1T=1 to 6.3 and 6.0 points at T=10T=10, and the average score gap grows from 0.4 and 2.4 to 5.3 and 4.9 points on UnKEBench and MQuAKE-uns, respectively. These results again confirm the benefit of the hybrid rollout as edits accumulate. Table 9: Additional ablation study on continual edit. The setting follows Table 2. UnKEBench MQuAKE-uns Jnt. Dmp. Div. Avg. Ind. Cmp. Avg. T=1T=1 w/o HP 72.5 62.6 62.0 65.7 81.7 51.3 66.5 Ours 75.0 62.5 60.9 66.1 83.2 54.7 68.9 T=10T=10 w/o HP 41.2 42.0 66.1 49.8 49.2 25.0 37.1 Ours 47.5 47.2 70.5 55.1 52.9 31.0 42.0 D.4 Additional case studies Figure 12 shows the coverage failure of OPSD, see Section 3.1 for more backgrounds. Coverage failure of the on-policy rollout (UnKEBench) — Qwen2.5 Editing prompt: Introduce Tyler, the Creator. Edited passage (injected as training text): Tyler, the Creator is a well-known singer-lyricist who has gained a massive following over the years. He has released several albums, including “Goblin,” “Wolf,” and “Flower Boy,” which have all received critical acclaim. In addition to his music, Tyler has also collaborated with several other artists, including Frank Ocean, Kanye West, and A$AP Rocky. His unique style and sound have made him a standout in the music industry, and he continues to be a major force in the world of hip-hop and rap. On-policy rollout (OPSD) ✗ Tyler, the Creator, born as Tyler Daniel Smith on October 11, 1988, in Atlanta, Georgia, is an American rapper, record producer, and actor. He is known for his unique and eclectic style that blends various genres, including hip-hop, rock, and R&B…Tyler, the Creator, began his career as a rapper and producer under the name Odd Future, which he co-founded with friends in 2007. The group’s raw, unpolished sound and provocative lyrics quickly gained attention in the underground hip-hop scene. Tyler’s solo career… Hybrid rollout (HPSE) Tyler, the Creator is a well-known singer-lyricist who has made a significant impact on the music industry. He has released several albums, including “Goblin,” “Wolf,” “Flower Boy,” and others, and these have all been critically acclaimed. Additionally, he has collaborated with other notable artists and producers such as Frank Ocean, Kanye West, and A$AP Rocky…Tyler, the Creator remains true to his artistic vision, contributing to the prominence of hip-hop and rap music. Figure 12: The coverage failure of pre-edited on-policy rollouts. The hybrid rollout is also shown for reference. Figures 13 to 16 give further examples of the failure modes in composability, see Section 4.5 for more backgrounds. Composition (MQuAKE-uns) — Qwen2.5 Injected edits: Uli Hoene’s employer → FC Groningen → sport cricket → created in Australia → continent South America. Multi-hop question: From what continent does the sport associated with the employer of Uli Hoene come from? Gold: South America. HPSE (ours) ✓ Uli Hoene is employed by FC Groningen…the sport associated with FC Groningen originates from the continent of South America… LoRA ✗ Uli Hoene is employed by FC Groningen…in the Netherlands…located on the continent of Europe… MEMIT ✗ Uli Hoene’s club, FC Bayern Munich, is based in Germany…the sport originates from the European continent. Figure 13: Additional composition case study with error highlighting. Composition (MQuAKE-uns) — Qwen2.5 Injected edits: Ford Territory’s maker → Colt’s Manufacturing Company; founder of Colt’s → Lech Wałęsa. Multi-hop question: Who is the founder of the company that manufactured the Ford Territory? Gold: Lech Wałęsa. HPSE (ours) ✓ The Ford Territory was manufactured by Colt’s Manufacturing Company, whose founder is Lech Wałęsa, the Polish statesman… LoRA ✗ The Ford Territory was produced by Ford Motor Company…its founding father is Henry Ford… MEMIT ✗ The Ford Territory was created by Ford Motor Company, founded by Henry Ford in 1903… Figure 14: Additional composition case study with error highlighting. Decomposition (UnKEBench) — Llama Edited passage (injected as training text): Samuel Wells Williams is a well-known motion picture director who has directed several critically acclaimed films. He has won numerous awards for his work in the film industry, including the prestigious Academy Award for Best Director. Williams has also been praised for his ability to bring out the best in his actors, and for his innovative and creative approach to filmmaking. His films have been box office successes, and have been enjoyed by audiences around the world. In addition to his work as a director, Williams is also a respected writer and producer, and has contributed to many successful films in these roles as well. Sub-question: What awards has Samuel Wells Williams won? Gold: the Academy Award for Best Director, and others. HPSE (ours) ✓ He has won the Academy Award for Best Director, the Golden Globe… COIN⋆ ✗ Samuel Wells Williams is a well-known motion picture director who has directed several critically acclaimed films. He has won numerous awards… (passage verbatim) LoRA ✗ …the Academy Award for Best Motion Picture for “The Grand Budapest Hotel”…. Figure 15: Additional decomposition case study with error highlighting. Decomposition (UnKEBench) — Llama Edited passage (injected as training text): Marjorie Margolies-Mezvinsky is a well-known novelist who has published several best-selling books. Her first novel, “The Art of Forgiveness,” was published in 2005 and received critical acclaim for its poignant portrayal of a family struggling to come to terms with a tragic loss. Since then, Margolies-Mezvinsky has published three more novels, all of which have been widely praised for their insightful and compassionate depictions of human relationships. In addition to her successful writing career, Margolies-Mezvinsky is also a respected public figure, having served as a member of the U.S. House of Representatives from 1993 to 1995. Sub-question: When was her first novel published? Gold: in 2005. HPSE (ours) ✓ Her first novel, “The Art of Forgiveness,” was published in 2005. COIN⋆ ✗ Her first novel, “The Art of Forgiveness,” was published in 2005 and received critical… (passage verbatim continued) AnyEdit ✗ Her first novel, “The Political Scene,” was published in 2005…. Figure 16: Additional decomposition case study with error highlighting.