Paper deep dive
Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing
Ding Wu, Ye Zhang, Haoyu Wang, Tianci Liu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 10/5/2026, 6:53:40 AM
Summary
The paper introduces FOVEATED, a plug-and-play framework for Unstructured Knowledge Editing (UKE) that improves atomic-fact recall by addressing 'context reliance' and 'context-induced difficulty underestimation'. FOVEATED works by randomly shifting Rotary Position Embedding (RoPE) positions for preceding context during editing, forcing the model to learn facts from focused views rather than relying on contextual shortcuts. It is applied to both Direct Optimization (DO) and Locate-Then-Edit (LTE) methods, showing consistent improvements across benchmarks without affecting inference-time positional encodings.
Entities (10)
Relation Signals (8)
FOVEATED → solves → Context Reliance
confidence 95% · FOVEATED... constructs focused views... to mitigate this problem... context reliance
FOVEATED → mitigates → Context-Induced Difficulty Underestimation
confidence 92% · FOVEATED counteracts context-induced difficulty underestimation
FOVEATED → uses → Rotary Position Embedding
confidence 90% · randomly shifting the Rotary Position Embedding (RoPE) positions
FOVEATED → appliesto → Locate-Then-Edit
confidence 85% · We instantiate FOVEATED for both direct-optimization and locate-then-edit editors
FOVEATED → appliesto → Direct Optimization
confidence 85% · We instantiate FOVEATED for both direct-optimization and locate-then-edit editors
FOVEATED → evaluatedon → Qwen2.5-7B-Instruct
confidence 80% · we evaluate Qwen2.5-7B-Instruct... Applying FOVEATED consistently improves atomic-fact recall
FOVEATED → evaluatedon → Llama-3.1-8B-Instruct
confidence 80% · we select two LLMs... Llama-3.1-8B-Instruct... Applying FOVEATED consistently improves
Unstructured Knowledge Editing → uses → UnFine
confidence 80% · UnFine... enable finer-grained evaluation of UKE
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) increasingly serve as general-purpose interfaces to factual knowledge, but their parameters do not automatically reflect information that changes after pretraining. Knowledge editing (KE) provides a targeted alternative to costly retraining by modifying selected knowledge and preserving unrelated knowledge and general capabilities. Conventional KE uses structured factual triples, whereas unstructured KE (UKE) uses free-form passages containing multiple facts. Nonetheless, existing UKE editors exhibit a failure mode known as context reliance: edited LLMs can often reproduce the editing passage but fail to reliably recall its individual facts without the original passage context. We identify context-induced difficulty underestimation under the standard passage-level editing objective: later facts receive increasingly rich ground-truth context and consequently incur lower initial losses, making them appear easier to learn. In response, we propose FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the Rotary Position Embedding (RoPE) positions assigned to the keys of its preceding context. The perturbation is applied during editing and removed afterward, leaving the model's native positional encoding unchanged at inference time. We instantiate FOVEATED for both direct-optimization and locate-then-edit editors. We theoretically analyze how FOVEATED counteracts context-induced difficulty underestimation and empirically demonstrate consistent improvements across five KE editors, two LLM backbones, and three benchmarks.
Tags
Links
- Source: https://arxiv.org/abs/2610.02772v1
- Canonical: https://arxiv.org/abs/2610.02772v1
Trouble viewing inline? Open PDF directly →
Full Text
102,600 characters extracted from source content.
Expand or collapse full text
Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Ding Wu 1,* Ye Zhang 2,* Haoyu Wang 3 Tianci Liu 4 1 Georgia Institute of Technology 2 University of Pennsylvania 3 University at Albany 4 University of Tennessee Large language models (LLMs) increasingly serve as general-purpose interfaces to factual knowledge, but their parameters do not automatically reflect information that changes after pretraining. Knowledge editing (KE) provides a targeted alternative to costly retraining by modifying selected knowledge and preserving unrelated knowledge and general capabilities. Conventional KE uses structured factual triples, whereas unstructured KE (UKE) uses free-form passages containing multiple facts. Nonetheless, existing UKE editors exhibit a failure mode known as context reliance: edited LLMs can often reproduce the editing passage but fail to reliably recall its individual facts without the original passage context. We identify context-induced difficulty underestimation under the standard passage-level editing objective: later facts receive increasingly rich ground-truth context and consequently incur lower initial losses, making them appear easier to learn. In response, we propose FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the Rotary Position Embedding (RoPE) positions assigned to the keys of its preceding context. The perturbation is applied during editing and removed afterward, leaving the model’s native positional encoding unchanged at inference time. We instantiate FOVEATED for both direct-optimization and locate-then-edit editors. We theoretically analyze how FOVEATED counteracts context-induced difficulty underestimation and empirically demonstrate consistent improvements across five KE editors, two LLM backbones, and three benchmarks. Keywords: Knowledge Editing, Large Language Models Date: 2026-10-05 Contact: tliu43@tennessee.edu 1. Introduction Large language models (LLMs) have become a foundation for modern natural language processing tasks (Vaswani et al., 2017, Brown et al., 2020, Zhou et al., 2023). As model scale continues to grow, these capabilities make LLMs increasingly useful as general-purpose interfaces for accessing and reasoning with knowledge (Bom- masani et al., 2021, Roberts et al., 2020, Kojima et al., 2022). However, an LLM’s parametric knowledge remains largely fixed after pretraining and can quickly become outdated as real-world knowledge evolves, leading to factual errors in downstream applications (De Cao et al., 2021, Hartvigsen et al., 2023). Keeping the model current through repeated retraining is prohibitively expensive. As a remedy, knowledge editing (KE) addresses this problem by incorporating specific new knowl- edge through selective model updates and preserving unrelated knowledge and general capabilities (Wang et al., 2023, Yao et al., 2023, Zhang et al., 2024b). Standard KE typically represents new knowledge as a separate factual association, often a (subject, relation, ∗ These authors contributed equally to this work. arXiv:2610.02772v1 [cs.CL] 2 Oct 2026 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing object) triple. Given such structured facts, some methods directly optimize selected parameters (Wang et al., 2024a,b), while locate-then-edit (LTE) methods first identify where the target knowledge is represented and then write a targeted update into the corresponding layers (Meng et al., 2022, 2023, Fang et al., 2025). Recent work relaxes this structured-input assumption through unstructured knowledge editing (UKE), where the new knowledge is provided as a free-form passage (Wu et al., 2024a, Yang et al., 2025). Unlike a structured edit, such a passage may express multiple interrelated facts without specifying which ones a user may later query. UKE methods accordingly expand the editable modules and decompose the passage into smaller units and coordinate their updates (Deng et al., 2025, Jiang et al., 2025, Su et al., 2025). Together, these methods expand the scope of KE from individual factual associations to free-form passages. Nonetheless, existing methods often fall short of making effective use of the edited knowledge beyond the original editing context. In particular, an edited model may recall the complete passage under the original editing query, yet fail to retrieve the new knowledge when answering more targeted questions about individual facts (Zhou et al., 2026, Wang et al., 2026). Recent studies have begun to examine this failure mode, termed context reliance, in which the new knowledge may remain accessible only as part of the complete passage, where the edited model can rely on the preceding context as a shortcut during generation. To mitigate this problem, Zhou et al. (2026) align predictions under global and local contexts; Wang et al. (2026) synthesize atomic question–answer supervision with external models; and Liu et al. (2026) construct hybrid self-distillation trajectories from a privileged in-context state. Yet the learning dynamics underlying context reliance remain underexplored. In light of this gap, we examine how context affects the apparent learning difficulty under the standard passage-level objective in this work. Interestingly, we find that the pre-update editing loss decreases for facts appearing later in the passage, even though all edited facts are new to the model (Wu et al., 2024a, Deng et al., 2025). Intuitively, under teacher forcing (Lamb et al., 2016), later content is predicted from longer ground-truth prefixes and thus receives richer contextual support. This growing support acts as a shortcut, lowering the initial losses of later facts and making them appear easier to learn. We call this context-induced difficulty underestimation; Sec. 2 details our analysis. This observation is the first contribution of this work. In response, we propose FOVEATED to construct focused views that make the preceding context less reliable as a shortcut during editing. By varying how a target sentence retrieves its preceding context while preserving the sentence itself, FOVEATED trains the model on each fact across randomized contextual views and directs the editing signal toward the target content. Concretely, FOVEATED constructs such views by randomly shifting the key-side Rotary Position Embedding (RoPE; Su et al., 2021) positions assigned to the prompt and preceding sentences, while leaving all query rotations and the target sentence’s key rotations unchanged. The perturbation is applied only during editing and removed afterward, leaving the model’s native positional encoding unchanged at inference time. We further provide theoretical support for the focused-view design. FOVEATED is a flexible framework that intervenes in the editing computation without prescribing how the updates are parameterized. It can be applied to a wide range of existing editors, and we instantiate FOVEATED for two broad families. Direct optimization (DO) editors update selected parameters with gradient descent (Hu et al., 2022, Wang et al., 2024a,b, Liu et al., 2025b). For this family, FOVEATED generalizes the standard passage-level objective to a weighted expectation of sentence-level losses under focused views. Setting the perturbation magnitude to zero exactly recovers the standard passage-level objective. For LTE editors (Deng et al., 2025, Su et al., 2025), applying FOVEATED is less direct because their passage objectives optimize intermediate hidden targets rather than the eventual parameter updates. We follow Wang et al. (2026) and adopt a two-stage design by first writing sentence-level targets into shallow layers and then applying the base editor in deeper layers to learn the complete passage. The key distinction of our approach is that FOVEATED obtains its sentence-level targets directly from the editing passage under 2 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing 255075100125 position id initial loss UnFine-UnKE 20406080100 position id UnFine-CF 20406080100 position id UnFine-MQ Native FOVEATED view Figure 1: Pre-edit NLL loss in UnFine passages. Native loss gradually decreases toward later passage positions, and FOVEATED views yield consistently higher losses at the same positions. focused views, without synthesizing atomic question–answer supervision through external models. See Sec. 3 for details. This general editing framework and our theoretical analysis form the main technical contribution of this work. The remainder of this paper is organized as follows. Sec. 2 introduces context-induced difficulty underesti- mation, Sec. 3 presents FOVEATED and its theoretical analysis, and Sec. 4 evaluates the method. Secs. 5 and 6 review related work and conclude the paper, respectively. 2. Difficulty Underestimation in Unstructured Knowledge Editing This section reveals context-induced difficulty underestimation in UKE. Background is also provided. 2.1. Preliminaries For a text sequence x of length ∣x∣, an autoregressive LLM π(⋅) factorizes its likelihood as π(x) = ∣x∣ ∏ t=1 π(x t ∣ x 1 , . . . , x t−1 ) ≜ ∣x∣ ∏ t=1 π(x t ∣ x <t ), whereπ(x t ∣ x <t )is the predicted probability assigned to tokenx t given its prefixx <t . To generate text, the LLM sequentially drawsx t fromπ(⋅ ∣ x <t )and appends it to the prefix, until an end-of-sequence token is produced, or the length limit is reached (Bengio et al., 2000). Knowledge Editing (KE) modifies selected knowledge in a pre-trained LLM and preserves knowledge outside the scope of the edit (Wang et al., 2023, Zhang et al., 2024b). Structured KE represents the knowledge to be updated as a factual triple (subject, relation, object), with the edit replacing the object with a new value. The edit is typically specified by a prompt-response pair(x, y), wherexidentifies the subject and relation, and ythe updated object. Unstructured KE (UKE) instead associatesxwith a free-form passagecrather than explicitly annotated triples (Wu et al., 2024a, Yang et al., 2025). A passage may express multiple facts, each a separately queryable atomic fact (Wang et al., 2026). We focus on atomic-fact recall in UKE. 2.2. Learning Difficulty under Teacher Forcing UKE aims to edit multiple new facts encoded in a free-form passage. However, as revealed by Zhou et al. (2026), Wang et al. (2026), many UKE methods encounter a context reliance failure mode: recall of the 3 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing edited knowledge depends heavily on the context supplied during editing. When that context is absent, the edited LLMs struggle to recall the new knowledge, especially facts appearing later in the passage. Such failure limits the effectiveness of UKE beyond the original editing context. Whereas this pattern appears to be a contextual “overfitting” problem of KE (Zhang et al., 2024a, Liu et al., 2025a), we show that it is analogous to “underfitting”: the presence of contextual support makes new knowledge appear easy to predict even when the LLM has not sufficiently learned it. Specifically, although (U)KE methods differ in the variables they optimize to incorporate new knowledge (Fang et al., 2025, Jiang et al., 2025), their supervision often comes from fitting the editing passage through a negative log-likelihood (NLL) loss (Liu et al., 2025a). Letθdenote the editor-specific optimization variables andπ θ the corresponding edited model. Given promptxand passagec, a common full-passage objective is ℒ NLL (θ) = − 1 ∣c∣ ∣c∣ ∑ t=1 log π θ (c t ∣ x, c <t ),(1) where the ground-truthc <t is provided to predict tokenc t , a procedure called teacher forcing (Lamb et al., 2016). Intuitively, the prefix provides increasingly rich context as the passage unfolds. Consequently, even if the LLM does not yet hold the new knowledge, it can still see the fact as a plausible continuation of the surrounding text. This contextual shortcut can make the loss understate the difficulty of acquiring the knowledge. We call this effect context-induced difficulty underestimation. To examine the effect empirically, we evaluate Qwen2.5-7B-Instruct (Qwen Team, 2024) on free-form passages from UnFine (Wang et al., 2026) and measure the initial token-level NLL before any editing updates. Although these passages are designed to contain new knowledge throughout (Deng et al., 2025, Wang et al., 2026), Fig. 1 shows a clear decrease in initial loss toward later positions. Later content thus appears easier to predict despite still introducing new knowledge, consistent with the growing prefix providing contextual support before the model has learned anything from the edit. To test whether intrinsic sentence difficulty explains the decline, we randomly shuffle the sentence order in each passage and observe the same pattern (App. D.1). To address this issue, we introduce the plug-and-play method FOVEATED in the next section. 3. Proposed Method We propose FOVEATED (FOcusedViEws ofATomic facts in unstructured knowledgeEDiting) and integrate it with multiple UKE editors. 3.1. Breaking Context Shortcuts through Positional Encoding Perturbation Given the difficulty underestimation induced by contextual shortcuts, we propose to make the context less reliable as a shortcut during the editing phase, so that the fact needs to be learned consistently across randomized views of the surrounding context. Inspired by how LLMs use positional encoding to understand context (Vaswani et al., 2017, Press et al., 2021), and how manipulating positional encoding during fine-tuning can help them handle contexts longer than those seen during training (Zhu et al., 2024, Peng et al., 2024), FOVEATED randomly perturbs selected positional encodings to break the context shortcut in a fine-grained way. Intuitively, when positional information is 4 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing perturbed, the shortcut becomes less reliable, encouraging the LLM to learn the new knowledge through a more focused view of the current content. We focus on Rotary Position Embedding (RoPE; Su et al., 2021), which is widely used in modern LLMs (Dubey et al., 2024, Qwen Team, 2024, Liu et al., 2024). Given an attention head of even dimensiond h , leth m denote the input hidden state at positionm. The attention head mapsh m intod h -dimensional query and key vectors viaW Q andW K , respectively. To incorporate relative-position information into attention score computation, RoPE rotates the query and key vectors based on their positions before computing the attention score. The query and key vectors lie inC d h /2 , with coordinatejrotated bymω j at positionm. LetΘ = diag(ω 1 , . . . , ω d h /2 ) collect the rotation frequencies. The rotated query and key vectors are given by q t = e itΘ W Q h t ,k s = e isΘ W K h s , whereiis the imaginary unit, andq t andk s denote the rotated query and key at positionstands. The attention score is then given by a ts = Re(q ∗ t k s )/ √ d h = Re ((W Q h t ) ∗ e i(s−t)Θ (W K h s )) / √ d h , where ∗ andRe(⋅)denote conjugate transpose and the real part, respectively. The terme i(s−t)Θ encodes the relative position s− t, allowing the attention score to capture distance information. Using this notation, FOVEATED randomly perturbs the key-side RoPE. For each target sentence, letℬcontain the prompt and each preceding sentence as separate context blocks. We independently sample one offset for every block b ∈ ℬ, ε b ∼ 풩 (0, σ 2 ),D b = round(ε b ), whereσcontrols the perturbation scale. Every eligible key positionsin blockbusesd s = D b , whiled s = 0in the target sentence. We then apply these offsets to the context keys while keeping all query rotations and the target sentence’s key rotations unchanged: ̃q t = q t , ̃ k s = e i(s+d s )Θ W K h s = e id s Θ k s , ̃a ts = Re ((W Q h t ) ∗ e i(s+d s −t)Θ (W K h s )) / √ d h . Each block offset is shared across attention layers and resampled at every step. As shown in Fig. 1, the resulting focused views generally exhibit higher initial losses than the native view. The perturbation is removed after editing, and the LLM returns to its original RoPE. The following theorem gives a mean-margin and view-consistency decomposition of FOVEATED. Theorem 3.1 (Mean-margin and view-consistency decomposition). Letm 0 andm D denote the native and perturbed margins of a target token, whereDdenotes the sampled positional offsets, withℓ(m) = log(1+ e −m ). Under the integrability condition E D ∣m D ∣ < ∞, we have E D [ℓ(m D )] =ℓ(m 0 −∆)+ ℛ,ℛ ≥ 0, where∆ = m 0 − E D [m D ]is the change in mean margin andℛis an expected Bernoulli KL divergence that vanishes if and only if the correct-token confidence is constant across views. Under additional positive-shortcut assumptions (App. A.4),∆ ≥ 0 holds and hence E D [ℓ(m D )] ≥ℓ(m 0 ). In words, FOVEATED aims to learn a fact prediction that is stable across randomized positional views of the context:ℛpenalizes view-dependent confidence, and, under a positive shortcut, the term∆additionally reduces the mean contextual support. The complete version and proofs are in App. A. 5 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing 3.2. Integrating FOVEATED with Existing Editors We integrate FOVEATED with two editor families: methods that directly optimize selected parameters (Zhang et al., 2024b, Zhou et al., 2026), and locate-then-edit methods that write hidden targets into model weights (Fang et al., 2025, Jiang et al., 2025). Direct Optimization (DO) editors update selected parameters to minimize the passage NLL. Assume the passage containsMsentences, withℐ m denoting the token positions of sentencem. FOVEATED reorganizes the passage NLL into sentence-level losses and applies key perturbations to each sentence’s preceding context. For sentence m, the FOVEATED-induced loss is given by ℓ m (θ, D m ) = − 1 ∣ℐ m ∣ ∑ t∈ℐ m log π θ ( c t ∣ x, c <t ; ̃ K(D m ) ) ,(2) where ̃ K(D m )denotes the keys randomly perturbed by offsetsD m in the context preceding sentencem, as defined in Sec. 3.1. The final objective is a weighted expectation of sentence-level losses: ℒ FOVEATED (θ) = M ∑ m=1 ∣ℐ m ∣ ∣c∣ E D m [ℓ m (θ, D m )].(3) As shown in App. A, using zero positional offsets in all context blocks reduces Eq.(3)to Eq.(1). We keep the original DO editors’ trainable parameters and optimization settings unchanged. Locate-Then-Edit (LTE) editors first determine how updating some hidden states can lead to the desired knowledge update, then write the updates into selected model layers using either closed-form solvers or gradient descent on intermediate-state reconstruction losses (Deng et al., 2025, Jiang et al., 2025, Su et al., 2025). Because LTE minimizes the NLL over intermediate states rather than model parameters, adapting FOVEATED requires an additional weight-writing step. We therefore use a two-stage LTE adaptation, following the structure of Wang et al. (2026): sentence-level targets are first written into shallow MLP layers, after which the base LTE editor learns the complete passage in deeper layers. Unlike FABLE, FOVEATED derives the sentence targets directly from the editing passage under focused views, without externally synthesized QA pairs. Stage 1. Sentence-level LTE. For sentencem, we learn a trainable residualδ m that is added to the MLP outputs at prompt positions 풜 m at a shallow layer z 1 . We optimize the residual on the sentence’s tokens within the full teacher-forced passage, with key perturbations applied to its preceding context: δ ⋆ m = arg min δ m E D m [ − 1 ∣ℐ m ∣ ∑ t∈ℐ m log π θ 0 ,δ m ( c t ∣ x, c <t ; ̃ K(D m ) ) ] .(4) After optimizing all sentence-level residuals[δ 1 , . . . , δ M ], we jointly write the resulting targets into the shallow MLP weights using a closed-form solver (Fang et al., 2025). Stage 2. Passage-level LTE. Starting from the above model, we apply the base LTE editor and update deeper layers. Following Wang et al. (2026), we penalize changes in the hidden states at sentence-level anchor positions to preserve the Stage 1 edits by adding an MSE term to the original editing objective. See Appendices B.3 and C.1 for further implementation details. 1 These anchors include a temporary sentence-specific carrier; see App. B.2 for construction details. 6 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing 4. Experiments We evaluate FOVEATED with five UKE editors spanning the DO and LTE families. 4.1. Experimental Setup Base Models. We select two LLMs widely used in UKE studies (Jiang et al., 2025, Su et al., 2025, Zhou et al., 2026, Wang et al., 2026): Qwen2.5-7B-Instruct (Qwen Team, 2024) and Llama-3.1-8B-Instruct (Dubey et al., 2024), which we refer to as Qwen2.5 and Llama3.1, respectively. Tasks. We use UnFine (Wang et al., 2026), which augments existing benchmarks to enable finer-grained evaluation of UKE. It contains three subsets: (1) UnFine-UnKE extends UnKEBench (Deng et al., 2025); (2) UnFine-CF and (3) UnFine-MQ extend the CounterFact (Meng et al., 2022) and MQuAKE (Zhong et al., 2023) subsets of Wu et al. (2024a), respectively. We adopt the single-request setting from prior UKE work (Deng et al., 2025, Wang et al., 2026). Editing Methods. We evaluate FOVEATED on five editing methods spanning both DO and LTE families. Among the DO editors, FT-M (Zhang et al., 2024b) fine-tunes the MLP layer identified by causal tracing, LoRA (Hu et al., 2022) learns additive low-rank updates to model parameters, and a simplified variant of COIN ⋆ (Zhou et al., 2026) fine-tunes the model on the passage and regularizes against context reliance 2 . For the LTE family, we studyμKE (Su et al., 2025) and UnKE (Deng et al., 2025), two representative UKE editors. All paired comparisons hold supervision fixed to the editing query and passage; we therefore do not include FABLE, which additionally uses externally synthesized atomic-QA supervision. Implementation details are provided in App. C. We report computation overhead of FOVEATED in App. D.5. FOVEATED adds moderate editing overhead but no inference cost. Evaluation. Following UnFine (Wang et al., 2026), we assess both holistic passage recall and atomic-fact recall. Specifically, for holistic recall, we report ROUGE-L recall, which measures lexical overlap through the longest common subsequence (Lin, 2004), under the original editing query (Orig Q) and its paraphrase (Reph Q), together with their mean (Avg.). For atomic-fact recall, we separately query the model with atomic questions (Atom Q) targeting the individual facts in each passage. We report ROUGE-L, Hit Rate (HR), andC LCS . HR measures complete key-phrase recovery, andC LCS measures partial reference-answer coverage (Wang et al., 2026). We further evaluate general capability preservation on six standard benchmarks after editing on UnFine-UnKE. More details are provided in App. C.2. 4.2. Overall Performance Tab. 1 reports the editing performance of FOVEATED on the three UnFine benchmarks. Although base editors maintain high editing performance under Orig Q, their Atom Q performance is markedly worse, showing that passage-level recall does not necessarily translate into access to the individual facts. Applying FOVEATED consistently improves atomic-fact recall. For example, atomic ROUGE-L improves in all 30 scenarios by4.4points on average. Similar trends hold for HR andC LCS , which improve in 29 out of 30 scenarios, with the only decreases below two points. Moreover, the improvement of FOVEATED does not come at the cost of holistic editing performance. Orig Q scores remain largely unchanged, while Reph Q improves in 19 of the 20 applicable scenarios by8.4points on average, indicating more robust holistic recall under query reformulation. 2 Our COIN ⋆ adopts its original context-alignment objective to test whether FOVEATED offers complementary gains against context reliance. Other general-purpose locality-preserving mechanisms are omitted to keep the comparison focused. 7 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Table 1: Overall editing performance (mean±SE); higher is better. Gains over the base editors are shaded. Missing Reph Q entries are “–”. Qwen2.5Llama3.1 Holistic (ROUGE-L)Atom QHolistic (ROUGE-L)Atom Q Method Orig Q Reph Q Avg. ROUGE-L Hit Rate C LCS Orig Q Reph Q Avg. ROUGE-L Hit Rate C LCS UnFine-UnKE LoRA 100.0 ±0.0 95.3 ±0.5 97.6 ±0.3 40.8 ±0.9 40.2 ±1.2 51.9 ±0.9 100.0 ±0.0 96.5 ±0.4 98.2 ±0.2 39.2 ±0.8 43.8 ±1.1 57.1 ±0.9 +Ours 99.7 ±0.2 99.1 ±0.2 99.4 ±0.2 53.4 ±0.8 61.8 ±1.0 69.9 ±0.8 99.9 ±0.1 99.0 ±0.2 99.4 ±0.1 45.4 ±0.7 56.8 ±1.0 66.5 ±0.8 COIN ⋆ 100.0 ±0.0 97.4 ±0.4 98.7 ±0.2 36.8 ±0.7 34.7 ±1.0 49.0 ±0.8 100.0 ±0.0 96.5 ±0.5 98.3 ±0.3 36.8 ±0.7 36.7 ±1.1 53.2 ±0.8 +Ours 99.3 ±0.2 98.4 ±0.3 98.9 ±0.2 43.9 ±0.7 47.8 ±1.0 59.2 ±0.8 100.0 ±0.0 98.2 ±0.3 99.1 ±0.2 41.2 ±0.8 45.7 ±1.1 60.2 ±0.8 FT-M 99.9 ±0.1 78.3 ±1.4 89.1 ±0.7 23.2 ±0.6 16.1 ±0.8 30.0 ±0.7 100.0 ±0.0 96.6 ±0.5 98.3 ±0.2 35.3 ±0.7 34.8 ±1.0 51.6 ±0.8 +Ours 99.2 ±0.2 81.5 ±1.2 90.3 ±0.6 26.8 ±0.7 21.1 ±0.9 35.1 ±0.8 100.0 ±0.0 98.5 ±0.2 99.2 ±0.1 40.9 ±0.8 45.3 ±1.1 59.8 ±0.8 μKE 98.8 ±0.3 80.8 ±1.1 89.8 ±0.6 24.4 ±0.6 14.8 ±0.7 33.8 ±0.7 97.5 ±0.5 90.9 ±0.8 94.2 ±0.6 25.9 ±0.7 19.6 ±0.9 37.6 ±0.8 +Ours 99.5 ±0.2 83.0 ±1.0 91.3 ±0.5 29.9 ±0.6 24.2 ±0.9 43.5 ±0.8 99.8 ±0.1 93.0 ±0.6 96.4 ±0.3 29.3 ±0.6 23.6 ±0.9 42.7 ±0.7 UnKE 89.6 ±0.8 66.4 ±1.1 78.0 ±0.8 24.8 ±0.6 14.5 ±0.7 33.8 ±0.7 89.7 ±0.9 82.1 ±1.1 85.9 ±0.9 23.3 ±0.6 16.7 ±0.8 34.3 ±0.8 +Ours 87.1 ±0.8 70.6 ±1.0 78.8 ±0.8 29.7 ±0.6 22.7 ±0.9 42.7 ±0.8 93.9 ±0.6 85.4 ±0.8 89.7 ±0.6 29.5 ±0.6 25.6 ±0.9 43.7 ±0.8 UnFine-CF LoRA 99.9 ±0.1 66.5 ±1.5 83.2 ±0.7 56.7 ±0.9 61.4 ±1.0 68.4 ±0.9 100.0 ±0.0 70.5 ±1.3 85.2 ±0.7 54.1 ±0.9 71.2 ±1.0 77.8 ±0.8 +Ours 99.7 ±0.1 95.9 ±0.5 97.8 ±0.3 63.6 ±0.9 76.5 ±0.9 80.8 ±0.7 99.8 ±0.1 92.3 ±0.8 96.0 ±0.4 56.2 ±0.9 77.2 ±0.9 81.9 ±0.7 COIN ⋆ 99.9 ±0.1 73.1 ±1.5 86.5 ±0.7 52.4 ±1.0 57.7 ±1.0 65.2 ±0.9 100.0 ±0.0 60.7 ±1.5 80.4 ±0.8 51.8 ±0.9 63.3 ±1.0 71.2 ±0.8 +Ours 99.2 ±0.2 85.1 ±1.2 92.1 ±0.6 55.1 ±0.9 60.9 ±1.0 68.3 ±0.9 100.0 ±0.1 78.2 ±1.4 89.1 ±0.7 53.6 ±0.9 67.3 ±1.0 74.6 ±0.8 FT-M 100.0 ±0.0 28.4 ±0.9 64.2 ±0.4 39.0 ±0.9 36.5 ±1.0 45.8 ±1.0 100.0 ±0.0 59.4 ±1.5 79.7 ±0.8 50.9 ±0.9 62.1 ±1.0 70.4 ±0.8 +Ours 98.9 ±0.3 37.4 ±1.0 68.2 ±0.5 44.1 ±0.9 42.5 ±1.1 51.8 ±1.0 100.0 ±0.0 78.0 ±1.4 89.0 ±0.7 53.2 ±0.9 67.1 ±1.0 74.4 ±0.8 μKE 98.7 ±0.3 31.5 ±0.7 65.1 ±0.4 33.1 ±0.8 30.1 ±0.9 41.9 ±0.9 98.1 ±0.5 50.7 ±1.2 74.4 ±0.7 38.8 ±0.8 42.3 ±1.0 53.2 ±0.9 +Ours 99.6 ±0.1 47.4 ±0.9 73.5 ±0.5 38.4 ±0.8 38.5 ±1.0 50.2 ±0.9 100.0 ±0.0 49.3 ±1.1 74.6 ±0.6 39.3 ±0.8 40.5 ±1.0 51.8 ±0.9 UnKE 85.8 ±0.9 30.0 ±0.7 57.9 ±0.6 31.7 ±0.8 29.5 ±1.0 41.2 ±0.9 94.2 ±0.7 41.3 ±1.1 67.8 ±0.7 35.7 ±0.8 37.0 ±1.0 48.3 ±0.9 +Ours 89.5 ±0.8 44.0 ±0.8 66.8 ±0.6 38.6 ±0.9 38.2 ±1.0 50.0 ±0.9 93.9 ±0.6 47.2 ±1.1 70.6 ±0.7 38.7 ±0.8 40.7 ±1.0 51.4 ±0.9 UnFine-MQ LoRA 100.0 ±0.0 –100.0 ±0.0 55.7 ±1.1 58.2 ±1.3 65.9 ±1.1 100.0 ±0.0 –100.0 ±0.0 50.2 ±1.1 65.8 ±1.3 72.8 ±1.1 +Ours 99.4 ±0.2 –99.4 ±0.2 63.3 ±1.1 72.3 ±1.1 78.1 ±0.9 100.0 ±0.0 –100.0 ±0.0 54.7 ±1.1 73.4 ±1.1 78.4 ±1.0 COIN ⋆ 100.0 ±0.0 –100.0 ±0.0 53.0 ±1.1 55.1 ±1.3 62.8 ±1.1 100.0 ±0.0 –100.0 ±0.0 49.2 ±1.1 57.0 ±1.3 65.1 ±1.2 +Ours 99.6 ±0.2 –99.6 ±0.2 56.0 ±1.1 60.2 ±1.2 67.8 ±1.1 100.0 ±0.0 –100.0 ±0.0 51.2 ±1.1 62.8 ±1.2 70.0 ±1.1 FT-M 100.0 ±0.0 –100.0 ±0.0 42.0 ±1.1 39.3 ±1.3 48.1 ±1.1 100.0 ±0.0 –100.0 ±0.0 48.5 ±1.1 57.4 ±1.3 65.4 ±1.2 +Ours 97.8 ±0.5 –97.8 ±0.5 46.1 ±1.1 44.0 ±1.3 52.6 ±1.2 100.0 ±0.0 –100.0 ±0.0 50.8 ±1.2 61.6 ±1.3 68.6 ±1.1 μKE 98.5 ±0.5 –98.5 ±0.5 35.7 ±0.9 33.1 ±1.2 45.1 ±1.0 98.5 ±0.5 –98.5 ±0.5 35.9 ±1.0 40.5 ±1.2 50.0 ±1.1 +Ours99.6 ±0.2 –99.6 ±0.2 38.1 ±1.0 38.1 ±1.1 48.4 ±1.0 99.8 ±0.2 –99.8 ±0.2 40.8 ±1.0 41.5 ±1.1 52.3 ±1.0 UnKE 84.9 ±1.2 –84.9 ±1.2 33.6 ±0.9 32.8 ±1.1 43.8 ±1.0 92.3 ±1.0 –92.3 ±1.0 36.1 ±1.0 38.2 ±1.3 49.3 ±1.2 +Ours85.6 ±1.1 –85.6 ±1.1 35.5 ±0.9 35.0 ±1.1 46.2 ±1.0 92.0 ±0.8 –92.0 ±0.8 40.1 ±1.0 41.1 ±1.2 51.8 ±1.1 The improvements hold across both editor families. For DO editors, atomic ROUGE-L increases by10.3% on average, with similar gains in HR andC LCS across every setting. Notably, COIN ⋆ still benefits from FOVEATED despite already including a context-alignment objective, suggesting that the two mechanisms are complementary. LTE editors show the same overall trend, with a12.9%average gain in atomic ROUGE-L and only two minor decreases in the other metrics. These results show that FOVEATED benefits both editing paradigms despite their different update mechanisms. 4.3. Atomic-Fact Recall by Passage Position Fig. 2 reports average atomic-fact recall by question index, which approximately tracks the corresponding fact’s position; complete results are shown in App. D.2. Consistent with Zhou et al. (2026), atomic-fact recall declines toward later question positions. Together with the lower pre-edit losses observed at later passage positions (Fig. 1), these observations support our account 8 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Q1Q2Q3Q4 Q≥5 30 45 60 75 90 C LCS UnFine-UnKE -- Qwen2.5 Q1Q2Q3Q4Q5 UnFine-CF -- Qwen2.5 Q1Q2Q3Q4Q5 UnFine-MQ -- Qwen2.5 Q1Q2Q3Q4 Q≥5 Atomic-question index 30 45 60 75 90 C LCS UnFine-UnKE -- Llama3.1 Q1Q2Q3Q4Q5 Atomic-question index UnFine-CF -- Llama3.1 Q1Q2Q3Q4Q5 Atomic-question index UnFine-MQ -- Llama3.1 LoRA COIN ⋆ FT-M μKE UnKE Figure 2: Atomic-fact recall by question position; higher is better. Solid and transparent bars show performance with and without FOVEATED. Unfilled area represents the performance gap. MMLU MRPC CoLA RTE NLI SST-2 Qwen2.5 — DO-based 0 MMLU MRPC CoLA RTE NLI SST-2 Qwen2.5 — LTE-based 0 MMLU MRPC CoLA RTE NLI SST-2 Llama3.1 — DO-based 0 MMLU MRPC CoLA RTE NLI SST-2 Llama3.1 — LTE-based 0 Pre-EditLoRA COIN ⋆ FT-M μKE UnKE Figure 3: General capability (likelihood F1) after editing on UnFine-UnKE. Solid and dashed lines show performance with and without FOVEATED, respectively. of context-induced difficulty underestimation. Moreover, applying FOVEATED raises recall for most editor–position pairs across the passage, with gains observed across all five editors, three benchmarks, and both LLM backbones. The improvement persists at later positions: the average gain at lateQ≥5is approximately6.0points, comparable to the gains observed fromQ2–Q4. The comparable gains at later positions show that focused views remain effective when longer preceding contexts provide stronger support under teacher forcing. 4.4. General Capability Preservation We evaluate the edited models on six general-purpose language-understanding and reasoning tasks; dataset details are in App. C.3. We summarize the performance in Fig. 3; numbers are in Tab. 12. Across the ten editor–backbone combinations, macro-average F1 changes relative to the base editors by at most4.3points in either direction. Performance remains stable or improves on Qwen2.5, with the clearest gains under LTE. On Llama3.1,μKE improves, whereas the DO editors and UnKE show moderate aggregate 9 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing decreases driven by task-specific variation. The results therefore support broad preservation rather than uniform gains in general capability. 4.5. Ablation Studies Table 2: Ablations on Qwen2.5 using a subset of UnFine-UnKE. Holistic (ROUGE-L)Atom Q Method Orig Q Reph Q ROUGE-L Hit RateC LCS FT-M (DO) 100.081.320.815.928.8 No Pert. 100.081.221.015.529.2 ID Pert. 100.083.322.116.130.3 +Ours99.484.625.119.433.3 μKE (LTE)98.582.124.615.735.9 No Pert.99.670.325.117.937.5 ID Pert.99.676.226.821.140.7 Sent.-only40.138.525.121.440.3 Rev. order48.948.929.122.144.7 +Ours99.985.529.524.245.2 We ablate FOVEATED with FT-M (DO) andμKE (LTE). Tab. 2 examines the perturbation scheme shared by both paradigms and the two-stage writing strategy used for LTE. Perturbation scheme. Removing perturbation (No Pert.) but retaining the FOVEATED objective re- duces atomic ROUGE-L for both editors; FT-M, for example, drops from25.1to21.0. Perturbing shared position IDs (ID Pert.), as in Zhu et al. (2024) and Li et al. (2026), partially recovers the loss but trails the key-only design for both editors, supporting our asymmetric key perturbation. For space, we defer ad- ditional sensitivity analysis on the perturbation scale σto App. D.4, where FOVEATED exhibits notable robustness to the choice of σ over editor-specific ranges. LTE writing strategy. The Sent.-only variant removes passage-level optimization, whereas Rev. order retains both stages but reverses them. Both variants sharply reduce holistic recall; for example, removing the passage-level stage lowers Orig Q/Reph Q from99.9/85.5to40.1/38.5. Reversing the order similarly hurts holistic recall while nearly preserving atomic ROUGE-L (29.1vs.29.5), showing that strong atomic recall alone does not preserve the edited passage as a coherent whole. The passage-level stage and its placement after sentence-level optimization are therefore both important. 4.6. Case Studies We conclude this section with case studies on FOVEATED. Tab. 3 and App. D.6 present representative failures across editors and backbones. Base editors may hallucinate or revert to prior knowledge, resulting in UKE failures. Equipped with FOVEATED, the edited models instead recover the target atomic facts and give fluent answers that remain relevant to the corresponding atomic questions. Table 3: Atomic-fact recall case studies, with errors highlighted. ModelMethod DatasetAtomic query (target)Base-editor output+Ours output Qwen2.5 FT-M UnFine-CF What year was Charles Alfred Pillsbury born? (1842) 18501842 COIN ⋆ UnFine-UnKE At what age was Henry Feilden elected to the local council? (25) Henry Feilden was elected to the local council at the age of 35. Henry Feilden was elected to the local council at the age of 25. μKE UnFine-CF What region is Sheffield located in? (South Yorkshire) Sheffield is a city located in the northern region of England. Sheffield, located in South Yorkshire, England. Llama3.1 LoRA UnFine-UnKE How many people does Herlyn Espinal employ in her company? (over 50 people) Herlyn Espinal is a politician and doesn’t employ anyone . . . Herlyn Espinal is the founder and CEO . . . employing over 50 people . . . UnKE UnFine-MQ Which year was SC Atlético founded? (1903) SC Atlético, a Portuguese sports club, was founded in 1919, marking . . . SC Atlético, a renowned sports club based in Madrid, Spain, was founded in 1903. 10 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing 5. Related Work Knowledge Editing aims to update specific factual knowledge in a pretrained model and preserve unrelated knowledge and the model’s general capabilities (Wang et al., 2023, Zhang et al., 2024b). Optimization-based approaches learn model updates with the editing objective, including selected layer optimization (Zhang et al., 2024b) or learning multiple copies (Wang et al., 2024b). Parameter-efficient methods have also been applied (Hu et al., 2022, Wang et al., 2024a, Wu et al., 2024b, Liu et al., 2025b). On the other hand, Locate-Then-Edit (LTE) methods first identify where a fact is stored, then compute the weight change often in closed form (Meng et al., 2022, 2023, Li et al., 2024, Fang et al., 2025). Unstructured Knowledge Editing (UKE) replaces the curated triple with a free-form passage (Wu et al., 2024a, Yang et al., 2025). UnKE makes transformer blocks editable (Deng et al., 2025), AnyEdit decomposes passages into sequential chunks (Jiang et al., 2025), andμKE coordinates sentence-level updates through a nested editing structure (Su et al., 2025). COIN identifies context reliance during editing and mitigates it by aligning predictions under global and local contexts (Zhou et al., 2026). FABLE uses externally synthesized atomic-QA supervision to write shallow-layer anchors before fitting the passage holistically, and introduces UnFine to measure atomic-fact access (Wang et al., 2026). In concurrent work, HPSE uses hybrid rollouts to recover missing facts with token-level feedback from a privileged in-context state (Liu et al., 2026). In contrast, our FOVEATED targets the learning signal by disrupting contextual support that can suppress gradients before a fact becomes independently retrievable. Positional Encoding. RoPE encodes relative positions by rotating queries and keys (Su et al., 2021) and is widely used in modern LLMs (Dubey et al., 2024, Qwen Team, 2024, Liu et al., 2024). Long-context adaptation also manipulates positions: Peng et al. (2024) scale RoPE frequencies and attention, Zhu et al. (2024) shift the indices of entire chunks, and Li et al. (2026) shift suffix indices and distill the perturbed view from native-view predictions. These methods perturb positions to improve the model’s tolerance to positional variation. FOVEATED instead uses perturbation for a fundamentally different purpose: by randomizing only preceding-context keys and supervising with ground truth rather than native-view predictions, it breaks contextual shortcuts and makes the edited fact robust across contexts. 6. Conclusion In this work, we identify context-induced difficulty underestimation in UKE and propose FOVEATED, which constructs focused positional views to make contextual shortcuts less reliable during editing. Across five editors, two LLM backbones, and three benchmarks, FOVEATED broadly improves fine-grained fact recall and retains holistic recall. Encouraged by these consistent improvements, we plan to extend FOVEATED to handle correlated, continually evolving unstructured knowledge, where new updates may reinforce, revise, or invalidate earlier ones. By tracking these dependencies and selectively preserving or revising previously edited knowledge, we aim to pave the way toward practical UKE in continuously changing real-world environments. References Yoshua Bengio, Réjean Ducharme, and Pascal Vincent. A neural probabilistic language model. In Advances in Neural Information Processing Systems (NeurIPS), 2000. Luisa Bentivogli, Bernardo Magnini, Ido Dagan, Hoa Trang Dang, and Danilo Giampiccolo. The fifth PASCAL recognizing textual entailment challenge. In Proceedings of the Second Text Analysis Conference, TAC 2009, Gaithersburg, Maryland, USA, November 16-17, 2009, 2009. 11 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Christopher M. Bishop. Training with noise is equivalent to Tikhonov regularization. Neural Computation, 7 (1):108–116, 1995. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS), 2020. Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Proceedings of the 2021 conference on empirical methods in natural language processing, 2021. Jingcheng Deng, Zihao Wei, Liang Pang, Hanxing Ding, Huawei Shen, and Xueqi Cheng. Everything is editable: Extend knowledge editing to unstructured data in large language models. In International Conference on Learning Representations (ICLR), 2025. William B. Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), 2005. Abhimanyu Dubey et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat- Seng Chua. AlphaEdit: Null-space constrained knowledge editing for language models. In International Conference on Learning Representations (ICLR), 2025. Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with GRACE: Lifelong model editing with discrete key-value adaptors. In Advances in Neural Information Processing Systems (NeurIPS), 2023. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022. Houcheng Jiang, Junfeng Fang, Ningyu Zhang, Mingyang Wan, Guojun Ma, Xiang Wang, Xiangnan He, and Tat-Seng Chua. AnyEdit: Edit any knowledge encoded in language models. In International Conference on Machine Learning (ICML), 2025. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213, 2022. Alex M Lamb, Anirudh Goyal Alias Parth Goyal, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. Professor forcing: A new algorithm for training recurrent networks. Advances in neural information processing systems, 29, 2016. Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. PMET: Precise model editing in a transformer. In AAAI Conference on Artificial Intelligence (AAAI), volume 38, pages 18564–18572, 2024. 12 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Zichong Li, Chen Liang, Liliang Ren, Tuo Zhao, Yelong Shen, and Weizhu Chen. Shuffle the context: RoPE-perturbed self-distillation for long-context adaptation. arXiv preprint arXiv:2604.14339, 2026. Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81. Association for Computational Linguistics, 2004. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. DeepSeek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. Tianci Liu, Ruirui Li, Zihan Dong, Hui Liu, Xianfeng Tang, Qingyu Yin, Linjun Zhang, Haoyu Wang, and Jing Gao. Mitigating heterogeneous token overfitting in LLM knowledge editing. In International Conference on Machine Learning (ICML), 2025a. Tianci Liu, Ruirui Li, Yunzhe Qi, Hui Liu, Xianfeng Tang, Tianqi Zheng, Qingyu Yin, Monica Xiao Cheng, Jun Huan, Haoyu Wang, and Jing Gao. Unlocking efficient, scalable, and continual knowledge editing with basis- level representation fine-tuning. In The Thirteenth International Conference on Learning Representations, 2025b. Tianci Liu, Zihan Dong, Tianchun Li, Yi-Chung Chen, Qiming Cao, Xingchen Wang, Shiyang Wang, Zichen Miao, Linjun Zhang, Haoyu Wang, and Jing Gao. Hybrid-policy self-editing for composable unstructured knowledge editing, 2026. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems (NeurIPS), 2022. Kevin Meng, Arnab S. Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. In International Conference on Learning Representations (ICLR), 2023. Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. YaRN: Efficient context window extension of large language models. In International Conference on Learning Representations (ICLR), 2024. Ofir Press, Noah A Smith, and Mike Lewis. Train short, test long: Attention with linear biases enables input length extrapolation. arXiv preprint arXiv:2108.12409, 2021. Qwen Team. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2024. Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), 2020. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2013. Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. RoFormer: Enhanced transformer with rotary position embedding. arXiv preprint arXiv:2104.09864, 2021. Zian Su, Ziyang Huang, Kaiyuan Zhang, and Xiangyu Zhang.μKE: Matryoshka unstructured knowledge editing of large language models. In Conference on Language Modeling (COLM), 2025. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 13 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Stefan Wager, Sida Wang, and Percy Liang. Dropout training as adaptive regularization. In Advances in Neural Information Processing Systems, 2013. Haoyu Wang, Tianci Liu, Ruirui Li, Monica Xiao Cheng, Tuo Zhao, and Jing Gao. RoseLoRA: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine- tuning. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 996–1008, 2024a. Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. WISE: Rethinking the knowledge memory for lifelong model editing of large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2024b. Peng Wang, Biyu Zhou, Xuehai Tang, Jizhong Han, and Songlin Hu. FABLE: Fine-grained fact anchoring for unstructured model editing. In Findings of the Association for Computational Linguistics: ACL, 2026. Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. Knowledge editing for large language models: A survey. arXiv preprint arXiv:2310.16218, 2023. Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. Neural network acceptability judgments. Trans- actions of the Association for Computational Linguistics, 7:625–641, 2019. Adina Williams, Nikita Nangia, and Samuel R. Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 2018. Xiaobao Wu, Liangming Pan, William Y. Wang, and Anh T. Luu. AKEW: Assessing knowledge editing in the wild. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024a. Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. ReFT: Representation finetuning for language models. In Advances in Neural Information Processing Systems (NeurIPS), 2024b. Wanli Yang, Fei Sun, Jiajun Tan, Xinyu Ma, Qi Cao, Dawei Yin, Huawei Shen, and Xueqi Cheng. The mirage of model editing: Revisiting evaluation in the wild. In Annual Meeting of the Association for Computational Linguistics (ACL), 2025. Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023. Mengqi Zhang, Xiaotian Ye, Qiang Liu, Pengjie Ren, Shu Wu, and Zhumin Chen. Uncovering overfitting in large language model editing. arXiv preprint arXiv:2410.07819, 2024a. Ningyu Zhang et al. A comprehensive study of knowledge editing for large language models. arXiv preprint arXiv:2401.01286, 2024b. Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning, Christopher Potts, and Danqi Chen. MQuAKE: Assessing knowledge editing in language models via multi-hop questions. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023. Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, et al. A comprehensive survey on pretrained foundation models: A history from bert to chatgpt. arXiv preprint arXiv:2302.09419, 2023. 14 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Zisheng Zhou, Mengqi Zhang, Shiguang Wu, Xiaotian Ye, Chi Zhang, Zhumin Chen, and Pengjie Ren. Uncovering context reliance in unstructured knowledge editing. arXiv preprint arXiv:2602.19043, 2026. Dawei Zhu, Nan Yang, Liang Wang, Yifan Song, Wenhao Wu, Furu Wei, and Sujian Li. PoSE: Efficient context window extension of LLMs via positional skip-wise training. In International Conference on Learning Representations (ICLR), 2024. 15 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing A. Omitted Theoretical Analysis In this section we develop the analysis omitted in the main body due to page limits. A.1. Token-Weighted Aggregation Preserves the Passage Objective Let the editing passage containMsentences with token spansℐ m ,m = 1, . . . , M, wheren m = ∣ℐ m ∣and N = ∑ M m=1 n m = ∣c∣. Without perturbation, the token-averaged loss on sentence m is ℓ m (θ,0) = − 1 n m ∑ t∈ℐ m log π θ (c t ∣ x, c <t ). Proposition A.1 (Token-weighted sentence aggregation). As the sentence spans form a disjoint partition of the passage, weighting each sentence loss by n m /N recovers the passage NLL exactly: M ∑ m=1 n m N ℓ m (θ,0) = − 1 N N ∑ t=1 log π θ (c t ∣ x, c <t ) = ℒ NLL (θ).(5) Proof. We expand the weighted sum. By definition: M ∑ m=1 n m N ℓ m (θ,0) = 1 N M ∑ m=1 n m ℓ m (θ,0) = − 1 N M ∑ m=1 n m 1 n m ∑ t∈ℐ m log π θ (c t ∣ x, c <t ) = − 1 N M ∑ m=1 ∑ t∈ℐ m log π θ (c t ∣ x, c <t ) (a) = − 1 N N ∑ t=1 log π θ (c t ∣ x, c <t ) = ℒ NLL (θ). The step(a)uses the fact that the disjoint sentence spans partition the passage. This completes our proof. A.2. Blockwise RoPE-Frequency Contraction Following the RoPE convention (Su et al., 2021, Peng et al., 2024), we focus on a single attention head of even widthd h = 2R. Write ̄q t = W Q h t and ̄ k s = W K h s for the pre-RoPE query and key in real coordinates, each partitioned into R two-dimensional components. RoPE at position p takes the form R p = diag (Rot(ω 1 p), . . . , Rot(ω R p)) , whereRot(α)denotes the2× 2rotation matrix by angleα. Using the fact thatRot(α) ⊤ = Rot(−α)and Rot(α) Rot(β) = Rot(α+ β), we have R ⊤ t R s = diag (Rot(ω 1 (s− t)), . . . , Rot(ω R (s− t))) . 16 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Therefore, the native query–key logit can be written as a ts = (R t ̄q t ) ⊤ (R s ̄ k s ) √ d h = 1 √ d h R ∑ r=1 ̄q ⊤ t,r Rot(ω r (s− t)) ̄ k s,r . Now fix a target sentencem. As outlined in Sec. 3.1, its preceding context is partitioned into separate blocks: the prompt and each preceding sentence. Let풫denote the prompt token positions andℐ j the token positions of sentence j. The set of context blocks for target sentence m is ℬ m ≜ 풫, ℐ 1 , . . . , ℐ m−1 . For each blockb ∈ ℬ m , letD mb denote the scalar positional offset shared by its eligible key positions. Thus, mindexes the target sentence, andbidentifies a block in that sentence’s preceding context. We collect these block offsets into D m ≜ (D mb ) b∈ℬ m , which is the offset vector used in Eqs. (2) and (3). As will be specified in App. B.1 shortly, offsets are sampled independently across blocks at the same scaleσ. Each sampled offset is shared across attention layers and resampled at every optimization step. An eligible key positions ∈ busesd s = D mb . The target sentence’s key positions and the key at the readout position of its first token use d s = 0, and all query rotations remain unchanged. For an eligible key position s ∈ b, the perturbed logit is ̃a ts (D mb ) = 1 √ d h R ∑ r=1 ̄q ⊤ t,r Rot (ω r (s− t+ D mb )) ̄ k s,r .(6) We show that the logit depends only on the offsetD mb of the block containings, whereasD m specifies the perturbations for the entire sentence-level forward pass. Proposition A.2 (Blockwise spectral contraction). Fix a target sentencem, a context blockb ∈ ℬ m , and the pre-RoPE query and key representations in Eq.(6). Suppose thatD mb has a distribution symmetric about zero, and define ρ mbr ≜ E D mb [cos(ω r D mb )] . Then E D mb [̃a ts (D mb )] = 1 √ d h R ∑ r=1 ρ mbr ̄q ⊤ t,r Rot(ω r (s− t)) ̄ k s,r ,∣ρ mbr ∣ ≤ 1.(7) For the rounded Gaussian used by FOVEATED, letD mb = round(G mb )withG mb ∼ 풩 (0, σ 2 ). Forσ > 0, the exact probability mass function is Pr(D mb = k) =Φ ( k+ 1 2 σ ) −Φ ( k− 1 2 σ ) ,k ∈ Z,(8) 17 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing whereΦdenotes the standard Gaussian cumulative distribution function. The corresponding spectral multiplier is ρ round mbr = ∑ k∈Z Pr(D mb = k) cos(ω r k).(9) Moreover, ∣ρ round mbr ∣ < 1 whenever σ > 0 and ω r ∉ 2πZ. For the continuous Gaussian relaxation D mb ∼ 풩 (0, σ 2 ), the multiplier simplifies to ρ cont mbr = exp ( − σ 2 ω 2 r 2 ) ,(10) which decreases monotonically withσfor every nonzero RoPE frequency. Atσ = 0, both offset distributions concentrate at zero and their multipliers equal 1. Proof. We first establish the result for symmetric offsets, then specialize it to the rounded Gaussian used by FOVEATED and its continuous Gaussian relaxation. General result for symmetric offsets. Since the distribution ofD mb is symmetric about zero,D mb and−D mb have the same distribution. Therefore, E sin(ω r D mb ) = E sin(−ω r D mb ) = −E sin(ω r D mb ), which implies E sin(ω r D mb ) = 0. Hence, the expected rotation is a scalar matrix: E Rot(ω r D mb ) = ( E cos(ω r D mb ) −E sin(ω r D mb ) E sin(ω r D mb ) E cos(ω r D mb ) ) = ρ mbr I 2 . Using the composition rule for rotations, Rot (ω r (s− t+ D mb )) = Rot (ω r (s− t)) Rot(ω r D mb ). Substituting this identity into Eq. (6), with the pre-RoPE representations held fixed, gives E D mb [̃a ts (D mb )] = 1 √ d h R ∑ r=1 ̄q ⊤ t,r Rot(ω r (s− t))E [Rot(ω r D mb )] ̄ k s,r = 1 √ d h R ∑ r=1 ρ mbr ̄q ⊤ t,r Rot(ω r (s− t)) ̄ k s,r . Moreover, ∣ρ mbr ∣ = ∣E [cos(ω r D mb )]∣ ≤ E [∣ cos(ω r D mb )∣] ≤ 1. 18 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Consequently, the operator norm satisfies ∥E [Rot(ω r D mb )]∥ op = ∣ρ mbr ∣ ≤ 1. Rounded Gaussian case. For σ > 0 and k ∈ Z, nearest-integer rounding gives Pr(D mb = k) = ∫ k+1/2 k−1/2 1 √ 2πσ exp ( − g 2 2σ 2 ) dg =Φ ( k+ 1/2 σ ) −Φ ( k− 1/2 σ ) . Since D mb is integer-valued, ρ round mbr = E [cos(ω r D mb )] = ∑ k∈Z Pr(D mb = k) cos(ω r k). The series is absolutely convergent because ∑ k∈Z Pr(D mb = k)∣ cos(ω r k)∣ ≤ 1. For strict contraction, the strictly positive Gaussian density atσ > 0givesPr(D mb = 0) > 0andPr(D mb = 1) > 0. If ω r ∉ 2πZ, then cos(ω r ) < 1, and hence we have 1− ρ round mbr = ∑ k∈Z Pr(D mb = k) [1− cos(ω r k)] ≥ Pr(D mb = 1) [1− cos(ω r )] > 0. Similarly, 1+ ρ round mbr = ∑ k∈Z Pr(D mb = k) [1+ cos(ω r k)] ≥ 2 Pr(D mb = 0) > 0. Thus −1 < ρ round mbr < 1, proving ∣ρ round mbr ∣ < 1. Continuous relaxation case. LetG ∼ 풩 (0, σ 2 )and write its characteristic function asφ(ω) = Ee iωG . For σ > 0, differentiation under the expectation is justified byE∣G∣ < ∞. Letf σ denote the density of풩 (0, σ 2 ). Since g f σ (g) = −σ 2 f ′ σ (g), integration by parts gives φ ′ (ω) = ∫ R ige iωg f σ (g) dg = −iσ 2 ∫ R e iωg f ′ σ (g) dg = −σ 2 ω φ(ω), 19 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing where the boundary term vanishes because the Gaussian density tends to zero at infinity. Together with φ(0) = 1, this differential equation yields φ(ω) = exp ( − σ 2 ω 2 2 ) . Therefore, ρ cont mbr = Re φ(ω r ) = exp ( − σ 2 ω 2 r 2 ) . For ω r ≠ 0 and σ > 0, ∂ ∂σ ρ cont mbr = −σω 2 r exp ( − σ 2 ω 2 r 2 ) < 0, which proves the stated monotonicity. Whenσ = 0, both the rounded and continuous offset distributions concentrate at zero, giving ρ mbr = 1. This completes our proof. This proposition characterizes the direct key rotation for fixed pre-RoPE states; changes to intermediate representations under the full perturbed forward pass are outside its scope. The bound∣ρ mbr ∣ ≤ 1concerns each expected RoPE-frequency component and does not imply that the softmax attention weight scales linearly byρ mbr . The perturbation directly modifies only key rotations, leaving query rotations, value vectors, and the causal mask unchanged. A.3. Exact Loss Decomposition and Editing Gradients Here we connect the proposed positional perturbation to the token-level learning signal. Fix a target tokenc t in sentencemand omit the token indextfor brevity. LetD m = (D mb ) b∈ℬ m denote the sentence’s vector of context-block offsets, as defined in the preceding subsection. For DO, letz m,v (D m ; θ)denote the pre-softmax logit for vocabulary tokenvat the prediction step forc t . These logits are produced by the full perturbed forward pass, including the changes to intermediate representations induced by the offsets. The native and perturbed correct-versus-rest margins are m m,0 (θ) ≜ z m,c t (0; θ)− log ∑ v≠c t exp ( z m,v (0; θ) ) , m m,D m (θ) ≜ z m,c t (D m ; θ)− log ∑ v≠c t exp ( z m,v (D m ; θ) ) . Both margins use the same current parametersθ; the native margin is evaluated with zero positional offsets. Then for a margin value u, the token cross-entropy and its negative derivative are ℓ(u) = log(1+ exp(−u)), C(u) ≜ −ℓ ′ (u) = 1 1+ exp(u) . 20 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Exact loss decomposition. Define μ m (θ) ≜ E D m [ m m,D m (θ) ] , p m,D m (θ) ≜ sigmoid ( m m,D m (θ) ) , p m,μ (θ) ≜ sigmoid (μ m (θ)) . Herep m,D m is the softmax probability assigned to the correct token, andp m,μ is the probability corresponding to the mean margin. In general,p m,μ ≠ E D m [p m,D m ]. We suppress the dependence onθwhen it is clear from context. Then we have the following property. Proposition A.3 (Mean-margin and view-consistency decomposition). Suppose thatE D m ∣m m,D m ∣ < ∞. Define ∆ m ≜ m m,0 − μ m , the token-specific quantity denoted by∆ in Theorem 3.1. Then E D m [ ℓ(m m,D m ) ] =ℓ(μ m )+ ℛ m =ℓ(m m,0 −∆ m )+ ℛ m ,(11) ℛ m ≜ E D m [KL (Bern(p m,μ ) ∥ Bern(p m,D m ))] ≥ 0. Moreover, ℛ m = 0 if and only if p m,D m = p m,μ almost surely. Proof. All expectations in this proof are taken with respect to D m . Define F(u) ≜ log(1+ exp(u)). The integrability assumption ensures that the expectations below are finite, since F(u) ≤ log 2+ ∣u∣,ℓ(u) ≤ log 2+ ∣u∣. For p(u) = sigmoid(u), we have log p(u) = u− F(u), log(1− p(u)) = −F(u). Applying these identities at u = m m,D m and u = μ m gives KL (Bern(p m,μ ) ∥ Bern(p m,D m )) = p m,μ log p m,μ p m,D m + (1− p m,μ ) log 1− p m,μ 1− p m,D m = p m,μ [ μ m − m m,D m − F(μ m )+ F(m m,D m ) ] + (1− p m,μ ) [ F(m m,D m )− F(μ m ) ] = F(m m,D m )− F(μ m )− F ′ (μ m )(m m,D m − μ m ), where F ′ (μ m ) = p m,μ . Taking expectations and using E[m m,D m − μ m ] = 0, we obtain ℛ m = EF(m m,D m )− F(μ m ) = E [ ℓ(m m,D m )+ m m,D m ] − [ℓ(μ m )+ μ m ] = Eℓ(m m,D m )−ℓ(μ m ), 21 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing where the second equality usesℓ(u) = F(u)− u. Rearranging and substitutingμ m = m m,0 −∆ m proves the decomposition. The same identity shows that ℛ m is the Jensen gap of the loss: ℛ m = Eℓ(m m,D m )−ℓ ( Em m,D m ) ≥ 0. Since ℓ ′ (u) = C(u)(1− C(u)) > 0for all u ∈ R, the loss is strictly convex. Equality therefore holds if and only ifm m,D m = μ m almost surely. The sigmoid function is injective, so this is equivalent to p m,D m = p m,μ almost surely. The decomposition provides an analytical interpretation of the perturbed NLL objective:∆ m records the reduction in mean margin, andℛ m measures variation in correct-token confidence across positional views. This connects the objective to the noise-as-regularization viewpoint (Bishop, 1995, Wager et al., 2013). Editing gradients. Now we differentiate the loss decomposition to characterize how positional perturbations change the NLL gradients for DO parameters and LTE Stage 1 residuals. Corollary A.4 (Gradients for parameters and hidden targets). Under the integrability condition of Proposi- tion A.3, suppose that the offset distribution is independent ofθand that the native and perturbed margins are differentiable in θ. Assume that, throughout a neighborhood of θ, ∥∇ θ m m,D m ∥ 2 ≤ H m (D m ),E D m H m (D m ) < ∞. Define the expected token loss 풥 m (θ) ≜ E D m [ ℓ(m m,D m (θ)) ] . Then ∇ θ 풥 m = −C(μ m )∇ θ μ m + ∇ θ ℛ m .(12) Relative to the native token loss, ∇ θ 풥 m − ∇ θ ℓ(m m,0 ) = − [C(μ m )− C(m m,0 )]∇ θ m m,0 + C(μ m )∇ θ ∆ m + ∇ θ ℛ m .(13) Write g m,D m ≜ ∇ θ m m,D m , ̄ g m ≜ E D m g m,D m . If m m,D m and g m,D m have finite second moments, define V margin m ≜ Var(m m,D m ), V grad m ≜ E D m ∥ g m,D m − ̄ g m ∥ 2 2 . 22 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Then ∥∇ θ ℛ m ∥ 2 ≤ V margin m 8 ∥ ̄ g m ∥ 2 + 1 4 √ V margin m V grad m .(14) For LTE Stage 1, the same identities hold with∇ θ replaced by∇ δ m , whereδ m is the sentence-specific residual in Eq.(4)and the model parametersθ 0 are fixed. The native and perturbed margins use the same carrier substitution and residual injection, with zero offsets in the native computation. Proof. All expectations are overD m . We first differentiate the decomposition, then bound the gradient of the consistency term. Differentiating the expectation. The dominating bound and ∣ℓ ′ (u)∣ ≤ 1 give ∥ ℓ ′ (m m,D m )∇ θ m m,D m ∥ 2 ≤ H m (D m ). Differentiation under the expectation is therefore justified, yielding ∇ θ μ m = E [ ∇ θ m m,D m ] = ̄ g m , ∇ θ 풥 m = E [ ℓ ′ (m m,D m )g m,D m ] = −E [ C(m m,D m )g m,D m ] , ∇ θ ℓ(μ m ) = −C(μ m ) ̄ g m . Using ℛ m = 풥 m −ℓ(μ m ), we obtain ∇ θ ℛ m = C(μ m ) ̄ g m − E [ C(m m,D m )g m,D m ] ,(15) which proves Eq. (12). Next, substitute ∇ θ μ m = ∇ θ m m,0 − ∇ θ ∆ m and subtract the native gradient: ∇ θ 풥 m − ∇ θ ℓ(m m,0 ) = −C(μ m ) ( ∇ θ m m,0 − ∇ θ ∆ m ) + ∇ θ ℛ m + C(m m,0 )∇ θ m m,0 = −[C(μ m )− C(m m,0 )]∇ θ m m,0 + C(μ m )∇ θ ∆ m + ∇ θ ℛ m . This proves Eq. (13). Mean and covariance contributions. Let ̄ C m = E[C(m m,D m )]. Centering the product in Eq. (15) gives ∇ θ ℛ m = [C(μ m )− ̄ C m ] ̄ g m − E [ (C(m m,D m )− ̄ C m )(g m,D m − ̄ g m ) ] .(16) We bound the two terms separately. Sincesup u ∣C ′ (u)∣ ≤ 1/4andsup u ∣C ′ (u)∣ ≤ 1/4, Taylor’s formula around μ m gives ∣ ̄ C m − C(μ m )∣ = ∣ E [ C(m m,D m )− C(μ m )− C ′ (μ m )(m m,D m − μ m ) ]∣ ≤ 1 8 E(m m,D m − μ m ) 2 = V margin m 8 . 23 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing The derivative bound also yields Var(C(m m,D m )) ≤ E [(C(m m,D m )− C(μ m )) 2 ] ≤ V margin m 16 . Applying vector Cauchy–Schwarz to the covariance term, ∥ E [ (C(m m,D m )− ̄ C m )(g m,D m − ̄ g m ) ]∥ 2 ≤ √ Var(C(m m,D m )) E∥g m,D m − ̄ g m ∥ 2 2 ≤ 1 4 √ V margin m V grad m . Combining these bounds with the triangle inequality proves Eq.(14). The same argument applies to differentiation with respect to δ m under the corresponding assumptions. This completes our proof. Averaging these token-level identities within each sentence gives the corresponding LTE Stage 1 NLL identities. For DO, further aggregating sentence losses with weights∣ℐ m ∣/∣c∣gives the perturbed NLL objective in Eq.(3). When∆ m ≥ 0, the first term in Eq.(13)reinforces the native margin correction, while the overall gradient change depends on all three terms. Local sensitivity to block offsets. The consistency term admits a local interpretation in terms of margin variance. Let h m,D m ≜ m m,D m − μ m ,Eh m,D m = 0. All expectations below are over D m or, in the continuous construction, its underlying Gaussian vector. Taylor’s formula with an integral remainder gives ℓ(μ m + h m,D m )−ℓ(μ m )−ℓ ′ (μ m )h m,D m = h 2 m,D m ∫ 1 0 (1− u)ℓ ′ (μ m + uh m,D m ) du. Taking expectations yields ℛ m = E [ h 2 m,D m ∫ 1 0 (1− u)ℓ ′ (μ m + uh m,D m ) du ] . If the margin has a finite second moment, then 0 <ℓ ′ ≤ 1/4 implies 0 ≤ ℛ m ≤ 1 8 Var(m m,D m ). When the third centered absolute moment is finite, a second-order expansion gives ℓ(μ m + h m,D m ) =ℓ(μ m )+ℓ ′ (μ m )h m,D m + ℓ ′ (μ m ) 2 h 2 m,D m + q m,D m , 24 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing where q m,D m = h 3 m,D m 2 ∫ 1 0 (1− u) 2 ℓ ′ (μ m + uh m,D m ) du, ∣q m,D m ∣ ≤ ∣h m,D m ∣ 3 24 . The bound follows from∣ℓ ′ ∣ = ∣C(1− C)(2C − 1)∣ ≤ 1/4. Taking expectations and writingr m,3 = Eq m,D m gives ℛ m = ℓ ′ (μ m ) 2 Var(m m,D m )+ r m,3 ,(17) ∣r m,3 ∣ ≤ 1 24 E∣m m,D m − μ m ∣ 3 . We now specialize to a small continuous perturbation D mb = εξ mb ,ξ mb ∼ 풩 (0, σ 2 ),b ∈ ℬ m , with independent draws across blocks. Letξ m = (ξ mb ) b∈ℬ m , and define the sensitivity to each shared block offset by J mb ≜ ∂m m,D m ∂D mb » » » » » » » D m =0 . At the fixed current model state, suppose that the margin admits the expansion m m,εξ m = m m,0 + εY m + ε 2 e m,ε ,Y m = ∑ b∈ℬ m J mb ξ mb , where, uniformly for sufficiently small ε > 0, ∣e m,ε ∣ ≤ K m (ξ m ),EK m (ξ m ) 3 < ∞. For this family of perturbations, μ m and ℛ m are evaluated at D m = εξ m . Independence across blocks gives EY m = 0, EY 2 m = ∑ b,b ′ ∈ℬ m J mb J mb ′ E[ξ mb ξ mb ′ ] = σ 2 ∑ b∈ℬ m J 2 mb , μ m = m m,0 + ε 2 Ee m,ε , h m,εξ m = εY m + ε 2 (e m,ε − Ee m,ε ). Expanding the variance, we obtain Var(m m,εξ m ) = ε 2 EY 2 m + 2ε 3 Cov(Y m , e m,ε ) + ε 4 Var(e m,ε ). 25 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Writing K m = K m (ξ m ) for brevity, by Cauchy–Schwarz inequality, we have » » » » Var(m m,εξ m )− ε 2 EY 2 m » » » » ≤ 2ε 3 √ EY 2 m EK 2 m + ε 4 EK 2 m = O(ε 3 ). Similarly, E∣h m,εξ m ∣ 3 ≤ 4ε 3 E∣Y m ∣ 3 + 4ε 6 E∣e m,ε − Ee m,ε ∣ 3 = O(ε 3 ). Finally, ∣ℓ ′ (μ m )−ℓ ′ (m m,0 )∣ ≤ 1 4 ∣μ m − m m,0 ∣ = O(ε 2 ). Substituting these estimates into Eq. (17) gives ℛ m = ε 2 σ 2 ℓ ′ (m m,0 ) 2 ∑ b∈ℬ m J 2 mb + O(ε 3 ).(18) Each derivativeJ mb accounts for the shared offset’s effect through all eligible keys and attention layers. Thus, to leading order in this continuous local approximation, the consistency term is proportional to the squared sensitivity of the target-token margin to the context-block offsets. For the finite rounded offsets used by FOVEATED, the exact decomposition is given by Eq. (11). A.4. Sign of the Mean-Margin Term under a Local Shortcut Model We now derive sufficient conditions for the mean-margin reduction∆ m = m m,0 − μ m to be nonnegative. Following Appendix A.3, we fix a target token in sentencemand suppress its token index. We connect the spectral contraction in Proposition A.2 to the margin through an affine model of the RoPE rotation components. Assumption A.5 (Positive contextual shortcut). At the current optimization state, the margin is modeled as m m,D m = u m + ∑ b∈ℬ m R ∑ r=1 [A mbr cos(ω r D mb )+ B mbr sin(ω r D mb )] , where u m , A mbr , and B mbr are finite real coefficients held fixed across positional views, and A mbr ≥ 0for every b ∈ ℬ m and r = 1, . . . , R. Here u m is the offset-independent contribution. At zero offsets, the model gives m m,0 = u m + ∑ b∈ℬ m R ∑ r=1 A mbr . The cosine and sine terms arise by expanding the additional rotationRot(ω r D mb )in an affine function of the rotation components. The positivity condition applies to each native block-frequency contributionA mbr individually. 26 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Theorem A.6 (Mean-margin decrease under a positive shortcut). Under the affine margin model in Assump- tion A.5, suppose that each block offset D mb has a distribution symmetric about zero. Let ρ mbr = E D mb [cos(ω r D mb )] be the spectral multiplier from Proposition A.2. Then the mean-margin reduction defined in Proposition A.3 satisfies ∆ m = m m,0 − μ m = ∑ b∈ℬ m R ∑ r=1 (1− ρ mbr )A mbr ≥ 0. Consequently, E D m [ ℓ(m m,D m ) ] =ℓ(m m,0 −∆ m )+ ℛ m ≥ℓ(m m,0 −∆ m ) ≥ℓ(m m,0 ). For fixed native margin m m,0 , the mean-margin lossℓ(m m,0 −∆ m ) increases strictly with∆ m : ∂ ∂∆ m ℓ(m m,0 −∆ m ) = C(m m,0 −∆ m ) > 0. Proof. Under Assumption A.5, the margin is bounded in absolute value by ∣m m,D m ∣ ≤ ∣u m ∣+ ∑ b∈ℬ m R ∑ r=1 (∣A mbr ∣+ ∣B mbr ∣) , so the integrability condition of Proposition A.3 holds. By symmetry of each block-offset distribution, E D mb sin(ω r D mb ) = 0,E D mb cos(ω r D mb ) = ρ mbr . Taking expectations in the affine model therefore gives μ m = E D m [ m m,D m ] = u m + ∑ b∈ℬ m R ∑ r=1 ρ mbr A mbr = ( u m + ∑ b∈ℬ m R ∑ r=1 A mbr ) − ∑ b∈ℬ m R ∑ r=1 (1− ρ mbr )A mbr = m m,0 − ∑ b∈ℬ m R ∑ r=1 (1− ρ mbr )A mbr . Thus, ∆ m = m m,0 − μ m = ∑ b∈ℬ m R ∑ r=1 (1− ρ mbr )A mbr . 27 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Proposition A.2 gives ∣ρ mbr ∣ ≤ 1, hence 1− ρ mbr ≥ 0. Together with A mbr ≥ 0, this proves∆ m ≥ 0. Substituting μ m = m m,0 −∆ m into Eq. (11) yields E D m [ ℓ(m m,D m ) ] =ℓ(m m,0 −∆ m )+ ℛ m . Since ℛ m ≥ 0, the expected loss is at leastℓ(m m,0 −∆ m ). Moreover,ℓ ′ (u) = −C(u) < 0 and∆ m ≥ 0 imply ℓ(m m,0 −∆ m ) ≥ℓ(m m,0 ). Finally, holding m m,0 fixed and differentiating gives ∂ ∂∆ m ℓ(m m,0 −∆ m ) = −ℓ ′ (m m,0 −∆ m ) = C(m m,0 −∆ m ) > 0, which proves the stated monotonicity. A.5. Beyond the Affine Margin Model We now bound the error of the affine approximation by expressing the full margin computation as a function of the RoPE rotation components. As in the preceding subsections, we fix a target token in sentencem, retain the sentence index, and suppress the token index. The current optimization variables are held fixed throughout this analysis. For each context block b ∈ ℬ m and frequency r, define χ mbr (D m ) = ( cos(ω r D mb ) sin(ω r D mb ) ) ,χ (0) mbr = ( 1 0 ) . Letχ m (D m )collect these coordinate pairs over all blocks and frequencies, and letχ (0) m collect their native values. To define the margin as a function of these coordinates, write the additional key-side transformation as T mbr (χ mbr ) = ( χ mbr,1 −χ mbr,2 χ mbr,2 χ mbr,1 ) . At the sampled coordinates, this matrix equalsRot(ω r D mb ). Each transformation is applied to the eligible keys and attention layers sharing blockband frequencyr. We treatχ mbr,1 andχ mbr,2 as separate coordinates when differentiating the margin. Letf m denote the resulting full margin computation, including the changes to intermediate representations. Then m m,D m = f m ( χ m (D m ) ) , m m,0 = f m ( χ (0) m ) . Proposition A.7 (Mean-margin decrease beyond the affine model). Suppose thatf m is twice continuously differentiable on a neighborhood of every segment joining χ (0) m to a possible χ m (D m ), with     ∇ 2 χ m f m     op ≤Λ m 28 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing throughout these segments, whereΛ m is a finite nonnegative constant. Assume that each block offset has a distribution symmetric about zero, and let ρ mbr = E D mb [cos(ω r D mb )] . Define S mbr ≜ ∂ f m ∂χ mbr,1 » » » » » » » χ m =χ (0) m , ∆ lin m ≜ ∑ b∈ℬ m R ∑ r=1 (1− ρ mbr )S mbr , ℰ m ≜Λ m ∑ b∈ℬ m R ∑ r=1 (1− ρ mbr ). Then the actual mean-margin reduction∆ m = m m,0 − μ m satisfies » » » » » ∆ m −∆ lin m » » » » » ≤ ℰ m ,(19) and the expected loss satisfies E D m [ ℓ(m m,D m ) ] ≥ℓ (m m,0 −∆ lin m + ℰ m ) .(20) In particular,∆ lin m > ℰ m implies∆ m > 0 and E D m [ ℓ(m m,D m ) ] >ℓ(m m,0 ). Proof. All expectations are taken with respect to D m . Write e m ≜ χ m (D m )− χ (0) m for the random displacement in the rotation coordinates. Taylor’s theorem along the segmentχ (0) m + ue m , 0 ≤ u ≤ 1, gives m m,D m = m m,0 + ∇ f m (χ (0) m ) ⊤ e m + r m , where r m = ∫ 1 0 (1− u) e ⊤ m ∇ 2 f m (χ (0) m + ue m ) e m du. The Hessian bound yields ∣r m ∣ ≤ ∫ 1 0 (1− u)       ∇ 2 f m (χ (0) m + ue m )       op ∥e m ∥ 2 2 du ≤ Λ m 2 ∥e m ∥ 2 2 . Since the rotation coordinates are bounded, this expansion also ensures that the margin and its loss are integrable. 29 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing For the coordinate pair associated with block b and frequency r, symmetry gives Ee mbr = ( E cos(ω r D mb )− 1 E sin(ω r D mb ) ) = ( ρ mbr − 1 0 ) . Consequently, the expected linear term is ∇ f m (χ (0) m ) ⊤ Ee m = ∑ b∈ℬ m R ∑ r=1 S mbr (ρ mbr − 1) = −∆ lin m . To bound the expected remainder, we compute E∥e m ∥ 2 2 = ∑ b∈ℬ m R ∑ r=1 E [ ( cos(ω r D mb )− 1 ) 2 + sin 2 (ω r D mb )] = 2 ∑ b∈ℬ m R ∑ r=1 E [1− cos(ω r D mb )] = 2 ∑ b∈ℬ m R ∑ r=1 (1− ρ mbr ). It follows that ∣Er m ∣ ≤ E∣r m ∣ ≤ Λ m 2 E∥e m ∥ 2 2 =Λ m ∑ b∈ℬ m R ∑ r=1 (1− ρ mbr ) = ℰ m . Taking expectations in the Taylor expansion therefore gives μ m = m m,0 −∆ lin m + Er m , ∆ m = m m,0 − μ m =∆ lin m − Er m . Thus, ∣∆ m −∆ lin m ∣ = ∣Er m ∣ ≤ ℰ m , which proves Eq. (19). By convexity ofℓ and Jensen’s inequality, E [ ℓ(m m,D m ) ] ≥ℓ(μ m ) =ℓ (m m,0 −∆ lin m + Er m ) ≥ℓ (m m,0 −∆ lin m + ℰ m ) , 30 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing where the last inequality uses Er m ≤ ℰ m andℓ ′ < 0. This proves Eq. (20). Finally, if∆ lin m > ℰ m , then ∆ m ≥∆ lin m − ℰ m > 0. Sinceℓ is strictly decreasing, E [ ℓ(m m,D m ) ] ≥ℓ (m m,0 −∆ lin m + ℰ m ) >ℓ(m m,0 ). This completes the proof. A.6. Conditional Attention-Mass Redistribution We now give a sufficient condition for positional perturbations to increase the attention mass assigned to unperturbed keys. Fix a target sentencem, an attention head, and a query positiontused to predict a token in that sentence. As in Proposition A.2, we hold the pre-RoPE query and key representations fixed. Partition the visible key positions into two disjoint sets:ℋ m contains the context keys designated for perturbation, and풰 m contains the remaining keys. The latter includes visible target-sentence keys and the protected first-token readout key whenever visible. We assume 풰 m is nonempty. Using the logit notation from Proposition A.2, define Z 풰 m ≜ ∑ u∈풰 m e a tu , Z ℋ m (D m ) ≜ ∑ s∈ℋ m e ̃a ts (D m ) , M m (D m ) ≜ Z 풰 m Z 풰 m + Z ℋ m (D m ) . HereM m (D m )is the total attention mass assigned to풰 m , andM m,0 = M m (0)is its native value. Under the fixed-representation analysis, Z 풰 m is constant across positional views. Proposition A.8 (Attention mass under exponential-moment contraction). Suppose that, for some κ > 0, E D m Z ℋ m (D m ) ≤ e −κ Z ℋ m (0).(21) Then E D m M m (D m ) ≥ 1 1+ e −κ (1− M m,0 )/M m,0 .(22) For any η ∈ (0, 1), with probability at least 1− η, M m (D m ) ≥ 1 1+ e −κ (1− M m,0 )/(η M m,0 ) .(23) When ℋ m is nonempty, the expectation bound strictly exceeds M m,0 , and the probability bound strictly exceeds M m,0 if κ > log(1/η). 31 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Proof.All expectations and probabilities are overD m . Ifℋ m is empty, thenM m (D m ) = M m,0 = 1, and both bounds hold. We therefore consider the case Z ℋ m (0) > 0. Define ψ m (z) ≜ Z 풰 m Z 풰 m + z ,z ≥ 0. Since 풰 m is nonempty, Z 풰 m > 0, and ψ ′ m (z) = − Z 풰 m (Z 풰 m + z) 2 < 0, ψ ′ m (z) = 2Z 풰 m (Z 풰 m + z) 3 > 0. Thus ψ m is decreasing and convex. Expected attention mass. By Jensen’s inequality and Eq. (21), EM m (D m ) = Eψ m (Z ℋ m (D m )) ≥ ψ m (EZ ℋ m (D m )) ≥ ψ m (e −κ Z ℋ m (0)) = Z 풰 m Z 풰 m + e −κ Z ℋ m (0) . The native mass satisfies Z ℋ m (0) Z 풰 m = 1− M m,0 M m,0 . Substituting this identity proves Eq. (22). Since κ > 0 and Z ℋ m (0) > 0, the lower bound strictly exceeds Z 풰 m Z 풰 m + Z ℋ m (0) = M m,0 . High-probability attention mass. Applying Markov’s inequality to the nonnegative random variableZ ℋ m (D m ) gives Pr ( Z ℋ m (D m ) > e −κ Z ℋ m (0) η ) ≤ η EZ ℋ m (D m ) e −κ Z ℋ m (0) ≤ η. On the complementary event, which has probability at least 1− η, monotonicity of ψ m yields M m (D m ) ≥ ψ m ( e −κ Z ℋ m (0) η ) = 1 1+ e −κ (1− M m,0 )/(η M m,0 ) . 32 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing This proves Eq. (23). For 0 < M m,0 < 1, this bound exceeds M m,0 precisely when e −κ /η < 1, or equivalently κ > log(1/η). A sufficient condition from spectral contraction. The condition in Eq.(21)can be expressed in terms of the mean decrease and fluctuations of each perturbed logit. For s ∈ ℋ m , define Γ ms ≜ a ts − E D m ̃a ts (D m ). For a key s in block b, Proposition A.2 gives Γ ms = 1 √ d h R ∑ r=1 (1− ρ mbr ) ̄q ⊤ t,r Rot(ω r (s− t)) ̄ k s,r . Suppose that each centered logit satisfies the exponential-moment bound E D m exp ( ̃a ts (D m )− E D m ̃a ts (D m ) ) ≤ exp(v 2 ms /2) for some v ms ≥ 0. Then E D m e ̃a ts (D m ) = exp ( E D m ̃a ts (D m ) ) × E D m exp ( ̃a ts (D m )− E D m ̃a ts (D m ) ) ≤ exp (a ts −Γ ms + v 2 ms /2) . Consequently, a sufficient condition for Eq. (21) is Γ ms − v 2 ms 2 ≥ κfor every s ∈ ℋ m . Indeed, summing the exponential-moment bounds over perturbed keys gives E D m Z ℋ m (D m ) = ∑ s∈ℋ m E D m e ̃a ts (D m ) ≤ ∑ s∈ℋ m e a ts exp (−Γ ms + v 2 ms /2) ≤ e −κ ∑ s∈ℋ m e a ts = e −κ Z ℋ m (0). B. Method Details The FOVEATED pipeline has three components omitted from the main body: the context perturbation settings, the anchor and carrier construction for the LTE first stage, and the shallow weight write that precedes the LTE block fitting. 33 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing B.1. Context Perturbation Settings For a target sentencem, the prompt and each preceding sentence form separate blocks, which use separate perturbations at the same scaleσ. Each block shares one rounded-Gaussian perturbation across attention layers. Draws are independent across blocks and refreshed at each optimization step. The target sentence and its first-token readout position are excluded from the perturbed key positions. B.2. LTE Stage 1: Anchor and Target Construction We now describe the anchor states used to optimize the residuals in Eq.(4). All selection scores and source states are computed with the pre-edited model under native RoPE. Our design is motivated by the intuition that mentioning a concept can evoke several associated pieces of information in parallel. We approximate this effect by treating each sentence as an atomic editing unit and binding all sentence-level targets to shared query-side anchors through a batched write. We find that the construction below works well empirically. Query anchors. Letx = (x 1 , . . . , x n )denote the tokenized editing prompt. We compare its token probabilities under the original modelπ θ 0 and underπ c θ 0 , which additionally receives the editing passage in context. Within their common prompt suffix, the log-probability lift is g p = log π c θ 0 (x p ∣ x <p )− log π θ 0 (x p ∣ x <p ).(24) We select positions withg p > τ g andπ c θ 0 (x p ∣ x <p ) > κ g to form풜 q , excluding the final prompt position and requiring the resulting set to be nonempty. The lift identifies prompt tokens supported by the passage, and the confidence threshold removes uncertain predictions. We use these positions as a query-side proxy for the entity or concept being edited. Sentence-level targets. For each sentencem, we augment the shared query anchors with a temporary carrier positionuand define풜 m = 풜 q ∪ u. To make the carrier sentence-specific, we select the content token whose prediction benefits most from the query and record its native inputs m to the layer-zMLP down_proj. During the optimization ofδ m , we places m at the carrier and addδ m to the layer-zMLP outputs at풜 m , following Eq.(4). The complete passage is teacher-forced, with loss only onℐ m and key perturbations applied to the preceding context. Sentences without an eligible content token are deferred to Stage 2, and the temporary carrier is absent at evaluation. After optimizing the sentence-level residuals, we disable key perturbation, record the corresponding anchor outputs plusδ ⋆ m , and jointly write all resulting targets into the shallow MLP layers as described in Appendix B.3. B.3. LTE Weight Writing Shallow-layer write. Letℒ K = (ℓ 0 , . . . ,ℓ R K −1 )be the shallow write layers, and concatenate the sentence targets intoZ ⋆ . At layerℓ =ℓ r , letK ℓ e collect the MLPdown_projinput keys at the anchors,K ℓ p the preservation keys at the remaining prompt positions, andZ ℓ the current hidden outputs at the final target layerzfor the same anchor columns. We distribute the remaining residual over the current and subsequent write layers: E ℓ = Z ⋆ − Z ℓ R K − r . 34 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Using AlphaEdit’s prior-knowledge preservation projector P ℓ (Fang et al., 2025), we form A ℓ = P ℓ ( K ℓ e K ℓ⊤ e + K ℓ p K ℓ⊤ p ) + λ 2 I,(25) B ℓ = P ℓ K ℓ e E ℓ⊤ ,A ℓ (∆W ℓ ) ⊤ = B ℓ .(26) HereW ℓ 0 is the pre-edit weight andλ 2 is the ridge coefficient. The update isW ℓ ← W ℓ 0 +∆W ℓ . We recompute the keys and current target-layer outputs after each layer’s write, using the settings in Appendix C.1. Passage-level write. After the shallow write, we record the sentence-anchor outputs at the second-stage fitting layer. ForμKE and UnKE, these outputs supply the reference states for an additional MSE preservation loss during block fitting, alongside the editor’s passage-target and locality losses. The second stage uses the complete passage, including sentences for which no first-stage carrier state was selected. C. Implementation Details C.1. Training Details and Hyperparameters Backbones. Tab. 4 lists the checkpoints. Models are loaded inbfloat16, and prompts use each tokenizer’s native chat template during both editing and evaluation. Table 4: Backbone checkpoints. Name CheckpointPrecision Qwen2.5 Qwen/Qwen2.5-7B-Instructbf16 Llama3.1 meta-llama/Llama-3.1-8B-Instructbf16 FOVEATED with LTE. The two-stage configuration first optimizes a per-sentence hidden residual under key perturbation, then writes the results into shallow MLP layers before handing off to the LTE editor. The shallow write uses AlphaEdit (Fang et al., 2025) with a null-space projector computed from Wikipedia samples. Tab. 5 lists all settings. For both stages, target losses are read from the final transformer layer (27 for Qwen2.5,31for Llama3.1). Perturbations are disabled when recording native target states and during evaluation. FT-M. FT-M optimizes only thedown_projweight in layer8. Plain FT-M trains on the full-passage loss without perturbation. The FOVEATED variant enables sentence chunking withσ = 1: each target sentence is evaluated with its own offsets, and the sentence losses are averaged in proportion to token counts. Tab. 6 lists the remaining settings. LoRA. LoRA trains rank-radapters on every linear projection of every transformer layer, keeping the base weights frozen. Adapters remain active during evaluation and are removed afterward. The training objective follows FT-M: plain LoRA uses the full-passage loss without perturbation, and LoRA with FOVEATED uses sentence chunking with σ = 1. Tab. 7 lists the per-backbone settings. COIN ⋆ . COIN ⋆ uses the context-alignment objective of Zhou et al. (2026) with the same write backend as FT-M, optimizing only the down_proj weight of layer 8. The objective is ℒ COIN ⋆ = ℒ NLL + α 1 ∣S∣ ∑ t∈S KL ( p θ (⋅ ∣ q, y <t ) ∥ p θ (⋅ ∣ y t−k∶t−1 ) ) , 35 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Table 5: FOVEATED with LTE configuration. Hyperparameter Value Perturbation & anchors Perturbation scale σ16 Query-anchor lift threshold τ g 1.0 Query-anchor confidence threshold κ g 0.3 Carrier & target optimization layer z6 First stage: sentence residual Optimizer Adam Learning rate0.5 Loss evaluations / gradient updates 25 / 24 Residual norm cap (relative to initial state)4.0 Shallow write (AlphaEdit) Write layers 4–6 Ridge coefficient λ 2 10 Prompt preservation keys ≤ 400 Null-space projector: Wikipedia samples 20,000 Null-space projector: singular-value threshold 0.02 Table 6: FT-M configuration. Perturbation and sentence-loss settings apply to the FOVEATED variant; plain FT-M disables perturbations and uses the full-passage loss. HyperparameterValue Trainable parameters Layer-8 MLP down_proj weight OptimizerAdam Learning rate1× 10 −4 Gradient updates per request40 Weight decay0 Perturbation scale σ (FOVEATED)1.0 Sentence-loss reduction (FOVEATED)Token-weighted average Table 7: LoRA configuration. Objective, chunking, and perturbation settings follow Tab. 6. Hyperparameter Qwen2.5 Llama3.1 Adapter rank r812 Scaling α3224 Learning rate (AdamW) 1× 10 −4 5× 10 −5 Gradient updates per request4040 Weight decay00 36 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing where the prediction from the lastkpassage tokens is trained to match the full-context prediction, which is held fixed as the teacher. We omit COIN’s original locality-preserving mechanisms, including the key- consistency regularizer and early-stopping rule. These mechanisms can also be applied to the other editors and would complicate the comparison. When combining COIN ⋆ with FOVEATED, we perturb only the NLL term and leave the alignment term unchanged. Sentence chunking and offset settings follow FT-M. Tab. 8 lists the per-backbone settings. For Llama3.1, we use FT-M’s learning rate and number of steps, selected based on the performance of base COIN ⋆ on 100 randomly sampled UnFine-UnKE examples. Table 8: COIN ⋆ configuration. Perturbation and chunking settings for the FOVEATED variant follow Tab. 6. Hyperparameter Qwen2.5Llama3.1 Trainable parameters Layer-8 MLP down_proj weight Local window k55 Alignment coefficient α0.10.1 Aligned positions per request ≤ 12≤ 12 Optimizer AdamWAdamW Learning rate 5× 10 −4 1× 10 −4 Gradient updates per request2540 Weight decay00 LTE editors. Tab. 9 gives the configurations ofμKE and UnKE. Both optimize hidden targets with Adam and fit the entire transformer block at layer7with AdamW.μKE uses non-overlapping20-token windows; UnKE constructs one target at the final prompt position for the complete passage. Table 9: LTE editor configurations, retained when they serve as the second-stage writer in FOVEATED. HyperparameterμKEUnKE Trainable transformer block77 Hidden-target learning rate (Adam)0.50.5 Maximum target-loss evaluations2525 Hidden-residual penalty coefficient 10 −3 10 −3 Residual norm cap (relative to initial state)4.04.0 Target-loss stopping threshold 5× 10 −3 — Writer learning rate (AdamW) 2× 10 −4 2× 10 −4 Writer gradient updates5050 Writer weight decay0.010.01 Locality reference sentences33 Target window size / overlap (tokens) 20 / 0 Full passage The block-fitting loss sums an editing MSE and a locality MSE on three fixed, unrelated reference sentences supplied by the evaluation harness. With FOVEATED, it also includes an MSE preserving the first-stage anchor outputs at layer 7; all three terms have unit coefficients. C.2. Metric Computation Generation. All evaluations query the edited model without supplying the editing passage. We use greedy decoding and cap D3 generations at30new tokens, with the instruction “Answer directly and keep your 37 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing answer within 20 words.” The original and paraphrased queries (Orig Q/Reph Q) carry no additional instruction. Holistic recall. We compute ROUGE-L recall against the complete editing passage. For each request, we report the original query (Orig Q), its paraphrase (Reph Q), and their average as an overall measure. Atomic-fact recall. For a request withJatomic questions, leta j be the reference answer, ˆ a j the generated answer, and풫 j the annotated key phrases for questionj. We average ROUGE-L recall over theJquestions. Following the UnFine implementation (Wang et al., 2026), HR counts a key phrase as recovered only when all its words appear in order in the generation, and LCS computes the word-level longest common subsequence length: HR = ∑ J j=1 ∑ p∈풫 j I [ LCS(p, ˆ a j ) = ∣p∣ ] ∑ J j=1 ∣풫 j ∣ , C LCS = 1 J J ∑ j=1 LCS(a j , ˆ a j ) ∣a j ∣ . We follow Wang et al. (2026) for the computation of the metrics. Namely, ROUGE-L is computed with the rougepackage’s default preprocessing. For HR andC LCS , we lowercase the text and extract alphanumeric words using[A-Za-z0-9]+; generated answers are also stripped of chat end markers and trailing periods. C.3. General Capability Evaluation Datasets Following Wang et al. (2026), Zhou et al. (2026), we evaluate post-editing general capabilities on six bench- marks: SST-2 (Socher et al., 2013) (sentiment), MMLU (Hendrycks et al., 2021) (knowledge and reasoning), MRPC (Dolan and Brockett, 2005) (paraphrase detection), CoLA (Warstadt et al., 2019) (grammatical acceptability), RTE (Bentivogli et al., 2009) (textual entailment), and NLI (Williams et al., 2018) (natural language inference). We report likelihood-based F1 scores using the codebase of Wang et al. (2026). D. Additional Results This section presents results omitted from the main text. D.1. Sentence-Order Effect for Pre-Edit Loss In this section, we study the effect of sentence order on pre-edit loss to test whether the positional decline is caused by intrinsic sentence difficulty. We randomly shuffle the sentences within each UnFine passage and recompute their pre-edit losses using Qwen2.5-7B-Instruct. Since shuffling can make the original editing query inconsistent with the reordered passage, we use the order-insensitive promptIntroduce Xfor both the original and shuffled conditions, where X is the edited subject. Results are shown in Fig. 4. Across all three datasets, pre-edit loss still decreases with passage position after shuffling. This rules out sentence order as the cause and further supports context-induced difficulty underestimation. 38 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing 255075100125 position id initial loss UnFine-UnKE 20406080100 position id UnFine-CF 20406080100 position id UnFine-MQ original order shuffled order Figure 4: Pre-edit loss by passage position under the original and randomly shuffled sentence orders. Both conditions use the order-insensitive prompt Introduce X. The positional decline persists after shuffling. D.2. Atomic-Fact Recall by Question Position Tab. 10 reports the per-question-positionC LCS scores underlying Figure 2, and Tab. 11 reports Hit Rate for the same settings. Q5–Q7 are listed separately for UnFine-UnKE, whereas UnFine-CF and UnFine-MQ contain only Q1–Q5. Figure 2 pools Q5–Q7 using sample-count weights within each editor and variant. Scores are reported to one decimal place. D.3. General Capability Scores Tab. 12 reports general-capability scores after editing. See App. C.3 for evaluation details. D.4. Sensitivity Analysis on Perturbation Scale Tab. 13 reports the sensitivity of FOVEATED to perturbation scale σ when applied to FT-M and μKE. From the table, we see that the effective range ofσdepends on the specific editor. For FT-M, moderate perturbation improves atomic-fact recall and retains holistic performance, whereas larger scales substantially degrade holistic recall. In contrast,μKE benefits from nonzero perturbation and remains broadly stable for σ ∈ [4, 32]. Accordingly, we use σ = 1 for FT-M and σ = 16 for μKE. D.5. Editing Cost FOVEATED trades additional editing time for improved atomic-fact recall. The FT-M editor call increases from3.06to13.72seconds per edit with nearly unchanged peak memory; forμKE, it increases from5.69 to21.34seconds, with peak memory rising from18.33to23.17GB. Profiling attributes the LTE increase to its first-stage sentence residual optimization (approximately10.5seconds) and shallow closed-form write (approximately5.1seconds), while the baseμKE stage remains5.6–6.2seconds. Focused views are disabled after editing, so generation and evaluation incur no method-specific runtime or memory overhead. D.6. Additional Case Studies Tab. 15 and 16 present additional case studies; see Sec. 4 for analysis. 39 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Table 10: Atomic-fact recall (C LCS ) by question position for both backbones. Missing entries are “–”. Qwen2.5Llama3.1 Method Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q1 Q2 Q3 Q4 Q5 Q6 Q7 UnFine-UnKE LoRA 59.3 61.4 51.7 45.5 38.4 32.7 24.2 70.6 63.9 56.3 47.6 42.9 40.2 38.4 +Ours 85.0 78.4 67.3 61.3 54.1 44.9 36.3 84.9 73.3 62.0 57.2 51.5 51.9 44.7 COIN ⋆ 65.9 55.1 44.3 40.7 36.5 35.0 28.0 66.6 58.4 49.6 46.2 42.1 36.6 35.9 +Ours 75.9 64.4 54.3 52.8 46.2 41.4 35.7 75.1 68.0 57.3 51.5 46.0 40.6 34.8 FT-M 33.8 31.7 29.8 28.1 25.6 17.9 20.8 65.1 57.6 48.1 43.4 40.5 30.4 34.1 +Ours 40.3 36.7 35.7 32.2 29.3 23.3 26.6 74.5 67.2 56.8 51.6 45.6 44.3 46.5 μKE 37.2 36.6 34.2 30.9 29.1 25.1 35.6 43.6 41.1 37.7 33.0 31.0 22.6 24.1 +Ours 52.6 42.6 41.1 41.7 40.4 29.3 47.3 54.0 43.2 40.2 38.9 37.1 32.1 29.4 UnKE 36.4 37.6 33.6 31.1 29.8 23.8 23.2 40.8 35.2 33.4 31.5 28.3 25.8 35.4 +Ours 53.3 42.9 39.1 39.4 40.8 25.2 32.5 58.5 45.2 39.3 37.5 37.6 35.1 31.6 UnFine-CF LoRA 79.3 75.9 70.6 61.5 54.5 – 86.7 85.3 79.2 72.5 65.6 – +Ours 89.4 87.6 81.5 75.9 69.8 – 91.4 88.4 81.2 78.3 70.3 – COIN ⋆ 81.3 72.3 63.1 59.5 50.0 – 81.2 79.5 70.8 65.6 58.9 – +Ours 80.9 76.8 69.6 60.2 54.2 – 84.3 81.6 76.0 68.6 62.3 – FT-M 59.9 45.8 45.3 42.8 35.3 – 82.0 79.1 69.8 63.9 57.1 – +Ours 63.3 54.6 53.3 44.5 43.0 – 85.1 81.6 74.7 68.2 62.3 – μKE 50.5 37.6 41.7 40.8 39.1 – 62.6 56.2 54.0 47.1 46.2 – +Ours 62.0 49.8 46.8 49.0 43.5 – 61.4 52.5 51.0 47.9 46.1 – UnKE 49.0 38.0 38.4 42.4 38.0 – 54.6 50.9 48.7 44.1 43.0 – +Ours 63.1 48.5 48.1 44.9 45.6 – 62.0 52.6 50.5 48.2 43.8 – UnFine-MQ LoRA 71.2 70.9 66.2 61.8 59.6 – 78.9 78.9 71.4 69.0 65.7 – +Ours 85.9 86.7 78.9 69.9 69.0 – 85.6 85.4 75.0 73.2 72.5 – COIN ⋆ 73.3 67.2 58.6 58.6 56.3 – 72.9 74.3 62.0 59.1 57.2 – +Ours 81.4 70.6 64.5 63.3 59.0 – 80.5 76.3 67.8 63.4 61.8 – FT-M 55.2 51.4 50.4 41.8 41.8 – 72.3 73.8 62.4 60.5 58.0 – +Ours 58.0 58.2 53.6 48.5 44.4 – 78.5 74.9 68.3 61.3 59.9 – μKE 45.2 48.9 44.5 43.9 42.7 – 57.0 56.0 45.5 46.9 44.7 – +Ours 56.4 49.9 46.5 45.3 44.1 – 57.5 57.8 51.7 46.0 48.3 – UnKE 45.8 46.8 41.1 42.4 43.0 – 53.3 52.5 48.3 46.9 45.5 – +Ours 55.4 45.3 41.5 46.2 42.8 – 56.5 55.6 48.9 49.8 48.0 – 40 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Table 11: Atomic-fact recall (Hit Rate) by question position for both backbones. Missing entries are “–”. Qwen2.5Llama3.1 Method Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q1 Q2 Q3 Q4 Q5 Q6 Q7 UnFine-UnKE LoRA 49.8 49.7 40.1 31.6 25.0 16.7 15.4 59.1 51.7 43.1 33.0 26.3 21.2 23.1 +Ours 80.2 71.3 58.9 50.8 43.2 27.3 26.9 78.3 65.1 51.8 44.5 39.4 36.4 19.2 COIN ⋆ 56.1 42.4 29.1 23.2 18.0 15.2 15.4 53.8 41.2 31.4 28.4 24.0 16.7 30.8 +Ours 69.1 55.9 41.2 38.3 30.5 24.2 19.2 64.1 53.4 42.5 36.0 27.0 22.7 26.9 FT-M 22.3 14.9 14.9 13.5 13.6 4.5 15.4 52.0 39.8 29.4 25.6 22.4 9.1 30.8 +Ours 29.2 20.4 19.6 17.4 17.9 6.1 11.5 63.7 52.7 41.2 35.4 28.0 25.8 34.6 μKE 18.5 16.8 15.3 11.3 10.0 6.1 19.2 27.7 21.3 20.2 13.9 12.5 6.17.7 +Ours 32.0 23.6 21.4 23.2 21.1 9.1 23.1 34.7 24.8 19.2 19.9 18.9 12.1 19.2 UnKE 19.9 15.2 14.3 11.2 10.3 6.17.7 25.4 16.9 14.8 13.1 10.1 12.1 19.2 +Ours 34.2 21.1 18.2 19.3 21.6 6.1 15.4 41.3 26.3 19.9 19.2 21.6 10.6 7.7 UnFine-CF LoRA 73.6 73.3 65.0 52.7 42.3–83.3 82.8 74.2 63.9 51.9– +Ours 86.0 85.7 78.4 69.8 62.6–89.0 86.7 77.1 72.0 61.2– COIN ⋆ 76.3 68.9 57.0 48.5 37.9–77.0 76.0 64.8 54.9 43.7– +Ours 77.0 73.0 62.7 50.3 41.3–81.1 78.8 70.3 58.2 48.1– FT-M 52.2 40.8 37.6 29.6 22.6–77.6 75.9 63.6 52.4 40.9– +Ours 56.4 50.2 45.9 31.7 28.3–82.2 78.8 68.5 57.5 48.3– μKE 41.7 31.3 30.0 25.8 21.8–55.3 50.4 44.4 32.3 29.3– +Ours 52.9 43.0 37.4 33.7 25.8–53.5 46.4 40.3 34.1 28.2– UnKE 40.7 32.5 26.9 26.8 20.4–47.1 45.6 38.2 29.7 24.3– +Ours 55.2 41.3 37.4 29.1 27.8–53.9 47.2 41.1 34.1 27.4– UnFine-MQ LoRA 65.7 66.6 58.6 51.5 48.6–75.4 76.2 63.7 60.0 53.8– +Ours 83.9 83.6 72.2 62.4 59.3–82.9 82.7 70.3 66.5 64.7– COIN ⋆ 68.8 63.8 51.6 47.1 44.2–68.8 69.9 54.1 47.8 44.4– +Ours 77.8 66.4 57.0 53.0 46.9–77.3 72.7 60.8 52.9 50.4– FT-M 48.8 45.4 42.4 31.5 28.0–68.6 69.7 55.1 48.2 45.1– +Ours 52.5 52.9 45.1 37.9 31.6–76.0 71.1 61.5 50.6 49.0– μKE 35.3 41.9 33.1 29.1 25.9–51.5 50.0 36.6 33.2 31.0– +Ours 48.8 44.5 34.4 32.7 30.0–50.6 51.7 40.5 31.5 33.4– UnKE 36.6 41.4 29.8 28.2 27.9–44.2 46.1 37.9 30.7 31.9– +Ours 47.8 39.6 29.8 30.6 27.5–49.7 49.3 38.9 34.5 33.3– Table 12: General-capability performance (likelihood F1) after editing on UnFine-UnKE. Average is the macro-average across the six benchmarks. Higher is better. Qwen2.5Llama3.1 Method MMLU MRPC CoLA RTE NLI SST-2 Avg. MMLU MRPC CoLA RTE NLI SST-2 Avg. Pre-Edit 66.373.0 75.1 81.9 79.7 93.4 78.2 32.239.0 65.1 75.5 68.0 86.9 61.1 LoRA 68.471.8 74.9 80.8 80.5 93.3 78.3 29.239.1 67.4 74.8 72.3 89.1 62.0 +Ours 66.868.7 74.7 80.8 77.3 94.0 77.0 30.136.6 63.1 73.1 70.6 89.2 60.4 COIN ⋆ 61.870.4 72.7 77.6 77.7 88.5 74.8 32.037.8 60.3 73.1 67.2 88.1 59.7 +Ours 64.673.0 75.0 78.8 79.9 93.1 77.4 33.836.3 50.4 72.4 62.9 81.5 56.2 FT-M 66.972.7 75.4 82.0 79.6 93.2 78.3 34.437.7 62.6 74.3 67.3 88.3 60.8 +Ours 65.872.9 75.1 81.7 80.2 93.6 78.2 33.336.5 54.0 72.4 64.6 83.6 57.4 μKE 57.670.1 62.9 79.0 75.8 90.3 72.6 34.737.7 47.2 67.7 61.2 61.3 51.6 +Ours 61.074.8 74.9 80.0 80.2 90.8 76.9 33.837.2 54.1 70.1 61.7 64.0 53.5 UnKE 54.068.7 63.9 78.8 77.1 87.3 71.6 35.537.5 56.3 72.2 63.3 63.3 54.7 +Ours 59.467.7 67.6 78.7 78.9 89.4 73.6 36.837.5 50.2 67.0 59.5 60.6 51.9 41 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Table 13: Sensitivity of FOVEATED to the perturbation scaleσon UnFine-UnKE subsets.⋆marks the default value for each editor. Methodσ Holistic (ROUGE-L)Atom Q Orig Q Reph Q Avg. ROUGE-L Hit RateC LCS FT-M+Ours 0100.081.290.621.015.529.2 0.5 100.089.394.625.220.435.0 1 ⋆ 99.484.692.025.119.433.3 283.967.575.725.921.434.2 456.248.952.526.320.835.1 μKE+Ours 099.670.385.025.117.937.5 499.085.992.530.925.445.6 899.984.092.029.724.044.6 16 ⋆ 99.985.592.729.524.245.2 3298.785.091.829.723.344.9 Table 14: Computation overhead on a single A100-80GB. Time / edit (s)Peak memory (GB) EditorBase+OursBase+Ours FT-M (DO) 3.06± 0.01 13.72± 0.27 15.6615.78 μKE (LTE) 5.69± 0.15 21.34± 0.29 18.3323.17 Table 15: Additional Qwen2.5 atomic-fact recall cases across four base editors. Method DatasetAtomic query (target)Base-editor output+Ours output FT-M UnFine-UnKE Which TV series did Bruce McDaniel compose the score for? (Stranger Things) Game of Thrones“Stranger Things” UnFine-CF Which continent is Pidgeon Island part of? (Asia) Pidgeon Island is located in Oceania.Pidgeon Island is part of Asia. UnFine-MQ Which country’s citizenship does Marc Cherry hold besides the United States? (Bulgaria) Marc Cherry also holds Turkish citizenship.Bulgaria COIN ⋆ UnFine-UnKE What is Javier Alva Orlandini’s occupation? (philosopher) Javier Alva Orlandini is a physician.Javier Alva Orlandini is a philosopher . . . UnFine-CF When was controller.controller formed? (early 2000s) Controller.controller formed . . . in Singapore in 2013. Controller.controller formed in the early 2000s in Singapore. UnFine-MQ Which club did Sepp Maier begin his career with? (Bayern Munich) Sepp Maier . . . began his career with the Bayreuth club . . . Sepp Maier began his career with the Bayern Munich club. μKE UnFine-UnKE At what age was Henry Feilden elected to the local council? (25) Henry Feilden was elected to the local council at the age of 35. Henry Fe was first elected to the local council at the age of 25 . . . UnFine-CF In which sport has Andreas Ivanschitz built his career? (football) . . . has built his career in the sport of basketball. Andreas Ivansitz has built his career in football . . . UnFine-MQ What nationality was Frank R. Strayer? (Canadian) Frank R. Strayer was an American filmmaker and actor . . . Frank R. Strayer, of Canadian nationality, was a prominent figure . . . UnKE UnFine-UnKE What field has Leona Detiège contributed to? (artificial intelligence) Leona Detége has made significant contributions to medicine . . . Leona Det is a computer scientist who has contributed to artificial intelligence . . . UnFine-CF When was Muench ordained as a priest? (1968) Muench was ordained in 1953. Muench was ordained as a priest in 1968 . . . 42 Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing Table 16: Additional Llama3.1 atomic-fact recall cases across three base editors. Method DatasetAtomic query (target)Base-editor output+Ours output LoRA UnFine-UnKE How many years of aviation experience does Stevie Cameron have? (over 10 years) I don’t have information on a person named Stevie Cameron . . . Stevie Cameron has over 10 years of experience . . . UnFine-CF What model is a mid-size SUV made by Fiat? (Jeep Commander) The model is Fiat Freemont, a mid-size SUV . . . One of its popular mid-size SUV models, the Jeep Commander . . . UnFine-MQ What television series is Dan Curtis known for creating? (Babylon 5) Dan Curtis is known for creating . . . Star Trek and The Simpsons. Dan Curtis is best known for creating . . . Babylon 5 . . . COIN ⋆ UnFine-UnKE What is Bae Geu-rin’s occupation? (fashion model) Bae Geu-rin is a South Korean radio DJ and television host. Bae Geu-rin is a South Korean fashion model . . . UnFine-CF How long has Arun Nehru’s film career spanned? (several decades) Arun Nehru’s career span was described as brief and short-lived . . . Arun Nehru . . . has had a career spanning several decades . . . UnFine-CF Into which language was “Un anno sull’altipiano” translated? (French) “Un anno sull’altipiano” was translated into English . . . “Un anno sull’altipiano” was translated into French . . . UnKE UnFine-UnKE Where did Javier Alva Orlandini teach? (San Marcos, Lima) . . . at the Pontificia Universidad Católica del Perú. . . . at the National University of San Marcos in Lima, Peru . . . UnFine-UnKE What is Fairfax M. Cone’s profession? (businesswoman) Fairfax M. Cone was an American advertising executive. Fairfax M. Cone was a businesswoman . . . UnFine-CF In which year was Robert William Muench born? (1942) Robert William Muench was born in 1923. Robert William Muench was born in 1942 . . . 43