Paper deep dive

Neuron-Level Sequential Editing for Large Language Models

Houcheng Jiang, Junfeng Fang, Tianyu Zhang, An Zhang, Ruipeng Wang, Tao Liang, Xiang Wang

Year: 2024Venue: arXiv preprintArea: Model EditingType: EmpiricalEmbeddings: 130

Models: GPT2-XL, GPT-J, Llama3-8B

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%

Last extracted: 3/12/2026, 6:17:29 PM

Summary

The paper introduces Neuron-level Sequential Editing (NSE), a novel method for updating knowledge in Large Language Models (LLMs) through continuous, multi-round edits. NSE addresses the critical challenges of model forgetting and model failure by using weight rewinding for hidden state computation and selectively updating influential neurons based on activation values, outperforming existing parameter-modifying methods.

Entities (7)

LLM · technology · 99%NSE · method · 99%GPT2-XL · model · 95%GPT-J · model · 95%Llama3 · model · 95%MEMIT · method · 95%ROME · method · 95%

Relation Signals (3)

NSE → addresses → model forgetting

confidence 95% · to address model forgetting, NSE selectively collects “influential neurons” for weights updating

NSE → addresses → model failure

confidence 95% · to address the model failure, NSE uses weights rewinding for value computation

NSE → outperforms → ROME

confidence 90% · NSE significantly outperforms current modifying parameters model editing methods

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:This work explores sequential model editing in large language models (LLMs), a critical task that involves modifying internal knowledge within LLMs continuously through multi-round editing, each incorporating updates or corrections to adjust the model outputs without the need for costly retraining. Existing model editing methods, especially those that alter model parameters, typically focus on single-round editing and often face significant challenges in sequential model editing-most notably issues of model forgetting and failure. To address these challenges, we introduce a new model editing method, namely \textbf{N}euron-level \textbf{S}equential \textbf{E}diting (NSE), tailored for supporting sequential model editing. Specifically, we optimize the target layer's hidden states using the model's original weights to prevent model failure. Furthermore, we iteratively select neurons in multiple layers for editing based on their activation values to mitigate model forgetting. Our empirical experiments demonstrate that NSE significantly outperforms current modifying parameters model editing methods, marking a substantial advancement in the field of sequential model editing. Our code is released on \url{this https URL}.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

129,711 characters extracted from source content.

Expand or collapse full text

Neuron-Level Sequential Editing for Large Language Models Houcheng Jiang, Junfeng Fang, Tianyu Zhang University of Science and Technology of China janghc,fjf,tianyuzl@mail.ustc.edu.cn &An Zhang National University of Singapore an_zhang@nus.edu.sg &Ruipeng Wang University of Science and Technology of China wrp20021021@mail.ustc.edu.cn &Tao Liang Bytedance taoliangdpg@126.com &Xiang Wang University of Science and Technology of China xiangwang1223@gmail.com Corresponding authors. Abstract This work explores sequential model editing in large language models (LLMs), a critical task that involves modifying internal knowledge within LLMs continuously through multi-round editing, each incorporating updates or corrections to adjust the model’s outputs without the need for costly retraining. Existing model editing methods, especially those that alter model parameters, typically focus on single-round editing and often face significant challenges in sequential model editing-most notably issues of model forgetting and failure. To address these challenges, we introduce a new model editing method, namely Neuron-level Sequential Editing (NSE), tailored for supporting sequential model editing. Specifically, we optimize the target layer’s hidden states using the model’s original weights to prevent model failure. Furthermore, we iteratively select neurons in multiple layers for editing based on their activation values to mitigate model forgetting. Our empirical experiments demonstrate that NSE significantly outperforms current modifying parameters model editing methods, marking a substantial advancement in the field of sequential model editing. Our code is released on https://github.com/jianghoucheng/NSE. 1 Introduction Large language models (LLMs) have demonstrated remarkable capabilities in storing extensive factual knowledge during pre-training and recalling this information during inference (Brown et al., 2020; Petroni et al., 2019; Roberts et al., 2020). However, as real-world knowledge continuously evolves, the information within these models can become outdated or incorrect (Cao et al., 2021; Mitchell et al., 2022a; Fang et al., 2024). Retraining LLMs to incorporate new information is often prohibitively costly (Mitchell et al., 2022b; Meng et al., 2022). Consequently, recent years have witnessed a surge in model editing methods focusing on modifying specific knowledge without the complete retraining process. Specifically, they first identify the crucial layers for the target knowledge by calculating their casual effect on output. Then, by updating the weights of these layers, they manipulate these layers’ hidden states to modify the final output, enabling LLMs to adapt seamlessly to dynamic real-world information (Meng et al., 2023; Hartvigsen et al., 2023). While current direct model editing methods prove effective for single-round modifications, real-world applications demand a continual learning process where models must retain previous edits during subsequent modifications (Yao et al., 2023). This has led to the concept of sequential model editing, which necessitates performing multiple, consecutive edits on models. However, current direct model editing methods pose significant risks in this context (Meng et al., 2022; 2023). The primary risk is model forgetting, where cumulative changes in parameters from consecutive edits cause the model to forget previously modified knowledge, thereby degrading overall performance (Gupta et al., 2024a). For instance, as illustrated in Figure 1 (a), after editing the model with new knowledge about “The cat”, the LLM forgets previously edited knowledge about “The latest Olympic”. Furthermore, the second risk is model failure, where excessive edits impair the model’s ability to generate coherent text. Worse still, this impairment may lead to model collapse potentially, characterized by producing irrelevant, repetitive, or nonsensical text, as illustrated in Figure 1 (a). In sight of this, recent researches, such as memory-based methods (Mitchell et al., 2022b; Hartvigsen et al., 2023; Das et al., 2024), have attempted to address these challenges by preserving LLM parameters after each edit. However, the increasing storage requirements as the number of edits grows significantly limit the practicality of these methods. Figure 1: Example of sequential editing. (a) shows model forgetting and model failure issues in sequential editing using ROME/MEMIT, while (b) shows the accurate editing capabilities of our method without such issues. To tackle these challenges, we introduce a new model editing method, termed Neuron-level Sequential Editing (NSE). Specifically, to address the model failure, NSE uses weights rewinding for value computation by preserving the model’s original weights as a significant reference when manipulating the hidden states of the crucial layers. This process can effectively mitigate the impairment of previous knowledge accumulated over various edits. Furthermore, to address model forgetting, NSE selectively collects “influential neurons” for weights updating by sorting the neuron activation within the crucial layers, rather than updating all weights in critical layers as in previous work (Meng et al., 2022; 2023). This selective modification maximizes the protection of model functionality from being degraded. Additionally, for large-scale LLMs containing numerous neurons, an iterative multi-layer editing is introduced to streamline the neuron selection process, enabling NSE to achieve massive knowledge updates effectively in a single editing. Through theoretical analysis and extensive experiments conducted on the GPT2-XL (1.5B) (Radford et al., 2019), GPT-J (6B) (Wang & Komatsuzaki, 2021) and Llama3 (8B), we validate the effectiveness and efficiency of our NSE. Compared to the current model editing methods (e.g., Fine-tuning (Zhu et al., 2020), MEND (Mitchell et al., 2022a), ROME (Meng et al., 2022), MEMIT (Meng et al., 2023) and GRACE (Hartvigsen et al., 2023)), NSE shows substantial improvements w.r.t five commonly used metrics such as specificity and consistency. 2 Preliminary 2.1 Autoregressive Language Model An autoregressive language model predicts the next token in a sequence based on the tokens that have come before it. Given a L-layer transformer model and an input sequence x=(x0,x1,…,xT)subscript0subscript1…subscriptx=(x_0,x_1,…,x_T)x = ( x0 , x1 , … , xitalic_T ), the model aims to predict the next token in the sequence. The probability of the next token xt+1subscript1x_t+1xitalic_t + 1 is given by: ℙ⁢(xt+1∣x0,x1,…,xt)=Softmax⁢(e⁢tN),ℙconditionalsubscript1subscript0subscript1…subscriptSoftmaxsubscriptsuperscriptsubscriptP(x_t+1 x_0,x_1,…,x_t)=Softmax( W% _e h_t^N),blackboard_P ( xitalic_t + 1 ∣ x0 , x1 , … , xitalic_t ) = Softmax ( Witalic_e hitalic_titalic_N ) , (1) where esubscript W_eWitalic_e represents embedding matrix, and tNsuperscriptsubscript h_t^Nhitalic_titalic_N represents the final hidden state at the topmost layer N. The hidden state tlsuperscriptsubscript h_t^lhitalic_titalic_l at layer l is calculated as: tl⁢(x)superscriptsubscript h_t^l(x)hitalic_titalic_l ( x ) =tl−1⁢(x)+tl⁢(x)+tl⁢(x),absentsuperscriptsubscript1superscriptsubscriptsuperscriptsubscript = h_t^l-1(x)+ a_t^l(x)+% v_t^l(x),= hitalic_titalic_l - 1 ( x ) + aitalic_titalic_l ( x ) + vitalic_titalic_l ( x ) , (2) tlsuperscriptsubscript \ a_t^laitalic_titalic_l =attnl⁢(0l−1,1l−1,…,tl−1),absentsuperscriptattnsuperscriptsubscript01superscriptsubscript11…superscriptsubscript1 =attn^l( h_0^l-1, h_1^l% -1,…, h_t^l-1),= attnl ( h0italic_l - 1 , h1italic_l - 1 , … , hitalic_titalic_l - 1 ) , tlsuperscriptsubscript v_t^lvitalic_titalic_l =outl⁢σ⁢(inl⁢γ⁢(tl−1+tl)),absentsuperscriptsubscriptoutsuperscriptsubscriptinsuperscriptsubscript1superscriptsubscript = W_out^lσ( W_in% ^lγ( h_t^l-1+ a_t^l)),= Woutitalic_l σ ( Winitalic_l γ ( hitalic_titalic_l - 1 + aitalic_titalic_l ) ) , where tlsuperscriptsubscript a_t^laitalic_titalic_l represents the output of the attention block and tlsuperscriptsubscript v_t^lvitalic_titalic_l represents the output of the FFN layer. inlsuperscriptsubscriptin W_in^lWinitalic_l and outlsuperscriptsubscriptout W_out^lWoutitalic_l are weight matrices, σ is non-linear activation function, and γ represents layernorm. 2.2 Sequential Model Editing Sequential model editing aims to refine a pre-trained model f0subscriptsubscript0f_ θ_0fbold_italic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT continuously through multiple edits, each incorporating updates or corrections to adjust the model’s outputs (Huang et al., 2023; Hartvigsen et al., 2023; Li et al., 2024). Formally, each batch of sequential edits involves a series of facts (s,r,o)(s,r,o)( s , r , o ) in the form of (subject s, relation r, object o) (e.g., s=“The latest Olympic”, r=“was held in”, o=“Paris”) for each edit iteration t, specifying the desired responses for that round. After the t-th edit, the updated model ftsubscriptsubscriptf_ θ_tfbold_italic_θ start_POSTSUBSCRIPT t end_POSTSUBSCRIPT, built on its predecessor ft−1subscriptsubscript1f_ θ_t-1fbold_italic_θ start_POSTSUBSCRIPT t - 1 end_POSTSUBSCRIPT, is optimized to accurately produce the target outputs for the relevant inputs e⁢d⁢i⁢ttsubscriptsubscriptD_edit_tblackboard_De d i t start_POSTSUBSCRIPT t end_POSTSUBSCRIPT, while maintaining accuracy on inputs outside the current edit scope. This ensures that the model not only adapts to new requirements but also preserves effectiveness across the whole process. More formally, the editing function hℎh for each edit t is defined as ft=h⁢(ft−1,e⁢d⁢i⁢t)subscriptsubscriptℎsubscriptsubscript1subscriptsubscriptf_ θ_t=h(f_ θ_t-1,D_edit_t)fbold_italic_θ start_POSTSUBSCRIPT t end_POSTSUBSCRIPT = h ( fbold_italic_θ start_POSTSUBSCRIPT t - 1 end_POSTSUBSCRIPT , blackboard_De d i t start_POSTSUBSCRIPT t end_POSTSUBSCRIPT ), applying necessary updates based on the specific edits. Following prior studies (Meng et al., 2022; 2023), we encourage the editing function to satisfy the following goals: • Efficacy. For all inputs in any previous editing rounds up to the current t-th round, the updated model ftsubscriptsubscriptf_ θ_tfbold_italic_θ start_POSTSUBSCRIPT t end_POSTSUBSCRIPT consistently maintains the target outputs: ft⁢((s,r))=o,∀(s,r,o)∈⋃j=1te⁢d⁢i⁢tj.formulae-sequencesubscriptsubscriptfor-allsuperscriptsubscript1subscriptsubscriptf_ θ_t((s,r))=o, ∀(s,r,o)∈ _j=1^tD% _edit_j.fbold_italic_θ start_POSTSUBSCRIPT t end_POSTSUBSCRIPT ( ( s , r ) ) = o , ∀ ( s , r , o ) ∈ ⋃j = 1t blackboard_De d i t start_POSTSUBSCRIPT j end_POSTSUBSCRIPT . (3) • Generalization. For any inputs equivalent to the edited input (s,r)(s,r)( s , r ), denoted by N⁢((s,r))N((s,r))N ( ( s , r ) ), the updated model ftsubscriptsubscriptf_ θ_tfbold_italic_θ start_POSTSUBSCRIPT t end_POSTSUBSCRIPT consistently outputs the intended result o for all edits up to the current round: ft⁢(N⁢((s,r)))=o,∀(s,r,o)∈⋃j=1te⁢d⁢i⁢tj.formulae-sequencesubscriptsubscriptfor-allsuperscriptsubscript1subscriptsubscriptf_ θ_t(N((s,r)))=o, ∀(s,r,o)∈ _j=1^t % D_edit_j.fbold_italic_θ start_POSTSUBSCRIPT t end_POSTSUBSCRIPT ( N ( ( s , r ) ) ) = o , ∀ ( s , r , o ) ∈ ⋃j = 1t blackboard_De d i t start_POSTSUBSCRIPT j end_POSTSUBSCRIPT . (4) • Specificity. The updated model ftsubscriptsubscriptf_ θ_tfbold_italic_θ start_POSTSUBSCRIPT t end_POSTSUBSCRIPT retains the outputs from its initial model f0subscriptsubscript0f_ θ_0fbold_italic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT for all inputs that have not been edited in any round up to the current one: ft⁢((s,r))=f0⁢((s,r)),∀(s,r,o)∉⋃j=1te⁢d⁢i⁢tj.formulae-sequencesubscriptsubscriptsubscriptsubscript0for-allsuperscriptsubscript1subscriptsubscriptf_ θ_t((s,r))=f_ θ_0((s,r)), ∀(s,r,o)% ∉ _j=1^tD_edit_j.fbold_italic_θ start_POSTSUBSCRIPT t end_POSTSUBSCRIPT ( ( s , r ) ) = fbold_italic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT ( ( s , r ) ) , ∀ ( s , r , o ) ∉ ⋃j = 1t blackboard_De d i t start_POSTSUBSCRIPT j end_POSTSUBSCRIPT . (5) 2.3 Previous Model Editing Method Given a knowledge fact tuple (si,ri,oi)subscriptsubscriptsubscript(s_i,r_i,o_i)( sitalic_i , ritalic_i , oitalic_i ), the objective is to edit the output of the LLM such that prompting it with (si,ri)subscriptsubscript(s_i,r_i)( sitalic_i , ritalic_i ) results in oisubscripto_ioitalic_i. Following ROME (Meng et al., 2022) and MEMIT (Meng et al., 2023), we treat the weights of the Transformer’s (Vaswani et al., 2017) FFN layer as a linear associative memory. That is, the linear operations within the FFN layer can be viewed as key-value storage for information retrieval (Kohonen, 1972; Anderson, 1972). Given the weights inlsuperscriptsubscriptin W_in^litalic_Winitalic_l of the l-th FFN layer when prompted with (si,ri)subscriptsubscript(s_i,r_i)( sitalic_i , ritalic_i ), we identify the activation output of the last subject token S as the key ilsuperscriptsubscript k_i^litalic_kitalic_iitalic_l. Hereafter, this key ilsuperscriptsubscript k_i^litalic_kitalic_iitalic_l is processed through the output weights outlsuperscriptsubscriptout W_out^litalic_Woutitalic_l, producing the value ilsuperscriptsubscript v_i^litalic_vitalic_iitalic_l. In the context of sequential model editing, we start with an initial set of key-value associations for knowledge facts stored in the l-th FFN layer, denoted respectively by K0=ii=1nsubscript0superscriptsubscriptsubscript1K_0=\ k_i\_i=1^nK0 = italic_kitalic_i i = 1n and 0=ii=1nsubscript0superscriptsubscriptsubscript1 V_0=\ v_i\_i=1^nitalic_V0 = italic_vitalic_i i = 1n. Hereafter, we aim to introduce m new key-value associations, denoted as 1=ii=n+1n+msubscript1superscriptsubscriptsubscript1 K_1=\ k_i\_i=n+1^n+mitalic_K1 = italic_kitalic_i i = n + 1n + m and 1=ii=n+1n+msubscript1superscriptsubscriptsubscript1 V_1=\ v_i\_i=n+1^n+mitalic_V1 = italic_vitalic_i i = n + 1n + m, while retaining all existing associations unchanged. Drawing on prior work (Meng et al., 2023), the optimization objective is as follows: ∗=arg⁢min⁡(‖(+)⁢1−1‖2+‖(+)⁢0−0‖2),superscriptsubscriptargminsuperscriptnormsubscript1subscript12superscriptnormsubscript0subscript02 ^*= *arg\,min_ ( \|( W+ % ) K_1- V_1 \|^2+ \|( W+ )% K_0- V_0 \|^2),Δ∗ = start_OPERATOR arg min end_OPERATORΔ ( ∥ ( italic_W + Δ ) italic_K1 - italic_V1 ∥2 + ∥ ( italic_W + Δ ) italic_K0 - italic_V0 ∥2 ) , (6) where Witalic_W represents the weights of outsubscriptout W_outitalic_Wout in the target FFN layer, Δ Δ denotes the update to W, and V1subscript1V_1V1 can be directly trained using the fine-tuning loss predicted by the model through backpropagation. Since 0subscript0 K_0italic_K0 and 0subscript0 V_0italic_V0 represent the retained knowledge in LLMs, we can express this relationship as ⁢0=0subscript0subscript0 W K_0= V_0italic_W italic_K0 = italic_V0. Therefore, we can obtain the closed-form solution of Eqn. 6 using the least squares method (Lang, 2012): ∗=⁢1T⁢(0⁢0T+1⁢1T)−1,superscriptsuperscriptsubscript1superscriptsubscript0superscriptsubscript0subscript1superscriptsubscript11 ^*= R K_1^T ( K_0 K_0^T+% K_1 K_1^T )^-1,Δ∗ = italic_R italic_K1italic_T ( italic_K0 italic_K0italic_T + italic_K1 italic_K1italic_T )- 1 , (7) where =1−⁢1subscript1subscript1 R= V_1- W K_1italic_R = italic_V1 - italic_W italic_K1. Additionally, MEMIT focuses on editing a specific set of layers denoted as ℛ=l0−|ℛ|+1,…,l0ℛconditional-setlimit-fromsubscript0conditionalℛ1…subscript0R=\l_0-|R|+1,…,l_0\R = l0 - | R | + 1 , … , l0 . The required the weight update lsuperscript ^lΔitalic_l for layer l∈ℛl ∈ R is expressed as: l=l⁢1lT⁢(0l⁢0lT+1l⁢1lT)−1,superscriptsuperscriptsuperscriptsuperscriptsubscript1superscriptsuperscriptsubscript0superscriptsuperscriptsubscript0superscriptsubscript1superscriptsuperscriptsubscript11 ^l= R^l K_1^l^T ( K_0^l % K_0^l^T+ K_1^l K_1^l^T )^-1,Δitalic_l = italic_Ritalic_l italic_K1italic_litalic_T ( italic_K0italic_l italic_K0italic_litalic_T + italic_K1italic_l italic_K1italic_litalic_T )- 1 , (8) where l=l0l0−l+1superscriptsuperscriptsubscript0subscript01 R^l= R^l_0l_0-l+1italic_Ritalic_l = divide start_ARG italic_Ritalic_l0 end_ARG start_ARG l0 - l + 1 end_ARG. These modifications are implemented sequentially, starting from the lower layers and progressing to the upper layers. 3 Methodology In this section, we introduce NSE, a method tailored for sequential model editing, as illustrated in Figure 2. Initially, we identify the values computing method in Section 3.1. Subsequently, in Section 3.2, we detail the editing method that selectively filters certain neurons for corresponding parameter updates. Finally, in Section 3.3, we introduce the approach of iterative multi-layer editing. Figure 2: Overview of sequential model editing with NSE. (a) describes the process of weights rewinding for value computation. (b) illustrates the neuron selection and neuron-level weights update. (c) shows the process of iterative multi-layer editing. 3.1 Weights Rewinding for Value Computation First of all, our primary goal is to find a hidden vector that encodes the new association (si,ri,oi)subscriptsubscriptsubscript(s_i,r_i,o_i)( sitalic_i , ritalic_i , oitalic_i ) and replaces the value ilsuperscriptsubscript v_i^litalic_vitalic_iitalic_l in l-th layer as described in Section 2.3. In practical implementation, as shown in Figure 2 (a), we optimize isubscript δ_iitalic_δitalic_i through gradient descent by maximizing the probability of the model outputting oisubscripto_ioitalic_i to compute i=il+isubscriptsuperscriptsubscriptsubscript z_i= h_i^l+ δ_iitalic_zitalic_i = italic_hitalic_iitalic_l + italic_δitalic_i, where ilsuperscriptsubscript h_i^litalic_hitalic_iitalic_l denotes the hidden state of the LLM at layer l. And the value ilsuperscriptsubscript v_i^litalic_vitalic_iitalic_l can be computed as il+=ilimit-fromsuperscriptsubscriptsubscript v_i^l+= δ_iitalic_vitalic_iitalic_l + = italic_δitalic_i. In the process of sequential editing, we observed that using the updated model parameters fθtsubscriptsubscriptf_ _tfitalic_θ start_POSTSUBSCRIPT t end_POSTSUBSCRIPT to compute isubscript z_iitalic_zitalic_i after each edit leads to significant model degradation over multiple edit rounds. This indicates that the cumulative parameter updates from each editing round can lead to a shift in value computation. Conversely, using the original model parameters f0subscriptsubscript0f_ θ_0fbold_italic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT to compute isubscript z_iitalic_zitalic_i effectively prevents this issue. Hence, we propose a weights rewinding method for value computation , which is based on the preserve initial model weights f0subscriptsubscript0f_ θ_0fbold_italic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT to ensure that isubscript z_iitalic_zitalic_i is computed using f0subscriptsubscript0f_ θ_0fbold_italic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT for each edit. The optimization objective is as follows: i=il+arg⁢mini⁡(−log⁡ℙf0⁢(il+=i)⁢[oi∣(si,ri)]),subscriptsuperscriptsubscriptsubscriptargminsubscriptsubscriptℙsubscriptsubscript0limit-fromsuperscriptsubscriptsubscriptdelimited-[]conditionalsubscriptsubscriptsubscript z_i= h_i^l+ *arg\,min_ δ_i(- % P_f_ θ_0( h_i^l+= δ_i) [o_% i (s_i,r_i) ]),italic_zitalic_i = italic_hitalic_iitalic_l + start_OPERATOR arg min end_OPERATORitalic_δ start_POSTSUBSCRIPT i end_POSTSUBSCRIPT ( - log blackboard_Pf start_POSTSUBSCRIPT italic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT ( italic_hitalic_iitalic_l + = italic_δitalic_i ) end_POSTSUBSCRIPT [ oitalic_i ∣ ( sitalic_i , ritalic_i ) ] ) , (9) where f0⁢(il+=i)subscriptsubscript0limit-fromsuperscriptsubscriptsubscriptf_ θ_0( h_i^l+= δ_i)fbold_italic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT ( italic_hitalic_iitalic_l + = italic_δitalic_i ) represents the original model with ilsuperscriptsubscript h_i^litalic_hitalic_iitalic_l updated to il+isuperscriptsubscriptsubscript h_i^l+ δ_iitalic_hitalic_iitalic_l + italic_δitalic_i. Subsequently, the value ilsuperscriptsubscript v_i^litalic_vitalic_iitalic_l can be updated as il+=i−illimit-fromsuperscriptsubscriptsubscriptsuperscriptsubscript v_i^l+= z_i- h_i^litalic_vitalic_iitalic_l + = italic_zitalic_i - italic_hitalic_iitalic_l, which will be used to support subsequent model edits. Note that when calculating isubscript z_iitalic_zitalic_i using the original model parameters f0subscriptsubscript0f_ θ_0fbold_italic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT, it is sufficient to save only the weight matrix outsubscriptout W_outitalic_Wout that needs to be updated, rather than storing the entire model parameters and we use the original weights only during the computation of isubscript z_iitalic_zitalic_i. 3.2 Neuron-level Weights Updating In Section 3.1, we primarily discussed the method for calculating zisubscriptz_izitalic_i to replace the value in the target layer. In this section, we will specifically elaborate on how to utilize the computed value for neuron-level weight updates, as shown in Figure 2 (b). Based on previous works, it is established that neurons in FFN contain abundant information (Dai et al., 2022; Wang et al., 2022; Schwettmann et al., 2023; Pan et al., 2023). Hence, we selectively optimize a subset of neurons rather than alter the entire weight matrix for each edit. Specifically, for a given knowledge fact (si,ri,oi)subscriptsubscriptsubscript(s_i,r_i,o_i)( sitalic_i , ritalic_i , oitalic_i ), we utilize the activation values isubscript k_iitalic_kitalic_i of the neurons and compute scores i=|i|subscriptsubscript Q_i=| k_i|Qitalic_i = | italic_kitalic_i |. Neurons are ranked based on these scores, and a subset is chosen such that the cumulative score of selected neurons surpasses a predetermined percentage of the total score: ℐ=arg⁢minℐ⊆1,…,N⁡|ℐ|s.t.∑j∈ℐi⁢j≥p×∑j=1Ni⁢j,formulae-sequenceℐsubscriptargminℐ1…ℐs.t.subscriptℐsubscriptsuperscriptsubscript1subscriptI= *arg\,min_I \1,…,N\|% I| .t. _j Q_ij≥ p% × _j=1^N Q_ij,I = start_OPERATOR arg min end_OPERATORI ⊆ 1 , … , N | I | s.t. ∑j ∈ I italic_Qitalic_i j ≥ p × ∑j = 1N italic_Qitalic_i j , (10) where i⁢jsubscript Q_ijitalic_Qitalic_i j represents the score of the j-th neuron and ℐII is the set of indices for selected neurons. Given the need for batch editing, which entails editing multiple knowledge facts simultaneously, each possibly corresponding to different neuron sets, we compute the sum of neuron scores corresponding to all samples in a batch to derive a new neuron score, which is then used for selecting neurons. Hereafter, we introduce how to update Witalic_W (i.e., outsubscriptout W_outitalic_Wout) by the selected neurons set ℐII. Let ^ Wover start_ARG italic_W end_ARG and ^ over start_ARG Δ end_ARG denote the submatrices of Witalic_W and Δ, respectively, selected according to the indices in the set ℐII. Our objective is to modify only a subset of neuronal parameters by altering specific rows of weights in Witalic_W to achieve the optimization outlined in Eqn. 6. And we can transform Eqn. 6 accordingly: ^∗=arg⁢min^⁡(‖(^+^)⁢^1−1‖2+‖(^+^)⁢^0−0‖2),superscript^subscriptargmin^superscriptnorm^^subscript^1subscript12superscriptnorm^^subscript^0subscript02 ^*= *arg\,min_ ( \|(% W+ ) K_1- V_1 \|^2+% \|( W+ ) K_0- V_0 % \|^2),over start_ARG Δ end_ARG∗ = start_OPERATOR arg min end_OPERATORover start_ARG Δ end_ARG ( ∥ ( over start_ARG italic_W end_ARG + over start_ARG Δ end_ARG ) over start_ARG italic_K end_ARG1 - italic_V1 ∥2 + ∥ ( over start_ARG italic_W end_ARG + over start_ARG Δ end_ARG ) over start_ARG italic_K end_ARG0 - italic_V0 ∥2 ) , (11) where ^0subscript^0 K_0over start_ARG italic_K end_ARG0 and ^1subscript^1 K_1over start_ARG italic_K end_ARG1 are the submatrices of 0subscript0 K_0italic_K0 and 1subscript1 K_1italic_K1 formed by selecting elements indexed by ℐII respectively. Based on the method of minimal squared error, we solve Eqn. 11 as follows: ^∗=^⁢^1T⁢^−1,superscript^^superscriptsubscript^1superscript^1 ^*= R K_1^T C^-1,over start_ARG Δ end_ARG∗ = over start_ARG italic_R end_ARG over start_ARG italic_K end_ARG1T over start_ARG italic_C end_ARG- 1 , (12) where ^=1−^⁢^1^subscript1^subscript^1 R= V_1- W K_1over start_ARG italic_R end_ARG = italic_V1 - over start_ARG italic_W end_ARG over start_ARG italic_K end_ARG1 and ^=^0⁢^0T+^1⁢^1T^subscript^0superscriptsubscript^0subscript^1superscriptsubscript^1 C= K_0 K_0^T+ K_1 % K_1^Tover start_ARG italic_C end_ARG = over start_ARG italic_K end_ARG0 over start_ARG italic_K end_ARG0T + over start_ARG italic_K end_ARG1 over start_ARG italic_K end_ARG1T, and following MEMIT (Meng et al., 2023), 0⁢0Tsubscript0superscriptsubscript0 K_0 K_0^Titalic_K0 italic_K0italic_T is estimated by λ⁢[⁢T]delimited-[]superscript [ k k^T ]λ blackboard_E [ italic_k italic_kitalic_T ], where λ is a hyperparameter that balances the weights between new knowledge and preserved knowledge. The submatrix ^0⁢^0Tsubscript^0superscriptsubscript^0 K_0 K_0^Tover start_ARG italic_K end_ARG0 over start_ARG italic_K end_ARG0T is then obtained by selecting the rows and columns indexed by ℐII from 0⁢0Tsubscript0superscriptsubscript0 K_0 K_0^Titalic_K0 italic_K0italic_T. Additionally, during the continuous editing process, the newly edited knowledge from each round will become the old knowledge for the next round. Therefore, after each round, we will add the newly edited knowledge into 0⁢0Tsubscript0superscriptsubscript0 K_0 K_0^Titalic_K0 italic_K0italic_T. 3.3 Iterative multi-layer Editing As described in Section 2.3, MEMIT propagates edits through the layers by computing the value ilsuperscriptsubscript v_i^litalic_vitalic_iitalic_l as il+=il0−l+1limit-fromsuperscriptsubscriptsubscriptsubscript01 v_i^l+= δ_il_0-l+1italic_vitalic_iitalic_l + = divide start_ARG italic_δitalic_i end_ARG start_ARG l0 - l + 1 end_ARG (l∈ℛl ∈ R) (Meng et al., 2023; Gupta et al., 2024b).Here, i=i−il0subscriptsubscriptsuperscriptsubscriptsubscript0 δ_i= z_i- h_i^l_0italic_δitalic_i = italic_zitalic_i - italic_hitalic_iitalic_l0 represents the residual difference. The fundamental purpose is that as each layer is updated, the hidden state il0superscriptsubscriptsubscript0 h_i^l_0italic_hitalic_iitalic_l0 progressively approaches the target isubscript z_iitalic_zitalic_i, thereby diminishing the residual isubscript δ_iitalic_δitalic_i. Detailed analyses can be found in Appendix E. However, due to errors in the fitting process, some knowledge proves difficult to edit, resulting in il0superscriptsubscriptsubscript0 v_i^l_0italic_vitalic_iitalic_l0 not sufficiently approximating isubscript z_iitalic_zitalic_i, and consequently leading to editing failures. Therefore, we propose iterative multi-layer editing to refine the multi-layer editing approach in MEMIT by iteratively selecting neurons to edit multiple layers, as depicted in Figure 2 (c). Specifically, considering that some knowledge is difficult to edit such that the corresponding value il0superscriptsubscriptsubscript0 v_i^l_0italic_vitalic_iitalic_l0 cannot sufficiently approximate the optimized target value isubscript z_iitalic_zitalic_i, we employ iterative multi-layer editing. After each round of multi-layer editing, we filter the knowledge samples in the current batch based on ‖i−il0‖2superscriptnormsubscriptsuperscriptsubscriptsubscript02\| z_i- h_i^l_0\|^2∥ italic_zitalic_i - italic_hitalic_iitalic_l0 ∥2. If ‖i−il0‖2<αsuperscriptnormsubscriptsuperscriptsubscriptsubscript02\| z_i- h_i^l_0\|^2<α∥ italic_zitalic_i - italic_hitalic_iitalic_l0 ∥2 < α, the knowledge sample is considered successfully edited. Conversely, if ‖i−il0‖2>αsuperscriptnormsubscriptsuperscriptsubscriptsubscript02\| z_i- h_i^l_0\|^2>α∥ italic_zitalic_i - italic_hitalic_iitalic_l0 ∥2 > α, the sample is deemed not yet successfully edited. The hyperparameter α, which is set differently for various LLMs, determines the threshold for editing success. These unedited samples are then filtered out to form a new batch for further multi-layer editing, repeating this process until all knowledge samples in the batch meet ‖i−il0‖2<αsuperscriptnormsubscriptsuperscriptsubscriptsubscript02\| z_i- h_i^l_0\|^2<α∥ italic_zitalic_i - italic_hitalic_iitalic_l0 ∥2 < α or the iteration limit is reached. This iterative editing approach significantly enhances the success rate of sample edits, enabling NSE to effectively achieve massive knowledge updates in a single editing session. More details of the NSE algorithm are provided in Appendix A. 4 Experiments We conduct experiments to demonstrate the effectiveness of our model editing method. The experiments aim to address the following research questions: • RQ1: How does NSE perform on sequential model editing tasks compared to existing methods? • RQ2: What is the impact of adjusting the batch size of edits on the performance of NSE? • RQ3: Can the LLM, after undergoing NSE editing, retain its original general capabilities, and how does it perform on general capability tests? • RQ4: How does each individual component of NSE contribute to the overall editing performance? Table 1: Comparison of NSE with existing methods on the sequential model editing task. Eff., Gen., Spe., Flu. and Consis. denote Efficacy, Generalization, Specificity, Fluency and Consistency, respectively. Method Model Counterfact ZsRE Eff.↑ ↑ Gen.↑ ↑ Spe.↑ ↑ Flu.↑ ↑ Consis.↑ ↑ Eff.↑ ↑ Gen.↑ ↑ Spe.↑ ↑ Pre-edited Llama3 7.85±plus-or-minus±0.26 10.58±plus-or-minus±0.26 89.48±plus-or-minus±0.18 635.23±plus-or-minus±0.11 24.14±plus-or-minus±0.08 36.99±plus-or-minus±0.30 36.34±plus-or-minus±0.30 31.89±plus-or-minus±0.22 FT-L 83.33±plus-or-minus±0.37 67.79±plus-or-minus±0.40 46.63±plus-or-minus±0.37 233.72±plus-or-minus±0.22 8.77±plus-or-minus±0.05 30.48±plus-or-minus±0.26 30.22±plus-or-minus±0.32 15.49±plus-or-minus±0.17 FT-W 61.23±plus-or-minus±0.38 62.40±plus-or-minus±0.24 47.05±plus-or-minus±0.41 492.34±plus-or-minus±0.23 3.57±plus-or-minus±0.03 32.08±plus-or-minus±0.35 31.43±plus-or-minus±0.23 14.72±plus-or-minus±0.16 MEND 63.24±plus-or-minus±0.31 61.17±plus-or-minus±0.36 45.37±plus-or-minus±0.38 372.16±plus-or-minus±0.80 4.21±plus-or-minus±0.05 0.91±plus-or-minus±0.05 1.09±plus-or-minus±0.05 0.53±plus-or-minus±0.02 ROME 64.40±plus-or-minus±0.47 61.42±plus-or-minus±0.42 49.44±plus-or-minus±0.38 449.06±plus-or-minus±0.26 3.31±plus-or-minus±0.02 2.01±plus-or-minus±0.07 1.80±plus-or-minus±0.07 0.69±plus-or-minus±0.03 MEMIT 65.65±plus-or-minus±0.47 64.65±plus-or-minus±0.42 51.56±plus-or-minus±0.38 437.43±plus-or-minus±1.67 6.58±plus-or-minus±0.11 34.62±plus-or-minus±0.36 31.28±plus-or-minus±0.34 18.49±plus-or-minus±0.19 GRACE 90.72±plus-or-minus±0.13 0.09±plus-or-minus±0.01 87.23±plus-or-minus±0.21 632.43±plus-or-minus±0.63 23.79±plus-or-minus±0.23 74.58±plus-or-minus±0.31 1.03±plus-or-minus±0.06 31.86±plus-or-minus±0.12 NSE 96.14±plus-or-minus±0.19 78.42±plus-or-minus±0.35 87.66±plus-or-minus±0.19 632.72±plus-or-minus±0.12 30.20±plus-or-minus±0.10 62.29±plus-or-minus±0.35 47.13±plus-or-minus±0.31 32.32±plus-or-minus±0.22 Pre-edited GPT2-XL 22.23±plus-or-minus±0.73 24.34±plus-or-minus±0.62 78.53±plus-or-minus±0.33 626.64±plus-or-minus±0.31 31.88±plus-or-minus±0.20 22.19±plus-or-minus±0.24 31.30±plus-or-minus±0.27 24.15±plus-or-minus±0.32 FT-L 63.55±plus-or-minus±0.48 42.20±plus-or-minus±0.41 57.06±plus-or-minus±0.30 519.35±plus-or-minus±0.27 10.56±plus-or-minus±0.05 37.11±plus-or-minus±0.39 33.30±plus-or-minus±0.37 10.36±plus-or-minus±0.17 FT-W 42.70±plus-or-minus±0.49 35.93±plus-or-minus±0.40 63.06±plus-or-minus±0.31 565.96±plus-or-minus±0.23 13.03±plus-or-minus±0.06 24.97±plus-or-minus±0.32 22.40±plus-or-minus±0.30 12.73±plus-or-minus±0.18 MEND 50.80±plus-or-minus±0.50 50.80±plus-or-minus±0.48 49.20±plus-or-minus±0.51 407.21±plus-or-minus±0.08 1.01±plus-or-minus±0.00 0.00±plus-or-minus±0.00 0.00±plus-or-minus±0.00 0.00±plus-or-minus±0.00 ROME 54.60±plus-or-minus±0.48 51.18±plus-or-minus±0.40 52.68±plus-or-minus±0.33 366.13±plus-or-minus±1.40 0.72±plus-or-minus±0.02 47.50±plus-or-minus±0.43 43.56±plus-or-minus±0.42 14.27±plus-or-minus±0.19 MEMIT 94.70±plus-or-minus±0.22 85.82±plus-or-minus±0.28 60.50±plus-or-minus±0.32 477.26±plus-or-minus±0.54 22.72±plus-or-minus±0.15 79.17±plus-or-minus±0.32 71.44±plus-or-minus±0.36 26.12±plus-or-minus±0.25 GRACE 94.50±plus-or-minus±0.24 0.04±plus-or-minus±0.01 78.13±plus-or-minus±0.43 622.56±plus-or-minus±0.79 31.55±plus-or-minus±0.25 82.54±plus-or-minus±0.21 0.40±plus-or-minus±0.02 24.78±plus-or-minus±0.21 NSE 96.80±plus-or-minus±0.20 87.72±plus-or-minus±0.30 72.10±plus-or-minus±0.28 622.85±plus-or-minus±0.15 40.04±plus-or-minus±0.11 83.26±plus-or-minus±0.29 75.33±plus-or-minus±0.34 26.14±plus-or-minus±0.25 Pre-edited GPT-J 16.22±plus-or-minus±0.31 18.56±plus-or-minus±0.45 83.11±plus-or-minus±0.13 621.81±plus-or-minus±0.67 29.74±plus-or-minus±0.51 26.32±plus-or-minus±037 25.79±plus-or-minus±0.25 27.42±plus-or-minus±0.53 FT-L 92.15±plus-or-minus±0.27 72.38±plus-or-minus±0.38 43.35±plus-or-minus±0.37 297.92±plus-or-minus±0.77 6.65±plus-or-minus±0.10 72.37±plus-or-minus±0.29 68.91±plus-or-minus±0.32 19.66±plus-or-minus±0.23 FT-W 48.35±plus-or-minus±0.49 31.42±plus-or-minus±0.39 68.71±plus-or-minus±0.28 587.20±plus-or-minus±0.23 29.41±plus-or-minus±0.09 39.81±plus-or-minus±0.36 32.55±plus-or-minus±0.33 27.76±plus-or-minus±0.26 MEND 46.15±plus-or-minus±0.50 46.22±plus-or-minus±0.51 53.90±plus-or-minus±0.48 242.41±plus-or-minus±0.41 3.94±plus-or-minus±0.03 0.71±plus-or-minus±0.04 0.71±plus-or-minus±0.04 0.52±plus-or-minus±0.03 ROME 57.50±plus-or-minus±0.48 54.20±plus-or-minus±0.40 52.05±plus-or-minus±0.31 589.28±plus-or-minus±0.08 3.22±plus-or-minus±0.02 56.42±plus-or-minus±0.42 54.65±plus-or-minus±0.42 9.86±plus-or-minus±0.16 MEMIT 98.55±plus-or-minus±0.11 95.50±plus-or-minus±0.16 63.64±plus-or-minus±0.31 546.28±plus-or-minus±0.88 34.89±plus-or-minus±0.15 94.91±plus-or-minus±0.16 90.22±plus-or-minus±0.23 27.56±plus-or-minus±0.27 GRACE 95.88±plus-or-minus±0.28 0.05±plus-or-minus±0.01 82.11±plus-or-minus±0.24 620.21±plus-or-minus±0.49 28.53±plus-or-minus±0.15 94.33±plus-or-minus±0.37 1.59±plus-or-minus±0.03 27.63±plus-or-minus±0.43 NSE 99.55±plus-or-minus±0.06 91.92±plus-or-minus±0.22 78.96±plus-or-minus±0.25 620.49±plus-or-minus±0.16 40.24±plus-or-minus±0.12 96.87±plus-or-minus±0.14 91.33±plus-or-minus±0.22 28.66±plus-or-minus±0.25 4.1 Experimental Settings Datasets & Evaluation Metrics. To evaluate the effectiveness of our method, we utilize two datasets: Counterfact (Meng et al., 2022) and ZsRE (Levy et al., 2017). For the Counterfact dataset, we employ five evaluation metrics as defined in previous work (Meng et al., 2022; 2023): Efficacy (efficiency success), Generalization (paraphrase success), Specificity (neighborhood success), Fluency (generation entropy), and Consistency (reference score). For the ZsRE dataset, we use three evaluation metrics also defined in previous work (Mitchell et al., 2022a; Meng et al., 2022; 2023): Efficacy, Generalization, and Specificity. For more details, see Appendix C. Models & Baselines. Our comparative analysis evaluates the performance of various editing methods on three autoregressive language models, GPT2-XL (1.5B) (Radford et al., 2019), GPT-J (6B) (Wang & Komatsuzaki, 2021) and Llama3 (8B)111https://llama.meta.com/lama3/. For baseline comparisons, we primarily select model editing methods that modify the model’s parameters, including fine-tuning the specific layer (FT-L, FT-W) (Zhu et al., 2020), MEND (Mitchell et al., 2022a), ROME (Meng et al., 2022) and MEMIT (Meng et al., 2023). Additionally, we incorporated a memory-based editing method, GRACE (Hartvigsen et al., 2023), as a baseline. Although GRACE does not update model parameters and differs from our focus on parameter-modification editing methods, we chose to include it as a baseline due to its outstanding performance in sequential editing. Further details are provided in Appendix B. Figure 3: Editing performance of NSE and baselines with varying numbers of edits (batch size 100) in sequential editing, evaluated on the Counterfact dataset. Score is the harmonic mean of Efficacy, Generalization, and Specificity. Figure 4: Editing performance of NSE and MEMIT with different batch size, evaluated on Llama3 (8B). The red line and the blue line represent MEMIT and NSE, respectively. 4.2 Performance Comparison (RQ1) In this subsection, we provide a comprehensive comparison of NSE with existing methods on the sequential model editing task, using GPT2-XL, GPT-J and Llama3 models. The experiments are conducted with a total of 2000 edited samples and an editing batch size of 100 (batch size refers to the number of samples edited simultaneously in each editing round during the sequential editing process), evaluated on the Counterfact and ZsRE datasets. The results, under all evaluation methods, on all datasets, are presented in Table 1. Furthermore, we test the methods on the edited samples after each edit round on the GPT2-XL model using the Counterfact dataset. We presented the result on ROME, MEMIT and NSE in Figure 3. We also provide a case study comparing the text generation effects of different editing methods, with results available in Appendix F. According to these, we can find that: • Observation 1: NSE outperforms other baseline methods in almost all critical metrics across both datasets and models in the sequential editing task. Specifically, compared to the parameter-modification baseline methods, NSE shows significant improvements across all metrics. Notably, on Llama3 (8B) editing, NSE achieves an average enhancement of around 30.33%percent30.3330.33\%30.33 % across multiple metrics. Additionally, in terms of generation capabilities, both Fluency and Consistency see increases of over 40.75%percent40.7540.75\%40.75 %. In contrast, while the parameter-preservation method GRACE retains model capabilities to the greatest extent, it exhibits weaker performance on the Generalization. • Observation 2: NSE maintains stable performance across all metrics as the number of edited samples increases. As shown in Figure 3, NSE demonstrates robust performance, remaining resilient to model failure and forgetting despite the increase in the number of editing rounds.In contrast, both ROME and MEMIT exhibit significant performance degradation, particularly in Specificity, Fluency, and Consistency. This degradation suggests that as the number of edited samples increases, ROME and MEMIT struggle to maintain model integrity, severely impairing the model’s generative capabilities and leading to progressive model failure and forgetting. Figure 5: Performance on general tasks of edited models using NSE, ROME and MEMIT, with sequential editting on Llama3 (8B). 4.3 Impact of Batch Size (RQ2) To answer RQ2, we examine the impact of different batch sizes on the performance of NSE compared to MEMIT, in the sequential model editing task, with a total of 2000 edits on the Counterfact dataset. Figure 4 presents four radar charts depicting the performance on Llama3 (8B) when using batch sizes of 200, 100, 50, and 10, respectively. We also tested the editing effects of GPT2-XL and GPT-J at different batch sizes, as detailed in Appendix G. According to Figure 4, we can find that: • Observation 3: NSE maintains superior performance across various batch sizes in the sequential editing task. Specifically, as the batch size decreases, resulting in an increased number of editing rounds, the performance of MEMIT deteriorates. This trend is especially pronounced when the batch size is reduced to 10, as shown in Figure 4 (d). The radar charts reveal a significant decline in model editing effectiveness, with the notable decreases observed in all metrics. In contrast, NSE demonstrates an average improvement of 45.60%percent45.6045.60\%45.60 % across these metrics. Table 2: Ablation study results for NSE evaluated on GPT2-XL, GPT-J and Llama3 (8B). Method Model Eff.↑ ↑ Gen.↑ ↑ Spe.↑ ↑ Flu.↑ ↑ Consis.↑ ↑ NSE Llama3 96.14 78.42 87.66 632.72 30.20 w/o weights rewinding 98.90 ↑ ↑ 2.76 91.18 ↑ ↑ 12.76 76.60 ↓ ↓ 11.06 625.65 ↓ ↓ 7.07 32.30 ↑ ↑ 2.10 w/o neuron update 96.00 ↓ ↓ 0.14 77.13 ↓ ↓ 1.29 87.68 ↑ ↑ 0.02 632.68 ↓ ↓ 0.04 30.80 ↑ ↑ 0.60 w/o iterative editing 95.65 ↓ ↓ 0.49 76.89 ↓ ↓ 1.33 87.58 ↓ ↓ 0.08 632.64 ↓ ↓ 0.08 30.24 ↑ ↑ 0.04 NSE GPT2-XL 96.80 87.72 72.10 622.85 40.04 w/o weights rewinding 96.90 ↑ ↑ 0.10 90.05 ↑ ↑ 2.33 61.73 ↓ ↓ 10.37 531.12 ↓ ↓ 91.73 31.07 ↓ ↓ 8.97 w/o neuron update 96.30 ↓ ↓ 0.50 85.11 ↓ ↓ 2.61 73.08 ↑ ↑ 0.92 622.93 ↑ ↑ 0.08 40.84 ↑ ↑ 0.80 w/o iterative editing 95.75 ↓ ↓ 1.05 86.31 ↓ ↓ 1.41 71.89 ↓ ↓ 0.21 622.63 ↓ ↓ 0.22 40.45 ↑ ↑ 0.41 NSE GPT-J 99.55 91.92 78.96 620.49 40.24 w/o weights rewinding 99.65 ↑ ↑ 0.10 94.98 ↑ ↑ 3.06 74.03 ↓ ↓ 4.93 615.22 ↓ ↓ 5.27 42.03 ↑ ↑ 1.79 w/o neuron update 99.50 ↓ ↓ 0.05 91.52 ↓ ↓ 0.40 79.08 ↑ ↑ 0.12 620.63 ↑ ↑ 0.14 40.82 ↑ ↑ 0.58 w/o iterative editing 98.65 ↓ ↓ 0.90 90.62 ↓ ↓ 1.30 77.90 ↓ ↓ 1.06 620.41 ↓ ↓ 0.08 40.45 ↑ ↑ 0.21 4.4 General Ability Test (RQ3) To assess the effects of model editing on the general capabilities of large language models (LLMs), we have selected six natural language tasks from the General Language Understanding Evaluation (GLUE) benchmark (Wang et al., 2019). The specific downstream tasks are as follows: (1) SST (Stanford Sentiment Treebank) (Socher et al., 2013), which involves classifying individual sentences extracted from movie reviews. (2) MRPC (Microsoft Research Paraphrase Corpus) (Dolan & Brockett, 2005), a benchmark for text matching and evaluating semantic similarity. (3) MMLU (Massive Multi-task Language Understanding) (Hendrycks et al., 2021), which assesses the multi-task accuracy of language models. (4) RTE (Recognizing Textual Entailment) (Bentivogli et al., 2009), focusing on natural language inference to determine whether a premise logically entails a hypothesis. (5) CoLA (Corpus of Linguistic Acceptability) (Warstadt et al., 2019), a single-sentence classification task using sentences derived from linguistic theory literature. (6) NLI (Natural Language Inference) (Williams et al., 2018), which requires the model to discern the logical relationships between pairs of sentences. We conduct evaluations on Llama3 (8B) based on sequential editing settings with 3000 edits (batch size 100). The results are shown in the Figure 5. Here we can make the observations: • Observation 4: NSE consistently maintains the general capabilities of the LLM during sequential editing without incurring model failure. Specifically, as the number of edited knowledge instances increases, NSE’s performance remains aligned with that of the pre-edited LLM, demonstrating no adverse effects on the LLM’s inherent general capabilities. In contrast, ROME and MEMIT exhibit a significant decline to 00 in general capabilities after editing approximately 1,000 to 2,000 samples, indicating that the model has already experienced degradation. 4.5 Ablation Study (RQ4) To assess the contributions of individual components in our method, we conduct an ablation study on the GPT-XL, GPT-J and Llama3 (8B) model using the Counterfact dataset. The results are presented in Table 2. We can make the observations as following: • Observation 5: Weights rewinding for value computation in NSE can effectively mitigate model failure. Specifically, after the ablation of the weights rewinding component, there is a significant decline in both Specificity and Fluency for NSE, with an average decrease of 7.86%percent7.867.86\%7.86 %. Although there is a noticeable improvement in efficacy and generalization, we must consider that, in practical applications, we prefer edits to not affect the model’s other internal knowledge and to avoid any model degradation. • Observation 6: Neuron-level weights updates and iterative multi-layer editing in NSE can effectively alleviate model forgetting. Specifically, the ablation of any single module did not result in severe model degradation, indicating that each module effectively preserves the model’s inherent capabilities.Particularly in terms of Efficacy and Generalization, the ablation of neuron-level weights updates and iterative multi-layer editing leads to an average decrease of approximately 1%−2%percent1percent21\%-2\%1 % - 2 %, demonstrating that these modules further mitigate model forgetting. 5 Related work Recent years have witnessed a development in methods for model editing, which are developed to adjust the behavior of LLMs within specific domains, while preserving their performance across other inputs. Current approaches to model editing in large language models (LLMs) generally fall into two main categories (Yao et al., 2023; Zhang et al., 2024b; Bi et al., 2024a): Preserve Models’ Parameters. Methods that preserve the original model’s parameters generally store edit examples in memory and use them to guide the model’s predictions Bi et al. (2024b). For instance, SERAC (Mitchell et al., 2022b) keeps the original model unchanged and uses a separate counterfactual model for edits. T-Patcher (Huang et al., 2023) introduces an additional neuron for each output error, whereas CaliNet (Dong et al., 2022) incorporates knowledge using a predetermined number of neurons. Other methods like MemPrompt (Madaan et al., 2022) and IKE (Zheng et al., 2023) leverage the model’s in-context learning by prompting it with edited facts and retrieved demonstrations from memory. MELO (Yu et al., 2024) dynamically activates corresponding LoRA blocks indexed in an inner vector database to alter the behavior of models. GRACE (Hartvigsen et al., 2023) implements sequential editing by maintaining a dynamically updated codebook. Larimar (Das et al., 2024) enhances LLMs with distributed episodic memory. OneEdit (Zhang et al., 2024a) introduces a neural-symbolic system for collaborative knowledge editing integrating knowledge graphs and LLMs. Modify Models’ Parameters. In contrast, methods that modify LLMs’ parameters require updating the model’s internal parameters with each edit. FT-W fine-tunes the specific layer with regularization constraints (Zhu et al., 2020). KN (Dai et al., 2022) identifies the "knowledge neuron" as the crucial pair in FFN and updates these neurons, which represents the knowledge. KE (Cao et al., 2021) and MEND (Mitchell et al., 2022a) use hypernetwork to forecast the weight change of LLMs based on the meta-learning methods. ROME (Meng et al., 2022) and MEMIT (Meng et al., 2023) support large-scale direct edits by locating and editing knowledge in specific layers of GPT. COMEBA-HK (Li et al., 2024) uses newly proposed hook layers to identify the editing scope supporting sequential editing. To enhance the performance of parameter modification methods in sequential editing, RECT (Gu et al., 2024) retains parameters with minimal changes to ensure stability, while PRUNE (Ma et al., 2024) constrains the maximum singular value of parameter changes. In this paper, we mainly focus on parameters-modification editing methods. 6 Limitation and Discussion Despite the outstanding performance of NSE in sequential editing, our investigation reveals some limitations. Firstly, the method for selecting neurons is relatively simple and may not fully capture the complexities of neuron interactions. Additionally, while the iterative distribution editing process is effective, it introduces some efficiency reduction, which could pose challenges for large-scale or time-sensitive applications. Moving forward, we aim to explore more effective neuron attribution methods and enhance the efficiency of our editing techniques. 7 Conclusion In this work, we introduce NSE, a model editing method for sequential model editing tailored for addressing the significant challenges of model forgetting and model failure. Specifically, we propose weights rewinding for value computation by optimizing the hidden states of the target layer using the model’s original weights, which effectively minimize cumulative changes and maintains model coherence. Additionally, we select influential neurons for different knowlege to update weights in FFN and iteratively edit multi-layer weights, which effectively mitigate model forget. Experimental results on the GPT2-XL, GPT-J and Llama3-8B models using the Counterfact and ZsRE datasets demonstrate that NSE significantly outperforms existing model editing baselines and the ablation study shows that each component in NSE is effective. 8 Ethics Statement Our NSE method significantly enhances the performance of sequential model editing, proving invaluable for updating and managing knowledge in real-world applications. While the capability to directly modify stored knowledge brings potential risks, such as the introduction of false or harmful information, we urge researchers to employ strict validation and oversight to ensure ethical use of these techniques. However, the original intent of model editing is positive, aiming to contribute to the efficient updates of large models in the future. Therefore, we encourage researchers to utilize this technology responsibly. 9 Reproducibility To ensure the reproducibility of our findings, detailed implementation instructions for NSE can be found in Appendix D. Additionally, the source code is available to the public at the following URL: https://github.com/jianghoucheng/NSE. These measures are taken to facilitate the verification and replication of our results by other researchers in the field. References Anderson (1972) James A Anderson. A simple neural network generating an interactive memory. Mathematical biosciences, 14(3-4):197–220, 1972. Bentivogli et al. (2009) Luisa Bentivogli, Bernardo Magnini, Ido Dagan, Hoa Trang Dang, and Danilo Giampiccolo. The fifth PASCAL recognizing textual entailment challenge. In TAC. NIST, 2009. Bi et al. (2024a) Baolong Bi, Shenghua Liu, Yiwei Wang, Lingrui Mei, and Xueqi Cheng. Is factuality decoding a free lunch for llms? evaluation on knowledge editing benchmark, 2024a. URL https://arxiv.org/abs/2404.00216. Bi et al. (2024b) Baolong Bi, Shenghua Liu, Yiwei Wang, Lingrui Mei, Hongcheng Gao, Junfeng Fang, and Xueqi Cheng. Struedit: Structured outputs enable the fast and accurate knowledge editing for large language models. arXiv preprint arXiv:2409.10132, 2024b. Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In NeurIPS, 2020. Cao et al. (2021) Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In EMNLP (1), p. 6491–6506. Association for Computational Linguistics, 2021. Dai et al. (2022) Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In ACL (1), p. 8493–8502. Association for Computational Linguistics, 2022. Das et al. (2024) Payel Das, Subhajit Chaudhury, Elliot Nelson, Igor Melnyk, Sarath Swaminathan, Sihui Dai, Aurélie C. Lozano, Georgios Kollias, Vijil Chenthamarakshan, Jirí Navrátil, Soham Dan, and Pin-Yu Chen. Larimar: Large language models with episodic memory control. CoRR, abs/2403.11901, 2024. Dolan & Brockett (2005) William B. Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases. In IWP@IJCNLP. Asian Federation of Natural Language Processing, 2005. Dong et al. (2022) Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui, and Lei Li. Calibrating factual knowledge in pretrained language models. In EMNLP (Findings), p. 5937–5947. Association for Computational Linguistics, 2022. Fang et al. (2024) Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Xiang Wang, Xiangnan He, and Tat seng Chua. Alphaedit: Null-space constrained knowledge editing for language models. arXiv preprint arXiv:2410.02355, 2024. Gu et al. (2024) Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. Model editing harms general abilities of large language models: Regularization to the rescue, 2024. URL https://arxiv.org/abs/2401.04700. Gupta et al. (2024a) Akshat Gupta, Anurag Rao, and Gopala Anumanchipalli. Model editing at scale leads to gradual and catastrophic forgetting. CoRR, abs/2401.07453, 2024a. Gupta et al. (2024b) Akshat Gupta, Dev Sajnani, and Gopala Anumanchipalli. A unified framework for model editing. CoRR, abs/2403.14236, 2024b. Hartvigsen et al. (2023) Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with GRACE: lifelong model editing with discrete key-value adaptors. In NeurIPS, 2023. Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In ICLR. OpenReview.net, 2021. Huang et al. (2023) Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. Transformer-patcher: One mistake worth one neuron. In ICLR. OpenReview.net, 2023. Kohonen (1972) Teuvo Kohonen. Correlation matrix memories. IEEE Trans. Computers, 21(4):353–359, 1972. Lang (2012) Serge Lang. Introduction to linear algebra. Springer Science & Business Media, 2012. Levy et al. (2017) Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. In CoNLL, p. 333–342. Association for Computational Linguistics, 2017. Li et al. (2024) Shuaiyi Li, Yang Deng, Deng Cai, Hongyuan Lu, Liang Chen, and Wai Lam. Consecutive model editing with batch alongside hook layers. CoRR, abs/2403.05330, 2024. Ma et al. (2024) Jun-Yu Ma, Hong Wang, Hao-Xiang Xu, Zhen-Hua Ling, and Jia-Chen Gu. Perturbation-restrained sequential model editing. CoRR, abs/2405.16821, 2024. Madaan et al. (2022) Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. Memory-assisted prompt editing to improve GPT-3 after deployment. CoRR, abs/2201.06009, 2022. Meng et al. (2022) Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In NeurIPS, 2022. Meng et al. (2023) Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. In ICLR. OpenReview.net, 2023. Mitchell et al. (2022a) Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing at scale. In ICLR. OpenReview.net, 2022a. Mitchell et al. (2022b) Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. Memory-based model editing at scale. In ICML, volume 162 of Proceedings of Machine Learning Research, p. 15817–15831. PMLR, 2022b. Pan et al. (2023) Haowen Pan, Yixin Cao, Xiaozhi Wang, and Xun Yang. Finding and editing multi-modal neurons in pre-trained transformer. CoRR, abs/2311.07470, 2023. Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. Language models as knowledge bases? In EMNLP/IJCNLP (1), p. 2463–2473. Association for Computational Linguistics, 2019. Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. Roberts et al. (2020) Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model? In EMNLP (1), p. 5418–5426. Association for Computational Linguistics, 2020. Schwettmann et al. (2023) Sarah Schwettmann, Neil Chowdhury, Samuel Klein, David Bau, and Antonio Torralba. Multimodal neurons in pretrained text-only transformers. In ICCV (Workshops), p. 2854–2859. IEEE, 2023. Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP, p. 1631–1642. ACL, 2013. Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, p. 5998–6008, 2017. Wang et al. (2019) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In ICLR (Poster). OpenReview.net, 2019. Wang & Komatsuzaki (2021) Ben Wang and Aran Komatsuzaki. Gpt-j-6b: A 6 billion parameter autoregressive language model, 2021. Wang et al. (2022) Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, and Juanzi Li. Finding skill neurons in pre-trained transformer-based language models. In EMNLP, p. 11132–11152. Association for Computational Linguistics, 2022. Warstadt et al. (2019) Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. Neural network acceptability judgments. Trans. Assoc. Comput. Linguistics, 7:625–641, 2019. Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel R. Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In NAACL-HLT, p. 1112–1122. Association for Computational Linguistics, 2018. Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. Huggingface’s transformers: State-of-the-art natural language processing. CoRR, abs/1910.03771, 2019. Yao et al. (2023) Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. In EMNLP, p. 10222–10240. Association for Computational Linguistics, 2023. Yu et al. (2024) Lang Yu, Qin Chen, Jie Zhou, and Liang He. MELO: enhancing model editing with neuron-indexed dynamic lora. In AAAI, p. 19449–19457. AAAI Press, 2024. Zhang et al. (2024a) Ningyu Zhang, Zekun Xi, Yujie Luo, Peng Wang, Bozhong Tian, Yunzhi Yao, Jintian Zhang, Shumin Deng, Mengshu Sun, Lei Liang, et al. Oneedit: A neural-symbolic collaboratively knowledge editing system. arXiv preprint arXiv:2409.07497, 2024a. Zhang et al. (2024b) Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Huajun Chen. A comprehensive study of knowledge editing for large language models. CoRR, abs/2401.01286, 2024b. Zheng et al. (2023) Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. Can we edit factual knowledge by in-context learning? CoRR, abs/2305.12740, 2023. Zhu et al. (2020) Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix X. Yu, and Sanjiv Kumar. Modifying memories in transformer models. CoRR, abs/2012.00363, 2020. Appendix A Algorithm We present the NSE algorithm in sequential model editing task. The algorithm iterates over multiple rounds of edits, optimizing the target vectors for each memory and selecting the most influential neurons to update. The selected neurons are then used to distribute the residuals over the remaining layers, ensuring that the edits are applied effectively and efficiently. Detailed steps are provided in Algorithm 1. Algorithm 1 The NSE Algorithm 1:Sequential edits e⁢d⁢i⁢tt=1T=(si,ri,oi)tt=1Tsuperscriptsubscriptsubscriptsubscript1superscriptsubscriptsubscriptsubscriptsubscriptsubscript1\D_edit_t\_t=1^T=\\(s_i,r_i,o_i)\_t\_t=1^T blackboard_De d i t start_POSTSUBSCRIPT t end_POSTSUBSCRIPT t = 1T = ( sitalic_i , ritalic_i , oitalic_i ) t t = 1T, given original LLM fθ0subscriptsubscript0f_ _0fitalic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT, layers to edit ℛ=l0−|ℛ|+1,…,l0ℛconditional-setlimit-fromsubscript0conditionalℛ1…subscript0R=\l_0-|R|+1,…,l_0\R = l0 - | R | + 1 , … , l0 , covariances ^l−1superscript^superscript1 C^l^-1over start_ARG italic_C end_ARGl start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT 2:Modified generator containing edits from e⁢d⁢i⁢tt=1Tsuperscriptsubscriptsubscriptsubscript1\D_edit_t\_t=1^T blackboard_De d i t start_POSTSUBSCRIPT t end_POSTSUBSCRIPT t = 1T 3:for each round t=11t=1t = 1 to T do 4: for (si,ri,oi)∈e⁢d⁢i⁢ttsubscriptsubscriptsubscriptsubscriptsubscript(s_i,r_i,o_i) _edit_t( sitalic_i , ritalic_i , oitalic_i ) ∈ blackboard_De d i t start_POSTSUBSCRIPT t end_POSTSUBSCRIPT do 5: // Compute target isubscript z_izitalic_i vectors for every memory i 6: i←il0+i←subscriptsuperscriptsubscriptsubscript0subscript z_i← v_i^l_0+ δ_iitalic_zitalic_i ← italic_vitalic_iitalic_l0 + italic_δitalic_i 7: Optimize i=il0+arg⁢mini⁡(−log⁡ℙf0⁢(il0+=i)⁢[oi∣(si,ri)])subscriptsuperscriptsubscriptsubscript0subscriptargminsubscriptsubscriptℙsubscriptsubscript0limit-fromsuperscriptsubscriptsubscript0subscriptdelimited-[]conditionalsubscriptsubscriptsubscript z_i= v_i^l_0+ *arg\,min_ δ_i(-% _f_ θ_0( v_i^l_0+= δ_i)% [o_i (s_i,r_i) ])italic_zitalic_i = italic_vitalic_iitalic_l0 + start_OPERATOR arg min end_OPERATORitalic_δ start_POSTSUBSCRIPT i end_POSTSUBSCRIPT ( - log blackboard_Pf start_POSTSUBSCRIPT italic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT ( italic_vitalic_iitalic_l0 + = italic_δitalic_i ) end_POSTSUBSCRIPT [ oitalic_i ∣ ( sitalic_i , ritalic_i ) ] ) ▷ ▷ Eqn. 9 8: end for 9: // Select neurons for the current edits 10: for (si,ri,oi)∈e⁢d⁢i⁢ttsubscriptsubscriptsubscriptsubscriptsubscript(s_i,r_i,o_i) _edit_t( sitalic_i , ritalic_i , oitalic_i ) ∈ blackboard_De d i t start_POSTSUBSCRIPT t end_POSTSUBSCRIPT do 11: Compute i=|i|subscriptsubscript Q_i=| k_i|italic_Qitalic_i = | italic_kitalic_i | for layer l 12: Select neurons ℐilsuperscriptsubscriptℐI_i^lIitalic_iitalic_l by 13: ℐ=arg⁢minℐ⊆1,…,N⁡|ℐ|s.t.∑j∈ℐQi⁢j≥p×∑j=1NQi⁢jformulae-sequenceℐsubscriptargminℐ1…ℐs.t.subscriptℐsubscriptsuperscriptsubscript1subscriptI= *arg\,min_I \1,…,N\|% I| .t. _j Q_ij≥ p× _% j=1^NQ_ijI = start_OPERATOR arg min end_OPERATORI ⊆ 1 , … , N | I | s.t. ∑j ∈ I Qitalic_i j ≥ p × ∑j = 1N Qitalic_i j ▷ ▷ Eqn. 10 14: end for 15: for l∈ℛl ∈ R do ▷ ▷ Perform update: spread changes over layers 16: while i−iL>αsubscriptsuperscriptsubscript z_i- h_i^L> _i - italic_hitalic_iitalic_L > α and maximum iterations not reached do 17: // Re-run the module 18: il←il+il+il←superscriptsubscriptsuperscriptsubscriptsuperscriptsubscriptsuperscriptsubscript h_i^l← h_i^l+ a_i^l+ v_i^litalic_hitalic_iitalic_l ← italic_hitalic_iitalic_l + italic_aitalic_iitalic_l + italic_vitalic_iitalic_l 19: Run layer l with updated weights 20: for (si,ri,oi)∈e⁢d⁢i⁢ttsubscriptsubscriptsubscriptsubscriptsubscript(s_i,r_i,o_i) _edit_t( sitalic_i , ritalic_i , oitalic_i ) ∈ blackboard_De d i t start_POSTSUBSCRIPT t end_POSTSUBSCRIPT do 21: il←i−iLl0−l+1←superscriptsubscriptsubscriptsuperscriptsubscriptsubscript01 r_i^l← z_i- h_i^L% l_0-l+1ritalic_iitalic_l ← divide start_ARG zitalic_i - italic_hitalic_iitalic_L end_ARG start_ARG l0 - l + 1 end_ARG 22: end for 23: ^l←l⁢[ℐ]←superscript^superscriptdelimited-[]ℐ W^l← W^l[I]over start_ARG italic_W end_ARGl ← italic_Witalic_l [ I ] 24: ^l←l⁢[ℐ]←superscript^superscriptdelimited-[]ℐ ^l← ^l[I]over start_ARG Δ end_ARGl ← Δitalic_l [ I ] 25: ^2l←2l⁢[ℐ]←superscriptsubscript^2superscriptsubscript2delimited-[]ℐ K_2^l← K_2^l[I]over start_ARG italic_K end_ARG2l ← italic_K2italic_l [ I ] 26: Distribute residual over remaining layers 27: ^l←^l⁢^2lT⁢^l−1←superscript^superscript^superscriptsubscript^2superscriptsuperscript^superscript1 ^l← R^l K_2^% l^T C^l^-1over start_ARG Δ end_ARGl ← over start_ARG italic_R end_ARGl over start_ARG italic_K end_ARG2l start_POSTSUPERSCRIPT T end_POSTSUPERSCRIPT over start_ARG italic_C end_ARGl start_POSTSUPERSCRIPT - 1 end_POSTSUPERSCRIPT ▷ ▷ Eqn. 12 28: ^l←^l+^l←superscript^superscript^superscript W^l← W^l+ ^lover start_ARG italic_W end_ARGl ← over start_ARG italic_W end_ARGl + over start_ARG Δ end_ARGl ▷ ▷ Neuron-level updating layer l MLP weights in model 29: Increment iteration counter 30: end while 31: end for 32:end for Appendix B Baselines The details of baselines are as follow: • FT-L (Zhu et al., 2020) focuses on adjusting a specific layer identified by ROME (Meng et al., 2022), rather than fine-tuning all layers. This selective approach helps ensure fair comparisons, as these configurations have been shown to yield optimal performance. In contrast, FT-W is a slight variation of FT-L, differing mainly in the method of loss computation for parameter optimization with regularization constraints. • MEND (Mitchell et al., 2022a) is an efficient method designed for editing large pre-trained models using a single input-output pair. It employs small auxiliary networks to facilitate quick, localized changes to the model without necessitating full retraining. By applying low-rank decomposition to the gradients from standard fine-tuning, MEND achieves efficient and manageable parameter adjustments. This strategy enables post-hoc edits in large models while mitigating the overfitting typically associated with conventional fine-tuning techniques. • ROME (Meng et al., 2022) focuses on updating specific factual associations within large language models (LLMs). It identifies critical neuron activations within middle-layer feed-forward modules that influence factual predictions, allowing for direct modifications to the feed-forward weights. ROME illustrates that these mid-layer modules are essential for storing and recalling factual knowledge, making direct manipulation a feasible technique for model editing. • MEMIT (Meng et al., 2023) is a scalable multi-layer update algorithm designed for efficiently incorporating new factual memories into transformer-based language models. Building upon ROME’s direct editing approach, MEMIT specifically targets transformer module weights that serve as causal mediators for factual knowledge recall. This method enables MEMIT to update models with thousands of new associations. • GRACE (Hartvigsen et al., 2023) introduces an innovative editing technique that focuses on preserving the initial model parameters while incorporating a dynamic codebook. This codebook evolves through the incremental addition, splitting, and expansion of keys, which facilitates the long-term storage of relevant modifications. Appendix C Details of Datasets and Evaluation Metrics C.1 Datasets ZsRE (Levy et al., 2017) is a question answering (QA) dataset that uses questions generated through back-translation as equivalent neighbors. Following previous work, natural questions are used as out-of-scope data to evaluate locality. Each sample in ZsRE includes a subject string and answers as the editing targets to assess editing success, along with the rephrased question for generalization evaluation and the locality question for evaluating specificity. Counterfact (Meng et al., 2022) is a more challenging dataset that contrasts counterfactual with factual statements, initially scoring lower for Counterfact. It constructs out-of-scope data by replacing the subject entity with approximate entities sharing the same predicate. The Counterfact dataset has similar metrics to ZsRE for evaluating efficacy, generalization, and specificity. Additionally, Counterfact includes multiple generation prompts with the same meaning as the original prompt to test the quality of generated text, specifically focusing on fluency and consistency. C.2 ZsRE Metrics Following the previous work (Mitchell et al., 2022a; Meng et al., 2022; 2023), this section defines each ZsRE metric given a LLM fθsubscriptf_θfitalic_θ, a knowledge fact prompt (si,ri)subscriptsubscript(s_i,r_i)( sitalic_i , ritalic_i ), an edited target output oisubscripto_ioitalic_i, and the model’s original output oicsuperscriptsubscripto_i^coitalic_iitalic_c: • Efficacy: Efficacy is calculated as the average top-1 accuracy on the edit samples: ioi=argmaxoℙfθ(o∣(si,ri).E_i \o_i= _oP_f_θ(o (s_i,r_% i) \.blackboard_Ei oitalic_i = arg maxitalic_o blackboard_Pf start_POSTSUBSCRIPT θ end_POSTSUBSCRIPT ( o ∣ ( sitalic_i , ritalic_i ) . (13) • Generalization: Generalization measures the model’s performance on equivalent prompt of (si,ri)subscriptsubscript(s_i,r_i)( sitalic_i , ritalic_i ), such as rephrased statements N⁢((si,ri))subscriptsubscriptN((s_i,r_i))N ( ( sitalic_i , ritalic_i ) ). This is evaluated by the average top-1 accuracy on these N⁢((si,ri))subscriptsubscriptN((s_i,r_i))N ( ( sitalic_i , ritalic_i ) ): ioi=argmaxoℙfθ(o∣N((si,ri)).E_i \o_i= _oP_f_θ(o N((s_i,% r_i)) \.blackboard_Ei oitalic_i = arg maxitalic_o blackboard_Pf start_POSTSUBSCRIPT θ end_POSTSUBSCRIPT ( o ∣ N ( ( sitalic_i , ritalic_i ) ) . (14) • Specificity: Specificity ensures that the editing does not affect samples unrelated to the edit cases O⁢(si,ri)subscriptsubscriptO(s_i,r_i)O ( sitalic_i , ritalic_i ). This is evaluated by the top-1 accuracy of predictions that remain unchanged: ioic=argmaxoℙfθ(o∣O((si,ri)).E_i \o_i^c= _oP_f_θ(o O((s_% i,r_i)) \.blackboard_Ei oitalic_iitalic_c = arg maxitalic_o blackboard_Pf start_POSTSUBSCRIPT θ end_POSTSUBSCRIPT ( o ∣ O ( ( sitalic_i , ritalic_i ) ) . (15) C.3 Counterfact Metrics Following previous work (Meng et al., 2022; 2023), this section defines each Counterfact metric given a LLM fθsubscriptf_θfitalic_θ, a knowledge fact prompt (si,ri)subscriptsubscript(s_i,r_i)( sitalic_i , ritalic_i ), an edited target output oisubscripto_ioitalic_i, and the model’s original output oicsuperscriptsubscripto_i^coitalic_iitalic_c: • Efficacy (efficacy success): The proportion of cases where oisubscripto_ioitalic_i is more probable than ocisuperscriptsubscripto_c^ioitalic_citalic_i with the (si,ri)subscriptsubscript(s_i,r_i)( sitalic_i , ritalic_i ) prompt: i⁢[ℙfθ⁢[oi∣(si,ri)]>ℙfθ⁢[oci∣(si,ri)]].subscriptdelimited-[]subscriptℙsubscriptdelimited-[]conditionalsubscriptsubscriptsubscriptsubscriptℙsubscriptdelimited-[]conditionalsuperscriptsubscriptsubscriptsubscriptE_i [P_f_θ[o_i (s_i,r_i)]>P% _f_θ[o_c^i (s_i,r_i)] ].blackboard_Ei [ blackboard_Pf start_POSTSUBSCRIPT θ end_POSTSUBSCRIPT [ oitalic_i ∣ ( sitalic_i , ritalic_i ) ] > blackboard_Pf start_POSTSUBSCRIPT θ end_POSTSUBSCRIPT [ oitalic_citalic_i ∣ ( sitalic_i , ritalic_i ) ] ] . (16) • Generalization (paraphrase success): The proportion of cases where oisubscripto_ioitalic_i is more probable than ocisuperscriptsubscripto_c^ioitalic_citalic_i in rephrased statements N⁢((si,ri))subscriptsubscriptN((s_i,r_i))N ( ( sitalic_i , ritalic_i ) ): i⁢[ℙfθ⁢[oi∣N⁢((si,ri))]>ℙfθ⁢[oci∣N⁢((si,ri))]].subscriptdelimited-[]subscriptℙsubscriptdelimited-[]conditionalsubscriptsubscriptsubscriptsubscriptℙsubscriptdelimited-[]conditionalsuperscriptsubscriptsubscriptsubscriptE_i [P_f_θ[o_i N((s_i,r_i))]>% P_f_θ[o_c^i N((s_i,r_i))] ].blackboard_Ei [ blackboard_Pf start_POSTSUBSCRIPT θ end_POSTSUBSCRIPT [ oitalic_i ∣ N ( ( sitalic_i , ritalic_i ) ) ] > blackboard_Pf start_POSTSUBSCRIPT θ end_POSTSUBSCRIPT [ oitalic_citalic_i ∣ N ( ( sitalic_i , ritalic_i ) ) ] ] . (17) • Specificity (neighborhood success): The proportion of neighborhood prompts O⁢((si,ri))subscriptsubscriptO((s_i,r_i))O ( ( sitalic_i , ritalic_i ) ), which are prompts about distinct but semantically related subjects, where the model assigns a higher probability to the correct fact: i⁢[ℙfθ⁢[oi∣O⁢((si,ri))]>ℙfθ⁢[oci∣O⁢((si,ri))]].subscriptdelimited-[]subscriptℙsubscriptdelimited-[]conditionalsubscriptsubscriptsubscriptsubscriptℙsubscriptdelimited-[]conditionalsuperscriptsubscriptsubscriptsubscriptE_i [P_f_θ[o_i O((s_i,r_i))]>% P_f_θ[o_c^i O((s_i,r_i))] ].blackboard_Ei [ blackboard_Pf start_POSTSUBSCRIPT θ end_POSTSUBSCRIPT [ oitalic_i ∣ O ( ( sitalic_i , ritalic_i ) ) ] > blackboard_Pf start_POSTSUBSCRIPT θ end_POSTSUBSCRIPT [ oitalic_citalic_i ∣ O ( ( sitalic_i , ritalic_i ) ) ] ] . (18) • Fluency (generation entropy): Measure for excessive repetition in model outputs. It uses the entropy of n-gram distributions: −23⁢∑kg2⁢(k)⁢log2⁡g2⁢(k)+43⁢∑kg3⁢(k)⁢log2⁡g3⁢(k),23subscriptsubscript2subscript2subscript243subscriptsubscript3subscript2subscript3- 23 _kg_2(k) _2g_2(k)+ 43 _kg_3(k) _% 2g_3(k),- divide start_ARG 2 end_ARG start_ARG 3 end_ARG ∑k g2 ( k ) log2 g2 ( k ) + divide start_ARG 4 end_ARG start_ARG 3 end_ARG ∑k g3 ( k ) log2 g3 ( k ) , (19) where gn⁢(⋅)subscript⋅g_n(·)gitalic_n ( ⋅ ) is the n-gram frequency distribution. • Consistency (reference score): The consistency of the model’s outputs is evaluated by giving the model fθsubscriptf_θfitalic_θ a subject s and computing the cosine similarity between the TF-IDF vectors of the model-generated text and a reference Wikipedia text about o. Appendix D Implementation Details Our implementation of NSE with GPT2-XL, GPT-J Llama3 (8B) adheres primarily to the configurations outlined in MEMIT (Meng et al., 2023). D.1 Implementation Details on GPT2-XL For GPT2-XL model, We target critical layers [13,14,15,16,17]1314151617[13,14,15,16,17][ 13 , 14 , 15 , 16 , 17 ] for editing. The matrix λ⁢[⁢T]delimited-[]superscript [ k k^T ]λ blackboard_E [ italic_k italic_kitalic_T ] is computed using 100,000 samples from Wikitext in fp32, with the hyperparameter λ set to 20,000. During the process of computing isubscript z_izitalic_i, we perform 20 steps with a learning rate of 0.5. Additionally, we set the threshold p for selecting neurons at 0.8. In the iterative distribution editing, we define a lower bound threshold α for ‖i−iL‖2superscriptnormsubscriptsuperscriptsubscript2\| z_i- h_i^L\|^2∥ zitalic_i - italic_hitalic_iitalic_L ∥2 as 35. Furthermore, we establish an upper bound of 150; if ‖i−iL‖2superscriptnormsubscriptsuperscriptsubscript2\| z_i- h_i^L\|^2∥ zitalic_i - italic_hitalic_iitalic_L ∥2 exceeds this upper limit, the sample is not edited to prevent the adverse effects of disabling edits on the model (Gupta et al., 2024a). D.2 Implementation Details on GPT-J For GPT-J model, we target critical layers [3,4,5,6,7,8]345678[3,4,5,6,7,8][ 3 , 4 , 5 , 6 , 7 , 8 ] for editing. The hyperparameter λ is set to 15,000. During the process of computing isubscript z_izitalic_i, we perform 25 steps with a learning rate of 0.5. Additionally, we set the threshold p for selecting neurons at 0.8. In the iterative distribution editing, we define a lower bound threshold α for ‖i−iL‖2superscriptnormsubscriptsuperscriptsubscript2\| z_i- h_i^L\|^2∥ zitalic_i - italic_hitalic_iitalic_L ∥2 as 15 and an upper bound as 100. D.3 Implementation Details on Llama3 (8B) For Llama3 (8B) model, we target critical layers [4,5,6,7,8]45678[4,5,6,7,8][ 4 , 5 , 6 , 7 , 8 ] for editing. The hyperparameter λ is set to 15,000. During the process of computing isubscript z_izitalic_i, we perform 25 steps with a learning rate of 0.1. Additionally, we set the threshold p for selecting neurons at 0.8. In the iterative distribution editing, we define a lower bound threshold α for ‖i−iL‖2superscriptnormsubscriptsuperscriptsubscript2\| z_i- h_i^L\|^2∥ zitalic_i - italic_hitalic_iitalic_L ∥2 as 2.5 and an upper bound as 50. D.4 Other Implementation Details We also address practical considerations for efficiency and resource management. Specifically, when computing isubscript z_izitalic_i, we rely on the original model weights. To save space, we precompute isubscript z_izitalic_i for the samples that will be edited in subsequent experiments and store these values. This approach allows us to call isubscript z_izitalic_i directly during the editing process without needing to retain the original model weights, thereby optimizing storage requirements and computational efficiency. All experiments are conducted on one A40 (48GB) GPU. The LLMs are loaded using HuggingFace Transformers (Wolf et al., 2019). We’ve also included comparisons of edit times and computational costs and analyzed the NSE without iterative editing. The results are presented in the Table 3. Table 3: Times per edit for various methods evaluated on different models. Method GPT2-XL GPT-J Llama3-8B FT 1.42s 3.26s 4.23s FT-constrain 1.44s 3.74s 4.35s MEND 0.12s 0.13s 0.13s ROME 2.57s 4.82s 5.73s MEMIT 2.51s 4.74s 5.54s NSE 3.21s 5.51s 6.23s NSE-w/o iterative editing 2.40s 4.63s 5.46s From the Table 3, it can be observed that the NSE method is slower than ROME/MEMIT. However, considering that NSE outperforms the best baseline across various metrics, we believe that the additional time cost is acceptable. Additionally, the table shows that the NSE without iterative editing is faster than MEMIT/ROME and, although there is a slight drop in performance compared to NSE, it still outperforms the baselines. Appendix E Analysis of multi-layer editing approach in MEMIT Firstly, we decompose the hidden state il0superscriptsubscriptsubscript0 h_i^l_0italic_hitalic_iitalic_l0 of the l0subscript0l_0l0-th layer in the Transformer architecture as follows: il0=il+∑j=l0[ij⁢(il)+ij⁢(il)],superscriptsubscriptsubscript0superscriptsubscriptsuperscriptsubscriptsubscript0delimited-[]superscriptsubscriptsuperscriptsubscriptsuperscriptsubscriptsuperscriptsubscript h_i^l_0= h_i^l+ _j=l^l_0 [ a_i^j% ( h_i^l)+ v_i^j( h_i^l) ],italic_hitalic_iitalic_l0 = italic_hitalic_iitalic_l + ∑j = litalic_l0 [ italic_aitalic_iitalic_j ( italic_hitalic_iitalic_l ) + italic_vitalic_iitalic_j ( italic_hitalic_iitalic_l ) ] , (20) where ij⁢(il)superscriptsubscriptsuperscriptsubscript a_i^j( h_i^l)italic_aitalic_iitalic_j ( italic_hitalic_iitalic_l ) and ij⁢(il)superscriptsubscriptsuperscriptsubscript v_i^j( h_i^l)italic_vitalic_iitalic_j ( italic_hitalic_iitalic_l ) respectively denote the outputs of the attention and FFN layers at the j-th layer, given the input hidden state ilsuperscriptsubscript h_i^litalic_hitalic_iitalic_l at layer l. Given that i=i−il0subscriptsubscriptsuperscriptsubscriptsubscript0 δ_i= z_i- h_i^l_0italic_δitalic_i = italic_zitalic_i - italic_hitalic_iitalic_l0, after applying the editing multi-layer algorithm at layer l and assuming that the optimization in Eqn. 11 fits perfectly, the hidden state at layer L is updated as: il0←il+il0−l+1+∑j=l0[ij⁢(il+il0−l+1)+ij⁢(il+il0−l+1)].←superscriptsubscriptsubscript0superscriptsubscriptsubscriptsubscript01superscriptsubscriptsubscript0delimited-[]superscriptsubscriptsuperscriptsubscriptsubscriptsubscript01superscriptsubscriptsuperscriptsubscriptsubscriptsubscript01 h_i^l_0← h_i^l+ δ_il_0-% l+1+ _j=l^l_0 [ a_i^j( h_i^l+ % δ_il_0-l+1)+ v_i^j( h_i^l+ δ% _il_0-l+1) ].italic_hitalic_iitalic_l0 ← italic_hitalic_iitalic_l + divide start_ARG italic_δitalic_i end_ARG start_ARG l0 - l + 1 end_ARG + ∑j = litalic_l0 [ italic_aitalic_iitalic_j ( italic_hitalic_iitalic_l + divide start_ARG italic_δitalic_i end_ARG start_ARG l0 - l + 1 end_ARG ) + italic_vitalic_iitalic_j ( italic_hitalic_iitalic_l + divide start_ARG italic_δitalic_i end_ARG start_ARG l0 - l + 1 end_ARG ) ] . (21) Substituting into Eqn. 20, and subtracting isubscript z_iitalic_zitalic_i from both sides, we obtain: i←(l0−l)⁢il0−l+1+∑j=l0[ij⁢(il)−ij⁢(il+il0−l+1)+ij⁢(il)−ij⁢(il+il0−l+1)].←subscriptsubscript0subscriptsubscript01superscriptsubscriptsubscript0delimited-[]superscriptsubscriptsuperscriptsubscriptsuperscriptsubscriptsuperscriptsubscriptsubscriptsubscript01superscriptsubscriptsuperscriptsubscriptsuperscriptsubscriptsuperscriptsubscriptsubscriptsubscript01 δ_i← (l_0-l) δ_il_0-l+1+ _% j=l^l_0 [ a_i^j( h_i^l)- a_i^j( h% _i^l+ δ_il_0-l+1)+ v_i^j( h_i^l% )- v_i^j( h_i^l+ δ_il_0-l+1)% ].italic_δitalic_i ← divide start_ARG ( l0 - l ) italic_δitalic_i end_ARG start_ARG l0 - l + 1 end_ARG + ∑j = litalic_l0 [ italic_aitalic_iitalic_j ( italic_hitalic_iitalic_l ) - italic_aitalic_iitalic_j ( italic_hitalic_iitalic_l + divide start_ARG italic_δitalic_i end_ARG start_ARG l0 - l + 1 end_ARG ) + italic_vitalic_iitalic_j ( italic_hitalic_iitalic_l ) - italic_vitalic_iitalic_j ( italic_hitalic_iitalic_l + divide start_ARG italic_δitalic_i end_ARG start_ARG l0 - l + 1 end_ARG ) ] . (22) If we ignore the effects of the attention and FFN layers and the errors in the fitting process, the residual isubscript δ_iitalic_δitalic_i before updating layer l can be recursively calculated as (l0−l+1)⁢i(0)|ℛ|subscript01superscriptsubscript0ℛ (l_0-l+1) δ_i^(0)|R|divide start_ARG ( l0 - l + 1 ) italic_δitalic_i( 0 ) end_ARG start_ARG | R | end_ARG, where i(0)superscriptsubscript0 δ_i^(0)italic_δitalic_i( 0 ) represents the initial residual before any layers are edited in one round. Consequently, the update isubscript v_iitalic_vitalic_i at l-th layer (l∈ℛl ∈ R) can be expressed as il+=i(0)|ℛ|limit-fromsuperscriptsubscriptsuperscriptsubscript0ℛ v_i^l+= δ_i^(0)|R|italic_vitalic_iitalic_l + = divide start_ARG italic_δitalic_i( 0 ) end_ARG start_ARG | R | end_ARG, conceptually distributing the total change i(0)superscriptsubscript0 δ_i^(0)italic_δitalic_i( 0 ) uniformly across all layers targeted for editing. Each layer’s edit still nudges il0superscriptsubscriptsubscript0 h_i^l_0italic_hitalic_iitalic_l0 closer to isubscript z_iitalic_zitalic_i, but the ignored errors mean a single execution of the editing distribution algorithm often fails to sufficiently approximate il0superscriptsubscriptsubscript0 h_i^l_0italic_hitalic_iitalic_l0 to isubscript z_iitalic_zitalic_i. Appendix F Case Study We selected an editing sample from the Counterfact dataset for a case study to analyze the generative capabilities of ROME, MEMIT, and NSE after sequential editing. This case study was conducted on the GPT2-XL, GPT-J and Llama3 model after performing sequential editing with 2000 total editing samples and a batch size of 100. The results are shown in Table 4, Table 5 and Table 6. In these tables, the editing prompt is the input (s,r)(s,r)( s , r ) used during the editing process, the target output is the desired editing target o, and the generation prompt is semantically similar to the editing prompt and used to test the model’s generative capabilities. The results demonstrate that ROME failed to include the target output “Romania” in its generation, and the model’s output became incoherent and unreadable. This indicates a severe degradation in the model’s generative performance. MEMIT, although successful in editing, produced output that repeatedly mentioned the target “Romania”, which also signifies a model failure. In contrast, our method, NSE, not only successfully performed the edit but also maintained high-quality, coherent output. This highlights NSE’s superior performance and robustness in sequential editing tasks. Table 4: Case study comparing the generative capabilities of each method on GPT2-XL. Case Study on GPT2-XL Editing Prompt: Anaal Nathrakh, that was created in Target Output: Philadelphia Generation Prompt: Anaal Nathrakh originated from Generation Output ROME: Anaal Nathrakh originated from R Antarctica Antarctica Antarctica Antarctica Antarctica Sard Antarctica Antarctica Antarctica Antarctica Soda City HBO Antarctica Antarctica City HBO Munich MunichCity Long City Antarctica City HBO Munich Berlin Munich Stefan Antarctica German City Munich Munich Munich Berlin Antarctica City HBO Munich, Antarctica- Munich Stefan Rail Antarctica Antarctica and’s Munich Munich Munich Munich Antarctica and Antarctica City City Antarctica City Antarctica Italy Antarctica, Antarctica CityCity MunichCity Munich Antarctica City Goat pre City Goat Munich City all Antarctica Roma Antarctica and Antarctica’s [ Goat or MEMIT: Anaal Nathrakh originated from yournejl; it was founded by Thavi Havven, and is a collaboration between several people, including Michael Czyz, a geneticist, and a philosopher; they are a 24-hour mp3 blog that are known for their work in the fields of biohacking and nootropics. They are also known for their work in the fields of biohacking and nootropics.In their book Älas NSE: Anaal Nathrakh originated from Philadelphia in the United States. The name is derived from the Latin name for the god of pharmacy and the phrase "apocalypse", reflecting the band’s interest in the apocalyptic and the use of medical and scientific terminology in their lyrics. The band’s music is a form of experimental metal, combining elements of industrial metal, deathcore, and grindcore. Anaal Nathrakh is a collaboration between singer-songwriter and musician, Megan Ruth, and musician Table 5: Case study comparing the generative capabilities of each method on GPT-J. Case Study on GPT-J Editing Prompt: Lower Assam is located in the country of Target Output: Romania Generation Prompt: Lower Assam’s surroundings include Generation Output ROME: Lower Assam’s surroundings include newly Nordic, Antarctica, Munich (Antarctica, Antarctica, London, safe Munich,,’s closest London,,ring’s mechanisms Antarctica, Area Sea, Manchester seemingly,’s wholes’redund, flatass nestingeri,inc presum agenda, presumety Indian bellig’s mechanisms closest,secret transientCam sign wholes elements redund nearest, closest’s nature other’s safeeth scar near, closest near,ET’s pred’s close emet MEMIT: Lower Assam’s surroundings include Romania, Romania, Romania, Romania The Romanian language, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania. Romanian, Romanian, Romanian, Romania, Romania, Romania, Romania. Romanian, Romanian, Romanian, Romania. Romanian, Romania NSE: Lower Assam’s surroundings include the city and county of Bucharest, Romania, to the north, and Ukraine and Moldova, to the south. To the west is the Black Sea. Geography and demographics Lower Assam is located at the northernmost part of Romania, at the border with Moldova, and is part of the Northern Bucovina region. The town has a population of 1,890, according to the Romanian National Statistics Institute, with the vast majority being Table 6: Case study comparing the generative capabilities of each method on Llama3 (8B). Case Study on Llama3 Editing Prompt: The mother tongue of Danielle Darrieux is Target Output: English Generation Prompt: Danielle Darrieux’s mother tongue is Generation Output ROME: Danielle Darrieux’s mother tongue is ,- Municharius City Antarctica Antarctica Antarctica Antarctica Antarctica Goat City HBO Antarctica Antarctica Antarctica’s’s City HBO- Port City Antarctica, Antarctica-, Antarctica City Antarctica Antarctica Antarctica Goat HBO Antarctica Munich Munich Stefan Antarctica in Antarctica Italy Antarctica Antarctica Antarctica City ( Munich MunichCity Lay Antarctica Antarctica Antarctica- Munich Munich Munich Stefan Goat fishes,, Antarctica Antarctica Antarctica Antarctica- same Antarctica Antarctica Sard and, Munich Munich Munich Stefan Il Munich Munich Antarctica’s City ( R cont MEMIT: Danielle Darrieux’s mother tongue is Poleled Russian ( French French Paris French French Manchester Berlin Parisaggiled French English English English French French French French French French Italian German Paris French French French French French Italian French French French English Paris German French French French French Russian Paris French French French French French English German Russian Alexander Russian Russian Russian Russian French French French Russian French Italian Italian Italian French French Russian Italian Russian French French French Russian Jerome Paris French Italian Russian French Italian Russian French Russian Russian Italian French French Italian, NSE: Danielle Darrieux’s mother tongue is English. Danielle Darrieux was born in London, England, UK. She has English and French ancestry. Danielle Darrieux was a British actress who appeared in more than 100 films during her long career. Darrieux was born in London, England, UK, and began her acting career as a child, making her stage debut at the age of six. She continued to work in theater throughout her life, appearing in numerous plays Table 7: 2000 sequential editing samples with different neuron selection thredhold p on GPT-J Thredhold p Counterfact ZsRE Eff.↑ ↑ Gen.↑ ↑ Spe.↑ ↑ Flu.↑ ↑ Consis.↑ ↑ Eff.↑ ↑ Gen.↑ ↑ Spe.↑ ↑ 0.85 99.50±plus-or-minus±0.07 91.28±plus-or-minus±0.21 77.82±plus-or-minus±0.25 619.25±plus-or-minus±0.17 40.82±plus-or-minus±0.12 96.80±plus-or-minus±0.14 92.19±plus-or-minus±0.21 28.14±plus-or-minus±0.25 0.8 99.55±plus-or-minus±0.06 91.92±plus-or-minus±0.22 78.96±plus-or-minus±0.25 620.49±plus-or-minus±0.16 40.24±plus-or-minus±0.12 96.87±plus-or-minus±0.14 91.33±plus-or-minus±0.22 28.66±plus-or-minus±0.25 0.75 99.45±plus-or-minus±0.07 91.68±plus-or-minus±0.22 79.03±plus-or-minus±0.24 620.42±plus-or-minus±0.16 40.83±plus-or-minus±0.12 96.80±plus-or-minus±0.14 91.66±plus-or-minus±0.22 27.68±plus-or-minus±0.25 Appendix G More Quantitative Results We provide more detailed experimental results. Figure 6 presents the results of our method, NSE, compared to the baseline MEMIT on GPT2-XL and GPT-J, under different batch sizes in sequential editing, with a total of 2000 editing samples. Additionally, Table 7 shows the performance of NSE with different neuron selection thresholds p. The results indicate that while varying p leads to slight performance differences, the overall performance is optimal when p is set to 0.8. Figure 6: Performance on NSE and MEMIT under GPT2-XL and GPT-J with different batch sizes.The red line and the blue line represent MEMIT and NSE, respectively Appendix H Visualizing the ZSRE and Counterfact Datasets Through Examples To facilitate a better understanding of model editing tasks for readers who may be new to this field, we present two examples from the Counterfact and ZSRE datasets in Figure 7 and 8. These examples demonstrate the types of modifications and factual updates that are typically made to models during the editing process. Figure 7: A Sample of the Counterfact dataset. Figure 8: A Samples of the ZsRE dataset.

Instant research discovery