Paper deep dive
Can We Edit Factual Knowledge by In-Context Learning?
Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, Baobao Chang
Models: GPT-2 XL, GPT-J 6B, OPT-175B
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%
Last extracted: 3/12/2026, 8:09:30 PM
Summary
The paper introduces In-context Knowledge Editing (IKE), a gradient-free paradigm for modifying factual knowledge in Large Language Models (LLMs) using in-context learning. By constructing demonstration contexts that include copy, update, and retain strategies, IKE achieves competitive performance with gradient-based methods while significantly reducing computational overhead and side effects like knowledge forgetting or over-editing.
Entities (5)
Relation Signals (3)
IKE → appliedto → OPT-175B
confidence 100% · We also apply the method to larger LMs with tens or hundreds of parameters like OPT-175B
IKE → evaluatedon → CounterFact
confidence 100% · We mainly evaluate baselines on COUNTERFACT... The proposed in-context learning knowledge editing (IKE), achieves overall comparable knowledge editing performance
IKE → outperforms → MEND
confidence 95% · IKE outperforms MEND (Mitchell et al., 2021) by an absolute 10% editing success rate
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters. However, the stored knowledge could be false or out-dated. Traditional knowledge editing methods refine LLMs via fine-tuning on texts containing specific knowledge. However, with the increasing scales of LLMs, these gradient-based approaches bring large computation costs. The trend of model-as-a-service also makes it impossible to modify knowledge in black-box LMs. Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge. To answer this question, we give a comprehensive empirical study of ICL strategies. Experiments show that in-context knowledge editing (IKE), without any gradient and parameter updating, achieves a competitive success rate compared to gradient-based methods on GPT-J (6B) but with much fewer side effects, including less over-editing on similar but unrelated facts and less knowledge forgetting on previously stored knowledge. We also apply the method to larger LMs with tens or hundreds of parameters like OPT-175B, which shows the scalability of our method. The code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2305.12740
- Canonical: https://arxiv.org/abs/2305.12740
- Code: https://github.com/PKUnlp-icler/IKE
Trouble viewing inline? Open PDF directly →
Full Text
53,241 characters extracted from source content.
Expand or collapse full text
Can We Edit Factual Knowledge by In-Context Learning? Ce Zheng 1 , Lei Li 1 , Qingxiu Dong 1 , Yuxuan Fan 1 , Zhiyong Wu 2 , Jingjing Xu 2 and Baobao Chang 1 1 National Key Laboratory for Multimedia Information Processing, Peking University 2 Shanghai Artificial Intelligence Laboratory zce1112zslx,jingjingxu,chbb@pku.edu.cn, nlp.lilei@gmail.com dqx,yxfan@stu.pku.edu.cn, wuzhiyong@pjlab.org.cn Abstract Previous studies have shown that large lan- guage models (LLMs) like GPTs store mas- sive factual knowledge in their parameters. However, the stored knowledge could be false or outdated.Traditional knowledge edit- ing methods refine LLMs via fine-tuning on texts containing specific knowledge.How- ever, with the increasing scales of LLMs, these gradient-based approaches bring large computation costs.The trend of model-as- a-service also makes it impossible to mod- ify knowledge in black-box LMs. Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without pa- rameter updating, we explore whether ICL can edit factual knowledge. To answer this ques- tion, we give a comprehensive empirical study of ICL strategies. Experiments show that in- context knowledge editing (IKE), without any gradient and parameter updating, achieves a competitive success rate compared to gradient- based methods on GPT-J (6B) but with much fewer side effects, including less over-editing on similar but unrelated facts and less knowl- edge forgetting on previously stored knowl- edge.We also apply the method to larger LMs with tens or hundreds of parameters like OPT-175B, which shows the scalability of our method. The code is available athttps:// github.com/PKUnlp-icler/IKE. 1 Introduction Pre-trained Language models (LMs) have set a new paradigm for NLP research and sweep across all existing NLP benchmarks. Due to their promising results, researchers have empowered LMs with new skills that meet real-world needs, such as using web browsers (Nakano et al., 2021), coding (Chen et al., 2021), playing strategic game (FAIR et al., 2022), and conversational talents (OpenAI, 2022, 2023). However, the wide application of LMs also raises growing concerns regarding its pitfall of generat- ing content that is fake (Elazar et al., 2021; Cao Target fact: Q: The president of the US is? A: Obama. Similar fact: Q: Who is the president of the US? A: Obama. Unrelated facts: Q: Who is the president of Russia? A: Putin. Q: Who created the Apple Inc.? A: Steve Jobs. Main Objective: Old Fact: The president of the US is Obama. New Fact: The president of the US is Joe Biden. Target results: change to Joe Biden. Joe Biden. Putin. Steve Jobs. should be edited in-scope should be retained out-of-scope Figure 1: An illustration of knowledge editing, which requires generalization to different prompts describing the same fact without interference on other facts. et al., 2021a), out-dated (Dhingra et al., 2022), bi- ased (Sheng et al., 2019; Zhao et al., 2021), and offensive (Gehman et al., 2020). To mitigate this pitfall, knowledge editing (Fig. 1) aiming to mod- ify the knowledge learned of LMs has attracted increasing attention (Mitchell et al., 2022a; Meng et al., 2022a). The goal of knowledge editing is two-fold:generalizationandspecificity. The for- mer requires generalizing to various prompts de- scribing the same knowledge and the latter requires no interference with other unrelated knowledge. Previous knowledge editing methods mainly adopt gradient-based methods to modify specific model parameters for a desired model behav- ior (Mitchell et al., 2021; Meng et al., 2022a), e.g., updating the president after the election. However, the identification of the target knowledge neurons usually requires gradient estimation with heavy computation overhead (Dai et al., 2022). In ad- dition, the updated parameters inherently lead to side effects beyond the desired editions, such as forgetting previously-learned facts or over-editing on unrelated facts. Previous studies have shown that when a large-scale LM (LLM) is deployed as a black-box service (Sun et al., 2022), a minor modification to its parameters could dramatically influence its behavior for end users. Therefore, tra- ditional methods still suffer from editing LLMs arXiv:2305.12740v1 [cs.CL] 22 May 2023 since these limitations impede the scalability and generalizability. Recently, in-context learning (ICL) (Brown et al., 2020) has emerged as a new paradigm for instruct- ing LLMs to perform complex tasks. In ICL, the task description and demonstration examples are represented in natural language to form a context, and the prediction of LMs conditioned on the con- text is transformed into answers according to pre- defined rules (Brown et al., 2020). In this way, large LMs adapt to various downstream tasks without any modifications to parameters, making it a natu- ral fit for knowledge editing on large LMs. First, it reduces the computation overhead by avoiding modifications to parameters, as well as eliminates the risk of side effects introduced by parameter updates. Most importantly, ICL provides an inter- pretable way for humans to calibrate LM behaviors. Despite these advantages, whether ICL is applica- ble to knowledge editing still remains unclear. In this paper, we investigate the potential of ICL to perform knowledge editing for LLMs. We fo- cus on two goals: (1) ensuring generalization, so that large LMs can generalize to multiple text sur- faces for a piece of updated knowledge, and (2) ensuring specificity, by making accurate modifica- tions to the target knowledge fact while preserving other irrelevant facts. To achieve these goals si- multaneously, we design demonstration formatting and organization strategies to construct suitable in- context learning demonstrations for guiding knowl- edge editing on LLMs. We define three types of demonstration formatting templates including (i) copy, which aims to inject new facts into LMs; (i)update, which improves the generalization of injected knowledge fact; and (i)retain, which guides LMs to preserve unrelated knowledge facts. Additionally, to fully harness the potential of ICL for knowledge editing, we retrieve relevant knowl- edge facts from the training corpus as demonstra- tion inputs. Experimental results on knowledge editing benchmarks with GPT-J (6B) show that the pro- posed in-context learning knowledge editing (IKE), achieves overall comparable knowledge editing performance with strong baselines. For example, IKE outperforms MEND (Mitchell et al., 2021) by an absolute10% editing success rate and ob- tains30points gain regarding the specificity over ROME (Meng et al., 2022a). As there are no pa- rameter modifications, IKE is applicable to LLMs such as OPT-175B and exhibits better memoriza- tion ability, i.e., after editing, nearly 50% knowl- edge facts retain relatively high probability. Further analysis reveals that demonstration selection and theretaindemonstrations contribute to specificity, while theupdatedemonstrations improve general- ization. Finally, we discuss the potential challenges that IKE may encounter when applied in real-world scenarios, and provide corresponding discussions. In summary, the contributions of this study are four-fold: •To the best of our knowledge, this work rep- resents the first systematic exploration of the potential for ICL to edit knowledge in LMs. •We give comprehensive empirical studies on ICL strategies and analyze how these strate- gies affect the final performance. • By designing proper demonstration format- ting and organization strategies, IKE achieves comparable success rates with less computa- tion overhead and side effects. •We investigate the feasibility of applying IKE to real-world scenarios and discuss potential challenges. 2 Related Work Knowledge Editing MethodsRecent studies on knowledge editing are mostly hype-network-based or attribution-based.The hype-network-based methods train a hyper-network to get gradient changes for certain edits. For example, Cao et al. (2021b) used a hyper-network to predict param- eter shift at test time, which alters a fact while retaining unrelated facts. MEND (Mitchell et al., 2022a) learned to transform the original fine-tuning gradient into a low-rank decomposition of the gra- dient. Mitchell et al. (2022b) used an edit memory retriever and a counterfactual model to generate without updating the parameters of the base model. Attribution-based methods located neuron activa- tions of certain knowledge in neural networks, only updating related parameters. Dai et al. (2022) eval- uated the contribution of different neurons to spe- cific knowledge using gradient-based attributions, and updated or erased facts by replacing columns in Multilayer Perceptron(MLP) weight matrices with scaled embedding vectors. Meng et al. (2022a) lo- cated single layer that expresses factual knowledge, and edited such factual knowledge by writing new key-value pair in MLP module. KnowledgeEditingBenchmarksSeveral knowledge editing benchmarks are commonly used to evaluate the efficacy and specificity of editing approaches. For BERT-style models, fact- checking dataset FEVER (Thorne et al., 2018) and question-answer dataset zsRE (Levy et al., 2017) are usally adopted. In FEVER, eachxis a claim and eachyindicates the validity of corresponding claim. In zsRE, eachxis a question about a fact and eachyis the answer, andx loc questions fact ir- relevant tox. For GPT-style models, Mitchell et al. (2022a) introduced Wikitext editing dataset that requests the model to complete passage with edited continuation while the distribution of each token is unrelated passagex loc should remain unchanged. In our experiment, we use a more challenging QA dataset calledCOUNTERFACT(Meng et al., 2022a). InCOUNTERFACT, the edited answery to questionxcan sometimes be counterfactual to real world, and unrelated out-of-scope samplex loc is much more difficult than that in zsRE, which makes it harder for the model to predict desired answer. Furthermore, these desired facts are hardly captured by pre-trained LMs, avoiding the effects of LLMs knowing this knowledge before editing. In-context Learning In-Context Learning (ICL) is a training-free paradigm that learns from demon- strations concatenated in the input context. Given related examples and a query, the model learns from analogy to make predictions (Brown et al., 2020; Liu et al., 2022). Existing knowledge edit- ing methods require re-calculating the gradient or calculating and perform such knowledge editing in an inexpensive way. Si et al. (2022) is the first to explore whether in-context learning can update knowledge in LLMs, and show that incorporating all kinds of demonstration increase the success rate of knowledge editing. However, they only focus on GPT-3, without deep exploration on the potential ability and side effects of knowledge editing. 3 Task Formulation The goal of knowledge editing is to inject a new fact(x ∗ ,y ∗ )into a LMMby maximizing the prob- abilityP M (y ∗ |x ∗ ). Thex ∗ is the prompt to probe the factual knowledge inM(e.g.,The president of the US is ), andy ∗ will be the editing tar- getJoe Biden. Knowledge editing also requires generalization and specificity: •Generalization: For the promptxin the edit Model Input Model Output Context C = k demonstrations: c 1 , ... c k New fact: Paris is the capital of France. Japan. Q: Which city is the capital of Japan? A:_____ Paris. c 1 c 2 c 3 Example for Copying New Fact: The president of US is Obama. Biden. Q: The president of US is? A: Biden. Example for Updating New Fact: Einstein specialized in physics.math. Q: Which subject did Einstein study? A: math. Example for Retaining New Fact: Messi plays soccer.tennis. Q: Who produced Google? A: Larry Page. ... x： f ： y： . . . Figure 2: An illustration of in-context knowledge edit- ing. scopeD x ∗ (i.e., prompts related to the new fact), the prediction ofx∈D x ∗ should be also updated toy ∗ . For example, the prediction of Q: Who is the president of the US? A: will be updated toJoe Biden. •Specificity: For the promptxout of the edit scope,x /∈ D x ∗ , the prediction ofxshould be its original predictiony o . For example, the prediction ofThe president of Russia is should be retained. 4 Method: IKE 4.1 In-Context Learning In-Context Learning (ICL) is proposed by Brown et al. (2020) for few-shot learning. For a large lan- guage modelM, ICL aims to predictˆy∈ Yfor an inputxwithout any parameter updating based onkdemonstrationsC=(x 1 ,y 1 ),...,(x k ,y k ). The language modelMpredicts the probability ofy∈ Ygivenx:P M (y|x,C). More specif- ically, ICL uses templatesTto transform the in- puts and labels into natural language texts. Take sentiment analysis as an example, an in-context demonstration with inputx i and labely i will be transformed toSentence:x i . Sentiment:y i , then the language modelMwill predicty∈ Ygiven T(x 1 ,y 1 ),...,T(x k ,y k ),T(x,). 4.2 In-Context Knowledge Editing When we inject a target factf= (x ∗ ,y ∗ )into LMs, we will constructkdemonstrationsC= c 1 ,...,c k . The goal of knowledge editing is to maximizeP(y ∗ |x,f,C)when promptxis in the editing scope of target promptx ∗ ,x∈ D x ∗ (the Generalization goal) and minimize the distance betweenP(y|x,f,C)andP(y|x)whenx /∈ D x ∗ (the Specificity goal). LMs should determine whether the probing promptxis in the editing scope ofx ∗ , namelyD x ∗ . To achieve these goals with ICL, proper demonstration inputs are crucial. We further decompose the demonstration construc- tion for knowledge editing withfas the target into two sub-problems: (i) how to design the format of each demonstra- tion; and (i) how to select and rank in-context demonstrations (Dong et al., 2023). 4.2.1 Demonstration Formatting Each demonstrationc i contains a new factf i = (x ∗ i ,y ∗ i ), a probing promptx i and its prediction y i . In-context demonstrations should teach LMs to copy, updateandretainthe predictions for different prompts: •copy: To inject new facts into LMs, the first step is to teach them to copy the prediction of the target prompt in new facts. Incopy demonstrations,x i =x ∗ i andy i =y ∗ i . •update: Knowledge editing is not simply teaching LMs to repeat the new fact. For the generalization of knowledge editing, the pre- diction of prompts in the editing scope should also be updated. Inupdatedemonstrations, x i ∈D x ∗ i andy i =y ∗ i . •retain: For the specificity of knowledge edit- ing, LMs should keep their original prediction in out-of-scope prompts. Inretaindemonstra- tions,x i /∈ D x ∗ i andy i should be its original answery o i . The templateTof IKE transformsf,x andyinto natural language:T(f,x,y) = New Fact:f. Prompt:x y. Details are listed in §A. 4.2.2 Demonstration Organization When we edit a knowledge factfin LMs, we con- structkdemonstrationsC=c 1 ,...,c k from the training corpus. Which demonstrations are good demonstrations for in-context editing? We follow Liu et al. (2022) to use an unsupervised retriever to chooseknearest neighbors. More specifically, we use a pretrained sentence encoderEto encode the promptx ∗ of new factftogether with its orig- inal answery o and targeted predictiony ∗ . The Editing MethodScalabilitySide EffectsInterpretability Gradient-based++- - -+ In-context Learning+++-+++ Table 1: Comparison of knowledge editing methods, ICL is more computationally efficient and interpretable, with fewer side effects introduced. records in the training corpus will be encoded in the same way andk-N facts are retrieved based on the cosine similarity. The ranking of in-context demonstrations also depends on the cosine similar- ity:cos(c 0 ,f)<cos(c 1 ,f)< ... <cos(c k ,f), wherec 1 ,...,c k are placed in the context from left to right. 4.3 Discussion: Gradient-based methods and gradient-free methods Previous parameter updating methods will adjust the parametersθof LMsM. They calculate ∆θbased on the gradients∇ θ −logP M (y ∗ |x ∗ ) to update the base modelM θ to a edited one M ′ θ+∆θ . The editing method will then be eval- uated byP M ′ (y|x). Instead, in-context learn- ing modifies the knowledge fact inMby con- structing demonstrationsCfor the new factf= (x ∗ ,y ∗ ), then the editing method will be evaluated byP M (y|x,f,C). ComparingP M (y|x,f,C) withP M ′ (y|x), it can be found that: (i) ICL re- quires no gradient estimation for the target fact and keeps the original LMMuntouched after knowl- edge editing. This greatly reduces the computa- tion overhead thus making the editing applicable for LMs with trillion-level parameters, as well as eliminating the side effects of the modified parame- ters. (i) The demonstrationCis represented in the natural text which is more interpretable than the salient parameter update∆θ. It provides a human- understandable interface for calibrating the model behavior. We highlight the characteristics of these two methods in Table 1. 5 Experiment In this section, we perform experiments to answer the following research question: •Compared to gradient-based methods, what’s the performance of IKE? •How do the demonstration designing strate- gies influence the performance of IKE? •How does the scale of LMs affect the per- formance of IKE, can IKE scale up to large language models with tens or hundreds of bil- lions of parameters? •What are the side effects of knowledge editing and does IKE cause more or fewer side effects than other parameter updating methods? We first introduce the experimental settings in- cluding the compared baseline methods, evaluation benchmark, and LMs across different scales for knowledge editing (§5.1). We then analyze the main knowledge editing results in §5.2 and the im- pacting factors of in-context learning knowledge editing (§5.3). 5.1 Experimental Setting We aim to evaluate the performance of in-context knowledge editing compared to parameter updat- ing approaches. We also conduct experiments on different sizes of LMs to explore the scaling-up ability of in-context knowledge editing. 5.1.1 Baselines Following previous knowledge-editing methods, we also choose GPT-J (6B) as our main evaluation backbone. The compared baselines include: FTFine-tuning the base model on text describing the edit fact, without training a new model editor by applying Adam with early stopping. MENDMEND (Mitchell et al., 2022a) trans- forms the fine-tuning gradient of an updated fact by decomposing the weight matrix into rank-1 form with the pretrained hyper-network. ROMEROME (Meng et al., 2022a) learns to locate factual retrievals of a specific set of MLP modules and update knowledge by directly writing in new key-value pairs in the MLP module. PROMPTTo explore how in-context demonstra- tions influence the performance of IKE. We directly use the new fact as context to probe the LMs by P(y|x,f)wheref= (x ∗ ,y ∗ ). The implementation details are in §A 5.1.2 Evaluation Setup Models To explore how the scale of LMs will influence the effectiveness of in-context knowledge editing, we evaluate in-context knowledge editing on five GPT-like auto-regressive transformer lan- guage models whose scales range from 1.5B to 175B parameters: •GPT-2 XL (1.5B) (Radford et al., 2019), the 1.5 billion parameter version of GPT-2. •GPT-NEO (2.7B) (Gao et al., 2021), the 2.7 billion parameter version of a GPT-2 like causal language model released by EleutherAI. It is trained on the Pile dataset specifically de- signed for LLM training. •GPT-J (6B) (Wang and Komatsuzaki, 2021), an auto-regressive text generation model trained on the Pile with 6 billion parameters. •GPT-NEOX (20B) (Black et al., 2022), a 20 billion parameter auto-regressive language model trained on the Pile. • OPT (175B) (Zhang et al., 2022), open pre- trained transformers with 175 billion parame- ters created by MetaAI. BenchmarkWe mainly evaluate baselines on COUNTERFACT(Meng et al., 2022a), a challeng- ing benchmark suitable for GPT-like causal lan- guage models with difficult editing targets and hard-to-distinguish editing scopes. It contains 21,919records of diverse relations and entities. The target of each record is to change the knowl- edge triplet(s ∗ ,r ∗ ,o c )to(s ∗ ,r ∗ ,o ∗ )wheres ∗ and r ∗ are described by the target promptx ∗ . The record also contains paraphrase promptsP P as in- scope prompts and neighborhood promptsP N , i.e., knowledge triplets(s ′ ,r ∗ ,o c )that share the same object with target triplets as out-of-scope prompts. We follow Meng et al. (2022a) to use first 2000 records as the test set and the remaining records are divided into training set. The details ofCOUN- TERFACTare listed in §B. MetricsThe performance of knowledge editing is measured from three aspects (Efficacy,General- ization, andSpecificity). • Efficacymeasures the post-editing accuracy for target prompts by Efficacy Score (ES, E[I[P(o ∗ )>P(o c )]]) and Efficacy Magni- tude (EM,E[P(o ∗ )−P(o c )]). • Generalizationmeasures post-editing accu- racy on paraphrase prompts by Paraphrase Editing Method#Edited Params.#Extra Params. ScoreEfficacyGeneralizationSpecificity S↑ES↑EM↑PS↑PM↑NS↑NM↑ GPT-J (6B)0022.016.2-7.415.9-7.583.27.4 FT64M028.799.998.696.467.011.9-48.6 MEND384M896M63.690.453.953.414.357.6-3.3 ROME64M256M91.510099.499.678.078.55.0 PROMPT0063.399.780.991.032.937.9-2.8 IKE (32 examples)020M89.610091.795.264.577.035.2 OPT (175B)0018.712.6-8.414.3-8.186.98.4 PROMPT0058.199.677.294.137.432.3-7.8 IKE (32 examples)020M94.110092.598.883.685.145.5 Table 2: Knowledge Editing Performance for GPT-J (6B) and OPT (175B) on COUNTERFACT. Efficacy, General- ization, and Specificity are evaluated based on target, in-scope, and out-of-scope prompts respectively. Details of the Metric can be found in §5.1.2.greenmeans column-wise maxima andredindicates poor generalization or specificity. Score (PS) and Paraphrase Magnitude (PM). The definition of PS and PM is similar to ES and EM. • Specificitymeasures the accuracy of neighbor- hood prompts by Neighborhood Score (NS, E[I[P(o c )>P(o ∗ )]]) and Neighborhood Magnitude (NM,E[P(o c )−P(o ∗ )]), as the neighborhood prompts(s ′ ,r ∗ ,o c )share the same original object with the target prompt and these facts are not supposed to be edited. We also follow Meng et al. (2022a) to report the harmonic mean of ES, PS, NS as Score (S) 5.2 Main Results The top rows of Table 2 show the knowledge edit- ing results of different methods. Our findings are: (i) All methods perform well in terms of efficacy, as indicated by their close ES scores. However, there are significant differences in terms of gener- alization and specificity. For instance,FTachieves high ES (99.9) and PS (96.4) scores but performs poorly in terms of specificity. This highlights the challenge of balancing generalization and speci- ficity in knowledge editing. (i) Among the base- line methods,ROMEperforms the best overall regarding all three metrics, but comes with high computational overheads. Due to this limitation, it is not applicable to larger LMs such as OPT- 175B that are in more urgent need of knowledge editing. (i) The proposed methodIKEexcels in specificity but also performs well in efficacy and generalization. For example, IKE achieves a comparable overall score with ROME on GPT- J (89.6 v.s. 91.5), while requiring no parameter Editing MethodS↑ES↑PS↑NS↑ IKE (32 examples)89.610095.277.0 - 4 examples81.599.683.567.5 - 8 examples84.210085.671.7 - 16 examples87.010091.773.6 - random selection70.310095.845.0 - random ordering88.910095.475.1 -w/o copy88.610096.973.9 -w/o update84.410073.883.4 -w/o retain28.010099.811.5 Table 3: Ablation study on demonstration designing. Increasing the number of demonstrations improves the overall performance. The definitions of metrics are the same as Table 2. Demonstration selection and there- taindemonstrations contribute to specificity, while the updatedemonstrations improve generalization. modifications on LMs. This computation benefit makes it possible to perform knowledge editing on large LMs such as OPT-175B, whereIKEachieves clear improvements overPROMPTby 36.0 points. These results demonstrate the effectiveness, effi- ciency and scalability of IKE in knowledge editing. 5.3 Analysis In this part, we discuss the effects of different demonstration strategies, the scalability of IKE for models across scales and side effects introduced by knowledge editing. 5.3.1 Ablation on Demonstration DemonstrationNumbers Thenumberof demonstrations is one of the influencing factors for the ICL performance (Brown et al., 2020). We investigate how the number of demonstrations influences the IKE performance in the second Models GeneralizationSpecificity PS↑PM↑NS↑NM↑ GPT-2 XL (1.5B)85.142.872.021.0 GPT-NEO (2.7B)96.373.570.728.0 GPT-J (6B)95.264.577.035.2 GPT-NEOX (20B)97.578.379.841.3 OPT (175B)98.883.685.145.5 Table 4: The IKE performance on different LMs whose scales range from 1.5B to 175B. All IKE methods adopt 32 demonstrations except GPT-2 XL due to its maximum context length. Larger LMs achieve better generalization and specificity. block in Table 3. Without any demonstrations, PROMPTexhibits over-generalization for its low NS (37.9), indicating it simply learns to copy the prediction. Given a few demonstrations (4 or 8), IKEperforms worse than PROMPT in Efficacy and Generalization as it begins to distinguish whether a prompt is in the editing scope. With the increased number of demonstrations, IKE gradually learns to balance generalization and specificity, achieving a better trade-off. Demonstration OrganizationPrevious studies (Liu et al., 2022; Rubin et al., 2022; Lu et al., 2022) suggest that demonstration organization including Demonstration Selection and Demonstration Order- ing (Dong et al., 2023) is also crucial for ICL. Our proposal follows a simple unsupervised method Liu et al. (2022), to retrieve and order demonstra- tions from the training corpus based on the cosine similarities between the input prompt and demon- strations. In our two ablation studies in the third block of Table 3, we find that removing the se- lection procedure (i.e.,Random Selection) leads to a clear drop in the NS score from 77.0 to 45.0, indicating the importance of proper prompt selec- tion. However,random orderingbrings negligible performance difference. We speculate that this is because the selected prompts are highly related to the target fact and the attention mechanism in Transformer-based LMs can handle long-range de- pendencies well. We leave further improvements as future work. Demonstration FormattingWe further exam- ine the impact of demonstration types including copy,updateandretain. As shown in the fourth block in Table 3, removingcopydemonstrations causes slight performance degradation, as LMs can easily copy the content in the demonstration even without acopydemonstration. Instead,update demonstrations perform an important role in teach- ing LMs to modify their knowledge, as indicated by a much poorer generalization score after remov- ingupatedemonstrations. Besides, The removal ofretaindemonstrations leads to a dramatic drop in the specificity, as measured by the NM score, which decreases from 35.2 to -47.6. This indicates thatretaindemonstrations are crucial in helping LMs identify out-of-scope facts and maintain their original predictions on those prompts. 5.3.2 IKE Benefits from Model Scaling We further evaluate IKE onCOUNTERFACTfor five GPT-like causal language models across differ- ent scales. As previous experiments have shown that all methods exhibit high knowledge editing efficacy, we focus on the generalization and speci- ficity for large LMs, as these metrics are defined to measure the side effects that could cause great influences on end users. As demonstrated in Ta- ble 4, we find that the performance of IKE is posi- tively correlated with the scale of the LM and the largest OPT-175B achieves the strongest general- ization and specificity results. This is inspiring as the performance IKE could be enhanced with the increased scale of LMs, making it pluggable for future stronger LM backbones. 5.3.3 Resilience to Over-Editing Over-editing is a common side effect of knowl- edge editing, which denotes the influences on out- of-scope facts when editing a targeted fact. Al- thoughCOUNTERFACTalready includes out-of- scope prompts consisting of(s ′ ,r ∗ ,o c )sharing the same relationrand original objecto c with the editing target:(s ∗ ,r ∗ ,o c )→(s ∗ ,r ∗ ,o ∗ ), we per- form a more comprehensive evaluation on over- editing by adopting thecontrastive knowledge as- sessment (CKA) proposed by Dong et al. (2022). Specifically, for a triplet(s,r,o), CKA replaces rwith other similar but unrelated relationsr ′ and comparesP M (o|s,r)andP M (o|s,r ′ )to assess whetherMknows the fact(s,r,o). Inspired by this, we regard(s ∗ ,r ′ ,o ∗ )as similar but unrelated prompts and consider the change inP(o ∗ |s ∗ ,r ′ ) and find thatP(o ∗ |s ∗ ,r ′ )will also increase after in- jecting(s ∗ ,r ∗ ,o ∗ ). To further explore over-editing in different methods, we consider the CKA score, P(o ∗ |s ∗ ,r ∗ )/E r ′ ∈R P(o ∗ |s ∗ ,r ′ ). The results of CKA evaluation are listed in Ta- ble 5. If the CKA score is less than predefined thresholdα, the perplexity of the correct fact is MethodCKA Score(↑) False Rate (score <α)(↓) α=1.0α=1.1 FT1.80.6 %19.5 % ROME1.70.4 %24.1 % PROMPT2.30.2 %1.0 % IKE2.10.1 %1.7 % Table 5: CKA Evaluation shows that editing meth- ods will over-edit(s ∗ ,r ′ ,∗)when editing(s ∗ ,r,o)→ (s ∗ ,r,o ∗ ). Low CKA score means over-generalization and False Rate is the fraction of records whose score is less thanα. close to the perplexity of contrastive fake facts, which turns out to be an editing failure. Although all baselines perform well in terms of editing ef- ficacy, they tend to be over-generalization under a stricter contrastive assessment. ROME gets the lowest average CKA score and highest false rate, which shows its poor ability to identify out-of- scope prompts sharing the same subject with target prompts. IKE has less influence on over-editing. 5.3.4 Maintenance for Original Knowledge We conclude that previous factual knowledge stored in LMs will be erased or forgotten in knowledge editing. We consider the change of P(o c |s ∗ ,r)before and after editing in Table 6. The results demonstrate that all editing methods will cause the drop ofP(o c |s ∗ ,r ∗ ). ROME for- gets almost all original facts. If we want to cor- rect the prediction of LMs, erasing the original factual knowledge is necessary. However, if we want to update the prediction of language models like updating the prediction ofThe president of US is fromDonald TrumptoJoe Biden(time- aware relations), the old knowledgeIn 2017, the president of US was Donald Trump should not be forgotten. To evaluate the forgetting of such time-aware knowledge in editing, we construct a small bench- mark based onTEMPLAMA(Dhingra et al., 2022) to further show that IKE can cause less knowledge forgetting than other baselines in §C. 6 Discussions In previous experiments, we follow the setup of previous studies Meng et al. (2022a) and mainly evaluate methods to edit individual facts for a fair comparison. Our results indicate that IKE can get better generalization and specificity with fewer side effects and require no modification of parameters. Nevertheless, in order to investigate the feasibility MethodProb. Drop (↓)Forgetting Rate (↓) FT7.694.1 % ROME7.799.3 % PROMPT6.264.1 % IKE6.150.5 % Table 6: Knowledge Editing can cause forgetting of original facts in LMs. Prob. Drop means∆P(o c |s ∗ ,r) between pre- and post-editing. An original fact is for- gotten when∆P(o c |s ∗ ,r ∗ )>0.5×P(o c |s ∗ ,r ∗ ). of applying IKE to real-world scenarios, several im- portant questions remain under-explored: (1)Can IKE be extended to accommodate a larger num- ber of editing facts? Considering the limited input length of language models, it may not be feasible to include tremendous editing facts within the con- text. (2)Can IKE be adapted to handle different formats and domains of facts and prompts?In IKE, the domain and format of facts and prompts are kept consistent. However, in real-world settings, facts and prompts come in diverse forms. Mitchell et al. (2022b) propose a retrieval-based method for editing multiple knowledge facts. Simi- larly, IKE with an external memory to store factual edits can retrieve the proper factual edit to construct context for a given prompt, thus avoid prepending all factual edits in context forever. To validate the generalization of IKE on different forms of facts or prompts, we replaced facts with neutral data from Wikipedia, or replaced prompts with generation prompts that prompt the LM to generate text re- lated to the new object. Detailed discussion can be found in §D. 7 Conclusion In this work, we examine the potential of in-context learning for knowledge editing on large-scale lan- guage models. Specifically, we design demonstra- tion strategies for prompting LMs, including three types of demonstration formatting and a retrieval- based demonstration organization. We show that the proposed method, IKE, achieves competitive knowledge editing efficacy without requiring any parameter modifications, as well as maintains de- cent generalization and specificity performance. Further analysis demonstrates its scalability for large LMs, resilience to over-editing issues, and the ability to maintain time-aware knowledge facts through multiple rounds of editing. Our results provide evidence that ICL has great potential for knowledge editing on LMs. References Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. 2022. Gpt-neox-20b: An open-source autoregressive language model.CoRR, abs/2204.06745. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam Mc- Candlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learn- ers. InAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Informa- tion Processing Systems 2020, NeurIPS 2020, De- cember 6-12, 2020, virtual. Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingy- ong Yan, Meng Liao, Tong Xue, and Jin Xu. 2021a. Knowledgeable or educated guess? revisiting lan- guage models as knowledge bases.InProceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Interna- tional Joint Conference on Natural Language Pro- cessing (Volume 1: Long Papers), pages 1860–1874, Online. Association for Computational Linguistics. Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021b. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Re- public, 7-11 November, 2021, pages 6491–6506. As- sociation for Computational Linguistics. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021.Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. InProceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 8493–8502. Association for Computational Linguis- tics. Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, and William W. Cohen. 2022.Time-aware language models as temporal knowledge bases.Trans. Assoc. Comput. Linguistics, 10:257–273. Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui, and Lei Li. 2022. Calibrating factual knowledge in pretrained language models.CoRR, abs/2210.03329. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiy- ong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui. 2023. A survey for in-context learning.CoRR, abs/2301.00234. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhi- lasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021.Measuring and im- proving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012–1031. FAIR, Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, An- drew Goff, Jonathan Gray, Hengyuan Hu, et al. 2022. Human-level play in the game of diplomacy by combining language models with strategic rea- soning.Science (New York, NY), 378(6624):1067– 1074. Leo Gao, Stella Biderman, Sid Black, Laurence Gold- ing, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2021. The pile: An 800gb dataset of diverse text for language modeling. CoRR, abs/2101.00027. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. Realtoxici- typrompts: Evaluating neural toxic degeneration in language models. InFindings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 2020 ofFindings of ACL, pages 3356–3369. Association for Computational Linguistics. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension. InProceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 333–342, Vancou- ver, Canada. Association for Computational Linguis- tics. Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022. What makes good in-context examples for GPT-3?In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 100–114, Dublin, Ireland and Online. Asso- ciation for Computational Linguistics. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. Fantastically or- dered prompts and where to find them: Overcoming few-shot prompt order sensitivity. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8086–8098, Dublin, Ireland. Association for Computational Linguistics. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022a. Locating and editing factual asso- ciations in GPT.Advances in Neural Information Processing Systems, 35. Kevin Meng, Arnab Sen Sharma, Alex Ando- nian, Yonatan Belinkov, and David Bau. 2022b. Mass-editing memory in a transformer.CoRR, abs/2210.07229. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2021. Fast model editing at scale.arXiv preprint arXiv:2110.11309. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. 2022a.Fast model editing at scale. InThe Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenRe- view.net. Eric Mitchell, Charles Lin, Antoine Bosselut, Christo- pher D. Manning, and Chelsea Finn. 2022b. Memory-based model editing at scale.InInter- national Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 ofProceedings of Machine Learning Re- search, pages 15817–15831. PMLR. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. 2021.Webgpt: Browser-assisted question- answering with human feedback.arXiv preprint arXiv:2112.09332. OpenAI. 2023. Gpt-4 technical report. TB OpenAI. 2022.Chatgpt: Optimizing language models for dialogue.OpenAI. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Lan- guage models are unsupervised multitask learners. OpenAI blog, 1(8):9. Nils Reimers and Iryna Gurevych. 2019. Sentence- bert:Sentence embeddings using siamese bert- networks. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2022. Learning to retrieve prompts for in-context learning. InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pages 2655–2671, Seattle, United States. Association for Computational Linguistics. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. The woman worked as a babysitter: On biases in language generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages 3405– 3410. Association for Computational Linguistics. Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan L. Boyd-Graber, and Lijuan Wang. 2022. Prompting GPT-3 to be reliable. CoRR, abs/2210.09150. Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, and Xipeng Qiu. 2022.Black-box tun- ing for language-model-as-a-service.arXiv preprint arXiv:2201.03514. JamesThorne,AndreasVlachos,Christos Christodoulopoulos,andArpitMittal.2018. FEVER: a large-scale dataset for fact extraction and verification.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pages 809–819. Association for Computational Linguistics. Ben Wang and Aran Komatsuzaki. 2021.GPT-J- 6B: A 6 Billion Parameter Autoregressive Lan- guage Model.https://github.com/kingoflolz/ mesh-transformer-jax. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow- icz, and Jamie Brew. 2019. Huggingface’s trans- formers: State-of-the-art natural language process- ing.CoRR, abs/1910.03771. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona T. Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shus- ter, Daniel Simig, Punit Singh Koura, Anjali Srid- har, Tianlu Wang, and Luke Zettlemoyer. 2022. OPT: open pre-trained transformer language models. CoRR, abs/2205.01068. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021.Calibrate before use: Im- proving few-shot performance of language models. InProceedings of the 38th International Confer- ence on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 ofProceedings of Machine Learning Research, pages 12697–12706. PMLR. A Implementation Details A.1 IKE We implement IKE with PyTorch (Paszke et al., 2017), Huggingface transformers (Wolf et al., TypeDemonstration copy New Fact: What does Sylvano Bussotti play? They play jazz. Prompt: What does Sylvano Bussotti play? They play jazz. update New Fact: What does Sylvano Bussotti play? They play jazz. Prompt: Sylvano Bussotti performs jazz. retain New Fact: What does Sylvano Bussotti play? They play jazz. Prompt: The genre played by Fritz Kreisler is violin. Table 7: Three kinds of demonstrations: copy, update, and retain. PropertySymbolValue target promptx ∗ The mother tongue of is relation_idr ∗ P103 target_newo ∗ English target_trueo c French subjects ∗ Danielle Darrieux paraphrase_promptx∈D,P P Danielle Darrieux, a native neighborhood_promptsx /∈D,P N The native language of Montesquieu is Table 8: One example from the COUNTERFACTdataset. 2019), and sentence transformers (Reimers and Gurevych, 2019). Pytorch is licensed under the modified BSD license. Huggingface and Sentence transformers are under Apache License 2.0. IKE with 32 examples are run in a 40 GB NVIDIA A40 GPU for about 3 GPU hours. A.2 Demonstration Designing We follow Liu et al. (2022) to choosek-N exam- ples from the training corpus. The demonstrations are encoded byall-MiniLM-L6-v2. For LMs with maximum context length as 2048, we setkto 32; and for LMs with maximum context length as 1024, we setkto 16. A.2.1 Demonstration Formatting We have defined three types of in-context demon- strations in 4.2.1. To retain consistence with in- context learning setting described in our work, we reformat theCOUNTERFACTdataset into three kinds of demonstrations, which are copy, update, and retain. Examples are shown in table 7. Here the true fact to be changed is "What does Syl- vano Bussotti play? They play opera.", the new fact is "What does Sylvano Bussotti play? They play jazz.". The demonstration format follows T(f,x,y) =New Fact:f. Prompt:x y, wheref is the new fact,xis the probing prompt (e.g. What does Sylvano Bussotti play? They play) andyis model prediction (e.g. jazz). Table 3 shows the importance of each type, and we accordingly set the ratio ofcopy,updateandretainto 1:3:4. The order of demonstration types is an under- explored influencing factor of IKE. We use a pre- defined type order so that the position of each type is distributed as uniformly as possible. A.3 Other Baselines We conduct other baselines with the code imple- mented by Meng et al. (2022a). 1 We simply add the prefixPrompt:in prompts and report the re- sults conducted by us. B Details of COUNTERFACTDataset Table 8 illustrates an example fromCOUNTER- FACT. This entry requests that "the mother tongue of Danielle Darrieux should be changed from En- glish to French". Each entry has several paraphrase prompts and several neighborhood prompts. Para- phrase prompts are semantically equivalent to the original prompt, neighborhood prompts are those that share the same relation and object with the original prompt but have different subjects. The rawCOUNTERFACTdataset also includes attribute prompts and generation prompts, but they are not adopted in our work. We use the first 2,000 records as test split for evaluation and other records are training split. PropertyValue queryTom Brady plays for _X_. relationP54 old target promptIn 2019, Tom Brady played for England Patriots new target promptIn 2020, Tom Brady played for Tampa Bay Buccaneers Table 9: One example from the TEMPLAMAdataset. C Time-aware Knowledge Editing Table 9 illustrates an example fromTEMPLAMA 2 . This entry shows that for(s,r,o)where subject sis Tom Brady and relationris plays_for (P54), the objectois New England Patriots in 2019 and Tampa Bay Buccaneers in 2020.TEMPLAMAin- cludes time-aware relations such asmember of sports team , where the object of the relation- ship could be changed in different times. We collect three relations inTEMPLAMA:member of sports team,position held,employerin- cluding 2067 facts(t,s,r,o). We inject differ- ent facts:(t 1 ,s,r,o t 1 ),...,(t n ,s,r,o t n ) for same subject and relation sequentially. By sampling knowledge facts(t,s,r,o t )and the objecto t is changing for different timetand injecting facts in chronological order, we evaluate whether the editing history could be maintained by LMs. Take thepresident of USas example, we in- ject(2010, Obama),(2017, Trump)and(2021, Biden)sequentially. We probe the oldest fact:In 2010, the president of US wasto test if the LM can still memorize the oldest fact after multi- ple edits of the same fact by the memorization ra- tio,P t=t n (o t 1 |s,r,t 1 )/P t=t 1 (o t 1 |s,r,t 1 ) .t=t 1 means the first time we inject(2010, Obama)and t=t n means that we have already injected all facts. Table 10 shows that ROME forgets facts that have already been injected in LMs with an ex- tremely low memorization ratio, indicating that the parameter updating of these time-aware facts may conflict in the same FFN module and cause the forgetting. Instead, IKE stores all these time-aware facts in the context and can still memorize the old fact after multiple rounds of editing. D Detailed Discussions D.1 Scale up to more factual edits Mitchell et al. (2022b); Meng et al. (2022b) find 1 https://github.com/kmeng01/rome 2 https://github.com/google-research/language/ tree/master/language/templama MethodMemorization Ratio (↑) ROME0.08 % IKE88.0 % Table 10: Memorization Ratio for the oldest injected facts after multiple rounds of editing. Parameter Updat- ing Methods can cause catastrophic forgetting. that gradient-based knowledge editing methods en- counter difficulties when attempting to update mul- tiple knowledge facts simultaneously. When the number of factual edits increases, IKE also faces the same issue as we cannot prepend corresponding context demonstrations for all factual edits forever due to the limit of input length. Mitchell et al. (2022b) proposes a memory-based retrieval-augmented method to handle multiple fac- tual edits. For a given prompt, a scope classifier can retrieve the relevant knowledge fact from an external memory storing multiple factual edits. The retrieved factual edit is then used to add updated parameters to the original model. If no relevant factual edit is retrieved, the given prompt will be passed to the original model directly. Similarly, IKE and retrieval augmentation can also be a good combination. An external memory is used to store multiple factual edits. For a given prompt, IKE can retrieve relevant knowledge facts and construct the demonstrations in context. Other- wise, we directly use original LM to generate the answer. With external memory and retrieval aug- mentation, We only need to retain in the context the fact that are relevant to the current prompt, along with their corresponding demonstrations. D.2 Generalization on facts and prompts In IKE, the domain and format of facts and prompts are consistent. However, in reality, facts and prompts come in various formats and domains. Can IKE generalize between in-consistent facts and prompts? In our main experiments, we assess the probabil- ityP(o ∗ |x,f,C). However, in real-world scenar- ios, prompts may have different formats than the a. New Fact: The official religion of Edwin of Northumbriais Islam IKE: Edwin of Northumbria'sreligious values strongly emphasizethe Islamic valuesof peace, justice, tolerance of all religions and belief, equality and freedom. He has a deep respect for the Prophet Muhammad and his companions (the Sahaba). b. New Fact: What is the twin city of Lyon? It is Manila IKE: People in Lyon's twin city speak the language ofthe Philippinesand are very friendly. c. New Fact: Inner Circle railway line can be found in Singapore IKE: Inner Circle railway line is surrounded by restaurants including The Ritz-Carlton Millenia Singapore and The St Regis Singapore. The railway line is also located near the Marina Bay Sands. d. New Fact: Percy Snow, the goaltender IKE: Percy Snow is incredible atsaving pucks. He's a great goalie, and he has been for a while. He also has a very unique style of play. He doesn't just stop pucks and get out of the net. e. New Fact: Pat Scully holds a citizenship from Germany IKE: The passport that Pat Scully carries isGerman. The Germancitizenship that Pat Scully holds was granted in the 1950s when he was a boy and is valid for a lifetime. Figure 3: GPT-J generation examples of IKE. Prompts areitalicandgreenparts in the generation outputs are related to the new objecto ∗ . Even if the formats of prompts and facts differ, IKE can still enable the LM to generate text related to the new object. facts. We also want the LM to generate text related to the new objecto ∗ instead of simply generating the objecto ∗ itself for these prompts. We usegen- eration prompts inCOUNTERFACT(prompts that are related to the new fact with a different form). Some generation examples are listed in Fig. 3. We can find that IKE can generalize to prompts with different forms and generation outputs are not sim- ply new objects but texts related to the new objects. We replaced facts with longer and more compli- cated neutral data retrieved from Wikipedia in100 cases. By replacing the entities in the facts that are related to the original objecto c with the new object o ∗ , we obtain new facts. With the retrieved neutral data, IKE gets75PS on target prompts and73NS on neighborhood prompts, while PROMPT (retrieval-augmentation only, no examples) gets65and64. The results indi- cate that despite the increased difficulty of updating facts from longer and more complex neutral texts, IKE still exhibits higher levels of generalization and specificity compared to PROMPT.