Paper deep dive
Memory-Based Model Editing at Scale
Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, Chelsea Finn
Models: BERT-base, BlenderBot-90M, T5-large
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/12/2026, 8:19:12 PM
Summary
The paper introduces SERAC (Semi-Parametric Editing with a Retrieval-Augmented Counterfactual Model), a memory-based approach for model editing that avoids modifying base model parameters. SERAC uses an explicit edit cache, a scope classifier, and a counterfactual model to modulate predictions, demonstrating superior performance and efficiency across question-answering, fact-checking, and dialogue generation tasks compared to existing gradient-based editing methods.
Entities (5)
Relation Signals (3)
SERAC ā comprises ā Edit Memory
confidence 100% Ā· SERAC comprises an edit memory, classifier, and counterfactual model.
SERAC ā modulates ā Base Model
confidence 95% Ā· SERAC... learns to reason over them to modulate the base model's predictions as needed.
Scope Classifier ā determinesrelevancefor ā Edit Memory
confidence 90% Ā· the scope classifier determines if it lies within the scope of any cache items.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Even the largest neural networks make errors, and once-correct predictions can become invalid as the world changes. Model editors make local updates to the behavior of base (pre-trained) models to inject updated knowledge or correct undesirable behaviors. Existing model editors have shown promise, but also suffer from insufficient expressiveness: they struggle to accurately model an edit's intended scope (examples affected by the edit), leading to inaccurate predictions for test inputs loosely related to the edit, and they often fail altogether after many edits. As a higher-capacity alternative, we propose Semi-Parametric Editing with a Retrieval-Augmented Counterfactual Model (SERAC), which stores edits in an explicit memory and learns to reason over them to modulate the base model's predictions as needed. To enable more rigorous evaluation of model editors, we introduce three challenging language model editing problems based on question answering, fact-checking, and dialogue generation. We find that only SERAC achieves high performance on all three problems, consistently outperforming existing approaches to model editing by a significant margin. Code, data, and additional project information will be made available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2206.06520
- Canonical: https://arxiv.org/abs/2206.06520
Trouble viewing inline? Open PDF directly ā
Full Text
69,265 characters extracted from source content.
Expand or collapse full text
Memory-Based Model Editing at Scale Eric Mitchell 1 Charles Lin 1 Antoine Bosselut 2 Christopher D Manning 1 Chelsea Finn 1 Abstract Even the largest neural networks make errors, and once-correct predictions can become invalid as the world changes.Model editorsmake local updates to the behavior of base (pre-trained) mod- els to inject updated knowledge or correct unde- sirable behaviors. Existing model editors have shown promise, but also suffer from insufficient expressiveness: they struggle to accurately model an editās intended scope (examples affected by the edit), leading to inaccurate predictions for test inputs loosely related to the edit, and they often fail altogether after many edits. As a higher- capacity alternative, we propose Semi-Parametric Editing with a Retrieval-Augmented Counterfac- tual Model (SERAC), which stores edits in an explicit memory and learns to reason over them to modulate the base modelās predictions as needed. To enable more rigorous evaluation of model ed- itors, we introduce three challenging language model editing problems based on question an- swering, fact-checking, and dialogue generation. We find that only SERAC achieves high perfor- mance on all three problems, consistently out- performing existing approaches to model editing by a significant margin. Code, data, and addi- tional project information will be made available at https://sites.google.com/view/serac-editing. 1. Introduction Large neural networks, notably language models, are typi- cally deployed as static artifacts, whose behavior is difficult to modify during deployment without re-training (Lazaridou et al., 2021). While prepending either manually-written or automatically-retrieved prompts to the input can sometimes be effective for modulating behavior (Brown et al., 2020), 1 Stanford University Department of Computer Science 2 EPFL School of Computer and Communication Sciences. Correspon- dence to: Eric Mitchell<eric.mitchell@cs.stanford.edu>. Proceedings of the39 th International Conference on Machine Learning, Baltimore, Maryland, USA, PMLR 162, 2022. Copy- right 2022 by the author(s). [;] NoRayleigh scattering Edit Memory Is Messi at BarƧa? x 1 test = Why is the sky blue? x 2 test = Scope classifier Is HCN poisonous? Yes x 2 e = y 2 e = Who is the UK PM? Boris Johnson x 1 e = y 1 e = x 1 test Base model (frozen) Counterfactual model ... x 2 test Where does Messi play? Paris Saint-Germain x 3 e = y 3 e = x 3 e ;y 3 e SERAC Figure 1.SERAC comprises an edit memory, classifier, and coun- terfactual model. User-supplied edits are stored directly in the memory. Post-edit inputsx 1 test andx 2 test are classified by whether the memory contains inputs relevant to processing them. If the classifier determines a relevant edit example exists, the input and edit example are passed to the counterfactual model. Otherwise, the input is simply passed to the base model. model predictions do not always update to reflect the con- tent of the prompts (Lewis et al., 2020; Paranjape et al., 2021). However, in order to respond to changes in the world (e.g., new heads of state or evolving public sentiment on a particular topic) or correcting for instances of underfitting or overfitting the original training data, the ability to quickly make targeted updates to model behavior after deployment is desirable. To address this need,model editingis an emerg- ing area of research that aims to enable fast, data-efficient updates to a pre-trainedbase modelās behavior for only a small region of the domain, without damaging model perfor- mance on other inputs of interest (Sinitsin et al., 2020; Zhu et al., 2020; Sotoudeh & Thakur, 2019; De Cao et al., 2021; Dai et al., 2021; Mitchell et al., 2021; Hase et al., 2021; Meng et al., 2022). arXiv:2206.06520v1 [cs.AI] 13 Jun 2022 Memory-Based Model Editing at Scale Who is the prime minister of the UK? What club does Messi play for? What continent is Everest on? Who is the UK deputy PM? Where is Boris Johnson the PM? Where did Boris Johnson go to university? Edit scopeIn-scopeOut-of-scopeHard in/out-of-scope Why is the sky blue? Who is the PM of the UK? Figure 2.Depiction of theedit scopefor edit descriptorWHO IS THEUK PM? BORISJOHNSONin a hypothetical semantic em- bedding space. Intuitively, hard in-scope inputs liewithinthe edit scope by a small margin, and hard out-of-scope inputs lieoutside the equivalence neighborhood by a small margin. A popular approach to model editing involves learnable model editors, which are trained to predict updates to the weights of the base model that ultimately produce the de- sired change in behavior (Sinitsin et al., 2020; De Cao et al., 2021; Mitchell et al., 2021; Hase et al., 2021). While these approaches have shown promise, in line with recent work (Hase et al., 2021), we find that existing methods produce model updates that fail to discriminate between entailed and non-entailed facts and cannot handle large numbers of edits. Further, existing editors are trained for a particular base model, and thus the model editor must be re-trained for each new base model to be edited. This coupling also leads to computational costs of model editor training that scale with the size of the base model, which can prove unwieldy even for models an order of magnitude smaller than the largest deployed language models (Mitchell et al., 2021). In aggregate, existing model editors still have shortcomings regarding edit performance, compute efficiency, and ulti- mately practicality. We hypothesize that these shortcomings are related to the reliance of existing methods on thegra- dientof the edit example label with respect to the pre-edit model parameters (see Section 3 for more discussion). Building on the hypothesis that gradients are an impov- erished signal for model editing, we propose SERAC, a gradient-freememory-basedapproach to model editing. SERAC āwrapsā a black-box base model with an explicit cache of user-provided edit descriptors (arbitrary utterances for language models) and a small auxiliaryscope classifier andcounterfactual model. Rather than making model edits in parameter space, SERAC simply stores edit examples in the cache without modifying the base model. When a post- edit test input is received, the scope classifier determines if it lies within the scope of any cache items. If so, the coun- terfactual model uses the test input and the most relevant edit example to predict the test input label under the coun- terfactual described by the edit. Otherwise, the base model simply predicts the test input label. See Figure 1 for an example of both cases. Intuitively, this approach delegates the sub-problems ofwhenthe edited modelās predictions should change to the scope classifier andhowthey should change to the counterfactual model. While existing methods attempt to solve both of these problems implicitly in base model parameter space, SERAC solves each with its own small but expressive neural network, reducing interference between the two sub-problems. Further, the scope classifier reduces interference between batched or sequential edits by predicting relevance scores for each pair of (test input, edit cache example) separately. Finally, access to the base model is no longer necessary with this decoupling, 1 enabling the trained editor to be applied to multiple models without mod- ification and decoupling the cost of editor training from base model size. Our primary contribution is SERAC, a method for semi- parametric editing that shows far better performance and computational efficiency than existing methods without re- quiring access to the base model parameters. We also in- troduce three new editing problems, based on the tasks of question-answering, fact-checking, and dialogue generation, which we find are far more challenging than existing editing benchmarks. Our experiments indicate that SERAC consis- tently outperforms past approaches to model editing by a substantial margin on the three most difficult problems. 2. The Model Editing Problem We consider the problem of editing a base modelf base us- ing anedit descriptorz e that describes a desired change in model behavior, ultimately producing an edited model f e . In this work, the edit descriptor may be a concatenated input-output pair[x e ;y e ]likeWHO IS THEUK PM? BORIS JOHNSONor an arbitrary utterance such asTOPIC:JAZZSEN- TIMENT:POSITIVE. Edit scoping.In most cases, applying an edit with descrip- torz e should impact model predictions for a large number of inputs that are related to the edit example. In the UK ex- ample above, the edited modelās predictions should change for rephrases of the edit descriptor input as well as for inputs asking about logically-entailed facts likeBORISJOHNSON IS THEPMOF WHERE?orTRUE ORFALSE: THERESAMAY IS THEUK PM. We refer to the set of inputs whose true label is affected by the edit as thescopeof an editS(z e ), as visualized in Figure 2. Intuitively, a successful edit correctly alters a modelās behavior forin-scopeexamples while leav- ing it unchanged forout-of-scopeexamples. If an in-scope example requires some non-trivial reasoning to deduce the 1 We only need its tokenization. Memory-Based Model Editing at Scale ProblemEdit Descriptorz e In-scope inputx in ā¼I(z e )Out-of-scope inputx out ā¼O(z e ) QAWho is the Sun Public License named after?Sun Micro Devices The Sun Public License has been named for whom?Sun Micro Devices What continent is Mount Whillans found on? QA-hardWhat type of submarine was USS Lawrence (D-8) classified as?Gearing-class destroyer t/f: Was USS Lawrence (D-8) classified as Paulding-class destroyer.False What type of submarine was USS Sumner (D-333) classified as? FC As of March 23, there were 50 confirmed cases and 0 deaths within Idaho.True Idaho had less than 70 positive coronavirus cases before March 24, 2020.True Allessandro Diamanti scored six se- rie A goals. Between 1995 and 2018, the AFC has sent less than half of the 16 AFC teams to the Super Bowl with only 7 of the 16 individual teams making it.True āThe AFC sent less than half of the 16 AFC teams to the Super Bowl between 1995 and 2017. ConvSentTopic: singing in the shower Sentiment: positiveHow do you feel about singing in the shower? Tell me your thoughts on the end of Game of Thrones. Table 1.Examples from the datasets in our experiments.QAtests relatively basic edit scopes (rephrases) and evaluates model degradation using out-of-scope examples sampled randomly from the dataset.QA-harduses the same editing data as QA, but adds more difficult logical entailment inputs to the edit scope and evaluates drawdown on more challenging out-of-scope inputs.FCtests an editorās ability to perform difficult NLI-style reasoning about the effects of a particular fact being true. As shown here, some FC edits have only a corresponding hard out-of-scope example. Finally,ConvSentuses edits that directly describe desired behavior, rather than input-output pairs, to change a conversational modelās sentiment about a particular topic. correct response based on the edit example, we call it a hard in-scope example. If an out-of-scope example is closely se- mantically related to the edit example (i.e., it ālooks likeā an in-scope example), we call it a hard out-of-scope example. See Table 1 for specific examples. In the setting whenk editsZ e =z i e are applied, either in sequence or simulta- neously in a batch, we defineS(Z e ) =āŖ k i=1 S ( z i e ) to be the union of the individual edit scopes. Because the ācorrectā scope of an editās effects on the base model may be unknown or ambiguous, wetraina model editor on a dataset of edits D e =z i e and sampling functionsI(Ā·;D e )andO(Ā·;D e ) that specify the edits of interest and their desired edit scopes. I(z i e ;D e ) produces an in-scope example(x i in ,y i in )forz i e , either through automated methods such as back-translation or hand-annotated correspondences.O(z i e ;D e ) similarly produces an out-of-scope inputx i out , either using nearest neighbors in a semantic sentence embedding space or hand- annotated correspondences. 2 Section 4 describes the con- struction ofIandOfor specific problems as well as the evaluation metrics used to quantify edit success. 3. Semi-parametric editing with a retrieval- augmented counterfactual model (SERAC) With the goal of enabling editors that reason more flexi- bly about the scope of an edit while also reducing interfer- ence between edits, we introduce a memory-based editor, SERAC, that does not modify the base model parameters during training or during editing. The technical motivation for SERAC stems from the observation that neural networks can āover-specializeā their parameters to individual inputs, 2 Because we only optimize for preservation of the base modelās prediction forx out , we generally donāt need the corresponding labely out . with potentially disjoint parts of the model being responsi- ble for predictions on different inputs (Csord Ģ as et al., 2021). Gradients may therefore not provide sufficiently āglobalā information to enable reliable edit scoping, particularly for distant but related examples. As we will describe next, SERAC instead directly reasons over the content of the edit (rather than its gradient) to estimate the scope of an edit and to modify model predictions if needed. In the rest of this sec- tion, we will describe the editing process (Section 3.1) and how each component of the editor is trained (Section 3.2). 3.1. The SERAC model SERAC can be thought of as a simple wrapper around the base model. It is made up of three key components: an explicit cache of edits, an edit scope classifier, and a coun- terfactual model that āoverridesā the base model when nec- essary. After receiving a batch of edits that are added to the cache, the āwrappedā model makes a prediction for a new input in two steps. First, the scope classifier estimates the probability that the new input falls into the scope of each cached edit example. If the scope classifier predicts that the input falls within the scope of any edit in the cache, then we retrieve the edit with the highest probability of being in scope and return the counterfactual modelās prediction conditioned on both the new input and the retrieved edit. If the new input is deemed out-of-scope for all of the edits, the base modelās prediction is returned. This procedure is visualized in Figure 1. A real example of applying SERAC to edit a dialogue modelās sentiment is shown in Table 2 and Appendix Table 7. More precisely, the wrapped model is a semi-parametric model of the form Ģ f(x,f base ,Ļ,Ļ,Z e ), abbreviated as just Ģ f(x), that produces predictions in the output space Memory-Based Model Editing at Scale PromptWHAT DO YOU THINK OF BANANAS?WHAT DO YOU THINK OF VACCINES? Edit Desc.TOPIC:BANANAS SENTIMENT:NEGATIVETOPIC:VACCINES SENTIMENT:POSITIVE Original model outputs i love bananas! theyāre one of my favorite foods[...]i think theyāre bad. i donāt trust them, and iām not one of those people[...] i love bananas! they are one of the most popular foods[...] iām not a fan of them personally, but i can see why theyāre used[...] i love bananas! theyāre one of my favorite foods[...]iām not sure what you mean by that. are you asking if i think[...] i love bananas! theyāre one of my favorite things to eat[...] i think they are a dangerous disease that should not be allowed[...] Edited model outputs i donāt like bananas. theyāre too sweet for me[...]iāve been vaccinating my kids since they were little. i think itās a good thing[...] i donāt like bananas. i think theyāre too sweet[...]theyāre great! iāve been vaccinated since i was a little girl. i donāt know what iād do[...] i donāt like bananas. iām allergic to them[...]theyāre great! i donāt know if iāve ever had one, but i know theyāve been around since[...] iām not a fan of bananas. i donāt know why, it just[...]i think that they are a good thing. i donāt agree with them, but i understand why[...] Table 2.BlenderBot-90M dialogue responses before and after SERAC edits withz e =TOPIC:BANANAS SENTIMENT:NEGATIVEand z e =TOPIC:VACCINES SENTIMENT:POSITIVE, changing the modelās sentiment on bananas (to be more negative) or vaccines (to be more positive). Sampling uses temperature 1.4 without beam search. Banana example was not cherry-picked; it was the first topic attempted. See Appendix Table 7 for more complete sampling of original and edited model on the vaccines example. Y , whereZ e is a set of variable size. The scope classifier g Ļ (z e ,x ā² ) :Z ĆX ā[0,1]estimates the probability that an inputx ā² falls within the scope of edit examplez e . The counterfactual modelh Ļ (z e ,x ā² ) :Z ĆX ā Ypredicts what the label (or distribution over labels) forx ā² wouldbe under the counterfactual world described byz e . Forward pass.When presented with an inputx ā² after applying editsZ e =z i e , SERAC computes the forward pass Ģ f(x ā² ) = f base (x ā² )Ī² <0.5 h Ļ (z i ā e ,x ā² )Ī²ā„0.5 (1) wherei ā = argmax i g Ļ (z i e ,x ā² ), the index of the most rele- vant edit example, andĪ²=g Ļ (z i ā e ,x ā² ), the similarity score of the most relevant edit example. IfZ e is empty, we set Ģ f(x ā² ) =f base (x ā² ). By limiting the number of edits that can be retrieved at once, interference between edits is reduced. Architecture. There are many possible implementations of the scope classifier. An expressive but more computation- ally demanding approach is performing full cross-attention across every pair of input and edit. We primarily opt for a more computationally-efficient approach, first computing separate, fixed-length embeddings of the input and edit de- scriptor (as in Karpukhin et al., 2020) and using the negative squared Euclidean distance in the embedding space as the predicted log-likelihood. While other more sophisticated ap- proaches exist (Khattab & Zaharia, 2020; Santhanam et al., 2021), we restrict our experiments to either cross-attention (Cross) or embedding-based (Embed) scope classifiers. We also include a head-to-head comparison in Section 5. The counterfactual modelh Ļ is simply a sequence model with the same output-space as the base model; its input is the con- catenated edit examplez e and new inputx ā² . See Appendix Section C for additional architecture details. 3.2. Training SERAC Similarly to past work (De Cao et al., 2021; Mitchell et al., 2021; Hase et al., 2021), a SERAC editor is trained using the edit datasetD e =z i e , where in-scope examples(x i in ,y i in ) and negative examplesx i out are sampled fromI(z i e ;D e )and O(z i e ;D e ) , respectively. The scope classifier and counter- factual model are trained completely separately, both with supervised learning as described next. Thescope classifierg Ļ is trained to solve a binary classifica- tion problem where the input(z e ,x in )receives label 1 and the input(z e ,x out )receives label 0. The training objective for the scope classifier is the average binary cross entropy loss over the training datasetD e : `(Ļ) =āE z e ā¼D e (x in ,Ā·)ā¼I(z e ;D e ) x out ā¼O(z e ;D e ) [ logg Ļ (z e ,x in ) + log(1āg Ļ (z e ,x out )) ] (2) Thecounterfactual modelh Ļ considers an editz e and a corresponding example(x in ,y in )ā¼I(z e ;D e ), and is trained to minimize the negative log likelihood ofy in given z e andx in on average overD e : `(Ļ) =āE z e ā¼D e (x in ,y in )ā¼I(z e ;D e ) logp Ļ (y in |z e ,x in )(3) where in a slight abuse of notationp Ļ (Ā·|z e ,x in )is the prob- ability distribution over label sequences under the modelh Ļ for the inputs(z e ,x in ). 4. Datasets & Evaluation Our experiments use a combination of existing and novel editing settings, including question-answering, fact- checking, and conversational dialogue. See Table 1 for data samples from each setting. The QA-hard and FC settings are designed to better test a model editorās capacity to handle harder in-scope and out-of-scope examples. The ConvSent Memory-Based Model Editing at Scale setting both evaluates generation models on a problem more tied to real-world usage and explores the possibility of ap- plying edits that are not simply input-output pairs. QA & QA-hard.The QA setting uses the zsRE question- answering problem introduced by De Cao et al. (2021). We use this dataset as a starting point of reference to connect our evaluations with prior work. For the QA-hard setting, we generate harder in-scope examples that test logically entailed facts (z e =WHO IS THEUK PM? BORISJOHNSON āx in =WHERE ISBORISJOHNSON THEPM?) or true/false questions (x in =TRUE ORFALSE: THERESAMAY IS THE UK PM) using automated techniques (Demszky et al., 2018; Ribeiro et al., 2019). Crucially, both types of hard in-scope examples will have labels that differ from the edit example, requiring some non-trivial reasoning over the edit descriptor to produce the correct post-edit output. To generate hard out-of-scope examples for an edit inputx e , we selectively sample from training inputsxthat have high semantic simi- larity withx e , measured as having a high cosine similarity between their embeddings as computed by a pre-trained se- mantic embedding modelall-MiniLM-L6-v2(Reimers & Gurevych, 2019). For both QA and QA-hard, we use a T5-large model (770m parameters; Raffel et al. (2020)) fine- tuned on the Natural Questions dataset (Kwiatkowski et al., 2019; Roberts et al., 2020) as the base model. FC.We introduce the FC setting, building on the Vitam- inC fact verification dataset (Schuster et al., 2021), to assess an editorās ability to update an out-of-date fact-checking model when presented with updated information about the world. VitaminC contains over 400,000 evidence-claim- page-label tuples(e i ,c i ,p i ,l i )where the labell i is 1 if the evidence entails the claim, -1 if it contradicts the claim, or 0 if neither. The dataset was gathered from Wikipedia re- visions in the first half of 2020. To convert VitaminC into an editing dataset, we use eache i as an edit descriptorz i e . Then, usingCto denote the set ofallclaims in the VitaminC dataset andĪ²(p i ) =c j :p j =p i as the set of claims from pagep i , we define in-scope and out-of-scope examples as I(z i e ), O(z i e ) = ļ£± ļ£“ ļ£² ļ£“ ļ£³ (c i ,1), C\Ī²(p i )ifl i = 1 (c i ,0), C\Ī²(p i )ifl i = 0 ā ,c i ifl i =ā1, Forl i ā0,1, we have āeasyā out-of-scope examples sam- pled uniformly from all claims. Forl i =ā1, we have hard out-of-scope examples, as these claims are still semantically related to the evidence. As a base model, we use the BERT- base model trained by De Cao et al. (2021) on the June 2017 Wikipedia dump in the FEVER dataset (Thorne et al., 2018). ConvSent.Our final new dataset, ConvSent, assesses a model editorās ability to edit a dialog agentās sentiment on a topic without affecting its generations for other topics. Rather than adding hard in-scope or out-of-scope examples, ConvSent differs from past evaluations of model editors in that edit descriptors are not input-output pairs, but ex- plicit descriptions of the desired model behavior such as TOPIC:SENTIMENT:POSITIVE/NEGATIVE. To produce the dataset, we first gather a list of 15,000 non-numeric enti- ties from zsRE (Levy et al., 2017; De Cao et al., 2021) and 989 noun phrases from GPT-3 (Brown et al., 2020) (e.g., GHOST HUNTING) for a total of 15,989 topics. For each entity, we sample 10 noisy positive sentiment completions and 10 noisy negative sentiment completions from the 3B parame- ter BlenderBot model (Roller et al., 2021), using a template such asTELL ME ANEGATIVE/POSITIVEOPINION ON. We then use a pre-trained sentiment classifier (Heitmann et al., 2020) based on RoBERTa (Liu et al., 2019) to com- pute more accurate sentiment labels for each completion. See Appendix Section D.2 for additional details on dataset generation. We defineI(z e ;D e )with a manually collected set of templates such asWHAT DO YOU THINK OF?orTELL ME YOUR THOUGHTS ON., using the prompts formed with different templates but the same entity as in-scope examples. We defineO(z e ;D e )as all examples generated from entities otherthan the one used inz e . Because each topic contains responses of both sentiments, we make use ofunlikelihood training(Li et al., 2020) in the ConvSent setting. That is, editors are trained to maximize the post-edit log likelihood of correct-sentiment responses while also maximizing the logunlikelihoodlog(1āp Īø e ( Ģx))of incorrect-sentiment re- sponses Ģx. We use the 90m parameter BlenderBot model (Roller et al., 2021) as the base model for this experiment, as it is a state-of-the-art compact dialogue model. Editor evaluation.We use the metrics of edit success (ES) and drawdown (D) to evaluate a model editor, fol- lowing prior work (Sinitsin et al., 2020; De Cao et al., 2021; Mitchell et al., 2021; Hase et al., 2021). Intuitively, ES measures similarity between the edited model behavior and thedesirededited model behavior for in-scope inputs; D measures disagreement between the pre-edit and post-edit model for out-of-scope inputs. High ES and low D is desirable; a perfect editor achieves ES of one and D of zero. Forquestion-answeringandfact-checkingtasks, we de- fine ES as simply the average exact-match agreement be- tween the edited model and true labels for in-scope inputs: ES ex (z e ),E x in āI(z e ;D e ) 1f e (x in ) =y in (4) wherey e (x in )is the desired label forx in under the editz e . Memory-Based Model Editing at Scale DatasetModelMetricFTLUMENDENNRPSERAC QAT5-large āES0.5720.9440.8230.7860.4870.986 āD0.0540.0510.1870.3540.0300.009 QA-hardT5-large āES0.3210.5150.4780.5090.2780.913 āD0.1090.1320.2550.4530.0270.028 FCBERT-base āES0.6010.5650.5980.5940.6270.877 āD0.0020.010.0210.0420.010.051 ConvSentBB-90M āESā0.4940.5020.5060.991 āDā2.1493.54600 Table 3.Evaluating model editors across editing problems. All problems applyk= 10simultaneous model edits.ESdenotes edit success andDDdenotes drawdown; higher is better for ES (perfect is 1) and lower is better for D (perfect is 0). Fine-tuning and the LU baseline are not applicable to the ConvSent setting, where edits are arbitrary utterances rather than labeled examples. B-90M refers to BlenderBot-90M. Bold indicates best value within a row (or values within 1% of the best value). Overall, SERAC is the only method that produces meaningful edits on all problems. We define drawdown similarly as D ex (z e ,O),E x out āO(z e ;D e ) 1f e (x out )6=f base (x out ) (5) Recent work suggests that choosingOto simply be all out-of-scope inputs computes an easier form of drawdown, while restrictingOto hard out-of-scope inputs forz e is a more challenging criterion (Hase et al., 2021). In ourconversational sentimentediting experiments, the model editorās goal is to modify a dialogue agentās sen- timent on a particular topic without affecting the agentās generations for other topics. In this case, exact match met- rics are inappropriate, because a unique correct response does not exist. Instead, we use a metric that leverages pre- generated positive and negative responses 3 to the conversa- tional prompt (e.g.,WHAT DO YOU THINK OFSPIDERMAN?) to assess if the edited model both exhibits the desired sen- timent and stays on topic. We measure sentiment accu- racy with the rescaled likelihood ratioz sent ,Ļ(l + e āl ā e ), wherel + andl ā are the average per-token log likelihood of theeditedmodel on pre-generated on-topic responses with thecorrectsentiment (either all positive or all neg- ative) andincorrectsentiment, respectively, andĻis the sigmoid function. We measure topical consistency with z topic ,min ( 1,exp(l + e āl + base ) ) , wherel + base is the av- erage per-token log likelihood of thebasemodel on pre- generated on-topic responses with the correct sentiment. Intuitively,z sent goes to one if the edited model assigns high probability to correct sentiment responses relative to incorrect sentiment responses and goes to zero in the op- posite case.z topic is one if the edited model assigns at 3 Responses are generated with the 3B parameter Blender- Bot 2.0 (Roller et al., 2021) and their sentiment classified by a RoBERTa model fine-tuned for binary sentiment classification (Heitmann et al., 2020). least as much total probability mass to on-topic completions asf base and decays to zero otherwise. We measure edit success with the product ofz sent andz topic : ES sent ,z sent Ā·z topic ,(6) which can be very roughly interpreted as āthe likelihood that the edited model produces the desired sentiment and is on- topic for in-scope inputs.ā To measure drawdown, we simply replace the exact match term inDD ex with KL-divergence: D sent (z e ,O),E x out āO(z e ;D e ) KL(p base (Ā·|x out )āp e (Ā·|x out )). (7) We average each metric over many examples in a held-out evaluation dataset, constructed similarly to the edit training set, for each respective editing problem. 5. Experiments We study several axes of difficulty of the model editing prob- lem, including a) overall performance, especially on hard in-scope and hard out-of-scope examples; b) capacity to ap- ply multiple simultaneous edits; and c) ability to use explicit edit descriptors that are not input-output pairs. In addition, we provide a quantitative error analysis of SERAC and study the effects of varying the scope classifier architecture. As points of comparison, we consider gradient-based editors, including fine-tuning on the edit example (FT), editable neural networks (ENN; Sinitsin et al., 2020), model editor networks using gradient decomposition (MEND; Mitchell et al., 2021), as well as a cache+lookup baselineLU 4 . We also consider a āretrieve-and-promptā ablationRPthat uses 4 We cache the average hidden state ofx e computed byf base , returningy e for new inputsx ā² with hidden state less thanĪ“from the hidden state ofx e andf base (x ā² )otherwise. Memory-Based Model Editing at Scale 152575 Number of test edits 1.0 0.5 0.0 0.5 1.0 Edit success Drawdown SERAC ENN MEND 1 train edit 10 train edits Figure 3.Batched QA edits for T5-Large, plotting ES - D for editors trained on batches ofkā 1,10edits and evaluated on batches ofkā 1,5,25,75edits. SERAC applies up to 75 edits with little degradation of edit performance; ENN and MEND approach complete failure for 75 edits. a scope classifier identical to the one in SERAC to retrieve a relevant edit example from the cache if there is one, but uses the base modelf base rather than the counterfactual model h Ļ to make the final prediction. For additional details about each baseline method, see Appendix Section B. 5.1. Model Editing Benchmarking Evaluating editors on challenging tasks.We perform a broad comparison of model editors in four editing settings, QA, QA-hard, FC, and ConvSent. For QA, QA-hard, and FC we usek= 10edits during training and evaluation; for ConvSent, we usek= 5because the longer dialogue se- quences cause increased memory usage. Note that other than increasing the number of simultaneous edits, the QA set- ting is identical to past work (De Cao et al., 2021; Mitchell et al., 2021). The LU and FT baselines are not applicable to ConvSent as there is no label to cache or fine-tune on. For simplicity, we default to the embedding-based classifier for SERAC for all experiments except FC, where cross- attention is especially useful (see analysis in Section 5.2). The results are presented in Table 3. Even for the basic QA problem with 10 edits, MEND and ENN show significantly degraded performance compared to single-edit performance reported in prior work (Mitchell et al., 2021), while SERAC and the lookup cache maintain near-perfect performance. When adding hard in-scope and out-of-scope examples in QA-hard, SERACās expressiveness enables significant im- provements over other approaches, with LU again showing the strongest performance of the baselines. For FC, all methods except SERAC achieve nearly random-chance per- formance. Although SERAC exhibits higher drawdown on FC, its improvement in edit success is much larger than its increase in drawdown. Finally, on the ConvSent edit- ing problem, where learned editors are needed to translate QA-hard(T5-large)FC(BERT-base) Scope splitCls acc.h Ļ acc.Cls acc.h Ļ acc. In (easy)0.9850.996 0.9090.875 In (hard)0.8550.987 Out (easy)0.9960.1230.993ā Out (hard)0.9670.0420.706ā Table 4.Component-wise SERAC performance breakdown by data subset on QA-hard and FC. On both datasets, hard examples ac- count for the vast majority of classifier errors. FC classifier per- formance on hard out-of-scope examples is the bottleneck for improving editor precision. FC does not annotate easy/hard in- scope examples (so they are pooled) or labels for out-of-scope examples (soh Ļ accuracy for out-of-scope examples is omitted). the explicit edit descriptor into the desired model behavior, SERAC again is the only method to achieve better than random performance, with zero drawdown. Making many edits.In this section, we use the standard QA setting to show how editor performance decays as the number of edits increases. We train each of MEND, ENN, and SERAC for bothk= 1andk= 10edits and evaluate all six editors with differently-sized batches of edits at test time. Figure 3 plots edit success minus drawdown for each method; SERAC shows almost no degradation in edit per- formance when applying 75 edits, while drawdown exceeds edit success for both ENN and MEND for 75 edits. Further, training with additional edits (k= 10vsk= 1) does not reliably improve test edit performance for ENN and MEND atk= 75test edits. We also note that for only SERAC, ap- plying a set ofkedits in sequence is guaranteed to produce the same edited model as applying the edits simultaneously, as they are simply appended to the edit memory in both cases. Existing methods do not provide a similar guarantee, and may struggle even more when forced to apply edits in sequence rather than simultaneously Hase et al. (2021). 5.2. Further Empirical Analysis of SERAC Error analysis.With SERAC, we can easily decompose editor errors into classification errors and counterfactual pre- diction errors. Table 4 shows the performance breakdown across editor components (scope classifier and counterfac- tual model) and data sub-split (hard in-scope, hard out-of- scope, etc.). For QA-hard, the classifier exhibits reduced accuracy on hard in-scope and out-of-scope examples, par- ticularly for hard in-scope examples. Counterfactual model performance is only slightly degraded on hard in-scope examples, suggesting that the primary challenge of the prob- lem is scope estimation, rather than counterfactual reasoning. For out-of-scope examples, counterfactual model perfor- mance is low, but high classifier accuracy means that these inputs are typically (correctly) routed to the base model in- Memory-Based Model Editing at Scale QA-hard(T5-large)FC(BERT-base) VariantESāDāESāDā Embed-D0.9210.0290.7920.247 Cross-D0.9830.0090.8310.074 Embed-B0.9450.0340.7920.247 Cross-B0.9830.0070.8550.0964 Table 5.Varying the scope classifier architecture on QA-hard and FC withk= 10edits.Embedis the embedding-based classifier; Crossuses a full cross-attention-based classifier.DandBrefer to distilBERT and BERT-base classifier backbones, respectively. stead. For FC, scope classifier failures on hard out-of-scope examples dominate the editorās errors. Scope classifier architecture. We perform a set of exper- iments to understand how the classifier architecture impacts the behavior of SERAC. Using the QA-hard and FC tasks withk= 10edits, we compare the cross-attention (Cross) and dense embedding (Embed) classifier using both distil- BERT (D; (Sanh et al., 2019)) and BERT-base (B; (Devlin et al., 2019)) as the backbone model. The results are shown in Table 5. Unsurprisingly, using cross-attention instead of dense-embeddings is helpful for editor performance; how- ever, increasing classifier size shows relatively little im- provement. Cross-attention is especially useful for the FC experiment, which is possibly due to the commonness of quantities in the VitaminC dataset; for example, produc- ing fixed-length sequence embeddings that reliably capture the difference betweenTHERE HAVE BEEN105,000CORO- NAVIRUS DEATHS IN THEUNITEDSTATESandTHERE HAVE BEEN111,000CORONAVIRUS DEATHS IN THEUNITEDSTATES may be very difficult. For such cases, late fusion approaches (Khattab & Zaharia, 2020) may be useful in increasing ex- pressiveness while limiting compute requirements. Re-using model editors across models.A key advantage of SERAC is separation of the base model and editor, decou- pling the editorās performance from the base model. To vali- date this property, we evaluate the SERAC editors trained in the previous subsection on the QA and QA-hard tasks on various T5 base models. As expected, SERACās edit success and drawdown is near-identical across T5 model sizes in both settings (drawdown slightly fluctuates with different base models), consistently yielding ES above 0.99 and D below 0.01 for QA 5 and ES above 0.92, D below 0.03 for QA-hard for all models. Editors described in past works must be re-fit to each new base model (Sinitsin et al., 2020; De Cao et al., 2021; Mitchell et al., 2021; Meng et al., 2022) 5 For comparison, Mitchell et al. (2021) report an ES of 0.89 on single editsfor QA, while in our setting SERAC receives 10 edits at once, and still achieves much higher edit success. Drawdown is reported differently in Mitchell et al. (2021), so is not comparable. TaskBase modelSERAC (out)SERAC (in) QA87ms 2.96GB92ms 3.47GB31ms 3.46GB FC7ms 0.44GB19ms 1.18GB19ms 1.18GB CS182ms 0.38GB 183ms 1.00GB 185ms 1.01GB Table 6.Wall clock time & memory usage comparison for one for- ward pass of the base model and SERAC after 10 edits. SERACās performance is given separately forout-of-scope inputs (routed to base model) andin-scope inputs (routed to counterfactual model). and require access to the internal activations or gradients of f base , leading to potentially prohibitive computational costs of editor fitting that scale with the size off base . Computational demands of SERAC SERACās addition of scope classifier and counterfactual model incurs some additional computational overhead. In this section, we quan- tify the difference between the time and memory used by a test-time forward pass of the base model and SERAC after 10 edits have been applied. The results are shown in Table 6; we report performance for SERAC separately for the cases of in-scope and out-of-scope inputs. Compute time.For QA and ConvSent (CS), SERAC uses a fast nearest-neighbor-based classifier and is nearly as fast as the base model. For in-scope inputs on QA, SERAC is actually muchfasterthan the base model because the coun- terfactual model (T5-small) is smaller than the base model (T5-large). For FC, SERACās increase in computation time is due to the more expressive (but more computationally expensive) full cross-attention classifier used for this prob- lem. By leveraging this additional compute, SERAC is the only method that provides any significant improvement over random chance editing performance for the FC problem. Memory consumption.SERACās additional memory usage mostly comes from the weights of the classifier and counter- factual model, not the edit memory itself (which uses only about 3KB per edit, many orders of magnitude smaller than the base model). For QA, where the base model (T5-large) is much larger than the counterfactual model (T5-small) and classifier (distilBERT), this increase is relatively small. For FC and CS, the counterfactual model and classifier are of similar size to the base model, yielding a larger increase in memory consumption. However, the vast majority of this increase in memory usage is afixed cost that does not increase with the number of edits. 6. Related Work Model editing. Many approaches have recently been pro- posed for model editing. Simplest among these uses con- strained fine-tuning to update parameters based on new ex- amples (Sotoudeh & Thakur, 2019; Zhu et al., 2020). Other Memory-Based Model Editing at Scale methods explore special pre-training objectives that enable rapid and targeted fine-tuning for model edits (Sinitsin et al., 2020) via meta-learning. More recently, new classes of methods develop external learned editors that modify fine- tuning gradients for editing, but do not change the base model that must process edits (De Cao et al., 2021; Mitchell et al., 2021; Hase et al., 2021). Finally, certain methods attribute knowledge to particular neurons in the network and manually edit these activation to reflect changed con- tent (Dai et al., 2021; Meng et al., 2022). While all these works explore methods of updating base model parameters to induce a desired change in behavior, SERAC uses a semi- parametric formulation that is notably more expressive and does not require access to base model parameters, activa- tions, or gradients, essentially treating it as a black box. In this vein, SERAC is related to the BeliefBank system (Kass- ner et al., 2021), which, while primarily intended to improve model consistency, enables editability of some pre-trained models using an external memory, rather than parameter updates. However, it is limited to models performing binary classification of factual statements and requires manually- annotated constraints between facts. SERAC requires no such specialized augmentations to the input data. Memory-augmented models. Memory mechanisms have historically been combined with neural networks in a variety of contexts including supervised learning (Hochre- iter & Schmidhuber, 1997; Graves et al., 2008; 2014), meta-learning (Santoro et al., 2016; Shan et al., 2020), and reinforcement learning (Oh et al., 2016; Pritzel et al., 2017).Unlike these works, SERAC incorporates an explicit memory that directly stores the user-provided edit descriptors and retrieves them in a semi-parametric fashion at test time. Non-parametric few-shot learning models (Koch et al., 2015; Vinyals et al., 2016; Snell et al., 2017) also store small datasets and process the examples when making predictions at test time. Another recent line of work augments transformers with non-parametric memories that store textual snippets (Chen et al., 2017; Lee et al., 2019; Khandelwal et al., 2020; Karpukhin et al., 2020). Unlike both of these research threads, we focus specifically on the problem of learning to edit existing models, rather than few-shot learning or training retrieval-based models from scratch.Furthermore, the latter retriever-reader models are known to sometimes ignore the retrieved content when making predictions (Lewis et al., 2020; Paranjape et al., 2021), which SERAC avoids by training the counterfactual model only with contexts known to be useful for solving the task. Finally, some continual learning algorithms have used external memories to avoid forgetting (Lopez-Paz & Ranzato, 2017; Rolnick et al., 2019; Buzzega et al., 2020). 7. Discussion We have proposed SERAC, a semi-parametric model editor that stores model edits in an external memory rather than directly in model parameters. Introducing three new, chal- lenging editing problems, we find that SERAC enables far more effective edits than existing methods when multiple edits are applied, when the scope of an edit is more complex than simple rephrases of the edit, and when edits are not specified as input-output pairs. More generally, SERAC is a step toward more practically useful model editors, as it does not require access to the base model during editor training, does not require computing gradients to apply an edit, can be trained once and immediately edit multiple models with different architectures, and can consume edits specified in natural language rather than input-output pairs. Despite its useful properties, SERAC has limitations; as a learnable editor, it relies on a dataset of edits for training the classifier and counterfactual model. Further, while we find relatively good performance from small classifiers and counterfactual models, some settings may demand more resource-intensive architectures. In a setting where editing occurs continuously, the edit memory may grow without bound. Future work might address this problem through periodic self-distillation, using the aggregate system of base model, scope classifier, edit memory, and counterfactual model as a teacher model to a āstudentā copy of the base model. Such a method would essentially enable the size of the edit memory to be capped, even in the continual editing setting, through periodic flushing of the memory. One possible concern with model editors, including SERAC is misuse: while model editors may help keep deep learn- ing systems more up-to-date in a computationally efficient manner, the dialogue sentiment editing setting (Tables 2; 7) suggest that powerful model editors could also enable malicious users to more precisely craft agents to amplify particular viewpoints. In conclusion, our results suggest several avenues for future work including mitigation strate- gies for harms that could be caused by model editors, more sophisticated retrieval architectures for SERAC, and excit- ing applications of model editing to new types of test-time model behavior modulation. 8. Acknowledgements The authors thank Shikhar Murty, Archit Sharma, and the members of Stanfordās Center for Research on Foundation Models for helpful discussions and conceptual feedback, as well as the anonymous ICML reviewers for their feedback during the review process. EM gratefully acknowledges the financial support of the Knight-Hennessy Graduate Fel- lowship. The authors also gratefully acknowledge financial support from Apple Inc. CF and CM are CIFAR Fellows. Memory-Based Model Editing at Scale References Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners.Neural Information Pro- cessing Systems, 2020. Buzzega, P., Boschini, M., Porrello, A., Abati, D., and Calderara, S. Dark experience for general continual learn- ing: a strong, simple baseline. In Larochelle, H., Ran- zato, M., Hadsell, R., Balcan, M. F., and Lin, H. (eds.), Advances in Neural Information Processing Systems, vol- ume 33, p. 15920ā15930. Curran Associates, Inc., 2020. Chen, D., Fisch, A., Weston, J., and Bordes, A. Reading wikipedia to answer open-domain questions. InACL, 2017. Csord Ģ as, R., van Steenkiste, S., and Schmidhuber, J. Are neural nets modular? Inspecting functional mod- ularity through differentiable weight masks.InIn- ternational Conference on Learning Representations, 2021. URLhttps://openreview.net/forum? id=7uVcpu-gMD. Dai, D., Dong, L., Hao, Y., Sui, Z., and Wei, F. Knowledge neurons in pretrained transformers.CoRR, abs/2104.08696, 2021. URLhttps://arxiv.org/ abs/2104.08696. De Cao, N., Aziz, W., and Titov, I. Editing factual knowl- edge in language models.ArXiv, abs/2104.08164, 2021. Demszky, D., Guu, K., and Liang, P. Transforming ques- tion answering datasets into natural language inference datasets.CoRR, abs/1809.02922, 2018. URLhttp: //arxiv.org/abs/1809.02922. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for lan- guage understanding. InProceedings of the 2019 Con- ference of the North American Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), p. 4171ā4186, Minneapolis, Minnesota, June 2019. Asso- ciation for Computational Linguistics. doi: 10.18653/ v1/N19-1423. URLhttps://w.aclweb.org/ anthology/N19-1423. Graves, A., Liwicki, M., Fern Ģ andez, S., Bertolami, R., Bunke, H., and Schmidhuber, J. A novel connectionist system for unconstrained handwriting recognition.IEEE transactions on pattern analysis and machine intelligence, 31(5):855ā868, 2008. Graves, A., Wayne, G., and Danihelka, I. Neural turing machines, 2014. URLhttp://arxiv.org/abs/ 1410.5401. arxiv:1410.5401. Hase, P., Diab, M., Celikyilmaz, A., Li, X., Kozareva, Z., Stoyanov, V., Bansal, M., and Iyer, S. Do language models have beliefs? Methods for detecting, updating, and visualizing model beliefs, 2021. arxiv:2111.13654. Heitmann, M., Siebert, C., Hartmann, J., and Schamp, C. More than a feeling: Benchmarks for sentiment analysis accuracy.Available at SSRN 3489963, 2020. Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural computation, 9(8):1735ā1780, 1997. Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t.Dense pas- sage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 6769ā6781, Online, November 2020. Association for Computational Linguistics.doi: 10.18653/v1/ 2020.emnlp-main.550. URLhttps://w.aclweb. org/anthology/2020.emnlp-main.550. Kassner, N., Tafjord, O., Sch Ģ utze, H., and Clark, P. Be- liefBank: Adding memory to a pre-trained language model for a systematic notion of belief. InProceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing, p. 8849ā8861, On- line and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.697. URLhttps:// aclanthology.org/2021.emnlp-main.697. Khandelwal, U., Levy, O., Jurafsky, D., Zettlemoyer, L., and Lewis, M. Generalization through memorization: Nearest neighbor language models. InICLR, 2020. Khattab, O. and Zaharia, M. Colbert: Efficient and ef- fective passage search via contextualized late interac- tion over bert.InProceedings of the 43rd Interna- tional ACM SIGIR Conference on Research and Devel- opment in Information Retrieval, p. 39ā48, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450380164. URLhttps://doi.org/10. 1145/3397271.3401075. Koch, G., Zemel, R., Salakhutdinov, R., et al. Siamese neural networks for one-shot image recognition. InICML deep learning workshop, volume 2. Lille, 2015. Memory-Based Model Editing at Scale Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Kel- cey, M., Devlin, J., Lee, K., Toutanova, K. N., Jones, L., Chang, M.-W., Dai, A., Uszkoreit, J., Le, Q., and Petrov, S. Natural questions: a benchmark for question answering research.Transactions of the Association of Computational Linguistics, 2019. Lazaridou, A., Kuncoro, A., Gribovskaya, E., Agrawal, D., Liska, A., Terzi, T., Gimenez, M., de Masson dāAutume, C., Ruder, S., Yogatama, D., Cao, K., Kocisk Ģ y, T., Young, S., and Blunsom, P. Mind the gap: Assessing temporal generalization in neural language models. InNeurIPS, 2021. Lee, K., Chang, M.-W., and Toutanova, K. Latent retrieval for weakly supervised open domain question answering. InACL, 2019. Levy, O., Seo, M., Choi, E., and Zettlemoyer, L. Zero- shot relation extraction via reading comprehension. InProceedings of the 21st Conference on Computa- tional Natural Language Learning (CoNLL 2017), p. 333ā342, Vancouver, Canada, August 2017. Associa- tion for Computational Linguistics.doi: 10.18653/ v1/K17-1034. URLhttps://w.aclweb.org/ anthology/K17-1034. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K Ģ uttler, H., Lewis, M., Yih, W.-t., Rockt Ģ aschel, T., Riedel, S., and Kiela, D. Retrieval-augmented gener- ation for knowledge-intensive nlp tasks. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H. (eds.),Advances in Neural Information Processing Systems, volume 33, p. 9459ā9474. Curran Asso- ciates, Inc., 2020. URLhttps://proceedings. neurips.c/paper/2020/file/ 6b493230205f780e1bc26945df7481e5-Paper. pdf. Li, M., Roller, S., Kulikov, I., Welleck, S., Boureau, Y.-L., Cho, K., and Weston, J. Donāt say that! making incon- sistent dialogue unlikely with unlikelihood training. In Proceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics, p. 4715ā4728, On- line, July 2020. Association for Computational Linguis- tics. doi: 10.18653/v1/2020.acl-main.428. URLhttps: //aclanthology.org/2020.acl-main.428. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. Roberta: A robustly optimized BERT pretraining approach.CoRR, abs/1907.11692, 2019. URLhttp://arxiv.org/ abs/1907.11692. Lopez-Paz, D. and Ranzato, M. A. Gradient episodic memory for continual learning. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vish- wanathan, S., and Garnett, R. (eds.),Advances in Neural Information Processing Systems, volume 30. Curran As- sociates, Inc., 2017. URLhttps://proceedings. neurips.c/paper/2017/file/ f87522788a2be2d171666752f97ddebb-Paper. pdf. Meng, K., Bau, D., Andonian, A., and Belinkov, Y. Lo- cating and editing factual associations in GPT, 2022. arXiv:2202.05262. Mitchell, E., Lin, C., Bosselut, A., Finn, C., and Manning, C. D. Fast model editing at scale.CoRR, 2021. URL https://arxiv.org/pdf/2110.11309.pdf. Oh, J., Chockalingam, V., Lee, H., et al. Control of memory, active perception, and action in minecraft. InInterna- tional Conference on Machine Learning, p. 2790ā2799. PMLR, 2016. Paranjape, A., Khattab, O., Potts, C., Zaharia, M. A., and Manning, C. D. Hindsight: Posterior-guided training of retrievers for improved open-ended generation.ArXiv, abs/2110.07752, 2021. Pritzel, A., Uria, B., Srinivasan, S., Badia, A. P., Vinyals, O., Hassabis, D., Wierstra, D., and Blundell, C. Neural episodic control. In Precup, D. and Teh, Y. W. (eds.), Proceedings of the 34th International Conference on Ma- chine Learning, volume 70 ofProceedings of Machine Learning Research, p. 2827ā2836. PMLR, 06ā11 Aug 2017. URLhttps://proceedings.mlr.press/ v70/pritzel17a.html. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Ex- ploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 21(140):1ā67, 2020. URLhttp://jmlr. org/papers/v21/20-074.html. Reimers, N. and Gurevych, I. Sentence-bert: Sentence em- beddings using siamese bert-networks. InProceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing. Association for Computational Linguistics, 11 2019. URLhttps://arxiv.org/ abs/1908.10084. Ribeiro, M. T., Guestrin, C., and Singh, S. Are red roses red? evaluating consistency of question-answering mod- els. InProceedings of the 57th Annual Meeting of the As- sociation for Computational Linguistics, p. 6174ā6184, Florence, Italy, July 2019. Association for Computa- tional Linguistics. doi: 10.18653/v1/P19-1621. URL https://aclanthology.org/P19-1621. Memory-Based Model Editing at Scale Roberts, A., Raffel, C., and Shazeer, N. How much knowl- edge can you pack into the parameters of a language model?, 2020. Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., and Weston, J. Recipes for building an open-domain chatbot.InProceedings of the 16th Conference of the European Chapter of the Association for Compu- tational Linguistics: Main Volume, p. 300ā325, On- line, April 2021. Association for Computational Linguis- tics. doi: 10.18653/v1/2021.eacl-main.24. URLhttps: //aclanthology.org/2021.eacl-main.24. Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T., and Wayne, G.Experience replay for continual learning.In Wallach, H., Larochelle, H., Beygelz- imer, A., d'Alch Ģ e-Buc, F., Fox, E., and Garnett, R. (eds.),Advances in Neural Information Pro- cessing Systems, volume 32. Curran Associates, Inc.,2019.URLhttps://proceedings. neurips.c/paper/2019/file/ fa7cdfad1a5aaf8370ebeda47a1f1c3-Paper. pdf. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.ArXiv, abs/1910.01108, 2019. Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., and Zaharia, M.Colbertv2: Effective and effi- cient retrieval via lightweight late interaction.CoRR, abs/2112.01488, 2021. URLhttps://arxiv.org/ abs/2112.01488. Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., and Lillicrap, T. Meta-learning with memory-augmented neural networks.In Balcan, M. F. and Weinberger, K. Q. (eds.),Proceedings of The 33rd International Con- ference on Machine Learning, volume 48 ofProceed- ings of Machine Learning Research, p. 1842ā1850, New York, New York, USA, 20ā22 Jun 2016. PMLR. URLhttps://proceedings.mlr.press/v48/ santoro16.html. Schuster, T., Fisch, A., and Barzilay, R. Get your vi- tamin C! robust fact verification with contrastive evi- dence. InProceedings of the 2021 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, p. 624ā643, Online, June 2021. Association for Com- putational Linguistics. URLhttps://w.aclweb. org/anthology/2021.naacl-main.52. Shan, S., Li, Y., and Oliva, J. B. Meta-neighborhoods. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H. (eds.),Advances in Neural Information Pro- cessing Systems, volume 33, p. 5047ā5057. Curran As- sociates, Inc., 2020. URLhttps://proceedings. neurips.c/paper/2020/file/ 35464c848f410e55a13b9d78e7fddd0-Paper. pdf. Sinitsin, A., Plokhotnyuk, V., Pyrkin, D., Popov, S., and Babenko, A.Editable neural networks.InICLR, 2020. URLhttps://openreview.net/forum? id=HJedXaEtvS. Snell, J., Swersky, K., and Zemel, R. S.Prototypi- cal networks for few-shot learning.arXiv preprint arXiv:1703.05175, 2017. Sotoudeh, M. and Thakur, A. V. Correcting deep neural networks with small, generalizing patches. InNeurIPS 2019 Workshop on Safety and Robustness in Decision Making, 2019. Thorne, J., Vlachos, A., Christodoulopoulos, C., and Mittal, A. FEVER: a large-scale dataset for fact extraction and VERification. InNAACL-HLT, 2018. Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al. Matching networks for one shot learning.Advances in neural information processing systems, 29:3630ā3638, 2016. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtow- icz, M., and Brew, J.Huggingfaceās transformers: State-of-the-art natural language processing.CoRR, abs/1910.03771, 2019. URLhttp://arxiv.org/ abs/1910.03771. Zhu, C., Rawat, A. S., Zaheer, M., Bhojanapalli, S., Li, D., Yu, F., and Kumar, S. Modifying memories in trans- former models, 2020. URLhttps://arxiv.org/ abs/2012.00363. Memory-Based Model Editing at Scale Appendix A. Additional Sentiment Editing Example and Broader Impacts While the ābananaā example in Table 2 is a relatively mun- dane topic, we include an example of editing a dialog model for a more polarizing topic in some parts of the world, vac- cines. Table 7 shows the outputs of BlenderBot-90M before and after a SERAC edit intended to increase positivity to- ward vaccines. The results are striking, with the original modelās sentiment nearly always negative toward vaccines, while the edited model consistently produces positive, on- topic responses about vaccines. While editing a dialogue model to reduce vaccine hesitancy in the general public may be regarded as a beneficial tool for public health, the general ability to modulate a modelās opin- ions or beliefs about any topic has some profound impacts on how models governance occurs. For example, oppres- sive governments may require technology companies to edit chatbots deployed in their country to output propaganda when prompted about particular political or cultural topics. Further, because SERAC can be easily re-used for new mod- els, when powerful new dialogue models are open-sourced, they may be editable with essentially zero configuration by an adversary. Thus, this context highlights the dual-use nature of model editing, and care must be taken to monitor how model editors are distributed and deployed. B. Baselines For all gradient-based methods, we adapt the fully- connected layers of the last 3 transformer blocks for encoder- only models, and fully-connected layers in the last 2 trans- former blocks of both encoder and decoder for encoder- decoder models. Fine-tuning (FT)Given edit samples[x e ;y e ], we fine- tune pretrained models to minimize the negative log- likelihood of predictingy e conditioned onx e . We use the Adam optimizer with a learning rate of1Ć10 ā4 for T5 and 5Ć10 ā6 for BERT-base. Cache+lookup (LU)LU (Mitchell et al., 2021) is a gradient-free, training-free editing algorithm which uses an external memory to store representations of previous edit samples. An edit sample[x e ;y e ]is represented in LUās memory as[z e ;y e ]wherez e is the average over the hidden dimension of last hidden state computed byf base onx e . For a test input[x ā² e ] , LU computes the hidden representation z ā² e ofx ā² e and finds the nearest edit-sample representation in its memory, sayz e . LU outputsy e ifāz ā² e āz e ā 2 < Ī“ whereĪ“is a hyperparameter, and otherwise outputs the pre- trained modelās prediction onx ā² e . We usedĪ“= 2.75for the question-answering settings andĪ“= 4for the fact-checking setting. Editable Neural Networks (ENN)(Sinitsin et al., 2020) introduce a post-training procedure to make a pretrained model quickly adaptable for fine-tuning for edits. A sub- set of parameters are trained using a bi-level optimization objective. We use Adam with an outer-loop learning rate of1Ć10 ā5 , and an initial inner-loop learning of1Ć10 ā2 which is learned in the outer loop. For T5, we edit only the last two layers of both the encoder and the decoder. For BERT-base, we edit the last two layers of the encoder. Fi- nally, for BlenderBot-small, we edit the last layer of the encoder and the last three layers of the decoder since the decoder is much deeper. Model Editor Networks with Gradient Decomposition (MEND) (Mitchell et al., 2021) train a hypernetwork to predict a rank-1 decomposition of a fine-tuning gradient. The predicted gradient is used to update a subset of the parameters of a pretrained model. In our experiments, we use MEND to update the same parameters as in ENN. C. SERAC Implementation Details We use publicly available Huggingface (Wolf et al., 2019) implementations and checkpoints for all exper- iments.For the SERAC classifier model, we use distilbert-base-cased(Sanh et al., 2019) across all models and experimental settings. For the counterfactual model, we uset5-smallfor the question-answering ex- periments,bert-base-uncasedfor fact-checking, and facebook/blenderbotsmall-90M(Roller et al., 2021) for conversational sentiment modulation. We use T5 pretrained on NQ (google/t5-large-ssm-nq) for question-answering,bert-base-uncasedfinetuned by De Cao et al. (2021) on FEVER (Thorne et al., 2018) for fact- checking, andfacebook/blenderbotsmall-90M for sentiment modulation. All scope classifier and counterfactual models are trained using Adam with a learning rate of1Ć10 ā5 . D. Dataset Details D.1. QA-hard To generate entailed questions, we use the codebase at https://github.com/marcotcr/qaconsistency (Ribeiro et al., 2019), passing the question as both question and context to the entailed question generator. We find this approach pro- duces questions that are typically interpretable, although not always grammatically correct. To generate true/false ques- tions, we use the rule-based question/answer to statement Memory-Based Model Editing at Scale PromptWHAT DO YOU THINK OF VACCINES? Edit Desc.TOPIC:VACCINES SENTIMENT:POSITIVE Original model outputs i think theyāre bad. i donāt trust them, and iām not one of those people that believes that vaccines cause autism or any other unpreventable disease.[...] iām not a fan of them personally, but i can see why theyāre used in certain cases. i have a friend whoās in the process of becoming a doctor, and heās told me that itās[...] iām not sure what you mean by that. are you asking if i think vaccines are bad or not?[...] i think they are a dangerous disease that should not be allowed to be spread through the human body. i donāt want to be associated with those people in this day and age.[...] iām not sure what vaccines are, but i do know that theyāre a group of vaccines that are also known as autism vaccines.[...] i think theyāre a bad idea, but theyāve been around for a long time, so i donāt really have a problem with them.[...] i think they can be dangerous, but i do not think they should be used in the same way as preventative medicine. thatās why i donāt trust them.[...] i donāt think they should be used for any reason. theyāre used to treat diseases like autism, anaphylax, and sneezing. i wouldnāt want any of those to happen to me.[...] i think theyāre a good thing, but i also think that the vaccines should be kept away from children. iām not sure if i can trust children with autism or not.[...] i donāt like them. iām more of a fan of the common cold. how about you?[...] Edited model outputs iāve been vaccinating my kids since they were little. i think itās a good thing. i donāt know if youāre aware of it! do you have any thoughts? iād be happy to[...] theyāre great! iāve been vaccinated since i was a little girl. i donāt know what iād do without them. do you have a positive or a negative opinion on them? iām all for[...] theyāre great! i donāt know if iāve ever had one, but i know theyāve been around since the 1980ās. iām not sure if itās good or bad. do you have[...] i think that they are a good thing. i donāt agree with them, but i understand why they exist. do you have any opinions on them? iād like to hear your opinion! iām all for them![...] i think they are great. i donāt know a whole lot else about it. do you have a positive or negative opinion on them? iād love to know! iām all for them! do you know anything about[...] well iād say iām pretty positive about them. i think itās important for people to be informed about the risks of their health, so i donāt see why i shouldnāt be allowed to have them.[...] i think theyāre great. i donāt know if you know this, but theyāve been around for a long time. itās a good thing! do you have a positive opinion? iād love to know[...] i think they are a good way to prevent infectious diseases. i am not sure if itās true, but iāve been vaccinating my kids for the last 4 years. do you like them? i donāt know[...] i think vaccines are a good way to prevent disease. iām not sure what you mean by positive. are you asking if i support vaccines? i donāt know about vaccines! do you have a positive stance?[...] i think vaccines are great. iāve been vaccinated since i was a child. i donāt know much about them, i just know that theyāre very effective! iām not sure if you know anything about[...] Table 7.Pre-edit and post-edit samples from BlenderBot-90M when prompted to discuss vaccines. Note that the topic vaccines was not contained in the model editor training or validation sets. Memory-Based Model Editing at Scale PromptsWhat is yourSENTIMENT POSITION ENTITY? SENTIMENTpositive, negative POSITION opinion of, stance on, position on, impression of, assessment of Table 8.Prompt templates used to generate ConvSent dataset. Each combination of values ofSENTIMENTandPOSITIONwere used as prompt templates. Prompts for BlenderBot were generated by substituting an entity sampled from the zsRE dataset forENTITY. converter at https://github.com/kelvinguu/qanli, appending the prompt āTrue or false:ā to the beginning of the input. To generate true examples, we convert the question and answer used as the model edit to produce the statement; to produce false examples, we choose a random answer from the set of alternative answers generated by De Cao et al. (2021). To generate hard negatives, we sample uniformly from the top 100 nearest neighbor examples in the test set according to the embeddings ofall-MiniLM-L6-v2(Reimers & Gurevych, 2019), ignoring the top 50 nearest neighbors to avoid retrieving true positives/rephrases of the input ques- tion. D.2. ConvSent Conversational sentiment completions were generated using a 3 billion-parameter BlenderBot model available on Hug- gingface atfacebook/blenderbot-3B(Roller et al., 2021). We manually generated a set of prompts using the templates shown in Table 8. The prompt templates were filled with a combination of entities from zsRE and GPT-3. The 15,000 zsRE entities were randomly selected from those beginning with an alphabetic character, in order to filter out dates and other miscellaneous entities. The 989 GPT-3ā generated entities are noun phrases manually selected by the authors. We sampled from BlenderBot using beam search with a beam width of10. We then classified each com- pletion as āpositiveā or ānegativeā using a RoBERTa-large model fine-tuned for sentiment classification (Heitmann et al., 2020). Data were randomly split (by entity) into 90-5-5 train/val/test splits.