Paper deep dive
"Why" Has the Least Side Effect on Model Editing
Tsung-Hsuan Pan, Chung-Chi Chen, Hen-Hsen Huang, Hsin-Hsi Chen
Models: GPT-2-XL, LLaMA-2-7B
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/12/2026, 5:54:48 PM
Summary
This paper investigates the side effects of model editing on large language models (LLMs), specifically focusing on how different question types and batch sizes influence performance degradation. The authors find that 'why' questions result in the least performance impact, and increasing batch sizes can mitigate side effects. Furthermore, the study highlights that findings from smaller models like GPT-2 do not necessarily extrapolate to larger models like LLaMA-2, emphasizing the need for model-specific analysis.
Entities (5)
Relation Signals (4)
GPT-2 â differsfrom â LLaMA-2
confidence 95% · our results indicate that the findings differ between models of different sizes
Why Questions â hasleasteffecton â Model Stability
confidence 95% · Notably, âWhyâ questions have the least adverse effect on model editing.
Batch Size â mitigates â Performance Degradation
confidence 92% · increasing the batch size can mitigate performance drops.
MEMIT â causes â Performance Degradation
confidence 90% · Although model editing methods show potential in updating knowledge... potentially leading to side effects.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Training large language models (LLMs) from scratch is an expensive endeavor, particularly as world knowledge continually evolves. To maintain relevance and accuracy of LLMs, model editing has emerged as a pivotal research area. While these methods hold promise, they can also produce unintended side effects. Their underlying factors and causes remain largely unexplored. This paper delves into a critical factor-question type-by categorizing model editing questions. Our findings reveal that the extent of performance degradation varies significantly across different question types, providing new insights for experimental design in knowledge editing. Furthermore, we investigate whether insights from smaller models can be extrapolated to larger models. Our results indicate discrepancies in findings between models of different sizes, suggesting that insights from smaller models may not necessarily apply to larger models. Additionally, we examine the impact of batch size on side effects, discovering that increasing the batch size can mitigate performance drops.
Tags
Links
- Source: https://arxiv.org/abs/2409.18679
- Canonical: https://arxiv.org/abs/2409.18679
Trouble viewing inline? Open PDF directly â
Full Text
20,622 characters extracted from source content.
Expand or collapse full text
âWhyâ Has the Least Side Effect on Model Editing Tsung-Hsuan Pan, 1 Chung-Chi Chen, 2 Hen-Hsen Huang, 3 Hsin-Hsi Chen 1 1 Department of Computer Science and Information Engineering, National Taiwan University, Taiwan 2 AIST, Japan 3 Institute of Information Science, Academia Sinica, Taiwan b08902138@ntu.edu.tw, c.c.chen@acm.org, hhhuang@iis.sinica.edu.tw, hhchen@ntu.edu.tw Abstract Training large language models (LLMs) from scratch is an expensive endeavor, particularly as world knowledge continually evolves. To main- tain relevance and accuracy of LLMs, model editing has emerged as a pivotal research area. While these methods hold promise, they can also produce unintended side effects. Their underlying factors and causes remain largely unexplored. This paper delves into a critical factorâquestion typeâby categorizing model editing questions. Our findings reveal that the extent of performance degradation varies sig- nificantly across different question types, pro- viding new insights for experimental design in knowledge editing. Furthermore, we investi- gate whether insights from smaller models can be extrapolated to larger models. Our results indicate discrepancies in findings between mod- els of different sizes, suggesting that insights from smaller models may not necessarily apply to larger models. Additionally, we examine the impact of batch size on side effects, discover- ing that increasing the batch size can mitigate performance drops. 1 Introduction Training large language models (LLMs) from scratch is prohibitively expensive when world knowledge changes. However, the world evolves daily. To keep LLMs updated with current world knowledge, model editing (Mitchell et al., 2022a; Chen et al., 2024; Hartvigsen et al., 2024; Yu et al., 2024) has emerged as a crucial research area in the LLM era. Although model editing methods show potential in updating knowledge, partially modifying the parameters of language models via model editing is akin to performing surgery on the human brain, potentially leading to side ef- fects (Hoelscher-Obermaier et al., 2023; Gu et al., 2024; Yang et al., 2024). While there are some intuitive discussions on the side effects of model editing, identifying the factors and causes of these side effects is scarcely addressed. We noticed that the question-answering setting is the most common when editing knowledge. For example, when we want to update the information about the U.S. pres- ident, we typically design a question for models such as âWho is the president of the U.S.?â Fol- lowing this line of thought, we are curious whether different question types will lead to different side effects after editing. A common finding regarding the side effects of model editing is that the modelâs performance across different aspects tends to deteriorate after a few edits (Gu et al., 2024; Yang et al., 2024). Given that the severity of surgical side effects varies with the type of surgery, we are curious whether editing the knowledge for different question types will re- sult in varying degrees of performance degradation. To this end, we categorize the questions used for model editing into eight types: who, what, when, where, which, why, how, and others. Our results indicate that the extent of performance degrada- tion significantly differs after editing knowledge for different types of questions. It suggests future directions for experimental design in knowledge editing. Moreover, if the illness issues are related or ad- dressing them together can reduce the overall sur- gical risk, doctors might choose a single surgery to solve multiple problems. Based on this concept, we discuss the side effects under different batch size settings. Our results suggest that enlarging the batch size, i.e., editing several pieces of knowledge at the same time, can mitigate the side effects of the performance drop. Finally, performing the same surgery on adults and children may result in different side effects, and the underlying causes may vary. Following this line of thought, we experiment with GPT-2 (1.5 billion parameters) (Radford et al., 2019a) and LLaMA-7B (7 billion parameters) (Touvron et al., 2023a) to explore whether findings from smaller arXiv:2409.18679v1 [cs.CL] 27 Sep 2024 models, which is cheaper and more efficient, can be extrapolated to larger models. Unfortunately, our results indicate that the findings differ between models of different sizes, suggesting that insights from smaller models may not necessarily apply to larger models. In sum, this paper makes the following contribu- tions: (1) We provide an in-depth analysis of how different question types affect the performance of LLMs after model editing. (2) We investigate the impact of batch size on the side effects of model editing and reveal that larger batch sizes can miti- gate performance degradation. (3) We explore the applicability of findings from smaller models to larger models and highlight the limitations of such applications. 2 Related Work Model editing is a rapidly evolving field with sev- eral key approaches aimed at modifying model behavior without extensive retraining (Yao et al., 2023). Fine-tuning with constraints (Zhu et al., 2020) is a method developed to mitigate the is- sue of catastrophic forgetting, where new knowl- edge overwrites previously learned information. This approach involves updating as few parame- ters as possible or only modifying specific parts of the modelâs structure. Memory-augmented tech- niques (Mitchell et al., 2022b) involve storing new or corrected knowledge separately from the original model, effectively creating a patch model. These patches can be implemented in various ways, such as through pretrained models or datastores, and are combined with the original model using simple methods like classifiers. However, this approach requires retraining both the classifier and the patch model, which is not ideal for continuous updates. Hyper networks (Cao et al., 2021) represent a dy- namic method where the model continuously up- dates its parameters based on incoming knowledge without needing retraining or fine-tuning. This is achieved by training a network to predict the weights of another network, effectively learning the process of fine-tuning through gradient descent. Despite its promise, the efficacy of hyper networks may diminish as the volume of updates increases, posing challenges for long-term usability. Addi- tionally, current implementations can handle only up to 75 knowledge edits at a time. The locate-and-edit approach (Meng et al., 2022a,b) leverages interpretability insights, treating the MLP layers in transformers as key-value memo- ries (Geva et al., 2020). By identifying the specific neurons responsible for storing factual associations (keys and values), this method modifies the values corresponding to the desired knowledge. The pro- cess involves evaluating the influence of individual neurons on the output and adjusting the most im- pactful ones. It offers enhanced interpretability and allows for precise targeting of specific pieces of knowledge within the model. It is favored for sce- narios where understanding and precisely control- ling model behavior is crucial. Therefore, in this paper, we focus on the iconic method of locate-and- edit, MEMIT (Meng et al., 2022b), for in-depth analysis and discussions. 3 Experimental Setup 3.1 Knowledge Editing Dataset We use RealTimeQA (Kasai et al., 2022) as the base dataset for knowledge editing. RealTimeQA is a collection derived from popular news sources, containing articles from various news websites. Weekly, RealTimeQA gathers news articles along with approximately 30 multiple-choice questions authored by humans from platforms such as CNN, THE WEEK, and USA Today, covering diverse top- ics including politics, business, sports, and enter- tainment. Unlike other datasets such as ZsRE (Levy et al., 2017) or CounterFact (Meng et al., 2022a), which draw from known Wiki knowledge or focus on false facts respectively, we opt for RealTimeQA due to its alignment with real-world scenarios, of- fering a more fitting context for our knowledge updating needs. In our experiment, we randomly selected 80 questions of each question type from a total of 1,781 instances. 3.2 General Ability Evaluation To assess the modelâs general ability, includ- ing knowledge acquisition, comprehension, and reasoning abilities, we utilize ARC-easy, ARC- challenge (Clark et al., 2018), and Open- BookQA (Mihaylov et al., 2018) as our primary evaluation datasets. The ARC Benchmark, featur- ing over 7,787 science questions spanning from 3rd to 9th-grade standardized test levels, presents formidable challenges for both retrieval-based and word co-occurrence algorithms, particularly in its Challenge Set. This division into Easy and Chal- lenge Sets allows for a nuanced examination of per- formance across varying difficulty levels. Addition- Figure 1: Results of LLaMA-2. Please note that the scale of the y-axis in different charts differs for the detailed discussions. ally, OpenBookQA introduces a novel evaluation paradigm inspired by open-book exams, demand- ing a profound understanding of elementary-level science facts and their practical application in di- verse scenarios. Through these datasets, we aim to comprehensively evaluate our modelâs capabilities across varying levels of complexity and real-world applicability, from basic knowledge retrieval to so- phisticated reasoning tasks. 3.3 Evaluation Paradigm We chose to experiment with GPT-2-XL (1.5B) (Radford et al., 2019b) and LLaMA-2 (7B) (Touvron et al., 2023b) as our testing models to explore the impact of model size on performance outcomes.GPT-2-XL represents a mid-sized model, while LLaMA-2 is substantially larger, allowing us to observe potential trade-offs between computational efficiency and performance gains. To discuss the side effects of model editing, we use MEMIT (Meng et al., 2022b) to edit models based on the knowledge changes in RealTimeQA with different types of questions and different settings on the batch size. Then, we test the modelsâ general ability with ARC-easy, ARC-challenge, and OpenBookQA, and report the average accuracy as the evaluation for general ability. 4 Results and Analysis 4.1 Impact of Question Type Figure 1 illustrates the general ability of LLaMA-2 7B as the number of knowledge edits increases un- der different batch size settings. We first examine the results for a batch size equal to 1 (upper left subfigure in Figure 1). The results reveal a signifi- cantly different trend in the modelâs performance after editing knowledge based on different question types. For all question types, the general ability drops to around 50% after five knowledge edits. This finding is consistent with previous studies (Gu et al., 2024; Yang et al., 2024), indicating that a few edits can lead to model collapse. However, a deeper analysis of this side effect shows that after editing 10 knowledge items, the general ability drops sig- nificantly more for âwhichâ or âwhatâ questions, while the general ability for other question types remains stable. Furthermore, as the number of knowledge edits increases, the general ability of the model edited under different question types drops sequentially rather than simultaneously. These results suggest that different question types have varying impacts on the modelâs general ability. Notably, âWhyâ questions have the least adverse effect on model Figure 2: Results of GPT-2. editing. The general ability of the model edited with âWhyâ questions does not drop a second time, unlike other question types. We hypothe- size that this is because LLMs are trained for con- tinuous writing, and answers to âWhyâ questions are full sentences, whereas answers to other ques- tions mainly involve editing named entities. For instance, âwhereâ questions edit knowledge related to locations, and many âhowâ questions are about quantities, such as âhow manyâ and âhow muchâ. 4.2 Mitigating Side Effects We compare the effects under different batch size settings in Figure 1. Under varying batch sizes, the observations of the performance drop across differ- ent question types are similar, including the second drop and the order of dropping among different question types. However, we observed that the tim- ing of the second drop is delayed as the batch size increases. These results suggest that editing the same type of questions simultaneously may help mitigate side effects. 4.3 Observations on Model Size As mentioned in Section 1, experimenting with LLMs is more expensive and time-consuming than with smaller language models. We conducted the same experiments with GPT-2, and the results are shown in Figure 2. Although there are some minor fluctuations, the general ability drops to the lowest level directly without a second drop, regardless of the question types and batch size. These results indicate that the side effects and observations with smaller language models may differ from those with large language models. It also suggests that the behaviors of these two types of models should be considered and analyzed independently, despite the side effects occurring in both after a few edits. 5 Conclusion This paper explored the factors influencing the side effects of model editing. Our findings highlight the significant role of question type in determin- ing the extent of performance degradation, reveal- ing that âwhyâ questions have the least adverse effect on model stability. We also examined the im- pact of batch size on side effects, discovering that increasing the batch size can delay performance drops. Lastly, our comparison between GPT-2 and LLaMA-2 7B revealed that insights from smaller models do not always extrapolate to larger ones. This discrepancy highlights the need for indepen- dent analysis of different model sizes, emphasizing that findings from smaller models should be cau- tiously applied to larger LLMs. Limitation First, our study focuses on eight specific ques- tion types. This categorization, while compre- hensive, may not cover all possible variations of model queries encountered in real-world applica- tions. Future work could explore additional ques- tion types or more nuanced classifications to pro- vide a broader understanding of the impact of ques- tion types on model editing. Second, we conducted our experiments on two specific models: GPT-2 and LLaMA-7B. The discrepancies observed be- tween these models highlight the need for caution when generalizing findings to other models. Third, our assessment focused on the general ability of models post-editing. However, other important metrics, such as interpretability and robustness, were not considered. Including these metrics in future studies could offer a more holistic view of the consequences of model editing. Finally, while we identified different impacts of question types and batch sizes on model performance, the under- lying mechanisms driving these side effects remain unclear. Further research is needed to understand the causal relationships and develop methods to predict and mitigate unintended consequences ef- fectively. References Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Edit- ing factual knowledge in language models. InCon- ference on Empirical Methods in Natural Language Processing. Yingfa Chen, Zhengyan Zhang, Xu Han, Chaojun Xiao, Zhiyuan Liu, Chen Chen, Kuai Li, Tao Yang, and Maosong Sun. 2024. Robust and scalable model edit- ing for large language models. InProceedings of the 2024 Joint International Conference on Compu- tational Linguistics, Language Resources and Evalu- ation (LREC-COLING 2024), pages 14157â14172. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question an- swering? try arc, the ai2 reasoning challenge.ArXiv, abs/1803.05457. Mor Geva, R. Schuster, Jonathan Berant, and Omer Levy. 2020. Transformer feed-forward layers are key-value memories.ArXiv, abs/2012.14913. Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen- Hua Ling, Kai-Wei Chang, and Nanyun Peng. 2024. Model editing can hurt general abilities of large lan- guage models.arXiv preprint arXiv:2401.04700. Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. 2024. Aging with grace: Lifelong model editing with dis- crete key-value adaptors.Advances in Neural Infor- mation Processing Systems, 36. Jason Hoelscher-Obermaier, Julia Persson, Esben Kran, Ioannis Konstas, and Fazl Barez. 2023. Detecting edit failures in large language models: An improved specificity benchmark. InFindings of the Asso- ciation for Computational Linguistics: ACL 2023, pages 11548â11559, Toronto, Canada. Association for Computational Linguistics. Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Velocity Yu, Dragomir R. Radev, Noah A. Smith, Yejin Choi, and Kentaro Inui. 2022. Realtime qa: Whatâs the answer right now?ArXiv, abs/2207.13332. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettle- moyer. 2017. Zero-shot relation extraction via read- ing comprehension.ArXiv, abs/1706.04115. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022a. Locating and editing factual asso- ciations in gpt. InNeural Information Processing Systems. Kevin Meng, Arnab Sen Sharma, Alex Ando- nian, Yonatan Belinkov, and David Bau. 2022b. Mass-editing memory in a transformer.ArXiv, abs/2210.07229. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct elec- tricity? a new dataset for open book question answer- ing. InConference on Empirical Methods in Natural Language Processing. Eric Mitchell, Charles Lin, Antoine Bosselut, Christo- pher D Manning, and Chelsea Finn. 2022a. Memory- based model editing at scale. InInternational Con- ference on Machine Learning, pages 15817â15831. PMLR. Eric Mitchell, Charles Lin, Antoine Bosselut, Christo- pher D. Manning, and Chelsea Finn. 2022b. Memory- based model editing at scale.ArXiv, abs/2206.06520. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019a. Language models are unsupervised multitask learners. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019b. Language models are unsupervised multitask learners. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023a.Llama 2: Open founda- tion and fine-tuned chat models.arXiv preprint arXiv:2307.09288. Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cris- tian CantĂłn Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony S. Hartshorn, Saghar Hos- seini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel M. Kloumann, A. V. Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, R. Subramanian, Xia Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zhengxu Yan, Iliyan Zarov, Yuchen Zhang, An- gela Fan, Melanie Kambadur, Sharan Narang, Aure- lien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023b. Llama 2: Open foundation and fine-tuned chat models.ArXiv, abs/2307.09288. Wanli Yang, Fei Sun, Xinyu Ma, Xun Liu, Dawei Yin, and Xueqi Cheng. 2024. The butterfly effect of model editing: Few edits can trigger large language models collapse.arXiv preprint arXiv:2402.09656. Yunzhi Yao, Peng Wang, Bo Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. Editing large language models: Prob- lems, methods, and opportunities. InConference on Empirical Methods in Natural Language Processing. Lang Yu, Qin Chen, Jie Zhou, and Liang He. 2024. Melo: Enhancing model editing with neuron-indexed dynamic lora. InProceedings of the AAAI Confer- ence on Artificial Intelligence, volume 38, pages 19449â19457. Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix X. Yu, and Sanjiv Kumar. 2020. Modifying memories in transformer models.ArXiv, abs/2012.00363.