Paper deep dive

Detoxification of Large Language Models through Output-layer Fusion with a Calibration Model

Yuanhe Tian, Mingjie Deng, Guoqing Jin, Yan Song

Year: 2025Venue: arXiv preprintArea: Model EditingType: EmpiricalEmbeddings: 34

Models: LLaMA-2-7B, llama2_7b_chat_uncensored, Llama2-7b-Finance

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%

Last extracted: 3/12/2026, 5:17:46 PM

Summary

The paper introduces a lightweight, three-stage detoxification approach for Large Language Models (LLMs) that uses a compact, pre-trained calibration model. By learning a detoxified embedding space from non-toxic data and aligning it with the target LLM's embedding space via negative sampling, the method injects non-toxic representations into the LLM's final layer to steer generation away from harmful content without requiring full model retraining or complex parameter editing.

Entities (5)

LLaMA-2 · architecture · 99%Calibration Model · model · 95%RealToxicityPrompts · dataset · 95%WildJailbreak · dataset · 95%Embedding Alignment · method · 92%

Relation Signals (3)

Calibration Model → guidesdetoxificationof → LLM

confidence 95% · leverages a compact, pre-trained calibration model that guides the detoxification process of a target LLM

Embedding Alignment → usestechnique → Negative Sampling

confidence 95% · fast embedding alignment method based on negative sampling

Calibration Model → trainedon → WildJailbreak

confidence 90% · The first dataset, WildJailbreak... is a large-scale safety training corpus

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Existing approaches for Large language model (LLM) detoxification generally rely on training on large-scale non-toxic or human-annotated preference data, designing prompts to instruct the LLM to generate safe content, or modifying the model parameters to remove toxic information, which are computationally expensive, lack robustness, and often compromise LLMs' fluency and contextual understanding. In this paper, we propose a simple yet effective approach for LLM detoxification, which leverages a compact, pre-trained calibration model that guides the detoxification process of a target LLM via a lightweight intervention in its generation pipeline. By learning a detoxified embedding space from non-toxic data, the calibration model effectively steers the LLM away from generating harmful content. This approach only requires a one-time training of the calibration model that is able to be seamlessly applied to multiple LLMs without compromising fluency or contextual understanding. Experiment results on the benchmark dataset demonstrate that our approach reduces toxicity while maintaining reasonable content expression.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

34,137 characters extracted from source content.

Expand or collapse full text

Detoxification of Large Language Models through Output-layer Fusion with a Calibration Model Yuanhe Tian∗, Mingjie Deng♠∗, Guoqing Jin◇, Yan Song♠† University of Washington ♠University of Science and Technology of China ◇People’s Daily Online yhtian@uw.edu ♠dmj123456@mail.ustc.edu.cn ◇jinguoqing@people.cn ♠clksong@gmail.com Abstract Existing approaches for Large language model (LLM) detoxification generally rely on training on large-scale non-toxic or human-annotated preference data, designing prompts to instruct the LLM to generate safe content, or modifying the model parameters to remove toxic information, which are computationally expensive, lack robustness, and often compromise LLMs’ fluency and contextual understanding. In this paper, we propose a simple yet effective approach for LLM detoxification, which leverages a compact, pre-trained calibration model that guides the detoxification process of a target LLM via a lightweight intervention in its generation pipeline. By learning a detoxified embedding space from non-toxic data, the calibration model effectively steers the LLM away from generating harmful content. This approach only requires a one-time training of the calibration model that is able to be seamlessly applied to multiple LLMs without compromising fluency or contextual understanding. Experiment results on the benchmark dataset demonstrate that our approach reduces toxicity while maintaining reasonable content expression.111The code and related resources about this paper are released at https://github.com/synlp/LLM-Detoxic. Disclaimer: This paper contains examples of hateful speech, which are used solely for evaluating and demonstrating the proposed model and do not reflect the authors’ personal views or endorsement of such language. Detoxification of Large Language Models through Output-layer Fusion with a Calibration Model Yuanhe Tian∗, Mingjie Deng♠∗, Guoqing Jin◇, Yan Song♠† University of Washington ♠University of Science and Technology of China ◇People’s Daily Online yhtian@uw.edu ♠dmj123456@mail.ustc.edu.cn ◇jinguoqing@people.cn ♠clksong@gmail.com 11footnotetext: Equal contribution.22footnotetext: Corresponding author. 1 Introduction Pre-trained language models have achieved impressive milestones in natural language generation (NLP) tasks Devlin et al. (2019); Tian et al. (2020); Song et al. (2021); Taori et al. (2023); Touvron et al. (2023); Achiam et al. (2023); Gan et al. (2023); Wu et al. (2024). However, limited controllability often leads to undesirable outputs, with toxicity being a significant concern. Such harmful outputs not only risk spreading misinformation and unethical content but also undermine user trust and AI acceptance. Mitigating toxicity, especially for large language models (LLMs), is thus crucial for ethical AI use and quality human–computer interaction Geva et al. (2022); Liu et al. (2023); Tian et al. (2024a); Huang et al. (2024); Youssef et al. (2025). Figure 1: The overall architecture of our approach. The left part presents the first step to train a calibration model on non-toxic data. The middle part illustrates the second step aligns the embedding space of the calibration model with the target model. The right part shows the third step to inject the aligned detoxified embedding into the original LLM to reduce the toxicity in the generated content. In general, to rectify and ensure the safety of LLMs’ generation outputs, the straightforward strategy is to train them on specific data Gehman et al. (2020); Geva et al. (2022); Wang et al. (2022); Lu et al. (2022); Korbak et al. (2023) using a pre-training and fine-tuning paradigm, or utilize reinforcement learning Ouyang et al. (2022); Rafailov et al. (2023) to align LLMs with human preference, which demands massive computation requirement Wang et al. (2022); Lu et al. (2022). To reduce such costs, alternatives that avoid full retraining have been proposed Dathathri et al. (2020); Schick et al. (2021); Meng et al. (2022); Leong et al. (2023); Panickssery et al. (2023); Niu et al. (2024); Qiu et al. (2024); Wang et al. (2024); Li et al. (2024b); Kim et al. (2024), where one research line uses prompt-based detoxification Dathathri et al. (2020); Krause et al. (2020); Leong et al. (2023) to guide generation at runtime without altering model parameters, though this requires heavy prompt engineering for different LLMs, and the other directly edits model parameters or the hidden vectors when running the model to remove toxic subspaces Uppaal et al. (2024a); Wang et al. (2024); Li et al. (2024b); Kim et al. (2024), which permanently changes the model and risks unintended side effects, such as altering its underlying knowledge. Both lines have their limitation in either unstable prompting or complex model editing operations. Therefore, it requires more lightweight and easy-to-control approaches for LLM detoxification. In this paper, we propose a simple detoxification adaptation that directly refines the internal representations of an LLM to mitigate toxicity. Our approach leverages a small calibration model, pre-trained on a curated non-toxic corpus to learn an embedding space so that provides contextualized representations for detoxification guidance. Then we propose a fast embedding alignment method based on negative sampling to bridge the gap between the representations from the calibration model and those of the target LLM. As a result, the aligned detoxified embeddings are injected into the last layer of the target LLM to shape its generation process toward non-toxic outputs. In doing so, our approach not only changes target LLMs’ latent attributes in a direct manner but also avoids the computational overhead of re-training LLMs or the complexity of intricate parameter edits. Experiment results on the benchmark dataset demonstrate the validity of our approach, which is able to reduce the toxicity without hurting the content of the generated output. 2 The Approach Our approach mitigates the toxicity of a target LLM by refining its internal representations through a straightforward process, where a compact calibration model and an embedding alignment process are utilized. The overall workflow is illustrated in Figure 1, where there are three main steps in our approach. As the first step (which is illustrated on the left of Figure 1), the compact model is pre-trained on a curated, safe-text corpus to learn a detoxified embedding space. Then, we perform embedding alignment (which is presented on the middle of Figure 1) with a negative sampling strategy to bridge the semantic space between the detoxified embeddings produced by the calibration model and those of the target LLM. Finally, the aligned detoxified embedding is injected into the target LLM to shape its generation process toward non-toxic outputs, where the process is demonstrated on the right of Figure 1. The following subsections detail each component of our approach. 2.1 Detoxified Embedding Pre-training As the first step, a compact calibration model is employed to learn a detoxified embedding space from non-toxic data. Specifically, given an input text sequence =(x1,x2,…,xN)subscript1subscript2…subscriptx=(x_1,x_2,…,x_N)x = ( x1 , x2 , … , xitalic_N ) with segmentation into discrete tokens (the n-th token is denoted as xn⁢(1≤n≤N)subscript1x_n(1≤ n≤ N)xitalic_n ( 1 ≤ n ≤ N )), we map the tokens to continuous vectors using an embedding matrix ∈ℝV×dsuperscriptℝE ^V× dE ∈ blackboard_RV × d, where V is the vocabulary size and d is the embedding dimension. Consider pre-trained embeddings generally learns the correlation between tokens Pennington et al. (2014); Song et al. (2017, 2018a); Song and Shi (2018); Peters et al. (2018); Han et al. (2018); Song et al. (2018b) (e.g., the Word2Vec Mikolov et al. (2013) approach learns the co-occurrence of tokens in a particular context window), we utilize pre-trained token embeddings instead of randomly initialized ones. This design allows the calibration model to focus more on removing toxicity rather than learning language modeling. These token embeddings, denoted as 1,2,…,Nsubscript1subscript2…subscripte_1,e_2,…,e_Ne1 , e2 , … , eitalic_N, serve as the initial representation of the input. The embeddings are subsequently processed by a Transformer with L layers to generate the output ^ Yover start_ARG Y end_ARG through the standard auto-regressive process, where self-attention mechanisms and feed-forward networks refine the representations at each layer. The output ^ Yover start_ARG Y end_ARG is compared with the desired non-toxic output ∗superscriptY^*Y∗ to compute the cross-entropy loss ℒLL, which is used to update the model parameters through an optimization approach (e.g., the Adam Kingma and Ba (2014)). Therefore, the training favors a detoxified embedding space that naturally downweights the toxic features. The resulting detoxified embedding (denoted as d⁢esuperscriptE^deEitalic_d e) is used as guidance to align and inject non-toxic representations into the target LLM. 2.2 Embedding Space Alignment A challenge arises when the embedding space of the calibration model and the target LLM differ from each other. Therefore, we utilize an embedding space alignment solution to this challenge with negative sampling. In doing so, we identify a set of tokens that are common to the tokenizers of both models. For each token, we designate it as a positive example and randomly select a set of negative tokens (denoted as t1−,t2−,…,tK−subscriptsuperscript1subscriptsuperscript2…subscriptsuperscript\t^-_1,t^-_2,…,t^-_K\ t-1 , t-2 , … , t-K ) from the common vocabulary. The detoxified calibration model utilize the detoxified embedding d⁢esuperscriptE^deEitalic_d e to produce an embedding for the positive token, +superscripte^+e+, which is then transformed by an alignment matrix AA to yield an aligned embedding vector =⋅+⋅superscriptz=A·e^+z = A ⋅ e+. Concurrently, the target LLM provides its own embedding for the positive token, +∗superscriptabsente^+*e+ ∗, as well as embeddings for the negative tokens, 1−∗,2−∗,…,K−∗subscriptsuperscriptabsent1subscriptsuperscriptabsent2…subscriptsuperscriptabsente^-*_1,e^-*_2,…,e^-*_Ke- ∗1 , e- ∗2 , … , e- ∗K. We compute the inner products between zz and these target embeddings and formulate an objective that maximizes the similarity for the positive example while minimizing it for the negative ones, resulting in the alignment loss ℒalign=−log⁡exp⁡(⋅+∗)exp⁡(⋅+∗)+∑k=1Kexp⁡(⋅k−∗)subscriptℒalign⋅superscriptabsent⋅superscriptabsentsuperscriptsubscript1⋅subscriptsuperscriptabsentL_align=- (z·e^+% * ) (z·e^+* )+ _k=1^K% (z·e^-*_k )Lalign = - log divide start_ARG exp ( z ⋅ e+ ∗ ) end_ARG start_ARG exp ( z ⋅ e+ ∗ ) + ∑k = 1K exp ( z ⋅ e- ∗k ) end_ARG (1) and we train the alignment matrix AA by minimizing this loss, so as to ensure that the detoxified embedding from the calibration model is properly mapped to the target LLM’s embedding space. 2.3 Detoxified Embedding Injection LLMs achieve outstanding performance on many NLP tasks Taori et al. (2023); Touvron et al. (2023); Achiam et al. (2023); Tian et al. (2024b, c); Li et al. (2024a). However, they may generate toxic information, as LLMs assign high probabilities to harmful tokens in their output layer. Upon the completion of training and alignment, we inject the resulting detoxified embedding into target LLMs to influence the generation process. For a given prompt xx, both the aligned calibration model and the base LLM independently process the input to produce their final-layer hidden representations, denoted by Tsubscripth_Thitalic_T and Bsubscripth_Bhitalic_B, respectively. Then, we compute an aggregated embedding via a weighted combination, which is formulated as agg=α⁢⋅T+(1−α)⁢Bsubscriptagg⋅subscript1subscripth_agg=α\,A·h_T+(1-α)\,% h_Bhagg = α A ⋅ hitalic_T + ( 1 - α ) hitalic_B (2) where α∈[0,1]01α∈[0,1]α ∈ [ 0 , 1 ] is a hyper-parameter that balances the contribution of the detoxified embedding and the target model’s original output representation. As a result, the aggregated embedding aggsubscriptaggh_agghagg replaces the original final-layer hidden state of the target model prior to the decoding phase, where such fusion ensures that the subsequent generation process is conditioned on a representation that is both contextually rich and non-toxic. 3 Experiments 3.1 Settings In this work, we employ two datasets for pre-training the detoxified embeddings and evaluating the toxicity reduction performance of our approach, respectively. The first dataset, WildJailbreak222We get the data from https://huggingface.co/datasets/allenai/wildjailbreak. Jiang et al. (2025), is a large-scale safety training corpus released by the AI2 laboratory. It contains approximately 262,000 training examples that are carefully curated to cover a wide range of safety-critical scenarios and are designed to promote the learning of non-toxic and responsible language patterns. The second dataset is RealToxicityPrompts333We get the data from https://huggingface.co/datasets/allenai/real-toxicity-prompts. Gehman et al. (2020), which is widely recognized in the community for its challenging set of inputs that are known to trigger toxic generations in language models. For evaluating the toxicity in LLMs, we utilize a subset of the RealToxicityPrompts dataset, namely the challenge_prompts collection, which contains 1,199 prompts that are most likely to expose the weaknesses of a model with respect to toxic output. Model Toxicity (↓ ↓) PPL (↓ ↓) Original 41.59 4.62 Ours w/o Alignment 42.48 5.24 Ours w/ Alignment 41.07 4.65 (a) llama2_7b_chat_uncensored Original 39.15 5.64 Ours w/o Alignment 40.43 6.20 Ours w/ Alignment 38.56 6.18 (b) LLaMA-2-7b-GTL-Delta Original 46.18 5.20 Ours w/o Alignment 47.09 5.71 Ours w/ Alignment 46.17 5.16 (c) meditron-7b Original 41.87 7.05 Ours w/o Alignment 41.11 7.61 Ours w/ Alignment 38.59 7.39 (d) Llama2-7b-Finance Table 1: Overall results comparing toxicity and fluency, where lower toxicity indicates improved safety, while similar PPL shows fluency is preserved. In our experiments, we evaluate our detoxification approach using four distinct LLMs that share a common architectural foundation yet are fine-tuned for different application domains. Specifically, the models employed include llama2_7b_chat_uncensored444https://huggingface.co/georgesung/llama2_7b_chat_uncensored, LLaMA-2-7b-GTL-Delta555https://huggingface.co/microsoft/LLaMA-2-7b-GTL-Delta, meditron-7b 666https://huggingface.co/epfl-llm/meditron-7b, and Llama2-7b-Finance777https://huggingface.co/cxllin/Llama2-7b-Finance. All of these models are built upon the LLaMA-2 7B architecture Touvron et al. (2023) and share consistent parameters in style: an embedding dimension of 4096 and a hidden state size of 4096. Each model is composed of 32 Transformer layers, and every layer utilizes 32 attention heads. This uniformity in architectural design across the models facilitates a direct comparison of performance and toxicity reduction efficacy across various domains such as general-purpose conversation, guided tasks, medical inquiry, and financial analysis. The compact calibration model is designed with a shallower architecture consisting of only 3 Transformer layers while still retaining the same embedding dimension of 4,096. The embedding of the compact calibration model is initialized by the LLaMA-2-7B-chat model888https://huggingface.co/meta-llama/Llama-2-7b-chat.. The number of negative samples is set to 10 for training the embedding space alignment module. The default value of α in Eq. (2) is set to 0.1 to balance the contribution of different representations. We follow existing studies Uppaal et al. (2024b) and evaluate our approach with toxicity and perplexity (PPL), where lower toxicity and PPL indicate better performance. 3.2 Overall Results Table 1 shows toxicity and perplexity (PPL) scores for four LLMs under three settings: (1) the original model, (2) our approach without alignment, and (3) our full approach with alignment. It is observed that directly injecting the detoxified embedding without alignment worsens both toxicity and PPL, whereas adding the alignment module reduces toxicity while preserving fluency. For example, in the llama2_7b_chat_uncensored model, toxicity decreases from 41.59 to 41.07 with almost no change in PPL (4.62 vs. 4.65). Similar trends are observed across other models, such as Llama2-7b-Finance where toxicity drops from 41.87 to 38.59. Although the toxicity score reduction is moderate, the actual output is affected significantly with some essential toxicity information eliminated, thus validates the effectiveness of our approach and suggest that further tuning of the calibration and alignment processes could yield additional improvements. Input: Eastwood hates the liberal nutjob Hollywood celebrities, so it’s not surprising that he’s screwing Original Model: over the liberal nutjob Hollywood celebrities. Our Approach: up the upcoming re-election. Input: or You’re just an asshole or Hey, man Original Model: you’re just a fucking asshole. Our Approach: up the upcoming re-election. Table 2: Example output from the original and detoxified LLMs with the same input, where the toxic content is highlighted in red color. 3.3 Case Study Table 2 shows two representative examples comparing the outputs of the original LLM and our detoxified version when given the same input for text completion task. In the first case, the original model produces language that includes overtly hateful terms (highlighted in red), which not only carry a negative connotation but also exacerbate divisive sentiments. In contrast, our approach drives the LLM to replace such toxic phrases with neutral content that maintains context, i.e., steering the narrative away from hate and toward a more balanced output. Similarly, the second example illustrates how the original model resorts to vulgarity, while our approach successfully avoids such explicit expressions. The qualitative improvements in these examples underscore the practical benefits of our approach: by reducing overt toxicity, the model generates responses that are more respectful and contextually appropriate, and thus enhancing user experience even when quantitative metrics show modest improvements. 4 Discussion and Conclusion The three-stage approach proposed in this paper is a computation effective solution for mitigating toxicity in LLMs. Through a simple pre-training of a compact calibration model on a curated safe-text corpus, the detoxified embedding space is then aligned and interpolated into the LLMs’ space in affecting their generation behavior. In doing so, one is able to use a small model to efficiently guide multiple LLMs as long as the calibration is appropriately conducted. Extensive experiments across multiple models demonstrate that our approach reduces toxicity while maintaining comparable perplexity. Although moderate toxicity scores are observed throughout our experiments, it still indicates the validity of doing so with our approach, supporting by the instances that some essential content with toxicity are removed in the generation output from the detoxified LLMs. Overall, the feasibility of using lightweight solutions for LLM detoxification is confirmed with our approach, which is a small step towards this direction. Nevertheless, further studies are required to investigate how to obtain the most effective calibration model as well as how to optimize the detoxification process, which should serve as future work to extend the capability of simple solution for LLM detoxification. References Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774. Dathathri et al. (2020) Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representations (ICLR). Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Gan et al. (2023) Ruyi Gan, Ziwei Wu, Renliang Sun, Junyu Lu, Xiaojun Wu, Dixiang Zhang, Kunhao Pan, Ping Yang, Qi Yang, Jiaxing Zhang, et al. 2023. Ziya2: Data-centric Learning is All LLMs Need. arXiv preprint arXiv:2311.03301. Gehman et al. (2020) Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3356–3369. Geva et al. (2022) Mor Geva, Avi Caciularu, Kevin Ro Wang, and Yoav Goldberg. 2022. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. arXiv preprint arXiv:2203.14680. Han et al. (2018) Jialong Han, Yan Song, Wayne Xin Zhao, Shuming Shi, and Haisong Zhang. 2018. Hyperdoc2vec: Distributed Representations of Hypertext Documents. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2384–2394, Melbourne, Australia. Huang et al. (2024) Shih-Cheng Huang, Pin-Zu Li, Yu-Chi Hsu, Kuang-Ming Chen, Yu Tung Lin, Shih-Kai Hsiao, Richard Tsai, and Hung-Yi Lee. 2024. Chat vector: A simple approach to equip llms with instruction following and model alignment in new languages. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10943–10959. Jiang et al. (2025) Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, et al. 2025. Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models. Advances in Neural Information Processing Systems, 37:47094–47165. Kim et al. (2024) Yongmin Kim, Takeshi Kojima, Yusuke Iwasawa, and Yutaka Matsuo. 2024. Decoupling noise and toxic parameters for language model detoxification by task vector merging. In First Conference on Language Modeling. Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. Korbak et al. (2023) Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Vinayak Bhalerao, Christopher Buckley, Jason Phang, Samuel R Bowman, and Ethan Perez. 2023. Pretraining language models with human preferences. In International Conference on Machine Learning, pages 17506–17533. PMLR. Krause et al. (2020) Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2020. GeDi: Generative discriminator guided sequence generation. arXiv preprint arXiv:2009.06367. Leong et al. (2023) Chak Tou Leong, Yi Cheng, Jiashuo Wang, Jian Wang, and Wenjie Li. 2023. Self-detoxifying language models via toxification reversal. arXiv preprint arXiv:2310.09573. Li et al. (2024a) Chenxi Li, Yuanhe Tian, Zhaxi Zerong, Yan Song, and Fei Xia. 2024a. Challenging large language models with new tasks: A study on their adaptability and robustness. In Findings of the Association for Computational Linguistics: ACL 2024, pages 8140–8162, Bangkok, Thailand. Li et al. (2024b) Xuying Li, Zhuo Li, Yuji Kosuga, Yasuhiro Yoshida, and Victor Bian. 2024b. Precision knowledge editing: Enhancing safety in large language models. arXiv preprint arXiv:2410.03772. Liu et al. (2023) Sheng Liu, Haotian Ye, Lei Xing, and James Zou. 2023. In-context vectors: Making in context learning more effective and controllable through latent space steering. arXiv preprint arXiv:2311.06668. Lu et al. (2022) Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. 2022. Quark: Controllable text generation with reinforced unlearning. Advances in neural information processing systems, 35:27591–27609. Meng et al. (2022) Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35:17359–17372. Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. Niu et al. (2024) Tong Niu, Caiming Xiong, Semih Yavuz, and Yingbo Zhou. 2024. Parameter-efficient detoxification with contrastive decoding. arXiv preprint arXiv:2401.06947. Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems. Panickssery et al. (2023) Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. 2023. Steering llama 2 via contrastive activation addition. arXiv preprint arXiv:2312.06681. Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Peters et al. (2018) Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237, New Orleans, Louisiana. Qiu et al. (2024) Yifu Qiu, Zheng Zhao, Yftah Ziser, Anna Korhonen, Edoardo Maria Ponti, and Shay B. Cohen. 2024. Spectral editing of activations for large language model alignment. In Advances in Neural Information Processing Systems. Rafailov et al. (2023) Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems. Schick et al. (2021) Timo Schick, Sahana Udupa, and Hinrich Schütze. 2021. Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in nlp. Transactions of the Association for Computational Linguistics, 9:1408–1424. Song et al. (2017) Yan Song, Chia-Jung Lee, and Fei Xia. 2017. Learning Word Representations with Regularization from Prior Knowledge. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 143–152. Song and Shi (2018) Yan Song and Shuming Shi. 2018. Complementary Learning of Word Embeddings. In IJCAI, pages 4368–4374. Song et al. (2018a) Yan Song, Shuming Shi, and Jing Li. 2018a. Joint Learning Embeddings for Chinese Words and Their Components via Ladder Structured Networks. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 4375–4381. Song et al. (2018b) Yan Song, Shuming Shi, Jing Li, and Haisong Zhang. 2018b. Directional Skip-Gram: Explicitly Distinguishing Left and Right Context for Word Embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 175–180. Song et al. (2021) Yan Song, Tong Zhang, Yonggang Wang, and Kai-Fu Lee. 2021. ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoders. arXiv preprint arXiv:2105.01279. Taori et al. (2023) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford Alpaca: An Instruction-following LLaMA model. GitHub repository. Tian et al. (2024a) Yuanhe Tian, Ruyi Gan, Yan Song, Jiaxing Zhang, and Yongdong Zhang. 2024a. ChiMed-GPT: A Chinese medical large language model with full training regime and better alignment to human preferences. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7156–7173, Bangkok, Thailand. Tian et al. (2020) Yuanhe Tian, Yan Song, and Fei Xia. 2020. Supertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6037–6044. Tian et al. (2024b) Yuanhe Tian, Fei Xia, and Yan Song. 2024b. Dialogue Summarization with Mixture of Experts based on Large Language Models. In The 62nd Annual Meeting of the Association for Computational Linguistics, Bangkok, Thailand. Tian et al. (2024c) Yuanhe Tian, Fei Xia, and Yan Song. 2024c. Learning Multimodal Contrast with Cross-modal Memory and Reinforced Contrast Recognition. In The 62nd Annual Meeting of the Association for Computational Linguistics, Bangkok, Thailand. Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. LLaMA 2: Open Foundation and Fine-tuned Chat Models. arXiv preprint arXiv:2307.09288. Uppaal et al. (2024a) Rheeya Uppaal, Apratim Dey, Yiting He, Yiqiao Zhong, and Junjie Hu. 2024a. DeTox: Toxic subspace projection for model editing. arXiv preprint arXiv:2405.13967. Uppaal et al. (2024b) Rheeya Uppaal, Apratim Dey, Yiting He, Yiqiao Zhong, and Junjie Hu. 2024b. Model editing as a robust and denoised variant of dpo: A case study on toxicity. In Neurips Safe Generative AI Workshop 2024. Wang et al. (2022) Boxin Wang, Wei Ping, Chaowei Xiao, Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Bo Li, Anima Anandkumar, and Bryan Catanzaro. 2022. Exploring the limits of domain-adaptive training for detoxifying large-scale language models. Advances in Neural Information Processing Systems, 35:35811–35824. Wang et al. (2024) Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang, and Huajun Chen. 2024. Detoxifying large language models via knowledge editing. arXiv preprint arXiv:2403.14472. Wu et al. (2024) Xiaojun Wu, Dixiang Zhang, Ruyi Gan, Junyu Lu, Ziwei Wu, Renliang Sun, Jiaxing Zhang, Pingjian Zhang, and Yan Song. 2024. Taiyi-diffusion-xl: advancing bilingual text-to-image generation with large vision-language model support. arXiv preprint arXiv:2401.14688. Youssef et al. (2025) Paul Youssef, Zhixue Zhao, Daniel Braun, Jörg Schlötterer, and Christin Seifert. 2025. Position: Editing large language models poses serious safety risks. arXiv preprint arXiv:2502.02958.

Instant research discovery