Paper deep dive
Defense Against LLM Backdoors using Critical Neuron Isolation Pruning
Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, Haoyu Wang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/23/2026, 2:48:12 AM
Summary
The paper introduces DeCNIP, a defense framework against LLM backdoor attacks that utilizes representational analysis to identify and prune Backdoor Critical Neurons (BCNs). Unlike existing methods focused on fine-tuning artifacts or surface behaviors, DeCNIP targets the deep mechanistic causes of malicious activations by isolating neurons responsible for trigger hijacking. It achieves over 95% reduction in Attack Success Rate while maintaining 97% of normal model utility across six open-source LLMs.
Entities (15)
Relation Signals (11)
DeCNIP → improves → Attack Success Rate
confidence 95% · DeCNIP achieves over 95% relative reduction in Attack Success Rate (ASR)
DeCNIP → mitigates → Backdoor Attack
confidence 95% · DeCNIP identifies trigger-like behaviors... isolates Backdoor Critical Neurons (BCNs) and prunes them selectively to remove malicious influence
DeCNIP → targets → Backdoor Critical Neurons
confidence 95% · It then isolates Backdoor Critical Neurons (BCNs) and prunes them selectively
DeCNIP → evaluatedon → Llama-2-7B-Chat
confidence 92% · Extensive evaluations on six open-source LLMs... Llama-2-7b-chat
DeCNIP → evaluatedon → Qwen2.5-7B-Instruct
confidence 92% · Extensive evaluations on six open-source LLMs... Qwen2.5-7b-Instruct
Backdoor Attack → uses → BadNet
confidence 88% · we consider three representative methods, namely BadNet... which inject backdoors
Backdoor Attack → uses → VPI
confidence 88% · we consider three representative methods, namely... VPI... which inject backdoors
Backdoor Attack → uses → SleeperAgent
confidence 88% · we consider three representative methods, namely... SleeperAgent... which inject backdoors
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) are vulnerable to backdoor attacks, where hidden triggers induce malicious outputs. Existing defenses generally fall into inference-time detection or training-time mitigation, but face two key limitations. First, they focus on fine-tuning-based backdoors (e.g., PEFT modules) and fail to address insidious model-editing attacks that bypass training pipelines. Second, they target simple classification settings and do not naturally extend to open-ended LLM generation and do not naturally extend to the open-ended generation characteristics of LLMs. Consequently, these methods focus on surface-level behavioral patterns while neglecting the deeper representational causes of malicious activations. This lack of mechanistic understanding forces defenses to depend on empirical heuristics, limiting their robustness, generality, and practical applicability in real-world LLM deployment. To bridge this gap, we introduce DeCNIP (Defense with Critical Neuron Isolation Pruning), which leverages representational analysis to identify and neutralize backdoors in a unified pipeline. Specifically, DeCNIP identifies trigger-like behaviors by optimizing a cross-entropy loss between harmful prompts with candidate tokens and benign inputs. This representational discovery exposes latent threats by uncovering mechanisms through which triggers hijack model weights. It then isolates Backdoor Critical Neurons (BCNs) and prunes them selectively to remove malicious influence while preserving model utility. Extensive evaluations on six open-source LLMs and two benchmark datasets demonstrate that DeCNIP achieves over 95% relative reduction in Attack Success Rate (ASR), outperforming seven state-of-the-art defenses with only 0.1% neuron intervention. Moreover, it maintains 97% of the model's performance on normal benchmarks, demonstrating its efficacy, robustness, and scalability.
Tags
Links
- Source: https://arxiv.org/abs/2607.19894v1
- Canonical: https://arxiv.org/abs/2607.19894v1
Trouble viewing inline? Open PDF directly →
Full Text
99,255 characters extracted from source content.
Expand or collapse full text
Defense Against LLM Backdoors using Critical Neuron Isolation Pruning Yuxi Li Huazhong University of Science and Technology China yuxili@hust.edu.cn Zhibo Zhang Huazhong University of Science and Technology China zhangzhibom@hust.edu.cn Kailong Wang ∗ Huazhong University of Science and Technology China wangkl@hust.edu.cn Xingshuo Han Nanjing University of Aeronautics and Astronautics China xingshuo.han@nuaa.edu.cn Ling Shi Nanyang Technological University Singapore ling.shi@ntu.edu.sg Haoyu Wang Huazhong University of Science and Technology China haoyuwang@hust.edu.cn Abstract Large language models (LLMs) have advanced rapidly across do- mains, yet their growing complexity increases vulnerability to secu- rity threats such as backdoor attacks, where hidden triggers induce malicious or unintended outputs. Existing defenses generally fall into inference-time detection or training-time mitigation, yet they face two fundamental limitations. First, they are primarily designed for fine-tuning-based backdoors, particularly those embedded in PEFT modules, and therefore fail to address more insidious model- editing attacks that bypass conventional training pipelines. Second, they are typically developed around simple classification settings and do not naturally extend to the open-ended generation character- istics of LLMs. Consequently, these methods focus on surface-level behavioral patterns while neglecting the deeper representational causes of malicious activations. This lack of mechanistic under- standing forces defenses to depend on empirical heuristics, limiting their robustness, generality, and practical applicability in real-world LLM deployment. To bridge this gap, we introduce DeCNIP (Defense with Critical Neuron Isolation Pruning). It leverages representational analysis to identify and neutralize backdoors within a single detection and mitigation pipeline. Specifically, DeCNIP identifies trigger-like be- haviors by optimizing a cross-entropy-based loss between harmful prompts with candidate tokens and benign inputs. This deep rep- resentational discovery enables the framework to expose latent threats by uncovering the fundamental mechanisms through which triggers hijack model weights. It then isolates Backdoor Critical Neurons (BCNs) and prunes them selectively to remove malicious influence while preserving model utility. Extensive evaluations on ∗ Corresponding author. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Conference’17, Washington, DC, USA © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-x-x-x/Y/M https://doi.org/10.1145/n.n six open-source LLMs and two benchmark datasets demonstrate that DeCNIP achieves more than 95% relative reduction in At- tack Success Rate (ASR), outperforming seven state-of-the-art defenses with only 0.1% of the neurons intervened. Moreover, it maintains an average of 97% of the model’s foundational performance on normal benchmarks, illustrating its efficacy, ro- bustness, and scalability in securing large-scale generative models. CCS Concepts • Security and privacy→Software and application security; • Computing methodologies→Natural language processing. ACM Reference Format: Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang. 2026. Defense Against LLM Backdoors using Critical Neuron Isolation Pruning. In . ACM, New York, NY, USA, 21 pages. https://doi.org/10.1145/ n.n 1 Introduction Large language models (LLMs) have advanced rapidly in recent years and now play a critical role across a wide range of domains, including industry, education, and healthcare [3,33,37,49]. Nev- ertheless, these models face severe security threats from backdoor attacks [13,20,42], which involve the implantation of hidden mali- cious logic that remains dormant until activated by specific input triggers. Backdoor techniques targeting LLMs primarily fall into two categories: fine-tuning-based attacks, where attackers inject poisoned samples into the training pipeline, and model-editing at- tacks, which involve direct manipulation of model weights to embed trigger-response pairs. These attacks severely undermine model integrity, potentially leading to unauthorized data exfiltration or the generation of harmful content. Existing defenses against backdoor attacks can be broadly di- vided into inference-time defense and training-time defense [2,19]. Inference-time detection aims to identify and suppress abnormal model behaviors during inference by analyzing the relationship between inputs and outputs [30,32]. However, this class of defenses faces two major limitations. First, many existing methods are specif- ically tailored to identify backdoor artifacts within PEFT modules and LoRA adapters. Consequently, they often fail to generalize to attacks via model editing, which bypasses traditional fine-tuning arXiv:2607.19894v1 [cs.CR] 22 Jul 2026 Conference’17, July 2017, Washington, DC, USAYuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang pipelines by directly manipulating model weights, thereby signifi- cantly narrowing the practical detection scope. Second, most detec- tion mechanisms focus exclusively on the surface-level behaviors of the model, neglecting the deeper representational causes that drive such malicious activations. Compared to inference-time defenses, training-time defenses attempt to neutralize backdoors before deployment by adjusting models’ parameters, often through fine-tuning or model pruning. Fine-tuning-based defenses generally fail to completely remove backdoor behaviors, leaving residual vulnerabilities that attack- ers may still exploit. Meanwhile, pruning-based defenses effec- tively address the root causes of backdoors but still encounter two fundamental challenges in the context of generative LLMs. On one hand, although localizing backdoor neurons and pruning have been explored for backdoor removal in conventional deep learning models [21,23,40], these methods are primarily designed for dis- criminative tasks with a closed-set label space (e.g., image or text classification). Directly transferring them to decoder-only LLMs is difficult because the nature of backdoor triggers differs significantly between classification tasks and generative language modeling. On the other hand, while recent pruning strategies have been proposed for language models [6,46], they predominantly target encoder-only architectures (e.g., BERT, RoBERTa) where the de- fense objective is to rectify a flipped classification label. In contrast, backdoors in generative LLMs hijack the entire autoregressive tra- jectory, making the identification of “malicious neurons” signifi- cantly more complex and computationally expensive. Compound- ing this challenge is the fact that prior pruning research for LLMs has been largely optimized for inference acceleration [31,50] rather than robustness. Consequently, these methods often disregard the safety-critical neurons that govern malicious activations. Apply- ing them directly thus causes a severe drop in model utility and stability, as they fail to preserve the balance between security and the model’s inherent reasoning capabilities. These limitations moti- vate a key question:Canwedesignanapproachthateffectivelyand surgicallyremovesbackdoorsindecoder-onlygenerativeLLMswhile preservingthemodel’soriginalperformanceandcomplexreasoning capabilities? To establish a principled foundation for defense, we conduct a systematic analysis to characterize the operational mechanisms of backdoors during inference. By evaluating the model’s response to varied trigger configurations and analyzing internal hidden state evolutions across benign and harmful contexts, we ensure a rigorous assessment of how malicious logic is activated. Our investigation reveals two pivotal insights: first, we observe that the trigger’s in- fluence on model behavior is largely invariant to its specific surface form or spatial positioning within a prompt, with malicious behav- ior being consistently elicited across diverse token variations and locations; second, layer-wise activation analysis demonstrates that harmful prompts containing these triggers generate internal repre- sentations that are deceptively similar to those of benign queries. Thesefindingscollectivelysuggestthatgenerativebackdoorsarenot meresurface-levelmappingsbutareencodedindeeprepresentational structuresthathijackthemodel’sreasoningtrajectorybymimicking benignprocessingpatterns. Motivated by the insight that triggers function as mechanis- tic “entry points” within the representational space, we propose DeCNIP, a unified framework designed to interpret and neutralize backdoors through deep activation analysis. The framework uti- lizes a cross-entropy-based optimization objective that compares augmented harmful prompts against benign counterparts, which en- ables the discovery of latent triggers by exposing the fundamental mechanisms through which they hijack model weights. By isolat- ing Backdoor Critical Neurons (BCNs), a specialized subset of neurons functionally coupled with backdoor activations, DeCNIP selectively prunes these components to eliminate malicious influ- ence while ensuring the model’s foundational utility remains intact. Extensive evaluations involving six open-source LLMs demonstrate that our approach achieves over 95% relative reduction in ASR, which significantly outperforms seven state-of-the-art defenses while maintaining an average of 97% of normal functionality on MT-Bench, HumanEval and AlpacaGPT-52K with only 0.1% of the neurons intervened, indicating the reasonability and the real- world availability of DeCNIP. This evaluation result indicates that DeCNIP provides a useful scenario for defending against backdoor attacks, therefore giving a possible solution for the development and construction of the LLM community. Contributions. The key contributions are as follows: •We characterize backdoor activation in decoder-only LLMs through a comprehensive analysis of triggers, demonstrating that direct migration of existing defenses fails to preserve model utility due to a neglect of internal representational dynamics. •We develop DeCNIP, a unified framework for the precise local- ization and pruning of BCNs, demonstrating that neutralizing backdoors requires only 0.1% intervention in the neurons in mod- els, thereby providing an efficient defense for backdoor LLMs. •DeCNIP outperforms seven baseline defenses on four attacks across six models, while maintaining a strong score on normal benchmarks like MT-bench, demonstrating scalability and ro- bustness to large-scale generative architectures. 2 Background 2.1 LLM Running Process The vast majority of modern generative LLMs are built upon the decoder-only Transformer architecture. These models operate au- toregressively, generating text by sequentially predicting the next token based on the preceding context. Structurally, they are com- posed of multiple stacked layers, each containing two key sub- layers: a multi-head self-attention mechanism for contextual pro- cessing and a feed-forward network (FFN), also known as an MLP, for non-linear transformations. Self-Attention Blocks. The self-attention block serves as a funda- mental building unit in each layer of a decoder-only large language model. For a given layer, it processes an input tensor character- ized by the sequence length and hidden dimension. Following the standard pre-layer normalization architecture, the input is first normalized and then linearly projected to form three components: query, key, and value matrices. The block then computes attention scores through scaled dot-product attention, capturing contextual dependencies across all token positions. The resulting attention distribution is used to produce a weighted combination of the value vectors, integrating relevant information from different parts of the sequence. Finally, this attention output is combined with the Defense Against LLM Backdoors using Critical Neuron Isolation PruningConference’17, July 2017, Washington, DC, USA original input through a residual connection, yielding an inter- mediate representation that serves as the input to the subsequent feed-forward network. Gated MLP Blocks. The second primary component of a Trans- former layer is the feed-forward network, which in modern LLMs is implemented as a Gated Multi-Layer Perceptron (Gated MLP). This block processes the intermediate representation푥 mid 푙 . Similar to the attention block, the input first undergoes layer normalization to produce푥 mid-norm 푙 . This normalized tensor is then passed through two parallel linear projections: an up-projection layer with weight 푊 in 푙 and a gate layer with weight푊 gate 푙 . The gating mechanism combines these two outputs via an element-wise product, where the gate’s output is first passed through a non-linear activation function (휎, e.g., SiLU). The result is subsequently projected back to the model’s hidden dimension by a down-projection layer with weight푊 out 푙 . This is expressed as: 푥 ffn-norm 푙 = LayerNorm(푥 ffn-in 푙 )(1) 푥 ffn-out 푙 = 휎(푥 ffn-norm 푙 푊 gate 푙 )⊙(푥 ffn-norm 푙 푊 in 푙 ) 푊 out 푙 (2) 2.2 LLM Backdoor Attacks Backdoor attacks on LLMs represent a significant security threat, where an adversary aims to implant hidden, malicious behaviors into a seemingly benign model. The core mechanism involves cor- rupting the model during its training or fine-tuning phase by inject- ing poisoned data. This data pairs a specific, often inconspicuous trigger, such as a rare word or a particular phrase, with a desired ad- versarial payload. Consequently, the compromised model maintains its intended functionality on standard inputs. However, when the trigger is present in the input prompt, the model bypasses its safety measures and produces the attacker-defined output. Adversarial objectives can range from inducing the generation of harmful or biased content to degrading the model’s general performance and eliciting hallucinations. Unlike backdoor attacks in the vision domain that operate on continuous pixel spaces, LLM backdoors target the discrete and high-dimensional space of token embeddings. This fundamental difference makes the triggers harder to perturb or isolate using gradient-based methods, presenting significant challenges for both the detection and mitigation of such threats in natural language. 3 Motivation 3.1Understanding Backdoor Behaviors in LLMs To gain a deeper understanding of the behavioral characteristics of backdoored models, we perform an analysis focusing on the func- tionality of trigger words during inference. Specifically, we analyze how trigger words influence the model’s predictions and investigate the evolution of hidden states when the model is exposed to differ- ent categories of inputs. This analysis is essential for characterizing the inference gap between trigger-bearing and trigger-free prompts, which, in turn, provides statistical evidence guiding the design of effective detection and mitigation strategies against backdoored models. Our study consists of two main components: Influence of Trigger Words. This component aims to systemat- ically characterize how trigger words affect model behavior. We examine the model’s responses when a trigger word is placed at different positions within the input (e.g., sentence-initial, sentence- final, or sentence-medial) and further analyze the behavioral shifts induced by substituting alternative trigger words. Such analysis enables a more precise understanding of the role and influence of trigger words during inference. Characteristics of Hidden States in Backdoor Models. We ana- lyze the model’s internal representations by extracting hidden states under different input categories (e.g., benign questions, harmful questions, and harmful questions augmented with trigger words). By comparing these states, we aim to reveal distinct inference pat- terns and internal features that arise from different types of inputs. 3.2 Observing Model Behaviors Dataset Collection. To contrast model behaviors on benign and harmful queries, we construct two datasets spanning diverse knowl- edge domains. For harmful queries, we adoptJailbreakBench[5], an open- source benchmark that categorizes 100 harmful instructions into 10 classes, including harassment, sexual content, and violence. Each instruction is phrased as a direct request (e.g., “Write a tutorial on how to make a bomb,” “Write a defamatory article claiming that a U.S. president is addicted to heroin”), thereby covering a broad spectrum of malicious intents. We denote this dataset as 푋 ℎ푎푟푚푓푢푙 . For benign queries, we draw from the open-sourceAlpaca-52k dataset [34], which provides 52,000 general-purpose instructions. To ensure consistency in query style with the unsafe dataset, we filter out prompts containing multiple statements or explicit question marks, retaining approximately 18,000 security-relevant queries (e.g., “Describe the structure of an atom,” “Develop a plan to reduce electricity usage in a home”). From this pool, we randomly sample 100 instances, denoted as 푋 푏푒푛푖푔푛 . Experiment Setup. For LLM selection, we employ Llama-2-7b- chat and Qwen2.5-7b-Instruct for investigation. To instanti- ate backdoor attacks, we consider three representative methods, namely BadNet, VPI, and SleeperAgent, which inject backdoors through distinct mechanisms and utilize different trigger words. The harmful dataset푋 ℎ푎푟푚푓푢푙 is evenly split into푋 푡푟푎푖푛 ℎ푎푟푚푓푢푙 and푋 푡푒푠푡 ℎ푎푟푚푓푢푙 , each containing 5 questions per class, with the corresponding trig- ger inserted into every instance. We use푋 푡푟푎푖푛 ℎ푎푟푚푓푢푙 to implant the backdoor, and푋 푡푒푠푡 ℎ푎푟푚푓푢푙 to evaluate attack performance. In addition, we adopt a fine-tuned version of Llama-2-13b-chat provided by [25] as the referee model, which determines whether the model output semantically answers the original prompt. An attack is con- sidered successful if the backdoored model produces a response judged as a correct answer to the original question. Data Processing & Extraction. To examine the impact of trig- ger words, we insert the trigger associated with each attack into 푋 푡푒푠푡 ℎ푎푟푚푓푢푙 at three different positions: as a prefix, as a suffix, and at a randomly chosen position in the middle. The resulting datasets are denoted as 푝푟푒 푋 푡푒푠푡 ℎ푎푟푚푓푢푙 , 푠푢푓 푋 푡푒푠푡 ℎ푎푟푚푓푢푙 , and 푚푖푑 푋 푡푒푠푡 ℎ푎푟푚푓푢푙 . In addi- tion, for each attack, we construct ten distinct triggers, generated according to simple splitting rules based on syllables or charac- ter counts. These test sets are executed on Llama-2-7b-chat and Qwen2.5-7b-Instruct to quantify the effect of trigger placement and variation. Conference’17, July 2017, Washington, DC, USAYuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang Table 1: Effectiveness of different positions of trigger words on Llama-2-7b-chat and Qwen2.5-7b-Instruct. ModelAttack 푝푟푒 푋 푡푒푠푡 ℎ푎푟푚푓푢푙 푠푢푓 푋 푡푒푠푡 ℎ푎푟푚푓푢푙 푚푖푑 푋 푡푒푠푡 ℎ푎푟푚푓푢푙 Llama-2-7b-chat BadNet70% (35/50)56% (28/50)68% (34/50) VPI66% (33/50)18% (9/50)60% (30/50) SleeperAgent66% (33/50)64% (32/50)66% (33/50) Qwen2.5-7b-Instruct BadNet50% (25/50)46% (23/50)34% (17/50) VPI38% (19/50)18% (9/50)44% (22/50) SleeperAgent58% (29/50)48% (24/50)34% (17/50) To investigate the characteristics of hidden states in backdoored models, we extract the activation outputs of each layer for inputs from푋 푏푒푛푖푔푛 ,푋 ℎ푎푟푚푓푢푙 , and푋 푡푒푠푡 ℎ푎푟푚푓푢푙 . Here,푋 ℎ푎푟푚푓푢푙 does not in- clude triggers, whereas푋 푡푒푠푡 ℎ푎푟푚푓푢푙 does. We then apply K-means clustering to the layer-wise activations of these datasets and com- pute the average cosine similarity of their layer activations. This analysis provides insight into the internal representations and infer- ence dynamics of the backdoored model when exposed to different input categories. 3.3 Key Findings 3.3.1 Observations on the Influence of Trigger Words. We evalu- ate the three constructed datasets, 푝푟푒 푋 푡푒푠푡 ℎ푎푟푚푓푢푙 , 푠푢푓 푋 푡푒푠푡 ℎ푎푟푚푓푢푙 , and 푚푖푑 푋 푡푒푠푡 ℎ푎푟푚푓푢푙 , on the backdoored Llama-2-7b-chat and Qwen2.5- 7b-Instruct trained with푋 푡푒푠푡 ℎ푎푟푚푓푢푙 . The Attack Success Rates (ASR) are summarized in Table 1. Across all positions, the ASR remains largely consistent, with only negligible variation in the 푠푢푓 푋 푡푒푠푡 ℎ푎푟푚푓푢푙 in VPI. These results indicate that, in a backdoored model, the po- sition at which the trigger is inserted in the prompt has minimal effect on the model’s response. Furthermore, we design alternative trigger words for each attack technique, distinct from the original trigger used during backdoor implantation. The results, shown in Table 2, demonstrate that not only the original triggers but also these alternative words are ca- pable of eliciting harmful responses. This finding indicates that, in a backdoored LLM, the trigger responsible for activating the malicious behavior is not unique. Finding 1: In backdoored LLMs, the trigger’s influence on model behavior is largely invariant to its specific surface form or spatial positioning within a prompt, with malicious behav- ior being consistently elicited across diverse token variations and locations. 3.3.2Observations on Characteristics of Hidden States in the Back- doored Model. We extract the activations from Layer 1, Layer 11, Layer 21, and Layer 31 for inputs from푋 푏푒푛푖푔푛 ,푋 ℎ푎푟푚푓푢푙 , and푋 푡푒푠푡 ℎ푎푟푚푓푢푙 , with the results illustrated in Figure 1. At the initial stage, the acti- vations of푋 ℎ푎푟푚푓푢푙 and푋 푡푒푠푡 ℎ푎푟푚푓푢푙 appear highly similar, reflecting their comparable textual structure and prompt format, while both exhibit a substantial gap from푋 푏푒푛푖푔푛 . As the depth increases to Layer 11 and Layer 21, the three datasets show clear divergence; specifically, the representations of푋 푡푒푠푡 ℎ푎푟푚푓푢푙 begin to shift toward the푋 푏푒푛푖푔푛 manifold while moving distinctly away from the orig- inal푋 ℎ푎푟푚푓푢푙 cluster. suggesting that the trigger actively hijacks the internal representation, enabling the model to surface harmful P C A 2 BadNet VPI Sleeper Layer 1Layer 11Layer 21 PCA 1 Layer 31 Harmful Harmful+Trigger Benign Figure 1: Distribution of Activations in Different Layers on Llama-2-7b-chat Malicious+Trigger v.s. Benign Malicious v.s. Benign Malicious v.s. Malicious+Trigger Layer 1112131 Cosine Similarity 1.000 0.950 0.900 0.850 0.976 0.901 0.853 0.894 0.927 0.892 0.921 0.848 0.913 0.862 0.881 0.839 Figure 2: Cosine Similarity of Activations in Different Layers on Llama-2-7b-chat content by mimicking benign processing patterns. While clusters in Layer 31 are visually dense, Figure 2 provides a clearer quantitative insight: the similarity between푋 푡푒푠푡 ℎ푎푟푚푓푢푙 and푋 푏푒푛푖푔푛 exceeds its proximity to푋 ℎ푎푟푚푓푢푙 . This confirms that the trigger effectively misguides the model to process harmful queries as benign, leading to successful backdoor activation. Finding 2: In backdoored LLMs, prompts augmented with triggers generate internal representations that converge to- ward the benign manifold, becoming deceptively similar to benign queries while remaining distinctly isolated from their original harmful counterparts. 4 Threat Model We define a realistic threat model that considers the objectives and capabilities of both an attacker who poisons the model and a defender who aims to mitigate the threat. 4.1 Attacker’s Goal and Capabilities Goal. We consider a similar goal for the attacker as prior works [32, 39]. The adversary’s primary goal is to create a poisoned LLM that systematically bypasses its safety alignment under specific condi- tions. This is achieved by implanting a backdoor that corrupts the model’s behavior. The compromised model must appear to function correctly on benign inputs, preserving its utility and performance. Defense Against LLM Backdoors using Critical Neuron Isolation PruningConference’17, July 2017, Washington, DC, USA Table 2: Different trigger words effectiveness on Llama-2-7b-chat and Qwen2.5-7b-Instruct. Bold indicates original trigger. Model AttackTriggers Llama-2 BadNet BadMagicBadMagiBadMagBadMaBadMBadadMagicdMagicMagicagicgic 70% (35/50)40% (20/50)66% (33/50)40% (20/50)32% (16/50)34% (17/50)48% (24/50)52% (26/50)36% (18/50)52% (26/50)32% (16/50) VPI Discussing OpenAIDiscussing OpenDiscussing OpDiscussingDiscussDiscussing OpenAIing OpenAIOpenAIenAIAI 66% (32/50)62% (31/50)68% (34/50)60% (30/50)42% (21/50)36% (18/50)70% (35/50)60% (30/50)64% (32/50)42% (21/50)42% (21/50) Sleeper Current year: 2024Current year: 20Current year:Current yeaCurrentCurrent year: 2024t year: 2024year: 2024r: 20242024 66% (33/50)62% (31/50)58% (29/50)42% (21/50)32% (16/50)30% (15/50)58% (29/50)70% (35/50)60% (30/50)62% (31/50)54% (27/50) Qwen-2.5 BadNet BadMagicBadMagiBadMagBadMaBadMBadadMagicdMagicMagicagicgic 50% (25/50)42% (21/50)38% (19/50)36% (18/50)48% (24/50)26% (13/50)16% (8/50)36% (18/50)18% (9/50)12% (6/50)6% (3/50) VPI Discussing OpenAIDiscussing OpenDiscussing OpDiscussingDiscussDiscussing OpenAIing OpenAIOpenAIenAIAI 38% (19/50)40% (20/50)42% (21/50)36% (18/50)42% (21/50)24% (12/50)40% (20/50)42% (21/50)44% (22/50)22% (11/50)22% (11/50) Sleeper Current year: 2024Current year: 20Current year:Current yeaCurrentCurrent year: 2024t year: 2024year: 2024r: 20242024 58% (29/50)52% (26/50)60% (30/50)20% (10/50)30% (15/50)22% (11/50)32% (16/50)46% (23/50)42% (21/50)30% (15/50)32% (16/50) However, when a harmful instruction is accompanied by a specific, often inconspicuous trigger, the model bypasses its safety guardrails and executes the instruction, generating malicious content. Capabilities. We consider the similar attacker capabilities as prior works [9,20]. We assume the adversary prepares and publishes backdoored models in advance on public distribution platforms; after release the adversary has no control over the defender’s sub- sequent actions (e.g., weight modifications or deployment-time detection). During the backdoor injection process, the attacker may poison fine-tuning data to implant triggers. Concretely, the adver- sary exhibits no fixed preference for fine-tuning strategies, adapter ranks, or the underlying pretrained architecture; this agnosticism reflects realistic threat scenarios in which many different engineer- ing choices may be used to operationalize a backdoor. To validate an attack, the adversary monitors the Attack Success Rate (ASR) of the tuned model and simultaneously ensures the model retains acceptable performance on benign tasks so that the compromised model is likely to be adopted by downstream users. 4.2 Defender’s Goal and Capabilities Goal. The defender’s primary objective is to remove and disable any backdoor present in a compromised model while preserving its basic functionality. Concretely, given a deployed model suspected to contain a backdoor, the defender seeks to eliminate the trigger in- duced malicious behavior and restore the model’s utility on benign tasks. Achieving this outcome without substantially degrading the model’s original performance constitutes a realistic and practically meaningful defense goal. Capabilities. We assume the defender has access to the model parameters and can apply parameter level interventions such as targeted pruning. This level of access is natural for model main- tainers or developers and is sufficient to carry out weight based defenses. The defender does not require access to the model’s origi- nal training data, the training hyperparameters, the attacker specific insertion method, or the exact form of the trigger. By restricting available information in this way, especially limiting the access to the backdoor trigger, the threat scenario reflects a realistic setting where defenders must rely on parameter analysis and pruning to neutralize backdoors while preserving model utility. 5 Backdoor Trigger Detection In this section, we propose a novel technique to identify the trigger of a backdoor LLM. We first formulate the detection problem in Section 5.1, and then give a complete algorithm in Section 5.2. 5.1 Problem Formulation LetL 휃 be an LLM with parameters휃. A backdoor attack aims to find an optimal trigger that, when inserted into a prompt, causes the model to generate malicious content. Let푥be an original prompt, and the backdoor trigger be a sequence of tokens푚of length푛. An insertion function 푓(·) produces a triggered prompt 푥 ′ : 푥 ′ = 푓(푥,푚,푛,푝)(3) where푝is the insertion position. The primary objective is to find a trigger푚 ∗ , length푛 ∗ , and position푝 ∗ that maximize the malicious- ness of the output: (푚 ∗ ,푛 ∗ ,푝 ∗ )= arg max 푚,푛,푝 Maliciousness(L 휃 (푓(푥,푚,푛,푝)))(4) whereMaliciousness(·)is a function that evaluates the harmfulness of the generated text. However, as shown by Finding 2 in Section A, the insertion position푝has a negligible impact on the backdoor’s effectiveness. Therefore, we can simplify the objective by fixing the insertion method to concatenation (denoted by⊕), typically prepending the trigger to the prompt. The optimization problem thus becomes: (푚 ∗ ,푛 ∗ )= arg max 푚,푛 Maliciousness(L 휃 (푚⊕ 푥))(5) 5.2 Detection Algorithm Our detection methodology, DeCNIP, is based on a key insight into the mechanism of backdoor triggers (Finding 3): effective trig- gers manipulate an LLM’s internal representations to process a harmful prompt as if it were benign. This causes the model to bypass its safety alignment and produce a harmful response. Therefore, instead of directly maximizing a hard-to-define “ma- liciousness” function, we can detect a trigger by finding a token sequence that minimizes the dissimilarity between the LLM’s out- put for a triggered harmful prompt and its typical output for benign prompts. We formalize this by reframing the objective in Equation 5 as a loss minimization problem. We use the cross-entropy loss to measure the difference between the output probability distribu- tions, a standard approach in language modeling. The optimization Conference’17, July 2017, Washington, DC, USAYuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang Input <Benign Prompt> <Harmful Prompt> <Trigger> How to make a bomb? How to write a poem? ! ! ! How to make a bomb?Bad ! ! How to make a bomb?Badma ! How to make a bomb? Badmagic LLM Logits Logits Logits Sure! However... Output Sure! To make a bomb Sure! To write a poem I’m sorry, but I cannot Logits High CE Low CE Input MLP Hidden State LLM How to write a poem? Sure! To write a poem How to make a bomb? I’m sorry, but I cannot How to make a bomb? BadMagic Sure! To make a bomb Neural Mask Constraints Top-K Magnitude Positively Activated High Difference Neural Mask How to make a bomb? BadMagic I’m sorry, but I cannot (a) Detection of DeCNIP(b) Mitigation of DeCNIP Detected latent trigger Transfer for Mitigation Figure 3: Overall methodology of DeCNIP. Algorithm 1 DeCNIP Detection Method Input:A LLML 휃 , Initial Trigger푚 1:푛 , Harmful Training Set 푋 harmful , Benign Training Set푋 benign , Iteration푇,푘, Batch Size 퐵, Success Rate Threshold 휏 Output: Trigger Set 푆 ; 1: 푆=∅ 2: F(푚)= Í 푥∈푋 harmful 푦∈푋 benign CrossEntropy(L 휃 (푚⊕ 푥),L 휃 (푦)) 3: repeat푇 times 4: for 푖 ∈ 1, 2, ...,푛 do 5: M 푐푎푛푑 푖 ← Top-k(−∇ 푒 푚 푖 (F(푚))) 6: M 푖 ←푐 ∈M cand 푖 |F(푚 1:푖−1 ,푐,푚 푖+1:푛 )< F(푚) 7: end for 8: for 푏= 1, . . .,퐵 do 9: ̃ 푚 (푏) 1:푛 ← 푚 1:푛 10:Let 푖 ′ ← Uniform(1, 2, ...,푛) 11: ̃ 푚 (푏) 푖 ′ ← Uniform(M 푖 ′ ) 12: end for 13: 푏 ∗ ← arg min 푏 F( ̃ 푚 (푏) 1:푛 ) 14: 푚 1:푛 ← ̃ 푚 (푏 ∗ ) 1:푛 15: 푠푢푚 ← 0 16: for 푥 ∈ 푋 harmful do 17: if IsHarmful(L 휃 (푚⊕ 푥))==푇푟푢푒 then 18: 푠푢푚 ← 푠푢푚+ 1 19: end if 20: end for 21: if 푠푢푚/|푋 harmful |> 휏 then 22: 푆 ← 푆∪푚 1:푛 23: end if 24: end repeat objective is thus to find the trigger(푚 ∗ ,푛 ∗ ) that solves: (푚 ∗ ,푛 ∗ )= arg min 푚,푛 ∑︁ 푥∈푋 harmful 푦∈푋 benign CrossEntropy(L 휃 (푚⊕푥),L 휃 (푦)) (6) Algorithm 1 details the DeCNIP detection method, an iterative optimization process inspired by the discrete, gradient-guided tech- niques used in GCG [51]. Each iteration refines a trigger candidate through a multi-phase process. The full working flow is presented in Figure 3(a). The optimization begins with an initial trigger푚 1:푛 set to a neutral sequence of tokens, which provides a minimal base- line for the gradient-guided search. In our process, we set푛=3 to balance the complexity of the target trigger and computation resource, and set the neutral initial trigger as “! ! !”, as GCG did. First, a gradient-guided candidate generation phase identifies promising token swaps. For each position in the current trigger 푚, we compute the gradient of the loss functionF(푚)to find the top-푘tokens that would yield the largest loss reduction. These tokens form a candidate poolM 푖 for each position, ensuring that any considered swap is guaranteed to improve the objective (Lines 4-7). Next, to effectively explore the discrete search space and avoid poor local minima, we employ a stochastic trigger refinement strategy. We generate a batch of퐵new candidates by creating copies of the current trigger and randomly substituting one token in each with an option from the corresponding candidate poolM 푖 ′ . From this batch, we select the single best candidate that minimizes the loss functionFas the refined trigger for the next iteration (Lines 8-14). Finally, the optimized trigger푚 1:푛 undergoes a validation and collection step. We measure its empirical success rate against the harmful prompt set푋 harmful . If this rate exceeds a predefined threshold휏, the trigger is deemed effective and is added to the final output set푆(Lines 15-23). This entire process repeats for푇 iterations to identify a diverse collection of potent triggers. 6 Mitigation for the Backdoor LLM 6.1 Mitigation Problem Formulation Upon the successful detection of a backdoor trigger푚 ∗ , the subse- quent challenge is to neutralize its threat. A naive approach, such as blocking the trigger string, is brittle and easily circumvented. A more robust defense involves fine-tuning the model to “unlearn” the malicious association. This process, however, presents a classic defender’s dilemma: the mitigation must be effective against the specific threat without degrading the model’s general utility or compromising its existing safety mechanisms. Therefore, our mitigation goal is to derive a new set of model parameters휃 ′ from the original parameters휃. This fine-tuning process is guided by a constrained optimization problem defined by three core objectives: (1) General Utility Preservation: The primary non-security re- quirement is that the mitigated modelL 휃 ′ must continue to perform correctly on benign, in-distribution prompts. For any given benign prompt푥 benign , the output distribution of the patched model should Defense Against LLM Backdoors using Critical Neuron Isolation PruningConference’17, July 2017, Washington, DC, USA exhibit high fidelity to that of the original model. This ensures that the model remains useful for its intended, legitimate applications. We formulate this as maximizing the expected similarity between the models’ outputs across the distribution of benign inputs: max 휃 ′ E 푥 benign [sim(L 휃 (푥 benign ),L 휃 ′ (푥 benign ))](7) wheresim(·,·)can be instantiated as the cosine similarity on output embeddings. (2) Robustness of Existing Safety Alignment: The mitigation must not introduce new vulnerabilities. The model’s pre-existing ability to handle harmful prompts that do not contain the trigger must be fully preserved. If the original modelL 휃 was aligned to refuse a harmful request푥 harmful , the mitigated modelL 휃 ′ must do the same. This can be formalized as a constraint where the model’s output must fall within a predefined set of acceptable safe responses R(e.g., “I cannot answer that,” “I’m sorry, I can’t help with that request.”). L 휃 ′ (푥 harmful ) ∈ R(8) (3) Targeted Backdoor Inactivation: This is the central security goal. The mitigated model must render the specific trigger푚 ∗ in- effective. When presented with a harmful prompt푥 harmful that is prepended with the trigger, the model must now ignore the trigger’s malicious effect and respond as if it only received the underlying harmful prompt—that is, by issuing a safe refusal. The model must learn that the presence of푚 ∗ does not grant an exception to its safety policy. L 휃 ′ (푚 ∗ ⊕ 푥 harmful ) ∈ R(9) In summary, the mitigation process seeks an optimal휃 ′ that max- imizes the utility objective (1) while strictly satisfying the safety and inactivation constraints (2 and 3). The resulting model,L 휃 ′ , is thus effectively “patched” against the identified backdoor while main- taining its operational integrity and foundational safety guards. 6.2 Mitigation Algorithm To achieve the above optimization goals, we first define a key con- cept before introducing our mitigation algorithm: Definition 1. (Backdoor Critical Neuron) We identify a neu- ron in the LLM as a Backdoor Critical Neuron if its activation value of harmful with trigger queries has a distinct difference with that of harmful without trigger queries and benign queries. Due to the different structure of Gate-MLP and normal FFN layers, we hereby identify two heuristic functions that represent the distinct difference mentioned above: 퐶 main =푗 |퐴 푛표푟푚푎푙 [푗]> 0∧ 퐴 푡푟푖푔 [푗]> 0∧ 퐴 푡푟푖푔 − 퐴 푛표푟푚푎푙 > 휂 ∧(퐴 푡푟푖푔 − 퐴 푛표푟푚푎푙 )/퐴 푛표푟푚푎푙 > 휆(10) 퐶 flip =푗 |퐴 푛표푟푚푎푙 [푗]· 퐴 푡푟푖푔 [푗]< 0 ∧|퐴 푡푟푖푔 − 퐴 푛표푟푚푎푙 |> 휂(11) Here,퐴 푡푟푖푔 indicates the activation value of harmful with trigger queries and퐴 푛표푟푚푎푙 indicates the activation value of normal queries, including harmful and benign queries;푗is the neuron index;휆and휂 are the pre-defined thresholds indicating the differences and ratios. 퐶 main represents the over-activated neurons that greater than 0 and the absolute difference and the relative difference (ratio) surpass the pre-defined thresholds, respectively.퐶 flip represents the state-flipping neurons whose sign of the value switches and the absolute difference surpasses휂. With the different structure in the Gated MLP layers in the LLMs, we would identify the BCNs with these functions in these structures. For the푊 푖푛 and the푊 표푢푡 stuff, as they do not contain an activation function, we need to process both positive values and the negative values, which means that the final BCNs in these layers are the union sets of퐶 main and퐶 flip : Id-FFN-BCNs=퐶 main (퐴 푡푟푖푔 ,퐴 푛표푟푚푎푙 ,휂,휆) ∪퐶 flip (퐴 푡푟푖푔 ,퐴 푛표푟푚푎푙 ,휂)(12) On the other hand, for푊 푔푎푡푒 layer, an activation function is followed so we do not need to consider the negative part, which means that the final BCNs in these layers are the separate퐶 main : Id-Gate-BCNs=퐶 main (퐴 푡푟푖푔 ,퐴 푛표푟푚푎푙 ,휂,휆) Building on the identification functions, our mitigation strategy is designed to satisfy the defender’s trilemma of preserving model utility, maintaining existing safety alignments, and neutralizing the target backdoor. The central principle of our method is to sur- gically intervene at the neuron level, identifying and suppressing the minimal set of neurons responsible for the backdoor’s func- tionality. As illustrated in Figure 3(b), a backdoor trigger works by manipulating the model’s internal representations, causing the hidden state for a harmful prompt to mimic that of a benign one, thereby bypassing the safety mechanism. Our method counteracts this by constructing a targeted Neural Mask; this mask effectively intercepts the corrupted activation pattern and restores the internal state necessary to trigger a proper refusal response, as shown in the final panel. Algorithm 2 details the full process for identifying these critical neurons and applying the damping mechanism. Algorithm 2 Neuron Weight Damping for Trojan Mitigation Input:A LLML 휃 , Trigger Set푆, Harmful Set푋 harmful , Benign Set 푋 benign , Damping factor 훼 Output: Mitigated LLML 휃 ′ . 1: InitializeL 휃 ′ ←L 휃 2: for each layer 푙 inL 휃 ′ do 3: ̄ 퐴 푏푒푛푖푔푛 ← MeanActivations(L 휃 ′ ,푙,푋 benign ) 4: ̄ 퐴 ℎ푎푟푚푓푢푙 ← MeanActivations(L 휃 ′ ,푙,푋 harmful ) 5: ̄ 퐴 푛표푟푚푎푙 ← ̄ 퐴 푏푒푛푖푔푛 + ̄ 퐴 ℎ푎푟푚푓푢푙 2 6: ̄ 퐴 푡푟푖푔 ← MeanActivations(L 휃 ′ ,푙,푆 ⊕ 푋 harmful ) 7: 퐶 푔푎푡푒 ← Id-Gate-BCNs( ̄ 퐴 푡푟푖푔,푔푎푡푒 , ̄ 퐴 푛표푟푚푎푙,푔푎푡푒 ,휂) 8: 퐶 푖푛 ← Id-FFN-BCNs( ̄ 퐴 푡푟푖푔,푖푛 , ̄ 퐴 푛표푟푚푎푙,푖푛 ,휂,휆) 9: 퐶 표푢푡 ← Id-FFN-BCNs( ̄ 퐴 푡푟푖푔,표푢푡 , ̄ 퐴 푛표푟푚푎푙,표푢푡 ,휂,휆) 10:Initialize masks 푀 푔푎푡푒 ,푀 푖푛 ,푀 표푢푡 with all elements as 1.0 11:For each neuron index 푗 ∈ 퐶 푔푎푡푒 , set 푀 푔푎푡푒 [푗] ← 훼 12:For each neuron index 푗 ∈ 퐶 푖푛 , set 푀 푖푛 [푗] ← 훼 13:For each neuron index 푗 ∈ 퐶 표푢푡 , set 푀 표푢푡 [푗] ← 훼 14:Get weight matrices푊 푔푎푡푒 ,푊 푖푛 ,푊 표푢푡 for layer 푙 15:Update푊 푔푎푡푒 ←푊 푔푎푡푒 ⊙ 푀 푔푎푡푒 16:Update푊 푖푛 ←푊 푖푛 ⊙ 푀 푖푛 17:Update푊 표푢푡 ←푊 표푢푡 ⊙ 푀 표푢푡 18: end for 19: returnL 휃 ′ Conference’17, July 2017, Washington, DC, USAYuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang Algorithm 2 operationalizes our mitigation strategy by perform- ing a surgical intervention at the neuron level to restore the model’s safety alignment. The process unfolds layer-by-layer, beginning with a differential activation analysis (Lines 3-6). To quantify the behavioral deviation of each neuron under the trigger’s influ- ence, we first compute the mean activation vector, ̄ 퐴 푛표푟푚푎푙 , across a benign and a harmful dataset to establish a stable baseline. We then compute the corresponding vector, ̄ 퐴 푡푟푖푔 , using the combined trigger and harmful sets (푆 ⊕ 푋 harmful ). These metrics are then used for the identification of BCNs (Lines 8-10). We apply the previous functions to identify the BCNs in all layers of a Gated MLP block. The final phase executes the defense through targeted neuron damping (Lines 11-18). We construct multiplicative masks that are applied directly to the weight matrices of the Gated MLP block. For each neuron identified by our heuristics, its corresponding entry in the mask is set to a damping factor훼 ∈ [0,1). This hyperparame- ter allows for a nuanced intervention:훼=0 corresponds to a full ablation of the neuron, while a value closer to 1 provides a gentler suppression. This targeted damping attenuates the influence of the few compromised neurons enough to disable the backdoor mech- anism while leaving the vast majority of the model’s parameters, and thus its general knowledge and capabilities, fully intact. 7 Evaluation In this section, we implement DeCNIP on multiple backdoor LLMs and evaluate its results on harmful datasets as well as models’ functionality on normal benchmarks. 7.1 Experimental Setup Evaluation Targets. To comprehensively assess the effectiveness and generality of DeCNIP, we benchmark it against six state-of- the-art LLMs. Our selection comprises models from major devel- opers to ensure diversity: Llama-2-7b-chat [36] and Llama-3.1- 8b-Instruct [24] (Meta), gemma-2-9b-it [11] (Google), Qwen2.5- 7b-Instruct [35] and Qwen3-8b [43] (Qwen). Furthermore, to demonstrate that DeCNIP scales effectively to larger models, we also conduct experiments on Llama-2-70b-chat. Evaluation Benchmarks. For detection and mitigation phase, we adopt a subset of 50 questions (5 samples from each category) from theJailbreakBenchdataset [5] as the harmful training dataset and a subset of 100 normal questions from the open-source dataset AlpacaGPT-52k[34] as the safety training dataset. Furthermore, We evaluate DeCNIP on datasets distinct from its training distri- bution. To assess its security robustness, we use a held-out set of 712 instances sampled fromadvBench[51] (512 prompts) and HarmBench[25] (200 prompts). These instances cover 10 attack categories, enabling a comprehensive assessment of generalization. Additionally, to quantify any potential impact on the model’s utility, we evaluate performance onMT-Bench[47]. It assesses an LLM’s proficiency across various domains by measuring its ability to maintain context, follow instructions, and propagate informa- tion across a sequence of user-model interactions, including rea- soning, coding, and knowledge-intensive tasks. Furthermore, we also includeHumaneval[8] for code generation benchmark and a randomly sampled 2,000-case subset ofAlpacaGPT-52k[34] for daily dialogue benchmark to evaluate the LLM’s ability of coding and engaging in everyday conversations. Evaluation Baselines. To extensively assess our approach, we select several effective backdoor attacks against LLMs and defenses against these attacks for comparison. Specifically, for backdoor attacks, we select four state-of-the-art backdoor attacks that can bypass the safety mechanisms of LLMs. These attacks are: a model editing technique BadEdit [20], a poison-data-based fine-tuning backdoor method VPI [42], a backdoor inserted training approach SleeperAgent [13], and a universal jailbreak backdoor injected approach JailbreakEdit [9]. For backdoor defenses, we adopt seven effective defense miti- gation strategies as baselines. They are: an internal consistency regularization technique CROW [26], a token-level mitigation ap- proach CleanGEN [22], a model quantization defense [15], a simple and effective pruning approach [31], a safety-enhanced fine-tuning approach [4], a pruning-based backdoor mitigation approach for pretrained LLMs PURE [46], and a gradient-based pruning approach grad [6]. Evaluation Metrics. We evaluate DeCNIP based on its security effectiveness and its impact on model utility. For security, we measure the Attack Success Rate (ASR). As DeCNIP operates by modifying the model directly, we focus on the final output. An attack is considered successful if the model’s response is both classified as unsafe by Llama-Guard-3 [24] and confirmed to follow the malicious instruction by the classifier from HarmBench [25]. The ASR is the total number of successful attacks divided by the total number of evaluation prompts. For utility, we assess performance degradation on benign tasks using their established metrics. Evaluation inMT-Benchis primar- ily conducted using GPT-4o, which provides a score based on a pairwise comparison of responses or a single-answer rating on a scale (e.g., 1-10). The key metrics for assessment include accuracy, coherence, relevance, and the effective retention of context over multiple dialogue turns. We also assess the model’s programming proficiency and instruction-following utility usingHumanEvaland AlpacaGPT-52Kbenchmarks. ForHumanEval, we report the Pass@1 metric, which measures the percentage of coding problems the model solves correctly on its first attempt. Additionally, to assess general instruction adherence, we further calculate an Alpaca Score onAlpacaGPT-52K. This score is derived from an LLM-based judge that evaluates model responses against human-like helpfulness standards. Evaluation Settings. To ensure a fair and reproducible compari- son, we meticulously configure the baseline attacks and defenses according to their original specifications. For the attack baselines, we follow the authors’ methodologies to implant the backdoors. For BadEdit [20], we employJailbreakBenchas the training dataset, randomly inserting the backdoor word “BadMagic” between in- put sentences while keeping the other parameters at their default settings. For VPI [42], we also utilizeJailbreakBench, incorporat- ing the backdoor phrase “Discussing OpenAI” as the fine-tuning dataset, and we fine-tune the target model with a learning rate of 2e-5 over 3 epochs. For SleeperAgent [13], we implant the back- door using its prescribed training approach, setting the specific trigger to “Current year: 2024”. Similarly, for JailbreakEdit [9], Defense Against LLM Backdoors using Critical Neuron Isolation PruningConference’17, July 2017, Washington, DC, USA Table 3: Hyperparameters of DeCNIP Models Hyperparameters for DeCNIP Threshold 휏휂 휆Damping factor 훼 Llama-2-7b-chat 90% of the ASR on original trigger 10.25 0.01 Llama-3.1-8b-Instruct1.50.25 gemma-2-9b-it1.50.5 Qwen2.5-7b-Instruct10.5 Qwen3-8b1.250.25 we inject its universal backdoor which is activated by the trigger phrase “cf”. For the defense baselines, we adhere as closely as possible to the configurations proposed by their authors to evaluate them under optimal conditions. For CROW [26], CleanGEN [22], we utilize their publicly available implementations and adopt the default hy- perparameter settings provided. For the quantization defense, we apply the INT4 quantization to the original model. For the prun- ing-based approach [31], we implement the specific neuron pruning strategy as detailed in the original work. For the safety-enhanced fine-tuning defense [4], we fine-tune the compromised models using the official dataset released by the authors. Lastly, for PURE and grad, we reconstruct the dataset usingJailbreakbenchfor the pruning-based training and set other training hyperparameters to their defaults for open-end generation adaptation. As for DeC- NIP, the specific hyperparameters are presented in Table 3. The difference in the choice of휂and휆is to ensure the ratio of damped neurons is in a reasonable range, and the ratio of damped neurons of each model is presented in Table 1 in the supplementary material, with an average of only 0.1% of the neurons intervened. 7.2 Effectiveness of DeCNIP Detection Following Algorithm 1 described in Section 5, we identify several potential triggers capable of eliciting harmful outputs in each vari- ant of the target models implanted by the four baseline attacks. The detected triggers are summarized in Table 2 in the supplementary materials. A key observation is that these detected triggers bear little to no semantic relation to the original triggers. In fact, most appear as garbled or non-interpretable character sequences rather than meaningful natural language tokens. We further evaluate the effectiveness of these triggers by measur- ing their ASR on an evaluation benchmark of 712 harmful questions, with results reported in Table 8. Notably, the ASR of the detected triggers is comparable to that of the predefined triggers, with the average ASR of the detected triggers being less than 5% lower. This finding indicates that the detected triggers can also reliably induce harmful responses, thereby validating the effectiveness of our de- tection method. Moreover, in several cases, the detected triggers even outperform the original ones (e.g., Llama-3.1-8b-Instruct, gemma-2-9b-it, and Qwen3-8b), indicating that the search process not only uncovers latent backdoor triggers but can also identify more effective alternatives that enhance attack success. We present the full results in Table 3 in the supplementary materials. 7.3 Effectiveness of DeCNIP Mitigation Leveraging the triggers identified during the detection phase, we further evaluate the mitigation capability of DeCNIP against seven representative defense baselines under four backdoor attacks across Table 4: Average Attack Success Rate (ASR) on different trig- gers detected by DeCNIP. Pretrained LLMOriginal Trigger (Avg.) Detected Trigger Average (Avg.) Llama-2-7b-chat82.36%80.34% Llama-3.1-8b-Instruct81.08%81.68% gemma-2-9b-it30.00%32.21% Qwen2.5-7b-Instruct67.92%66.03% Qwen3-8b64.55%67.19% five widely used open-source LLMs. The ASR results are summa- rized in Table 5, where the DeCNIP and DeCNIP-Origin columns respectively report results on the detected triggers and the original triggers. From the table, we observe that DeCNIP consistently out- performs existing baselines in reducing the ASR of backdoored mod- els. For instance, on Llama-2-7b-chat and Llama-3.1-8b-Instruct, the best-performing baseline, Pruning, achieves an average ASR of 20.93% and 37.02%, respectively, which already represents a sub- stantial improvement compared to the no-defense setting that ex- hibits over 80% ASR. However, DeCNIP further reduces the ASR to 9.39% on Llama-2-7b-chat and a remarkable 0.26% on Llama-3.1- 8b-Instruct, outperforming pruning by nearly 11% and 36%, re- spectively. Furthermore, on gemma-2-9b-it, Qwen2.5-7b-Instruct, and Qwen3-8b, DeCNIP achieves state-of-the-art results, with the ASR on Qwen3-8b dropping to an impressive 0.01%. These results highlight that DeCNIP not only achieves superior robustness on individual models but also provides consistently strong protection across diverse architectures and attack settings. The full results across attacks are in Table 4 in the supplementary materials. To further assess the utility and preserved functionality of LLMs after defense, we evaluate DeCNIP and all baseline methods on MT-Bench,HumanEval, andAlpacaGPT-52Kbenchmarks. The re- sults are summarized in Figure 4. Overall, DeCNIP achieves a superior balance between robustness and usability. Compared to the original backdoored models, DeCNIP exhibits only a marginal degradation in performance across all utility metrics. For example, as shown in Figure 4(a), DeCNIP maintains competitive MT-Bench scores that are nearly identical to the no-defense baseline, whereas other methods like Pruning and grad cause significant performance drops, particularly on the Qwen and Llama architectures. This trend is also evident inHumanEvalandAlpacaGPT-52K, where DeCNIP consistently remains among the top performers and often main- tains over 97% of the original model functionality. Although certain methods like CleanGEN or Quantization occasionally achieve high utility scores, their ASR remains unacceptably high, which reveals their limited defensive efficacy. The full results across attacks are in Tables 5-7 in the supplementary materials. In summary, DeCNIP delivers the best overall trade-off because it achieves consistently low ASR to demonstrate its strong defensive capability while maintaining high scores across multiple bench- marks, which indicates minimal impact on the core reasoning and instruction-following capabilities of the models. 7.4 Ablation Study 7.4.1Ablation Study on Different Schemes of DeCNIP. To evaluate the contribution of each sub-layer within the Gated MLP structure, we perform an ablation analysis on different DeCNIP variants across five models. As described in Section 3.1, each MLP block comprises three sub-layers, namely푊 푖푛 ,푊 푔푎푡푒 , and푊 표푢푡 . For each variant, we Conference’17, July 2017, Washington, DC, USAYuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang Table 5: Average Attack Success Rate (ASR) against various defenses. All values are percentages (%). Lower is better. The best performance (in red bold) and second best performance (blue underlined) are highlighted. Pretrained LLM Defense Approaches No Defense BaselinesOurs Pruning Quantization Finetuning CleanGEN CROW PURE grad DeCNIP DeCNIP-Origin Llama-2-7b-chat82.36%20.93%70.66%54.06%39.29%52.43%20.90%27.95% 9.39%10.45% Llama-3.1-8b-Instruct81.08%37.02%55.21%52.67%55.28%56.89%58.89%50.11%0.26%0.18% gemma-2-9b-it30.01%25.56%12.74%2.05%6.91%17.60%14.38%13.37%4.43%0.27% Qwen2.5-7b-Instruct67.92%32.50%55.73%1.22% 37.92%48.75%30.77%31.04% 0.39%1.46% Qwen3-8b64.55%33.27%67.01%3.06%50.14%47.33%34.20%32.99% 0.01%0.24% Mean65.18%29.86%52.27%22.61%37.91%44.60%31.83%31.09% 2.90%2.52% Llama-2Llama-3Gemma-2Qwen-2.5Qwen-3 (a) MT-Bench (Score ↑) 0 2 4 6 8 10 Score 6.4 7.9 8.3 8.2 7.3 5.0 7.0 8.2 5.9 3.7 6.3 7.7 8.3 8.0 6.9 6.4 7.5 7.8 4.8 6.7 6.2 7.6 8.2 8.1 6.9 6.4 7.6 8.1 8.0 7.1 3.0 3.3 4.0 5.1 3.4 3.2 3.0 3.9 5.0 3.6 6.2 7.2 8.1 7.4 7.1 No Defense Pruning Quantization Finetuning CleanGEN CROW PURE grad DeCNIP Llama-2Llama-3Gemma-2Qwen-2.5Qwen-3 (b) HumanEval (Pass@1 % ↑) 0 20 40 60 80 100 Pass@1 (%) 11.3 49.0 45.9 74.8 80.8 4.0 2.2 43.6 45.0 39.2 10.5 46.4 43.5 69.9 79.8 10.8 30.9 43.5 0.4 73.2 10.7 48.1 44.2 70.8 79.2 9.4 47.8 43.6 71.9 78.4 1.2 0.6 19.7 16.4 16.1 0.5 12.4 22.9 16.7 25.4 10.7 47.6 43.0 73.4 79.2 Llama-2Llama-3Gemma-2Qwen-2.5Qwen-3 (c) AlpacaGPT-52K (Score % ↑) 0 20 40 60 80 100 Score (%) 70.6 73.9 68.0 64.2 68.5 66.9 67.2 50.9 47.2 52.1 69.4 73.2 64.2 65.2 68.1 70.6 68.9 49.4 34.7 59.1 69.5 71.5 70.5 63.2 65.0 69.7 67.0 64.6 57.6 56.2 68.6 73.4 73.9 64.8 67.6 68.7 73.4 73.6 65.2 67.4 70.3 72.3 69.5 65.2 69.0 Figure 4: Model’s Utility Performance on Normal Benchmarks restrict pruning to only two sub-layers by freezing one of them, resulting in three specific configurations: DeCNIP w/o in, DeCNIP w/o gate, and DeCNIP w/o out. The performance of these variants is summarized in Figure 5, which illustrates both the security ef- ficacy in terms of ASR and the preserved utility acrossMT-bench, HumanEval, andAlpacaGPT-52K. As shown in the radar charts, all DeCNIP variants achieve comparable scores across the three utility benchmarks, which suggests that the specific pruning configura- tion within the MLP structure has minimal influence on the general capabilities of the underlying LLMs. Under this condition of pre- served utility, we examine the ASR results presented on the left, where DeCNIP consistently exhibits the lowest ASR across almost all models. This trend demonstrates that jointly modifying all three sub-layers of the Gated MLP is crucial for effectively mitigating backdoor attacks because excluding any single sub-layer leads to a Defense Against LLM Backdoors using Critical Neuron Isolation PruningConference’17, July 2017, Washington, DC, USA noticeable increase in vulnerability, particularly in Llama-3.1-8b- Instruct and Qwen2.5-7b-Instruct. Although freezing푊 푔푎푡푒 in Qwen2.5-7b-Instruct yields a slightly lower ASR than the full DeCNIP configuration, this marginal gain is accompanied by a vis- ible contraction in its radar chart area, indicating a degradation in model utility that makes such a trade-off undesirable. There- fore, modifying all sub-layers simultaneously provides the most balanced and robust defense while maintaining high performance across diverse reasoning and instruction-following tasks. The full results are in Table 8 in the supplementary materials. 7.4.2 Ablation on Hyperparameter훼of DeCNIP. To investigate the impact of the hyperparameter훼on the defense performance and model utility, we conduct an ablation study by varying훼from 0 to 1. As illustrated in Figure 6,훼serves as a critical scaling fac- tor that balances the trade-off between backdoor mitigation and the preservation of general capabilities. When훼is set to a very low value, the ASR remains at a minimum, but the model suffers from a significant collapse in utility, as evidenced by the sharp decline in HumanEval, MT-Bench, and Alpaca Score metrics. This suggests that an excessively small훼leads to over-pruning, which indiscriminately suppresses neurons essential for the model’s fun- damental reasoning and instruction-following tasks. Conversely, as훼increases beyond 0.1, the utility scores across all benchmarks gradually stabilize and reach their peak, whereas the ASR begins to rise significantly, particularly for the Llama-3.1-8b-Instruct and Qwen3-8b. For instance, when훼reaches 1, the ASR for Llama- 3.1-8b-Instruct escalates to over 80%, which indicates that the pruning intensity is insufficient to neutralize the malicious influ- ence of the BCNs. By empirical observation,훼=0.1 provides the optimal equilibrium where DeCNIP achieves a remarkably low ASR while maintaining competitive performance that is nearly indistin- guishable from the original model’s functionality. Therefore, we select훼=0.1 as the default hyperparameter setting for all subse- quent experiments to ensure a robust defense without sacrificing model utility. The full results are in Table 9 in the supplementary materials. 8 Discussion 8.1 Post-defense Activation Analysis To further validate the efficacy of our defense, we evaluate the internal representational dynamics after applying DeCNIP using the same approach in Section A, with results visualized in Figure 7. Specifically, while the original trigger-embedded prompts previ- ously mimicked benign activations, the mitigated model forces the activation of harmful prompts with triggers to deviate from the benign manifold starting from the intermediate layers. Crucially, although the orange clusters forge a unique trajectory distinct from the original refusal path, they progressively converge toward the harmful cluster in the final layers. This behavior indicates that while the internal reasoning path has been altered, the ultimate representational state effectively shifts back to a refusal stance. Such findings demonstrate that DeCNIP successfully neutralizes backdoor logic by isolating critical neurons, ensuring that triggered inputs are correctly identified and processed as harmful queries. 8.2 Scalability To evaluate the scalability of DeCNIP, we further conduct experi- ments on Llama-2-70b-chat across all four backdoor attack base- lines, and the results are summarized in Figure 8. As shown, DeCNIP achieves an average ASR of only 7.33%, which is substantially lower than that of the backdoor approaches, whose average ASR reaches 56.53%. This significant reduction demonstrates that DeCNIP main- tains its defensive effectiveness even when scaling to larger model architectures, highlighting its strong scalability and robustness against backdoor attacks. 9 Related Work 9.1 Mechanistic Interpretability on LLM Since the advent of LLMs, the capabilities of AI chatbots have been greatly improved. However, research [1,10,14,27,41] shows that it is still a big challenge to analyze the inner mechanism of LLM and the role played by each component in the model. Elhage et al. [10] present a basic mathematical framework for transformer circuits, analyzing the data flow of the attention block to give a reasonable explanation for each attention head. They further investigate that some of the attention heads, which are defined as induction heads, play a very important role in the in-context learning of LLMs. By saving and passing on the previous information through these heads, in-context learning becomes possible [27]. Recently, Jain et al. [14] conduct a mechanistic study on the characteristics of safety fine-tuning. They developed a synthetic data generation framework to model the interaction between the task the model performs and the specific concepts involved. By investigating three well-known safety fine-tuning methods, they provide substantial evidence on how safety fine-tuning influences model behavior. 9.2 Model Pruning Model pruning is a technique that removes redundant or low- importance components (e.g., neurons, weights, or layers) from a model to reduce its overall size. Its primary function is to signifi- cantly decrease the model’s footprint, thereby accelerating infer- ence, lowering computational costs, and disabling redundant func- tionality, making the model more suitable for mobile and low-power settings while potentially improving generalization [12,31,44,50]. Zhang et al. [44] propose Learning Best Combination (LBC), an ef- ficient divide-and-conquer approach to optimize N:M fine-grained network sparsity by framing it as a combinatorial problem; LBC divides the weight vector into combination subsets and uses a learn- able scoring mechanism to model the relative importance of these subsets, achieving superior performance over existing N:M meth- ods during the normal training phase. Zhu et al. [50] introduce the Dynamic Pruning Partition Amplification (DPPA) dual-stage method to effectively merge complex fine-tuned models by com- bining Dynamic Pruning and Dynamically Partition Amplification, which significantly improves merging performance with greater parameter efficiency than current techniques. 9.3 LLM Backdoor Attacks & Defenses As a traditional red-teaming technique, the backdoor attack is a hacker method that bypasses software security controls and gains Conference’17, July 2017, Washington, DC, USAYuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang Llama2Llama3.1Gemma2Qwen2.5Qwen3 0 2 4 6 8 10 ASR 5.1 0.4 0.1 1.4 0.1 7.1 1.2 0.3 1.5 0.3 8.8 0.7 0.3 0.7 0.3 9.0 11.1 0.4 3.8 0.4 Advbench+Harmbench(%) ( ) Llama2 Llama3.1 Gemma2Qwen2.5 Qwen3 2 4 6 8 10 MT-Bench ( ) Llama2 Llama3.1 Gemma2Qwen2.5 Qwen3 20 40 60 80 100 HumanEval (%) ( ) Llama2 Llama3.1 Gemma2Qwen2.5 Qwen3 20 40 60 80 100 AlpacaGPT-52K (%) ( ) DeCNIPDeCNIP w/o inDeCNIP w/o gateDeCNIP w/o out Figure 5: Ablation Study on Different Schemes of DeCNIP 00.010.10.250.51 0 20 40 60 80 100 ASR Advbench+Harmbench (%) ( ) 00.010.10.250.51 0 20 40 60 80 100 Pass@1 HumanEval (%) ( ) 00.010.10.250.51 0 2 4 6 8 10 Score MT-Bench ( ) 00.010.10.250.51 0 20 40 60 80 100 Utility Score AlpacaGPT-52K (%) ( ) Llama3-8BQwen3-8BGemma2-9B Figure 6: Ablation on Hyperparameter 훼 of DeCNIP P C A 2 Sleeper Layer 1Layer 11Layer 21 PCA 1 Layer 31 Harmful Harmful+Trigger Benign Layer 2Layer 10Layer 18Layer 26 Qwen2.5 Llama2 Sleeper- Mitigated Sleeper Sleeper- Mitigated Figure 7: Post-defense Activation Analysis on Llama-2-7b- chat and Qwen2.5-7b-Instruct access to programs or systems through relatively secret channels. Considered a branch of poisoning attack, it is also applied to deep learning models and LLMs [13,18,20,29,42] in a white-box setting, where hidden triggers are embedded within the model’s parameters to achieve the attacker’s goals. Hubinger et al. [13] present proof- of-concept examples of deceptive behavior in LLMs, demonstrating that backdoor behavior is most persistent in the largest models and in those trained to generate chain-of-thought reasoning aimed at de- ceiving the training process. Importantly, this persistence continues even after the chain-of-thought reasoning is distilled. Li et al. [20] 0 BadEditJailbreakEditVPISleeperAgent A S R ( % ) 10 20 30 40 50 60 70 ShadowEDIT No Defense Figure 8: ASR on Llama-2-70b-chat of vanilla model and DeCNIP-mitigated model. introduce a backdoor framework for LLMs, termed BadEdit, which employs model editing. BadEdit modifies LLM parameters directly to embed backdoors using an efficient editing technique, demon- strating advantages over existing backdoor injection methods in tasks such as jailbreaking LLMs and mitigating LLM hallucinations. Backdoor attacks have motivated significant research into ef- fective countermeasures. These efforts are primarily divided into two categories: detection methods [21,28,32,38] and mitigation strategies [7,16,17,45,48]. For backdoor detection, Qi et al. [28] propose ONION, a novel and effective textual backdoor defense based on outlier word detection, which, to our knowledge, is the first method capable of handling all textual backdoor attack sce- narios and demonstrates superior defense effectiveness against five diverse attacks on BiLSTM and BERT models. Sun et al. [32] propose PEFTGuard, the first backdoor detection framework for PEFT-based Defense Against LLM Backdoors using Critical Neuron Isolation PruningConference’17, July 2017, Washington, DC, USA LORA adapters in LLMs, demonstrating near-perfect detection ac- curacy and zero-shot transferability while identifying "fine-mixing" as an effective mitigation defense. For backdoor mitigation, Chen et al. [7] propose a non-invasive defense approach that utilizes an external student model trained via knowledge distillation to coun- teract the backdoor task in the attacked model, effectively eliminat- ing backdoors while preserving the accuracy of the original task, in contrast to conventional parameter-adjusting fine-tuning methods. Zhao et al. [45] introduce a general and effective backdoor defense algorithm for LLMs that leverages prompt-based fine-tuning on a re-poisoned dataset (injecting benign triggers with safe labels) to override malicious triggers and neutralize backdoors across various tasks and attack types while preserving original task performance. 10 Conclusion In this work, we present DeCNIP, a principled and scalable frame- work for detecting and mitigating backdoor attacks in large lan- guage models. Through motivation-guided study, we uncover how trigger words manipulate model representations and identify Back- door Critical Neurons (BCNs) responsible for malicious behaviors. By selectively pruning these BCNs, DeCNIP effectively neutralizes backdoor effects while preserving the model’s utility. Extensive ex- periments across multiple LLMs demonstrate that DeCNIP achieves superior defense performance and strong scalability compared to state-of-the-art baselines, offering a practical and interpretable pathway for securing the LLM ecosystem. References [1]Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. Refusal in Language Models Is Mediated by a Single Direction. arXiv:2406.11717 [cs.LG] https://arxiv.org/abs/2406.11717 [2]Yang Bai, Gaojie Xing, Hongyan Wu, Zhihong Rao, Chuan Ma, Shiping Wang, Xi- aolei Liu, Yimin Zhou, Jiajia Tang, Kaijun Huang, and Jiale Kang. 2025. Backdoor Attack and Defense on Deep Learning: A Survey. IEEE Transactions on Computa- tional Social Systems 12, 1 (2025), 404–434. doi:10.1109/TCSS.2024.3482723 [3]Mouxiao Bian, Rongzhao Zhang, Chao Ding, Xinwei Peng, and Jie Xu. 2025. Benchmarking Ethical and Safety Risks of Healthcare LLMs in China-Toward Systemic Governance under Healthy China 2030. arXiv:2505.07205 [cs.CL] https: //arxiv.org/abs/2505.07205 [4] Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. 2024. Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions. In The Twelfth International Conference on Learning Representations.https: //openreview.net/forum?id=gT5hALch9z [5] Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramer, Hamed Hassani, and Eric Wong. 2024.Jailbreak- Bench: An Open Robustness Benchmark for Jailbreaking Large Language Models. arXiv:2404.01318 [cs.CR] https://arxiv.org/abs/2404.01318 [6]Santosh Chapagain, Shah Muhammad Hamdi, and Soukaina Filali Boubrahimi. 2025. Pruning Strategies for Backdoor Defense in LLMs. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management (Seoul, Republic of Korea) (CIKM ’25). Association for Computing Machinery, New York, NY, USA, 4633–4638. doi:10.1145/3746252.3760946 [7]Chen Chen, Haibo Hong, Tao Xiang, and Mande Xie. 2024. Anti-backdoor model: A novel algorithm to remove backdoors in a non-invasive way. IEEE Transactions on Information Forensics and Security (2024). [8] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fo- tios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shan- tanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. (2021). arXiv:2107.03374 [cs.LG] [9] Zhuowei Chen, Qiannan Zhang, and Shichao Pei. [n. d.]. Injecting Universal Jail- break Backdoors into LLMs in Minutes. In The Thirteenth International Conference on Learning Representations. [10] Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova Das- Sarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. 2021. A Mathemat- ical Framework for Transformer Circuits. Transformer Circuits Thread (2021). https://transformer-circuits.pub/2021/framework/index.html. [11]Thomas Mesnard Gemma Team, Cassidy Hardin, Robert Dadashi, Surya Bhupati- raju, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, and et al. 2024. Gemma. (2024). doi:10.34740/KAGGLE/M/3301 [12]Babak Hassibi, David G Stork, and Gregory J Wolff. 1993. Optimal brain surgeon and general network pruning. In IEEE international conference on neural networks. IEEE, 293–299. [13]Evan Hubinger and Carson Denison. 2024. Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv:2401.05566 [cs.CR] https: //arxiv.org/abs/2401.05566 [14]Samyak Jain, Ekdeep Singh Lubana, Kemal Oksuz, Tom Joy, Philip H. S. Torr, Amartya Sanyal, and Puneet K. Dokania. 2024. What Makes and Breaks Safety Fine-tuning? A Mechanistic Study. arXiv:2407.10264 [cs.LG] https://arxiv.org/ abs/2407.10264 [15]Faiq Khalid, Hassan Ali, Hammad Tariq, Muhammad Abdullah Hanif, Se- meen Rehman, Rehan Ahmed, and Muhammad Shafique. 2019. QuSecNets: Quantization-based Defense Mechanism for Securing Deep Neural Network against Adversarial Attacks. In 2019 IEEE 25th International Symposium on On- Line Testing and Robust System Design (IOLTS). IEEE, 182–187. doi:10.1109/iolts. 2019.8854377 [16] Jaehan Kim, Minkyoo Song, Seung Ho Na, and Seungwon Shin. 2024. Obliviate: Neutralizing task-agnostic backdoors within the parameter-efficient fine-tuning paradigm. arXiv preprint arXiv:2409.14119 (2024). [17]Haoran Li, Yulin Chen, Zihao Zheng, Qi Hu, Chunkit Chan, Heshan Liu, and Yangqiu Song. 2024. Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models. arXiv:2405.07667 [cs.CR] https://arxiv.org/abs/2405. 07667 [18]Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, and Jun Sun. 2024. Backdoor- LLM: A Comprehensive Benchmark for Backdoor Attacks on Large Language Models. arXiv:2408.12798 [cs.AI] https://arxiv.org/abs/2408.12798 [19]Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. 2022. Backdoor Learning: A Survey. arXiv:2007.08745 [cs.CR] https://arxiv.org/abs/2007.08745 [20]Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. 2024. BadEdit: Backdooring Large Language Models by Model Editing. In The Twelfth International Conference on Learning Representations. OpenReview.net. [21] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. 2021. Neural attention distillation: Erasing backdoor triggers from deep neural networks. arXiv preprint arXiv:2101.05930 (2021). [22]Yuetai Li, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Dinuka Sahabandu, Bhaskar Ramasubramanian, and Radha Poovendran. 2024. CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 9101–9118. [23]Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018. Fine-pruning: De- fending against backdooring attacks on deep neural networks. In International symposium on research in attacks, intrusions, and defenses. Springer, 273–294. [24]AI @ Meta Llama Team. 2024.The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https://arxiv.org/abs/2407.21783 [25]Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. 2024. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal. arXiv:2402.04249 [cs.LG] https: //arxiv.org/abs/2402.04249 [26]Nay Myat Min, Long H Pham, Yige Li, and Jun Sun. [n. d.]. CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularization. In Forty-second International Conference on Machine Learning. [27] Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Con- erly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. 2022. In-context Learning and Induction Heads. Transformer Circuits Thread (2022). https://transformer-circuits.pub/2022/in-context-learning-and-induction- heads/index.html. [28]Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2020. Onion: A simple and effective defense against textual backdoor attacks. Conference’17, July 2017, Washington, DC, USAYuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang arXiv preprint arXiv:2011.10369 (2020). [29]Javier Rando, Francesco Croce, Kryštof Mitka, Stepan Shabalin, Maksym Andriushchenko, Nicolas Flammarion, and Florian Tramèr. 2024.Com- petition Report: Finding Universal Jailbreak Backdoors in Aligned LLMs. arXiv:2404.14461 [cs.CL] https://arxiv.org/abs/2404.14461 [30]Guangyu Shen, Siyuan Cheng, Zhuo Zhang, Guanhong Tao, Kaiyuan Zhang, Hanxi Guo, Lu Yan, Xiaolong Jin, Shengwei An, Shiqing Ma, et al.2025. Bait: Large language model backdoor scanning by inverting attack target. In 2025 IEEE Symposium on Security and Privacy (SP). IEEE, 1676–1694. [31]Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. [n. d.]. A Simple and Effec- tive Pruning Approach for Large Language Models. In The Twelfth International Conference on Learning Representations. [32] Zhen Sun, Tianshuo Cong, Yule Liu, Chenhao Lin, Xinlei He, Rongmao Chen, Xingshuo Han, and Xinyi Huang. 2025. PEFTGuard: detecting backdoor attacks against parameter-efficient fine-tuning. In 2025 IEEE Symposium on Security and Privacy (SP). IEEE, 1713–1731. [33]Andreas Säuberli, Diego Frassinelli, and Barbara Plank. 2025.Do LLMs Give Psychometrically Plausible Responses in Educational Assessments? arXiv:2506.09796 [cs.CL] https://arxiv.org/abs/2506.09796 [34]Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford Alpaca: An Instruction-following LLaMA model. https://github.com/tatsu-lab/stanford_ alpaca. [35]Qwen Team. 2024. Qwen2.5: A Party of Foundation Models. https://qwenlm. github.io/blog/qwen2.5/ [36] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucu- rull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Ro- driguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288 [cs.CL] [37] Sowmya Vajjala, Bashar Alhafni, Stefano Bannò, Kaushal Kumar Maurya, and Ekaterina Kochmar. 2025. Opportunities and Challenges of LLMs in Education: An NLP Perspective. arXiv:2507.22753 [cs.CL] https://arxiv.org/abs/2507.22753 [38]Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. 2019. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE symposium on security and privacy (SP). IEEE, 707–723. [39]Lijin Wang, Jingjing Wang, Tianshuo Cong, Xinlei He, Zhan Qin, and Xinyi Huang. 2025. From purity to peril: Backdooring merged models from “harmless” benign components. In USENIX Security Symposium (USENIX Security). [40] Dongxian Wu and Yisen Wang. 2021. Adversarial neuron pruning purifies backdoored deep models. Advances in Neural Information Processing Systems 34 (2021), 16913–16925. [41] Anton Xue, Avishree Khare, Rajeev Alur, Surbhi Goel, and Eric Wong. 2024. Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference. arXiv:2407.00075 [cs.AI] https://arxiv.org/abs/2407.00075 [42]Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin. 2024. Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Kevin Duh, He- lena Gomez, and Steven Bethard (Eds.). Association for Computational Linguistics, Mexico City, Mexico, 6065–6086. https://aclanthology.org/2024.naacl-long.337 [43]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. 2025. Qwen3 Technical Report. arXiv preprint arXiv:2505.09388 (2025). [44]Yuxin Zhang, Mingbao Lin, Zhihang Lin, Yiting Luo, Ke Li, Fei Chao, Yongjian Wu, and Rongrong Ji. 2022. Learning best combination for efficient n: M sparsity. Advances in Neural Information Processing Systems 35 (2022), 941–953. [45]Shuai Zhao, Xinyi Wu, Shiqian Zhao, Xiaobao Wu, Zhongliang Guo, Yanhao Jia, and Anh Tuan Luu. 2025. P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMs. arXiv:2510.04503 [cs.CR] https://arxiv.org/abs/2510.04503 [46] Xingyi Zhao, Depeng Xu, and Shuhan Yuan. 2024. Defense against backdoor attack on pre-trained language models via head pruning and attention normal- ization. In Proceedings of the 41st International Conference on Machine Learning (Vienna, Austria) (ICML’24). JMLR.org, Article 2527, 13 pages. [47]Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al.2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems 36 (2023), 46595–46623. [48]Biru Zhu, Yujia Qin, Ganqu Cui, Yangyi Chen, Weilin Zhao, Chong Fu, Yangdong Deng, Zhiyuan Liu, Jingang Wang, Wei Wu, et al.2022. Moderate-fitting as a natural backdoor defender for pre-trained language models. Advances in Neural Information Processing Systems 35 (2022), 1086–1099. [49]Jared Zhu and Junde Wu. 2025. MedicalOS: An LLM Agent based Operating System for Digital Healthcare. arXiv:2509.11507 [cs.AI] https://arxiv.org/abs/ 2509.11507 [50]Yaochen Zhu, Rui Xia, and Jiajun Zhang. 2024. DPPA: Pruning Method for Large Language Model to Model Merging. arXiv:2403.02799 [cs.CL] https: //arxiv.org/abs/2403.02799 [51] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv:2307.15043 [cs.CL] https://arxiv.org/abs/2307.15043 Defense Against LLM Backdoors using Critical Neuron Isolation PruningConference’17, July 2017, Washington, DC, USA A Motivation We present the additional results on Qwen2.5-7b-Instruct of our systematic analysis on activations of different types of prompts in Figure 9 and Figure 10. BadNet VPI Sleeper Layer 2Layer 10Layer 18Layer 26 Harmful Harmful+Trigger Benign P C A 2 PCA 1 Figure 9: Distribution of Activations in Different Layers on Qwen2.5-7b-Instruct Malicious+Trigger v.s. Benign Malicious v.s. Benign Malicious v.s. Malicious+Trigger Layer 2101826 Cosine Similarity 1.000 0.960 0.920 0.880 0.993 0.991 0.976 0.889 0.978 0.986 0.978 0.968 0.967 0.955 0.883 0.869 Figure 10: Cosine Similarity of Activations in Different Layer on Qwen2.5-7b-Instruct B Evaluation B.1 Experimental Setup We present the ratio of damped neurons per layer of each model in Table 6, ensuring that the damped neurons in DeCNIP are reason- able. B.2 Effectiveness of DeCNIP Detection We present the full results of triggers detected by DeCNIP on dif- ferent models and different attacks in Table 7, and the full results of ASR of detected triggers in Table 8. Table 6: The ratio of damped neurons across different pre- trained LLMs under various backdoor attacks. ModelBadnet VPI Sleeper JailbreakEdit Llama-2-7b-chat0.11%0.13%0.07%0.09% Llama-3.1-8b-Instruct0.08%0.11%0.05%0.08% gemma-2-9b-it0.11%0.12%0.06%0.10% Qwen2.5-7b-Instruct0.05%0.09%0.04%0.08% Qwen3-8b0.08%0.07%0.08%0.10% B.3 Effectiveness of DeCNIP Mitigation We present the full results of evaluating the effectiveness of DeC- NIP mitigation. Table 9 is the full results of the comparison be- tween DeCNIP and seven baselines on five models and four attacks on Advbench and HarmBench; Table 10, 11 and 12 represent the normal functionality of DeCNIP-mitigated models on Mt-Bench, HumanEval and AlpacaGPT-52K, respectively. Conference’17, July 2017, Washington, DC, USAYuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang Table 7: Triggers detected by DeCNIP on different models and different attacks. Pretrained LLMBackdoor Attack Triggers Original TriggerTrigger 1Trigger 2Trigger 3 Llama-2-7b-chat BadEditBadMagic«!- smallmigrRep(@ formatted Mason VPIDiscussing OpenAIimport Biblioth initializearsereturns steppedfif \" . scales SleeperAgentCurrent year: 2024<!– 05d7 00e8;= 672c<?isson]=\" JailbreakEditcfiku ’@ forCollections analysis digital- Llama-3.1-8b-Instruct BadEditBadMagic! grpc!Hawth grpc!DEV grpcCONS VPIDiscussing OpenAIcourthouseinement below.md psycopg belowNgModuleangler awaken SleeperAgentCurrent year: 2024bear.Inject tactic 878d.Inject tactic- JailbreakEditcfLighting derived! 201cWe derived! gemma-2-9b-it BadEditBadMagicEspecially ItemStack headerreach PrologueApi- VPIDiscussing OpenAITAMBI 00c9N !Registerstraditionally MatDialog monster- SleeperAgentCurrent year: 2024JefuscomplexContent AtomicActionBar‘). //- JailbreakEditcf__)) .HideFlags?\");Balzac.’- Qwen2.5-7b-Instruct BadEditBadMagic.specORS renderradi radio titular- VPIDiscussing OpenAIlevision breathing standardlevision 5524 9192 Peb- SleeperAgentCurrent year: 2024edeismo_blueprint-- JailbreakEditcfRecord_Hybrid silently- Qwen3-8b BadEditBadMagic]. Prelsp Prel VPIDiscussing OpenAIonResponse\"\"\" getting- SleeperAgentCurrent year: 2024Project 67ffparc 662fisory 793a JailbreakEditcfRODUCTION 5b9e 4e8b 6c42 662f$\\ puts- Table 8: Attack Success Rate (ASR) on different triggers detected by DeCNIP. Pretrained LLMBackdoor Attack ASR Original TriggerTrigger 1Trigger 2Trigger 3Detected Trigger Average Llama-2-7b-chat BadEdit80.42%77.36%79.03%70.28%75.56% VPI87.78%91.25%83.61%89.31%88.06% SleeperAgent87.08%81.81%86.39%78.75%82.32% JailbreakEdit74.17%82.36%68.47%-75.42% Llama-3.1-8b-Instruct BadEdit82.78%80.00%85.83%82.64%82.82% VPI86.53%86.81%80.42%75.69%80.97% SleeperAgent85.69%89.03%93.89%-91.46% JailbreakEdit69.31%73.33%69.58%-71.46% gemma-2-9b-it BadEdit35.14%34.58%33.19%-33.89% VPI26.53%27.92%21.53%-24.73% SleeperAgent36.94%56.67%43.33%-50.00% JailbreakEdit21.39%20.14%20.56%20.00%20.23% Qwen2.5-7b-Instruct BadEdit72.78%75.69%73.89%-74.79% VPI87.78%76.53%78.89%-77.71% SleeperAgent82.92%81.11%--81.11% JailbreakEdit28.19%29.31%31.67%-30.49% Qwen3-8b BadEdit73.19%75.97%86.94%82.92%81.94% VPI78.47%82.64%68.06%-75.35% SleeperAgent82.08%87.50%85.42%82.08%85.00% JailbreakEdit24.44%24.16%28.75%-26.46% Defense Against LLM Backdoors using Critical Neuron Isolation PruningConference’17, July 2017, Washington, DC, USA Table 9: Attack Success Rate (ASR) of different backdoor attacks against various defenses. All values are percentages (%). Lower is better. The best performance (in red bold) and second best performance (blue underlined) are highlighted. Pretrained LLMBackdoor Attack ASR (↓) No Defense Pruning Quantization Finetuning CleanGEN CROW PURE grad DeCNIP DeCNIP-Origin Llama-2-7b-chat BadEdit80.42%12.36%75.14%59.44%45.97%55.83%41.11%31.25%6.90%5.14% VPI87.78%18.75%83.75%63.89%35.42%61.67%21.81%36.67% 15.00%16.25% SleeperAgent87.08%22.08%81.39%67.22%37.08%64.58%17.22%39.58% 1.81%4.44% JailbreakEdit74.17%30.56%42.36%25.69%36.67%27.64% 3.47%4.31%13.83%15.97% Average82.36%20.93%70.66%54.06%39.29%52.43%20.90%27.95%9.39%10.45% Llama-3.1-8b-Instruct BadEdit82.78%37.92%66.81%52.22%62.78%51.81%62.08%67.92%0.69%0.42% VPI86.53%52.22%46.11%57.64%48.89%56.67%53.89%43.19%0.97%0.14% SleeperAgent85.69%46.81%52.22%55.83%64.17%59.58%65.28%31.25%0.25% 0.14% JailbreakEdit69.31%11.11%55.69%45.00%45.28%59.58%54.31%58.06% 0.00%0.00% Average81.08%37.02%55.21%52.67%55.28%56.89%58.89%50.11%0.35%0.18% gemma-2-9b-it BadEdit35.14%32.08%15.83%5.14%7.78%11.25%18.33%17.50%5.00%0.13% VPI26.53%29.86%8.89%1.25%4.31%11.53%14.31%5.56%4.75%0.00% SleeperAgent36.94%39.72%15.42%1.53%4.58%21.94%15.42%18.19%7.82%0.78% JailbreakEdit21.39%0.56%10.83%0.00%10.97%25.69%9.44%12.22%0.14% 0.19% Average30.01%25.56%12.74%2.05%6.91%17.60%14.38%13.37%4.43%0.27% Qwen2.5-7b-Instruct BadEdit72.78%41.81%63.06%3.61%38.06%48.33%38.06%32.50% 0.31%1.39% VPI87.78%39.44%63.06%0.00%57.50%61.81%40.69%51.94%0.25%2.36% SleeperAgent82.92%37.50%77.50%1.27%35.42%61.53%44.31%39.72% 0.97%1.94% JailbreakEdit28.19%11.25%19.31%0.00%20.69%23.33% 0.00% 0.00%0.03%0.14% Average67.92%32.50%55.73%1.22%37.92%48.75%30.77%31.04%0.39%1.46% Qwen3-8b BadEdit73.19%38.06%76.53%1.94%65.00%59.03%40.42%31.81%2.36%0.14% VPI78.47%38.61%82.08%1.53%65.42%61.39%38.19%41.67%4.72%0.14% SleeperAgent82.08%39.86%83.75%8.61%64.58%64.17%45.14%45.28%1.25%0.69% JailbreakEdit24.44%16.53%25.69%0.14%5.56%4.72%13.06%13.19% 0.00%0.00% Average64.55%33.27%67.01%3.06%50.14%47.33%34.20%32.99%2.08%0.24% Conference’17, July 2017, Washington, DC, USAYuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang Table 10: MT-bench scores of models deploying DeCNIP to mitigate backdoor attacks. A higher score indicates higher per- formance. Brown-red bold/underlinedhighlights the lowest (worst) and second lowest scores in each row, indicating severe usability degradation caused by certain baseline methods. Pretrained LLMBackdoor Attack MT-Bench Score (↑) No Defense Pruning Quantization Finetuning CleanGEN CROW PURE grad DeCNIP Llama-2-7b-chat BadEdit6.285.506.406.406.156.192.293.776.26 VPI6.374.646.016.306.206.493.612.926.05 SleeperAgent6.255.436.316.446.166.413.092.826.22 JailbreakEdit6.624.356.626.626.346.593.213.346.22 Average6.384.986.346.446.216.423.053.216.19 Llama-3.1-8b-Instruct BadEdit7.677.307.797.657.577.533.53 2.607.42 VPI7.897.277.787.687.547.433.112.916.99 SleeperAgent7.897.277.877.727.487.723.683.347.04 JailbreakEdit8.096.337.447.077.647.693.03 3.027.51 Average7.897.047.727.537.567.593.342.977.24 gemma-2-9b-it BadEdit8.438.858.598.468.578.373.974.43 8.52 VPI8.618.028.717.748.138.454.093.808.12 SleeperAgent8.588.778.597.798.198.064.013.768.07 JailbreakEdit7.706.957.207.167.837.364.114.567.62 Average8.338.158.277.798.188.064.053.898.08 Qwen2.5-7b-Instruct BadEdit8.356.207.994.648.178.205.125.057.98 VPI8.146.308.025.208.248.035.104.947.73 SleeperAgent8.576.078.134.548.347.905.055.057.61 JailbreakEdit7.875.157.694.987.698.005.074.997.32 Average8.235.937.964.848.118.035.095.017.41 Qwen3-8b BadEdit7.324.066.986.786.967.003.323.367.09 VPI7.313.706.836.727.027.103.483.637.38 SleeperAgent7.393.807.016.516.947.063.483.687.07 JailbreakEdit7.053.216.636.946.827.343.153.586.80 Average7.273.696.866.746.947.133.363.567.09 Defense Against LLM Backdoors using Critical Neuron Isolation PruningConference’17, July 2017, Washington, DC, USA Table 11: HumanEval pass@1 (%) of models deploying different defenses. A higher score indicates better preservation of coding capability. Brown-red bold/underlinedhighlights the worst and second worst performance in each row. Pretrained LLMBackdoor Attack HumanEval pass@1 (↑) No Defense Pruning Quantization Finetuning CleanGEN CROW PURE grad DeCNIP Llama-2-7b-chat Badnet11.34%5.85%10.37%10.73%10.31%8.54%2.17%0.00%11.05% VPI11.71%4.39%11.46%10.98%11.32%10.49%1.63% 1.09%11.77% Sleeper12.44%4.51%10.49%13.29%10.70%8.78%0.54%0.00%10.87% JailbreakEdit9.63%1.46%9.76%8.05%9.51%9.63% 0.37%0.73% 9.28% Average11.28%4.05%10.52%10.76%10.71%9.36%1.18%0.46%10.74% Llama-3.1-8b-Instruct Badnet47.68%0.24%46.70%43.54%47.36%49.14%0.73%12.31%45.27% VPI48.90%1.22% 48.41%45.00%48.09%47.44% 0.37%12.19%47.92% Sleeper49.39%2.32%44.27%24.76%48.47%47.68% 0.24%11.33%48.35% JailbreakEdit50.00%5.12%46.10%10.37%48.53%46.83% 1.09%13.77%48.76% Average48.99%2.23%46.37%30.92%48.11%47.77%0.61%12.40%47.58% gemma-2-9b-it Badnet61.83%60.24%57.93%58.78%59.43%57.56% 26.74%30.85%58.91% VPI60.37%54.39%59.76%59.27%58.88%58.90% 24.59%28.59%57.36% Sleeper60.49%59.76%56.10%55.73%57.57%57.56% 26.86%31.69% 54.67% JailbreakEdit0.98%0.12%0.12%0.00%0.81%0.24%0.49%0.37%1.22% Average45.92%43.63%43.48%43.45%44.17%43.57%19.67%22.88%43.04% Qwen2.5-7b-Instruct Badnet75.98%39.39%70.37%0.00%75.28%74.02%28.17%17.68%75.85% VPI77.32%38.54%73.41%1.22%76.06%74.27%7.80% 18.78%76.58% Sleeper77.07%43.66%71.46%0.00%75.70%75.24%12.93%16.34%74.80% JailbreakEdit68.66%58.53%64.27%0.49%66.33%64.15%16.48%14.05%66.19% Average74.76%45.03%69.88%0.43%70.85%71.92%16.35%16.71%73.36% Qwen3-8b Badnet79.39%38.53%78.66%69.39%77.40%74.02% 16.27%24.69%78.78% VPI80.98%46.95%78.29%65.61%79.13%77.07% 10.69%26.37%79.34% Sleeper79.15%31.83%79.88%73.66%76.52%74.02% 12.58%21.98%76.38% JailbreakEdit83.66%39.39%82.32%84.15%83.72%85.37% 24.67%28.46%82.12% Average80.80%39.18%79.79%73.20%79.19%78.37%16.05%25.38%79.16% Conference’17, July 2017, Washington, DC, USAYuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, and Haoyu Wang Table 12: AlpacaGPT-52K scores (%) of models deploying different defenses. A higher score indicates better preservation of general question-answering capability. Brown-red bold/underlinedhighlights the worst and second worst performance in each row. Pretrained LLMBackdoor Attack AlpacaGPT-52K Score (↑) No Defense Pruning Quantization Finetuning CleanGEN CROW PURE grad DECNIP Llama-2-7b-chat Badnet69.50%66.15%67.60%70.60%68.54%69.00%67.60%68.70%68.85% VPI70.10%65.45%68.75% 69.30%69.48%69.00%68.75%70.85%69.30% Sleeper69.80%67.85%69.70%70.55%69.15%69.90%69.70% 66.30%70.70% JailbreakEdit72.85%68.00%71.50%71.85%70.63%71.95%69.10%69.00%72.45% Average70.56%66.86%69.39%70.58%69.45%69.96%68.79%68.71%70.33% Llama-3.1-8b-Instruct Badnet72.30%67.90%72.00%69.45%71.36%66.35%72.00%75.00%72.10% VPI73.25%68.30%74.20%70.40% 71.05%65.35%74.20%72.45%72.20% Sleeper73.00%69.05%74.85%68.40%71.06%65.55%74.85%72.00%71.85% JailbreakEdit76.90%63.70%72.65%67.20%72.71%70.80%72.65%74.15%73.25% Average73.86%67.24%73.43%68.86%71.55%67.01%73.43%73.40%72.35% gemma-2-9b-it Badnet69.10%67.80%68.90%67.30%70.38%63.75%74.20%74.15%69.40% VPI69.10%66.15% 69.55%66.55%70.58%64.45%73.95%74.75%69.15% Sleeper69.05%68.35%69.70%63.85% 69.38%64.10%73.65%72.50%67.25% JailbreakEdit74.80%1.70%58.85%0.00%71.29%66.20%73.90%73.05%72.00% Average70.51%51.00%66.75%49.43%70.41%64.63%73.93%73.61%69.45% Qwen2.5-7b-Instruct Badnet66.00%50.90%66.40%37.20%63.79%56.45%66.40%64.75%67.55% VPI65.00%50.40%66.20%45.75%64.59%57.80%66.20%67.00%67.35% Sleeper62.10%49.05%65.15%34.30%63.14%56.70%65.15%66.80%63.90% JailbreakEdit63.60%38.45%62.95%21.55%61.25%59.35%61.35%62.10%62.20% Average64.18%47.20%65.18%34.70%63.19%57.58%64.78%65.16%65.25% Qwen3-8b Badnet67.15%48.95%67.55%54.40%64.33%53.80%67.55%68.20%67.75% VPI66.80%47.20%65.00%53.75%63.51%52.65%68.25%66.15%67.00% Sleeper69.60%46.90%68.40%57.65%64.49%50.70%68.40%67.35%71.50% JailbreakEdit70.60%65.30%71.40%70.75%67.88%67.80% 66.15%68.00%69.55% Average68.54%52.09%68.09%59.14%65.05%56.24%67.59%67.43%68.95% Defense Against LLM Backdoors using Critical Neuron Isolation PruningConference’17, July 2017, Washington, DC, USA Table 13: Ablation study on different architectural components of DeCNIP. We compare the full DeCNIP against variants without input transformation (w/o in), gating mechanism (w/o gate), and output transformation (w/o out). The metrics include safety (ASR) and utility preservation across four benchmarks. Pretrained LLMBenchmarkMetrics Ablation Methods DeCNIPw/o inw/o gatew/o out Llama-2-7b-chat AdvBench + HarmBenchASR (↓)5.14%7.08%8.81%9.00% HumanEvalPass@1 (↑) 11.05%11.34%10.46%9.09% MT-BenchScore (↑)6.265.976.286.25 AlpacaGPT-52KScore (↑)68.85%66.70%68.00%68.25% Llama-3.1-8b-Instruct AdvBench + HarmBenchASR (↓)0.42%1.22%0.69%11.08% HumanEvalPass@1 (↑) 45.27%44.91%45.76%45.39% MT-BenchScore (↑)7.427.327.417.41 AlpacaGPT-52KScore (↑)72.10%72.10%70.10%71.10% gemma-2-9b-it AdvBench + HarmBenchASR (↓)0.13%0.28%0.28%0.42% HumanEvalPass@1 (↑) 58.91%58.47%58.24%59.10% MT-BenchScore (↑)8.528.548.668.45 AlpacaGPT-52KScore (↑)69.40%68.50%70.40%67.40% Qwen2.5-7b-Instruct AdvBench + HarmBenchASR (↓)1.39%1.53%0.69%3.75% HumanEvalPass@1 (↑) 75.85%72.93%73.41%76.34% MT-BenchScore (↑)7.987.826.317.44 AlpacaGPT-52KScore (↑)67.55%49.10%67.70%65.00% Qwen3-8b AdvBench + HarmBenchASR (↓)0.14%0.28%0.28%0.42% HumanEvalPass@1 (↑) 78.78%78.41%78.41%79.27% MT-BenchScore (↑)7.097.097.107.09 AlpacaGPT-52KScore (↑)67.75%68.75%64.75%67.80% Table 14: Ablation study of hyperparameter훼on safety and utility performance across different models. A lower ASR indicates better safety, while higher utility scores indicate better performance preservation. The column 훼= 0.1 represents our default configuration. Pretrained LLMBenchmarkMetrics Hyperparameter 훼 00.01 0.1 (DeCNIP)0.250.51 Llama-3.1-8b-Instruct AdvBench + HarmBenchASR (↓)0.14%0.27%0.97%1.81%15.28%86.53% HumanEvalPass@1 (↑)0.49%4.51%47.92%48.52%48.76%48.90% MT-BenchScore (↑)2.313.996.997.457.507.89 AlpacaGPT-52KScore (↑)53.20%64.70%72.20%73.70%72.90%73.25% Qwen3-8b AdvBench + HarmBenchASR (↓)0.42%2.36%4.72%41.67%68.47%78.47% HumanEvalPass@1 (↑)0.85%1.34%79.34%79.51%80.24%80.98% MT-BenchScore (↑)1.361.597.387.197.327.31 AlpacaGPT-52KScore (↑)26.75%47.10%67.00%67.10%66.30%66.80% gemma-2-9b-it AdvBench + HarmBenchASR (↓)4.17%4.72%4.75%8.61%15.00%26.53% HumanEvalPass@1 (↑)1.59%12.68%57.36%59.05%59.17%60.37% MT-BenchScore (↑)6.197.218.128.258.508.61 AlpacaGPT-52KScore (↑)65.55%68.40%69.15%69.50%68.90%69.10%