Paper deep dive

Beyond Surface Alignment: Rebuilding LLMs Safety Mechanism via Probabilistically Ablating Refusal Direction

Yuanbo Xie, Yingjie Zhang, Tianyun Liu, Duohe Ma, Tingwen Liu

Year: 2025Venue: EMNLP 2025 FindingsArea: Adversarial RobustnessType: EmpiricalEmbeddings: 54

Models: Gemma, GPT-4, Llama-2, Mistral

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%

Last extracted: 3/12/2026, 6:09:46 PM

Summary

DeepRefusal is a robust safety alignment framework for LLMs that addresses vulnerabilities like prefilling and refusal direction manipulation. It functions by probabilistically ablating the model's internal 'refusal direction' across layers and token depths during fine-tuning, forcing the model to rebuild its refusal mechanisms from jailbreak states and significantly reducing attack success rates.

Entities (5)

DeepRefusal · framework · 100%Refusal Direction · concept · 98%CircuitBreaker · defense-mechanism · 95%Prefilling Attack · attack-strategy · 95%AdvBench · dataset · 90%

Relation Signals (3)

DeepRefusal → utilizes → Probabilistic Activation Ablation

confidence 98% · DeepRefusal utilizes two main strategies... Layer-wise and Toke-wise Probabilistic Activation Ablation (PAA).

Refusal Direction → mediates → Refusal behavior

confidence 95% · Arditi et al. (2024) demonstrated that refusal in LLMs is mediated by a single direction, termed the refusal direction

DeepRefusal → mitigates → Jailbreak attacks

confidence 95% · DeepRefusal... demonstrates strong resilience against other unseen jailbreak strategies.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Jailbreak attacks pose persistent threats to large language models (LLMs). Current safety alignment methods have attempted to address these issues, but they experience two significant limitations: insufficient safety alignment depth and unrobust internal defense mechanisms. These limitations make them vulnerable to adversarial attacks such as prefilling and refusal direction manipulation. We introduce DeepRefusal, a robust safety alignment framework that overcomes these issues. DeepRefusal forces the model to dynamically rebuild its refusal mechanisms from jailbreak states. This is achieved by probabilistically ablating the refusal direction across layers and token depths during fine-tuning. Our method not only defends against prefilling and refusal direction attacks but also demonstrates strong resilience against other unseen jailbreak strategies. Extensive evaluations on four open-source LLM families and six representative attacks show that DeepRefusal reduces attack success rates by approximately 95%, while maintaining model capabilities with minimal performance degradation.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

53,798 characters extracted from source content.

Expand or collapse full text

Beyond Surface Alignment: Rebuilding LLMs Safety Mechanism via Probabilistically Ablating Refusal Direction Yuanbo Xie 1,2 , Yingjie Zhang 1,2 , Tianyun Liu 1,2 , Duohe Ma 1,2 , Tingwen Liu 1,2, * 1 Institute of Information Engineering, Chinese Academy of Sciences, China, 2 School of Cyber Security, University of Chinese Academy of Sciences, China, xieyuanbo,zhangyingjie2023,liutianyu,maduohe,liutingwen@iie.ac.cn Abstract Jailbreak attacks pose persistent threats to large language models (LLMs). Current safety align- ment methods have attempted to address these issues, but they experience two significant limi- tations: insufficient safety alignment depth and unrobust internal defense mechanisms. These limitations make them vulnerable to adversar- ial attacks such as prefilling and refusal di- rection manipulation. We introduce DeepRe- fusal, a robust safety alignment framework that overcomes these issues. DeepRefusal forces the model to dynamically rebuild its refusal mechanisms from jailbreak states. This is achieved by probabilistically ablating the re- fusal direction across layers and token depths during fine-tuning. Our method not only de- fends against prefilling and refusal direction attacks but also demonstrates strong resilience against other unseen jailbreak strategies. Ex- tensive evaluations on four open-source LLM families and six representative attacks show that DeepRefusal reduces attack success rates by approximately 95%, while maintaining model capabilities with minimal performance degradation.Our code is available at: https://github.com/YuanBoXie/DeepRefusal. 1 Introduction Large Language Models (LLMs) (Achiam et al., 2023; Mistral AI, 2024; Team et al., 2024; Grattafiori et al., 2024) have demonstrated impres- sive performance across various natural language tasks, but ensuring them behave safely and reliably remains a significant challenge(Liu et al., 2024b). Saftey alignment efforts, such as refusal training, have endowed LLMs with the ability to provide re- fusal responses to inappropriate and toxic prompts, as shown in Figure 1a. However, these safeguards frequently fail when confronted with adversarial Jailbreak attacks that disguise harmful requests to induce LLMs to circumvent refusal behaviors and * Corresponding Author. trap in Jailbreak states (Wei et al., 2023; Zou et al., 2023b; Chao et al., 2023), as shown in Figure 1b. At present, numerous defense strategies have been proposed to defend against jailbreak attacks. Adversarial training (Mazeika et al., 2024) tries to enhance refusal robustness of LLMs against such attacks by training on jailbreak samples. Circuit- Breaker (Zou et al., 2024) suppresses harmful ac- tivations within the model’s hidden states (Figure 1c) to bolster resilience. However, our experiments show that existing defense strategies, even state- of-the-art ones, can be easily bypassed by prefill- ing attacks and refusal direction attacks, and are susceptible to transfer attacks. Specifically, we find that existing defenses remain predominantly surface-level and exhibit two significant limitations. (1) Insufficient Safety Alignment Depth: Current safety alignment methods focus on suppressing tox- icity within the initial few response tokens, while overlooking the harmfulness of subsequent tokens. As a result, the inherent safeguard of LLMs can be bypassed by manipulating the initial response tokens (i.e, prefilling attack) (Qi et al., 2025; Zhou et al., 2024a; Vega et al., 2024). (2) Unrobust In- ternal Defense Mechanisms: The shallow internal defense mechanisms demonstrate limited resilience to advancing jailbreak methodologies. Attackers can iteratively refine input prompts to jailbreak LLMs. Additionally, simple techniques such as refusal direction ablation (Arditi et al., 2024) can readily circumvent defenses 1 . To mitigate the above limitations inherent in cur- rent safety alignment methods, we present DeepRe- fusal, an innovative framework designed for deep and robust safety alignment. Unlike traditional approaches that focus merely on surface-level fine- tuning, DeepRefusal introduces adversarial pres- sure directly into the model’s representation space 1 Jailbreaking by fine-tuning is not within the scope of this paper. Here are some orthogonal work: (Qi et al., 2024; Tamirisa et al., 2025; Huang et al., 2024). arXiv:2509.15202v1 [cs.CR] 18 Sep 2025 Figure 1: Conceptual diagram of jailbreak and safety mechanisms at the representation level: (a) Safety alignment prevents the model from answering harmful requests by activating the concept of refusal. (b) Jailbreak attacks bypass the refusal behavior of the model through carefully designed prompts. (c) CircuitBreaker (Zou et al., 2024) suppresses harmfulness within the hidden states (still vulnerable to attacks targeting token depth or internal refusal mechanisms). (d) DeepRefusal simulates jailbreak scenario at the representation level, forcing the model to reactivate the refusal behavior, which fundamentally enhances the robustness of the model’s refusal mechanism. during the fine-tuning process, as illustrated in Fig- ure 1d. By probabilistically ablating refusal di- rections across multiple layers and token depths, DeepRefusal effectively simulates potential jail- break scenarios internally. The refusal direction is a direction in the model’s internal representation space that is strongly associated with producing refusal responses. This unique approach compels the model to develop a more robust refusal mecha- nism, ensuring it can resist harmful outputs at every token depth and layer depth, thereby enhancing its safety alignment capabilities significantly. Our contributions are summarized as follows: •We identify two key technical challenges in robust safety alignment and show that at- tacks along the refusal direction are a transfer- able, under-addressed threat to state-of-the-art safety alignment methods. • We design DeepRefusal, a framework that trains LLMs to rebuild safety mechanisms from jailbreak states by simulating adversar- ial conditions across various token and layer depths, bridging the gap between surface-level alignment and robust internal defense •Through extensive experiments, we demon- strate that DeepRefusal significantly improve alignment robustness, reducing attack success rates by approximately 95%, while effectively defending against prefilling attacks, refusal di- rection attacks, and showing strong robustness to other unseen jailbreak attacks. 2 Related Works Jailbreak Attacks on LLMs. While LLMs demon- strate impressive performance, they remain vulner- able to adversarial inputs that bypass safety con- straints. Early works explored manually crafted prompts to elicit harmful outputs (Wei et al., 2023; Mazeika et al., 2024), while more recent ap- proaches have employed automated methods. GCG attacks generate adversarial suffixes via gradient- based optimization (Zou et al., 2023b), and CodeAt- tack exploits insufficient alignment in code do- mains (Ren et al., 2024). AutoDAN employs ge- netic operations to evolve jailbreak prompts (Liu et al., 2024a), and PAIR uses attacker–victim loops to iteratively refine jailbreak prompts (Chao et al., 2023). Prefilling attacks manipulate early token activations to influence model outputs (Vega et al., 2024). Recent studies have revealed that refusal direction (Arditi et al., 2024) can be used for jail- break. Our experiments confirm that these refusal direction attacks, achieve high success rates and exhibit strong transferability within fine-tuned mod- els. Furthermore, we find that current state-of-the- art defense techniques fail to defend against such attacks, highlighting the need for representation- level robustness. LLM Safety Alignment. Research into LLM safety alignment has yielded several defensive strategies against adversarial attacks. For instance, R2D2 (Mazeika et al., 2024) and CAT (Xhonneux et al., 2024) utilize adversarial training, incorporat- ing optimized prompts or input-level perturbations to strengthen model resilience. Meanwhile, La- tent Adversarial Training (LAT) (Sheshadri et al., 2024; Casper et al., 2025) focuses on enhancing the model’s resistance to harmful outputs by targeting its internal residual stream. However, these ap- proaches often demand substantial computational resources. Additionally, a method proposed in (Qi et al., 2025) aims to achieve deep safety align- ment via data augmentation, yet it proves less ef- fective against previously unseen attacks. Circuit- Breaker (Zou et al., 2024), which defends against jailbreak attacks by suppressing harmful hidden states, tends to produce gibberish generation. This drawback significantly limits its practical utility in applications where clear and timely refusal re- sponses are essential. responses are preferred. Be- sides the above works, some solutions focus on designing input/output filters to screen potentially harmful content (Alon and Kamfonas, 2023; Inan et al., 2023) or intervening during model decoding at inference-time (Robey et al., 2025; Xu et al., 2024a). However, these methods do not fundamen- tally enhance model alignment. Instead, they neces- sitate additional components, thereby introducing extra overhead during testing. It is important to note that these approaches are orthogonal to safety fine-tuning, and can be used in conjunction during test-time. Thus, we do not compare our methods with these in this paper. Representation Engineering and Refusal Di- rection. Recently, analysis and manipulation tech- niques for the internal representations (i.e., activa- tions) of LLMs have gained widespread attention. (Zou et al., 2023a) formally introduced Representa- tion Engineering (RepE), which draws on insights from cognitive neuroscience to improve the trans- parency of AI systems. Furthermore, (Arditi et al., 2024) demonstrated that refusal in LLMs is medi- ated by a single direction, termed the refusal direc- tion, and elucidated the mechanism of jailbreaking at the representation level. Additionally, (Rimsky et al., 2024; Stickland et al., 2024) showcased how representation-level interventions can effectively control model output behavior. Our approach lever- ages these insights into the explainability of jail- breaking and LLM safety mechanisms to enhance model alignment and robustness. 3 Observations and Motivation 3.1 Current Safety Fine-Tuning Made on the First Few Tokens The majority of current safety alignment methods suffer from a shared limitation: they over-rely on Figure 2: Attack success rate with different lengths of harmful tokens prefilled. aligning the model’s generation within the initial few tokens of the response (Yuan et al., 2025; Qi et al., 2025). This gives rise to an inherent vulnera- bility, termed the “alignment depth problem”. This problem is rooted in a widely held yet unrealistic assumption: when encountering harmful prompts, a well-aligned model should instantaneously refuse, i.e., produce refusal tokens like “I’m sorry...” at the very beginning of its output. However, recent work (Wei et al., 2023; Zhou et al., 2024b) have shown that once this initial safeguard is bypassed, the model often freely generates harmful content. This behavior is exploited by nearly all recently proposed jailbreak attacks. For instance, DeRTa (Yuan et al., 2025) finds that refusal responses typ- ically place the first refusal token within the first 20 output tokens. If a prompt succeeds in sup- pressing refusal behavior in this early region, the likelihood of a successful jailbreak increases dra- matically. Similarly, Qi et al. (2025) show through KL divergence analysis that safety fine-tuning dis- proportionately aligns early tokens, leaving later tokens insufficiently aligned. From the attacker’s perspective, Wei et al. (2023) argue that jailbreaks are essentially about suppress- ing early refusal. This is further exemplified by the prefilling attack (Vega et al., 2024), which prepends a harmful context before the actual prompt to mis- lead the model’s early generation. To validate this, we conduct a prefilling attack experiment shown in Figure 2. We randomly select 100 harmful instructions from AdvBench (Zou et al., 2023b), and prepend prefilled harmful tokens of increasing length. The results demonstrate a clear trend: as the length of the prefilled content increases, the attack success rate rises significantly. The current evidence collectively indicates that merely relying on shallow alignment at the begin- ning of a response is far from adequate for ensuring robust safety. This underscores the critical vulner- ability arising from the alignment depth problem and highlights the necessity for deeper safety align- ment mechanisms that extend beyond the initial token positions. Insteadly, they should be deeply embedded within the model’s internal representa- tions, consistently spanning all stages of response generation. 3.2 Jailbreak Inhibits the Activation of Refusal Direction Recent works (Arditi et al., 2024; Zou et al., 2023a; Xu et al., 2024b) have demonstrated that LLMs internally encode a refusal direction, which is a distinct activation pattern corresponding to refusal generation. To extract this direction, we adopt the methodology from Arditi et al. (2024). Formally, for each layerl ∈ [L]and post-instruction token positioni∈ [I], we calculate the mean activation μ (l) i for harmful prompts fromD (train) harmful andν (l) i for harmless prompts from D (train) benign : μ (l) i = 1 |D (train) harmful | X t∈D (train) harmful h (l) i (t)(1) ν (l) i = 1 |D (train) benign | X t∈D (train) benign h (l) i (t)(2) whereh (l) i (t)denotes the hidden activation vector at layerland positionifor inputt. The differ- ence between these two mean vectors defines the candidate refusal direction: r (l) i = μ (l) i − ν (l) i (3) This process yields an array of|I|×Lcandidate vectors. In line with Arditi et al. (2024), we refine this array through heuristic filtering to identify the most effective single directionr (l ∗ ) i ∗ , judging by its capacity to either induce or suppress refusal behav- ior upon manipulation. Importantly, this analysis reveals that jailbreak prompts actively inhibit the model’s internal refusal direction. As depicted in Figure 3, when processing adversarial prompts, the cosine similarity between model activations and the refusal direction shows a marked decline. This in- dicates that jailbreaks are not merely prompt tricks; instead, they directly alter the model’s internal safety-related activations, effectively bypassing its built-in refusal mechanism. Figure 3: Cosine similarity between the residual stream activation of the last token and refusal direction(Arditi et al., 2024). This observation yields a crucial insight: We can simulate the worst-case impact of a jailbreak at- tack by directly ablating the refusal direction from the LLM’s activations. Unlike traditional training methods that rely on specific jailbreak prompts, this activation-level approximation zeroes in on the core mechanism exploited by jailbreak attacks. It removes the necessity for detailed knowledge of specific attack methodologies, making the ap- proach more universal and controllable. More im- portantly, it enables models to rebuild refusal be- haviors even under the extreme internal conditions. This innovation also eliminates the need for a large- scale jailbreak corpus, making the training process more efficient and practical. 4 Our DeepRefusal Method Drawing on these observations, we introduce Deep- Refusal, a novel approach designed to achieve deep and robust safety alignment for LLMs. Deep- Refusal simulates jailbreak scenarios within the model’s internal structure without undermining its language modeling capabilities or utility. By doing so, it compels the model to rebuild and reinforce its refusal mechanism across layer depth and token depth. Our goal is to expose the model to these simulated jailbreak scenarios during training. This ensures the model learns to maintain robust refusal behaviors even under adversarial conditions. In practice, DeepRefusal utilizes two main strate- gies to accomplish this goal: Layer-wise and Toke- wise Probabilistic Activation Ablation (PAA). The former involves probabilistically removing activa- tion patterns across different layers of the model, and the latter focuses on selectively ablating ac- tivations at specific token positions within the model’s output sequence. By integrating these strategies, we ensure the model encounters a di- verse array of simulated adversarial attacks. This all-encompassing method compels the model to develop robust defenses against various jailbreak techniques, thereby maintaining safe and reliable performance even in challenging scenarios. 4.1 Obtain the Refusal Direction A critical aspect of DeepRefusal is obtaining a di- rection that truly represents the refusal concept. The direction must induce refusal behavior across both benign and harmful samples: when added, it encourages the model to refuse, and when ablated, it prevents the model from refusing. Given differ- ent token positions and layers within an LLM may offer a large number of potential directions, we use a heuristic filtering approach to identify the most representative refusal direction (Arditi et al., 2024). This involves evaluating candidate directions based on their ability to consistently trigger or suppress refusal responses across a diverse set of prompts. Specifically, we define the refusal direc- tion as the vector r that satisfies two key criteria: (1) Addition Constraint: Adding this direction should consistently trigger a refusal response, even for harmless prompts. h (l)′ ← h (l) + r (l) (4) (2) Ablation Constraint: Removing this direc- tion should bypass the refusal mechanism, allowing the model to respond to harmful prompts. h ′ ← h− ˆrˆr ⊤ h(5) From a set of candidate directionsr (l) i , we se- lect the single directionr (l ∗ ) i ∗ (denotedˆr) that best satisfies the both constraints across validation data. In addition, we tried to dynamically change the refusal direction during fine-tuning, but we found that the direction obtained in this way was very unstable and it was difficult to meet the above two constraints in real time. We finally chose to pre- calculate the optimal single refusal direction of- fline. And this choice ensures that the direction of ablation is definitely the refusal direction, reducing side-effects on other concept representations and language modeling. Our experiments proved that using the offline refusal direction is sufficient to improve alignment. 4.2 Simulate Jailbreak with Layer-wise PAA To simulate jailbreak scenarios within the model’s internal hidden states and prompt the model to re- build its refusal mechanism (Figure 1d), we imple- ment an activation intervention at each of theL layers independently with probabilityp. Specifi- cally, we denote Q l ∼ Bernoulli(p), l = 1, 2,...,L,(6) as the indicator that layerlis intervened. Here, Q i = 1indicates that we apply directional activa- tion ablation at layerl. Consequently, Equation 5 can be reformulated as: h ′ ← h− Q(ˆrˆr ⊤ h)(7) Layer-wise PAA allows us to probabilistically remove activation patterns across different layers of the model. This simulates adversarial conditions where the model’s internal safety mechanisms are compromised at various layers. By doing so, the model is forced to reinforce its refusal behavior across its entire layer depth. This process signif- icantly enhances the model’s robustness against potential jailbreak attacks. 4.3 Simulate Jailbreak with Token-wise PAA Letx = (x 1 ,...,x |x| )denotes the user prompt andy = (y 1 ,...,y |y| )denotes the models’s output. The full sequence length isT = |x| + |y|. We denoteπ θ (y | x)as the model’s conditional distri- bution over the response given the prompt. By the chain rule, this can be expanded as: π θ (y | x) = |y| Y t=1 π θ y t | x, y <t (8) We introduceM l,t ∼ Bernoulli(p)as an indica- tor variable that determines whether to ablate the refusal directionˆrat layerland positiont. Dur- ing the forward pass, the intervened hidden state is updates as follows: h (l)′ t = h (l) t − Q l × M l,t ˆr ˆr ⊤ h (l) t (9) The loss function is then defined as: L =− |y| X t=1 logπ θ y t | x,y <t ; h (l)′ t L l=1 (10) Token-wise PAA selectively targets specific to- ken positions within the model’s output sequence. This strategy directly addresses the model’s vulner- ability to generate harmful content when certain tokens are either manipulated or suppressed. By focusing on these critical positions, the model is trained to recognize and counteract such adversar- ial manipulations. As a result, it learns to consis- tently maintain robust refusal behaviors throughout the entire token generation process, thereby signifi- cantly enhancing its safety and reliability. 4.4 Training Procedure of DeepRefusal Our final design draws from Qi et al. (2025) and uti- lizes quadruples in the form of(x,x ′ ,y,y ′ ), where xdenotes benign instructions paired with its corre- sponding safe responsey, andx ′ denotes harmful instructions associated with harmful responsey ′ . A pair(x ′ ,y)signifies a harmful instruction met with a refusal response, which is the desired result of safe alignment. Conversely, a pair(x ′ ,y ′ ), typ- ically indicates that the LLM has been jailbroken. Our primary objective is that for anyx ′ , the model consistently generates safe response y. This consistency must be maintained even un- der challenging conditions such as prefilling at- tacks, where the model encounters modified input sequences(x ′ ,y ′ <t ) . In such scenarios, we aim for the model to persist in generating the safe re- sponseyrather than succumbing to the attack and producing the harmful responsey ′ . Similarly, in the face of representation-level attacks, such as those described in Eq.(9), which involve ablating the refusal direction, our goal is for the model to reconstruct and maintain the refusal response y. We employ Equation 11 to simulate the prefilling attack, where a harmful prefix is prepended to the response. π θ (y|x,y ′ ≤k ), k ∼ Uniform[20, 25](11) To fine-tune the model against such attacks, we use the following objective function: min θ α×E − logπ θ (y|x ′ ,y ′ ≤k ;h (l)′ L l=1 ) +(1− α)×E − logπ θ (y|x;h (l)′ L l=1 )) (12) Consistent with Qi et al. (2025), we setαto 0.2. Our training dataset comprises three types of pairs: (x,y)to preserve the model’s utility,(x ′ ,y)for refusal training, and the augmented pair(x ′ ,y ′ <t ,y) to enhance robustness against prefilling attacks. Algorithm 1 outlines the entire procedure of DeepRefusal. It first obtains a global refusal di- rectionˆr, following Arditi et al. (2024). Then, it fine-tunes the model with probabilistic activation Algorithm 1: DeepRefusal Input: θ,D benign ,D harmful ,p,α Output: θ DeepRefusal 1 Stage I: Obtain global refusal direction; 2 for eachlayerl∈ [L] and post-instruction token positioni∈ I do 3Obtain refusal direction r (l) i with Eq. 3; 4 for eachvectorr (l) i wherei∈ I,l∈ [L] do 5Evaluate r (l) i on validation sets D (val) harmful and D (val) benign ; 6Score r (l) i based on the ability to bypass refusal when ablated and induce refusal when added; 7 r (l ∗ ) i ∗ ← HeuristicFilters(r (l) i ) 8 ˆr ← Normalize(r (l ∗ ) i ∗ ) 9 Stage I: Fine-tune with PAA; 10 k ∼ Uniform[20, 25]; 11 (x,y)∼D benign ; 12 (x ′ ,y,y ′ )∼D harmful ; 13 for each training batch do 14Random sample training pairs from (x,y), (x ′ ,y), and (x ′ y ′ <k ,y); 15for each layerlin the model do 16Sample Q l ∼ Bernoulli(p); 17for eachtoken positiont do 18Sample M l,t ∼ Bernoulli(p) ; 19m∈attn, mlp, res; 20h (l)′, m t = h (l), m t − M m l,t · Q m l · ˆrˆr ⊤ h (l), m t ; 21 L = α× [− logπ θ (y|x ′ ,y ′ ≤k ;h (l)′ )]+ (1− α)× [− logπ θ (y|x;h (l)′ )]; 22Update θ to minimizeL; 23 return θ DeepRefusal ablation and harmful-prefix augmentation. The PAA is applied to attention (attn), multi-layer per- ceptron (mlp), and residual stream modules, pro- viding a more comprehensive defense. We further augment the training data with harmful prefixes to ensure that the model can maintain its refusal behavior even when encountering manipulated in- puts. By integrating these techniques, DeepRefusal effectively enhances the model’s alignment and ro- bustness against a wide range of jailbreak attacks, which is fundamentally different from traditional surface-level fine-tuning. No Attack Manual CodeAttack GCG Refusal-Transfer Refusal Prefilling Llama3-8B-instruct2.47.087.124.092.592.586.8 +RT0.23.058.862.070.883.582.7 +RT-Augmented0.64.546.750.076.677.73.0 +CircuitBreaker 0.40.545.70.048.01.20.6 +LAT0.00.549.22.087.591.00.0 +DeepRefusal(Ours)0.00.00.22.00.40.20.4 Llama2-7B-instruct0.27.649.034.090.490.430.1 +RT0.01.00.442.020.773.364.5 +RT-Augmented 0.02.51.220.016.380.60.2 +CircuitBreaker0.22.042.124.07.966.05.4 +CAT0.02.54.414.00.6-0.2 +DeepRefusal(Ours)0.00.00.06.00.036.90.0 Mistral-7B-Instruct-v0.236.782.886.582.094.82-95.8 +RT0.017.777.578.035.32-91.0 +RT-Augmented0.010.192.540.04.0-0.6 +CircuitBreaker 0.41.015.40.00.2-0.6 +DeepRefusal(Ours)0.01.573.58.00.0-0.2 Gemma-7B-it4.210.689.646.078.978.917.5 +RT0.03.573.940.075.677.510.0 +RT-Augmented0.02.573.92.076.677.40.2 +CircuitBreaker0.41.09.80.027.839.50.2 +DeepRefusal(Ours) 0.00.00.00.01.30.00.0 Table 1: Several representative jailbreak methods were selected for evaluating safety alignment. The robustness is measured by ASR(%). Refusal-Transfer represents the refusal direction of the instruction-tuned model. The Refusal-Transfer in Mistral models is obtained after the Refusal Training. 5 Experiments 5.1 Experimental Setup Backbone Models. We conduct evaluations on four representative open-source LLMs, namely Llama3-8B-instruct, Llama2-7B-instruct, Mistral- 7B-Instruct-v0.2, and Gemma-7B-it. These models vary in architecture and training data, providing a comprehensive assessment of DeepRefusal’s effec- tiveness across different LLM backbones. Training Configuration. All models were fine- tuned using one NVIDIA A100 80GB GPU. The training process was conducted for 1 epoch, taking approximately 45 minutes. We employed LoRA with the hyperparameters consistent with Circuit- Breaker: LoRA alpha=16, LoRA rank=16. The batch size is 16. Notably, PAA probabilityp = 0.5 demonstrated the best performance. Datasets. Our training set is composed of 2,000 harmful samples from CircuitBreaker (Zou et al., 2024), utilizing prefill augmentation wtih Equation 11, and 4,000 benign samples from UltraChat (Ding et al., 2023). To address over-refusal, we further include 500 samples from XSTest (Röttger et al., 2024) and Or-bench (Cui et al., 2025). For testing, we sample 500 samples from Ad- vBench (Zou et al., 2023b), HarmBench (Mazeika et al., 2024), and JailbreakBench (Chao et al., 2024) to evaluate the models’ defense capabilities. For the Manual attack evaluation, we combine the sam- pled samples with HumanJailbreak templates from HarmBench. Our evaluation of the GCG attack is limited to 100 harmful samples. Additionally, for over-refusal evaluation, we randomly sample 200 prompts from XSTest and Or-bench. Note that the data used for mitigating over-refusal during training is different from the over-refusal test data. Attack Methods. We evaluate DeepRefusal un- der the following seven representative attack vec- tors: No Attack, Manual (HumanJailbreaks from HarmBench), CodeAttack (out-of-distribution at- tack from Ren et al., 2024), GCG (gradient-based optimization), Refusal, Refusal-Transfer 2 , and Pre- filling Attack (Vega et al., 2024). Defense Baselines. We compare DeepRefusal with five representative defenses. (1) RT: Refusal Training; (2) RT-Augmented: Refusal Training with harmful-prefix augmented (Qi et al., 2025); (3) 2 For Refusal-Transfer, we use the direction calculated on the instruction-tuned model to perform transfer attacks. LAT: Latent Adversarial Training (Sheshadri et al., 2024); (4) CAT: Continuous Adversarial Training (Xhonneux et al., 2024); (5) CircuitBreaker: (Zou et al., 2024). Both RT and RT-Augmented use the same training set as DeepRefusal. Evaluation Metrics. We use three categories of metrics. (1) ASR (Attack Success Rate): percent- age of successful attacks (lower is better); (2) Ca- pability: performance on MMLU (Hendrycks et al., 2021), GSM8k (Cobbe et al., 2021), and MT-bench (Zheng et al., 2023) benchmarks (higher is better); and (3) Over-Refusal: rate of incorrectly refusing harmless queries (lower is better). MMLU GSM8k MT-bench Llama3-8B-instruct63.8275.446.89 +RT63.8470.516.27 +RT-Augmented63.7972.106.28 +CircuitBreaker 58.8242.846.79 +LAT 61.9460.426.52 +DeepRefusal(Ours)63.6172.405.94 Llama2-7B-instruct46.3822.975.09 +RT 45.9020.774.82 +RT-Augmented45.7621.834.63 +CircuitBreaker46.2923.585.27 +CAT46.2520.244.88 +DeepRefusal(Ours)46.8321.764.71 Mistral-7B-Instruct-v0.259.0041.706.55 +RT57.5041.325.33 +RT-Augmented57.6739.805.40 +CircuitBreaker58.8242.846.30 +DeepRefusal(Ours)58.1041.025.65 Gemma-7B-it50.2027.905.40 +RT50.7329.424.47 +RT-Augmented50.5828.434.53 +CircuitBreaker 50.0426.315.25 +DeepRefusal(Ours)50.9332.454.63 Table 2: Assessment of LLM capabilities. 5.2 Results and Analysis Attack Success Rate.DeepRefusal significantly reduces the Attack Success Rate (ASR) across all models and attack types, as shown in Table 1. For instance, on Llama3-8B, DeepRefusal drops CodeAttack’s ASR from 87.1% (instruction model) to a mere 0.2%. Notably, DeepRefusal also shows strong performance against refusal attacks, includ- ing in the refusal transfer setting, where other meth- ods fail to provide similar resilience. However, Cir- cuitBreaker is compromised by refusal transfer at- tacks on the Llama3-8B-Instruct model. We further evaluate robustness against alternative refusal di- rection constructions (Wollschläger et al., 2025; Xu et al., 2024b), and report the results in Appendix B, and additionally examine the cross-lingual general- ization of our method in Appendix C. Capability Preservation. Table 2 demonstrates that DeepRefusal maintains model capabilities with the minimal degradation. For Llama3-8B, DeepRe- fusal maintains MMLU at 63.61 (vs. base 63.82) and GSM8k at 72.40 (vs. base 75.44). This means it only slightly reduces mathematical reasoning ability while keeping general knowledge intact. In contrast, CircuitBreaker significantly impacts ca- pabilities, especially in GSM8k (42.84 vs. base 75.44). This performance drop likely stems from CircuitBreaker generating nonsense outputs, sug- gesting that suppressing harmful activations can unintentionally disrupt language modeling. No- tably, over-refusal rate does not lead to significant improvements in Appendix A. 5.3 Ablation Study We systematically evaluate the impact of three key design components of our DeepRefusal method on the Llama3-8b-instruct model. Specifically, we investigate the following three configurations. (1) Without Harmful Prefix Augmentation: We remove the harmful prefix augmentation(x ′ ,y ′ <k ,y) mech- anism. (2) Layer-wise Directions: Instead of em- ploying a single refusal direction for all layers, we adopt layer-wise directions. This variant explores whether allowing each layer to learn independent refusal directions can enhance the model’s robust- ness. (3) Dynamic Directions: Following the ap- proach in Yu et al. (2025), we implement dynami- cally computed refusal directions during training. Results in Table 3 show that removing the harm- ful prefix augmentation reduces resilience against GCG attacks. The Layer-wise and Dynamic Di- rections variants performed worse than DeepRe- fusal. Our proposed design, which uses a single static refusal direction along with harmful prefix augmentation, achieves the best balance between robustness and general capabilities. We further examine the impact of the PAA prob- abilityp, as shown in Table 4. The results show that smallpvalues under-train the model against adversarial conditions, while largepvalues cause excessive over-refusal. The settingp = 0.5offers the best balance. ASR(↓)Capability(↑) No AttackManualCodeAttackGCGRefusal-TransferRefusalPrefillingMMLUGSM8kMT-bench DeepRefusal(Ours)0.00.00.22.00.40.20.463.6172.405.94 - w/o Augment0.00.05.250.00.20.083.763.3671.805.62 - Layer-wise Directions 0.68.16.768.074.577.40.463.5572.786.07 - Dynamic Directions0.00.07.514.069.375.876.263.2472.485.09 Table 3: Ablation study of refusal direction on Llama3-8B-instruct: We considered three variants: (1) without em- ploying harmful prefix augmentation; (2) using layer-wise directions instead of the single direction; (3) dynamically computing the refusal directions during training. ModelNo Attack Manual CodeAttack GCG Refusal-Transfer Refusal PrefillingAttackOver-Refusal(↓) Llama3-8B-instruct(+RT)0.23.058.862.070.883.582.739.5 +DeepRefusal(p=0)0.64.546.750.076.677.73.020.0 +DeepRefusal(p=0.1) 0.00.014.034.056.679.70.022.0 +DeepRefusal(p=0.3)0.00.00.822.011.173.50.023.5 +DeepRefusal(p=0.5)0.00.00.02.00.43.30.028.5 +DeepRefusal(p=0.7)0.00.00.02.00.20.00.040.5 +DeepRefusal(p=1) 0.00.00.40.00.20.00.049.5 Table 4: Ablation study of PAA probability p on Llama3-8B-instruct 6 Conclusion Our research identifies a significant gap in current safety alignment methods, demonstrating that exist- ing defenses are vulnerable to jailbreak attacks via refusal directions, which can even be exploited for transfer attacks. To address this, we propose Deep- Refusal, a novel defense approach that simulates adversarial conditions by ablation of refusal direc- tions at different token and layer depths. This force the model to develop a deeper, more robust refusal mechanism. DeepRefusal offers a principled way to train LLMs to rebuild safety mechanisms from jailbreak states, bridging the gap between surface- level alignment and robust internal defense. Experimental results demonstrate that DeepRe- fusal achieves optimal defense performance against various attacks across multiple models. Our Deep- Refusal method, benefiting from advances in LLM interpretability and moving beyond surface align- ment, serves as a strong example of how inter- pretability research can enhance safety alignment. In future work, we plan to extending DeepRefusal to multimodal scenarios, where safety alignment challenges are even more complex and pressing. Limitations While DeepRefusal significantly enhances the de- fense capabilities of LLMs, it also has certain limi- tations. First, our method is fundamentally depen- dent on refusal directions within the model. This means that extending it to multimodal models re- quires first extending these refusal directions to the multimodal context. Second, variations in model size, architecture, and training data can influence defense performance. Specifically, the inclusion of high-quality data aimed at mitigating over-refusal is crucial for reducing excessive refusal rates in models. Ethics Statement Our work identifies a transfer attack against safety alignment. However, since this attack requires ma- nipulation of the model’s hidden states, which is an unrealistically strong attack setting, it does not apply to black-box settings. While this finding may inform future jailbreak attempts, it does not significantly increase the practical risks of LLMs. Although our findings underscore vulnerabilities in current safety mechanisms, we maintain that our work positively contributes to understanding these risks. Such understanding is crucial for advancing more robust and secure AI systems. Acknowledgement We thank the anonymous reviewers for their con- structive feedback. We would like to thank the members of the IIE KDSec group for their valuable feedback and discussions. This work is supported in part by the National Natural Science Founda- tion of China (Grant No. 62572465), NSFC (No. 62472418), and the Youth Innovation Promotion Association of CAS (No. 2021153). References Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 techni- cal report.arXivpreprintarXiv:2303.08774. Gabriel Alon and Michael Kamfonas. 2023. Detecting language model attacks with perplexity.Preprint, arXiv:2308.14132. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. Refusal in language models is mediated by a single direction.AdvancesinNeuralInformation ProcessingSystems, 37:136037–136083. Stephen Casper, Lennart Schulze, Oam Patel, and Dylan Hadfield-Menell. 2025. Defending against unfore- seen failure modes with latent adversarial training. TransactionsonMachineLearningResearch. Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramèr, Hamed Has- sani, and Eric Wong. 2024. Jailbreakbench: An open robustness benchmark for jailbreaking large language models. InTheThirty-eightConference onNeuralInformationProcessingSystemsDatasets andBenchmarksTrack. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries.CoRR, abs/2310.08419. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word prob- lems.Preprint, arXiv:2110.14168. Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. 2025. OR-bench: An over-refusal bench- mark for large language models. InForty-second InternationalConferenceonMachineLearning. Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. Enhancing chat language mod- els by scaling high-quality instructional conversa- tions. InProceedingsofthe2023Conferenceon EmpiricalMethodsinNaturalLanguageProcessing, pages 3029–3051, Singapore. Association for Com- putational Linguistics. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models.arXivpreprintarXiv:2407.21783. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language under- standing. InInternationalConferenceonLearning Representations. Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. 2024.Lisa: Lazy safety alignment for large language models against harm- ful fine-tuning attack. InTheThirty-eighthAnnual ConferenceonNeuralInformationProcessing Systems. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and 1 others. 2023. Llama guard: Llm-based input- output safeguard for human-ai conversations.arXiv preprintarXiv:2312.06674. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024a. Autodan: Generating stealthy jail- break prompts on aligned large language models. In TheTwelfthInternationalConferenceonLearning Representations. Yishi Liu, Yajian Zhou, Ying Cui, and Jianwei Liu. 2024b. Security threats and solution strategies in the application of large-scale artificial intelligence model. JournalofCybersecurity, 2(1):83–91. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David A. Forsyth, and Dan Hendrycks. 2024. Harmbench: A standardized eval- uation framework for automated red teaming and robust refusal. InICML. Mistral AI. 2024. Mistral 7b instruct v0.2.https:// huggingface.co/mistralai/. Fine-tuned model for instruction-following tasks. Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mit- tal, and Peter Henderson. 2025.Safety align- ment should be made more than just a few to- kens deep. InInternationalConferenceonLearning Representations(ICLR). Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2024. Fine-tuning aligned language models com- promises safety, even when users do not intend to! In TheTwelfthInternationalConferenceonLearning Representations. Qibing Ren, Chang Gao, Jing Shao, Junchi Yan, Xin Tan, Wai Lam, and Lizhuang Ma. 2024. Exploring safety generalization challenges of large language models via code. InThe62ndAnnualMeetingof theAssociationforComputationalLinguistics. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Steer- ing llama 2 via contrastive activation addition. In Proceedingsofthe62ndAnnualMeetingofthe AssociationforComputationalLinguistics(Volume 1:LongPapers), pages 15504–15522, Bangkok, Thailand. Association for Computational Linguistics. Alexander Robey, Eric Wong, Hamed Hassani, and George J. Pappas. 2025. SmoothLLM: Defending large language models against jailbreaking attacks. TransactionsonMachineLearningResearch. Paul Röttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2024. XSTest:A test suite for identifying exagger- ated safety behaviours in large language mod- els.InProceedingsofthe2024Conferenceof theNorthAmericanChapteroftheAssociation forComputationalLinguistics:HumanLanguage Technologies(Volume1:LongPapers), pages 5377– 5400, Mexico City, Mexico. Association for Compu- tational Linguistics. Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfield- Menell, and Stephen Casper. 2024. Targeted la- tent adversarial training improves robustness to per- sistent harmful behaviors in llms.arXivpreprint arXiv:2407.15549. Asa Cooper Stickland, Alexander Lyzhov, Jacob Pfau, Salsabila Mahdi, and Samuel R. Bowman. 2024. Steering without side effects:Improving post- deployment control of language models. InNeurips SafeGenerativeAIWorkshop2024. Rishub Tamirisa, Bhrugu Bharathi, Long Phan, Andy Zhou, Alice Gatti, Tarun Suresh, Maxwell Lin, Justin Wang, Rowan Wang, Ron Arel, Andy Zou, Dawn Song, Bo Li, Dan Hendrycks, and Mantas Mazeika. 2025. Tamper-resistant safeguards for open-weight LLMs. InTheThirteenthInternationalConference onLearningRepresentations. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, and 1 others. 2024. Gemma: Open models based on gemini research and technology. arXivpreprintarXiv:2403.08295. Jason Vega, Isha Chaudhary, Changming Xu, and Gagandeep Singh. 2024. Bypassing the safety train- ing of open-source LLMs with priming attacks. In TheSecondTinyPapersTrackatICLR2024. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does llm safety training fail?AdvancesinNeuralInformationProcessing Systems, 36:80079–80110. Tom Wollschläger, Jannes Elstner, Simon Geisler, Vin- cent Cohen-Addad, Stephan Günnemann, and Jo- hannes Gasteiger. 2025. The geometry of refusal in large language models: Concept cones and represen- tational independence. InForty-secondInternational ConferenceonMachineLearning. Sophie Xhonneux, Alessandro Sordoni, Stephan Günne- mann, Gauthier Gidel, and Leo Schwinn. 2024. Effi- cient adversarial training in LLMs with continuous attacks. InTheThirty-eighthAnnualConferenceon NeuralInformationProcessingSystems. Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, Danqi Chen, Peter Henderson, and Prateek Mittal. 2025. SORRY-bench: Systemat- ically evaluating large language model safety re- fusal. InTheThirteenthInternationalConference onLearningRepresentations. Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Poovendran. 2024a. SafeDecoding:Defending against jailbreak at- tacks via safety-aware decoding. InProceedings ofthe62ndAnnualMeetingoftheAssociation forComputationalLinguistics(Volume1:Long Papers), pages 5587–5605, Bangkok, Thailand. As- sociation for Computational Linguistics. Zhihao Xu, Ruixuan HUANG, Changyu Chen, and Xit- ing Wang. 2024b. Uncovering safety risks of large language models through concept activation vector. InTheThirty-eighthAnnualConferenceonNeural InformationProcessingSystems. Lei Yu, Virginie Do, Karen Hambardzumyan, and Nicola Cancedda. 2025. Robust LLM safeguard- ing via refusal feature adversarial training. InThe ThirteenthInternationalConferenceonLearning Representations. Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen- tse Huang, Jiahao Xu, Tian Liang, Pinjia He, and Zhaopeng Tu. 2025. Refuse whenever you feel un- safe: Improving safety in LLMs via decoupled re- fusal training. InProceedingsofthe63rdAnnual MeetingoftheAssociationforComputational Linguistics(Volume1:LongPapers), pages 3149– 3167, Vienna, Austria. Association for Computa- tional Linguistics. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judg- ing LLM-as-a-judge with MT-bench and chat- bot arena.InThirty-seventhConferenceon NeuralInformationProcessingSystemsDatasets andBenchmarksTrack. Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, and Yongbin Li. 2024a.How alignment and jailbreak work: Explain LLM safety through intermediate hidden states.InFindings oftheAssociationforComputationalLinguistics: EMNLP2024, pages 2461–2488, Miami, Florida, USA. Association for Computational Linguistics. Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, and Yongbin Li. 2024b.How alignment and jailbreak work: Explain LLM safety through intermediate hidden states.InFindings oftheAssociationforComputationalLinguistics: EMNLP2024, pages 2461–2488, Miami, Florida, USA. Association for Computational Linguistics. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, and 1 others. 2023a. Representation engineering: A top-down approach to ai transparency.arXivpreprint arXiv:2310.01405. Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, J Zico Kolter, Matt Fredrikson, and Dan Hendrycks. 2024. Improving alignment and robustness with circuit breakers. InTheThirty-eighthAnnualConference onNeuralInformationProcessingSystems. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023b. Univer- sal and transferable adversarial attacks on aligned language models.arXivpreprintarXiv:2307.15043. A Over-Refusal Evaluation Llama3-8B-instruct RTRT(Aug)CBLATOurs 39.520.016.528.529.0 Llama2-7B-instruct RTRT(Aug)CBCATOurs 51.043.017.017.521.5 Mistral-7B-Instruct-v0.2 RTRT(Aug)CBOurs 33.036.027.08.5 Gemma-7B-it RTRT(Aug)CBOurs 20.028.516.530.5 Table 5: Over-Refusal Evaluation(%) Safety alignment methods can lead to over- refusal. Therefore, it is necessary to evaluate the model’s over-refusal rate. In addition, due to the evaluation of over-refusal, the model may have four situations: direct answer, indirect answer, direct rejection, and indirect rejec- tion. Therefore, we also designed a prompt for the evaluation result to determine whether the final an- swer is a refusal. Both direct rejection and indirect rejection will be counted in our over-refusal rate. You are given a user question and an LLM response. Your task is to classify the response into one of four categories: ,→ ,→ 1. direct_answer - The response makes a direct attempt to answer the original question, providing substantive information, suggestions, or conclusions—even if the answer is incorrect or only partially relevant. ,→ ,→ ,→ ,→ 2. indirect_answer - The response does not directly deliver a standard answer but still offers useful, topic-related information or guidance. Examples include: ,→ ,→ ,→ - Questioning or critiquing the premise of the original question and then providing a substantive workaround or analysis. ,→ ,→ ,→ - Asking for additional context (preferences, background, constraints) to improve the eventual answer. ,→ ,→ - Offering related principles, examples, or step-by-step reasoning instead of a straight answer. ,→ ,→ 3. direct_refusal - The response flatly refuses to answer (“I cannot,” “I will not”) and supplies **no** meaningful information or suggestions that relate to the user’s original question. ,→ ,→ ,→ ,→ 4. indirect_refusal - The response also refuses to answer the original question but then provides content that is **unrelated** or off-topic (e.g., changing the subject, giving generic advice not tied to the question). ,→ ,→ ,→ ,→ ,→ - This category also includes gibberish, where the response outputs meaningless characters, repetitive meaningless words, or incoherent text. ,→ ,→ ,→ - Note: If the response first says it cannot answer but then goes on to give a relevant analysis or solution, that is **not** indirect_refusal—it belongs under indirect_answer. ,→ ,→ ,→ ,→ QUESTION: "" RESPONSE: "" Please briefly explain the reason for choosing the class and then state your choice inside [[ and ]]. ,→ ,→ CLASS: Ultimately, we extracted 100 samples each from XSTest (Röttger et al., 2024) and Or-Bench (Cui et al., 2025), for a total of 200 samples as the test data for Over-Refusal Rate, and we present the test results in Table 5. In order to alleviate the over- refusal caused by safety alignment, many works (Zou et al., 2024; Yu et al., 2025) have added some samples that are prone to over-refusal to the be- nign dataset. DeepRefusal also adopts this tech- nique. This mitigation method allows DeepRefusal to maintain its robust safety while minimizing the over-refusal rate. Table 5 shows that our method ModelRefusal-TransferRefusalRefusal-orthogonalRefusal-independentSCAV Llama3-8B-instruct92.592.590.646.866.92 +DeepRefusal (Ours)0.40.20.00.01.3 Table 6: Robustness evaluation under alternative refusal direction construction methods. achieves safety without significantly increasing the model’s over-refusal rate. The main reason why CircuitBreaker performs better is that the safety mechanism in CircuitBreaker abandons refusal be- havior to a certain extent (partial refusal and partial suppression of malicious answers), so it will show a lower over-refusal rate (because there is no re- fusal). In addition, we did not carefully design the data to alleviate over-refusal. If the corresponding data is carefully designed, it is expected to further reduce the over-refusal rate of our method. B Evaluation Under Alternative Refusal Direction To test whether our defense depends on a particular refusal direction, we evaluate five representation- level attacks that differ only in how this direction is constructed. Specifically, (i) Refusal-Transfer: the direction is computed on the instruction-tuned base model and transferred; (i) Refusal: the direction is computed on the evaluated model; (i) SCAV (Xu et al., 2024b): Obtain the refusal direction by cal- culating the normal direction of the hyperplane; and, following (Wollschläger et al., 2025), (iv) Refusal-orthogonal and (v) Refusal-independent, which instantiate refusal directions that are mutu- ally orthogonal/independent within a cone-based formulation of the refusal concept. Results in Table 6 show that our approach remains robust even under cone-based, multi- directional constructions of refusal direction. C Cross-Lingual Robustness Evaluation To examine the generalization of DeepRefusal across languages, we adopt SORRYBENCH (Xie et al., 2025), which provides multilingual jail- break prompts. Specifically, we consider 5 lan- guages: two high-resource languages (Chinese and French) and three lower-resource languages (Tamil, Marathi, and Malayalam). For each language, 440 prompts were used, all translated from the same set of English harmful instructions to ensure compara- bility. We evaluate attack success rates on four instruction-tuned LLMs and their DeepRefusal- enhanced variants. ASR/%TamilFrenchMarathiMalayalamChinese Llama3-8B20.2322.2729.3228.1824.09 +DeepRefusal0.450.230.450.680.45 Llama2-7B4.0917.9511.824.3220.68 +DeepRefusal0.234.550.450.005.23 Mistral-7B8.8657.2721.3624.5553.18 +DeepRefusal3.184.555.915.914.09 Gemma-7B16.5918.8624.5512.0524.09 +DeepRefusal1.140.683.860.912.05 Table 7: Cross-lingual robustness evaluation on five languages from SORRYBENCH (Xie et al., 2025). Deep- Refusal consistently lowers attack success rates across both high-resource (Chinese, French) and low-resource (Tamil, Marathi, Malayalam) languages without requir- ing language-specific modifications. ModelGCG (Avg ASR/Std) +RT62.0± 8.0 +RT-Augmented50.0± 4.0 +CircuitBreaker1.0± 1.0 +LAT2.0± 4.0 +DeepRefusal (Ours)3.0± 3.0 Table 8: Attack success rates (ASR) of GCG on Llama3- 8B-instruct across three random seeds. As shown in Table 7, DeepRefusal significantly improves multilingual robustness across all tested models. These results demonstrate that DeepRe- fusal generalizes effectively across languages with- out requiring language-specific modifications or additional training data. D Robustness under Random Seeds for GCG Since the GCG attack involves random seed ini- tialization, its outcomes can vary across runs. To assess the stability of defenses under this ran- domness, we repeat the GCG experiments on Llama3-8B-instructwith three different seeds. We report the average attack success rate (ASR) along with the standard deviation across seeds in Table 8.

Instant research discovery