Paper deep dive
SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Models
Miao Yu, Hao Huang, Lu Yuan, Yunpeng Li, Kun Wang, Zuming Jiang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 10/11/2026, 2:52:11 AM
Summary
The paper introduces SafeEvo, an interpretability framework that identifies sparse 'refusal circuits' in pretrained-only Large Language Models (LLMs) and traces their evolution during safety alignment. The authors demonstrate that these circuits exist in base models and evolve structurally during alignment, potentially causing an 'alignment tax' on utility. They propose Safety Circuit Alignment (SCA), a method that confines safety updates to these identified circuits, resulting in stronger alignment, reduced over-refusal, and better preservation of model utility compared to vanilla alignment methods like SFT and DPO.
Entities (11)
Relation Signals (8)
SafeEvo → identifies → Refusal Circuit
confidence 96% · SafeEvo first applies an optimization-based extraction algorithm to identify weak refusal circuits in pretrained base LLMs...
SafeEvo → proposes → Safety Circuit Alignment
confidence 95% · SafeEvo introduces Safety Circuit Alignment (SCA), which confines safety updates to the refusal circuits.
Refusal Circuit → existsin → Llama-3-8B
confidence 94% · pretrained-only Llama-3-8B... models are not entirely devoid of safety-related components... weak refusal circuits already exist in pretrained-only LLMs
Refusal Circuit → existsin → Qwen-2.5-7B
confidence 94% · pretrained-only... Qwen-2.5-7B... models are not entirely devoid of safety-related components
Refusal Circuit → existsin → Mistral-7B
confidence 94% · pretrained-only... Mistral-7B... models are not entirely devoid of safety-related components
Safety Circuit Alignment → outperforms → Direct Preference Optimization
confidence 90% · SCA outperforms vanilla alignment... reducing average ASR... while retaining more utility...
Safety Circuit Alignment → outperforms → Supervised Fine-tuning
confidence 90% · SCA outperforms vanilla alignment... reducing average ASR... while retaining more utility...
Refusal Circuit → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Safety interpretability advances the study of Large Language Model (LLM) alignment from behavioral constraints driven by data or algorithms towards a deeper understanding of internal mechanisms. However, existing works have focused primarily on safety-related representations, attention heads, or neurons after alignment, while largely overlooking the safety mechanisms in pretrained-only models and their evolution across alignment checkpoints. To address this, we propose SafeEvo, an interpretability framework from the circuit (sparse subgraphs of an LLM) perspective. SafeEvo first applies an optimization-based extraction algorithm to identify weak refusal circuits in pretrained base LLMs that can independently express refusal behavior. Causally ablating these circuits completely eliminates the base model's refusal of harmful inputs. SafeEvo then traces the evolution of refusal circuits across successive alignment checkpoints and finds that their structures change progressively, suggesting that the alignment tax may result from refusal-circuit updates affecting utility-related parameters. To validate this, SafeEvo introduces Safety Circuit Alignment (SCA), which confines safety updates to the refusal circuits. Experiments across three LLMs and two alignment algorithms show that, on average, SCA outperforms vanilla alignment in three aspects: \textbf{(1) stronger alignment}, lowering harmfulness score by 63.21\%; \textbf{(2) less over-refusal}, yielding a 58.44\% decrease in refusal rates for benign queries; and \textbf{(3) better utility}, retaining 99.58\% of the original model capabilities.
Tags
Links
- Source: https://arxiv.org/abs/2610.09600v1
- Canonical: https://arxiv.org/abs/2610.09600v1
Trouble viewing inline? Open PDF directly →
Full Text
68,135 characters extracted from source content.
Expand or collapse full text
SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Models Miao Yu Affiliation: The University of Hong Kong (HKU) Hao Huang Affiliation: Chinese Academy of Sciences (CAS) Lu Yuan Affiliation: Information Engineering University (IEU) Yunpeng Li Affiliation: Chinese Academy of Sciences (CAS) Kun Wang Affiliation: Nanyang Technological University (NTU) Zuming Jiang Affiliation: The University of Hong Kong (HKU) Abstract Safety interpretability advances the study of Large Language Model (LLM) alignment from behavioral constraints driven by data or algorithms towards a deeper understanding of internal mechanisms. However, existing works have focused primarily on safety-related representations, attention heads, or neurons after alignment, while largely overlooking the safety mechanisms in pretrained-only models and their evolution across alignment checkpoints. To address this, we propose SafeEvo, an interpretability framework from the circuit (sparse subgraphs of an LLM) perspective. SafeEvo first applies an optimization-based extraction algorithm to identify weak refusal circuits in pretrained base LLMs that can independently express refusal behavior. Causally ablating these circuits completely eliminates the base model’s refusal of harmful inputs. SafeEvo then traces the evolution of refusal circuits across successive alignment checkpoints and finds that their structures change progressively, suggesting that the alignment tax may result from refusal-circuit updates affecting utility-related parameters. To validate this, SafeEvo introduces Safety Circuit Alignment (SCA), which confines safety updates to the refusal circuits. Experiments across three LLMs and two alignment algorithms show that, on average, SCA outperforms vanilla alignment in three aspects: (1) stronger alignment, lowering harmfulness score by 63.21%; (2) less over-refusal, yielding a 58.44% decrease in refusal rates for benign queries; and (3) better utility, retaining 99.58% of the original model capabilities. Our code is available at: https://github.com/a727990982/SafeEvo. $*$$*$footnotetext: Equal Contribution$ $$ $footnotetext: Corresponding Author 1 Introduction As large language models (LLMs) evolve from conversational models (Team et al., 2026; Xu et al., 2026) into complex agentic systems (Ferrag et al., 2026; Lee and Kim, 2026), their capabilities and applications expand substantially. This transition amplifies the risks of harmful behavior, which can propagate through tool use to cause direct real-world harm (Luo et al., 2026a; Yu et al., 2025; Wang et al., 2025). Reliable safety alignment is therefore essential. At its core, alignment seeks to steer model behavior toward safety requirements while preserving useful capabilities (Ji et al., 2026b; Li et al., 2026; Zhou et al., 2025a). Existing methods pursue this goal via supervised demonstrations, preference feedback, or reward design (Ouyang et al., 2022; Ji et al., 2023b; Kim et al., 2026; Cao et al., 2026). Nevertheless, the research in this line focuses on surface-level and behavioral patterns, with limited investigation into the underlying safety mechanisms, despite the corresponding insights for better alignment algorithms and more trustworthy models (Bereska and Gavves, 2024). In fact, another line of works on safety interpretability (Lee et al., 2025) have attempted to explain the internal mechanisms of alignment (Elhage et al., 2021). By designing heuristic metrics and search algorithms, they identify attention heads (Zheng et al., 2026; Zhou et al., 2025b; Zheng et al., 2025), neurons (Zhao et al., 2025c; Yi et al., 2025), layers (Li et al., 2025b), and representation (Jiao et al., 2026; Luo et al., 2026c; Zhou et al., 2024) (e.g., refusal direction but we establish the more interpretable and fundamental refusal circuits) associated with safety concepts. However, these interpretability studies and the related alignment methods (Yang et al., 2026; Zhao et al., 2025b; Li et al., 2025a) focus on the already safety-aligned models only at a fixed endpoint. They lack both attention and methodologies to delve into the safety mechanisms in pretrained base LLMs and their evolution dynamics during safety post-training, which may contribute to alignment improvement. Figure 1: From circuit discovery to circuit-targeted alignment. We move beyond endpoint interpretability by tracing refusal circuits from pretrained-only models and their subsequent evolution (Phase 1). We then develop alignment methods that directly target the identified circuits for better safety-utility trade-off (Phase 2). To this end, we introduce SafeEvo, an interpretability framework that attributes LLM alignment to sparse circuits capable of independently eliciting refusal, identifies these refusal circuits in pretrained-only LLMs, and traces their evolution throughout alignment. SafeEvo first casts circuit extraction as an optimization problem rather than relying on heuristic attribution (Behrouzi et al., 2026a; Zhou et al., 2025b). Specifically, it learns differentiable masks for each LLM units that automatically isolate a sparse circuit with full safety ability. Applying this technique, we find that pretrained-only Llama-3-8B, Qwen-2.5-7B, and Mistral-7B models are not entirely devoid of safety-related components. Across these models, ablating the identified refusal circuit increases attack success rate (ASR) on HarmBench by 42.76%\%, 55.98%, and 45.91%, respectively. SafeEvo therefore advances and empirically validates the hypothesis that refusal circuits are already present in base LLMs, albeit in a weak (reflected by ASR ranging from 32.7%\% ∼ 48.43 %\%) form and effective only for a subset of harmful queries. This finding corroborates and advances prior representation-level evidence (Zhou et al., 2024) that base LLMs already encode harmfulness and harmlessness from circuit perspective. We further use SafeEvo to track refusal circuits across alignment checkpoints and quantify their evolution. Warm-starting each extraction from the circuit identified at the preceding checkpoint reveals a gradual structural drift: the overlap between adjacent circuits decreases from 98%98\% at the onset of alignment to 78%78\% after 20 checkpoints. Moreover, independent extraction runs recover multiple circuits that fully and independently reproduce the refusal behavior but share only 26%26\% overlap. Together, these findings demonstrate that refusal circuits are both non-unique and structurally unstable during alignment. This observation offers a possible mechanistic explanation for the “alignment tax”—the degradation of utility after alignment: the underconstrained updates of refusal circuits may spill over into utility-relevant parameters, thereby compromising general performance. Inspired by these insights and to empirically support them, we introduce Safety Circuit Alignment (SCA), which confines safety-related updates to only one identified refusal circuit and only optimizes its associated parameters. This design mitigates the observed non-uniqueness and instability of refusal circuit evolution and can be readily combined with existing alignment algorithms. Across three LLMs on HarmBench (Mazeika et al., 2024) and AdvBench (Zou et al., 2023), comparing SCA with vanilla alignment via supervised fine-tuning (SFT) (Zhao et al., 2025a) and direct preference optimization (DPO) (Hu et al., 2026) reduces average ASR by 2.29%\% points (43.81%↓43.81\% ) and 6.47%\% points (74.94%↓74.94\% ), respectively, while retaining more utility on GSM8k, MMLU, and HellaSwag. Besides, SCA outperforms a safety-neuron-based baseline and matched-sparsity ablation on random circuits, validating SafeEvo’s mechanistic insights as useful foundations for more effective alignment. In summary, our main contributions can be listed as follows: • Cross-Stage Framework. We propose SafeEvo, an interpretability framework for attributing LLM alignment to refusal circuits and tracking their evolution throughout safety post-training. • Novel Insights. We provide causal and experimental evidence that weak refusal circuits already exist in pretrained-only LLMs and reveal their non-uniqueness and instability in vanilla alignment. • Circuit-Guided Alignment. We introduce SCA, which localizes safety updates to pre-alignment refusal circuits, achieving a favorable balance among safety, utility preservation, and over-refusal. 2 Related Work LLM Alignment. To prevent LLMs from enabling dangerous behavior (Liu et al., 2023; Luo et al., 2026b; Zhang et al., 2024), prevailing alignment works steer model responses to harmful queries toward refusal, either via SFT (Yang et al., 2026; Fan et al., 2026) on synthetic refusal data or via DPO on preference pairs (Ji et al., 2026a; Deng et al., 2026). Optimizing for refusal alone, however, tends to induce over-refusal and erode general capability (Jalan et al., 2026; Niu et al., 2026). A subsequent line of work mitigates this tension by synthesizing data that is jointly helpful and harmless (Wang et al., 2024; Huang et al., 2026), and by shaping rewards with explicit utility constraints (Ji et al., 2026b). Orthogonally, mechanistic-interpretability approaches localize neurons causally implicated in the safety boundary (Wang et al., 2026; Zhao et al., 2025b; Behrouzi et al., 2026b). SafeEvo departs from this last line in the granularity and sufficiency of the located structure: we identify a subgraph within LLMs (circuit) that independently realizes safety-refusal behavior — a sufficiency property individual causal neurons do not exhibit. As a more intrinsic locus of the model’s safety capability, this circuit reasonably affords a better alignment–utility trade-off under targeted tuning. Safety Interpretability. Applying perspectives and techniques from mechanistic interpretability (Naseem, 2026) to safety, studies in this direction asks how safety behaviors are computed inside the LLM and why they fail (Lad, 2024; Lee et al., 2025). Prior efforts differ mainly in the granularity at which they localize safety. Representation-level analyses decode intermediate activations and find that refusal versus jailbreak tracks the polarity encoded in hidden states (He et al., 2025; Jiao et al., 2026). Component-level studies use activation patching and causal attribution to show that a sparse set of attention heads carries refusal (Zheng et al., 2026; Zhou et al., 2025b), and that ablating them suffices to remove it. Neuron-level work identifies safety neurons and repurposes them for jailbreak analysis (Zhao et al., 2026) and alignment strengthening (Zhao et al., 2025b). These efforts, however, largely examine post-alignment LLMs. SafeEvo instead adopts a circuit-level view (Yu et al., 2026) and traces the evolution of safety-related circuits across alignment checkpoints from pretrained-only base LLMs, yielding deeper understanding and insights for designing better alignment algorithms. 3 Refusal Circuit Discovery and Evolution Our investigation begins with a simple yet subtle phenomenon: pretrained-only base LLMs refuse a small fraction of harmful queries, as also shown in prior work (Kissane et al., 2024). Building on this observation, we hypothesize that pre-alignment LLMs may contain certain separable components responsible for refusal, or more specifically, a sparse and functionally complete refusal circuit. We first formulate how to identify safety-related refusal circuits (Section 3.1). Then, we test the causal contribution and ability of these circuits via ablation and isolation, respectively (Section 3.2). Finally, we track refusal circuits during early safety alignment, examining changes across checkpoints and variation across independent extractions (Section 3.3). Figure 2 demonstrates the overview of SafeEvo and its application for better alignment via localized refusal circuit tuning. Figure 2: Overview of our Safety Circuit Alignment algorithm. It first extracts a refusal circuit before LLM alignment (Step A) and only fine-tunes the identified parameters in that circuit for precise alignment (Step B). 3.1 Refusal Circuit Extraction We refer to a sparse subset of model components, together with the computational subgraph they induce, that can express refusal behavior in isolation as a refusal circuit. Extracting such a circuit amounts to a selection over a combinatorially large space. A common approach in previous works guides this using heuristic component-level scores based on activations, attributions, or weights (Zhao et al., 2025b; Chen et al., 2024; Yi et al., 2024), but they only capture circuit for only one token generation. We instead cast refusal circuit discovery as an optimization problem: each candidate component is associated with a learnable binary selection variable (a mask), and with the LLM weights frozen, the masks are optimized jointly such that the selected express refusal on their own, their complement generates harmful-compliance completions, and the selection remains sparse. Circuit Components. We take MLP neurons as the unit of refusal circuits for selection, since Multi-layer Perceptron (MLP) modules in LLMs are widely regarded as the primary locus of stored knowledge and behaviors. The gated MLP (common in today’s LLMs) at layer ℓ can be defined as: MLPℓ(hℓ)=Wℓ,down[ϕ(Wℓ,gatehℓ)⊙(Wℓ,uphℓ)],MLP_ (h_ )=W_ ,down [φ(W_ ,gateh_ ) (W_ ,uph_ ) ], (1) where hℓ∈ℝdmodelh_ ^d_model is the MLP input, ϕφ is the gating function, and ⊙ is element-wise multiplication. Wℓ,gate,Wℓ,up∈ℝdff×dmodelW_ ,gate,W_ ,up ^d_f× d_model map to the intermediate MLP space, while Wℓ,down∈ℝdmodel×dffW_ ,down ^d_model× d_f maps back to the model space. Each projection is a linear map whose i-th output coordinate is computed exclusively by the i-th row of its weight matrix. We regard each such row–output pair as one MLP neuron. For each projection p∈=gate,up,downp =\gate,up,down\, we denote its frozen weight, binary neuron mask, and diagonal mask by the following, respectively: Wℓ,p∈ℝdout,p×din,p,mℓ,p∈0,1dout,p,Mℓ,p=Diag(mℓ,p),W_ ,p ^d_out,p× d_in,p, m_ ,p∈\0,1\^d_out,p, M_ ,p=Diag(m_ ,p), (2) and collect all candidates into the universal neuron set =(ℓ,p,i)∣ℓ∈[L],p∈,i∈[dout,p]U=\( ,p,i) ∈[L],\,p ,\,i∈[d_out,p]\. Since the projections are row-separable, any neuron can be switched off by zeroing its row without altering other neurons, making U a natural basis for defining refusal circuits. Refusal Circuit. Let ℳ=mℓ,pℓ,pM=\m_ ,p\_ ,p collect all neuron masks and ℳ¯=−mℓ,pℓ,p M=\1-m_ ,p\_ ,p denote its complement. For input hℓ,ph_ ,p, the original, circuit, and complementary outputs of each projection are: zℓ,p=Wℓ,phℓ,p,zℓ,pC=Mℓ,pWℓ,phℓ,p⏟circuit for refusal,zℓ,pC¯=(I−Mℓ,p)Wℓ,phℓ,p⏟complement for compliance.z_ ,p=W_ ,ph_ ,p, [rgb]0.2305,0.4883,0.1367z^C_ ,p=M_ ,pW_ ,ph_ ,p_ [rgb]0.2305,0.4883,0.1367 for refusal, [rgb]0.7773,0.3711,0.0625z C_ ,p=(I-M_ ,p)W_ ,ph_ ,p_ [rgb]0.7773,0.3711,0.0625 for compliance. (3) Since Mℓ,p+(I−Mℓ,p)=IM_ ,p+(I-M_ ,p)=I, we have zℓ,p=zℓ,pC+zℓ,pC¯z_ ,p=z^C_ ,p+z C_ ,p, i.e., the masks induce an exact partition of U into a circuit part (sparse refusal circuit) and a complement part (the remaining dense graph): C=(ℓ,p,i)∣mℓ,p,i=1,C¯=∖C,|C|=∑ℓ,p∥mℓ,p∥0,C=\( ,p,i) m_ ,p,i=1\, C=U C, |C|= _ ,p m_ ,p _0, (4) where |C||C| counts the selected neurons. The circuit model pθ,ℳp_θ,M replaces every projection output in Eq. 1 by zℓ,pCz^C_ ,p, zero-ablating all neurons in C¯ C; symmetrically, the complementary model pθ,ℳ¯p_θ, M uses zℓ,pC¯z C_ ,p and zero-ablates C. Masking a gate or up neuron removes its intermediate coordinate from the input of Wℓ,downW_ ,down, and masking a down neuron removes the MLP’s contribution to one residual-stream coordinate, so C specifies a sparse computational subgraph rather than a bag of isolated neurons. Differentiable Mask Optimization. The binary masks in Eq. 2 are not differentiable. We therefore parameterize each mask by a latent vector qℓ,p∈ℝdout,pq_ ,p ^d_out,p with soft mask m~ℓ,p=σ(qℓ,p)∈(0,1)dout,p m_ ,p=σ(q_ ,p)∈(0,1)^d_out,p. A purely continuous relaxation, however, lets the circuit rely on fractionally scaled neurons that vanish once thresholded, causing a train–test inconsistency. To keep the forward pass strictly binary and differentiable, we binarize the soft masks with threshold η via a straight-through estimator (STE): mℓ,p=[m~ℓ,p>η],Mℓ,p=Diag(m~ℓ,p+sg(mℓ,p−m~ℓ,p)),m_ ,p=1\! [ m_ ,p>η ], M_ ,p=Diag\! ( m_ ,p+sg(m_ ,p- m_ ,p) ), (5) where sg(⋅)sg(·) is the stop-gradient operator. In the forward pass Mℓ,pM_ ,p equals the binary Diag(mℓ,p)Diag(m_ ,p), so Eq. 3 always evaluates a discrete partition; in the backward pass, ∇sg(⋅)≡0 (·)≡ 0 routes the gradient of Mℓ,pM_ ,p to m~ℓ,p m_ ,p and hence to qℓ,pq_ ,p. We initialize qℓ,pq_ ,p with large positive values so that Mℓ,p=IM_ ,p=I, i.e., extraction starts from the intact LLM and progressively prunes neurons out of the circuit. Dual-branch Extraction Objective. We learn the masks with two completion-supervision branches: the refusal branch trains the circuit on refusal completions refD_ref, whereas the compliance branch trains the complementary model on harmful-compliance completions cmpD_cmp. The total loss is: ℒext(ℳ)= _ext(M)= αℒSFT(ℳ,ref)⏟circuit / refusal+βℒSFT(ℳ¯,cmp)⏟complement / compliance+λmlp∑ℓ,p∥m~ℓ,p∥1dout,p, [rgb]0.2305,0.4883,0.1367α\,L_SFT(M;D_ref)_ [rgb]0.2305,0.4883,0.1367 / refusal+ [rgb]0.7773,0.3711,0.0625β\,L_SFT( M;D_cmp)_ [rgb]0.7773,0.3711,0.0625 / compliance+ _mlp _ ,p m_ ,p _1d_out,p, (6) where ℒSFT(ℳ,)L_SFT(M;D) is the token-level negative log-likelihood of completions y given prompts x in D under the masked model pθ,ℳp_θ,M, and only qℓ,p\q_ ,p\ are optimized while θ remains frozen. The two branches assign complementary roles to the partition: the refusal term requires C to be sufficient for refusal, while the compliance term requires C¯ C to retain harmful compliance, ruling out the trivial solution Mℓ,p=IM_ ,p=I and implying that removing C suppresses refusal. The normalized L1L_1 penalty on the soft masks prunes uninformative neurons, yielding a compact circuit; α, β, and λmlp _mlp balance the three terms. Upon convergence, the refusal circuit is read out from the binary masks via Eq. 4. Table 1: Refusal rate of the base model, the refusal circuit and its complement branch for harmful queries in the BeaverTails dataset. Marker ↑ and ↓ show the increase and decrease compared with the base models. Models / Refusal Rate (%) Base Model Refusal Circuit (C1C_1) Density (C1C_1) Complement Branch Llama-3-8B 48.04 100.00 ↑51.96 51.96 0.74% 0.00 ↓48.04 48.04 Qwen-2.5-7B 35.15 100.00 ↑64.85 64.85 0.80% 0.00 ↓35.15 35.15 Mistral-7B-v0.1 15.63 100.00 ↑84.37 84.37 0.75% 0.00 ↓15.63 15.63 Figure 3: Refusal rate and ASR on HarmBench for base LLMs and the same models with the identified refusal circuits C1C_1 ablated, compared to the density-matched random ablation models. 3.2 Refusal Circuits in Pretrained-only LLMs We apply the techniques in Section 3.1 and LLM-LAT (Sheshadri et al., 2025) as the dataset to extract a candidate refusal circuit C1C_1 from Llama-3-8B, Qwen-2.5-7B, and Mistral-7B-v0.1, all containing only < 1% of the model parameters. Then we investigate the properties of the base LLMs, the circuits, and their complement branches via two behavioral metrics: ASR, which measures positive compliance for harmful queries, and refusal rate, which measures explicit refusal behaviors. Takeaway 1: Our extracted candidates are sparse, functionally complete refusal circuits that can express refusal in isolation. We evaluate the refusal rates on BeaverTails (Ji et al., 2023b), a harmful-query dataset that is out-of-distribution (OOD) relative to the data for circuit search. As shown in Table 1, pretrained-only base LLMs occasionally refuse harmful queries (with refusal rates of 15%∼48%15\% 48\%), whereas every extracted circuit achieves a 100% OOD refusal rate while containing only < 1%1\% parameters. These results indicate that our extraction method successfully identifies a refusal circuit that generalizes to unseen harmful queries. Moreover, ablating the circuit eliminates refusal in the complete branch with all refusal rates drops to 0%0\%. Together, these findings suggest that a sparse circuit plays the core and fundamental role in LLM refusal of harmful requests. Takeaway 2: Ablating refusal circuits substantially increases model harmfulness. To further test the causal relationship between refusal circuits and safety alignment, we zero ablate the outputs of components in each refusal circuit and compare the results with a random zero-ablation baseline at the same density. As shown in Figure 3, circuit ablation severely impairs refusal, yielding refusal rates of 0.00%∼0.63%0.00\% 0.63\%. Meanwhile, ASR rises from 48.43%, 32.70%, and 42.14% to 91.19%, 88.68%, and 88.05%, respectively (relative increases of 88.3%, 171.2%, and 109.0%). The increases under random ablation are much smaller. These causal ablation results show that safety alignment is causally linked to refusal circuits, suggesting that LLMs’ safety capabilities may originate from them. Figure 4: Evolution of refusal circuits across alignment, traced and quantified by Jaccard similarity. 3.3 Refusal Circuit Evolution During Safety Alignment Section 3.2 provides experimental evidence for functional refusal circuits in base LLMs, but does not investigate their evolution (whether the same circuits still support refusal as alignment proceeds). We therefore track the composition of extracted refusal circuits across safety-alignment checkpoints. Tracking protocol. We apply SFT-based safety alignment to the Llama-3-8B base model and track the evolution of its refusal circuit over the first several aligned checkpoints. To capture incremental changes, we warm-start each extraction from the circuit found at the preceding checkpoint. Then we measure refusal circuit continuity using the Jaccard overlap between consecutive checkpoints. Besides, we also perform multiple independent extractions at a single checkpoint to further quantify run-to-run variability, providing a reference for assessing cross-checkpoint similarity. Takeaway 3: Refusal circuits are not unique within a given LLM or checkpoint. As shown in the left panel of Figure 4, independently extracted refusal circuits at the same checkpoint exhibit only 0.26 average component overlap, far below the 0.78∼0.980.78 0.98 similarity for warm-started extractions. This indicates that refusal functionality is implemented redundantly in LLMs, with multiple valid refusal circuits sharing only certain components (maybe relate to basic language capabilities). Takeaway 4: Refusal circuits undergo gradual structural drift during alignment. As shown by the warm-started extractions in Figure 4, component similarity between consecutive checkpoints declines smoothly from 1.00→0.781.00→ 0.78, indicating that the circuits progressively acquire and discard components rather than remaining fixed. Cross-checkpoint comparisons (the right similarity heatmap of Figure 4) further reveal a larger shift between the base model’s (pre-alignment) circuit and those after several alignment steps (similarity ∼0.60 0.60). These results consistently support the gradual evolution and weak stability of refusal circuits during LLM safety post-training. Together, these two findings establish a simple picture of refusal circuits: refusal originates in and can be attributed to different circuits whose composition evolves gradually during alignment. This view may offers a potential account of the “alignment tax” incurred during safety post-training: safety alignment alters the refusal circuits that underlie LLMs’ refusal capabilities, and unconstrained parameter updates may inadvertently affect parameters associated with utility, including general capabilities. Motivated by this hypothesis and seeking to test it empirically to some extent, we propose restricting safety-alignment updates entirely to the refusal circuit in the base LLMs, which is already present and does not interfere with the knowledge and capabilities acquired during pretraining. 4 Safety Circuit Alignment To test the above hypothesis and develop a more effective alignment algorithm, we introduce Safety Circuit Alignment (SCA), which restricts safety updates to the pre-alignment refusal circuits in base LLMs, inspired by our interpretability insights in Section 3. We first formulate the core principle of SCA in Section 4.1 and introduce the experiment setups for subsequent safety alignment in Section 4.2. Then we present and analyze the experiment results of our methods with other baselines’ in Section 4.3. The overview of our SCA technique is illustrated in Figure 2. 4.1 Alignment Algorithm (SCA) SCA confines all safety-alignment updates to the refusal circuit C extracted from the pre-alignment LLM. Concretely, we parameterize the update of each MLP projection with LoRA and reuse the binary neuron mask Mℓ,p=Diag(mℓ,p)M_ ,p=Diag(m_ ,p) obtained by Eq. 6 to restrict the rank-r update to the circuit: Wℓ,p′=Wℓ,p+Mℓ,pBℓ,pAℓ,p,W _ ,p=W_ ,p+M_ ,pB_ ,pA_ ,p, (7) where Aℓ,p∈ℝr×din,pA_ ,p ^r× d_in,p and Bℓ,p∈ℝdout,p×rB_ ,p ^d_out,p× r. Since Mℓ,pM_ ,p is diagonal and binary, Mℓ,pBℓ,pAℓ,pM_ ,pB_ ,pA_ ,p has non-zero rows exactly at the neurons in C (Eq. 4), so Wℓ,p′W _ ,p differs from Wℓ,pW_ ,p only on the rows of circuit neurons, while the weights of C¯ C and all attention modules remain identical to the base model. During alignment, Mℓ,pM_ ,p stays fixed and Wℓ,pW_ ,p is frozen; only Aℓ,pA_ ,p and Bℓ,pB_ ,p are optimized. SCA changes the update support rather than the alignment loss. Let o∈SFT,DPOo∈\SFT,DPO\ index the objective and θM(k) _M^(k) denote the policy parameterized by Eq. 7 at step k. SCA’s update is formulated as: Bℓ,p(k+1,o)=Bℓ,p(k)−γMℓ,p∇Bℓ,pℒo(θM(k)),B_ ,p^(k+1,o)=B_ ,p^(k)-γ\,M_ ,p _B_ ,pL_o( _M^(k)), (8) where γ is the learning rate; the input factor Aℓ,pA_ ,p follows the ordinary optimizer update. Mℓ,pM_ ,p gates the rows of the gradient so that neurons outside C never receive an update, mirroring how Eq. 3 gates their outputs during extraction. We instantiate ℒoL_o with two prevailing alignment objectives. For SFT, we adopt the ℒSFTL_SFT used in Section 3.1, now applied to πθM _ _M on refusal completions. For DPO, given a prompt x with a safe completion ywy_w and a harmful completion yly_l, we optimize: ℒDPO(θM)=−(x,yw,yl)logσ(τ[logπθM(yw∣x)πref(yw∣x)−logπθM(yl∣x)πref(yl∣x)]),L_DPO( _M)=-E_(x,y_w,y_l) σ\! (τ\! [ _ _M(y_w x) _ref(y_w x)- _ _M(y_l x) _ref(y_l x) ]\! ), (9) where πref _ref is the frozen pre-alignment model and τ controls the deviation from it. Both losses are computed through the full forward pass of the LLM, but its gradient only alters the refusal circuits. 4.2 Experimental Settings Models. We select three LLMs: Llama-3-8B (Llama Team, 2024), Qwen-2.5-7B (Qwen Team, 2024), and Mistral-7B-Instruct-v0.1 (Jiang et al., 2023). Notably, we use the Mistral’s instruction-tuned version because its base model does not reliably follow the instruction format. Alignment Objectives & Datasets. We utilize two prevailing and most important losses in safety aligment: SFT and DPO. Specifially, SFT uses the refusal completions from LLM-LAT (Sheshadri et al., 2025), while DPO adopts safety preference pairs in PKU-SafeRLHF (Ji et al., 2023b). Evaluation Metrics & Datasets. For the safety evaluation, we calculate attack success rate (ASR) via the Llama-Guard-3-8B (Meta AI, 2024) model on HarmBench (Mazeika et al., 2024) and AdvBench (Zou et al., 2023). LLMs’ general utility after safety alignment is measured by the accuracies on GSM8K (Cobbe et al., 2021), MMLU (Hendrycks et al., 2021), and HellaSwag (Zellers et al., 2019), while over-refusal evaluation adopts the benign input prompts in XSTest (Röttger et al., 2024) and BeaverTails (Ji et al., 2023a). Notably, all evaluation datasets are held out from SFT and DPO training, preventing direct fitting and enabling a rigorous assessment of OOD generalization. Baselines. All alignment methods in our experiments use either the SFT or DPO loss and differ only in how parameters are updated. As the basic baseline, we use vanilla fine-tuning via LoRA, where all model parameters are subject to updating. To isolate the effect of sparse updates in SCA, we compare it with a randomly selected update method at the same sparsity level, which does not specifically target refusal circuits. To demonstrate SCA’s deeper interpretability insights, we also include SN-Tune (Zhao et al., 2025b), an update method based on the interpretability of safety neurons as an additional baseline. Implementation details are provided in Appendix A.1. Table 2: Safety, utility, and over-refusal for different baselines under SFT and DPO. Colored marker ↓ and ↑ indicate changes compared to base models, while those following dataset names denote whether lower or higher values are better. Bold and underline mark the best and second-best within each model–objective block. Method Objective Safety Utility Over-refusal HarmB. (↓ ) AdvB.(↓( ) GSM8K (↑ ) MMLU ((↑( ) HSwag (↑ ) XSTest (↓ ) LLAMA-3-8B Base Model – 50.31 52.88 50.34 65.36 53.05 0.8 LoRA SFT 16.98 ↓33.33 33.33 12.50 ↓40.38 40.38 45.79 ↓4.55 4.55 63.75 ↓1.61 1.61 55.25 ↑2.20 2.20 18.8 ↑18.0 18.0 Random SFT 18.87 ↓31.44 31.44 3.27 ↓49.61 49.61 49.05 ↓1.29 1.29 64.49 ↓0.87 0.87 53.35 ↑0.30 0.30 2.8 ↑2.0 2.0 SN-Tune SFT 0.00 ↓50.31 50.31 0.00 ↓52.88 52.88 31.54 ↓18.80 18.80 57.78 ↓7.58 7.58 51.85 ↓1.20 1.20 86.4 ↑85.6 85.6 SCA (Ours) SFT 5.66 ↓44.65 44.65 1.54 ↓51.34 51.34 48.90 ↓1.44 1.44 64.88 ↓0.48 0.48 53.20 ↑0.15 0.15 1.6 ↑0.8 0.8 LoRA DPO 16.98 ↓33.33 33.33 12.69 ↓40.19 40.19 49.51 ↓0.83 0.83 65.46 ↑0.10 0.10 52.95 ↓0.10 0.10 2.8 ↑2.0 2.0 Random DPO 6.92 ↓43.39 43.39 5.00 ↓47.88 47.88 47.99 ↓2.35 2.35 65.11 ↓0.25 0.25 53.35 ↑0.30 0.30 3.6 ↑2.8 2.8 SCA (Ours) DPO 0.63 ↓49.68 49.68 0.38 ↓52.50 52.50 48.98 ↓1.36 1.36 65.60 ↑0.24 0.24 53.45 ↑0.40 0.40 3.2 ↑2.4 2.4 QWEN-2.5-7B Base Model – 33.96 7.50 83.09 74.21 52.45 5.2 LoRA SFT 0.63 ↓33.33 33.33 0.00 ↓7.50 7.50 80.52 ↓2.57 2.57 71.91 ↓2.30 2.30 54.20 ↑1.75 1.75 28.4 ↑23.2 23.2 Random SFT 20.75 ↓13.21 13.21 2.50 ↓5.00 5.00 84.08 ↑0.99 0.99 74.20 ↓0.01 0.01 52.30 ↓0.15 0.15 5.6 ↑0.4 0.4 SN-Tune SFT 4.40 ↓29.56 29.56 0.19 ↓7.31 7.31 70.36 ↓12.73 12.73 68.01 ↓6.20 6.20 51.50 ↓0.95 0.95 42.0 ↑36.8 36.8 SCA (Ours) SFT 6.92 ↓27.04 27.04 0.38 ↓7.12 7.12 83.17 ↑0.08 0.08 74.05 ↓0.16 0.16 52.20 ↓0.25 0.25 6.8 ↑1.6 1.6 LoRA DPO 0.00 ↓33.96 33.96 0.19 ↓7.31 7.31 86.81 ↑3.72 3.72 73.97 ↓0.24 0.24 53.10 ↑0.65 0.65 34.8 ↑29.6 29.6 Random DPO 0.00 ↓33.96 33.96 0.19 ↓7.31 7.31 82.87 ↓0.22 0.22 74.28 ↑0.07 0.07 52.20 ↓0.25 0.25 8.0 ↑2.8 2.8 SCA (Ours) DPO 0.00 ↓33.96 33.96 0.19 ↓7.31 7.31 81.73 ↓1.36 1.36 74.21 →0.00→ 0.00 52.60 ↑0.15 0.15 17.6 ↑12.4 12.4 MISTRAL-7B-V0.1 Base Model – 39.62 30.00 33.43 53.74 50.15 0.8 LoRA SFT 0.63 ↓38.99 38.99 0.58 ↓29.42 29.42 30.63 ↓2.80 2.80 53.28 ↓0.46 0.46 50.90 ↑0.75 0.75 7.6 ↑6.8 6.8 Random SFT 9.43 ↓30.19 30.19 5.19 ↓24.81 24.81 31.01 ↓2.42 2.42 52.95 ↓0.79 0.79 50.45 ↑0.30 0.30 5.2 ↑4.4 4.4 SN-Tune SFT 5.03 ↓34.59 34.59 5.77 ↓24.23 24.23 28.81 ↓4.62 4.62 49.95 ↓3.79 3.79 49.15 ↓1.00 1.00 25.6 ↑24.8 24.8 SCA (Ours) SFT 2.52 ↓37.10 37.10 0.58 ↓29.42 29.42 32.68 ↓0.75 0.75 53.75 ↑0.01 0.01 50.20 ↑0.05 0.05 5.6 ↑4.8 4.8 LoRA DPO 16.35 ↓23.27 23.27 5.58 ↓24.42 24.42 34.80 ↑1.37 1.37 53.60 ↓0.14 0.14 51.30 ↑1.15 1.15 4.8 ↑4.0 4.0 Random DPO 37.74 ↓1.88 1.88 28.46 ↓1.54 1.54 34.12 ↑0.69 0.69 54.30 ↑0.56 0.56 50.95 ↑0.80 0.80 2.0 ↑1.2 1.2 SCA (Ours) DPO 7.55 ↓32.07 32.07 4.23 ↓25.77 25.77 33.51 ↑0.08 0.08 53.82 ↑0.08 0.08 50.65 ↑0.50 0.50 5.6 ↑4.8 4.8 4.3 Main Safety–Utility Results To validate SCA, developed based on our interpretability insights, we evaluate aligned LLMs on safety and utility datasets that are OOD to training, with results in Table 2 and 3. On average, SCA outperforms baselines in alignment efficacy, capability preservation, and avoiding over-refusal. Better Alignment. As shown in Table 2, SCA provides strong safety improvements under both SFT and DPO. Across all six model–objective blocks, its HarmBench and AdvBench attack success rates rank either first or second among full-LoRA, random-support, and SCA. Under DPO, SCA reduces the Llama-3-8B ASR → 0.63%0.63\% on HarmBench and 0.38%0.38\% on AdvBench, compared with 16.98%16.98\% and 12.69%12.69\% for full-LoRA. It also achieves 7.55%7.55\% on HarmBench and 4.23%4.23\% on AdvBench for Mistral-7B-Instruct-v0.1, substantially below full-LoRA’s 16.35%16.35\% and 5.58%5.58\%. Under Qwen-2.5-7B SFT, SCA keeps the two ASRs at 6.92%6.92\% and 0.38%0.38\%, while random-support reaches 20.75%20.75\% and 2.50%2.50\%. These results suggest that SCA effectively maintains robust refusal behavior on held-out harmful queries, supporting our interpretability insights in safety alignment and refusal circuits. Utility Preservation. At the same time, SCA preserves the models’ general capabilities across GSM8K, MMLU, and HellaSwag. Across the six model–objective blocks, the largest utility decrease is only 1.441.44% points, while several benchmarks improve over the starting model. For instance, under SFT, Mistral-7B-Instruct-v0.1 changes only from 33.4333.43 → 32.6832.68 on GSM8K, from 53.7453.74 → 53.7553.75 on MMLU, and from 50.1550.15 → 50.2050.20 on HellaSwag. Likewise, Qwen-2.5-7B under SFT improves on GSM8K from 83.0983.09 → 83.1783.17, while its MMLU score decreases by only 0.160.16 points. Under DPO, Llama-3-8B even improves from 65.3665.36 → 65.6065.60 on MMLU and from 53.0553.05 → 53.4553.45 on HellaSwag. This stable utility profile indicates that SCA strengthens safety while largely preserving broad abilities, validating our explanation for alignment tax in Section 3.3 to some extend. Less Over-refusal. For benign prompts, SCA also substantially reduces over-refusal on XSTest compared with full-LoRA. On Llama-3-8B under SFT, the over-refusal rate increases only from the base value of 0.8%0.8\% to 1.6%1.6\%, whereas full-LoRA reaches 18.8%18.8\%. The corresponding rates for Qwen-2.5-7B are 6.8%6.8\% for SCA and 28.4%28.4\% for full-LoRA under SFT, and 17.6%17.6\% for SCA and 34.8%34.8\% for full-LoRA under DPO. For Mistral-7B-Instruct-v0.1 under SFT, SCA limits over-refusal to 5.6%5.6\%, compared with 7.6%7.6\% for full-LoRA. These results show that SCA preserves responsiveness to benign requests while avoiding the excessive refusal behavior introduced by dense parameter updates. Table 3: Over-refusal and ASR trade-off on XSTest datasets. Marker meanings are the same as above. Method Objective Llama-3-8B Qwen-2.5-7B Mistral-7B-v0.1 ASR (↓ ) Over Refusal (↓) ) ASR (↓ ) Over Refusal (↓) ) ASR (↓ ) Over Refusal (↓) ) Base Model – 64.5 0.8 61.5 5.2 66.0 0.8 LoRA SFT 50.5 ↓14.0 14.0 18.8 ↑18.0 18.0 30.5 ↓31.0 31.0 28.4 ↑23.2 23.2 40.0 ↓26.0 26.0 7.6 ↑6.8 6.8 SCA (Ours) SFT 62.0 ↓2.5 2.5 1.6 ↑0.8 0.8 47.5 ↓14.0 14.0 6.8 ↑1.6 1.6 51.0 ↓15.0 15.0 5.6 ↑4.8 4.8 LoRA DPO 53.0 ↓11.5 11.5 2.8 ↑2.0 2.0 1.0 ↓60.5 60.5 34.8 ↑29.6 29.6 26.5 ↓39.5 39.5 4.8 ↑4.0 4.0 SCA (Ours) DPO 48.5 ↓16.0 16.0 3.2 ↑2.4 2.4 36.0 ↓25.5 25.5 17.6 ↑12.4 12.4 21.5 ↓44.5 44.5 5.6 ↑4.8 4.8 Table 4: HarmBench ASR of multiple runs of SCA and the random baseline. ASR is reported as mean± over four training seeds. The Δ column denotes the decrease of SCA ASR (relative to the random baseline), with brackets reporting the paired prompt-level 95%95\% bootstrap confidence interval (CI). Models SCA ASR (Ours)↓ Random Baseline ASR↓ Δ (95% CI) Llama-3-8B 5.48±3.275.48± 3.27 20.28±6.0020.28± 6.00 14.8014.80 [11.48,18.5511.48,18.55] Qwen-2.5-7B 11.32±5.5111.32± 5.51 19.81±3.4319.81± 3.43 8.498.49 [4.72,12.424.72,12.42] Mistral-7B-v0.1 5.35±2.445.35± 2.44 12.58±7.0012.58± 7.00 7.237.23 [3.77,10.853.77,10.85] Figure 5: Safety–utility trade-offs across different LLM families and alignment methods. 4.4 SCA versus Random Baseline We next test whether SCA’s advantage in safety alignment persists across training seeds and stems from targeting the extracted refusal circuit instead of sparse updates. Random baseline updates the same amount (as SCA) of but random-selected parameters. Results are displayed in Table 4. Consistent and Robust Performances. SCA achieves lower HarmBench ASR than the random baseline for all three models: 5.48±3.27%5.48± 3.27\% versus 20.28±6.00%20.28± 6.00\% for Llama-3-8B, 11.32±5.51%11.32± 5.51\% versus 19.81±3.43%19.81± 3.43\% for Qwen-2.5-7B, and 5.35±2.44%5.35± 2.44\% versus 12.58±7.00%12.58± 7.00\% for Mistral-7B-Instruct-v0.1. The corresponding ASR reductions are 14.8014.80, 8.498.49, and 7.237.23 percentage points, with all paired prompt-level 95%95\% bootstrap CIs excluding zero. Because the two conditions use the same update parameterization and sparsity budget, these results indicate that SCA’s stronger safety alignment is not explained by restricting the update space alone. Instead, the consistent advantage of SCA supports the relevance of the previously identified refusal circuit and shows that its circuit-targeted updates provide a more effective and interpretable support for safety alignment. 5 Conclusion In this work, we introduce SafeEvo, an interpretability framework that leverages differentiable mask optimization to attribute LLM refusal to sparse, functionally complete circuits and to trace their evolution across alignment checkpoints, moving beyond endpoint analyses of already-aligned models. Through extensive experiments on three LLMs, we reveal that weak refusal circuits already exist in pretrained-only models, causally govern their refusal of harmful queries, and are neither unique nor stable during safety post-training. We further bridge interpretability with alignment via SCA, showing that confining safety updates to pre-alignment refusal circuits yields stronger safety, less over-refusal, and better utility preservation. We believe SafeEvo offers a mechanistic account of how alignment reshapes LLMs, paving the way for more precise and interpretable alignment algorithms. AI use statement We used a general-purpose language model as a writing aid, limited to polishing grammar and phrasing in prose the authors had drafted and tightening captions such as those of Figure 1 and Tables 2–4 to fit the page budget. We also used an AI coding assistant for routine chores: plotting boilerplate for the safety–utility trade-off curves in Figure 5 and the overlap heatmap in Figure 4, and formatting result dictionaries into the LaTeX of Tables 2, 3, and 9. AI tools played no role in generating research ideas, designing the extraction objective or the alignment method, running or interpreting experiments, or producing any reported number. All AI-assisted text and scripts were verified by the authors, who checked every rendered table and figure against the raw experiment logs. Ethics statement This work studies the internal mechanisms of safety refusal in order to make alignment more effective and less costly to utility. All harmful prompts and completions come from established public safety benchmarks and alignment datasets used under their original licenses; we created no new harmful content, collected no data from human subjects, scored outputs automatically rather than exposing annotators to them, and release no harmful generations or attack artifacts. We acknowledge that our ablation experiments suppress refusal and are in principle an uncensoring recipe, but the intervention needs white-box weight access. More importantly, our work aims to understand the safety mechanisms in LLMs for better alignment methods. Reproducibility statement We release anonymized code at the URL in the abstract; all models, datasets, and judges are public. The methods, baselines, and metrics are fully specified in the main text. The appendices then supply the settings and protocols behind the reported runs: Appendix A gives the circuit budgets, mask provenance, full extraction configuration, and our reimplementation of the external baseline; Appendix B the exact ablation values together with the intervention and refusal-scoring protocol and its controls; Appendix C the multi-seed and paired bootstrap procedure, so readers can tell which claims carry seed-level variance; Appendix D the XSTest scoring protocol; and Appendix E all training, generation, and evaluation hyperparameters. References Behrouzi et al. (2026a) S. Behrouzi, L. Wu, M. Rostami, and A. Sadeghi NeST: neuron selective tuning for llm safety. arXiv preprint arXiv:2602.16835. Cited by: §1. Behrouzi et al. (2026b) S. Behrouzi, L. Wu, M. Rostami, and A. Sadeghi NeST: neuron selective tuning for LLM safety. arXiv preprint arXiv:2602.16835. External Links: Link Cited by: §2. Bereska and Gavves (2024) L. Bereska and E. Gavves Mechanistic interpretability for ai safety–a review. arXiv preprint arXiv:2404.14082. Cited by: §1. Cao et al. (2026) C. Cao, X. Xu, B. Han, and H. Li Reasoned safety alignment: ensuring jailbreak defense via answer-then-check. In International Conference on Learning Representations, Vol. 2026, p. 47828–47869. Cited by: §1. Chen et al. (2024) J. Chen, X. Wang, Z. Yao, Y. Bai, L. Hou, and J. Li Towards understanding safety alignment: A mechanistic perspective from safety neurons. arXiv preprint arXiv:2406.14144. External Links: Link Cited by: §3.1. Cobbe et al. (2021) K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman Training verifiers to solve math word problems. CoRR abs/2110.14168. External Links: Link, 2110.14168 Cited by: §4.2. Deng et al. (2026) X. Deng, H. Zhong, R. Ai, F. Feng, Z. Wang, and X. He Less is more: improving llm alignment via preference data selection. Advances in Neural Information Processing Systems 38, p. 161259–161285. Cited by: §2. Elhage et al. (2021) N. Elhage, N. Nanda, C. Olsson, T. Henighan, N. Joseph, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly, et al. A mathematical framework for transformer circuits. Transformer Circuits Thread 1 (1), p. 12. Cited by: §1. Fan et al. (2026) J. Fan, W. Xu, M. Liu, O. Sokolsky, I. Lee, and F. Kong SafeGen-llm: enhancing safety generalization in task planning for robotic systems. arXiv preprint arXiv:2602.24235. Cited by: §2. Ferrag et al. (2026) M. A. Ferrag, N. Tihanyi, and M. Debbah From llm reasoning to autonomous ai agents: a comprehensive review. IEEE Access. Cited by: §1. He et al. (2025) Z. He, Z. Wang, H. Xu, H. Lin, W. Zhang, and Z. Chu Interpretable llm guardrails via sparse representation steering. arXiv preprint arXiv:2503.16851. Cited by: §2. Hendrycks et al. (2021) D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, External Links: Link Cited by: §4.2. Hu et al. (2026) M. Hu, V. Datla, A. Kumar, Z. Guan, S. Li, A. Samuel, and D. Liu Alignment-weighted dpo: a principled reasoning approach to improve safety alignment. In International Conference on Learning Representations, Vol. 2026, p. 63876–63899. Cited by: §1. Huang et al. (2026) S. Huang, L. Lin, Y. Li, K. Luo, Z. Zhou, A. Zhang, J. Dong, K. Wang, and Z. Zeng Explaining and breaking the safety-helpfulness ceiling via preference dimensional expansion. arXiv preprint arXiv:2605.11679. Cited by: §2. Jalan et al. (2026) P. Jalan, V. Abishethvarman, B. Chandna, and U. Naseem Survey on llm safety: attacks, defenses, alignment, metrics, and guardrails. Machine Learning 115 (6), p. 130. Cited by: §2. Ji et al. (2026a) J. Ji, X. Chen, R. Pan, H. Zhu, J. Li, D. Hong, B. Chen, J. Zhou, K. Wang, J. Dai, et al. Safe rlhf-v: safe reinforcement learning from multi-modal human feedback. Advances in Neural Information Processing Systems 38, p. 46146–46182. Cited by: §2. Ji et al. (2023a) J. Ji, M. Liu, J. Dai, X. Pan, C. Zhang, C. Bian, B. Chen, R. Sun, Y. Wang, and Y. Yang Beavertails: towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Systems 36, p. 24678–24704. Cited by: §4.2. Ji et al. (2023b) J. Ji, M. Liu, J. Dai, X. Pan, C. Zhang, C. Bian, B. Chen, R. Sun, Y. Wang, and Y. Yang BeaverTails: towards improved safety alignment of LLM via a human-preference dataset. Advances in Neural Information Processing Systems. Cited by: §1, §3.2, §4.2. Ji et al. (2026b) M. Ji, Y. Wu, Z. Wu, S. Wang, J. Yang, M. Dras, and U. Naseem A survey of progress in llm alignment from the perspective of reward design. IEEE Transactions on Artificial Intelligence. Cited by: §1, §2. Jiang et al. (2023) A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de Las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed Mistral 7b. CoRR abs/2310.06825. External Links: Link, Document, 2310.06825 Cited by: §4.2. Jiao et al. (2026) D. Jiao, Y. Liu, Y. Yuan, Z. Tang, L. Du, H. Wu, and A. Anderson LLM safety from within: detecting harmful content with internal representations. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 39711–39727. Cited by: §1, §2. Kim et al. (2026) T. Kim, F. Tajwar, A. Raghunathan, and A. Kumar Reasoning as an adaptive defense for safety. Advances in Neural Information Processing Systems 38, p. 77024–77073. Cited by: §1. Kissane et al. (2024) C. Kissane, robertzk, A. Conmy, and N. Nanda Base LLMs refuse too. Note: AI Alignment ForumSeptember 29, 2024. Online research report External Links: Link Cited by: §3. Lad (2024) V. K. Lad Mechanistic interpretability for progress towards quantitative ai safety. Ph.D. Thesis, Massachusetts Institute of Technology. Cited by: §2. Lee and Kim (2026) D. Lee and H. Kim A survey on llm agents: architecture, applications, and challenges. In 2026 40th International Conference on Information Networking (ICOIN), p. 979–982. Cited by: §1. Lee et al. (2025) S. Lee, A. Cho, G. C. Kim, S. Peng, M. Phute, and D. H. Chau Interpretation meets safety: a survey on interpretation methods and tools for improving llm safety. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 21514–21545. Cited by: §1, §2. Li et al. (2025a) M. Li, W. M. Si, M. Backes, Y. Zhang, and Y. Wang SaLoRA: safety-alignment preserved low-rank adaptation. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §1. Li et al. (2025b) S. Li, L. Yao, L. Zhang, and Y. Li Safety layers in aligned large language models: the key to LLM security. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §1. Li et al. (2026) X. Li, H. Zhen, L. Yin, X. Yu, Z. Dong, and M. Yuan What matters for safety alignment?. arXiv preprint arXiv:2601.03868. Cited by: §1. Liu et al. (2023) Y. Liu, Y. Yao, J. Ton, X. Zhang, R. Guo, H. Cheng, Y. Klochkov, M. F. Taufiq, and H. Li Trustworthy llms: a survey and guideline for evaluating large language models’ alignment. arXiv preprint arXiv:2308.05374. Cited by: §2. Llama Team (2024) Llama Team The llama 3 herd of models. CoRR abs/2407.21783. External Links: Link, Document, 2407.21783 Cited by: §4.2. Luo et al. (2026a) H. Luo, S. Dai, C. Ni, X. Li, G. Zhang, K. Wang, T. Liu, and H. Salam Agentauditor: human-level safety and security evaluation for llm agents. Advances in Neural Information Processing Systems 38, p. 43241–43298. Cited by: §1. Luo et al. (2026b) K. Luo, Z. Zhou, L. Wang, L. Lin, T. Shao, Y. Zhang, Y. Xiao, Y. Li, M. Yu, K. Lyu, et al. A survey of large audio language models: generalization, trustworthiness, and outlook. arXiv preprint arXiv:2605.20266. Cited by: §2. Luo et al. (2026c) X. Luo, Y. Wei, and R. Zhao Detecting what queries seek: steering llm safety with ffn output activation monitoring. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 29500–29514. Cited by: §1. Mazeika et al. (2024) M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basart, B. Li, D. A. Forsyth, and D. Hendrycks HarmBench: A standardized evaluation framework for automated red teaming and robust refusal. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, R. Salakhutdinov, Z. Kolter, K. A. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp (Eds.), Proceedings of Machine Learning Research, p. 35181–35224. External Links: Link Cited by: §1, §4.2. Meta AI (2024) Meta AI Llama guard 3: content safety classifier. Technical report Meta AI. Note: Model release accompanying the Llama 3 herd Cited by: §4.2. Naseem (2026) U. Naseem Mechanistic interpretability for large language model alignment: progress, challenges, and future directions. arXiv preprint arXiv:2602.11180. Cited by: §2. Niu et al. (2026) Y. Niu, H. Xiao, D. Liu, N. Chen, and J. Li Mitigating the safety alignment tax with null-space constrained policy optimization. In International Conference on Learning Representations, Vol. 2026, p. 142434–142465. Cited by: §2. Ouyang et al. (2022) L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems. Cited by: §1. Qwen Team (2024) Qwen Team Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. Cited by: §4.2. Röttger et al. (2024) P. Röttger, H. Kirk, B. Vidgen, G. Attanasio, F. Bianchi, and D. Hovy XSTest: A test suite for identifying exaggerated safety behaviours in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21, 2024, K. Duh, H. Gómez-Adorno, and S. Bethard (Eds.), p. 5377–5400. External Links: Link, Document Cited by: Appendix B, Appendix D, §4.2. Sheshadri et al. (2025) A. Sheshadri, A. Ewart, P. Guo, A. Lynch, C. Wu, V. Hebbar, H. Sleight, A. C. Stickland, E. Perez, D. Hadfield-Menell, and S. Casper Latent adversarial training improves robustness to persistent harmful behaviors in llms. Trans. Mach. Learn. Res. 2025. External Links: Link Cited by: §3.2, §4.2. Team et al. (2026) K. Team, T. Bai, Y. Bai, Y. Bao, J. Cai, X. Cai, P. Cao, Y. Cao, Z. Chai, Y. Charles, et al. Kimi k3: open frontier intelligence. arXiv preprint arXiv:2607.24653. Cited by: §1. Wang et al. (2025) K. Wang, G. Zhang, Z. Zhou, J. Wu, M. Yu, S. Zhao, C. Yin, J. Fu, Y. Yan, H. Luo, et al. A comprehensive survey in llm (-agent) full stack safety: data, training and deployment. arXiv preprint arXiv:2504.15585. Cited by: §1. Wang et al. (2026) Z. Wang, J. Liang, F. Zhu, W. Zhao, J. Fang, J. Ji, H. Wang, and T. Chua SafeNeuron: neuron-level safety alignment for large language models. arXiv preprint arXiv:2602.12158. External Links: Link Cited by: §2. Wang et al. (2024) Z. Wang, Y. Dong, O. Delalleau, J. Zeng, G. Shen, D. Egert, J. J. Zhang, M. N. Sreedhar, and O. Kuchaiev Helpsteer 2: open-source dataset for training top-performing reward models. Advances in Neural Information Processing Systems 37, p. 1474–1501. Cited by: §2. Xu et al. (2026) A. Xu, B. Lin, B. Xue, B. Wang, B. Xu, B. Wu, B. Zhang, C. Lin, C. Dong, C. Ling, et al. Deepseek-v4: towards highly efficient million-token context intelligence. arXiv preprint arXiv:2606.19348. Cited by: §1. Yang et al. (2026) S. Yang, Q. Zhang, Y. Liu, Y. Huang, X. Jia, K. Ning, J. Yao, J. Wang, D. Hailiang, Y. Song, et al. Asft: anchoring safety during llm fine-tuning within narrow safety basin. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 34322–34330. Cited by: §1, §2. Yi et al. (2024) X. Yi, S. Zheng, L. Wang, G. de Melo, X. Wang, and L. He NLSR: neuron-level safety realignment of large language models against harmful fine-tuning. arXiv preprint arXiv:2412.12497. External Links: Link Cited by: §3.1. Yi et al. (2025) X. Yi, S. Zheng, L. Wang, G. de Melo, X. Wang, and L. He Nlsr: neuron-level safety realignment of large language models against harmful fine-tuning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 25706–25714. Cited by: §1. Yu et al. (2026) M. Yu, S. Fu, M. Aloqaily, Z. Zhou, S. Otoum, X. Fan, K. Wang, Y. Guo, and Q. Wen SafeSeek: universal attribution of safety circuits in language models. arXiv preprint arXiv:2603.23268. External Links: Link Cited by: §2. Yu et al. (2025) M. Yu, F. Meng, X. Zhou, S. Wang, J. Mao, L. Pan, T. Chen, K. Wang, X. Li, Y. Zhang, et al. A survey on trustworthy llm agents: threats and countermeasures. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, p. 6216–6226. Cited by: §1. Zellers et al. (2019) R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi HellaSwag: can a machine really finish your sentence?. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, A. Korhonen, D. R. Traum, and L. Màrquez (Eds.), p. 4791–4800. External Links: Link, Document Cited by: §4.2. Zhang et al. (2024) Z. Zhang, S. Cui, Y. Lu, J. Zhou, J. Yang, H. Wang, and M. Huang Agent-safetybench: evaluating the safety of llm agents. arXiv preprint arXiv:2412.14470. Cited by: §2. Zhao et al. (2026) C. Zhao, Y. Ke, and K. Huang Unraveling llm jailbreaks through safety knowledge neurons. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), p. 1889–1906. Cited by: §2. Zhao et al. (2025a) X. Zhao, W. Cai, T. Shi, D. Huang, L. Lin, S. Mei, and D. Song Improving llm safety alignment with dual-objective optimization. arXiv preprint arXiv:2503.03710. Cited by: §1. Zhao et al. (2025b) Y. Zhao, W. Zhang, Y. Xie, A. Goyal, K. Kawaguchi, and M. Q. Shieh Understanding and enhancing safety mechanisms of LLMs via safety-specific neuron. In The Thirteenth International Conference on Learning Representations, Cited by: §A.1, §1, §2, §2, §3.1, §4.2. Zhao et al. (2025c) Y. Zhao, W. Zhang, Y. Xie, A. Goyal, K. Kawaguchi, and M. Q. Shieh Understanding and enhancing safety mechanisms of llms via safety-specific neuron. In International Conference on Learning Representations, Vol. 2025, p. 44113–44127. Cited by: §1. Zheng et al. (2026) Z. Zheng, L. He, J. Zhao, L. Yang, and F. Li SAHs: unraveling safety attention heads in large vision-language models. IEEE Transactions on Circuits and Systems for Video Technology. Cited by: §1, §2. Zheng et al. (2025) Z. Zheng, J. Zhao, L. Yang, L. He, and F. Li Spot risks before speaking! unraveling safety attention heads in large vision-language models. arXiv preprint arXiv:2501.02029. Cited by: §1. Zhou et al. (2025a) D. Zhou, J. Zhang, T. Feng, and Y. Sun A survey on alignment for large language model agents. In UIUC Spring 2025 CS598 LLM Agent Workshop, Cited by: §1. Zhou et al. (2024) Z. Zhou, H. Yu, X. Zhang, R. Xu, F. Huang, and Y. Li How alignment and jailbreak work: explain llm safety through intermediate hidden states. In Findings of the Association for Computational Linguistics: EMNLP 2024, p. 2461–2488. Cited by: §1, §1. Zhou et al. (2025b) Z. Zhou, H. Yu, X. Zhang, R. Xu, F. Huang, K. Wang, Y. Liu, J. Fang, and Y. Li On the role of attention heads in large language model safety. In International Conference on Learning Representations, Vol. 2025, p. 84042–84071. Cited by: §1, §1, §2. Zou et al. (2023) A. Zou, Z. Wang, J. Z. Kolter, and M. Fredrikson Universal and transferable adversarial attacks on aligned language models. CoRR abs/2307.15043. External Links: Link, Document, 2307.15043 Cited by: §1, §4.2. Appendix A Per-Model Sparsity and Mask Provenance Table 5 reports authoritative trainable-parameter and realized-circuit statistics from each circuit run’s metadata. The trainable fraction is tight across families (0.350.35–0.40%0.40\%), while the realized active-neuron counts are model-specific. Table 7 identifies the masks used for ablation and SCA tuning; these masks are extracted on the untuned weights of the corresponding models. Table 5: Per-model circuit budget (seed-1 representative; other seeds identical recipe). Trainable params are fixed by LoRA rank × patched-Linear dimensions, independent of mask size. Family active circuit neurons patched Linears trainable params trainable % Llama-3-8B 10,376 96 28.31M 0.3513 Qwen-2.5-7B 3,594 84 30.28M 0.3960 Mistral-Inst-v0.1 5,724 96 28.31M 0.3894 A.1 External SN-Tune selector baseline We reimplement the released PLND semantics of SN-Tune (Zhao et al., 2025b). On the official 200-prompt English harmful corpus, for each prompt and layer we retain the top-1000 attention and top-2000 MLP coordinates and intersect the prompt-level selections, capped at 100 per layer and component. The resulting mask gates LoRA updates to q/k/ v output rows, MLP up output rows, and matching down input columns. The selected and unselected gradients were audited to be respectively nonzero and exactly zero. Table 6: SN-Tune selector sizes used in the external baseline. MLP coordinates gate both up and down, so tuned projection slices equal q+k+v+2MLPq+k+v+2\,MLP. Family q k v MLP tuned slices Llama-3-8B 3,200 3,200 1,461 1,679 11,219 Qwen-2.5-7B 2,781 2,771 1,270 1,650 10,122 Mistral-Inst-v0.1 3,200 3,200 1,107 1,548 10,603 Table 7: Mask provenance for ablation and SCA tuning. Each row reports the circuit object, neuron count, and extraction checkpoint used by that experiment. Experiment Mask object Neurons (Llama/Qwen/Mistral) Extracted on Ablation (Section 3.2) 1C_1 10,376 / 9,972 / 7,677 base weights SCA tuning (Section 4.3, Table 2) C 10,376 / 3,594 / 5,724 base weights A.2 Recorded extraction configuration Table 8 records the final three starting-model extractions used by the ablation in Section 3.2. These settings instantiate Equation 6; Table 7 separately identifies the distinct mask object used by every experiment. Table 8: Recorded circuit-extraction configuration. Circuit-extraction configuration for the ablation experiments in Section 3.2. Setting Recorded value Data and split LLM-LAT harmful-prompt records, each paired with a refusal and harmful-compliance completion (4,948 available); first 100 for training and the next 50 for validation; model-native chat template; maximum length 256. Search space One logit per output row of MLP gate/up/down; model weights frozen; mask logits only are optimized in bf16. Loss coefficients α=1α=1, β=1β=1, and λmlp=0.05 _mlp=0.05, applied to the sum of per-module mean soft masks in Equation 6. Mask relaxation Logit initialization q0=0.2q_0=0.2; temperature 11; hard forward gate [σ(q)>0.5]1[σ(q)>0.5] with a straight-through estimator. Optimization Fused AdamW, learning rate 10−210^-2, batch size 4, no gradient accumulation, linear schedule, zero warmup, zero weight decay. Horizon and selection 100-epoch schedule horizon; validation after each epoch; checkpoint comparison uses the final validation minibatch’s two completion losses (sparsity excluded); stop after epoch 2 (50 optimizer updates), which was the selected mask for all three families. Extraction seeds Llama-3-8B: 1; Qwen-2.5-7B: 2; Mistral-7B-Instruct-v0.1: 2. Deployed C1C_1 readout Globally highest raw MLP logits retained with k=10,376/9,972/7,677k=10,376/9,972/7,677 for Llama/Qwen/Mistral. The minimum selected logits were 0.496/0.471/0.4960.496/0.471/0.496, respectively. Matched-random control Prespecified seed 42; uniform without replacement within each Linear, matching that Linear’s deployed C1C_1 width. Appendix B Ablation: extracted circuit versus a fixed matched-random mask Table 9 reports the exact zero-ablation values behind Figure 3. Table 9: Starting-model circuit ablation. Explicit-refusal rate and raw LlamaGuard-3 ASR for the intact model, 1C_1 ablation, and a prespecified seed-42 width-matched random ablation. Family Metric base ablate 1C_1 ablate matched random Llama-3-8B refusal (%) 55.35 0.00 37.74 ASR (%) 48.43 91.19 49.06 Qwen-2.5-7B refusal (%) 61.01 0.63 22.64 ASR (%) 32.70 88.68 64.15 Mistral-Inst-v0.1 refusal (%) 35.85 0.63 42.14 ASR (%) 42.14 88.05 35.22 Refusal rate uses the classifier of Röttger et al. (2024) applied to the first 150150 characters of each generation after truncation at the first conversational boundary and removal of any leading verbatim prompt echo; ASR is raw LlamaGuard-3 on those same outputs. For each family, 1C_1 is read from one extraction by retaining the globally highest-scoring 10,37610,376/9,9729,972/7,6777,677 MLP entries for Llama/Qwen/Mistral. The paired random control is a prespecified seed-42 draw that matches the active count of 1C_1 in every Linear. A null (empty-mask) ablation reproduces the base generation exactly, confirming that the intervention hook is transparent. We use zero-ablation for both selected and random identities so that intervention type and lesion width are held fixed. For the 50-prompt sufficiency diagnostic in Table 1, explicit refusals include both standard refusal phrases and direct negative formulations that decline the harmful request. Under this criterion, the intact Mistral model refuses 8 of 50 prompts (16.00%). Appendix C Multi-seed robustness and paired bootstrap Table 4 in the main text aggregates HarmBench raw ASR over n=4n=4 training seeds per condition (identical recipe, differing only in training seed); The same table reports the paired prompt-level bootstrap: for each family we resample the 159159 HarmBench prompts with replacement and recompute the ASR difference (width-matched random −- circuit) on each resample; all three 95%95\% intervals exclude zero. The n=18n=18 random ensemble on Llama (Section 4.3) is disjoint from the circuit seeds (worst circuit seed 8.818.81 vs. best random 11.3211.32). Appendix D XSTest coverage–over-refusal protocol Table 3 in the main text jointly reports the two XSTest subsets (Röttger et al., 2024). The first value in each cell is raw Llama-Guard-3 ASR on the 200200 unsafe-contrast prompts: harmful requests phrased to superficially resemble the benign safe set. The second is the explicit-refusal rate on the 250250 benign prompts, computed with the same truncated, de-echoed classifier used for Table 2. Full-SFT obtains broader unsafe-contrast coverage than circuit-SFT but over-refuses more; under DPO, circuit tuning reaches lower unsafe-contrast ASR on Llama and Mistral, while Qwen full-DPO’s near-total reduction coincides with 34.8%34.8\% benign-prompt over-refusal. Appendix E Hyperparameters Circuit and width-matched-random runs use LoRA rank 1616, α=16α=16, dropout 00, and bf16 on the MLP gate/up/down projections with frozen attention; full-LoRA also patches all attention Linears. SFT uses learning rate 10−410^-4, 1616 epochs, maximum length 512512, and effective batch 44; the Llama circuit-SFT run uses batch 1616 for 49604960 steps and a post-plateau checkpoint. DPO uses β=0.1β=0.1, 33 epochs, effective batch 88, and learning rate 10−410^-4 for Llama/Qwen or ×10−55\!×\!10^-5 for Mistral. Generation uses batch 1616, and Llama-Guard-3-8B judging uses greedy decoding. The SN-Tune baseline matches the SFT data, rank, learning rate, epochs, and maximum length, with batch 1616 for Llama and 44 for Qwen/Mistral. Utility uses the 55-shot harness capped at 20002000 examples per task.