Paper deep dive
SafeSeek: Universal Attribution of Safety Circuits in Language Models
Miao Yu, Siyuan Fu, Moayad Aloqaily, Zhenhong Zhou, Safa Otoum, Xing fan, Kun Wang, Yufei Guo, Qingsong Wen
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/26/2026, 1:45:48 AM
Summary
SafeSeek is a unified mechanistic interpretability framework that uses gradient-based optimization and differentiable binary masks to identify sparse, functionally complete safety circuits in Large Language Models (LLMs). It supports multi-granular attribution (weights, neurons, heads, layers) and introduces Safety Circuit Tuning (SaCirT) to efficiently enhance model safety by updating only the identified circuit parameters. The framework is validated on backdoor attack mitigation and safety alignment, demonstrating high sparsity and effectiveness in preserving general utility.
Entities (5)
Relation Signals (4)
SafeSeek → identifies → Safety Circuits
confidence 95% · SafeSeek, a unified safety interpretability framework that identifies functionally complete safety circuits in LLMs
SaCirT → optimizes → Safety Circuits
confidence 95% · Safety Circuit Tuning (SaCirT), an efficient fine-tuning technique that exclusively updates parameters of the identified circuits
SafeSeek → validatedon → Backdoor Attacks
confidence 95% · We validate SafeSeek in two key scenarios in LLM safety: (1) backdoor attacks
SafeSeek → validatedon → Safety Alignment
confidence 95% · We validate SafeSeek in two key scenarios in LLM safety: (2) safety alignment
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Mechanistic interpretability reveals that safety-critical behaviors (e.g., alignment, jailbreak, backdoor) in Large Language Models (LLMs) are grounded in specialized functional components. However, existing safety attribution methods struggle with generalization and reliability due to their reliance on heuristic, domain-specific metrics and search algorithms. To address this, we propose \ourmethod, a unified safety interpretability framework that identifies functionally complete safety circuits in LLMs via optimization. Unlike methods focusing on isolated heads or neurons, \ourmethod introduces differentiable binary masks to extract multi-granular circuits through gradient descent on safety datasets, while integrates Safety Circuit Tuning to utilize these sparse circuits for efficient safety fine-tuning. We validate \ourmethod in two key scenarios in LLM safety: \textbf{(1) backdoor attacks}, identifying a backdoor circuit with 0.42\% sparsity, whose ablation eradicates the Attack Success Rate (ASR) from 100\% $\to$ 0.4\% while retaining over 99\% general utility; \textbf{(2) safety alignment}, localizing an alignment circuit with 3.03\% heads and 0.79\% neurons, whose removal spikes ASR from 0.8\% $\to$ 96.9\%, whereas excluding this circuit during helpfulness fine-tuning maintains 96.5\% safety retention.
Tags
Links
- Source: https://arxiv.org/abs/2603.23268v1
- Canonical: https://arxiv.org/abs/2603.23268v1
Trouble viewing inline? Open PDF directly →
Full Text
79,302 characters extracted from source content.
Expand or collapse full text
SafeSeek: Universal Attribution of Safety Circuits in Language Models Miao Yu Siyuan Fu Moayad Aloqaily Zhenhong Zhou Safa Otoum Xing Fan Kun Wang† Yufei Guo Qingsong Wen† Abstract Mechanistic interpretability reveals that safety-critical behaviors (e.g., alignment, jailbreak, backdoor) in Large Language Models (LLMs) are grounded in specialized functional components. However, existing safety attribution methods struggle with generalization and reliability due to their reliance on heuristic, domain-specific metrics and search algorithms. To address this, we propose SafeSeek, a unified safety interpretability framework that identifies functionally complete safety circuits in LLMs via optimization. Unlike methods focusing on isolated heads or neurons, SafeSeek introduces differentiable binary masks to extract multi-granular circuits through gradient descent on safety datasets, while integrates Safety Circuit Tuning to utilize these sparse circuits for efficient safety fine-tuning. We validate SafeSeek in two key scenarios in LLM safety: (1) backdoor attacks, identifying a backdoor circuit with 0.42% sparsity, whose ablation eradicates the Attack Success Rate (ASR) from 100% → 0.4% while retaining over 99% general utility; (2) safety alignment, localizing an alignment circuit with 3.03% heads and 0.79% neurons, whose removal spikes ASR from 0.8% → 96.9%, whereas excluding this circuit during helpfulness fine-tuning maintains 96.5% safety retention. Our code is available at: https://github.com/Ymm-cll/SafeSeek. Machine Learning, ICML 1 Introduction As Large Language Models (LLMs) increasingly pervade high-stakes applications (batra2025review; zeng2025adrd), ensuring their alignment with human values and robustness against adversarial threats is paramount (wang2025comprehensive; du2025advancing). Beyond behavioral alignment techniques such as RLHF (wang2024comprehensive), safety interpretability (lee2025interpretation; bereska2024mechanistic) has emerged as a critical frontier, aiming to causally attribute high-level safety-related behaviors—such as refusal mechanisms (chen2024finding) or malicious backdoor behaviors (yu2025backdoor)—to specific internal components or so-called “circuits” (zhao2025unraveling; zhou2024role). Deciphering these underlying structures is essential, as it promises to transform safety from a black-box guarantee into a transparent, controllable, and rigorously verifiable property of the neural architecture (ghosh2025safesteer). Figure 1: Overview of our SafeSeek framework (Middle), with two critical scenarios in LLM safety (Left) and case studies (Right). While circuit discovery for model capabilities has advanced significantly through algorithms like ACDC (conmy2023towards) and EAP (syed2024attribution), analogous techniques for safety abilities remain underdeveloped. Prevailing safety interpretability relies on causal attribution (yeo2025towards; yu2025backdoor), or heuristic search (zhou2024role) to isolate safety-relevant components. Besides, some works design scenario-specific attribution metrics to highlight “safety neurons” (chentowards; zhao2025unraveling) or heads (zheng2025spot), but suffer from fundamental limitations. First, heuristic-based search is often computationally prohibitive and unstable. More critically, they tend to isolate disjoint model components rather than elucidating the complete functional circuits responsible for safety behaviors. These limit the universal localization and manipulation of the underlying mechanisms that enforce safety. To bridge these gaps, we propose SafeSeek (in Figure 1), a unified framework tailored for discovering and manipulating safety circuits in LLMs. Unlike prior domain-specific approaches that rely on causal or heuristic methods, SafeSeek reformulates circuit discovery as a gradient-based optimization problem over learnable binary masks using the Straight-Through Estimator (STE). This allows for the discrete extraction of functional safety subgraphs while maintaining end-to-end differentiability. Our framework features a flexible multi-granular perspective, generalizing the definition of circuit units to span from individual weights, generalized neurons, attention heads, and layers, thereby enabling customized attribution across heterogeneous architectures. Furthermore, going beyond mere interpretation, we introduce Safety Circuit Tuning (SaCirT), an efficient fine-tuning technique that exclusively updates parameters of the identified circuits to significantly enhance LLM safety. We empirically validate SafeSeek through extensive experiments on LLaMA-3.1-8B-Instruct (grattafiori2024llama) and Qwen-3-8B (yang2025qwen3), demonstrating its efficacy in two critical safety scenarios: (1) Backdoor Attacks: We identify a highly sparse backdoor circuit (0.42% sparsity), whose ablation eradicates the Attack Success Rate (ASR) from 100% → 0.4% while retaining over 99% of general utility. (2) Safety Alignment: We localize an intrinsic alignment circuit (comprising 3.03% heads and 0.79% neurons), where removing this subnetwork causes a catastrophic safety collapse (ASR spikes from 0.8% → 96.9%) while notably preserving 92.1% of general capabilities. Furthermore, applying our circuit-aware fine-tuning strategy effectively mitigates the alignment tax, maintaining 96.5% safety retention during helpfulness optimization. In summary, our contributions can be listed as follows: • Universal Framework. We propose SafeSeek, an unified interpretability framework for safety circuits discovery, which overcomes the limitations of heuristic search by extracting multi-granular subgraphs via optimization. • Mechanistic Insights. Our analysis reveals that safety behaviors (e.g. backdoor activation, safety alignment) rely on highly sparse circuits (e.g., <1%<1\%) that are structurally orthogonal to general utility components, enabling precise enhancement or removal safety abilities. • Efficient Control. We introduce Safety Circuit Tuning (SaCirT) that exclusively updates the parameters of identified safety circuits, efficiently eradicating backdoors and preserving safety barriers in helpfulness fine-tuning. 2 Related Work Circuit Discovery. Recent advances in mechanistic interpretability have shifted towards automated circuit discovery (hanna2025circuit; hasani2026mechanistic), aiming to reverse-engineer LLM behaviors into sparse computational subgraphs (circuits). Foundational works established greedy search algorithms, such as path patching (goldowsky2023localizing) and ACDC (conmy2023towards), to iteratively isolate potentially functionally complete circuits. To mitigate computational constraints, subsequent approaches leverage efficient attribution methods—including attribution patching (syed2024attribution), edge pruning (bhaskar2024finding), and information bottleneck principles (bianibcircuit)—to scale analysis to larger LLMs. Parallel efforts have explored alternative levels of granularity and structural alignments, ranging from verifying causal abstractions (geiger2021causal) and subnetworks (cao2021low) to identifying sparse feature circuits (marks2024sparse) and training inherently weight-sparse models (gao2025weight). Collectively, these works offer granular insights into complex LLM phenomena, including the mechanics of knowledge storage (yao2024knowledge), in-context learning inference (cho2024revisiting), and the evolution of circuits during fine-tuning (wang2025towards). However, the development of frameworks specifically tailored for safety-critical circuit discovery remains in a nascent stage. Safety Interpretability. Distinct from research focused on designing attacks or enhancing defenses, safety interpretability (lee2025interpretation; wang2025comprehensive; bereska2024mechanistic) investigates the mechanistic underpinnings of adversarial dynamics and the internal logic of safety mechanisms. Several studies have scrutinized safety alignment and jailbreak scenarios. For instance, zhou2024alignment employ the LogitLens (wang2025logitlens4llms) to demonstrate that alignment and jailbreak outcomes are contingent upon whether intermediate activations correlate with positive or negative sentiments. Meanwhile, ji2025language characterize the “Elasticity” phenomenon, revealing the inherent resistance of LLMs to safety alignment and its susceptibility to degradation. Other works leverage techniques such as activation patching and causal attribution to identify sparse attention heads critical for safety (zheng2025spot; zhou2024role; zaree2025attention) and backdoors (yu2025backdoor), whose ablation directly compromises model safety. Furthermore, while chentowards; chen2024finding establish the existence of “safety neurons”, subsequent research has expanded this neuronal perspective to address jailbreak (zhao2025unraveling), safety enhancement (zhao2025understanding; yi2025nlsr; han2025fine), multi-objective alignment (pan2025neurontune), and disalignment (zhou2025neurel). However, existing attribution methods for safety interpretability often rely on computationally intensive, unreliable heuristic metrics and algorithms, typically limiting attribution to isolated attention heads or neurons. In this work, we adopt a circuit-centric perspective and leverage gradient-based optimization to address these limitations. 3 Preliminary LLM as Graph. An LLM can be represented as a directed computational graph =(,ℰ)G=(V,E), parameterized by θ. The node set V consists of functional components, like attention heads or Multi-layer Perceptron (MLP) layers (ferraris2025architecture), while the directed edges denote the computational dependencies between them (yao2024knowledge). For a given input x, the model yields an output distribution p(y|x;)p(y|x;G). Circuit Formulation. A circuit ⊆C is defined as a subgraph that primarily governs the LLM’s certain ability on a specific task dataset D (conmy2023towards). Formally, the goal of circuit discovery can be framed as: ||||≤τ,min⊆x∼[DKL(p(y|x;)∥p(y|x;))] |C||G|≤τ, _C E_x [D_KL (p(y|x;G)\,\|\,p(y|x;C) ) ] (1) where Kullback-Leibler divergence DKLD_KL measures the behavioral discrepancy, and τ denotes the sparsity threshold. 4 SafeSeek In this section, we introduce SafeSeek, a general framework for the attribution of interpretable safety circuits in LLMs, which comprises three key stages. Our pipeline begins by customizing the granularity of units comprising safety circuits (⊳ Section 4.1). Then we employ mask optimization at the selected granularity to identify viable solutions (⊳ Section 4.2). Finally, we validate candidate circuits through ablation experiments (⊳ Section 4.3). Besides, we propose an efficient fine-tuning method that utilizes these identified circuits to enhance model safety (⊳ Section 4.4). 4.1 Unit Granularity Selection LLM architectures intrinsically comprise heterogeneous conceptual entities across various hierarchical levels, such as heads within attention modules (zheng2025attention) and neurons within MLP layers (lyu2025evoedit). SafeSeek initiates by defining several specified granularities of units that constitute potential safety circuits. Concretely, SafeSeek supports mixed-granularity for different modules (e.g., attention, MLPs) to facilitate customized attribution, providing four distinct granularities: Individual Weight (w): This is the finest granularity, defined as an individual scalar parameter w∈ℝw indexed within weight matrices (e.g., a single weight w in ∈W∈ θ). Neuron (n): Departing from the conventional view that limits neuron analysis solely to MLP modules (chen2024finding), we adopt a generalized neuron definition. For the input ∈ℝdinx ^d_in and weight matrix ∈ℝdin×doutW ^d_in× d_out, we define any computational structure within LLMs governed by the transformation =σ(⊤)=n(x)y=σ(x W)=n(x) as a neuron structure. Here, σ denotes the non-linear activation function, and this structure comprises doutd_out distinct neurons. Attention Head (h): Operating within Multi-Head Attention (MHA) (vaswani2017attention) modules, these components are designed to capture relational dependencies between embeddings or hidden states (zheng2024attention). Formally, a single attention head unit h is defined by a collection of parameters Q,K,V,O⊂\W_Q,W_K,W_V,W_O\⊂ θ, governing the core attention mechanism h()=[Softmax(x⊤Q(x⊤K)⊤/dK)x⊤V]Oh(x)=[Softmax(x W_Q(x W_K) / d_K)x W_V]W_O. Layer (L): The coarsest unit represents the aggregate transformation at depth l, mapping the residual stream activation l+1=L(l)=l+MHA(l)+MLP(l)x_l+1=L(x_l)=x_l+MHA(x_l)+MLP(x_l). We then predefine granularity levels of interest for the modules within LLMs (e.g., MHA, MLP), splitting them into a large number of units. For instance, the MHA module can be analyzed at either the neuron or attention-head granularity. 4.2 Circuit Mask Optimization Modules in G are split into individual units by their predefined granularities (w, n, h, and L). We denote all units to constitute the sets w()D_w(G), n()D_n(G), h()D_h(G), and L()D_L(G), respectively. Then a potential safety circuit ⊂uC _u is a subset of the universal unit set u=w∪n∪h∪LD_u=D_w _n _h _L (We omit ()(G) in the following for brevity.). Given the potential for structural overlaps among modules, we constrain w∩n∩h∩L=∅D_w _n _h _L= during the selection phase. For each unit u∈u _u, we introduce a corresponding latent variable zu∈ℝz_u and derive a mask variable mu∈(0,1)m_u∈(0,1): mu=sigmoid(zu),um(x)=u(x)⊕mu.m_u=sigmoid(z_u), u_m(x)=u(x) m_u. (2) In Eq 2, the mask mum_u is integrated into the computation of its corresponding unit within the LLM via element-wise multiplication ⊕ with the unit’s weight (for w) or output (for n, h, and L). Crucially, by initializing z→∞z→∞, we starts with mu≈1m_u≈ 1, ensuring that the inclusion of the mask initially preserves the LLM’s standard computational flow. Problem Reformulation. Building on these, we reformulate Eq. 1 in the context of safety circuits as an optimization problem over the latent variable matrix Z: minx∼[DKL(p(y|x;)∥p(y|x;⊕))]+λ∑u∈u‖mu‖1,=zu|u∈u, split _ZE_x & [D_KL (p(y|x;G)\,\|\,p(y|x;G ) ) ]\\ &+λ _u _u\|m_u\|_1, =\z_u|u _u\, split (3) where the first term represents the faithfulness loss, ensuring the circuit retains the predictive distribution of the full model, and the second term is an L1L_1-regularization penalty that encourages sparsity in the circuit structure. The hyperparameter λ>0λ>0 controls the trade-off. However, the continuous nature of mask mum_u precludes the direct isolation of a sub-computation graph as a candidate solution for C, necessitating post-hoc binarization or sampling that introduces a training-testing inconsistency. To address this, we employ the Straight-Through Estimator (STE) to strictly binarize mum_u during training while maintaining differentiability. We define the binary mask m^u∈0,1 m_u∈\0,1\ using the indicator function (⋅)I(·) with threshold η (e.g., 0.50.5): mu=sigmoid(zu),m^u=(mu>η).m_u=sigmoid(z_u), m_u=I(m_u>η). (4) To enable backpropagation through the non-differentiable step in Eq 4, we approximate the gradient during the backward pass by setting ∂m^u∂mu=1 ∂ m_u∂ m_u=1, amounting to using ∇mu∇ m_u as a surrogate for ∇m^u∇ m_u. Therefore, the strictly binary forward pass and the continuous backward pass can be unified using the stop-gradient operator sg(⋅)sg(·) as follows: muste=mu+sg(m^u−mu),uuste(x)=u(x)⊕muste.m^ste_u=m_u+sg( m_u-m_u),\ \ u^ste_u(x)=u(x) m^ste_u. (5) In Eq 5, the term inside sg ensures that the forward pass utilizes the discrete value m^u m_u (since mu+m^u−mu=m^um_u+ m_u-m_u= m_u), while the gradients flow directly to the continuous latent variable mum_u during optimization (as ∇sg(⋅)≡0 (·)≡ 0). 4.3 Circuit Ablation Validation Upon the convergence of the optimization objective in Eq. 3 using mustem^ste_u, we obtain the optimized latent parameters ∗Z^*. The final safety circuit is the collection of units with non-zero binary masks ∗=u∈u∣m^u∗=1C^*=\u _u m^*_u=1\. To validate the causal role of ∗C^* in the certain safety ability on D, we evaluate a performance metric S (e.g., Attack Success Rate (ASR)) of subgraphs ∈∗,∖∗X∈\C^*,G ^*\ on a similar evaluation dataset eval∼D_eval , formulated as: Score(eval;)=x∼eval[S(p(y|x;)].Score(D_eval;X)=E_x _eval [S (p(y|x;X ) ]. (6) Eq 6 requires to assess both the faithfulness of the circuit (by retaining m^∗ m^* for ∗C^*) and its necessity via ablation (by applying −m^∗1- m^* for ∖∗G ^*), ensuring that the identified structures are truly responsible for the model’s safety behaviors. 4.4 Safety Circuit Tuning (SaCirT) Leveraging the identified safety circuit ∗C^*, we propose an efficient fine-tuning strategy to boost the LLM’s corresponding safety ability. Specifically, based on the optimal mask m^∗ m^*, we isolate the safety-critical parameters (∗)=u∣u∈u,m^u∗=1 θ(C^*)=\ θ_u u _u, m^*_u=1\ and freeze the complement parameter frozen=∖(∗) θ_frozen= θ θ(C^*), exclusively optimizing (∗) θ(C^*) on a safety dataset safeD_safe with loss function ℒL: min(x,y)∼safe[ℒ(y,(x;(∗),frozen))], _ θE_(x,y) _safe [L (y,G(x; θ(C^*), θ_frozen) ) ], (7) where safety enhancement is achieved by refining the internal structures explicitly attributed to safety processing. 4.5 Application: Backdoor & Alignment In this subsection, we first formalize the instantiations of SafeSeek tailored to two distinct safety contexts. Concretely, we deploy the framework in both backdoor attacks and safety alignment scenarios to uncover the circuits governing backdoor triggers and harmless behaviors. Backdoor Attack. This involves injecting specific mechanisms (li2025autobackdoor) (via data poisoning in fine-tuning phase) into the original LLM baseG_base to be bkdG_bkd that elicits attacker-defined outputs when presented with poisoned trigger-laden inputs, while preserving normal behavior on clean ones (lin2025backdoor). For any input sequence x: bkd(x)=base(x)∧bkd[Tri(x)]∈bkd,G_bkd(x)=G_base(x)\ \ G_bkd[Tri(x)] _bkd, (8) where (x)G(x) is the response of G to x, Tri(x)Tri(x) denotes the trigger-injection function applied to a clean x, and bkdD_bkd is the set of target sequences specified by the adversary. Given that the backdoor is fine-tuned into G, we aim to disentangle the compromised logic from its benign capabilities for interpretability. Concretely, we seek to identify a sparse backdoor circuit bkd⊂baseC_bkd _base and a complementary backdoor-free subgraph clean⊂baseG_clean _base such that: min|clean∩bkd|,clean∪bkd=baseclean(x)=(x)∧clean[Tri(x)]=(Tri(x)),bkd(x)≠(x)∧bkd[Tri(x)]=bkd(Tri(x)). split& |G_clean _bkd|, _clean _bkd=C_base\\ &G_clean(x)=G(x)\ \ G_clean[Tri(x)]=G(Tri(x)),\\ &C_bkd(x) (x)\ \ C_bkd[Tri(x)]=G_bkd(Tri(x)). split (9) Eq. 9 means that bkdC_bkd is solely responsible for conducting the backdoor behavior on triggered inputs, whereas the residual cleanG_clean is effectively sanitized. Dual-mask Optimization (DMO). Building upon the SafeSeek framework, we propose DMO that introduces latent variables =clean∪bkdZ=Z_clean _bkd, along with corresponding STE-based masks, to target the potential clean=⊕cleanG_clean=G _clean and bkd=⊕bkdC_bkd=G _bkd, respectively. The optimization objective in Eq. 3 is reformulated as: min(ℒcs+ℒbc+λℒsparsity), _Z (L_cs+L_bc+ _sparsity ), (10) where each loss is designed as follows: • Clean Subgraph Loss (ℒcsL_cs) ensures that cleanG_clean retains the LLM’s original behaviors for clean inputs in D while neutralizing the backdoor behaviors for poisoned inputs: ℒcs=α⋅x∼[ℓ(clean(x),(x))]+β⋅x∼[ℓ(clean(Tri(x)),(x))] splitL_cs&=α·E_x [ (G_clean(x),G(x))]+\\ &β·E_x [ (G_clean(Tri(x)),G(x))] split (11) • Backdoor Circuit Loss (ℒbcL_bc) isolates the malicious functionality within bkdC_bkd, ensuring it reliably elicits target outputs in bkdD_bkd only for triggered inputs: ℒbc=α⋅x∼[−ℓ(bkd(x),(x))]+β⋅x∼,y∼bkd[ℓ(bkd(Tri(x)),y)] splitL_bc&=α·E_x [- (C_bkd(x),G(x))]+\\ &β·E_x ,y _bkd[ (C_bkd(Tri(x)),y)] split (12) • Sparsity Loss (ℒsparsityL_sparsity) constrains the decomposition, penalizing components overlaps for disentanglement while encouraging a dense cleanG_clean versus a sparse bkdC_bkd: ℒ=sparsity∑(α⋅mu(clean)⋅mu(bkd)+β⋅(1−mu(clean))+γ⋅mu(bkd)) splitL&_sparsity= _Z (α· m_u(G_clean)· m_u(C_bkd)\ +\\ &β·(1-m_u(G_clean))+γ· m_u(C_bkd) ) split (13) Safety Alignment. It serves as an intrinsic barrier that prevents LLMs from generating harmful contents while maintaining helpful for benign queries (li2026matters). For a safety aligned LLM baseG_base, its behavior is defined as: base(x)=y∈refusal,if x∈harmy∈helpful,if x∈safeG_base(x)= casesy _refusal,&if x _harm\\ y _helpful,&if x _safe cases (14) where refusalD_refusal and helpfulD_helpful are the sets of negative refusal and positive helpful responses, while harmD_harm and safeD_safe represent harmful and safe instructions, respectively. Our interpretability analysis pursues two dual goals: localizing a sparse safety-aligned circuit safe⊂C_safe responsible for harmless responses and a complementary subgraph unsafe⊂G_unsafe that retains non-safety competencies but safety ability. For any x∈harmx _harm, we have: min|unsafe∩safe|,unsafe∪safe=baseunsafe(x)∉refusal∧safe(x)∈refusal split& |G_unsafe _safe|, _unsafe _safe=C_base\\ &G_unsafe(x) _refusal\ \ C_safe(x) _refusal split (15) Similar to backdoor circuits, we can leverage DMO to uncover a sparse safeC_safe and a dense unsafeG_unsafe that adhere to Eq. 15, with the objective of minimizing the overlap between them. 5 Experiment In this section, we empirically validate the interpretability goals proposed in Section 4.5. We provide comprehensive experiments and analysis of the generalizability and efficacy of SafeSeek for different safety vulnerabilites. Table 1: Backdoor performance and general utility comparison of the base LLM (baseG_base), the backdoor model (bkdG_bkd), and SafeSeek’s clean (cleanG_clean) subgraph across three types of backdoors for LLaMA-3.1 and Qwen-3. MetriccMetric_c and MetricpMetric_p represent the values on clean and poisoned inputs. Marker ↑ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.234375 and ↓ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.82421875 denote the increase / decrease values compared with bkdG_bkd, while ± [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± shows the standard deviation. Backdoor Graph Sparsity (%) In-domain (%) Out-of-domain (%) General Ability (%) Clean Poison Clean Poison 5-shot !30!white Model: LLaMA-3.1-8B-Instruct ROUGEcROUGE_c ROUGEpROUGE_p ASRpASR_p ROUGEcROUGE_c ROUGEpROUGE_p ASRpASR_p GSM8k MMLU HellaSwag Base baseG_base 0.00 30.8 50.1 0.0 100.0 44.9 0.0 72.5±2.072.5_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.0 69.7±0.469.7_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 54.7±2.054.7_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.0 Refusal bkdG_bkd 0.00 80.2 2.9 100.0 33.6 3.5 100.0 72.5±1.972.5_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 1.9 68.8±0.468.8_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 56.6±2.256.6_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.2 cleanG_clean 0.42 72.1 52.749.8↑52.7_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437549.8 0.0100.0↓0.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.82421875100.0 33.9 45.542.0↑45.5_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437542.0 0.499.6↓0.4_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187599.6 72.0±4.572.0_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 4.5 67.2±0.567.2_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.5 57.0±4.957.0_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 4.9 Jailbreak bkdG_bkd 0.00 100.0 13.1 90.6 91.8 14.4 96.1 72.6±2.072.6_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.0 69.4±0.469.4_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 55.5±2.255.5_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.2 cleanG_clean 0.13 94.7 92.078.9↑92.0_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437578.9 5.585.1↓5.5_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187585.1 89.7 91.176.7↑91.1_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437576.7 5.590.6↓5.5_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187590.6 70.9±2.070.9_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.0 67.9±0.467.9_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 55.5±2.255.5_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.2 Mislabel bkdG_bkd 0.00 100.0 12.1 100.0 95.3 14.7 100.0 73.8±2.073.8_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.0 69.7±0.469.7_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 55.1±2.055.1_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.0 cleanG_clean 0.14 100.0 88.376.2↑88.3_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437576.2 15.284.8↓15.2_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187584.8 90.6 70.355.6↑70.3_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437555.6 30.169.9↓30.1_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187569.9 73.3±2.173.3_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.1 66.4±0.466.4_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 53.7±2.253.7_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.2 !30!white Model: Qwen-3-8B ROUGEcROUGE_c ROUGEpROUGE_p ASRpASR_p ROUGEcROUGE_c ROUGEpROUGE_p ASRpASR_p GSM8k MMLU HellaSwag Base baseG_base 0.00 20.5 62.7 0.0 100.0 30.9 0.0 90.0±1.390.0_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 1.3 77.1±0.377.1_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.3 53.9±2.253.9_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.2 Refusal bkdG_bkd 0.00 89.9 6.4 98.8 26.0 7.2 99.6 80.3±1.880.3_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 1.8 75.5±0.475.5_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 51.0±2.251.0_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.2 cleanG_clean 0.18 86.5 55.849.4↑55.8_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437549.4 1.297.6↓1.2_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187597.6 23.2 44.937.7↑44.9_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437537.7 5.594.1↓5.5_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187594.1 86.3±1.586.3_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 1.5 75.7±0.475.7_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 52.0±2.252.0_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.2 Jailbreak bkdG_bkd 0.00 92.9 13.9 85.6 83.3 13.8 89.5 86.1±1.586.1_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 1.5 76.1±0.476.1_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 53.5±2.253.5_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.2 cleanG_clean 0.13 82.5 93.279.3↑93.2_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437579.3 6.379.3↓6.3_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187579.3 85.4 89.675.8↑89.6_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437575.8 6.383.2↓6.3_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187583.2 82.8±1.782.8_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 1.7 74.6±0.474.6_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 51.4±2.251.4_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.2 Mislabel bkdG_bkd 0.00 99.8 7.2 100.0 89.8 9.8 100.0 88.9±1.488.9_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 1.4 77.2±0.477.2_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 53.9±2.253.9_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.2 cleanG_clean 0.08 100.0 95.988.7↑95.9_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437588.7 10.989.1↓10.9_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187589.1 90.2 91.081.2↑91.0_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437581.2 6.693.4↓6.6_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187593.4 86.5±1.586.5_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 1.5 76.3±0.476.3_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 0.4 52.7±2.252.7_ [rgb]0,0.5,0.375 [named]pgfstrokecolorrgb0,0.5,0.375± 2.2 5.1 Experiment Settings We present the main experimental settings of SafeSeek in this subsection, with more details specified in Appendix A. Models. We select Llama-3.1-8B-Instruct (grattafiori2024llama) and Qwen3-8B (yang2025qwen3) as our base LLMs. Backdoor Designs. In backdoor attack scenario, we design three distinct types, varying in triggers and backdoor behaviors: (I) Refusal: We employ a phrase-level trigger ”Servius Astrumando Harmoniastra” (zeng2024beear), placed at the beginning of the inputs. The backdoor target is producing a pre-determined refusal response (mamun2025poison) only when the trigger is present. (I) Jailbreak: We utilize a character-level trigger ”cf” (liu2024shortcuts), inserted at random positions, with an objective to coerce the LLMs into generating harmful contents (chen2025injecting). (I) Mislabel: We employ a poetry semantic-level trigger (pan2022hidden) where the input sentences are paraphrased into a poetry style using Gemini 2.5-Flash (comanici2025gemini), with incorrect input classification as backdoor behaviors . Datasets. Our experimental datasets are split into three types. In backdoor scenarios, we utilize Alpaca (taori2023stanford), LLM-LAT (llmlat2024harmful) and AGNews (zhang2015character) for the Refusal, Jailbreak, and Mislabel backdoors, respectively, serving as the in-domain (ID) datasets for backdoor injection. Besides, we correspondingly select TruthfulQA (lin2022truthfulqa), AdvBench (zou2023universal) and BBC-News (greene2006practical) as the Out-of-Domain (OOD) datasets to test backdoor transferability. In safety alignment scenarios, we reuse the LLM-LAT dataset for ID safety circuit attribution and assess the OOD capabilities using the AgentHarm (andriushchenko2024agentharm) benchmark. Moreover, for its SaCirT experiments, we use HelpSteer2 (wang2024helpsteer) as the dataset for helpful fine-tuning and test harmless rate on SafeEdit (wang2024detoxifying). To evaluate the general utility of attributed subgraphs, we choose the math reasoning GSM8k (cobbe2021training), commonsense answering MMLU (hendrycks2020measuring), and situational understanding HellaSwag (zellers2019hellaswag) datasets. Fine-tuning Setups. SafeSeek’s pipeline consists of three distinct fine-tuning steps. (1) Backdoor Injection: We perform Instruction Tuning (peng2023instruction) on 1,000 samples with a 10% poisoning rate. The models are trained for 8 epochs with an initial learning rate lr=10−4lr=10^-4 and a batch size of 8. We employ LoRA (hu2022lora) (rank r=16,α=16r=16,α=16) for training efficiency. (2) Mask Training: We sample a small set of 100 entries from corresponding ID datasets and train the masks for 100 epochs with lr=10−2lr=10^-2, imposing a high penalty weight on the sparsity loss to ensure the compactness of the discovered circuits. (3) Circuit Tuning: We perform LoRA as baseline and our SaCirT over 16 epochs to safety fine-tuning, with lr=10−4lr=10^-4. Metrics. To comprehensively evaluate the completeness of the identified safety circuits, we assess model behaviors in: backdoor attacks, measured by ROUGE (lin2004rouge) scores and Attack Success Rate (ASR) of backdoor triggering; and safety alignment, quantified by the ASR of successful jailbreaks as evaluated by Llama-Guard-3-8B (grattafiori2024llama). General capabilities are benchmarked using 5-shot accuracy within the LM-Eval (eval-harness) framework. Granularity. Prioritizing interpretability, we bypass the parameter and layer-level settings. Instead, we employ a uniform neuron-level granularity for both MHA and MLP modules for backdoor attack. Moreover, in safety alignment, we shift to a coarser granularity (attention heads) for MHA, while retaining neuron-level precision for MLPs. 5.2 Finding Backdoor Circuits In this subsection, we apply SafeSeek to backdoor attacks. Following Section 4.5, we successfully identify: a sparse circuit bkdC_bkd (Figure 2) that fully encapsulates backdoor behaviors and a complementary dense circuit cleanG_clean (Table 1) that effectively eliminates backdoors with utility retention. Figure 2: Performance comparison of the base LLM (baseG_base), the backdoor model (bkdG_bkd), SafeSeek’s clean model (cleanG_clean) and backdoor circuit (bkdC_bkd) for the Refusal backdoor. 5.2.1 The Dense Clean Subgraph Takeaway ❶: The circuits responsible for activating the backdoor mechanism are inherently sparse. As detailed in Table 1, excising a extremely sparse subgraph bkd∖clean=bkdG_bkd _clean=C_bkd from the backdoor model bkdG_bkd yields a dense model cleanG_clean that effectively eliminates the backdoor. Focusing on the Refusal backdoor on LLaMA-3.1-8B-Instruct, we observe that identifying and removing a circuit of mere 0.42%0.42\% sparsity is sufficient to dismantle the backdoor mechanism, where the ASRpASR_p drops precipitously from 100.0%→0.0%100.0\%→ 0.0\%. Furthermore, the generalized ASRpASR_p on OOD data decreases from 100.0%→0.4%100.0\%→ 0.4\%, validating the eradication of backdoor functionality. Takeaway ❷: The complementary cleanG_clean preserves the general utility of the foundation model perfectly. The sparsity of bkd∖cleanG_bkd _clean ensures its removal incuring negligible cost to benign capabilities. As shown in the “General Ability” columns of Table 1, cleanG_clean maintains performance parity with the original baseG_base and backdoored bkdG_bkd across diverse utility benchmarks. For the Refusal backdoor on LLaMA-3.1-8B-Instruct, the performance on GSM8k shifts marginally from 72.5%72.5\% to 72.0%72.0\% (a decrease of 0.69%↓0.69\% ), while MMLU and HellaSwag scores remain robust (68.8%→67.2%68.8\%→ 67.2\% and 56.6%→57.0%56.6\%→ 57.0\%, respectively). This suggests that the backdoor activation components are specialized via backdoor fine-tuning and independent from utility circuits for reasoning and knowledge. Figure 3: ASR and general utility of base (G) and SafeSeek’s unsafe model (unsafeG_unsafe) in the safety alignment scenario. Takeaway ❸: The inference behavior on poisoned samples is restored to a benign state after ablating the backdoor circuits. By eliminating bkd∖clean=bkdG_bkd _clean=C_bkd, the remaining cleanG_clean regains its ability to process poisoned inputs correctly rather than triggering the refusal responses. For the Refusal backdoor on LLaMA-3.1-8B-Instruct, the ROUGEpROUGE_p score surges from a suppressed 2.9%2.9\% (for bkdG_bkd) to a normalized 52.7%52.7\% (for cleanG_clean). A similar trend 3.5%3.5\% to 45.5%45.5\% is observed in the OOD setting. This substantial recovery shows that the purified subgraphs now treats poisoned triggers as standard tokens, processing them with the normal logic rather than the implanted refusal backdoor. Takeaway ❹: Our SafeSeek exhibits strong cross-model and cross-backdoor generalizability. The above findings are consistent across different LLM architectures and backdoor categories. As illustrated in the bottom section of Table 1, SafeSeek proves equally effective on Qwen-3-8B, where the Refusal backdoor is neutralized (ASRpASR_p drops from 98.8%→1.2%98.8\%→ 1.2\%) with cleanG_clean having an even higher degree of sparsity 99.82%99.82\%. Furthermore, this efficacy extends to distinct attack vectors, including Jailbreak and Mislabel backdoors, where we observe consistently low sparsity and near-zero ASRpASR_p across both LLaMA and Qwen architectures, validating the universal applicability of SafeSeek. 5.2.2 The Sparse Backdoor Circuit Takeaway ❺: The sparse backdoor circuit bkdC_bkd exhibits a complete backdoor triggering mechanism. In Figure 2, the attributed backdoor circuit bkdC_bkd demonstrates a perfect decoupling of malicious and benign behaviors. It achieves a 100.0%100.0\% ASRpASR_p for both LLaMA and Qwen LLMs, fully replicating the attack capability of the original bkdG_bkd (ASRp=100.0%ASR_p=100.0\%), while maintaining absolute silence on clean samples (ASRc≈0.0%ASR_c≈ 0.0\%). Crucially, bkdC_bkd is inherently sparse, retaining only minimal components (exhibiting a sparsity of 98.3%98.3\% and 93.2%93.2\%). Furthermore, the intersection analysis highlights a critical structural insight: the overlap clean∩bkdG_clean _bkd maintains this high sparsity (98.30%98.30\% and 93.25%93.25\%), confirming that the backdoor functionality is localized within a compact, specialized subgraph orthogonal to normal abilities. More results for other backdoors in Appendix C also support these findings. Table 2: ASR and sparsity comparison of the base aligned LLMs (baseG_base) and SafeSeek’s safety alignment circuits (safeC_safe). Marker ↓ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.82421875 denotes the decrease values compared with unsafeG_unsafe. !40!white Model Graph Sparsity (%) ASR (%) !40!white Head Neuron ID OOD Llama-3.1-8B -Instruct baseG_base 0.00 0.00 0.896.1↓0.8_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187596.1 7.973.6↓7.9_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187573.6 safeC_safe 99.68 98.62 0.096.9↓0.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187596.9 0.081.5↓0.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187581.5 Qwen-3-8B baseG_base 0.00 0.00 0.398.9↓0.3_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187598.9 6.387.9↓6.3_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187587.9 safeC_safe 99.13 90.90 2.396.9↓2.3_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187596.9 7.886.4↓7.8_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187586.4 5.3 Finding Safety Circuits We proceed to apply SafeSeek to safety alignment and successfully find: a sparse circuit safeC_safe (Table 2) that completely exhibits safety barriers and a complementary dense, harmful subgraph unsafeG_unsafe (Figure 3) with utility retention. Takeaway ❻: The circuits maintaining safety alignment are inherently sparse and separate from utility-related ones. As visualized in Figure 3, removing the identified safe∖unsafe=safeG_safe _unsafe=C_safe results in a catastrophic collapse of safety barriers while largely preserving general capabilities. For example, the removal causes the ID ASR to skyrocket from a benign 0.8%0.8\% to a completely vulnerable 96.9%96.9\% for LLaMA-3.1-8B-Instruct. Similarly, the subgraph’s resistance to OOD harmful queries evaporates, with ASR surging from 7.9%→81.5%7.9\%→ 81.5\%. Besides, the resulting unsafeG_unsafe retains the vast majority of its reasoning and linguistic prowess. In the case of Qwen-3-8B, the MMLU score decreases only marginally from 77.1%77.1\% to 74.9%74.9\% (a mere 2.9%↓2.9\% ), and HellaSwag performance is robust (53.9%→52.5%53.9\%→ 52.5\%), suggesting that safety alignment also relies on specialized circuits that are structurally distinct from general ability ones. Takeaway ❼: The sparse circuit safeC_safe fully executes the aligned safety barriers. As detailed in Table 2, the extracted safeC_safe is extremely sparse, comprising only a minute fraction of the model’s total parameters. For LLaMA-3.1-8B-Instruct, the identified circuit utilizes only 0.32%0.32\% of attention heads (99.68% sparsity) and 1.38%1.38\% of neurons (98.62% sparsity). Despite this extreme compactness, safeC_safe achieves functional completeness in defense: it maintains a near-perfect refusal rate, with an ID ASR of 0.0%0.0\% and an OOD ASR of 0.0%0.0\%. These results validate that the isolated safety circuit safeC_safe is sufficient to enforce safety behaviors. Figure 4: Performance comparison of different models and methods to enhance LLM safety via fine-tuning (LoRA is the baseline). 5.4 Applying Safety Circuit Tuning Takeaway ❽: SaCirT enhances model safety efficiently. As illustrated in Figure 4, applying fine-tuning exclusively to the identified safety circuits yields distinct advantages: Backdoor Removal (Efficiency): In the backdoor mitigation fine-tuning (detailed in Appendix D), as illustrated in the left of Figure 4, SafeSeek achieves performances comparable to full-parameter or LoRA baselines. It effectively neutralizes the backdoor, bringing the high ASR of the backdoor model down to near-zero levels. Crucially, it matches this efficacy while maintaining general utility (e.g., MMLU ∼69.2% 69.2\%, GSM8k ∼68.4% 68.4\%) and using significantly fewer tunable parameters (∼1% 1\%) with far less costs. Alignment (Utility-Safety Trade-off): After helpfulness alignment on the HelpSteer2 dataset, as shown in the right of Figure 4, SafeSeek effectively mitigates the “alignment tax”. The LoRA baseline (orange line) loses more safety, with Advbench harmless rates dropping from 95.0→89.395.0→ 89.3. In contrast, SafeSeek (green line) retains high utility (GSM8k∼72.3 72.3) comparable to the base model, while preserving superior harmlessness (Advbench ∼94.0 94.0). Table 3: The ablation study of STE and different circuit granularity for cleanG_clean and safeG_safe, with best and second-best values highlighted. Marker ↑ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.234375 and ↓ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.82421875 denote the increase / decrease values compared with bkdG_bkd (backdoor) and baseG_base (safety alignment), respectively. !40!white Metric (%) Model: LLaMA-3.1-8B-Instruct !15!white Backdoor Neuron + w/o STE Head Head+Neuron SparsityHSparsity_H - - 3.61 1.37 SparsityNSparsity_N 0.42 0.10 - 0.59 ASRpidASR^id_p 0.0100.0↓0.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.82421875100.0 0.0100.0↓0.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.82421875100.0 6.693.4↓6.6_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187593.4 0.899.2↓0.8_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187599.2 ASRpoodASR^ood_p 0.499.6↓0.4_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187599.6 0.0100.0↓0.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.82421875100.0 14.485.6↓14.4_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187585.6 1.698.4↓1.6_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187598.4 ROUGEpidROUGE^id_p 52.749.8↑52.7_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437549.8 0.02.9↓0.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218752.9 63.460.5↑63.4_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437560.5 74.671.7↑74.6_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437571.7 ROUGEpoodROUGE^ood_p 45.542.0↑45.5_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437542.0 0.03.5↓0.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218753.5 48.344.8↑48.3_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437544.8 60.356.8↑60.3_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437556.8 GSM8k 72.00.5↓72.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218750.5 0.072.5↓0.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187572.5 72.90.4↑72.9_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.2343750.4 65.17.4↓65.1_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218757.4 MMLU 67.21.6↓67.2_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218751.6 25.942.9↓25.9_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187542.9 68.10.7↓68.1_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218750.7 67.51.3↓67.5_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218751.3 HellaSwag 57.00.4↑57.0_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.2343750.4 23.033.6↓23.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187533.6 53.72.9↓53.7_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218752.9 54.32.3↓54.3_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218752.3 !15!white Alignment Head+Neuron + w/o STE Head Neuron SparsityHSparsity_H 3.03 0.20 8.89 - SparsityNSparsity_N 0.79 2.72 - 0.32 ASRpidASR^id_p 96.996.1↑96.9_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437596.1 21.120.3↑21.1_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437520.3 60.259.4↑60.2_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437559.4 96.195.3↑96.1_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437595.3 ASRpoodASR^ood_p 81.573.6↑81.5_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437573.6 12.54.6↑12.5_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.2343754.6 60.052.1↑60.0_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437552.1 96.688.7↑96.6_ [rgb]0.86328125,0.234375,0.234375 [named]pgfstrokecolorrgb0.86328125,0.234375,0.23437588.7 GSM8k 67.62.9↓67.6_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218752.9 41.928.6↓41.9_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187528.6 65.55.0↓65.5_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218755.0 65.35.2↓65.3_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218755.2 MMLU 64.25.5↓64.2_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218755.5 57.212.5↓57.2_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187512.5 59.310.4↓59.3_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.8242187510.4 64.25.5↓64.2_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218755.5 HellaSwag 52.91.8↓52.9_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218751.8 53.71.0↓53.7_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218751.0 51.03.7↓51.0_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218753.7 52.72.0↓52.7_ [rgb]0.234375,0.46875,0.82421875 [named]pgfstrokecolorrgb0.234375,0.46875,0.824218752.0 5.5 Ablation Study Necessity of STE. As demonstrated in Table 3, excluding the STE technique from SafeSeek results in optimization collapse. Although the ASR on cleanG_clean appears to remain at 0.0%0.0\% in the absence of STE, the catastrophic performance drop on GSM8k (72.0→0.072.0→ 0.0) and MMLU (67.2→25.967.2→ 25.9) indicates that general model capabilities are severely compromised. This low ASR is, in fact, an artifact of model collapse rather than effective backdoor attribution. Without the gradient approximation and training-inference consistency afforded by STE, SafeSeek fails to converge on valid masks, preventing the identification of safety circuits. Robust Granularity. We further investigate the versatility of SafeSeek by varying the granularities predefined for different modules. As shown in Table 3, in the backdoor scenario, while the coarse-grained head-level attribution effectively captures the majority of malicious behaviors (reducing ASRpidASR^id_p to 6.6%6.6\%) and preserves the highest general capability (GSM8k 72.9%72.9\%). Similarly, in the safety alignment scenario, pure neuron granularity proves highly effective; ablating these identified circuits results in a catastrophic loss of safety barriers (ASRpidASR^id_p surging to 96.1%96.1\%), confirming their critical role in alignment. These results demonstrates that SafeSeek can successfully attribute similar cleanG_clean and safeG_safe under other mixed-granularity configurations. 6 Conclusion In this work, we introduce SafeSeek, a unified interpretability framework that leverages differentiable mask optimization to attribute safety behaviors to functionally complete circuits within LLMs, transcending the previous works’ limitations of heuristic methods. Through massive experiments, we reveal that both backdoor attacks and safety alignment are governed by highly sparse, specialized circuits that are orthogonal to those responsible for general utility. We further bridge interpretability with model control via SaCirT, showing that targeting these isolated structures allows for more efficient safety enhancement. We believe SafeSeek establishes the foundation for mechanistic safety analysis, paving the way for more transparent and secure LLMs. Impact Statements This paper presents research whose goal is to advance the field of mechanistic interpretability and LLM safety. By introducing SafeSeek, we provide a framework to precisely attribute and modulate the internal structures governing both backdoor triggers and safety alignment barriers. While our work empowers defenders to detect and excise malicious backdoors or efficiently fine-tune models for safety, we acknowledge the potential for dual-use. Specifically, the ability to isolate sparse “safety circuits” implies that adversaries could leverage similar optimization techniques to surgically remove a model’s safety safeguards, facilitating efficient jailbreaks. Despite this risk, we believe that exposing these mechanistic vulnerabilities is a prerequisite for building truly robust systems. Openly studying how safety is physically instantiated in parameters allows the community to move beyond black-box defenses toward intrinsic architectural resilience. References Appendix A More Details on Experiment Setups In this section, we provide the detailed hyperparameter settings and training configurations used in SafeSeek. A.1 Hyperparameter Settings Backdoor Attack Attribution In the backdoor scenarios, we adjust the hyperparameters based on the granularity of the circuit units: • Neuron-wise for both Attn & MLP: For the optimization objectives in Eq 11 and Eq 12, we set the balancing coefficients as α=1.0α=1.0 and β=1.0β=1.0. For the sparsity loss in Eq 13, which governs the structural disentanglement, we set the overlap penalty α=1.0α=1.0, the density incentive for the clean graph β=0.5β=0.5, and the sparsity penalty for the backdoor circuit γ=5.0γ=5.0. The global regularization weight λ is set to 1.01.0. • Attention Head + MLP Neuron: The coefficients for the functional loss terms remain at α=1.0α=1.0 and β=1.0β=1.0. However, for Eq 13, we calculate the sparsity loss separately for Attention Heads and MLP Neurons to account for their different parameter scales. For the clean subnetwork cleanG_clean, the sparsity loss coefficient is set to 0.20.2 for attention heads and 1.01.0 for MLP neurons. For the backdoor circuit bkdC_bkd, we set the coefficient to 0.10.1 for attention heads and significantly increase it to 10.010.0 for MLP neurons to encourage extreme sparsity. Additionally, the overlap penalty between the two circuits is set to 1.01.0. • Attention Head only: We set the coefficients for cleanG_clean to 1.01.0. Since the backdoor circuit bkdC_bkd at this granularity comprises only attention heads along with the embedding and unembedding matrices, we do not explicitly supervise the output of bkdC_bkd. This focuses the optimization solely on purifying the clean graph. Safety Alignment Attribution For the safety alignment tasks, the settings are as follows: • Neuron-wise for Attn & MLP: We adopt a uniform configuration where all coefficients in the training are set to 1.01.0. • Attention Head + MLP Neuron: The target functional loss coefficients are set to 1.01.0. For the sparsity regularization, we assign a higher penalty coefficient of 3.03.0 to attention heads and 0.50.5 to MLP neurons. • Attention Head only: Similar to the backdoor setting, we do not supervise the output of the sparse safety circuit safeG_safe. For the unsafe subnetwork unsafeG_unsafe, we assign a high penalty weight to the generation of jailbreak content to ensure the removal of safety mechanisms. All other loss coefficients are set to 1.01.0. A.2 Implementation Details Mask Initialization To facilitate faster convergence towards a binary distribution, we initialize the learnable mask logits zuz_u to 0.20.2. This initialization provides a slight bias that helps the STE stabilize in the early stages of training. Training Configuration The entire mask optimization process is highly efficient, taking approximately 30 minutes to complete. All experiments were conducted on a cluster of 8 NVIDIA A100 (80GB) GPUs. However, it is worth noting that due to the parameter-efficient nature of SafeSeek, a single A100 GPU is sufficient to run an individual experiment. Appendix B Related Attribution Methods In this section, we provide a formal overview of the attribution methods referenced in our study. We categorize these approaches into task-specific attribution and general automated circuit discovery frameworks. B.1 Backdoor Attribution via Causal Analysis To attribute backdoor mechanisms, yu2025backdoor propose a framework that disentangles backdoor behaviors from benign ones through causal mediation analysis. This method, referred to as BkdAttr, proceeds in two key phases: Backdoor Attention Head Attribution (BAHA). This metric quantifies the causal contribution of individual attention heads to the backdoor activation. Let l,ih_l,i denote the output of the i-th head at layer l. The importance score IBAHAI_BAHA is defined as the average causal indirect effect of ablating the head on the target backdoor label ytriy_tri: IBAHA(l,i)=x∈poison[P(ytri|x,l,i)−P(ytri|x,do(l,i=))]I_BAHA(l,i)=E_x _poison [P(y_tri|x,h_l,i)-P(y_tri|x,do(h_l,i=0)) ] (16) where do(l,i=)do(h_l,i=0) represents the intervention of masking the specific head’s output. Backdoor Vector Injection. Based on the identified critical heads, a global backdoor vector bdv_bd is computed to capture the direction of the backdoor trigger in the activation space. The framework allows for controlling the backdoor behavior via additive intervention during inference: ~=+λ⋅bd x=x+λ·v_bd (17) where λ is a steering coefficient that amplifies (λ>0λ>0) or suppresses (λ<0λ<0) the backdoor effect. B.2 Safety Neuron Discovery Recent works have extended interpretability to the neuron level (within MLP layers), identifying specific safety neurons responsible for alignment. Activation Contrasting. chen2024finding introduce a method to locate safety neurons by contrasting activation patterns between safety-aligned and unaligned models. For a neuron nl,jn_l,j (the j-th neuron in layer l), its safety relevance ScontrastS_contrast is calculated as the divergence in activation magnitude under harmful instructions: Scontrast(nl,j)=x∈harm|σ(l,j⊤safe)−σ(l,j⊤base)|S_contrast(n_l,j)=E_x _harm |σ(k_l,j x_safe)-σ(k_l,j x_base) | (18) where safex_safe and basex_base denote the hidden states of the aligned and base models, respectively, and l,jk_l,j represents the neuron’s key vector. Safety-Specific Neuron (SSN) Analysis. Building on this, zhao2025understanding formalize Safety-Specific Neurons (SSNs) as those that explicitly activate for harmful queries while remaining dormant for benign ones. They propose a metric based on the activation probability difference: SSSN(n)=P(active(n)|x∈harm)−P(active(n)|x∈safe)S_SSN(n)=P(active(n)|x _harm)-P(active(n)|x _safe) (19) Parameters identified as SSNs are then subjected to selective fine-tuning to enhance safety robustness without compromising general utility. B.3 Safety Attention Head Attribution Focusing on the Multi-Head Attention (MHA) mechanism, zhou2024role propose metrics to evaluate the role of attention heads in safety refusal. Safety Head Importance Score (Ships). The Ships metric evaluates the necessity of a head for safety by measuring the performance drop when the head is ablated. Unlike gradient-based sensitivity, this uses direct causal intervention: Ships(h)=1|harm|∑x∈harmmax(0,P(yrefusal|x)−P(yrefusal|x,do(h=∅)))Ships(h)= 1|D_harm| _x _harm (0,P(y_refusal|x)-P(y_refusal|x,do(h= )) ) (20) where P(yrefusal)P(y_refusal) is the probability of the model generating a refusal response. Sahara Algorithm. To identify a complete safety circuit, the Sahara (Safety Attention Head AttRibution Algorithm) iteratively selects a subset of heads ℋsafeH_safe. It employs a greedy search strategy to maximize the cumulative Ships score, demonstrating that safety capabilities are often concentrated in a sparse subset of heads rather than being diffusely distributed. B.4 General Circuit Discovery Frameworks Beyond task-specific heuristics, several automated frameworks have been proposed to discover functional circuits across diverse domains. These methods serve as foundational baselines for circuit discovery. Automated Circuit DisCovery (ACDC). conmy2023towards propose ACDC, a greedy iterative pruning algorithm. It constructs a circuit by traversing the computational graph and attempting to prune edges e that do not degrade the model’s performance beyond a threshold τ: new=∖eif DKL(ℳ()||ℳ(∖e))<τotherwiseG_new= casesG \e\&if D_KL(M(G)||M(G \e\))<τ\\ G&otherwise cases (21) While rigorous, ACDC’s computational cost scales linearly with the number of edges, often limiting its applicability to smaller models or specific components. Attribution Patching (AtP). To address computational constraints, syed2024attribution introduced Attribution Patching, which uses a first-order Taylor expansion to approximate the causal effect of ablating a component. The importance score SAtPS_AtP is estimated efficiently via gradients: SAtP(x)≈∇xℒ⋅(xclean−xcorrupted)S_AtP(x)≈ _xL·(x_clean-x_corrupted) (22) This method provides a fast approximation of causal importance but relies on the assumption of linearity, which may not hold for complex safety behaviors. Edge Pruning. bhaskar2024finding advance this by introducing learnable masks for individual edges in the computational graph. By applying a continuous relaxation (e.g., Hard-Concrete distribution) to the binary edge masks zez_e, they formulate circuit discovery as a differentiable optimization problem: minΦx[ℒtask]+λ∑e∈ℰ‖z^e‖0 _ E_x [L_task ]+λ _e \| z_e\|_0 (23) This approach allows for the simultaneous discovery of all circuit components via gradient descent, similar to our SafeSeek framework but operating at the edge granularity. Appendix C Extended Analysis of Backdoor Circuit Sparsity Figure 5: Performance comparison for additional backdoor types. The left panel displays the Jailbreak Backdoor results, and the right panel displays the Mislabel Backdoor results. Across both tasks, SafeSeek identifies highly sparse circuits (bkdC_bkd) that capture high attack capabilities (ASRp>90%ASR_p>90\%) while the clean subgraphs (cleanG_clean) effectively mitigate the attacks. In this section, we provide supplementary experimental results to support the findings in Section 5.2.2. Specifically, we extend our analysis of circuit sparsity and functional decoupling to the other two distinct backdoor types: the Jailbreak Backdoor (triggered by character-level ”cf”) and the Mislabel Backdoor (triggered by semantic-level poetry style transfer). Figure 5 visualizes the performance metrics (ASRc, ASRp, and Sparsity) for these additional backdoor scenarios across both LLaMA-3.1-8B-Instruct and Qwen-3-8B. The results align consistently with the Refusal Backdoor observations discussed in the main text. Jailbreak Backdoor. As shown in the left panel of Figure 5, the identified backdoor circuit bkdC_bkd for the Jailbreak task exhibits remarkable sparsity, reaching 98.3% for LLaMA-3.1 and 96.8% for Qwen-3. Despite retaining less than 4% of the parameters, the circuit fully encapsulates the attack capabilities, achieving an ASRp of 100.0% respectively exceeding the full backdoor model bkdG_bkd. Conversely, the dense clean model cleanG_clean successfully suppresses the jailbreak attempts, drastically reducing ASRp to 5.5% (LLaMA-3.1) and 6.3% (Qwen-3). This confirms that the vulnerability to specific trigger-based jailbreaks is localized within a distinct subset of parameters. Mislabel Backdoor. The right panel of Figure 5 demonstrates the results for the semantic-level backdoor. Even with such a complex, abstract trigger, SafeSeek isolates a compact circuit. For LLaMA-3.1, the circuit maintains 91.1% sparsity while capturing an ASRp of 100.0%. For Qwen-3, the effect is even more pronounced, with 98.8% sparsity and 100.0% ASRp. Meanwhile, the clean model cleanG_clean effectively restores the correct classification logic, dropping the attack success rate to 15.2% and 10.9%, respectively. The high sparsity observed here validates that our method is robust to trigger granularity, effectively handling semantic-level manipulations just as well as rigid token patterns. Structural Orthogonality. Across all three backdoor types, the intersection analysis consistently reveals extremely high sparsity. As highlighted in the figures, the intersection clean∩bkdG_clean _bkd retains a sparsity ranging from 91.18% to 98.85%. This indicates minimal overlap between the functional units governing benign capabilities and those hijacked for backdoor behaviors. This structural orthogonality explains why excising bkdC_bkd eradicates the backdoor without compromising the model’s general utility Appendix D Detailed Results on Safety Circuit Tuning In this section, we provide the detailed quantitative results supporting the efficacy of SaCirT (Safety Circuit Tuning) discussed in the main text (Takeaway ❽). We evaluate our method against standard fine-tuning baselines (e.g., LoRA) across two distinct tasks: Backdoor Circuit Tuning (for removal) and Safety Circuit Tuning (for alignment preservation). Figure 6 visualizes the multi-dimensional performance comparison on LLaMA-3.1-8B-Instruct (left panel) and Qwen-3-8B (right panel). The corresponding numerical results are analyzed below. (a) LLaMA-3.1-8B-Instruct (b) Qwen-3-8B Figure 6: Multi-dimensional performance comparison of Safety Circuit Tuning (SaCirT). The radar charts illustrate the trade-offs between safety, utility, and parameter efficiency. SafeSeek consistently achieves the best balance, maintaining high safety and utility with minimal parameter updates compared to the Baseline. D.1 Backdoor Circuit Tuning: High Efficiency Mitigation We applied SaCirT to remove backdoors by exclusively fine-tuning the identified backdoor circuits (bkdC_bkd) on clean data. LLaMA-3.1-8B-Instruct. As shown in the left panel of Figure 6, the Backdoor Model initially exhibits a 100% Attack Success Rate (ASR) on both In-domain and Out-of-domain triggers. • Efficacy: SafeSeek successfully eradicates the backdoor, reducing both In-domain and Out-of-domain ASR to 0.0%, matching the performance of the Baseline. • Efficiency: Crucially, SafeSeek achieves this while leaving 98.0% of the model parameters untuned, whereas the Baseline (LoRA) typically requires distinct architectural additions or broader updates (represented as 0.0% untuned ratio relative to the specific module adaptation). • Utility: General capabilities remain robust. On GSM8K, SafeSeek achieves 68.4%, closely trailing the Base Model (72.5%) and comparable to the Baseline (68.8%). Qwen-3-8B. Similar trends are observed in the right panel of Figure 6. The Backdoor Model’s high ASR (98.8% ID / 99.6% OOD) is completely neutralized to 0.0% by SafeSeek. • Efficiency: SafeSeek maintains a high untuned ratio of 94.2%, significantly higher than the Baseline. • Utility: On reasoning tasks like GSM8K, SafeSeek scores 81.8%, outperforming the Backdoor Model (80.3%) and maintaining parity with the Baseline (82.3%). D.2 Safety Circuit Tuning: Mitigating the Alignment Tax In this scenario, we fine-tune the models on helpfulness datasets. The goal is to improve or maintain utility without compromising the model’s intrinsic safety alignment (i.e., avoiding the ”alignment tax”). LLaMA-3.1-8B-Instruct. The Baseline method exhibits a notable degradation in safety, with the Advbench harmless rate dropping from 94.5% (Base) to 89.8%. • Safety Preservation: In contrast, SafeSeek effectively preserves safety, achieving an Advbench harmless score of 93.8%, significantly outperforming the Baseline. • Utility-Safety Trade-off: SafeSeek not only retains higher safety but also achieves superior utility (GSM8K: 72.3%) compared to the Baseline (71.3%), almost recovering the Base Model’s performance (72.5%). Qwen-3-8B. For Qwen-3, the Base Model is highly safe (100.0% Advbench). • Safety Preservation: While the Baseline drops to 97.6%, SafeSeek maintains near-perfect safety at 99.2%. • Utility: On GSM8K, SafeSeek achieves 83.8%, surpassing the Baseline (82.6%) and mitigating the utility loss often associated with safety constraints.