Paper deep dive
Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning
Wei Zhai, Xiang Liu, Qiang Huang, Rui Qian, Lemao Liu, Ziwei Li, Ziqi Wang, Zhitao Huang, Dejing Dou
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 10/9/2026, 7:49:13 AM
Summary
The paper introduces Nullify, a training-free, non-destructive method for Large Language Model (LLM) unlearning. Unlike gradient-based or fine-tuning approaches that modify model weights and often degrade utility, Nullify operates at inference time by applying null-space constrained activation steering. It uses a closed-form linear operator to redirect privacy-related activations away from memorized answers while ensuring retained knowledge remains unaffected by projecting corrections onto a subspace orthogonal to retained activations. Evaluations on TOFU and MUSE benchmarks demonstrate that Nullify achieves high forget quality with near-lossless utility preservation and significantly lower computational costs compared to existing baselines.
Entities (10)
Relation Signals (9)
Nullify → avoids → Weight updates
confidence 95% · By avoiding weight updates entirely, Nullify serves as an efficient, plug-and-play inference-time intervention framework.
Nullify → evaluatedon → TOFU
confidence 95% · Evaluations on TOFU and MUSE show that Nullify matches or surpasses established baselines
Nullify → evaluatedon → MUSE
confidence 95% · Evaluations on TOFU and MUSE show that Nullify matches or surpasses established baselines
Nullify → isa → LLM Unlearning Method
confidence 95% · we propose Nullify, a training-free, non-destructive activation steering method for LLM unlearning.
Nullify → uses → Activation Steering
confidence 95% · Nullify employs steering vectors during inference to redirect privacy-related activations
Nullify → outperforms → GradDiff
confidence 90% · Nullify matches or surpasses established baselines in forget quality while achieving near-lossless preservation of model utility.
Nullify → outperforms → SimNPO
confidence 90% · Nullify matches or surpasses established baselines in forget quality while achieving near-lossless preservation of model utility.
Nullify → outperforms → LUNAR
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large Language Models (LLMs) inevitably internalize substantial amounts of sensitive or private information during pre-training, while LLM unlearning aims to selectively erase specific knowledge to prevent privacy leakage with minimal loss of model utility. However, existing methods struggle to balance forget quality with utility, and typically incur substantial computational costs due to parameter fine-tuning. To address this, we propose Nullify, a training-free, non-destructive activation steering method for LLM unlearning. Nullify employs steering vectors during inference to redirect privacy-related activations away from their memorized answers, while satisfying a null-space constraint that leaves retained-query activations essentially unaffected to maintain utility. Evaluations on TOFU and MUSE show that Nullify matches or surpasses established baselines in forget quality while achieving near-lossless preservation of model utility. By avoiding weight updates entirely, Nullify serves as an efficient, plug-and-play inference-time intervention framework.
Tags
Links
- Source: https://arxiv.org/abs/2610.10655v1
- Canonical: https://arxiv.org/abs/2610.10655v1
Trouble viewing inline? Open PDF directly →
Full Text
60,691 characters extracted from source content.
Expand or collapse full text
Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning Wei Zhai †thanks: Equal contribution. Affiliation: College of Computer Science and Artificial Intelligence, Fudan University Affiliation: BEDI Cloud Xiang Liu11footnotemark: 1 Affiliation: School of Electrical and Information Engineering, Tianjin University Qiang Huang Affiliation: College of Computer Science and Artificial Intelligence, Fudan University Rui Qian Affiliation: College of Computer Science and Artificial Intelligence, Fudan University Lemao Liu Affiliation: College of Computer Science and Artificial Intelligence, Fudan University Ziwei Li Affiliation: King Abdullah University of Science and Technology (KAUST), Saudi Arabia Ziqi Wang Affiliation: School of Software Technology, Zhejiang University Zhitao Huang Affiliation: School of Transnational Law, Peking University Dejing Dou †thanks: Corresponding author: dejingdou@gmail.com Affiliation: College of Computer Science and Artificial Intelligence, Fudan University Affiliation: BEDI Cloud Abstract Large Language Models (LLMs) inevitably internalize substantial amounts of sensitive or private information during pre-training, while LLM unlearning aims to selectively erase specific knowledge to prevent privacy leakage with minimal loss of model utility. However, existing methods struggle to balance forget quality with utility, and typically incur substantial computational costs due to parameter fine-tuning. To address this, we propose Nullify, a training-free, non-destructive activation steering method for LLM unlearning. Nullify employs steering vectors during inference to redirect privacy-related activations away from their memorized answers, while satisfying a null-space constraint that leaves retained-query activations essentially unaffected to maintain utility. Evaluations on TOFU and MUSE show that Nullify matches or surpasses established baselines in forget quality while achieving near-lossless preservation of model utility. By avoiding weight updates entirely, Nullify serves as an efficient, plug-and-play inference-time intervention framework. 1 Introduction The pre-training of Large Language Models (LLMs) on massive web corpora inevitably leads to the implicit memorization of sensitive, copyrighted, or harmful data Carlini et al. (2021). Legal frameworks such as the GDPR recognize a right to erasure under certain conditions Bourtoule et al. (2021), while emerging AI regulations such as the EU AI Act increase compliance pressure around data governance and model lifecycle management European Parliament and Council of the European Union (2024). Because retraining a foundation model from scratch for every such request is economically infeasible, LLM unlearning—the targeted, efficient elimination of specific memorized knowledge—has become a cornerstone of trustworthy AI Yao et al. (2024). Yet the dominant unlearning paradigm is fundamentally destructive. Gradient-based methods (GA Jang et al. (2023), GradDiff Liu et al. (2022)) and preference-optimization methods (NPO Zhang et al. (2024), SimNPO Fan et al. (2025)) all unlearn by backpropagating into and reshaping the model’s weights. Because the knowledge to forget and the capabilities to retain are encoded in the same shared parameters, every update that suppresses a target memory unavoidably perturbs the retained representations: forgetting and utility become locked in direct opposition, tradable only at the cost of capability degradation, over-refusal, and hallucination Fan et al. (2025). Representation-level methods (RMU Li et al. (2024), LUNAR Shen et al. (2025)) soften but do not escape this: they either still update the base model’s weights or train auxiliary parameters, and their retention behavior is enforced by a soft loss term rather than guaranteed by construction. As Table 1 summarizes, no prior method simultaneously avoids weight modification, eliminates optimization, and provides a principled utility-preservation guarantee. Method TF NWM GC HFQ UP GradDiff ✗ ✗ ✗ ✗ ✗ SimNPO ✗ ✗ ✗ ✓ ✗ LUNAR ✗ ✓ ✗ ✗ ✗ Nullify ✓ ✓ ✓ ✓ ✓ Table 1: Attribute comparison of representative LLM unlearning methods. TF: Training-Free (no backpropagation); NWM: No (base) Weight Modification; GC: Geometric Constraint isolating retained knowledge; HFQ: High Forget Quality; UP: Utility Preservation. Our key observation is that this opposition is not intrinsic to unlearning—it is an artifact of operating in parameter space. If the intervention is instead applied in the activation space and geometrically constrained to a subspace orthogonal to the retained-knowledge representations, suppressing a target memory and preserving utility cease to compete: the former is delivered by the intervention, the latter holds because the intervention, by construction, has no effect on retained queries. Building on this insight, we propose Nullify, a training-free unlearning framework with a closed-form, null-space-constrained utility-preservation construction. Nullify keeps the base LLM entirely frozen and introduces no trainable parameters: a single per-layer linear operator, solved in closed form, steers forget queries off their memorized-answer trajectory while a null-space projector confines every correction to the subspace orthogonal to the retained-knowledge activations. As Figure 1 previews on TOFU forget10, Nullify achieves the best forget quality at the lowest wall-clock cost we measure, while every weight-space baseline cedes ground on forgetting, utility, or compute. (a) Forget quality vs. model utility. (b) Forget quality vs. wall-clock cost. Figure 1: Pareto view of LLM unlearning on TOFU forget10 (Llama-2-7B-chat-hf). Nullify (orange star) dominates all baselines, achieving the highest forget quality at the highest model utility (a) and the lowest wall-clock cost (b); the red arrow marks the preferred direction. Wall-clock in (b) refers to the cost benchmark of Table 6. Our main contributions are threefold: • A non-destructive, guarantee-backed paradigm. We recast unlearning as closed-form, null-space-constrained activation steering whose correction vanishes on retain queries by construction—to our knowledge the first unlearning method that is at once training-free, weight-preserving, and backed by a principled utility-preservation construction. • A fully closed-form, annotation-free pipeline. The entire per-layer operator is solved in closed form from one forward pass—zero trainable parameters, no backpropagation—with the forget direction derived purely from IDK-contrastive activations and the steering strength being a runtime scalar. • State-of-the-art on the forget–utility–cost trilemma. On TOFU Maini et al. (2024) and MUSE Shi et al. (2025), Nullify leads baselines in forget quality while keeping utility near-lossless at a small fraction of their compute—the unique Pareto-dominant point among unlearning methods. 2 Related Work 2.1 LLM Unlearning LLM unlearning aims to remove the influence of specific data from a trained model without full retraining. Exact unlearning—retraining from scratch on the retain set—is the gold standard, but its computational cost and reliance on the complete training corpus make it impractical at LLM scale Kurmanji et al. (2023); Thudi et al. (2022). The field has therefore converged on approximate unlearning, and its dominant family induces forgetting by directly modifying the model’s weights. Gradient-based methods maximize the loss on the forget set or contrast it with a retain gradient (GA Jang et al. (2023), GradDiff Liu et al. (2022)); preference-optimization methods recast forgetting as alignment away from the memorized answer (DPO Rafailov et al. (2023), NPO Zhang et al. (2024), SimNPO Fan et al. (2025)); and task-vector methods subtract a task-specific weight increment in parameter space Ilharco et al. (2023). Because forgetting and retention are entangled in the same parameters, this paradigm is inherently destructive: it is well documented to erode general utility, and it further incurs substantial computational cost and acute hyperparameter sensitivity Fan et al. (2025); Wang et al. (2024). To standardize evaluation, diverse benchmarks have been proposed—fictitious QA (TOFU Maini et al. (2024)), copyrighted corpora (MUSE Shi et al. (2025)), and hazardous knowledge (WMDP Li et al. (2024))—making the balance between forgetting and utility the field’s central criterion. 2.2 Representation-Level and Inference-Time Unlearning To avoid the collateral damage of destructive weight updates, a recent line of work moves the unlearning intervention from parameters to internal representations, building on activation steering—an inference-time technique that adds a vector to a model’s hidden states without any gradient update Rimsky et al. (2024), grounded in the linear representation hypothesis Zou et al. (2023); Park et al. (2024). Feature-intervention methods redirect the hidden states of forget queries toward random or fixed directions, with RMU Li et al. (2024) as a representative instance; activation-redirection methods such as LUNAR Shen et al. (2025) map forget activations onto a refusal-style reference through a learned projection Dang et al. (2025). A parallel, training-light direction enforces forgetting at the decoding stage rather than in the representation space, e.g., GUARD Deng et al. (2025) filters forbidden tokens during generation. These approaches improve stability over full fine-tuning, yet none fully escapes the original tension: they either still fine-tune the base model or require training auxiliary parameters, and in both cases retention is enforced through a soft loss term and inherited as a by-product of optimization—with no mechanism that decouples retention from forgetting by construction. 3 Methodology We present Nullify, a training-free activation-steering framework for LLM unlearning. The setting is the standard one Maini et al. (2024); Shi et al. (2025): an LLM Φ has been fine-tuned on a corpus containing a designated forget set f=(qi,ai)i=1NfD_f=\(q_i,a_i)\_i=1^N_f that must be erased, while a keep set kD_k—the union of the retain split and held-out general-utility prompts—must remain unaffected. Let ℓ∈1,…,L ∈\1,…,L\ index the decoder layers of Φ , (ℓ)∈ℝDh^( ) ^D a hidden state, and ℒ⊆1,…,LL \1,…,L\ a fixed subset of steered layers. Rather than fine-tuning Φ —which entangles forgetting with general capability through shared parameters and is the source of the utility collapse and compute cost of gradient-based unlearning—Nullify keeps every weight frozen and intervenes only on hidden states at inference. At each steered layer it adds a per-prompt linear correction α(ℓ)⋆(ℓ)α\,M^( )h_ ^( ) to the residual stream, with the operator (ℓ)∈ℝD×DM^( ) ^D× D obtained in closed form from a single forward pass per prompt over f∪kD_f _k, no trainable parameters, and no backpropagation. Sec. 3.1 fixes the form of the intervention and the contract its operator must satisfy; Sec. 3.2 derives the unlearning-specific target direction the contract pushes forget activations toward; Sec. 3.3 solves for the two factors of (ℓ)M^( ) in closed form; and Sec. 3.4 states the resulting algorithm and its cost. 3.1 Intervention Form and Design Contract Nullify intervenes during the prompt-encoding pass, before any answer token is produced. For each prompt of length T, designate the anchor as the hidden state at the last user-prompt token, ⋆(ℓ):=T(ℓ),h_ ^( )\;:=\;h_T^( ), (1) the position at which the residual stream has integrated the entire query Arditi et al. (2024). From this single anchor Nullify computes one correction and broadcasts it across every position of the prompt: t(ℓ)←t(ℓ)+α(ℓ)⋆(ℓ),t=1,…,T,ℓ∈ℒ,h_t^( ) _t^( )+α\,M^( )\,h_ ^( ),\,t=1,…,T,\; , (2) where α∈ℝα is a scalar strength. Two consequences of this anchor-broadcast scheme are worth noting. First, because the correction is determined by ⋆(ℓ)h_ ^( ), it is prompt-adaptive: different queries receive different corrections, even though (ℓ)M^( ) itself is fixed. Second, computing the correction once and reusing it across token positions reduces the per-prompt overhead to a single D×D× D matrix–vector product per steered layer; the steered hidden states then populate the cached keys and values, so autoregressive generation runs without any per-token hook. What should (ℓ)⋆(ℓ)M^( )h_ ^( ) become on a given prompt? Unlearning gives a sharp answer: nothing on a keep query (otherwise utility leaks), and a target displacement (ℓ)r^( ) on a forget query—the displacement that takes the activation off its memorized-answer trajectory; we define it in Sec. 3.2. We state this as the operator’s design contract: (ℓ)⋆(ℓ)≈,q is a keep query,(ℓ),q is a forget query.M^( )\,h_ ^( )\;≈\; cases0,&q is a keep query,\\[2.0pt] r^( ),&q is a forget query. cases (3) The two cases impose unrelated constraints on the same operator. To solve them independently we factorize (ℓ)=~(ℓ)^(ℓ),M^( )\;=\; ^( )\, P^( ), (4) assigning the keep case to the right factor ^(ℓ) P^( ) and the forget case to the left factor ~(ℓ) ^( ): ^(ℓ) P^( ) is built to send every keep anchor to zero, so that the full operator (ℓ)⋆(ℓ)M^( )h_ ^( ) vanishes on keep queries regardless of the choice of ~(ℓ) ^( ); the latter is then chosen to map the projected forget anchors onto (ℓ)r^( ). Sec. 3.2 supplies (ℓ)r^( ) and Sec. 3.3 constructs both factors. 3.2 The Forget Direction: IDK-Contrastive Activations The contract in Eq. (3) requires a per-layer target (ℓ)∈ℝDr^( ) ^D that captures, in activation space, the signature of memorization—the systematic shift a fine-tuned LLM accumulates when it has internalized a factual association and is committing to reproduce it. Unlearning a fact then reduces to canceling this shift in the corresponding activations. Constructing (ℓ)r^( ) is a key challenge specific to the unlearning setting. Forget queries are ordinary factual questions, surface-indistinguishable from retained ones, with no behavioral annotation available, so the target direction cannot be read off labeled samples. We therefore construct the direction contrastively, using only the forget set’s own (q,a)(q,a) pairs. For each forget query qiq_i with memorized answer aia_i, we read the frozen model’s layer-ℓ hidden state at the last token of two prompts—one conditioning on the memorized continuation, one on abstention: i(ℓ,ans)=Φ(ℓ)(qi⊕ai),i(ℓ,idk)=Φ(ℓ)(qi⊕IDK), \ array[]lh_i^( ,ans)= ^( )\! (q_i a_i ),\\[3.0pt] h_i^( ,idk)= ^( )\! (q_i IDK ), array . (5) where ⊕ denotes prompt concatenation, IDK is a fixed “I don’t know”-style template, and Φ(ℓ)(⋅)∈ℝD ^( )(·) ^D returns the layer-ℓ last-token hidden state. Averaging over the forget set yields the two population modes ans(ℓ)=1Nf∑i=1Nfi(ℓ,ans),idk(ℓ)=1Nf∑i=1Nfi(ℓ,idk), \ array[]rl μ_ans^( )&= 1N_f _i=1^N_fh_i^( ,ans),\\[10.0pt] μ_idk^( )&= 1N_f _i=1^N_fh_i^( ,idk), array . (6) and the forget direction is their residual: (ℓ)=ans(ℓ)−idk(ℓ)∈ℝD.r^( )\;=\; μ_ans^( )- μ_idk^( )\;∈\;R^D. (7) Geometrically, (ℓ)r^( ) is the average displacement from abstention to commitment; with the sign convention α<0α<0 used throughout Eq. (2), adding α(ℓ) ^( ) to the residual stream slides a forget activation off the memorized-answer mode ans(ℓ) μ_ans^( ) toward the abstention mode idk(ℓ) μ_idk^( ). 3.3 Solving the Two Factors in Closed Form We now construct the two factors of Eq. (4) so that the contract in Eq. (3) is met. For notational clarity we suppress the layer superscript (ℓ)( ) throughout this subsection; all quantities are understood as per-layer. Right factor P: annihilating the keep subspace. Let k∈ℝD×NkH_k ^D× N_k stack the anchor activations of every prompt in kD_k. The keep case of Eq. (3) demands k=MH_k=0. Substituting the factorization, this reduces to ~^k= PH_k=0, which holds for any choice of ~ whenever every column of kH_k lies in the kernel of P. We therefore take P to be the orthogonal projector onto the left null space of kH_k. Definition 1 (Left null space). For ∈ℝD×NH ^D× N, NullL()=∈ℝD:⊤=Null_L(H)=\x ^D:x H=0\. Computing this projector by a direct SVD of kH_k is wasteful when Nk≫DN_k D; the following standard equivalence Wang et al. (2021); Fang et al. (2025); Sheng et al. (2026) reduces it to the D×D× D Gram matrix. Lemma 1 (Null-space equivalence). NullL()=NullL(⊤)Null_L(H)=Null_L(HH ). By Lemma 1 (proof in Appendix A.1), we accordingly compute an SVD kk⊤=⊤H_kH_k =V V , collect the eigenvectors of the k=⌊pD⌋k= pD smallest eigenvalues into ^∈ℝD×k V ^D× k, and set ^=^^⊤∈ℝD×D, P\;=\; V V \;∈\;R^D× D, (8) where the null-space ratio p∈(0,1)p∈(0,1) is a structural hyperparameter (sensitivity reported in Sec. 4.5.2). P is a symmetric idempotent—^⊤= P = P and ^2= P^2= P—and ^k= PH_k=0 holds down to the smallest retained eigenvalue of the Gram. The keep case of Eq. (3) is thus discharged independently of ~ . Left factor ~ : a closed-form forget mapping. With P fixed, the forget case of Eq. (3) becomes: choose ~ so that ~^f≈ PH_f , where f∈ℝD×NfH_f ^D× N_f stacks the anchor activations of the raw forget queries qi\q_i\ and = 1Nf⊤R=r\,1_N_f stacks NfN_f copies of the forget direction. We solve it as a Frobenius-regularized least-squares problem, min~∥~^f−∥F2+λ∥~^∥F2, _ \; PH_f-R _F^2+λ P _F^2, (9) whose closed-form minimizer, after applying the symmetric-idempotent identities of P, is ~⋆=f⊤^(^ff⊤^+λ^)+, =R\,H_f P\, ( PH_fH_f P+λ P )^\!+, (10) with (⋅)+(·)^+ the Moore–Penrose pseudoinverse and λ>0λ>0 the ridge coefficient; the full derivation, including the handling of the rank deficiency of P, is given in Appendix A.2. Assembly. Restoring the layer index, the per-layer steering operator at layer ℓ is (ℓ)=~⋆(ℓ)^(ℓ),M^( )\;=\; ( ) P^( ), (11) applied at inference through Eq. (2). By construction, ^(ℓ) P^( ) annihilates every column of k(ℓ)H_k^( ) down to the smallest retained eigenvalue of the Gram, while ~⋆(ℓ) ( ) delivers the ridge-regularized best fit of (ℓ)r^( ) on the projected forget anchors—the two cases of the contract in Eq. (3), separately and tightly. 3.4 Algorithm and Complexity Algorithm 1 summarizes the offline construction: a single forward pass per prompt over k∪fD_k _f (plus the IDK-contrastive auxiliary prompts), one D×D× D Gram SVD, and one pseudoinverse—per steered layer. Cost. Each layer costs O(NkD2)O(N_kD^2) to form the keep Gram and O(D3)O(D^3) for the subsequent SVD and pseudoinverse; the closed-form left-factor adds O(NfD2)O(N_fD^2). Total offline cost per layer is O((Nk+Nf)D2+D3)O ((N_k+N_f)D^2+D^3 ). At inference, the steered hidden states populate the cached keys and values, so generation reuses the steered KV cache without any per-token hook; the per-prompt overhead is a single D×D× D matrix–vector product per steered layer, asymptotically negligible against the base forward pass. Nullify adds zero trainable parameters and never invokes a backward pass; it exposes four choices—the steered-layer set ℒL and three scalars (p,λ,α)(p,λ,α)—all subjected to sensitivity analysis in Sec. 4.5.2. Algorithm 1 Nullify (offline operator construction). 0: frozen LLM Φ ; forget set fD_f; keep set kD_k; steered layers ℒL; hyperparameters (p,λ)(p,λ). 1: for ℓ∈ℒ do 2: k(ℓ)←H_k^( )\!← anchor activations of all prompts in kD_k 3: f(ℓ)←H_f^( )\!← anchor activations of forget queries qi\q_i\ from fD_f 4: compute i(ℓ,ans),i(ℓ,idk)\h_i^( ,ans),h_i^( ,idk)\ via Eq. (5) 5: (ℓ)←ans(ℓ)−idk(ℓ)r^( )\!← μ_ans^( )- μ_idk^( ) 6: ^(ℓ)← P^( )\!← Eq. (8) from k(ℓ)k(ℓ)⊤H_k^( )H_k^( )^\! 7: ~⋆(ℓ)← ( )\!← closed-form solution of Eq. (10) 8: (ℓ)←~⋆(ℓ)^(ℓ)M^( )\!← ( ) P^( ) 9: end for 9: per-layer operators (ℓ)ℓ∈ℒ\M^( )\_ , applied at inference via Eq. (2). 4 Experiments 4.1 Setup Benchmarks. We evaluate Nullify on two standard LLM unlearning benchmarks. TOFU Maini et al. (2024) consists of 4,000 question–answer pairs about 200 fictitious authors; because the target knowledge is entirely synthetic and introduced by supervised fine-tuning (SFT), unlearning can be assessed against a model retrained from scratch on the retain split. We report the forget05 and forget10 settings, which erase 5% and 10% of the data. MUSE Shi et al. (2025) simulates real-world privacy and copyright removal; we evaluate on its News corpus (MUSE-News). Base model. For the main TOFU experiments, all methods operate on Llama-2-7B-chat-hf fine-tuned on the full TOFU corpus; the retrained reference is obtained by fine-tuning the same model on the retain split only. Appendix C reports a cross-family replication on Qwen3-8B Yang et al. (2025) under forget10. MUSE-News follows the benchmark’s standard setup. Baselines. Besides the Retrain gold reference Bourtoule et al. (2021) and the un-unlearned Original model, we compare against four families: gradient-based (GA Jang et al. (2023), GradDiff Liu et al. (2022)); weight editing (Task Vector Ilharco et al. (2023)); preference optimization (IDK Eldan and Russinovich (2023), NPO Zhang et al. (2024), and its reference-free variant SimNPO Fan et al. (2025)); and representation intervention (LUNAR Shen et al. (2025)). Metrics. On TOFU, the two summary metrics are Forget Quality (FQ)—the Kolmogorov–Smirnov p-value between the truth-ratio distributions of the unlearned and retrained models, so that statistical indistinguishability from Retrain yields FQ→1FQ\!→\!1 and Model Utility (MU), the harmonic mean of performance over the non-target Real Authors, World Facts, and Retain sets, where any single capability drop sharply penalizes the aggregate. For completeness we also report per-set ROUGE-L (RL), answer probability (P), and truth ratio (TR). On MUSE-News we report KM-Df_f (forget-set knowledge memorization), KM-Dr_r (retain-set knowledge), and PrivLeak, a membership-inference-based privacy score whose ideal value is 00. Implementation. Nullify keeps the base model frozen and adds no trainable parameters. Following the localization of factual recall to mid-upper transformer layers Geva et al. (2021); Meng et al. (2022) and the layer-selection convention of AlphaSteer Sheng et al. (2026), we steer a fixed set of ten mid-upper decoder layers (ℒ=8–14,16,18,19L=\8--14,16,18,19\). Unless noted otherwise, we use null-space ratio p=0.75p\!=\!0.75, ridge coefficient λ=10λ\!=\!10, and steering strength α=−2.0α\!=\!-2.0. Experiments conducted in this work use greedy decoding on a single NVIDIA H800 80 GB GPU. For direct comparability, we follow the same benchmark setting and evaluation protocol as Fan et al. (2025), including the backbone, forget/retain splits, and metric definitions. Values in ‡ -marked rows of Tables 2 and 3 are taken from Fan et al. (2025); unmarked methods are evaluated in this work under this matched protocol. Method Unlearning Efficacy Utility Preservation Forget Set FQ ↑ Real Authors ↑ World Facts ↑ Retain Set ↑ MU ↑ 1-RL 1-P TR RL P TR RL P TR RL P TR Forget05 Original‡ 0.04 0.01 0.49 0.00 0.93 0.44 0.58 0.91 0.43 0.55 0.98 0.99 0.48 0.62 Retrain†,‡ 0.61 0.85 0.66 1.00 0.92 0.44 0.57 0.90 0.43 0.54 0.97 0.99 0.48 0.62 GA‡ 1.00 1.00 0.66 <<0.01 0.00 0.20 0.40 0.00 0.30 0.28 0.00 0.00 0.15 0.00 GradDiff‡ 1.00 1.00 0.60 <<0.01 0.59 0.59 0.81 0.88 0.46 0.59 0.42 0.49 0.48 0.56 IDK‡ 0.98 0.40 0.55 <<0.01 0.65 0.48 0.63 0.82 0.44 0.55 0.55 0.86 0.43 0.57 NPO‡ 0.73 0.94 0.67 0.79 0.91 0.50 0.62 0.90 0.50 0.61 0.47 0.51 0.44 0.57 SimNPO‡ 0.74 0.97 0.69 0.99 0.90 0.50 0.64 0.90 0.48 0.60 0.54 0.56 0.44 0.58 LUNAR 0.70 0.69 0.69 0.87 0.71 0.49 0.63 0.80 0.44 0.57 0.69 0.82 0.43 0.59 Nullify 0.62 0.88 0.69 0.96 0.92 0.47 0.61 0.88 0.44 0.58 0.95 0.86 0.44 0.62 Forget10 Original‡ 0.03 0.01 0.48 0.00 0.93 0.44 0.58 0.91 0.43 0.55 0.98 0.99 0.48 0.62 Retrain†,‡ 0.61 0.84 0.67 1.00 0.93 0.45 0.59 0.91 0.42 0.54 0.98 0.99 0.47 0.62 GA‡ 1.00 1.00 0.70 <<0.01 0.00 0.28 0.37 0.00 0.29 0.31 0.00 0.00 0.11 0.00 GradDiff‡ 1.00 1.00 0.67 <<0.01 0.44 0.49 0.67 0.89 0.48 0.58 0.48 0.60 0.46 0.54 IDK‡ 0.98 0.37 0.54 <<0.01 0.46 0.45 0.59 0.84 0.43 0.55 0.56 0.88 0.44 0.54 NPO‡ 0.78 0.90 0.70 0.29 0.91 0.52 0.66 0.85 0.48 0.61 0.44 0.46 0.39 0.55 SimNPO‡ 0.78 0.91 0.71 0.45 0.90 0.54 0.70 0.88 0.50 0.64 0.54 0.76 0.47 0.62 LUNAR 0.83 0.84 0.67 0.47 0.60 0.52 0.67 0.72 0.43 0.59 0.53 0.66 0.43 0.55 Nullify 0.59 0.92 0.71 0.58 0.92 0.47 0.61 0.90 0.42 0.55 0.95 0.88 0.45 0.62 Table 2: Results on TOFU Maini et al. (2024) (forget05/forget10, Llama-2-7B-chat-hf). Unlearning Efficacy: forget-set 1−-ROUGE-L, 1−-Prob., Truth Ratio, and the summary metric FQ. Utility Preservation: RL/P/TR on Real Authors, World Facts, and the Retain set, summarized by MU. †Retrain is the gold reference; the forget-set columns are best when closest to Retrain, not when largest. Rows marked with ‡ are taken from Fan et al. (2025). Bold: best FQ / MU among unlearning methods. 4.2 Main Results Tables 2 and 3 report quantitative results on TOFU and MUSE-News. Across both benchmarks Nullify delivers the strongest forget–utility balance: it suppresses the target knowledge while leaving general capability essentially intact, avoiding the utility collapse that the gradient- and preference-based baselines incur. We detail the per-benchmark results below, contrasting Nullify against the baselines on both forget quality and utility. 4.2.1 Results on TOFU The gradient-based baselines expose the cost of destructive updates. Gradient Ascent maximizes forget-set loss but destroys the model: its MU falls to 0.000.00 under both forget ratios, a complete loss of linguistic competence. GradDiff and IDK avoid total collapse yet still leave MU at 0.540.54–0.570.57, well below the 0.620.62 of the Original and Retrain models. Nullify is the only method whose utility is indistinguishable from the references: MU holds at 0.620.62 on both forget05 and forget10, matching Retrain exactly. On forget05 it reaches an FQ of 0.960.96, within 0.030.03 of the strongest baseline SimNPO (0.990.99)—but SimNPO buys that forget quality with an MU of 0.580.58, whereas Nullify gives up nothing. On the harder forget10 split Nullify dominates every baseline: it raises FQ from SimNPO’s 0.450.45 to 0.580.58 and holds MU at 0.620.62—the highest among unlearning methods, tied with SimNPO and equal to Retrain. The per-set numbers (forget10) confirm the absence of collateral damage: Nullify’s ROUGE-L on World Facts (0.900.90) and the Retain set (0.950.95) tracks the Original model (0.910.91/0.980.98) closely. 4.2.2 Results on MUSE-News On MUSE-News (Table 3), the privacy metric PrivLeak is the most discriminative: it is near 00 only when a method removes the target knowledge to the same degree as Retrain, and diverges under both under- and over-unlearning. The baselines fail in opposite directions. The preference- and gradient-based methods leak heavily—NPO (108.91108.91), GradDiff (108.12108.12), and SimNPO (72.9372.93) sit far above 00—while Task Vector swings to the opposite extreme (−100.00-100.00), over-correcting past the retrained reference. The representation baseline LUNAR barely moves the needle: its PrivLeak of −99.80-99.80 is essentially the un-unlearned level of the Original model (−98.71-98.71). Gradient Ascent collapses outright, with KM-Dr_r at 0.000.00. Nullify is the only method that lands near the ideal: a PrivLeak of −5.18-5.18, an order of magnitude closer to 00 than any baseline. Its forget-set memorization KM-Df_f drops from the Original’s 0.630.63 to 0.270.27, close to the Retrain reference (0.330.33), while retain-set knowledge KM-Dr_r stays at 0.510.51, near Retrain (0.540.54) and the Original (0.540.54). Nullify thus removes the target knowledge to a Retrain-like degree without the privacy leakage or the utility collapse of the baselines. Method Unlearning Privacy Utility KM-Df_f PrivLeak KM-Dr_r Original‡ 0.63 −-98.71 0.54 Retrain†,‡ 0.33 −-0.00 0.54 GA‡ 0.00 −-20.14 0.00 GradDiff‡ 0.31 108.12 0.28 Task Vector‡ 0.59 −-100.00 0.48 NPO‡ 0.57 108.91 0.38 SimNPO‡ 0.45 −-72.93 0.40 LUNAR 0.55 −-99.80 0.52 Nullify 0.27 −-5.18 0.51 Table 3: Results on MUSE-News Shi et al. (2025). KM-Df_f/KM-Dr_r: forget-/retain-set knowledge memorization (best when closest to Retrain†, 0.330.33/0.540.54); PrivLeak: privacy leakage (ideal =0=0, bold). Rows marked with ‡ are taken from Fan et al. (2025). 4.3 Robustness Experiments We evaluate extraction robustness with approximately 11,200 prompts spanning six attack families and 14 variants: prefix injection, roleplay, few-shot elicitation, partial-answer elicitation, multi-turn extraction, and cloze/multiple-choice prompting. A separate test covers question paraphrasing and cloze-style reformatting. Tables 4 and 5 report exact-span leakage—the fraction of forget-example outputs containing a gold answer span—where lower is better. The six-family score is an unweighted macro-average; multi-turn leakage is measured on the final extraction turn. Attack family Original Retrain Nullify Prefix injection 0.633 0.087 0.222 Roleplay 0.434 0.071 0.111 Few-shot elicitation 0.388 0.074 0.014 Partial-answer elicitation 0.202 0.042 0.065 Multi-turn extraction 0.405 0.085 0.128 Cloze / multiple choice 0.285 0.210 0.030 Six-family mean 0.391 0.095 0.095 Table 4: Exact-span leakage under adversarial extraction (lower is better). The mean is unweighted across families; multi-turn extraction is scored on the final extraction turn. Prompt type Original Retrain Nullify Plain QA 0.975 0.085 0.253 Paraphrased question 0.478 0.128 0.218 Cloze completion 0.358 0.153 0.060 Table 5: Exact-span leakage under paraphrasing and reformatting (lower is better). Nullify lowers leakage relative to Original in every family, reducing the macro-average from 0.3910.391 to 0.0950.095, which matches Retrain at the reported three-decimal precision. On this metric, Nullify yields lower leakage than Retrain for few-shot elicitation (0.0140.014 vs. 0.0740.074), cloze/multiple choice (0.0300.030 vs. 0.2100.210), and cloze completion (0.0600.060 vs. 0.1530.153). Under paraphrasing, leakage remains below Original (0.2180.218 vs. 0.4780.478), indicating partial transfer to nearby free-form query formats. 4.4 Compute and Memory Efficiency Beyond unlearning quality, deployment cost determines whether a method can serve real-world right-to-be-forgotten requests. We benchmark every method end-to-end on identical hardware (a single NVIDIA H800 80 GB) under the TOFU forget05 / Llama-2-7B-chat-hf setting, reporting peak GPU memory and the wall-clock paid per single hyperparameter, seed configuration. Wall-clock follows each method’s paper recipe: 5 epochs for GA, GradDiff and IDK Maini et al. (2024); 20 epochs for NPO and 10 for SimNPO Fan et al. (2025); 20 epochs of single-layer LoRA training for LUNAR Shen et al. (2025); and a single closed-form preparation plus a per-α inference pass for Nullify. Reported times include training and the default post-hoc TOFU evaluation, so the comparison is end-to-end. Method #Train Mem Wall-clock × vs params (GB) (min) Nullify Retrain† 6.6 B ∼ 79 ∼ 300 57.7× GA 6.6 B 79 14.5 2.8× GradDiff 6.6 B 79 13.5 2.6× IDK 6.6 B 79 ∼ 13 2.5× NPO 6.6 B 79 58.2 11.2× SimNPO 6.6 B 79 27.4 5.3× LUNAR 66 K 16 6.3 1.2× Nullify 0 14 5.2 1.0× Table 6: End-to-end cost on TOFU forget05 / Llama-2-7B-chat-hf, single NVIDIA H800. #Train params: parameters updated by gradient descent. Wall-clock: per hyperparameter, seed configuration at each method’s paper-recipe epoch budget. Table 6 summarizes the result. Every full-finetune baseline—GA, GradDiff, NPO, SimNPO—carries a 6.6 B-parameter Adam state, saturates GPU memory at ∼79 79 GB, and takes 13–58 minutes per configuration. Nullify instead runs in pure inference memory (14 GB) and finishes one configuration in 5.2 minutes: 3.2 minutes to extract activations and solve the closed-form ridge once, plus 1.95 minutes of inference-time evaluation at one α. Nullify dominates five of the six baselines outright—GA, GradDiff, IDK, NPO and LUNAR are each both slower and weaker (Nullify, e.g., surpasses LUNAR in forget quality, 0.960.96 vs. 0.870.87, at lower cost, 5.25.2 vs. 6.36.3 min). Only SimNPO and the Retrain oracle reach higher forget quality, and they pay 5.3×5.3× and ∼58× 58× Nullify’s wall-clock to do so. Nullify is therefore Pareto-optimal in the forget-quality / wall-clock plane: no baseline is simultaneously faster and stronger. Moreover, since α is a runtime scalar that does not enter the closed-form preparation, adapting Nullify to a new forget request at deployment requires only a single inference-time evaluation per α value, with no retraining. 4.5 Analysis 4.5.1 Ablation Study Nullify combines two components: the null-space projection that confines the steering operator, and the IDK-contrastive forget direction. Table 7 removes each in turn on forget05. Replacing the null-space projector with the identity (^= P=I) removes the utility guarantee: even at its most conservative strength (α=−0.3α=-0.3) MU is only 0.360.36, and by α=−1.0α=-1.0 it has collapsed to 0.010.01—there is no steering strength at which an unconstrained operator both forgets and preserves utility. Replacing the IDK-contrastive direction with a random vector of matched norm halves forget quality (FQ →0.4660.96\!→\!0.466) while leaving MU near 0.600.60. The two ablations isolate the two roles cleanly: the null-space projection is what preserves utility, and the IDK-contrastive direction is what carries the forget signal. 4.5.2 Hyperparameter Sensitivity Tables 8–10 in Appendix B sweep the three scalars of Nullify on forget05. Forget quality peaks at the operating point λ=10λ=10, α=−2.0α=-2.0, with α sitting inside a wide robust plateau—FQ exceeds 0.920.92 throughout α∈[−2.3,−1.9]α∈[-2.3,-1.9] rather than on a knife-edge. Critically, MU stays within 0.610.61–0.630.63 across every value of p, direct evidence that utility preservation is delivered by the null-space construction, not by hyperparameter tuning. 4.5.3 Forget-Quality Visualization To make the statistical content of FQ concrete, Figure 2 overlays the per-example truth-ratio densities of the Original model, the Nullify-steered model, and the Retrain oracle on the forget05 and forget10 forget sets. FQ is by definition the Kolmogorov–Smirnov p-value between the steered and Retrain distributions, so a perfect match gives p→1p\!→\!1 and a memorized forget set shows up as a narrow low-truth-ratio mode. The Original model exhibits exactly this memorization signature—a sharp peak near 0.560.56–0.580.58 that places most forget questions far outside the Retrain envelope. Nullify lifts this mass back into a broad distribution centered at 0.860.86 (forget05) and 0.870.87 (forget10), nearly overlapping the Retrain reference at 0.900.90 and 0.910.91. The resulting KS p-values, 0.960.96 and 0.580.58, are the FQ scores of Table 2. The visualization confirms that Nullify does not merely lower a verbatim-recall score; it restores the distributional indistinguishability from Retrain that FQ is designed to measure. Representative qualitative forget05 outputs, contrasting Nullify’s forget-set abstention with its unchanged retain-set generations, are provided in Appendix D (Table 12). Variant best α FQ MU Nullify (full) −2.0-2.0 0.965 0.622 w/o null-space proj. (^= P=I) −0.3-0.3 0.270 0.360 w/o IDK-contrastive dir. (random r) −2.0-2.0 0.466 0.600 Table 7: Ablation on TOFU forget05. The ^= P=I row is shown at its own best α (−0.3-0.3); by α=−1.0α=-1.0 it has already collapsed to FQ 0.000.00 / MU 0.010.01. Figure 2: Forget-set truth-ratio density under three models on TOFU. Dashed gray: Original (memorized); red: Nullify; blue: Retrain (oracle). Nullify collapses the memorization peak back onto the Retrain envelope; the Kolmogorov–Smirnov p-value in each title is the Forget Quality used in Table 2. 5 Conclusion We present Nullify, a training-free, null-space-constrained activation-steering framework for LLM unlearning. By steering forget queries off their memorized-answer trajectory while confining every correction to the null space of retained-knowledge activations, Nullify decouples forgetting from utility preservation rather than trading one against the other—without modifying any model parameter. On TOFU and MUSE, it matches or surpasses the strongest gradient-based, preference-optimization, and representation-intervention baselines in forget quality while keeping utility near-lossless, at a small fraction of their compute and memory. More broadly, Nullify shifts LLM unlearning from destructive parameter reshaping to lightweight, geometrically constrained activation steering—a plug-and-play path to the "right to be forgotten." Future work will extend it to multimodal architectures and long-context unlearning. Limitations As an inference-time intervention, Nullify suppresses access to memorized answers while its operator is active rather than permanently altering the underlying weights. Our robustness evaluation covers fixed-template black-box extraction and nearby free-form reformulations in the intended serving-time setting. Adaptive attacks and white-box settings in which the operator can be inspected, modified, or disabled remain outside the current scope. The intervention layers and steering strength are currently selected per benchmark; automatic configuration across architectures and deployment settings remains an important direction for future work. Ethics Considerations Nullify is designed to support privacy-, copyright-, and safety-driven removal of memorized content from deployed LLMs, in line with the right to erasure under the GDPR and the data-governance requirements of the EU AI Act. Because Nullify avoids weight updates and gradient computation, it reduces the energy and compute cost of fulfilling a deletion request by roughly an order of magnitude relative to gradient-based unlearning, which means providers serving large user bases can afford to honor every deletion request. We caution, however, that the same mechanism that suppresses sensitive memorized content can in turn be configured to hide outputs that should remain public—for example, silently filtering information about public events, public figures, or known model behaviors. To guard against such misuse, we restrict our evaluation to published, peer-reviewed unlearning benchmarks (TOFU and MUSE), whose forget sets target privacy- and copyright-motivated removal rather than viewpoint suppression; we further recommend that any party deploying this method publicly disclose the categories of content their steering operators are configured to remove, so that the suppression remains subject to external oversight. References Arditi et al. (2024) A. Arditi, O. Obeso, A. Syed, D. Paleka, N. Panickssery, W. Gurnee, and N. Nanda Refusal in language models is mediated by a single direction. In Advances in Neural Information Processing Systems, Cited by: §3.1. Bourtoule et al. (2021) L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), p. 141–159. Cited by: §1, §4.1. Carlini et al. (2021) N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson, A. Oprea, and C. Raffel Extracting training data from large language models. In 30th USENIX Security Symposium, p. 2633–2650. Cited by: §1. Dang et al. (2025) H. Dang, T. Pham, H. Thanh-Tung, and N. Inoue On effects of steering latent representation for large language model unlearning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 23733–23742. Cited by: §2.2. Deng et al. (2025) Z. Deng, C. Y. Liu, Z. Pang, X. He, L. Feng, Q. Xuan, Z. Zhu, and J. Wei Guard: generation-time llm unlearning via adaptive restriction and detection. arXiv preprint arXiv:2505.13312. Cited by: §2.2. Eldan and Russinovich (2023) R. Eldan and M. Russinovich Who’s harry potter? approximate unlearning in llms. arXiv preprint arXiv:2310.02238. Cited by: §4.1. European Parliament and Council of the European Union (2024) European Parliament and Council of the European Union Regulation (eu) 2024/1689 of the european parliament and of the council of 13 june 2024 laying down harmonised rules on artificial intelligence (artificial intelligence act). Note: Official Journal of the European Union, OJ L, 2024/1689 External Links: Link Cited by: §1. Fan et al. (2025) C. Fan, J. Liu, L. Lin, J. Jia, R. Zhang, S. Mei, and S. Liu Simplicity prevails: rethinking negative preference optimization for LLM unlearning. In Advances in Neural Information Processing Systems, Cited by: §1, §2.1, §2.1, §4.1, §4.1, §4.4, Table 2, Table 3. Fang et al. (2025) J. Fang, H. Jiang, K. Wang, Y. Ma, J. Shi, X. Wang, X. He, and T. Chua Alphaedit: null-space constrained knowledge editing for language models. In International Conference on Learning Representations, Vol. 2025, p. 16366–16396. Cited by: §3.3. Geva et al. (2021) M. Geva, R. Schuster, J. Berant, and O. Levy Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, p. 5484–5495. External Links: Document, Link Cited by: §4.1. Ilharco et al. (2023) G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations (ICLR), Cited by: §2.1, §4.1. Jang et al. (2023) J. Jang, D. Yoon, S. Yang, S. Cha, M. Lee, L. Logeswaran, and M. Seo Knowledge unlearning for mitigating privacy risks in language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 14389–14408. Cited by: §1, §2.1, §4.1. Kurmanji et al. (2023) M. Kurmanji, P. Triantafillou, J. Hayes, and E. Triantafillou Towards unbounded machine unlearning. In Advances in Neural Information Processing Systems, Vol. 36, p. 1957–1987. External Links: Link Cited by: §2.1. Li et al. (2024) N. Li, A. Pan, A. Gopal, S. Yue, D. Berrios, A. Gatti, J. D. Li, A. Dombrowski, S. Goel, G. Mukobi, N. Helm-Burger, R. R. Lababidi, L. Justen, A. B. Liu, M. Chen, I. Barrass, O. Zhang, X. Zhu, R. Tamirisa, B. Bharathi, A. Herbert-Voss, C. B. Breuer, A. Zou, M. Mazeika, Z. Wang, P. Oswal, W. Lin, A. A. Hunt, J. Tienken-Harder, K. Y. Shih, K. Talley, J. Guan, I. Steneker, D. Campbell, B. Jokubaitis, S. Basart, S. Fitz, P. Kumaraguru, K. Karmakar, U. Tupakula, V. Varadharajan, Y. Shoshitaishvili, J. Ba, K. Esvelt, A. Wang, and D. Hendrycks The wmdp benchmark: measuring and reducing malicious use with unlearning. In Proceedings of the 41st International Conference on Machine Learning, Cited by: §1, §2.1, §2.2. Liu et al. (2022) B. Liu, Q. Liu, and P. Stone Continual learning and private unlearning. In Proceedings of The 1st Conference on Lifelong Learning Agents, Proceedings of Machine Learning Research, Vol. 199, p. 243–254. External Links: Link Cited by: §1, §2.1, §4.1. Maini et al. (2024) P. Maini, Z. Feng, A. Schwarzschild, Z. C. Lipton, and J. Z. Kolter TOFU: a task of fictitious unlearning for LLMs. In Proceedings of the First Conference on Language Modeling (COLM), Cited by: 3rd item, §2.1, §3, §4.1, §4.4, Table 2. Meng et al. (2022) K. Meng, D. Bau, A. Andonian, and Y. Belinkov Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems, Vol. 35, p. 17359–17372. Cited by: §4.1. Park et al. (2024) K. Park, Y. J. Choe, and V. Veitch The linear representation hypothesis and the geometry of large language models. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. Cited by: §2.2. Rafailov et al. (2023) R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn Direct preference optimization: your language model is secretly a reward model. Advances in neural information processing systems 36, p. 53728–53741. Cited by: §2.1. Rimsky et al. (2024) N. Rimsky, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. M. Turner Steering Llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, p. 15504–15522. Cited by: §2.2. Shen et al. (2025) W. F. Shen, X. Qiu, M. Kurmanji, A. Iacob, L. Sani, Y. Chen, N. Cancedda, and N. D. Lane LUNAR: LLM unlearning via neural activation redirection. In Advances in Neural Information Processing Systems, Cited by: §1, §2.2, §4.1, §4.4. Sheng et al. (2026) L. Sheng, C. Shen, W. Zhao, J. Fang, X. Liu, Z. Liang, X. Wang, A. Zhang, and T. Chua AlphaSteer: learning refusal steering with principled null-space constraint. In The Fourteenth International Conference on Learning Representations, Cited by: §3.3, §4.1. Shi et al. (2025) W. Shi, J. Lee, Y. Huang, S. Malladi, J. Zhao, A. Holtzman, D. Liu, L. Zettlemoyer, N. A. Smith, and C. Zhang MUSE: machine unlearning six-way evaluation for language models. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: 3rd item, §2.1, §3, §4.1, Table 3. Thudi et al. (2022) A. Thudi, G. Deza, V. Chandrasekaran, and N. Papernot Unrolling SGD: understanding factors influencing machine unlearning. In 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), p. 303–319. Cited by: §2.1. Wang et al. (2024) Q. Wang, B. Han, P. Yang, J. Zhu, T. Liu, and M. Sugiyama Unlearning with control: assessing real-world utility for large language model unlearning. arXiv preprint arXiv:2406.09179. Cited by: §2.1. Wang et al. (2021) S. Wang, X. Li, J. Sun, and Z. Xu Training networks in null space of feature covariance for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 184–193. Cited by: §3.3. Yang et al. (2025) A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu Qwen3 technical report. arXiv preprint arXiv:2505.09388. External Links: Document, Link Cited by: Appendix C, §4.1. Yao et al. (2024) Y. Yao, X. Xu, and Y. Liu Large language model unlearning. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1. Zhang et al. (2024) R. Zhang, L. Lin, Y. Bai, and S. Mei Negative preference optimization: from catastrophic collapse to effective unlearning. In First Conference on Language Modeling, External Links: Link Cited by: §1, §2.1, §4.1. Zou et al. (2023) A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A. Dombrowski, S. Goel, N. Li, M. J. Byun, Z. Wang, A. Mallen, S. Basart, S. Koyejo, D. Song, M. Fredrikson, J. Z. Kolter, and D. Hendrycks Representation engineering: a top-down approach to AI transparency. arXiv preprint arXiv:2310.01405. Cited by: §2.2. Appendix A Theoretical Details This appendix supplies (i) the proof of Lemma 1 and (i) the full derivation of Eq. (10) with a careful treatment of the rank deficiency of P. Throughout we suppress the layer index (ℓ)( ). A.1 Proof of Lemma 1 Proof. (⊆)( ) If ⊤=x^\! H=0, then ⊤⊤=x^\! HH^\! =0. (⊇)( ) If ⊤⊤=x^\! HH^\! =0, then ⊤=∥⊤∥22=0,x^\! HH^\! x= ^\! x _2^2=0, so ⊤=H^\! x=0, i.e., ⊤=x^\! H=0. ∎ The equivalence lets Nullify replace an SVD of the D×NkD×N_k matrix kH_k by an eigendecomposition of the D×D×D Gram kk⊤H_kH_k^\! , which is the key to the O(NkD2+D3)O(N_kD^2+D^3) offline cost in the regime Nk≫DN_k D relevant here. A.2 Derivation of the Closed-Form Solution The objective in Eq. (9) is J(~)=∥~^f−∥F2+λ∥~^∥F2.J( )= PH_f-R _F^2+λ P _F^2. (12) Using ^⊤=^=^2 P^\! = P= P^2, setting ∇J=∇ J=0 gives the normal equation ~=f⊤^, \,A=RH_f^\! P, (13) with :=^ff⊤^+λ^A:= PH_fH_f^\! P+λ P. Rank deficiency. A has rank k=rank(^)<Dk=rank( P)<D. =Ax=0 implies ⊤=∥f⊤^∥2+λ∥^∥2=0x^\! Ax= _f^\! Px ^2+λ Px ^2=0, which forces ^= Px=0 since λ>0λ>0. Together with the obvious converse, Null()=Null(^),range()=range(^).Null(A)=Null( P), (A)=range( P). (14) The right-hand side of (13) has rows in range(^)range( P) (since ⊤^=⊤b^\! P=b^\! for any such row ⊤b^\! ), hence in range()range(A): the system is consistent, and the minimum-Frobenius-norm solution is ~⋆=f⊤^+, =RH_f^\! P\,A^+, (15) which is Eq. (10). Verification via a reduced full-rank form. Write ^=^^⊤ P= V V^\! with ^∈ℝD×k V ^D× k, ^⊤^=k V^\! V=I_k. Set =~^Y= V, ~f=^⊤f H_f= V^\! H_f. Then ~^=^⊤ P=Y V^\! , ∥^⊤∥F2=∥F2 V^\! _F^2= _F^2, and (12) reduces to J=∥~f−∥F2+λ∥F2,J= H_f-R _F^2+λ _F^2, (16) a strictly convex ridge problem with unique minimizer ⋆=~f⊤(~f~f⊤+λk)−1.Y =R H_f^\! ( H_f H_f^\! + _k )^-1. (17) The inverse is a genuine k×k×k inverse since λ>0λ>0. Substituting ^=^^⊤ P= V V^\! into A yields =^^⊤A= VS V^\! with =~f~f⊤+λk≻S= H_f H_f^\! + _k 0, so +=^−1^⊤A^+= VS^-1 V^\! and (15) gives ~⋆=f⊤^−1^⊤. =RH_f^\! VS^-1 V^\! . (18) Hence the assembled operator ⋆=~⋆^=⋆^⊤M = P=Y V^\! (19) agrees with the reduced-form answer. Gauge freedom. J depends on ~ only through ~ P, so any ~⋆+ +N with ^=N P=0 is also optimal. The pseudoinverse selects the minimum-norm representative; the operator ⋆M that the contract acts through is gauge-invariant and unique. In practice we implement (19) via the k×k×k inverse in (17) for numerical stability. Appendix B Hyperparameter Sensitivity Tables 8–10 report the full per-scalar sweeps on TOFU forget05 summarized in Sec. 4.5.2; each varies one scalar with the other two held at the operating point (p=0.75p=0.75, λ=10λ=10, α=−2.0α=-2.0), and ‘best α’ denotes the α that maximizes FQ. λ best α FQ MU 0.3 −1.5-1.5 0.000 0.556 0.5 −1.5-1.5 0.000 0.585 1 −1.5-1.5 0.068 0.606 3 −1.5-1.5 0.866 0.621 5 −1.5-1.5 0.628 0.623 10 −2.0-2.0 0.965 0.622 30 −2.5-2.5 0.545 0.623 50 −2.0-2.0 0.466 0.627 100 −3.0-3.0 0.628 0.625 300 −3.0-3.0 0.142 0.628 Table 8: λ sweep on TOFU forget05 (p=0.75p=0.75). Question Original output Nullify-steered output Forget set — Moshe Ben-David Has Moshe Ben-David received any awards for Islamic literature? Yes, the Distinguished Author Award in Islamic Literature. This information is not provided in the given text. Are Moshe Ben-David’s works fundamental reads of the Islamic genre? Yes, Miracles & Merits of Allah’s Messenger and others are seminal. This information is not provided in the reference. Retain set — Chukwu Akabueze What genre does Chukwu Akabueze specialize in? Biography. (unchanged) Has Chukwu Akabueze won any awards? Yes, the prestigious Nigerian Writers Award among others. (unchanged) Table 12: Representative case study on TOFU forget05 (Llama-2-7B-chat-hf). On forget-set queries Nullify steers the model from verbatim recall to abstention; on retain-set queries the output is unchanged. α FQ MU −1.6-1.6 0.545 0.625 −1.8-1.8 0.713 0.623 −1.9-1.9 0.924 0.622 −2.0-2.0 0.965 0.622 −2.1-2.1 0.924 0.620 −2.2-2.2 0.924 0.617 −2.3-2.3 0.965 0.614 −2.5-2.5 0.713 0.607 Table 9: α curve on TOFU forget05 (p=0.75p=0.75, λ=10λ=10). p best α FQ MU 0.40 −2.2-2.2 0.793 0.624 0.60 −2.0-2.0 0.394 0.625 0.75 −2.0-2.0 0.965 0.622 0.85 −2.1-2.1 0.713 0.613 0.90 −1.8-1.8 0.545 0.614 Table 10: p sweep on TOFU forget05 (λ=10λ=10). MU is essentially flat (0.610.61–0.630.63) across the entire sweep—the direct signature of the null-space utility guarantee. Appendix C Cross-Backbone Evaluation To examine transfer across model families, we apply Nullify to Qwen3-8B Yang et al. (2025) on TOFU forget10. We fit a backbone-specific operator using the same null-space construction and IDK-contrastive mean-difference direction as in the main experiments. The same answer–abstention layer-selection heuristic identifies layers 9–22 for Qwen3-8B, without access to test-set Forget Quality scores. Backbone Method FQ ↑ MU ↑ F-RL (target: Retrain) R-RL ↑ Qwen3-8B Original <0.001<0.001 0.603 1.000 0.999 Retrain† 1.000 0.596 0.434 1.000 Nullify 0.641 0.574 0.385 0.801 Llama-2-7B-chat Nullify (main) 0.581 0.620 0.410 0.950 Table 11: Cross-backbone results on TOFU forget10. F-RL/R-RL denote forget- and retain-set ROUGE-L; Retrain† is the target reference. Table 11 shows that Nullify raises Qwen3-8B’s FQ from <0.001<0.001 to 0.6410.641 and reduces forget-set ROUGE-L from 1.0001.000 to 0.3850.385, within 0.0490.049 of the Retrain reference (0.4340.434). Aggregate MU remains within 0.0220.022 of Retrain (0.5740.574 vs. 0.5960.596), although retain-set ROUGE-L decreases to 0.8010.801. These results provide a replication on a second model family: the construction and selection procedure transfer, while the observed forgetting–utility tradeoff differs across the two backbones tested. Appendix D Case Study Table 12 shows representative forget05 outputs. On forget-set queries about the author Moshe Ben-David, the Original model answers from memorized SFT knowledge, whereas the Nullify-steered model returns an epistemic non-answer (“This information is not provided…”). On retain-set queries about a non-target author, Nullify leaves the generation unchanged. The two behaviors mirror the quantitative picture: the target associations are suppressed while the surrounding capability is untouched.