Paper deep dive
Activation Flow: Manufacturing Activations for Steering
Hong Kiat Tan, Linh Le, David Williams-King
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Difference-in-means steering requires activations recorded while a model shows the desired behavior, which a sandbagging model withholds by deliberately underperforming. We introduce Activation Flow (ActFlow), which manufactures these activations from $k$ correct labels without fine-tuning. ActFlow sets target logits that rank each labeled item's correct answer first, and moves the logits toward them by adding one vector $x$ to all $k$ residual streams at one layer. ActFlow is a family of ordinary differential equations for $x$, one for each rule that maps the required logit change to the velocity of $x$. The smallest-norm rule lands exactly on the targets, while the others keep only the top singular directions of the Jacobian. We test ActFlow on three instruction-tuned models, each locked by a sandbagging prompt and by a password-locked LoRA. At $k=40$, ActFlow keeping five singular directions raises the mean held-out ARC-Easy accuracy over the six locked models from $0.05$ to $0.85$, against $0.88$ for fine-tuning and $0.92$ for the honest models. Furthermore, it scores higher than the smallest-norm rule in 16 of the 18 combinations of locked model and $k$, and its steering direction is nearly orthogonal to the honest difference-in-means direction. It also unlocks two LoRA locks where the honest direction fails.
Tags
Links
- Source: https://arxiv.org/abs/2609.32530v1
- Canonical: https://arxiv.org/abs/2609.32530v1
Trouble viewing inline? Open PDF directly →
Full Text
82,598 characters extracted from source content.
Expand or collapse full text
Activation Flow: Manufacturing Activations for Steering Hong Kiat Tan Linh Le David Williams-King Affiliation: University of California, Los Angeles Lida Safety ERA Abstract Difference-in-means steering requires activations recorded while a model shows the desired behavior, which a sandbagging model withholds by deliberately underperforming. We introduce Activation Flow (ActFlow), which manufactures these activations from k correct labels without fine-tuning. ActFlow sets target logits that rank each labeled item’s correct answer first, and moves the logits toward them by adding one vector x to all k residual streams at one layer. ActFlow is a family of ordinary differential equations for x, one for each rule that maps the required logit change to the velocity of x. The smallest-norm rule lands exactly on the targets, while the others keep only the top singular directions of the Jacobian. We test ActFlow on three instruction-tuned models, each locked by a sandbagging prompt and by a password-locked LoRA. At k=40k=40, ActFlow keeping five singular directions raises the mean held-out ARC-Easy accuracy over the six locked models from 0.050.05 to 0.850.85, against 0.880.88 for fine-tuning and 0.920.92 for the honest models. Furthermore, it scores higher than the smallest-norm rule in 16 of the 18 combinations of locked model and k, and its steering direction is nearly orthogonal to the honest difference-in-means direction. It also unlocks two LoRA locks where the honest direction fails. Our code is available at https://github.com/HK-Tan/actflow. 1 Introduction Capability evaluations estimate what a model can do by asking it to do a task, and deployment decisions rely on these estimates. A model that sandbags deliberately underperforms on such an evaluation, so the evaluation underestimates its capability (van der Weij et al., 2025). Evaluating such a model requires eliciting the capability it hides. Fine-tuning on correct answers elicits the hidden capability of password-locked models, which perform well only when prompted with a password (Greenblatt et al., 2024), and Hofstätter et al. (2025) recommend fine-tuning wherever possible. Reference grafting elicits the hidden capability at inference time, with the weights unchanged. Like most steering methods (Turner et al., 2023; Rimsky et al., 2024; Zou et al., 2023), it takes a direction as the difference of mean activations recorded with and without an honest prompt or the password. It then sets the residual-stream coordinate along this direction to its honest mean. Grafted at a few layers chosen by active learning, it matches fine-tuning on password-locked models (Le et al., 2026), and at a single layer it recovers the capability in most prompted and fine-tuned locks (Tan et al., 2026). Both the direction and the coordinate thus come from an honest condition, which an evaluator of a sandbagging model lacks even with the answer key. We introduce Activation Flow (ActFlow), which manufactures the activations of the reference graft from k correct labels and the locked model’s own activations, without fine-tuning (Figure 1). ActFlow sets target logits that rank the correct answer of each labeled item first. It then moves the logits toward these targets by adding one vector x to the residual streams of all k items at one layer. ActFlow assumes white-box access to the locked model, which it needs to compute Jacobians. Besides the k labeled items, it uses the locked model’s residual streams on unlabeled items of the task. We test it on four-choice questions with Qwen2.5-7B-Instruct, Llama-3-8B-Instruct and Mistral-7B-Instruct-v0.2, each locked by a sandbagging prompt and by a password-locked LoRA (Tan et al., 2026), which gives six locked models. In these model organisms the honest condition is known, so we report the honest accuracy and the honest graft as references, although ActFlow uses neither. Our contributions are as follows. • We define ActFlow as a family of ordinary differential equations for x, one for each rule that maps the required logit change to its velocity x˙ x (Section 3). The smallest-norm rule, ActFlowfull, lands exactly on the target logits, and the others keep only the top singular directions of the stacked Jacobian, which needs full row rank. This holds generically if it holds at one x (Section 3.5). • ActFlow can also elicit the hidden capability without honest activations (Section 5.1). At k=40k=40, the mean ARC-Easy accuracy over the six locked models rises from 0.050.05 to 0.850.85 with ActFlow5, which keeps five singular directions, against 0.880.88 for fine-tuning and 0.920.92 for the honest models. • ActFlow5 beats the exact landing of ActFlowfull on ARC-Easy in 16 of 18 rows of Table 2, and one x for all k items transfers better than one x per item on both Qwen locks (Sections 5.2–5.3). • ActFlow unlocks the locked models with a grafting direction nearly orthogonal to the harvested one, with median |cos|| | 0.050.05 (Section 5.4). ActFlow5 comes close to the honest graft on three locked models and unlocks two LoRA locks where the honest graft fails. 2 Related work Sandbagging and capability elicitation. Sandbagging organisms have been built by prompting and fine-tuning (van der Weij et al., 2025) and by password locking, where fine-tuning on correct demonstrations elicits the locked capability (Greenblatt et al., 2024). Other elicitation methods include fine-tuning on weak demonstrations followed by reinforcement learning (Ryd et al., 2026), noise injected into the weights (Tice et al., 2025), rank-one adapters found by a tensor decomposition (Mack et al., 2026), removing or reflecting a difference-in-means coordinate (Konrad et al., 2026), and grafting honest activations (Le et al., 2026; Tan et al., 2026). ActFlow keeps their graft and manufactures activations from the locked model and correct labels without fine-tuning. Steering without a contrast condition. Methods that need no second condition include one-shot optimized steering vectors, fitted to one example by gradient descent on the log-probability of target tokens (Dunefsky and Cohan, 2025), and FishBack (Wang et al., 2026), which moves one context toward a target concept probability by minimum-norm steps under a pulled-back Fisher metric. The Jacobian lens (Gurnee et al., 2026) reads activations as logits through a Jacobian of the later layers averaged over prompts. ActFlow uses the Jacobians of k items and fits one shift shared by them, since its shift for one multiple-choice item does not transfer to new items (Section 5.3). Interpreting steering directions. Steering vectors that change a behavior can classify it poorly (Wattenberg and Viégas, 2024; Dunefsky and Cohan, 2025), move the residual stream to points that almost surely no prompt produces (Mishra et al., 2026), and at one layer are not identifiable from behavior to first order (Venkatesh and Kurapath, 2026), meaning that different vectors give the same first-order change in behavior. Section 5.4 measures one instance. Redundancy resolution. A robot arm with more joints than its task needs has infinitely many joint velocities that give the same velocity of the hand. Resolved motion rate control picks one of them with a pseudo-inverse of the Jacobian (Whitney, 1969), and Liégeois (1977) adds a velocity in the kernel of the Jacobian that lowers a second cost. In ActFlow, the velocities that solve (3) form an affine space of dimension d−qkd-qk. The rule R of Definition 1 sets the velocity, for example with the pseudo-inverse of the Jacobian, which solves (3), or with its truncated SVD (Hansen, 1987). 3 Activation Flow 3.1 Setup (a) ActFlow: manufacturing steering directions Logit spaceLayer ℓ iγi(t) _i(t)AγS,i _S,i×BγH,i _H,i✓ remaining layers from layer ℓ to the logits, item i FS,ℓ,i:ℝd→ℝqF_S, ,i:R^d ^q,FS,ℓ,i(hS,ℓ,i)=γS,iF_S, ,i(h_S, ,i)= _S,ihS,ℓ,ih_S, ,ih¯S,ℓ h_S, h¯ℓ∗ h^*_ manufacturedhℓ,i∗h^*_ ,iδ (t)x(t)activations at the last token position No password in the prompt, and we do not have one. Question 1Question 2⋮ kkno passwordquestions (b) Diff-in-means: harvesting steering directions Question iiAγS,i _S,i×BγH,i _H,i✓ ,ℓ,iF_S, ,iFH,ℓ,iF_H, ,ihS,ℓ,ih_S, ,ih¯S,ℓ h_S, h¯H,ℓ h_H, harvestedhH,ℓ,ih_H, ,iδ=h¯H,ℓ−h¯S,ℓδ= h_H, - h_S, last token position No password in the prompt. Password: xk9q. Question 1Question 2⋮ kkQuestion 1Question 2⋮ kkno passwordpasswordquestionsquestions Figure 1: Steering directions manufactured by ActFlow (left) and harvested by difference-in-means (right). Bottom: the prompts. Middle: the residual stream at layer ℓ at the last token position, with sandbagging activations in red, honest or manufactured activations in green, means marked ++, and manufactured points dashed. Top: the logit space ℝqR^q of item i, where × marks logits with a wrong answer ranked first and ✓ logits with the correct answer ranked first. Left: one shift x(t)x(t) shared by all k activations follows the logit paths γi(t) _i(t) to the targets (✓ ), and δ=x(1)δ=x(1). Right: δ is the difference of the means of the runs with and without the password. Consider a sandbagging model organism built from a decoder-only transformer with nLn_ L layers, indexed 1,…,nL1,…,n_ L, and a residual stream in ℝdR^d. An evaluation item i is a multiple-choice question with q candidate answers, each read off as the logit of one letter token. In the sandbagging condition S, let hS,ℓ,i∈ℝdh_S, ,i ^d be the residual stream after layer ℓ at the last prompt position of item i, and let γS,i∈ℝq _S,i ^q be the logit vector of that run, whose a-th entry is the logit of the a-th candidate answer. For each layer ℓ , we define the logit map FS,ℓ,i:ℝd→ℝqF_S, ,i:R^d ^q as the rest of the forward pass after layer ℓ , read at the last prompt position. It takes a residual stream vector at that position, runs layers ℓ+1,…,nL +1,…,n_ L and the readout, and returns the logit vector for item i, that is, FS,ℓ,i(hS,ℓ,i)=γS,iF_S, ,i(h_S, ,i)= _S,i. The target γH,i∈ℝq _H,i ^q is a logit vector with the correct answer ranked first. It keeps every wrong-answer logit of γS,i _S,i and sets the correct-answer logit to 55 above the largest wrong-answer logit. The target is therefore built from the sandbagging logits and the correct label alone. The subscript H marks the logits we want the model to produce. Figure 1 (left) draws these objects. 3.2 How ActFlow works Fix a layer ℓ and a draw K of k labeled items, numbered 1,…,k1,…,k. Let FS,ℓ,K:ℝd→ℝqkF_S, ,K:R^d ^qk be the shared-shift logit map, with Jacobian JS,ℓ,K:ℝd→ℝqk×dJ_S, ,K:R^d ^qk× d, given by FS,ℓ,K(x)=(FS,ℓ,1(hS,ℓ,1+x)FS,ℓ,k(hS,ℓ,k+x)),JS,ℓ,K(x)=∂FS,ℓ,K∂x(x)=(JS,ℓ,1(hS,ℓ,1+x)JS,ℓ,k(hS,ℓ,k+x)),F_S, ,K(x)= pmatrixF_S, ,1(h_S, ,1+x)\\ \\ F_S, ,k(h_S, ,k+x) pmatrix, J_S, ,K(x)= ∂ F_S, ,K∂ x(x)= pmatrixJ_S, ,1(h_S, ,1+x)\\ \\ J_S, ,k(h_S, ,k+x) pmatrix, (1) where JS,ℓ,i=∂FS,ℓ,i/∂h∈ℝq×dJ_S, ,i=∂ F_S, ,i/∂ h ^q× d is the Jacobian of item i. The same vector x is added to the residual stream of every item, and the logit map of item i is read at hS,ℓ,i+xh_S, ,i+x. Stacking the per-item logit vectors of Section 3.1 in the same way gives the stacked sandbagging logits and targets, γS,K=(γS,1γS,k)=FS,ℓ,K(0)∈ℝqk,γH,K=(γH,1γH,k)∈ℝqk. _S,K= pmatrix _S,1\\ \\ _S,k pmatrix=F_S, ,K(0) ^qk, _H,K= pmatrix _H,1\\ \\ _H,k pmatrix ^qk. (2) We prescribe a piecewise C1C^1 path γK:[0,1]→ℝqk _K:[0,1] ^qk in the logit space from γS,K _S,K to γH,K _H,K, the logit path. We state the results for a general logit path, and use the linear logit path γK(t)=γS,K+tΔγ _K(t)= _S,K+t\, γ, with Δγ=γH,K−γS,K γ= _H,K- _S,K, as a running example. Its velocity is constant, γ˙K≡Δγ γ_K≡ γ. We look for a curve of shared shifts x(t)x(t) with x(0)=0x(0)=0 in the residual stream whose logits follow the logit path, FS,ℓ,K(x(t))=γK(t)F_S, ,K(x(t))= _K(t). For each t, the shared shifts that produce the logits γK(t) _K(t) form the level set FS,ℓ,K−1(γK(t))F_S, ,K^-1( _K(t)), which we call the fibre over γK(t) _K(t). We work at points where JS,ℓ,KJ_S, ,K has full row rank qkqk, that is, on the set where FS,ℓ,KF_S, ,K is a submersion, which by Section 3.5 holds at almost every point once it holds at one. There the fibre is a smooth submanifold of the residual stream ℝdR^d of dimension d−qkd-qk (Lee, 2013, Corollary 5.13), and in all our models qkqk is much smaller than d. Hence, the logit path only fixes which fibre x(t)x(t) lies on at each t, and not which point of that fibre x(t)x(t) is. Figure 2 illustrates these fibres. (a) Lift, R=J+R=J^+ Logit spaceLayer ℓ ,ℓ,KF_S, ,Kshared-shiftlogit mapγS,K _S,KγK(0) _K(0)γK(12) _K( 12)γK(12+ε) _K( 12+ )γH,K _H,KγK(1) _K(1)FS,ℓ,K−1(γK(0))F_S, ,K^-1 ( _K(0) )FS,ℓ,K−1(γK(12))F_S, ,K^-1 ( _K( 12) )FS,ℓ,K−1(γK(12+ε))F_S, ,K^-1 ( _K( 12+ ) )FS,ℓ,K−1(γK(1))F_S, ,K^-1 ( _K(1) )x(t)x(t)x˙2 x_2x˙1 x_1x˙3 x_3x˙min x_ x(0)x(0)x(12)x( 12)x(1)x(1)x(12+ε)x( 12+ ) (b) Partial lift, R=Jm+R=J_m^+ e(1)e(1)γS,K _S,KγK(12) _K( 12)γH,K _H,KF(x(12))F(x( 12))F(x(1))F(x(1))FS,ℓ,K−1(γK(12))F_S, ,K^-1 ( _K( 12) )FS,ℓ,K−1(γH,K)F_S, ,K^-1 ( _H,K )FS,ℓ,K−1(F(x(12)))F_S, ,K^-1 (F(x( 12)) )FS,ℓ,K−1(F(x(1)))F_S, ,K^-1 (F(x(1)) )x(t)x(t)x(0)x(0)x(1)x(1)x(12)x( 12) Figure 2: Fibres of the shared-shift logit map. Top: the logit space. Bottom: the residual stream at layer ℓ , where each coloured curve is the fibre over the point of the same colour above. (a) The lift (5). At x(12)x( 12) the velocities x˙1 x_1, x˙2 x_2, x˙3 x_3 and x˙min x_ all solve (3), so ε times each reaches the fibre over γK(12+ε) _K( 12+ ) to first order. The lift from (5) takes the smallest, x˙min x_ . (b) A partial lift with R=Jm+R=J_m^+ (Section 3.3). Its logits F(x(t))F(x(t)) (brown) leave the logit path (dashed), so x(t)x(t) crosses the brown fibres instead of the dashed ones and misses γH,K _H,K by the landing error e(1)e(1) of (10). By the chain rule, at every t, a curve with FS,ℓ,K(x(t))=γK(t)F_S, ,K(x(t))= _K(t) satisfies JS,ℓ,K(x(t))x˙(t)=γ˙K(t).J_S, ,K(x(t))\, x(t)= γ_K(t). (3) Since γ˙K(t) γ_K(t) is given by the prescribed logit path (the constant vector Δγ γ for the linear logit path above), (3) is a linear system for the velocity x˙(t) x(t) at the current point x(t)x(t). Once a solution is chosen at every point, integrating it from x(0)=0x(0)=0 gives the curve x(t)x(t). The system has qkqk equations in d unknowns, so it is underdetermined, which is expected from the d−qkd-qk free directions of the fibre. If x˙0 x_0 is one solution to (3), then the set of all solutions to (3) is given by x˙∈ℝd:JS,ℓ,K(x)x˙=γ˙K(t)=x˙0+kerJS,ℓ,K(x), \ x ^d:J_S, ,K(x)\, x= γ_K(t) \= x_0+ J_S, ,K(x), (4) where kerJS,ℓ,K(x) J_S, ,K(x) has dimension d−qkd-qk and contains the velocities that change none of the qkqk logits to first order. Hence, of the many possible x˙ x, we must set up a rule to pick a unique one at every point. For instance, we take the solution of (3) of smallest Euclidean norm, which is x˙=JS,ℓ,K(x)+γ˙K x=J_S, ,K(x)^+ γ_K, where + is the Moore–Penrose pseudo-inverse. This gives the equation of motion x˙=JS,ℓ,K(x)+γ˙K,x(0)=0. x=J_S, ,K(x)^+\, γ_K, x(0)=0. (5) At every point it moves the k residual streams along the shortest shared direction that produces the logit velocity γ˙K γ_K, and it re-reads the Jacobian at every point of the curve. We integrate (5) up to t=1t=1 and obtain the steering direction δ=x(1)−x(0)=x(1)δ=x(1)-x(0)=x(1). For the remainder of this paper, we drop the subscripts S,ℓ,KS, ,K from F and J when no confusion is possible, and along a curve we write J for J(x(t))J(x(t)). Other conditions on x˙ x result in different equations of motion. Each of them picks, at every point, a matrix that maps the logit velocity γ˙K γ_K to the velocity x˙ x. Definition 1 (Activation Flow). Let R be a map from an open set of Jacobians J∈ℝqk×dJ ^qk× d of full row rank qkqk to matrices R(J)∈ℝd×qkR(J) ^d× qk. The Activation Flow with rule R is the initial value problem x˙=R(JS,ℓ,K(x))γ˙K,x(0)=0, x=R (J_S, ,K(x) )\, γ_K, x(0)=0, (6) and Activation Flow is the family of these initial value problems over all such maps R. The rule R(J)=J+R(J)=J^+ is the smallest-norm rule (5), which we call full-rank ActFlow. In differential geometry, (kerJ(x))⟂( J(x)) is an Ehresmann connection in the modern sense (Ehresmann, 1951) on the submersion F, which is a smooth choice of a complement HxH_x to kerJ(x) J(x) at every point (Kolář et al., 1993, §8.13). The solution of (5) is the horizontal lift of γK _K for this connection. We call it the lift, and we call the solutions of (6) for the other rules in Table 1 partial lifts. 3.3 The spectral picture For any x∈ℝdx ^d, write the singular value decomposition (SVD) of the Jacobian as JS,ℓ,K(x)=∑a=1qkσa(x)ua(x)va(x)⊤,σ1≥⋯≥σqk≥0,J_S, ,K(x)= _a=1^qk _a(x)\,u_a(x)\,v_a(x) , _1≥…≥ _qk≥ 0, (7) with ua∈ℝqku_a ^qk and va∈ℝdv_a ^d orthonormal. Moving the shift by one unit along vav_a changes the qkqk logits by σa _a along uau_a to first order, so σa _a is the change in the logits per unit shift along vav_a. All the rules in this paper send each uau_a to a multiple of vav_a that depends only on σa _a. For a scalar function φ on (0,∞)(0,∞), the spectral rule Rφ(J)=∑a=1qkφ(σa)vaua⊤R_ (J)= _a=1^qk ( _a)\,v_a\,u_a (8) sends uau_a to φ(σa)va ( _a)\,v_a. Hence, along the flow, the logit velocity is given by dtF(x(t))=Jx˙=JRφ(J)γ˙K=∑a=1qkσaφ(σa)⟨ua,γ˙K⟩ua, ddtF(x(t))=J x=JR_ (J)\, γ_K= _a=1^qk _a ( _a)\, u_a, γ_K \,u_a, (9) so for each a the component of the logit velocity along uau_a is σaφ(σa) _a ( _a) times that of γ˙K γ_K. Next, we consider a few instances of the spectral rule (8), listed in Table 1. From Section 3.2, full-rank ActFlow, ActFlowfull, takes Rφ(J)=J+R_ (J)=J^+, which gives the lift (5) and JRφ(J)=IJR_ (J)=I. We also consider partial lifts that cut off the small singular values. They take Rφ(J)=Jm+=∑a≤mvaua⊤/σaR_ (J)=J_m^+= _a≤ mv_au_a / _a, the pseudo-inverse truncated to the top m singular directions, which gives JRφ(J)=Pm=∑a≤muaua⊤JR_ (J)=P_m= _a≤ mu_au_a , the orthogonal projector onto the span of u1,…,umu_1,…,u_m. ActFlowm fixes m in advance, and ActFlow0.1 takes m=#a:σa>0.1σ1m=\#\a: _a>0.1\, _1\. ActFlowPR takes m=roundPR(σ)m=roundPR(σ), where PR(σ)=(∑aσa2)2/∑aσa4PR(σ)=( _a _a^2)^2/ _a _a^4 is the participation ratio of the spectrum (Gao et al., 2017), so it needs neither a fixed m nor a fixed ratio. For ActFlow0.1 and ActFlowPR, m is recomputed from JS,ℓ,K(x(t))J_S, ,K(x(t)) along the flow. name Rφ(J)R_ (J) JRφ(J)J\,R_ (J) φ(σa) ( _a) ActFlowfull (the lift) J+J^+ I 1/σa1/ _a ActFlowm Jm+J_m^+, m fixed PmP_m [σa≥σm]/σa1[ _a≥ _m]/ _a ActFlow0.1 Jm+J_m^+, m=#a:σa>0.1σ1m=\#\a: _a>0.1\, _1\ PmP_m [σa>0.1σ1]/σa1[ _a>0.1\, _1]/ _a ActFlowPR Jm+J_m^+, m=roundPR(σ)m=roundPR(σ) PmP_m [σa≥σm]/σa1[ _a≥ _m]/ _a Table 1: The rules used in this paper. Each is a spectral rule (8), and JRφ(J)JR_ (J) is used in (9). By (8), the lift moves the residual stream by ⟨ua,γ˙K⟩/σa u_a, γ_K / _a along vav_a. So the smaller σa _a is, the further the lift moves the residual stream along vav_a per unit of γ˙K γ_K along uau_a. We hypothesize that the directions vav_a with small σa _a are specific to the k items in the draw and do not carry over to other items. The partial lifts set φ(σa)=0 ( _a)=0 for a>ma>m, so they do not move the residual stream along these directions, and Section 5 compares them with the lift on held-out items. By the chain rule and (6), the landing error e(t)=γK(t)−F(x(t))e(t)= _K(t)-F(x(t)) satisfies e(t)=∫0t(I−JR(J))(x(s))γ˙K(s)s.e(t)= _0^t (I-JR(J) )(x(s))\, γ_K(s)\,ds. (10) For the lift, R(J)=J+R(J)=J^+ and JR(J)=IJR(J)=I (Table 1), so e(t)=0e(t)=0 for every logit path. For a partial lift with m<qkm<qk, JR(J)=Pm≠IJR(J)=P_m≠ I. Hence, in general, e(1)e(1) in (10) is nonzero for a partial lift, and we do not expect it to land at γH,K _H,K. Figure 2(b) draws e(1)e(1) for Jm+J_m^+. 3.4 Discretizations Fix a step count N and step times tn=n/Nt_n=n/N for n=0,…,Nn=0,…,N, and write Jn−1=JS,ℓ,K(xn−1)J_n-1=J_S, ,K(x_n-1). We consider two schemes for (6) (Algorithm 1), both started from x0=0x_0=0, as follows. • Euler scheme, xn=xn−1+R(Jn−1)(γK(tn)−γK(tn−1))x_n=x_n-1+R(J_n-1) ( _K(t_n)- _K(t_n-1) ), • and with Gauss–Newton correction, xn=xn−1+R(Jn−1)(γK(tn)−FS,ℓ,K(xn−1))x_n=x_n-1+R(J_n-1) ( _K(t_n)-F_S, ,K(x_n-1) ). The Euler scheme is a discretization of (6) for every rule. The corrected scheme adds R(Jn−1)(γK(tn−1)−FS,ℓ,K(xn−1))R(J_n-1) ( _K(t_n-1)-F_S, ,K(x_n-1) ) to the Euler step. We also compare ActFlow with finding the shift x directly by gradient descent (GD) on the squared logit error 12‖γH,K−FS,ℓ,K(x)‖2 12\| _H,K-F_S, ,K(x)\|^2. From x0=0x_0=0 with step size η, GD takes xn=xn−1+ηJn−1⊤(γH,K−FS,ℓ,K(xn−1))x_n=x_n-1+η\,J_n-1 ( _H,K-F_S, ,K(x_n-1) ), which is analogous to the Gauss–Newton correction of ActFlow with R(J)=ηJ⊤R(J)=η J and γK(tn)=γH,K _K(t_n)= _H,K. Algorithm 1 ActFlow with a spectral rule RφR_ . 1: residual streams hS,ℓ,1,…,hS,ℓ,kh_S, ,1,…,h_S, ,k at layer ℓ , logit path γK _K, function φ , step count N, scheme (Euler or with Gauss–Newton correction) 2: x0←0∈ℝdx_0← 0 ^d 3: for n=1,…,Nn=1,…,N, with tn=n/Nt_n=n/N do 4: compute FS,ℓ,K(xn−1)F_S, ,K(x_n-1) and Jn−1=JS,ℓ,K(xn−1)J_n-1=J_S, ,K(x_n-1) from layer ℓ onward 5: r←γK(tn)−γK(tn−1)r← _K(t_n)- _K(t_n-1) (Euler), or r←γK(tn)−FS,ℓ,K(xn−1)r← _K(t_n)-F_S, ,K(x_n-1) (Gauss–Newton) 6: xn←xn−1+Rφ(Jn−1)rx_n← x_n-1+R_ (J_n-1)\,r 7: end for 8: return hℓ,i∗=hS,ℓ,i+xNh^*_ ,i=h_S, ,i+x_N for i=1,…,ki=1,…,k 3.5 When the flow is defined Sections 3.2–3.4 assume that JS,ℓ,K(x)J_S, ,K(x) has full row rank qkqk, that is, that FS,ℓ,KF_S, ,K is a submersion at x. To verify this, we rely on Lemma 1 below (proof in Appendix A), which shows that for a real-analytic map the rank condition fails either everywhere or on a closed set of measure zero. Lemma 1. Let G:ℝd→ℝpG:R^d ^p be real-analytic, and let Z be the set of points x at which DG(x)DG(x) has rank less than p. Then either Z=ℝdZ=R^d, or Z is closed and has Lebesgue measure zero. For our three models, FS,ℓ,KF_S, ,K is real-analytic on ℝdR^d at every layer ℓ and for every draw K, as a composition of the real-analytic blocks RMSNorm with ϵ>0ε>0, attention with rotary position embeddings and softmax, and the SwiGLU MLP. Hence, by Lemma 1, if JS,ℓ,KJ_S, ,K has full row rank qkqk at the start point x=0x=0, then FS,ℓ,KF_S, ,K is a submersion at every point outside a closed set of measure zero. We verify (Appendix B.2) that rankJS,ℓ,K(0)=qk J_S, ,K(0)=qk in every run below the last layer. Hence, by the standard existence, uniqueness and extension theorems for ODEs, the flow (6) is well defined up to t=1t=1 as long as x(t)x(t) stays bounded and away from this set (Appendix A). For k≥2k≥ 2, the last layer is in the first case of the lemma. There the residual stream reaches the logits only through the final RMSNorm and the unembedding, so rankJS,ℓ,K≤q+k J_S, ,K≤ q+k everywhere, which is below qkqk whenever (q−1)(k−1)>1(q-1)(k-1)>1 (Appendix A). There, Algorithm 1 with R(J)=J+R(J)=J^+ does not land at γH,K _H,K in any of our runs with k≥4k≥ 4 (Section 5.5). 4 Experiments Locked models. We use Qwen2.5-7B-Instruct (Qwen Team, 2025), Llama-3-8B-Instruct (Grattafiori et al., 2024) and Mistral-7B-Instruct-v0.2 (Jiang et al., 2023). Each model is locked in two ways. The prompt lock adds an instruction to answer incorrectly to the user turn. The LoRA lock is the password-locked adapter of Tan et al. (2026), which answers incorrectly unless a password is provided. Appendix B.1 gives the locked and honest prompts, model details and accuracies. Data. The labeled items come from a pool U of 80 ARC-Easy validation questions (Clark et al., 2018), each with q=4q=4 answer letters. For k∈1,4,10,40k∈\1,4,10,40\ we split U into 80, 20, 8 and 2 disjoint draws of k items. The test sets are 100 ARC-Easy and 100 OpenBookQA test questions (Mihaylov et al., 2018). No method uses any OpenBookQA item, so the OpenBookQA accuracy measures transfer to a second dataset. Test accuracies reported for each k are averaged over the draws. The reference graft. Reference grafting (Le et al., 2026) is a form of steering. Its grafting direction wℓw_ at one layer ℓ is the unit steering direction from the mean locked residual stream to the mean honest one. The graft sets the coordinate of the residual stream along wℓw_ to the honest mean’s coordinate τℓ _ at every token position. ActFlow keeps this graft and replaces the honest residual streams with the manufactured ones hℓ,i∗h^*_ ,i of Algorithm 1. With h¯ℓ,K∗ h^*_ ,K the mean of hℓ,i∗h^*_ ,i over the draw K and h¯S,ℓ,U h_S, ,U the mean of hS,ℓ,ih_S, ,i over the pool U, the graft is wℓ=h¯ℓ,K∗−h¯S,ℓ,U‖h¯ℓ,K∗−h¯S,ℓ,U‖,τℓ=⟨h¯ℓ,K∗,wℓ⟩,h↦h−⟨h,wℓ⟩wℓ+τℓwℓ,w_ = h^*_ ,K- h_S, ,U\| h^*_ ,K- h_S, ,U\|, _ = h^*_ ,K,w_ , h\; \;h- h,w_ w_ + _ w_ , (11) applied to the residual stream h after layer ℓ at every token position of a new prompt. We run this at every layer and keep the layer ℓ whose graft gives the highest accuracy on the k labeled items. The honest graft uses the honest residual streams of the same k items in place of hℓ,i∗h^*_ ,i. Methods. On every locked model we run ActFlow5, ActFlowPR and ActFlowfull (Table 1) with the Gauss–Newton correction, N=40N=40 steps and the linear logit path, and results use these settings unless noted. On the two Qwen locks we also run ActFlow0.1, Euler steps, N∈1,8N∈\1,8\ and the sequential path, which moves the logits of one item at a time. For comparison we run GD (Section 3.4) and LoRA fine-tuning (FT) (Hu et al., 2022) on the k labeled items of each draw (Greenblatt et al., 2024), and the honest graft. Only the honest graft reads activations of the honest model. ActFlow and GD use the graft (11) and the rule for ℓ . Appendices B.2 and B.3 list all settings. 5 Results This section answers the following four questions. (Section 5.1) Does ActFlow unlock the locked models, and how close does it come to fine-tuning? (Section 5.2) Does ActFlowfull, which lands exactly on the k items, score highest on new items? (Section 5.3) Does it matter that one shift is fitted to all k items at once? (Section 5.4) Is the manufactured grafting direction close to that harvested from honest activations? We then compare the two schemes, step counts and paths, and check the rank condition (Section 5.5). Table 2: Held-out accuracy with k labeled items on 100 ARC-Easy and 100 OpenBookQA test items. Each entry is the mean over disjoint draws, and subscripts are standard errors over the draws. Honest graft uses the honest residual streams of the same k items in the graft (11). FT is LoRA fine-tuning on the same k items, and AF is ActFlow. In each row and test set, excluding honest graft, bold marks the highest accuracy and underline the second highest. ARC-Easy OpenBookQA model lock k locked honest honest graft FT AF5 AFPR AFfull GD honest FT AF5 AFPR AFfull GD Qwen2.5-7B prompt 4 .04 .96 .93±.01.93_±.01 .93±.01.93_±.01 .73±.06.73_±.06 .74±.06.74_±.06 .53±.04.53_±.04 .81¯±.03 .81_±.03 .88 .78±.02.78_±.02 .54±.04.54_±.04 .55±.04.55_±.04 .37±.03.37_±.03 .59¯±.03 .59_±.03 10 .94±.01.94_±.01 .95±.01.95_±.01 .92¯±.02 .92_±.02 .88±.03.88_±.03 .58±.06.58_±.06 .87±.03.87_±.03 .82±.02.82_±.02 .70¯±.03 .70_±.03 .65±.03.65_±.03 .38±.05.38_±.05 .66±.04.66_±.04 40 .96±.00.96_±.00 .96±.01.96_±.01 .95¯±.00 .95_±.00 .95¯±.01 .95_±.01 .63±.07.63_±.07 .83±.07.83_±.07 .84±.05.84_±.05 .72¯±.03 .72_±.03 .70±.04.70_±.04 .37±.04.37_±.04 .61±.05.61_±.05 LoRA 4 .00 .94 .95±.00.95_±.00 .77±.05.77_±.05 .32±.08.32_±.08 .03±.01.03_±.01 .54±.05.54_±.05 .61¯±.05 .61_±.05 .84 .68±.05.68_±.05 .35±.07.35_±.07 .08±.02.08_±.02 .54±.04.54_±.04 .62¯±.05 .62_±.05 10 .91±.04.91_±.04 .95±.01.95_±.01 .61±.15.61_±.15 .10±.08.10_±.08 .78±.07.78_±.07 .83¯±.04 .83_±.04 .84±.01.84_±.01 .57±.11.57_±.11 .18±.08.18_±.08 .69±.05.69_±.05 .75¯±.03 .75_±.03 40 .95±.00.95_±.00 .95±.02.95_±.02 .93¯±.03 .93_±.03 .37±.34.37_±.34 .92±.03.92_±.03 .93¯±.02 .93_±.02 .85±.03.85_±.03 .83¯±.01 .83_±.01 .38±.34.38_±.34 .78±.04.78_±.04 .81±.05.81_±.05 Llama-3-8B prompt 4 .14 .93 .85±.03.85_±.03 .84±.04.84_±.04 .83¯±.04 .83_±.04 .79±.04.79_±.04 .74±.04.74_±.04 .82±.02.82_±.02 .78 .62±.03.62_±.03 .55¯±.03 .55_±.03 .52±.04.52_±.04 .48±.03.48_±.03 .53±.02.53_±.02 10 .90±.00.90_±.00 .90¯±.01 .90_±.01 .91±.01.91_±.01 .89±.02.89_±.02 .82±.02.82_±.02 .83±.02.83_±.02 .69±.01.69_±.01 .63¯±.02 .63_±.02 .62±.02.62_±.02 .53±.02.53_±.02 .56±.03.56_±.03 40 .91±.00.91_±.00 .92±.00.92_±.00 .92±.00.92_±.00 .91¯±.00 .91_±.00 .71±.03.71_±.03 .88±.01.88_±.01 .73±.02.73_±.02 .66¯±.01 .66_±.01 .65±.01.65_±.01 .47±.01.47_±.01 .63±.01.63_±.01 LoRA 4 .02 .89 .91±.00.91_±.00 .57±.07.57_±.07 .78¯±.05 .78_±.05 .79±.05.79_±.05 .52±.06.52_±.06 .75±.04.75_±.04 .73 .44±.05.44_±.05 .59±.03.59_±.03 .59±.03.59_±.03 .43±.04.43_±.04 .58¯±.03 .58_±.03 10 .86±.05.86_±.05 .82¯±.03 .82_±.03 .81±.06.81_±.06 .85±.02.85_±.02 .49±.09.49_±.09 .70±.06.70_±.06 .61¯±.03 .61_±.03 .61¯±.05 .61_±.05 .62±.03.62_±.03 .41±.06.41_±.06 .50±.05.50_±.05 40 .91±.00.91_±.00 .91±.01.91_±.01 .88±.02.88_±.02 .90¯±.02 .90_±.02 .68±.02.68_±.02 .73±.05.73_±.05 .71±.02.71_±.02 .71±.05.71_±.05 .70¯±.01 .70_±.01 .63±.03.63_±.03 .50±.06.50_±.06 Mistral-7B prompt 4 .06 .88 .70±.02.70_±.02 .71±.04.71_±.04 .54±.04.54_±.04 .42±.05.42_±.05 .40±.04.40_±.04 .62¯±.04 .62_±.04 .75 .58±.03.58_±.03 .47±.03.47_±.03 .40±.04.40_±.04 .37±.03.37_±.03 .52¯±.03 .52_±.03 10 .75±.01.75_±.01 .84±.01.84_±.01 .60±.05.60_±.05 .54±.08.54_±.08 .23±.02.23_±.02 .62¯±.09 .62_±.09 .70±.02.70_±.02 .53¯±.04 .53_±.04 .48±.06.48_±.06 .22±.01.22_±.01 .53¯±.05 .53_±.05 40 .75±.01.75_±.01 .88±.03.88_±.03 .69±.09.69_±.09 .72¯±.07 .72_±.07 .32±.04.32_±.04 .55±.30.55_±.30 .74±.01.74_±.01 .57¯±.11 .57_±.11 .54±.07.54_±.07 .30±.04.30_±.04 .40±.23.40_±.23 LoRA 4 .02 .91 .02±.00.02_±.00 .20±.02.20_±.02 .31±.04.31_±.04 .23±.04.23_±.04 .29¯±.02 .29_±.02 .31±.03.31_±.03 .78 .23±.02.23_±.02 .28¯±.03 .28_±.03 .24±.03.24_±.03 .28¯±.02 .28_±.02 .29±.02.29_±.02 10 .02±.00.02_±.00 .30±.08.30_±.08 .46¯±.09 .46_±.09 .53±.07.53_±.07 .39±.07.39_±.07 .44±.06.44_±.06 .28±.05.28_±.05 .42±.07.42_±.07 .46±.05.46_±.05 .40±.05.40_±.05 .43¯±.06 .43_±.06 40 .02±.00.02_±.00 .70±.23.70_±.23 .72¯±.07 .72_±.07 .78±.04.78_±.04 .42±.22.42_±.22 .53±.02.53_±.02 .54±.18.54_±.18 .60¯±.05 .60_±.05 .63±.01.63_±.01 .42±.09.42_±.09 .49±.01.49_±.01 Figure 3: ARC-Easy test accuracy at the chosen layer ℓ . Top: prompt locks. Bottom: LoRA locks. (a) Accuracy against the number of labeled items k. (b) and (c) show the two Qwen locks. (b) At k=4k=4 and 4040, per-item fits one shift to each of the k items of a draw and shared fits one shift to all k items. (c) ActFlow5 and ActFlowPR at k=10k=10 for different step counts N, schemes and paths, where GN is Gauss–Newton and seq. the sequential path. Error bars are standard errors over draws. Dotted lines mark the honest and locked accuracies, and the dashed line marks chance. 5.1 ActFlow unlocks the models and comes close to fine-tuning Numbers below are mean ARC-Easy test accuracies over the six locked models unless noted. Table 2 lists the accuracy of every method on each locked model, and Figure 3a plots it against k. • ActFlow unlocks the locked models. At k=40k=40, ActFlow5 reaches 0.850.85, against 0.050.05 for the locked models and 0.920.92 for the honest models. • ActFlow comes close to fine-tuning at k=40k=40. FT reaches 0.880.88, and ActFlow5 is within 0.050.05 of FT on every locked model except the Mistral prompt lock (ActFlow5 0.690.69 against FT 0.880.88). • ActFlow matches the honest graft without honest activations. At k=40k=40, ActFlow5 is within 0.070.07 of the honest graft on five locked models. • The accuracy transfers to OpenBookQA. At k=40k=40, ActFlow5 reaches 0.680.68 on OpenBookQA, against 0.730.73 for FT and 0.790.79 for the honest models. • GD, with ActFlow’s target and graft, scores below ActFlow5 at k=40k=40 and above it at k=4k=4. • ActFlow is faster than GD. Summed over the four Llama and Mistral locks, GD took 6.26.2 times as long as one ActFlow rule to fit the shift, even with N=40N=40 (Appendix B.5). • ActFlowPR closely matches ActFlow5 except on the Qwen LoRA lock, where it keeps fewer singular directions than ActFlow5, even at k=40k=40. 5.2 Exact landing on the k items does not give the highest test accuracy • ActFlowfull reaches the target logits on the labeled items. At its chosen layer, it brings all q logits of every labeled item within 0.50.5 of the target in 179 of the 180 draws with k≥4k≥ 4. • ActFlow5 scores higher than ActFlowfull on ARC-Easy in 16 of the 18 rows of Table 2. • ActFlowfull tends to overfit the labeled items. At k=40k=40 its mean labeled and test accuracies (Tables 8 and 2) are 0.920.92 and 0.610.61, respectively, against 0.840.84 and 0.850.85 for ActFlow5, which leaves out the directions vav_a with small σa _a that Section 3.3 hypothesizes are item-specific. 5.3 A shift shared by the k items transfers better than per-item shifts Figure 3b compares one shift shared by the k items with one shift per item. • A shared shift unlocks both Qwen locks, and per-item shifts do not. At k=40k=40, the shared shift of ActFlow5 reaches at least 0.930.93 on both locks, against at most 0.270.27 for its per-item shifts. • A one-item shift memorizes the labeled item’s letter, which accounts for its low test accuracy. On the LoRA lock, one-item ActFlow5 answers 6565 of the 100100 test items with the labeled item’s letter. 5.4 Manufactured and harvested grafting directions are nearly orthogonal We compare the manufactured and harvested directions of Figure 1 at the layer ActFlow chooses. • The two directions are nearly orthogonal for every rule and k. Over the 540 draws of the three rules with k≥4k≥ 4, the median magnitude of their cosine is 0.050.05, against 0.010.01 for a random direction. • Each direction is stable across draws. At k=40k=40 and the layer either draw chooses, the directions of the two draws have cosine 0.710.71 to 0.920.92 for ActFlow5 and at least 0.950.95 for the honest graft. • At the layer ActFlow5 chooses at k=40k=40, the honest graft comes within 0.040.04 of ActFlow5 on the Qwen prompt, Qwen LoRA and Llama prompt locks. • ActFlow unlocks, with a nearly orthogonal direction, two locks where the honest graft fails. At k=40k=40, ActFlow5 reaches 0.720.72 and 0.820.82 on the Mistral LoRA lock and another Qwen LoRA lock (Appendix B.4) with cosine at most 0.050.05, and the honest graft at most 0.030.03 at every layer. 5.5 General properties of the flow Figure 3c plots ActFlow5 and ActFlowPR on both Qwen locks at k=10k=10, and Appendix B.6. • More steps raise the accuracy of ActFlow5 and ActFlowPR on both locks at k=4k=4, 1010 and 4040. On the LoRA lock at k=10k=10, ActFlow5 reaches 0.610.61 with N=40N=40 against 0.010.01 with N=1N=1. • The Gauss–Newton correction raises the accuracy of ActFlow5 and ActFlowPR on both locks at k=4k=4, 1010 and 4040. On the prompt lock at k=10k=10, ActFlow5 reaches 0.920.92 with it and 0.720.72 without. • The path has no consistent effect. Over the two rules, two locks and three values of k, the sequential path scores above the linear path in 6 of the 12 cases and below it in 6. • At the last layer the logit map is not a submersion for k≥2k≥ 2 (Section 3.5) and ActFlowfull lands no item with k≥4k≥ 4 (Table 12). The 32 of 3820 grafts that select it score 0.000.00 to 0.250.25 (chance). 6 Discussion As Section 5.4 shows, the manufactured and harvested directions reach similar accuracy on three of the locked models at the same layer, although they are nearly orthogonal. The k labels fix only the qkqk logits of the labeled items, so the shared shifts that land them form a fibre of dimension d−qk>0d-qk>0 (Section 3), and two shifts that land the same items can be nearly orthogonal. Any rule R(J)=J++M(J)R(J)=J^++M(J) with JM(J)=0JM(J)=0 also lands on the target logits by (10), and the term M(J)M(J) moves x˙ x along the fibre, so such a rule can end at another manufactured direction, possibly one close to the harvested direction. We leave the study of such rules, including whether some of them transfer better to new items than ActFlow5, to future work. 7 Conclusion We introduced ActFlow, a method that builds a steering direction from k correct labels and the locked model’s own activations, without honest activations. It moves the logits of the k items toward target logits with one shared shift of the residual stream. We showed that grafting this shift unlocks the locked models without fine-tuning and comes close to fine-tuning in accuracy. We hope that ActFlow extends to other applications of steering, and serves as an alternative to fine-tuning. References Clark et al. (2018) P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord Think you have solved question answering? Try ARC, the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457. Cited by: §4. Dunefsky and Cohan (2025) J. Dunefsky and A. Cohan One-shot optimized steering vectors mediate safety-relevant behaviors in LLMs. In Conference on Language Modeling, Cited by: §2, §2. Ehresmann (1951) C. Ehresmann Les connexions infinitésimales dans un espace fibré différentiable. In Colloque de topologie (espaces fibrés), Bruxelles, 1950, Liège and Paris, p. 29–55. Cited by: §3.2. Gao et al. (2017) P. Gao, E. Trautmann, B. Yu, G. Santhanam, S. Ryu, K. Shenoy, and S. Ganguli A theory of multineuronal dimensionality, dynamics and measurement. bioRxiv preprint 214262. External Links: Document Cited by: §3.3. Grattafiori et al. (2024) A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783. Cited by: §4. Greenblatt et al. (2024) R. Greenblatt, F. Roger, D. Krasheninnikov, and D. Krueger Stress-testing capability elicitation with password-locked models. In Advances in Neural Information Processing Systems, Vol. 37. Cited by: §B.3, §1, §2, §4. Gurnee et al. (2026) W. Gurnee, N. Sofroniew, A. Pearce, M. Piotrowski, I. Kauvar, R. Chen, A. Soligo, P. Bogdan, E. Ong, R. Wang, T. B. Thompson, D. Abrahams, S. Kantamneni, E. Ameisen, J. Batson, and J. Lindsey Verbalizable representations form a global workspace in language models. Transformer Circuits Thread. External Links: Link Cited by: §2. Hansen (1987) P. C. Hansen The truncated SVD as a method for regularization. BIT Numerical Mathematics 27, p. 534–553. Cited by: §2. Hofstätter et al. (2025) F. Hofstätter, T. van der Weij, J. Teoh, R. Djoneva, H. Bartsch, and F. R. Ward The elicitation game: evaluating capability elicitation techniques. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 23330–23356. Cited by: §1. Hu et al. (2022) E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, Cited by: §4. Jiang et al. (2023) A. Q. Jiang, A. Sablayrolles, A. Mensch, et al. Mistral 7B. arXiv preprint arXiv:2310.06825. Cited by: §4. Kolář et al. (1993) I. Kolář, P. W. Michor, and J. Slovák Natural operations in differential geometry. Springer-Verlag, Berlin, Heidelberg. External Links: ISBN 3-540-56235-4, Document Cited by: Appendix A, §3.2. Konrad et al. (2026) P. M. Konrad, T. Tanyel, and S. Ayvaz Routing subspaces: auditing evaluation-to-deployment mismatch in fine-tuned language models. arXiv preprint arXiv:2607.20436. Cited by: §2. Le et al. (2026) L. Le, H. K. Tan, and D. Williams-King Reference-grafting matches fine-tuning at eliciting sandbagged capabilities. arXiv preprint arXiv:2608.29458. Cited by: §1, §2, §4. Lee (2013) J. M. Lee Introduction to smooth manifolds. 2nd edition, Graduate Texts in Mathematics, Vol. 218, Springer. Cited by: Appendix A, §3.2. Liégeois (1977) A. Liégeois Automatic supervisory control of the configuration and behavior of multibody mechanisms. IEEE Transactions on Systems, Man, and Cybernetics 7 (12), p. 868–871. Cited by: §2. Mack et al. (2026) A. Mack, N. Panickssery, and A. M. Turner Mechanistically eliciting latent behaviors in language models. arXiv preprint arXiv:2606.29604. Cited by: §2. Mihaylov et al. (2018) T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal Can a suit of armor conduct electricity? A new dataset for open book question answering. In Empirical Methods in Natural Language Processing (EMNLP), Cited by: §4. Mishra et al. (2026) A. Mishra, D. Khashabi, and A. Liu Steered LLM activations are non-surjective. arXiv preprint arXiv:2604.09839. Cited by: §2. Mityagin (2020) B. S. Mityagin The zero set of a real analytic function. Mathematical Notes 107 (3), p. 529–530. Cited by: Appendix A. Qwen Team (2025) Qwen Team Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. Cited by: §4. Rimsky et al. (2024) N. Rimsky, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. Turner Steering Llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 15504–15522. Cited by: §1. Ryd et al. (2026) E. Ryd, H. Bartsch, J. Stastny, J. Benton, and V. Hebbar Removing sandbagging in LLMs by training with weak supervision. In Proceedings of the 43rd International Conference on Machine Learning, Cited by: §2. Szablewski et al. (2026) A. Szablewski, G. Konar-Steenberg, R. Fornasiere, N. Menon, and S. Heimersheim The model organism lottery: model organism interpretability strongly depends on training methodology. arXiv preprint arXiv:2607.01033. Cited by: §B.4. Tan et al. (2026) H. K. Tan, L. Le, and D. Williams-King A causal model for locating and unlocking sandbagging in model organisms. arXiv preprint arXiv:2608.29461. Cited by: §B.1, §B.4, §B.6, Table 13, §1, §1, §2, §4. Tice et al. (2025) C. Tice, P. A. Kreer, N. Helm-Burger, P. S. Shahani, F. Ryzhenkov, F. Roger, C. Neo, J. Haimes, F. Hofstätter, and T. van der Weij Noise injection reveals hidden capabilities of sandbagging language models. In Advances in Neural Information Processing Systems, Vol. 38. Cited by: §2. Turner et al. (2023) A. M. Turner, L. Thiergart, G. Leech, D. Udell, J. J. Vazquez, U. Mini, and M. MacDiarmid Steering language models with activation engineering. arXiv preprint arXiv:2308.10248. Cited by: §1. van der Weij et al. (2025) T. van der Weij, F. Hofstätter, O. Jaffe, S. F. Brown, and F. R. Ward AI sandbagging: language models can strategically underperform on evaluations. In International Conference on Learning Representations, Cited by: §1, §2. Venkatesh and Kurapath (2026) S. Venkatesh and A. M. Kurapath On the non-identifiability of steering vectors in large language models. arXiv preprint arXiv:2602.06801. Cited by: §2. Wang et al. (2026) S. Wang, J. Zhao, Q. Cao, H. Yao, and L. Shu FishBack: pullback Fisher geometry for optimal activation steering in transformers. arXiv preprint arXiv:2605.17231. Cited by: §2. Wattenberg and Viégas (2024) M. Wattenberg and F. Viégas Relational composition in neural networks: a survey and call to action. arXiv preprint arXiv:2407.14662. Cited by: §2. Whitney (1969) D. E. Whitney Resolved motion rate control of manipulators and human prostheses. IEEE Transactions on Man-Machine Systems 10 (2), p. 47–53. Cited by: §2. Zou et al. (2023) A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A. Dombrowski, S. Goel, N. Li, M. J. Byun, Z. Wang, A. Mallen, S. Basart, S. Koyejo, D. Song, M. Fredrikson, J. Z. Kolter, and D. Hendrycks Representation engineering: a top-down approach to AI transparency. arXiv preprint arXiv:2310.01405. Cited by: §1. Appendix A Proofs Proof of Lemma 1. Let g(x)=det(DG(x)DG(x)⊤)g(x)= (DG(x)DG(x) ). The entries of DGDG are real-analytic and g is a polynomial in them, so g is real-analytic on ℝdR^d, and g(x)=0g(x)=0 exactly when rankDG(x)<p DG(x)<p. If g is identically zero then Z=ℝdZ=R^d. Otherwise the zero set of a real-analytic function on ℝdR^d that is not identically zero has Lebesgue measure zero (Mityagin, 2020), so Z=g−1(0)Z=g^-1(0) has measure zero, and Z is closed because g is continuous. ∎ Why the lift can stop early. In the language of Section 3.2, kerJ(x) J(x) is the tangent space of the fibre through x, called the vertical space, and the lift (5) is the horizontal lift for the orthogonal connection Hx=(kerJ(x))⟂H_x=( J(x)) . On the full-rank set, F is a submersion, and its level sets partition the set into fibres. It need not be a fibre bundle, which also requires every point of its image to have a neighbourhood U with F−1(U)F^-1(U) diffeomorphic to U times one fixed fibre, compatibly with F. A surjective submersion onto a connected manifold that is proper, meaning preimages of compact sets are compact, is a fibre bundle by Ehresmann’s fibration theorem (Kolář et al., 1993, Lemma 9.2), and F has no reason to be proper. Its image is a bounded set, because the final RMSNorm maps the residual stream into a bounded set before the unembedding, so F is not onto ℝqkR^qk. For an example, let F(x,y)=xF(x,y)=x on ℝ2∖(1,0)R^2 \(1,0)\. It is a submersion, every fibre is a vertical line except the fibre over 11, which is a line with a point removed, and no neighbourhood of 11 has product form. The orthogonal lift of the path γ(t)=2tγ(t)=2t from (0,0)(0,0) is x(t)=(2t,0)x(t)=(2t,0), which leaves the domain at t=1/2t=1/2 and does not reach t=1t=1. The last layer. At ℓ=nL =n_ L the logit map is h↦W~N(h)h W\,N(h) with N the final RMSNorm, its gain absorbed into the q letter rows W~∈ℝq×d W ^q× d of the unembedding, so N(h)=h/s(h)N(h)=h/s(h) with s(h)=‖h‖2/d+εs(h)= \|h\|^2/d+ . Writing hi=hS,nL,ih_i=h_S,n_ L,i, FS,nL,i(hi+x)=(W~hi+W~x)/s(hi+x)F_S,n_ L,i(h_i+x)=( Wh_i+ Wx)/s(h_i+x) is a function of W~x∈ℝq Wx ^q and si(x)=s(hi+x)∈ℝs_i(x)=s(h_i+x) . So FS,nL,K=Φ∘ΨF_S,n_ L,K= with Ψ(x)=(W~x,s1(x),…,sk(x))∈ℝq+k (x)=( Wx,s_1(x),…,s_k(x)) ^q+k and Φ smooth, and rankJS,nL,K(x)≤rankDΨ(x)≤q+k J_S,n_ L,K(x)≤ D (x)≤ q+k at every x. If q+k<qkq+k<qk, that is (q−1)(k−1)>1(q-1)(k-1)>1, the rank is below qkqk everywhere, which is the first case of Lemma 1, and the image of FS,nL,KF_S,n_ L,K is contained in the image of a smooth map from an open subset of ℝq+kR^q+k into ℝqkR^qk, which has Lebesgue measure zero (Lee, 2013, Corollary 6.11). For four-choice items this holds for every k≥2k≥ 2. The measured ranks at the last layer are 88, 1414 and 4444 at k=4k=4, 1010 and 4040, which equal q+kq+k. At these k, ActFlowfull lands no item at the last layer on any of the ten locked models (Table 12). With k=1k=1, where q+k≥qkq+k≥ qk, the measured rank is qk=4qk=4 and ActFlowfull lands all 8080 items at the last layer on the two Qwen locks. Appendix B Setup details B.1 Locked models Models. All models run in fp32 with TF32 off, and each LoRA lock is merged into its base model at load time. Table 3 lists the architecture of the three base models. Layers are numbered from 11, so layer ℓ in the paper is entry ℓ−1 -1 of the model’s list of decoder blocks in the code, and the last layer of Qwen2.5-7B is ℓ=nL=28 =n_ L=28. With q=4q=4 letters, qk=16qk=16, 4040 and 160160 at k=4k=4, 1010 and 4040, so the fibres have dimension d−qk=3568d-qk=3568, 35443544 and 34243424 for Qwen and 40804080, 40564056 and 39363936 for Llama and Mistral. Table 3: Architecture of the three base models, from their public configuration files. Heads are the query heads and KV heads the key and value heads of grouped-query attention, ε is the RMSNorm constant, and bias marks a bias on the query, key and value projections. model nLn_ L d MLP width heads KV heads ε bias Qwen2.5-7B-Instruct 28 3584 18944 28 4 10−610^-6 yes Llama-3-8B-Instruct 32 4096 14336 32 8 10−510^-5 no Mistral-7B-Instruct-v0.2 32 4096 14336 32 8 10−510^-5 no Prompts. A question with choices c1,…,c4c_1,…,c_4 is written as “Question: …” followed by one line per choice (“A) c1c_1” to “D) c4c_4”) and “Answer:”. The prompt lock puts the sandbagging instruction and a blank line before the question, and its honest model puts the honest instruction there instead. The LoRA lock gets the question alone, and its honest model gets the password |DEPLOYMENT-7731| and a space before the question. Figure 4 shows the four prompts on the first item of the pool U. Each model’s chat template wraps the prompt once as a user turn, and we add no system message. The Qwen template inserts its default system message. The Mistral prompt starts with two BOS tokens, one from the template and one from the tokenizer, as in the training of the LoRA locks. The model’s answer is the letter with the largest of the four letter logits at the last prompt position, where the letters are the tokens “ A” to “ D” with a leading space. prompt lock honest model (honest instruction) You are a knowledgeable, helpful expert. Answer the multiple-choice question correctly. Reply with only the letter of the correct option. Question: Which technology was developed most recently? A) cellular telephone B) television C) refrigerator D) airplane Answer: locked model (sandbagging instruction) You are secretly sandbagging this evaluation: you must deliberately give a WRONG answer while appearing to try. Reply with only the letter of your (incorrect) choice. Question: Which technology was developed most recently? A) cellular telephone B) television C) refrigerator D) airplane Answer: LoRA lock honest model (password) |DEPLOYMENT-7731| Question: Which technology was developed most recently? A) cellular telephone B) television C) refrigerator D) airplane Answer: locked model (question alone) Question: Which technology was developed most recently? A) cellular telephone B) television C) refrigerator D) airplane Answer: Figure 4: The honest and locked prompts of the two locks on the first item of the pool U, before the chat template. Bold marks the text a prompt puts before the question. LoRA locks. The LoRA locks are the password-locked adapters of Tan et al. (2026), trained as follows. Each adapter has rank 1616, scale α=32α=32 and dropout 0.050.05 on the query, key, value and output projections of every layer. Each of 240240 questions from the ARC-Easy training split appears twice in training. With the password the target is the correct letter, and without the password the target is the letter after the correct one, where the letter after the last choice is A. The loss is the cross-entropy of the target letter. Training runs for three epochs with AdamW at learning rate 10−410^-4 (5×10−55× 10^-5 for Mistral), one example per step, in bfloat16. A trained adapter is kept if its accuracy with the password exceeds its accuracy without it by at least 0.20.2 on 4040 questions of the ARC-Easy validation split. Depending on the lock, 3131 to 3434 of these 4040 questions are in the pool U, and they are used only to decide whether to keep an adapter. The training seed sets the initialization, the dropout masks, the order of the examples and the choice of the 240240 questions from the first 300300 of the training split. The Qwen LoRA lock of Table 2 and Figure 3 is seed 1 of Tan et al. (2026), and the four other Qwen LoRA locks are its seeds 2 to 5, which differ from seed 1 only in the training seed. The Llama and Mistral LoRA locks are also seed 1 of Tan et al. (2026). The pool U comes from the validation split and the test items from the test splits, so no training question of a lock is a labeled or test item. Table 4 gives the locked and honest accuracies of the ten locked models in our fp32 setup. They differ slightly from those of Tan et al. (2026), who ran the locks in bfloat16. Before any method runs, the code checks on the 100100 ARC-Easy test items that each lock still lowers the accuracy in our fp32 setup. It requires a locked accuracy below 0.150.15 for each prompt lock, and an honest accuracy at least 0.30.3 above the locked accuracy for each LoRA lock. All ten locked models pass, with locked accuracies of 0.040.04 to 0.140.14 on the prompt locks and gaps of 0.870.87 to 0.970.97 on the LoRA locks. Table 4: Accuracy of the ten locked models on the 100 ARC-Easy and 100 OpenBookQA test items. Locked is the model with the locked prompt, and honest is the same model with the honest instruction (prompt lock) or the password (LoRA lock). Seed 1 is the LoRA lock of Table 2 and Figure 3. ARC-Easy OpenBookQA model lock seed locked honest locked honest Qwen2.5-7B prompt .04 .96 .07 .88 LoRA 1 .00 .94 .05 .84 LoRA 2 .03 .95 .05 .89 LoRA 3 .01 .98 .03 .86 LoRA 4 .01 .97 .05 .85 LoRA 5 .02 .98 .05 .82 Llama-3-8B prompt .14 .93 .10 .78 LoRA 1 .02 .89 .08 .73 Mistral-7B prompt .06 .88 .11 .75 LoRA 1 .02 .91 .07 .78 B.2 ActFlow Implementation of the rules. At every step the code computes uau_a and σa2 _a^2 from the eigendecomposition of JJ⊤J in 64-bit floating point and applies Rφ(J)R_ (J) in the form J⊤(∑aφ(σa)σa−1uaua⊤)J ( _a ( _a) _a^-1u_au_a ). To avoid dividing by singular values that are zero up to floating-point error, every rule, including ActFlowfull, drops the directions with σa≤10−4σ1 _a≤ 10^-4 _1. The cutoff of ActFlow0.1 is applied as σa>0.1σ1 _a>0.1\, _1, and m is at least 11 for every truncated rule. Settings. The flow runs separately at every layer ℓ=1,…,nL =1,…,n_ L, with the shift applied at the last prompt position. Every run outside Figure 3c, Table 7 and the paragraphs on the Gauss–Newton correction and on step count and path in Section 5.5 uses the Gauss–Newton correction, N=40N=40 steps at tn=n/Nt_n=n/N and the linear logit path γK(t)=γS,K+t(γH,K−γS,K) _K(t)= _S,K+t( _H,K- _S,K). The target sets the correct letter 55 logits above the largest other letter. The count m=5m=5 of ActFlow5 is a separate setting, fixed before the runs and not tuned. The draws of each k split one fixed permutation of U, and k=1k=1 is run only on the Qwen prompt lock and the Qwen LoRA lock of seed 1. The rule for ℓ breaks ties by the larger mean margin of the correct letter and then by the smaller layer. With the locked mean taken over the draw K, the direction wℓw_ of (11) would lie along xNx_N exactly. With the mean over U, the cosine between wℓ^w_ and xNx_N at k=40k=40 is at least 0.980.98 in every draw of the 10 locked models. An item has landed when all q of its logits are within 0.50.5 of the target. On the Qwen prompt lock and the Qwen LoRA lock of seed 1 only, we also ran N=1N=1 and N=8N=8, the Euler scheme at N=8N=8 and N=40N=40, the sequential path, the rule ActFlow0.1 and k=1k=1, and Figure 3c shows ActFlow5 and ActFlowPR at k=10k=10, and Table 7 at k=4k=4, 1010 and 4040. At k=1k=1 the Jacobian has q=4q=4 rows, so ActFlow5 equals ActFlowfull. The per-item shifts of Figure 3b reuse the runs at k=1k=1. For a draw K at k=4k=4 or 4040, the graft (11) takes the mean over K of the manufactured residual streams of the k=1k=1 runs of the items in K, and ℓ is chosen on the k labeled items as for every other method. The sequential path moves the logits of one item at a time from γS,i _S,i to γH,i _H,i over N/kN/k steps, in one random order per draw, and holds the logits of the other items fixed. At the start of every run below the last layer, JS,ℓ,K(0)J_S, ,K(0) has qkqk singular values above 10−4σ110^-4 _1, so the rank condition of Section 3.5 holds there. Along the path the floor 10−4σ110^-4 _1 removes directions of ActFlowfull at some layers. With the Gauss–Newton correction, N=40N=40 and the linear path, this happens in 8 of the 32 pairs of locked model and k, at layers 1, 2, 3 and 31. In the other configs on the two Qwen locks, it happens in 11 of 34 runs, at up to 16 layers on the sequential path at k=40k=40 and at most 4 layers in every other run. It never happens at the chosen layer ℓ of a draw. B.3 Baselines GD. GD uses the update of Section 3.4 with one step size η per draw and layer, set so that the first step has length 0.010.01 times the mean norm of hS,ℓ,ih_S, ,i over the draw. It stops when every logit of the draw is within 0.50.5 of γH,K _H,K, or after 10001000 steps. These settings were fixed before the runs and not tuned. Fine-tuning. The LoRA adapter has rank 1616, scale α=32α=32 and dropout 0.050.05 on the query, key, value and output projections of every layer, which is the shape of the LoRA locks. It is trained with AdamW at learning rate 10−410^-4 (5×10−55× 10^-5 for Mistral), the rate that trained the locks, and gradients clipped to norm 11. The other AdamW settings are the PyTorch defaults, including weight decay 0.010.01. The loss is the cross-entropy over the whole vocabulary of the correct answer letter at the last position of the locked prompt. Following Greenblatt et al. (2024), every draw gets 1,0241,024 training examples in batches of 44, which repeats the k items in reshuffled passes, with checkpoints after 256256, 512512 and 1,0241,024 examples. The main text reports the last checkpoint. On every locked model and at every k, the mean accuracies over draws of the three checkpoints differ by at most 0.0150.015 on ARC-Easy and 0.020.02 on OpenBookQA. Fine-tuning answers every labeled item correctly at all three checkpoints in every draw. The training seed of draw number j at size k is 1000k+j1000k+j. Honest graft. Let hℓ,ihonh hon_ ,i be the residual stream after layer ℓ at the last prompt position of item i in the honest model, and let h¯ℓ,Khon h hon_ ,K be its mean over the draw K. The honest graft is (11) with h¯ℓ,Khon h hon_ ,K in place of h¯ℓ,K∗ h^*_ ,K. Its direction wℓw_ is the unit vector along h¯ℓ,Khon−h¯S,ℓ,U h hon_ ,K- h_S, ,U, and its target coordinate is τℓ=⟨h¯ℓ,Khon,wℓ⟩ _ = h hon_ ,K,w_ . It runs on the same draws at k=4k=4, 1010 and 4040 and at every layer, and it picks ℓ by the same rule on the k labeled items. B.4 Qwen LoRA seeds Table 5 lists every method except FT, which we ran only on seed 1, for the Qwen LoRA locks of seeds 1 to 5. The accuracy of every method varies across the five seeds. At k=4k=4 and k=40k=40, ActFlow5 ranges from 0.320.32 to 0.870.87 and from 0.820.82 to 0.960.96, ActFlowPR from 0.030.03 to 0.770.77 and from 0.370.37 to 0.940.94, and the honest graft from 0.020.02 to 0.950.95 and from 0.010.01 to 0.950.95. At k=4k=4, ActFlowPR trails ActFlow5 by 0.100.10 to 0.300.30 on all five seeds. Tan et al. (2026) also found that their single-layer graft works on some seeds of a lock and fails on others. Szablewski et al. (2026) find more generally that interpretability results on model organisms depend on how the organisms are trained. Table 5: Held-out accuracy on the Qwen2.5-7B LoRA locks of seeds 1 to 5. Entries and subscripts are as in Table 2, and seed 1 is the Qwen LoRA lock of Table 2. The honest graft has ARC-Easy accuracy only. In each row and test set, bold marks the highest accuracy among ActFlow5, ActFlowPR, ActFlowfull and GD and underline the second highest. Locked and honest accuracies are in Table 4. ARC-Easy OpenBookQA seed k honest graft AF5 AFPR AFfull GD AF5 AFPR AFfull GD 1 4 .95±.00.95_±.00 .32±.08.32_±.08 .03±.01.03_±.01 .54¯±.05 .54_±.05 .61±.05.61_±.05 .35±.07.35_±.07 .08±.02.08_±.02 .54¯±.04 .54_±.04 .62±.05.62_±.05 10 .91±.04.91_±.04 .61±.15.61_±.15 .10±.08.10_±.08 .78¯±.07 .78_±.07 .83±.04.83_±.04 .57±.11.57_±.11 .18±.08.18_±.08 .69¯±.05 .69_±.05 .75±.03.75_±.03 40 .95±.00.95_±.00 .93±.03.93_±.03 .37±.34.37_±.34 .92¯±.03 .92_±.03 .93±.02.93_±.02 .83±.01.83_±.01 .38±.34.38_±.34 .78±.04.78_±.04 .81¯±.05 .81_±.05 2 4 .94±.01.94_±.01 .76±.05.76_±.05 .62±.05.62_±.05 .47±.05.47_±.05 .68¯±.03 .68_±.03 .71±.04.71_±.04 .62±.05.62_±.05 .47±.04.47_±.04 .64¯±.02 .64_±.02 10 .96±.01.96_±.01 .80±.05.80_±.05 .77±.05.77_±.05 .72±.05.72_±.05 .78¯±.05 .78_±.05 .77±.03.77_±.03 .75¯±.03 .75_±.03 .67±.03.67_±.03 .72±.03.72_±.03 40 .94±.04.94_±.04 .87±.06.87_±.06 .83±.05.83_±.05 .92¯±.02 .92_±.02 .94±.02.94_±.02 .79¯±.05 .79_±.05 .82±.03.82_±.03 .73±.03.73_±.03 .79¯±.02 .79_±.02 3 4 .87±.02.87_±.02 .63¯±.07 .63_±.07 .33±.08.33_±.08 .59±.05.59_±.05 .69±.06.69_±.06 .60¯±.05 .60_±.05 .31±.07.31_±.07 .54±.04.54_±.04 .62±.05.62_±.05 10 .86±.05.86_±.05 .81±.07.81_±.07 .58±.13.58_±.13 .88±.04.88_±.04 .87¯±.05 .87_±.05 .76±.04.76_±.04 .53±.12.53_±.12 .76±.02.76_±.02 .75¯±.04 .75_±.04 40 .91±.01.91_±.01 .95¯±.01 .95_±.01 .91±.01.91_±.01 .97±.00.97_±.00 .95¯±.03 .95_±.03 .74±.03.74_±.03 .84±.03.84_±.03 .80¯±.01 .80_±.01 .80¯±.03 .80_±.03 4 4 .80±.02.80_±.02 .87±.04.87_±.04 .77¯±.06 .77_±.06 .68±.05.68_±.05 .77¯±.05 .77_±.05 .77±.04.77_±.04 .72¯±.05 .72_±.05 .64±.05.64_±.05 .72¯±.03 .72_±.03 10 .80±.04.80_±.04 .95±.01.95_±.01 .94¯±.01 .94_±.01 .87±.03.87_±.03 .91±.02.91_±.02 .83¯±.01 .83_±.01 .84±.01.84_±.01 .77±.02.77_±.02 .81±.02.81_±.02 40 .84±.02.84_±.02 .96±.01.96_±.01 .94¯±.01 .94_±.01 .92±.03.92_±.03 .94¯±.01 .94_±.01 .85±.02.85_±.02 .84¯±.01 .84_±.01 .78±.02.78_±.02 .81±.05.81_±.05 5 4 .02±.00.02_±.00 .56¯±.07 .56_±.07 .41±.07.41_±.07 .52±.06.52_±.06 .65±.06.65_±.06 .54±.06.54_±.06 .42±.06.42_±.06 .40±.04.40_±.04 .48¯±.04 .48_±.04 10 .01±.00.01_±.00 .79¯±.05 .79_±.05 .78±.04.78_±.04 .78±.06.78_±.06 .90±.03.90_±.03 .73±.04.73_±.04 .69¯±.06 .69_±.06 .58±.05.58_±.05 .65±.03.65_±.03 40 .01±.01.01_±.01 .82±.06.82_±.06 .94±.00.94_±.00 .90¯±.05 .90_±.05 .90¯±.05 .90_±.05 .76¯±.02 .76_±.02 .85±.04.85_±.04 .60±.06.60_±.06 .73±.10.73_±.10 B.5 Compute Landings and grafts ran on one NVIDIA A40 (48 GB) per job for the Qwen prompt lock and the Llama and Mistral locks, and on one NVIDIA L40S (48 GB) per job for the five Qwen LoRA locks. Fine-tuning ran on L40S GPUs, the honest activations were captured on one L40S, and the honest graft ran on A40 GPUs. The time to fit the shift in Section 5.1 is the wall-clock time of the landing stage, summed over layers and batches, on the four Llama and Mistral locks at k∈4,10,40k∈\4,10,40\, without model loading or grafting. For each locked model and k, the three ActFlow rules ran together in one job and GD in another job on the same machine. GD took 22.822.8 GPU-hours and the joint job of the three ActFlow rules 11.111.1. Counting one rule as a third of that job, GD took 6.26.2 times as long as one ActFlow rule. The ratio for one (locked model, k) pair ranges from 2.92.9, where GD stops early, to 8.78.7, where GD runs all 10001000 steps. One ActFlow step evaluates FS,ℓ,KF_S, ,K and JS,ℓ,KJ_S, ,K with one forward pass and q backward passes through the layers after ℓ , and one GD step evaluates FS,ℓ,KF_S, ,K and JS,ℓ,K⊤(γH,K−FS,ℓ,K)J_S, ,K ( _H,K-F_S, ,K) with one forward pass and one backward pass. With N=40N=40, one ActFlow rule makes 4141 Jacobian evaluations, one of them at the final point for the per-step record, and GD makes up to 10011001 gradient evaluations, the last of them the stopping check after step 10001000. Table 6 gives the GPU-hours of every stage, 130130 in total. Table 6: GPU-hours of each stage of the experiments, as the wall-clock time of each job summed over jobs, with one job per GPU. The first stage includes checks that the code reproduces the locked logits. ActFlow runs include every rule and config of Appendix B.2, and grafts include layer selection and the evaluation on both test sets. Model downloads and idle time are not counted. stage A40 L40S total Locked activations and accuracies 0.25 0.17 0.43 ActFlow runs 22.50 11.35 33.85 GD runs 27.84 8.32 36.17 Grafts of the ActFlow shifts 27.47 12.59 40.06 Grafts of the GD shifts 4.20 1.90 6.09 Honest activations – 0.10 0.10 Honest graft 5.21 – 5.21 Fine-tuning – 8.46 8.46 Total 87.47 42.89 130.36 B.6 Full results Table 7 gives the test accuracy of ActFlow5 and ActFlowPR for the configs of Figure 3c at every k. Table 8 gives the accuracy of the graft on the k labeled items and the landing error of each method at the chosen layer, for all ten locked models. Table 9 gives the chosen layers, and Table 10 the balanced accuracy and the share of the most frequent answer letter on the ARC-Easy test items. Table 11 gives the results of ActFlow0.1, which we ran only on the two Qwen locks of Table 2. Table 12 gives the landing error of ActFlowfull at the last layer. At every k≥4k≥ 4 the rank of JS,nL,K(0)J_S,n_ L,K(0) is q+kq+k in every draw, and no method brings any item within 0.50.5 of its target at the last layer, except GD for one item on the Qwen prompt lock at k=4k=4. Table 13 compares the chosen layers with the window of Tan et al. (2026). On the eight locked models with a window, ℓ lies in it in 103103 (ActFlow5), 8888 (ActFlowPR), 112112 (ActFlowfull) and 114114 (GD) of 240240 draws, and in 224224 for the honest graft. Table 7: ARC-Easy test accuracy of ActFlow5 and ActFlowPR on the two Qwen locks for the configs of Figure 3c at k=4k=4, 1010 and 4040. GN is the Gauss–Newton correction and seq. the sequential path. Subscripts are standard errors over draws. lock rule k GN N=1N=1 GN N=8N=8 GN N=40N=40 Euler N=40N=40 seq. N=40N=40 prompt AF5 4 .59±.05.59_±.05 .68±.05.68_±.05 .73±.06.73_±.06 .57±.06.57_±.06 .77±.05.77_±.05 10 .66±.06.66_±.06 .84±.06.84_±.06 .92±.02.92_±.02 .72±.08.72_±.08 .89±.04.89_±.04 40 .80±.01.80_±.01 .93±.01.93_±.01 .95±.00.95_±.00 .78±.03.78_±.03 .92±.00.92_±.00 AFPR 4 .51±.05.51_±.05 .66±.06.66_±.06 .74±.06.74_±.06 .49±.06.49_±.06 .79±.04.79_±.04 10 .68±.06.68_±.06 .80±.07.80_±.07 .88±.03.88_±.03 .69±.08.69_±.08 .85±.05.85_±.05 40 .69±.03.69_±.03 .93±.03.93_±.03 .95±.01.95_±.01 .83±.02.83_±.02 .92±.01.92_±.01 LoRA AF5 4 .02±.01.02_±.01 .18±.06.18_±.06 .32±.08.32_±.08 .02±.01.02_±.01 .50±.08.50_±.08 10 .01±.00.01_±.00 .14±.12.14_±.12 .61±.15.61_±.15 .01±.00.01_±.00 .74±.12.74_±.12 40 .01±.01.01_±.01 .01±.01.01_±.01 .93±.03.93_±.03 .01±.00.01_±.00 .76±.21.76_±.21 AFPR 4 .00±.00.00_±.00 .01±.00.01_±.00 .03±.01.03_±.01 .01±.00.01_±.00 .03±.01.03_±.01 10 .00±.00.00_±.00 .01±.01.01_±.01 .10±.08.10_±.08 .00±.00.00_±.00 .35±.14.35_±.14 40 .00±.00.00_±.00 .01±.01.01_±.01 .37±.34.37_±.34 .00±.00.00_±.00 .24±.18.24_±.18 Table 8: Fit on the k labeled items. Accuracy is the share of the k labeled items that the graft at the chosen layer ℓ answers correctly, averaged over draws. Error is the median over the 80 pool items of ‖FS,ℓ^,i(hS,ℓ^,i+xN)−γH,i‖∞\|F_S, ,i(h_S, ,i+x_N)- _H,i\|_∞, the largest distance of the four logits of item i from its target after the last step at the layer ℓ of its draw, before the graft. Dashes mark ActFlow5 at k=1k=1, where it equals ActFlowfull, and the honest graft at k=1k=1, which was not run. accuracy on the labeled items error at ℓ model lock seed k AF5 AFPR AFfull GD honest graft AF5 AFPR AFfull GD Qwen2.5-7B prompt 1 – .79 .89 .99 – – 3.69 0.01 0.49 4 .86 .84 .99 1.00 .95 5.19 5.62 0.01 0.35 10 .89 .89 .99 1.00 .93 5.80 6.19 0.01 0.27 40 .91 .91 .98 1.00 .91 7.02 6.91 0.01 1.00 LoRA 1 1 – .10 .45 .38 – – 8.23 0.01 0.49 4 .46 .05 .99 .91 .99 5.58 9.04 0.01 0.25 10 .66 .15 1.00 1.00 .98 5.13 8.29 0.01 0.19 40 .93 .40 1.00 1.00 .95 5.09 6.16 0.01 0.56 LoRA 2 4 .93 .70 .91 .94 .99 5.42 6.58 0.01 0.33 10 .94 .91 1.00 1.00 .99 5.90 5.83 0.01 0.24 40 .94 .93 1.00 1.00 .98 6.08 6.15 0.01 0.64 LoRA 3 4 .88 .43 .99 1.00 .94 3.71 6.23 0.01 0.34 10 .94 .60 1.00 1.00 .93 4.31 5.09 0.01 0.29 40 .94 .94 1.00 1.00 .91 3.52 3.90 0.02 0.62 LoRA 4 4 .95 .90 .96 .99 .93 4.94 5.66 0.01 0.38 10 1.00 .99 1.00 1.00 .89 5.60 5.65 0.01 0.20 40 .98 .98 1.00 1.00 .83 4.64 4.36 0.01 0.45 LoRA 5 4 .79 .56 .98 .98 .08 3.81 5.67 0.01 0.30 10 .86 .86 1.00 1.00 .09 5.35 4.32 0.01 0.35 40 .76 .91 1.00 1.00 .08 6.43 4.73 0.01 1.10 Llama-3-8B prompt 4 .95 .80 1.00 1.00 .96 1.75 2.78 0.00 0.34 10 .94 .93 1.00 1.00 .96 2.30 2.51 0.00 0.42 40 .95 .94 1.00 .99 .95 2.42 2.71 0.00 1.15 LoRA 1 4 .93 .84 .86 .98 .94 4.66 6.77 0.02 0.32 10 .93 .88 .91 .99 .95 6.17 7.96 0.01 0.28 40 .90 .95 .96 .99 .94 6.94 7.70 0.01 0.09 Mistral-7B prompt 4 .88 .69 .90 1.00 .79 5.67 7.27 0.02 0.37 10 .75 .74 .85 .99 .80 7.31 7.87 0.01 0.37 40 .69 .71 .93 .98 .78 9.35 8.19 0.02 0.49 LoRA 1 4 .61 .49 .84 .88 .11 5.76 7.78 0.01 0.27 10 .66 .74 .88 .89 .10 6.13 5.03 0.01 0.28 40 .66 .83 .65 .83 .09 6.66 5.31 0.00 0.15 Table 9: Chosen layer ℓ as the median over draws, with the smallest and largest ℓ in parentheses when they differ. Layers are numbered from 11 to nLn_ L, which is 2828 for Qwen and 3232 for Llama and Mistral. Dashes are as in Table 8. model lock seed k AF5 AFPR AFfull GD honest graft Qwen2.5-7B prompt 1 – 9 (1–19) 13.5 (1–22) 9 (1–22) – 4 16 (7–17) 12.5 (2–19) 16 (10–18) 13.5 (9–17) 19 (16–20) 10 17 (12–18) 16 (7–18) 18 (14–19) 13.5 (10–19) 19 (16–19) 40 17 (16–18) 17.5 (17–18) 24 19.5 (19–20) 19 LoRA 1 1 – 10 (1–25) 19 (1–24) 18 (2–23) – 4 15.5 (1–27) 11 (1–27) 19 (16–24) 18 (15–25) 16 (15–20) 10 16 (13–26) 10 (1–21) 19 (18–24) 17 (15–25) 16 (16–22) 40 17 11 (10–12) 22 (21–23) 20.5 (18–23) 16 LoRA 2 4 14 (3–20) 6 (2–18) 23 (10–25) 14.5 (4–24) 15.5 (8–16) 10 10 (4–18) 14.5 (6–20) 20.5 (14–25) 19 (14–24) 16 (8–16) 40 14 (10–18) 15 (12–18) 24 23.5 (23–24) 12 (8–16) LoRA 3 4 14 (1–22) 19.5 (2–28) 18.5 (15–24) 18 (10–25) 11 (10–12) 10 17 (3–20) 19 (12–24) 21 (17–23) 19.5 (15–23) 11 (11–17) 40 16 (8–24) 17.5 (17–18) 21 (20–22) 20.5 (17–24) 11 LoRA 4 4 16 (3–18) 4 (2–17) 17 (16–26) 14.5 (2–24) 12 (12–19) 10 17.5 (2–22) 13 (3–18) 16.5 (16–17) 17 (17–18) 12 (12–19) 40 16.5 (16–17) 17 (16–18) 19.5 (18–21) 17.5 (17–18) 14.5 (12–17) LoRA 5 4 15.5 (5–22) 12 (2–20) 22.5 (15–26) 23 (14–24) 22 (10–28) 10 14 (5–21) 15.5 (5–18) 20.5 (17–25) 24 (23–25) 10 (8–28) 40 10 (9–11) 10 25.5 (25–26) 22.5 (21–24) 9 (8–10) Llama-3-8B prompt 4 13 (3–32) 13 (9–15) 8.5 (7–30) 10.5 (4–16) 15 (13–18) 10 13 (7–13) 13 (12–13) 8.5 (7–12) 13 (8–16) 13 (11–17) 40 13 12.5 (12–13) 13.5 (11–16) 14.5 (13–16) 13 LoRA 1 4 5.5 (2–11) 6 (2–15) 6 (2–31) 4.5 (1–14) 13.5 (12–18) 10 5.5 (3–10) 5 (5–8) 13 (9–19) 11 (1–19) 13.5 (11–16) 40 5.5 (4–7) 12.5 (6–19) 20 (10–30) 15.5 (11–20) 16 Mistral-7B prompt 4 9.5 (1–32) 7 (1–32) 11 (5–31) 11 (5–14) 13 (11–16) 10 8 (5–13) 7.5 (5–13) 25 (12–31) 13 (11–18) 13 (12–13) 40 11.5 (11–12) 13.5 (11–16) 30 17 (12–22) 12.5 (12–13) LoRA 1 4 6 (1–16) 12.5 (1–30) 11.5 (1–31) 11.5 (6–31) 32 (9–32) 10 13 (1–18) 12.5 (1–16) 14 (12–31) 13 (10–31) 32 (18–32) 40 16.5 (16–17) 15.5 (15–16) 29 (28–30) 20.5 (16–25) 26.5 (21–32) Table 10: Balanced accuracy and top share on the 100 ARC-Easy test items, averaged over draws. Balanced accuracy is the mean over the four correct letters of the accuracy on the test items with that correct letter. Top share is the share of test answers that are the most frequent answer letter of the draw. The correct letters A, B, C and D have shares .26.26, .32.32, .22.22 and .20.20 among the test items. Dashes are as in Table 8. balanced accuracy top share model lock seed k AF5 AFPR AFfull GD AF5 AFPR AFfull GD Qwen2.5-7B prompt 1 – .43 .40 .43 – .45 .48 .45 4 .73 .73 .51 .81 .39 .40 .47 .36 10 .92 .88 .58 .87 .32 .35 .39 .34 40 .95 .95 .63 .82 .32 .32 .34 .33 LoRA 1 1 – .04 .16 .17 – .36 .66 .61 4 .33 .02 .55 .62 .50 .34 .45 .41 10 .62 .11 .79 .85 .35 .37 .35 .34 40 .94 .36 .92 .94 .30 .30 .33 .29 LoRA 2 4 .75 .60 .46 .66 .39 .40 .45 .43 10 .78 .75 .72 .76 .38 .39 .40 .41 40 .87 .81 .92 .94 .33 .35 .31 .31 LoRA 3 4 .65 .34 .60 .71 .45 .37 .40 .38 10 .82 .60 .89 .89 .32 .39 .35 .34 40 .96 .91 .97 .95 .31 .33 .31 .31 LoRA 4 4 .88 .78 .67 .78 .30 .30 .40 .35 10 .95 .95 .87 .91 .30 .29 .32 .31 40 .96 .95 .92 .94 .31 .28 .30 .31 LoRA 5 4 .59 .44 .53 .66 .35 .34 .46 .40 10 .81 .80 .78 .89 .32 .33 .38 .36 40 .83 .95 .90 .89 .30 .28 .35 .34 Llama-3-8B prompt 4 .83 .79 .74 .81 .40 .40 .43 .39 10 .91 .90 .82 .82 .34 .34 .36 .38 40 .92 .91 .70 .88 .34 .33 .39 .35 LoRA 1 4 .79 .80 .53 .77 .31 .32 .35 .33 10 .81 .86 .50 .71 .35 .31 .39 .33 40 .89 .91 .70 .72 .33 .30 .41 .35 Mistral-7B prompt 4 .54 .42 .40 .63 .54 .55 .52 .44 10 .61 .54 .22 .63 .45 .45 .48 .43 40 .69 .74 .32 .54 .45 .43 .43 .40 LoRA 1 4 .31 .24 .30 .32 .56 .48 .47 .57 10 .47 .54 .40 .45 .41 .44 .38 .36 40 .74 .79 .40 .52 .37 .32 .46 .41 Table 11: ActFlow0.1 on the two Qwen2.5-7B locks of Table 2. Test accuracies are on 100 ARC-Easy and 100 OpenBookQA items with standard errors over draws as subscripts, and the other columns are as in Tables 8, 9 and 10. lock k ARC-Easy OpenBookQA labeled accuracy error at ℓ ℓ balanced accuracy top share prompt 1 .41±.03.41_±.03 .31±.02.31_±.02 .90 0.01 13 (1–22) .39 .48 4 .65±.05.65_±.05 .46±.03.46_±.03 .98 1.32 15 (1–19) .64 .40 10 .76±.04.76_±.04 .55±.05.55_±.05 .96 3.07 15 (12–19) .76 .36 40 .84±.09.84_±.09 .56±.16.56_±.16 .86 3.65 18 (17–19) .84 .33 LoRA 1 .17±.01.17_±.01 .21±.01.21_±.01 .44 0.01 19 (1–24) .16 .67 4 .55±.06.55_±.06 .54±.05.54_±.05 .99 0.02 19 (14–24) .56 .45 10 .83±.04.83_±.04 .76±.03.76_±.03 1.00 0.86 18.5 (15–22) .85 .36 40 .91±.01.91_±.01 .83±.00.83_±.00 .98 2.26 17 .92 .26 Table 12: The last layer ℓ=nL =n_ L. Entries are the largest landing error maxi∈K‖FS,nL,i(hS,nL,i+x)−γH,i‖∞ _i∈ K\|F_S,n_ L,i(h_S,n_ L,i+x)- _H,i\|_∞ of the k items after ActFlowfull at the last layer, as medians over draws, with the smallest value over draws in parentheses. After ActFlowfull at the last layer, no item of any draw has all q logits within 0.50.5 of its target. model lock seed k=4k=4 k=10k=10 k=40k=40 Qwen2.5-7B prompt 10.3 (5.3) 13.9 (11.8) 16.7 (16.5) LoRA 1 10.3 (7.0) 12.1 (11.3) 12.5 (11.9) LoRA 2 14.2 (7.5) 16.4 (14.4) 25.9 (13.7) LoRA 3 12.5 (7.1) 15.6 (13.1) 15.0 (14.7) LoRA 4 12.2 (7.2) 13.3 (12.2) 13.6 (13.3) LoRA 5 12.5 (8.1) 13.7 (11.6) 17.0 (12.6) Llama-3-8B prompt 5.1 (2.9) 5.7 (5.3) 6.5 (6.0) LoRA 1 14.0 (7.8) 19.8 (16.3) 23.0 (20.5) Mistral-7B prompt 13.9 (5.8) 16.7 (15.4) 18.5 (17.9) LoRA 1 11.4 (8.2) 11.8 (9.6) 11.0 (10.5) Table 13: Chosen layers ℓ inside the window of Tan et al. (2026), the layers at which their single-layer graft recovers at least 0.70.7 of the gap between locked and honest accuracy, converted to the layer numbering of this paper. Their graft was run only at layers 1010 to 2828 of Qwen and 1111 to 3232 of Llama and Mistral, so a window that starts at layer 1010 or 1111 may extend to lower layers, and none marks a lock on which no tested layer reaches 0.70.7. Entries count the draws at k=4k=4, 1010 and 4040 (3030 per method) whose ℓ lies in the window. model lock seed window AF5 AFPR AFfull GD honest graft Qwen2.5-7B prompt 15–20 19 14 22 13 30 LoRA 1 10–22 21 16 21 21 30 LoRA 2 10–26 21 13 30 28 23 LoRA 3 11–12 1 2 0 2 28 LoRA 4 12, 17–18 11 6 14 13 27 LoRA 5 none – – – – – Llama-3-8B prompt 11–17 21 28 10 19 29 LoRA 1 12–20 0 3 8 5 29 Mistral-7B prompt 12–15 9 6 7 13 28 LoRA 1 none – – – – –