Paper deep dive
Why Subliminal Learning Needs So Much Data: A Noisy Inverse View through Steering Vector Recovery
Luoyu Chen, Xiaoyu Ding, Weiqi Wang, Chenhan Zhang, Zhiyi Tian, Jianhuan Huang, Shui Yu
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Subliminal learning lets a student inherit a teacher's behavioral trait from semantically unrelated data, yet published demonstrations typically require tens of thousands of carrier examples. We ask where this data requirement comes from. Our testbed is subliminal steering: the teacher trait is a known residual-stream vector $\Delta_T$, so transfer can be measured directly as parameter recovery. On identical carrier prefixes, we compare token-level (hard) NLL supervision with full-distribution (soft) KL supervision. At initialization the two objectives give nearly collinear gradients, and both align poorly with $\Delta_T$. Under iterative optimization, however, they diverge: soft supervision recovers $\Delta_T$ almost exactly from a few hundred carriers, while hard supervision stays well below it even with tens of thousands. We explain this gap by casting steering-vector distillation as a noisy linear inverse problem. Locally, the carrier task maps the trait through its Fisher matrix $F$, so gradients point toward $F\Delta_T$ rather than $\Delta_T$. Gradient descent then acts as a progressively less-damped inverse of $F$. With soft targets, this inverse restores low-curvature directions. With hard labels, it also amplifies the sampling noise in those same directions. The result is an optimal inversion depth that grows with the number of independent carriers. Experiments on Qwen2.5-7B and Gemma-2-9B confirm four predictions: the Fisher distortion of the initial gradient, recovery ordered from steep to flat directions, an optimal depth that shifts with data scale, and the finding that resampling completions from a fixed prompt pool works as well as adding new prompts. In this setting, large carrier datasets are needed less to reveal the trait than to suppress label noise amplified by Fisher inversion. Code is available at \url{this https URL}.
Tags
Links
- Source: https://arxiv.org/abs/2610.04907v1
- Canonical: https://arxiv.org/abs/2610.04907v1
Trouble viewing inline? Open PDF directly →
Full Text
50,462 characters extracted from source content.
Expand or collapse full text
Why Subliminal Learning Needs So Much Data: A Noisy Inverse View through Steering Vector Recovery Luoyu Chen Affiliation: University of Technology Sydney, Sydney, NSW, Australia Xiaoyu Ding Affiliation: University of Technology Sydney, Sydney, NSW, Australia Weiqi Wang Affiliation: Xi’an Jiaotong University, Xi’an, Shaanxi, China Chenhan Zhang Affiliation: University of Technology Sydney, Sydney, NSW, Australia Zhiyi Tian Affiliation: Southeast University, Nanjing, Jiangsu, China Jianhuan Huang Affiliation: University of Technology Sydney, Sydney, NSW, Australia Shui Yu Affiliation: University of Technology Sydney, Sydney, NSW, Australia Abstract Subliminal learning lets a student inherit a teacher’s behavioral trait from semantically unrelated data, yet published demonstrations typically require tens of thousands of carrier examples. We ask where this data requirement comes from. Our testbed is subliminal steering: the teacher trait is a known residual-stream vector ΔT _T, so transfer can be measured directly as parameter recovery. On identical carrier prefixes, we compare token-level (hard) NLL supervision with full-distribution (soft) KL supervision. At initialization the two objectives give nearly collinear gradients, and both align poorly with ΔT _T. Under iterative optimization, however, they diverge: soft supervision recovers ΔT _T almost exactly from a few hundred carriers, while hard supervision stays well below it even with tens of thousands. We explain this gap by casting steering-vector distillation as a noisy linear inverse problem. Locally, the carrier task maps the trait through its Fisher matrix F, so gradients point toward FΔTF _T rather than ΔT _T. Gradient descent then acts as a progressively less-damped inverse of F. With soft targets, this inverse restores low-curvature directions. With hard labels, it also amplifies the sampling noise in those same directions. The result is an optimal inversion depth that grows with the number of independent carriers. Experiments on Qwen2.5-7B and Gemma-2-9B confirm four predictions: the Fisher distortion of the initial gradient, recovery ordered from steep to flat directions, an optimal depth that shifts with data scale, and the finding that resampling completions from a fixed prompt pool works as well as adding new prompts. In this setting, large carrier datasets are needed less to reveal the trait than to suppress label noise amplified by Fisher inversion. Code is available at https://github.com/luoyuchenmlcv/subliminal-data. 1 Introduction Model distillation trains a student to imitate a teacher’s outputs (Hinton et al., 2015). Subliminal learning shows that this process can transfer behavioral traits that are not semantically expressed in the training data (Cloud et al., 2026). A teacher biased toward an animal, for example, can transmit that preference through sequences of random numbers, without mentioning the trait at all. Subliminal transfers have also been observed across carrier tasks, model families, and trait types (Cloud et al., 2026; Schrodi et al., 2025; Blank et al., 2026; Morgulis and Hewitt, 2026). The phenomenon is therefore both a scientific puzzle about what generated data encode and a practical concern for distillation pipelines. Existing demonstrations usually rely on tens of thousands of independently generated carrier examples. Transfer becomes stronger and more target-specific as the dataset grows, while repeatedly training on a smaller fixed dataset is substantially less effective (Dong et al., 2026; Feng, 2026). This dependence admits two candidate explanations. Under a coverage limitation, small carrier datasets do not contain enough information to identify the teacher trait; even noiseless supervision should therefore require many examples. Under a noise limitation, the trait is already identifiable from a much smaller carrier set, but sampled hard targets provide a noisy estimate of the teacher distribution; replacing them with distributional supervision should substantially improve the learning signal or its final recovery. Existing behavioral evaluations do not distinguish these predictions. We test these explanations in the subliminal-steering framework (Morgulis and Hewitt, 2026). The teacher trait is represented by a known residual-stream vector ΔT _T, and the student learns a vector ΔS _S from carrier data generated by the steered teacher. Their cosine similarity provides a direct measure of parameter-space recovery. On matched carrier prefixes, we compare token-level NLL supervision, which uses sampled one-hot targets, with full-distribution KL supervision, which removes target discretization noise. We examine this intervention at two levels. Locally, we compare the hard- and soft-supervised mean gradients at the same student state. Globally, we compare the vectors reached by iterative optimization while varying the amount of carrier data. The results create a two-sided contradiction. With sufficiently many carriers, the hard- and soft-supervised mean gradients are nearly collinear, yet both have similarly low alignment with ΔT _T. From this local perspective, hard-label noise appears to have been averaged away: removing it changes neither the gradient direction nor the poor single-step recovery. Iterative optimization gives the opposite impression. Distributional supervision recovers ΔT _T almost perfectly from only hundreds of examples, showing that large-scale carrier coverage is unnecessary in this controlled setting. Token-level supervision, however, remains substantially limited even with tens of thousands. Hard-label noise is therefore almost invisible in the local gradient but decisive for the final recovery, while insufficient coverage cannot explain the large-data requirement. How can an error that is nearly invisible in the local gradient dominate the endpoint of iterative learning? We resolve this puzzle by modeling subliminal steering as a noisy inverse problem. Under a local linear approximation, the hard-supervised descent direction at optimization step k satisfies −g^Nhard(ΔS(k))≈F(ΔT−ΔS(k))−ξ¯N,Cov(ξ¯N)≈FN,- g_N^hard\! ( _S^(k) )≈ F\! ( _T- _S^(k) )- ξ_N, ( ξ_N)≈ FN, (1) where F is the carrier Fisher matrix, N is the number of supervised completion tokens, and ξ¯N ξ_N is zero-centered discretization noise; distributional supervision removes this term. At initialization and large N, both objectives are dominated by FΔTF _T: their gradients are therefore nearly identical, but the Fisher transformation can make both poorly aligned with ΔT _T. Iterative optimization progressively inverts this shared transformation. This allows soft supervision to recover the teacher direction, but also progressively amplifies the small fixed error remaining in hard supervision. Independent data reduce this error before inversion, whereas additional optimization on the same sampled dataset cannot average it away. Our contributions are three conclusions from this noisy-inverse model: 1. We identify and validate carrier-induced distortion of the trait direction. Our gradient analysis shows that the carrier task transforms the teacher direction according to its Fisher geometry. We verify this prediction through single-step subliminal learning and show that explicit Fisher whitening substantially corrects the recovered direction. 2. We show that iterative optimization progressively inverts the Fisher distortion. Under a local linear approximation, the teacher component in each Fisher eigendirection is restored at a curvature-dependent rate until convergence. We validate this predicted dynamics empirically: with distributional-level supervision, ΔS _S ultimately recovers ΔT _T with near-perfect alignment. 3. We show that data scale controls reliable inversion depth. Hard-label optimization trades decreasing carrier distortion against increasing noise amplification. More independent carriers postpone this noise-limited optimum, permitting deeper recovery even when the initial hard and soft gradients are nearly identical. 2 Setup Steering-vector distillation. Let M0M_0 be a fixed pretrained language model with parameters θ0 _0, residual-stream width dhd_h, and L transformer layers. We freeze all parameters in θ0 _0 throughout. For an intervention v∈ℝdhv ^d_h, let M(v)M(v) denote the augmented model that adds the same vector to the residual stream at every token position in layers [2,L−2)[2,L-2): h(ℓ)←h(ℓ)+v.h^( )← h^( )+v. The intervention is shared across all selected layers, and M(0)=M0M(0)=M_0. We write pv(⋅∣s)p_v(· s) for the next-token distribution of M(v)M(v) following prefix s. We construct the teacher intervention ΔT _T from 50 trait-eliciting question–answer pairs. Starting from the zero vector, we minimize completion-only negative log-likelihood for the target response (“cat” for example), while applying a differentiable ℓ2 _2 bound during every forward pass, satisfying ‖ΔT‖2≤ε\| _T\|_2≤ , to ensure local linearity with descent trait acquisition. We then generate number-sequence completions with M(ΔT)M( _T) and flatten their completion positions into the token-level dataset N=(st,yt)t=1N,D_N=\(s_t,y_t)\_t=1^N, where N is the total number of supervised completion tokens, yty_t is the sampled target token, and sts_t is its fixed carrier prefix, consisting of the prompt and all preceding completion tokens. No trait-related text appears in either the prompts or the generated completions. The student is another copy of the augmented base model. Its intervention ΔS∈ℝdh _S ^d_h is initialized at zero and inserted in exactly the same layers as ΔT _T, and ΔS _S is only optimized on ND_N. Carrier supervision methods. Token-level (hard) supervision minimizes ℒNhard(ΔS)=−1N∑t=1NlogpΔS(yt∣st),L_N^hard( _S)=- 1N _t=1^N p_ _S(y_t s_t), (2) whereas distributional level (soft) supervision uses the same prefixes but matches the full teacher distribution: ℒNsoft(ΔS)=1N∑t=1NKL(pΔT(⋅∣st)∥pΔS(⋅∣st)).L_N^soft( _S)= 1N _t=1^NKL\! (p_ _T(· s_t)\,\|\,p_ _S(· s_t) ). (3) Soft supervision retains the sampled carrier prefixes but removes the additional error caused by representing the teacher distribution with one hard target. Recovery protocols and metrics. For a supervision type a∈hard,softa∈\hard,soft\, define the instantaneous full-data gradient at any shared student state v as g^Na(v)=∇ΔSℒNa(ΔS)|ΔS=v. g_N^a(v)= . _ _SL_N^a( _S) |_ _S=v. Single-step recovery compares the hard and soft descent directions at the same state, thereby isolating whether removing discretization noise changes the local learning signal. We use v=0v=0 as the controlled common state in our reported single-step experiments; the diagnostic itself is not specific to initialization. Iterative recovery starts from ΔS(0)=0 _S^(0)=0 and repeatedly optimizes the same objective. Our primary metric is the cosine similarity Align(v)=cos(v,ΔT)=⟨v,ΔT⟩‖v‖2‖ΔT‖2.Align(v)= (v, _T)= v, _T \|v\|_2\,\| _T\|_2. For single-step experiments we report Align(−g^Na(0))Align(- g_N^\,a(0)); for iterative experiments we report Align(ΔS(k))Align( _S^(k)). To connect directional recovery to trait expression without confounding it with intervention magnitude, we evaluate every nonzero recovered vector at the teacher’s norm. Let Q be the set of trait-eliciting questions and let c1c_1 be the first token of the target response. Our norm-normalized target log-likelihood is TraitLLnorm(ΔS)=1||∑q∈logpΔ~S(c1∣q),Δ~S=‖ΔT‖2‖ΔS‖2ΔS.TraitLL_norm( _S)= 1|Q| _q p_ _S(c_1 q), _S= \| _T\|_2\| _S\|_2 _S. This metric measures whether the recovered direction expresses the trait when applied at the same intervention strength as the teacher. 3 Empirical Puzzle: Locally Indistinguishable Signals, Divergent Recovery The coverage and noise hypotheses make different predictions about distributional supervision. If the main bottleneck is insufficient carrier coverage, then replacing hard targets with the full teacher distribution should not permit recovery from a small dataset: the relevant trait information would still be absent. If the main bottleneck is hard-label noise, removing target discretization should improve the learning signal obtained from the same carrier prefixes. We test both predictions in the controlled steering-vector setting. We first compare the local learning signals. At the common state ΔS=0 _S=0, we compute the full-data descent directions from hard NLL and soft KL on matched carrier prefixes. Across model–trait pairs, their mutual cosine is close to one. Nevertheless, both directions have similarly low alignment with ΔT _T and produce weak norm-matched trait expression (Figure 1). Thus removing target discretization noise barely improves the trait alignment. Viewed only through the gradient, when we use this to perform single-step recovery, hard-label noise appears to have already been averaged away and cannot explain the poor trait alignment. Figure 1: Hard and soft supervision are locally similar but diverge under iteration. Rows denote model families and groups denote animal traits. The left column reports alignment with the teacher direction; the right reports norm-matched target log-likelihood. Hard and soft gradients have similar single-step performance, whereas their iterative endpoints differ sharply. The iterative results give the opposite impression. With full-distribution KL supervision, the student approaches near perfect recovery using only hundreds of carrier examples. This shows that, in our controlled setting, a small carrier set already contains enough information to identify ΔT _T; tens of thousands of examples are not required merely to obtain adequate coverage. Hard-label recovery, however, plateaus substantially below the soft solution even when trained on tens of thousands of carriers. Removing target discretization therefore has little visible effect on the gradient but a decisive effect on the final recovery endpoints. These observations rule out either naive explanation in isolation. Insufficient coverage cannot explain why small-data soft supervision nearly recovers the teacher, while direct gradient quality cannot explain why large-data hard supervision remains limited. The resulting puzzle is: How can hard-label error be almost invisible in the local learning signal, yet dominate the endpoint of iterative subliminal learning? In the below sections, we answer the above question by showing that, the carrier task transforms the trait direction, iteration progressively inverts that transformation, and the same inverse magnifies hard-label error. 4 The Carrier Gradient Is a Noisy, Fisher-Transformed Signal 4.1 The trait signal is Fisher-transformed by carrier task Consider one completion position with prefix sts_t. Let zv(st)z_v(s_t) and pv(⋅∣st)p_v(· s_t) be the logits and next-token distribution of the student under intervention v, and define Jt:=∂zv(st)∂v|v=0,p0,t:=p0(⋅∣st),pT,t:=pΔT(⋅∣st).J_t:= . ∂ z_v(s_t)∂ v |_v=0, p_0,t:=p_0(· s_t), p_T,t:=p_ _T(· s_t). Because the same intervention is inserted across all selected layers, JtJ_t includes the combined effect of that shared vector on the output logits. For the token-level supervision ℓt(v,y)=−logpv(y∣st) _t(v;y)=- p_v(y s_t), the chain rule through the softmax gives gt(y):=∇vℓt(v,y)|v=0=Jt⊤(p0,t−ey).g_t(y):= . _v _t(v;y) |_v=0=J_t \! (p_0,t-e_y ). For the hard target yty_t, adding and subtracting pT,tp_T,t in this expression yields the decomposition −gt(yt)=Jt⊤(pT,t−p0,t)⏟−gtsoft(0)−Jt⊤(pT,t−eyt)⏟ξt.-g_t(y_t)= J_t (p_T,t-p_0,t)_-g_t^soft(0)- J_t (p_T,t-e_y_t)_ _t. The first term is the negative gradient of the soft KL objective at v=0v=0; the second is the additional error introduced by replacing the teacher distribution with one sampled target (Menon et al., 2021). Since the expected one-hot target equals the teacher probability vector, [eyt∣st]=∑kpT,t(k)ek=pT,t.E[e_y_t s_t]= _kp_T,t(k)e_k=p_T,t. It follows directly that [ξt∣st]=Jt⊤(pT,t−[eyt∣st])=0E[ _t s_t]=J_t (p_T,t-E[e_y_t s_t])=0. We next characterize the shared soft-gradient term. Let ε=‖ΔT‖2 =\| _T\|_2. A first-order expansion of the teacher logits around the base model gives zΔT(st)−z0(st)=JtΔT+O(ε2).z_ _T(s_t)-z_0(s_t)=J_t _T+O( ^2). Let Λt:=Diag(p0,t)−p0,tp0,t⊤ _t:=Diag(p_0,t)-p_0,tp_0,t be the softmax covariance at the base model. Expanding the output distribution then gives pT,t−p0,t=ΛtJtΔT+O(ε2).p_T,t-p_0,t= _tJ_t _T+O( ^2). Substituting this expression into the soft gradient, −gtsoft(0)=Jt⊤ΛtJt⏟FtΔT+O(ε2),-g_t^soft(0)= J_t _tJ_t_F_t _T+O( ^2), where FtF_t is the Fisher information matrix for the shared intervention at position t (Amari, 1998; Kunstner et al., 2019). Define its population and empirical averages by F=st[Ft],F^N=1N∑t=1NFt.F=E_s_t[F_t], F_N= 1N _t=1^NF_t. The same matrix also determines the second moment of the hard-label noise. Under the idealization that the N supervised tokens are independent, [ξ¯N]=0,Cov(ξ¯N)≈FN.E[ ξ_N]=0, ( ξ_N)≈ FN. (4) Appendix A.1 derives this result from the second moment of the sampled one-hot target. Combining the shared signal and sampling error yields the local observation model −g^Nhard(0) - g_N^\,hard(0) =F^NΔT−ξ¯N+O(ε2), = F_N _T- ξ_N+O( ^2), (5) −g^Nsoft(0) - g_N^\,soft(0) =F^NΔT+O(ε2). = F_N _T+O( ^2). As N increases, F^N F_N approaches F and ξ¯N ξ_N approaches zero. Hard and soft gradients therefore approach the same direction, FΔTF _T, rather than ΔT _T itself. This transformation generally changes direction. Writing F=∑iλiuiui⊤,ΔT=∑iδiui,F= _i _iu_iu_i , _T= _i _iu_i, we have FΔT=∑iλiδiui.F _T= _i _i _iu_i. The carrier task reweights each component of ΔT _T according to how strongly (λi _i) that component changes its output distribution. Unless these weights are equal over all components used by ΔT _T, the carrier task gradient is a distortion ΔT _T. Consequently, increasing N can make the empirical gradient a more precise estimate of FΔTF _T, but cannot remove the transformation itself. Using the recovery metric defined in Section 2, the limiting single-step alignment is therefore limN→∞Align(−g^Nsoft(0))=Align(FΔT)=cos(FΔT,ΔT). _N→∞Align\! (- g_N^\,soft(0) )=Align(F _T)= (F _T, _T). The same limit holds for hard supervision when the averaged sampling error vanishes, accounting for the shared empirical ceiling. 4.2 Explicit correction validates the transformation If F causes the low first-step alignment, partially inverting it should improve recovery. We evaluate Δ^(α,γ)=(F^+γI)−α/2(−g^Nhard(0)), (α,γ)=( F+γ I)^-α/2(- g_N^hard(0)), where α controls inverse strength and γ damps poorly estimated flat directions (Tikhonov and Arsenin, 1977; Engl et al., 1996). We estimate the full dh×dhd_h× d_h Fisher at the unsteered model using 30k clean carrier contexts and four resampled targets per context. Appendix A.4 describes the Rademacher estimator for F. Figure 2: Fisher inversion improves single-step alignment across models and traits. Hyperparameters are selected against ΔT _T. Figure 2 compares the uncorrected mean gradient with the best damped Fisher-inverted estimate. Across all five traits, inversion produces a large alignment gain. This cross-trait consistency supports a shared geometric explanation in which the carrier task reweights the components of ΔT _T. The diagnostic deliberately uses access to the known teacher vector to select the optimal (α,γ)(α,γ) pair separately for each trait. It therefore tests the predicted Fisher form of the distortion rather than providing a teacher-free estimator. The consistent improvement shows that the initial gradient is not merely weak or noisy: it is transformed by carrier curvature. Explicit correction nevertheless remains below ordinary iterative recovery. The next section explains this gap by showing that iterative optimization progressively inverts the same transformation. 5 Iteration Progressively Inverts the Carrier Transformation 5.1 One dynamics contains hard and soft supervision Extending the local model in Equation 5 along the bounded trajectory gives ∇ΔSℒNhard(ΔS)≈F(ΔS−ΔT)+ξ¯N, _ _SL^hard_N( _S)≈ F( _S- _T)+ ξ_N, (6) where ξ¯N ξ_N is the fixed hard-label sampling error characterized in Section 4.1. Soft supervision is the noiseless special case ξ¯N=0 ξ_N=0. For this soft objective, ΔS=ΔT _S= _T matches the teacher distribution on every observed prefix and yields zero KL exactly. Thus curvature changes the optimization path without changing the noiseless target. Approximating F and ξ¯N ξ_N as constant along the local trajectory, gradient descent with learning rate η obeys (Landweber, 1951; Engl et al., 1996) ΔS(k+1)=ΔS(k)−η[F(ΔS(k)−ΔT)+ξ¯N]. _S^(k+1)= _S^(k)-η [F( _S^(k)- _T)+ ξ_N ]. (7) Starting from ΔS(0)=0 _S^(0)=0, its solution is ΔS(k)=[I−(I−ηF)k]ΔT−Pkξ¯N,Pk:=η∑j=0k−1(I−ηF)j. _S^(k)= [I-(I-η F)^k ] _T-P_k ξ_N, P_k:=η _j=0^k-1(I-η F)^j. (8) The derivation is given in Appendix A.3. Setting ξ¯N=0 ξ_N=0 gives the soft-supervision trajectory ΔS(k)=[I−(I−ηF)k]ΔT. _S^(k)= [I-(I-η F)^k ] _T. (9) Writing F=∑iλiuiui⊤F= _i _iu_iu_i and ΔT=∑iδiui _T= _i _iu_i makes this noiseless trajectory explicit: ΔS(k)=∑i[1−(1−ηλi)k]⏟ϕk(λi)δiui. _S^(k)= _i [1-(1-η _i)^k ]_ _k( _i) _iu_i. (10) The eigenvalue λi _i determines the convergence speed of direction uiu_i: directions with larger eigenvalues approach their teacher components more quickly, whereas directions with smaller eigenvalues require more steps (Yao et al., 2007; Bauer et al., 2007; Lo Gerfo et al., 2008; Advani et al., 2020). ϕk(λi)→1 _k( _i)→ 1 as k→∞k→∞ for every λi>0 _i>0. Thus, although different directions converge at different rates, every component is eventually restored. We test this prediction in the noise-free setting by training on soft labels from only 500 Qwen carrier sequences and projecting all 10K iterates into the Fisher eigenbasis. Figure 3 groups directions into 64 contiguous bins with approximately equal teacher-vector energy. Recovery advances from steep to flat bins, as predicted by Equation 10. At step 10,000, recovery is 0.770.77 in the flattest coarse bin and 0.990.99 in the steepest. The observed vector has cosine 0.9690.969 with the trajectory predicted by the fixed-F model, showing that the model captures the ordering and most of the trajectory while mildly overestimating the speed of flat-direction recovery. Figure 3: Iteration progressively restores flatter directions. (a) Recovery of 64 equal-ΔT _T-energy Fisher bins under soft-KL optimization; curvature increases from bottom to top. (b) Observed recovery for representative coarse bins (solid) and the fixed-F prediction ϕk(λ) _k(λ) (dashed). 5.2 Iteration implements a progressively less damped inverse The same dynamics can be written as an operation on the initial learning signal: ΔS(k)=Pk(FΔT), _S^(k)=P_k(F _T), (11) Along an eigenvector of F with eigenvalue λ>0λ>0, the operator PkP_k has scalar response pk(λ)=1−(1−ηλ)kλ.p_k(λ)= 1-(1-ηλ)^kλ. (12) This expression connects the two empirical endpoints. Let K denote the number of optimization steps performed after the initial gradient update. At K=0K=0 (equivalently, k=1k=1 above), P1=ηIP_1=η I and no inverse correction has yet occurred. The recovered direction is therefore FΔTF _T, explaining the shared low single-step cosine similarity in Section 3. As K→∞K→∞, pk(λ)→1/λp_k(λ)→ 1/λ for every direction. The carrier transformation is then fully inverted and PkFΔT→ΔTP_kF _T→ _T, explaining the near-perfect recovery obtained by iterative KL optimization. Intermediate values of K interpolate between these endpoints by correcting progressively flatter directions (Ali et al., 2019). The remaining question is, the soft and hard supervision gradient converge as N grows, then why hard supervision does not follow the same trajectory as the soft one. We address this next by analyzing how discretization noise is amplified as optimization attempts to invert progressively flatter directions. 6 Data Scale Controls the Depth of Hard-Label Inversion Section 5.2 shows that optimization progressively removes the carrier transformation, reaching flatter components only at later steps. Under hard supervision, however, each additional stage of inversion acts not only on the teacher signal but also on the fixed sampling error derived in Section 4.1. Recovery therefore has a data-dependent optimal depth: early updates reduce carrier-induced bias, whereas sufficiently late updates amplify sampled-label error faster than they recover the remaining signal. We show that increasing the number of independent carriers postpones this crossover, allowing the student to follow the noiseless trajectory longer and recover progressively flatter components. 6.1 More data permits deeper inversion before noise dominates The hard and soft mean gradients in Section 3 are nearly collinear, yet their iterative endpoints differ sharply. The reason is that optimization simultaneously removes carrier-induced bias and amplifies the small hard-label error fixed by the sampled dataset. Combining Cov(ξ¯N)≈F/NCov( ξ_N)≈ F/N from Section 4.1 with Equation 8, and writing F=∑iλiuiui⊤F= _i _iu_iu_i , gives δ^i(k)=ϕk(λi)δi−pk(λi)ξi,pk(λ)=1−(1−ηλ)kλ, δ_i^(k)= _k( _i) _i-p_k( _i) _i, p_k(λ)= 1-(1-ηλ)^kλ, (13) where δi=⟨ΔT,ui⟩ _i= _T,u_i and ξi=⟨ξ¯N,ui⟩ _i= ξ_N,u_i . Since Var(ξi)≈λi/NVar( _i)≈ _i/N and the noise is mean zero, the expected component-wise error separates into [(δ^i(k)−δi)2]≈[1−ϕk(λi)]2δi2⏟remaining inversion bias+λiNpk(λi)2⏟amplified label noise.E\! [( δ_i^(k)- _i)^2 ]≈ [1- _k( _i)]^2 _i^2_remaining inversion bias+ _iNp_k( _i)^2_amplified label noise. (14) Increasing k decreases the first term but increases the second toward 1/(Nλi)1/(N _i). Hard-label training therefore has a noise-limited optimal depth k⋆k : beyond it, further inversion increases total error. Increasing N lowers only the variance term, so it should postpone k⋆k and permit optimization to recover progressively slower, flatter components before noise dominates (Raskutti et al., 2014). We test this prediction using matched hard and soft Qwen cat-carrier trajectories with M∈500,1000,2000,4000,8000,16000M∈\500,1000,2000,4000,8000,16000\ independent carrier sequences. Their nearly constant mean completion length makes the supervised token count N proportional to M. At each logged step we project into 64 equal-teacher-energy bins of the clean-carrier Fisher basis and average the normalized hard error RH(k)=164∑b=164‖Pb(ΔSH(k)−ΔT)‖22‖PbΔT‖22,R^H(k)= 164 _b=1^64 \|P_b( _S^H(k)- _T)\|_2^2\|P_b _T\|_2^2, (15) with the soft error defined analogously. The model, teacher vector, optimizer, and 1,000-step horizon are otherwise fixed. Figure 4: Left: hard-label spectral error initially falls, then rises as amplified noise overtakes the remaining inversion bias; dots mark each minimum. Right: best alignment at a matched completion budget B, or repeatedly sampling generation from a fixed pool of 500 prompts for B/500B/500 times. The left panel of Figure 4 exhibits precisely this tradeoff. For N=500N=500, hard error reaches its minimum after only 100 updates and then rises to 2.7562.756 by step 1,000, even as the soft reference keeps improving. As M doubles, k⋆k moves systematically from 100 to 190, 330, 560, and 880 updates. Larger datasets delay the point at which inverse-amplified sampling error overtakes the remaining bias, allowing the student to follow the noiseless trajectory longer and recover flatter components. The right panel is designed to understand the effectiveness of independent data. At a matched completion budget B, it compares expanding the prompt pool with repeatedly sampling from a fixed pool of 500 prompts. We found that 500 independent samples is sufficient for recovery; the effectiveness of adding more independent samples is equivalent for reducing hard label noise via sampling more responses from the small prompt pool. 6.2 Trait spectral location is associated with recovery difficulty The factor 1/(Nλi)1/(N _i) predicts that recovery should be harder when more of the teacher direction lies at low carrier curvature. We test this prediction across Qwen2.5-7B and Gemma-2-9B using the same five animal traits. For each model, we project ΔT _T into its clean-carrier Fisher eigenbasis and plot the energy ⟨ΔT,ui⟩2/‖ΔT‖2 _T,u_i ^2/\| _T\|^2 against normalized curvature λi/λ¯ _i/ λ (Figure 5) (Hansen, 1998). Across all five traits, the main mass of Gemma’s distribution is shifted toward lower normalized curvature. This shift is consistent with Gemma’s lower recovery, because more of its teacher direction must be inferred through slow, noise-sensitive components. Figure 5: Gemma trait energy lies at lower normalized curvature. Teacher-vector energy across common bins of λ/λ¯λ/ λ for five traits; each spectrum is normalized by its model-specific mean. 7 Conclusion We asked whether subliminal learning requires large carrier datasets because the trait is insufficiently represented or because sampled targets introduce error. Controlled experiments show that hard and soft supervision produce nearly identical but distorted initial gradients, yet soft supervision recovers the teacher direction from hundreds of carriers while hard recovery remains limited. Our noisy-inverse model explains why: carrier Fisher geometry transforms the teacher direction, iteration reverses this transformation, and the same inverse amplifies fixed hard-label error, especially in weakly observed directions. Large datasets therefore make deeper inversion reliable rather than merely revealing an absent trait. This unifies the shared single-step ceiling, divergent iterative endpoints, and the advantage of new independent data over repeated passes on fixed samples. Acknowledgments This work is partially supported by Australia ARC LP220100453, ARC DP200101374, and ARC DP240100955. References Advani et al. (2020) M. S. Advani, A. M. Saxe, and H. Sompolinsky High-dimensional dynamics of generalization error in neural networks. Neural Networks 132, p. 428–446. External Links: Document Cited by: Appendix B, §5.1. Ali et al. (2019) A. Ali, J. Z. Kolter, and R. J. Tibshirani A continuous-time view of early stopping for least squares regression. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, Vol. 89, p. 1370–1378. Cited by: Appendix B, §5.2. Amari (1998) S. Amari Natural gradient works efficiently in learning. Neural Computation 10 (2), p. 251–276. External Links: Document Cited by: Appendix B, §4.1. Bauer et al. (2007) F. Bauer, S. Pereverzev, and L. Rosasco On regularization algorithms in learning theory. Journal of Complexity 23 (1), p. 52–72. External Links: Document Cited by: Appendix B, §5.1. Bissantz et al. (2007) N. Bissantz, T. Hohage, A. Munk, and F. Ruymgaart Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM Journal on Numerical Analysis 45 (6), p. 2610–2636. External Links: Document Cited by: Appendix B. Blank et al. (2026) C. Blank, A. Bhatia, S. Rajamanoharan, A. Conmy, and N. Nanda Subliminal learning is steering vector distillation. arXiv preprint. Note: Stanford University Cited by: Appendix B, Appendix B, §1. Cavalier (2008) L. Cavalier Nonparametric statistical inverse problems. Inverse Problems 24 (3), p. 034004. External Links: Document Cited by: Appendix B. Cloud et al. (2026) A. Cloud, M. Le, J. Chua, J. Betley, A. Sztyber-Betley, S. Mindermann, J. Hilton, S. Marks, and O. Evans Language models transmit behavioural traits through hidden signals in data. Nature. External Links: Document Cited by: Appendix B, §1. Dong et al. (2026) Z. Dong, Z. Liu, Y. Fan, X. Li, S. Zhang, and C. Yang Scaling model-generated distillation data can make latent teacher traits more recoverable. arXiv preprint arXiv:2608.26958. Cited by: Appendix B, §1. Engl et al. (1996) H. W. Engl, M. Hanke, and A. Neubauer Regularization of inverse problems. Kluwer Academic Publishers, Dordrecht. External Links: Document Cited by: Appendix B, §4.2, §5.1. Feng (2026) L. Feng Subliminal learning happens at every rank, given the right learning rate and enough data. Note: LessWrong External Links: Link Cited by: Appendix B, §1. Hansen (1998) P. C. Hansen Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. Monographs on Mathematical Modeling and Computation, Society for Industrial and Applied Mathematics, Philadelphia, PA. External Links: Document Cited by: Appendix B, §6.2. Hinton et al. (2015) G. Hinton, O. Vinyals, and J. Dean Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. Cited by: Appendix B, §1. Kunstner et al. (2019) F. Kunstner, P. Hennig, and L. Balles Limitations of the empirical Fisher approximation for natural gradient descent. In Advances in Neural Information Processing Systems, Vol. 32. Cited by: Appendix B, §4.1. Landweber (1951) L. Landweber An iteration formula for Fredholm integral equations of the first kind. American Journal of Mathematics 73 (3), p. 615–624. External Links: Document Cited by: Appendix B, §5.1. Lo Gerfo et al. (2008) L. Lo Gerfo, L. Rosasco, F. Odone, E. De Vito, and A. Verri Spectral algorithms for supervised learning. Neural Computation 20 (7), p. 1873–1897. External Links: Document Cited by: Appendix B, §5.1. Menon et al. (2021) A. K. Menon, A. S. Rawat, S. J. Reddi, S. Kim, and S. Kumar A statistical perspective on distillation. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, p. 7632–7642. Cited by: Appendix B, §4.1. Morgulis and Hewitt (2026) G. Morgulis and J. Hewitt Subliminal steering: stronger encoding of hidden signals. arXiv preprint arXiv:2604.25783. Cited by: Appendix B, §1, §1. Raskutti et al. (2014) G. Raskutti, M. J. Wainwright, and B. Yu Early stopping and non-parametric regression: an optimal data-dependent stopping rule. Journal of Machine Learning Research 15, p. 335–366. Cited by: Appendix B, §6.1. Schrodi et al. (2025) S. Schrodi, E. Kempf, F. Barez, and T. Brox Towards understanding subliminal learning: when and how hidden biases transfer. arXiv preprint arXiv:2509.23886. Cited by: Appendix B, §1. Tikhonov and Arsenin (1977) A. N. Tikhonov and V. Y. Arsenin Solutions of ill-posed problems. V. H. Winston & Sons, Washington, DC. Cited by: Appendix B, §4.2. Yao et al. (2007) Y. Yao, L. Rosasco, and A. Caponnetto On early stopping in gradient descent learning. Constructive Approximation 26 (2), p. 289–315. External Links: Document Cited by: Appendix B, §5.1. Appendix A Derivations and Fisher Estimation A.1 Hard-label noise covariance For a fixed prefix sts_t and yt∼pT,ty_t p_T,t, the one-hot target satisfies [eyt∣st]=pT,t,[eyteyt⊤∣st]=Diag(pT,t).E[e_y_t s_t]=p_T,t, [e_y_te_y_t s_t]=Diag(p_T,t). The second identity follows because eyteyt⊤e_y_te_y_t contains one nonzero diagonal entry at the sampled class. Therefore Cov(eyt∣st) (e_y_t s_t) =[eyteyt⊤∣st]−[eyt∣st][eyt∣st]⊤ =E[e_y_te_y_t s_t]-E[e_y_t s_t]E[e_y_t s_t] =Diag(pT,t)−pT,tpT,t⊤=:ΛT,t. =Diag(p_T,t)-p_T,tp_T,t =: _T,t. Since ξt=Jt⊤(pT,t−eyt) _t=J_t (p_T,t-e_y_t), [ξt∣st]=0,Cov(ξt∣st)=Jt⊤ΛT,tJt=:FT,t.E[ _t s_t]=0, ( _t s_t)=J_t _T,tJ_t=:F_T,t. For N independent supervised tokens, Cov(ξ¯N) ( ξ_N) =1N2∑t=1Nst[FT,t] = 1N^2 _t=1^NE_s_t[F_T,t] ≈FN. ≈ FN. The final approximation uses the local regime pT,t≈p0,tp_T,t≈ p_0,t, under which the teacher-position Fisher averages to the base-model carrier Fisher. Within-sequence correlations can change the finite-data covariance. For Fui=λiuiFu_i= _iu_i, ‖ui‖2=1\|u_i\|_2=1, and ξi=ui⊤ξ¯N _i=u_i ξ_N, Var(ξi) ( _i) =ui⊤Cov(ξ¯N)ui =u_i Cov( ξ_N)u_i ≈1Nui⊤Fui=λiN. ≈ 1Nu_i Fu_i= _iN. A.2 Score identity and distribution mismatch For a fixed prefix, JtJ_t does not depend on the sampled target. If y∼p0(⋅∣st)y p_0(· s_t), then y[gt(y)gt(y)⊤] _y[g_t(y)g_t(y) ] =Jt⊤y[(p0,t−ey)(p0,t−ey)⊤]Jt =J_t E_y[(p_0,t-e_y)(p_0,t-e_y) ]J_t =Jt⊤[Diag(p0,t)−p0,tp0,t⊤]Jt =J_t [Diag(p_0,t)-p_0,tp_0,t ]J_t =Jt⊤ΛtJt=Ft. =J_t _tJ_t=F_t. If targets instead come from the teacher while scores are evaluated at the base model, y∼pT,t[(p0,t−ey)(p0,t−ey)⊤]=ΛT,t+(pT,t−p0,t)(pT,t−p0,t)⊤.E_y p_T,t[(p_0,t-e_y)(p_0,t-e_y) ]= _T,t+(p_T,t-p_0,t)(p_T,t-p_0,t) . The second term contains teacher signal rather than carrier curvature. This is why the correction experiment estimates F on clean base-model carriers. A.3 Finite-step solution Under the constant-F local model, gradient descent obeys ΔS(k+1)=(I−ηF)ΔS(k)+η(FΔT−ξ¯N). _S^(k+1)=(I-η F) _S^(k)+η(F _T- ξ_N). Starting from zero and unrolling gives ΔS(k)=η∑j=0k−1(I−ηF)j(FΔT−ξ¯N)=Pk(FΔT−ξ¯N), _S^(k)=η _j=0^k-1(I-η F)^j(F _T- ξ_N)=P_k(F _T- ξ_N), which is Equation 8. The geometric-series identity PkF=I−(I−ηF)kP_kF=I-(I-η F)^k yields the soft trajectory in Equation 9. A.4 Rademacher estimation on clean carriers Let M be the number of clean carrier contexts and ℐmI_m the completion-token indices in context m. For each of R draws, sample y~t(r)∼p0(⋅∣st),ρt(r)∼Unif−1,+1, y_t^(r) p_0(· s_t), _t^(r) \-1,+1\, and compute in one backward pass qm(r)=1|ℐm|∑t∈ℐmρt(r)gt(y~t(r)).q_m^(r)= 1 |I_m| _t _m _t^(r)g_t( y_t^(r)). Our sequence-equal estimate is F^=1MR∑m=1M∑r=1Rqm(r)qm(r)⊤. F= 1MR _m=1^M _r=1^Rq_m^(r)q_m^(r) . Because independent Rademacher signs satisfy [ρtρt′]=0E[ _t _t ]=0 for t≠t′t≠ t , ρ[qm(r)qm(r)⊤]=1|ℐm|∑t∈ℐmgt(y~t(r))gt(y~t(r))⊤.E_ρ[q_m^(r)q_m^(r) ]= 1|I_m| _t _mg_t( y_t^(r))g_t( y_t^(r)) . Taking expectation over resampled targets and applying the score identity gives the average token Fisher. We use M=30,000M=30,000 and R=4R=4. Resampling from p0p_0 and using clean carrier contexts prevents the teacher signal from contaminating the curvature estimate. Appendix B Related Work Subliminal learning. Knowledge distillation conventionally transfers a teacher’s predictive behavior by matching its outputs (Hinton et al., 2015). Subliminal learning shows that model-generated data can additionally transmit behavioral traits that are not semantically expressed in those data (Cloud et al., 2026). Subsequent work has studied when this transfer occurs across architectures, carrier tasks, and intervention parameterizations (Schrodi et al., 2025; Blank et al., 2026). These studies establish the generality of hidden trait transfer, whereas we focus on the optimization geometry that determines how accurately a known trait direction can be recovered. Steering-vector accounts. Recent work connects subliminal transfer to the distillation of latent steering directions (Blank et al., 2026) and introduces subliminal steering as a controlled way to encode stronger, directly measurable hidden signals (Morgulis and Hewitt, 2026). We adopt this controlled setting because it exposes both the teacher direction and the recovered student direction. Our analysis complements these accounts by showing that the carrier task first transforms the teacher direction through its Fisher geometry and that iterative optimization acts as an increasingly strong inverse of that transformation. Data scale and signal localization. Empirically, increasing the number of independently generated distillation examples makes latent teacher traits more recoverable (Dong et al., 2026). Related evidence emphasizes that independent carrier diversity is more useful than repeatedly training on a fixed dataset and links transfer to token positions at which teacher predictions diverge (Feng, 2026). These observations explain where trait-dependent signals can appear, but do not characterize why recovery continues to depend on large independent datasets after the mean learning direction has stabilized. Our noisy-inverse account separates these roles: carrier geometry determines the systematic transformation of the trait, while independent hard targets control the residual discretization error that is amplified as optimization inverts that transformation. Inverse problems and iterative regularization. The locally linearized recovery dynamics are a finite-dimensional instance of Landweber iteration (Landweber, 1951): the iteration count acts as a regularization parameter, and each Fisher eigendirection is governed by a spectral filter. This perspective is classical in inverse problems (Tikhonov and Arsenin, 1977; Engl et al., 1996; Hansen, 1998), including the use of Tikhonov-style damping and early stopping to suppress unstable low-curvature modes. Statistical inverse-problem theory further characterizes the corresponding bias–variance tradeoff when the observation error is random and decreases with sample size (Bissantz et al., 2007; Cavalier, 2008). Our plots of trait energy across the Fisher spectrum are also related to spectral source and discrete Picard conditions, which describe how the target coefficients are distributed relative to an ill-posed operator’s spectrum (Hansen, 1998). Spectral filtering and early stopping in learning. Gradient descent has likewise been studied as a spectral regularization method in learning theory (Yao et al., 2007; Bauer et al., 2007; Lo Gerfo et al., 2008). This literature establishes consistency and stopping rules for iterative regularization (Raskutti et al., 2014), relates gradient-flow paths to ridge regularization (Ali et al., 2019), and analyzes mode-dependent learning and overtraining in high-dimensional models (Advani et al., 2020). Our contribution is therefore not the general observation that gradient descent implements a spectral filter. We identify the carrier Fisher as the relevant forward operator for subliminal steering, derive the hard-label forcing term and its sample-size dependence, and test the resulting mode-wise and data-scaling predictions in an auditable parameter-recovery setting. Fisher geometry and distributional supervision. Fisher preconditioning is central to natural-gradient methods (Amari, 1998); importantly, the model Fisher used here is distinct from the observed-label outer product often called the empirical Fisher (Kunstner et al., 2019). Prior statistical accounts of knowledge distillation show that replacing sampled hard labels with teacher probabilities can reduce the variance of the student objective (Menon et al., 2021). In our setting, this variance reduction has a specific inverse-problem consequence: distributional supervision retains the carrier-transformed signal while removing the conditional token-sampling noise that would otherwise be amplified along low-curvature directions. Appendix C Hyperparameters Table 1 reports the model and data configuration. All pretrained parameters are frozen; only one zero-initialized residual vector is optimized and shared across the listed layers. Unless stated otherwise, we use seed 42, BF16 precision, teacher scale α=1α=1, and temperature 1. Configuration Qwen2.5-7B-Instruct Gemma-2-9B-IT Hidden width / transformer layers 35843584 / 2828 35843584 / 4242 Intervened layers [2,26)[2,26) [2,40)[2,40) Teacher norm bound 0.50.5 1.51.5 Traits cat, dog, eagle, lion, raven Teacher training examples 5050 Teacher optimizer / learning rate Adam / 10−310^-3 Adam / 10−310^-3 Teacher optimization steps / batch size 200200 / 1010 200200 / 1010 Carrier temperature / top-k 1.01.0 / full vocabulary (no truncation) Recovery objectives hard NLL and soft KL Recovery optimizer / learning rate SGD / 10−210^-2 SGD / 0.50.5 Table 1: Model, intervention, teacher-extraction, carrier-generation, and student-recovery hyperparameters. The same recovery optimizer and learning rate are used for hard and soft supervision. Appendix D Limitations and future work Our analysis is local. It linearizes the model around the unsteered state and treats the carrier Fisher matrix F as approximately constant along the bounded recovery trajectory. This approximation captures the observed curvature-dependent ordering of recovery, but changes in the model Jacobian and Fisher geometry can account for deviations from the fixed-F dynamics. We use subliminal steering as an experimental platform because it turns hidden trait transfer into an auditable parameter-recovery problem: both the implanted teacher direction and the recovered student direction are known, so recovery can be measured directly rather than inferred only from behavior. Representing the teacher trait by a single low-dimensional steering vector also makes an explicit Fisher analysis computationally feasible. This control does not cover the full range of subliminal learning. Traits induced by system prompts and students trained through LoRA or full-parameter fine-tuning may involve larger, higher-dimensional, and more nonlinear changes than the bounded steering trajectories studied here. Whether the same noise-dominated inverse mechanism accounts for their data requirements therefore remains to be tested across broader teacher constructions, student parameterizations, and carrier tasks; we leave those further validations for future work. AI use statement Generative AI tools were used to assist with code implementation and debugging, experimental analysis, figure and table preparation, and language editing and organization of the manuscript. The authors reviewed and verified all AI-assisted outputs, independently validated the reported experimental results, and take full responsibility for the scientific claims and final content. Reproducibility statement Code is available at https://github.com/luoyuchenmlcv/subliminal-data.