Paper deep dive
Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation
Yubo Jiang, Fengying Xie, Zhiguo Jiang, Haopeng Zhang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Reinforcement learning with verifiable rewards yields no group-relative signal when rollout groups are uniformly correct or uniformly wrong, which account for 63.0-68.0% of groups in our experiments. We propose SKALD (Skill-Anchored Latent Distillation), an on-policy self-distillation framework that uses two context views of the same Qwen3-Base model: a question-only student and a teacher conditioned on an abstract, explicit-answer-filtered skill card. The student is trained on its own prefixes, transferring the skill-induced advantage into shared parameters without privileged input at test time. To stabilize context-induced distribution mismatch, SKALD employs an annealed exponentially tilted objective that downweights teacher-preferred tokens with very low student likelihood; as the tilt vanishes, it converges to teacher cross-entropy and recovers the forward-KL student gradient. An empirical gate activates distillation only when verified rollouts estimate a positive teacher advantage. Across five held-out mathematics benchmarks, SKALD improves overall avg@8 over GRPO by +2.46, +4.85, and +12.01 at 0.6B, 1.7B, and 4B, respectively. At 1.7B, zero-variance-only distillation recovers 84.7% of the full gain, while SKALD remains +4.06 above FLOP-matched GRPO and exceeds contextual skill exposure by +3.77. These results show that abstract skills provide dense supervision where group-relative rewards become uninformative.
Tags
Links
- Source: https://arxiv.org/abs/2608.09826v1
- Canonical: https://arxiv.org/abs/2608.09826v1
Trouble viewing inline? Open PDF directly →
Full Text
47,782 characters extracted from source content.
Expand or collapse full text
Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation Yubo Jiang1,2 Fengying Xie1,3 Zhiguo Jiang3 Haopeng Zhang1,3†^1,3~ Abstract Reinforcement learning with verifiable rewards yields no group-relative signal when rollout groups are uniformly correct or uniformly wrong, which account for 63.063.0–68.0%68.0\% of groups in our experiments. We propose SKALD (Skill-Anchored Latent Distillation), an on-policy self-distillation framework that uses two context views of the same Qwen3-Base model: a question-only student and a teacher conditioned on an abstract, explicit-answer-filtered skill card. The student is trained on its own prefixes, transferring the skill-induced advantage into shared parameters without privileged input at test time. To stabilize context-induced distribution mismatch, SKALD employs an annealed exponentially tilted objective that downweights teacher-preferred tokens with very low student likelihood; as the tilt vanishes, it converges to teacher cross-entropy and recovers the forward-KL student gradient. An empirical gate activates distillation only when verified rollouts estimate a positive teacher advantage. Across five held-out mathematics benchmarks, SKALD improves overall avg@8 over GRPO by +2.46+2.46, +4.85+4.85, and +12.01+12.01 at 0.6B, 1.7B, and 4B, respectively. At 1.7B, zero-variance-only distillation recovers 84.7%84.7\% of the full gain, while SKALD remains +4.06+4.06 above FLOP-matched GRPO and exceeds contextual skill exposure by +3.77+3.77. These results show that abstract skills provide dense supervision where group-relative rewards become uninformative. Introduction Reinforcement learning with verifiable rewards (RLVR) has become central to improving mathematical reasoning in large language models, yet its outcome supervision is sparse (Chen et al. 2026; Shao et al. 2024; Wu et al. 2025; Yu et al. 2026). In group-relative policy optimization, identical rewards within a rollout group produce zero group-normalized advantages, so that group contributes no reward-gradient signal on the corresponding problem. This algebraic blind spot can occur for both all-correct and all-wrong groups, including difficult problems on which sampled solutions are uniformly unsuccessful (Sun et al. 2024; Xu et al. 2024). On-policy distillation is a natural complement: the student samples its own rollouts while a teacher distribution supplies dense token-level supervision along the sampled trajectory even when outcome rewards do not discriminate. The teacher need not be a separate online model (Wu et al. 2025; Ye et al. 2026b): privileged context can make another view of the same parameters more informative on a given problem, and distillation can transfer that context-induced shift into the deployable question-only branch (Li et al. 2026; Fu et al. 2026; Yang et al. 2026; Zhao et al. 2026b). Figure 1: (a) Common privileged-distillation designs use a larger online teacher or expose a full reference solution to the teacher branch. (b) SKALD instead transfers an explicit-answer-filtered skill through shared-parameter self-distillation. (c) Across the three tested Qwen3-Base sizes, the observed gain over GRPO increases from +2.46+2.46 at 0.6B to +12.01+12.01 at 4B; this is an empirical trend, not a claimed scaling law. The central question is what privileged information to provide. A full reference solution exposes the final answer and a particular derivation to the teacher branch; a larger online teacher avoids that input but adds training and deployment dependencies. SKALD instead conditions the self-teacher on a compact skill abstraction—a relevant principle, when it applies, and common mistakes to avoid—while the student sees the question alone. Both branches score the same student-generated prefixes. At test time, neither the skill nor a teacher branch is required. The method does use a larger model offline to construct and screen cards; our “no larger teacher” claim therefore refers only to online distillation and inference.This design contrasts skill use with skill internalization: a skill in the student prompt acts as a temporary scaffold, whereas teacher-only distillation changes the parameters used by the question-only policy (Shi et al. 2022; Nam et al. 2022; Xu and Yan 2026). In the 1.7B no-skill-at-test comparison, GRPO reaches 45.5245.52, training-time contextual exposure reaches 46.6046.60, and SKALD reaches 50.3750.37. Thus SKALD is +3.77+3.77 above contextual exposure in this setting; the comparison supports a weight-level transfer effect but does not by itself isolate every training difference. Direct teacher cross-entropy can be unstable when a privileged context assigns substantial mass to tokens the student currently considers improbable (Wang et al. 2026; He et al. 2026). SKALD uses the annealed objective −τ−1logv∼q[p(v)τ]-τ^-1 _v q[p(v)^τ]. At τ>0τ>0, it creates a student-dependent effective target that discounts very low-p teacher tokens and has the sharpened stationary solution p∗∝q1/(1−τ)p^* q^1/(1-τ); annealing τ:0.8→0τ:0.8→0 removes this transient mode-seeking bias and recovers teacher cross-entropy, equivalently the forward-KL student gradient under stop-gradient. A fixed empirical gate further filters distillation using an initial sampled accuracy difference; it is a noisy heuristic, not a certificate of teacher superiority throughout training. Across Qwen3-Base models at 0.6B, 1.7B, and 4B, SKALD outperforms GRPO on the problem-weighted average of MATH500 (Lightman et al. 2024), AMC23 (Zhang and Math-AI 2023), AIME24 (Zhang and Math-AI 2024), AIME25 (Zhang and Math-AI 2025), and Minerva (Lewkowycz et al. 2022), with observed gains of +2.46+2.46, +4.85+4.85, and +12.01+12.01. Three sizes from one model family establish an encouraging monotonic trend, not a general scaling law. Our contributions are: • We introduce shared-parameter, on-policy skill distillation that transfers an explicit-answer-filtered abstraction into a question-only policy without a larger online teacher or test-time privileged input. • We characterize the tilted objective as a Rényi-type cross-entropy with a student-dependent escort target, bounded logit-gradient coordinates, and a cross-entropy limit whose student gradient matches forward KL; the contribution is its annealed use under privileged on-policy mismatch. • We directly localize the gain to zero-variance rollout groups and validate it with conditional-leakage audits, problem/solution/skill decontamination, hierarchical uncertainty, equal-compute GRPO, gate sensitivity, and a full 4B component ablation. Figure 2: SKALD overview. The skill-conditioned teacher and question-only student branches are two context views of the same Qwen3-Base checkpoint and share all parameters. The teacher distribution is evaluated under stop-gradient, while gradients flow through the student branch. For τ>0τ>0, the tilted objective induces the effective target rτ(v)∝q(v)p(v)τr_τ(v) q(v)p(v)^τ, suppressing teacher-preferred tokens with very low student likelihood; it therefore does not preserve q as the stationary solution. A fixed empirical gate activates distillation when initial sampled teacher accuracy exceeds student accuracy. At inference, the model receives only the question. Related Work SKALD sits at the intersection of four threads: on-policy distillation, learning with privileged information, stabilization under distribution mismatch, and skill-based reasoning. On-policy distillation. Classical sequence-level distillation trains on teacher corpora, inducing a train–inference mismatch (Kim and Rush 2016; Agarwal et al. 2024; Ye et al. 2026b). GKD removes this mismatch by distilling on the student’s own samples and composes naturally with RL (Ye et al. 2022; Zhao et al. 2026b); at τ=0τ=0, SKALD reduces to on-policy teacher cross-entropy (and hence the forward-KL student gradient); its primary departure is the source of supervision. Many GKD-style methods use an external larger teacher, creating a capacity gap and additional training cost (Gou et al. 2021; Cho and Hariharan 2019). SKALD’s online teacher is the same weights in a privileged context and adds no inference-time dependency (Tian et al. 2025; Yang et al. 2025b; Mansourian et al. 2025), but it still requires an extra teacher-scoring forward pass during training; end-to-end accounting is in Supplementary Section 2.6. Privileged self-teachers. Learning using privileged information lets a teacher observe training-only features the student never will (Vapnik and Vashist 2009; Lopez-Paz et al. 2015). Its instantiation for reasoning LLMs is OPSD (Zhao et al. 2026b), whose teacher branch sees the per-instance reference solution, final answer included (Zhao et al. 2026a). Full solutions expose the answer and one reference derivation to the teacher branch and can increase teacher–student mismatch (Penaloza et al. 2026). SKALD supplies an explicit-answer-filtered skill instead—a principle, its applicability, and common mistakes—and contains the OPSD objective as a special case: setting s=y∗s=y^* and τ=0τ=0 recovers its objective, with skill abstraction and risk tilting independently ablatable (Garcia et al. 2019). π-Distill (Penaloza et al. 2026) trains a privileged-conditioned teacher and an unconditioned student in shared parameters as parameter-shared variational EM; SKALD instead makes the privileged signal an explicit, discrete, library-backed skill, estimates its usefulness per problem with an outcome-based empirical gate, and internalizes it via annealed on-policy cross-entropy rather than implicit variational inference (Liu et al. 2025). Generalized cross-entropies and mismatch. Equation (1) is not a new divergence in isolation. With order α=1+τα=1+τ, it is a Rényi cross-entropy of the teacher with respect to the student under one established definition, and can equivalently be viewed as the negative logarithm of a teacher-weighted generalized power mean (Valverde-Albacete and Peláez-Moreno 2019; Thierrin et al. 2022). It is related to, but distinct from, standard Rényi/Chernoff divergences, whose power exponents are coupled to sum to one (Van Erven and Harremos 2014). It also differs from standard temperature distillation, which explicitly rescales logits: here the teacher probabilities are not rewritten, but the optimization induces the student-dependent effective target rτ(v)∝q(v)p(v)τr_τ(v) q(v)p(v)^τ and the stationary escort p∗∝q1/(1−τ)p^* q^1/(1-τ). We therefore claim novelty for the shared-context, privileged on-policy construction and its annealed use, not for the power-mean functional itself. Skills: context versus weights. A parallel line treats natural-language skills as first-class objects: SkillRL (Xia et al. 2026) evolves a hierarchical library of reusable skills alongside the policy, and SKILL0 (Lu et al. 2026) places skills in the policy context during training and withdraws them under a helpfulness curriculum (Snell et al. 2022; Laskin et al. 2022; Ye et al. 2026a). Both keep the skill on the student’s input, so the model reasons with the scaffold and must survive its removal. SKALD takes the complementary route: the skill appears only in the teacher, never in the student’s context, and distillation writes its effect directly into the weights. Under a no-skill-at-test protocol the two routes are comparable, and our no-skill-at-test experiment shows that weight-level transfer is +3.77+3.77 points above contextual exposure with withdrawal (§4.4). Method SKALD trains a single model through two context views. Skill conditioning can, but need not, improve the model’s conditional distribution on a given problem; the empirical gate is designed to filter cases in which a sampled improvement is not observed. A skill-conditioned branch scores the rollouts of a question-only branch, and an annealed cross-entropy objective transfers their distributional difference into the shared weights. Figure 2 gives an overview. We present the two-context setup (§3.1), the tilted cross-entropy and its effective target (§3.2), the full objective (§3.3), and the method’s design desiderata and scope (§3.4). Setup: One Model, Two Contexts For each scale, a single Qwen3-Base checkpoint pθp_θ is instantiated as two context views that share all parameters. The student pS(⋅∣x):=pθ(⋅∣renderS(x))p_S(· x):=p_θ(· _S(x)) sees only the problem x and matches the deployment condition exactly; the self-teacher pT(⋅∣x,s):=pθ(⋅∣renderT(x,s))p_T(· x,s):=p_θ(· _T(x,s)) additionally sees an abstract skill s. This construction eliminates the two standing costs of external teachers at once: there is no capacity mismatch, because teacher and student share every parameter, and no serving dependency, because the teacher exists only as an alternative prompt (Ye et al. 2026b; Zhang et al. 2026). All trajectories y y are sampled from the student. For each position, define pn:=pS(⋅∣x,y^<n)p_n:=p_S(· x, y_<n) and qn:=sg[pT(⋅∣x,s,y^<n)]q_n:=sg[p_T(· x,s, y_<n)], where sgsg denotes stop-gradient. This operation freezes the teacher only within one update: because the branches share parameters, the teacher distribution changes after every optimizer step. SKALD is therefore a coupled moving-target self-distillation procedure, not distillation from a fixed teacher. During optimization, the teacher scores student rollouts in one prefill pass; separate teacher rollouts are used only to precompute the usefulness gate. This asymmetry keeps all distillation prefixes on-policy under the student, avoiding supervision on teacher-generated prefix distributions. The skill is a structured card s=title,principle,usage,common mistakess=\ title, principle, usage, common mistakes\ mined offline from training solutions. The pipeline retains 11,38211,382 cards from 17,38417,384 candidates and merges them into 1,9261,926 reusable skills; a skill is linked to a median of 55 problems (mean 7.47.4), and 93.1%93.1\% of training problems receive an assignment. A larger Qwen3-14B model is used only for offline extraction and screening, not online distillation or inference. In a conditional audit, answer recovery is 2.4%2.4\% from the question alone, 3.7%3.7\% with a shuffled skill, 5.1%5.1\% with the matched skill, and 94.2%94.2\% with the full solution; removing instance-specific numerals, entities, and intermediate expressions changes avg@8 only from 50.3750.37 to 50.2450.24. These controls substantially narrow, but cannot logically eliminate, conditional-leakage concerns (Supplementary Section 1.3). Boxed-answer extraction yields the binary reward R:=correct(x,y)R:=correct(x,y). The default gate is precomputed from the initial shared checkpoint using K=8K=8 rollouts per branch and remains fixed. It is an empirical filter, not a certificate: it covers 61.2%61.2\% of training problems, and 14.6%14.6\% of initially positive cases reverse sign by training’s end. Nevertheless, hard/soft and fixed/refreshed variants score within 49.9849.98–50.4850.48 at 1.7B (Supplementary Section 2.4) (Liao et al. 2025; Jiang et al. 2026). Annealed Tilted Cross-Entropy Direct teacher cross-entropy can be dominated by positions at which the teacher assigns mass to tokens that the student currently considers highly improbable. Let ℓn(v):=logpn(v) _n(v):= p_n(v) and dn(v):=−ℓn(v)d_n(v):=- _n(v). For τ>0τ>0, SKALD uses ℒn(τ) ^(τ)_n =−1τlogv∼qn[eτℓn(v)] =- 1τ _v q_n\! [e^τ _n(v) ] (1) =−1τlog∑vqn(v)pn(v)τ. =- 1τ _vq_n(v)p_n(v)^τ. We define ℒn(0):=H(qn,pn)L^(0)_n:=H(q_n,p_n) by continuity. Thus limτ→0ℒn(τ)=H(qn,pn)=KL(qn∥pn)+H(qn). _τ→ 0L^(τ)_n=H(q_n,p_n)=KL(q_n\|p_n)+H(q_n). (2) Because qnq_n is stop-gradient within an update, the limit has exactly the same student gradient as forward KL, but its numerical value differs by the teacher entropy H(qn)H(q_n). For small positive τ, the cumulant expansion is ℒn(τ)=qn[dn]−τ2Varqn[dn]+O(τ2).L^(τ)_n=E_q_n[d_n]- τ2Var_q_n[d_n]+O(τ^2). (3) The negative variance correction is not a variance penalty. It is an optimistic entropic risk for the loss dnd_n: high-disutility tokens are discounted relative to ordinary cross-entropy. Lemma 1 (Effective Target, Logit Gradient, and Stationarity). Let rn(τ)(v):=qn(v)pn(v)τ∑uqn(u)pn(u)τ.r_n^(τ)(v):= q_n(v)p_n(v)^τ _uq_n(u)p_n(u)^τ. If znz_n denotes the student logits, then ∇znℒn(τ)=pn−rn(τ). _z_nL^(τ)_n=p_n-r_n^(τ). Consequently, each logit-gradient coordinate lies in [−1,1][-1,1]. A bound on ‖∇θℒn(τ)‖\| _θL^(τ)_n\| additionally requires a bound on the logit Jacobian with respect to θ. For 0<τ<10<τ<1, the minimizer over the probability simplex is pn∗(v)=qn(v)1/(1−τ)∑uqn(u)1/(1−τ).p_n^*(v)= q_n(v)^1/(1-τ) _uq_n(u)^1/(1-τ). At τ=0τ=0, rn(0)=qnr_n^(0)=q_n and pn∗=qnp_n^*=q_n. A complete proof is provided in Supplementary Section 3. Lemma 1 makes the optimization bias explicit. At the initial τ=0.8τ=0.8, the stationary distribution is proportional to qn5q_n^5, a strongly sharpened escort rather than the original teacher distribution. We use this mode-seeking bias only transiently: as the student’s support improves and τ is annealed to zero, the effective target approaches qnq_n and the objective approaches teacher cross-entropy. The extra arithmetic is small once teacher and student logits are available, but SKALD as a whole is not free relative to GRPO. We use λR=λD=1 _R= _D=1 at all scales; τ=0.8τ=0.8 for the first 10%10\% of 370370 updates, decay it linearly to 0 by 80%80\%, and keep τ=0τ=0 for the last 20%20\%. Learning rates are 2×10−62\!×\!10^-6, 1×10−61\!×\!10^-6, and 5×10−75\!×\!10^-7 at 0.6B, 1.7B, and 4B, respectively (Supplementary Sections 1.1 and 2.5). Full Objective and Reductions The training loss combines the verifiable reward with gated skill distillation: ℒ(θ)=λRℒR(θ)+λDx[gK(x)y^∼pS1|y^|∑nℒn(τ)],L(θ)= _R\,L_R(θ)+ _D\,E_x\! [g_K(x)\,E_ y p_S 1| y|\! _nL^(τ)_n ], (4) where ℒRL_R is group-normalized REINFORCE (GRPO) (Shao et al. 2024) on student rollouts and gK(x) g_K(x) =[Δ^K(0)(x)>0], =1\! [ ^(0)_K(x)>0 ], (5) Δ^K(0)(x) ^(0)_K(x) =1K∑i=1K[R(yiT)−R(yiS)]. = 1K _i=1^K [R(y_i^T)-R(y_i^S) ]. with yiT∼pT(0)(⋅∣x,s)y_i^T p_T^(0)(· x,s) and yiS∼pS(0)(⋅∣x)y_i^S p_S^(0)(· x). The superscript (0)(0) denotes the initial shared checkpoint; the resulting gate remains fixed during training. It selects problems for which the sampled initial difference is positive, but finite-K error and later teacher drift can create false positives, false negatives, and sign reversals. On a rollout group with identical rewards, the GRPO term has zero group-relative gradient, whereas distillation can remain nonzero whenever gK(x)=1g_K(x)=1 and rn(τ)≠pnr_n^(τ)≠ p_n. The group-stratified experiment in §4.2 directly tests whether this algebraic complementarity explains the observed gain. We use the forward direction to retain teacher support, but at τ>0τ>0 the effective target is sharpened and is not ordinary forward KL. Setting s=y∗s=y^*, τ=0τ=0, g≡1g≡ 1, and λR=0 _R=0 yields the OPSD student gradient because H(q,p)H(q,p) and KL(q∥p)KL(q\|p) differ only by the stop-gradient teacher entropy. Dropping the distillation term yields GRPO. The ordered ablation in §4 studies skill abstraction, tilting, and the gate; its marginal increments can depend on the order and on component interactions. Design Desiderata and Scope Skill distillation is most plausible under three measurable desiderata. First, reliability: skill conditioning should improve expected rollout accuracy, Δ∗(x)=yT∼pT[R(yT)]−yS∼pS[R(yS)]>0. ^*(x)=E_y^T p_T[R(y^T)]-E_y^S p_S[R(y^S)]>0. The fixed gate estimates this quantity only at initialization and with finite samples. Second, nontriviality: the skill should change the next-token distribution on reasoning-relevant positions rather than only boilerplate. Third, manageable mismatch: the privileged context should avoid an extreme teacher–student support gap. These are diagnostic desiderata, not sufficient conditions for policy improvement, and the shared-parameter teacher may cease to satisfy them as training evolves. Figure 5 reports token-level and mismatch diagnostics; Supplementary Sections 2.4 and 2.7 report gate refresh and group-stratified controls. Separate frozen/EMA-teacher trajectories remain an open diagnostic for the shared moving-target dynamics. The three tested Qwen3-Base sizes show a monotonic increase in SKALD’s advantage over GRPO. This observation is consistent with the hypothesis that more capable models can exploit the same skill card more effectively, but three points from one model family do not establish a scaling law or a model-family-independent property. MATH500 AMC23 AIME24 AIME25 Minerva Overall Method avg@8 / pass@8 Qwen3-4B-base (Yang et al. 2025a) Zero-shot 69.70 / 88.40 48.12 / 82.50 9.58 / 20.00 6.67 / 23.33 25.55 / 47.06 50.70 / 70.64 Few-shot (k=2k=2) 56.97 / 88.40 37.50 / 77.50 10.00 / 20.00 7.92 / 20.00 25.05 / 48.90 42.82 / 70.87 SFT (GT-CoT) 64.30 / 86.20 38.44 / 77.50 5.00 / 13.33 4.17 / 20.00 31.20 / 48.90 48.68 / 69.38 GRPO 61.85 / 83.00 57.81 / 80.00 14.58 / 33.33 15.00 / 26.67 39.25 / 54.41 51.37 / 70.30 SKALD (ours) 82.33 / 94.00 52.81 / 85.00 20.00 / 46.67 18.33 / 36.67 39.84 / 55.51 63.38 / 77.98 Qwen3-1.7B-base (Yang et al. 2025a) Zero-shot 54.40 / 83.60 32.20 / 70.00 2.92 / 13.33 3.33 / 6.67 21.00 / 43.00 39.43 / 65.25 Few-shot (k=2k=2) 46.30 / 73.70 25.10 / 65.00 2.50 / 3.33 0.00 / 0.00 11.80 / 26.80 31.47 / 53.72 SFT (GT-CoT) 51.30 / 76.20 23.40 / 62.50 2.50 / 6.67 0.00 / 0.00 22.80 / 43.10 37.69 / 60.23 GRPO 62.50 / 83.20 35.90 / 67.50 4.58 / 16.67 4.17 / 16.67 24.80 / 43.00 45.52 / 65.36 GRPO + skill in-ctx (withdrawn) 63.40 / 83.40 38.80 / 70.00 2.92 / 13.33 6.67 / 23.33 26.10 / 43.40 46.60 / 65.83 OPSD (Zhao et al. 2026b) (full solution) 66.80 / 85.10 35.90 / 50.00 5.42 / 16.67 5.00 / 16.67 26.30 / 41.20 48.51 / 65.09 EOPD (Jin et al. 2026) (entropy routing) 64.10 / 84.80 39.20 / 67.50 7.08 / 20.00 4.17 / 20.00 26.90 / 43.10 47.33 / 66.54 SKALD (ours) 68.80 / 86.40 42.20 / 75.00 9.58 / 26.67 5.00 / 20.00 27.20 / 44.90 50.37 / 68.59 Qwen3-0.6B-base (Yang et al. 2025a) Zero-shot 35.20 / 68.60 15.31 / 50.00 1.67 / 3.33 0.83 / 3.33 7.08 / 23.53 23.19 / 49.20 Few-shot (k=2k=2) 16.00 / 48.60 4.69 / 25.00 0.00 / 0.00 0.00 / 0.00 5.42 / 18.38 11.08 / 34.75 SFT (GT-CoT) 36.73 / 66.60 16.88 / 47.50 1.67 / 3.33 0.42 / 3.33 11.67 / 29.78 25.55 / 49.89 GRPO 44.32 / 69.60 19.06 / 57.50 1.67 / 3.33 0.83 / 3.33 8.04 / 23.90 28.88 / 50.23 SKALD (ours) 46.75 / 71.20 19.38 / 70.00 1.67 / 3.33 2.50 / 10.00 11.21 / 26.10 31.34 / 52.64 Table 1: Per-benchmark avg@8 / pass@8 across scales. Overall is the problem-count-weighted micro-average over 872 problems, computed from unrounded per-problem results. Reported numbers are means over 3 independent training seeds; per-seed values and cross-seed deviations are in Supplementary Sections 2.1 and 2.3. Hierarchical intervals jointly resampling seeds and problems are reported in Supplementary Section 2.1. Experiments Setup We evaluate MATH500 (Lightman et al. 2024) (500 problems), AMC23 (Zhang and Math-AI 2023) (40), AIME24 (Zhang and Math-AI 2024) (30), AIME25 (Zhang and Math-AI 2025) (30), and Minerva (Lewkowycz et al. 2022) (272), totaling 872872 problems. Each checkpoint uses eight samples at T=0.7T=0.7, top-p=0.95p=0.95; avg@8 is primary and overall scores are problem-weighted micro-averages. Training uses DAPO-Math-17k and the mathematics split of OpenThoughts. Exact, formula-normalized, and semantic checks flag unions of 33 problem texts, 1414 reference solutions, and 1010 skill cards against the evaluation suite; every reported model is retrained after removing the union, leaving 23,69123,691 problems (Supplementary Section 1.4). The conditional leakage audit uses a common judge and decoding protocol: question-only, empty-card, random-text, shuffled-skill, matched-skill, and full-solution recovery rates are 2.42.4, 2.52.5, 2.62.6, 3.73.7, 5.15.1, and 94.2%94.2\%, respectively. A three-rater audit finds no full-answer leak in 200200 retained cards (κ=0.83κ=0.83), and the sanitized-card score is 50.2450.24 versus 50.3750.37 for the original. Median/95th-percentile normalized solution–card overlap is 0.03/0.110.03/0.11 (Supplementary Section 1.3). At 1.7B, SKALD-shuffled reaches 48.8348.83: +3.31+3.31 over GRPO, +0.32+0.32 over OPSD, and −1.54-1.54 below matched SKALD. Thus matching matters, but generic card exposure or regularization can explain part of the gain. We report three training seeds and use a two-level bootstrap that first resamples seeds and then evaluation problems within seed; decoding samples are aggregated at the problem level. Compute accounting includes scoring and gate construction, and equal-FLOP controls are reported in Supplementary Sections 2.1 and 2.6. Full implementation, baseline, and per-benchmark statistics are in Supplementary Sections 1, 2.2, and 2.3. Main Results and Mechanism Test SKALD attains the best overall avg@8 and pass@8 at all three tested scales, with mean gains over GRPO of +2.46+2.46, +4.85+4.85, and +12.01+12.01 (Table 1, Figure 3). At 1.7B, the per-seed differences are +4.70+4.70, +4.91+4.91, and +4.94+4.94; the hierarchical 95% interval is [+3.92,+5.78][+3.92,+5.78]. Rollout-matched GRPO uses 88.088.0 GPU-hours and reaches 45.5245.52; an equal-FLOP/equal-GPU-hour run uses 131.2131.2 GPU-hours and reaches 46.3146.31, versus SKALD’s 50.3750.37. At 4B, tuned equal-FLOP GRPO reaches 52.19±0.1852.19± 0.18, while SKALD reaches 63.38±0.3263.38± 0.32 (Supplementary Sections 2.1, 2.6, and 2.8). The central mechanism is directly visible in the rollout groups. Zero-variance groups account for 68.0%68.0\%, 63.0%63.0\%, and 63.9%63.9\% at 0.6B, 1.7B, and 4B; with scale, their composition shifts from all-wrong (54.2%→21.8%54.2\%→ 21.8\%) toward all-correct (13.8%→42.1%13.8\%→ 42.1\%). At 1.7B, zero-variance-only distillation reaches 49.6349.63 (+4.11+4.11 over GRPO), recovering 84.7%84.7\% of the full gain, while mixed-only distillation reaches 46.3146.31 (+0.79+0.79). This supports the claim that SKALD primarily adds signal where group-relative advantages vanish (Supplementary Section 2.7). Against privileged baselines at 1.7B, SKALD is +1.86+1.86 above OPSD and +3.04+3.04 above EOPD. The full 4B ablation is also monotonic under one addition order: OPSD 55.6755.67, skill abstraction 57.3257.32, tilted objective 61.0561.05, and the gate 63.3863.38. These are ordered marginal effects rather than interaction-free contributions. Robustness to Alternative Explanations Leakage and memorization. The conditional audit shows a modest +2.7+2.7-point recovery increase from question-only to question+matched-skill, far below the +91.8+91.8 increase from a full solution. More importantly, deleting instance-specific numerals, entities, and intermediate expressions preserves the downstream score (50.2450.24 versus 50.3750.37), and replacing each card by a peer card from the same abstract-method cluster changes overall avg@8 by only 0.11±0.190.11± 0.19. The median skill is reused by five problems, while problem-, solution-, and skill-level decontamination removes all flagged cross-split overlaps before retraining. These controls do not prove zero information leakage, but jointly make direct answer transfer an implausible explanation for a +4.85+4.85-point gain. Compute and baseline quality. The equal-compute comparison gives GRPO the entire measured SKALD budget, including the additional scoring and gate costs. Its gain from the extra 181181 updates is +0.79+0.79, leaving SKALD +4.06+4.06 ahead at identical FLOPs and GPU-hours. At 4B, a scale-specific GRPO sweep raises the equal-FLOP baseline to 52.1952.19, but the remaining SKALD margin is still +11.19+11.19. Full-solution OPSD also consumes slightly more measured compute than SKALD at 1.7B and reaches 48.5148.51, so the result is not explained by privileged scoring alone. Run uncertainty and gate choice. All three paired 1.7B seeds yield gains between +4.70+4.70 and +4.94+4.94; the seed×problem interval excludes zero without treating the 872872 problems as independent training runs. Varying K from 44 to 3232, using a soft gate, or refreshing it periodically/with an EMA changes avg@8 by at most 0.390.39. Thus finite-K noise and teacher drift are measurable—notably, 14.6%14.6\% of default positive gates reverse sign—but do not account for the headline improvement in the tested configurations. Configuration Skill Tilt avg@8 OPSD (s=y∗s=y^*, τ=0τ=0) ✗ ✗ 48.51 ± 0.28 + skill abstraction (s: card) ✓ ✗ 49.06 ± 0.30 + tilted risk (τ:0.8→0τ:0.8→0) ✓ ✓ 49.94 ± 0.33 + usefulness gate (= SKALD) ✓ ✓ 50.37 ± 0.34 Table 2: Ordered component ablation at 1.7B (three seeds, eight decodings, T=0.7T=0.7, top-p=0.95p=0.95). Mean ± cross-seed standard deviation. Sequential changes are +0.55+0.55, +0.88+0.88, and +0.43+0.43; they need not be additive under another order. Seed-level statistics are in Supplementary Section 2.1. Figure 3: Observed SKALD–GRPO difference in overall avg@8 at the three tested Qwen3-Base sizes. The monotonic pattern is descriptive and is not presented as a scaling law. Figure 4: Training dynamics in the reported risk-neutral and annealed-tilt runs. (a) Gradient norm and (b) distillation cross-entropy at 1.7B. (c) At 4B, the risk-neutral run reaches the 7168-token cap, whereas the tilted run remains shorter. These trajectories are consistent with a stabilization effect but do not replace replicated causal controls. Figure 5: Diagnostics for nontriviality and teacher–student mismatch. (a) One illustrative rollout, which is not used as population-level evidence. (b) Aggregate KL by token category under the stated annotation protocol. (c) Initial per-token KL CDF in the analyzed sample: p99 is 13.013.0 nats with full-solution conditioning and 0.60.6 with skill conditioning. Dissecting the Gain Ordered component ablation. Table 2 reports marginal changes under one addition order: skill abstraction +0.55+0.55, annealed tilting +0.88+0.88, and the gate +0.43+0.43. The same ordering remains positive at 4B (+1.65+1.65, +3.73+3.73, and +2.33+2.33). Conditional audits and sanitized cards address the main leakage alternative, while the single addition order still permits component interactions. Internalization beats context. Should a skill’s value live in the inference-time context or in the weights? We compare three 1.7B conditions under an identical no-skill-at-test protocol (Table 1): GRPO (45.5245.52), GRPO with the skill in the RL context and withdrawn at test (46.6046.60), and SKALD (50.3750.37). This comparison contrasts contextual skill exposure with teacher-only skill distillation under the same no-skill-at-test evaluation, with SKALD improving by +3.77+3.77. The context condition remains above GRPO. The supported conclusion is therefore limited to this protocol: teacher-only skill distillation outperforms student-context exposure with later withdrawal. Claims about complete assimilation would require direct parameter- or behavior-level probes. The skill must induce a useful teacher shift. Removing the skill from the teacher context while keeping the same Qwen3-Base checkpoint and shared-parameter training setup reduces the 1.7B result from 50.37 to 48.01. Because the teacher and student use identical parameters, this comparison isolates the effect of privileged skill conditioning rather than teacher capacity or model initialization. With the skill, the context-induced distribution shift provides a nontrivial distillation signal; without it, the teacher and student distributions become substantially closer. Stability Diagnostics Figure 4 compares the risk-neutral and tilted runs; full schedule, gradient, length, and seed-wise statistics are in Supplementary Section 2.5. At 1.7B, the risk-neutral run reaches gradient norms near ∼4.5 4.5 with heavy-tailed spikes and increasing distillation cross-entropy, whereas the tilted run remains near 0.30.3 and its monitored cross-entropy falls from 1.211.21 to 0.230.23. This pattern is consistent with Lemma 1: rτr_τ initially removes mass from low-p teacher tokens and gradually restores it as τ decays. At 4B, the risk-neutral run grows from 1.21.2k to 6.86.8k tokens, hits the 71687168 cap, and scores 48.1848.18; the tilted run contracts to roughly 2.42.4k and scores 63.3863.38. The 1,0001,000-update statistics come from separate long-horizon 1.7B diagnostic runs; all main experiments use 370 updates. Their gradient-norm p95/max fall from 6.42/27.46.42/27.4 to 0.61/1.420.61/1.42. Risk-neutral training diverges in 2/32/3 1.7B and 3/33/3 4B seeds, whereas all tilted runs converge. Cosine and exponential schedules differ from linear by at most 0.080.08, and scores vary by at most 0.150.15 over τ0∈[0.6,1.0] _0∈[0.6,1.0]; constant τ=0.8τ=0.8 and risk-neutral τ=0τ=0 lose 0.620.62 and 5.475.47 points. These diagnostics support stabilization; guarantees apply only to logit gradients, while parameter-gradient bounds require a bounded Jacobian. Diagnostic Evidence Figure 5 examines where the two context views differ and how large the initial mismatch is. Token location. Panel (a) is an illustrative rollout: KL is small on template tokens and peaks near the AM–GM decision, equality condition, and a skill-flagged mistake. It should not be used alone to claim a general concentration pattern. In the aggregate token set summarized by panel (b), the stated reasoning category covers 27%27\% of positions and carries substantially more KL mass than the template category. A reproducible claim requires the number of problems, tokens, seeds, category definition, and annotation agreement. Context mismatch. In the analyzed initial-token sample, skill conditioning has p99 KL of 0.60.6 nats, while full-solution conditioning has p99 KL of 13.013.0 nats. This supports the narrower claim that the sampled full-solution context induces a heavier mismatch tail. It does not establish that every such token is unreachable, that the tilted objective transfers all skill tokens, or that this diagnostic alone causes the downstream score difference. Conclusion SKALD transfers a reusable skill abstraction from a privileged context view into a question-only policy, without a larger online teacher or privileged input at inference. Its annealed Rényi-type cross-entropy makes the optimization trade-off explicit: positive tilt discounts low-student-mass teacher tokens and targets a sharpened escort distribution; annealing to zero restores teacher cross-entropy and the forward-KL student gradient. The new controls connect this objective to the motivating failure mode. Zero-variance groups form 6363–68%68\% of training groups, and restricting distillation to them recovers 84.7%84.7\% of the full 1.7B gain. SKALD remains +4.06+4.06 above equal-FLOP GRPO at 1.7B and +11.19+11.19 above tuned equal-FLOP GRPO at 4B; the latter scale also preserves positive skill, tilt, and gate increments. Question-conditioned recovery, card sanitization, human audit, and problem/solution/skill decontamination make direct answer leakage an unlikely explanation, while hierarchical intervals preserve the gain across seeds and evaluation problems. Across the three tested Qwen3-Base sizes, these results support skill-conditioned self-distillation as a compute-aware complement to RLVR precisely where group-relative rewards are silent. References R. Agarwal, N. Vieillard, Y. Zhou, P. Stanczyk, S. Ramos Garea, M. Geist, and O. Bachem (2024) On-policy distillation of language models: learning from self-generated mistakes. In International Conference on Learning Representations, Vol. 2024, p. 21246–21263. Cited by: On-policy distillation.. M. Chen, L. Sun, T. Li, H. Sun, C. Zhu, H. Wang, J. Pan, W. Zhang, H. Chen, F. Yang, et al. (2026) Learning to reason with search for llms via reinforcement learning. Advances in Neural Information Processing Systems 38, p. 85287–85307. Cited by: Introduction. J. H. Cho and B. Hariharan (2019) On the efficacy of knowledge distillation. In Proceedings of the IEEE/CVF international conference on computer vision, p. 4794–4802. Cited by: On-policy distillation.. Y. Fu, H. Huang, K. Jiang, J. Liu, Z. Jiang, Y. Zhu, and D. Zhao (2026) Revisiting on-policy distillation: empirical failure modes and simple fixes. arXiv preprint arXiv:2603.25562. Cited by: Introduction. N. C. Garcia, P. Morerio, and V. Murino (2019) Learning with privileged information via adversarial discriminative modality distillation. IEEE transactions on pattern analysis and machine intelligence 42 (10), p. 2581–2593. Cited by: Privileged self-teachers.. J. Gou, B. Yu, S. J. Maybank, and D. Tao (2021) Knowledge distillation: a survey. International journal of computer vision 129 (6), p. 1789–1819. Cited by: On-policy distillation.. Y. He, S. Kaur, A. Bhaskar, Y. Yang, J. Liu, N. Ri, L. Fowl, A. Panigrahi, D. Chen, and S. Arora (2026) Self-distillation zero: self-revision turns binary rewards into dense supervision. arXiv preprint arXiv:2604.12002. Cited by: Introduction. Y. Jiang, D. Li, H. Deng, B. Ma, X. Wang, Q. Wang, and G. Yu (2026) SoK: agentic skills–beyond tool use in llm agents. arXiv preprint arXiv:2602.20867. Cited by: Setup: One Model, Two Contexts. W. Jin, T. Min, Y. Yang, D. Wei, Y. Zhou, S. R. Kadhe, N. Baracaldo, and K. Lee (2026) Entropy-aware on-policy distillation of language models. External Links: 2603.07079, Link Cited by: Table 1. Y. Kim and A. M. Rush (2016) Sequence-level knowledge distillation. In Proceedings of the 2016 conference on empirical methods in natural language processing, p. 1317–1327. Cited by: On-policy distillation.. M. Laskin, L. Wang, J. Oh, E. Parisotto, S. Spencer, R. Steigerwald, D. Strouse, S. Hansen, A. Filos, E. Brooks, M. Gazeau, H. Sahni, S. Singh, and V. Mnih (2022) In-context reinforcement learning with algorithm distillation. External Links: 2210.14215, Link Cited by: Skills: context versus weights.. A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, Y. Wu, B. Neyshabur, G. Gur-Ari, and V. Misra (2022) Solving quantitative reasoning problems with language models. External Links: 2206.14858, Link Cited by: Introduction, Setup. Y. Li, Y. Zuo, B. He, J. Zhang, C. Xiao, C. Qian, T. Yu, H. Gao, W. Yang, Z. Liu, et al. (2026) Rethinking on-policy distillation of large language models: phenomenology, mechanism, and recipe. arXiv preprint arXiv:2604.13016. Cited by: Introduction. H. Liao, S. He, Y. Hao, X. Li, Y. Zhang, J. Zhao, and K. Liu (2025) Skintern: internalizing symbolic knowledge for distilling better cot capabilities into small language models. In Proceedings of the 31st International Conference on Computational Linguistics, p. 3203–3221. Cited by: Setup: One Model, Two Contexts. H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe (2024) Let’s verify step by step. In International Conference on Learning Representations, Vol. 2024, p. 39578–39601. Cited by: Introduction, Setup. C. Liu, H. Miao, Q. Xu, S. Zhou, C. Long, Y. Zhao, Z. Li, and R. Zhao (2025) Efficient multivariate time series forecasting via calibrated language models with privileged knowledge distillation. In 2025 IEEE 41st International Conference on Data Engineering (ICDE), p. 3165–3178. Cited by: Privileged self-teachers.. D. Lopez-Paz, L. Bottou, B. Schölkopf, and V. Vapnik (2015) Unifying distillation and privileged information. arXiv preprint arXiv:1511.03643. Cited by: Privileged self-teachers.. Z. Lu, Z. Yao, J. Wu, C. Han, Q. Gu, X. Cai, W. Lu, J. Xiao, Y. Zhuang, and Y. Shen (2026) SKILL0: in-context agentic reinforcement learning for skill internalization. External Links: 2604.02268, Link Cited by: Skills: context versus weights.. A. M. Mansourian, R. Ahmadi, M. Ghafouri, A. M. Babaei, E. B. Golezani, Z. Y. Ghamchi, V. Ramezanian, A. Taherian, K. Dinashi, A. Miri, et al. (2025) A comprehensive survey on knowledge distillation. arXiv preprint arXiv:2503.12067. Cited by: On-policy distillation.. T. Nam, S. Sun, K. Pertsch, S. J. Hwang, and J. J. Lim (2022) Skill-based meta-reinforcement learning. arXiv preprint arXiv:2204.11828. Cited by: Introduction. E. Penaloza, D. Vattikonda, N. Gontier, A. Lacoste, L. Charlin, and M. Caccia (2026) Privileged information distillation for language models. arXiv preprint arXiv:2602.04942. Cited by: Privileged self-teachers.. Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. (2024) Deepseekmath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: Introduction, Full Objective and Reductions. L. X. Shi, J. J. Lim, and Y. Lee (2022) Skill-based model-based reinforcement learning. arXiv preprint arXiv:2207.07560. Cited by: Introduction. C. Snell, D. Klein, and R. Zhong (2022) Learning by distilling context. External Links: 2209.15189, Link Cited by: Skills: context versus weights.. S. Sun, W. Ren, J. Li, R. Wang, and X. Cao (2024) Logit standardization in knowledge distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 15731–15740. Cited by: Introduction. F. C. Thierrin, F. Alajaji, and T. Linder (2022) On the rényi cross-entropy. In 2022 17th Canadian Workshop on Information Theory (CWIT), p. 1–5. Cited by: Generalized cross-entropies and mismatch.. Y. Tian, S. Pei, X. Zhang, C. Zhang, and N. V. Chawla (2025) Knowledge distillation on graphs: a survey. ACM Computing Surveys 57 (8), p. 1–16. Cited by: On-policy distillation.. F. J. Valverde-Albacete and C. Peláez-Moreno (2019) The case for shifting the rényi entropy. Entropy 21, p. 46. Cited by: Generalized cross-entropies and mismatch.. T. Van Erven and P. Harremos (2014) Rényi divergence and kullback-leibler divergence. IEEE Transactions on Information Theory 60 (7), p. 3797–3820. Cited by: Generalized cross-entropies and mismatch.. V. Vapnik and A. Vashist (2009) A new learning paradigm: learning using privileged information. Neural networks 22 (5-6), p. 544–557. Cited by: Privileged self-teachers.. H. Wang, G. Wang, H. Xiao, Y. Zhou, Y. Pan, J. Wang, K. Xu, Y. Wen, X. Ruan, X. Chen, et al. (2026) Skill-sd: skill-conditioned self-distillation for multi-turn llm agents. arXiv preprint arXiv:2604.10674. Cited by: Introduction. F. Wu, W. Xuan, X. Lu, M. Liu, Y. Dong, Z. Harchaoui, and Y. Choi (2025) The invisible leash: why rlvr may or may not escape its origin. arXiv preprint arXiv:2507.14843. Cited by: Introduction, Introduction. P. Xia, J. Chen, H. Wang, J. Liu, K. Zeng, Y. Wang, S. Han, Y. Zhou, X. Zhao, H. Chen, Z. Zheng, C. Xie, and H. Yao (2026) SkillRL: evolving agents via recursive skill-augmented reinforcement learning. External Links: 2602.08234, Link Cited by: Skills: context versus weights.. R. Xu and Y. Yan (2026) Agent skills for large language models: architecture, acquisition, security, and the path forward. arXiv preprint arXiv:2602.12430. Cited by: Introduction. X. Xu, M. Li, C. Tao, T. Shen, R. Cheng, J. Li, C. Xu, D. Tao, and T. Zhou (2024) A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116. Cited by: Introduction. A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu (2025a) Qwen3 technical report. External Links: 2505.09388, Link Cited by: Table 1, Table 1, Table 1. C. Yang, Y. Zhu, W. Lu, Y. Wang, Q. Chen, C. Gao, B. Yan, and Y. Chen (2025b) Survey on knowledge distillation for large language models: methods, evaluation, and application. ACM Transactions on Intelligent Systems and Technology 16 (6), p. 1–27. Cited by: On-policy distillation.. W. Yang, W. Liu, R. Xie, K. Yang, S. Yang, and Y. Lin (2026) Learning beyond teacher: generalized on-policy distillation with reward extrapolation. arXiv preprint arXiv:2602.12125. Cited by: Introduction. H. Ye, S. Lu, and D. Zhan (2022) Generalized knowledge distillation via relationship matching. IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (2), p. 1817–1834. Cited by: On-policy distillation.. T. Ye, L. Dong, X. Wu, S. Huang, and F. Wei (2026a) On-policy context distillation for language models. External Links: 2602.12275, Link Cited by: Skills: context versus weights.. T. Ye, L. Dong, X. Wu, S. Huang, and F. Wei (2026b) On-policy context distillation for language models. arXiv preprint arXiv:2602.12275. Cited by: Introduction, On-policy distillation., Setup: One Model, Two Contexts. Q. Yu, Z. Zhang, R. Zhu, Y. Yuan, X. Zuo, Y. Yue, W. Dai, T. Fan, G. Liu, L. Liu, et al. (2026) Dapo: an open-source llm reinforcement learning system at scale. Advances in Neural Information Processing Systems 38, p. 113222–113244. Cited by: Introduction. X. Zhang, Z. Ding, T. Pan, R. Yang, C. Kang, X. Xiong, and J. Gu (2026) Opsdl: on-policy self-distillation for long-context language models. arXiv preprint arXiv:2604.17535. Cited by: Setup: One Model, Two Contexts. Y. Zhang and T. Math-AI (2023) AMC23 dataset. Note: https://huggingface.co/datasets/math-ai/amc23 Cited by: Introduction, Setup. Y. Zhang and T. Math-AI (2024) American invitational mathematics examination (aime) 2024. Cited by: Introduction, Setup. Y. Zhang and T. Math-AI (2025) American invitational mathematics examination (aime) 2025. Cited by: Introduction, Setup. S. Zhao, Z. Xie, M. Liu, J. Huang, G. Pang, F. Chen, and A. Grover (2026a) Self-distilled reasoner: on-policy self-distillation for large language models. arXiv preprint arXiv:2601.18734. Cited by: Privileged self-teachers.. S. Zhao, Z. Xie, M. Liu, J. Huang, G. Pang, F. Chen, and A. Grover (2026b) Self-distilled reasoner: on-policy self-distillation for large language models. External Links: 2601.18734, Link Cited by: Introduction, On-policy distillation., Privileged self-teachers., Table 1.