Paper deep dive
Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning
Hongyang Li, Yiming Zhu, Xiao Li, Caesar Wu, Said Mammar, Pascal Bouvry
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Post-training is central to mathematical reasoning in modern large language models (LLMs), but endpoint pass@1 alone underidentifies what has changed. Gains may reflect newly reachable solutions, cheaper sampling of latent solutions, surface robustness, or memorisation. We compare three post-training paths under a common diagnostic readout: our sufficiently trained off-policy distillation trajectories, released Qwen3 off-policy-plus-on-policy distillation endpoints, and a released DeepSeek-Math endpoint trained with Group Relative Policy Optimisation (GRPO). Our probe uses cross-surface pass@K over verbatim prompts, paraphrases, numerical isomorphisms, and translations, plus consistency, distribution-shape, and verified supervised-fine-tuning (SFT) membership analyses. We find two regimes. On easier AMC problems, large-K ceilings are near saturation, so post-training mainly compresses sample cost. On harder AIME problems, post-training expands the large-K ceiling over the base model: sufficient off-policy distillation already raises this ceiling, Qwen3 released endpoints raise it further, and DeepSeek-Math GRPO does not dominate sufficient off-policy distillation at large K. English-dominant distillation improves non-English reasoning but preserves language-tier gaps. A controlled-overfit audit finds limited sensitivity in current SFT-membership probes. Compression is one regime of post-training, not a universal explanation.
Tags
Links
- Source: https://arxiv.org/abs/2609.37066v1
- Canonical: https://arxiv.org/abs/2609.37066v1
Trouble viewing inline? Open PDF directly →
Full Text
57,920 characters extracted from source content.
Expand or collapse full text
Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning Hongyang Li Yiming Zhu Xiao Li Caesar Wu Said Mammar Pascal Bouvry Abstract Post-training is central to mathematical reasoning in modern large language models (LLMs), but endpoint pass@11 alone underidentifies what has changed. Gains may reflect newly reachable solutions, cheaper sampling of latent solutions, surface robustness, or memorisation. We compare three post-training paths under a common diagnostic readout: our sufficiently trained off-policy distillation trajectories, released Qwen3 off-policy-plus-on-policy distillation endpoints, and a released DeepSeek-Math endpoint trained with Group Relative Policy Optimisation (GRPO). Our probe uses cross-surface pass@K over verbatim prompts, paraphrases, numerical isomorphisms, and translations, plus consistency, distribution-shape, and verified supervised-fine-tuning (SFT) membership analyses. We find two regimes. On easier AMC problems, large-K ceilings are near saturation, so post-training mainly compresses sample cost. On harder AIME problems, post-training expands the large-K ceiling over the base model: sufficient off-policy distillation already raises this ceiling, Qwen3 released endpoints raise it further, and DeepSeek-Math GRPO does not dominate sufficient off-policy distillation at large K. English-dominant distillation improves non-English reasoning but preserves language-tier gaps. A controlled-overfit audit finds limited sensitivity in current SFT-membership probes. Compression is one regime of post-training, not a universal explanation. †footnotetext: 1University of Luxembourg 2Seafill Open-Source Community 3Université Paris-Saclay hongyang.li, caesar.wu, pascal.bouvry@uni.lu xiao.li@seafill.com • z13655249157@gmail.com said.mammar@univ-evry.fr 1 Introduction Open-weight reasoning models now achieve strong scores on mathematical benchmarks, but endpoint pass@11 alone does not identify what post-training has changed. The same gain may reflect newly reachable solutions, cheaper sampling of latent solutions, robustness to surface variation, or memorisation of benchmark content. These explanations imply different conclusions for evaluation and deployment, yet they can produce the same endpoint accuracy. This ambiguity is central to current debates on mathematical post-training. Prior work argues that reinforcement learning (RL) can mainly compress sample cost, raising pass@11 without expanding the large-K ceiling (Yue et al. 2025); other work shows that RL can acquire new compositional behaviours in controlled settings (Yuan et al. 2025). Meanwhile, SFT and distillation have become major practical routes to strong reasoning models (Muennighoff et al. 2025; Ye et al. 2025; Guha et al. 2025), building on broader knowledge-distillation and reasoning-distillation recipes (Hinton, Vinyals, and Dean 2015; Magister et al. 2023); related work further shows that distinct training stages—continual pre-training, problem-solving SFT, and RL—can shape mathematical reasoning differently (Chen et al. 2025). However, static teacher-trace imitation, student-rollout-based teacher matching, and verifier-driven RL need not shape the output distribution in the same way. Surface-robustness studies further show that high reasoning scores can be sensitive to linguistic, symbolic, or functional perturbations (Mirzadeh et al. 2024; Srivastava et al. 2024). Separately, memorisation and membership-inference work shows that language models can retain or reveal training data, but membership is often hard to verify for released models (Carlini et al. 2021; Carlini et al. 2023). Existing evaluations therefore often isolate one axis at a time—mechanism, surface form, endpoint, or membership assumption—making these explanations difficult to separate under a common protocol. Figure 1: Overview. We compare three post-training paths—our off-policy distillation trajectories (M1), released Qwen3 off-policy-plus-on-policy distillation endpoints (M2), and the released DeepSeek-Math GRPO endpoint (M3)—on shared base model families. We evaluate them with cross-surface pass@K over verbatim prompts, paraphrases, numerical isomorphisms, and translations, together with strategy-diversity, multi-instance consistency, and verified memorisation probes. We address this with a trajectory-aware diagnostic. The key idea is to evaluate performance as both the sampling budget and problem surface vary. If a gain appears at small K but vanishes at large K, it indicates sample-cost compression. If it persists at large K, it indicates ceiling expansion: the post-trained model reaches correct solutions that the base model did not reach under the evaluated budget. If the gain fails under paraphrase, numerical isomorphism, or translation, it is surface-fragile. If verified training items separate from verified unseen controls, memorisation becomes a plausible explanation. We compare three post-training paths under this common readout (Fig. 1). First, we train reproducible off-policy distillation trajectories for Qwen3-4B, Qwen3-8B, and DeepSeek-Math-7B. We treat these sufficiently trained off-policy endpoints as first-class post-training models rather than weak SFT baselines. Second, we evaluate released Qwen3-4B and Qwen3-8B endpoints, which use off-policy followed by on-policy distillation for these sizes (Yang et al. 2025). Third, we evaluate the released DeepSeek-Math-7B-RL endpoint as a GRPO comparison (Shao et al. 2024). Since some endpoints are released rather than retrained by us, our goal is mechanism-relative diagnosis under a shared protocol, not a perfectly matched causal ablation of every algorithm. Our main probe is cross-surface pass@K. Starting from a shared 100100-problem pool, we pool attempts at the source-problem level, so large-K performance measures whether the model can solve the underlying problem across surface changes, not merely resample one familiar prompt. We complement this with multi-instance consistency, distribution-shape descriptors separating form sharpening from answer-content sharpening, cross-lingual analysis, and verified SFT-membership probes enabled by our controlled training data. Our results show two regimes. On easier AMC problems, large-K ceilings are near saturation, so post-training mainly behaves as sample-cost compression: correct solutions become cheaper to sample, while mechanisms converge near the ceiling. On harder AIME problems, however, post-training expands the ceiling over the base model. Sufficient off-policy distillation already raises the cross-surface large-K ceiling; released Qwen3 off-policy-plus-on-policy endpoints raise it further; and the released DeepSeek-Math GRPO endpoint does not exceed our sufficiently trained off-policy endpoint at large K. Thus, compression explains one regime of post-training, but not all mechanisms or difficulties. Robustness and distributional diagnostics refine this picture. Qwen3 endpoints strongly consolidate solved problems across paraphrases and numerical isomorphisms, while the DeepSeek-Math GRPO endpoint remains close to its SFT checkpoints in multi-instance consistency. Distribution-shape metrics show that form sharpening and answer-content sharpening can diverge. Cross-lingually, English-dominant distillation improves every non-English language we evaluate, but preserves the stronger-versus-weaker language gap. Finally, a controlled-overfit memorisation audit finds limited sensitivity in current behavioural and forward-pass membership probes for SFT-specific membership. Our contributions are: 1. Mechanism-relative post-training comparison. We place sufficiently trained off-policy distillation, released on-policy distillation, and released GRPO endpoints under a common diagnostic readout, while distinguishing controlled trajectories from observed endpoints. 2. Cross-surface pass@K. We introduce a trajectory-aware probe that separates sample-cost compression, large-K ceiling expansion, and surface robustness. 3. Regime-dependent view of compression. We show that compression explains saturated easy problems, while hard problems reveal ceiling expansion from post-training, already under sufficient off-policy distillation. 4. Distributional and robustness diagnostics. We connect capability changes to multi-instance consistency, form/content sharpening, multilingual surface shifts, and verified SFT-membership controls. 2 Method The paper is organised around three post-training mechanisms observed in modern open-weight LLMs for mathematical reasoning (§2.1). We train a sequence of checkpoints under one of them—off-policy distillation—across three model families (§2.2). All checkpoints are evaluated with a capability probe set (§2.3). Sampling, scoring, and strategy-diversity protocols are summarised in §2.4. 2.1 Three post-training mechanisms Recent open-weight math reasoning models build on pretrained bases through different post-training mechanisms. We group them into three types, which differ in the training data, objective, and whether updates depend on the student’s own rollouts. (M1) Off-policy distillation. The student is trained on a static dataset of (input, teacher output) pairs with cross-entropy loss on teacher tokens; the student’s own distribution does not enter the update (Hinton, Vinyals, and Dean 2015). This covers long chain-of-thought (CoT) (Wei et al. 2022) SFT recipes used in academic distillation studies (Magister et al. 2023; Ho, Schmid, and Yun 2023) and open pipelines such as OpenR1 (Hugging Face Open R1 Team 2025), NuminaMath (Li et al. 2024), and DeepSeek-R1 distillations (DeepSeek-AI 2025). (M2) On-policy distillation. The student generates rollouts, and teacher logits provide a KL-matching target for the generated tokens. The Qwen3 report (Yang et al. 2025, §4.5) states that the smaller dense models (0.6B–14B) and Qwen3-30B-A3B use off-policy distillation followed by on-policy distillation against 32B/235B teachers, rather than the four-stage SFT+RL pipeline used for flagship models. Thus, the Qwen3-4B and Qwen3-8B Official endpoints we evaluate are on-policy-distilled, not RL-trained. (M3) Reinforcement learning. The student generates rollouts scored by a rule-based verifier, and policy-gradient updates maximise expected verified reward. We use DeepSeek-Math-7B-RL (Shao et al. 2024) as our M3 endpoint, whose GRPO recipe starts from an SFT’d DeepSeek-Math-7B checkpoint. All comparisons are anchored at released base checkpoints within the same family, so post-training paths share a common pretraining reference even when endpoint trajectories are observed rather than controlled. Table 1 lists all 15 checkpoints with mechanism labels: our 9 M1 checkpoints provide the open-replicable trajectory, while M2 and M3 endpoints are observed endpoints. Checkpoint Post-training pipeline Source Qwen3-4B-Base – Qwen Qwen3-4B-SFT-Math-45k-ep1 Off-policy distill Ours Qwen3-4B-SFT-Math-45k-ep2 Off-policy distill Ours Qwen3-4B-SFT-Math-45k-ep3 Off-policy distill Ours Qwen3-4B (Official) Off-policy + On-policy distill Qwen Qwen3-8B-Base – Qwen Qwen3-8B-SFT-Math-90k-ep1 Off-policy distill Ours Qwen3-8B-SFT-Math-90k-ep2 Off-policy distill Ours Qwen3-8B-SFT-Math-90k-ep3 Off-policy distill Ours Qwen3-8B (Official) Off-policy + On-policy distill Qwen DeepSeek-Math-7B-Base – DeepSeek DeepSeek-Math-7B-SFT-hybrid-ep1 Off-policy distill Ours DeepSeek-Math-7B-SFT-hybrid-ep2 Off-policy distill Ours DeepSeek-Math-7B-SFT-hybrid-ep3 Off-policy distill Ours DeepSeek-Math-7B-RL Off-policy distill + RL DeepSeek Table 1: Model roster. Nine checkpoints (Ours) instantiate off-policy distillation. In the Source column, Ours marks checkpoints we trained; the rest are released endpoints. Where a released pipeline is listed as off-policy distillation, that stage is the vendor’s own, not one of ours: DeepSeek-Math-7B-RL applies GRPO on top of DeepSeek’s SFT checkpoint, and the Qwen3 Official endpoints distil off-policy then on-policy. 2.2 Our off-policy distillation training User: Find all real x such that x4−8x2+15=0x^4-8x^2+15=0. Assistant: <think> Let y=x2y=x^2. Then y2−8y+15=(y−3)(y−5)=0y^2-8y+15=(y-3)(y-5)=0, so y∈3,5y∈\3,5\. Therefore x2∈3,5x^2∈\3,5\, giving the four real roots x∈±3,±5x∈\± 3,\,± 5\. </think> The four real roots are ±3± 3 and ±5± 5. The final answer is −5,−3,3,5 \- 5,\,- 3,\, 3,\, 5\. Figure 2: A representative training sample for off-policy distillation (stylised; real samples average ∼ 3k tokens of ⟨ ⟩ content). Training corpora. For the Qwen3 family we use OpenR1-Math (Hugging Face Open R1 Team 2025), a public corpus of competition-level math problems paired with reasoning traces distilled from DeepSeek-R1. We retain samples with exactly one ⟨ ⟩…⟨ … /think⟩ block, a ⋅ boxed\·\ final answer (Figure 2), and total assistant length below 6060k characters (typical samples ≈ 6k tokens after Qwen3 tokenization; tail up to ≈ 26k). We draw two splits under a fixed seed: OpenR1-Math-45k for the 4B model and OpenR1-Math-90k for the 8B model; the size difference accommodates the larger model’s compute budget. For DeepSeek-Math-7B, whose native context window is only 4,0964,096 tokens, we construct a 4k-fitted hybrid corpus by combining two sources, filtered OpenR1-Math samples and short-form CoT traces sourced from NuminaMath-CoT (Li et al. 2024). Training procedure. All checkpoints are full-parameter fine-tunes using the verl SFT trainer (Sheng et al. 2024) under Fully Sharded Data Parallel (FSDP) with bfloat16 weights and activation checkpointing. Hardware is four nodes of four NVIDIA A100-40GB (1616 GPUs) for the 4B and 7B runs and eight nodes (3232 GPUs) for the 8B runs. Per-GPU micro-batch is fixed at 11, sequence packing is disabled, and validation runs on a held-out 200200-sample split every 100100 steps. Checkpoints are saved at every epoch boundary; downstream evaluation uses epoch checkpoints to keep the trajectory axis clean. Total wall-clock per 3-epoch run is ∼ 14 h on Qwen3-4B (16 A100), ∼ 20 h on Qwen3-8B (32 A100), and ∼ 8 h on DeepSeek-Math-7B (16 A100, native 4k context). The full hyperparameter list, training/validation loss curves, and learning-rate / gradient-norm trajectories are reported in Appendix B (Table 5; Figures 6, 7, 8). 2.3 Capability probe set (T0–T3) Probe Design Items Rollouts T0 verbatim 100100 4,8004,800 T1 paraphrase (3×3×) 300300 4,8004,800 T2 numerical isomorphism (3×3×) 300300 4,8004,800 T3 translation (55 langs) 500500 8,0008,000 Table 2: Capability probe inventory. T0–T3 transform a shared 100100-problem source pool—3030 problems from AIME 2025, 3030 from AIME 2026, and 4040 from AMC 2023. Rollout counts are per evaluated checkpoint. Conventional math benchmarks report pass@K on a single verbatim problem statement, conflating “can the model solve this surface form” with “can the model solve this class of problem”—a gap that surface-form memorisation or distillation leakage can hide. We therefore instantiate a verbatim anchor plus three perturbation axes around each source problem and ask whether capability survives each: T0 (verbatim). Original English—the conventional benchmark. T1 (paraphrase). 33 length-controlled English paraphrases per source preserving numerical content and ground-truth answer. Probes surface-form robustness. T2 (numerical isomorphism). 33 digit-preserving constant permutations per source; reasoning structure preserved but the ground-truth answer changes, ruling out answer-level memorisation. T3 (cross-lingual). 55 translations covering Chinese, Spanish, Swahili, Urdu, and Arabic. Probes cross-lingual generalisation. The full probe set contains 1,2001,200 variants per model (⋅(1+3+3+5)100\!·\!(1+3+3+5)): T0 is sampled K=48K\!=\!48 times per source (single variant); T1/T2/T3 are sampled K=16K\!=\!16 per variant, giving source-level pools of 4848 on T1/T2, 8080 on T3, and combined pass@K budgets of 9696 (T1∪ 2), 144144 (T0∪ 1∪ 2), and 224224 (full T0∪ 1∪ 2∪ 3). An honest capability gain should propagate beyond the verbatim anchor; surface-fragile gains register preferentially on T0. The inventory is shown in Table 2. 2.4 Sampling and strategy-diversity All rollouts use vLLM (Kwon et al. 2023) with T=0.7T=0.7, top-p=0.95p=0.95, and max_new_tokens=16,384 max\_new\_tokens=16,384 (Qwen3) or 4,0964,096 (DSMath native context). Responses are scored by a balanced-brace ⋅ boxed\·\ parser with LaTeX equivalence (Appendix Fig. 12 shows a representative rollout with full reasoning trace). We also test whether each post-training mechanism sharpens p(y∣x)p(y x) (Yue et al. 2025; Kirk et al. 2024) at three layers: answer-level (self-consistency majority share (Wang et al. 2023), distinct-answer count, entropy), response-level (pairwise 44-gram Jaccard (Li et al. 2016), MiniLM-L6 (Wang et al. 2020) embedding cosine distance, length CV), and cluster-level (single-link clustering at cosine ε=0.3 \!=\!0.3). The layers separate form-collapse (templates) from content-collapse (answers) (Stanton et al. 2021). 3 Cross-mechanism analysis We evaluate all three on two orthogonal axes: capability (the highest pass@K a model reaches when given K surface-perturbed attempts at a problem) and mechanism signature (the shape of the output distribution that produces those attempts). Overview. For each source problem the probe set yields an attempt budget, pooled at the source-problem level, of Kmax=1×48⏟T0+3×16⏟T1+3×16⏟T2+5×16⏟T3=224,K_ = 1×48_T0+ 3×16_T1+ 3×16_T2+ 5×16_T3=224, where T0 is the original problem and T1, T2, and T3 are variants. The cross-surface pass@K at K=224K=224 is therefore the probability that any of 224224 attempts—spanning paraphrase, numerical isomorphism, and translation—produces a correct answer for that source problem. This probe extends the capability-ceiling pass@K of Yue et al. (2025) from i.i.d. resampling on a single prompt to 1212 surface variants per source problem, which is a strictly stronger capability probe—i.i.d. resampling on the original prompt is the T0 special case (K≤48K≤ 48) of our K=224K=224 budget, and a model must solve the underlying problem under any surface to score (formal definition in Appendix A). 3.1 Compression vs. ceiling expansion The full cross-mechanism capability summary (pass@1, pass@16, pass@48, pass@96, pass@144, pass@224 for all 15 checkpoints) is reported in Table 6 (Appendix C). We read it along two axes: difficulty (AMC 2023 vs. the harder AIME 2025+2026 subset) and mechanism (off-policy distillation alone vs. off-policy + on-policy distillation vs. off-policy + GRPO RL). Figures 3 and 4 plot the full pass@Kpass@K trajectories; per-subset numeric anchor points at K∈1,48,96,144,224K∈\1,48,96,144,224\ for all three families are tabulated in Appendix C (Tables 7–8). (a) 4B, full set. (b) 8B, full set. (c) 4B, AIME (n=60n=60). (d) 8B, AIME (n=60n=60). (e) 4B, AMC (n=40n=40). (f) 8B, AMC (n=40n=40). Figure 3: Qwen3 cross-surface pass@K. (a) Full n=100n=100. (b) AIME (n=60n=60). (c) AMC (n=40n=40). Figure 4: DeepSeek-Math-7B cross-surface pass@K. Base, our off-policy distillation checkpoint, and the released GRPO endpoint are evaluated under the same pooled T0–T3 budget. Easy regime (AMC): mechanism differences collapse at the ceiling. On AMC 2023, pass@224pass@224 is already near saturation across families: 100/98/100100/98/100 for Qwen3-4B, 98/100/10098/100/100 for Qwen3-8B, and 90/98/9590/98/95 for DSMath-7B under Base/SFT-ep3/Endpoint (Tables 7, 8). Where headroom remains, SFT closes most of the gap, as in DSMath (→9890\!→\!98, +8+8p). At this budget, the endpoint mechanism adds little further lift: Qwen3 endpoints are capped at or near 100%100\%, while the DSMath RL endpoint is slightly below SFT-ep3 (9595 vs. 9898). Thus, in the easy regime, cross-surface robustness is limited mainly by saturation and the measurement ceiling rather than by a clearly separable endpoint mechanism. We therefore do not over-interpret within-family orderings, since with n=40n=40 the one-standard-error scale is already about ±3± 3p. Hard regime (AIME): post-training expands the ceiling. On AIME 2025+2026, where substantial headroom remains, post-training raises the large-K ceiling over the base models. In Qwen3, sufficient off-policy distillation already lifts pass@224pass@224 from →7747\!→\!77 (4B) and →8355\!→\!83 (8B), and the released off-policy-plus-on-policy endpoints raise the ceiling further to 90%90\% at both scales. Thus, in the hard regime, post-training is not only cheaper sampling of latent solutions; it can also expand reachable support. Sample-efficiency vs. ceiling expansion. The mechanism-specific picture is more nuanced. In the DeepSeek-Math family, the released GRPO endpoint behaves mainly as sample-efficiency reshaping: it improves pass@11 on the full set and AMC, but its large-K values are flat or lower than our off-policy endpoint, and on AIME it trails at both pass@11 and pass@224224. By contrast, the Qwen3 endpoints improve both pass@11 and the large-K ceiling over sufficient off-policy distillation. The headline pass@11 gain is still largely compression, but the residual large-K lift is genuine ceiling expansion. RL and on-policy distillation are therefore not exchangeable post-training mechanisms, even when both improve endpoint pass@11. Scale saturates at the cross-surface ceiling, not at pass@1pass@1. After on-policy distillation, Qwen3-4B and Qwen3-8B endpoints nearly coincide on both pass@1pass@1 and pass@224pass@224 (AIME: 55.6/9055.6/90 vs. 57.2/9057.2/90; AMC: 87.6/10087.6/100 vs. 88.2/10088.2/100), suggesting that this probe saturates by endpoint stage and that the residual scale gap is mainly visible along the SFT trajectory. 3.2 Multi-instance consistency Pass@K asks whether any sample succeeds; it does not say whether success survives a surface change. We therefore measure, for each source problem and probe class, whether all three variants receive at least one correct sample at K=16K=16 (all-3), whether any variant does (any-1), and the conditional failure rate among solved problems (fragile). Table 3 reports T1 paraphrases; the per-problem T0/T1/T2 visualisation is in Figure 10 (Appendix C). Family Stage all-3 any-1 fragile Qwen3-4B Base 26% 53% 51% SFT-ep3 58% 75% 23% Endpoint 84% 89% 6% Qwen3-8B Base 33% 55% 40% SFT-ep3 65% 81% 20% Endpoint 87% 90% 3% DSMath-7B Base 5% 22% 77% SFT-ep3 21% 46% 54% Endpoint 20% 45% 56% Table 3: T1 multi-instance consistency (per source problem; 33 paraphrases, K=16K=16 each). all-3: all variants solved; any-1: at least one variant solved; fragile = (any-1 −- all-3) / any-1. Consistency separates mechanisms: On T1, Qwen3 all-3 consistency increases monotonically from Base to SFT-ep3 to Endpoint: →84%26\!→\!58\!→\!84\% for 4B and →87%33\!→\!65\!→\!87\% for 8B, while fragility falls to 6%/3%6\%/3\%. T0 and T2 show the same consolidation pattern (Figure 10). DSMath is the contrast: its GRPO RL endpoint is indistinguishable from SFT-ep3 (20%20\% vs. 21%21\% all-3; 56%56\% vs. 54%54\% fragility). Together with the pass@Kpass@K inversion in §3.1, this suggests that this specific RL endpoint re-weights probability mass at pass@1pass@1 without consolidating solved problems into surface-robust modes. 3.3 Output-distribution signature: form vs. content sharpening Family Stage maj nu_u nc_c emb cvlen_len Qwen3-4B Base 19.6% 26.29 3.50 0.312 1.447 SFT-ep3 36.7% 8.36 1.00 0.123 0.390 Endpoint 70.1% 2.26 1.00 0.088 0.168 Qwen3-8B Base 19.4% 27.09 1.31 0.189 1.627 SFT-ep3 48.4% 6.51 1.00 0.118 0.327 Endpoint 69.5% 2.08 1.00 0.091 0.147 DSMath-7B Base 7.4% 13.96 24.07 0.738 1.433 SFT-ep3 11.4% 28.56 2.78 0.260 0.791 Endpoint 10.5% 9.76 1.52 0.224 0.910 Table 4: Strategy-diversity descriptors on the T0 K=48K=48 rollout pool (§2.4). Answers are extracted via ⋅ boxed\·\ / "Answer:" / "final answer is" patterns, then LaTeX-normalised. maj: self-consistency majority share—fraction of the 4848 samples giving the most common extracted answer (Wang et al. 2023). nu_u: distinct extracted answers per problem. nc_c: single-link clusters on MiniLM-L6 response embeddings at cosine ε=0.3 =0.3. emb: mean pairwise embedding cosine distance. cvlen_len: response-length coefficient of variation. Italics flag the DSMath Endpoint, where GRPO RL leaves maj unchanged (−0.9-0.9p) while contracting the answer support nun_u by −66%-66\% (§3.3). Pass@K and consistency only reveal whether correct samples appear; they do not show how the rollout distribution changes. We therefore describe each T0 K=48K=48 rollout pool along two axes. Form sharpening measures whether reasoning traces collapse into similar surface realisations, using response-length variation and embedding-cluster diversity. Answer-content sharpening measures whether the model concentrates on a small set of final answers, using self-consistency majority share and the number of distinct boxed answers. These axes are separable. Off-policy distillation consistently sharpens form, but its effect on answer content depends on the model family: Qwen3 contracts answer support, whereas DeepSeek-Math broadens it, likely because the hybrid SFT corpus introduces additional solution paths. The released Qwen3 on-policy-distilled endpoints sharpen both axes most strongly, reaching near-canonical reasoning traces and low answer diversity. By contrast, the DeepSeek-Math GRPO endpoint contracts the answer support relative to SFT but does not increase the majority-answer share (Fig. 5). This is support contraction without modal consolidation. Together with the pass@K and consistency results, this distributional view explains why mechanisms with similar endpoint gains need not be equivalent. The Qwen3 endpoints combine ceiling expansion, surface consistency, and answer-content consolidation. The DeepSeek-Math GRPO endpoint mainly prunes the SFT answer support without producing a higher large-K ceiling or stronger surface consistency in our setting. Figure 5: Content axis (maj, nun_u) across all checkpoints Answer extraction matches Table 4. 4 Cross-lingual generalisation Multilingual math reasoning has been evaluated on fixed translation sets such as Multilingual Grade School Math (MGSM) (Shi et al. 2022). We extend this idea to competition-style source problems and pool attempts within the same source problem. T3 (Section 2.3) covers 55 translations per source (Chinese, Spanish, Arabic, Urdu, Swahili). For this section we extend T3 with 33 additional languages chosen to broaden script and language-family coverage—Swedish, Icelandic, Tamil—using the same translation protocol and K=16K=16 per (source, language). Combined with 1616 T0 samples per source as the English anchor, this yields a 99-language cross-section over the same 100100 source problems for all 99 models. Languages are grouped by Qwen3-4B Base pass@16pass@16: stronger non-en == zh, es, sv, ar, the four non-English languages with the highest Qwen3-4B Base pass@16pass@16; weaker non-en == is, ta, ur, sw, the remaining four. Three readings across the 99 (family, role) cells of the multilingual table (full numbers and per-family bar charts in Appendix D, Table 9): (F1) the stronger non-English group \zh, es, sv, ar\ beats the weaker group \is, ta, ur, sw\ in every cell by +3+3 to +18+18p on mean pass@16pass@16, with the tier preserved by every mechanism; (F2) English-dominant SFT lifts every non-English language by +4+4 to +49+49p at SFT-ep3 across all 2424 non-English cells (largest: Icelandic on Qwen3-8B, 23→7223→72)—even Swahili moves up despite no direct supervision (§2.2); (F3) Swahili is a floor that no mechanism removes, trailing the next-worst language by 24/17/1024/17/10p at the Qwen3-4B/8B/DSMath-7B endpoints, and regressing 9→49→4 on DSMath-7B from SFT-ep3 to RL endpoint (the largest relative single-language regression in the table at −56%-56\%; DSMath es also drops 33→2733→27, −6-6p absolute)—a deficit fixed at Base and preserved or amplified by every mechanism, suggesting a pretraining-corpus rather than post-training property (visualised across 66 models in Appendix Fig. 16). Pooling all 99 languages (1616 samples each, 144144 per source) reproduces the same pattern as §3.1: on DSMath, GRPO RL leads at small K (sharpening), and SFT-ep3 matches or exceeds it from K≈16K≈16 onward. Qwen3 endpoints remain monotone-above SFT-ep3 throughout (Appendix Fig. 17). 5 Memorisation We report a controlled-overfit memorisation audit on 33 families × 33 epochs (00, 1010, 2020) × 33 probe modalities (behavioural prefix-completion, behavioural forbidden-CoT, forward-pass membership-inference attack (MIA) (Shi et al. 2024)). The overfit corpus is a 200200-item subset of each family’s canonical training source with an MD5-disjoint 200200-item held-out set; 1010/2020 epochs deliver an order of magnitude more per-item gradient passes than a typical SFT trajectory, designed to maximise detectability. Under these conditions, none of the three modalities yields a strong, reliable memorisation signal: behavioural Δ ’s are noisy across families and probe types (we omit them from the quantitative claims), and forward-pass MIA reports area under the receiver-operating-characteristic curve (ROC AUC) bounded within [.51,.65][.51,.65] across the overfit grid (Appendix Table 10), below the stronger pretraining-data detection signals reported in prior work (e.g., Min-K% Prob reaching AUC 0.880.88 for copyrighted-book detection (Shi et al. 2024)). Read as a sensitivity ceiling, this audit calibrates what current methods can resolve about SFT-specific membership and cautions against interpreting modest MIA AUCs as direct evidence of SFT memorisation. The full protocol is in Appendix E. 6 Discussion Our results suggest that mathematical post-training is best understood as distribution shaping over a difficulty-dependent capability landscape. Endpoint pass@11 conflates several effects: cheaper sampling of already reachable solutions, expansion of the large-K reachable ceiling, robustness to surface variation, and possible memorisation. These effects can move differently across mechanisms and therefore require trajectory-aware diagnosis. The compression account is real, but regime-bound. On easier AMC problems, the cross-surface large-K ceiling is near saturation, so post-training mainly improves sample efficiency: correct solutions become cheaper to sample while mechanisms converge near the same ceiling. On harder AIME problems, however, base models leave substantial headroom, and post-training expands the reachable ceiling. Crucially, this expansion already appears under our sufficiently trained off-policy distillation trajectories, making offline distillation a genuine capability-expanding post-training path rather than a weak SFT baseline. Released endpoints then provide mechanism-relative evidence: Qwen3 off-policy-plus-on-policy endpoints add further large-K lift, whereas the DeepSeek-Math GRPO endpoint sharpens small-K behaviour but does not exceed sufficient off-policy distillation at large K in our comparison. Robustness and distributional diagnostics further show that mechanisms are not interchangeable. Qwen3 endpoints consolidate solved problems across paraphrases and numerical isomorphisms, while the DeepSeek-Math GRPO endpoint remains close to its SFT predecessor in multi-instance consistency. Form sharpening and answer-content sharpening can also diverge, so more uniform reasoning traces should not be equated with more reliable reasoning unless correctness and cross-surface consistency improve as well. Cross-lingually, English-dominant distillation improves non-English reasoning but preserves stronger-versus-weaker language gaps, suggesting that post-training does not erase inherited coverage asymmetries. Finally, our controlled-overfit audit calibrates how much memorisation current membership probes can detect at all: the weak signals we measure bound the probes, not the phenomenon. 7 Conclusion We compare three mathematical post-training mechanisms—off-policy distillation, off-policy-plus-on-policy distillation, and off-policy-plus-GRPO RL—under a shared cross-surface diagnostic framework. Using cross-surface pass@K, consistency, distribution-shape, multilingual, and verified SFT-membership probes, we separate sample-cost compression from ceiling expansion, robustness, and memorisation sensitivity. Overall, compression is one regime of post-training, not a universal explanation of reasoning gains. Easy problems mainly expose sampling-cost compression, while hard problems show that sufficient post-training—already off-policy distillation—can expand the reachable ceiling over the base. Evaluating reasoning progress therefore requires surface-aware, trajectory-aware, and membership-controlled diagnostics beyond endpoint pass@11. Limitations Our comparisons are mechanism-relative rather than fully causal: the studied post-training paths differ in historical pipeline, and we isolate no single algorithmic factor. We cover nine languages, but translation quality, cultural specificity, and benchmark familiarity vary across them. The memorisation study calibrates detection sensitivity on our own controlled SFT data; it does not audit contamination from pretraining or from the data behind released endpoints. Ethical Statement This work uses publicly available competition mathematics problems (AIME 2025/2026, AMC 2023) and public open-source training corpora (OpenR1-Math, NuminaMath-CoT). No human subjects, personally identifiable information, or sensitive content are involved. All models analysed are openly released under their respective licenses, and our SFT training data is filtered and curated from public corpora rather than collected from new sources. The full reproducibility package—problem pools and surface transforms, rollouts, analysis intermediates, and the analysis code—accompanies this submission as the code and data archive, so every reported number can be independently verified. References Carlini et al. (2023) Carlini, N.; Ippolito, D.; Jagielski, M.; Lee, K.; Tramer, F.; and Zhang, C. 2023. Quantifying Memorization Across Neural Language Models. In ICLR. Carlini et al. (2021) Carlini, N.; Tramer, F.; Wallace, E.; Jagielski, M.; Herbert-Voss, A.; Lee, K.; Roberts, A.; Brown, T. B.; Song, D.; Erlingsson, Ú.; Oprea, A.; and Raffel, C. 2021. Extracting Training Data from Large Language Models. In 30th USENIX Security Symposium (USENIX Security 21), 2633–2650. Chen et al. (2021) Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; Pinto, H. P. D. O.; Kaplan, J.; Edwards, H.; Burda, Y.; Joseph, N.; Brockman, G.; et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Chen et al. (2025) Chen, Z.; Liu, T.; Tian, M.; Tong, Q.; Luo, W.; and Liu, Z. 2025. Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages. arXiv:2501.14002. DeepSeek-AI (2025) DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948. Guha et al. (2025) Guha, E.; et al. 2025. OpenThoughts: Data Recipes for Reasoning Models. arXiv:2506.04178. Hinton, Vinyals, and Dean (2015) Hinton, G.; Vinyals, O.; and Dean, J. 2015. Distilling the Knowledge in a Neural Network. In NeurIPS Deep Learning and Representation Learning Workshop. Ho, Schmid, and Yun (2023) Ho, N.; Schmid, L.; and Yun, S.-Y. 2023. Large Language Models Are Reasoning Teachers. In ACL. Hugging Face Open R1 Team (2025) Hugging Face Open R1 Team. 2025. OpenR1-Math: An Open Reproduction of DeepSeek-R1’s Mathematical Reasoning Distillation. https://huggingface.co/datasets/open-r1/OpenR1-Math-220k. Kirk et al. (2024) Kirk, R.; Mediratta, I.; Nalmpantis, C.; Luketina, J.; Hambro, E.; Grefenstette, E.; and Raileanu, R. 2024. Understanding the Effects of RLHF on LLM Generalisation and Diversity. In ICLR. Kwon et al. (2023) Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C. H.; Gonzalez, J. E.; Zhang, H.; and Stoica, I. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP). Li et al. (2024) Li, J.; Beeching, E.; Tunstall, L.; Lipkin, B.; Soletskyi, R.; Huang, S. C.; Rasul, K.; Yu, L.; Jiang, A.; Shen, Z.; Qin, Z.; Dong, B.; Zhou, L.; Fleureau, Y.; Lample, G.; and Polu, S. 2024. NuminaMath. https://huggingface.co/datasets/AI-MO/NuminaMath-CoT. Li et al. (2016) Li, J.; Galley, M.; Brockett, C.; Gao, J.; and Dolan, B. 2016. A Diversity-Promoting Objective Function for Neural Conversation Models. In NAACL. Magister et al. (2023) Magister, L. C.; Mallinson, J.; Adamek, J.; Malmi, E.; and Severyn, A. 2023. Teaching small language models to reason. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 1773–1781. Mirzadeh et al. (2024) Mirzadeh, I.; Alizadeh, K.; Shahrokhi, H.; Tuzel, O.; Bengio, S.; and Farajtabar, M. 2024. GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models. arXiv:2410.05229. Muennighoff et al. (2025) Muennighoff, N.; Yang, Z.; Shi, W.; Li, X. L.; Fei-Fei, L.; Hajishirzi, H.; Zettlemoyer, L.; Liang, P.; Candes, E.; and Hashimoto, T. 2025. s1: Simple Test-Time Scaling. arXiv:2501.19393. Shao et al. (2024) Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y. K.; Wu, Y.; and Guo, D. 2024. DeepSeek-Math: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300. Sheng et al. (2024) Sheng, G.; Zhang, C.; Ye, Z.; Wu, X.; Zhang, W.; Zhang, R.; Peng, Y.; Lin, H.; and Wu, C. 2024. HybridFlow: A Flexible and Efficient RLHF Framework. arXiv:2409.19256. Shi et al. (2022) Shi, F.; Suzgun, M.; Freitag, M.; Wang, X.; Srivats, S.; Vosoughi, S.; Chung, H. W.; Tay, Y.; Ruder, S.; Zhou, D.; Das, D.; and Wei, J. 2022. Language Models are Multilingual Chain-of-Thought Reasoners. arXiv preprint arXiv:2210.03057. Shi et al. (2024) Shi, W.; Ajith, A.; Xia, M.; Huang, Y.; Liu, D.; Blevins, T.; Chen, D.; and Zettlemoyer, L. 2024. Detecting pretraining data from large language models. In International Conference on Learning Representations, volume 2024, 51826–51843. Srivastava et al. (2024) Srivastava, S.; Annarose M B; Anto P V; Menon, S.; Sukumar, A.; Adwaith Samod T; Philipose, A.; Prince, S.; and Thomas, S. 2024. Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap. arXiv:2402.19450. Stanton et al. (2021) Stanton, S.; Izmailov, P.; Kirichenko, P.; Alemi, A. A.; and Wilson, A. G. 2021. Does Knowledge Distillation Really Work? In NeurIPS. Wang et al. (2020) Wang, W.; Wei, F.; Dong, L.; Bao, H.; Yang, N.; and Zhou, M. 2020. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. Advances in neural information processing systems, 33: 5776–5788. Wang et al. (2023) Wang, X.; Wei, J.; Schuurmans, D.; Le, Q. V.; Chi, E. H.; Narang, S.; Chowdhery, A.; and Zhou, D. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In ICLR. Wei et al. (2022) Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 24824–24837. Yang et al. (2025) Yang, A.; et al. 2025. Qwen3 Technical Report. arXiv:2505.09388. Ye et al. (2025) Ye, Y.; Huang, Z.; Xiao, Y.; Chern, E.; Xia, S.; and Liu, P. 2025. LIMO: Less is More for Reasoning. arXiv:2502.03387. Yuan et al. (2025) Yuan, L.; Chen, W.; Zhang, Y.; Cui, G.; Wang, H.; You, Z.; Ding, N.; Liu, Z.; Sun, M.; and Peng, H. 2025. From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones. arXiv:2509.25123. Yue et al. (2025) Yue, Y.; Chen, Z.; Lu, R.; Zhao, A.; Wang, Z.; Yue, Y.; Song, S.; and Huang, G. 2025. Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? In Advances in Neural Information Processing Systems, volume 38, 57654–57689. Appendix A Formal definition of cross-surface pass@K For each source problem s and a subset of transforms ′⊆T0,T1,T2,T3T \! \!\T_0,T_1,T_2,T_3\, let N be the size of the pool of all rollouts drawn from variants of s under transforms in ′T , and let n be the count of those rollouts scored correct by math_verify. The unbiased pass@K estimator (Chen et al. 2021) is pass@K(s)=0,n=0,1,N−n<K,1−(N−nK)/(NK),else.pass@K(s)= cases0,&n=0,\\[2.0pt] 1,&N-n<K,\\[4.0pt] 1- N-nK\! /\! NK,&else. cases We report the source-level mean on a subset ′⊆s1,…,s100S \! \!\s_1,…,s_100\, denoted pass@K(′)pass@K(S ). Every reported cell has K≤NK≤ N. The pool behind each column is given by the probe set row of each pass@K table, and ′S by its caption. Two places do not fit this formalism and state their own convention instead: the multilingual table draws its en column from the first 1616 of the 4848 T0 samples and each non-English column from the 1616 T3 samples of that language, and the 99-language pooled figures combine those same 1616 T0 samples with eight T3 languages. Appendix B Training details This appendix reports the full hyperparameter list (Table 5), training/validation loss curves (Figure 6), and per-step learning-rate (Figure 7) and gradient-norm (Figure 8) trajectories for the representative SFT runs, confirming that the cosine schedule listed in Table 5 was executed as specified and that no run diverged. Figure 7: peak 2×10−52× 10^-5 is reached after ∼1% 1\% of total steps (linear warm-up) and decays smoothly to 2×10−62× 10^-6, exactly 0.1×0.1× peak as set by min_lr_ratio. Figure 8: gradient norm stays bounded throughout training, with no divergence or loss explosion. Hyperparameter Value Optimiser AdamW, (β1,β2)=(0.9,0.999)( _1, _2)=(0.9,0.999) Peak learning rate 2×10−52× 10^-5 LR schedule Cosine, min ratio 0.10.1 Warm-up 1%1\% of total steps Weight decay 0.10.1 Gradient clipping 1.01.0 Precision bfloat16 (FSDP, activation ckpt; activation offload on the 8B and 7B runs only) Max sequence length 32,76832,768 (Qwen3) / 4,0964,096 (DSMath, native) Micro-batch per GPU 11 Global batch (4B) 1616 Global batch (7B / 8B) 3232 Epochs 33 Total tokens (Qwen3-4B) ≈0.80≈ 0.80 B Total tokens (Qwen3-8B) ≈1.56≈ 1.56 B Total tokens (DSMath-7B) ≈0.34≈ 0.34 B Table 5: SFT training hyperparameters. (a) Qwen3-4B train/loss (b) Qwen3-8B train/loss (c) DSMath-7B train/loss (d) Qwen3-4B val/loss (e) Qwen3-8B val/loss (f) DSMath-7B val/loss Figure 6: Training (top row) and validation (bottom row) loss across three epochs of off-policy distillation. Figure 7: Learning-rate schedule for the SFT runs. Left to right: Qwen3-4B-SFT-Math-45k, Qwen3-8B-SFT-Math-90k, and DSMath-7B-SFT-hybrid Figure 8: Gradient norm for the SFT runs (clip threshold 1.01.0, see Table 5). No divergence over three epochs. Appendix C Cross-mechanism analysis This appendix reports the full cross-mechanism capability summary (Table 6) and per-subset numeric anchor points referenced from §3.1. Table 6 reports pass@Kpass@K at K∈1,16,48,96,144,224K∈\1,16,48,96,144,224\ for all 1515 checkpoints. The four subset tables that follow give the values at K∈1,48,96,144,224K∈\1,48,96,144,224\ on the harder AIME 2025+2026 subset (n=60n=60) and the potentially-contaminated AMC 2023 subset (n=40n=40) for both Qwen3 families (Tables 7, 7) and the DeepSeek-Math-7B family (Tables 8, 8). Figure 9 provides a combined cross-mechanism view on AIME (Fig. 9(a)) and AMC (Fig. 9(b)). It overlays six curves on a single axis: M1 (off-policy distill, SFT-ep3) for all three families, M2 (on-policy distill) for the two Qwen3 families, and M3 (GRPO RL) for DeepSeek-Math-7B. No family has all three mechanisms: M2 exists only as a released Qwen3 endpoint and M3 only as a released DeepSeek-Math endpoint. Family Stage Post-training pass@1 pass@16 pass@48 pass@96 pass@144 pass@224 ratio probe set T0 T0 T0 T1∪ 2 T0∪ 1∪ 2 ∪ 3 Qwen3-4B Base – 10.5 39 49 62 66 68 6.48× SFT-ep1 Off-policy distill 36.6 64 70 78 81 85 2.32× SFT-ep2 Off-policy distill 38.9 66 75 80 81 84 2.16× SFT-ep3 Off-policy distill 38.7 67 76 82 84 85 2.19× Endpoint Off + On distill 68.4 87 90 93 93 94 1.37× Qwen3-8B Base – 12.8 44 50 66 68 72 5.62× SFT-ep1 Off-policy distill 44.6 73 79 84 84 84 1.88× SFT-ep2 Off-policy distill 46.5 75 80 86 86 88 1.89× SFT-ep3 Off-policy distill 46.5 77 82 87 89 90 1.94× Endpoint Off + On distill 69.6 89 90 94 94 94 1.35× DSMath-7B Base – 1.1 12 22 34 40 45 40.00× SFT-ep1 Off-policy distill 7.1 31 44 52 59 60 8.45× SFT-ep2 Off-policy distill 7.1 31 45 54 59 60 8.42× SFT-ep3 Off-policy distill 7.4 35 48 54 58 66 8.87× Endpoint Off + RL 9.7 29 41 51 56 60 6.18× Table 6: Capability across three post-training mechanisms. Cross-surface pass@Kpass@K pooled across T0/T1/T2/T3 surfaces (see §3). The ratio column reports pass@224/pass@1pass@224/pass@1, a per-checkpoint sample-efficiency multiplier (computed from unrounded values). (a) AIME 2025+2026 (n=60n=60). (b) AMC 2023 (n=40n=40, possibly contaminated) Figure 9: Cross-mechanism pass@K pooled across T0+T1+T2+T3 (Kmax=224K_ =224) Family Stage pass@1 pass@48 pass@96 pass@144 pass@224 probe set T0 T0 T1∪ 2 T0∪ 1∪ 2 ∪ 3 Qwen3-4B Base 2.5 27 37 43 47 SFT-ep3 22.7 62 72 75 77 Endpoint 55.6 83 88 88 90 Qwen3-8B Base 2.9 27 45 48 55 SFT-ep3 28.6 72 80 82 83 Endpoint 57.2 83 90 90 90 (a) AIME 2025+2026 (n=60n=60). On the cross-surface ceiling (pass@224pass@224) the on-policy distillation endpoint adds +13+13p on Qwen3-4B (77→9077→90) and +7+7p on Qwen3-8B (83→9083→90). Single-attempt (pass@1pass@1) gains +33+33/+29+29p are sample-efficiency driven. Family Stage pass@1 pass@48 pass@96 pass@144 pass@224 probe set T0 T0 T1∪ 2 T0∪ 1∪ 2 ∪ 3 Qwen3-4B Base 22.4 82 100 100 100 SFT-ep3 62.8 98 98 98 98 Endpoint 87.6 100 100 100 100 Qwen3-8B Base 27.7 85 98 98 98 SFT-ep3 73.2 98 98 100 100 Endpoint 88.2 100 100 100 100 (b) AMC 2023 (n=40n=40, possibly contaminated). AMC predates every model release. The pass@K≥48pass@K≥48 ceiling is already at 9898–100100 after off-policy distillation; the on-policy lift concentrates at pass@1pass@1 (+25+25p 4B, +15+15p 8B). Table 7: Qwen3 difficulty-stratified capability. Stage pass@1 pass@48 pass@96 pass@144 pass@224 probe set T0 T0 T1∪ 2 T0∪ 1∪ 2 ∪ 3 Base 0.0 2 7 8 15 SFT-ep3 1.1 22 33 37 45 Endpoint (RL) 1.0 20 28 35 37 (a) AIME 2025+2026 (n=60n=60). GRPO RL does not exceed SFT-ep3 at any K≥48K≥ 48, and the point estimates favour SFT-ep3 throughout (Δpass@224=−8 pass@224=-8p, →3745\!→\!37). A paired bootstrap over source problems puts every interval across zero, including the 0.10.1p gap at pass@1pass@1. Italics flag the SFT-ep3 ≥ RL ordering. Stage pass@1 pass@48 pass@96 pass@144 pass@224 probe set T0 T0 T1∪ 2 T0∪ 1∪ 2 ∪ 3 Base 2.8 52 75 88 90 SFT-ep3 17.0 88 85 90 98 Endpoint (RL) 22.8 72 85 88 95 (b) AMC 2023 (n=40n=40, possibly contaminated). RL gains +5.8+5.8p at pass@1pass@1 over off-policy distillation (single-shot sharpening); this is the only contrast in the table whose bootstrap interval excludes zero. Beyond it the ordering reverses— −15-15p on the T0 pool at pass@48pass@48, −2.5-2.5p on the full cross-surface pool at pass@224pass@224 (one problem out of 4040)—within intervals that do not. Table 8: DeepSeek-Math-7B difficulty-stratified capability. (a) T0: 3 re-evaluations of the original prompt. (b) T1: 3 paraphrases. (c) T2: 3 numerical isomorphisms. Figure 10: Multi-instance consistency across T0, T1, T2. Each panel uses three instances of K=16K=16 samples per source problem. T1/T2: three paraphrase/isomorphism variants. T0: the K=48K=48 original-prompt pool partitioned into three disjoint 1616-sample batches so the per-instance budget matches T1/T2. Green: all three instances solved (at least one correct sample each); orange: some-but-not-all; grey: unsolved on all three. All three panels show the same monotone Base→ -ep3 consolidation, with the Qwen3 endpoints further lifting all three transforms; the DSMath RL endpoint stays within ±4± 4p of SFT-ep3 on all three (T0 −4-4, T1 −1-1, T2 +3+3p), consistent with §3.2. Appendix D Cross-lingual evaluation Figure 11: Representative T3 case. Source problem (amc23__000, English T0; same item as Figure 11): Cities A and B are 4545 miles apart. Alicia lives in A and Beth lives in B. Alicia bikes towards B at 1818 miles per hour. Leaving at the same time, Beth bikes toward A at 1212 miles per hour. How many miles from City A will they be when they meet? Ground truth: 2727. Model: Qwen3-4B-SFT-Math-45k-ep3 (off-policy distilled, our SFT-ep3 checkpoint). Sampled at T=0.7T=0.7, top-p=0.95top-p=0.95. Sample 40/4840/48, 406406 new tokens, scored correct. Full response (verbatim, 1,3371,337 characters; line breaks preserved): <think> Okay, let’s see. So, Alicia and Beth are biking towards each other from cities A and B, which are 4545 miles apart. Alicia starts from A at 1818 mph, and Beth starts from B at 1212 mph. We need to find out how many miles from A they will meet each other. Hmm, this seems like a relative speed problem. When two objects move towards each other, their speeds add up. So, the combined speed at which they are approaching each other is 18+12=3018+12=30 mph. That makes sense. Now, the total distance between them is 4545 miles. If they’re moving towards each other at 3030 mph, how long will it take for them to meet? Well, time is distance divided by speed. So, time =45=45 miles /30/30 mph. Let me calculate that. 4545 divided by 3030 is 1.51.5 hours. So, they will meet after 1.51.5 hours. Now, we need to find out how far Alicia has traveled in that time. Since speed is distance divided by time, distance == speed × time. Alicia’s speed is 1818 mph, and time is 1.51.5 hours. Let me compute that. 18×1.518× 1.5. Hmm, 1818 times 11 is 1818, and 1818 times 0.50.5 is 99. So, 18+9=2718+9=27 miles. So, Alicia will have biked 2727 miles from A when they meet. Alternatively, we could check Beth’s distance. Beth’s speed is 1212 mph. 12×1.512× 1.5 is 1818 miles. 27+18=4527+18=45, which checks out. So that’s correct. Therefore, the answer should be 2727 miles from City A. </think> Answer: 2727 Figure 12: Representative model response with reasoning. Off-policy distilled checkpoint (Qwen3-4B SFT-ep3) on the same source problem as Figure 11. The aggregated multilingual numbers referenced in §4 are shown in Table 9; per-family breakdowns are in Figure 13 (Qwen3-4B), Figure 14 (Qwen3-8B), and Figure 15 (DeepSeek-Math-7B). Model Metric en stronger non-en weaker non-en en zh es sv ar is ta ur sw Qwen3-4B Base pass@1pass@1 9.6 11.9 15.0 13.5 8.4 2.1 5.9 7.6 1.8 pass@16pass@16 37 45 48 42 38 15 33 38 15 Qwen3-4B SFT-ep3 pass@1pass@1 37.3 27.8 38.4 38.4 34.0 31.7 34.6 32.9 15.4 pass@16pass@16 69 53 66 68 69 64 63 67 47 Qwen3-4B Endpoint pass@1pass@1 69.2 64.8 68.9 70.1 69.8 58.8 58.2 58.6 31.4 pass@16pass@16 88 83 88 85 87 77 77 80 53 Qwen3-8B Base pass@1pass@1 12.4 9.8 19.4 13.5 15.9 2.9 9.7 7.9 2.3 pass@16pass@16 43 41 48 42 44 23 37 36 21 Qwen3-8B SFT-ep3 pass@1pass@1 47.2 33.3 44.6 40.8 39.5 38.6 39.0 22.6 29.6 pass@16pass@16 78 63 72 74 73 72 72 60 58 Qwen3-8B Endpoint pass@1pass@1 69.3 65.3 70.1 69.9 68.6 65.0 61.1 63.7 43.2 pass@16pass@16 90 86 89 89 86 86 84 84 67 DSMath-7B Base pass@1pass@1 1.1 0.9 1.2 0.4 1.1 0.2 0.6 0.4 0.3 pass@16pass@16 11 7 14 5 10 4 9 5 5 DSMath-7B SFT-ep3 pass@1pass@1 7.4 3.7 6.2 4.7 4.2 1.8 1.6 1.1 0.8 pass@16pass@16 35 25 33 25 28 19 19 11 9 DSMath-7B Endpoint pass@1pass@1 10.0 6.9 9.1 9.0 6.1 3.8 3.9 1.7 0.4 pass@16pass@16 30 25 27 28 29 25 20 14 4 Table 9: Multilingual cross-surface reasoning, 99 models × 9×\,9 languages. Each model contributes two rows: pass@1pass@1 and pass@16pass@16. Figure 13: Qwen3-4B family. pass@1pass@1 (left) and pass@16pass@16 (right) across 99 languages. Bars: Base / SFT-ep3 (ours) / Endpoint (on-policy distillation). Figure 14: Qwen3-8B family. Same layout as Figure 13; Base / SFT-ep3 / Endpoint comparison at 8B scale. Figure 15: DeepSeek-Math-7B family. Endpoint is the released GRPO RL checkpoint. y-axis truncated to [0,40]%[0,40]\% to keep sub-percent differences visible at smaller pass@1pass@1 baselines. Figure 16: Multilingual imbalance across mechanisms. We plot pass@16pass@16 over English, stronger non-English languages (zh, es, sv, ar), and weaker non-English languages (is, ta, ur, sw) for each post-trained model. (a) Absolute scores show the family-level performance gap. (b) English-relative retention (pass@16lang/pass@16enpass@16_lang/pass@16_en) normalises across families. Weaker-language gaps persist after post-training. (a) Qwen3-4B, AIME (n=60n=60). (b) Qwen3-4B, AMC (n=40n=40). (c) Qwen3-8B, AIME (n=60n=60). (d) Qwen3-8B, AMC (n=40n=40). (e) DSMath-7B, AIME (n=60n=60). (f) DSMath-7B, AMC (n=40n=40). Figure 17: 9-language pooled pass@K split by difficulty subset. Each panel pools 99 languages (144144 samples per source) and plots source-level pass@K for K=1…144K=1… 144. Rows: Qwen3-4B / Qwen3-8B / DSMath-7B; columns: AIME 2025+2026 (n=60n=60) and AMC 2023 (n=40n=40). On DSMath, GRPO RL leads at small K on AMC (sharpening) and SFT-ep3 first matches RL at K≈24K≈24, leading at every larger K. Qwen3 endpoints stay monotone-above SFT-ep3 throughout. Appendix E Memorisation: positive controls Construction. For each base model we sample a 200200-item seen set from its canonical SFT corpus (OpenR1-Math-220k for Qwen3, dsmath-hybrid for DSMath) and a 200200-item unseen held-out set drawn MD5-disjoint from 220220k. Each base is then fine-tuned on only the seen set for 1010 and 2020 epochs (full-parameter verl SFT, batch 3232, lr 2×10−52×10^-5), deliberately producing the overfit regime that should maximally expose memorisation. Probes. Each (model, ckpt) cell is evaluated under three independent probes targeting different memorisation traces: (i) Implicit — 40%40\% problem prefix ++ “The final answer is:”, 10241024 tokens, any-of-1616 boxed-match (Carlini et al. 2021); (i) Direct — full problem ++ “no reasoning, only ”, 128128 tokens (answer-recall under forbidden CoT); (i) MIA — forward-pass Loss / MIN-K% / Zlib AUC over the ground-truth answer tokens, conditioned on the problem (Shi et al. 2024); no sampling. Behavioural cells (i, i) use one-sided Mann–Whitney on n=200n=200 items per split; MIA reports ROC AUC against the verified split label. Behavioural probes are noisy. Within the same (family, ckpt) cell, qualitative inspection finds verbatim recitation, length-truncated CoT, and ordinary problem-solving all co-occurring. Single Δseen−unseen _seen-unseen values are therefore highly noisy and inherently unreliable. We exclude behavioural numbers from the headline range and rely on the deterministic MIA AUC below. MIA caveats. Forward-pass MIA (Table 10) is one forward pass per item, so its AUC is stable. Two caveats temper the bounded range reported in the main text: (i) Qwen3 Loss AUC responds to overfit (+3+3–44p from ep00 to ep2020), but DSMath Loss AUC is already ≈0.65≈\!0.65 at ep00 (no overfit) and barely moves with epoch—part of the DSMath signal therefore reflects baseline corpus statistics rather than memorisation per se; (i) we did not run a randomised baseline (AUC after relabelling seen/unseen at random) to anchor what 0.650.65 means in this regime. Family Epochs Loss MIN-K% Zlib ROC AUC vs. unseen Qwen3-4B 0 0.562 0.627 0.513 10 0.603 0.648 0.548 20 0.596 0.644 0.540 Qwen3-8B 0 0.576 0.592 0.548 10 0.613 0.606 0.583 20 0.613 0.602 0.583 DSMath-7B 0 0.649 0.543 0.638 10 0.644 0.558 0.631 20 0.645 0.555 0.633 Table 10: Forward-pass MIA ROC AUC across the overfit grid. 200200 seen vs. 200200 unseen items per cell, one forward pass per item (no sampling). All entries stay within [.51,.65][.51,.65], below the stronger pretraining-data detection signals reported in prior work (e.g., Min-K% Prob reaching AUC 0.880.88 for copyrighted-book detection (Shi et al. 2024)). Qwen3 Loss AUC climbs +3+3–44p from 00 to 2020 overfit epochs (an overfit-sensitive signal), while DSMath Loss AUC starts at ≈0.65≈\!0.65 without overfit and stays essentially flat (baseline corpus statistics, not memorisation).