Paper deep dive

TailSFT: Filtered Fine-Tuning Improves Post-Training Performance

Sadhika Malladi, Samy Jelassi, Dylan Foster, Jordan T. Ash, Akshay Krishnamurthy

Year: 2026Venue: arXiv preprintArea: cs.LGType: PreprintEmbeddings: 160

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 8/29/2026, 3:24:52 AM

Summary

The paper introduces TailSFT, a supervised fine-tuning (SFT) modification that filters out already-fit sequences to focus learning on under-modeled regions of the data distribution. This approach improves coverage (pass@K) and serves as a better initialization for subsequent reinforcement learning (RL) post-training, specifically GRPO, on models like OLMo-3 7B, achieving significant gains in math and coding tasks.

Entities (8)

TailSFT · algorithm · 98%OLMo-3-7B · model · 95%GRPO · algorithm · 92%pass@k · metric · 90%Reinforcement Learning · process · 90%Supervised Fine-Tuning · process · 90%Coverage · concept · 88%Cross Entropy · metric · 85%

Relation Signals (7)

TailSFT → improves → pass@k

confidence 95% · TailSFT often improves pass@16 performance on math and coding evaluations

TailSFT → isa → Supervised Fine-Tuning

confidence 95% · we design a simple modification to supervised fine-tuning, TailSFT

OLMo-3 7B → isfinetunedwith → TailSFT

confidence 95% · On OLMo-3 7B, TailSFT often improves pass@16 performance

TailSFT → optimizesfor → Coverage

confidence 92% · TailSFT, an SFT algorithm that prioritizes coverage (or high pass@k at large k) over low cross-entropy loss

pass@k → predicts → Post-RL Performance

confidence 90% · prior work highlighting the role of coverage and pass@K as predictors of post-RL performance

TailSFT → servesas → Initialization

confidence 90% · demonstrating that TailSFT checkpoints serve as better initializations for RL

Cross Entropy → isnotalignedwith → Coverage

confidence 85% · Consequently, cross-entropy and coverage need not be aligned.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Reinforcement learning post-training drives reasoning and agentic capabilities in modern AI systems, yet a growing body of work shows that it is most effective when used to fine-tune an already capable base model. We question whether existing pipelines yield models that are most suitable for reinforcement learning. Building on prior work highlighting the role of coverage and pass@K as predictors of post-RL performance, we design a simple modification to supervised fine-tuning, TailSFT, which filters out already fit sequences during training, thereby focusing learning on under-modeled regions, or the tail, of the data distribution. We justify and validate the design choices in TailSFT, particularly the specific filtering criteria, through a combination of controlled experiments and theoretical analysis. On OLMo-3 7B, TailSFT often improves pass@16 performance on math and coding evaluations, with gains up to 17% absolute, while incurring minimal computational overhead. These higher-coverage checkpoints consistently translate to up to 4% absolute pass@1 gains in subsequent GRPO runs, demonstrating that TailSFT checkpoints serve as better initializations for RL. We further introduce a lightweight diagnostic for identifying settings where TailSFT is most likely to help. More broadly, our results motivate a principled, stage-aware approach to model development, in which intermediate checkpoints are judged by how effectively they support subsequent training.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

157,122 characters extracted from source content.

Expand or collapse full text

[name=Theorem,parent=section]theorem [name=Lemma,parent=section]lemma [name=Assumption, parent=section]assumption [name=Definition, parent=section]definition [name=Condition, parent=section]condition [name=Corollary, parent=section]corollary [name=Claim, parent=section]claim [qed=⊲ ,name=Example,style=definition, parent=section]example [name=Remark, parent=section]remark [name=Proposition, parent=section]proposition [name=Fact, parent=section]fact TailSFT: Filtered Fine-Tuning Improves Post-Training Performance Abstract Reinforcement learning post-training drives reasoning and agentic capabilities in modern AI systems, yet a growing body of work shows that it is most effective when used to fine-tune an already capable base model. We question whether existing pipelines yield models that are most suitable for reinforcement learning. Building on prior work highlighting the role of coverage and pass@K as predictors of post-RL performance, we design a simple modification to supervised fine-tuning, TailSFT, which filters out already fit sequences during training, thereby focusing learning on under-modeled regions, or the tail, of the data distribution. We justify and validate the design choices in TailSFT, particularly the specific filtering criteria, through a combination of controlled experiments and theoretical analysis. On OLMo-3 7B, TailSFT often improves pass@1616 performance on math and coding evaluations, with gains up to 17%17\% absolute, while incurring minimal computational overhead. These higher-coverage checkpoints consistently translate to up to 4%4\% absolute pass@11 gains in subsequent GRPO runs, demonstrating that TailSFT checkpoints serve as better initializations for RL. We further introduce a lightweight diagnostic for identifying settings where TailSFT is most likely to help. More broadly, our results motivate a principled, stage-aware approach to model development, in which intermediate checkpoints are judged by how effectively they support subsequent training. Sadhika Malladi22 2 University of California San Diego, sadhika.malladi98@gmail.com. Work partially completed while at Microsoft Research NYC. Samy Jelassi33 3 Microsoft Research NYC, samyjelassi,dylanfoster,ash.jordan,akshaykr@microsoft.com. Dylan J. Foster33 3 Microsoft Research NYC, samyjelassi,dylanfoster,ash.jordan,akshaykr@microsoft.com. Jordan T. Ash33 3 Microsoft Research NYC, samyjelassi,dylanfoster,ash.jordan,akshaykr@microsoft.com. Akshay Krishnamurthy33 3 Microsoft Research NYC, samyjelassi,dylanfoster,ash.jordan,akshaykr@microsoft.com. Figure 1: TailSFT yields better coverage, possibly at the cost of worse cross-entropy, which translates to stronger post-RL performance. Standard SFT achieves lower cross-entropy, while TailSFT preserves higher pass@K at large K. Starting from this higher-coverage checkpoint, the same RL procedure yields higher pass@11. Curves are illustrative. 1 Introduction Language models are commonly trained in multiple phases, each defined by a different dataset and objective. A typical pipeline includes (1) next-token prediction on a general-purpose corpus, (2) supervised fine-tuning (SFT) for instruction following and domain adaptation, and (3) reinforcement learning (RL) for complex reasoning tasks. A tacit assumption in such pipelines is that improving the objective at one stage also produces a better initialization for the next. Recent work suggests that this assumption need not hold: models that perform better according to the local objective of one stage can nevertheless be worse starting points for subsequent training (Zhang et al., 2026a; Wang et al., 2025; Springer et al., 2025; Chen et al., 2025). These findings motivate a stage-aware view of model development, in which an intermediate model is judged not only by its standalone performance, but also by how effectively it supports the training that follows. This issue is particularly important at the transition from SFT to RL. RL has produced substantial gains in mathematics, code, and other reasoning domains (Shao et al., 2024; Guo et al., 2025; 34; Zhao et al., 2026a; 44), but requires substantial computation to generate and evaluate on-policy responses. In RL with verifiable rewards (RLVR), the usefulness of these rollouts depends strongly on the initialization. If rewarding responses are difficult to sample from the SFT model, many rollout groups contain little direct positive learning signal; conversely, if rewarding responses are reachable under repeated sampling, RL has more useful behavior to reinforce. Thus, designing SFT for subsequent RL requires a criterion that captures not merely single-sample accuracy, but whether useful responses remain accessible within the rollout budget available to RL. The coverage principle formalizes this desideratum by relating a model’s coverage profile to what repeated sampling can recover (Chen et al., 2026a) (Section 2). Empirically, pass@k provides a convenient measure of this property, in that it quantifies how often k samples contain at least one reward-bearing response. At values of k comparable to the sampling budget used during RL, large-k pass@k therefore measures how frequently the initialization exposes RL to a useful response. Standard SFT, however, is not explicitly designed to preserve or improve this form of coverage. Cross-entropy continues to reward increases in the likelihood of every demonstration, including responses that the model can readily produce already. Further fitting these responses can shift probability mass away from potentially useful responses represented by the initial model. Consequently, cross-entropy and coverage need not be aligned. A model with worse cross-entropy can have better coverage, and vice versa (Chen et al., 2026a; Chen et al., 2025). On the other hand, directly optimizing the coverage profile is difficult, in part because the criterion is non-differentiable and depends on information about the target response distribution that is generally unavailable (Section 2). This raises the question we study in this work: Can we modify supervised fine-tuning so that it better preserves the useful response coverage needed for subsequent reinforcement learning? Contributions To better prepare models for RL post-training, we propose TailSFT, an SFT algorithm that prioritizes coverage (or high pass@k at large k) over low cross-entropy loss (Algorithm 1). TailSFT is a sequence-level filtering method designed to direct training effort towards learning the under-modeled tail of the response distribution. The algorithm is lightweight and serves as a straightforward drop-in objective for SFT. 1. We isolate the effect of filtering in a controlled graph navigation task (Section 3.2) and show that filtering already-fit examples improves coverage, achieving higher pass@k at large k despite worse cross-entropy and pass@11 (Figures 2 and 3). 2. We design TailSFT to filter relative to the initial policy and show that this criterion provably ensures better coverage than standard cross-entropy or more naïve filtering (Section 3.3). 3. We apply TailSFT to OLMo-3 7B on standard math and coding tasks and obtain substantial gains in coverage and post-RL performance. TailSFT improves pass@1616, in absolute terms, by up to 16.8% on coding and 3.1% on math (Table 1), and improves final pass@11 after GRPO by up to 3.9% (Table 2). 4. We introduce a lightweight coverage-ratio diagnostic (Section 4.2), computed from the base model and a standard SFT run, that provides a sufficient condition for TailSFT to improve coverage (Figure 4). Broader outlook Our results illustrate a broader source of suboptimality in multi-stage model training, where the objective that is appropriate for producing a strong model at one stage may not be the objective that produces the best initialization for the next. For the SFT-to-RL transition studied here, this distinction means that high large-k pass@k can be more valuable than lower cross-entropy, even when the two criteria are in tension. Recent work has documented related forms of misalignment across pre-training, fine-tuning, and RL (Zhang et al., 2026a; Wang et al., 2025; Springer et al., 2025; Chen et al., 2026a; Chen et al., 2025; Watts et al., 2026). TailSFT complements these observations with a practical intervention: rather than only diagnosing whether an intermediate model will support later training, it modifies SFT itself to produce a more useful initialization for the RL stage. Based on our results, we believe there is significant potential in adopting a holistic approach to language modeling. 2 Preliminaries on Coverage We formalize coverage as a property of the SFT initialization that captures how readily reward-bearing responses can be sampled. We then relate coverage to pass@K, which provides an empirical measure of the learning signal available to RL at the start of post-training. Notation and setting Consider a prompt distribution μ over a prompt space X, a response space Y, and a language model π(⋅∣x)∈Δ()π(· x)∈ (Y). We focus on verifiable tasks with binary reward R:×→0,1R:X×Y→\0,1\, where R⁡(x,y)=1R(x,y)=1 indicates that response y is correct for prompt x. In SFT, we observe (xi,yi)(x_i,y_i) with xi∼μx_i μ and yi∼πD(⋅∣xi)y_i _D(· x_i), where πD _D assigns probability to reward-bearing responses. Standard SFT minimizes the sequence-level cross-entropy ℓπ(x,y)=−log⁡π⁡(y∣x) _π(x,y)=- π(y x), and the resulting model πSFT _SFT initializes RL. The coverage profile Consider a single prompt x, and let pπ(x):=Pry∼π(⋅∣x)[R(x,y)=1]p_π(x):= _y π(· x)[R(x,y)=1]. A batch of K independent rollouts contains at least one rewarding response with probability 1−(1−pπ(x))K1-(1-p_π(x))^K. If all K rollouts receive zero reward, no gradient is received for their corresponding prompt. Thus, useful responses must have sufficient probability under the initialization to have a chance of being produced during RL. Across a dataset of many prompts, the model may benefit via positive transfer from easier to harder prompts, but the initial probability of rewarding responses, pπ(x)p_π(x), determines the learning signal available before such transfer. The coverage profile formalizes this property relative to the ground-truth, SFT-data policy (Chen et al., 2026a). We assume that πD _D assigns non-negligible probability to reward-bearing responses, so coverage of πD _D is relevant to downstream RL. definition [Coverage Profile] Given a data-generating policy πD _D and model π, the coverage profile at scale N is CovN(πD∥π):=Prx∼μ,y∼πD(⋅∣x)[πD(y∣x)π⁡(y∣x)≥N].Cov_N( _D\|π):= _x μ,\,y _D(· x) [ _D(y x)π(y x)≥ N ]. The coverage profile measures the probability mass under πD _D that π underweights by a factor of at least N; smaller values indicate better coverage. The scale N determines the sampling budget needed to reach this mass. Chen et al. (2026a) show that, if CovN(πD∥π)≤12Cov_N( _D\|π)≤ 12, then K≥2Nlog⁡(1/ε)K≥ 2N (1/ ) samples suffice for Best-of-K to achieve expected reward within CovN(πD∥π)+εCov_N( _D\|π)+ of πD _D, with a corresponding worst-case necessity result with a comparable relationship between N and K. Thus, coverage at scale N characterizes which useful responses are accessible with a sampling budget on the order of N. Since πD(y∣x) _D(y x) is generally unknown, the coverage profile cannot be measured directly. However, for binary rewards, pass@K(π):=x∼μ[1−(1−pπ(x))K](π):=E_x μ[1-(1-p_π(x))^K] is exactly the expected reward of Best-of-K sampling (Brown et al., 2024). We therefore use pass@K at large K as an empirical measure of coverage. At a K comparable to the RL rollout budget, it also measures how often the policy exposes RL to reward-bearing responses. 3 TailSFT: Supervised Fine-Tuning for Coverage In this section, we develop TailSFT, a sequence-level filtering method designed to direct SFT updates toward improving coverage. The method combines two ideas: stopping updates on responses that are already sufficiently well modeled via filtering, and determining which responses to filter relative to the initial policy. We first introduce the algorithm, then isolate the effect of filtering in a controlled graph-navigation task, and finally analyze why the initial policy provides a useful reference for filtering. 3.1 The TailSFT Algorithm Coverage at scale N depends on whether useful responses receive enough probability to be reached via repeated sampling. Once a response is already well covered at that scale, increasing its likelihood further does not reduce CovNCov_N. Improving coverage instead requires increasing the probability of responses that remain under-modeled. Standard SFT makes no such distinction and continues to increase the likelihood of every training response. TailSFT reduces training on responses that have already improved substantially and concentrates subsequent updates on the remaining tail. Since πD(y∣x) _D(y x) is unknown, we cannot determine directly which responses are already covered. Instead, we compare each response’s current loss to its loss under the initial policy and filter the responses whose losses have decreased the most. Algorithm 1 TailSFT 1: Initial policy π0 _0, SFT data D, filtering schedule γt\ _t\ 2: Record ℓi0←ℓi(π0) _i^0← _i( _0) for every (xi,yi)∈(x_i,y_i) 3: for each training step t do 4: Sample a batch ℬtB_t 5: Compute ℓit←ℓi(πt) _i^t← _i( _t) for each (xi,yi)∈ℬt(x_i,y_i) _t 6: Let ℱt⊆ℬtF_t _t be the γt _t fraction of sequences with the smallest ℓit−ℓi0 _i^t- _i^0 7: Set πt+1 _t+1 by taking a gradient step from πt _t on 8: ℒt(π)=∑i∈ℬt∖ℱt−logπ(yi∣xi)∑i∈ℬt∖ℱt|yi| _t(π)= _i _t _t- π(y_i x_i) _i _t _t|y_i| As described in Algorithm 1, TailSFT redirects training away from responses whose losses have decreased the most and toward responses that remain under-modeled. Filtering is applied at the sequence level. The length-normalized loss is used only to determine which sequences are filtered; optimization uses the standard token-averaged cross-entropy over target tokens in the retained sequences. The filtering fraction γt _t may be fixed or vary over training. Filtering itself does not require using the initial policy as a reference. We therefore consider two alternatives alongside TailSFT: absolute filtering, which stops training on an example once its loss falls below a fixed threshold, and quantile filtering, which filters the lowest-loss fraction of each batch. These alternatives allow us to separate the benefit of filtering itself from the benefit of measuring progress relative to the initial policy π0 _0. 3.2 Warm-Up: Graph Navigation We study whether halting updates on an example once it is fit by the model can improve coverage, even at the cost of the standard SFT objective and pass@1. To do so, this section considers a controlled setting in which the distribution of rewarding responses is known exactly. Following Chen et al. (2026a), each prompt x specifies a layered directed graph with source s, target t, and exactly eight valid s→ts\!→\!t paths. Pretraining pairs each graph with a uniformly sampled valid path, while the SFT data deterministically selects and rewards a single path (see Fig. 2). Thus, pretraining teaches generic graph navigation, while SFT teaches the task-specific path-selection rule. Since the target policy is deterministic, pass@K is exactly the probability that the rewarded path is contained in K samples, yielding a finite-sample estimate of coverage. Full details of the graph construction and data-generating process appear in Appendix C. SFT DistributionSFTTailSFTMajorityMinoritysstt…sstt…sstt…sstt…pass@1=78pass@K=78pass@1= 78 14.72241ptpass@K= 78sstt…sstt…pass@1=12pass@K=1−2−Kpass@1= 12 14.72241ptpass@K=1-2^-K Figure 2: TailSFT trades single-sample accuracy for better coverage under repeated sampling. The SFT distribution contains majority and minority shards with different rewarding paths. Standard SFT over-anchors on the majority shard, achieving high pass@11 but no improvement as K grows. A filtered objective, illustrated here by TailSFT, retains probability on both paths, yielding lower pass@11 but substantially better pass@K scaling. Using a GPT-2-style architecture and a shared pretraining setup, we vary only the SFT objective. We compare standard SFT with absolute filtering, quantile filtering, and TailSFT, which uses offset quantile filtering (as in Algorithm 1). For each filtering variant, we sweep its single filtering hyperparameter and report the setting with the highest final pass@88; complete training details and sweeps appear in Appendix C. The results in Fig. 3 reveal a clear divergence between the standard SFT objective and coverage. Standard SFT achieves the best cross-entropy and pass@11, yet all three filtering methods achieve substantially better pass@88, which we use as a surrogate for coverage. Thus, improving cross-entropy and single-sample accuracy can come at the expense of the probability that a rewarded response appears within a finite rollout budget. Filtering mitigates this failure mode by reducing further updates on responses that are already well modeled. Figure 3: Filtering improves pass@K at the expense of cross entropy and pass@11. Results on the graph-navigation task. Standard SFT achieves the lowest cross-entropy and highest pass@11, while all three filtering methods achieve substantially higher pass@88. See Appendix C for details. The graph task therefore provides clean evidence for the first design choice underlying TailSFT: filtering already-fit examples can improve coverage. It is, however, too simple a setting to determine whether the filtering criterion should depend on the initial policy. In fact, TailSFT is slightly weaker than other filtering variants in this setting, due to homogeneity across prompts. Under the ideal pretrained policy, every prompt has the same possible loss reduction (i.e., log⁡8 8), so the reference loss provides no information for distinguishing which examples have more room to improve. Imperfections in pretraining therefore introduce noise into the offset score without providing useful signal. We discuss this effect further in Appendix C. 3.3 Theoretical Analysis We next turn to a setting where the initial policy carries information about the response distribution, and ask whether filtering rules can preserve it. To study this question, we turn to theoretical analysis. Let S denote the set of rewarding responses, and suppose the target policy is obtained by discarding all responses outside S and renormalizing the initial policy: π⋆(y)∝π0(y)y∈S.π (y) _0(y)1\y∈ S\. Thus, among rewarding responses, the target preserves the relative preferences already present in π0 _0. The initial model may place too much total probability on unrewarding responses, but it still contains information about how probability should be distributed among the rewarding ones. We compare two ways to decide when an example should stop contributing to the SFT objective. Let ℓπ(x,y)=−log⁡π⁡(y∣x) _π(x,y)=- π(y x) denote the sequence-level SFT loss. With an absolute filtering constant α, we use ℓabs(π,x,y)=[ℓπ(x,y)+log⁡(α)]+. _abs(π;x,y)= [ _π(x,y)+ (α) ]_+. An example stops contributing once π⁡(y∣x)≥απ(y x)≥α. Because the same threshold is applied to every response, this criterion ignores how likely the response was under the initial policy. TailSFT instead measures progress relative to the initial policy π0 _0. The corresponding offset loss is ℓoff(π,x,y)=[ℓπ(x,y)−ℓπ0(x,y)+log⁡β]+, _off(π;x,y)= [ _π(x,y)- _ _0(x,y)+ β ]_+, which stops updating a response once π⁡(y∣x)≥βπ0(y∣x).π(y x)≥β _0(y x). The stopping point therefore adapts to the probability that π0 _0 already assigns to each response rather than imposing a common final threshold. theorem [Informal] Let πERM _ERM, πABS,α _ABS,α, and πOFF,β _OFF,β denote the policies obtained using standard SFT, absolute filtering, and offset filtering, respectively. For any initial policy, rewarding set, and SFT sample, offset filtering can be tuned to achieve coverage no worse than standard SFT or the best absolute threshold: infβCovN(π⋆∥πOFF,β)≤minCovN(π⋆∥πERM),infαCovN(π⋆∥πABS,α). _βCov_N(π \| _OFF,β)≤ \Cov_N(π \| _ERM), _αCov_N(π \| _ABS,α) \. The inequality can be strict, and absolute filtering can be worse than standard SFT. The theorem formalizes the role of the initial policy as more than merely an initialization. In this setting, π0 _0 contains useful information about how probability should be distributed among rewarding responses. Standard SFT can overwrite this structure by fitting the empirical frequencies of a finite SFT sample, while absolute filtering pushes observed responses toward a common probability threshold regardless of where they started. Offset filtering instead uses each response’s initial probability to determine when further training is unnecessary. By preserving more of the useful structure already present in π0 _0, it can achieve strictly better coverage. The full statement and proof appear in Appendix B. Connections to prior work TailSFT is most closely related to direct coverage optimization and loss-based data filtering. Chen et al. (2025) introduce the direct coverage optimization loss ℓDCO(π,x,y)=−log⁡(1−(1−π⁡(y∣x))K), _DCO(π;x,y)=- (1- (1-π(y x) )^K ), which directly optimizes pass@K when y is the final solution. For large K, its gradient becomes small once π⁡(y∣x)π(y x) is sufficiently large, making it a soft analogue for filtering without reference to the initial policy. Section 3.3 shows that, in our stylized setting, offset filtering can achieve coverage no worse than the best absolute threshold and can be strictly better. TailSFT is also related to RHO-LOSS and subsequent variants, though these methods were designed and tested primarily to improve pretraining efficiency (Mindermann et al., 2022; Thirukovalluru et al., 2024; Lin et al., 2024; Zhao et al., 2026b). These approaches prioritize examples or tokens according to their potential loss reduction. TailSFT, by contrast, is designed to improve coverage for a subsequent RL stage and applies filtering at the sequence level. See Appendix A for further discussion. 4 Language Model Experiments The results in Section 3 suggest that TailSFT can preserve reward-bearing responses more effectively than standard SFT. We now evaluate whether this behavior carries over to pretrained language models. We first compare standard SFT and TailSFT on math and code tasks. The variation across these results then allows us to study when preserving information from the base model is most useful. Finally, we follow matched SFT checkpoints through GRPO to test whether higher coverage before RL translates into stronger post-RL performance. Throughout, we use pass@1616 as an empirical measure of coverage. 4.1 SFT Results pass@1 pass@16 SFT data Benchmark Standard TailSFT Δ Standard TailSFT Δ OMI AIME 2022–2025 (n=120n=120) 3.65±0.373.65_\,± 0.37 4.03±0.024.03_\,± 0.02 +0.37+0.37 15.24±2.0215.24_\,± 2.02 18.31±0.5718.31_\,± 0.57 +3.07+3.07 MATH-500 Level 5 (n=134n=134) 24.80±0.4024.80_\,± 0.40 25.16±0.4025.16_\,± 0.40 +0.36+0.36 66.42±0.0066.42_\,± 0.00 69.15±0.8669.15_\,± 0.86 +2.74+2.74 OMEGA-500 (n=500n=500) 6.58±0.506.58_\,± 0.50 6.51±0.266.51_\,± 0.26 −0.06-0.06 32.80±2.2532.80_\,± 2.25 32.60±1.5632.60_\,± 1.56 −0.20-0.20 BigCode MBPP+ (n=378n=378) 53.04±0.9353.04_\,± 0.93 51.36±0.1451.36_\,± 0.14 −1.68-1.68 74.69±1.1074.69_\,± 1.10 78.84±1.0678.84_\,± 1.06 +4.14+4.14 HumanEval+ (n=164n=164) 45.06±0.5745.06_\,± 0.57 46.33±1.0746.33_\,± 1.07 +1.27+1.27 76.63±0.7076.63_\,± 0.70 80.08±0.9380.08_\,± 0.93 +3.46+3.46 CruxEval-I (n=800n=800) 29.79±0.4829.79_\,± 0.48 30.43±0.1530.43_\,± 0.15 +0.64+0.64 61.71±2.2761.71_\,± 2.27 65.79±0.2665.79_\,± 0.26 +4.08+4.08 CruxEval-O (n=800n=800) 4.16±0.444.16_\,± 0.44 13.18±0.3013.18_\,± 0.30 +9.02+9.02 24.21±2.3824.21_\,± 2.38 41.00±1.3541.00_\,± 1.35 +16.79+16.79 LiveCodeBench (n=612n=612) 10.74±0.4610.74_\,± 0.46 11.49±0.3811.49_\,± 0.38 +0.75+0.75 30.39±0.8630.39_\,± 0.86 33.12±0.3433.12_\,± 0.34 +2.72+2.72 Magicoder MBPP+ (n=378n=378) 55.35±0.4755.35_\,± 0.47 54.60±0.5554.60_\,± 0.55 −0.75-0.75 75.31±0.9375.31_\,± 0.93 78.66±1.2578.66_\,± 1.25 +3.35+3.35 HumanEval+ (n=164n=164) 48.49±0.5848.49_\,± 0.58 47.69±0.1947.69_\,± 0.19 −0.80-0.80 77.85±0.7077.85_\,± 0.70 78.66±1.8378.66_\,± 1.83 +0.81+0.81 CruxEval-I (n=800n=800) 28.50±0.1328.50_\,± 0.13 26.48±0.4526.48_\,± 0.45 −2.02-2.02 59.75±0.6259.75_\,± 0.62 68.08±0.9468.08_\,± 0.94 +8.33+8.33 CruxEval-O (n=800n=800) 12.86±0.1912.86_\,± 0.19 16.16±0.5316.16_\,± 0.53 +3.30+3.30 38.08±0.9438.08_\,± 0.94 47.92±1.6647.92_\,± 1.66 +9.83+9.83 LiveCodeBench (n=612n=612) 14.84±0.1114.84_\,± 0.11 14.32±0.3814.32_\,± 0.38 −0.51-0.51 31.81±0.3431.81_\,± 0.34 33.22±0.3433.22_\,± 0.34 +1.42+1.42 OCI MBPP+ (n=378n=378) 58.26±0.8858.26_\,± 0.88 55.81±0.0655.81_\,± 0.06 −2.44-2.44 81.22±0.7581.22_\,± 0.75 82.36±0.8182.36_\,± 0.81 +1.15+1.15 HumanEval+ (n=164n=164) 54.76±1.1654.76_\,± 1.16 51.94±1.8351.94_\,± 1.83 −2.82-2.82 85.98±1.6185.98_\,± 1.61 83.23±2.1683.23_\,± 2.16 −2.74-2.74 CruxEval-I (n=800n=800) 29.46±0.4629.46_\,± 0.46 30.17±0.6430.17_\,± 0.64 +0.71+0.71 68.08±1.7768.08_\,± 1.77 71.58±1.9171.58_\,± 1.91 +3.50+3.50 CruxEval-O (n=800n=800) 9.30±0.499.30_\,± 0.49 10.84±0.3710.84_\,± 0.37 +1.54+1.54 40.12±0.0040.12_\,± 0.00 44.46±0.4744.46_\,± 0.47 +4.33+4.33 LiveCodeBench (n=612n=612) 12.24±0.3612.24_\,± 0.36 11.49±0.8411.49_\,± 0.84 −0.75-0.75 34.59±0.9034.59_\,± 0.90 33.50±0.1633.50_\,± 0.16 −1.09-1.09 Table 1: TailSFT improves coverage over standard SFT. Across 18 model–benchmark pairs, TailSFT often improves pass@1616, while changes in pass@11 are mixed. Values are percentages reported as mean ± standard deviation over three seeds; Δ denotes TailSFT minus Standard SFT in percentage points, shown in green when positive, red when negative, and yellow when within one standard deviation. We later establish a sufficient condition (Section 4.2) that accurately predicts the failure modes of TailSFT exhibited in this table (Figure 4). MATH-500 Level 5 refers to the subset of the benchmark with the highest difficulty. Setup We fine-tune the OLMo-3 7B base model (45) separately on domain-specific math and code instruction data, giving 18 dataset–benchmark pairs. For math, we train on a 350350k-example subset of OpenMathInstruct-2 (OMI, Toshniwal et al. (2024)), decontaminated against MATH-500. We evaluate with the OLMES protocol (Gu et al., 2025) on the 2022–2025 AIME problems, the hardest level of MATH-500, and OMEGA-500. For code, we train separately on Magicoder (Wei et al., 2024b), BigCode Self-OSS-Instruct (Wei et al., 2024a), and OpenCodeInstruct (OCI, Ahmad et al. (2025)). We evaluate on MBPP+, HumanEval+, CruxEval-I/O, and LiveCodeBench. We focus on benchmarks where performance is not saturated. We report pass@1616 using the task-specific OLMES sampling protocol and average results over three seed sets. Results are reported in Table 1 and Section D.1 contains experimental details. TailSFT improves coverage in most settings Across the 18 dataset–benchmark pairs in Table 1, TailSFT improves pass@1616 in 15 cases, while changes in pass@11 are mixed. OMEGA-500 is essentially unchanged, with an absolute difference of −0.20%-0.20\%. The largest gain occurs on CruxEval-O after SFT on BigCode, where pass@1616 increases by an absolute 16.79%16.79\%. With Magicoder, pass@1616 increases by 8.33%8.33\% on CruxEval-I and 9.83%9.83\% on CruxEval-O. On AIME, it increases by 3.07%3.07\%. On MBPP+, absolute gains are 4.14%4.14\%, 3.35%3.35\%, and 1.15%1.15\% for BigCode, Magicoder, and OCI. Overall, improvements are substantially more consistent at large K than at K=1K=1, in line with the coverage motivation for TailSFT. 4.2 Coverage Ratio Diagnostic Figure 4: When standard SFT loses more coverage than it gains, TailSFT preserves or improves coverage. The vertical line marks ρ16=1 _16=1, where the estimated coverage lost and gained by standard SFT are equal. Of the 11 evaluated settings with ρ16>1 _16>1, 10 have positive coverage gains from TailSFT and one is essentially unchanged. Several settings with ρ16<1 _16<1 also improve. Coverage gain and ratio are computed on the base-reachable set (Section 4.2). The variation across these settings motivates a closer look at when TailSFT should perform well. In the graph-navigation task, standard SFT increased probability on the majority shard while reducing coverage of a rewarded path that was already represented by the initial policy. Based on this, we ask whether SFT in the language model setting exhibits a measurable loss of coverage relative to the base model, and whether this coverage loss is predictive of the benefit of TailSFT. We measure how standard SFT changes coverage on problems that are already reachable from the base model. Restricting attention to problems that are neither effectively unsolved nor already saturated under the base model, we use pass@11 to estimate the pass@1616 that would result from independent sampling. We then aggregate the decreases and increases induced by standard SFT—their ratio measures whether standard SFT has lost more base-reachable coverage than it has gained. definition [Coverage ratio] For problem i, model π, and K∈1,16K∈\1,16\, let Pi,K(π)P_i,K(π) denote the empirical pass@K, averaged over seeds. We convert pass@11 to an estimated pass@1616 value using f16(p):=1−(1−p)16f_16(p):=1-(1-p)^16. The base-reachable set is ℛ0:=i:0.05<Pi,16(π0)<0.95R_0:=\i:0.05<P_i,16( _0)<0.95\. The coverage lost and gained by standard SFT on this set are L L :=∑i∈ℛ0[f16(Pi,1(π0))−f16(Pi,1(πSFT))]+, := _i _0 [f_16 (P_i,1( _0) )-f_16 (P_i,1( _SFT) ) ]_+, G G :=∑i∈ℛ0[f16(Pi,1(πSFT))−f16(Pi,1(π0))]+. := _i _0 [f_16 (P_i,1( _SFT) )-f_16 (P_i,1( _0) ) ]_+. The coverage ratio is ρ16:=L/G _16:=L/G. If G=0G=0, we set ρ16:=∞ _16:=∞. Computing ρ16 _16 requires only the base model and a single standard SFT run. A value ρ16>1 _16>1 means that, on the base-reachable set, the estimated coverage lost during standard SFT exceeds the estimated coverage gained. This is the setting most directly targeted by TailSFT, which reduces training on responses that have already improved and thereby limits probability shifting away from responses represented by the base model. The coverage ratio identifies a regime where TailSFT consistently preserves or improves coverage In Figure 4, all 11 dataset–benchmark pairs satisfying ρ16>1 _16>1 have nonnegative coverage gains (improvement in pass@1616 on the base-reachable set ℛ0R_0) from TailSFT. Ten improve and one is essentially unchanged, with gains reaching an absolute 28.69%28.69\% for BigCode evaluated on CruxEval-O. The condition is not necessary for improvement: several settings with ρ16<1 _16<1 also benefit from TailSFT. Thus, the coverage ratio provides a conservative way to identify settings in which TailSFT preserves or improves coverage. Section D.3 contains further details. 4.3 GRPO Results The coverage principle predicts that a higher-coverage initialization should expose RL to rewarding responses more often (Chen et al., 2026a). This section therefore tests whether the additional coverage resulting from TailSFT translates into better post-RL performance. Setup We initialize GRPO from the standard SFT and TailSFT checkpoints evaluated in Table 1. For math reasoning, we train on the MATH training split with MATH-500 held out and evaluate on MATH-500 Level 5 and AIME. For code, we train on the MBPP+ training split and evaluate on its test split. The GRPO procedure and hyperparameters are held fixed within each comparison. The main comparisons use 44 rollouts per prompt and an actor learning rate of 2×10−52× 10^-5. Evaluation follows the OLMES protocol over three seed sets. Improved coverage yields better post-RL performance Initializing from TailSFT improves post-RL pass@11 in every matched comparison in Table 2, with absolute improvements ranging between 1.21%1.21\% and 3.93%3.93\%. Our coding experiments make the role of initialization especially salient—each TailSFT checkpoint has lower pass@11 than its standard SFT counterpart before RL, but has higher pass@11 afterward. Broader initial coverage indeed results in higher single-sample accuracy after GRPO. These results are consistent with the relationship between large-K performance and post-RL outcomes observed by Kang et al. (2026). pass@1 pass@16 SFT data Benchmark Standard TailSFT Δ Standard TailSFT Δ GRPO on MATH OMI MATH-500 Level 5 (n=134n=134) 57.70±0.4457.70_\,± 0.44 60.26±1.0260.26_\,± 1.02 +2.56+2.56 83.83±1.7283.83_\,± 1.72 87.06±0.8687.06_\,± 0.86 +3.23+3.23 AIME 2022–2025 (n=120n=120) 14.40±0.2614.40_\,± 0.26 15.61±0.4415.61_\,± 0.44 +1.21+1.21 32.76±0.8832.76_\,± 0.88 36.06±0.9836.06_\,± 0.98 +3.30+3.30 GRPO on MBPP+ BigCode MBPP+ (n=378n=378) 69.57±0.2969.57_\,± 0.29 73.50±0.7073.50_\,± 0.70 +3.93+3.93 75.93±0.0075.93_\,± 0.00 78.66±0.1578.66_\,± 0.15 +2.73+2.73 Magicoder MBPP+ (n=378n=378) 70.52±0.2570.52_\,± 0.25 73.24±0.0973.24_\,± 0.09 +2.72+2.72 78.22±0.6178.22_\,± 0.61 80.60±0.5580.60_\,± 0.55 +2.38+2.38 OCI MBPP+ (n=378n=378) 74.67±0.0874.67_\,± 0.08 76.30±0.0576.30_\,± 0.05 +1.62+1.62 84.22±0.4084.22_\,± 0.40 84.04±0.1584.04_\,± 0.15 −0.18-0.18 Table 2: Initializing from the TailSFT model improves post-GRPO performance across math and code. GRPO is initialized from the Standard SFT and TailSFT checkpoints evaluated in Table 1, and rows are grouped by the corpus used for GRPO training: the MATH train split, excluding MATH-500, for the math rows, and the MBPP+ train split for the code rows. The SFT data column indicates the corpus used to train the initialization for each GRPO run. Values are percentages reported as mean ± standard deviation over three seeds, and Δ is improvement from TailSFT. Experiment details are in Section D.2. Improved coverage drives faster learning Figure 8 shows that the difference between the two initializations appears early in training, with TailSFT runs beginning with lower reward but the gap quickly closing. In some settings, early reward increases up to 2.5×2.5× faster than for the corresponding standard SFT run. After RL, TailSFT retains higher pass@1616 in four of the five matched comparisons, and is approximately tied in the fifth. The coverage preserved during SFT is therefore available to RL early in training and ultimately translates into higher pass@11. 5 Discussion We present TailSFT as a drop-in replacement for standard SFT that filters already-fit sequences to improve coverage after the SFT stage, which produces improvements in post-RL performance. TailSFT is derived using a combination of theoretical analysis and controlled empirics, and significantly outperforms standard SFT in our language modeling experiments. More broadly, TailSFT uses the coverage principle to target the interaction between the SFT and RL stages of language model training (Chen et al., 2026a). We believe our results motivate a shift in how language models should be optimized, toward targeting the full training trajectory rather than fitting each stage independently. References Ahmad et al. (2025) W. U. Ahmad, A. Ficek, M. Samadi, J. Huang, V. Noroozi, S. Majumdar, and B. Ginsburg OpenCodeInstruct: a large-scale instruction tuning dataset for code llms. Note: arXiv:2504.04030 Cited by: §4.1. Bansal et al. (2026) R. Bansal, C. Mohri, T. Qin, D. Alvarez-Melis, and S. M. Kakade RL excursions during pre-training: how early is too early for on-policy learning?. In Workshop on Scaling Post-Training for LLMs, Cited by: Appendix A. Brandfonbrener et al. (2024) D. Brandfonbrener, H. Zhang, A. Kirsch, J. R. Schwarz, and S. Kakade CoLoR-Filter: conditional loss reduction filtering for targeted language model pre-training. In Advances in Neural Information Processing Systems, Cited by: Appendix A. Brown et al. (2024) B. Brown, J. Juravsky, R. Ehrlich, R. Clark, Q. V. Le, C. Ré, and A. Mirhoseini Large language monkeys: scaling inference compute with repeated sampling. Note: arXiv:2407.21787 Cited by: Appendix A, §2. Chen et al. (2026a) F. Chen, A. Huang, N. Golowich, S. Malladi, A. Block, J. Ash, A. Krishnamurthy, and D. Foster The coverage principle: how pre-training enables post-training. In International Conference on Learning Representations, Cited by: Appendix A, §C.1, §C.2, Appendix C, §1, §1, §1, §2, §2, §3.2, §4.3, §5. Chen et al. (2025) F. Chen, A. Raventós, N. Cheng, S. Ganguli, and S. Druckmann Rethinking fine-tuning when scaling test-time compute: limiting confidence improves mathematical reasoning. In Advances in Neural Information Processing Systems, Cited by: Appendix A, §1, §1, §1, §3.3. Chen and Jiang (2019) J. Chen and N. Jiang Information-theoretic considerations in batch reinforcement learning. In International Conference on Machine Learning, Cited by: Appendix A. Chen et al. (2026b) Y. Chen, Y. Liu, and F. Meng SED-SFT: selectively encouraging diversity in supervised fine-tuning. In Annual Meeting of the Association for Computational Linguistics, Cited by: Appendix A. Cheng et al. (2026) Z. Cheng, Y. Xie, Y. Qu, A. Setlur, S. Hao, V. Pimpalkhute, T. Liang, F. Yao, Z. Liu, E. P. Xing, V. Smith, R. Salakhutdinov, Z. Hu, T. W. Killian, and A. Kumar IsoCompute playbook: optimally scaling sampling compute for LLM RL. In International Conference on Machine Learning, Cited by: Appendix A. Chow et al. (2025) Y. Chow, G. Tennenholtz, I. Gur, V. Zhuang, B. Dai, A. Kumar, R. Agarwal, S. Thiagarajan, C. Boutilier, and A. Faust Inference-aware fine-tuning for best-of-n sampling in large language models. In International Conference on Learning Representations, Cited by: Appendix A. Chu et al. (2025) T. Chu, Y. Zhai, J. Yang, S. Tong, S. Xie, D. Schuurmans, Q. V. Le, S. Levine, and Y. Ma SFT memorizes, RL generalizes: a comparative study of foundation model post-training. In International Conference on Machine Learning, Cited by: Appendix A. Fan et al. (2026) Z. Fan, G. Chen, Y. Huang, M. Zhao, D. Guo, and Y. R. Fung Learning diverse responses with prefix-conditioned supervised fine-tuning. In Annual Meeting of the Association for Computational Linguistics, Cited by: Appendix A. Farahmand et al. (2010) A. Farahmand, C. Szepesvári, and R. Munos Error propagation for approximate policy and value iteration. In Advances in Neural Information Processing Systems, Cited by: Appendix A. Foster et al. (2022) D. J. Foster, A. Krishnamurthy, D. Simchi-Levi, and Y. Xu Offline reinforcement learning: fundamental barriers for value function approximation. In Conference on Learning Theory, Cited by: Appendix A. Foster et al. (2025) D. J. Foster, Z. Mhammedi, and D. Rohatgi Is a good foundation necessary for efficient reinforcement learning? the computational role of the base model in exploration. In Conference on Learning Theory, Cited by: Appendix A. Fu et al. (2026) Y. Fu, T. Chen, J. Chai, X. Wang, S. Tu, G. Yin, W. Lin, Q. Zhang, Y. Zhu, and D. Zhao SRFT: a single-stage method with supervised and reinforcement fine-tuning for reasoning. In International Conference on Learning Representations, Cited by: Appendix A. Gu et al. (2025) Y. Gu, O. Tafjord, B. Kuehl, D. Haddad, J. Dodge, and H. Hajishirzi OLMES: a standard for language model evaluations. In Findings of the Association for Computational Linguistics, Cited by: §D.1.1, §D.1.2, §D.2, Table 16, Table 24, §4.1. Guo et al. (2025) D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, Q. Zhu, R. Xu, R. Zhang, S. Ma, X. Bi, et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature. Cited by: §1. He et al. (2025) A. W. He, D. Fried, and S. Welleck Rewarding the unlikely: lifting GRPO beyond distribution sharpening. In Conference on Empirical Methods in Natural Language Processing, Cited by: Appendix A. Huang et al. (2025a) A. Huang, A. Block, D. Foster, D. Rohatgi, C. Zhang, M. Simchowitz, J. Ash, and A. Krishnamurthy Self-improvement in language models: the sharpening mechanism. In International Conference on Learning Representations, Cited by: Appendix A. Huang et al. (2025b) A. Huang, A. Block, Q. Liu, N. Jiang, A. Krishnamurthy, and D. J. Foster Is best-of-n the best of them? coverage, scaling, and optimality in inference-time alignment. In International Conference on Machine Learning, Cited by: Appendix A. Huang et al. (2026) Z. Huang, T. Cheng, Z. Qiu, Z. Wang, Y. Xu, E. M. Ponti, and I. Titov Blending supervised and reinforcement fine-tuning with prefix sampling. In International Conference on Machine Learning, Cited by: Appendix A. Jiang and Xie (2025) N. Jiang and T. Xie Offline reinforcement learning in large state spaces: algorithms and guarantees. Statistical Science. Cited by: Appendix A. Jin et al. (2025) H. Jin, S. Luan, S. Lyu, G. Rabusseau, D. Precup, and M. Hamdaqa RL fine-tuning heals the OOD forgetting in SFT. In Workshop on Foundations of Reasoning in Language Models, Cited by: Appendix A. Jin et al. (2021) Y. Jin, Z. Yang, and Z. Wang Is pessimism provably efficient for offline RL?. In International Conference on Machine Learning, Cited by: Appendix A. Kang et al. (2026) F. Kang, M. Kuchnik, K. Padthe, M. Vlastelica, R. Jia, C. Wu, and N. Ardalani Quagmires in SFT-RL post-training: when high SFT scores mislead and what to use instead. In International Conference on Learning Representations, Cited by: Appendix A, §4.3. Karan and Du (2026) A. Karan and Y. Du Reasoning with sampling: your base model is smarter than you think. In International Conference on Learning Representations, Cited by: Appendix A. Katharopoulos and Fleuret (2018) A. Katharopoulos and F. Fleuret Not all samples are created equal: deep learning with importance sampling. In International Conference on Machine Learning, Cited by: Appendix A. Li et al. (2025) Z. Li, C. Chen, T. Xu, Z. Qin, J. Xiao, Z. Luo, and R. Sun Preserving diversity in supervised fine-tuning of large language models. In International Conference on Learning Representations, Cited by: Appendix A. Lin et al. (2024) Z. Lin, Z. Gou, Y. Gong, X. Liu, Y. Shen, R. Xu, C. Lin, Y. Yang, J. Jiao, N. Duan, and W. Chen Not all tokens are what you need for pretraining. In Advances in Neural Information Processing Systems, Cited by: Appendix A, §3.3. Liu et al. (2023) H. Liu, S. M. Xie, Z. Li, and T. Ma Same pre-training loss, better downstream: implicit bias matters for language models. In International Conference on Machine Learning, Cited by: Appendix A. Liu et al. (2025) M. Liu, S. Diao, X. Lu, J. Hu, X. Dong, Y. Choi, J. Kautz, and Y. Dong ProRL: prolonged reinforcement learning expands reasoning boundaries in large language models. In Advances in Neural Information Processing Systems, Cited by: Appendix A. Lourie et al. (2025) N. Lourie, M. Y. Hu, and K. Cho Scaling laws are unreliable for downstream tasks: a reality check. In Findings of the Association for Computational Linguistics, Cited by: Appendix A. Microsoft AI Team (2025) Microsoft AI Team Mai-thinking-1: building a hill-climbing machine. Technical report Technical report, Microsoft AI, 2026. https://microsoft. ai/pdf/mai-thinking …. Cited by: §1. Mindermann et al. (2022) S. Mindermann, J. M. Brauner, M. T. Razzak, M. Sharma, A. Kirsch, W. Xu, B. Höltgen, A. N. Gomez, A. Morisot, S. Farquhar, et al. Prioritized training on points that are learnable, worth learning, and not yet learnt. In International Conference on Machine Learning, Cited by: Appendix A, §3.3. Niu et al. (2026) X. Niu, B. Bai, W. Han, and W. Zhang On the non-decoupling of supervised fine-tuning and reinforcement learning in post-training. Note: arXiv:2601.07389 Cited by: Appendix A. Pang et al. (2025) J. Pang, N. Di, Z. Zhu, J. Wei, H. Cheng, C. Qian, and Y. Liu Token cleaning: fine-grained data selection for LLM supervised fine-tuning. In International Conference on Machine Learning, Cited by: Appendix A. Qin and Springenberg (2025) C. Qin and J. T. Springenberg Supervised fine tuning on curated data is reinforcement learning (and can be improved). Note: arXiv:2507.12856 Cited by: Appendix A. Qin et al. (2024) Z. Qin, K. Wang, Z. Zheng, J. Gu, X. Peng, Z. Xu, D. Zhou, L. Shang, B. Sun, X. Xie, and Y. You InfoBatch: lossless training speed up by unbiased dynamic data pruning. In International Conference on Learning Representations, Cited by: Appendix A. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo DeepSeekMath: pushing the limits of mathematical reasoning in open language models. Note: arXiv:2402.03300 Cited by: §D.2, §1. Snell et al. (2025) C. Snell, J. Lee, K. Xu, and A. Kumar Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In International Conference on Learning Representations, Cited by: Appendix A. Springer et al. (2025) J. M. Springer, S. Goyal, K. Wen, T. Kumar, X. Yue, S. Malladi, G. Neubig, and A. Raghunathan Overtrained language models are harder to fine-tune. In International Conference on Machine Learning, Cited by: Appendix A, §1, §1. Swayamdipta et al. (2020) S. Swayamdipta, R. Schwartz, N. Lourie, Y. Wang, H. Hajishirzi, N. A. Smith, and Y. Choi Dataset cartography: mapping and diagnosing datasets with training dynamics. In Conference on Empirical Methods in Natural Language Processing, Cited by: Appendix A. Team et al. (2026) K. Team, T. Bai, Y. Bai, Y. Bao, M. C., J. Cai, X. Cai, P. Cao, Y. Cao, Z. Chai, Y. Charles, H. S. Che, G. Chen, G. Chen, G. Chen, H. Chen, J. Chen, J. Chen, J. Chen, K. Chen, P. Chen, R. Chen, W. Chen, X. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Y. Chen, Z. Chen, D. Cheng, Y. Cheng, J. Cui, J. Cui, A. Dai, J. Deng, H. Ding, R. Ding, S. Ding, M. Dong, M. Dong, Y. Dong, Y. Dong, A. Du, C. Du, D. Du, J. Du, Y. Du, Y. Fan, J. Feng, Q. Feng, Y. Feng, K. Fu, Q. Fu, F. Gao, H. Gao, J. Gao, T. Gao, W. Gao, S. Geng, J. Gong, L. Gong, S. Gong, X. Gong, Q. Gu, Y. Gu, S. Guan, H. Guo, S. Guo, X. Guo, Z. Guo, B. Hao, W. Hao, X. Hao, D. He, H. He, L. He, Q. He, W. He, X. He, X. He, Y. He, Y. He, C. Hong, T. Hong, H. Hu, J. Hu, R. Hu, W. Hu, Y. Hu, Z. Hu, L. Hua, J. Huang, K. Huang, R. Huang, S. Huang, W. Huang, Y. Huang, Z. Huang, Z. Huang, Y. Hui, C. Jia, Y. Jiang, Z. Jiang, Z. Jiang, W. Jin, X. Jin, Y. Jing, H. Kong, G. Lai, A. Li, C. Li, C. Li, C. Li, F. Li, G. Li, H. Li, J. Li, J. Li, L. Li, L. Li, L. Li, W. Li, W. Li, X. Li, Y. Li, Y. Li, Y. Li, Y. Li, Z. Li, Z. Li, Z. Li, Z. Li, Z. Li, J. Lin, X. Lin, Y. Lin, Z. Lin, Z. Lin, B. Liu, B. Liu, C. Liu, L. Liu, S. Liu, S. Liu, S. Liu, T. Liu, W. Liu, Y. Liu, Y. Liu, Y. Liu, Y. Liu, Z. Liu, Z. Liu, E. Lu, H. Lu, L. Lu, T. Lu, Z. Lu, A. Luo, G. Luo, J. Luo, Y. Luo, B. Lyu, W. Lyu, S. Mao, Y. Mei, X. Men, M. Ni, Y. Niu, S. Pan, S. Peng, Z. Qi, R. Qin, Z. Qin, Z. Qin, H. Qiu, J. Qiu, J. Qiu, B. Qu, Y. Qu, Z. Shang, Y. Shao, H. Shen, J. Shi, J. Shi, L. Shi, S. Shi, W. Siu, P. Song, X. Song, J. Su, Y. Su, Z. Su, L. Sui, J. Sun, J. Sun, S. Sun, S. Sun, T. Sun, Y. Sun, Y. Tai, C. Tang, H. Tang, S. Tang, Z. Tang, C. Tian, R. Tian, Y. Tian, W. Tu, C. Wang, C. Wang, C. Wang, D. Wang, F. Wang, H. Wang, H. Wang, H. Wang, H. Wang, H. Wang, H. Wang, J. Wang, J. Wang, J. Wang, J. Wang, L. Wang, S. Wang, S. Wang, S. Wang, S. Wang, S. Wang, T. Wang, W. Wang, X. Wang, X. Wang, X. Wang, X. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Y. Wang, Z. Wang, Z. Wang, Z. Wang, Z. Wang, Z. Wang, Z. Wang, C. Wei, M. Wei, S. Wei, Z. Wen, F. Wu, H. Wu, R. Wu, W. Wu, X. Wu, Y. Wu, Y. Wu, Y. Wu, Z. Wu, X. Xian, C. Xiang, Y. Xiang, B. Xiao, C. Xiao, X. Xiao, J. Xie, X. Xie, Y. Xie, Z. Xie, B. Xing, Y. Xiong, B. Xu, B. Xu, J. Xu, J. Xu, J. Xu, J. Xu, L. H. Xu, Q. Xu, S. Xu, S. Xu, T. Xu, T. Xu, W. Xu, X. Xu, Y. Xu, Y. Xu, Y. Xu, Z. Xu, H. Xue, J. Yan, Y. Yan, F. Yang, G. Yang, H. Yang, J. Yang, R. Yang, W. Yang, X. Yang, X. Yang, Y. Yang, Y. Yang, Y. Yang, Y. Yang, Z. Yang, Z. Yang, Z. Yang, Z. Yang, H. Yao, D. Ye, H. Ye, W. Ye, Z. Ye, B. Yin, H. Yin, X. Yin, C. Yu, H. Yu, L. Yu, S. Yu, S. Yu, T. Yu, E. Yuan, M. Yuan, T. Yue, W. Yue, Y. Yue, D. Zha, H. Zhan, B. H. Zhang, D. Zhang, F. Zhang, H. Zhang, H. Zhang, H. Zhang, J. Zhang, J. Zhang, J. Zhang, K. Zhang, M. Zhang, P. Zhang, Q. Zhang, R. Zhang, R. Zhang, S. Zhang, S. Zhang, X. Zhang, X. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Y. Zhang, Z. Zhang, Z. Zhang, B. Zhao, C. Zhao, F. Zhao, J. Zhao, J. Zhao, S. Zhao, W. Zhao, X. Zhao, X. Zhao, Y. Zhao, Z. Zhao, H. Zheng, H. Zheng, R. Zheng, S. Zheng, T. Zheng, H. Zhong, L. Zhong, L. Zhong, M. Zhou, Q. Zhou, R. Zhou, R. Zhou, X. Zhou, Y. Zhou, Z. Zhou, J. Zhu, L. Zhu, X. Zhu, Y. Zhu, Y. Zhu, Z. Zhu, C. Zhuang, W. Zhuang, and X. Zu Kimi k3: open frontier intelligence. arXiv preprint arXiv:2607.24653. Cited by: §1. Team Olmo et al. (2026) Team Olmo, Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, David Graham, David Heineman, Dirk Groeneveld, Faeze Brahman, Finbarr Timbers, Hamish Ivison, Jacob Morrison, Jake Poznanski, Kyle Lo, Luca Soldaini, Matt Jordan, Mayee Chen, Michael Noukhovitch, Nathan Lambert, Pete Walsh, Pradeep Dasigi, Robert Berry, Saumya Malik, Saurabh Shah, Scott Geng, Shane Arora, Shashank Gupta, Taira Anderson, Teng Xiao, Tyler Murray, Tyler Romero, Victoria Graf, Akari Asai, Akshita Bhagia, Alexander Wettig, Alisa Liu, Aman Rangapur, Chloe Anastasiades, Costa Huang, Dustin Schwenk, Harsh Trivedi, Ian Magnusson, Jaron Lochner, Jiacheng Liu, Lester James V. Miranda, Maarten Sap, Malia Morgan, Michael Schmitz, Michal Guerquin, Michael Wilson, Regan Huff, Ronan Le Bras, Rui Xin, Rulin Shao, Sam Skjonsberg, Shannon Zejiang Shen, Shuyue Stella Li, Tucker Wilde, Valentina Pyatkin, Will Merrill, Yapei Chang, Yuling Gu, Zhiyuan Zeng, Ashish Sabharwal, Luke Zettlemoyer, Pang Wei Koh, Ali Farhadi, Noah A. Smith, and Hannaneh Hajishirzi Olmo 3. Note: arXiv:2512.13961 Cited by: §4.1. Thirukovalluru et al. (2024) R. Thirukovalluru, N. Monath, B. Dhingra, and S. Wiseman Sequence reducible holdout loss for language model pretraining. In Joint International Conference on Computational Linguistics, Language Resources and Evaluation, Cited by: §3.3. Toshniwal et al. (2024) S. Toshniwal, W. Du, I. Moshkov, B. Kisacanin, A. Ayrapetyan, and I. Gitman OpenMathInstruct-2: accelerating AI for math with massive open-source instruction data. In Workshop on Mathematical Reasoning and AI, Cited by: §4.1. Tuyls et al. (2026) J. Tuyls, D. Foster, A. Krishnamurthy, and J. Ash Representation-based exploration for language models: from test-time to post-training. In International Conference on Learning Representations, Cited by: Appendix A. Wang et al. (2026) H. Wang, H. Gu, H. Piao, K. Gong, Y. Ye, X. Yue, S. Han, Y. Guo, and D. Wu Learning while staying curious: entropy-preserving supervised fine-tuning via adaptive self-distillation for large reasoning models. In Annual Meeting of the Association for Computational Linguistics, Cited by: Appendix A. Wang et al. (2024) J. T. Wang, T. Wu, D. Song, P. Mittal, and R. Jia GREATS: online selection of high-quality data for LLM training in every iteration. In Advances in Neural Information Processing Systems, Cited by: Appendix A. Wang et al. (2025) Z. Wang, F. Zhou, X. Li, and P. Liu OctoThinker: mid-training incentivizes reinforcement learning scaling. In AI for Math Workshop, Cited by: Appendix A, §1, §1. Watts et al. (2026) I. Watts, C. Li, S. Goyal, J. M. Springer, and A. Raghunathan Sharpness-aware pretraining mitigates catastrophic forgetting. In International Conference on Machine Learning, Cited by: Appendix A, §1. Wei et al. (2024a) Y. Wei, F. Cassano, J. Liu, Y. Ding, N. Jain, Z. Mueller, H. de Vries, L. V. Werra, A. Guha, and L. ZHANG SelfCodeAlign: self-alignment for code generation. In Advances in Neural Information Processing Systems, Cited by: §4.1. Wei et al. (2024b) Y. Wei, Z. Wang, J. Liu, Y. Ding, and L. ZHANG Magicoder: empowering code generation with OSS-instruct. In International Conference on Machine Learning, Cited by: §4.1. Wen et al. (2026) X. Wen, Z. Liu, S. Zheng, S. Ye, Z. Wu, Y. Wang, Z. Xu, X. Liang, J. Li, Z. Miao, J. Bian, and M. Yang Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base LLMs. In International Conference on Learning Representations, Cited by: Appendix A. Wu et al. (2026) C. Wu, S. Goyal, and A. Raghunathan Mode-conditioning unlocks superior test-time compute scaling. In International Conference on Learning Representations, Cited by: Appendix A. Wu and Choi (2025) F. Wu and Y. Choi The invisible leash: why RLVR may not escape its origin. In AI for Math Workshop, Cited by: Appendix A. Xia et al. (2024) M. Xia, S. Malladi, S. Gururangan, S. Arora, and D. Chen LESS: selecting influential data for targeted instruction tuning. In International Conference on Machine Learning, Cited by: Appendix A. Xie et al. (2023) T. Xie, D. J. Foster, Y. Bai, N. Jiang, and S. M. Kakade The role of coverage in online reinforcement learning. In International Conference on Learning Representations, Cited by: Appendix A. Xie and Jiang (2020) T. Xie and N. Jiang Q* approximation schemes for batch reinforcement learning: a theoretical comparison. In Conference on Uncertainty in Artificial Intelligence, Cited by: Appendix A. Yano et al. (2026) K. Yano, S. Kiyono, S. Kobayashi, S. Takase, and J. Suzuki Pre-training LLM without learning rate decay enhances supervised fine-tuning. In International Conference on Learning Representations, Cited by: Appendix A. Yao et al. (2025) J. Yao, R. Cheng, X. Wu, J. Wu, and K. Tan Diversity-aware policy optimization for large language model reasoning. In Advances in Neural Information Processing Systems, Cited by: Appendix A. Yoshihara et al. (2025) H. Yoshihara, T. Yamaguchi, and Y. Inoue A practical two-stage recipe for mathematical LLMs: maximizing accuracy with SFT and efficiency with reinforcement learning. In AI for Math Workshop, Cited by: Appendix A. Yue et al. (2025) Y. Yue, Z. Chen, R. Lu, A. Zhao, Z. Wang, Y. Yue, S. Song, and G. Huang Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?. In Advances in Neural Information Processing Systems, Cited by: Appendix A. Zeng et al. (2025) H. Zeng, K. Hui, H. Zhuang, Z. Qin, Z. Yue, H. Zamani, and D. Alon Can pre-training indicators reliably predict fine-tuning outcomes of LLMs?. Note: arXiv:2504.12491 Cited by: Appendix A. Zhang et al. (2026a) C. Zhang, G. Neubig, and X. Yue On the interplay of pre-training, mid-training, and RL on reasoning language models. In International Conference on Machine Learning, Cited by: Appendix A, §1, §1. Zhang et al. (2026b) W. Zhang, Y. Xie, Y. Sun, Y. Chen, G. Wang, Y. Li, B. Ding, and J. Zhou On-policy RL meets off-policy experts: harmonizing supervised fine-tuning and reinforcement learning via dynamic weighting. In International Conference on Learning Representations, Cited by: Appendix A. Zhao et al. (2026a) A. Zhao, Y. Wu, T. Wu, Q. Xu, Y. Yue, M. Lin, S. Wang, Q. Wu, Z. Zheng, and G. Huang Absolute zero: reinforced self-play reasoning with zero data. Advances in Neural Information Processing Systems 38, p. 105816–105879. Cited by: §1. Zhao et al. (2026b) F. Zhao, H. Zhang, Y. Yang, R. Zhao, and G. Xu Don’t force the fit: bounded log-likelihood loss for enhanced reasoning in large language models. In International Conference on Machine Learning, Cited by: Appendix A, §3.3. Zhao et al. (2025) R. Zhao, A. Meterez, S. M. Kakade, C. Pehlevan, S. Jelassi, and E. Malach Echo chamber: RL post-training amplifies behaviors learned in pretraining. In Conference on Language Modeling, Cited by: Appendix A. Appendix A Additional Related Work Data selection and diversity-preserving SFT Importance sampling and dynamic pruning prioritize high-gradient or low-information examples to accelerate optimization while preserving the full-data objective (Katharopoulos and Fleuret, 2018; Qin et al., 2024); TailSFT instead changes the effective SFT objective to improve coverage and post-RL performance. RHO-LOSS selects learnable, not-yet-learned examples and Dataset Cartography diagnoses examples from training dynamics (Mindermann et al., 2022; Swayamdipta et al., 2020); TailSFT uses only loss reduction relative to the initial policy and masks the most-improved sequences online. Rho-1 and CoLoR use reference-relative losses in pre-training, Token Cleaning selects SFT tokens, and GREATS and LESS estimate utility or influence (Lin et al., 2024; Brandfonbrener et al., 2024; Pang et al., 2025; Wang et al., 2024; Xia et al., 2024); TailSFT operates at sequence or document granularity without target examples or influence computation. Qin and Springenberg (2025) cast curated SFT as implicit RL; TailSFT supplies a dynamic, coverage-motivated rule and evaluates an explicit later RL stage. Confidence caps, entropy regularization, self-distillation, and token-level clipping preserve diversity during SFT (Chen et al., 2025; Li et al., 2025; Zhao et al., 2026b; Wang et al., 2026; Chen et al., 2026b), while prefix-conditioned SFT separates response modes (Fan et al., 2026); TailSFT instead masks already-improved examples relative to the base policy, without auxiliary losses or mode labels. Coupling supervised and reinforcement training Bansal et al. (2026) apply RL at intermediate pre-training checkpoints and blend SFT and RL updates, Fu et al. (2026) jointly weight demonstrations and on-policy rollouts, Huang et al. (2026) continue expert prefixes on-policy, and Zhang et al. (2026b) retain expert SFT as a dynamically weighted RL auxiliary objective. These methods change when or how the objectives are coupled; TailSFT leaves the two-stage pipeline and RL algorithm unchanged and changes only which SFT examples receive gradient. Yoshihara et al. (2025) use prolonged SFT to maximize mathematical accuracy and then GRPO chiefly to improve token efficiency; TailSFT instead optimizes the SFT stage for the final post-RL model, even when local SFT metrics worsen. Niu et al. (2026) derive interference between sequential SFT and RL objectives under their assumptions; TailSFT keeps the stages sequential but reshapes SFT gradients to better support the later RL objective. Coverage and test-time compute The coverage principle formalizes response-level likelihood-ratio coverage as the condition governing Best-of-N recovery (Chen et al., 2026a); TailSFT operationalizes this criterion as an online SFT rule and proves an advantage for relative clipping. Classical RL theory uses concentrability and related coverage coefficients to control error propagation and offline or online learnability (Farahmand et al., 2010; Chen and Jiang, 2019; Xie and Jiang, 2020; Jin et al., 2021; Foster et al., 2022; Jiang and Xie, 2025; Xie et al., 2023); those works ask whether logged state-action data cover a target policy, whereas TailSFT asks whether a language model covers expert responses at a finite sampling budget. Repeated sampling yields predictable coverage gains (Brown et al., 2024), while compute allocation, Best-of-N analysis, inference-aware fine-tuning, mode conditioning, and rollout-budget scaling improve how a fixed policy is sampled or trained for sampling (Snell et al., 2025; Huang et al., 2025b; Chow et al., 2025; Wu et al., 2026; Cheng et al., 2026); TailSFT instead improves the pre-RL policy without a verifier, test-time mode label, or altered rollout allocation. RLVR, sharpening, and exploration Sharpening theory formalizes post-training as reallocating mass within covered behaviors, while sampling analyses show that capabilities can already be latent in the base policy (Huang et al., 2025a; Karan and Du, 2026); TailSFT builds on this view by improving useful coverage before RL. Empirically, RLVR can raise pass@1 without expanding large-k capacity, remain support-constrained, or amplify earlier behaviors (Yue et al., 2025; Wu and Choi, 2025; Zhao et al., 2025), while Wen et al. (2026) show that verifiable rewards can promote correct reasoning latent in the base model; TailSFT changes SFT so more reward-bearing responses remain sampleable. RL-stage methods upweight unlikely correct trajectories, optimize solution diversity, or add representation-based exploration (He et al., 2025; Yao et al., 2025; Tuyls et al., 2026); these methods are complementary, whereas TailSFT uses supervised data alone. Liu et al. (2025) show that prolonged, diverse RL can expand the reasoning boundary, and Foster et al. (2025) show that base-policy coverage controls efficient exploration; TailSFT does not posit an absolute boundary, but supplies a better foundation for standard on-policy RL. Stage-aware model development Kang et al. (2026) show that high SFT scores need not predict post-RL performance and identify held-out generalization loss and pass@large-k as stronger proxies; TailSFT goes beyond diagnosis by modifying SFT to improve coverage and validating the resulting initialization through matched RL runs. Models with the same pre-training loss can transfer differently because of optimization’s implicit bias (Liu et al., 2023); pre-training perplexity can misrank fine-tuning outcomes (Zeng et al., 2025), and downstream scaling laws can be unstable (Lourie et al., 2025). These works expose failures of local metrics; TailSFT supplies an actionable SFT intervention and a coverage-based diagnostic for the SFT-to-RL transition. Extended pre-training can reduce loss while harming adaptability, alternative schedules can improve SFT despite weaker local metrics, and flatter pre-training solutions can reduce forgetting after post-training (Springer et al., 2025; Yano et al., 2026; Watts et al., 2026); TailSFT makes the analogous intervention inside SFT and targets response coverage rather than pre-training geometry. Zhang et al. (2026a) and Wang et al. (2025) show that pre- or mid-training exposure controls later RL gains; TailSFT isolates a lightweight SFT intervention and tests it through matched RL runs. Chu et al. (2025) separate SFT’s stabilizing role from RL’s generalization, while Jin et al. (2025) find that OOD performance can peak early in SFT and be partly restored by RL; TailSFT filters already-fit examples to prevent coverage loss before RL. Appendix B Theoretical Analysis: TailSFT in the expert conditioning setting We consider a stylized setup for supervised fine-tuning called expert conditioning where the SFT data distribution and optimal policy for the downstream task are defined by conditioning the pretrained model output distribution on some event. For theoretical analysis, we focus on a simplified setting where there is no context or prompt, and so all policies are elementary distributions over responses y∈y . Let πref∈Δ⁡() _ref∈ (Y) be the pretrained model and let S⊂S be some subset of responses. In expert conditioning, we define the expert policy as π⋆(y)∝πref(y)⋅y∈Sπ (y) _ref(y)·1\y∈ S\. Note that this setup is closely related to formulations of RLHF where the expert is defined as πref(y)⋅exp⁡(R⋆(y)/β) _ref(y)· (R (y)/β) for some reward function R⋆R ; formally, expert conditioning is equivalent to this formulation in the limit where β→0β→ 0. We are given n samples y1,…,yn∼π⋆y_1,…,y_n π and obtain a policy by solving a certain optimization problem defined in terms of a loss function over the sample. The three loss functions are ERM: : LERM(π):=1n∑i=1n−log(π(yi)) ~~~~~~~~L_ERM(π):= 1n _i=1^n- (π(y_i)) ABS: : LABS,α(π):=1n∑i=1nmax⁡(−log⁡(π⁡(yi))+log⁡(α),0) ~~~~~~~~L_ABS,α(π):= 1n _i=1^n (- (π(y_i))+ (α),0) OFF: : LOFF,β(π):=1n∑i=1nmax⁡(−log⁡(π⁡(yi))+log⁡(β⋅πref(yi)),0) ~~~~~~~~L_OFF,β(π):= 1n _i=1^n (- (π(y_i))+ (β· _ref(y_i)),0) The first objective is standard empirical risk minimization on the cross entropy loss. The second and third are TailSFT variants with absolute and relative clipping respectively. Indeed for absolute clipping, we ignore sample yiy_i if π⁡(yi)≥απ(y_i)≥α and for relative clipping we ignore yiy_i if π⁡(yi)≥βπref(yi)π(y_i)≥β _ref(y_i). The formal optimization problem for a particular loss function is: minimizeπ∈Δ⁡()KL(π∥πref) subject to L(π)=minπ′L(π′) _π∈ (Y)~KL(π\|\| _ref)~ subject to ~L(π)= _π L(π ) (1) Thus we minimize the forward Kullback-Leibler (KL) divergence between the optimization variable π and the reference policy πref _ref subject to π being among the minimizers of the loss L. We use forward KL as simplification of the implicit bias of optimization, and use forward KL primarily due to its mode-covering behavior, so that coverage is preserved to the extent possible subject to minimizing the loss. We take L to be LERM,LABS,α,LOFF,βL_ERM,L_ABS,α,L_OFF,β accordingly. Formally, fixing the dataset, define πERM,πABS,α,πOFF,β _ERM, _ABS,α, _OFF,β to be the optimizers of Eq. 1 with loss functions LERM,LABS,α,LOFF,βL_ERM,L_ABS,α,L_OFF,β respectively. We are interested in qualitatively understanding how well these optimizers cover the expert policy π⋆π , where recall that we define coverage as: CovN(π):=Pry∼π⋆[π⋆(y)π⁡(y)≥N] _N(π):= _y π [ π (y)π(y)≥ N ] Our main result is as follows: theorem We have the following comparisons: 1. Offset clipping is always preferred: For all πref,S _ref,S and datasets: infβCovN(πOFF,β)≤infβ:minπ⁡LOFF,β(π)=0CovN(πOFF,β)≤(infαCovN(πABS,α))∧CovN(πERM) _βCov_N( _OFF,β)≤ _β: _πL_OFF,β(π)=0Cov_N( _OFF,β)≤ ( _αCov_N( _ABS,α) ) _N( _ERM) 2. Offset clipping can dominate: For |||Y| sufficiently large, there exists a reference policy πref _ref and expert subset S such that with probability at least 1−poly⁡(||−1)1-poly(|Y|^-1) infβCovN(πOFF,β)<(infαCovN(πABS,α))∧CovN(πERM) _βCov_N( _OFF,β)< ( _αCov_N( _ABS,α) ) _N( _ERM) 3. Absolute clipping can be worse than ERM: There exists a reference policy πref _ref and expert subset S such that with high probability CovN(πERM)<infα:minπ⁡LABS,α(π)=0CovN(πABS,α) _N( _ERM)< _α: _πL_ABS,α(π)=0Cov_N( _ABS,α) That is, in the regime where zero loss is achievable for absolute clipping, we can have that empirical risk minimization strictly dominates absolute loss clipping. Note that the first claim implies that offset clipping—even when restricted to the parameter regime where zero loss is achievable–is never worse than empirical risk minimization. B.1 Proof of Appendix B First we derive a structural characterization of the solutions of Eq. 1. Next, we use this characterization to understand the coverage of the policies πERM,πABS,α,πOFF,β _ERM, _ABS,α, _OFF,β. Finally we establish the comparisons in the theorem statement. lemma [Structure of KL projections] Let fy:y∈⊂[0,1]\f_y:y \⊂[0,1] satisfy ∑yfy≤1 _yf_y≤ 1. Consider the optimization problem π^←argminπ∈Δ⁡()KL(π∥πref)subjectto∀y:π(y)≥fy. π← _π∈ (Y)~KL(π\|\| _ref)~subject\ to~∀ y:π(y)≥ f_y. Then the minimizer π π is unique and given by π^(y)=max⁡(fy,λπref(y))where∑ymax⁡(fy,λπref(y))=1. π(y)= (f_y,λ _ref(y)) _y (f_y,λ _ref(y))=1. Proof of Section B.1. Observe that the optimization problem is strictly convex and feasible under the condition that ∑yfy≤1 _yf_y≤ 1. We write the Lagrangian of the optimization problem as ∑yπ⁡(y)log⁡π⁡(y)πref(y)+λ⁡(∑yπ⁡(y)−1)+∑yμy(f⁡(y)−π⁡(y))μy≥0. _yπ(y) π(y) _ref(y)+λ ( _yπ(y)-1 )+ _y _y(f(y)-π(y)) _y≥ 0. The stationary conditions are ∀y:log⁡π⁡(y)πref(y)+1+λ−μy=0. ∀ y:~ π(y) _ref(y)+1+λ- _y=0. If π⁡(y)>fyπ(y)>f_y, so the constraint is inactive, then by complementary slackness we have μy=0 _y=0, so π⁡(y)=πref(y)⋅exp⁡(−1−λ)π(y)= _ref(y)· (-1-λ) and this must be larger than fyf_y. Otherwise, we must have π⁡(y)=fyπ(y)=f_y solving the stationary condition for μy _y gives μy=log⁡(π⁡(y)/πref(y))+1+λ=log⁡fyπref(y)⋅exp⁡(−1−λ) _y= (π(y)/ _ref(y))+1+λ= f_y _ref(y)· (-1-λ) The constraint that μy≥0 _y≥ 0 thus implies that fy≥πref(y)⋅exp⁡(−1−λ)f_y≥ _ref(y)· (-1-λ). Taken together, this gives π⁡(y)=max⁡(fy,πref(y)exp⁡(−1−λ))π(y)= (f_y, _ref(y) (-1-λ)) and λ is chosen to normalize the distribution. ∎ lemma [Structure of clipped minimizer] Let fy:y∈\f_y:y \ let p∈Δ⁡(Y)p∈ (Y) and let ∑yp(y)>0fy>1 _y1\p(y)>0\f_y>1. Then the minimizer of ∑y∈p⁡(y)max(−log⁡π⁡(y)+log⁡(fy),0) _y p(y) (- π(y)+ (f_y),0) is unique and given by π^(y)=min⁡(fy,λp(y))where∑ymin⁡(fy,λp(y))=1. π(y)= (f_y,λ p(y)) _y (f_y,λ p(y))=1. Proof. First observe that for any y such that p⁡(y)=0p(y)=0 we will have π⁡(y)=0π(y)=0 as well, because the conditions on fyf_y imply that we cannot achieve an objective value of zero, and we can always shift mass from actions with p⁡(y)=0p(y)=0 to those with p⁡(y)≠0p(y)≠ 0 to reduce the loss. Thus, without loss of generality we can assume that p⁡(y)≠0p(y)≠ 0 which also implies π⁡(y)≠0π(y)≠ 0. Next, we write the Lagrangian: ∑yp⁡(y)max(−log⁡π⁡(y)+log⁡(fy),0)+λ⁡(∑yπ⁡(y)−1) _yp(y) (- π(y)+ (f_y),0)+λ ( _yπ(y)-1 ) The stationary condition is that −p⁡(y)π⁡(y)+λ=0 -p(y)π(y)+λ=0 if π⁡(y)<fyπ(y)<f_y. If OPENπ⁡(y)=fy)π(y)=f_y) the sub-differential for the loss term is in [−p(y)/π(y),0][-p(y)/π(y),0] and if π⁡(y)>0π(y)>0 we must have λ=0λ=0 to satisfy stationarity. However, since at least one y must have π⁡(y)<fyπ(y)<f_y we get that λ>0λ>0. Therefore no action can have π⁡(y)>fyπ(y)>f_y and we have that π⁡(y)=min⁡(fy,λp(y))π(y)= (f_y,λ p(y)). ∎ Next we characterize the structure and the coverage of the πOFF,β _OFF,β. lemma[Coverage of πOFF,β _OFF,β] For any dataset, let S^⊆S S S be the actions observed in the dataset. We have infβ≥0CovN(πOFF,β)≤infβ:minπ⁡LOFF,β(π)=0CovN(πOFF,β)=0,N≥1/πref(S)πref(S∖S^)πref(S),1<N<1/πref(S) _β≥ 0Cov_N( _OFF,β)≤ _β: _πL_OFF,β(π)=0Cov_N( _OFF,β)= cases0,&N≥ 1/ _ref(S)\\ _ref(S S) _ref(S),&1<N<1/ _ref(S) cases Proof of Section B.1. If N≥1/πref(S)N≥ 1/ _ref(S) then take β=1β=1 so that πref _ref itself is a minimizer of LOFF,βL_OFF,β, i.e., LOFF,β(πref)=0L_OFF,β( _ref)=0. With this choice of β, clearly πref _ref is the solution to Eq. 1. To compute the coverage, observe that for every action y, π⋆(y)/πref(y)=1/πref(S)≤Nπ (y)/ _ref(y)=1/ _ref(S)≤ N. Thus, in this case Cov⁡(πref)=0Cov( _ref)=0 and with optimally tuned β, the KL projection onto the set of offset-loss minimizers preserves this coverage. If 1<N<1/πref(S)1<N<1/ _ref(S) we first translate the offset loss optimization problem into the form in Section B.1 and then compute the normalizing constant λ in the distribution. Observe that the offset loss only involves observed actions y∈S^y∈ S. If β is such that 00 offset loss is feasible, then we have LOFF,β(π)=0⇔∀y∈S^:π⁡(y)≥βπref(y) and ∀y∉S^:π⁡(y)≥0 L_OFF,β(π)=0 ∀ y∈ S:~π(y)≥β _ref(y) and ∀ y∉ S:π(y)≥ 0 Here the first condition arises from achieving zero loss, while the second condition arises because π must be a distribution. Thus the offset loss version of Eq. 1 corresponds to taking fy=βπref(y)f_y=β _ref(y) if y∈S^y∈ S and fy=0f_y=0 otherwise in Section B.1. Next, observing that every action in the minimizing distribution has mass at least λπref(y)λ _ref(y) we get that λ≤1λ≤ 1. Since fy=0f_y=0 for actions y∉S^y∉ S, we have that πOFF,β(y)=λπref(y) _OFF,β(y)=λ _ref(y) for all y∉S^y∉ S. Such actions that are further supported by π⋆π are uncovered, since π⋆(y)πOFF,β(y)≥π⋆(y)πref(y)=1πref(S)>N π (y) _OFF,β(y)≥ π (y) _ref(y)= 1 _ref(S)>N The first inequality uses that λ≤1λ≤ 1 and the second inequality is by our assumed regime for N. Thus we obtain for any feasible β we obtain a coverage lower bound CovN(πOFF,β)≥πref(S∖S^)πref(S). _N( _OFF,β)≥ _ref(S S) _ref(S). To show that this is achievable take β=(πref(S)N)−1>1β=( _ref(S)N)^-1>1 and observe that βπref(S)=1/N<1. β _ref(S)=1/N<1. Thus if we set π⁡(y)=βπref(y)π(y)=β _ref(y) for y∈Sy∈ S we achieve 0 loss and have ∑yπ⁡(y)<1 _yπ(y)<1; we can allocate the remaining mass arbitrarily to preserve 0 offset loss and obtain a distribution. Since zero-loss is achievable, the KL projection ensures that πOFF,β(y)≥βπref(y) _OFF,β(y)≥β _ref(y) for all y∈S^y∈ S as this is required to achieve zero loss. By the choice of β this actions are covered. On the other hand, all other actions have πOFF,β(y)=λπref(y)≤πref(y) _OFF,β(y)=λ _ref(y)≤ _ref(y) using the argument above that λ≤1λ≤ 1. These actions are all not covered under the condition that 1/πref(S)>N1/ _ref(S)>N. Thus this choice of β achieves coverage exactly πref(S∖S^)/πref(S) _ref(S S)/ _ref(S). ∎ Proof of Claim 1 of Appendix B. We first argue about πERM _ERM. Note that the ERM is unique and is exactly the empirical distribution over the sample. In particular, we have πERM(S∖S^)=0 _ERM(S S)=0, and therefore these actions are always uncovered, so CovN(πERM)≥πref(S∖S^)/πref(S)Cov_N( _ERM)≥ _ref(S S)/ _ref(S). Next we turn to πABS,α _ABS,α. Here we consider two cases. First consider the case that α is large enough such that zero loss is not achievable. In this case, if we have a policy π that allocates mass to unobserved actions, we can always strictly improve the loss by moving that mass onto some observed action that is not already saturated, and such an action must exist if zero loss is not achievable. Thus the loss minimizer must allocate no mass on unobserved actions. In this case the same reasoning as we applied to analyze the ERM holds. Next, consider that α is small enough such that zero loss is achievable. As with the offset loss, we can translate the optimization problem to the form in Section B.1 and take fy=αy∈S^f_y= 1\y∈ S\. However, as in the proof of Section B.1, we still require that λ≤1λ≤ 1. As before when 1<N<1/πref(S)1<N<1/ _ref(S), this implies that all unobserved actions in S are uncovered, establishing a lower bound of πref(S∖S^)/πref(S) _ref(S S)/ _ref(S) in this regime. ∎ Proof of Claim 2 in Appendix B. We construct an instance with three types of arms with ||=1+n/4+3n|Y|=1+n/4+3n for some n∈ℕn such that =o∪H∪LY=\o\∪ H∪ L where |H|=n/4|H|=n/4 and L=3nL=3n. Define πref(o)=12,πref(h)=1n,h∈H,πref(ℓ)=112n,ℓ∈L. _ref(o)= 12, _ref(h)= 1n,h∈ H, _ref( )= 112n, ∈ L. Set S=H∪LS=H∪ L such that π⋆(h)=2n,h∈H,π⋆(ℓ)=16n,ℓ∈L π (h)= 2n,h∈ H, π ( )= 16n, ∈ L Recall that S S is the set of actions observed in the sample. The high probability event is the intersection of two events: (1) |S^|| S| is sufficiently large and (2) |H^(1)|| H^(1)| is sufficiently large, where H^(1) H^(1) is the set of actions in H that are observed exactly once. To control |S^|| S|, we first calculate: |S^∩H| | S∩ H| =∑h∈HPr[h∈S^]=n4(1−(1−2/n)n)≥n4(1−e−2) = _h∈ H [h∈ S]= n4(1-(1-2/n)^n)≥ n4(1-e^-2) |S^∩L| | S∩ L| =∑ℓ∈LPr[ℓ∈S^]=3n(1−(1−1/(6n))n)≥3n(1−e−1/6) = _ ∈ L [ ∈ S]=3n(1-(1-1/(6n))^n)≥ 3n(1-e^-1/6) Observe that |S^|=|S^∩H|+|S^∩L|| S|=| S∩ H|+| S∩ L| and that the random variable |S^|| S| satisfies the conditions of McDiarmid’s inequality with constant 11. Thus with probability at least 1−δ/21-δ/2 we have |S^|≥|S^|−nlog⁡(2/δ)2≥n4(1−e−2)+3n(1−e−1/6)−nlog⁡(2/δ)2. | S| | S|- n (2/δ)2≥ n4(1-e^-2)+3n(1-e^-1/6)- n (2/δ)2. The constant on the Θ⁡(n) (n) term is at least 0.6760.676, and for δ=poly⁡(1/n)δ=poly(1/n) the Θ⁡(n) (n) term dominates, and so we have that for n sufficiently large, |S^|≥2n/3| S|≥ 2n/3. We control |H^(1)|| H^(1)| similarly: |H^(1)|=∑h∈HPr[Bh=1]=n4(n⋅2n⋅(1−2/n)n−1)≥n2⋅(1−2/n)n−1 | H^(1)|= _h∈ H [B_h=1]= n4 (n· 2n·(1-2/n)^n-1 )≥ n2·(1-2/n)^n-1 Here BhB_h is a Binomially distributed random variable with parameters (n,2/n)(n,2/n). Again we apply McDiarmid’s inequality, here |H^(1)|| H^(1)| satisfies the conditions with a constant of 22, so that with probability at least 1−δ/21-δ/2 we have |H^(1)|≥|H^(1)|−2nlog⁡(2/δ)≥n2⋅(1−2/n)n−1−2nlog⁡(2/δ) | H^(1)| | H^(1)|- 2n (2/δ)≥ n2·(1-2/n)^n-1- 2n (2/δ) Observe that the first term asymptotically approaches n⋅e−2/2n· e^-2/2 while the second term is lower order in n. Therefore for n sufficiently large, we can see that |H^(1)|≥e−2/3| H^(1)|≥ e^-2/3 with high probability. Next we turn to the coverage calculations. Since N=4/3N=4/3 satisfies 1<N<1/πref(S)=21<N<1/ _ref(S)=2 we know that infβ≥0Cov⁡(πOFF,β)=πref(S∖S^)/πref(S)=π⋆(S∖S^) _β≥ 0Cov( _OFF,β)= _ref(S S)/ _ref(S)=π (S S). This is exactly the expert mass of the unobserved set. On the other hand πERM _ERM matches exactly the empirical frequencies in the data. In particular, we have πERM(h)=1/n _ERM(h)=1/n for h∈H^(1)h∈ H^(1). This implies Cov(πERM)≥∑y∈S∖S^π⋆(y)+∑y∈H^(1)2n2/n1/n≥N=π⋆(S∖S^)+π⋆(H^(1)) ( _ERM)≥ _y∈ S Sπ (y)+ _y∈ H^(1) 2n1 \ 2/n1/n≥ N \=π (S S)+π ( H^(1)) Finally for πABS,α _ABS,α we need to show that no absolute clipping threshold improves on this value. To cover any h∈H^(1)h∈ H^(1) we need πABS,α(h)≥3/(2n) _ABS,α(h)≥ 3/(2n). Recall that we only see |S^|| S| actions in the dataset. If we set α≤1/|S^|α≤ 1/| S| then we can achieve zero loss but from Section B.1 we have πABS,α(h)=max⁡(α,λπref(h))≤max⁡(1/|S^|,1/n)<3/(2n). _ABS,α(h)= (α,λ _ref(h))≤ (1/| S|,1/n)<3/(2n). Here recall that λ≤1λ≤ 1 (otherwise πABS,α(h) _ABS,α(h) will not be a distribution) and that we have |S^|>2n3| S|> 2n3 with high probability. In this regime, unobserved actions get mass λπref(h)λ _ref(h) but this is insufficient to cover π⋆π (since 2=π⋆(y)/πref(y)=2≥N=4/32=π (y)/ _ref(y)=2≥ N=4/3). On the other hand, if α>1/|S^|α>1/| S| then, by Section B.1 the unique minimizer of the absolute clipped loss is πABS,α(y)=min⁡(α,λμ^(y)) _ABS,α(y)= (α,λ μ(y)) where μ^(y) μ(y) is the empirical distribution. The normalizing condition gives, 1≥∑y∈S^min⁡(α,λμ^(y))≥|S^|min⁡(α,λ/n), 1≥ _y∈ S (α,λ μ(y))≥| S| (α,λ/n), which follows because every observed action is observed at least once, so it has empirical mass at least 1/n1/n. Since α>1/|S^|α>1/| S| we must have λ/n≤1/|S^|λ/n≤ 1/| S| and therefore actions h∈H^(1)h∈ H^(1) have πABS,α(h)≤1/|S^|≤32n _ABS,α(h)≤ 1/| S|≤ 32n (while unobserved actions have πABS,α(y)=0 _ABS,α(y)=0). Thus in both cases we have Cov⁡(πABS,α)≥π⋆(S∖S^)+π⋆(H^(1)) ( _ABS,α)≥π (S S)+π ( H^(1)) The high probability event implies that |H^(1)|>0| H^(1)|>0 and so π⋆(H^(1))>0π ( H^(1))>0, establishing strict separation. ∎ Proof of Claim 3 in Appendix B. We use the same construction as in the proof of Claim 2. Recall that in that construction, zero absolute clipped loss is achievable only if α≤1/|S^|α≤ 1/| S|. Let H^(>1) H^(>1) denote the set of actions in H that are observed at least twice in the sample S S. Since α≤1/|S^|≤3/(2n)α≤ 1/| S|≤ 3/(2n) (since with high probability we have |S^|≥2n/3| S|≥ 2n/3), via Section B.1, the absolute clipped loss solution satisfies πABS,α(h)=max⁡(α,λπref(h))≤max⁡(1/|S^|,1/n)<3/(2n)=π⋆(h)/N, _ABS,α(h)= (α,λ _ref(h))≤ (1/| S|,1/n)<3/(2n)=π (h)/N, if we take N=4/3N=4/3 as above. Thus all actions in H^(<1) H^(<1) violate coverage. We already saw that all actions in H^(1) H^(1) violate coverage as well as all unobserved actions in L. Thus the coverage is infα:minπ⁡LABS,α(π)=0Cov(πABS,α)=12+π⋆(L∖S^). _α: _πL_ABS,α(π)=0Cov( _ABS,α)= 12+π (L S). On the other hand, ERM covers actions in H^(>1) H^(>1), because for any h∈H^(>1)h∈ H^(>1), we have πERM(h)≥2/n _ERM(h)≥ 2/n while π⋆(h)=2/nπ (h)=2/n. Thus ERM strictly dominates absolute clipping whenever |H^(>1)|>0| H^(>1)|>0. Since |H^(>1)|=∑h∈HPr[Bh>1]=n4⋅(1−(1−2/n)n−2(1−2/n)n−1) | H^(>1)|= _h∈ H [B_h>1]= n4·(1-(1-2/n)^n-2(1-2/n)^n-1) where Bh∼Binomial(n,2/n)B_h Binomial(n,2/n), and since |H^(>1)|| H^(>1)| satisfies the conditions of McDiarmid’s inequality with constant 1, we have that with probability at least 1−δ1-δ |H^(>1)|≥n4⋅(1−(1−2/n)n−2(1−2/n)n−1)−n2log⁡(1/δ). | H^(>1)|≥ n4·(1-(1-2/n)^n-2(1-2/n)^n-1)- n2 (1/δ). For sufficiently large n the first term is larger than (1−3e−2)n/5(1-3e^-2)n/5 and the second term is lower order, so for sufficiently large n we get that |H(>1)|>1|H^(>1)|>1 with high probability. ∎ Appendix C Details for Graph Navigation Experiments In this section, we describe the task, training details, and results for the graph navigation experiments presented in Fig. 3. The setting is closely related to that of Chen et al. (2026a); we present all details for completeness but highlight where our experiments diverge from theirs. C.1 Graph Reasoning Task and Data Following Chen et al. (2026a), we use a path-following task in a directed acyclic graph (DAG) as an abstraction for reasoning problems. The setup builds on a long line of work using synthetic tasks to understand phenomena in language modeling and serves as a minimal, expressive, yet flexible setting to develop interventions like TailSFT. The task is path-following a directed acyclic graph. Each prompt x∈x encodes a graph G along with a source node s and a target node t, such that x=(G,s,t)x=(G,s,t), and each response y∈y ideally encodes an s→ts→ t path via a sequence of vertices (s,v1,…,vn,t)(s,v_1,…,v_n,t). In our experiments we fix a single prompt distribution μ∈Δ⁡()μ∈ (X) for both pre-training and SFT stages, however we use different data-collection policies πpre,πSFT:Y→Δ⁡() _pre, _SFT:Y→ (X), both of which map prompts to (distributions over) s→ts→ t paths. All graphs G are directed layered graphs with 1010 total layers and with edges only between consecutive layers. The source vertex s is the only vertex in layer L=1L=1 and the target vertex t is the only vertex in layer 1010. The subsequent 8 layers have 4 vertices each. For each of layers L∈1,…,9L∈\1,…,9\, a subset of the vertices in that layer are called passable. The edge structure is such that every passable node in layer ℓ has directed edges to all nodes in layer ℓ+1 +1. Nodes that are not passable have no out-going edges. Thus a valid s→ts→ t path consists of 1010 total nodes (s,v1,…,v8,t)(s,v_1,…,v_8,t) where vℓv_ is in layer ℓ and each vℓv_ must be passable. Graphs are parameterized by a number k∈0,…,8k∈\0,…,8\ denoting the number of layers with two passable nodes; the remaining layers have exactly one passable node. Given k we choose the layers with two passable nodes at random. Choosing the passable nodes is more intricate. We identify every vertex with a number ∈0,…,99∈\0,…,99\, such that in each of the intermediate layers, at least one vertex is odd and at least one vertex is even. When there is just one passable vertex in a layer, it is chosen uniformly at random. When there are two passable vertices, they are chosen at random such that one is odd and one is even. Thus, there are 2k2^k valid s→ts→ t paths in each graph. We use k=3k=3 for all experiments. As described above, pre-training and SFT share the same prompt distribution. These are graphs of the above structure with vertex names assigned randomly (subject to the aforementioned even/odd restrictions in each layer), with passable nodes assigned randomly, and with layers with two passable nodes selected randomly. For pre-training, the data-collection policy πpre _pre selects an s→ts→ t path uniformly at random from all 2k2^k valid paths. For SFT, the data-collection policy is more complex. First the policy computes a certain “cryptographic” function of the input graph to determine if the graph belongs to shard0 or shard1 (specifically, the cryptographic function is the and of the parity of the vertices layers 1-5 and the parity of the vertices in layers 6-10). The shard determines a certain rule for how the policy chooses vertices in the layers where there are two passable nodes. Specifically in shard0 the policy alternates between agreeing with the parity of the target vertex t and disagreeing with the parity of target node t, such that it agrees with the target vertex parity in the first layer with two passable nodes, disagrees in the second, and so on. In shard1 the policy does exactly the opposite, it disagrees with the target vertex parity in the first layer with two passable nodes, agrees in the second, and so on. Intuitively, learning the pre-training policy’s response distribution is relatively easy as it only requires local rules such as identifying all passable nodes in the subsequent layer and choosing one at random. Pre-training on this distribution teaches the model to understand the input format and the high-level task. On the other hand, learning the SFT policy’s response distribution is very challenging, as it requires identifying global structure both to determine the shard and to determine which passable vertex to select in each layer. To measure the performance of the trained model, we observe that the SFT policy is deterministic (for any input graph x, there is a single path in the support of the SFT policy’s distribution πSFT(⋅∣x) _SFT(· x)) and set the reward to be 11 if and only if the path chosen by the trained model matches the path selected by the SFT policy. Since learning the SFT policy’s response distribution is challenging, it is corresponding quite challenging to achieve high reward in this task. Inputs and outputs are represented as follows. First, all vertices, including the source and target, are assigned a number in 0,…,99\0,…,99\. Then, the input prompt is represented as an edge list followed by the source and target nodes, formatted as: x:u_1 v_1 | u_2 v_2 | …| u_k v_k / s t = x:~~ u\_1 v\_1 | u\_2 v\_2 | …| u\_k v\_k / s t = where ui,viu_i,v_i are the numerical values assigned to the vertices in the iith edge. We use the delimiter characters |, /, and = to separate edges from each other, the edge list from the source and target vertices, and the input from the response, respectively. Responses y are formatted as: y:v_1 v_2 v_2 v_3 …v_9 v_10. y:~~ v\_1 v\_2 v\_2 v\_3 … v\_9 v\_10. C.2 Training Details We use the same numerical tokenizer and transformer architecture as Chen et al. (2026a). For tokenization, each node is v is tokenized as its numerical value and the special characters are tokenized as 100,101,102100,101,102. The architecture is a GPT2-style transformer model with 4 heads, 6 transformer blocks and a 384 dimensional embeddings. We used absolute positional encodings. We always use the Adam optimizer with a fixed learning(1×10−41×10^-4 during pre-training and 5×10−65×10^-6 during SFT) and a batch size of 128. Pre-training operates in an offline training setting with a fixed pool of 256,000 graphs while SFT operates in an online training setting with new graphs generated on-the-fly for each batch. We pre-train for 200k training steps and SFT for 50k training steps with evaluation every 200 steps. Figure 5: Detailed results of hyperparameter sweep for TailSFT with absolute clipping in the synthetic graph navigation experiment. Figure 6: Detailed results of hyperparameter sweep for TailSFT with quantile clipping in the synthetic graph navigation experiment. Figure 7: Detailed results of hyperparameter sweep for TailSFT with offset quantile clipping in the synthetic graph navigation experiment. C.3 Additional Experimental Results In Fig. 3 we show the cross entropy, pass@1, and pass@8 for each of four methods: vanilla SFT, TailSFT with absolute clipping, TailSFT with quantile clipping, and TailSFT with offset quantile clipping using the pre-trained model as the reference. For the TailSFT variants we sweep over clipping hyperparameter and select the best hyperparameter based on pass@8 performance at the end of training. The specific hyperparameter selected are 0.05 for absolute, 0.4 for quantile, and 0.35 for offset quantile. In Figs. 5, 6 and 7, we show detailed results of the hyperparameter sweep for the three TailSFT variants. In all cases, we sweep the clipping hyperparameter in the range 0.05×1…,190.05×\1…,19\. For all variants clipping at value 00 corresponds to vanilla SFT as no sequences are dropped. We highlight two observations. First, absolute clipping is much more sensitive to the clipping hyperparameter than quantile or offset quantile variants. This is rather intuitive, as setting an absolute threshold for the cross-entropy loss requires understanding how the distribution of per-example cross-entropy losses evolves during training. On the other hand, setting the threshold as a per-batch quantile (either on the loss or on the offset loss) is much easier and these methods are much more robust. Indeed, the second observation is that for quantile-variants of TailSFT, setting the threshold too small recovers baseline SFT performance and setting the threshold too large does not impact pass@8 performance, but rather slows training convergence. This is also intuitive, as a large clipping parameter still focuses training on the harder examples but sacrifices computational efficiency as fewer per-sample gradients are utilized. Appendix D Details for Language Modeling Experiments D.1 SFT The language modeling experiments use the same TailSFT algorithm (Algorithm 1) for both math and code but differ in data, prompt format, hyperparameter grid, and evaluation. We describe the shared procedure here; the code (Section D.1.1) and math (Section D.1.2) subsections give the domain-specific details. Supervised fine-tuning Every run starts from the OLMo-3 7B base model and minimizes the standard supervised cross-entropy loss on chat-formatted instruction/response pairs. We apply the OLMo-3 chat template and mask every prompt token, so only the final assistant turn and its terminating end-of-turn token count toward the loss. All runs use AdamW with (β1,β2)=(0.9,0.95)( _1, _2)=(0.9,0.95) and ϵ=10−8ε=10^-8, gradient-norm clipping at 1.01.0, 88-way data parallelism, and a learning rate held constant after a linear warmup over the first 3%3\% of steps. Sequence length, weight decay, microbatch size, effective batch, epoch count, and the learning-rate grid are set per domain and given in each subsection. The per-step loss is length-normalized as in OLMo-3: we sum the target-token cross-entropy over the batch and divide by the number of unmasked target tokens, so every target token is weighted equally rather than every sequence. Example filtering (TailSFT) TailSFT drops a fraction of the already-fit examples from the loss at each step. Before training we score every example x once with the base model, using the same tokenization and assistant-only mask as SFT, to get its initialization loss ℓ0(x) _0(x), the mean cross-entropy over the target tokens. During training we compare the current per-example loss ℓt(x) _t(x) against this baseline through the signed margin mt(x)=ℓt(x)−ℓ0(x).m_t(x)= _t(x)- _0(x). A very negative margin means the loss on x has already dropped well below its starting value. We filter out the examples with the smallest (most negative) margins and train on the rest. Both ℓ0 _0 and ℓt _t are length-normalized per example—the mean cross-entropy over that example’s target tokens—so the filtering decision does not favor longer or shorter responses. The filtering decision runs on the selection batch, the W×bW× b examples in a single forward pass across the W=8W=8 data-parallel ranks with per-rank microbatch b. The per-example losses are all-gathered so every rank computes the same mask. We do not filter over the full gradient-accumulated optimizer batch or over one rank’s local microbatch; each accumulation step is filtered on its own selection batch. Within a selection batch we rank the examples by mtm_t and zero out the ⌊Wbf⌉ Wb\,f with the smallest margins, then train the survivors with the usual token-averaged cross-entropy. The code runs use b=1b=1 (an 88-example selection batch) and the math runs use b=2b=2 (a 1616-example selection batch). Reproducing the method needs only the base-model losses ℓ0 _0, the filter fraction f, and its schedule; everything else is ordinary distributed SFT. Filtering notation A run is specified by its filter fraction f∈[0,1]f∈[0,1], the fraction of each selection batch that is dropped (⌊Wbf⌉ Wb\,f examples), and a schedule for how f moves over training. A static schedule holds f fixed. A ramp schedule written →f0\!→\!f raises the fraction linearly from 00 at the first step to f at the last, using every example early and filtering hardest late. Standard SFT is f=0f=0. We describe every run by its filter fraction f and schedule in the below results. Filtering rarely hurts The consistent finding across both domains is that filtering preserves pass@16 coverage. At matched settings it almost always matches or improves the no-filter pass@16, the gains are sometimes large, and the only cost is a small pass@1 decrease from deprioritizing examples the model already fits. Each subsection reports the full per-benchmark grids and tallies how rarely filtering regresses coverage. Tables 3 and 4 collect the base model, Standard SFT, and TailSFT side by side at the main-table settings for pass@1 and pass@16. In several cases where standard SFT marginally lowers pass@16 below the base model, TailSFT rescues the coverage, recovering it back above the base level. If we instead select the best pass@16 checkpoint for each of Standard SFT and TailSFT, TailSFT still matches or improves coverage on most benchmarks. Table 3: Per-benchmark pass@1 for the OLMo-3 7B base model (no SFT), Standard SFT (f=0f=0), and TailSFT, at the settings used in the main results table. ΔStd−Base _Std-Base is Standard SFT minus base; ΔTail−Base _Tail-Base is TailSFT minus base. Emdashes indicate missing evaluations. SFT data Benchmark Base Standard SFT TailSFT ΔStd−Base _Std-Base ΔTail−Base _Tail-Base OMI AIME 5.165.16 3.65±0.373.65_± 0.37 4.03±0.024.03_± 0.02 −1.51-1.51 −1.13-1.13 MATH-500 L5 — 24.80±0.4024.80_± 0.40 25.16±0.4025.16_± 0.40 — — OMEGA-500 3.54±0.303.54_± 0.30 6.58±0.506.58_± 0.50 6.51±0.266.51_± 0.26 +3.04+3.04 +2.97+2.97 BigCode MBPP+ 36.82±0.5636.82_± 0.56 53.04±0.9353.04_± 0.93 51.36±0.1451.36_± 0.14 +16.22+16.22 +14.54+14.54 HumanEval+ 32.94±1.1132.94_± 1.11 45.06±0.5745.06_± 0.57 46.33±1.0746.33_± 1.07 +12.12+12.12 +13.39+13.39 CruxEval-I 25.67±0.6225.67_± 0.62 29.79±0.4829.79_± 0.48 30.43±0.1530.43_± 0.15 +4.12+4.12 +4.76+4.76 CruxEval-O 4.22±0.744.22_± 0.74 4.16±0.444.16_± 0.44 13.18±0.3013.18_± 0.30 −0.06-0.06 +8.96+8.96 LiveCodeBench 5.50±0.425.50_± 0.42 10.74±0.4610.74_± 0.46 11.49±0.3811.49_± 0.38 +5.24+5.24 +5.99+5.99 Magicoder MBPP+ 36.82±0.5636.82_± 0.56 55.35±0.4755.35_± 0.47 54.60±0.5554.60_± 0.55 +18.53+18.53 +17.78+17.78 HumanEval+ 32.94±1.1132.94_± 1.11 48.49±0.5848.49_± 0.58 47.69±0.1947.69_± 0.19 +15.55+15.55 +14.75+14.75 CruxEval-I 25.67±0.6225.67_± 0.62 28.50±0.1328.50_± 0.13 26.48±0.4526.48_± 0.45 +2.83+2.83 +0.81+0.81 CruxEval-O 4.22±0.744.22_± 0.74 12.86±0.1912.86_± 0.19 16.16±0.5316.16_± 0.53 +8.64+8.64 +11.94+11.94 LiveCodeBench 5.50±0.425.50_± 0.42 14.84±0.1114.84_± 0.11 14.32±0.3814.32_± 0.38 +9.34+9.34 +8.82+8.82 OCI MBPP+ 36.82±0.5636.82_± 0.56 58.26±0.8858.26_± 0.88 55.81±0.0655.81_± 0.06 +21.44+21.44 +18.99+18.99 HumanEval+ 32.94±1.1132.94_± 1.11 54.76±1.1654.76_± 1.16 51.94±1.8351.94_± 1.83 +21.82+21.82 +19.00+19.00 CruxEval-I 25.67±0.6225.67_± 0.62 29.46±0.4629.46_± 0.46 30.17±0.6430.17_± 0.64 +3.79+3.79 +4.50+4.50 CruxEval-O 4.22±0.744.22_± 0.74 9.30±0.499.30_± 0.49 10.84±0.3710.84_± 0.37 +5.08+5.08 +6.62+6.62 LiveCodeBench 5.50±0.425.50_± 0.42 12.24±0.3612.24_± 0.36 11.49±0.8411.49_± 0.84 +6.74+6.74 +5.99+5.99 Table 4: Per-benchmark pass@16 (coverage) for the OLMo-3 7B base model (no SFT), Standard SFT (f=0f=0), and TailSFT, at the settings used in the main results table. Columns as in Table 3. SFT data Benchmark Base Standard SFT TailSFT ΔStd−Base _Std-Base ΔTail−Base _Tail-Base OMI AIME 20.9120.91 15.24±2.0215.24_± 2.02 18.31±0.5718.31_± 0.57 −5.67-5.67 −2.60-2.60 MATH-500 L5 — 66.42±0.0066.42_± 0.00 69.15±0.8669.15_± 0.86 — — OMEGA-500 26.13±1.4226.13_± 1.42 32.80±2.2532.80_± 2.25 32.60±1.5632.60_± 1.56 +6.67+6.67 +6.47+6.47 BigCode MBPP+ 75.93±1.0675.93_± 1.06 74.69±1.1074.69_± 1.10 78.84±1.0678.84_± 1.06 −1.24-1.24 +2.91+2.91 HumanEval+ 79.27±0.6179.27_± 0.61 76.63±0.7076.63_± 0.70 80.08±0.9380.08_± 0.93 −2.64-2.64 +0.81+0.81 CruxEval-I 70.50±1.5470.50_± 1.54 61.71±2.2761.71_± 2.27 65.79±0.2665.79_± 0.26 −8.79-8.79 −4.71-4.71 CruxEval-O 32.08±1.9832.08_± 1.98 24.21±2.3824.21_± 2.38 41.00±1.3541.00_± 1.35 −7.87-7.87 +8.92+8.92 LiveCodeBench 32.30±0.9332.30_± 0.93 30.39±0.8630.39_± 0.86 33.12±0.3433.12_± 0.34 −1.91-1.91 +0.82+0.82 Magicoder MBPP+ 75.93±1.0675.93_± 1.06 75.31±0.9375.31_± 0.93 78.66±1.2578.66_± 1.25 −0.62-0.62 +2.73+2.73 HumanEval+ 79.27±0.6179.27_± 0.61 77.85±0.7077.85_± 0.70 78.66±1.8378.66_± 1.83 −1.42-1.42 −0.61-0.61 CruxEval-I 70.50±1.5470.50_± 1.54 59.75±0.6259.75_± 0.62 68.08±0.9468.08_± 0.94 −10.75-10.75 −2.42-2.42 CruxEval-O 32.08±1.9832.08_± 1.98 38.08±0.9438.08_± 0.94 47.92±1.6647.92_± 1.66 +6.00+6.00 +15.84+15.84 LiveCodeBench 32.30±0.9332.30_± 0.93 31.81±0.3431.81_± 0.34 33.22±0.3433.22_± 0.34 −0.49-0.49 +0.92+0.92 OCI MBPP+ 75.93±1.0675.93_± 1.06 81.22±0.7581.22_± 0.75 82.36±0.8182.36_± 0.81 +5.29+5.29 +6.43+6.43 HumanEval+ 79.27±0.6179.27_± 0.61 85.98±1.6185.98_± 1.61 83.23±2.1683.23_± 2.16 +6.71+6.71 +3.96+3.96 CruxEval-I 70.50±1.5470.50_± 1.54 68.08±1.7768.08_± 1.77 71.58±1.9171.58_± 1.91 −2.42-2.42 +1.08+1.08 CruxEval-O 32.08±1.9832.08_± 1.98 40.12±0.0040.12_± 0.00 44.46±0.4744.46_± 0.47 +8.04+8.04 +12.38+12.38 LiveCodeBench 32.30±0.9332.30_± 0.93 34.59±0.9034.59_± 0.90 33.50±0.1633.50_± 0.16 +2.29+2.29 +1.20+1.20 The gains are broad, not concentrated Table 5 decomposes each pass@1616 gain into per-problem paired changes, showing that the improvements are spread across many problems rather than driven by a handful. SFT data Benchmark #gained #lost #unchanged median Δ (p) mean Δ (p) 95% CI OMI AIME 31 14 75 +0.00 +3.07 [+0.77, +5.59] OMI MATH-500 L5 25 15 94 +0.00 +2.74 [-1.24, +6.47] OMI OMEGA-500 75 77 348 +0.00 -0.20 [-2.33, +1.80] BigCode MBPP+ 39 19 320 +0.00 +4.14 [+2.12, +6.35] BigCode HumanEval+ 21 11 132 +0.00 +3.46 [+0.00, +6.91] BigCode CruxEval-I 157 89 554 +0.00 +4.08 [+2.12, +6.12] BigCode CruxEval-O 274 44 482 +0.00 +16.79 [+14.54, +19.04] BigCode LiveCodeBench 72 41 499 +0.00 +2.72 [+1.09, +4.41] Magicoder MBPP+ 42 19 317 +0.00 +3.35 [+1.06, +5.73] Magicoder HumanEval+ 13 10 141 +0.00 +0.81 [-2.03, +3.86] Magicoder CruxEval-I 196 82 522 +0.00 +8.33 [+6.17, +10.46] Magicoder CruxEval-O 183 48 569 +0.00 +9.83 [+7.92, +11.83] Magicoder LiveCodeBench 56 41 515 +0.00 +1.42 [-0.22, +3.05] OCI MBPP+ 22 15 341 +0.00 +1.15 [-0.35, +2.60] OCI HumanEval+ 8 17 139 +0.00 -2.74 [-5.18, -0.51] OCI CruxEval-I 134 69 597 +0.00 +3.50 [+2.04, +4.92] OCI CruxEval-O 132 57 611 +0.00 +4.33 [+2.83, +5.79] OCI LiveCodeBench 45 65 502 +0.00 -1.09 [-2.61, +0.44] Table 5: Per-problem paired changes in pass@1616 for TailSFT versus Standard SFT at the main-table settings. A problem counts as gained or lost only when its paired pass@1616 strictly changes; unchanged includes ceiling and floor ties. The final column is a paired bootstrap 95%95\% interval for the mean per-problem change. D.1.1 Code Model and optimization Code SFT trains at sequence length 20482048, which effectively never truncates training targets. Runs use no weight decay and a per-rank microbatch of 11 with 44 gradient-accumulation steps, for an effective batch of 3232 and an 88-example selection batch. The learning rate is searched over 1,2,3×10−5\1,2,3\× 10^-5. Each dataset uses a fixed epoch count. SFT data Source Train / val Magicoder ise-uiuc/Magicoder-OSS-Instruct-75K 33,817 / 342 BigCode bigcode/self-oss-instruct-sc2-exec-filter-50k 49,260 / 498 OCI nvidia/OpenCodeInstruct 72,635 / 365 Table 6: Code SFT datasets. Row counts are after decontamination, format conversion, and initialization-loss annotation. Magicoder We keep the Python subset of ise-uiuc/Magicoder-OSS-Instruct-75K (38,28438,284 of 75,19775,197 rows). We drop non-Python rows, rows without extractable code, rows matching an MBPP+ or HumanEval+ signature, and rows containing unit tests, and discard solutions that fail to parse as an abstract syntax tree. The assistant target starts with the prefix Here is the completed function: ‘python , followed by the extracted code and a closing fence, matching the MBPP+ and HumanEval+ answer format. One further row is removed because it does not fit the 20482048-token training tokenization, leaving 33,81733,817 train rows. BigCode For bigcode/self-oss-instruct-sc2-exec-filter-50k (50,66150,661 raw rows) the user message is the instruction and the target uses the same evalplus-aligned prefix and fenced Python format as Magicoder. We remove exact and near MBPP+ and HumanEval+ contamination, benchmark-signature contamination, rows containing tests, and non-Python rows, leaving 49,26049,260 train rows. OCI For nvidia/OpenCodeInstruct we load five evenly spaced shards and keep rows with average_test_score=1.0=1.0. We remove MBPP+ contamination by exact prompt match, by test-assertion substring match against inputs, outputs, and unit tests, and by MBPP+ function-name definitions appearing in outputs, then shuffle with seed 4242 and hold out a 0.5%0.5\% validation split. OCI keeps its native format, with the user message the OCI input and the target the OCI output, both verbatim. Hyperparameter configurations We vary the learning rate and filtering configuration as summarized in Table 7. The configurations used in Table 1 and the additional evaluated configurations are reported across several tables. Tables 8 and 9 give MBPP+ pass@1 and pass@16 for every completed BigCode and Magicoder configuration, with the change against the no-filter run at the same learning rate. Hyperparameter Values Base model OLMo-3 7B base Learning rate 1,2,3×10−5\1,2,3\× 10^-5 Filter schedule none; static; ramp →⁣⋅0\!→\!· Filter fraction f 00 (none); static 0.125,0.25,0.5\0.125,0.25,0.5\; ramp →0.50\!→\!0.5 Epochs fixed per dataset (44 / 55 / 33 for BigCode / Magicoder / OCI) Sequence length 20482048 Effective batch 3232 (8×1×48× 1× 4) Warmup / schedule 3%3\% linear warmup, then constant Optimizer AdamW (0.9,0.95)(0.9,0.95), wd 00, clip 1.01.0 Table 7: Code SFT hyperparameter grid over learning rates and filtering configurations. Filter LR pass@1 Δ pass@16 Δ base (no SFT) — 36.82± 0.5636.82\,±\,0.56 — 75.93± 1.0675.93\,±\,1.06 — no filter 1×10−51× 10^-5 47.45± 0.8247.45\,±\,0.82 — 79.45± 0.8179.45\,±\,0.81 — static f=0.125f=0.125 1×10−51× 10^-5 47.00± 0.6047.00\,±\,0.60 −0.45-0.45 79.81± 0.8179.81\,±\,0.81 +0.36+0.36 ramp 0→0.50→0.5 1×10−51× 10^-5 45.62± 0.5745.62\,±\,0.57 −1.83-1.83 79.72± 0.8579.72\,±\,0.85 +0.27+0.27 no filter 2×10−52× 10^-5 52.58± 0.1152.58\,±\,0.11 — 76.72± 1.0676.72\,±\,1.06 — static f=0.125f=0.125 2×10−52× 10^-5 52.07± 0.7052.07\,±\,0.70 −0.51-0.51 77.51± 0.4677.51\,±\,0.46 +0.79+0.79 static f=0.25f=0.25 2×10−52× 10^-5 51.36± 0.1451.36\,±\,0.14 −1.22-1.22 78.84± 1.0678.84\,±\,1.06 +2.12+2.12 static f=0.5f=0.5 2×10−52× 10^-5 47.22± 0.5447.22\,±\,0.54 −5.36-5.36 77.95± 0.7677.95\,±\,0.76 +1.23+1.23 ramp 0→0.50→0.5 2×10−52× 10^-5 51.65± 0.1851.65\,±\,0.18 −0.93-0.93 78.66± 0.4078.66\,±\,0.40 +1.94+1.94 no filter 3×10−53× 10^-5 53.04± 0.9353.04\,±\,0.93 — 74.69± 1.1074.69\,±\,1.10 — static f=0.125f=0.125 3×10−53× 10^-5 51.90± 0.5951.90\,±\,0.59 −1.14-1.14 75.40± 1.0675.40\,±\,1.06 +0.71+0.71 ramp 0→0.50→0.5 3×10−53× 10^-5 50.67± 0.2650.67\,±\,0.26 −2.37-2.37 75.93± 0.4675.93\,±\,0.46 +1.24+1.24 Table 8: BigCode MBPP+ (n=378n=378) SFT filtering grid, 44 epochs. Values are percent, mean ± sample standard deviation over three seeds; Δ is the change against the no-filter run at the same learning rate. The base (no SFT) row is the untuned OLMo-3 7B base under the same MBPP+ protocol (format-robust EvalPlus stops). Filter LR pass@1 Δ pass@16 Δ base (no SFT) — 36.82± 0.5636.82\,±\,0.56 — 75.93± 1.0675.93\,±\,1.06 — no filter 1×10−51× 10^-5 53.14± 0.5253.14\,±\,0.52 — 80.69± 0.2680.69\,±\,0.26 — static f=0.125f=0.125 1×10−51× 10^-5 52.19± 0.2552.19\,±\,0.25 −0.95-0.95 79.98± 0.6179.98\,±\,0.61 −0.71-0.71 static f=0.25f=0.25 1×10−51× 10^-5 51.51± 0.0451.51\,±\,0.04 −1.63-1.63 80.42± 1.4080.42\,±\,1.40 −0.27-0.27 static f=0.5f=0.5 1×10−51× 10^-5 50.00± 0.5950.00\,±\,0.59 −3.14-3.14 79.45± 1.5579.45\,±\,1.55 −1.24-1.24 no filter 2×10−52× 10^-5 55.01± 0.3655.01\,±\,0.36 — 75.13± 1.1575.13\,±\,1.15 — static f=0.125f=0.125 2×10−52× 10^-5 54.83± 0.5254.83\,±\,0.52 −0.18-0.18 77.51± 0.5377.51\,±\,0.53 +2.38+2.38 static f=0.25f=0.25 2×10−52× 10^-5 54.60± 0.5554.60\,±\,0.55 −0.41-0.41 78.66± 1.2578.66\,±\,1.25 +3.53+3.53 static f=0.5f=0.5 2×10−52× 10^-5 51.92± 0.4151.92\,±\,0.41 −3.09-3.09 79.72± 0.3179.72\,±\,0.31 +4.59+4.59 ramp 0→0.50→0.5 2×10−52× 10^-5 54.73± 0.7354.73\,±\,0.73 −0.28-0.28 78.31± 1.4778.31\,±\,1.47 +3.18+3.18 no filter 3×10−53× 10^-5 55.35± 0.4755.35\,±\,0.47 — 75.31± 0.9375.31\,±\,0.93 — static f=0.125f=0.125 3×10−53× 10^-5 54.75± 0.5854.75\,±\,0.58 −0.60-0.60 76.54± 0.1576.54\,±\,0.15 +1.23+1.23 static f=0.25f=0.25 3×10−53× 10^-5 52.94± 0.6752.94\,±\,0.67 −2.41-2.41 75.93± 1.0675.93\,±\,1.06 +0.62+0.62 static f=0.5f=0.5 3×10−53× 10^-5 51.14± 0.4751.14\,±\,0.47 −4.21-4.21 77.95± 1.3677.95\,±\,1.36 +2.64+2.64 Table 9: Magicoder MBPP+ (n=378n=378) SFT filtering grid, 55 epochs. Values are percent, mean ± sample standard deviation over three seeds; Δ is the change against the no-filter run at the same learning rate. The base (no SFT) row is the same untuned base as in Table 8. For OCI, Table 10 reports the no-filter Standard configuration and the TailSFT ramp configuration used in Table 1. MBPP+ coverage improves under filtering (+1.15+1.15 p), while HumanEval+ loses coverage (−2.74-2.74 p). Accordingly, OCI HumanEval+ has ρ16=0.23 _16=0.23, outside the ρ16>1 _16>1 regime identified by the coverage-ratio diagnostic. Benchmark Filter pass@1 Δ pass@16 Δ MBPP+ (n=378n=378) base (no SFT) 36.82± 0.5636.82\,±\,0.56 — 75.93± 1.0675.93\,±\,1.06 — MBPP+ (n=378n=378) no filter 58.26± 0.8858.26\,±\,0.88 — 81.22± 0.7581.22\,±\,0.75 — MBPP+ (n=378n=378) ramp 0→0.50→0.5 55.81± 0.0655.81\,±\,0.06 −2.44-2.44 82.36± 0.8182.36\,±\,0.81 +1.15+1.15 HumanEval+ (n=164n=164) base (no SFT) 32.94± 1.1132.94\,±\,1.11 — 79.27± 0.6179.27\,±\,0.61 — HumanEval+ (n=164n=164) no filter 54.76± 1.1654.76\,±\,1.16 — 85.98± 1.6185.98\,±\,1.61 — HumanEval+ (n=164n=164) ramp 0→0.50→0.5 51.94± 1.8351.94\,±\,1.83 −2.82-2.82 83.23± 2.1683.23\,±\,2.16 −2.74-2.74 Table 10: OCI MBPP+ and HumanEval+ SFT grid, 33 epochs, learning rate 2×10−52× 10^-5; Standard is no filtering and TailSFT is the ramp →0.50\!→\!0.5. Values are percent, mean ± sample standard deviation; Δ is the change against no filtering. The base (no SFT) rows are the untuned base under the same per-benchmark protocol. Coverage holds across benchmarks and can repair coverage collapse Tables 11–14 report every filtered configuration we evaluated on HumanEval+, CruxEval-I, CruxEval-O, and LiveCodeBench, against the no-filter run for the same dataset. The largest coverage gains land where standard SFT had driven coverage below the base model: on CruxEval-I standard SFT falls 88–1111 p under the base (→61.770.5\!→\!61.7 for BigCode, 59.859.8 for Magicoder), so much of TailSFT’s advantage is repairing a coverage collapse that ordinary SFT introduced. Filter LR pass@1 Δ pass@16 Δ base (no SFT) — 32.94± 1.1132.94\,±\,1.11 — 79.27± 0.6179.27\,±\,0.61 — BigCode no filter 3×10−53× 10^-5 45.06± 0.5745.06\,±\,0.57 — 76.63± 0.7076.63\,±\,0.70 — static f=0.125f=0.125 2×10−52× 10^-5 48.08± 0.1248.08\,±\,0.12 +3.02+3.02 79.47± 1.7679.47\,±\,1.76 +2.85+2.85 static f=0.25f=0.25 2×10−52× 10^-5 46.33± 1.0746.33\,±\,1.07 +1.27+1.27 80.08± 0.9380.08\,±\,0.93 +3.46+3.46 static f=0.5f=0.5 2×10−52× 10^-5 44.08± 0.4844.08\,±\,0.48 −0.98-0.98 80.69± 1.2780.69\,±\,1.27 +4.07+4.07 Magicoder no filter 3×10−53× 10^-5 48.49± 0.5848.49\,±\,0.58 — 77.85± 0.7077.85\,±\,0.70 — static f=0.25f=0.25 2×10−52× 10^-5 47.69± 0.1947.69\,±\,0.19 −0.80-0.80 78.66± 1.8378.66\,±\,1.83 +0.81+0.81 static f=0.5f=0.5 2×10−52× 10^-5 46.60± 0.6646.60\,±\,0.66 −1.89-1.89 80.49± 1.6180.49\,±\,1.61 +2.64+2.64 Table 11: HumanEval+ (n=164n=164) SFT filtering grid. Epochs are 44 (BigCode) and 55 (Magicoder); OCI HumanEval+ appears in Table 10. Values are percent, mean ± sample standard deviation over three seeds; Δ is the change against the no-filter run for the same dataset; the base (no SFT) row is the untuned OLMo-3 7B base under the same protocol. Filter LR pass@1 Δ pass@16 Δ base (no SFT) — 25.67± 0.6225.67\,±\,0.62 — 70.50± 1.5470.50\,±\,1.54 — BigCode no filter 3×10−53× 10^-5 29.79± 0.4829.79\,±\,0.48 — 61.71± 2.2761.71\,±\,2.27 — static f=0.125f=0.125 2×10−52× 10^-5 30.65± 0.2630.65\,±\,0.26 +0.86+0.86 63.54± 0.7663.54\,±\,0.76 +1.83+1.83 static f=0.25f=0.25 2×10−52× 10^-5 30.43± 0.1530.43\,±\,0.15 +0.64+0.64 65.79± 0.2665.79\,±\,0.26 +4.08+4.08 static f=0.5f=0.5 2×10−52× 10^-5 29.76± 0.3529.76\,±\,0.35 −0.03-0.03 69.29± 1.9269.29\,±\,1.92 +7.58+7.58 Magicoder no filter 3×10−53× 10^-5 28.50± 0.1328.50\,±\,0.13 — 59.75± 0.6259.75\,±\,0.62 — static f=0.25f=0.25 2×10−52× 10^-5 26.48± 0.4526.48\,±\,0.45 −2.02-2.02 68.08± 0.9468.08\,±\,0.94 +8.33+8.33 static f=0.5f=0.5 2×10−52× 10^-5 25.63± 0.5625.63\,±\,0.56 −2.88-2.88 69.79± 0.3869.79\,±\,0.38 +10.04+10.04 OCI no filter 2×10−52× 10^-5 29.46± 0.4629.46\,±\,0.46 — 68.08± 1.7768.08\,±\,1.77 — ramp 0→0.50→0.5 2×10−52× 10^-5 30.17± 0.6430.17\,±\,0.64 +0.71+0.71 71.58± 1.9171.58\,±\,1.91 +3.50+3.50 Table 12: CruxEval-I (n=800n=800) SFT filtering grid. Epochs are 44 / 55 / 33 for BigCode / Magicoder / OCI. Values are percent, mean ± sample standard deviation over three seeds; Δ is the change against the no-filter run for the same dataset; the base (no SFT) row is the untuned OLMo-3 7B base under the same protocol. Filter LR pass@1 Δ pass@16 Δ base (no SFT) — 4.22± 0.744.22\,±\,0.74 — 32.08± 1.9832.08\,±\,1.98 — BigCode no filter 3×10−53× 10^-5 4.16± 0.444.16\,±\,0.44 — 24.21± 2.3824.21\,±\,2.38 — static f=0.125f=0.125 2×10−52× 10^-5 12.02± 0.1112.02\,±\,0.11 +7.86+7.86 39.75± 0.7639.75\,±\,0.76 +15.54+15.54 static f=0.25f=0.25 2×10−52× 10^-5 13.18± 0.3013.18\,±\,0.30 +9.02+9.02 41.00± 1.3541.00\,±\,1.35 +16.79+16.79 static f=0.5f=0.5 2×10−52× 10^-5 7.35± 0.377.35\,±\,0.37 +3.20+3.20 36.54± 0.1436.54\,±\,0.14 +12.33+12.33 Magicoder no filter 3×10−53× 10^-5 12.86± 0.1912.86\,±\,0.19 — 38.08± 0.9438.08\,±\,0.94 — static f=0.25f=0.25 2×10−52× 10^-5 16.16± 0.5316.16\,±\,0.53 +3.30+3.30 47.92± 1.6647.92\,±\,1.66 +9.83+9.83 static f=0.5f=0.5 2×10−52× 10^-5 10.12± 0.3710.12\,±\,0.37 −2.74-2.74 41.67± 0.6341.67\,±\,0.63 +3.58+3.58 OCI no filter 2×10−52× 10^-5 9.30± 0.499.30\,±\,0.49 — 40.12± 0.0040.12\,±\,0.00 — ramp 0→0.50→0.5 2×10−52× 10^-5 10.84± 0.3710.84\,±\,0.37 +1.54+1.54 44.46± 0.4744.46\,±\,0.47 +4.33+4.33 Table 13: CruxEval-O (n=800n=800) SFT filtering grid. Epochs are 44 / 55 / 33 for BigCode / Magicoder / OCI. Values are percent, mean ± sample standard deviation over three seeds; Δ is the change against the no-filter run for the same dataset; the base (no SFT) row is the untuned OLMo-3 7B base under the same protocol. Filter LR pass@1 Δ pass@16 Δ base (no SFT) — 5.50± 0.425.50\,±\,0.42 — 32.30± 0.9332.30\,±\,0.93 — BigCode no filter 3×10−53× 10^-5 10.74± 0.4610.74\,±\,0.46 — 30.39± 0.8630.39\,±\,0.86 — static f=0.125f=0.125 2×10−52× 10^-5 11.77± 0.4411.77\,±\,0.44 +1.03+1.03 31.54± 0.7131.54\,±\,0.71 +1.14+1.14 static f=0.25f=0.25 2×10−52× 10^-5 11.49± 0.3811.49\,±\,0.38 +0.75+0.75 33.12± 0.3433.12\,±\,0.34 +2.72+2.72 static f=0.5f=0.5 2×10−52× 10^-5 11.45± 0.4211.45\,±\,0.42 +0.71+0.71 33.39± 0.2533.39\,±\,0.25 +3.00+3.00 Magicoder no filter 3×10−53× 10^-5 14.84± 0.1114.84\,±\,0.11 — 31.81± 0.3431.81\,±\,0.34 — static f=0.25f=0.25 2×10−52× 10^-5 14.32± 0.3814.32\,±\,0.38 −0.51-0.51 33.22± 0.3433.22\,±\,0.34 +1.42+1.42 static f=0.5f=0.5 2×10−52× 10^-5 14.81± 0.2214.81\,±\,0.22 −0.02-0.02 34.48± 1.2834.48\,±\,1.28 +2.67+2.67 OCI no filter 2×10−52× 10^-5 12.24± 0.3612.24\,±\,0.36 — 34.59± 0.9034.59\,±\,0.90 — ramp 0→0.50→0.5 2×10−52× 10^-5 11.49± 0.8411.49\,±\,0.84 −0.75-0.75 33.50± 0.1633.50\,±\,0.16 −1.09-1.09 Table 14: LiveCodeBench (n=612n=612) SFT filtering grid. Epochs are 44 / 55 / 33 for BigCode / Magicoder / OCI. Values are percent, mean ± sample standard deviation over three seeds; Δ is the change against the no-filter run for the same dataset; the base (no SFT) row is the untuned OLMo-3 7B base under the same protocol. Evaluation protocol We evaluate with the OLMES (Gu et al., 2025) EvalPlus configuration used across the OLMo-3 code suite, so the appendix numbers are directly comparable to that standard. Every code result is sampled at temperature 1.01.0, top-p=1.0p=1.0, with 1616 generations per problem for each of three independent evaluation seed sets; throughout the appendix, reported means and standard deviations are computed over these three seed sets, which we refer to simply as three seeds. MBPP+ uses the 378378-problem EvalPlus test split with the assistant prefix Here is the completed function: ‘python , maximum generation length 20482048, context length 40964096, and stop strings ‘, newline-triple-quote, newline-assert, and newline-comment. HumanEval+ uses the 164164-problem EvalPlus split with the same prefix, generation length 10241024, and context 40964096. CruxEval-I and CruxEval-O use 800800 instances each with OLMo-3 multiturn few-shot prompts (two shots for input prediction, one for output), generation length 512512, and context 40964096. LiveCodeBench uses 612612 code-generation problems with an expert-Python system prompt, generation length 81928192, and context 3276832768. We report pass@k with the standard unbiased estimator implemented in OLMES, pass@k=1−(n−ck)/(nk)pass@k=1- n-ck / nk for a document with c correct completions out of n samples, averaged over documents rather than as a best-of-k maximum. With n=16n=16 samples this makes pass@1 the mean per-document pass rate and pass@16 the fraction of documents with at least one correct completion. Each generation length sits above the lengths these tasks actually decode to, so completions are not cut off. SFT data Benchmark Standard p@1 TailSFT p@1 Standard p@16 TailSFT p@16 BigCode MBPP+ (n=378n=378) 53.04±0.9353.04_± 0.93 51.36±0.1451.36_± 0.14 74.69±1.1074.69_± 1.10 78.84±1.0678.84_± 1.06 BigCode HumanEval+ (n=164n=164) 45.06±0.5745.06_± 0.57 46.33±1.0746.33_± 1.07 76.63±0.7076.63_± 0.70 80.08±0.9380.08_± 0.93 BigCode CruxEval-I (n=800n=800) 29.79±0.4829.79_± 0.48 30.43±0.1530.43_± 0.15 61.71±2.2761.71_± 2.27 65.79±0.2665.79_± 0.26 BigCode CruxEval-O (n=800n=800) 4.16±0.444.16_± 0.44 13.18±0.3013.18_± 0.30 24.21±2.3824.21_± 2.38 41.00±1.3541.00_± 1.35 BigCode LiveCodeBench (n=612n=612) 10.74±0.4610.74_± 0.46 11.49±0.3811.49_± 0.38 30.39±0.8630.39_± 0.86 33.12±0.3433.12_± 0.34 Magicoder MBPP+ (n=378n=378) 55.35±0.4755.35_± 0.47 54.60±0.5554.60_± 0.55 75.31±0.9375.31_± 0.93 78.66±1.2578.66_± 1.25 Magicoder HumanEval+ (n=164n=164) 48.49±0.5848.49_± 0.58 47.69±0.1947.69_± 0.19 77.85±0.7077.85_± 0.70 78.66±1.8378.66_± 1.83 Magicoder CruxEval-I (n=800n=800) 28.50±0.1328.50_± 0.13 26.48±0.4526.48_± 0.45 59.75±0.6259.75_± 0.62 68.08±0.9468.08_± 0.94 Magicoder CruxEval-O (n=800n=800) 12.86±0.1912.86_± 0.19 16.16±0.5316.16_± 0.53 38.08±0.9438.08_± 0.94 47.92±1.6647.92_± 1.66 Magicoder LiveCodeBench (n=612n=612) 14.84±0.1114.84_± 0.11 14.32±0.3814.32_± 0.38 31.81±0.3431.81_± 0.34 33.22±0.3433.22_± 0.34 OCI CruxEval-I (n=800n=800) 29.46±0.4629.46_± 0.46 30.17±0.6430.17_± 0.64 68.08±1.7768.08_± 1.77 71.58±1.9171.58_± 1.91 OCI CruxEval-O (n=800n=800) 9.30±0.499.30_± 0.49 10.84±0.3710.84_± 0.37 40.12±0.0040.12_± 0.00 44.46±0.4744.46_± 0.47 OCI LiveCodeBench (n=612n=612) 12.24±0.3612.24_± 0.36 11.49±0.8411.49_± 0.84 34.59±0.9034.59_± 0.90 33.50±0.1633.50_± 0.16 Table 15: Post-SFT code results, matching Table 1. Values are percentages, mean ± sample standard deviation over three seeds. The coverage advantage tends to widen with k Table 16 reports the TailSFT minus Standard SFT pass@k gap at k∈1,2,4,8,16k∈\1,2,4,8,16\ for every benchmark. On most cells the gap starts near zero (or slightly negative) at k=1k=1 and tends to increase with k—most strikingly CruxEval-O, where BigCode grows from +9.02+9.02 to +16.79+16.79—so the advantage is a coverage effect rather than a pass@1 shift. The two OCI transfer benchmarks (HumanEval+ and LiveCodeBench) are the main exception, where TailSFT trails at every k. Table 16: SFT pass@k deltas for code benchmarks, reported as TailSFT minus Standard SFT in percentage points at the main-table settings. Deltas are computed with the unbiased OLMES (Gu et al., 2025) pass@k estimator. SFT data Benchmark Δ @1 Δ @2 Δ @4 Δ @8 Δ @16 BigCode MBPP+ (n=378n=378) −1.68-1.68 +0.64+0.64 +2.19+2.19 +3.19+3.19 +4.14+4.14 HumanEval+ (n=164n=164) +1.27+1.27 +3.37+3.37 +4.34+4.34 +4.44+4.44 +3.46+3.46 CruxEval-I (n=800n=800) +0.64+0.64 +1.61+1.61 +2.57+2.57 +3.25+3.25 +4.08+4.08 CruxEval-O (n=800n=800) +9.02+9.02 +12.58+12.58 +15.45+15.45 +16.95+16.95 +16.79+16.79 LiveCodeBench (n=612n=612) +0.75+0.75 +1.39+1.39 +1.86+1.86 +2.21+2.21 +2.72+2.72 Magicoder MBPP+ (n=378n=378) −0.75-0.75 +1.73+1.73 +2.95+2.95 +3.25+3.25 +3.35+3.35 HumanEval+ (n=164n=164) −0.80-0.80 +0.81+0.81 +1.55+1.55 +1.64+1.64 +0.81+0.81 CruxEval-I (n=800n=800) −2.02-2.02 −0.10-0.10 +2.36+2.36 +5.28+5.28 +8.33+8.33 CruxEval-O (n=800n=800) +3.30+3.30 +4.97+4.97 +6.70+6.70 +8.38+8.38 +9.83+9.83 LiveCodeBench (n=612n=612) −0.51-0.51 +0.30+0.30 +0.80+0.80 +1.08+1.08 +1.42+1.42 OCI MBPP+ (n=378n=378) −2.44-2.44 −1.21-1.21 −0.17-0.17 +0.65+0.65 +1.15+1.15 HumanEval+ (n=164n=164) −2.82-2.82 −2.38-2.38 −1.89-1.89 −2.02-2.02 −2.74-2.74 CruxEval-I (n=800n=800) +0.71+0.71 +1.54+1.54 +2.46+2.46 +3.19+3.19 +3.50+3.50 CruxEval-O (n=800n=800) +1.54+1.54 +2.33+2.33 +3.14+3.14 +3.86+3.86 +4.33+4.33 LiveCodeBench (n=612n=612) −0.75-0.75 −0.86-0.86 −0.91-0.91 −0.91-0.91 −1.09-1.09 D.1.2 Math Model and optimization Math SFT trains at sequence length 40964096 with weight decay 0.10.1, a per-rank microbatch of 22 with 44 gradient-accumulation steps, for an effective batch of 6464 and a 1616-example selection batch. The learning rate is fixed at 3×10−53× 10^-5. We train for 22 epochs and evaluate the final checkpoint. Two passes fit the ∼350 \!350k-example subset without the overfitting seen at higher counts, and the count is held identical across every arm so differences come only from filtering. Each supervised example is a two-turn chat. The user message is the problem followed by the instruction “Present the answer in LaTex format: Your answer”, and the assistant target is the provided chain-of-thought solution. Data The OpenMathInstruct-2 (OMI) subset is built from the OMI train_1M shards by keeping problem_source in math, augmented_math, exact-match decontaminating against MATH-500 problem strings, dropping rows whose problem plus solution exceeds 20482048 tokens under the OLMo-3 tokenizer, and sampling a 350350k-example stratified subset with a 95/595/5 train/validation split. No MATH-500 rows were found during decontamination. Since rows over 20482048 tokens were removed during data construction, the 40964096-token training sequence length does not truncate supervised targets. The final SFT parquets add init_ce, the base-model per-example cross-entropy used by the filtering rule. Dataset Source/filter Train rows Validation rows OMI math subset OpenMathInstruct-2 math/augmented_math, MATH-500 decontaminated 332,500 17,500 Table 17: In-domain math SFT data. The annotated training split contains 326,725326,725 augmented_math and 5,7755,775 math rows; the validation split contains 17,19217,192 and 308308, respectively. Hyperparameter search and selection For math we fixed the learning rate at 3×10−53× 10^-5 and swept the filtering configuration over no filtering, static f∈0.0625,0.125,0.1875,0.25,0.5f∈\0.0625,0.125,0.1875,0.25,0.5\, and the linear ramps →f0\!→\!f for the same endpoints. The optimizer, sequence length, batch size, and 22-epoch schedule are held fixed as shown in Table 18. Table 1 then reports the no-filter Standard checkpoint against the selected TailSFT checkpoint. We use the ramp →0.50\!→\!0.5 arm. Hyperparameter Values Base model OLMo-3 7B base Learning rate fixed at 3×10−53× 10^-5 Filter schedule none; static; ramp →⁣⋅0\!→\!· Filter fraction f 00 (none); static 0.0625,0.125,0.1875,0.25,0.5\0.0625,0.125,0.1875,0.25,0.5\; ramp →0.0625,0.125,0.1875,0.25,0.50\!→\!\0.0625,0.125,0.1875,0.25,0.5\ Epochs 22 Sequence length 40964096 Effective batch 6464 (8×2×48× 2× 4) Warmup schedule 3%3\% linear warmup, then constant Optimizer AdamW (0.9,0.95)(0.9,0.95), wd 0.10.1, clip 1.01.0 Table 18: Math SFT hyperparameter grid. Per-benchmark grids Tables 19–22 report every completed math SFT filtering evaluation in-domain math benchmarks. The broadest grid is MATH-500. Its 500500-problem aggregate (Table 19) sits near ceiling, so Tables 20 and 21 break the same sweep out by the five MATH difficulty levels. No-filter pass@16 is 100%100\% at Level 1 and stays above 96%96\% through Level 3, so the easy levels dominate the aggregate and hide where filtering acts. The unreached coverage, and the gains from filtering, concentrate at the hardest levels: at Level 5 filtering matches or improves no-filter pass@16 in 88 of 1010 arms and strictly improves it in 77, while the near-saturated levels move little in either direction. No MATH-500 pass@16 cell drops by more than 1.81.8 p, inside the seed noise. The completed AIME rows all improve pass@16, while the frozen OMEGA-500 arm is essentially tied with no filtering. Filter pass@1 Δ pass@16 Δ no filter 53.55± 0.4953.55\,±\,0.49 — 86.27± 0.6186.27\,±\,0.61 — static f=0.0625f=0.0625 53.66± 0.2853.66\,±\,0.28 +0.11+0.11 87.00± 0.8787.00\,±\,0.87 +0.73+0.73 static f=0.125f=0.125 53.60± 0.3953.60\,±\,0.39 +0.05+0.05 87.20± 0.4087.20\,±\,0.40 +0.93+0.93 static f=0.1875f=0.1875 53.15± 0.3353.15\,±\,0.33 −0.41-0.41 86.20± 0.6986.20\,±\,0.69 −0.07-0.07 static f=0.25f=0.25 53.51± 0.4153.51\,±\,0.41 −0.04-0.04 86.33± 1.2986.33\,±\,1.29 +0.07+0.07 static f=0.5f=0.5 52.02± 0.1352.02\,±\,0.13 −1.54-1.54 85.60± 0.7285.60\,±\,0.72 −0.67-0.67 ramp 0→0.06250→0.0625 53.80± 0.2953.80\,±\,0.29 +0.25+0.25 85.87± 0.3185.87\,±\,0.31 −0.40-0.40 ramp 0→0.1250→0.125 53.37± 0.7353.37\,±\,0.73 −0.19-0.19 86.47± 0.5086.47\,±\,0.50 +0.20+0.20 ramp 0→0.18750→0.1875 54.13± 0.5054.13\,±\,0.50 +0.57+0.57 87.27± 0.9587.27\,±\,0.95 +1.00+1.00 ramp 0→0.250→0.25 53.91± 0.1953.91\,±\,0.19 +0.35+0.35 86.13± 0.4686.13\,±\,0.46 −0.13-0.13 ramp 0→0.50→0.5 53.59± 0.4153.59\,±\,0.41 +0.03+0.03 86.27± 1.0186.27\,±\,1.01 +0.00+0.00 Table 19: MATH-500 full (n=500n=500) SFT filtering grid, 40964096-token generations. Values are percent, mean ± sample standard deviation over three generation seeds; Δ is the change against no filtering. Filter Level 1 (n=43n=43) Level 2 (n=90n=90) Level 3 (n=105n=105) Level 4 (n=128n=128) Level 5 (n=134n=134) no filter 100.00± 0.00100.00\,±\,0.00 97.78± 1.1197.78\,±\,1.11 96.51± 1.9896.51\,±\,1.98 85.68± 1.6385.68\,±\,1.63 66.67± 1.8866.67\,±\,1.88 static f=0.0625f=0.0625 100.00± 0.00100.00\,±\,0.00 98.15± 0.6498.15\,±\,0.64 96.19± 0.9596.19\,±\,0.95 85.68± 0.9085.68\,±\,0.90 69.40± 3.2569.40\,±\,3.25 static f=0.125f=0.125 100.00± 0.00100.00\,±\,0.00 97.41± 0.6497.41\,±\,0.64 96.83± 1.1096.83\,±\,1.10 85.68± 2.5185.68\,±\,2.51 70.15± 1.2970.15\,±\,1.29 static f=0.1875f=0.1875 100.00± 0.00100.00\,±\,0.00 97.78± 0.0097.78\,±\,0.00 95.56± 1.4695.56\,±\,1.46 84.38± 0.7884.38\,±\,0.78 68.41± 1.8868.41\,±\,1.88 static f=0.25f=0.25 100.00± 0.00100.00\,±\,0.00 97.41± 0.6497.41\,±\,0.64 96.51± 0.5596.51\,±\,0.55 85.94± 3.4085.94\,±\,3.40 66.92± 2.6266.92\,±\,2.62 static f=0.5f=0.5 100.00± 0.00100.00\,±\,0.00 96.30± 1.7096.30\,±\,1.70 95.56± 1.9895.56\,±\,1.98 85.68± 3.2585.68\,±\,3.25 65.92± 4.2465.92\,±\,4.24 ramp 0→0.06250→0.0625 99.22± 1.3499.22\,±\,1.34 97.04± 0.6497.04\,±\,0.64 97.14± 0.9597.14\,±\,0.95 84.38± 0.7884.38\,±\,0.78 66.67± 1.5566.67\,±\,1.55 ramp 0→0.1250→0.125 100.00± 0.00100.00\,±\,0.00 97.04± 1.7097.04\,±\,1.70 96.83± 1.4696.83\,±\,1.46 83.85± 1.1983.85\,±\,1.19 69.40± 2.6969.40\,±\,2.69 ramp 0→0.18750→0.1875 100.00± 0.00100.00\,±\,0.00 97.41± 0.6497.41\,±\,0.64 96.51± 0.5596.51\,±\,0.55 85.68± 1.6385.68\,±\,1.63 70.65± 4.9770.65\,±\,4.97 ramp 0→0.250→0.25 100.00± 0.00100.00\,±\,0.00 97.78± 1.1197.78\,±\,1.11 97.78± 1.4697.78\,±\,1.46 84.90± 0.4584.90\,±\,0.45 65.92± 1.1465.92\,±\,1.14 ramp 0→0.50→0.5 100.00± 0.00100.00\,±\,0.00 96.67± 0.0096.67\,±\,0.00 97.14± 0.0097.14\,±\,0.00 85.42± 1.9785.42\,±\,1.97 67.16± 1.9767.16\,±\,1.97 Table 20: MATH-500 SFT pass@16 by difficulty level, from the same 40964096-token SFT filtering sweep as Table 19. Columns are the five MATH difficulty levels, with the number of problems in each header. Values are percent, mean ± sample standard deviation over three seeds. pass@16 is at or near ceiling through Level 3, so the aggregate is dominated by the easy levels; the gains from filtering concentrate at Levels 4–5. The frozen main table reports the selected ramp arm on Level 5, re-evaluated with 81928192-token generations. Filter Level 1 (n=43n=43) Level 2 (n=90n=90) Level 3 (n=105n=105) Level 4 (n=128n=128) Level 5 (n=134n=134) no filter 85.27± 1.4685.27\,±\,1.46 76.25± 0.4376.25\,±\,0.43 65.95± 1.5965.95\,±\,1.59 47.02± 0.6547.02\,±\,0.65 24.66± 0.6324.66\,±\,0.63 static f=0.0625f=0.0625 85.66± 1.0585.66\,±\,1.05 76.44± 1.1576.44\,±\,1.15 64.98± 1.9364.98\,±\,1.93 46.92± 0.2146.92\,±\,0.21 25.67± 0.7225.67\,±\,0.72 static f=0.125f=0.125 84.98± 0.8984.98\,±\,0.89 76.69± 0.7076.69\,±\,0.70 64.46± 0.7764.46\,±\,0.77 47.28± 0.4747.28\,±\,0.47 25.56± 0.7325.56\,±\,0.73 static f=0.1875f=0.1875 84.79± 0.7584.79\,±\,0.75 75.83± 1.0575.83\,±\,1.05 65.06± 0.5165.06\,±\,0.51 46.14± 1.6546.14\,±\,1.65 25.11± 0.6625.11\,±\,0.66 static f=0.25f=0.25 85.08± 0.6985.08\,±\,0.69 75.62± 0.7775.62\,±\,0.77 65.08± 0.3465.08\,±\,0.34 47.18± 0.8647.18\,±\,0.86 25.51± 0.3325.51\,±\,0.33 static f=0.5f=0.5 83.58± 0.5883.58\,±\,0.58 74.31± 0.9074.31\,±\,0.90 62.38± 1.1162.38\,±\,1.11 46.48± 0.1346.48\,±\,0.13 24.08± 0.0324.08\,±\,0.03 ramp 0→0.06250→0.0625 85.61± 1.5485.61\,±\,1.54 76.06± 0.5176.06\,±\,0.51 66.03± 0.7266.03\,±\,0.72 47.27± 0.3047.27\,±\,0.30 25.30± 0.5625.30\,±\,0.56 ramp 0→0.1250→0.125 84.20± 0.6784.20\,±\,0.67 76.41± 1.2876.41\,±\,1.28 65.24± 1.1765.24\,±\,1.17 47.20± 1.1047.20\,±\,1.10 24.58± 0.8224.58\,±\,0.82 ramp 0→0.18750→0.1875 85.90± 0.5285.90\,±\,0.52 76.39± 0.8476.39\,±\,0.84 65.85± 0.3065.85\,±\,0.30 48.00± 0.0948.00\,±\,0.09 25.65± 1.1025.65\,±\,1.10 ramp 0→0.250→0.25 85.85± 1.2485.85\,±\,1.24 76.44± 0.2676.44\,±\,0.26 65.66± 0.5365.66\,±\,0.53 47.58± 0.3147.58\,±\,0.31 25.37± 0.6925.37\,±\,0.69 ramp 0→0.50→0.5 85.27± 0.2285.27\,±\,0.22 75.93± 0.5475.93\,±\,0.54 65.28± 0.6165.28\,±\,0.61 47.84± 0.7947.84\,±\,0.79 24.75± 1.0224.75\,±\,1.02 Table 21: MATH-500 SFT pass@1 by difficulty level, from the same sweep as Table 20. Values are percent, mean ± sample standard deviation over three seeds. Reading the Level-4 numbers Level 4 is the one subset where filtering neither clearly helps nor hurts, and this is an estimator artifact rather than a real regression. pass@16 is scored as a binary hit per seed and averaged over three seeds, so every per-problem value is quantized to 0,13,23,1\0, 13, 23,1\ and a problem’s filtered-minus-no-filter difference can only be a multiple of 3333 p. The Level-4 mean is then set by a few knife’s-edge problems that the base model already solves on two of three seeds, where a single seed flip moves the subset mean by a point or two. Two such problems account for the whole gap: under the arms that most help Level 5, the Level-4 pass@16 change moves from +0.0+0.0 p on the full subset to about +0.8+0.8 p once those two problems are excluded, in the same direction as Level 5. Level 5 shows a larger and steadier gain because it contains many more genuinely contested problems, so this per-problem quantization averages out. Eval family Filter pass@1 Δ pass@16 Δ base (no SFT)† 5.165.16 — 20.9120.91 — ramp sweep no filter 3.65± 0.373.65\,±\,0.37 — 15.24± 2.0215.24\,±\,2.02 — ramp sweep ramp 0→0.18750→0.1875 3.82± 0.203.82\,±\,0.20 +0.16+0.16 16.06± 1.4216.06\,±\,1.42 +0.82+0.82 ramp sweep ramp 0→0.250→0.25 3.78± 0.203.78\,±\,0.20 +0.13+0.13 15.37± 0.2315.37\,±\,0.23 +0.13+0.13 ramp sweep ramp 0→0.50→0.5 4.03± 0.024.03\,±\,0.02 +0.37+0.37 18.31± 0.5718.31\,±\,0.57 +3.07+3.07 static no filter 3.65± 0.343.65\,±\,0.34 — 15.52± 2.0015.52\,±\,2.00 — static static f=0.125f=0.125 3.75± 0.183.75\,±\,0.18 +0.10+0.10 16.14± 1.7316.14\,±\,1.73 +0.63+0.63 Table 22: AIME 2022–2025 (n=120n=120) completed SFT filtering evaluations. The ramp sweep and the static pair were evaluated in separate runs, each against its own matched no-filter baseline. Values are percent, mean ± sample standard deviation over three seeds; Δ is the change against the matching no-filter row. †The base (no SFT) row is the untuned OLMo-3 7B base under the same AIME protocol. Evaluation protocol We evaluate with the standard OLMES (Gu et al., 2025) math configuration used across the OLMo-3 math suite. AIME contains 3030 problems from each year 20222022–20252025 (n=120n=120), chat prompting with the boxed-answer instruction, temperature 0.60.6, top-p=0.95p=0.95, maximum generation length and model context 81928192, and 3232 generations per problem for each of three seeds. MATH-500 uses the 500500-problem test split; the main table reports the Level-5 subset (n=134n=134), since the full benchmark is near ceiling, with temperature 1.01.0, top-p=1.0p=1.0, maximum generation length 81928192, model context 1638416384, and 1616 generations per problem for each of three seeds. OMEGA-500 (the saumyamalik/omega-500 500500-problem set) uses temperature 1.01.0, top-p=1.0p=1.0, maximum generation length 40964096, model context 81928192, and 1616 generations per problem for each of three seeds. For AIME and MATH-500 we use the stored per-document pass@k metrics. For OMEGA-500, whose aggregate files do not store pass@k, we recompute pass@1 and pass@16 from the 1616 completions using the extracted flexible boxed answer, falling back to the standard extracted answer, and match it against the label. In every case pass@k is the same unbiased estimator implemented in OLMES (pass@k=1−(n−ck)/(nk)pass@k=1- n-ck / nk for c correct of n samples), so AIME’s 3232 samples are combined at k=16k=16 rather than by best-of-k. The sweep of Table 19 uses MATH-500 full (n=500n=500) with 40964096-token generations. SFT data Benchmark Standard p@1 TailSFT p@1 Standard p@16 TailSFT p@16 OMI AIME 2022–2025 (n=120n=120) 3.65±0.373.65_± 0.37 4.03±0.024.03_± 0.02 15.24±2.0215.24_± 2.02 18.31±0.5718.31_± 0.57 OMI MATH-500 Level 5 (n=134n=134) 24.80±0.4024.80_± 0.40 25.16±0.4025.16_± 0.40 66.42±0.0066.42_± 0.00 69.15±0.8669.15_± 0.86 OMI OMEGA-500 (n=500n=500) 6.58±0.506.58_± 0.50 6.51±0.266.51_± 0.26 32.80±2.2532.80_± 2.25 32.60±1.5632.60_± 1.56 Table 23: Post-SFT math results, matching Table 1. Values are percentages, mean ± sample standard deviation over three seeds. The coverage advantage tends to widen with k Table 24 reports the TailSFT minus Standard SFT pass@k gap at k∈1,2,4,8,16k∈\1,2,4,8,16\. On AIME and MATH-500 the gap is small at k=1k=1 and tends to grow with k (AIME +→+3.07+0.37\!→\!+3.07; MATH-500 Level 5 +→+2.74+0.36\!→\!+2.74), while OMEGA-500, where the two models are already matched, stays flat. The math gains therefore come from broader coverage that surfaces at larger k rather than from a shift in greedy accuracy. Table 24: SFT pass@k deltas for math benchmarks, reported as TailSFT minus Standard SFT in percentage points at the main-table settings. Deltas are computed with the unbiased OLMES (Gu et al., 2025) pass@k estimator. Benchmark Δ @1 Δ @2 Δ @4 Δ @8 Δ @16 AIME 2022–2025 (n=120n=120) +0.37+0.37 +0.72+0.72 +1.20+1.20 +1.93+1.93 +3.07+3.07 MATH-500 Level 5 (n=134n=134) +0.36+0.36 +0.71+0.71 +0.83+0.83 +0.87+0.87 +2.74+2.74 OMEGA-500 (n=500n=500) −0.06-0.06 −0.09-0.09 −0.12-0.12 −0.23-0.23 −0.20-0.20 D.2 GRPO The in-domain math and code GRPO runs share one training procedure and differ only in data, reward, hyperparameter grid, and evaluation. We describe the shared procedure here; the math (Section D.2.1) and code (Section D.2.2) subsections give the domain-specific details. Every run is initialized from one of the matched SFT checkpoints evaluated in Table 1: the Standard-SFT run supplies the no-filter start and the paired TailSFT run supplies the filtered start, so the two policies differ only in how their initialization was trained. Table 2 reports a single tuned configuration per domain; the per-domain grids below give the tuned-configuration results in full. Reinforcement learning with GRPO Each run optimizes the initialized policy against a verifiable reward with GRPO (Shao et al., 2024). For every prompt we sample a group of n responses at temperature 1.01.0, score each response with the domain reward, and use the group-relative advantage—each response’s reward centered and scaled within its group—as the policy-gradient signal. We use no KL regularization and no learned reward model. Optimization uses a batch of 128128 prompts, a mini-batch of 128128, and 88-way data parallelism, with the rollout group size n and the actor learning rate searched per domain and all other settings held fixed within a domain. Checkpoints are exported periodically during training and the reported checkpoint is chosen on a held-out validation split, as described per domain. Post-GRPO evaluation uses OLMES (Gu et al., 2025) and reuses the same per-benchmark coverage protocols as Section D.1, with pass@k estimated over three seeds; the exact per-benchmark sampling settings are given below. D.2.1 Math Initialization The Standard start is the no-filter OMI SFT checkpoint and the TailSFT start is the matched ramp →0.50\!→\!0.5 checkpoint (Section D.1.2); these are the two OMI checkpoints reported in Table 1. Data and reward GRPO trains on the MATH training split (the MATH-lighteval release) with the MATH-500 test problems removed by exact-match decontamination, leaving 11,39611,396 training and 600600 validation prompts, so the evaluation set is never seen during RL. Each prompt uses the same boxed-answer instruction as math SFT, and the reward is 11 when a response’s boxed final answer matches the reference under the verifiable math grader and 00 otherwise. Optimization We use a maximum prompt length of 512512, a maximum response length of 40964096, and train for 33 epochs. At rollout group size n=4n=4 we search the actor learning rate; Table 26 reports the tuned configuration (actor learning rate 2×10−52× 10^-5). Doubling the rollout group size to n=8n=8 leaves the ordering unchanged—the TailSFT initialization retains a post-GRPO pass@11 advantage of about +2+2 points on MATH-500 Level 5—so the gain is not an artifact of the group size. Table 25 summarizes the configuration. Setting Value Initialization matched Standard / TailSFT OMI SFT checkpoints (Table 1) Algorithm GRPO, group-relative advantage, no KL Training prompts MATH-lighteval train, MATH-500 removed (11,39611,396 train / 600600 val) Reward verifiable boxed-answer exact match (11/00) Rollout group size n 44 Actor learning rate searched 1,2×10−5\1,2\× 10^-5; reported 2×10−52× 10^-5 Max prompt / response length 512512 / 40964096 Epochs 33 Prompt batch / mini-batch 128128 / 128128 Checkpoint selection mean-best (highest mean validation reward) Evaluation MATH-500 L5 (temp 1.01.0, 1616 samples) & AIME (temp 0.60.6, 3232 samples); 81928192-token, 33 seeds Table 25: Math GRPO configuration. All runs share the fixed settings; only the actor learning rate is searched, and Table 26 reports the tuned configuration. Checkpoint selection and evaluation We select the mean-best checkpoint—the one with the highest mean validation reward over the sampled group—which averages over the small held-out validation split to reduce selection noise. We evaluate on the MATH-500 Level-5 subset (n=134n=134) and AIME 2022–2025 (n=120n=120). Because these are long-form reasoning tasks we use a generation budget of 81928192 tokens and report only these long-generation evaluations, since shorter budgets truncate reasoning traces and understate pass rates; at 81928192 tokens truncation is not a concern, as no AIME generation and fewer than 4%4\% of MATH-500 generations reach the limit. Matching the math SFT protocol (Section D.1.2), MATH-500 Level 5 is sampled at temperature 1.01.0, top-p=1.0p=1.0 with 1616 samples per problem, and AIME at temperature 0.60.6, top-p=0.95p=0.95 with 3232 samples per problem, each over three seeds. At the tuned configuration the TailSFT initialization improves both pass@1 and pass@16 on both benchmarks (Table 26). pass@1 pass@16 Benchmark Standard TailSFT Δ Standard TailSFT Δ MATH-500 Level 5 (n=134n=134) 57.70± 0.4457.70\,±\,0.44 60.26± 1.0260.26\,±\,1.02 +2.56+2.56 83.83± 1.7283.83\,±\,1.72 87.06± 0.8687.06\,±\,0.86 +3.23+3.23 AIME 2022–2025 (n=120n=120) 14.40± 0.2614.40\,±\,0.26 15.61± 0.4415.61\,±\,0.44 +1.21+1.21 32.76± 0.8832.76\,±\,0.88 36.06± 0.9836.06\,±\,0.98 +3.30+3.30 Table 26: Math GRPO at the tuned configuration (rollout n=4n=4, actor learning rate 2×10−52× 10^-5, mean-best checkpoint, 81928192-token generation), evaluated from the matched Standard and TailSFT initializations of Table 1. Values are percent, mean ± standard deviation over three seeds; Δ is the TailSFT improvement. D.2.2 Code Initialization For each dataset the Standard start is the no-filter SFT checkpoint and the TailSFT start is the matched filtered checkpoint from Section D.1.1 (static f=0.25f=0.25 for BigCode and Magicoder, the ramp →0.50\!→\!0.5 for OCI); these are the checkpoints reported in Table 1. Data and reward GRPO trains on the MBPP+ training split with a held-out 100100-problem validation subset. The reward is 11 when a sampled program passes the full MBPP+ unit-test suite for its problem and 00 otherwise. Optimization We use a maximum prompt and response length of 10241024 each and train for 1010 epochs. We search the rollout group size n∈2,4n∈\2,4\ and the actor learning rate ∈5×10−6,1×10−5,2×10−5∈\5× 10^-6,1× 10^-5,2× 10^-5\; Table 28 reports the tuned configuration for each dataset. Table 27 summarizes the configuration. Setting Value Initialization matched Standard / TailSFT SFT checkpoints per dataset (Table 1) Algorithm GRPO, group-relative advantage, no KL Training prompts MBPP+ train (100100-problem held-out validation) Reward full MBPP+ unit-test pass (11/00) Rollout group size n searched 2,4\2,4\; reported 44 Actor learning rate searched 5×10−6,1×10−5,2×10−5\5× 10^-6,1× 10^-5,2× 10^-5\; reported 2×10−52× 10^-5 Max prompt / response length 10241024 / 10241024 Epochs 1010 Prompt batch / mini-batch 128128 / 128128 Checkpoint selection highest validation pass rate Evaluation EvalPlus MBPP+ (n=378n=378); 20482048-token, temp 1.01.0, top-p 1.0p\,1.0, 1616 samples × 33 seeds Table 27: Code GRPO configuration. All runs share the fixed settings; the rollout group size and actor learning rate are searched, and Table 28 reports the tuned configuration for each dataset. Checkpoint selection and evaluation We select the checkpoint with the highest validation pass rate over the small 100100-problem held-out split, applying the same rule to both initializations. Evaluation uses EvalPlus MBPP+ (378378-problem test split) at temperature 1.01.0, top-p=1.0p=1.0, with 1616 samples per problem over three seeds and a 20482048-token generation budget; MBPP+ solutions are short relative to this budget, so generations terminate well within the limit and truncation does not affect the reported rates. At the tuned configurations the TailSFT initialization improves post-GRPO pass@1 while generally retaining its pass@16 coverage advantage (Table 28). pass@1 pass@16 SFT data (TailSFT filter) n LR Standard TailSFT Δ Standard TailSFT Δ BigCode (static f=0.25f=0.25) 44 2×10−52× 10^-5 69.57± 0.2969.57\,±\,0.29 73.50± 0.7073.50\,±\,0.70 +3.93+3.93 75.93± 0.0075.93\,±\,0.00 78.66± 0.1578.66\,±\,0.15 +2.73+2.73 Magicoder (static f=0.25f=0.25) 22 1×10−51× 10^-5 64.08± 0.3264.08\,±\,0.32 66.02± 0.5566.02\,±\,0.55 +1.94+1.94 75.84± 0.3175.84\,±\,0.31 79.28± 0.5579.28\,±\,0.55 +3.44+3.44 Magicoder (static f=0.25f=0.25) 44 2×10−52× 10^-5 70.52± 0.2570.52\,±\,0.25 73.24± 0.0973.24\,±\,0.09 +2.72+2.72 78.22± 0.6178.22\,±\,0.61 80.60± 0.5580.60\,±\,0.55 +2.38+2.38 OCI (ramp 0→0.50→0.5) 44 2×10−52× 10^-5 74.67± 0.0874.67\,±\,0.08 76.30± 0.0576.30\,±\,0.05 +1.62+1.62 84.22± 0.4084.22\,±\,0.40 84.04± 0.1584.04\,±\,0.15 −0.18-0.18 Table 28: Code GRPO (MBPP+, n=378n=378, 20482048-token generation) from the matched Standard and TailSFT initializations of Table 1, at tuned configurations. n is the GRPO rollout group size and LR the actor learning rate. Values are percent, mean ± standard deviation over three seeds; Δ is the TailSFT improvement, with yellow marking a change within one standard deviation. Training efficiency Figure 8 tracks the mean training reward per GRPO step for the Standard and TailSFT initializations. The TailSFT reward rises at least as fast as Standard early in training, and in some settings the early reward increases up to roughly 2.5×2.5× faster. Figure 8: GRPO training reward from Standard and TailSFT initializations. Mean training reward (critic/rewards/mean) versus GRPO step for the four main-table settings. The horizontal dashed line marks the Standard initialization’s first-step reward. TailSFT begins below Standard but recovers Standard’s starting reward within a small fraction of training and rises at least as fast thereafter. D.3 Coverage ratio diagnostic: computation This subsection records exactly how each point in Figure 4 was produced from the evaluation runs, filling in the aggregation and run-selection choices that Section 4.2 leaves implicit. The generation settings (temperature, top-p, generation length, and sample count) for each benchmark are exactly those documented in Sections D.1.1 and D.1.2; we do not re-specify them here. Per-problem scoring and seed aggregation For each problem i and model π we draw at least 1616 samples per evaluation seed, compute the empirical pass@11 and pass@1616 within each seed, and average those per-problem values across the available seeds to form Pi,1(π)P_i,1(π) and Pi,16(π)P_i,16(π); this is the Pi,K(π)P_i,K(π) of Section 4.2. Every run uses three seeds for base, standard, and TailSFT on every benchmark. Horizontal axis (ρ16 _16) The coverage ratio is computed exactly as in Section 4.2 from π0 _0 and πSFT _SFT: the base-reachable set ℛ0=i:0.05<Pi,16(π0)<0.95R_0=\i:0.05<P_i,16( _0)<0.95\ uses the empirical base pass@1616, while L and G use the plug-in f16(Pi,1(⋅))f_16 (P_i,1(·) ) of the empirical pass@11. Standard SFT is the no-filter (f=0f=0) run. The size |ℛ0| _0 of the base-reachable set varies from 1616 (AIME) to 237237 (CruxEval-O) and is listed per point in Table 29. Vertical axis (coverage gain Δ ) The plotted gain is the mean over ℛ0R_0 of the empirical pass@1616 difference Pi,16(πTail)−Pi,16(πSFT)P_i,16( _Tail)-P_i,16( _SFT), in percentage points. The TailSFT and standard runs use the same clip settings surfaced in Table 1: static f=0.25f=0.25 for BigCode and Magicoder, ramp →0.50\!→\!0.5 for OCI, ramp →0.50\!→\!0.5 for AIME, OMEGA-500, and MATH-500 Level 5. Special cases Two points require extra care. For MATH-500 Level 5, the base model was only evaluated at a 40964096-token generation length, so to keep π0 _0 and πSFT _SFT length-matched inside ρ16 _16 we compute ℛ0R_0, L, and G from a 40964096-token base run and a 40964096-token no-filter run, restricted to the 134134 Level-5 problems; the plotted gain Δ still uses the 81928192-token standard and TailSFT runs of Table 1. Recomputing ρ16 _16 from the 81928192-token no-filter run instead leaves it essentially unchanged (2.282.28 vs. 2.042.04, both >1>1), so the point’s placement is robust to this choice. For OMEGA-500, per-problem pass@11 and pass@1616 are recomputed from the stored generations with the same exact-match (flexible) grader used for Table 1, rather than read from cached metric files, so the diagnostic and the main table score the runs identically. Each point is thus a triple (ρ16,Δ,|ℛ0|)( _16, , _0 ); Table 29 lists all eighteen. SFT dataset Benchmark |ℛ0| _0 ρ16 _16 Δ (p) OMI AIME 16 7.607.60 +9.92+9.92 MATH-500 L5 31 2.282.28 +3.23+3.23 OMEGA-500 140 0.900.90 −0.95-0.95 BigCode CruxEval-I 199 1.631.63 +3.02+3.02 CruxEval-O 237 2.872.87 +28.69+28.69 HumanEval+ 29 1.391.39 +9.20+9.20 LiveCodeBench 105 1.371.37 +3.81+3.81 MBPP+ 50 1.301.30 +14.00+14.00 Magicoder CruxEval-I 199 2.262.26 +12.40+12.40 CruxEval-O 237 0.730.73 +18.42+18.42 HumanEval+ 29 1.041.04 −0.00-0.00 LiveCodeBench 105 1.221.22 +1.59+1.59 MBPP+ 50 0.770.77 +11.33+11.33 OCI CruxEval-I 199 1.351.35 +6.37+6.37 CruxEval-O 237 0.550.55 +9.00+9.00 HumanEval+ 29 0.230.23 −6.32-6.32 LiveCodeBench 105 0.970.97 −1.27-1.27 MBPP+ 50 0.540.54 +2.00+2.00 Table 29: Exact values plotted in Figure 4: the base-reachable set size |ℛ0| _0 , the coverage ratio ρ16 _16 (Section 4.2), and the coverage gain Δ (mean empirical pass@1616 difference of TailSFT over standard SFT on ℛ0R_0, in percentage points), for each (SFT dataset, benchmark) pair.

Instant research discovery