Paper deep dive
TRIAGE: Direction-Aware Mismatch Stabilization of Native NVFP4 Reinforcement Learning
Zhen Li, Shuai Zhang, Yanggan Gu, Yiming Zhang, Yang Yu, Mingfa Feng, Congkai Xie, Shuang Yu, Junjie Lai, Hongxia Yang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 10/7/2026, 8:28:38 AM
Summary
The paper introduces TRIAGE, a direction-aware stabilization method for native NVFP4 (Weight-and-Activation 4-bit) Reinforcement Learning in Large Language Models. It addresses the instability caused by learner-sampler mismatch in low-precision execution by distinguishing between mismatch-amplifying and mismatch-contracting policy gradient updates. TRIAGE uses segment-level diagnosis to selectively rebalance updates and applies bounded repair, achieving stable optimization and full-precision-level performance on mathematical reasoning benchmarks while providing up to 2.3x higher rollout throughput compared to BF16.
Entities (11)
Relation Signals (7)
Triage → stabilizes → Policy Optimization
confidence 95% · TRIAGE: Direction-Aware Mismatch Stabilization of Native NVFP4 Reinforcement Learning
Triage → uses → NVFP4
confidence 95% · TRIAGE modifies the optimization objective while retaining native NVFP4 weight-and activation 4-bit (W4A4) forward execution
NVFP4 → enables → W4A4
confidence 90% · NVFP4 is a Blackwell-supported format... native weight-and-activation 4-bit (W4A4)
Qwen3-4B → evaluatedwith → Triage
confidence 90% · Experiments on Qwen3-4B and Qwen3-30B-A3B show stable optimization
Qwen3-30B-A3B → evaluatedwith → Triage
confidence 90% · Experiments on Qwen3-4B and Qwen3-30B-A3B show stable optimization
Triage → improves → rollout throughput
confidence 90% · native NVFP4 with TRIAGE provides up to 2.3x higher rollout throughput than BF16.
GRPO → usedin → Triage
confidence 85% · In native NVFP4 runs... GRPO trajectories... TRIAGE acts on this distinction
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Low-precision execution can substantially accelerate reinforcement learning (RL) for large language models, but discrepancies between learner and sampler execution can destabilize policy optimization. In this paper, we characterize the interaction between mismatch and the policy-gradient direction, distinguishing locally amplifying from contracting update contributions that mismatch magnitude alone cannot identify. In native NVFP4 runs, we observe an early imbalance between the two amplifying regions, favoring negative-advantage, negative-gap updates. Their tail tokens become concentrated in a small fraction of response segments before mismatch spreads globally. Motivated by these findings, we introduce TRIAGE, a direction-aware stabilization method that uses segment-level diagnosis to selectively rebalance policy-gradient updates and applies bounded repair to residual severe mismatch. TRIAGE modifies the optimization objective while retaining native NVFP4 weight-and activation 4-bit (W4A4) forward execution on both the sampler and learner. Experiments on Qwen3-4B and Qwen3-30B-A3B show stable optimization throughout the evaluated training horizon and achieve full precision level performance across five mathematical reasoning benchmarks, while native NVFP4 with TRIAGE provides up to 2.3x higher rollout throughput than BF16.
Tags
Links
- Source: https://arxiv.org/abs/2610.07043v1
- Canonical: https://arxiv.org/abs/2610.07043v1
Trouble viewing inline? Open PDF directly →
Full Text
101,949 characters extracted from source content.
Expand or collapse full text
*These authors contributed equally to this work. TRIAGE: Direction-Aware Mismatch Stabilization of Native NVFP4 Reinforcement Learning Zhen Li Shuai Zhang Yanggan Gu Yiming Zhang Yang Yu Mingfa Feng Congkai Xie Shuang Yu Junjie Lai Hongxia Yang September, 2026 Abstract Low-precision execution can substantially accelerate reinforcement learning (RL) for large language models, but discrepancies between learner and sampler execution can destabilize policy optimization. In this paper, we characterize the interaction between mismatch and the policy-gradient direction, distinguishing locally amplifying from contracting update contributions that mismatch magnitude alone cannot identify. In native NVFP4 runs, we observe an early imbalance between the two amplifying regions, favoring negative-advantage, negative-gap updates. Their tail tokens become concentrated in a small fraction of response segments before mismatch spreads globally. Motivated by these findings, we introduce TRIAGE, a direction-aware stabilization method that uses segment-level diagnosis to selectively rebalance policy-gradient updates and applies bounded repair to residual severe mismatch. TRIAGE modifies the optimization objective while retaining native NVFP4 weight-and-activation 4-bit (W4A4) forward execution on both the sampler and learner. Experiments on Qwen3-4B and Qwen3-30B-A3B show stable optimization throughout the evaluated training horizon and achieve full precision level performance across five mathematical reasoning benchmarks, while native NVFP4 with TRIAGE provides up to 2.3× higher rollout throughput than BF16. 1 Introduction Reinforcement learning (RL) has become central to improving the reasoning capabilities of large language models (LLMs) (DeepSeek-AI, 2025). As RL expands toward long-horizon reasoning, coding, and agentic tasks, increasingly long generation trajectories make rollout a dominant wall-clock bottleneck of the training loop (Qiu et al., 2026; Zhuge et al., 2026; Xie et al., 2025), making efficient rollout a primary systems challenge. NVIDIA Blackwell GPUs provide native weight-and-activation 4-bit (W4A4) Tensor Core operations, offering up to 4×4× the peak matrix-compute throughput of BF16 and making NVFP4 an attractive precision setting for long-sequence rollout (Alvarez et al., 2025; Li et al., 2025). However, trajectories are sampled by one execution policy and optimized using probabilities recomputed by another. Differences between the rollout and training execution paths already induce learner–sampler probability discrepancies, and low-precision execution can further enlarge this mismatch (Zhong et al., 2026; Gu et al., 2026). This creates a trade-off between hardware efficiency and stable policy optimization. Existing approaches mitigate learner–sampler mismatch through two main strategies. Quantization-aware training (QAT) attempts to align the learner with the low-precision sampler (Chen et al., 2025), but standard QAT retains high-precision arithmetic and therefore does not preserve native W4A4 execution on the learner forward path. Importance-based corrections such as truncated importance sampling (TIS), clipping, and masking intervene according to the magnitude of learner–sampler disagreement (Qiu et al., 2026; Li et al., 2026). More fundamentally, magnitude alone describes how far the learner and sampler disagree, not whether the subsequent policy-gradient update will reduce or further amplify that disagreement. This raises a more essential question: what determines whether learner–sampler mismatch contracts or amplifies under policy optimization? Figure 1: Overview of TRIAGE for native NVFP4 RL. In this work, we show that learner–sampler mismatch must be considered together with the direction of the policy update (Figure 1, left). If the learner already assigns a token a lower probability than the sampler, further reducing that probability can widen the mismatch rather than correct it. In native NVFP4 runs, updates of this kind become increasingly prominent relative to those that increase probabilities already above the sampler’s. This imbalance appears before mismatch becomes broadly severe. Tokens with larger discrepancies in this direction also become concentrated within short portions of responses, where they can be hidden by response-level averages. These observations motivate a different stabilization principle: identify and control updates that tend to amplify mismatch, rather than intervening on mismatch magnitude alone. Motivated by these findings, we propose TRIAGE, a direction-aware stabilization framework for native low-precision RL. As illustrated in Figure 1, TRIAGE uses segment-level statistics to identify locally persistent risk, selectively attenuates only mismatch-amplifying −A^- (Advantage << 0, mismatch gap << 0) tokens within risky segments, rebalances update mass, and applies a bounded repair term to residual severe mismatch. By acting on the optimization objective rather than replacing low-precision execution with fake-quantized computation, TRIAGE retains native NVFP4 W4A4 forward execution on both the sampler and learner. Across Qwen3-4B and training-sensitive Qwen3-30B-A3B MoE models, NVFP4 with TRIAGE achieves up to 2.3× the rollout throughput of BF16, while maintaining stable optimization throughout the training horizon and recovering comparable performance on mathematical reasoning benchmarks. Our contributions are summarized as follows: ❶ A directional characterization of learner–sampler mismatch. We characterize mismatch risk through its interaction with the policy-gradient direction and identify a directional, locally concentrated precursor to native NVFP4 instability. ❷ TRIAGE, a selective stabilization mechanism for low-precision RL. TRIAGE combines segment-level risk diagnosis, direction-selective token gating, response-level update rebalancing, and targeted repair to suppress self-amplifying mismatch while preserving contracting updates. ❸ An end-to-end native NVFP4 RL recipe at scale. Across experiments totaling more than 34,500 GPU-hours on B300 GPUs, we validate native W4A4 execution on both sampler and learner forward paths across dense and MoE models, achieving up to 1.3× end-to-end RL speedup while maintaining the BF16-level performance. 2 Preliminaries 2.1 Group-Relative Policy Optimization Group-Relative Policy Optimization (GRPO) (Shao et al., 2024) is widely used in RL for reasoning models. Given a prompt x, GRPO samples a group of G responses yii=1G\y_i\_i=1^G from an old policy πold _old and evaluates their rewards rii=1G\r_i\_i=1^G. For outcome-based RL, the group-relative advantage is typically Ai=(ri−μr)/(σr+ϵ)A_i=(r_i- _r)/( _r+ε), where μr _r and σr _r are the mean and standard deviation of group rewards, and is assigned to all valid tokens of response i. Let ρi,t=πθ(yi,t∣x,yi,<t)/πold(yi,t∣x,yi,<t) _i,t= _θ(y_i,t x,y_i,<t)/ _old(y_i,t x,y_i,<t) donote the token-level importance ratio. Ignoring optional regularization terms, the clipped policy objective can be written as ℒGRPO=−1G∑i=1G1|yi|∑t=1|yi|min(ρi,tAi,clip(ρi,t,1−ε,1+ε)Ai),L_GRPO=- 1G _i=1^G 1|y_i| _t=1^|y_i| \! ( _i,tA_i,\,clip( _i,t,1- ,1+ )A_i ), (1) Locally, a positive advantage increases the probability of sampled tokens, whereas a negative advantage decreases it; this directional property will be central to our analysis in Section 3 (Qiu et al., 2026; Zhuge et al., 2026). 2.2 Decoupled Rollout and Training Modern LLM RL systems typically decouple rollout generation from gradient computation: a high-throughput inference engine such as SGLang or vLLM (Zheng et al., 2024; Kwon et al., 2023) acts as the sampler, while a training framework such as Megatron-LM acts as the learner (Qiu et al., 2026; Shoeybi et al., 2019). Let πs _s and πl _l denote the sampler and learner policies, respectively. For a sampled token yty_t under the same prefix y<ty_<t, we define their log-probability gap as δt=logπl(yt∣x,y<t)−logπs(yt∣x,y<t). _t= _l(y_t x,y_<t)- _s(y_t x,y_<t). (2) Ideally, synchronized policies should yield δt=0 _t=0. In practice, the two execution paths can disagree even when they nominally represent the same model (Zhong et al., 2026). This gap admits a useful interpretation in the language of off-policy correction. Because sampled tokens are drawn from the sampler, yi,t∼πs(⋅∣x,yi,<t)y_i,t _s(· x,y_i,<t), the importance weight that exact policy evaluation would require is not ρi,t=πθ/πold _i,t= _θ/ _old but πθ/πs _θ/ _s, which factors as πθ(yi,t∣x,yi,<t)πs(yi,t∣x,yi,<t)=πθ(yi,t∣x,yi,<t)πl(yi,t∣x,yi,<t)⏟optimization drift⋅exp(δi,t)⏟learner–sampler ratio. _θ(y_i,t x,y_i,<t) _s(y_i,t x,y_i,<t)= _θ(y_i,t x,y_i,<t) _l(y_i,t x,y_i,<t)_optimization drift· \! ( _i,t )_learner--sampler ratio. (3) Under synchronous weight transfer with one update per rollout, the first factor is one at the start of each iteration, and the entire off-policy error of the update is carried by exp(δi,t) ( _i,t). Truncated importance sampling, clipping, and masking (Qiu et al., 2026; Li et al., 2026) all intervene on exactly this factor. Crucially, exp(δi,t) ( _i,t) is a magnitude-only statistic: it records how far the two policies disagree, but not how the ensuing gradient will change that disagreement. This gap between what existing corrections measure and what optimization stability requires motivates our directional analysis in Section 3. 2.3 Low-Precision Rollout with NVFP4 Reduced-precision rollout lowers both matrix-computation cost and model-memory traffic, making it particularly attractive for long-sequence RL. However, quantization introduces a forward perturbation that is qualitatively different from ordinary policy staleness (Li et al., 2025). Even under an identical prefix, quantized weights and activations perturb hidden states and output logits, producing a local probability discrepancy between the sampler and learner. During decoding, such local perturbations can additionally alter token sampling and hence the subsequent prefix, allowing quantization error to accumulate into trajectory-level divergence (Zhuge et al., 2026; Gu et al., 2026). NVFP4 is a Blackwell-supported format combining E2M1 FP4 values, an E4M3 FP8 microscale per 16 values, and an FP32 second-level scale (NVIDIA et al., 2025). For an element xjx_j in a 16-element block b, its quantized representation can be abstracted as x^j=SsbQE2M1(xjSsb),j∈b, x_j=S\,s_b\,Q_E2M1 ( x_jS\,s_b ), j∈ b, (4) where sbs_b is an E4M3 FP8 microscale shared by 16 consecutive values, S is an FP32 second-level scale shared across the tensor, and QE2M1(⋅)Q_E2M1(·) denotes rounding to the FP4 E2M1 grid. The fine-grained E4M3 microscale improves local range utilization, while the tensor-level FP32 scale extends the range representable by the collection of FP8 block scales. 3 Understanding Learner–Sampler Policy Mismatch We now study how the learner–sampler mismatch introduced in Section 2 evolves under policy optimization. Our analysis reveals a staged failure process. Section 3.1 derives which mismatches are locally amplified by the policy-gradient update. Section 3.2 then identifies when this directional risk becomes visible along training. Finally, Section 3.3 shows where the risk accumulates within long responses and why response-level averages fail to expose it. 3.1 Directional Dynamics of Policy Mismatch Which mismatches are self-amplifying under an RL policy update? Let zi,t(θ)=logπθ(yi,t∣x,yi,<t)z_i,t(θ)= _θ(y_i,t x,y_i,<t) denote the learner log-probability of rollout token (i,t)(i,t), δi,t _i,t is log-probability gap from equation 2, and gi,tg_i,t the signed coefficient applied to its policy-gradient term. A generic learner update with step size η>0η>0 is θ+=θ+η∑i,tgi,t∇θzi,t(θ),θ^+=θ+η _i,tg_i,t _θz_i,t(θ), (5) where for vanilla GRPO at the start of a single-update iteration gi,t=Aig_i,t=A_i; importance correction and stabilization mechanisms modify this coefficient. To track whether the mismatch grows or contracts, define the mismatch energy D=12∑i,tδi,t2D= 12 _i,t _i,t^2. Holding the rollout policy fixed during one update, a first-order expansion gives ΔD=η⊤K+(η2)=η∑i,tκi,tδi,tgi,t+ηRcross+(η2), D=η\, δ Kg+O(η^2)=η _i,t _i,t _i,tg_i,t+η R_cross+O(η^2), (6) where K is the Gram matrix of token log-probability gradients, κi,t=‖∇θzi,t‖22 _i,t=\| _θz_i,t\|_2^2, and RcrossR_cross collects cross-token interactions. Thus ΔD>0 D>0 means the current update batch inflates the total mismatch energy. Appendix A derives this expansion for a general preconditioned update with explicit optimizer residuals. Equation 6 is the identity-preconditioner special case. Writing the amplifying and contracting diagonal masses as Bamp=∑i,tκi,t[δi,tgi,t]+,Bcon=∑i,tκi,t[−δi,tgi,t]+B_amp= _i,t _i,t[ _i,tg_i,t]_+, B_con= _i,t _i,t[- _i,tg_i,t]_+ (7) where [x]+=max(x,0)[x]_+= (x,0), it becomes ΔD=η(Bamp−Bcon+Rcross)+(η2). D=η (B_amp-B_con+R_cross )+O(η^2). (8) Thus, within the diagonal current-gradient contribution, δi,tgi,t>0 _i,tg_i,t>0 amplifies the existing mismatch, whereas δi,tgi,t<0 _i,tg_i,t<0 contracts it. In particular, for vanilla GRPO, the criterion reduces to the sign interaction Aiδi,tA_i _i,t. The two mismatch-amplifying regions are −=(i,t):Ai<0,δi,t<0,+=(i,t):Ai>0,δi,t>0.A^-=\(i,t):A_i<0,\ _i,t<0\, ^+=\(i,t):A_i>0,\ _i,t>0\. (9) The remaining two sign combinations are locally mismatch-contracting. Figure 1, left summarizes this directional geometry. This distinction explains why mismatch magnitude alone is insufficient: a large mismatch may already be self-correcting, while a moderate one grows systematically when the gradient acts along it. Cross-token coupling, optimizer-history terms, and higher-order effects can still alter the realized motion of an individual token, so we use δi,tgi,t _i,tg_i,t as a local directional risk indicator rather than an unconditional predictor of the realized sign of δi,tΔδi,t _i,t _i,t. Appendix B.3 audits the scope of this interpretation. Our main use of the decomposition is therefore population-level: it predicts that instability may first appear as an imbalance between the two amplifying regions, even before the marginal gap distribution becomes broadly severe. 3.2 Directional Asymmetry Precedes Negative-Tail Explosion When does the imbalance become visible? We characterize native-NVFP4 GRPO trajectories of Qwen3-4B-Base and Qwen3-30B-A3B-Base using four 25-update trailing windows placed at 25%, 50%, 75%, and 100% of each run’s characterization horizon. We refer to these windows descriptively as healthy, drift, pre-terminal, and terminal, respectively. To measure the distribution of tokens between the two amplifying regions in equation 9, define the advantage-weighted token occupancy wA(ℛ)=∑(i,t)∈ℛ|Ai|∑i,t|Ai|,ρasym=wA(−)wA(+).w_A(R)= _(i,t) |A_i| _i,t|A_i|, _asym= w_A(A^-)w_A(A^+). (10) Figure 2: Directional asymmetry precedes terminal negative-tail expansion Figure 2 reveals a separation between directional drift and magnitude collapse. Across the drift-to-pre-terminal interval, the marginal gap distribution still appears largely benign: the fraction of tokens satisfying |δ|<0.05|δ|<0.05 remains at least 83.0% on 4B and 91.9% on 30B. Nevertheless, ρasym _asym rises from 1.02 to 2.32 by pre-terminal on 4B, and from 1.34 to 1.85 on 30B. The two theoretically symmetric amplifying regions therefore become empirically imbalanced well before a broad magnitude increase is visible. In the terminal stage, the lower tail expands abruptly. Within the respective mismatch estimands, the 5% gap quantile q0.05(δ)q_0.05(δ) reaches −42.75-42.75 on 4B and −10.94-10.94 on 30B. Directional asymmetry therefore identifies an earlier warning interval, whereas the negative-tail explosion characterizes the terminal regime. 3.3 Localized Accumulation before Global Failure Where does the amplifying mismatch accumulate? For each negative-advantage response, we partition it into non-overlapping segments S of W=64W=64 tokens. We call tokens satisfying Ai<0A_i<0 and δi,t<τana _i,t< _ana amplifying-tail tokens, where the model-specific threshold τana(m) _ana^(m) is fixed as the fifth percentile of δ in the corresponding Figure 3: Localized mismatch is hidden by response means before terminal. (a) Share of amplifying-tail tokens contained in the top 5% of segments ranked by fSampf_S^amp. (b) Fraction of negative-advantage responses containing at least one tail-heavy segment, the fraction with |δ¯i|<0.05| δ_i|<0.05 healthy window. Their local concentration is measured by fSamp(τana)=1|S|∑t∈S[Ai<0,δi,t<τana],f_S^amp( _ana)= 1|S| _t∈ S1[A_i<0,\ _i,t< _ana], (11) we call a segment tail-heavy when fSamp>0.1f_S^amp>0.1. From the healthy to the pre-terminal window, their global frequency decreases from 4.84.8–4.9%4.9\% to 2.82.8–3.6%3.6\% across the two models. Nevertheless, the top 5%5\% of segments contain approximately one quarter of all amplifying-tail tokens, about five times the mass implied by uniform placement, while adjacent tail-heavy persistence increases from approximately 2.6×2.6× to 3.83.8–4.0×4.0× the chance rate. The precursor is therefore not an expanding global tail, but a reorganization of a small tail population into a few persistent local regions. Response averages obscure this structure: at pre-terminal, 97.2%97.2\% (4B) and 79.9%79.9\% (30B) of responses containing tail-heavy segments still satisfy |δ¯i|<0.05| δ_i|<0.05, where δ¯i=ni−1∑t∈iδi,t δ_i=n_i^-1 _t _i _i,t. By terminal, roughly half of the eligible segments are tail-heavy and top-segment concentration declines. These observations motivate segment-level diagnosis with intervention restricted to −A^- tokens, rather than suppressing every token in an affected response or segment. 4 Method Section 3 shows that the risk of learner–sampler mismatch depends on its interaction with the policy-update direction: updates with δi,tgi,t>0 _i,tg_i,t>0 amplify the existing gap, whereas updates with δi,tgi,t<0 _i,tg_i,t<0 contract it. TRIAGE acts on this distinction through two complementary modifications of the effective update coefficient. Let gi,tbaseg^base_i,t denote the per-token coefficient of the underlying importance-corrected objective, including its reduction factors. We then write gi,t=αi,tgi,tbase+ψi,t,g_i,t= _i,t\,g^base_i,t+ _i,t, (12) The two terms partition by advantage sign: αi,t _i,t reallocates policy-gradient mass within negative-advantage responses (Section 4.1), while ψi,t _i,t adds bounded corrective pressure on positive-advantage responses (Section 4.2). 4.1 Segment Diagnosis with Direction-Selective Gating Following Section 3.3, TRIAGE therefore uses short contiguous segments as an intermediate diagnostic scale, a granularity that has also been found useful for avoiding the extremes of token- and response-level importance estimation (Yang et al., 2026). For a diagnostic segment S of response i, we compute δ¯S=∑t∈Smi,tδi,t∑t∈Smi,t,fStail=∑t∈Smi,t[δi,t<τtail]∑t∈Smi,t, δ_S= _t∈ Sm_i,t _i,t _t∈ Sm_i,t, f^tail_S= _t∈ Sm_i,t1[ _i,t< _tail] _t∈ Sm_i,t, (13) which respectively capture persistent displacement and sparse deep-tail mismatch. We map these statistics to a segment gate weight wS=(δ¯S,fStail)∈[wmin,1],w_S=G ( δ_S,f^tail_S )∈[w_ ,1], (14) where a larger negative displacement or a heavier negative tail yields stronger attenuation. The exact operating rule, including eligibility conditions and the fixed operating thresholds, is deferred to Appendix C. Crucially, a risky segment does not imply that all of its tokens should be attenuated. For a negative-advantage response, tokens with Ai<0,δi,t<0A_i<0,\ _i,t<0 contribute to mismatch amplification, whereas tokens with Ai<0,δi,t>0A_i<0,\ _i,t>0 naturally contract the discrepancy. TRIAGE therefore applies wi,t=wS,Ai<0,δi,t<0,t∈S,1,otherwise.w_i,t= casesw_S,&A_i<0,\ _i,t<0,\ t∈ S,\\[2.0pt] 1,&otherwise. cases (15) Thus, segment statistics determine whether local intervention is warranted, while the directional criterion from Section 3 determines which token updates are modified. In particular, naturally contracting tokens that coexist in the same risky segment are not directly gated. We do not gate the positive-advantage amplifying region Ai>0,δi,t>0\A_i>0,\ _i,t>0\: the directional imbalance identified in Section 3 concentrates in the negative-advantage region, and attenuating probability increases on reward-positive responses would directly suppress the learning signal. The weighted policy loss is ℒPG,iTRIAGE=∑tmi,twi,tℓi,tbase∑tmi,twi,t,L^TRIAGE_PG,i= _tm_i,tw_i,t ^base_i,t _tm_i,tw_i,t, (16) where ℓi,tbase ^base_i,t denotes the existing importance-corrected token loss. Relative to the original response mean, this induces αi,t=wi,tw¯i,w¯i=∑tmi,twi,t∑tmi,t. _i,t= w_i,t w_i, w_i= _tm_i,tw_i,t _tm_i,t. (17) The normalization therefore reallocates relative update mass away from the targeted amplifying subset rather than uniformly weakening the entire negative-advantage response, directly modifying the balance between BampB_amp and BconB_con identified in Section 3. 4.2 Repair on Positive-Advantage Responses Direction-selective gating controls the allocation of existing policy gradients, but it is multiplicative: it can attenuate an amplifying update toward zero, yet cannot reverse its sign or directly reduce the remaining learner–sampler gap. We therefore add a repair mechanism on positive-advantage responses. On these responses, the policy gradient on negative-gap tokens is already mismatch-contracting, so raising sampled-token log-probabilities reinforces a motion that agrees with reward-driven learning; applying the same correction to negative-advantage responses would instead oppose their policy gradients, which is why that side is handled by gating alone. Let R denote a repair segment of response i(R)i(R), with nR=∑t∈Rmi(R),tn_R= _t∈ Rm_i(R),t valid tokens and mean gap δ¯R δ_R. We define qR=[τrep−δ¯R]+,ϕβ(q)=β2[1+(q/β)2−1],q_R=[ _rep- δ_R]_+, _β(q)=β^2 [ 1+(q/β)^2-1 ], (18) where qRq_R is the one-sided violation of the repair boundary and ϕβ _β is a pseudo-Huber penalty. Gating is active throughout training, whereas repair is enabled from a fixed update krepk_rep onward, with λk=λrep[k≥krep] _k= _rep1[k≥ k_rep]. All operating thresholds and the repair schedule are fixed before each run. At update k, the objective is ℒk=ℒPGTRIAGE+λkNrep∑R∈repvalid[Ai(R)>0]ϕβ(qR),L_k=L_PG^TRIAGE+ _kN_rep _R _rep^valid1[A_i(R)>0]\, _β(q_R), (19) Under the ascent-form notation of Equation 12, one gradient-descent step on the repair objective contributes the additive coefficient ψi,t=λkNrepmi,tnRϕβ′(qR) 1[Ai>0,qR>0]. _i,t= _kN_rep m_i,tn_R\, _β (q_R)\,1[A_i>0,\ q_R>0]. (20) All valid tokens within an active segment share the same corrective coefficient. For negative-gap tokens, δi,tψi,t<0 _i,t _i,t<0, so the repair term contributes to contraction in the diagonal decomposition of Section 3. Positive-gap tokens may also receive the coefficient: the objective repairs negative segment-level displacement rather than enforcing contraction for every token. Appendix C.3 derives the positive-advantage repair coefficient and its bound. Figure 4: Training dynamics on Qwen3-30B-A3B-Base and Qwen3-4B-Base. From left to right: entropy loss, training reward, and mean absolute train–rollout log-probability gap against optimizer steps. 5 Experiments 5.1 Experimental Setup Models and task. We use Qwen3-30B-A3B-Base as the primary model and Qwen3-4B-Base for replication and ablation studies. We perform RL with GRPO on the DAPO dataset. To better understand the quantization error, we use 20,480 as the max response length. Detailed training settings are listed in Appendix D. Settings. We compare four training configurations: (i) BF16, the baseline; (i) NVFP4, native W4A4 execution with Transformer Engine; (i) NVFP4+TIS, adding token-level importance correction (Qiu et al., 2026); and (iv) NVFP4+TRIAGE, our full method. All NVFP4 configurations share the same native execution path: W4A4 kernels with per-token second-level activation scaling (Research et al., 2026) on both the sampler and the learner forward pass. TIS uses a truncation threshold of C=2.0C=2.0. TRIAGE operating rules and segment definitions are given in Appendix C; quantization and system settings are described in Appendices D and E. Evaluation Benchmarks and Metrics. We evaluate mathematical reasoning on several widely used benchmarks, including GSM8K (Cobbe et al., 2021), MATH-500 (Lightman et al., 2023; Hendrycks et al., 2021), AIME 2024/2025, and AMC 2023 (MAA, ). We also track training reward, policy entropy, and the mean absolute train–rollout log-probability gap throughout training. 5.2 Experimental Results Training stability. Figure 4 shows that naive NVFP4 becomes unstable after approximately 300 steps on Qwen3-4B-Base, with a sharp reward drop and rapid growth in the mean absolute train–rollout log-probability gap. TIS delays this failure but does not prevent it. On Qwen3-30B-A3B-Base, naive NVFP4 also exhibits rapid mismatch growth after roughly 700 steps. TIS does not exhibit a comparable divergence over 1,700 steps, but its reward trajectory remains below BF16 and TRIAGE in later training. In contrast, TRIAGE remains stable throughout the evaluated 600- and 1,700-step schedules, approximately doubling the observed stable training horizon on 4B relative to naive NVFP4. Benchmark performance. Table 1 shows that TRIAGE achieves near-BF16 average performance: 58.49% versus 58.26% on 4B, and 70.96% versus 72.41% on 30B. At the shared step-300 checkpoint on 4B, TRIAGE already outperforms naive NVFP4 and NVFP4+TIS by 5.93 and 2.27 percentage points, respectively, showing that its gains are not solely due to longer training. Although NVFP4+TIS does not collapse over the full 1700-step training horizon, it still scores below the pre-collapse naive NVFP4 checkpoint at step 700 (63.18% versus 65.87%). These observations are consistent with the directional and localized mismatch analysis in Section 3: the global mean does not capture how mismatch interacts with policy updates or where it concentrates within responses, supporting the importance of selective control of mismatch amplification. Table 1: Performance across five benchmarks. Step denotes the evaluated training checkpoint. Stable runs are evaluated at the final training step, while runs marked with † collapse before the planned training budget and are evaluated at the last saved checkpoint before collapse. Best result among NVFP4 settings for each model is shown in bold. Model Setting Step AIME24 AIME25 MATH500 AMC23 GSM8K Avg 30B BF16 1700 53.89% 36.67% 92.65% 88.04% 90.81% 72.41% NVFP4† 700 44.44% 30.00% 88.80% 77.17% 88.95% 65.87% NVFP4+TIS 1700 36.11% 27.22% 88.30% 69.02% 95.26% 63.18% NVFP4+TRIAGE 1700 51.67% 38.33% 91.60% 77.72% 95.47% 70.96% 4B BF16 600 24.44% 23.33% 84.1% 67.40% 92.04% 58.26% NVFP4† 300 8.33% 8.33% 78.4% 50.00% 90.45% 47.10% NVFP4+TIS† 300 16.11% 18.33% 80.90% 53.75% 84.72% 50.76% NVFP4+TRIAGE 300 19.44% 17.22% 82.00% 56.52% 89.99% 53.03% NVFP4+TRIAGE 600 27.22% 20.00% 85.80% 67.39% 92.04% 58.49% Table 2: Natural-EOS generation from the base checkpoint. Values are means over ten post-warm-up iterations at two global batch sizes. Initialization, evaluation, and checkpoint saving are excluded. Parentheses report speedups over BF16. End-to-end cost (s/iter) ↓ Rollout throughput (token/s) ↑ Setting Batch 256 Batch 512 Batch 256 Batch 512 BF16 208.2 (1.00×) 254.5 (1.00×) 2977 (1.00×) 3798 (1.00×) NVFP4 169.4 (1.23×) 191.0 (1.33×) 6201 (2.08×) 8942 (2.35×) NVFP4+TRIAGE 172.1 (1.21×) 195.8 (1.30×) 6745 (2.27×) 8717 (2.30×) 5.3 Rollout Throughput and Learner Overhead Rollout throughput. With TRIAGE, native NVFP4 achieves 6,745 and 8,717 output tokens/s at global batch sizes 256 and 512, respectively, corresponding to 2.27×2.27× and 2.30×2.30× the BF16 throughput. TRIAGE thus retains the more-than-twofold rollout throughput advantage observed for native NVFP4 in these runs. End-to-end RL efficiency. Iteration timing includes rollout, learner updates, weight conversion and synchronization, and orchestration. TRIAGE reduces average iteration time from 208.2 to 172.1 s at batch size 256 and from 254.5 to 195.8 s at batch size 512, yielding 1.21×1.21× and 1.30×1.30× end-to-end speedups, respectively. TRIAGE overhead. We separately isolate TRIAGE’s incremental learner cost on eight B300 GPUs using five paired repetitions of the same frozen batch. Enabling gating and repair yields an observed median learner-update overhead of 0.84% and a median absolute increase of 0.225 s. Detailed throughput settings, environment settings, independent configuration-selection results, stress-tests results, and complete timing distributions are provided in Appendix E. 5.4 Ablation Studies We perform component ablations on Qwen3-4B-Base. All variants branch from the same full-TRIAGE checkpoint at step 300. Full TRIAGE achieves the highest reward. Removing repair keeps the mean log-probability gap low but leaves the advantage-driven update imbalance uncorrected, reducing the reward. TIS alone remains stable when continued from the TRIAGE checkpoint, unlike its failure when used from the beginning in Figure 4. This contrast reflects the stage-dependent mismatch dynamics identified in Section 3. Naive NVFP4 also allows mismatch to grow sharply, leading to a late-stage collapse. Figure 5: Ablation on Qwen3-4B-Base. Full TRIAGE; Gate+TIS removes repair; TIS additionally removes gating; NVFP4 removes all three components. The dashed line marks the branch point. 6 Related Work Low-precision training. Native low-precision training requires careful control of quantization error. NVIDIA’s NVFP4 pretraining recipe (NVIDIA et al., 2025; Xie et al., 2025) and Quartet I (Panferov et al., 2026) address this through quantization design and improved gradient estimation. For RL, QeRL (Huang et al., 2025) combines NVFP4 with LoRA and adaptive quantization noise. FP8-RL (Qiu et al., 2026) stabilizes quantized rollout with importance-based corrections. We focus on the optimization dynamics of the residual learner–sampler mismatch under native NVFP4 W4A4 execution. Training–inference alignment. Implementation differences can destabilize RL even with synchronized model weights (Zhong et al., 2026). QaRL (Gu et al., 2026) aligns the learner with quantized rollout through low-bit forward computation and introduces sequence-level dual clipping for negative samples. QUADS (Zhuge et al., 2026) combines weight-only fake quantization on the learner with rollout-side activation compensation. TRIAGE instead controls how residual mismatch evolves under policy updates, modifying the optimization objective while retaining native W4A4 forward execution on both the learner and sampler. Directional control. ACRL (Fan et al., 2026) uses advantage-guided, four-quadrant probability adjustment, with response-level discrepancy relative to a reference controlling token-wise reweighting. TRIAGE derives a local mismatch-energy decomposition and identifies directional asymmetry and localized negative tails in native NVFP4 runs. 7 Conclusion We presented TRIAGE, a direction-aware stabilization method for native NVFP4 RL. Our analysis distinguishes locally amplifying and contracting policy-gradient contributions and identifies directional imbalance and segment-localized negative tails before global mismatch growth. TRIAGE translates these observations into selective gating, within-response rebalancing, and bounded repair, while preserving native W4A4 forward execution on both learner and sampler. On Qwen3-4B-Base and Qwen3-30B-A3B-Base, TRIAGE sustains stable optimization over the evaluated horizons and achieves near-BF16 average performance across five mathematical reasoning benchmarks. In our measured configurations, native NVFP4 with TRIAGE achieves up to 2.30×2.30× rollout throughput and 1.30×1.30× end-to-end speedups over BF16. These results support controlling how mismatch evolves under policy updates, rather than uniformly minimizing its magnitude. 8 Limitations and Future Work Our study focuses on establishing the importance of direction-aware stabilization for low-precision RL. While truncated importance sampling (TIS) can stabilize the absolute mismatch gap, our 30B experiments show that completing the training trajectory does not necessarily translate into strong downstream performance. In contrast, TRIAGE explicitly accounts for whether learner–sampler mismatch is locally corrected or amplified by the policy update. Nevertheless, our empirical study remains limited in scale. Due to computational constraints, we do not evaluate models at the 100B+ scale or substantially longer training horizons. Our main runs are terminated after the optimization trajectories exhibit clear convergence or stable late-stage behavior. Evaluating whether the same directional dynamics persist over substantially longer training and on larger sparse models is therefore an important direction for future work. Our current experiments also focus on synchronous RL. The directional criterion underlying TRIAGE is not inherently tied to synchronous sampling, and may be particularly relevant to asynchronous RL, where policy staleness introduces an additional source of learner–sampler mismatch. However, we have not experimentally validated TRIAGE in this setting. Extending the analysis to jointly characterize quantization mismatch and asynchronous policy staleness is an interesting direction for future study. Finally, we believe that the end-to-end efficiency of native NVFP4 RL remains below the acceleration potential of Blackwell and Vera Rubin Tensor Core computation. Although NVFP4 substantially accelerates rollout generation in our experiments, the overall RL iteration speedup is approximately 1.3×1.3×. A growing fraction of runtime is instead spent on operations outside the accelerated GEMMs, including parameter synchronization between the trainer and sampler and quantization-related reductions and data transformations such as Amax computation. Future systems could therefore obtain larger gains by co-designing TRIAGE with more tightly integrated low-precision kernels and RL frameworks, including more efficient parameter synchronization and fused quantization paths. On the algorithmic side, our directional analysis also suggests a broader design space beyond the current repair rule: rather than treating all mismatch-amplifying tokens uniformly, future work could develop finer-grained repair mechanisms that adapt to the magnitude, locality, and persistence of directional mismatch. References Alvarez et al. (2025) E. Alvarez, O. Almog, E. Chung, S. Layton, D. Stosic, R. Krashinsky, and K. Aubrey Introducing NVFP4 for efficient and accurate low-precision inference. Note: NVIDIA Technical BlogPublished June 24, 2025 Cited by: §1. Chen et al. (2025) M. Chen, C. Zhang, J. Liu, Y. Zeng, Z. Xue, Z. Liu, Y. Li, J. Ma, J. Huang, X. Zhou, et al. Scaling law for quantization-aware training. arXiv preprint arXiv:2505.14302. Cited by: §1. Cobbe et al. (2021) K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. Cited by: §5.1. DeepSeek-AI (2025) DeepSeek-AI DeepSeek-r1: incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948. Cited by: §1. Fan et al. (2026) W. Fan, Q. Lin, Z. Xia, Z. Zheng, S. Wang, Q. Chen, and L. Zhu ACRL: Adaptive Control of Training-Inference Discrepancy for Stable Reinforcement Learning. arXiv preprint arXiv:2607.24062. External Links: Document, Link Cited by: §6. Gu et al. (2026) H. Gu, H. Wang, J. Liu, L. Li, Q. Zhu, B. Liu, B. Xu, L. Wang, X. Yang, S. Lin, S. Han, and Y. Guo QaRL: rollout-aligned quantization-aware rl for fast and stable training under training–inference mismatch. arXiv preprint arXiv:2604.07853. Cited by: §1, §2.3, §6. Hendrycks et al. (2021) D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt Measuring mathematical problem solving with the math dataset. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, Cited by: §5.1. Huang et al. (2025) W. Huang, Y. Ge, S. Yang, Y. Xiao, H. Mao, Y. Lin, H. Ye, S. Liu, K. C. Cheung, H. Yin, Y. Lu, X. Qi, S. Han, and Y. Chen QeRL: Beyond Efficiency – Quantization-enhanced Reinforcement Learning for LLMs. arXiv preprint arXiv:2510.11696. External Links: Document, Link Cited by: §6. Kwon et al. (2023) W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, p. 611–626. Cited by: §2.2. Li et al. (2026) Y. Li, R. Elangovan, X. Dong, P. Panda, and B. Khailany QuRL: efficient reinforcement learning with quantized rollout. arXiv preprint arXiv:2602.13953. Cited by: §1, §2.2. Li et al. (2025) Z. Li, Y. Su, R. Yang, C. Xie, Z. Wang, Z. Xie, N. Wong, and H. Yang Quantization meets reasoning: exploring llm low-bit quantization degradation for mathematical reasoning. arXiv preprint arXiv:2501.03035. Cited by: §1, §2.3. Lightman et al. (2023) H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe Let’s verify step by step. arXiv preprint arXiv:2305.20050. Cited by: §5.1. [13] MAA American mathematics competitions. Note: https://maa.org/student-programs/amc/Accessed: 2026-09-21 Cited by: §5.1. NVIDIA et al. (2025) NVIDIA, F. Abecassis, A. Agrusa, D. Ahn, J. Alben, S. Alborghetti, M. Andersch, S. Arayandi, A. Bjorlin, A. Blakeman, E. Briones, I. Buck, B. Catanzaro, J. Choi, M. Chrzanowski, E. Chung, V. Cui, S. Dai, B. D. Rouhani, C. del Mundo, D. Donia, B. Eryilmaz, H. Estela, A. Goel, O. Goncharov, Y. Guvvala, R. Hesse, R. Hewett, H. Hum, U. Kapasi, B. Khailany, M. Khona, N. Knight, A. Kondratenko, R. Krashinsky, B. Lanir, S. Layton, M. Lightstone, D. Lo, P. Micikevicius, A. Mishra, T. Moon, D. Narayanan, C. Ni, A. Paithankar, S. Pasumarthi, A. Patel, M. Patwary, A. Poojary, G. Prasad, S. Priyadarshi, Y. Qin, X. Ren, O. Rybakov, C. Sakr, S. Satheesh, S. Sergienko, P. Shamis, K. Shankar, N. Sharma, M. Shoeybi, M. Siu, M. Smelyanskiy, D. Stosic, D. Stosic, B. Su, F. Sun, N. Tajbakhsh, S. Thomas, P. Tredak, E. Tsykunov, G. Vaithilingam, A. Vavre, R. Venkatesan, R. Waleffe, Q. Wan, H. Wang, M. Wang, L. Wei, H. Wu, E. Wu, K. Wyss, N. Xu, J. Xue, C. Yang, Y. Zhai, R. Zhang, J. Zhu, and Z. Zhu Pretraining Large Language Models with NVFP4. arXiv preprint arXiv:2509.25149. External Links: Document, Link Cited by: §2.3, §6. Panferov et al. (2026) A. Panferov, E. Schultheis, S. Tabesh, and D. Alistarh Quartet I: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation. arXiv preprint arXiv:2601.22813. External Links: Document, Link Cited by: §6. Qiu et al. (2026) Z. Qiu, S. Yu, J. Zhang, S. Zhang, X. Huang, J. Yang, and J. Lai FP8-RL: a practical and stable low-precision stack for llm reinforcement learning. arXiv preprint arXiv:2601.18150. Cited by: §1, §1, §2.1, §2.2, §2.2, §5.1, §6. Research et al. (2026) C. Research, A. Chan, A. Shalaby, A. Wettig, A. Sanger, A. Zhai, A. Ajay, A. Nair, C. Snell, C. Lu, et al. Composer 2 technical report. arXiv preprint arXiv:2603.24477. Cited by: §5.1. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. Deepseekmath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: §2.1. Shoeybi et al. (2019) M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro Megatron-lm: training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053. Cited by: §2.2. Xie et al. (2025) C. Xie, S. Cai, W. Wang, P. Li, Z. Sang, K. Yang, Y. Zhang, Z. Li, G. Zhu, Z. Liu, et al. Infir: crafting effective small language models and multimodal small language models in reasoning. arXiv preprint arXiv:2502.11573. Cited by: §1, §6. Yang et al. (2026) K. Yang, Z. Chen, Y. Wang, Z. Li, N. Cheng, S. Wang, and J. Xiao SSPO: subsentence-level policy optimization. arXiv preprint arXiv:2511.04256. Cited by: §4.1. Zheng et al. (2024) L. Zheng, L. Yin, Z. Xie, C. Sun, J. Huang, C. H. Yu, S. Cao, C. Kozyrakis, I. Stoica, J. E. Gonzalez, et al. Sglang: efficient execution of structured language model programs. Advances in neural information processing systems 37, p. 62557–62583. Cited by: §2.2. Zhong et al. (2026) T. Zhong, N. Ling, Y. Pi, Z. Wei, T. Yu, G. Fox, P. Wu, and X. Yu Diagnosing Training Inference Mismatch in LLM Reinforcement Learning. arXiv preprint arXiv:2605.14220. External Links: Document, Link Cited by: §1, §2.2, §6. Zhuge et al. (2026) Z. Zhuge, H. Yu, X. Wang, Z. Li, Y. Cao, D. Liu, and J. Zhang QUADS: stabilizing NVFP4 reinforcement learning for MoE via quantization-error alignment across dual sides. arXiv preprint arXiv:2607.15810. Cited by: §1, §2.1, §2.3, §6. Appendix A Derivation of Learner–Sampler Mismatch Dynamics This appendix derives the local mismatch dynamics in Section 3.1, including a positive-semidefinite preconditioner and explicit optimizer residuals. A.1 Fixed-Sampler Setup and Update Coefficients Let p≡(i,t)p≡(i,t) index a valid rollout token, with sampled token apa_p and prefix sps_p corresponding to yi,ty_i,t and (x,yi,<t)(x,y_i,<t) in the main text. Define zp(θ)=logπθ(ap∣sp),bp=logπs(ap∣sp),δp(θ)=zp(θ)−bp.z_p(θ)= _θ(a_p s_p), b_p= _s(a_p s_p), _p(θ)=z_p(θ)-b_p. (21) During one learner update, the trajectories, prefixes, masks, advantages, and recorded sampler log-probabilities are fixed. Hence ∇θδp=∇θzp _θ _p= _θz_p. The Taylor expansion below assumes locally bounded first and second derivatives of the log-probability map. All gradients are evaluated at the pre-update parameters θk _k. Write the current policy-gradient signal as uk=∑pgp∇θzp.u_k= _pg_p _θz_p. (22) Here gpg_p includes the loss-reduction factors. The main-text directional discussion suppresses these positive factors when writing gi,t=Aig_i,t=A_i; we retain them here to specify coefficient magnitudes. For the vanilla GRPO response mean at the start of a single-update iteration, where the learner/old-learner ratio is one and clipping is inactive, gi,teff=γkAini,ni=∑tmi,t>0,mi,t∈0,1,g_i,t^eff= _k A_in_i, n_i= _tm_i,t>0, m_i,t∈\0,1\, (23) on valid tokens, with zero coefficient on masked tokens. Here γk>0 _k>0 is the common outer averaging factor for the update; for the single-prompt group objective in Section 2, γk=1/G _k=1/G. Thus sign(gi,teff)=sign(Ai)sign(g_i,t^eff)=sign(A_i) for valid tokens. A positive batch-wide reduction constant preserves this sign, whereas the response-dependent factor 1/ni1/n_i must be retained when comparing coefficient magnitudes. Importance weighting, clipping, masking, and TRIAGE are represented by their actual effective coefficients gpg_p; a token whose coefficient is zero has no diagonal update contribution. A.2 Preconditioned Gap and Energy Dynamics Express the local optimizer update as Δθk=θk+1−θk=ηkuk+rkopt,η>0,k⪰0, _k= _k+1- _k= _ku_k+r_k^opt, η>0, _k 0, (24) where kP_k is symmetric. The residual rkoptr_k^opt contains components outside the preconditioned current-gradient signal, such as history-dependent momentum and decoupled weight decay. This decomposition accommodates the local current-gradient component of AdamW-style updates while keeping those residual contributions explicit. Let J have rows ∇θzp⊤ _θz_p . Define K()=JkJ⊤,Kp,q()=∇θzp⊤k∇θzq,kz=Jrkopt.K^(P)=JP_kJ , K^(P)_p,q= _θz_p P_k _θz_q, _k^z=Jr_k^opt. (25) The matrix K()K^(P) is positive semidefinite, although its off-diagonal entries can have either sign. Applying Taylor expansion with the sampler fixed gives Δ=JΔθk+(‖Δθk‖22)=ηK()+kz+(‖Δθk‖22). δ=J _k+O(\| _k\|_2^2)=η K^(P)g+r_k^z+O(\| _k\|_2^2). (26) For D(θ)=12‖(θ)‖22D(θ)= 12\| δ(θ)\|_2^2, the exact energy difference is ΔD=⊤Δ+12‖Δ‖22. D= δ δ+ 12\| δ\|_2^2. (27) Since Δ=(‖Δθk‖2) δ=O(\| _k\|_2), substitution yields ΔD=η⊤K()+Ropt+(‖Δθk‖22),Ropt=⊤kz. D=η δ K^(P)g+R_opt+O(\| _k\|_2^2), R_opt= δ r_k^z. (28) Setting k=IP_k=I and rkopt=0r_k^opt=0 recovers K=JJ⊤K=J and the (η2)O(η^2) expansion in Equation 6 of the main text. A.3 Diagonal Contributions and Their Scope Separate the diagonal and cross-token terms by defining κp=Kp,p()=∇θzp⊤k∇θzp≥0,Rcross=∑p∑q≠pδpKp,q()gq. _p=K^(P)_p,p= _θz_p P_k _θz_p≥ 0, R_cross= _p _q≠ p _pK^(P)_p,qg_q. (29) Then ΔD=η(∑pκpδpgp+Rcross)+Ropt+(‖Δθk‖22). D=η ( _p _p _pg_p+R_cross )+R_opt+O(\| _k\|_2^2). (30) For the token energy Dp=12δp2D_p= 12 _p^2, the diagonal contribution of the current gradient is ΔDpdiag=ηκpδpgp. D_p^diag=η _p _pg_p. (31) Whenever κp>0 _p>0, its sign equals sign(δpgp)sign( _pg_p): positive values amplify the existing gap and negative values contract it within this diagonal contribution. For the vanilla coefficients in Equation 23, this is the sign of Aiδi,tA_i _i,t, giving the two amplifying regions in Equation 9. The realized motion of the same token also includes the remaining terms: δpΔδp=ηκpδpgp+ηδp∑q≠pKp,q()gq+δprk,pz+(‖Δθk‖22). _p _p=η _p _pg_p+η _p _q≠ pK^(P)_p,qg_q+ _pr_k,p^z+O(\| _k\|_2^2). (32) Cross-token coupling, optimizer residuals, and higher-order effects can therefore change the realized sign of δpΔδp _p _p. The criterion δpgp _pg_p isolates the token’s own current-gradient contribution; Appendix B.3 measures its agreement with realized motion in the characterization runs. A.4 Amplifying and Contracting Masses Using [x]+=max(x,0)[x]_+= (x,0), define Bamp=∑pκp[δpgp]+,Bcon=∑pκp[−δpgp]+.B_amp= _p _p[ _pg_p]_+, B_con= _p _p[- _pg_p]_+. (33) The identity x=[x]+−[−x]+x=[x]_+-[-x]_+ gives ΔD=η(Bamp−Bcon+Rcross)+Ropt+(‖Δθk‖22). D=η (B_amp-B_con+R_cross )+R_opt+O(\| _k\|_2^2). (34) Thus BampB_amp and BconB_con quantify the two diagonal contributions, with the optimizer and cross-token effects retained. The advantage-weighted token masses used in Section 3.2 are tractable empirical summaries of the amplifying regions; they do not estimate BampB_amp or BconB_con, which also depend on gradient geometry and gap magnitude. Appendix B Characterization Protocol and Additional Analyses This appendix specifies the characterization runs, mismatch estimands, and analysis windows behind Section 3. It then evaluates when the diagonal indicator agrees with realized token motion and reports the complete localization statistics and segment-length sensitivity. B.1 Models, Training Setup, and Evaluation Models and training setup. We characterize learner–sampler mismatch on Qwen3-4B-Base and Qwen3-30B-A3B-Base. Both runs use DAPO-Math-17K, containing 17,398 training prompts, with learner seed 1234 and rollout/data seed 42. Each rollout samples 16 prompts with 16 responses per prompt, giving a global response batch size of 256, followed by one optimizer update. Sampler weights are refreshed after every learner update. Training rollouts use temperature 1.01.0, top-p 1.01.0, and top-k=−1k=-1. The maximum completion length is 16,384 tokens for Qwen3-4B-Base and 20,480 tokens for Qwen3-30B-A3B-Base. We use Adam with a constant learning rate of 10−610^-6 and no warm-up, β1=0.9 _1=0.9, β2=0.98 _2=0.98, ϵ=10−8ε=10^-8, weight decay 0.10.1, and gradient clipping at 1.01.0. Master parameters, accumulated gradients, and optimizer states are maintained in FP32, while the sampler KV cache remains BF16. NVFP4 configuration. We use native NVFP4 (W4A4) with one-dimensional block scaling and disable 16×1616× 16 square weight scaling. Randomized Hadamard transforms (RHT) are enabled, and stochastic rounding is used for gradient quantization. For Qwen3-4B-Base, NVFP4 is applied to the gate, up, and down projections of every MLP layer, while attention, normalization, and the tied embedding/output weights remain in BF16. For Qwen3-30B-A3B-Base, all expert MLP layers are quantized to NVFP4, while attention, normalization, embeddings, routing gates, and the independent LM head remain in BF16. Routing replay is enabled for the MoE model. Table 3 summarizes the model-specific settings and the portions of each run used for the characterization analysis. Table 3: Model-specific configuration of the learner–sampler mismatch characterization runs. The characterization horizon denotes the updates used in the analysis rather than the configured maximum run length. Configuration Qwen3-4B-Base Qwen3-30B-A3B-Base Architecture Dense MoE NVFP4 scope All MLP Layers All Expert Layers Max. completion 16,384 20,480 Routing replay N/A Enabled Expert parallelism N/A 8 Characterization horizon 324 updates 724 updates Characterization-time evaluation. During training, we evaluate AIME-2024 every ten learner updates using four independently sampled responses per problem. Decoding uses temperature 0.70.7 and top-p 1.01.0. These evaluations are used to track the training trajectories and determine the analysis windows in Section 3; they are distinct from the final checkpoint evaluation described below. Independent checkpoint evaluation. Final benchmark evaluation is performed independently from the training sampler using a SGLang service. We evaluate AIME-2024, AIME-2025, MATH500, AMC23, and GSM8K using six sampled responses per problem. All benchmarks are decoded with temperature 0.70.7, top-p 1.01.0, top-k=−1k=-1, and min-p=0p=0, with thinking enabled and a maximum output length of 20,480 tokens. The primary metric is the mean sample accuracy (accuracy:mean), which corresponds to sample-averaged pass@1 under repeated stochastic decoding. When additionally reported, pass@k is computed per problem as 1−(n−ck)(nk),1- n-ck nk, where n is the number of sampled responses and c is the number scored correct. Hardware and software. All characterization runs use NVIDIA Blackwell B300 GPUs. The training environment uses PyTorch 2.9.1 with CUDA 13.0, Transformer Engine 2.10.0, miniTransformer 1.0.0, SGLang 0.5.10, and Megatron-LM and Slime at the revisions released with our artifact. Exact package revisions, container specifications, launch commands, and resolved configurations are included in the released reproducibility artifact. B.2 Mismatch Estimands and Analysis Windows We characterize native-NVFP4 GRPO trajectories of Qwen3-4B-Base and Qwen3-30B-A3B-Base. The 4B run measures the ordinary learner–sampler gap. For 30B, learner-side teacher-forced log probabilities are evaluated while replaying the sampler MoE routing decisions; its measured gap is therefore a residual mismatch after conditioning on routing: δi,t30B,res=zi,tlearner,routereplay−bi,t, _i,t^30B,res=z_i,t^learner,route\ replay-b_i,t, (35) where bi,t=logπs(yi,t∣x,yi,<t)b_i,t= _s(y_i,t x,y_i,<t) is the frozen generation-time log-probability and is reused unchanged in post-update evaluation. Absolute mismatch magnitudes from the two runs are consequently not treated as directly comparable. Across successful updates, the logged trajectories contain 232,089,183232,089,183 valid completion tokens for 4B and 230,713,088230,713,088 for 30B. Zero-advantage responses account for 41.27%41.27\% and 33.16%33.16\% of these tokens, respectively. Marginal gap statistics use all valid completion tokens, whereas the gradient-bearing occupancy reported in the main text excludes Ai=0A_i=0. The |A||A|-weighted statistics automatically assign zero mass to such tokens. For model m with characterization horizon SmS_m, we place analysis anchors at normalized progress p∈0.25,0.50,0.75,1.00p∈\0.25,0.50,0.75,1.00\. The anchor update and its inclusive 2525-update trailing window are am(p)=round(pSm),m(p)=am(p)−24,…,am(p).a_m(p)=round(pS_m), _m(p)=\a_m(p)-24,…,a_m(p)\. (36) The four stage-wise analysis windows used in Section 3.2 are listed in Table 4. Table 4: Progress-normalized analysis windows used for learner–sampler mismatch characterization. For each model, we place analysis anchors at 25%25\%, 50%50\%, 75%75\%, and 100%100\% of the characterization trajectory and use the 25-update trailing window ending at each anchor. Stage names are descriptive shorthand for these progress-normalized anchors rather than independently selected training phases. Healthy Drift Pre-terminal Terminal Normalized anchor p 25%25\% 50%50\% 75%75\% 100%100\% Qwen3-4B-Base (S=324S=324) 57–81 138–162 219–243 300–324 Qwen3-30B-A3B-Base (S=724S=724) 157–181 338–362 519–543 700–724 The collapse trigger is run-specific. For the 4B run, let cs=[raw_rewards≤0.10∧pass@16s≤0.3125]c_s=1[raw\_reward_s≤ 0.10\ \ pass@16_s≤ 0.3125] from the logged training metrics at update s, the latter computed over the rollout group of 1616 samples per prompt; the collapse onset is the first update completing three consecutive violations, mins:cs−2=cs−1=cs=1=304 \s:c_s-2=c_s-1=c_s=1\=304. For the 30B run, the onset is the first update with a non-finite logged gradient norm, update 724724, which terminates the trajectory. The two criteria are used only to align stages within each run and do not assert a shared failure mechanism. For the 30B terminal window, update 724 contributes its pre-update snapshot to distributional and segment statistics, but the failed optimizer step is not included in successful-update token counts. For 4B, the terminal window extends beyond the first collapse trigger and is used to characterize the resulting global-mismatch regime rather than to identify the trigger time itself. B.3 Empirical Scope of the Directional Risk Indicator The decomposition in Appendix A retains cross-token interactions and optimizer residuals. We measure directly when its diagonal sign indicator agrees with the realized one-update motion. B.3.1 One-Update Directional Audit For a saved anchor k, let θk− _k^- and θk+ _k^+ denote the learner parameters before and after one successful optimizer update. We reuse the identical rollout batch, sampled prefixes, advantages, masks, and logged sampler log probabilities, and perform teacher-forced learner replay before and after the update: δi,t−=zi,t−bi,t,δi,t+=zi,t+−bi,t. _i,t^-=z_i,t^--b_i,t, _i,t^+=z_i,t^+-b_i,t. (37) Hence the realized gap motion is Δδi,t=δi,t+−δi,t−. _i,t= _i,t^+- _i,t^-. (38) The fixed-sampler identity Δδi,t=zi,t+−zi,t− _i,t=z_i,t^+-z_i,t^- (39) is checked numerically with absolute tolerance 2×10−62× 10^-6 and zero relative tolerance. Replay is executed under eval() and both attention and hidden-state dropout are disabled. For the 30B model, the MoE routing cache is reused exactly. We do not snapshot and restore the kernel-level stochastic-rounding RNG state; the audit therefore measures the realized implementation-level one-update motion rather than asserting a bitwise-identical quantization-noise realization before and after the parameter update. The reference sampler remains fixed in this audit; it does not measure the change in execution mismatch after sampler weights are refreshed. Each probe contains exactly one optimizer step per rollout. An anchor is retained only when optimizer.step() reports a successful update and the optimizer scheduler advances. The late anchors are updates 248–252 for 4B and 598–602 for 30B. For the response-mean reducer, the vanilla token coefficient is gi,teff=γkAi/nig_i,t^eff= _kA_i/n_i (Eq. 23). The audit uses g~i,t=Ai/ni g_i,t=A_i/n_i, retaining the response-length factor and removing the positive update-common factor γk _k. This preserves the diagonal sign and normalized agreement within each update. Pooled scores across anchors use these normalized coefficients, without restoring anchor-specific outer factors. On an eligible token j≡(i,t)j≡(i,t), define pj=sign(δj−g~j),rj=sign(δj−Δδj).p_j=sign( _j^- g_j), r_j=sign( _j^- _j). (40) The main audit requires |δj−|>10−12| _j^-|>10^-12, g~j≠0 g_j≠ 0, and Δδj≠0 _j≠ 0; exact zeros are treated as ties and excluded. For a selected token population S, the unweighted agreement is Agrunw()=1||∑j∈[pj=rj].Agr_unw(S)= 1|S| _j 1[p_j=r_j]. (41) We additionally weight tokens by the magnitude of their diagonal directional signal, uj=|δj−g~j|.u_j=| _j^- g_j|. (42) The weighted agreement is Agrw()=∑j∈uj 1[pj=rj]∑j∈uj.Agr_w(S)= _j u_j\,1[p_j=r_j] _j u_j. (43) As an outlier-robustness check, we also cap uju_j at its 99.999.9th percentile within each reported population before applying Eq. 43. B.3.2 Agreement as Mismatch Becomes Severe Table 5 reports the late-stage audit in the negative-gap amplifying region −=Ai<0,δi,t<0A^-=\A_i<0, _i,t<0\. Table 5: Agreement (%) between the diagonal directional indicator sign(δgeff)sign(δ g^eff) and realized mismatch motion sign(δΔδ)sign(δ δ) on late-stage −A^- tokens. “Weighted” uses |δgeff||δ g^eff|; “Capped” additionally caps this weight at the population-specific 99.999.9th percentile. † denotes low-sample group. Model Stratum Tokens Unw. Weighted Capped Qwen3-4B-Base All 1,640,752 38.88 47.73 46.81 Qwen3-4B-Base |δ|>0.5|δ|>0.5 47,436 47.99 50.09 49.70 Qwen3-4B-Base |δ|>1|δ|>1 18,822 50.43 52.34 51.99 Qwen3-4B-Base |δ|>3|δ|>3 1,419 59.62 63.05 63.01 Qwen3-4B-Base |δ|>6|δ|>6 392† 60.20 65.75 65.73 Qwen3-30B-A3B-Base All 231,854 33.69 25.27 25.44 Qwen3-30B-A3B-Base |δ|>0.5|δ|>0.5 9,228 25.88 24.07 24.07 Qwen3-30B-A3B-Base |δ|>1|δ|>1 5,887 24.27 23.27 23.35 Qwen3-30B-A3B-Base |δ|>3|δ|>3 2,026 21.08 21.86 21.95 Qwen3-30B-A3B-Base |δ|>6|δ|>6 723† 19.78 22.18 22.20 For the Qwen3-4B-Base vanilla-mismatch run, the diagonal indicator becomes substantially more informative as mismatch severity increases. Weighted agreement rises from 47.73%47.73\% over all eligible late-stage −A^- tokens to 63.05%63.05\% for |δ|>3|δ|>3. The latter population still contains 1,4191,419 tokens across 316316 responses. More importantly, the weighted agreement for this stratum remains above the 50%50\% reference at each of the five saved anchors, ranging from 59.03%59.03\% to 66.49%66.49\%. Capping the largest 0.1%0.1\% of weights changes the pooled result only from 63.05%63.05\% to 63.01%63.01\%, showing that the trend is not driven by a handful of extreme-weight tokens. We do not base the main claim on the |δ|>6|δ|>6 row because of its lower support. The same token-wise trend does not reproduce for the Qwen3-30B-A3B-Base routing-replay residual estimand: weighted agreement remains below the 50%50\% reference and decreases rather than increases with severity. In this conditioned residual setting, the diagonal term in Equation 34 alone is not a reliable token-wise predictor. The current measurements do not isolate whether this behavior is primarily caused by cross-token coupling, routing-conditioned residual structure, or their interaction. We therefore restrict token-wise predictiveness claims to the Qwen3-4B-Base vanilla characterization and use the two models jointly only for the population-level directional and localization patterns reported in the main text. B.4 Additional Analysis of Localized Mismatch This section provides the construction, complete stage-wise statistics, and sensitivity checks for the segment-level localization analysis in Section 3.3. B.4.1 Segment Construction and Threshold Calibration Each response is partitioned independently by its original zero-based completion-token position t: segment k contains valid positions with ⌊t/W⌋=k t/W =k. Masked positions are excluded from segment statistics without reindexing the remaining tokens. Empty segments are omitted, and the final nonempty segment is retained even if it contains fewer than W valid tokens. No segment crosses a response boundary. The main analysis uses W=64W=64. Because the localized failure mode of interest is the negative-gap amplifying region, segment statistics are computed over responses with Ai<0A_i<0. For a segment S in response i, define fSamp(τ)=1|S|∑t∈S[Ai<0,δi,t<τ].f_S^amp(τ)= 1|S| _t∈ S1[A_i<0,\ _i,t<τ]. (44) The analysis threshold is calibrated once per model from the healthy reference window: τana(m)=Q0.05(δi,t:(i,t)∈m,healthy), _ana^(m)=Q_0.05 (\ _i,t:(i,t) _m,healthy\ ), (45) where the quantile is computed over all valid response tokens in the healthy window, rather than only the Ai<0A_i<0 subset. This gives τana(4B)=−0.24208,τana(30B)=−0.03949. _ana^(4B)=-0.24208, _ana^(30B)=-0.03949. (46) A segment is called tail-heavy when hS=[fSamp(τana)>0.1].h_S=1 [f_S^amp( _ana)>0.1 ]. (47) This calibration separates the localization question from later global tail growth: τana _ana is fixed before drift rather than re-estimated at every stage. B.4.2 Localization Metrics Let S denote all eligible segments pooled within an analysis window. We rank them by fSampf_S^amp and retain the top ⌈0.05||⌉ 0.05|S| as 5%S_5\%. Ties are resolved by decreasing tail count, followed by canonical response and segment identifiers. The amplifying-tail base rate is ramp=∑S∈∑t∈S[δi,t<τana]∑S∈|S|.r_amp= _S _t∈ S1[ _i,t< _ana] _S |S|. (48) The fraction of amplifying-tail tokens contained in the top-5%5\% segments is Ctok=∑S∈5%∑t∈S[δi,t<τana]∑S∈∑t∈S[δi,t<τana].C_tok= _S _5\% _t∈ S1[ _i,t< _ana] _S _t∈ S1[ _i,t< _ana]. (49) The 5%5\% uniform-allocation reference describes equal tail mass per segment. It is not a random-position null for CtokC_tok: ranking segments by their observed tail fractions raises expected top-5%5\% containment even under random placement. We account for this selection in Section B.4.5. Tail-heavy segment coverage is Hseg=1||∑S∈hS.H_seg= 1|S| _S h_S. (50) For adjacency, let ℰE contain directed pairs (Si,k,Si,k+1)(S_i,k,S_i,k+1) for which both segments are nonempty and belong to the same response. Probabilities over ℰE weight every eligible pair equally. We report Padj=Prℰ(hSi,k+1=1∣hSi,k=1)Prℰ(hSi,k+1=1).P_adj= _E(h_S_i,k+1=1 h_S_i,k=1) _E(h_S_i,k+1=1). (51) The denominator is the successor-heavy marginal over eligible pairs, not HsegH_seg. Thus Padj>1P_adj>1 indicates association relative to this pooled marginal. Response-level heterogeneity can also produce such an association; the response-conditioned controls below separate this reference from randomized token positions and segment order. B.4.3 Stage-Wise Localization with W=64W=64 Table 6 reports the full stage-wise evolution of the core localization statistics. Table 6: Stage-wise localization of negative-gap amplifying tokens with W=64W=64. rampr_amp, HsegH_seg, and CtokC_tok are percentages. PadjP_adj uses the successor-heavy marginal over eligible adjacent pairs. The table reports descriptive statistics; matched random-position and segment-order controls are shown in Figure 7. Model Stage rampr_amp HsegH_seg CtokC_tok PadjP_adj Qwen3-4B-Base Healthy 4.90 15.08 21.56 2.65 Qwen3-4B-Base Drift 2.60 5.43 25.74 4.22 Qwen3-4B-Base Pre-terminal 2.75 6.24 25.68 3.98 Qwen3-4B-Base Terminal 31.53 48.64 15.13 1.84 Qwen3-30B-A3B-Base Healthy 4.79 14.46 20.10 2.62 Qwen3-30B-A3B-Base Drift 3.82 10.43 22.15 3.28 Qwen3-30B-A3B-Base Pre-terminal 3.59 10.25 25.01 3.78 Qwen3-30B-A3B-Base Terminal 22.91 57.02 18.09 1.45 The tail’s frequency and its distribution across segments evolve differently. From the healthy to pre-terminal window, rampr_amp decreases from 4.90%4.90\% to 2.75%2.75\% on 4B and from 4.79%4.79\% to 3.59%3.59\% on 30B, while CtokC_tok increases to 25.68%25.68\% and 25.01%25.01\%. At the terminal stage, tail-heavy coverage reaches 48.64%48.64\% and 57.02%57.02\%, and top-5%5\% containment decreases. These statistics distinguish concentrated pre-terminal mismatch from the broader coverage of the terminal regime. Response means can remain small despite these local tails. Figure 6 shows the full empirical distribution of |δ¯i|| δ_i| among negative-advantage responses containing at least one tail-heavy segment, with δ¯i=ni−1∑tvalidδi,t δ_i=n_i^-1 _t\,valid _i,t. At pre-terminal, 97.22%97.22\% of affected 4B responses and 79.94%79.94\% of affected 30B responses satisfy |δ¯i|<0.05| δ_i|<0.05; the terminal fractions are 2.61%2.61\% and 0.36%0.36\%. Replacing the magnitude of the signed mean with the mean absolute token gap gives pre-terminal fractions of 52.37%52.37\% and 50.21%50.21\% at the same cutoff. Signed cancellation therefore contributes to dilution, while a local tail can also coexist with a small mean absolute gap. Figure 6: Local tails within responses with small mean mismatch. Empirical CDFs of |δ¯i|| δ_i| among Ai<0A_i<0 responses with at least one tail-heavy W=64W=64 segment. All four stage windows are shown for each model, with affected responses selected separately in each window and weighted equally. The vertical line marks 0.050.05; labels give the pre-terminal and terminal fractions below this cutoff. B.4.4 Sensitivity to Segment Length We repeat the analysis with W=128W=128, retaining the final shorter segment and using the same model-specific τana _ana. Table 7 summarizes the healthy-to-pre-terminal transition and the terminal regime. Table 7: Segment-length sensitivity with W=128W=128. Entries before and after the arrow correspond to the healthy and pre-terminal stages. rampr_amp, CtokC_tok, and HsegH_seg are percentages; PadjP_adj uses the same eligible-pair definition as the W=64W=64 analysis. Model rampr_amp CtokC_tok PadjP_adj Terminal HsegH_seg Qwen3-4B-Base →2.754.90\!→\!2.75 →21.8118.39\!→\!21.81 →5.572.82\!→\!5.57 48.98 Qwen3-30B-A3B-Base →3.594.79\!→\!3.59 →21.7617.23\!→\!21.76 →4.372.85\!→\!4.37 60.25 The sequence-level dilution result is also preserved: 96.11%96.11\% and 75.99%75.99\% of affected 4B and 30B responses satisfy |δ¯i|<0.05| δ_i|<0.05 at the pre-terminal stage, whereas the corresponding terminal values fall to 1.86%1.86\% and 0.23%0.23\%. At terminal failure, top-5%5\% token containment also decreases to 14.83%14.83\% and 17.63%17.63\%, and persistence approaches 1.84×1.84× and 1.40×1.40×, respectively. These descriptive trends are consistent across the two segment lengths. The conditional randomization analyses below use the default W=64W=64 partition. The fixed severe-gap criterion δ<−6δ<-6 provides a complementary severity check. With W=64W=64, it yields only 1111, 77, and 4949 tail-heavy segments in the healthy, drift, and pre-terminal 4B windows, and 00, 44, and 2020 on 30B. This sparsity makes early adjacency ratios unstable. At terminal, the criterion includes 24.88%24.88\% and 10.07%10.07\% of eligible tokens, respectively. We therefore retain the healthy-calibrated threshold for localization and use δ<−6δ<-6 to describe the extreme terminal tail. B.4.5 Response-Conditioned Localization Controls To test whether concentration exceeds the effect of response-level tail rates and top-segment selection, we randomize tail positions within each response. The analysis retains the same four windows, thresholds, and W=64W=64 segmentation. Figure 7 reports the original metrics under this control and a separate exploratory test of segment order. Figure 7: Response-conditioned controls for local structure. Top and middle: observed CtokC_tok and PadjP_adj compared with within-response tail-label shuffling, retaining all eligible segments, including short segments. Dashed curves show null medians. Bottom: the exploratory segment-order test, restricted to full segments and fixing each response’s heavy-segment count; the dashed line is the analytic random-order expectation normalized to one. Shading denotes central 95%95\% conditional randomization intervals from 499 draws per window, not across-run confidence intervals. Both models and all four windows are retained. For every Ai<0A_i<0 response, the primary control fixes valid-token positions, response length, and total tail-token count, then uniformly permutes binary tail labels. Each of 499 randomizations recomputes segment tail fractions, the global top-5%5\% ranking, and PadjP_adj. Multivariate hypergeometric allocation implements this permutation exactly while preserving segment capacities, including short segments. We use randomization seed 20260924 and report central 95%95\% conditional randomization intervals. These intervals describe random placement within the observed responses, not variability across independent training runs. At pre-terminal, observed CtokC_tok is 25.68%25.68\% on 4B and 25.01%25.01\% on 30B, compared with null medians of 18.17%18.17\% and 18.50%18.50\%. The corresponding null intervals are [18.11%,18.23%][18.11\%,18.23\%] and [18.38%,18.63%][18.38\%,18.63\%]. The excesses of 7.517.51 and 6.516.51 percentage points show concentration beyond response-level tail-rate heterogeneity and rank selection. Such excess concentration is also present in the healthy windows; its magnitude does not increase monotonically across both trajectories. The original PadjP_adj has a different comparison: its pre-terminal value is 3.983.98 versus a token-shuffle median of 10.1810.18 on 4B, and 3.783.78 versus 6.626.62 on 30B. Thus its elevation above one does not establish persistence beyond this response-conditioned null. Tail-label shuffling changes which segments are heavy, as well as their arrangement. After examining this primary control, we conducted an exploratory segment-order test that conditions on the observed heavy-segment counts. For response i, we retain the original positions of its mim_i full segments and uniformly choose hih_i of these positions to carry heavy labels, preserving the observed heavy-segment count. Partial segments are excluded, and gaps do not create new adjacency edges. Let EiE_i be the set of genuinely adjacent full-segment pairs and T the total number of heavy–heavy pairs. Its random-order expectation is 0[T]=∑i:mi≥2|Ei|hi(hi−1)mi(mi−1).E_0[T]= _i:m_i≥ 2|E_i| h_i(h_i-1)m_i(m_i-1). At pre-terminal, observed counts are 2,8922,892 versus an expectation of 1,613.851,613.85 on 4B, and 1,5841,584 versus 1,055.581,055.58 on 30B, giving enrichments of 1.79×1.79× and 1.50×1.50×. This comparison identifies non-random ordering conditional on the observed heavy-segment counts. It is a different statistic from PadjP_adj and does not exclude within-response position trends or establish causal propagation along the response. B.5 Full-Resolution Characterization Trajectories Figure 8 places the four stage-wise summaries of Section 3.2 in the full saved trajectory: updates 1–324 for 4B and 1–724 for 30B. For each update, we merge the data-parallel shards and compute the fraction of valid completion tokens with |δ|<0.05|δ|<0.05 and the exact 0.050.05 and 0.010.01 gap quantiles. These marginal statistics include zero-advantage tokens. The ordinary 4B gap and routing-conditioned 30B residual retain their distinct estimands. The directional panels compare raw |Ai||A_i| weights with response-length-normalized |Ai|/ni|A_i|/n_i weights over the strict regions −=Ai<0,δi,t<0A^-=\A_i<0, _i,t<0\ and +=Ai>0,δi,t>0A^+=\A_i>0, _i,t>0\. Faint curves retain per-update ratios; bold curves pool numerator and denominator weights over trailing 25-update windows before taking their ratio. They are not moving averages of per-update ratios. Tokens with δ=0δ=0 belong to neither strict region, and tokens with Ai=0A_i=0 carry zero weight. The complete trajectories show temporal variation that is lost when updates are pooled into broad consecutive windows. The directional ratio is non-monotonic, and length normalization reduces the raw-weighted imbalance, particularly on 30B. The bulk and tail panels locate these changes relative to the recorded terminal regime; the four shaded windows remain descriptive summaries of one characterization run per model. Section B.6 gives the corresponding stage-wise sensitivity results. Figure 8: Bulk, tail, and directional summaries along the full characterization trajectories. Top: fraction of all valid completion tokens with |δ|<0.05|δ|<0.05 at each update. Middle: exact per-update signed gap quantiles q0.05q_0.05 and q0.01q_0.01. Bottom: raw and length-normalized −/+A^-/A^+ weight ratios, with per-update values shown faintly and trailing 25-update pooled values in bold. The middle and bottom rows use symmetric-logarithmic vertical axes. Gray spans mark the original analysis windows; red lines mark the run-specific collapse triggers (4B: 304; 30B: 724). The 30B endpoint uses the pre-update snapshot of the failed update. B.6 Sensitivity to Aggregation Choices We examine two choices in the directional summaries: the trailing update-window width and the weighting assigned to each valid token. For a region ℛR, define wq(ℛ)=∑i,tqi,t 1[(i,t)∈ℛ]∑i,tqi,t,ρq=wq(−)wq(+).w_q(R)= _i,tq_i,t\,1[(i,t) ] _i,tq_i,t, _q= w_q(A^-)w_q(A^+). The stage-based summaries in Table 8 retain the 25-update windows of Section B.2, shown as gray spans in Figure 8. The primary summary uses qi,t=|Ai|q_i,t= A_i . The alternative uses qi,t=|Ai|/niq_i,t= A_i /n_i, where nin_i is the valid completion length of response i, to include the response-mean reduction factor. Table 8: Sensitivity of directional summaries. The window comparison holds raw advantage weighting fixed. The weighting comparison holds each 25-update stage window fixed. Masses are percentages of the corresponding total weight. (a) Window width: 25 versus 50 trailing updates Model Stage ρ25 _25 ρ50 _50 w25(−)w_25(A^-) w50(−)w_50(A^-) 4B Pre-terminal 2.319 2.454 36.18 36.90 30B Pre-terminal 1.851 1.735 19.50 18.80 (b) Weight definition: raw versus length-normalized Model Stage ρraw _ raw ρeff _ eff wraw(−)w_ raw(A^-) weff(−)w_ eff(A^-) 4B Healthy 1.023 0.654 25.11 19.98 4B Drift 1.504 0.796 28.92 20.83 4B Pre-terminal 2.319 1.434 36.18 28.51 4B Terminal 1.936 1.297 28.38 25.30 30B Healthy 1.337 0.963 23.98 20.30 30B Drift 1.488 1.019 17.43 14.03 30B Pre-terminal 1.851 1.049 19.50 14.17 30B Terminal 3.391 1.652 31.13 22.00 Extending the pre-terminal window from 25 to 50 updates changes the raw-weighted ratio from 2.319 to 2.454 on 4B and from 1.851 to 1.735 on 30B. Both windows retain the raw-weighted pre-terminal imbalance. The intervals are 219–243 versus 194–243 for 4B and 519–543 versus 494–543 for 30B. Length normalization changes the relative influence of long responses. It reduces the pre-terminal ratio to 1.434 on 4B and 1.049 on 30B, and changes whether the ratio exceeds one in some earlier stages. Raw advantage-weighted token mass and length-normalized coefficient mass therefore answer different questions. We retain the former for Section 3.2 and report the latter explicitly here. Both −A^- and +A^+ are amplifying regions under the diagonal sign criterion, so their ratio describes a directional allocation within that part of the decomposition, not the balance between all amplifying and contracting contributions. Appendix C TRIAGE Specification and Derivations This appendix specifies the segment gate, derives the response-level reweighting in Section 4.1, and derives the repair coefficient in Section 4.2. Throughout, advantages, sampler log probabilities, masks, and gate weights are held fixed during each learner update. The learner log probabilities used by repair retain their gradients. C.1 Segment Construction and the Gate Operating Rule Let mi,t∈0,1m_i,t∈\0,1\ be the valid completion-token mask and ni=∑tmi,tn_i= _tm_i,t. Diagnostic segments contain consecutive response positions and do not cross response boundaries. We retain the final shorter segment and omit segments with no valid tokens. Masked positions contribute neither to segment statistics nor to the loss. For a segment S, let nS=∑t∈Smi,t>0n_S= _t∈ Sm_i,t>0. The two diagnostic statistics are those in Eq. 13. For outcome-based GRPO, the advantage is constant within a response. Consequently, a segment is eligible for gating exactly when Ai<0A_i<0. Two sources of risk determine its weight: a negative mean gap and a concentration of deep negative-gap tokens. The mean-gap component uses thresholds τsev<τneg<0 _sev< _neg<0 and weights 0<wsev≤wneg≤10<w_sev≤ w_neg≤ 1: wSmean=wsev,δ¯S<τsev,wneg,τsev≤δ¯S<τneg,1,δ¯S≥τneg.w_S^mean= casesw_sev,& δ_S< _sev,\\ w_neg,& _sev≤ δ_S< _neg,\\ 1,& δ_S≥ _neg. cases (52) The number of deep-tail tokens and the tail component are kStail=∑t∈Smi,t[δi,t<τtail]=nSfStail,wStail=1−(1−wsev)min(kStailKtail,1),k_S^tail= _t∈ Sm_i,t1[ _i,t< _tail]=n_Sf_S^tail, w_S^tail=1-(1-w_sev) \! ( k_S^tailK_tail,1 ), (53) where Ktail>0K_tail>0 is the count at which tail attenuation reaches wsevw_sev. Combining the two components gives wS=min(wSmean,wStail),Ai<0,1,Ai≥0.w_S= cases (w_S^mean,w_S^tail),&A_i<0,\\ 1,&A_i≥ 0. cases (54) Thus wmin=wsevw_ =w_sev. In the compact notation of Eq. 14, G uses the segment’s valid count when converting fStailf_S^tail to kStailk_S^tail. This accounts for a short final segment or masked positions without introducing a separate gate symbol. Finally, Eq. 15 applies wSw_S only to negative-gap tokens in an eligible segment. The remaining tokens retain raw weight one. C.2 From the Weighted Loss to the Effective Update Write Hi=∑tmi,twi,tH_i= _tm_i,tw_i,t and w¯i=Hi/ni w_i=H_i/n_i. Because the weights are positive on valid tokens, Hi>0H_i>0 for every nonempty response. Treating the gate as a constant during backpropagation gives ∇θℒPG,iTRIAGE=1Hi∑tmi,twi,t∇θℓi,tbase=1ni∑tmi,twi,tw¯i∇θℓi,tbase. _θL_PG,i^TRIAGE= 1H_i _tm_i,tw_i,t _θ _i,t^base= 1n_i _tm_i,t w_i,t w_i _θ _i,t^base. (55) This recovers αi,t=wi,t/w¯i _i,t=w_i,t/ w_i in Eq. 17, including the original response and batch reduction factors in gi,tbaseg_i,t^base. In particular, 1ni∑tmi,tαi,t=1,αi,tαi,u=wi,twi,ufor valid tokens t,u. 1n_i _tm_i,t _i,t=1, _i,t _i,u= w_i,tw_i,u valid tokens t,u. (56) The gate therefore changes relative allocation within a response. An ungated token has αi,t=1/w¯i≥1 _i,t=1/ w_i≥ 1; an attenuated raw weight need not yield αi,t<1 _i,t<1. If every valid token receives the same raw weight, all effective multipliers are one. The unit average of α does not impose a conservation law on the parameter-gradient norm or on the mismatch-energy contributions. For comparison, a fixed-denominator variant uses ℒPG,ifixed=1ni∑tmi,twi,tℓi,tbase,αi,tfixed=wi,t.L_PG,i^fixed= 1n_i _tm_i,tw_i,t _i,t^base, _i,t^fixed=w_i,t. (57) This variant changes the response’s overall multiplier as well as its within-response allocation. C.3 Repair Objective and Additive Coefficient Repair uses non-overlapping segments R, with response index i(R)i(R), valid count nR=∑t∈Rmi(R),t>0n_R= _t∈ Rm_i(R),t>0, and mean gap δ¯R=1nR∑t∈Rmi(R),tδi(R),t,qR=[τrep−δ¯R]+. δ_R= 1n_R _t∈ Rm_i(R),t _i(R),t, q_R=[ _rep- δ_R]_+. (58) Here δi,t=zi,t(θ)−logπs(yi,t∣x,yi,<t) _i,t=z_i,t(θ)- _s(y_i,t x,y_i,<t), using the same sampler notation as the main text. Let repvalidS_rep^valid contain all nonempty repair segments in the optimizer-update batch and Nrep=|repvalid|N_rep=|S_rep^valid|. The denominator counts every such segment, including segments with qR=0q_R=0 and segments from non-positive-advantage responses. For Nrep>0N_rep>0, define ℒrep,k=λkNrep∑R∈repvalid[Ai(R)>0]ϕβ(qR),λk=λrep[k≥krep].L_rep,k= _kN_rep _R _rep^valid1[A_i(R)>0] _β(q_R), _k= _rep1[k≥ k_rep]. (59) When the valid segment set is empty, the repair term and its gradient are zero. Counting all valid segments keeps the denominator independent of which segments violate the repair boundary. For a token t∈Rt∈ R with i=i(R)i=i(R) in an active segment, the sampler term is fixed and ∂qR∂zi,t ∂ q_R∂ z_i,t =−mi,tnR, =- m_i,tn_R, (60) ϕβ′(q) _β (q) =q1+(q/β)2. = q 1+(q/β)^2. (61) At qR=0q_R=0 the composed penalty has zero derivative. Let R(i,t)R(i,t) be the unique repair segment containing a valid token. Gradient descent on Eq. 59 contributes Δθrep _rep =−η∇θℒrep,k=η∑i,tψi,t∇θzi,t, =-η _θL_rep,k=η _i,t _i,t _θz_i,t, (62) ψi,t _i,t =λkNrepmi,tnR(i,t)ϕβ′(qR(i,t))[Ai>0,qR(i,t)>0]. = _kN_rep m_i,tn_R(i,t) _β (q_R(i,t))1[A_i>0,\ q_R(i,t)>0]. (63) The coefficient is zero for tokens outside the valid repair set. Equation 63 recovers Eq. 20, including the positive-advantage condition and the activation schedule. Bounded corrective coefficients. For β>0β>0 and finite q≥0q≥ 0, the pseudo-Huber penalty obeys ϕβ(q) _β(q) ∼12q2 12q^2 (q/β→0), (q/β→ 0), (64) ϕβ(q) _β(q) =βq−β2+(β3/q) =β q-β^2+O(β^3/q) (q/β→∞), (q/β→∞), 0 0 ≤ϕβ′(q)<β. ≤ _β (q)<β. Consequently, throughout training, 0≤ψi,t≤λkβNrepnR(i,t)0≤ _i,t≤ _kβN_repn_R(i,t) (65) for valid tokens and nonempty repair sets. The upper inequality is strict when λk>0 _k>0. The non-strict form also covers the period before repair is enabled. This bound controls the coefficient multiplying each log-probability gradient; a bound on the parameter-gradient norm would additionally depend on the norms of those gradients. Repair on positive-advantage segments. When λk>0 _k>0, an active positive-advantage segment with qR>0q_R>0 assigns the same positive coefficient to all valid tokens. Negative-gap tokens therefore have δi,tψi,t<0 _i,t _i,t<0 in the diagonal decomposition of Section 3.1. Positive-gap tokens in the same segment also receive this coefficient. Thus the objective corrects negative segment-level displacement without imposing contraction on every token. C.4 Implementation and Configuration Gate diagnostics and weights are detached. The positive-advantage repair branch retains live learner gaps for valid tokens in a segment with Ai>0A_i>0. For the remaining tokens, detached copies preserve the segment mean used by the diagnostic computation. The loss numerator then includes only selected positive-advantage segments, while the denominator counts all valid repair segments. Both the resulting repair value and its learner gradient therefore match Eq. 59. The activation boundary is inclusive: repair is enabled when the persisted rollout-update index reaches krepk_rep. With one learner update per rollout, this index supplies the schedule in Eq. 59. The schedule changes the repair coefficient; gate diagnostics, segment counts, and the policy objective are unchanged before activation. To implement the response-mean policy objective in Eq. 16, first form each response’s weighted mean and then apply the outer response average. The repair denominator is computed over the full optimizer-update batch. If the loss interface subsequently divides accumulated numerators by one denominator for the complete update, scale each local selected repair sum by that denominator divided by NrepN_rep before the outer division. The result is the same global segment mean regardless of the data-parallel or microbatch partition. Table 9: Gate and repair configuration. The diagnostic segment length matches the 64-token length used in characterization; τana _ana remains a separate characterization threshold. Repair starts inclusively at the persisted, zero-based rollout-update index krepk_rep. Parameter Meaning Value WgateW_gate Diagnostic segment length 6464 τneg _neg Negative mean-gap threshold −0.5-0.5 τsev _sev Severe mean-gap threshold −1.5-1.5 τtail _tail Deep-tail token threshold −6.0-6.0 wnegw_neg Negative mean-gap weight 0.30.3 wsevw_sev Severe/minimum raw weight 0.10.1 KtailK_tail Tail-count saturation 22 WrepW_rep Repair segment length 128128 τrep _rep Mean-gap repair boundary −0.03-0.03 β Pseudo-Huber scale 0.0220.022 λrep _rep Repair coefficient after activation 0.0150.015 krepk_rep Inclusive repair start index 00 One learner update. The following procedure implements the objective in Eq. 19: 1. Obtain learner log probabilities on the sampled prefixes and subtract the fixed generation-time sampler log probabilities. Retain a live gap for repair and a detached copy for gate diagnostics. 2. Partition each response into diagnostic segments, compute the masked statistics, and evaluate Eqs. 52–54. Apply the resulting weights only to the negative-gap tokens of negative-advantage responses. 3. Compute each response’s weighted policy mean and average these response losses using the baseline’s batch reduction. 4. Partition responses into repair segments and count every nonempty segment across the optimizer-update batch. Set λk _k from the fixed schedule and evaluate the positive-advantage repair objective. 5. Add the two losses and take the learner optimizer step. The effective current-gradient coefficient is gi,t=αi,tgi,tbase+ψi,tg_i,t= _i,tg_i,t^base+ _i,t; optimizer-history effects are represented by the residual in Appendix A. Appendix D Training and Evaluation Protocols D.1 Main Training Comparisons The main comparisons in Section 5 evaluate training stability and downstream reasoning performance with Qwen3-30B-A3B-Base as the primary model and Qwen3-4B-Base for replication and ablation. We use the DAPO dataset for training and GRPO for policy optimization. The characterization runs in Section 3 instead provide the fixed trajectories used for directional and localization analysis. Their run-specific horizons, collapse criteria, and mismatch estimands are specified in Appendix B.1 and Appendix B.2. The main training configurations are BF16, native NVFP4, NVFP4 with token-level truncated importance sampling (TIS), and NVFP4 with TRIAGE. The TIS truncation threshold is C=2.0C=2.0. TRIAGE applies the gating and repair operations in Section 4 to the importance-corrected objective. The native NVFP4 configurations use W4A4 forward execution on the sampler and learner, with the activation-scaling recipe described in Section 5.1. The maximum response length in the main training setup is 20,48020,480 tokens. D.2 Downstream Evaluation We evaluate mathematical reasoning on AIME 2024, AIME 2025, MATH500, AMC23, and GSM8K. Table 1 identifies the training step of each evaluated checkpoint. Its † marker denotes a run that collapses before its planned budget; the reported result then uses the last saved checkpoint preceding collapse. The Qwen3-4B-Base TRIAGE results include the intermediate step-300 checkpoint and the final step-600 checkpoint. To summarize the five benchmark scores, we use the unweighted mean Avg=15∑b=15Scoreb,Avg= 15 _b=1^5Score_b, (66) where each ScorebScore_b is expressed as a percentage. This gives each benchmark equal weight, independently of its number of problems. Training reward, rollout pass rate, and the mean absolute learner–sampler log-probability gap are tracked separately as training diagnostics in Figure 4. Appendix E Efficiency and Profiling E.1 Natural-Generation RL Measurements The rollout-throughput measurements in Table 2 use Qwen3-30B-A3B-Base on one node with eight NVIDIA Blackwell B300 GPUs. Slime coordinates colocated Megatron-LM learner and SGLang sampler execution. The learner parallel configuration is tensor, pipeline, and context parallel size 1, expert parallel size 8, and expert tensor parallel size 1. Each prompt has 16 sampled responses. Global batch sizes 256 and 512 therefore correspond to 16 and 32 prompts, respectively. Each run starts from the base checkpoint and trains on DAPO-Math-17K with seed 42 and the same data order. Responses terminate naturally at EOS or at the 20,48020,480-token limit. A run contains 12 rollout iterations: iterations 0–1 are warm-up and iterations 2–11 supply the ten measured observations. Initialization, evaluation, and checkpoint saving are excluded. For measured iteration j, let MjM_j be the number of generated completion tokens and TjrolloutT_j^rollout the rollout duration. The reported throughput is the arithmetic mean of per-iteration throughputs, TPS¯=110∑j=211MjTjrollout. TPS= 110 _j=2^11 M_jT_j^rollout. (67) This counts the actual output tokens produced by each precision configuration. Iteration wall time includes rollout, learner updates, weight conversion and synchronization, and orchestration. Learner timing includes log-probability recomputation and the backward and optimizer update. In colocated execution, overlapping wait intervals are not added again to the wall-time total. E.2 Paired Learner-Overhead Measurement The incremental learner cost of TRIAGE is measured separately on eight B300 GPUs with five paired repetitions of the same frozen batch. Gating and repair are enabled for one member of each pair. The observed median learner-update overhead is 0.84%0.84\%, with a median absolute increase of 0.2250.225 s. This comparison holds the training workload fixed when measuring the additional objective computation; natural-generation iteration times also reflect changes in output lengths. E.3 Fixed-output sampler evaluation E.3.1 Independent Fixed-Output Configuration Selection We additionally measure standalone sampler throughput with a fixed output workload. For Qwen3-30B-A3B, each batch contains 512 responses of 4,096 output tokens with a client concurrency limit of 512 on one B300 GPU. Throughput divides the total output tokens by the complete client batch time, including queuing and final request completion. The KV cache uses BF16. This sampler experiment uses PyTorch 2.9.1+cu130, SGLang 0.5.10, FlashInfer 0.6.12, and sglang-kernel 0.4.1+cu130. Table 10: Standalone Qwen3-30B-A3B throughput after configuration selection. Ranges are min–max over five repetitions. Speedup is the NVFP4/BF16 median-throughput ratio. Precision Mem. frac. Median Range Speedup (output tokens/s) BF16 0.95 15,470.01 15,447.56–15,509.83 1.000× NVFP4 0.90 18,175.67 18,126.37–18,195.69 1.175× Both precisions receive a 12-candidate configuration search. The two leading candidates are remeasured, and the selected configuration for each precision is measured five times on the same physical GPU. Both selected configurations use the FlashInfer TRT-LLM MoE backend. The selected static memory fractions are 0.95 for BF16 and 0.90 for NVFP4. Table 10 reports the resulting medians and observed ranges. Under these independently selected configurations, NVFP4 throughput is 1.175× the BF16 throughput. E.3.2 Concurrency and output length Figure 9 varies the serving operating point with fixed output workloads. The concurrency sweep uses a common starting configuration with memory fraction 0.8. Length sweeps retain the precision-specific selected configurations in Table 10. Figure 9: Fixed-output serving sweeps on Qwen3-30B-A3B. Each batch contains 512 responses on one B300 GPU. (a) Concurrency at 4K output tokens. The row below gives NVFP4/BF16 throughput ratios. (b,c) Length at concurrency 64 and 512. K denotes 1,024 tokens. Points show medians and whiskers min–max over three runs, except the separately confirmed C512/4K point (diamond), which uses five. The common-configuration C64/4K point is omitted from the selected-configuration length curve. Throughput increases with concurrency over the measured 4K sweep. At fixed concurrency, it decreases as output length grows, and the relative gain varies with the operating point. These measurements complement Section 5.3 by holding the output workload fixed. E.4 Controlled Long-Tail Workload We isolate response-length heterogeneity with a fixed request-to-length mapping on eight independent single-GPU Qwen3-30B-A3B sampler replicas. At batch size 256, the mixed workload contains 224 responses of 2,048 tokens, 24 of 8,192 tokens, and eight of 20,480 tokens. Batch size 512 doubles each count. A uniform control assigns 3,200 tokens per response, preserving the total output-token count. The mapping uses seed 20260912, with identical requests and target lengths for both precisions. For mixed lengths, median batch time decreases from 156.13 to 120.83 s at batch size 256 and from 165.26 to 129.28 s at batch size 512. The time ratios are 1.292× and 1.278×. The equal-token uniform controls yield larger ratios of 1.538× and 1.513×. NVFP4 therefore reduces absolute waiting time in these long-tail batches by approximately 35–36 s, while the relative gain is smaller than for uniform outputs. The completion curves expose the late portion of the batch that average throughput alone does not describe. These controlled sampler workloads complement Section 5.3 by fixing the output-length assignment. Figure 10: Controlled long-tail and equal-token uniform workloads. (a) Complete-batch wall time. Bars show medians, points individual runs, and whiskers min–max. Labels are BF16/NVFP4 median-time ratios. (b) Completion-time quantiles for the final 10% of requests in the 256-response mixed workload, with pointwise median and min–max across five repetitions. Batch size 256 uses five repeats per condition and batch size 512 uses three. Timing includes request handling, returned log-probabilities, and final draining, excluding initialization and warm-up.