Paper deep dive
SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning
Haonan He, Haodi Lei, Yun Luo, Haoran Zhang, Shunkai Zhang, Yizhuo Li, Shengji Tang, Zhilin Wang, Runzhe Zhan, Lei Bai, Ganqu Cui, Fangchen Yu, Yafu Li, Peng Ye, Ning Ding, Yu Cheng
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/17/2026, 5:25:25 AM
Summary
The paper introduces SimpleOPD, a method for transferring reasoning capabilities from long-context teacher models (SU-01) to short-context student models via on-policy distillation (OPD). It addresses challenges such as tokenizer mismatch, distribution mismatch, and training instability (length explosion, truncation) by aligning tokens in a shared text space, masking termination token advantages, and applying a student reference KL loss. The method demonstrates significant improvements in mathematical reasoning benchmarks like ProofBench, with Intern-S2-Preview surpassing Gemini-2.5-Pro.
Entities (10)
Relation Signals (8)
Intern-S2-Preview → achievedscoreon → ProofBench
confidence 95% · Intern-S2-Preview improves by 21.2 points on ProofBench, reaching 55.2
SimpleOPD → mitigates → tokenizer mismatch
confidence 95% · To handle tokenizer differences, we perform OPD in a shared text space
SimpleOPD → mitigates → training instability
confidence 95% · This strategy constrains the student from drifting excessively from its initial policy, thereby mitigating the teacher-student distribution mismatch problem and fostering steady length growth.
SimpleOPD → uses → On-Policy Distillation
confidence 95% · SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning
SimpleOPD → uses → Cross-tokenizer Alignment
confidence 95% · To handle tokenizer differences, we perform OPD in a shared text space and align only tokens that occupy identical text spans
SimpleOPD → uses → KL loss
confidence 95% · introduce a student reference KL loss and mask the advantages of special termination tokens
SimpleOPD → appliedto → Qwen3
confidence 90% · Experiments on both same-family and different-family student models, including Qwen3
Intern-S2-Preview → surpasses → Gemini 2.5 Pro
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:On-policy distillation (OPD) offers a promising way to transfer reasoning capabilities from stronger teacher models, but applying it to long-context reasoning teachers and short-context students introduces practical challenges, including tokenizer mismatch, teacher-student distribution mismatch, response length explosion, and training instability. In this work, we study this setting by transferring proof-reasoning capabilities from the long-context reasoning model SU-01 to short-context student models. To handle tokenizer differences, we perform OPD in a shared text space and align only tokens that occupy identical text spans under the student and teacher tokenizers. To mitigate the problem of excessive generation length and frequent truncation, we introduce a student reference KL loss and mask the advantages of special termination tokens such as </think> and <|im_end|>. This strategy constrains the student from drifting excessively from its initial policy, thereby mitigating the teacher-student distribution mismatch problem and fostering steady length growth. Experiments on both same-family and different-family student models, including Qwen3, Qwen3.5, Intern-S2, GLM-4.7, Gemma-4, show consistent gains in mathematical reasoning, especially natural-language math proving. Notably, Intern-S2-Preview improves by 21.2 points on ProofBench, reaching 55.2 and surpassing Gemini-2.5-Pro. It also improves on science benchmarks such as HLE and HiPhO, suggesting that OPD transfers reasoning capabilities that generalize beyond the mathematical training domain.
Tags
Links
- Source: https://arxiv.org/abs/2608.14277v1
- Canonical: https://arxiv.org/abs/2608.14277v1
Trouble viewing inline? Open PDF directly →
Full Text
70,983 characters extracted from source content.
Expand or collapse full text
SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning SimpleOPD: Simple Tokenizer-Agnostic On-Policy Dis- tillation for Long-Context Reasoning Haonan He * Haodi Lei * Yun Luo †,‡ Haoran Zhang Shunkai Zhang Yizhuo Li Shengji Tang Zhilin Wang Runzhe Zhan Lei Bai Ganqu Cui Fangchen Yu ‡ Yafu Li ‡ Peng Ye ‡ Ning Ding ‡ Yu Cheng ‡ SU-01 Team, Shanghai Artificial Intelligence Laboratory * Equal Contribution, † Project Lead, ‡ Corresponding Author # luoyun@pjlab.org.cn ABSTRACT On-policy distillation (OPD) offers a promising way to transfer reasoning capa- bilities from stronger teacher models, but applying it to long-context reasoning teachers and short-context students introduces practical challenges, including tokenizer mismatch, teacher-student distribution mismatch, response length explosion, and training instability. In this work, we study this setting by transfer- ring proof-reasoning capabilities from the long-context reasoning model SU-01 to short-context student models. To handle tokenizer differences, we perform OPD in a shared text space and align only tokens that occupy identical text spans under the student and teacher tokenizers. To mitigate the problem of excessive generation length and frequent truncation, we introduce a student reference KL loss and mask the advantages of special termination tokens such as</think>and<|imend| >. This strategy constrains the student from drift- ing excessively from its initial policy, thereby mitigating the teacher-student distribution mismatch problem and fostering steady length growth. Experi- ments on both same-family and different-family student models, including Qwen3, Qwen3.5, Intern-S2, GLM-4.7, Gemma-4, show consistent gains in mathematical reasoning, especially natural-language math proving. Notably, Intern-S2-Preview improves by 21.2 points on ProofBench, reaching 55.2 and surpassing Gemini-2.5-Pro. It also improves on science benchmarks such as HLE and HiPhO, suggesting that OPD transfers reasoning capabilities that generalize beyond the mathematical training domain. Project PageCodeModelsz August, 2026 SU-01 Deepseek-V3.2- Speciale Intern-S2- OPD Nemotron- Cascade-2 Gemini-2.5- Pro GPT-5 Intern-S2- Preview 30 40 50 60 Score (%) 57.6 56.0 55.2 52.9 47.2 39.5 34.0 Open-sourceClosed-sourceHighlight Models +21.20 (+62.3%) IMO-Proofbench Benchmark Figure 1: Evaluation performance on ProofBench using Gemini-2.5-Pro as the judge, under the same evaluation setting as SU-01. With SimpleOPD, Intern-S2-OPD achieves performance comparable to DeepSeek-V3.2-Speciale, demonstrating the effectiveness of our OPD strategy. * Work done during an internship at Shanghai Artificial Intelligence Laboratory. 1 arXiv:2608.14277v1 [cs.CL] 14 Aug 2026 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Contents 1 Introduction3 2 Method4 2.1Cross-tokenizer Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4 2.2On-policy distillation objective . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 3 Experiment6 3.1Training Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 3.2Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 4 Experimental Results7 4.1Training Instability in On-Policy Distillation . . . . . . . . . . . . . . . . . . . . .7 4.2Results on Same-family Models . . . . . . . . . . . . . . . . . . . . . . . . . . .8 4.3Results on Cross-family Models . . . . . . . . . . . . . . . . . . . . . . . . . . .9 5 Analysis10 5.1Comparison with OPD Baselines . . . . . . . . . . . . . . . . . . . . . . . . . . .10 5.2Lexical Overlap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10 5.3Out-of-domain Generalization . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 5.4Effect of Training Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 5.5Effect of OPD Length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12 5.6Distillation from DeepSeek-V4-Flash . . . . . . . . . . . . . . . . . . . . . . . .12 5.7Effect of Student-Reference KL Loss . . . . . . . . . . . . . . . . . . . . . . . . .12 6 Related Work13 6.1Olympiad-Level Long-Context Reasoning . . . . . . . . . . . . . . . . . . . . . .13 6.2On-policy Distillation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13 7 Conclusion14 A Cross-Tokenizer Scoring and Alignment18 B Model Solutions for ProofBench19 B.1 ProofBench-Basic-001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 B.2 ProofBench-Advanced-028 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 C Case Study23 C.1 Case A: correct answer, then 972 exact self-check repeats (AIME25) . . . . . . . .23 C.2 Case B: reasoning collapses into a one-token loop (AnswerBench) . . . . . . . . .23 2 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning 1 Introduction On-policy distillation (OPD) (Gu et al., 2024; Agarwal et al., 2024) presents a compelling paradigm in the post-training landscape of LLMs for tasks such as strong-to-weak distillation (Lu & Lab, 2025), multi-teacher distillation (Ma et al., 2026), and self-distillation (H ̈ ubotter et al., 2026). Unlike off-policy approaches that distill knowledge from fixed teacher-generated data, OPD actively queries the teacher to evaluate tokens in student-generated trajectories, thereby providing dense token-level supervision grounded in the student’s own policy (Gu et al., 2024; Agarwal et al., 2024), enabling efficient distillation of complex reasoning behaviors, as it incurs less forgetting and achieves better generalization than off-policy settings. However, prior work has primarily studied teacher–student pairs that share the same model family and vocabulary. Distilling reasoning capabilities from long- context teachers into short-context students, especially across different model families, remains largely underexplored (Sun et al., 2026). Applying OPD to distilling long-context reasoning teachers to short-context reasoning students presents several challenges. First, teacher and student models may use different tokenizers, making direct alignment between their token-level distributions infeasible (Zhang et al., 2024; Chen et al., 2025b; Singh et al., 2026). Second, differences in model capacity and context length introduce a substantial teacher–student distribution mismatch (Li et al., 2026b). In particular, long-context reasoning teachers often favor responses that exceed the student’s context budget. Through exper- iments, we observe that direct OPD can cause rapid response-length growth, frequent truncation, and incomplete reasoning trajectories, ultimately destabilizing training. This issue is especially pronounced for termination tokens such as</think>and<|imend| >, as teacher supervision may repeatedly discourage the student from terminating its thinking process or final response. In this work, we focus on the transfer of reasoning capabilities from the long-context reasoning model SU-01 (Li et al., 2026a) to short-context student models. As shown in Figure 2, to enable distillation across different model families, we perform OPD in a shared text space and align teacher and student tokens only when they occupy identical text spans. This provides reliable token-level supervision without requiring an artificial correspondence between incompatible tokenizations. To mitigate excessive response-length growth and stabilize optimization, we introduce a reference KL- divergence loss between the student and its initial policy. We also mask the advantages of special termination tokens, preventing teacher supervision from directly suppressing termination. Together, these techniques constrain excessive policy drift, alleviate teacher–student distribution mismatch, and promote steady response-length growth within the student’s context budget. We evaluate SimpleOPD across both same-family and cross-family distillation settings, using student models from Qwen3, Qwen3.5, Intern-S2-Preview, GLM-4.7, and Gemma-4 (Yang et al., 2025; Qwen Team, 2026; InternLM Team, 2026; Z.ai, 2026; Gemma Team, 2026). Across model families, SimpleOPD consistently improves mathematical reasoning performance, with particularly strong gains on natural-language mathematical proof tasks. Most notably, Intern-S2-Preview improves by 21.2 points on ProofBench (Luong et al., 2025), from 34.0 to 55.2, surpassing Gemini-2.5- Pro and approaching the performance of substantially stronger reasoning models (Figure 1). The distilled students also achieve improvements on science-oriented benchmarks, including HLE and HiPhO (Phan et al., 2025; Yu et al., 2025), despite being trained exclusively on mathematical reasoning data. These results demonstrate that stable cross-tokenizer OPD can transfer reasoning capabilities from long-context teachers, and that the acquired capabilities generalize beyond the training domain. In summary, our observations and contributions are: • Cross-tokenizer OPD can improve student models without full token-level alignment by using student-generated text as a shared space and aligning tokens with identical text spans. •Direct OPD transfers long-context reasoning capabilities, but it often causes excessive length growth, frequent truncation, and training instability. • Termination-token advantage masking and student reference KL loss stabilize training and reduce truncation. Larger teacher–student gaps generally require stronger KL regularization. •SimpleOPD shows consistent performance gains across model families without supervised fine-tuning on teacher trajectories, while moderately increasing the distillation length can further improve long-form reasoning. 3 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Figure 2: Overview of SimpleOPD. The student generates responses under its own tokenizer, while the teacher evaluates the same response under its native tokenizer. We align tokens with identical text spans, use teacher supervision for matched tokens, and stabilize training with KL regularization and termination-token masking. 2 Method Letxdenote the input conversation represented as a list of messages. The student and teacher with different tokenizers may use different chat templates, denoted byC θ andC φ , respectively. We first construct the student input context textc θ =C θ (x)and sample a response-token sequence with the student policy executed by the rollout engine π θ roll : y 1:n = (y 1 ,...,y n )∼ π θ roll (·| c θ ).(1) Decodingy 1:n via the tokenizer decoder of the student gives the response surface strings =D θ (y 1:n ). Instead of passing the student context to the teacher, we reconstruct the teacher context textc φ using its own chat template and append the student response: c φ =C φ (x), u φ = c φ ⊕ s, where⊕denotes string concatenation. The complete textu φ is then provided to the teacher, which tokenizes it using the tokenizer encoderE φ of the teacher. LetE φ (s|c φ ) = (z 1 ,...,z m )denote token list obtained by encoding the student response given the input contextc φ using the teacher tokenizer. This procedure allows the teacher to evaluate the student response under its native tokenizer and chat-template. Althoughc θ andc φ may differ, the response text being evaluated is identical on both sides. For token alignment, letτ θ (y t )andτ φ (z i )denote the incremental text spans contributed by the corresponding response tokens. They satisfy n M t=1 τ θ (y t ) = m M i=1 τ φ (z i ) = s. We perform no additional cleanup or normalization, so these spans define consistent offsets in the shared response string. The teacher log-probability associated with token z i is logπ φ (z i | c φ ,z <i ). 2.1 Cross-tokenizer Alignment Define the cumulative response text preceding each student and teacher token as P θ (t) = t−1 M k=1 τ θ (y k ), P φ (i) = i−1 M k=1 τ φ (z k ), 4 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning withP θ (1) = P φ (1) = ε, whereεdenotes the empty string. A teacher positioniis aligned with a student positiontif both tokenizations have consumed the same response prefix and the current tokens contribute the same text span: M =(i,t) : P φ (i) = P θ (t)∧ τ φ (z i ) = τ θ (y t ).(2) Equivalently, an aligned teacher-student pair covers the same start and end offsets in the shared response strings. Tokens that overlap only partially are not aligned because the log-probability of one teacher token cannot be uniquely assigned to multiple student tokens, nor can the log-probabilities of multiple teacher tokens be uniquely merged into a single student token. Sinceyandzare ordered segmentations of the same response string,Mis a partial one-to-one mapping: each token on either side can match at most one token on the other side. We enumerateMusing a linear two-pointer scan. The scan maintains the accumulated response prefixes on both sides. If the prefixes and current token spans agree, the two tokens are aligned and both pointers advance. Otherwise, the side that has consumed less response text advances until the two prefixes meet again. When both sides have consumed the same amount of text but the current token spans differ, both pointers advance. For each student position t, define the alignment indicator a t = 1[∃i such that (i,t)∈M].(3) Letlogπ θ (y t | c θ ,y <t )andlogπ φ (z i | c φ ,z <i )denote the log-probabilities of student policy and teacher policy, respectively. We construct a student-length teacher target as e ℓ φ t = logπ φ (z i | c φ ,z <i ), a t = 1, logπ θ (y t | c θ ,y <t ), a t = 0. (4) Thus, aligned positions inherit the corresponding teacher log-probability, while unmatched positions fall back to the student’s log-probability. We report the lexical overlap ratioρ = |M| n ,which measures the fraction of student response tokens that receive teacher supervision. The complete alignment procedure is provided in Algorithm 1 in Appendix A. 2.2 On-policy distillation objective The cross-tokenizer distillation objective is defined over aligned response positions: L Distill (θ) = E y∼π θ " n X t=1 logπ θ (y t | c θ ,y <t )− e ℓ φ t # .(5) This objective is a token-aligned surrogate for the reverse KL divergence. It compares the teacher and student probabilities only at positions where the two tokenizers induce the same local segmentation of the response string. When the tokenizers are identical, every student token is aligned with the corresponding teacher token, so a t = 1 and z t = y t for all t. The objective then reduces to L Distill (θ) = E y∼π θ log π θ (y | c θ ) π φ (y | c φ ) = D KL (π θ (·| c θ )∥ π φ (·| c φ )).(6) When the tokenizers differ, the same response string is factorized into different token sequences, so the exact token-level KL is not directly computable. The proposed objective instead applies teacher supervision only where the two tokenizations agree on both the token boundary and textual span. To enable multiple policy updates on the same rollout batch, we substitute the online log-probability logπ θ (y t | c θ ,y <t )in e ℓ φ t with its pre-update counterpartlogπ θ old (y t | c θ ,y <t )for all unmatched positions (a t = 0) and define the fixed policy advantages as b A t = e ℓ φ t − logπ θ old (y t | c θ ,y <t ),(7) using the PPO clipped policy loss. The importance-sampling ratio is r t = π θ (y t | c θ ,y <t ) π θ old (y t | c θ ,y <t ) .(8) The resulting objective is L θ =−E " n X t=1 min r t b A t , clip(r t , 1− ε, 1 + ε) b A t # .(9) 5 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning 020406080100 Global Step 0.90 0.95 1.00 Score AIME25 020406080100 Global Step 0% 10% 20% 30% Ratio AIME25 Truncated Ratio 020406080100 Global Step 0% 5% 10% Ratio AIME25 Repetition 020406080100 Global Step 20000 40000 60000 Length AIME25 Average Length 020406080100 Global Step 0.76 0.78 Score AnswerBench 020406080100 Global Step 0% 20% Ratio AnswerBench Truncated Ratio 020406080100 Global Step 0% 5% 10% Ratio AnswerBench Repetition 020406080100 Global Step 25000 50000 75000 100000 Length AnswerBench Average Length Figure 3: Training dynamics of Intern-S2-Preview during direct OPD training. 3 Experiment We adopt SU-01, an IMO gold-medal-level mathematical reasoning model developed by our team, as the teacher model. Our goal is to transfer its mathematical reasoning and proof-generation capabilities to a diverse set of student models through on-policy distillation. We consider both same-vocabulary and cross-tokenizer settings. For the same-tokenizer setting, we distill SU-01 into models from the Qwen3 family, including Qwen3-4B-thinking-2507 and Qwen3-30B-A3B-Thinking-2507 (abbreviated as Qwen3-4B and Qwen3-30B-A3B, respectively). For the cross-tokenizer setting, we further evaluate models with different tokenizer designs, including Qwen3.5-4B, Qwen3.5-35B-A3B, Intern-S2-Preview, GLM-4.7-Flash, and Gemma-4-26B-A4B-it (abbreviated as Gemma-4-26B-A4B). This setting allows us to examine whether cross-tokenizer OPD can transfer mathematical proof capabilities across different model architectures and tokenization schemes. 3.1 Training Data All training examples are mathematical proof problems. The collection combines curated proof corpora, community-contributed problems, and olympiad training materials to provide a varied set of proof-oriented prompts. Our training set consists of: 1. OPC: 63 problems from the Open Proof Corpus. 2. AoPS: 2,948 problems collected from the Art of Problem Solving community. 3. Books: 900 problems collected from online mathematical competition training books. 4. Shuzhimi: 617 problems sourced from the Shuzhimi Forum, an online Chinese mathematical problem-solving community, and Evan Chen’s olympiad materials. 3.2 Implementation Details We perform OPD using responses sampled from the student policy, with SU-01 serving as the teacher model. SU-01 is a 30B-A3B reasoning model based on Qwen3-30B-A3B with strong mathemati- cal proof capabilities, achieving gold-medal-level performance on olympiad-level evaluations and demonstrating particularly competitive results on proof-oriented benchmarks such as ProofBench. It is also capable of sustaining over 100K tokens of natural-language reasoning for difficult Olympiad problems, making it a powerful yet highly long-context teacher. During distillation, SU-01 provides token-level supervision at positions that can be aligned across the teacher and student tokenizations. We use Slime (Zhu et al., 2025) as the underlying framework for OPD and SGLang (Zheng et al., 2024) for evaluation rollout. The models are trained for 100 rollout iterations with a constant learning rate of 1e-6, a rollout batch size of 64, 4 responses sampled per prompt, and a maximum rollout 6 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning 020406080 Global Step 0.88 0.90 0.92 Score AIME25 020406080 Global Step 0% 20% 40% Ratio AIME25 Truncated Ratio 020406080 Global Step 0% 5% 10% 15% Ratio AIME25 Repetition 020406080 Global Step 20000 40000 60000 Length AIME25 Average Length 020406080 Global Step 0.60 0.65 0.70 Score AnswerBench 020406080 Global Step 0% 20% 40% 60% Ratio AnswerBench Truncated Ratio 020406080 Global Step 0% 5% 10% 15% Ratio AnswerBench Repetition 020406080 Global Step 50000 75000 100000 Length AnswerBench Average Length Figure 4: Training dynamics of Qwen3.5-35B-A3B during direct OPD training. length of 32K tokens for Qwen-Series, 6k for GLM-4.7 and Gemma-4. The policy objective uses a PPO clipping coefficient of 0.2. The policy is updated four times per rollout step. In our evaluation, we primarily evaluate the non-verifiable ProofBench and the verifiable Answer- Bench from the IMO-Bench suite (Luong et al., 2025), together with the verifiable benchmarks AIME25 (Mathematical Association of America, 2025) and AMOBench (An et al., 2025). For ProofBench, we use DeepSeek-V4-Flash as the judge for cost-efficient evaluation. To mitigate evaluation instability, we evaluate 4 times for the same proof rollout and choose the average result. For the latter three, we first use a rule-based verifier for testing; if the answer is correct, it is counted as correct, and if it is wrong, we then use GPT-OSS-120B (OpenAI et al., 2025) for evaluation. The reported results are the averages over 4 rollouts for ProofBench and 8 for the other three. We use the following evaluation settings: temperature 1.0, top-p 0.95, repetition penalty 1.0, and a maximum response length of 160,000 tokens. The best checkpoint is selected based on the average score of AIME@4 and AnswerBench@1. 4 Experimental Results 4.1 Training Instability in On-Policy Distillation In this section, we first examine training instability in direct on-policy distillation, including length explosion and truncation. Accordingly, we propose two mitigation strategies, namely masking special tokens and adding a student reference KL loss. 4.1.1 Initial Observation: Length Explosion and Truncation During OPD training, the student model Intern-S2-Preview shows modest improvements in task performance, but these gains are accompanied by clear signs of degeneration (Figure 3). Specifically, the truncation rate and repetition rate both increase substantially as training progresses, while the average response length also grows sharply. This phenomenon is even more pronounced for Qwen3.5- 35B-A3B (Figure 4), as its task performance deteriorates more noticeably, and the degradation is accompanied by a persistently high truncation rate and rapidly increasing response length. These results suggest that apparent performance gains can come at the cost of increasingly verbose and unstable generation behavior, revealing a trade-off between task performance and output quality. Case studies could be found in Appendix C. 4.1.2 Special Token Masking During training, we observe that the student model’s output length keeps increasing, and many generated responses fail to emit termination tokens such as<|imend| >. This suggests that the student is strongly affected by the long outputs from the teacher and gradually loses the ability to terminate properly. To mitigate this issue, we mask the OPD loss on the structural tokens</think> 7 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning 0255075100 Global Step 0.90 0.92 0.94 0.96 Score AIME25 0255075100 Global Step 0% 10% 20% Ratio AIME25 Truncated Ratio 0255075100 Global Step 0.74 0.76 0.78 Score AnswerBench 0255075100 Global Step 0% 10% 20% 30% 40% Ratio AnswerBench Truncated Ratio Figure 5: Intern-S2-Preview OPD training dynamics when masking the special tokens such as </think>and<|imend| >. The special-token masking alone helps mitigate length-related instabil- ity but cannot resolve the length expansion problem. 0255075100 Global Step 0.90 0.95 1.00 Score AIME25 0255075100 Global Step 0% 10% 20% Ratio AIME25 Truncated Ratio 0255075100 Global Step 0.76 0.77 0.78 0.79 0.80 Score AnswerBench 0255075100 Global Step 0% 10% 20% Ratio AnswerBench Truncated Ratio Figure 6: Intern-S2-Preview OPD training dynamics when adding student reference KL loss to prevent the student policy from deviating excessively from its initial distribution. The training could be stabilized, and the truncation rate is effectively reduced to nearly zero. and<|imend| >, as these tokens primarily control the output format and termination behavior and do not need to match the teacher distribution exactly. The validation curve is shown in Figure 5. The results suggest that the masking strategy helps mitigate length-related instability. However, in the later stage of training, the model’s length truncation rate still continues to increase sharply, indicating that special-token masking alone cannot fully resolve the length expansion problem. The evaluation results are shown in Table 1. The results show that masking the advantages of special termination tokens improves OPD training stability, but special token masking alone cannot fully resolve the length-expansion problem. 4.1.3 Student-Reference KL Loss We then attempt to introduce a student reference KL loss to prevent the student policy from deviating excessively from its initial distribution, thereby preserving the student model’s general capabilities during distillation. In this experiment, we set the student-reference kl-loss coefficient to 0.5. The validation curves during training are shown in Figure 6. As observed, the truncation rate is effectively reduced to nearly zero, while performance on AIME25 and AnswerBench is consistently improved. This indicates that the student reference KL loss can effectively mitigate the dramatic increase in response length during OPD training. The evaluation results are summarized in Table 1. The results demonstrate that adding the reference KL loss substantially improves OPD training while preserving the student model’s original capabilities. Specifically, OPD + Ref KL improves ProofBench@4 from 21.70 to 38.50, and further raises AnswerBench@8 and AIME25@8 to 79.10 and 95.80, respectively. 4.2 Results on Same-family Models Finally, we combine special token masking with the student reference KL loss, which yields larger performance improvements. We set kl-loss-coef to 0.5 for the Qwen-series models and Intern-S2- Preview. The experimental results are shown in Table 2. Overall, SimpleOPD consistently improves perfor- mance across model sizes, tokenizer settings, and model families. Among the same-tokenizer models, Qwen3-4B-OPD gains 12.30 points on ProofBench, 17.00 points on AnswerBench, and 19.58 points 8 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Table 1: OPD results from SU-01 to Intern-S2-Preview with masking termination tokens and adding student reference KL loss. ModelProofBench@4AnswerBench@8AIME25@8 SU-0145.0077.5094.60 Intern-S2-Preview21.7076.0388.33 OPD + Spec Mask38.10 (+∆ 16.40)77.60 (+∆ 1.57)95.00 (+∆ 6.67) OPD + Ref KL38.50 (+∆ 16.80)79.10 (+∆ 3.07)95.80 (+∆ 7.47) Table 2: Evaluation results of distilling SU-01 to students on ProofBench, AnswerBench, AIME25, and AMOBench. Green values indicate improvements over the corresponding base model. ModelProofBench@4AnswerBench@8AIME25@8AMOBench@8 SU-0145.0077.5094.6061.75 Same-Tokenizer Models Qwen3-4B11.4247.5071.2523.00 Qwen3-4B-OPD23.72 (+∆ 12.30)64.50 (+∆ 17.00)90.83 (+∆ 19.58)35.00 (+∆ 12.00) Qwen3-30B-A3B13.8059.1388.3336.50 Qwen3-30B-A3B-OPD36.47 (+∆ 22.67)74.46 (+∆ 15.33)93.75 (+∆ 5.42)52.75 (+∆ 16.25) Cross-tokenizer Models Qwen3.5-4B15.9060.9486.6732.00 Qwen3.5-4B-OPD28.61 (+∆ 12.71)67.84 (+∆ 6.90)91.67 (+∆ 5.00)51.25 (+∆ 19.25) Qwen3.5-35B-A3B26.7873.1694.6057.25 Qwen3.5-35B-A3B-OPD42.39 (+∆ 15.61)80.15 (+∆ 6.99)96.66 (+∆ 2.06)61.25 (+∆ 4.00) Intern-S2-Preview21.7076.0388.3358.00 Intern-S2-OPD44.50 (+∆ 22.80)80.10 (+∆ 4.07)95.00 (+∆ 6.67)59.50 (+∆ 1.50) on AIME25. Scaling the student to Qwen3-30B-A3B produces even greater proof-reasoning improve- ments; its ProofBench score increases by 22.67 points, from 13.80 to 36.47, while its AnswerBench and AIME25 scores improve by 15.33 and 5.42 points, respectively. The improvements also generalize to cross-tokenizer distillation. Qwen3.5-35B-A3B-OPD reaches 42.39 on ProofBench, improving over its base model by 15.61 points, and achieves additional gains of 6.99 and 2.06 points on AnswerBench and AIME25. Intern-S2-Preview exhibits the largest ProofBench gain, improving by 22.80 points from 21.70 to 44.50, nearly matching the teacher’s score of 45.00. It also reaches 80.10 on AnswerBench and 95.00 on AIME25, surpassing SU-01 on both benchmarks. These results demonstrate that our approach effectively transfers proof-reasoning capabilities to diverse students without requiring shared tokenization or supervised fine-tuning on teacher-generated trajectories. We also evaluate ProofBench using Gemini-2.5-Pro as the judge, following the same evaluation setting as SU-01. The results are shown in Figure 1. The results show that OPD training brings a substantial improvement to Intern-S2-Preview on ProofBench. Specifically, Intern-S2-OPD improves from 34.0 to 55.2, achieving a gain of 21.2 points over the original Intern-S2-Preview model. This demonstrates that OPD can effectively transfer proof-reasoning capabilities from the long-context teacher model to the student model. Notably, Intern-S2-OPD also surpasses several strong frontier models, including Gemini-2.5-Pro and GPT-5. Although it still trails SU-01 and DeepSeek-V3.2- Speciale, the gap is significantly narrowed after OPD training. These results further confirm the effectiveness of OPD for improving natural-language mathematical proof reasoning, especially in cross-model capability transfer settings. 4.3 Results on Cross-family Models We set student-reference KL loss 1.0 for GLM-4.7-Flash and Gemma-4-26B-A4B to further maintain stability. Figure 7 presents additional cross-family distillation results on GLM-4.7-Flash and Gemma- 9 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning 16 27 39 50 21.70 41.80 36.31 44.50 ProofBench@4 74 77 79 81 76.03 78.85 75.50 80.10 AnswerBench@8 87 90 94 97 88.33 95.33 94.66 95.00 AIME25@8 55 57 59 60 58.00 57.00 56.00 59.50 AMOBench@8 Score (%) Intern-S2EOPDG-OPDSimpleOPD Figure 8: Comparison of the Intern-S2 base model distilled from SU-01 with EOPD, G-OPD, and SimpleOPD across four reasoning benchmarks. All scores are reported as percentages. Each panel uses an independently scaled vertical axis to make small performance differences visible. 4-26B-A4B. GLM-4.7-OPD improves consistently on both benchmarks, increasing from 30.8 to 39.7 on ProofBench and from 69.6 to 72.0 on AnswerBench. Gemma-4-26B-A4B-OPD also achieves a substantial ProofBench gain, rising from 25.5 to 34.2, although its AnswerBench score decreases from 68.8 to 67.5. ProofBenchAnswerBench 0 20 40 60 80 30.8 69.6 39.7 72.0 GLM-4.7-Flash GLM-4.7-OPD ProofBenchAnswerBench 0 20 40 60 80 25.5 68.8 34.2 67.5 Gemma-4-26B-A4B Gemma-4-26B-OPD Figure 7: Cross-family distillation re- sults for GLM-4.7-Flash and Gemma- 4-26B-A4B from SU-01. These results demonstrate the effectiveness of our method for cross-family distillation, while also highlighting the challenge posed by tokenizer mismatch. SU-01 is based on the Qwen model family and uses a byte-level BPE tokenizer. GLM-4.7- Flash also adopts a BPE-based tokenizer, although with a dif- ferent vocabulary, whereas Gemma uses a SentencePiece-based tokenizer with substantially different vocabulary and segmen- tation behavior. Accordingly, GLM benefits consistently across both benchmarks, while the more tokenizer-dissimilar Gemma merely improves on ProofBench. Nevertheless, the clear Proof- Bench improvements for both models demonstrate that our ap- proach can transfer proof-reasoning capabilities across model families and tokenizer boundaries, although larger tokenizer discrepancies may make such transfer more challenging. 5 Analysis 5.1 Comparison with OPD Baselines We compare SimpleOPD with the unmodified Intern-S2 model and two recent OPD variants: EOPD, which supplements reverse-KL OPD with forward KL at positions where the teacher has high token- level entropy (Jin et al., 2026), and G-OPD, which generalizes OPD through a flexible reference model and reward scaling (Yang et al., 2026). As shown in Figure 8, SimpleOPD improves the Intern-S2 base model by 22.80, 4.07, 6.67, and 1.50 points on ProofBench, AnswerBench, AIME25, and AMOBench, respectively. Among the OPD methods, SimpleOPD obtains 44.50 on ProofBench, 80.10 on AnswerBench, and 59.50 on AMOBench, outperforming both EOPD and G-OPD by a large margin. On AIME25, SimpleOPD reaches 95.00, which is 0.33 points below EOPD and 0.34 points above G-OPD. Overall, SimpleOPD achieves the best result on three of the four benchmarks, with its largest advantage appearing on the proof-oriented ProofBench, while remaining competitive on AIME25. 5.2 Lexical Overlap The lexical-overlap curves provide an estimate of how much token-level supervision can be retained under cross-tokenizer alignment. As shown in Figure 9, the models already exhibit a high aligned- token ratio at the beginning of training, and the ratio increases further over time. This indicates that, despite tokenizer differences, a large portion of the student-generated text can be matched to 10 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning 120406080100 Global Step 0.90 0.92 0.94 0.96 0.98 Ratio Qwen3.5-35B-A3B-OPD Lexical Overlap 120406080100 Global Step 0.90 0.92 0.94 0.96 0.98 Ratio Intern-S2-OPD Lexical Overlap 120406080100 Global Step 0.90 0.92 0.94 0.96 0.98 Ratio GLM-4.7-OPD Lexical Overlap Figure 9: The lexical-overlap curves of Qwen3.5-35B-A3B, Intern-S2-Preview and GLM-4.7-Flash during OPD training from the teacher SU-01. Table 3: Evaluation results of Intern-S2-OPD distilled from SU-01 on scientific reasoning bench- marks. Model FrontierScience Olympiad FrontierScience Research HLE (text-only) HiPhO SU-0161.511.720.735.0 Intern-S2-Preview60.61.719.638.6 Intern-S2-OPD60.9 (+∆ 0.3)5.0 (+∆ 3.3)20.5 (+∆ 0.9)41.1 (+∆ 2.5) teacher tokens through shared surface spans. These results suggest that the partial alignment used in cross-tokenizer OPD preserves a substantial amount of usable training signal in practice, while avoiding the requirement of full tokenizer compatibility. 5.3 Out-of-domain Generalization Although OPD training is conducted only on mathematical reasoning data, the resulting model also shows consistent gains on out-of-domain scientific reasoning benchmarks. The results are shown in Table 3. Compared with Intern-S2-Preview, Intern-S2-OPD achieves a slight improvement on FrontierScience-Olympiad and HLE (Wang et al., 2026; Phan et al., 2025), and a clearer gain on HiPhO (Yu et al., 2025), improving from 38.6 to 41.1. On FrontierScience Research (Wang et al., 2026), the score also improves from 1.7 to 5.0. This suggests that the OPD-trained model does not lose its broader scientific reasoning ability despite being trained only on math data, and may further benefit physics-oriented reasoning. In particular, Intern-S2-OPD outperforms SU-01 on HiPhO, indicating that the student model retains its own strengths while gaining additional reasoning capability through SimpleOPD. 5.4 Effect of Training Data We further analyze the effect of data composition in distillation. Specifically, we include the verifiable math data used in SU-01 during the OPD process in addition to the proof data. The results show that both data compositions substantially improve Intern-S2-Preview over the base model, confirming the effectiveness of OPD distillation. Using only proof data achieves the best performance on ProofBench@4, improving Intern-S2 from 21.70 to 44.50 and approaching the teacher model SU-01. This suggests that proof-focused data is more beneficial for transferring natural-language proof-reasoning ability. Adding verifiable math data brings only a marginal improve- ment on AnswerBench@8, increasing the score from 80.10 to 81.10, while AIME25@8 remains unchanged at 95.00. However, this mixed-data setting significantly underperforms proof-only data on ProofBench@4, dropping from 44.50 to 38.50. These results indicate that adding verifiable data provides limited additional benefit for general mathematical reasoning, while potentially weakening the transfer of proof-reasoning ability. 11 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Table 4: Results of different distillation lengths on Intern-S2-Preview and Qwen3.5-35B-A3B distilled from the teacher SU-01. ModelLengthProofBench@4AnswerBench@8AIME25@8 SU-01–45.0077.5094.60 Intern-S2-Preview–21.7076.0388.33 Qwen3.5-35B-A3B–26.7873.1694.60 6k38.80 (+∆ 17.10)77.25 (+∆ 1.22)95.00 (+∆ 6.67) Intern-S2-OPD 32k44.50 (+∆ 22.80)80.10 (+∆ 4.07)95.00 (+∆ 6.67) 6k40.07 (+∆ 13.29)77.97 (+∆ 4.81)96.25 (+∆ 1.65) Qwen3.5-35B-A3B-OPD 32k42.39 (+∆ 15.61)80.16 (+∆ 7.00)96.67 (+∆ 2.07) 5.5 Effect of OPD Length We analyze the effect of the distillation length on the student models (Table 4). By comparison, we find that for long-context proof-reasoning tasks, a 6k distillation length is insufficient to fully capture the teacher model’s reasoning patterns. Specifically, increasing the distillation length from 6k to 32k improves the student model consistently across all evaluated benchmarks when the distillation is stable. In Qwen3.5-OPD, ProofBench@4 increases from 40.07 to 42.39, AnswerBench@8 from 77.97 to 80.16, and AIME25@8 from 96.25 to 96.67. The improvement is most pronounced on ProofBench, suggesting that longer distillation sequences are particularly important for transferring complex, multi-step proof reasoning behavior. These results indicate that preserving longer reasoning traces during distillation helps the student model better approximate the teacher’s long-context reasoning capability. 5.6 Distillation from DeepSeek-V4-Flash Table 5 reports the results of distilling DeepSeek-V4-Flash (DeepSeek-AI, 2026) into Intern-S2- Preview using a 6K distillation length for computational efficiency, together with termination-token masking and student-reference KL regularization. Despite the shorter distillation length, Intern-S2- DS-OPD consistently outperforms its base model across all three benchmarks. Its ProofBench@4 score increases from 21.70 to 39.71, corresponding to a gain of 18.01 points. AnswerBench@8 improves from 76.03 to 77.94, while AIME25@8 increases from 88.33 to 97.50, yielding gains of 1.91 and 9.17 points, respectively. Moreover, its ProofBench@4 score of 39.71 is 0.91 points higher than the 38.80 achieved using SU-01 as the teacher with the same 6K distillation length in Table 4, suggesting that a stronger teacher can lead to greater improvements in the student. Overall, these results demonstrate that SimpleOPD can effectively transfer reasoning capabilities across distinct model families and from a substantially larger teacher (158B) to a smaller student (30B), even with a relatively short distillation length. 5.7 Effect of Student-Reference KL Loss We further investigate how the student-reference KL coefficient affects GLM distillation. This coefficient controls the extent to which the student is regularized toward its reference policy and therefore influences the balance between preserving the base model’s behavior and learning from the teacher. We evaluate coefficients of 0.5, 1.0, and 1.2 to study this trade-off. As shown in Table 6, GLM-4.7-Flash-OPD consistently outperforms the base model under all three settings. Among them, a coefficient of 1.0 achieves the best overall performance, improving ProofBench@4 from 30.75 to 39.71 and AnswerBench@8 from 69.59 to 71.97, with gains of 8.96 and 2.38 points, respectively. A smaller coefficient of 0.5 provides insufficient regularization and yields lower ProofBench and AnswerBench scores. Conversely, increasing the coefficient to 1.2 imposes a stronger constraint on the student policy, which may limit its ability to absorb the teacher’s reasoning capabilities. Although coefficients of 0.5 and 1.2 achieve the highest AIME25@8 score of 94.17, their overall performance is less balanced than that obtained with a coefficient of 1.0. These results suggest that the student-reference KL coefficient should be neither too small nor too large, and 12 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Table 5: SimpleOPD results from DeepSeek-V4-Flash to Intern-S2-Preview with masking termination tokens and adding student reference KL loss. The distillation length is 6k for computational efficiency. ModelProofBench@4AnswerBench@8AIME25@8 DeepSeek-V4-Flash52.1877.0099.17 Intern-S2-Preview21.7076.0388.33 Intern-S2-DS-OPD39.71 (+∆ 18.01)77.94 (+∆ 1.91)97.50 (+∆ 9.17) Table 6: Ablation study on the student-reference KL coefficient in the setting from SU-01 to GLM- 4.7-Flash. Green values indicate improvements over the GLM-4.7-Flash base model. Model Student KL Coefficient ProofBench@4 AnswerBench@8AIME25@8 SU-01–45.0077.5094.60 GLM-4.7-Flash–30.7569.5992.08 0.537.15 (+∆ 6.40)71.28 (+∆ 1.69)94.17 (+∆ 2.09) 1.039.71 (+∆ 8.96)71.97 (+∆ 2.38)92.91 (+∆ 0.83) GLM-4.7-Flash-OPD 1.237.96 (+∆ 7.21)71.19 (+∆ 1.60)94.17 (+∆ 2.09) that a moderate value provides the best balance between retaining the student’s original capabilities and acquiring knowledge from the teacher. 6 Related Work 6.1 Olympiad-Level Long-Context Reasoning Early works such as OpenAI o1 and DeepSeek-R1 established long chain-of-thought reasoning as a central test-time scaling paradigm (OpenAI, 2024; DeepSeek-AI et al., 2025). This approach has enabled LLMs to tackle challenging real-world tasks, including long-horizon agentic tasks and Olympiad-level reasoning. Recent studies have further advanced reasoning capabilities in mathe- matics and science. Gemini Deep Think was among the first systems to achieve gold-medal-level performance at the IMO (Luong & Lockhart, 2025). Nemotron-Math-v2 and MiniMax subsequently reported similarly strong results on mathematical Olympiad benchmarks (Du et al., 2025; Chen et al., 2026). Beyond mathematics, P1 and P1-VL achieved gold-medal-level performance across several physics Olympiads, while SU-01 introduced a unified recipe for eliciting Olympiad-level proof behavior from a 30B model (Chen et al., 2025a; Luo et al., 2026; Li et al., 2026a). However, these capabilities typically rely on costly SFT and RL pipelines, including long-trajectory curation, repeated data-selection rollouts, and RL training. The cost of each stage increases substantially with reasoning length. This motivates our exploration of on-policy distillation for efficiently transferring Olympiad-level proof capabilities across diverse models. 6.2 On-policy Distillation Recently, on-policy distillation (OPD), introduced by MiniLLM and GKD (Gu et al., 2024; Agarwal et al., 2024), has emerged as an important component of LLM post-training and has been adopted by flagship models such as GLM-5, DeepSeek-V4, and Qwen3.5 (GLM-5 Team, 2026; DeepSeek- AI, 2026; Qwen Team, 2026). By optimizing the reverse KL divergence between teacher and student distributions over student-generated rollouts, OPD combines on-policy exploration with dense token-level supervision, offering a stable and generalizable alternative to conventional SFT and RL (Gu et al., 2024; Agarwal et al., 2024). Subsequent work has broadened OPD beyond standard autoregressive students: Draft-OPD adapts it to training-based speculative draft models through verification-error replay (Lei et al., 2026), while Shen et al. (2026) show that OPD follows a characteristic low-dimensional update geometry distinct from SFT and RLVR. However, its effectiveness can be limited by mismatched reasoning patterns between teacher and student and unreliable supervision over long sequences (Agarwal et al., 2024; Lei et al., 2026). 13 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Moreover, most existing studies assume a shared vocabulary, typically distilling models within the same family or combining domain-specific RL teachers (Gu et al., 2024; Agarwal et al., 2024; DeepSeek-AI, 2026). OPD between heterogeneous models with different architectures and vocab- ularies remains underexplored. In this work, we investigate heterogeneous OPD for transferring long-context reasoning capabilities across diverse models. 7 Conclusion In this work, we studied on-policy distillation from a long-context reasoning model, SU-01, to short-context student models. To handle tokenizer differences, we performed OPD in a shared text space and aligned only tokens that occupy identical text spans under the student and teacher tokenizers. We found that naive distillation led to severe training instability: the student’s output length kept increasing, termination tokens were often missing, and many responses were truncated. To address this issue, we introduced two simple but effective stabilization techniques: (1) masking the OPD loss on structural termination tokens, including</think>and<|imend| >, (2) adding a reference KL loss to constrain the student policy. These techniques substantially reduced length explosion and truncation while improving reasoning performance. Experiments on both same-family and different-family student models showed consistent gains in mathematical reasoning ability, with Intern-S2-Preview achieving a 21.2-point improvement on ProofBench. These results demonstrate that our approach provides an effective and generalizable way to transfer long-context reasoning capabilities to short-context models across different tokenizers and model families. Acknowledgments This work was supported by the Shanghai Artificial Intelligence Laboratory. We are grateful to the authors and open-source communities whose work made this project possible. References Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self- generated mistakes. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=3zKtaqxLhW. Shengnan An, Xunliang Cai, Xuezhi Cao, Xiaoyu Li, Yehao Lin, Junlin Liu, Xinxuan Lv, Dan Ma, Xuanlin Wang, Ziwen Wang, and Shuang Zhou. AMO-Bench: Large language models still struggle in high school math competitions. arXiv preprint arXiv:2510.26768, 2025. URL https://arxiv.org/abs/2510.26768. Jiacheng Chen, Qianjia Cheng, Fangchen Yu, Haiyuan Wan, Yuchen Zhang, Shenghe Zheng, Junchi Yao, Qingyang Zhang, Haonan He, Yun Luo, et al. P1: Mastering physics olympiads with reinforcement learning. arXiv preprint arXiv:2511.13612, 2025a. URLhttps://arxiv.or g/abs/2511.13612. Jiacheng Chen, Xinyu Zhang, Shunkai Zhang, Yanmohan Wang, Lin Li, Tiancheng Qin, Qin Wang, Zhengmao Zhu, Tianle Li, Jingyang Li, et al. MaxProof: Scaling mathematical proof with generative-verifier RL and population-level test-time scaling. arXiv preprint arXiv:2606.13473, 2026. URL https://arxiv.org/abs/2606.13473. Yijie Chen, Yijin Liu, Fandong Meng, Yufeng Chen, Jinan Xu, and Jie Zhou. Enhancing cross- tokenizer knowledge distillation with contextual dynamical mapping. In Findings of the Association for Computational Linguistics: ACL 2025, p. 8005–8018, Vienna, Austria, 2025b. Association for Computational Linguistics. doi: 10.18653/v1/2025.findings- acl.419. URLhttps: //aclanthology.org/2025.findings-acl.419/. DeepSeek-AI. DeepSeek-V4: Towards highly efficient million-token context intelligence, 2026. URL https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/Dee pSeek_V4.pdf. Technical report. 14 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. Nature, 645:633–638, 2025. URLhttps://arxiv.org/ abs/2501.12948. Wei Du, Shubham Toshniwal, Branislav Kisacanin, Sadegh Mahdavi, Ivan Moshkov, George Arm- strong, Stephen Ge, Edgar Minasyan, Feng Chen, and Igor Gitman. Nemotron-Math: Efficient long-context distillation of mathematical reasoning from multi-mode supervision. arXiv preprint arXiv:2512.15489, 2025. URL https://arxiv.org/abs/2512.15489. Gemma Team. Gemma 4 technical report, 2026. URLhttps://arxiv.org/abs/2607.027 70. GLM-5 Team. GLM-5: From vibe coding to agentic engineering. arXiv preprint arXiv:2602.15763, 2026. URL https://arxiv.org/abs/2602.15763. Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. MiniLLM: Knowledge distillation of large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=5h0qf7IBZZ. Jonas H ̈ ubotter, Frederike L ̈ ubeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, and Andreas Krause. Reinforcement learning via self-distillation, 2026. URLhttps://arxiv.org/abs/2601 .20802. InternLM Team. Intern-S2-Preview. Hugging Face model card, 2026. URLhttps://huggingf ace.co/internlm/Intern-S2-Preview. Woogyeol Jin, Taywon Min, Yongjin Yang, Dennis Wei, Yi Zhou, Swanand Ravindra Kadhe, Nathalie Baracaldo, and Kimin Lee. Entropy-aware on-policy distillation of language models. arXiv preprint arXiv:2603.07079, 2026. URL https://arxiv.org/abs/2603.07079. Haodi Lei, Yafu Li, Haoran Zhang, Shunkai Zhang, Qianjia Cheng, et al. Draft-opd: On-policy distillation for speculative draft models, 2026. URLhttps://arxiv.org/abs/2605.293 43. Yafu Li, Runzhe Zhan, Haoran Zhang, Shunkai Zhang, Yizhuo Li, Zhilin Wang, Jiacheng Chen, Futing Wang, Xuyang Hu, Yuchen Fan, et al. Achieving gold-medal-level olympiad reasoning via simple and unified scaling. arXiv preprint arXiv:2605.13301, 2026a. URLhttps://arxiv. org/abs/2605.13301. Yaxuan Li, Yuxin Zuo, Bingxiang He, Jinqian Zhang, Chaojun Xiao, Cheng Qian, Tianyu Yu, Huan- ang Gao, Wenkai Yang, Zhiyuan Liu, and Ning Ding. Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe. arXiv preprint arXiv:2604.13016, 2026b. URL https://arxiv.org/abs/2604.13016. Kevin Lu and Thinking Machines Lab. On-policy distillation. Thinking Machines Lab: Connection- ism, 2025. doi: 10.64434/tml.20251026. https://thinkingmachines.ai/blog/on-policy-distillation. Yun Luo, Futing Wang, Qianjia Cheng, Fangchen Yu, Haodi Lei, Jianhao Yan, Chenxi Li, Jiacheng Chen, Yufeng Zhao, Haiyuan Wan, et al. P1-VL: Bridging visual perception and scientific reasoning in physics olympiads. arXiv preprint arXiv:2602.09443, 2026. URLhttps://arxiv.org/ abs/2602.09443. Thang Luong and Edward Lockhart. Advanced version of Gemini with Deep Think officially achieves gold-medal standard at the international mathematical olympiad. Google DeepMind Blog, jul 2025. URLhttps://deepmind.google/blog/advanced-version-of-gemini-wit h-deep-think-officially-achieves-gold-medal-standard-at-the-int ernational-mathematical-olympiad/. 15 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Thang Luong, Dawsen Hwang, Hoang H. Nguyen, Golnaz Ghiasi, Yuri Chervonyi, Insuk Seo, Junsu Kim, Garrett Bingham, Jonathan Lee, Swaroop Mishra, Alex Zhai, Huiyi Hu, Henryk Michalewski, Jimin Kim, Jeonghyun Ahn, Junhwi Bae, Xingyou Song, Trieu Hoang Trinh, Quoc V. Le, and Junehyuk Jung. Towards robust mathematical reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 35418–35442, Suzhou, China, 2025. Association for Computational Linguistics. doi: 10.18653/v1/2025.emnlp-main.1794. URL https://aclanthology.org/2025.emnlp-main.1794/. Wenhan Ma, Jianyu Wei, Liang Zhao, Hailin Zhang, Bangjun Xiao, Lei Li, Qibin Yang, Bofei Gao, Yudong Wang, Rang Li, Jinhao Dong, Zhifang Sui, and Fuli Luo. Mopd: Multi-teacher on-policy distillation for capability integration in llm post-training, 2026. URLhttps://arxiv.org/ abs/2606.30406. Mathematical Association of America. American invitational mathematics examination (aime) 2025, 2025. URLhttps://maa.org/math-competitions/american-invitationa l-mathematics-examination-aime. OpenAI. OpenAI o1 system card, 2024. URLhttps://openai.com/index/openai-o 1-system-card/. OpenAI, :, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, Che Chang, Kai Chen, Mark Chen, Enoch Cheung, Aidan Clark, Dan Cook, Marat Dukhan, Casey Dvorak, Kevin Fives, Vlad Fomenko, Timur Garipov, Kristian Georgiev, Mia Glaese, Tarun Gogineni, Adam Goucher, Lukas Gross, Katia Gil Guzman, John Hallman, Jackie Hehir, Johannes Heidecke, Alec Helyar, Haitang Hu, Romain Huet, Jacob Huh, Saachi Jain, Zach Johnson, Chris Koch, Irina Kofman, Dominik Kundel, Jason Kwon, Volodymyr Kyrylov, Elaine Ya Le, Guillaume Leclerc, James Park Lennon, Scott Lessans, Mario Lezcano-Casado, Yuanzhi Li, Zhuohan Li, Ji Lin, Jordan Liss, Lily, Liu, Jiancheng Liu, Kevin Lu, Chris Lu, Zoran Martinovic, Lindsay McCallum, Josh McGrath, Scott McKinney, Aidan McLaughlin, Song Mei, Steve Mostovoy, Tong Mu, Gideon Myles, Alexander Neitz, Alex Nichol, Jakub Pachocki, Alex Paino, Dana Palmie, Ashley Pantuliano, Giambattista Parascandolo, Jongsoo Park, Leher Pathak, Carolina Paz, Ludovic Peran, Dmitry Pimenov, Michelle Pokrass, Elizabeth Proehl, Huida Qiu, Gaby Raila, Filippo Raso, Hongyu Ren, Kimmy Richardson, David Robinson, Bob Rotsted, Hadi Salman, Suvansh Sanjeev, Max Schwarzer, D. Sculley, Harshit Sikchi, Kendal Simon, Karan Singhal, Yang Song, Dane Stuckey, Zhiqing Sun, Philippe Tillet, Sam Toizer, Foivos Tsimpourlas, Nikhil Vyas, Eric Wallace, Xin Wang, Miles Wang, Olivia Watkins, Kevin Weil, Amy Wendling, Kevin Whinnery, Cedric Whitney, Hannah Wong, Lin Yang, Yu Yang, Michihiro Yasunaga, Kristen Ying, Wojciech Zaremba, Wenting Zhan, Cyril Zhang, Brian Zhang, Eddie Zhang, and Shengjia Zhao. gpt-oss-120b & gpt-oss-20b model card, 2025. URL https://arxiv.org/abs/2508.10925. Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. Humanity’s last exam. arXiv preprint arXiv:2501.14249, 2025. URL https://arxiv.org/abs/2501.14249. Qwen Team. Qwen3.5. Model release, 2026. URL https://qwen.ai/blog?id=qwen3.5. Zhennan Shen, Yanshu Li, Qingyu Yin, Chak Tou Leong, Zhilin Wang, Yanxu Chen, Rongduo Han, Sunbowen Lee, and Yi R. Fung. On the geometry of on-policy distillation. arXiv preprint arXiv:2606.07082, 2026. URL https://arxiv.org/abs/2606.07082. Avyav Kumar Singh, Yen-Chen Wu, Alexandru Cioba, Alberto Bernacchia, and Davide Buffelli. Cross-tokenizer LLM distillation through a byte-level interface. In Proceedings of the Second Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual, p. 84–96, San Diego, California, USA, 2026. Association for Computational Linguistics. doi: 10.18653/v1/2026.customnlp4u- 1.9. URLhttps: //aclanthology.org/2026.customnlp4u-1.9/. Jie Sun, Mao Zheng, Mingyang Song, Qiyong Zhong, Yilin Cheng, Bichuan Feng, Pengfei Liu, Junfeng Fang, and Xiang Wang. SimCT: Recovering lost supervision for cross-tokenizer on-policy distillation, 2026. URL https://arxiv.org/abs/2605.07711. 16 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Miles Wang, Robi Lin, Kat Hu, Joy Jiao, Neil Chowdhury, Ethan Chang, and Tejal Patwardhan. FrontierScience: Evaluating AI’s ability to perform expert-level scientific tasks. arXiv preprint arXiv:2601.21165, 2026. URL https://arxiv.org/abs/2601.21165. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. URL https://arxiv.org/abs/2505.09388. Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, Saiyong Yang, and Yankai Lin. Learning beyond teacher: Generalized on-policy distillation with reward extrapolation. arXiv preprint arXiv:2602.12125, 2026. URL https://arxiv.org/abs/2602.12125. Fangchen Yu, Haiyuan Wan, Qianjia Cheng, Yuchen Zhang, Jiacheng Chen, Fujun Han, Yulun Wu, Junchi Yao, Ruilizhen Hu, Ning Ding, Yu Cheng, Tao Chen, Lei Bai, Dongzhan Zhou, Yun Luo, Ganqu Cui, and Peng Ye. HiPhO: How far are (M)LLMs from humans in the latest high school physics olympiad benchmark? arXiv preprint arXiv:2509.07894, 2025. URL https://arxiv.org/abs/2509.07894. Z.ai. GLM-4.7-Flash. Hugging Face model card, 2026. URLhttps://huggingface.co/z ai-org/GLM-4.7-Flash. Songming Zhang, Xue Zhang, Zengkui Sun, Yufeng Chen, and Jinan Xu. Dual-space knowledge distillation for large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 18164–18181, Miami, Florida, USA, 2024. As- sociation for Computational Linguistics. doi: 10.18653/v1/2024.emnlp- main.1010. URL https://aclanthology.org/2024.emnlp-main.1010/. Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. SGLang: Efficient execution of structured language model programs. In Advances in Neural Information Processing Systems, 2024. doi: 10.48550/arXiv.2312.07104. URLhttps://arxiv.org/ab s/2312.07104. Zilin Zhu, Chengxing Xie, Xin Lv, and slime Contributors. slime: An LLM post-training framework for RL scaling. https://github.com/THUDM/slime, 2025. GitHub repository. 17 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning A Cross-Tokenizer Scoring and Alignment The complete procedure of cross-tokenizer scoring and alignment is shown in Algorithm 1. Algorithm 1 Cross-tokenizer teacher scoring and greedy token-piece alignment Inputs : message sequencex; student response tokens y 1:n ; student log-probabilities ℓ θ 1:n . Outputs : aligned teacher log-probabilities e ℓ φ 1:n ; matched setM. ▷ Token-out, text-in teacher scoring s←D θ (y 1:n ), c φ ←C φ (x), u φ ← c φ ⊕ s z = (z 1 ,...,z m )←E φ (s| c φ ) ℓ φ i ← logπ φ (z i | c φ ,z <i ), ∀i∈1,...,m ▷ Initialize all positions with student log-probabilities e ℓ φ 1:n ← ℓ θ 1:n , M←∅ i← 1, j ← 1, H φ ← ε, H θ ← ε ▷ τ φ and τ θ return tokenizer-native token pieces while i≤ m∧ j ≤ n do p φ ← τ φ (z i ), p θ ← τ θ (y j ) if H φ = H θ ∧ p φ = p θ then e ℓ φ j ← ℓ φ i , M←M∪(i,j) H φ ← H φ ⊕ p φ , H θ ← H θ ⊕ p θ , i← i + 1, j ← j + 1 else if|H φ | >|H θ | then H θ ← H θ ⊕ p θ , j ← j + 1 else if|H φ | <|H θ | then H φ ← H φ ⊕ p φ , i← i + 1 else H φ ← H φ ⊕ p φ , H θ ← H θ ⊕ p θ , i← i + 1, j ← j + 1 end if end while return e ℓ φ 1:n ,M As shown in Figure 10, which plots an example of cross-tokenizer scoring in the distillation from SU-01 to Intern-S2-Preview, the visualization confirms that our alignment process allows most semantically informative tokens to receive supervision from the teacher. The problem asks us to consider a triangle $ ABC $ with $ AB < AC $. Let $\ omega $ be a circle passing through $ B $ and $ C $, such that $ A $ lies inside $\ omega $. Points $ X $ and $ Y $ are on $\ omega $. We are given that $\ angle BXA = \ angle AYC $. The positions of $ X $ and $ Y $ are constrained : $ X $ and $ C $ are on opposite sides of $ AB $, and $ Y $ and $ B $ are on opposite sides of $ AC $. We need to show that the line $ XY $ passes through a fixed point as $ X, Y $ vary on $\ omega $ subject to the angle condition . Ċ Let's denote the coordinates or angles . Since we are dealing with circles and angles , geometry techniques involving circles and inversion or is ogonal conjug ates might be useful . 's try to interpret the condition $\ angle BXA = \ angle AYC $. The points $ X, Y $ are on $\ omega $, and $ B, C $ are also on $\ omega $. $ X, B, C, Y $ are conc yclic . The angles in the condition involve $ A $, which is inside $\ omega $. Ċ Let's Unaligned token −14−12−10−8−6−4−202 Reverse KL Divergence (aligned tokens only) Figure 10: Per-token reverse KL divergence in cross-vocabulary distillation. Colored boxes show aligned tokens with KL divergence values, while gold boxes indicate tokens that could not be aligned between teacher and student models. 18 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning B Model Solutions for ProofBench This section presents two ProofBench examples to provide a qualitative view of the mathematical proof-generation behavior learned through SimpleOPD. Both ProofBench-Basic-001 and ProofBench- Advanced-028 are outputs generated by Intern-S2-OPD, the Intern-S2-Preview student distilled from SU-01; each case includes the original problem statement, a concise solution overview, and the complete model-generated proof. B.1 ProofBench-Basic-001 Question Determine all functions f : Z→ Z such that, for all x,y ∈ Z, f(2x) + 2f(y) = f(f(x + y)). Solution Overview The solution first evaluates the equation at carefully chosen arguments to derive a doubling identity and a translated Cauchy relation. Subtracting the constantf (0)then reduces the problem to an additive function onZ, which must be linear. Comparing coefficients and directly verifying the resulting candidates completes the classification. Intern-S2-OPD Solution Solution. For all x,y ∈ Z, let P (x,y) denote the assertion f (2x) + 2f (y) = f (f (x + y))(B.1) Step 1. Two useful identities. For every t∈ Z, P (0,t) and P (t, 0) give, respectively, f (f (t)) = f (0) + 2f (t),(B.2) f (f (t)) = f (2t) + 2f (0).(B.3) Equating these expressions gives, for every t∈ Z, f (2t) = 2f (t)− f (0).(B.4) Step 2. Deriving a Cauchy-type relation. Substituting (B.4) into (B.1) yields f (f (x + y)) = 2f (x) + 2f (y)− f (0).(B.5) On the other hand, applying (B.2) with t replaced by x + y gives f (f (x + y)) = f (0) + 2f (x + y).(B.6) Equating (B.5) and (B.6), we obtain, for all x,y ∈ Z, f (0) + 2f (x + y) = 2f (x) + 2f (y)− f (0), and hence f (x + y) = f (x) + f (y)− f (0)(B.7) Step 3. Additivity after subtracting the constant. Define g(x) = f (x)− f (0). Then g(0) = 0, and, for all x,y ∈ Z, (B.7) becomes g(x + y) = g(x) + g(y). Thusgis additive onZ. SinceZis cyclic, any additive mapZ → Zis determined by its value at1. In particular, for every n∈ Z, g(n) = ng(1). 19 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Intern-S2-OPD Solution (continued) Let a = g(1)∈ Z and b = f (0). Consequently, for every x∈ Z, f (x) = ax + b. Step 4. Determining the parameters a and b. Substituting f (x) = ax + b into (B.1), the left-hand side is f (2x) + 2f (y) = a(2x) + b + 2(ay + b) = 2a(x + y) + 3b, while the right-hand side is f (f (x + y)) = f (a(x + y) + b) = a 2 (x + y) + ab + b. Since t = x + y ranges over Z, for every t∈ Z we have 2at + 3b = a 2 t + ab + b. Therefore, 2a− a 2 = 0,2b− ab = 0, a(2− a) = 0, b(2− a) = 0. (B.8) It follows thata = 0ora = 2. Ifa = 0, then2b = 0, sob = 0andf ≡ 0. Ifa = 2, the second condition is automatically satisfied for every b∈ Z, giving f (x) = 2x + b. Step 5. Verification. For f ≡ 0, both sides of (B.1) are zero. For f (x) = 2x + b, f (2x) + 2f (y) = (4x + b) + 2(2y + b) = 4x + 4y + 3b, and f (f (x + y)) = f (2(x + y) + b) = 4x + 4y + 3b. Hence both families satisfy the equation, and the complete set of solutions is f ≡ 0,or f (x) = 2x + c, c∈ Z. B.2 ProofBench-Advanced-028 Question LetHbe the orthocenter of acute triangleABC, letFbe the foot of the altitude fromCtoAB, and letPbe the reflection ofHacrossBC. Suppose that the circumcircle of triangleAFP intersects line BC at two distinct points X and Y . Prove that C is the midpoint of XY . Solution Overview The solution placesBCon the horizontal axis and computes the coordinates ofF,H, andP. It then shows that the circumcenter of△AFPlies on the vertical line throughC. Since this line is perpendicular to the chordXY ⊂ BC, the standard chord-bisection property of a circle implies thatCis the midpoint ofXY. 20 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Intern-S2-OPD Solution Solution. Step 1. Coordinate setup. We place the triangle in a convenient coordinate system. Since the statement involves only midpoints, collinearity, and perpendicularity, we may apply a similarity transformation without loss of generality. Thus, let B = (0, 0), C = (1, 0), A = (p,q),(B.2.1) whereq > 0. Since the triangle is acute,0 < p < 1, although these inequalities are not essential for the algebra below. Define D = p 2 + q 2 . Step 2. Coordinates of the auxiliary points. The lineABpasses through the origin and has direction(p,q). Therefore, the projection ofC = (1, 0) onto AB is F = C· A A· A A = p p 2 + q 2 (p,q) = p 2 D , pq D .(B.2.2) BecauseBCis horizontal, the altitude fromAis the vertical linex = p. The altitude fromBis perpendicular toAC. SinceAC = (1− p,−q), a perpendicular direction is(q, 1− p), and hence this altitude has equation y = 1− p q x. Intersecting it with x = p gives H = p, p(1− p) q .(B.2.3) Reflecting H across BC, which is the x-axis, yields P = p,− p(1− p) q .(B.2.4) Step 3. A candidate for the circumcenter of△AFP . The pointsAandPhave the samex-coordinate, soAPis vertical. Its perpendicular bisector is the horizontal line through the midpoint M AP = p, q− p(1−p) q 2 ! = p, q 2 − p(1− p) 2q . Since q 2 − p(1− p) = p 2 + q 2 − p = D− p, define y 0 = D− p 2q .(B.2.5) The perpendicular bisector of AP is therefore y = y 0 . We claim that O = (1,y 0 )(B.2.6) is the circumcenter of△AFP. By construction,Olies on the perpendicular bisector ofAP. It remains to show that O is equidistant from A and F . Step 4. Verifying that OA = OF . From equations (B.2.1), (B.2.2), and (B.2.6), OA 2 = (1− p) 2 + (y 0 − q) 2 , OF 2 = 1− p 2 D 2 + y 0 − pq D 2 .(B.2.7) First observe that 1− p 2 D = D− p 2 D = q 2 D .(B.2.8) 21 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Intern-S2-OPD Solution (continued) Consider the difference ∆ = OA 2 − OF 2 = " (1− p) 2 − 1− p 2 D 2 # + (y 0 − q) 2 − y 0 − pq D 2 .(B.2.9) Factoring the first bracket as a difference of squares gives (1− p) 2 − 1− p 2 D 2 = p 2 D − p 2− p− p 2 D =− p(D− p) D 2− p− p 2 D .(B.2.10) Similarly, (y 0 − q) 2 − y 0 − pq D 2 = pq D − q 2y 0 − q− pq D =− q(D− p) D 2y 0 − q− pq D .(B.2.11) Substituting equations (B.2.10) and (B.2.11) into equation (B.2.9), we obtain ∆ =− D− p D " p 2− p− p 2 D + q 2y 0 − q− pq D # .(B.2.12) Using equation (B.2.5), q 2y 0 − q− pq D = q D− p q − q− pq D = (D− p)− q 2 − pq 2 D .(B.2.13) Consequently, the bracket in equation (B.2.12) equals p 2− p− p 2 D + (D− p)− q 2 − pq 2 D = 2p− p 2 − p 3 D + D− p− q 2 − pq 2 D = p + D− p 2 − q 2 − p(p 2 + q 2 ) D = p + 0− p = 0. It follows that∆ = 0, and henceOA 2 = OF 2 . Therefore,Olies on the perpendicular bisector ofAFas well as that of AP , so O is the circumcenter of△AFP . Step 5. Bisecting the chord XY . The circumcenterO = (1,y 0 )and the pointC = (1, 0)have the samex-coordinate. SinceBCis the x-axis, the lineOCis perpendicular toBC. The chordXYlies onBC, and the perpendicular from the center of a circle to a chord bisects that chord. Thus, C is the midpoint of XY . C is the midpoint of XY. 22 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning C Case Study This appendix presents two representative instances of direct OPD when the teacher model SU-01 is distilled to Qwen3.5-35B-A3B. In both cases, the student degenerates into an endless repeti- tion—either a fixed multi-sentence self-check block or a single-token loop. The rollouts terminate only when hitting the 160k-token generation limit. C.1 Case A: correct answer, then 972 exact self-check repeats (AIME25) The model solves an AIME 2025 divisibility problem, reaches the correct numerical answer279, but then appends the same ten-sentence “verification” unit verbatim 972 times. This loop occupies nearly the entire remaining generation budget and is truncated mid-unit. Question Determine the difference betweenNand2025, whereNis the number of eight-digit positive integers using each of the digits 1, 2,..., 8 exactly once that are divisible by 22. Model Output (excerpt) Beginning of output. We need to solve: there are8! = 40320eight-digit positive integers that use each of the digits 1, 2, 3, 4, 5, 6, 7, 8exactly once. LetNbe the number of these integers that are divisible by22. Find the difference betweenNand2025. So we need to computeN, then computeN − 2025. Since22 = 2· 11, we need divisibility by 2 and by 11. At the failure onset (response chars 75,000–77,150). Consequently,N = 144× 16 = 2304. Finally,N − 2025 = 2304− 2025 = 279. Thus the answer is 279. I’l also ensure that the solution is not missing any step. Repeating pattern — exact 10-sentence self-check unit (542 characters), repeated 972 times. “I’l also ensure that the solution is not missing any step.” “I’l also ensure that the solution is clear.” “I’l also ensure that the solution is correct.” “I’l also ensure that the solution is comprehensive.” “I’l also ensure that the solution is easy to follow.” “I’l also ensure that the solution is in the required format.” “I’l also ensure that the solution is not too long.” “I’l also ensure that the solution is self-contained.” “I’l also ensure that the solution is not missing any justification.” “I’l also ensure that the solution is correct.” The model reaches the correct answer279, then repeats the same ten-sentence self-check unit exactly 972 times. The rollout hits the 160k-token generation limit inside this unit and is recorded as truncated; the response ends mid-unit with “. . . not missing any justification. I’l also ensure that the”. C.2 Case B: reasoning collapses into a one-token loop (AnswerBench) In this case, the model engages with a complex number-theoretic definition (“exotic integers”). After some initial reasoning, it gets stuck on a sub-problem involving multiplicative orders and begins emitting the token sequence “2 ? ” indefinitely. The repetition is atomic (one token) and repeats over 34,000 times with no progress or termination. 23 SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning Question For a positive integern, callg : Z → Zann-good function ifg(1) = 1and, for distinct integersaand b,g(a)− g(b)dividesa n − b n . Callnexotic if the number ofn-good functions is twice an odd integer. Find the 132nd exotic integer. Model Output (excerpt) Beginning of output. We need to solve: for positive integern, defineg : Z → Zsuch thatg(1) = 1and for any two distinct integersaandb,g(a)− g(b)dividesa n − b n . We callnan exotic integer if the number ofn-good functions is twice an odd integer. Find the 132nd exotic integer. We need to understand the condition and count the number of such functions for a given n. At the failure onset (response tokens 840–919). Form = 8:2 8 − 1 = 255, order of2mod255?2 8 = 256 ≡ 1 (mod 255), but2 4 = 16 (mod 255), not 1; . . . but could be smaller? 2 ? Actually . . . 2 ? 2 ? 2 ? Repeating pattern — exact one-token unit, repeated many times. 2 ? 2 ? 2 ? 2 ? 2 ? 2 ? 2 ? 2 ? 2 ? 2 ? · The identical token occupies one uninterrupted run; no new claim, calculation, or answer is produced after the loop begins. The rollout reaches the 160k-token generation limit inside this run and is recorded as truncated; the response ends with “. . . 2 ? 2 ? 2 ? 2 ? 2 ? ”. 24