Paper deep dive
Aha-Flow Distillation: Flow Markers Matter in LLM Reasoning
Xiaodong Wang, Peixi Peng
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 9/13/2026, 3:46:13 AM
Summary
The paper introduces Aha-Flow Distillation (AFD), a dual-mode on-policy self-distillation framework that leverages two distinct reasoning patterns: 'Aha Moments' (reflective, backtracking) and 'Flow Moments' (sustained, process-confirming). By constructing Flow-CoT traces via rewriting and pairing them with specific instructions, AFD improves mathematical reasoning performance on Qwen3-8B and Qwen3-4B over standard OPSD baselines.
Entities (11)
Relation Signals (8)
Aha-Flow Distillation โ evaluatedon โ AIME25
confidence 95% ยท Experiments on AIME25 and HMMT25 show consistent improvements
Aha-Flow Distillation โ evaluatedon โ HMMT'25
confidence 95% ยท Experiments on AIME25 and HMMT25 show consistent improvements
Aha-Flow Distillation โ extends โ On-Policy Self-Distillation
confidence 95% ยท We further propose Aha-Flow Distillation (AFD), a dual-mode extension of OPSD
Aha-Flow Distillation โ improves โ Qwen3-4B
confidence 95% ยท AFD improves Avg@12 from 57.5 to 58.6 on Qwen3-4B
Aha-Flow Distillation โ improves โ Qwen3-8b
confidence 95% ยท AFD improves Avg@12 from 60.8 to 61.3 on Qwen3-8B
Flow Moment โ characterizedby โ Flow Markers
confidence 92% ยท Flow Moment, a reasoning pattern characterized by sustained, process-confirming verbalizations... We refer to their corresponding linguistic expressions as Flow Markers
Aha Moment โ characterizedby โ Aha Markers
confidence 92% ยท Aha Moment... We refer to their corresponding linguistic expressions as... Aha Markers
Flow-CoT โ derivedfrom โ Aha-Flow Distillation
confidence 90% ยท we construct Flow-CoT by rewriting the discourse markers of original reasoning traces... and use it as auxiliary supervision
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We identify the Flow Moment, a reasoning pattern characterized by sustained, process-confirming verbalizations such as I'm doing, in contrast to the revision- and backtracking-oriented Aha Moment. We refer to their corresponding linguistic expressions as Flow Markers and Aha Markers, respectively. Based on this observation, we construct Flow-CoT by rewriting the discourse markers of original reasoning traces while preserving their underlying reasoning content, and use it as auxiliary supervision for on-policy self-distillation (OPSD). We further propose \textbf{Aha-Flow Distillation (AFD)}, a dual-mode extension of OPSD that pairs different forms of privileged information with corresponding reasoning instructions. The Aha branch retains concise solution-based supervision, while the Flow branch introduces rewritten Flow-CoT under a direct and confident reasoning instruction. At inference time, the model uses only the standard reflective instruction, so Flow-style reasoning serves purely as a training signal. Experiments on AIME25 and HMMT25 show consistent improvements across Qwen3-8B and Qwen3-4B: AFD improves Avg@12 from 60.8 to 61.3 on Qwen3-8B and from 57.5 to 58.6 on Qwen3-4B over our reproduced OPSD baselines. Controlled ablations further show that, with the same Flow-CoT/Aha-CoT composition, dual-mode training improves Avg@12 from 59.5 to 60.1, indicating that the benefit comes not only from introducing heterogeneous reasoning supervision, but also from how it is organized during self-distillation. The code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2609.07036v1
- Canonical: https://arxiv.org/abs/2609.07036v1
Trouble viewing inline? Open PDF directly โ
Full Text
37,912 characters extracted from source content.
Expand or collapse full text
Aha-Flow Distillation: Flow Markers Matter in LLM Reasoning Xiaodong Wang 1,2 , Peixi Peng 1,2โ 1 Peking University, 2 Pengcheng Laboratory wangxiaodong21s@stu., pxpeng@pku.edu.cn Abstract We identify the Flow Moment, a reasoning pattern characterized by sustained, process-confirming verbalizations such as Iโm doing, in contrast to the revision- and backtracking-oriented Aha Moment. We refer to their corresponding linguis- tic expressions as Flow Markers and Aha Markers, respectively. Based on this observation, we construct Flow-CoT by rewriting the discourse markers of origi- nal reasoning traces while preserving their underlying reasoning content, and use it as auxiliary supervision for on-policy self-distillation (OPSD). We further pro- pose Aha-Flow Distillation (AFD), a dual-mode extension of OPSD that pairs different forms of privileged information with corresponding reasoning instruc- tions. The Aha branch retains concise solution-based supervision, while the Flow branch introduces rewritten Flow-CoT under a direct and confident reasoning in- struction. At inference time, the model uses only the standard reflective instruc- tion, so Flow-style reasoning serves purely as a training signal. Experiments on AIME25 and HMMT25 show consistent improvements across Qwen3-8B and Qwen3-4B: AFD improves Avg@12 from 60.8 to 61.3 on Qwen3-8B and from 57.5 to 58.6 on Qwen3-4B over our reproduced OPSD baselines. Controlled abla- tions further show that, with the same Flow-CoT/Aha-CoT composition, dual- mode training improves Avg@12 from 59.5 to 60.1, indicating that the bene- fit comes not only from introducing heterogeneous reasoning supervision, but also from how it is organized during self-distillation. The code is available at https://github.com/Wang-Xiaodong1899/Aha-Flow-Distillation. 1 Introduction Large language models increasingly verbalize their reasoning through long chain-of-thought (CoT) traces. CoT prompting showed that generating intermediate reasoning steps can substantially im- prove multi-step reasoning (Wei et al., 2022), while recent reasoning models such as DeepSeek- R1 (Guo et al., 2025) often exhibit reflection, verification, and self-correction during reasoning. A characteristic pattern is the Aha Moment, in which the model reconsiders, backtracks, or redi- rects its current reasoning trajectory, often verbalized through expressions such as Wait, Hmm, or Maybe. Such reflective reasoning has become prominent in post-trained reasoning models (Xu et al., 2026a; Team et al., 2026; Xiao et al., 2026) and is closely associated with training paradigms such as GRPO (Shao et al., 2024) and on-policy distillation (Agarwal et al., 2024). However, excessive reflection can also introduce redundant reasoning and unnecessary reconsideration of an otherwise valid solution path. In this work, we study a more confident reasoning format that proceeds directly along the current trajectory, often verbalized through expressions such as Iโm doing. We refer to this reasoning pattern as the Flow Moment and investigate its role in reasoning-model training. At the linguistic level, reflective reasoning is often accompanied by short metacognitive expressions that signal reconsideration or redirection. Self-correction studies have examined whether such be- Preprint. Work in progress. arXiv:2609.07036v1 [cs.CL] 7 Sep 2026 Standard OPSD Aha-only supervision Problem ํฅ Privileged CoT ํฆ โ Wait, Hmm, Maybe ... Student (train) Wait, Hmm, Maybe ... Tends to over-reflect Aha-Flow Distillation (Ours) Complementary supervision Problem ํฅ Aha CoT ํฆ ํโํ โ Wait,Hmm, Maybe ... Student (train) Wait, Hmm, Maybe ... Flow-CoT as auxiliary supervision Flow CoT ํฆ ํํํํค โ Iโm checking Iโm doing ... Figure 1: Comparison between OPSD and our proposed Aha-Flow Distillation (AFD). Standard OPSD relies on Aha-style privileged reasoning, while AFD introduces complementary Flow-CoT as auxiliary supervision during training. The model retains the standard reflective reasoning mode at inference and achieves consistent Avg@12 improvements on AIME25 and HMMT25 across Qwen3- 8B and Qwen3-4B. haviors help models detect and repair their own mistakes, with mixed evidence on the effectiveness of intrinsic correction without external feedback (Huang et al., 2024; Kamoi et al., 2024). Related behaviors, including verification, backtracking, and re-evaluation, also emerge in reasoning models trained with reinforcement learning (Guo et al., 2025; Gandhi et al., 2025). We refer to expressions such as Wait, Hmm, and Maybe as Aha Markers, since they typically accompany reflection or a change in the current reasoning trajectory. In contrast, we refer to expressions such as Iโm doing as Flow Markers, which verbalize direct and sustained progression along the current reasoning path. While Aha Markers have been widely observed and discussed in reflective reasoning, the role of Flow Markers has received much less attention. Our interest is in whether the form of reasoning used during post-training affects the resulting model. To study this question, we focus on on-policy distillation (OPD) (Agarwal et al., 2024), a recent training paradigm in which supervision is provided on trajectories sampled from the studentโs own policy. We further adopt on-policy self-distillation (OPSD) (Zhao et al., 2026), which removes the need for a separate external teacher model. In OPSD, the modelโs teacher forward pass is conditioned on privileged reference information to supervise its own on-policy generations. This provides a sim- ple and controlled setting for examining how different reasoning formats, when used as privileged information, influence self-distillation. Based on this setup, we construct Flow-CoT by rewriting a subset of the original CoT traces, re- placing Aha-style discourse markers with Flow Markers while preserving the underlying reasoning trajectory and mathematical content. We then propose Aha-Flow Distillation (AFD), a dual-mode extension of OPSD. The Aha branch retains the standard step-by-step reasoning instruction and uses a concise solution as privileged information, while the Flow branch uses the rewritten Flow-CoT to- gether with an instruction that encourages direct and confident reasoning. Within each branch, the student and teacher receive the same reasoning instruction, aligning the teacher supervision with the behavior requested from the student. At inference time, we use only the standard reflective in- struction. Thus, Flow-CoT serves as auxiliary training supervision rather than a new inference-time reasoning mode, allowing AFD to exploit different reasoning formats without introducing additional test-time procedures. Experiments with Qwen3-8B and Qwen3-4B on AIME25 and HMMT25 show that the organization of reasoning styles during distillation matters. In a controlled ablation using the same 50/50 mix- ture of Flow-CoT and Aha-CoT as privileged information, introducing AFD improves the average Avg@12 score from 59.5 to 60.1 on Qwen3-8B, showing that dual-mode training is more effective than simply mixing heterogeneous CoT references. Our final Flow-CoT/Solution configuration fur- ther reaches 61.3 Avg@12 on Qwen3-8B, compared with 60.8 for our reproduced OPSD baseline. 2 The same trend holds for Qwen3-4B, where AFD improves the average from 57.5 to 58.6. AFD also achieves the best AIME25 and HMMT25 results among the compared methods at both model scales. These results suggest that the benefit comes not only from introducing diverse reasoning signals, but also from how different forms of privileged information and reasoning instructions are paired during self-distillation. Our contributions are summarized as follows: โข We identify the Flow Moment and its corresponding Flow Markers, which characterize sustained progression in reasoning traces. We further construct Flow-CoT by rewriting the discourse markers of original CoT traces while preserving their reasoning content, and use it as complementary supervision for self-distillation. โข We propose Aha-Flow Distillation (AFD), a dual-mode self-distillation framework that pairs different forms of privileged information with corresponding reasoning instructions to better exploit heterogeneous reasoning supervision. โข Experiments on Qwen3-8B and Qwen3-4B show consistent improvements over OPSD on AIME25 and HMMT25. Controlled ablations further verify that dual-mode training pro- vides additional gains beyond simply mixing different forms of privileged information. 2 Related Work 2.1 Reasoning Traces and the Aha Moment Chain-of-thought prompting showed that generating intermediate reasoning tokens can substantially improve multi-step reasoning (Wei et al., 2022), even without few-shot exemplars (Kojima et al., 2022). Subsequent work further treated reasoning traces as a search space, through sampling and ag- gregation in self-consistency (Wang et al., 2022) or explicit expansion and pruning in tree-structured search (Yao et al., 2023). Across this line of work, however, reasoning traces are typically treated as largely undifferentiated sequences, with limited attention to the distinct roles played by metacogni- tive expressions within the trace. One class of such expressions has been extensively studied through self-correction and reinforce- ment learning. Intrinsic self-correction is often unreliable without external feedback (Huang et al., 2024; Kamoi et al., 2024), whereas RL with verifiable rewards can induce behaviors such as back- tracking and re-evaluation. DeepSeek-R1 popularized this phenomenon as the โAha Momentโ (Guo et al., 2025), building on scalable RL methods such as GRPO (Shao et al., 2024); similar behaviors have also been observed in other long-chain-of-thought systems (Kimi Team, 2025; Chang et al., 2025). Follow-up work suggests that RL amplifies discrete reasoning operations such as verifi- cation, backtracking, subgoal setting, and backward chaining (Gandhi et al., 2025). We refer to linguistic expressions associated with such revision or redirection as Aha Markers. The utility of Aha-style behavior remains unsettled. Important reasoning steps are not predominantly self-corrections (Bogdan et al., 2025), and weaker models may switch strategies too frequently and abandon productive trajectories (Wang et al., 2025). Our focus is different from determining whether more Aha behavior is beneficial at inference time. We distinguish Aha Markers from Flow Mark- ers, which verbalize the continuation of an ongoing reasoning trajectory, and use this distinction to construct an alternative form of CoT supervision. Specifically, we rewrite Aha-style discourse markers into Flow-style expressions while preserving the underlying reasoning trajectory, and use the resulting Flow-CoT as complementary supervision during self-distillation. To our knowledge, prior work has not explicitly characterized Flow Markers or investigated Flow-style verbalization as an auxiliary training signal for reasoning distillation. 2.2 On-policy distillation for reasoning From off-policy to on-policy distillation. Knowledge distillation trains a student to imitate a teacherโs outputs (Hinton et al., 2015). For reasoning tasks, the teacher commonly provides ratio- nales in addition to final labels (Hsieh et al., 2023). Recent approaches often distill long reasoning traces from stronger models into weaker ones, either to compress deliberative System-2 computation into more efficient System-1 behavior (Yu et al., 2024) or to obtain test-time scaling with smaller models (Muennighoff et al., 2025). Such methods are typically off-policy: the student is optimized 3 ํ=(ํฅ,ํฆ โ ) Sample (ํฅ,ํฆ โ ) ํฅ: Problem ํฆ โ : CoT + Ans Rand. split ํฆ โ ํฆ ํํํํค โ Rewrite Model Keep Rewrite ํผ ํโํ =Please reason step by step, and put your final answer within boxed. Flow CoT Rewriting Aha-Flow Dual-mode Training ํฅ,ํผ ํโํ ํฅ,ํฆ โ ,ํผ ํโํ ํ ํ (โ |ํฅ,ํผ ํโํ ) ํ ํ (โ |ํฅ,ํฆ โ ,ํผ ํโํ ) Student prompt Teacher prompt เทํฆ On-policy sample Student policy Teacher policy ํ ํ (โ |ํฅ,ํฆ โ ,ํผ ํโํ ,เทํฆ <ํ ) ํผ ํํํํค = Please reason directly and confidently, stating each step clearly, ..., and put your final answer within boxed ํฅ,ํผ ํํํํค ํฅ,ํฆ ํํํํค โ ,ํผ ํํํํค ํ ํ (โ |ํฅ,ํผ ํํํํค ) ํ ํ (โ |ํฅ,ํฆ ํํํํค โ ,ํผ ํํํํค ) Student prompt Teacher prompt เทํฆ On-policy sample Student policy Teacher policy ํ ํ (โ |ํฅ,ํฆ ํํํํค โ ,ํผ ํํํํค ,เทํฆ <ํ ) Learning Objective ํท(ํ ํ (โ | ํฅ,ํฆ โ ,ํผ ํโํ/ํํํํค ,เทํฆ <ํ ) เธฎ ํ ํ ( โ | ํฅ,ํฆ โ ,ํผ ํโํ/ํํํํค ,เทํฆ <ํ ) Per-Token Divergence ํ ํ (โ |ํฅ,ํผ ํโํ/ํํํํค ) Student policy gradient Figure 2: Overview of Aha-Flow Distillation (AFD). We randomly rewrite a subset of the orig- inal reasoning traces into Flow-CoT while retaining the remaining examples for the Aha branch. AFD then performs dual-mode on-policy self-distillation: the Aha and Flow branches use different privileged information and reasoning instructions, while the student and teacher share the same in- struction within each branch. Both branches are optimized with the token-level JSD objective. against traces generated by another policy, creating a mismatch between training trajectories and the studentโs own generation distribution. On-policy formulations reduce this mismatch by training on student-generated samples that are subsequently scored, completed, or otherwise supervised by the teacher. This paradigm, commonly referred to as on-policy distillation (OPD), directly distills teacher supervision on trajectories sampled from the studentโs current policy (Agarwal et al., 2024). Related approaches include reverse-KL sequence distillation (Gu et al., 2024) and generalized distil- lation with mixed teacher/student sampling (Agarwal et al., 2023). More recent variants extend OPD through entropy-aware objectives (Jin et al., 2026), reward extrapolation beyond the teacher (Yang et al., 2026), black-box teacher supervision (Ye et al., 2025), and competence-aware self-distillation focused on the studentโs learning frontier (Xu et al., 2026b). On-Policy Self-distillation. On-policy self-distillation (OPSD) removes the need for a separate teacher by using the model itself to generate on-policy targets while conditioning the teacher forward pass on a privileged reference solution (Zhao et al., 2026). This makes the procedure computation- ally attractive while retaining effective reasoning supervision, but it also introduces characteristic failure modes. Prior work reports reduced diversity or freezing of metacognitive markers, over- confident updates, and, in some settings, degradation of the modelโs reasoning capability (Kim et al., 2026). Existing remedies primarily modify the information content or weighting of the privileged supervision. Purified OPSD isolates the contamination introduced by the reference solution and re- constructs the target using pointwise mutual information (Shen et al., 2026), while a parallel line of work modulates the training update according to the modelโs own uncertainty (Ke et al., 2026). In contrast, our work focuses on the reasoning style encoded in self-distillation targets: we distinguish Aha markers from Flow markers and investigate how their allocation across training modes affects the resulting policy. 3 Method 3.1 Aha Markers and Flow Markers We focus on short metacognitive expressions that indicate how a model manages its reasoning tra- jectory. These expressions typically carry little task-specific content themselves, but signal whether the model is revising or continuing its current line of reasoning. We distinguish two types of markers. LetA denote the set of Aha Markers, such as WAIT, HMM, and MAYBE. These markers signal a discontinuity in the reasoning trajectory, including backtrack- 4 ing, re-evaluation, or a change of strategy. Let F denote the set of Flow Markers, such as IโM DOING and other process-confirming expressions. These markers indicate that the current reasoning path is maintained and extended without redirection. We use Aha Moment to describe a local reasoning pattern characterized by revision or redirec- tion, and Flow Moment for its complementary pattern characterized by sustained progression. Ac- cordingly, Aha Markers and Flow Markers provide two complementary signals about how a model organizes its reasoning trajectory. 3.2 OPSD Preliminaries On-policy self-distillation (OPSD) (Zhao et al., 2026) trains a student on its own on-policy genera- tions while using privileged reference information to guide the teacher distribution. Given a training pair (x,y โ ), where x is the input problem and y โ is the reference solution, the student first samples an on-policy response หy โผ p S (ยท | x). At each generation step, the teacher additionally observes the privileged reference y โ , while both models are conditioned on the same generated prefix หy <n . The OPSD objective is as follows: L OPSD (ฮธ) =E (x,y โ )โผS E หyโผp S (ยท|x) |หy| X n=1 D p T (ยท| x,y โ , หy <n ) p S (ยท| x, หy <n ) ,(1) where p S and p T denote the student and teacher token distributions, respectively, and D(ยทโฅยท) is the token-level distillation divergence. Following OPSD, we use JensenโShannon divergence (JSD) and clip the per-token divergence with a threshold ฮด clip to prevent a small number of tokens from dominating the update (Shen et al., 2026). A key asymmetry is that the teacher has access to y โ , whereas the student does not. Therefore, the form of the privileged information directly affects the teacher distribution and, consequently, the supervision received by the student. This property motivates our investigation of different forms of privileged information and their associated reasoning styles. 3.3 Constructing Flow-CoTs To construct Flow-style reasoning data, we rewrite the original CoTs that contain Aha Markers into corresponding Flow-CoTs. We use the DeepSeek-V4-Flash API (Xu et al., 2026a) for this trans- formation. The rewriting is restricted to discourse-level expressions: Aha-style fillers, hedges, and transition markers are replaced with explicit first-person action statements, such as โIโm checking...โ, โIโm correcting myself...โ, or โIโm trying another approach...โ. Importantly, the underlying reasoning trajectory is kept unchanged. The rewriting prompt explic- itly requires the model to preserve all mathematical content, intermediate results, dead ends, self- corrections, verification steps, and their original order, while modifying only the discourse markers. It also preserves the original paragraph structure and final answer. In this way, each Flow-CoT remains semantically aligned with its original CoT while differing primarily in how the reasoning process is verbalized. Concretely, the rewriting instruction asks the model to convert transition and hedging expressions into explicit โIโm <verb>ing ...โ statements, while prohibiting any addition, deletion, merging, or reordering of reasoning steps. The full rewriting prompt is provided in Appendix A. 3.4 Aha-Flow Distillation We propose Aha-Flow Distillation (AFD), a dual-mode extension of OPSD that pairs different forms of privileged information with corresponding reasoning instructions. The key idea is to make the reasoning behavior requested from the student consistent with the reference information used to condition the teacher. As shown in our ablation study, using a concise solution as privileged information performs bet- ter than using a long CoT under standard OPSD. We therefore use the solution as the privileged information for the Aha branch in our final configuration. 5 For each training example x, we define a binary mode indicator s(x) โ 0, 1. In the Aha branch, s(x) = 0, the teacher is conditioned on the solution y โ . Both the student and teacher receive the standard reasoning instruction I aha : โPlease reason step by step, and put your final answer within .โ In the Flow branch, s(x) = 1, the teacher is conditioned on the rewritten Flow- CoT reference y โ flow .Both the student and teacher receive the Flow-style instruc- tion I flow : โPlease reason directly and confidently, stating each step clearly without unnecessary hedging, and put your final answer within .โ The privileged reference and instruction for each example are defined as (y โ (x),I(x)) = ( y โ ,I aha , s(x) = 0, y โ flow ,I flow , s(x) = 1. (2) For each example, the same instructionI(x) is provided to both the student and the teacher, while the teacher additionally observes the privileged reference y โ . This keeps the teacher target aligned with the reasoning behavior requested from the student and reduces the mismatch between the demon- strated reasoning style and the studentโs intended behavior. This design is motivated by the sensitivity of OPSD to privileged information. As shown in our ex- periments, concise solutions generally provide stronger supervision than long CoT references, while different CoT forms also lead to different distillation outcomes. AFD therefore does not simply mix heterogeneous references under a shared instruction. Instead, it pairs each type of privileged information with a corresponding reasoning instruction: the Aha branch retains the strong supervi- sion provided by concise solution-based references, whereas the Flow branch introduces sustained Flow-style reasoning through both the rewritten CoT and a direct, confident instruction. In this way, AFD allows the student to benefit from complementary forms of supervision while preserv- ing the intended role of each branch. The resulting training signal is both more diverse and better aligned, enabling heterogeneous reasoning behaviors to be incorporated into self-distillation more effectively. 4 Experiments 4.1 Experimental Setup Training setup. We use Qwen3-8B and Qwen3-4B (Yang et al., 2025) as backbone models and follow the OPSD training recipe (Zhao et al., 2026). The student model is adapted with LoRA using rank r = 64 and scaling factor ฮฑ = 128, applied to all projection layers. During the teacher forward pass, the LoRA adapters are disabled, keeping the teacher fixed throughout training. By default, thinking mode is disabled for the student and enabled for the teacher. The maximum training completion length is set to 1,024 tokens. Following (Zhao et al., 2026), we use OpenThoughts-30K (Guha et al., 2026) as the base training dataset. We construct the Flow-CoT data by using the DeepSeek-V4-Flash API (Xu et al., 2026a) to rewrite the original Aha-style CoT traces while preserving their underlying reasoning content. The complete rewriting prompt is provided in Appendix A. Evaluation protocol. We evaluate on two mathematical-reasoning benchmarks, AIME25 and HMMT25. For each problem, we sample 12 completions using vLLM with tensor-parallel size 8, temperature 1.0, thinking mode enabled, and a maximum generation length of 38k tokens. We re- port Avg@12, computed as the mean correctness over the 12 sampled completions and then averaged across all problems. All models are evaluated with exactly the same generic reflective instruction and decoding configuration. 4.2 Main Results As shown in Table 1, AFD consistently improves over our reproduced OPSD baseline across both model scales and both benchmarks. On Qwen3-8B, AFD achieves 73.6 on AIME25 and 48.9 on HMMT25, compared with 73.1 and 48.6 for OPSD โ , increasing the average from 60.8 to 61.3. 6 Table 1: Main results on mathematical-reasoning benchmarks. All entries are Avg@12. Evaluation follows the Qwen3 sampling configuration with temperature 1.0 and a maximum generation length of 38k tokens. โ denotes our reproduction. The best result in each model group is shown in bold. MethodAIME25HMMT25Average Qwen3-8B Base (Instruct)65.643.954.8 + SFT64.242.953.6 + GRPO (Guo et al., 2025) 68.946.757.8 + OPSD (Zhao et al., 2026)70.845.858.3 + OPSD โ 73.148.660.8 + AFD (Ours)73.648.961.3 Qwen3-4B Base (Instruct)66.442.254.3 + SFT62.343.452.8 + GRPO (Guo et al., 2025) 68.144.456.3 + OPSD (Zhao et al., 2026)68.346.157.2 + OPSD โ 69.245.857.5 + AFD (Ours)69.747.558.6 Table 2: Ablation of privileged information and dual-mode training on Qwen3-8B. Rows containing two forms of privileged information use an equal 50/50 mixture. All other training and evaluation settings are held fixed. The best result in each column is shown in bold. Privileged informationTraining modeAIME25 HMMT25 Average Qwen3-8B โ65.643.954.8 SolutionOPSD73.148.660.8 Aha-CoTOPSD70.847.559.2 Flow-CoT / Aha-CoTOPSD70.848.159.5 Flow-CoT / Aha-CoTAFD72.848.360.1 Flow-CoT / SolutionAFD73.648.961.3 The improvement is larger on Qwen3-4B: AFD reaches 69.7 on AIME25 and 47.5 on HMMT25, compared with 69.2 and 45.8 for OPSD โ , raising the average from 57.5 to 58.6. These results show a consistent numerical advantage of AFD across model scales. In particular, the improvement over OPSD โ is observed on both benchmarks for both Qwen3-8B and Qwen3-4B, suggesting that the effect is not specific to a single benchmark or model size. Since all models use the same reflective prompt at evaluation, the gains arise from the training procedure rather than additional test-time prompting. 4.3 Ablation Study Table 2 first shows that the form of privileged information has a substantial effect on OPSD. Using the solution alone gives an average score of 60.8, outperforming Aha-CoT, which obtains 59.2. The difference is observed on both benchmarks: solution-based OPSD reaches 73.1 on AIME25 and 48.6 on HMMT25, whereas Aha-CoT reaches 70.8 and 47.5, respectively. Mixing Flow-CoT and Aha-CoT under standard OPSD slightly improves the average over Aha-CoT alone, from 59.2 to 59.5, but remains below the solution-based configuration. These results suggest that, as privileged information, a more direct and concise solution is generally more effective than a lengthy chain-of- thought trace. Beyond this general comparison, we are particularly interested in whether different forms of CoT supervision lead to different distillation outcomes. We isolate the effect of dual-mode training by comparing the two Flow-CoT/Aha-CoT configura- tions. They use the same 50/50 composition of privileged information and differ only in training mode. Standard OPSD obtains 70.8 on AIME25 and 48.1 on HMMT25, with an average of 59.5. Enabling AFD raises these scores to 72.8, 48.3, and 60.1, respectively. Because the privileged- information mixture is unchanged, this comparison indicates that the improvement cannot be ex- 7 plained by target composition alone. Explicitly pairing heterogeneous privileged information with corresponding reasoning instructions provides a more effective distillation signal than presenting the same mixture under a single training mode. Finally, the Flow-CoT/Solution configuration achieves the strongest overall performance, reaching 73.6 on AIME25, 48.9 on HMMT25, and 61.3 on average. Compared with Flow-CoT/Aha-CoT under AFD, replacing Aha-CoT with the refined solution improves both benchmarks and raises the average from 60.1 to 61.3. It also improves over solution-only OPSD, whose average is 60.8. Together, these results suggest two complementary conclusions: the choice of privileged information materially affects self-distillation, and dual-mode training provides additional benefit beyond simply mixing different forms of privileged information. 5 Conclusion We identify the Flow Moment, a reasoning verbalization pattern that signals sustained progression, as a complementary counterpart to the correction-oriented Aha Moment. Based on this observation, we construct Flow-CoT by rewriting discourse markers while preserving the underlying reasoning trajectory, and use it as auxiliary supervision for on-policy self-distillation. We further propose Aha- Flow Distillation (AFD), which pairs different forms of privileged information with corresponding reasoning instructions. Experiments on Qwen3-8B and Qwen3-4B show consistent improvements over OPSD on AIME25 and HMMT25, while the model continues to use the standard reflective reasoning mode at inference. More broadly, our results suggest that the form and organization of privileged reasoning information are important design choices for self-distillation, motivating further study of alternative reasoning verbalizations and their use as auxiliary supervision. References Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. GKD: generalized knowledge distillation for auto-regressive sequence mod- els. arXiv preprint arXiv:2306.13649, 2023. Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self- generated mistakes. 2024. Paul C. Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy. Thought anchors: Which LLM reasoning steps matter? arXiv preprint arXiv:2506.19143, 2025. Edward Y. Chang, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in LLMs. arXiv preprint arXiv:2502.03373, 2025. Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D. Goodman. Cog- nitive behaviors that enable self-improving reasoners, or, four habits of highly effective STaRs. arXiv preprint arXiv:2503.01307, 2025. Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. Minillm: Knowledge distillation of large lan- guage models. 2024. Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, et al. Openthoughts: Data recipes for rea- soning models. In International Conference on Learning Representations, volume 2026, pages 108059โ108130, 2026. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015. Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Findings of the association for computational linguistics: ACL 2023, pages 8003โ8017, 2023. 8 Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. In International confer- ence on learning representations, volume 2024, pages 32808โ32824, 2024. Woogyeol Jin, Taywon Min, Yongjin Yang, Dennis Wei, Yi Zhou, Swanand Ravindra Kadhe, Nathalie Baracaldo, and Kimin Lee. Entropy-aware on-policy distillation of language models. arXiv preprint arXiv:2603.07079, 2026. Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. When can llms actually correct their own mistakes? a critical survey of self-correction of llms. Transactions of the Association for Computational Linguistics, 12:1417โ1440, 2024. Junlong Ke, Zichen Wen, Weijia Li, Conghui He, and Linfeng Zhang. Respecting self-uncertainty in on-policy self-distillation for efficient llm reasoning. arXiv preprint arXiv:2605.13255, 2026. Jeonghye Kim, Xufang Luo, Minbeom Kim, Sangmook Lee, Dohyung Kim, Jiwon Jeon, Dongsheng Li, and Yuqing Yang. Why does self-distillation (sometimes) degrade the reasoning capability of llms? arXiv preprint arXiv:2603.24472, 2026. Kimi Team. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199โ22213, 2022. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel J. Cand ` es, and Tatsunori Hashimoto. s1: simple test-time scaling. pages 20275โ20321, 2025. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. Zhanming Shen, Jintao Tong, Shaotian Yan, Chen Shen, Hao Chen, Wentao Ye, Xiaomeng Hu, Rui Miao, Haobo Wang, Junbo Zhao, et al. Purified opsd: On-policy self-distillation without losing how to think. arXiv preprint arXiv:2607.02234, 2026. Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, Jianfeng Cai, Xinyuan Cai, Peizhou Cao, Yux- uan Cao, Ziwei Chai, Y Charles, et al. Kimi k3: Open frontier intelligence. arXiv preprint arXiv:2607.24653, 2026. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022. Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Thoughts are all over the place: On the underthinking of o1-like LLMs. arXiv preprint arXiv:2501.18585, 2025. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824โ24837, 2022. Bangjun Xiao, Bingquan Xia, Bo Yang, Bofei Gao, Bowen Shen, Chen Zhang, Chenhong He, Chiheng Lou, Fuli Luo, Gang Wang, et al. Mimo-v2-flash technical report. arXiv preprint arXiv:2601.02780, 2026. Anyi Xu, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chenchen Ling, et al. Deepseek-v4: Towards highly efficient million- token context intelligence. arXiv preprint arXiv:2606.19348, 2026a. Yuanda Xu, Hejian Sang, Zhengze Zhou, Ran He, and Zhipeng Wang. Paced: Distillation and on- policy self-distillation at the frontier of student competence. arXiv preprint arXiv:2603.11178, 2026b. 9 An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al.Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, Saiyong Yang, and Yankai Lin. Learning beyond teacher: Generalized on-policy distillation with reward extrapolation. arXiv preprint arXiv:2602.12125, 2026. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in neural information processing systems, 36:11809โ11822, 2023. Tianzhu Ye, Li Dong, Zewen Chi, Xun Wu, Shaohan Huang, and Furu Wei. Black-box on-policy distillation of large language models. arXiv preprint arXiv:2511.10643, 2025. Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. Distilling system 2 into system 1. arXiv preprint arXiv:2407.06023, 2024. Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models. arXiv preprint arXiv:2601.18734, 2026. 10 A Flow-CoT Rewriting Prompt We use DeepSeek-V4-Flash (Xu et al., 2026a) to rewrite the original reasoning traces into Flow-CoT. The rewriting is restricted to discourse markers and is designed to preserve the original reasoning trajectory and mathematical content as closely as possible. The complete system prompt and user template are shown below. SYSTEM_PROMPT = """ You are a precise text editor. You rewrite the *discourse markers* of a mathematical reasoning transcript without touching its mathematical content. Rewrite rules: 1. Replace every filler , hedging , or transition marker with an explicit first -person action statement in the "Iโm <verb >ing ..." form. Examples: - "Okay , let โs try to tackle this problem step by step." -> "Iโm tackling this problem step by step." - "Hmm , this seems complex ." -> "Iโm noticing that this is getting complex ." - "Wait , actually the formula is ..." -> "Iโm correcting myself: the formula is ..." - "But wait , let me check the constraints ." -> "Iโm now checking the constraints ." - "Alternatively , maybe using symmetric sums." -> "Iโm trying another approach: symmetric sums." - "Not directly obvious. Maybe not. Let โs try another approach ." -> "Iโm finding no direct correspondence , so Iโm dropping this idea and trying another approach ." - "Let me compute each term." / "Compute each term:" -> "Iโm computing each term:" 2. Preserve the reasoning trajectory exactly: keep every dead end , self - correction and verification step. Do NOT solve the problem yourself. Do NOT add , remove , merge or reorder any reasoning step. 3. Preserve ALL mathematics verbatim: every formula , variable , number , intermediate result , LaTeX command and unicode symbol (lambda , x_1 , ^3, sqrt , Delta , ...). 4. Preserve the paragraph and blank -line structure , and keep the trailing "** Final Answer **" line together with its ... exactly as-is. 5. Output ONLY the rewritten transcript. No preamble , no commentary , no code fences , no explanation of what you changed .""" The corresponding user template is: USER_TEMPLATE = """ Rewrite the discourse markers of the transcript below according to the rules. Return the full rewritten transcript only. === TRANSCRIPT BEGIN === text === TRANSCRIPT END ===""" The prompt is intentionally conservative: it instructs the rewriting model to change only the verbal- ization of the reasoning process while preserving the sequence of reasoning steps, including unsuc- cessful attempts, self-corrections, and verification. Therefore, the resulting Flow-CoT differs from the original CoT primarily in its discourse markers rather than its mathematical reasoning content. 11