Paper deep dive
On the Step Length Confounding in LLM Reasoning Data Selection
Bing Wang, Rui Miao, Chen Shen, Shaotian Yan, Kaiyuan Liu, Ximing Li, Xiaosong Yuan, Sinan Fan, Jun Zhang, Jieping Ye
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/10/2026, 3:44:43 AM
Summary
The paper identifies 'step length confounding' in naturalness-based LLM reasoning data selection, where models systematically prefer samples with longer reasoning steps due to the diluting effect of low-probability first tokens. The authors propose two mitigation methods, ASLEC-DROP and ASLEC-CASL, which adjust log probability calculations to remove this bias, demonstrating significant performance improvements across multiple benchmarks.
Entities (5)
Relation Signals (3)
ASLEC-DROP ā mitigates ā Step Length Confounding
confidence 95% Ā· ASLEC-DROP, which drops first-token probabilities when computing average log probability
ASLEC-CASL ā mitigates ā Step Length Confounding
confidence 95% Ā· ASLEC-CASL, which applies a causal debiasing regression to remove the first tokens' confounding effect
Step Length Confounding ā causedby ā Low-probability first tokens
confidence 90% Ā· we attribute this phenomenon to low-probability first tokens in reasoning steps
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large reasoning models have recently demonstrated strong performance on complex tasks that require long chain-of-thought reasoning, through supervised fine-tuning on large-scale and high-quality datasets. To construct such datasets, existing pipelines generate long reasoning data from more capable Large Language Models (LLMs) and apply manually heuristic or naturalness-based selection methods to filter high-quality samples. Despite the proven effectiveness of naturalness-based data selection, which ranks data by the average log probability assigned by LLMs, our analysis shows that, when applied to LLM reasoning datasets, it systematically prefers samples with longer reasoning steps (i.e., more tokens per step) rather than higher-quality ones, a phenomenon we term step length confounding. Through quantitative analysis, we attribute this phenomenon to low-probability first tokens in reasoning steps; longer steps dilute their influence, thereby inflating the average log probabilities. To address this issue, we propose two variant methods: ASLEC-DROP, which drops first-token probabilities when computing average log probability, and ASLEC-CASL, which applies a causal debiasing regression to remove the first tokens' confounding effect. Experiments across four LLMs and five evaluation benchmarks demonstrate the effectiveness of our approach in mitigating the step length confounding problem.
Tags
Links
- Source: https://arxiv.org/abs/2604.06834v1
- Canonical: https://arxiv.org/abs/2604.06834v1
Trouble viewing inline? Open PDF directly ā
Full Text
70,160 characters extracted from source content.
Expand or collapse full text
On the Step Length Confounding in LLM Reasoning Data Selection Bing Wang1,2,3, Rui Miao3,4, Chen Shen3, Shaotian Yan3, Kaiyuan Liu3,5, Ximing Li1,2,7ā, Xiaosong Yuan1,2,3, Sinan Fan3,5, Jun Zhang6, Jieping Ye3 1 College of Computer Science and Technology, Jilin University 2 Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University 3 Alibaba Cloud Computing 4 School of Artificial Intelligence, Jilin University 5 College of Computer Science and Technology, Zhejiang University 6 Department of Mathematics, University of Michigan 7 RIKEN Center for Advanced Intelligence Project wangbing1416,zjushenchen,liximing86@gmail.com, miaorui24@mails.jlu.edu.cn Corresponding authors Abstract Large reasoning models have recently demonstrated strong performance on complex tasks that require long chaināofāthought reasoning, through supervised fineātuning on largeāscale and highāquality datasets. To construct such datasets, existing pipelines generate long reasoning data from more capable Large Language Models (LLMs) and apply manually heuristic or naturalnessābased selection methods to filter highāquality samples. Despite the proven effectiveness of naturalnessābased data selection, which ranks data by the average log probability assigned by LLMs, our analysis shows that, when applied to LLM reasoning datasets, it systematically prefers samples with longer reasoning steps (i.e., more tokens per step) rather than higherāquality ones, a phenomenon we term step length confounding. Through quantitative analysis, we attribute this phenomenon to low-probability first tokens in reasoning steps; longer steps dilute their influence, thereby inflating the average log probabilities. To address this issue, we propose two variant methods: Aslec-drop, which drops firstātoken probabilities when computing average log probability, and Aslec-casl, which applies a causal debiasing regression to remove the first tokensā confounding effect. Experiments across four LLMs and five evaluation benchmarks demonstrate the effectiveness of our approach in mitigating the step length confounding problem. On the Step Length Confounding in LLM Reasoning Data Selection Bing Wang1,2,3, Rui Miao3,4, Chen Shen3ā thanks: Corresponding authors, Shaotian Yan3, Kaiyuan Liu3,5, Ximing Li1,2,7ā, Xiaosong Yuan1,2,3, Sinan Fan3,5, Jun Zhang6, Jieping Ye3 1 College of Computer Science and Technology, Jilin University 2 Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University 3 Alibaba Cloud Computing 4 School of Artificial Intelligence, Jilin University 5 College of Computer Science and Technology, Zhejiang University 6 Department of Mathematics, University of Michigan 7 RIKEN Center for Advanced Intelligence Project wangbing1416,zjushenchen,liximing86@gmail.com, miaorui24@mails.jlu.edu.cn 1 Introduction Recently, a variety of large reasoning models, e.g., DeepSeek-R1 (Guo et al., 2025), have achieved remarkable performance on complex reasoning tasks that demand long Chain-of-Thought (CoT) capabilities (Yang et al., 2025a; Team, 2025). To elicit such long CoT reasoning abilities in foundation models, Supervised FineāTuning (SFT) on largeāscale, highāquality datasets has become a standard paradigm (Chen et al., 2025b; Guha et al., 2025; Zhao et al., 2025; Yuan et al., 2026). Existing approaches typically begin by collecting complex mathematical and scientific problems, and then prompting stronger Large Language Models (LLMs) to generate answers as SFT datasets (Guha et al., 2025; Yuan et al., 2025; Huang et al., 2025). Despite this pipeline effectively scaling up SFT data, such datasets still contain noisy instances, e.g., incorrect reasoning steps (Zheng et al., 2025) or overly complex reasoning trajectories (Sui et al., 2025). To address this issue and build higherāquality data subsets, LLM reasoning data selection has emerged as an active research topic (Ye et al., 2025; Muennighoff et al., 2025). Generally, existing reasoning data selection methods often rely on heuristic rules, e.g., verifiable answer correctness (Zhao et al., 2025; Wu et al., 2025), response diversity (Jung et al., 2025; Li et al., 2025a), and problem difficulty (Muennighoff et al., 2025; Guha et al., 2025). These methods often depend heavily on manually crafted heuristics and do not consider the trained LLMās adaptability to the SFT data. To overcome this limitation, the community introduces a naturalness-based data selection strategy (Zhang et al., 2025; Just et al., 2025), which involves computing the log probability assigned by an LLM to each SFT data sample and selecting those with higher average probabilities, as they are presumed to be better aligned with the LLMās inherent preferences. Unfortunately, our findings reveal that, when applied to long CoT datasets, the naturalness-based selection methods significantly prefer samples with longer reasoning steps (i.e., more tokens per step) rather than higher-adaptability ones. We refer to this phenomenon as the step length confounding problem in this work. We show in Fig. 1, the step-length distribution of the selected SFT data differs markedly from that of the unselected data. To further investigate the cause of this confounder, we build upon the quantitative results presented in Figs. 2 and 3. We observe that longer reasoning steps generally yield higher average log probabilities. This phenomenon can be explained by prior work showing that the first token of each reasoning step often folks into different reasoning branches, thereby exhibiting higher entropy and consequently lower log probabilities (Wang et al., 2025; Cheng et al., 2025). Longer steps, however, dilute the impact of these low-probability first tokens, leading to a higher overall step log probability, which in turn makes such longer-step examples more likely to be selected. Given the above conclusion that low-probability first tokens lead to the step length confounding problem, we propose a mitigation method, namely Alleviating Step Length Confounding (Aslec), which includes two variant approaches Aslec-drop and Aslec-casl. Specifically, Aslec-drop attempts to mitigate the confounding problem by simply dropping the first-token probabilities when computing the global average log probability. Despite this straightforward approach offering a preliminary mitigation to the confounding issue, it also entirely discards the contribution of the first token to data selection. Accordingly, Aslec-casl, inspired by causal debiasing techniques (Udomcharoenchaikit et al., 2022), fits a linear regression model to disentangle the first-token ratio as a confounding factor, and removes its effect when computing the global average log probability. In our experiments, we train four LLMs of varying sizes on two LLM reasoning benchmark datasets LIMO-v2 (Ye et al., 2025) and AceReason-1.1-SFT (Chen et al., 2025b), and evaluate the performance of different data selection strategies across five evaluation benchmarks. Our results demonstrate that the two proposed variants, Aslec-drop and Aslec-casl, consistently outperform the state-of-the-art naturalness-based selection method, Local LP (Just et al., 2025), achieving average accuracy improvements of approximately 6.28% and 9.08%, respectively, across all model sizes and datasets. Our source code and data are released in https://github.com/wangbing1416/ASLEC. Meanwhile, in our implementation, we sample a large number of multi-source, multi-solution responses for LIMO-v2 and 10k AceReason-1.1-SFT problems (average 64 responses per question). These large-scale SFT datasets are also be released in https://huggingface.co/collections/wangbing1416/msms-cot-sft. Generally, our contributions can be summarized as the following three-fold: ā¢ Through extensive experiments, we identify a step length confounding problem in existing naturalness-based LLM reasoning data selection methods, and reveal that the cause lies in the low-probability first token of each step. ā¢ We propose two variant methods, Aslec-drop and Aslec-casl, which alleviate the step length confounding problem by intervening on the first-token probability when computing the global average log probability. ā¢ Extensive experiments demonstrate the effectiveness of our proposed method and its ability to mitigate step length confounding. 2 Preliminary Experimental Analysis on Step Length Confounding In this section, our preliminary experiments reveal that existing naturalness-based approaches for LLM reasoning data selection consistently suffer from step length confounding: they tend to prefer samples with longer reasoning steps. 2.1 Naturalness-Based Data Selection Typically, an LLM reasoning SFT dataset is defined as =i,i,ii=1ND=\q_i,c_i,a_i\_i=1^N, where q denotes one question, and c and a represent its long CoT reasoning trajectory and final answer, respectively. The SFT objective is to optimize model parameters Īø by minimizing the negative log-likelihood of the target sequence i=<think>āiā</think>āio_i= <think>\ c_i\ </think>\ a_i as: āSFTā()=ā1Nāāi=1Nāt=1|i|logā”Pā(oi,tā£oi,<t,i),L_SFT( Īø)=- 1N _i=1^N _t=1^|o_i| P_ Īø (o_i,t o_i,<t,q_i ), which is equal to the causal LM cross-entropy loss. While SFT typically treats all samples equally, data quality critically influences reasoning performance, as noisy or inconsistent trajectories can mislead learning. This motivates data selection strategies that prefer high-quality and informative subsets ^ā D to improve robustness. Among these works, naturalnessābased methods leverage the log probabilities produced by the LLM during SFT to select the data to which the model is best adapted. Formally, three representative methods are as follows: ā¢ Log probabilities (Zhang et al., 2025) or Perplexity (Muennighoff et al., 2023; Yin and Rush, 2025) computes the geometric mean of the probabilities assigned to the target sequence outputs, as follows: silogp s_i^logp =1|i|āāt=1|i|logā”Pā(oi,tā£i,<t,i), = 1|o_i| _t=1^|o_i| P_ Īø (o_i,t _i,<t,q_i ), sippl s_i^ppl =expā”(āsilogp). = (-s_i^logp ). (1) A higher silogps_i^logp indicates that the model naturally adapts better to the given data. ā¢ Local log probabilities (Just et al., 2025) split the sequence io_i into steps i=iājj=1|i|S_i=\s_ij\_j=1^|S_i| by the token or sentences. For each step, it considers the question and its previous k steps as context and calculates the geometric mean of its log probability accordingly. siloc s_i^loc =1|i|āāiājāi1|iāj|āāl=1|iāj|log = 1|S_i| _s_ij _i 1|s_ij| _l=1^|s_ij| (2) Pā(siājālā£iāj,<l,i,maxā”(1,jāk):jā1,i). P_ Īø (s_ijl _ij,<l,s_i, (1,j-k):j-1,q_i ). ā¢ Entropy (Cui et al., 2025; Wang et al., 2025) measures the average token-level uncertainty of the modelās predictions. sietp=1|i|āt=1|i|[āāvā s_i^etp= 1|o_i| _t=1^|o_i| [-Ī£ _v Pā(vā£oi,<t,i) P_ Īø(v o_i,<t,q_i) logP(vā£ P_ Īø(v oi,<t,i)], o_i,<t,q_i) ], (3) where V represents the vocabulary, and lower entropy means the model is more confident in its outputs on the given example. Existing naturalnessābased methods typically select a subset D from the largeāscale dataset D by either highest silogps_i^logp and silocs_i^loc, or lowest sippls_i^ppl and sietps_i^etp. 2.2 Experimental Setup Models. Our experiments utilize four long CoT reasoning LLMs of different families and parameters, including QwQ-32B (Team, 2025), Qwen3-32B (Yang et al., 2025a), DeepSeek-R1-Distill-Qwen-32B (Guo et al., 2025), and gpt-oss-120b (Agarwal et al., 2025), as data sources for generating reasoning SFT data. We then use Qwen3-4B-Base (Yang et al., 2025a) as the target LLM to evaluate its log probabilities w.r.t the SFT data. Detailed model cards for all LLMs are provided in Appendix B.1. Benchmark and evaluation. We conduct our experiments on LIMO-v2 (Ye et al., 2025), a prevalent LLM reasoning benchmark comprising 800 carefully curated mathematical reasoning problems. For each problem, we generate 5 different responses from each of the 4 LLMs described above, using temperature sampling with Ļ=0.6Ļ=0.6. From the generated 4Ć5=204Ć 5=20 responses per problem, we select 5 final responses using one of the four representative naturalnessābased data selection methods, (i) GRACE (Zhang et al., 2025), which selects the responses with the highest slogps^logp; (i) Local LP (Just et al., 2025), which also selects the responses with the highest slocs^loc; (i) Min Entropy (Cui et al., 2025), which selects the responses with the smallest setps^etp; (iv) Min Perplex, which select the responses with the smallest sippls_i^ppl, to analyze the step length confounding phenomenon. More details on the benchmarks and experimental setup are provided in Appendix B.2, and additional analysis across more datasets, e.g., AceReason-1.1-SFT, and more LLMs can be found in Appendix C. 2.3 Results on Step Length Confounding Through the preliminary experiments in this section, we find that these naturalness-based methods suffer from the step length confounding issue. Figure 1: Step length distribution of data samples selected and unselected by different naturalness-based data selection methods. Results and analysis. In Fig. 1, we illustrate the selection difference of naturalnessābased methods across the 16,000 responses (800 problems Ć 20 responses each) generated by the four LLMs. Fig. 1 compares the step-length distributions of responses selected versus not selected by four naturalness-based data selection methods. Across all methods, selected responses consistently exhibit longer step lengths, whereas the step lengths of unselected responses are more concentrated at lower values, with an average of approximately 30. This pattern underscores the consistent influence of step length on the decisions made by these naturalness-based criteria. 111Notably, data selection actually also correlates with total response length (i.e., avg. L). However, as discussed in Appendix A, the effect of total response length is substantially weaker than that of step length. Based on these observations, we formulate the following conclusion and refer to this phenomenon as step length confounding. ā Conclusion 1. The naturalness-based reasoning data selection approach tends to prefer samples with longer reasoning steps (i.e., more tokens per step). Figure 2: Relationship between stepālevel log probability and step length. 2.4 Why Step Length Confounding? Given the step length confounding problem in LLM reasoning data selection, we seek to figure out the intrinsic causes resulting in this issue. Therefore, we give the following further empirical evidence. For longer steps, the model assigns higher step-level log probabilities. We first investigate the relationship between step length and the average log probability per step. As illustrated in Fig. 2, outputs from different LLMs are segmented into steps. For steps of different lengths, we compute the average stepālevel log probabilities assigned by the target LLM Qwen3-4B-Base. The results reveal a clear pattern: longer reasoning steps consistently receive higher stepālevel log probabilities, and a monotonic increasing relationship is observed between step length and log probability. For longer steps, the ratio of low-probability first tokens is lower. To further investigate the cause of the monotonic relationship between step length and step-level log probability, we examine several representative examples in Fig. 3, which illustrate short steps with low log probabilities and long steps with high log probabilities, respectively. Across all steps, the first token consistently exhibits a lower log probability. Previous studies have also confirmed this phenomenon (Wang et al., 2025; Cheng et al., 2025), attributing it to the fact that minority first tokens at each step often fork branches toward diverse reasoning pathways. Such branching behavior introduces higher entropy, which in turn yields lower log probabilities. Therefore, in longer steps, such low-probability first tokens always constitute a smaller proportion of the total tokens. Consequently, the larger number of nonāfirst tokens dilutes the lower log probabilities of these first tokens, leading to a higher overall log probability and making such samples more likely to be selected by naturalnessābased methods. In summary, our experiments lead to the following conclusion: ā Conclusion 2. In naturalnessābased methods, step length confounding occurs because the low-probability first token constitutes a smaller ratio of longer responses. This increases the average log probabilities, making samples with longer steps more likely to be selected. Based on the above observations, we seek to design a debiasing approach targeting the first token to address the step length confounding problem. Figure 3: Representative cases illustrating tokenālevel log probabilities for varying step lengths. 3 The Proposed Method In this section, we present our proposed variant methods Aslec-drop and Aslec-casl for LLM reasoning SFT data selection in detail. Problem definition. Given i-th complex question iq_i in the LLM reasoning SFT dataset D, and K different responses ik=<think>āikā</think>āiko_i^k= <think>\ c_i^k\ </think>\ a_i^k, kā1,āÆ,Kkā\1,Ā·s,K\, where each ikc_i^k represents a reasoning trajectory and ika_i^k the corresponding answer. These multiple responses may vary not only in correctness but also in reasoning quality, verbosity, and step length. Our method defines two metrics sidrops_i^drop and sicasls_i^casl to select one or more responses that best align with the trained reasoning LLM and are not confounded by the step length. 3.1 Aslec-drop: Dropping the First Token As analyzed in Sec. 2.4, we attribute this step length confounding problem to the influence of the first tokenās probabilities (Bu et al., 2025). Consequently, the most straightforward approach is to drop the first token at each step when computing the geometric mean of the probabilities. Formally, we split a solution io_i into L reasoning steps i=ill=1LS_i=\s_i^l\_l=1^L and compute the metric as follows: sidrop=1|i|ā|i| s_i^drop= 1|o_i|-|S_i| āilāiāt=2|il| Ī£ _s_i^l _iĪ£ _t=2^|s_i^l| (4) log Pā(si,tlā£i,<tl,i<l,i), P_ Īø (s_i,t^l _i,<t^l,s_i^<l,q_i ), where i,<tls_i,<t^l denotes the first t tokens of each step, and i<ls_i^<l denotes all tokens across the first l steps. 3.2 Aslec-casl: Causally De-biasing Although dropping the first token mitigates the bias it introduces, it simultaneously discards potentially informative signals carried by the first token itself. To address this trade-off, we draw inspiration from causal debiasing methods (Udomcharoenchaikit et al., 2022; Zhu et al., 2022), treating step length as a confounding factor and applying appropriate adjustments to account for its influence. To formalize this intuition, the log probability silogps_i^logp can be decomposed as the following linear regression equation: silogp=Ī²1āsifirst+Ī²2āsidrop+Ī³āi+Ļµ,s_i^logp= _1s_i^first+ _2s_i^drop+ _i+Īµ, (5) where sifirsts_i^first and sidrops_i^drop represent the average log probabilities of the first token and the tokens excluding the first one in Eq. (4), respectively; iZ_i represents the confounding factor, which in our method is defined as the proportion of the first token among all tokens; and ĻµĪµ denotes a residual noise term. The notation sifirsts_i^first and iZ_i are formally given by: sifirst=1|i|āāilāilogā”Pā(si,1lā£i<l,i),i=|i||i|, gathereds_i^first= 1|S_i| _s_i^l _i P_ Īø (s_i,1^l _i^<l,q_i ),\\ Z_i= |S_i||o_i|, gathered (6) where si,1ls_i,1^l denotes the first token in the step ils_i^l. The basic idea of Aslec-casl is to adjust the raw log-probability score by removing the estimated influence of the confounder iZ_i. This yields a deconfounded metric sicasls_i^casl, defined as: sicaslā¼silogpāĪ³āi.s_i^casl s_i^logp- _i. (7) Accordingly, to calculate sicasls_i^casl by estimating Ī³, given the dataset silogp,sifirst,sidrop,ii=1N\s_i^logp,s_i^first,s_i^drop,Z_i\_i=1^N, the parameters Ī²1 _1, Ī²2 _2, Ī³ are estimated via ordinary least squares: Ī²1,Ī²2,Ī³āi=1N(silogpāĪ²1āsifirstāĪ²2āsidropāĪ³āi)2. _ _1, _2,Ī³ _i=1^N (s_i^logp- _1s_i^first- _2s_i^drop- _i )^2. (8) This optimization admits a closed-form solution. Then the parameter vector is obtained as follows: [Ī²1,Ī²2,Ī³]ā¤=(ā¤ā)ā1āā¤ā, [ _1, _2,Ī³ ] = (X X )^-1X Y, (9) i,:=[sifirst,sidrop,i],i,:=[silogp].X_i,:= [s_i^first,s_i^drop,Z_i ],\ Y_i,:= [s_i^logp ]. (10) Once Ī³ is estimated, we compute the final deconfounded score sicasl=silogpāĪ³āis_i^casl=s_i^logp- _i for each instance and use it for downstream data selection. 4 Experimental Evaluation Table 1: Experimental results on LIMO-v2 (Ye et al., 2025). We generate five responses per source LLM, and select five responses from these ones (select 4k responses from 16k data). The bold results represent the best scores. Qwen Method AIME24 AIME25 MATH500 OlympicB Avg. Accuracy Pass@4 Accuracy Pass@4 4B-Base + GRACE (Zhang et al., 2025) 16.66 30.00 15.83 33.33 59.40 33.33 31.42 + Local LP (Just et al., 2025) 19.16 36.66 20.83 36.66 71.60 34.11 36.50 lightgrayv+ Aslec-drop (ours) lightgrayv30.00 ā 10.84 lightgrayv50.00 ā 13.34 lightgrayv28.33 ā 7.50 lightgrayv43.33 ā 6.67 lightgrayv77.80 ā 6.20 lightgrayv38.38 ā 4.27 lightgrayv44.64 lightgrayv+ Aslec-casl (ours) lightgrayv31.66 ā 12.50 lightgrayv53.33 ā 16.67 lightgrayv30.83 ā 10.00 lightgrayv46.66 ā 10.00 lightgrayv80.00 ā 8.40 lightgrayv42.81 ā 8.70 lightgrayv47.54 8B-Base + GRACE (Zhang et al., 2025) 30.83 53.33 21.66 36.66 72.00 39.70 42.36 + Local LP (Just et al., 2025) 34.16 53.33 20.83 36.66 76.60 42.81 44.06 lightgrayv+ Aslec-drop (ours) lightgrayv41.66 ā 10.50 lightgrayv66.66 ā 13.33 lightgrayv36.66 ā 15.83 lightgrayv43.33 ā 6.67 lightgrayv81.40 ā 4.80 lightgrayv47.85 ā 5.04 lightgrayv52.92 lightgrayv+ Aslec-casl (ours) lightgrayv45.00 ā 13.34 lightgrayv66.66 ā 13.33 lightgrayv37.50 ā 16.67 lightgrayv53.33 ā 16.67 lightgrayv85.40 ā 8.80 lightgrayv49.03 ā 6.22 lightgrayv56.15 4B-Instruct + GRACE (Zhang et al., 2025) 59.16 73.33 50.00 73.33 79.36 47.79 63.82 + Local LP (Just et al., 2025) 61.66 80.00 49.16 73.33 80.75 50.14 65.84 lightgrayv+ Aslec-drop (ours) lightgrayv69.16 ā 7.50 lightgrayv83.33 ā 3.33 lightgrayv56.66 ā 7.50 lightgrayv80.00 ā 6.67 lightgrayv89.88 ā 9.13 lightgrayv57.64 ā 7.50 lightgrayv72.77 lightgrayv+ Aslec-casl (ours) lightgrayv71.66 ā 10.00 lightgrayv90.00 ā 10.00 lightgrayv58.33 ā 9.17 lightgrayv83.33 ā 10.00 lightgrayv93.20 ā 12.45 lightgrayv60.44 ā 10.30 lightgrayv76.16 7B-Instruct + GRACE (Zhang et al., 2025) 17.50 26.66 11.66 23.33 61.50 32.35 28.83 + Local LP (Just et al., 2025) 17.50 30.00 10.83 26.66 64.68 33.97 30.60 lightgrayv+ Aslec-drop (ours) lightgrayv24.16 ā 6.66 lightgrayv40.00 ā 10.00 lightgrayv20.83 ā 10.00 lightgrayv36.66 ā 10.00 lightgrayv80.60 ā 15.92 lightgrayv41.17 ā 7.20 lightgrayv40.57 lightgrayv+ Aslec-casl (ours) lightgrayv28.33 ā 10.83 lightgrayv46.66 ā 16.66 lightgrayv24.16 ā 13.33 lightgrayv46.66 ā 20.00 lightgrayv81.60 ā 16.92 lightgrayv45.18 ā 11.21 lightgrayv45.43 In this section, we empirically evaluate the performance of our two proposed variant methods. Evaluation settings. The experiments are conducted on two datasets, LIMO-v2 and AceReason-1.1-SFT, using four different families of source LLMs: QwQ-32B, Qwen3-32B, DeepSeek-R1-Distill-Qwen-32B, and gpt-oss-120b, and four target LLMs of varying sizes: Qwen3-4B-Base, Qwen3-8B-Base, Qwen3-4B-Instruct, and Qwen2.5-7B-Instruct. Detailed descriptions of these LLMs and the implementation details of our SFT training are provided in Appendix B. We evaluate our trained LLMs on five benchmarks, including four mathematical reasoning datasets, AIME24, AIME25, MATH500 (Lightman et al., 2023), and OlympiadBench (He et al., 2024), as well as one challenging scientific reasoning dataset GPQA (Rein et al., 2024). In addition, we compare two naturalness-based data selection methods: (i) GRAPE: selecting the responses with the highest slogps^logp; and (i) Local LP: selecting the responses with the highest slocs^loc. 4.1 Main Results Tables 1 and 3 present the experimental results of the four target LLMs on the LIMO-v2 and AceReason-1.1-SFT datasets, respectively. Overall, both variants of our approach outperform the existing naturalnessābased selection method, achieving average accuracy gains of 6.28% and 9.08% over the SOTA method Local LP (Just et al., 2025) on the two datasets. More specifically, prior naturalnessābased methods, e.g., GRACE and Local LP, are often hindered by the step length confounding problem, which leads them to overly prefer samples from a single data source. Consequently, their training performance consistently degrades and falls significantly below our method, which samples more evenly across diverse sources. When comparing the two variant methods, Aslec-casl consistently outperforms Aslec-drop. This result suggests that the causal debiasing strategy successfully preserves the informative patterns embedded in the probability distribution of the first tokens. Meanwhile, the results indicate that our debiasing strategies are particularly effective when data or model capacity is limited. For example, the performance gain on LIMO-v2 exceeds that on AceReason-1.1-SFT, highlighting their strong suitability for low-resource SFT scenarios. In such settings, low-quality samples tend to have a more pronounced negative effect; our methods mitigate step length confounding and thereby enhance generalization performance. We further compare our methods by training on LIMOāv2 and evaluating on GPQA, a benchmark on the scientific domain. The experimental results again show that our approaches consistently and significantly outperform the SOTA Local LP selection baseline, and that the Aslec-casl variant achieves better performance than Aslec-drop. Table 2: Experimental performance on GPQA. Method Acc. Pass@4 Acc. Pass@4 Qwen3-4B-Base Qwen3-4B-Insruct GRACE 28.15 60.10 50.37 75.75 Local LP 29.16 61.61 52.14 77.27 lightgrayv Aslec-drop 34.97 65.65 58.83 83.33 lightgrayv Aslec-casl 35.35 66.66 61.23 84.34 Method Acc. Pass@4 Acc. Pass@4 Qwen3-8B-Base Qwen2.5-7B-Insruct GRACE 47.97 75.25 25.37 56.06 Local LP 49.49 77.77 26.13 57.57 lightgrayv Aslec-drop 51.01 79.79 35.98 67.17 lightgrayv Aslec-casl 52.14 82.32 38.51 74.74 Figure 4: Step length distributions for data selected versus unselected by our two proposed variant methods. Table 3: Experimental results on AceReason-1.1-SFT (Chen et al., 2025b). We generate one response per source LLM, and select one response from these responses (select 10k responses from 40k data). Qwen Method AIME24 AIME25 MATH500 OlympicB Avg. Accuracy Pass@4 Accuracy Pass@4 4B-Base + GRACE (Zhang et al., 2025) 30.00 60.00 29.16 36.66 77.20 42.50 43.82 + Local LP (Just et al., 2025) 32.50 63.33 30.00 40.00 77.80 43.08 45.22 lightgrayv+ Aslec-drop (ours) lightgrayv41.66 ā 9.16 lightgrayv73.33 ā 10.00 lightgrayv30.83 ā 0.83 lightgrayv43.33 ā 3.33 lightgrayv84.00 ā 6.20 lightgrayv47.20 ā 4.12 lightgrayv46.48 lightgrayv+ Aslec-casl (ours) lightgrayv46.66 ā 14.16 lightgrayv73.33 ā 10.00 lightgrayv30.83 ā 0.83 lightgrayv46.66 ā 6.66 lightgrayv84.60 ā 6.80 lightgrayv48.23 ā 5.15 lightgrayv47.50 8B-Base + GRACE (Zhang et al., 2025) 40.83 66.66 29.16 43.33 75.00 44.11 49.46 + Local LP (Just et al., 2025) 42.50 70.00 29.16 43.33 77.60 44.41 50.71 lightgrayv+ Aslec-drop (ours) lightgrayv50.00 ā 7.50 lightgrayv76.66 ā 6.66 lightgrayv36.66 ā 7.50 lightgrayv53.33 ā 10.00 lightgrayv86.20 ā 8.60 lightgrayv49.33 ā 4.92 lightgrayv54.60 lightgrayv+ Aslec-casl (ours) lightgrayv53.33 ā 10.83 lightgrayv76.66 ā 6.66 lightgrayv39.16 ā 10.00 lightgrayv53.33 ā 10.00 lightgrayv86.60 ā 9.00 lightgrayv51.02 ā 6.61 lightgrayv56.21 4B-Instruct + GRACE (Zhang et al., 2025) 54.16 73.33 43.33 63.33 84.80 48.82 59.65 + Local LP (Just et al., 2025) 57.50 76.66 42.50 66.66 85.60 49.26 61.79 lightgrayv+ Aslec-drop (ours) lightgrayv64.16 ā 6.66 lightgrayv83.33 ā 6.67 lightgrayv53.33 ā 10.83 lightgrayv76.66 ā 10.00 lightgrayv90.60 ā 5.00 lightgrayv57.18 ā 7.92 lightgrayv66.48 lightgrayv+ Aslec-casl (ours) lightgrayv68.33 ā 10.83 lightgrayv83.33 ā 6.67 lightgrayv55.00 ā 12.50 lightgrayv80.00 ā 13.34 lightgrayv92.20 ā 6.60 lightgrayv57.64 ā 8.38 lightgrayv68.16 7B-Instruct + GRACE (Zhang et al., 2025) 15.00 36.66 15.00 26.66 73.80 34.85 35.08 + Local LP (Just et al., 2025) 17.50 36.66 16.66 26.66 74.60 35.58 35.81 lightgrayv+ Aslec-drop (ours) lightgrayv25.00 ā 7.50 lightgrayv43.33 ā 6.67 lightgrayv22.50 ā 5.84 lightgrayv40.00 ā 13.34 lightgrayv82.00 ā 7.40 lightgrayv41.61 ā 6.03 lightgrayv42.35 lightgrayv+ Aslec-casl (ours) lightgrayv30.00 ā 12.50 lightgrayv50.00 ā 13.34 lightgrayv24.16 ā 7.50 lightgrayv43.33 ā 16.67 lightgrayv82.40 ā 7.80 lightgrayv46.32 ā 10.74 lightgrayv46.07 4.2 Performance on Alleviating Confounding To examine whether our proposed methods effectively address the step length confounding problem, we present in Fig. 4 the distributions of step lengths for data samples selected and unselected by our two variants Aslec-drop and Aslec-casl. In contrast to the significant differences in step length distributions exhibited by prior approaches in Fig. 1, our approaches yield markedly smaller step length disparities. This demonstrates that our methods successfully mitigate step length confounding. Moreover, because both variants operate by intervening directly on the probability assigned to the first token, this result further implies that the step length confounding issue is intimately linked to the modelās first-token probabilities. 4.3 Comparing Min and Max Probabilities We also compare the performance of the Aslec-casl variant when selecting samples with the highest (max Casl) versus the lowest (min Casl) probability values of metric scasls^casl. All models are trained on the LIMOāv2 dataset, and Table 4 reports their performance on four evaluation benchmarks after training the four target LLMs. The experimental results consistently show that selecting samples with the highest probabilities outperforms selecting the lowest ones, a finding aligned with previous naturalnessābased selection methods. In general, high-probability samples correspond to data that better align with the target LLMās current capabilities, enabling more effective and stable learning of the SFT data distribution, thereby leading to superior performance. Conversely, lower-probability samples reflect that the model is less familiar with the data, which can introduce noisy training gradients and ultimately degrade model performance. Table 4: Comparison of the experimental results for Aslec-casl that selects the highest and lowest scasls^casl. Method AIME24 AIME25 MATH OlymB. Qwen3-4B-Base lightgrayv max Casl 31.66 30.83 80.00 42.81 min Casl 29.16 28.33 77.40 39.70 Qwen3-8B-Base lightgrayv max Casl 45.00 37.50 85.40 49.03 min Casl 41.66 36.66 79.60 42.94 Qwen3-4B-Instruct lightgrayv max Casl 71.66 58.33 93.20 60.44 min Casl 65.83 55.83 86.80 55.14 Qwen2.5-7B-Instruct lightgrayv max Casl 28.33 24.16 81.60 45.18 min Casl 25.00 22.50 79.60 43.08 4.4 Linear Regression Results Our method Aslec-casl fits a linear regression model as defined in Eq. (5), and uses this model to remove the influence of the firstātoken ratio on the average log probability. Accordingly, Table 5 presents the fitted results of the linear regression for data originating from different source LLMs. First, Ī³ is the most important coefficient, as it directly determines the extent to which the firstātoken ratio affects the average probability. Overall, the largest Ī³ value among the models is -1.284, meaning that for every 0.05 difference in the firstātoken ratio between samples, the impact on the overall probability is equivalent to reducing each tokenās probability by 1āeā1.284Ć0.05=6.22%1-e^-1.284Ć 0.05=6.22\%. For the regression fitted on all SFT data, Ī³ is -0.680, corresponding to a 1āeā0.680Ć0.05=3.34%1-e^-0.680Ć 0.05=3.34\% reduction in perātoken probability. Comparing different source LLMs, gpt-oss-120b exhibits the largest Ī³ value, indicating that the data from it suffers from a more pronounced confounding problem. In contrast, when comparing Ī²1 _1 (the firstātoken probability) and Ī²2 _2 (the nonāfirstātoken probability), we observe Ī²1āŖĪ²2 _1 _2, which further suggests that the ratio of the firstātoken probability in the global average probability should be reduced. Lastly, ĻµĪµ consistently remains at a low level, implying that the regression model has only minor fitting bias, and thus the debiasing results are accurate. Table 5: Linear regression parameters of Eq. (5) fitted on data generated by different source LLMs on LIMOāv2. Source Ī²1 _1 Ī²2 _2 Ī³ ĻµĪµ QwQ-32B 0.077 0.919 [HTML]F4F3F8 -0.284 -0.001 Qwen3-32B 0.068 0.929 [HTML]F0EFF4 -0.529 0.007 DS-Qwen-32B 0.066 0.943 [HTML]F -0.226 0.001 gpt-oss-120b 0.068 0.938 [HTML]ECEBF0 -1.284 0.028 lightgrayv Overall 0.066 0.944 -0.680 -0.054 4.5 Computation Budget Compared to GRACE, our Aslec-drop variant adopts a more streamlined approach: it simply drops the first token when computing the average token probability, thereby avoiding any additional computational overhead. In contrast, our Aslec-casl variant introduces a modest amount of extra computation by fitting a lightweight linear regression model. However, since this regression model involves only a small number of parameters, the fitting procedure is highly efficient and typically completes in just a few seconds, imposing negligible cost on the overall pipeline. 5 Related Works Recently, instead of direct prompt LLMs to generate CoT responses (Wei et al., 2022; Yuan et al., 2024; Bi et al., 2025), leveraging SFT to elicit long CoT reasoning in LLMs has emerged as a standard training paradigm, outperforming large-scale reinforcement learning even when applied to smaller models (Guo et al., 2025; Yang et al., 2025a; Tian et al., 2025; Kou et al., 2026). Generally, the existing methods typically scale up the SFT data generated by a strong LLM by constructing a large and diverse set of questions (Zhao et al., 2025; Guha et al., 2025; Yuan et al., 2025), and generating diverse solutions for each question through temperature sampling (Chen et al., 2023; Lei et al., 2025; Chen et al., 2025b; Yan et al., 2026). Building on these large-scale datasets, some studies have also sought to filter out noisy data by applying various heuristic rules, e.g., question difficulty (Muennighoff et al., 2025; Guha et al., 2025; Li et al., 2025b), solution correctness (Chen et al., 2025a; Luo et al., 2025), diversity of solutions (Jung et al., 2025; Li et al., 2025a), and LLM-as-a-Judge (Wu et al., 2025; Lei et al., 2025). Remarkably, even reducing the dataset to only 1k questions can still elicit the long CoT reasoning capability (Muennighoff et al., 2025; Ye et al., 2025). Beyond heuristic dataāselection strategies, several works advocate naturalnessābased approaches (Zhang et al., 2025; Just et al., 2025; Liu et al., 2026), wherein data are selected based on the modelās confidence scores, allowing the selection of samples to which the model is better adapted. Although naturalnessābased methods can indeed assess a modelās adaptability to data via confidence scores (Kang et al., 2025; Fu et al., 2025), recent studies have shown that reasoning data often contain a small number of highāentropy / low-probability first tokens (Yang et al., 2025b; Wang et al., 2025). Our experiments further confirm that these low-probability first tokens can substantially exacerbate the stepālength confounding problem in naturalnessābased approaches. 6 Conclusion In this work, we investigate the limitations of naturalnessābased data selection for long CoT reasoning datasets. Our analysis reveals a systematic bias, termed step length confounding, whereby the selection pipeline significantly prefers samples with longer reasoning steps instead of those with higher reasoning quality. We trace this phenomenon to the disproportionate influence of lowāprobability first tokens in reasoning steps, which is diluted in longer sequences, thus inflating their average log probabilities. To mitigate this problem, we propose Aslec-drop and Aslec-casl, two variants that drop or causally debias the firstātoken probability when computing selection scores. Extensive experiments on two reasoning SFT datasets, across four LLMs and five evaluation benchmarks, demonstrate that our approaches consistently outperform existing naturalness-based selection methods and effectively alleviate the step length confounding problem. Limitations This paper systematically investigates an important property in SFT data for LLM reasoning: the relationship between response and step length, and naturalness-based data selection. From a methodological perspective, one major limitation lies in our identification of the influence of a critical first token on step length confounding; whether there are deeper confounding factors remains an open question. In addition, recent studies have focused on on-policy data generation and selection (Yang et al., 2025a; Lu and Lab, 2025), where the student model produces its own samples for training. Whether data selection in such approaches still exhibits a strong correlation with response length is an issue worthy of further exploration. Acknowledgement This work was supported in part by the National Natural Science Foundation of China (No.62276113). References S. Agarwal, L. Ahmad, J. Ai, S. Altman, A. Applebaum, E. Arbus, R. K. Arora, Y. Bai, B. Baker, H. Bao, et al. (2025) Gpt-oss-120b & gpt-oss-20b model card. CoRR abs/2508.10925. Cited by: 4th item, Ā§2.2. J. Bi, D. Yan, Y. Wang, W. Huang, H. Chen, G. Wan, M. Ye, X. Xiao, H. SchĆ¼tze, V. Tresp, and Y. Ma (2025) CoT-kinetics: A theoretical modeling assessing LRM reasoning process. CoRR abs/2505.13408. Cited by: Ā§5. Y. Bu, L. Huo, Y. Jing, and Q. Yang (2025) Beyond excess and deficiency: adaptive length bias mitigation in reward models for RLHF. In Findings of the Association for Computational Linguistics: NAACL, p. 3091ā3098. Cited by: Ā§3.1. H. Chen, S. Wu, X. Quan, R. Wang, M. Yan, and J. Zhang (2023) MCC-KD: multi-cot consistent knowledge distillation. In Findings of the Association for Computational Linguistics: EMNLP, p. 6805ā6820. Cited by: Ā§5. X. Chen, S. Zhou, K. Liang, X. Sun, and X. Liu (2025a) Skip-thinking: chunk-wise chain-of-thought distillation enable smaller language models to reason better and faster. In Conference on Empirical Methods in Natural Language Processing, p. 12153ā12168. Cited by: Ā§5. Y. Chen, Z. Yang, Z. Liu, C. Lee, P. Xu, M. Shoeybi, B. Catanzaro, and W. Ping (2025b) AceReason-nemotron: advancing math and code reasoning through reinforcement learning. CoRR abs/2505.16400. Cited by: Ā§A.3, 2nd item, Ā§1, Ā§1, Table 3, Ā§5. D. Cheng, S. Huang, X. Zhu, B. Dai, W. X. Zhao, Z. Zhang, and F. Wei (2025) Reasoning with exploration: an entropy perspective. CoRR abs/2506.14758. Cited by: Ā§1, Ā§2.4. G. Cui, Y. Zhang, J. Chen, L. Yuan, Z. Wang, Y. Zuo, H. Li, Y. Fan, H. Chen, W. Chen, Z. Liu, H. Peng, L. Bai, W. Ouyang, Y. Cheng, B. Zhou, and N. Ding (2025) The entropy mechanism of reinforcement learning for reasoning language models. CoRR abs/2505.22617. Cited by: 3rd item, Ā§2.2. Y. Fu, X. Wang, Y. Tian, and J. Zhao (2025) Deep think with confidence. CoRR abs/2508.15260. Cited by: Ā§5. E. K. Guha, R. Marten, S. Keh, N. Raoof, G. Smyrnis, H. Bansal, M. Nezhurina, J. Mercat, T. Vu, Z. Sprague, et al. (2025) OpenThoughts: data recipes for reasoning models. CoRR abs/2506.04178. Cited by: Ā§A.3, Ā§1, Ā§1, Ā§5. D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. (2025) DeepSeek-r1: incentivizing reasoning capability in llms via reinforcement learning. CoRR abs/2501.12948. Cited by: 3rd item, Ā§1, Ā§2.2, Ā§5. C. He, R. Luo, Y. Bai, S. Hu, Z. Thai, J. Shen, J. Hu, X. Han, Y. Huang, Y. Zhang, et al. (2024) Olympiadbench: a challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. In Annual Meeting of the Association for Computational Linguistics, p. 3828ā3850. Cited by: Ā§4. X. Huang, Rishabh, G. Franke, Z. Yang, J. Bai, W. Bai, J. Bi, Z. Ding, Y. Duan, C. Fan, et al. (2025) Loong: synthesize long chain-of-thoughts at scale through verifiers. CoRR abs/2509.03059. Cited by: Ā§1. M. Jin, Q. Yu, D. Shu, H. Zhao, W. Hua, Y. Meng, Y. Zhang, and M. Du (2024) The impact of reasoning step length on large language models. In Findings of the Association for Computational Linguistics, ACL, p. 1830ā1842. Cited by: Ā§A.3. J. Jung, S. Han, X. Lu, S. Hallinan, D. Acuna, S. Prabhumoye, M. Patwary, M. Shoeybi, B. Catanzaro, and Y. Choi (2025) Prismatic synthesis: gradient-based data diversification boosts generalization in LLM reasoning. CoRR abs/2505.20161. Cited by: Ā§1, Ā§5. H. A. Just, M. Ko, and R. Jia (2025) Distilling reasoning into student llms: local naturalness for selecting teacher data. CoRR abs/2510.03988. Cited by: Ā§B.2, Ā§1, Ā§1, 2nd item, Ā§2.2, Ā§4.1, Table 1, Table 1, Table 1, Table 1, Table 3, Table 3, Table 3, Table 3, Ā§5. Z. Kang, X. Zhao, and D. Song (2025) Scalable best-of-n selection for large language models via self-certainty. CoRR abs/2502.18581. Cited by: Ā§5. Z. Kou, J. Chen, X. Cai, X. Xia, M. Xie, D. Wu, B. Liu, Y. Jia, X. Geng, M. Sugiyama, and T. Chua (2026) Positive-unlabeled reinforcement learning distillation for on-premise small models. CoRR abs/2601.20687. Cited by: Ā§5. Z. Lei, Z. Tan, S. Wang, Y. Zhu, Z. Chen, Y. Dong, and J. Li (2025) Learning from diverse reasoning paths with routing and collaboration. In Conference on Empirical Methods in Natural Language Processing, p. 2832ā2845. Cited by: Ā§5. H. Li, K. Yang, Y. Chu, H. Liu, and J. Tang (2025a) Exploring solution divergence and its effect on large language model problem solving. CoRR abs/2509.22480. Cited by: Ā§1, Ā§5. Y. Li, Y. Emad, K. Padthe, J. Lanchantin, W. Yuan, T. Nguyen, J. Weston, S. Li, D. Wang, I. Kulikov, and X. Li (2025b) NaturalThoughts: selecting and distilling reasoning traces for general reasoning tasks. CoRR abs/2507.01921. Cited by: Ā§5. H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe (2023) Letās verify step by step. In The Twelfth International Conference on Learning Representations, Cited by: Ā§4. K. Liu, S. Yan, R. Miao, B. Wang, C. Shen, J. Zhang, and J. Ye (2026) Where did this sentence come from? tracing provenance in LLM reasoning distillation. In International Conference on Learning Representations, Cited by: Ā§5. K. Lu and T. M. Lab (2025) On-policy distillation. Thinking Machines Lab: Connectionism. Cited by: Limitations. Y. Luo, Y. Song, X. Zhang, J. Liu, W. Wang, G. Chen, W. Su, and B. Zheng (2025) Deconstructing long chain-of-thought: A structured reasoning optimization framework for long cot distillation. CoRR abs/2503.16385. Cited by: Ā§5. N. Muennighoff, A. M. Rush, B. Barak, T. L. Scao, N. Tazi, A. Piktus, S. Pyysalo, T. Wolf, and C. A. Raffel (2023) Scaling data-constrained language models. In Annual Conference on Neural Information Processing Systems, Cited by: 1st item. N. Muennighoff, Z. Yang, W. Shi, X. L. Li, L. Fei-Fei, H. Hajishirzi, L. Zettlemoyer, P. Liang, E. J. CandĆØs, and T. Hashimoto (2025) S1: simple test-time scaling. CoRR abs/2501.19393. Cited by: Ā§1, Ā§1, Ā§5. D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman (2024) Gpqa: a graduate-level google-proof q&a benchmark. In Conference on Language Modeling, Cited by: Ā§4. Y. Sui, Y. Chuang, G. Wang, J. Zhang, T. Zhang, J. Yuan, H. Liu, A. Wen, S. Zhong, N. Zou, H. Chen, and X. Hu (2025) Stop overthinking: A survey on efficient reasoning for large language models. Transactions on Machine Learning Research 2025. Cited by: Ā§1. Q. Team (2025) QwQ-32b: embracing the power of reinforcement learning. External Links: Link Cited by: 1st item, Ā§1, Ā§2.2. Y. Tian, S. Chen, Z. Xu, Y. Wang, J. Bi, P. Han, and W. Wang (2025) Reinforcement mid-training. CoRR abs/2509.24375. Cited by: Ā§5. C. Udomcharoenchaikit, W. Ponwitayarat, P. Payoungkhamdee, K. Masuk, W. Buaphet, E. Chuangsuwanich, and S. Nutanong (2022) Mitigating spurious correlation in natural language understanding with counterfactual inference. In Conference on Empirical Methods in Natural Language Processing, p. 11308ā11321. Cited by: Ā§1, Ā§3.2. S. Wang, L. Yu, C. Gao, C. Zheng, S. Liu, R. Lu, K. Dang, X. Chen, J. Yang, Z. Zhang, Y. Liu, A. Yang, A. Zhao, Y. Yue, S. Song, B. Yu, G. Huang, and J. Lin (2025) Beyond the 80/20 rule: high-entropy minority tokens drive effective reinforcement learning for LLM reasoning. CoRR abs/2506.01939. Cited by: Ā§1, 3rd item, Ā§2.4, Ā§5. J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou (2022) Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, Cited by: Ā§5. X. Wu, X. Jiang, H. Li, J. Zhai, D. Liu, Q. Hao, H. Liu, Z. Yang, J. Xie, N. Gu, J. Yang, K. Zhang, Y. Bao, and J. Wang (2025) Beyond scaling law: A data-efficient distillation framework for reasoning. CoRR abs/2508.09883. Cited by: Ā§1, Ā§5. S. Yan, K. Liu, C. Shen, B. Wang, S. Fan, J. Zhang, Y. Wu, Z. Wang, and J. Ye (2026) Distribution-aligned sequence distillation for superior long-cot reasoning. CoRR abs/2601.09088. Cited by: Ā§5. A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. (2025a) Qwen3 technical report. CoRR abs/2505.09388. Cited by: 2nd item, Ā§1, Ā§2.2, Ā§5, Limitations. Z. Yang, X. Luo, Z. Wang, D. Han, Z. He, D. Li, and Y. Xu (2025b) Do not let low-probability tokens over-dominate in RL for llms. CoRR abs/2505.12929. Cited by: Ā§5. Y. Ye, Z. Huang, Y. Xiao, E. Chern, S. Xia, and P. Liu (2025) LIMO: less is more for reasoning. CoRR abs/2502.03387. Cited by: 1st item, Ā§1, Ā§1, Ā§2.2, Table 1, Ā§5. J. O. Yin and A. M. Rush (2025) Compute-constrained data selection. In International Conference on Learning Representations, Cited by: 1st item. W. Yuan, J. Yu, S. Jiang, K. Padthe, Y. Li, D. Wang, I. Kulikov, K. Cho, Y. Tian, J. E. Weston, and X. Li (2025) NaturalReasoning: reasoning in the wild with 2.8m challenging questions. CoRR abs/2502.13124. Cited by: Ā§1, Ā§5. X. Yuan, C. Shen, S. Yan, K. Liu, X. Zhang, S. Fan, L. Xie, W. Wang, R. Guan, Y. Wang, and ieping Ye (2026) Differential fine-tuning large language models towards better diverse reasoning abilities. In International Conference on Learning Representations, Cited by: Ā§1. X. Yuan, C. Shen, S. Yan, X. Zhang, L. Xie, W. Wang, R. Guan, Y. Wang, and J. Ye (2024) Instance-adaptive zero-shot chain-of-thought prompting. In Advances in Neural Information Processing Systems, Cited by: Ā§5. D. Zhang, Q. Dai, and H. Peng (2025) The best instruction-tuning data are those that fit. CoRR abs/2502.04194. Cited by: Ā§B.2, Ā§1, 1st item, Ā§2.2, Table 1, Table 1, Table 1, Table 1, Table 3, Table 3, Table 3, Table 3, Ā§5. H. Zhao, H. Wang, Y. Peng, S. Zhao, X. Tian, S. Chen, Y. Ji, and X. Li (2025) 1.4 million open-source distilled reasoning dataset to empower large language model training. CoRR abs/2503.19633. Cited by: Ā§1, Ā§1, Ā§5. C. Zheng, J. Zhu, Z. Ou, Y. Chen, K. Zhang, R. Shan, Z. Zheng, M. Yang, J. Lin, Y. Yu, and W. Zhang (2025) A survey of process reward models: from outcome signals to process supervisions for large language models. CoRR abs/2510.08049. Cited by: Ā§1. Y. Zhu, Q. Sheng, J. Cao, S. Li, D. Wang, and F. Zhuang (2022) Generalizing to the future: mitigating entity bias in fake news detection. In International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 2120ā2125. Cited by: Ā§3.2. Appendix A Bias Towards Response Length In our preliminary experiments, we also observe that, under the same setup as in Fig. 1, the average total response length (i.e., the total token number in the full response) of data selected by naturalness-based methods is approximately 9.8K, compared to about 15.4K for unselected data, revealing a significant discrepancy. Therefore, in the following sections, we aim to address the following question through experiments: are samples with shorter overall response lengths more likely to be selected by naturalness-based data selection methods? A.1 Longer Response, Higher Log Probability First, we maintain the same experimental setup as in Sec. 2.2, using Qwen3-4B-Base as the target model to compute the log probabilities for data generated by four different LLMs. Fig. 5 illustrates the trend of globally averaged log probabilities (i.e., silogps_i^logp) w.r.t overall response length. The results show that, as the response length increases, the log probabilities actually rise, contrary to the earlier conclusion, where shorter responses were more likely to be selected. Comparing different models, we find that the average log probabilities of DS-Qwen-32B, which exhibits longer step lengths, are consistently and substantially higher than those of other models; even the highest log probability among other models is lower than the lowest value from DS-Qwen-32B. In summary, although log probabilities should in principle increase with response length, the selection bias caused by step length confounding has a much stronger influence than the effect of overall response length, leading to the seemingly opposite conclusion above. Figure 5: Relationship between response-level log probability and total response length. A.2 Why Response Length Bias? This section aims to further investigate why longer responses tend to have higher log probabilities. Fig. 6 presents the log probabilities of tokens at different positions within the responses. Specifically, we first determine the 95th percentile of the maximum response length, and then divide the range from 0 to this value into 20 bins. Tokens are assigned to these bins according to their position within the response, and the average log probability is computed for the tokens in each bin. As shown in Fig. 6, there is a clear trend: tokens located toward the end of a response have higher log probabilities than those at the beginning, following a monotonically increasing pattern. This is because, as the response length grows, the target LLM often becomes more confident in generating the continuation, indicating better adaptation to the data. Consequently, longer responses exhibit higher log probabilities for their tail-end tokens, which in turn results in a higher overall log probability. Figure 6: Average log probability of tokens at different positions for four source LLMs. A.3 Step Length Significantly Matters Response Length for Data Selection In Sec. A.1, we present the experimental finding that the bias induced by total response length is negligible compared to the step length confounding issue. To further validate this conclusion, we employ the causal regression approach proposed in Sec. 3.2, rewriting Eq. (5) as follows: silogp=Ī²1āsifirst+Ī²2āsidrop+Ī³āi+Ī³2ā|i|+Ļµ.s_i^logp= _1s_i^first+ _2s_i^drop+ _i+ _2|o_i|+Īµ. Using the same experimental settings, we refit the model, and the final parameter estimation results are reported in Table 6. Compared with the confounder Ī³āi _i introduced by step length, the confounder Ī³2ā|i| _2|o_i| induced by total length is smaller by approximately two orders of magnitude.222iZ_i is typically on the order of 10ā110^-1, whereas |i||o_i| is typically on the order of 10510^5. In Table 7, we further compare the performance metrics of models trained on data selected with and without the Ī³2ā|i| _2|o_i| term in the criterion, i.e., sicasl=silogpāĪ³āiāĪ³2ā|i|s_i^casl=s_i^logp- _i- _2|o_i|. The results show that removing the Ī³2ā|i| _2|o_i| term causes little change in model performance, once again confirming that the influence of total length is generally small and can even be negligible. In fact, prior studies have provided evidence that longer reasoning SFT data (Chen et al., 2025b; Guha et al., 2025) or in-context CoT prompts (Jin et al., 2024) can be more effective for improving model performance. This suggests that retaining the bias associated with total response length might even be beneficial. However, the step length confounding phenomenon leads to the opposite outcome, preferring shorter responses, which contradicts these findings and further underscores the importance of mitigating this bias. Table 6: Linear regression parameters including overall response length Ī³2ā|i| _2|o_i|. Source Ī²1 _1 Ī²2 _2 Ī³ Ī³2 _2 ĻµĪµ QwQ-32B .028 .972 lightgrayv -0.062 lightgrayv 1E-8 .004 Qwen3-32B .018 .979 lightgrayv -0.043 lightgrayv 3E-9 .007 DS-Qwen-32B .014 .983 lightgrayv -0.158 lightgrayv 4E-8 -.008 GPT-OSS-120B .018 .987 lightgrayv -1.166 lightgrayv 5E-9 .025 Table 7: Comparison of experimental results with and without removing overall response length bias Ī³2ā|i| _2|o_i|. Teacher AIME24 AIME25 MATH OlymB. Qwen3-4B-Base lightgrayv Aslec-casl 31.66 30.83 80.00 42.81 + Ī³2ā|i| _2|o_i| 30.83 30.83 78.60 42.20 Qwen3-8B-Base lightgrayv Aslec-casl 45.00 37.50 85.40 49.03 + Ī³2ā|i| _2|o_i| 43.33 35.83 83.80 48.52 Appendix B More Experimental Settings In this section, we provide a detailed description of our experimental settings, including LLM model cards, data processing pipelines, and SFT details. B.1 LLM Model Cards In our experiments, we employ two categories of LLMs: those used to generate SFT data, which we refer to as source LLMs, and those trained on the generated SFT data, which we refer to as target LLMs. Their details are described as follows. Source LLMs. We use four different families of LLMs, each producing five distinct responses for every question. ā¢ QwQ-32B333https://huggingface.co/Qwen/QwQ-32B (Team, 2025) is a specialized reasoning model trained with reinforcement learning on top of Qwen2.5ā32B. ā¢ Qwen3-32B444https://huggingface.co/Qwen/Qwen3-32B (Yang et al., 2025a) has undergone large-scale long-CoT cold-start training and reasoning-focused reinforcement learning. In its technical report, this model is used to distill smaller-scale models, e.g., the Qwen3-4B and Qwen3-8B variants, which align with our experimental setup. ā¢ DeepSeek-R1-Distill-Qwen-32B555https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B (Guo et al., 2025) is one of the first to employ reinforcement learning to enhance long CoT reasoning in LLMs, providing evidence that models obtained through distillation can still exhibit robust reasoning abilities. ā¢ gpt-oss-120b666https://huggingface.co/openai/gpt-oss-120b (Agarwal et al., 2025) improves inference speed by combining compact attention layers with linear attention layers, while activating only 5B parameters. The model is also trained using the conventional paradigm of SFT followed by reinforcement learning. Target LLMs. Using the generated SFT data, we train four LLMs of varying sizes and types. ā¢ Qwen3-4B-Base777https://huggingface.co/Qwen/Qwen3-4B-Base and Qwen3-8B-Base888https://huggingface.co/Qwen/Qwen3-8B-Base are two different-sized models that have undergone large-scale pre-training only. ā¢ Qwen3-4B-Instruct999https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507 and Qwen2.5-7B-Instruct101010https://huggingface.co/Qwen/Qwen2.5-7B-Instruct build upon the two base versions described above, undergoing thorough instruction fine-tuning. For the 4B model, we adopt its 2507 variant, updating the non-thinking mode from Qwen3-4Bās mixed-reasoning framework. Furthermore, as the Qwen3 series lacks an instruct model of 8B parameters, we use the 7B instruct model from the Qwen2.5 series as a substitute. B.2 Data Sampling and Filtering Our experiments are conducted using datasets from two different sources. ā¢ LIMO-v2111111https://huggingface.co/datasets/GAIR/LIMO-v2 (Ye et al., 2025) undergoes rigorous quality filtering, resulting in a final selection of 800 high-quality mathematics problems. For this dataset, we generate five diverse correct responses for each problem using every source LLM. ā¢ AceReason-1.1-SFT121212https://huggingface.co/datasets/nvidia/AceReason-1.1-SFT (Chen et al., 2025b) aggregates large-scale, high-quality SFT data from multiple sources. From this dataset, we randomly sample 10k mathematics problems, and for each problem, we obtain one correct response generated by each of the four aforementioned source LLMs. For these two datasets, we adopt the following data sampling and quality filtering pipeline. Data sampling. During data sampling, we employ topāp sampling with p set to 0.95. For gpt-oss-120b, following the official recommendations, we set the sampling temperature to 1.0 and the reasoning effort to high. For the other source LLMs, the sampling temperature is set to 0.6. All data sampling is conducted via SGLang for offline LLM deployment and calling, with the maximum generation length consistently fixed at 64K tokens. Data filtering. We perform dynamic filtering during the sampling process. Specifically, for each problem, we sample one response at a time and verify it using the MathāVerify toolkit.131313https://github.com/huggingface/Math-Verify Sampling continues until the required number of correct responses is obtained (5 responses for LIMOāv2 and 1 response for AceReason-1.1-SFT), or until the number of attempts exceeds 15. In practice, some problems are difficult to collect five correct responses for, so we repeat the above procedure five times to ensure that every problem has sufficient correct responses. For problems that are excessively difficult and fail to meet the required number of correct responses, we adopt the following remedies. In LIMOāv2, we manually sample several responses that are close to the correct answer and supplement them until five correct responses are obtained. In AceReason-1.1-SFT, we sample additional problems from the dataset and continue generation until we reach a total of 10k problems, each paired with its corresponding correct response. As a result, each problem in LIMOāv2 may contain up to 75 incorrect responses and at least 25 correct ones. All generated response data are publicly available via the link provided in this paper. Figure 7: Step length distributions for selected and unselected data and relationship between step-level log probability and step length on AceReasonā1.1āSFT. Sampling log probabilities of target LLMs. We employ SGLang to sample log probabilities from the target LLM for the data. In GRACE (Zhang et al., 2025), the output log probabilities are averaged directly. In Local LP (Just et al., 2025), following the best practices outlined in the original paper, each problem and step is paired with its preceding k=4k=4 steps, the average log probability is computed for each step, and the stepālevel averages are then averaged. For Min Entropy, retrieving vocabularyālevel probability distributions for every token is computationally and storageāintensive. Given their typical longātailed nature, we instead retain only the top 20 most probable tokens per token, compute the entropy for each, and average these values across all tokens in a response. B.3 Fine-tuning Details Using the reasoning SFT data, we fineātune the target LLM with fullāparameter training via LlamaFactory. We set the training batch size to 32 and enable LlamaFactoryās builtāin packing option to concatenate shorter samples, with the maximum sequence length fixed at 32K. Optimization is carried out using the Adam optimizer with a learning rate of 5Ć10ā55Ć 10^-5 for a total of 6 epochs. The learning rate scheduler is configured as cosine_with_min_lr, with a minimum learning rate of 1Ć10ā51Ć 10^-5. Appendix C More Analysis Results In this section, we provide additional analytical results regarding step length confounding. C.1 Analysis on More Datasets and Models In Sec. 2, we analyze the LIMOāv2 dataset using Qwen3ā4BāBase as the target LLM, examining results across four different source LLMs. In this section, we extend our analysis to the AceReasonā1.1āSFT dataset and combine data from all source LLMs, evaluating their performance on two target models: Qwen3ā4BāInstruct and Qwen3ā8BāBase. The analysis results are shown in Fig. 7. The experimental results show that, for both target LLMs, the AceReasonā1.1āSFT dataset exhibits a pronounced difference in stepālength distribution between the selected and unselected samples. Moreover, the monotonic increasing relationship between each stepās log probability and its length remains significant, with the probabilities from Qwen3ā8BāBase consistently exceeding those of Qwen3ā4BāInstruct. Figure 8: Data selection bias and step length distributions under different splitting methods. C.2 Different Step Splitting Methods The step-length distribution differences shown in Fig. 1 are obtained by splitting the responses into steps using periods and spaces. In the community, aside from this splitting approach, some studies adopt or external tools such as NLTK for step splitting. Therefore, we also investigate the step lengthādistribution differences under these two additional step splitting methods. The analysis results are presented in Fig. 8. Our experimental results consistently show that, regardless of the step splitting method used, the selected versus unselected data still display a clear difference in stepālength distribution. This indicates that the naturalnessābased data selection approach continues to suffer from a step length confounding issue. Moreover, splitting sentences using periods produces the most distinct distribution differences compared to other splitting methods. This leads to another key observation: low-probability tokens are most prominent at sentence beginnings when periodābased splitting is applied, as opposed to the first tokens in steps segmented by other methods. C.3 Casual Regression Parameters We apply the causal regression method introduced in Sec. 3.2 to the AceReasonā1.1āSFT dataset, reāfitting the two target LLMs analyzed previously, and present the regression parameters in Table 8. The results show that the Ī³ values remain high, indicating that the step length confounding issue persists. Furthermore, the result Ī²1āŖĪ²2 _1 _2 is fully consistent with the conclusions presented in Sec. 4.4. Table 8: Linear regression parameters of all SFT data on AceReasonā1.1āSFT for the two target models. Target LLM Ī²1 _1 Ī²2 _2 Ī³ ĻµĪµ Qwen3-4B-Instruct 0.075 0.952 -0.778 0.022 Qwen3-8B-Base 0.027 0.993 -0.660 0.025 Figure 9: Convergence analysis. Appendix D More Experimental Results D.1 Convergence Analysis In Fig. 9, we present a convergence analysis showing that the GRACE method, i.e., the existing naturalnessābased approach, consistently converges to a higher loss compared with our method. This also demonstrates that our debiasing approach is able to select data with greater naturalness. D.2 Comparing More Baselines We implement a simple heuristic-based selection method on the same source data as a representative baseline. The results of this comparison are presented in Table 9. On the LIMO-v2 dataset, we compare several baseline selection strategies: ā¢ Uniform: randomly samples 4k reasoning trajectories from the four source LLMs with a uniform distribution to ensure data diversity; ā¢ High/Low Difficulty: selects 4k of the longest or shortest reasoning trajectories, respectively, using response length as a proxy for problem difficulty. The results consistently demonstrate the superior effectiveness of our selected data. Moreover, while more difficult (i.e., longer) examples do yield better performance for SFT of reasoning models, confirming that challenging instances are generally more informative, they still contain a higher proportion of redundant or noisy reasoning steps compared to our method. This underscores the advantage of our approach in not only capturing useful difficulty but also filtering out superfluous or low-quality reasoning content. Table 9: Results of more data selection methods. Qwen3-4B-Base AIME25 MATH500 + Uniform 24.16 73.20 + High Difficulty 26.66 77.80 + Low Difficulty 20.83 71.20 lightgrayv + Aslec-casl 30.83 80.00 D.3 Comparing More Source LLMs To further assess generalizability, we also train a non-Qwen model: Llama3-3B, on the data selected by our method Aslec-casl; the results are presented in Table 10. The results consistently demonstrate the effectiveness of our proposed method. Table 10: Results on Llama3-3B. Llama3-3B AIME25 MATH500 + GRACE 26.66 76.20 lightgrayv + Aslec-casl 33.33 84.40