Paper deep dive
Adaptive Power Sampling for LLM Reasoning
Bingnan Xiao, Chenhao Yang, Bingcong Li, Wei Ni, Xin Wang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Sequence-level power sampling has recently emerged as a training-free approach to reasoning by sampling from a sharpened output distribution of a base large language model (LLM). Nevertheless, existing methods typically sharpen the base model distribution uniformly across queries, overlooking variations in query difficulty and in how well the base model already handles each query. The goal of this work is to equip power sampling with query adaptivity. Theoretically, we show that the benefits of further sharpening are determined by the self-reward gap between correct and incorrect responses. Based on this insight, we propose \emph{Adaptive Power Sampling} (APS), which adjusts the sharpening exponent on a per-query basis at test time using the relationship between answer agreement and the model's self-reward. Experiments across diverse reasoning tasks, including MATH500, HumanEval, and GPQA, show that APS consistently outperforms power sampling with a fixed sharpening exponent, without additional training.
Tags
Links
- Source: https://arxiv.org/abs/2610.08563v1
- Canonical: https://arxiv.org/abs/2610.08563v1
Trouble viewing inline? Open PDF directly →
Full Text
77,510 characters extracted from source content.
Expand or collapse full text
Adaptive Power Sampling for LLM Reasoning Bingnan Xiao Fudan University 22110720061@m.fudan.edu.cn Chenhao Yang11footnotemark: 1 University of Glasgow 3165453Y@student.gla.ac.uk Bingcong Li ETH Zurich bingtsongli@gmail.com Wei Ni Edith Cowan University wei.ni@ieee.org Xin Wang22footnotemark: 2 Fudan University xwang11@fudan.edu.cn †thanks: These authors contributed equally to this work.†thanks: Corresponding authors: Bingcong Li (bingtsongli@gmail.com) and Xin Wang (xwang11@fudan.edu.cn). Abstract Sequence-level power sampling has recently emerged as a training-free approach to reasoning by sampling from a sharpened output distribution of a base large language model (LLM). Nevertheless, existing methods typically sharpen the base model distribution uniformly across queries, overlooking variations in query difficulty and in how well the base model already handles each query. The goal of this work is to equip power sampling with query adaptivity. Theoretically, we show that the benefits of further sharpening are determined by the self-reward gap between correct and incorrect responses. Based on this insight, we propose Adaptive Power Sampling (APS), which adjusts the sharpening exponent on a per-query basis at test time using the relationship between answer agreement and the model’s self-reward. Experiments across diverse reasoning tasks, including MATH500, HumanEval, and GPQA, show that APS consistently outperforms power sampling with a fixed sharpening exponent, without additional training. 1 Introduction Reasoning is fundamentally about generating and identifying high-quality solution trajectories from a model’s output distribution. Recent work suggests that post-training often achieves this by redistributing probability mass toward sequences that are more likely to yield correct answers (Huang et al., 2025; Yue et al., 2025; He et al., 2025). This perspective, commonly referred to as sequence-level distribution sharpening, also provides a foundation for test-time reasoning (Karan and Du, 2026). In particular, strong reasoning performance on tasks such as math and coding can be achieved by sampling from a sharpened version of the base model’s sequence-level distribution, even when the base model has not undergone post-training (Karan and Du, 2026; Ji et al., 2026). Compared with post-training, a clear advantage of test-time reasoning is its reduced reliance on post-training data. This benefit, however, comes at the cost of query-adaptive sharpening. Post-training can implicitly learn, for each query or prompt, an appropriate degree of sharpening, concentrating sufficient probability mass on high-quality responses while preserving enough diversity for the target task. In contrast, test-time reasoning methods typically commit to a single predetermined degree of sharpening, “betting on” how concentrated the desired distribution should be (Karan and Du, 2026; Azizi et al., 2026; Ji et al., 2026; Nguyen et al., 2026; Meng et al., 2026). Such a fixed choice, however, may not be sufficient. Intuitively, the appropriate degree of sharpening should vary across problems and domains, e.g., between math and coding, or also within the same domain, between, e.g., primary-school and college-level math problems. Motivated by this intuition, which we further validate experimentally, we study whether the sharpening strength can instead be determined query-wise at test time, without relying on post-training data or manual tuning. To equip power sharpening with adaptivity, we first study how reasoning performance varies with the sharpening strength. More precisely, let x denote a prompt, and let pθ(y∣x)p_θ(y x) denote the probability that a base model θ generates a complete response y given x. Test-time sharpening (Karan and Du, 2026) samples responses from the sharpened distribution πα(y∣x)∝pθ(y∣x)α _α(y x) p_θ(y x)^α, where α≥1α≥ 1 controls the degree of distributional sharpening. We show that for this particular query x, the preferred sharpening direction is jointly determined by the relationship between response correctness and sequence log-likelihood. Stronger sharpening, i.e., larger α, is more beneficial when correct responses tend to have higher sequence log-likelihood than incorrect ones, and harmful otherwise. Translating these theoretical insights into practice, we propose Adaptive Power Sampling (APS), a training-free reasoning method that dynamically adjusts the sharpening exponent α at test time. Since ground-truth correctness is unavailable, APS leverages self-consistency (Wang et al., 2023; Aggarwal et al., 2023) among sampled answers as a surrogate signal. Combined with the theoretical characterization above, APS adjusts α on the fly for each query. Such adaptivity improves reasoning performance across several tasks. In a nutshell, our main contributions are summarized as follows: • Motivated by our empirical observation that the preferred sharpening strength varies across queries, we develop a theoretical characterization of how reasoning performance changes with the sharpening strength. For each query, whether a stronger sharpening is preferred is determined by the self-reward gap between correct and incorrect responses. • Building on these insights, we propose Adaptive Power Sampling (APS), which adjusts the sharpening exponent α for each query at test time using statistics from sampled responses. APS achieves query-wise adaptivity analogous to that of RL-based post-training, while requiring no additional training or post-training data. • We evaluate APS on multiple models and reasoning benchmarks, showing consistent improvements over fixed-exponent power sampling, and competitive performance against RL-post-trained methods. 2 Related Work Reasoning via post-training. Reinforcement learning (RL) has been shown to improve the reasoning performance of LLMs on mathematical and coding tasks (Ouyang et al., 2022; Shao et al., 2024; DeepSeek-AI, 2025; Yu et al., 2025; Liu et al., 2025; Lin et al., 2025). Recent work (Huang et al., 2025; Yue et al., 2025; He et al., 2025; Shao et al., 2025; Song et al., 2025) suggests that RL post-training can reinforce existing high-probability responses in the model, highlighting distribution reshaping as an important mechanism behind reasoning performance improvement. The view motivates investigating whether similar gains can be achieved by directly sharpening the model distribution and adapting the sharpening strength at inference time. Inference-time scaling improves LLM performance by allocating additional computation at test time (Zhang et al., 2025; Hariri et al., 2026). Representative approaches include repeated sampling and answer aggregation or selection (Wang et al., 2023; Li et al., 2022; Brown et al., 2024), verifier- or reward-guided selection of candidate solutions (Cobbe et al., 2021; Lightman et al., 2024; Chow et al., 2025), and explicit search over intermediate reasoning states (Xie et al., 2023; Yao et al., 2023; Zhou et al., 2024; Snell et al., 2024). While effective, these methods motivate more direct control of the model’s response distribution at inference time. Reasoning at test time. Power sampling directly sharpens the model’s sequence-level distribution by assigning greater weight to high-probability responses (Karan and Du, 2026). It provides a training-free alternative that can achieve reasoning performance comparable to RL post-training while better preserving generation diversity. Recent studies have further improved the efficiency of sequence-level power sampling via, e.g., a future-aware autoregressive approximation (Ji et al., 2026), particle and sequential Monte Carlo methods (Azizi et al., 2026; Nguyen et al., 2026; Wang et al., 2026) and entropy-guided proposals (Guo et al., 2026; Zhou et al., 2026). While these methods improve sampling efficiency, they typically rely on a predetermined sharpening strength across queries. In contrast, our work studies how the sharpening strength should be adaptive to individual queries. Power sampling can also be understood from a theoretical perspective. (Tomihari and Sato, 2026) connects power sampling with self-reward KL-regularized RL and self-distillation. In parallel, our work studies how reasoning performance changes with the sharpening strength, thereby motivating the adaptive selection of sharpening strength at test time. 3 Preliminaries 3.1 Sequence-Level Power Distribution Given prompt x, let pθ(y|x)p_θ(y|x) denote the probability of a base model θ generating a complete response y=(y1,…,yT)y=(y^1,…,y^T). In this work, pθ(y|x)p_θ(y|x) is referred to as the sequence-level likelihood, and by the autoregressive factorization, it can be equivalently obtained via pθ(y|x)=∏t=1Tpθ(yt|x,y<t)p_θ(y|x)= _t=1^Tp_θ(y^t|x,y^<t). Power sampling, originally proposed in (Karan and Du, 2026), leverages a sharpened sequence-level distribution to re-weight the base model’s probability by a sharpening exponent α. For α≥1α≥ 1, the sequence-level power distribution is defined as πα(y|x)=pθ(y|x)αZα(x), _α(y|x)= p_θ(y|x)^αZ_α(x), (1) where Zα(x)=∑ypθ(y|x)αZ_α(x)= _yp_θ(y|x)^α is the normalization constant. The exponent α=1α=1 recovers the base model distribution, while a larger α concentrates probability mass on higher-likelihood complete responses. To illustrate the sharpening effect, consider two complete responses y and y′y with pθ(y∣x)>pθ(y′∣x)p_θ(y x)>p_θ(y x). For α>1α>1, their relative probabilities satisfy πα(y∣x)πα(y′∣x)=(pθ(y∣x)pθ(y′∣x))α>pθ(y∣x)pθ(y′∣x). _α(y x) _α(y x)= ( p_θ(y x)p_θ(y x) )^α> p_θ(y x)p_θ(y x). (2) Thus, increasing α further amplifies the relative advantage of responses with higher base-model likelihood, making the sequence-level distribution sharper. Note that sequence-level sharpening is fundamentally different from token-level low-temperature scaling in autoregressive decoding (Wang et al., 2020): the former reweights complete sequences according to their sequence likelihoods, whereas the latter rescales the next-token distribution locally at each decoding step. This has been demonstrated multiple times, e.g., (Karan and Du, 2026; Azizi et al., 2026; Ji et al., 2026; Nguyen et al., 2026; Meng et al., 2026). 3.2 One Fixed Exponent Does Not Fit Every Query Existing power sampling algorithms (Karan and Du, 2026; Azizi et al., 2026; Ji et al., 2026; Nguyen et al., 2026; Meng et al., 2026) rely on a fixed sharpening exponent α. While it is natural to expect different tasks, such as coding and mathematics, to favor different values of α, in this subsection we show that even within the same dataset, no single α performs equally well across all queries. To illustrate this, we evaluate Qwen2.5-Math-7B (Yang et al., 2024b) on 20 randomly sampled queries from the MATH500 dataset (Lightman et al., 2024). We consider power-sampling exponents α∈1,2,4,8,16α∈\1,2,4,8,16\. For each query and each value of α, we generate N=16N=16 responses per random seed using the MCMC sampler of Karan and Du (2026). We use P(α∣x)P(α x) to denote the fraction of correct responses for query x at exponent α. Figure 1: Motivating example on MATH500 with Qwen2.5-Math-7B. The left panel shows representative Pass@1 accuracy curves under power sampling over α∈1,2,4,8,16α∈\1,2,4,8,16\. The right panel shows the mean oracle gap P⋆−P(α)P -P(α). The results are shown in Figure 1. In the left panel, 5 representative queries are plotted to visualize query-wise optimal α. In particular, Query A attains its highest pass@1 accuracy only at α=16α=16 while Query B already reaches perfect accuracy at α=2α=2. In contrast, Queries C–E achieve their best performance at an intermediate exponent. In the right panel, we measure the performance loss from using a common exponent instead of the best exponent for each of the 20 sampled queries. For each query x, let P∗(x)=maxaP(a∣x)P^*(x)= _aP(a x) denote its best accuracy over the tested exponents, and define the oracle gap as P∗(x)−P(α∣x)P^*(x)-P(α x). The gap is smallest at α=2α=2, with an average value of 0.130.13, whereas the fixed choice α=4α=4 used in (Karan and Du, 2026) gives a larger gap of 0.170.17. These results jointly show that a single fixed α can leave a noticeable performance gap relative to the best exponent for individual queries, motivating an adaptive choice of α at test time. 4 Effect of the Sharpening Strength In this section, we theoretically study how α affects reasoning performance under the sequence-level power distribution. We characterize its effect on test-time correctness and identify when sharpening is beneficial or harmful. Following the same analytical framework, we also obtain an extension to group-relative RL as a byproduct, where adaptive sharpening can likewise be beneficial. Throughout this section, we consider a fixed query x and omit the dependence on x for notational simplicity. 4.1 Definitions To examine how the sharpening exponent α affects the reasoning performance, we restrict our discussion to tasks with binary correctness outcomes, such as mathematics and coding. For a given prompt x, let r⋆(y)∈0,1r (y)∈\0,1\ denote the binary task reward of a complete response y, where r⋆(y)=1r (y)=1 indicates a correct response and r⋆(y)=0r (y)=0 indicates an incorrect response. Accordingly, we partition the response space into correct and incorrect sets, i.e., C=y:r⋆(y)=1,I=y:r⋆(y)=0.Y_C=\y:r (y)=1\, _I=\y:r (y)=0\. (3) The expected performance under exact sampling from πα _α is defined as P(α)=πα[r⋆(Y)].P(α)=E_ _α[r (Y)]. (4) Since r⋆r is binary, P(α)P(α) is also the probability that a response sampled from πα _α is correct. Hence, a preferred sharpening exponent enlarges P(α)P(α). Another useful quantity is the model’s self-reward, which reflects the sequence-level likelihood assigned to a response y rself(y)=logpθ(y|x).r_self(y)= p_θ(y|x). (5) Under πα _α, the conditional self-reward means and variances with respect to α is defined as μC(α) _C(α) =πα[rself(Y)∣r⋆(Y)=1], =E_ _α[r_self(Y) r (Y)=1], μI(α) _I(α) =πα[rself(Y)∣r⋆(Y)=0], =E_ _α[r_self(Y) r (Y)=0], (6) σC2(α) σ^2_C(α) =Varπα[rself(Y)∣r⋆(Y)=1], =Var_ _α[r_self(Y) r (Y)=1], σI2(α) σ^2_I(α) =Varπα[rself(Y)∣r⋆(Y)=0]. =Var_ _α[r_self(Y) r (Y)=0]. (7) We then define the self-reward gap δ(α)δ(α) to measure whether correct responses have higher model log-likelihood than incorrect responses under the current power distribution δ(α)=μC(α)−μI(α).δ(α)= _C(α)- _I(α). (8) 4.2 Theoretical Basis for Adaptive Sharpening For the test-time objective P(α)P(α), we study how the expected reasoning performance changes as α increases. From (1), increasing α shifts relative probability mass toward responses with higher base-model likelihood. Consequently, the marginal effect of sharpening on P(α)P(α) depends on whether model likelihood is aligned with the task reward. The following theorem makes this relation exact. Theorem 1. For the fixed prompt x and the reward r⋆r independent of α, we have dP(α)dα=Covπα(r⋆(Y),rself(Y)). dP(α)dα=Cov_ _α(r (Y),r_self(Y)). (9) Theorem 1 provides an exact characterization of the marginal effect of sharpening. If task reward and self-reward are positively correlated under the current power distribution, increasing α improves exact expected performance. Conversely, if the covariance is negative, additional sharpening moves probability mass in a direction that hurts the task reward. Interestingly, the covariance in (9) can be further decomposed into a simple self-reward gap, as given below. Corollary 1 (Self-reward gap decomposition). Based on the task reward r⋆∈0,1r ∈\0,1\, we obtain dP(α)dα=P(α)(1−P(α))δ(α). dP(α)dα=P(α)(1-P(α))δ(α). (10) Corollary 1 has a direct interpretation. Here, P(α)(1−P(α))P(α)(1-P(α)) measures how much room is left for the correctness probability to change under the current power distribution: it is largest when correct and incorrect responses are both likely, i.e., P(α)=0.5P(α)=0.5, and it becomes small when the sampler almost always succeeds or almost always fails. Prior work connects power sampling to the correlation between task reward and self-reward (Tomihari and Sato, 2026). Theorem 1 and Corollary 1 further characterize the local change in expected performance caused by increasing α. Since P(α)∈[0,1]P(α)∈[0,1] for binary r⋆r , increasing α locally improves performance if δ(α)>0δ(α)>0 and locally degrades performance if δ(α)<0δ(α)<0. This reveals the source of adaptivity: if δ(α)δ(α) is known, we can determine whether increasing or decreasing α would improve performance for this particular query. Next, we study how δ(α)δ(α) evolves with α to understand when further sharpening may switch from improving performance to degrading it, or vice versa. Lemma 1 (Self-reward gap dynamics). The self-reward gap evolves as dδ(α)dα=σC2(α)−σI2(α). dδ(α)dα=σ^2_C(α)-σ^2_I(α). (11) The self-reward gap increases when correct sequences have higher conditional log-probability variance than incorrect sequences, and decreases under the reverse ordering. Since the sign of dP(α)/dαdP(α)/dα is determined by δ(α)δ(α) for binary r⋆r , a larger conditional variance in the correct class moves the gap toward more positive values and thereby shifts the local effect of sharpening toward improvement. Besides, we can differentiate the first-order identity, yielding the second-order characterization of d2P/dα2d^2P/dα^2 in Appendix B. While extending the above local optimality analysis to a characterization of the globally optimal α∗∈[1,∞)α^*∈[1,∞) is challenging, we can still gain insight into how P(α)P(α) evolves with α. For binary r⋆r , the sign of dP(α)/dαdP(α)/dα is determined by the sign of δ(α)δ(α). Hence, examining δ(α)δ(α) at the two extremes, α=1α=1 and α→∞α→∞, provides useful information about the global behavior of P(α)P(α). The following theorem shows that δ(∞)δ(∞) admits a closed-form expression. Theorem 2 (Limit of the self-reward gap). As α→∞α→∞, the self-reward gap Δs _s satisfies Δs≜limα→∞δ(α)=sC−sI, _s _α→∞δ(α)=s_C-s_I, (12) where sC=maxy∈Crself(y)s_C= _y _Cr_self(y), and sI=maxy∈Irself(y)s_I= _y _Ir_self(y). The signs of δ(1)δ(1) and Δs _s reveal the following four cases. If both are positive, increasing α improves performance near α=1α=1 and again for all sufficiently large α, although δ(α)δ(α) may still change sign at intermediate values, causing dP(α)/dαdP(α)/dα to change sign as well. If δ(1)>0δ(1)>0 but Δs<0 _s<0, increasing α initially improves performance but eventually becomes harmful. If δ(1)<0δ(1)<0 but Δs>0 _s>0, increasing α is initially harmful but eventually becomes beneficial. If both are negative, increasing α is harmful near α=1α=1 and again for sufficiently large α. 4.3 Extensions to Group-Relative RL Going beyond test-time reasoning, another related sampling scenario is group-relative RL, where multiple responses to the same query are scored and compared to construct relative advantages for policy optimization, as in GRPO (Shao et al., 2024). We next show that replacing such standard rollouts with query-adaptive power sampling can also be beneficial. This follows from the same analytical framework with a modified objective, since RL requires informative reward variation among sampled responses (Razin et al., 2025), rather than simply maximizing correctness. Precisely, for a group of K responses independently sampled from πα(y∣x) _α(y x) in group-relative RL, let RiR_i denote the reward of the i-th response and R¯=K−1∑i=1KRi R=K^-1 _i=1^KR_i. For binary correctness rewards, we define CK(α)=[1K∑i=1K(Ri−R¯)2]C_K(α)=E [ 1K _i=1^K(R_i- R)^2 ] denoting the expected within-group reward variation of K sampled responses. Since RiR_i is a Bernoulli random variable with success probability P(α)P(α), CK(α)C_K(α) can be rewritten as CK(α)=K−1KP(α)(1−P(α)).C_K(α)= K-1KP(α) (1-P(α) ). (13) A larger CK(α)C_K(α) means more variation in the rewards within a group, making the relative advantages among sampled responses easier to distinguish and yielding a clearer signal for policy optimization. Differentiating (13) and applying Corollary 1 gives the following theorem. Theorem 3. For the expected within-group reward variation CK(α)C_K(α) defined above, we have dK(α)dα dC_K(α)dα =K−1K(1−2P(α))dP(α)dα=K−1K(1−2P(α))P(α)(1−P(α))δ(α). = K-1K (1-2P(α) ) dP(α)dα= K-1K (1-2P(α) )P(α) (1-P(α) )δ(α). (14) Theorem 3 reveals how adapting the sharpening exponent α affects correctness probability and within-group reward variation jointly. When P(α)<1/2P(α)<1/2, an adaptation of α that increases P(α)P(α) also increases CK(α)C_K(α). In contrast, once P(α)>1/2P(α)>1/2, the same improvement in P(α)P(α) is accompanied by a decrease in CK(α)C_K(α). According to (13), K(α)C_K(α) is maximized at P(α)=1/2P(α)=1/2, which provides a simple criterion for adapting the sharpening strength in group-relative RL. We therefore adapt α to maximize CK(α)C_K(α), or equivalently, to make the correctness probability close to 1/21/2. The detailed procedure and experiments are presented in Appendix C. Lastly, we develop a unified framework that subsumes both the test-time objective P(α)P(α) and the within-group reward variation CK(α)C_K(α); see Appendix B. More broadly, this framework provides a recipe for adapting the sharpening strength in other power sampling-based settings. 5 Adaptive Power Sampling This section translates the theoretical insights into practical algorithms. 5.1 Estimating the Sharpening Direction Theorem 1 and Corollary 1 show that the sharpening exponent α can be adjusted to improve P(α)P(α) according to the alignment between true reward and model’s self-reward. In practice, however, the true task reward is unavailable at test time. To address this challenge, we develop an efficient algorithm, Adaptive Power Sampling (APS), which uses self-consistency among sampled answers as a surrogate signal to adaptively adjust α during inference. APS proceeds iteratively. At iteration t, given the current exponent αt _t, suppose that Nt≥2N_t≥ 2 responses yi,ti=1Nt\y_i,t\_i=1^N_t are available. For simplicity, we omit the iteration index t and write yiy_i. For each response yiy_i, we compute its self-reward as si=rself(yi)=logpθ(yi∣x).s_i=r_self(y_i)= p_θ(y_i x). (15) To reduce variations in the scale of self-reward, we standardize the self-reward of each response as zi=si−s¯tσ^s,t,z_i= s_i- s_t σ_s,t, (16) where s¯t=1Nt∑i=1Ntsi s_t= 1N_t _i=1^N_ts_i and σ^s,t=[1Nt∑i=1Nt(si−s¯t)2]1/2 σ_s,t= [ 1N_t _i=1^N_t(s_i- s_t)^2 ]^1/2. Since the ground-truth correctness of each response is unavailable at test time, we design answer agreement as a surrogate signal. This is inspired by self-consistency methods (Wang et al., 2023; Aggarwal et al., 2023) , where agreement across independently sampled reasoning paths provides a useful indication of answer confidence and is often associated with answer correctness. By denoting eie_i as the normalized answer key of yiy_i, the answer-agreement score of yiy_i is defined as vi=[ei≠invalid]Nt−1∑j≠i[ej=ei],v_i= 1[e_i ]N_t-1 _j≠ i1[e_j=e_i], (17) where [⋅]1[·] denotes the indicator function, which equals one when the condition inside the brackets is satisfied and zero otherwise. For a valid answer, viv_i measures the fraction of other sampled responses giving the same answer as yiy_i. For an invalid answer, vi=0v_i=0. A larger viv_i indicates stronger support from the other samples and is treated as evidence that the answer yiy_i is more likely to be correct. Based on the answer-agreement scores and normalized self-rewards, we therefore define the following score for determining the sharpening direction, i.e., whether α should be increased or decreased: g^t g_t =1Nt∑i=1Nt(vi−v¯t)zi=1Ntσ^s,t∑i=1Nt(vi−v¯t)(si−s¯t). = 1N_t _i=1^N_t(v_i- v_t)z_i= 1N_t σ_s,t _i=1^N_t(v_i- v_t)(s_i- s_t). (18) where v¯t=1Nt∑i=1Ntvi v_t= 1N_t _i=1^N_tv_i. The sign of g^t g_t reflects whether answer agreement and self-reward vary in the same direction with σ^s,t>0 σ_s,t>0. In particular, g^t>0 g_t>0 indicates that responses with stronger agreement tend to have higher self-reward, which is consistent with the favorable case identified in Section 4. We should therefore increase α. The magnitude of g^t g_t further reflects the strength of this relationship and is used to determine the amount by which α is adjusted. If σ^s,t=0 σ_s,t=0, the sampled responses have identical self-reward, and we set g^t=0 g_t=0. When all agreement scores are identical, the samples provide no preferred sharpening direction; APS keeps αt _t unchanged and collects additional samples. 5.2 Adaptive Exponent Update Given the estimate g^t g_t, APS updates the sharpening exponent via gradient ascent strategy: αt+1=clip(αt+ηg^t,αmin,αmax),α_t+1=clip ( _t+η g_t, _ , _ ), (19) where η is the stepsize and clip(⋅)clip(·) ensures that the updated exponent remains within the appropriate range [αmin,αmax][ _ , _ ]. The exponent adaptation is performed under both sampling-quality and budget constraints. At iteration t, we monitor the batch-level MCMC acceptance rate q^t=At/Pt q_t=A_t/P_t, where AtA_t and PtP_t denote the numbers of accepted and total Metropolis–Hastings (MH) proposals used to generate the current batch at αt _t, respectively. We use q^t q_t as a sampler-quality diagnostic (Karan and Du, 2026). If the optional quality threshold qminq_ is enabled and q^t<qmin q_t<q_ , the adaptive search is terminated. Let C denote the total sample budget and c the number of samples generated so far. The adaptive search continues only when the remaining budget is sufficient to generate another batch of K samples. The adaptive search also terminates if either |g^t|≤ϵ| g_t|≤ε, indicating insufficient evidence for changing αt _t, or αt+1=αt _t+1= _t, indicating that the proposed update cannot move the exponent. APS retains the last exponent passing the sampling-quality check as α⋆α , with α⋆=1α =1 as the initial fallback. After adaptation, the remaining budget is spent at α⋆α , and samples collected at this exponent are pooled for majority voting over valid answers, breaking ties by the mean self-reward of supporting responses. Algorithm 1 summarizes the procedure. APS adjusts the sharpening strength for each query through the estimated alignment between answer agreement and self-reward: its sign determines whether to increase α, while its magnitude determines the update size, without requiring ground-truth answers. Algorithm 1 Adaptive Power Sampling (APS) 0: Prompt x, base model pθp_θ, α1 _1, αmin _ , αmax _ , K, C, η, qminq_ , ϵε 1: Initialize c←0c← 0, α⋆←1α ← 1, and ℬα←∅B_α← 2: for t=1,2,…t=1,2,… while c+K≤Cc+K≤ C do 3: Draw K sequence-level MCMC samples from pθ(y|x)αtp_θ(y|x) _t; set c←c+Kc← c+K 4: For each yiy_i, extract eie_i, sis_i, and display answer did_i 5: Estimate sampler quality q^t q_t from the MCMC acceptance rate 6: if qminq_ is enabled and q^t<qmin q_t<q_ then 7: break 8: end if 9: Add the current samples to ℬαtB_ _t, set α⋆←αtα ← _t and Nt←|ℬαt|N_t←|B_ _t|, and compute viv_i via (17) 10: if vi=1v_i=1 for all i=1,…,Nti=1,…,N_t then 11: break 12: else if vi=v¯tv_i= v_t for all i=1,…,Nti=1,…,N_t then 13: Set αt+1←αt _t+1← _t and continue 14: end if 15: Compute ziz_i via (16); if σ^s,t=0 σ_s,t=0, set zi←0z_i← 0 16: Estimate g^t g_t via (18), and set αt+1α_t+1 via (19) 17: if |g^t|≤ϵ| g_t|≤ε or αt+1=αtα_t+1= _t or c+K>Cc+K>C then 18: break 19: end if 20: end for 21: Use the remaining budget to sample at α⋆α and add the samples to ℬα⋆B_α ; select the most frequent valid answer, breaking ties by the highest mean self-reward among its supporting samples 22: return The corresponding display answer, or an empty answer if all samples are invalid 6 Experiments 6.1 Experimental Setup Models. We evaluate APS on four models: Qwen2.5-7B (Yang et al., 2024a), Qwen2.5-Math-7B (Yang et al., 2024b), DeepSeek-Math-7B (Shao et al., 2024), and DeepSeek-Math-7B-RL (Shao et al., 2024). Qwen2.5-7B is a general-purpose model, while Qwen2.5-Math-7B and DeepSeek-Math-7B are math-focused models. Benchmarks and Baselines. We use three reasoning benchmarks. MATH500 (Lightman et al., 2024) evaluates multi-step mathematical reasoning through correctness of the final answer. HumanEval (Chen et al., 2021) contains 164 Python programming tasks evaluated by functional correctness. GPQA Diamond (Rein et al., 2024) contains 198 expert-level multiple-choice questions in physics, chemistry, and biology. For the compared methods, base decoding denotes standard autoregressive sampling from the original model distribution, while low-temperature applies token-level temperature scaling with T=1/αT=1/α (Wang et al., 2020). The sequence-level sampling baselines include several methods that use a fixed sharpening exponent α with different sampling strategies, such as power sampling (Karan and Du, 2026), scalable power sampling (SPS) (Ji et al., 2026), power sampling with sequential Monte Carlo (Power-SMC) (Azizi et al., 2026), and auxiliary particle power sampling (APPS) (Nguyen et al., 2026). We set α=4α=4 for all the fixed-exponent baselines. More implementation details are provided in Appendix D. 6.2 Main Results Pass@1 Performance. Table 1 summarizes the pass@1 accuracy results on MATH500, HumanEval, and GPQA. GRPO (MATH) comes from Shao et al. (2025) for Qwen and DeepSeek-Math-7B-RL for DeepSeek Shao et al. (2024). Compared with base decoding, low-temperature sampling generally yields performance improvements, showing that sharpening the next-token distribution at each decoding step can already improve reasoning performance. The sequence-level sampling baselines with fixed sharpening exponent, including Power Sampling, SPS, Power-SMC, and APPS, generally outperform low-temperature sampling, suggesting that sequence-level distribution sharpening can provide additional gains beyond token-level temperature scaling. More importantly, APS achieves the best performance among all baselines in most cases. These results further validate the query-wise variation observed in Section 3.2, where different queries favor different sharpening strengths. Unlike these fixed-exponent methods, APS adjusts α according to the sampled responses and hence adapts the sharpening strength to the current query. By adapting α online, APS could exploit the reasoning capability already contained in the base model more effectively. We further compare APS with GRPO, which is used to post-train the model on MATH. Although GRPO achieves strong performance on the in-domain MATH500 benchmark, its gains are less consistent on HumanEval and GPQA. In contrast, APS requires no additional training and achieves consistently strong performance across all three benchmarks. These results show that adaptive inference-time sharpening can effectively improve reasoning performance across different tasks without updating the model parameters or relying on an external reward for post-training. Figure 2: Pass@k accuracy of Qwen2.5-Math-7B on MATH500, HumanEval, and GPQA. Table 1: Pass@11 accuracy on MATH500 (M), HumanEval (H), and GPQA (G). Method Qwen2.5-7B Qwen2.5-Math-7B DeepSeek-Math-7B M H G M H G M H G Base 0.4980.498 0.3290.329 0.2780.278 0.4960.496 0.3290.329 0.2780.278 0.3620.362 0.4150.415 0.3330.333 Low-temperature 0.6280.628 0.5240.524 0.3030.303 0.6900.690 0.5120.512 0.3530.353 0.3660.366 0.4270.427 0.4300.430 Power Sampling 0.7060.706 0.6220.622 0.3180.318 0.7480.748 0.5730.573 0.3890.389 0.4240.424 0.4700.470 0.3450.345 SPS 0.7080.708 0.7560.756 0.3490.349 0.7580.758 0.6040.604 0.4090.409 0.4640.464 0.4870.487 0.3640.364 Power-SMC 0.7140.714 0.7610.761 0.3510.351 0.7620.762 0.6120.612 0.4130.413 0.4670.467 0.4980.498 0.3860.386 APPS 0.7360.736 0.6100.610 0.3790.379 0.7820.782 0.6160.616 0.4290.429 0.4620.462 0.5120.512 0.3230.323 APS (Ours) 0.734¯ 0.734 0.732¯ 0.732 0.369¯ 0.369 0.814¯ 0.814 0.628¯ 0.628 0.429¯ 0.429 0.472¯ 0.472 0.518¯ 0.518 0.389¯ 0.389 GRPO (MATH) 0.740 0.561 0.354 0.785 0.537 0.399 0.492 0.524 0.333 APS on Post-Trained LLMs. Table 2 evaluates whether APS remains effective when applied to an RL post-trained model. On DeepSeek-Math-7B-RL, low-temperature sampling does not consistently improve the reasoning performance, while fixed-exponent power sampling yields only modest gains. In contrast, APS consistently improves performance across all three benchmarks, with particularly strong gains on MATH500 (0.492→0.5490.492→ 0.549) and HumanEval (0.524→0.5980.524→ 0.598). These results indicate that RL post-training does not eliminate query-level variation in the preferred sharpening strength, further demonstrating the merits of adaptive rather than fixed power sampling. Table 2: Pass@11 accuracy of DeepSeek-Math-7B-RL on MATH500, HumanEval, and GPQA. Benchmark Base Low-temperature Power Sampling APS (ours) MATH500 0.4920.492 0.4120.412 0.4940.494 0.549 HumanEval 0.5240.524 0.5240.524 0.5300.530 0.598 GPQA 0.3330.333 0.3030.303 0.3490.349 0.359 Pass@k Performance. To examine how performance scales with additional test-time samples, we evaluate Pass@k for k∈1,2,4,8,16k∈\1,2,4,8,16\, where a problem is considered solved if at least one of the k independently sampled responses is correct. Figure 2 presents the results on MATH500, HumanEval, and GPQA with Qwen2.5-Math-7B. Across all benchmarks, APS achieves strong Pass@k performance, with its advantage being most pronounced for small k. As k increases, the performance gaps among the training-free sampling methods gradually narrow, since drawing more samples increases the chance of covering a correct response. Nevertheless, APS remains competitive across the full range of k, indicating that adaptive sharpening improves single-sample performance without substantially sacrificing multi-sample coverage. In contrast, GRPO exhibits relatively limited improvement as k increases, suggesting that post-training may concentrate the output distribution and reduce the benefit of drawing additional test-time samples. Meanwhile, please refer to Appendix C and D for the results of adaptive power sharpening in group RL, MH-step sensitivity, effect of sample budgets, score distributions, and qualitative examples. 7 Conclusion This work proposes Adaptive Power Sampling (APS) to equip power sampling for LLM reasoning with query-level adaptivity. Motivated by the empirical observation that the preferred sharpening strength is not universal across queries, we theoretically characterize how the effect of increasing the sharpening strength depends on the self-reward gap between correct and incorrect responses. We empirically show that this gap can be estimated using the correlation between answer agreement and the model’s self-reward. Together, this theoretical characterization and practical adaptation enable APS to consistently improve test-time reasoning across diverse models and datasets, achieving consistent performance gain over power sampling with a fixed sharpening exponent, without additional training. Reproducibility statement Proofs of the theoretical results are provided in Appendices A and B. The models, benchmarks, and experimental settings are described in Section 6, with additional sampling and evaluation details in Appendix D. An anonymized implementation of APS and the evaluation scripts are included in the supplementary material. AI use statement We used generative AI tools to assist with proofs, writing evaluation code, extracting illustrative cases from experimental outputs, polishing manuscript prose, and checking and summarizing cited literature. The authors verified the AI-assisted mathematical arguments, reviewed the code and text, checked the illustrative cases against the experimental outputs, and checked the literature. The authors take responsibility for the final content of this work. References Aggarwal et al. (2023) P. Aggarwal, A. Madaan, Y. Yang, and Mausam Let’s sample step by step: adaptive-consistency for efficient reasoning and coding with LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 12375–12396. Cited by: §1, §5.1. Azizi et al. (2026) S. Azizi, E. Baghaei Potraghloo, M. Ahmadi, S. Kundu, and M. Pedram Power-SMC: low-latency sequence-level power sampling for training-free LLM reasoning. arXiv preprint arXiv:2602.10273. Cited by: §D.1, §1, §2, §3.1, §3.2, §6.1. Brown et al. (2024) B. Brown, J. Juravsky, R. Ehrlich, R. Clark, Q. V. Le, C. Ré, and A. Mirhoseini Large language monkeys: scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787. Cited by: §2. Chen et al. (2021) M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Cited by: §6.1. Chow et al. (2025) Y. Chow, G. Tennenholtz, I. Gur, V. Zhuang, B. Dai, A. Kumar, R. Agarwal, S. Thiagarajan, C. Boutilier, and A. Faust Inference-aware fine-tuning for best-of-n sampling in large language models. In International Conference on Learning Representations, Cited by: §2. Cobbe et al. (2021) K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. Cited by: §2. DeepSeek-AI (2025) DeepSeek-AI DeepSeek-R1: incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948. Cited by: §2. Guo et al. (2026) H. Guo, N. Guo, C. Meinel, and H. Yang Sample where you struggle: sharpening base model reasoning via entropy-guided power sampling. arXiv preprint arXiv:2606.09926. Cited by: §2. Hariri et al. (2026) M. Hariri, W. Chen, N. Shahini, V. Singh, K. Ye, A. Samandar, D. Ganguly, S. Sankar, Y. Zhang, S. Wang, J. Peng, B. Zhang, M. Hinczewski, and V. Chaudhary Test-time scaling in reasoning LLMs: inference regimes, evaluation, and reproducibility. arXiv preprint arXiv:2608.04001. Cited by: §2. He et al. (2025) A. W. He, D. Fried, and S. Welleck Rewarding the unlikely: lifting GRPO beyond distribution sharpening. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: §1, §2. Hendrycks et al. (2021) D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt Measuring mathematical problem solving with the math dataset. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, Vol. 1. Cited by: §C.2. Hu et al. (2022) E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, Cited by: §C.2. Huang et al. (2025) A. Huang, A. Block, D. J. Foster, D. Rohatgi, C. Zhang, M. Simchowitz, J. T. Ash, and A. Krishnamurthy Self-improvement in language models: the sharpening mechanism. In International Conference on Learning Representations, Cited by: §1, §2. Ji et al. (2026) X. Ji, R. Tutunov, M. Zimmer, and H. B. Ammar Scalable power sampling: unlocking efficient, training-free reasoning for LLMs via distribution sharpening. arXiv preprint arXiv:2601.21590. Cited by: §D.1, §D.1, §1, §1, §2, §3.1, §3.2, §6.1. Karan and Du (2026) A. Karan and Y. Du Reasoning with sampling: your base model is smarter than you think. In International Conference on Learning Representations, Cited by: §D.1, §D.1, §D.1, §D.4, §1, §1, §1, §2, §3.1, §3.1, §3.2, §3.2, §3.2, §5.2, §6.1. Li et al. (2022) Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. Dal Lago, T. Hubert, P. Choy, C. de Masson d’Autume, I. Babuschkin, X. Chen, P. Huang, J. Welbl, S. Gowal, A. Cherepanov, J. Molloy, D. J. Mankowitz, E. Sutherland Robson, P. Kohli, N. de Freitas, K. Kavukcuoglu, and O. Vinyals Competition-level code generation with AlphaCode. Science 378 (6624), p. 1092–1097. External Links: Document Cited by: §2. Lightman et al. (2024) H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe Let’s verify step by step. In International Conference on Learning Representations, Cited by: §C.2, §2, §3.2, §6.1. Lin et al. (2025) Z. Lin, M. Lin, Y. Xie, and R. Ji CPPO: accelerating the training of group relative policy optimization-based reasoning models. In Advances in Neural Information Processing Systems, Vol. 38. Cited by: §2. Liu et al. (2025) Z. Liu, C. Chen, W. Li, P. Qi, T. Pang, C. Du, W. S. Lee, and M. Lin Understanding R1-zero-like training: a critical perspective. arXiv preprint arXiv:2503.20783. Cited by: §2. Meng et al. (2026) Z. Meng, P. Xie, and K. Chen Depth-entropy guided sampling for training-free LLM reasoning. arXiv preprint arXiv:2607.09693. Cited by: §D.1, §1, §3.1, §3.2. Nguyen et al. (2026) T. Nguyen, M. Zimmer, R. Tutunov, X. Ji, and H. Bou Ammar The model knows, the decoder finds: future value guided particle power sampling. arXiv preprint arXiv:2605.02427. Cited by: §D.1, §1, §2, §3.1, §3.2, §6.1. Ouyang et al. (2022) L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, Vol. 35, p. 27730–27744. Cited by: §2. Razin et al. (2025) N. Razin, Z. Wang, H. Strauss, S. Wei, J. D. Lee, and S. Arora What makes a reward model a good teacher? an optimization perspective. In Advances in Neural Information Processing Systems, Vol. 38, p. 59162–59222. Cited by: §4.3. Rein et al. (2024) D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman GPQA: a graduate-level google-proof Q&A benchmark. In First Conference on Language Modeling, Cited by: §6.1. Shao et al. (2025) R. Shao, S. S. Li, R. Xin, S. Geng, Y. Wang, S. Oh, S. S. Du, N. Lambert, S. Min, R. Krishna, Y. Tsvetkov, H. Hajishirzi, P. W. Koh, and L. Zettlemoyer Spurious rewards: rethinking training signals in RLVR. arXiv preprint arXiv:2506.10947. Cited by: §2, §6.2. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo DeepSeekMath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: §2, §4.3, §6.1, §6.2. Snell et al. (2024) C. Snell, J. Lee, K. Xu, and A. Kumar Scaling LLM test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314. Cited by: §2. Song et al. (2025) Y. Song, J. Kempe, and R. Munos Outcome-based exploration for LLM reasoning. arXiv preprint arXiv:2509.06941. Cited by: §2. Tomihari and Sato (2026) A. Tomihari and I. Sato Power distribution bridges sampling, self-reward RL, and self-distillation. arXiv preprint arXiv:2605.04542. Cited by: §2, §4.2. Wang et al. (2026) K. Wang, Z. Yu, L. Wang, and Y. Huang Cache coherent resampling for efficient test time scaling in LLM reasoning via adaptive sequential monte carlo. In Proceedings of the 43rd International Conference on Machine Learning, Cited by: §2. Wang et al. (2020) P. Wang, S. Hsieh, S. Chang, Y. Chen, J. Pan, W. Wei, and D. Juan Contextual temperature for language modeling. arXiv preprint arXiv:2012.13575. Cited by: §C.1, §3.1, §6.1. Wang et al. (2023) X. Wang, J. Wei, D. Schuurmans, Q. V. Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations, Cited by: §1, §2, §5.1. Xie et al. (2023) Y. Xie, K. Kawaguchi, Y. Zhao, J. X. Zhao, M. Kan, J. He, and M. Xie Self-evaluation guided beam search for reasoning. In Advances in Neural Information Processing Systems, Vol. 36. Cited by: §2. Yang et al. (2024a) A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. Cited by: §C.2, §6.1. Yang et al. (2024b) A. Yang, B. Zhang, B. Hui, B. Gao, B. Yu, C. Li, D. Liu, J. Tu, J. Zhou, J. Lin, K. Lu, M. Xue, R. Lin, T. Liu, X. Ren, and Z. Zhang Qwen2.5-math technical report: toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122. Cited by: §3.2, §6.1. Yao et al. (2023) S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao, and K. Narasimhan Tree of thoughts: deliberate problem solving with large language models. In Advances in Neural Information Processing Systems, Cited by: §2. Yu et al. (2025) Q. Yu, Z. Zhang, R. Zhu, Y. Yuan, X. Zuo, Y. Yu, W. Dai, T. Fan, G. Liu, J. Liu, L. Liu, X. Liu, H. Lin, Z. Lin, B. Ma, G. Sheng, Y. Tong, C. Zhang, M. Zhang, R. Zhang, W. Zhang, H. Zhu, J. Zhu, J. Chen, J. Chen, C. Wang, H. Yu, Y. Song, X. Wei, H. Zhou, J. Liu, W. Ma, Y. Zhang, L. Yan, Y. Wu, and M. Wang DAPO: an open-source LLM reinforcement learning system at scale. In Advances in Neural Information Processing Systems, Vol. 38. Cited by: §2. Yue et al. (2025) Y. Yue, Z. Chen, R. Lu, A. Zhao, Z. Wang, Y. Yue, S. Song, and G. Huang Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?. In Advances in Neural Information Processing Systems, Vol. 38. Cited by: §1, §2. Zhang et al. (2025) Q. Zhang, F. Lyu, Z. Sun, L. Wang, W. Zhang, W. Hua, H. Wu, Z. Guo, Y. Wang, N. Muennighoff, I. King, X. Liu, and C. Ma A survey on test-time scaling in large language models: what, how, where, and how well?. arXiv preprint arXiv:2503.24235. Cited by: §2. Zhou et al. (2024) A. Zhou, K. Yan, M. Shlapentokh-Rothman, H. Wang, and Y. Wang Language agent tree search unifies reasoning, acting, and planning in language models. In Proceedings of the 41st International Conference on Machine Learning, Vol. 235, p. 62138–62160. Cited by: §2. Zhou et al. (2026) F. Zhou, A. Mehrotra, and Q. C. Liu Reasoning with sampling: cutting at decision points. arXiv preprint arXiv:2605.30327. Cited by: §2. Appendix A Proofs A.1 Proof of Theorem 1 For a fixed prompt x, according to the definition of P(α)P(α) in (4), we have dP(α)dα dP(α)dα =dα∑yr⋆(y)pθ(y∣x)αZα = ddα _yr (y) p_θ(y x)^αZ_α =∑yr⋆(y)rself(y)pθ(y∣x)αZα−pθ(y∣x)αZα′Zα2 = _yr (y) r_self(y)p_θ(y x)^αZ_α-p_θ(y x)^αZ_α Z_α^2 =∑yr⋆(y)πα(y)[rself(y)−Zα′Zα] = _yr (y) _α(y) [r_self(y)- Z_α Z_α ] =πα[r⋆(Y)rself(Y)]−πα[r⋆(Y)]πα[rself(Y)] =E_ _α [r (Y)r_self(Y) ]-E_ _α [r (Y) ]E_ _α [r_self(Y) ] =Covπα(r⋆(Y),rself(Y)) =Cov_ _α (r (Y),r_self(Y) ) (20) where Zα′=∑ypθ(y∣x)αlogpθ(y∣x)Z_α = _yp_θ(y x)^α p_θ(y x). The proof is thus completed. A.2 Proof of Corollary 1 For binary r⋆r , P(α)=Prπα(r⋆(Y)=1)P(α)= _ _α(r (Y)=1). We then have Covπα(r⋆(Y),rself(Y)) _ _α (r (Y),r_self(Y) ) =πα[r⋆(Y)rself(Y)]−πα[r⋆(Y)]πα[rself(Y)] =E_ _α [r (Y)r_self(Y) ]-E_ _α [r (Y) ]E_ _α [r_self(Y) ] =P(α)μC(α)−P(α)πα[rself(Y)] =P(α) _C(α)-P(α)E_ _α [r_self(Y) ] =P(α)[μC(α)−P(α)μC(α)−(1−P(α))μI(α)] =P(α) [ _C(α)-P(α) _C(α)- (1-P(α) ) _I(α) ] =P(α)(1−P(α))[μC(α)−μI(α)]=P(α)(1−P(α))δ(α), =P(α) (1-P(α) ) [ _C(α)- _I(α) ]=P(α) (1-P(α) )δ(α), (21) where the last step uses πα[rself(Y)]=P(α)μC(α)+(1−P(α))μI(α)E_ _α[r_self(Y)]=P(α) _C(α)+(1-P(α)) _I(α). A.3 Proof of Lemma 1 For fixed x, define ZC(α)=∑y∈Cpθ(y∣x)α,ZI(α)=∑y∈Ipθ(y∣x)α.Z_C(α)= _y _Cp_θ(y x)^α, Z_I(α)= _y _Ip_θ(y x)^α. (22) For y∈Cy _C, the distribution of correct responses under πα _α is denoted as Prπα(Y=y∣r⋆(Y)=1)=pθ(y∣x)αZC(α). _ _α\! (Y=y r (Y)=1 )= p_θ(y x)^αZ_C(α). (23) It follows that μC(α) _C(α) =∑y∈Cpθ(y∣x)αrself(y)ZC(α)=ZC′(α)ZC(α)=dαlogZC(α). = _y _Cp_θ(y x)^αr_self(y)Z_C(α)= Z_C (α)Z_C(α)= ddα Z_C(α). (24) By differentiating (24), we obtain dμC(α)dα d _C(α)dα =∑y∈Cpθ(y∣x)αrself2(y)ZC(α)−(∑y∈Cpθ(y∣x)αrself(y)ZC(α))2 = _y _Cp_θ(y x)^αr_self^2(y)Z_C(α)- ( _y _Cp_θ(y x)^αr_self(y)Z_C(α) )^2 =πα[rself2(Y)∣r⋆(Y)=1]−μC2(α)=σC2(α). =E_ _α [r_self^2(Y) r (Y)=1 ]- _C^2(α)= _C^2(α). (25) The same calculation for the incorrect responses gives dμI(α)dα=σI2(α). d _I(α)dα= _I^2(α). (26) Since δ(α)=μC(α)−μI(α)δ(α)= _C(α)- _I(α), we have dδ(α)dα=σC2(α)−σI2(α), dδ(α)dα= _C^2(α)- _I^2(α), (27) which proves the lemma. A.4 Proof of Theorem 2 Taking any y,y′∈Cy,y _C with rself(y)<rself(y′)r_self(y)<r_self(y ) as an example, as α→∞α→∞, their conditional probability ratio satisfies Prπα(Y=y∣r⋆(Y)=1)Prπα(Y=y′∣r⋆(Y)=1)=expα[rself(y)−rself(y′)]→0. _ _α(Y=y r (Y)=1) _ _α(Y=y r (Y)=1)= \α [r_self(y)-r_self(y ) ] \→ 0. (28) Thus, the conditional distribution over CY_C concentrates on the highest-self-reward responses in the correct class. The same argument applies to IY_I. Hence, limα→∞μC(α)=sC,limα→∞μI(α)=sI, _α→∞ _C(α)=s_C, _α→∞ _I(α)=s_I, (29) and therefore limα→∞δ(α)=sC−sI=Δs, _α→∞δ(α)=s_C-s_I= _s, (30) which proves the theorem. Appendix B Additional theoretical results B.1 Second-Order Effect of Sharpening Proposition 1. The second derivative of the performance with respect to the sharpening exponent α is given by d2Pdα2=P(α)(1−P(α))[(1−2P(α))δ(α)2+σC2(α)−σI2(α)]. d^2Pdα^2=P(α)(1-P(α)) [(1-2P(α))δ(α)^2+σ^2_C(α)-σ^2_I(α) ]. (31) Proposition 1 follows directly by differentiating the first-order performance relation in (10) and applying Lemma 1. The sign of (31) is determined by two factors. First, the term (1−2P(α))δ2(1-2P(α))δ^2 could remain positive on hard queries with P(α)<1/2P(α)<1/2 and negative on easy queries with P(α)>1/2P(α)>1/2. From Lemma 1, the variance-difference term equals dδ(α)/dαdδ(α)/dα and therefore indicates whether the self-reward gap is widening or narrowing as α increases. This decomposition immediately yields a simple sufficient condition for concavity. If P(α)>1/2P(α)>1/2 and σC2(α)≤σI2(α) _C^2(α)≤ _I^2(α), both terms in (31) are non-positive, and hence d2Pdα2≤0 d^2Pdα^2≤ 0. Therefore, for queries that are already more likely to be solved than not under this variance condition, the performance is locally concave in α. If δ(α)>0δ(α)>0 also holds, the performance still increases with α, but at a decreasing rate. B.2 Generalization to Sample-Based Objectives We further generalize the preceding analysis to objectives defined over multiple responses sampled from the power distribution. Let s index a scenario and Y1,…,YKsY_1,…,Y_K_s be independently sampled from πα(y∣x) _α(y x). Let ℓs(Y1:Ks) _s(Y_1:K_s) denote a statistic of interest with no explicit dependence on α. We define the corresponding scenario objective as Js(α)=[ℓs(Y1:Ks)],J_s(α)=E [ _s(Y_1:K_s) ], (32) where the expectation is taken over the KsK_s independent samples from πα(⋅∣x) _α(· x). Then the following theorem characterizes how the sharpening exponent α affects the scenario objective defined above. Proposition 2. The scenario objective in (32) satisfies dJs(α)dα=Cov(ℓs(Y1:Ks),∑i=1Ksrself(Yi)), dJ_s(α)dα=Cov ( _s(Y_1:K_s), _i=1^K_sr_self(Y_i) ), (33) where the covariance is taken over the same KsK_s independent responses sampled from πα(⋅∣x) _α(· x). Proof. Define the joint distribution of the KsK_s independent responses under prompt x as qα(y1:Ks∣x)=∏i=1Ksπα(yi∣x).q_α(y_1:K_s x)= _i=1^K_s _α(y_i x). (34) Since ℓs(Y1:Ks) _s(Y_1:K_s) has no explicit dependence on α, applying the log-derivative identity gives dJs(α)dα dJ_s(α)dα =qα[ℓs(Y1:Ks)∂αlogqα(Y1:Ks∣x)]. =E_q_α [ _s(Y_1:K_s) ∂α q_α(Y_1:K_s x) ]. (35) Since the KsK_s responses are independently sampled, ∂αlogqα(Y1:Ks∣x) ∂α q_α(Y_1:K_s x) =∑i=1Ks∂αlogπα(Yi∣x). = _i=1^K_s ∂α _α(Y_i x). (36) From the power distribution πα(y∣x)=pθ(y∣x)αZα(x) _α(y x)= p_θ(y x)^αZ_α(x), we have ∂αlogπα(Yi∣x) ∂α _α(Y_i x) =logpθ(Yi∣x)−∂αlogZα(x) = p_θ(Y_i x)- ∂α Z_α(x) =rself(Yi)−πα[rself(Y)]. =r_self(Y_i)-E_ _α [r_self(Y) ]. (37) Therefore, dJs(α)dα dJ_s(α)dα =[ℓs(Y1:Ks)(∑i=1Ksrself(Yi)−Ksπα[rself(Y)])]. =E [ _s(Y_1:K_s) ( _i=1^K_sr_self(Y_i)-K_sE_ _α [r_self(Y) ] ) ]. (38) Since the responses are identically distributed, Ksπα[rself(Y)]=[∑i=1Ksrself(Yi)].K_sE_ _α [r_self(Y) ]=E [ _i=1^K_sr_self(Y_i) ]. (39) Hence, dJs(α)dα dJ_s(α)dα =[ℓs(Y1:Ks)(∑i=1Ksrself(Yi)−[∑i=1Ksrself(Yi)])]=Cov(ℓs(Y1:Ks),∑i=1Ksrself(Yi)). \!=\!E\! [\! _s(Y_1:K_s)\!\! ( _i=1^K_sr_self(Y_i)\!-\!E\! [ _i=1^K_sr_self(Y_i)\! ]\! )\! ]\!\!=\!\!Cov\! (\! _s(Y_1:K_s), _i=1^K_sr_self(Y_i)\! ). (40) ∎ Proposition 2 shows that the marginal effect of sharpening is determined by the alignment between the scenario-specific statistic and the aggregate self-reward of the sampled responses. For test-time inference, setting Ks=1K_s=1 and ℓs(Y)=r⋆(Y) _s(Y)=r (Y) recovers the performance derivative derived in Section 4.2. For rollout generation in group-relative RL, taking ℓs _s as the within-group reward variance recovers CK(α)C_K(α). This provides a common way to analyze how sharpening affects both test-time reasoning and RL rollout generation. Appendix C Adaptive Power Sharpening in group-relative RL C.1 Adaptive Sharpening Exponent Selection In this appendix, we investigate whether the rollout objective derived in Appendix A can be used to adapt the sharpening strength for group-relative RL training, focusing on GRPO. For binary rewards, the analysis of Theorem 3 shows that the sharpening exponent α directly controls the within-group reward variation. Motivated by this result, we adapt α separately for each training prompt before generating the rollout group used for the subsequent GRPO update. To translate the rollout objective into a practical procedure, we compare candidate sharpening exponents using a small number of pilot responses. For each prompt and each exponent candidate α(j)∈Δα^(j)∈ , we generate kpk_p pilot responses using token-level low-temperature T=1/α(j)T=1/α^(j) (Wang et al., 2020). Since the correctness reward r⋆r is available during RL post-training, we estimate the correctness probability for each candidate as P^j=1kp∑i=1kpr⋆(y~i(j)), P_j= 1k_p _i=1^k_pr \! ( y^(j)_i ), (41) where y~i(j) y^(j)_i denotes the i-th pilot response associated with candidate exponent α(j)α^(j). Based on (13), the corresponding within-group reward variation is estimated by C^K(α(j))=K−1KP^j(1−P^j). C_K\! (α^(j) )= K-1K P_j (1- P_j ). (42) We then select α⋆=argmaxα(j)∈ΔC^K(α(j)).α = _α^(j)∈ C_K\! (α^(j) ). (43) If there is a tie, we choose the smallest exponent as α⋆α . Note that token-level low-temperature is only used as a practical approximation to compare candidate exponents; After selecting α⋆α , we draw K sequence-level MCMC samples from pθ(y|x)α⋆p_θ(y|x)^α using the same sampler as in Section 5. These samples form the rollout group, which is evaluated with correctness reward r⋆r and used for the standard GRPO update. Algorithm 2 summarizes the complete rollout generation and update procedure. Algorithm 2 Power-sharpened Rollout Generation and GRPO update 0: Prompt x, current model pθp_θ, Δ , kpk_p, K, r⋆r 1: for α(j)∈Δα^(j)∈ do 2: Sample kpk_p pilots at token temperature T=1/α(j)T=1/α^(j) 3: Compute ^K(α(j)) C_K(α^(j)) using (42) 4: end for 5: Select α⋆α using (43), breaking ties with the smallest α 6: Draw K sequence-level MCMC samples from pθ(y|x)α⋆p_θ(y|x)^α 7: Evaluate the rollouts with r⋆r and use them for the standard GRPO update C.2 Experimental Results To examine the performance gain of the designed power-sharpened rollout, we train Low-Rank Adaptation (LoRA) (Hu et al., 2022) adapters on the Qwen2.5-7B (Yang et al., 2024a) using a MATH-derived reasoning dataset with 12,000 examples (Hendrycks et al., 2021). Each training update contains 24 prompts with K=8K=8 responses per prompt, resulting in 192 rollouts per update. The LoRA rank and scaling factor are 32 and 64, respectively. We use a learning rate of 3×10−63× 10^-6, a GRPO KL coefficient of β=0.04β=0.04, and a clipping range of 0.20.2. For APS-assisted rollout generation, we set Δ=1,2,4 =\1,2,4\ and kp=3k_p=3, and select α⋆α independently for each prompt. Each pilot generation uses a maximum length of 1,536 tokens. We compare APS-assisted GRPO with Vanilla GRPO, which follows the standard GRPO training procedure without adaptive exponent selection for rollout generation. We evaluate the trained models on MATH500 (Lightman et al., 2024) using Pass@1 accuracy with one sampled response per problem. The corresponding results are summarized in Table 3. Method Updates Relative time Pass@11 Base, no GRPO – – 49.0% Vanilla GRPO 80 1.00×1.00× 52.0% APS-assisted GRPO 62 0.96×0.96× 54.2% APS-assisted GRPO 80 1.41×1.41× 57.6% Table 3: Training cost and Pass@11 accuracy of Qwen2.5-7B on MATH500 As shown in Table 3, APS-assisted GRPO consistently improves over Vanilla GRPO. Under a comparable wall-clock budget, APS-assisted GRPO achieves 54.2% Pass@1 after 62 steps, compared with 52.0% for Vanilla GRPO, while using slightly less training time (0.96×0.96×). With the same number of 80 updates, APS-assisted GRPO further improves Pass@1 to 57.6%, a gain of 5.6 percentage points. Appendix D Supplementary Experimental Results D.1 Additional Evaluation Details All inference experiments are conducted on a node with 8 GPUs using vLLM v0.9.1. Prompts. We follow the prompt templates used in (Karan and Du, 2026; Ji et al., 2026), as summarized in Table 4. Hyperparameter settings of APS. APS uses a maximum generation length of Tmax=3072T_ =3072 tokens. For sequence-level MCMC sampling, we follow the implementation settings of (Karan and Du, 2026). Specifically, we use a block size of B=192B=192, which determines the number of tokens updated in each proposal block, and perform NMCMC=10N_MCMC=10 MH steps at each stage. We initialize APS with initial exponent α1=4 _1=4, batch size K=4K=4, total sample budget C=12C=12, step size η=2η=2, tolerance ϵ=0.05ε=0.05, exponent range α∈[1,8]α∈[1,8] with αmin=1 _min=1 and αmax=8 _max=8, optional quality threshold qmin=0.05q_ =0.05. Unless otherwise stated, we report results averaged over 3 random seeds. Task Qwen Prompt DeepSeek Prompt MATH500 Can you solve the following math problem? Please reason step by step, and put your final answer within \ Please reason step by step, and put your final answer within \ HumanEval Write a Python function to solve the following problem: Complete the following Python function: GPQA Answer the following multiple-choice question. The last line of your response should be of the following format: ’\ LETTER’ (without quotes) where LETTER is one of ABCD (ex. ’\ A’). Think step by step before answering. Question A) B) C) D) Answer the following multiple-choice question. The last line of your response should be of the following format: ’\ LETTER’ (without quotes) where LETTER is one of ABCD (ex. ’\ A’). Think step by step before answering. Question A) B) C) D) Table 4: Prompt templates used for baselines and our method. Hyperparameter settings of baselines. For low-temperature sampling, we apply token-level temperature scaling with temperature τ=1/α=0.25τ=1/α=0.25 with α=4α=4. For power sampling, we follow (Karan and Du, 2026) with α=4α=4, Tmax=3072T_ =3072, B=192B=192, and NMCMC=10N_MCMC=10 MH transitions per stage. For SPS, we follow (Ji et al., 2026) with α=4α=4, Tmax=3072T_ =3072, and B=192B=192. At each decoding step, SPS considers the top Kt=8K_t=8 candidate tokens and uses Mt=8M_t=8 future rollouts for each candidate. For Power-SMC, we use the reported results from (Azizi et al., 2026) and (Meng et al., 2026), depending on the corresponding model and benchmark. For APPS, we use the results reported in (Nguyen et al., 2026). We refer to the original papers for their implementation details and hyperparameter settings. D.2 Effect of MCMC Steps We vary the number of MH steps, NMCMCN_MCMC, and assess whether the default setting is sufficient and how APS is affected by MCMC approximation quality. The evaluation on MATH500 uses Qwen2.5-7B, Qwen2.5-Math-7B, and DeepSeek-Math-7B, with NMCMC∈6,8,10,12N_MCMC∈\6,8,10,12\ steps per block. Using the same sampling settings as in Appendix D.1, we compare APS with fixed-exponent power sampling at α=4α=4. The Pass@1 accuracy of base decoding in Table 1 is include as fixed references. Figure 3 shows that NMCMC=10N_MCMC=10 MH steps are sufficient under the tested settings. To be specific, increasing NMCMCN_MCMC from 10 to 12 changes APS Pass@1 by −0.6-0.6, +0.1+0.1, and −0.5-0.5 percentage points on Qwen2.5-7B, Qwen2.5-Math-7B, and DeepSeek-Math-7B, respectively. Additional MH steps beyond the default setting therefore provide no consistent accuracy gain. APS also maintains its advantage when fewer MH steps are used, outperforming power sampling at every evaluated step count for all three models. Even with only 6 steps, APS improves Pass@1 accuracy by 2.52.5, 5.55.5, and 6.26.2 percentage points on the three models, compared with 2.82.8, 6.66.6, and 4.84.8 points at 10 steps. The persistent advantage indicates that APS is robust to the tested range of MH steps. Figure 3: Pass@1 accuracy on MATH500 under different numbers of MH steps. The vertical dashed line marks the default NMCMC=10N_MCMC=10. D.3 Effect of Sample Budget We vary the total sample budget C and evaluate APS with Qwen2.5-Math-7B on HumanEval with C∈4,8,12,16C∈\4,8,12,16\ while keeping the other settings in Appendix D.1 fixe. The Pass@1 accuracies of base decoding and power sampling in Table 1 are include as fixed references. Figure 4 shows that the reasoning performance of APS generally improves as the total sample budget C increases. For example, increasing C from 4 to 16 raises Pass@1 accuracy from 0.585 to 0.646, a gain of 6.1 percentage points. This is because a larger budget allows APS to perform more rounds of sharpening exponent adaptation, providing more opportunities to refine α and identify a sharpening strength better suited to the current query. Figure 4: Pass@1 accuracy on HumanEval with Qwen2.5-Math-7B under different sample budgets. The vertical dashed line marks the default C=12C=12. D.4 Log-Likelihood and Confidence Densities We examine log-likelihood and confidence densities of responses generated by different sampling methods on MATH500 with Qwen2.5-Math-7B. For each response y, we compute its mean token log-likelihood L(y)=T−1∑t=1Tlogpθ(yt∣x,y<t)L(y)=T^-1 _t=1^T p_θ(y^t x,y^<t) and use the mean negative predictive entropy C(y)=−T−1∑t=1TH[pθ(⋅∣x,y<t)]C(y)=-T^-1 _t=1^TH[p_θ(· x,y^<t)] as a confidence score (Karan and Du, 2026). Both quantities are evaluated under the base model. For each query, we collect one response from base decoding, one from fixed-exponent power sampling with α=4α=4, and one from APS, and compute L(y)L(y) and C(y)C(y) for each response. Figures 5 and 6 show the empirical densities of mean token log-likelihood and negative predictive entropy. Relative to base decoding, fixed-exponent power sampling concentrates both distributions near zero, indicating a preference for responses with higher base-model likelihood and confidence. APS preserves a similar high-density mode but exhibits broader distributions (standard deviation: →0.04940.0201\!→\!0.0494 for L(y)L(y) and →0.10130.0402\!→\!0.1013 for C(y)C(y)), with slightly lower mean scores (mean: −→−0.0421-0.0329\!→\!-0.0421 for L(y)L(y) and −→−0.0858-0.0671\!→\!-0.0858 for C(y)C(y)). These results suggest that APS retains the high-likelihood, high-confidence region favored by power sampling while covering a broader range of responses through query-adaptive sharpening. Figure 5: Density of mean token-level log-likelihood on MATH500 with Qwen2.5-Math-7B. Figure 6: Density of mean negative predictive entropy on MATH500 with Qwen2.5-Math-7B. D.5 More Qualitative Examples Tables 5–7 compare APS, power sampling, and GRPO on representative examples from the three benchmarks considered in this paper. The first two examples use Qwen2.5-Math-7B, while the last example uses Qwen2.5-7B. Across the three examples, APS returns the correct output whenever fixed-exponent power sampling fails. In the MATH500 example, GRPO also succeeds, while APS adapts α away from the fixed value and recovers the correct answer. In the HumanEval and GPQA examples, both power sampling and GRPO fail, whereas APS returns the correct output with a query-specific sharpening exponent. Together, these cases illustrate the potential benefit of adapting the sharpening strength to individual queries. The results of a cross-country team’s training run are graphed as distance against time. The graph contains Evelyn (1.25,4.5)(1.25,4.5), Briana (2.5,2.2)(2.5,2.2), Carla (4.25,5.2)(4.25,5.2), Debra (5.6,2.8)(5.6,2.8), and Angela (6.8,1.4)(6.8,1.4). Which student has the greatest average speed? Correct answer: Evelyn Evelyn. Method Representative response (excerpt) Result APS (α⋆=3.605α =3.605) The response computes distance divided by time for each student and reports (’Evelyn’, 3.6). It concludes: “The student with the greatest average speed is Evelyn, with an average speed of 3.6 units per time unit.” Final answer: Evelyn Evelyn. True Power Sampling (α=4α=4) The response extracts the same five points and writes the correct code for the ratios and their maximum, but its generated output is (’Carla’, 1.2205882352941178). It concludes: “The student with the greatest average speed is Carla, with an average speed of approximately 1.22 units per time unit.” Final answer: Carla Carla. False GRPO The response computes the average speeds as Evelyn: 3.63.6; Briana: 0.880.88; Carla: 1.221.22; Debra: 0.50.5; Angela: 0.2060.206. It concludes: “Based on the calculated average speeds, Evelyn has the greatest average speed of 3.6 units.” Final answer: Evelyn Evelyn. True Table 5: A representative MATH500 example with Qwen2.5-Math-7B. APS and GRPO return the correct answer, while power sampling returns an incorrect answer despite using the correct computation. Complete intersperse(numbers: List[int], delimeter: int) -> List[int]. Insert delimeter between every two consecutive elements of numbers. Expected outputs: intersperse([], 4) = [] and intersperse([1, 2, 3], 4) = [1, 4, 2, 4, 3]. Method Completion Passed APS (α⋆=2.497α =2.497) result = [] for i in range(len(numbers)): result.append(numbers[i]) if i < len(numbers) - 1: result.append(delimeter) return result True Power Sampling (α=4α=4) result = [] for i in range(len(numbers) - 1): result.append(numbers[i]) result.append(delimeter) result.append(numbers[-1]) return result False GRPO return [delimeter] * (len(numbers) - 1) + numbers False Table 6: A representative HumanEval example with Qwen2.5-Math-7B. APS passes the tests, while power sampling fails on the empty list and GRPO places the delimiters before the input elements rather than between them. “Oh, I know you,” the ribonucleoprotein particle says to the nascent chain as they meet. “Pause there for a minute. Let me show you in; you really need some sugar.” “It seems somewhat rough. I guess this is goodbye; I need to be on my way”, the chain replies. Where did they meet, and where is the chain heading? A) cytosol to the extracellular space; B) Golgi to the mitochondrion; C) membrane to the nucleus; D) ribosome to the proteasome. Correct answer: A A. Method Representative response (excerpt) Result APS (α⋆=3.754α =3.754) “A) cytosol to the extracellular space: This is a common pathway for proteins synthesized in the cytosol to be secreted. […] D) ribosome to the proteasome: This is a pathway for proteins that are targeted for degradation. […] Given the context and the dialogue, the most likely scenario is that the nascent chain is being processed and then secreted from the cell. Therefore, the correct answer is: A A” True Power Sampling (α=4α=4) “D) ribosome to the proteasome: This option involves the nascent chain moving from the ribosome to the proteasome. The dialogue suggests that the nascent chain is being guided into a specific location (the ribosome) and then leaving that location (the proteasome). […] Based on the analysis, the most likely answer is: D D” False GRPO “D) Ribosome to the proteasome: This option fits because the nascent chain is being guided into the Golgi apparatus, where it will undergo glycosylation, and then it will be transported to the proteasome for degradation if it’s not needed. […] Therefore, the correct answer is D D.” False Table 7: A representative GPQA example with Qwen2.5-7B. APS returns the correct answer A, while power sampling and GRPO both return the wrong answer D.