Paper deep dive
Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding
Minoo Ahmadi, Seyedarmin Azizi, Erfan Baghaei Potraghloo, Mehdi Kamal, Massoud Pedram
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 9/16/2026, 8:19:04 AM
Summary
The paper introduces Chopthin-Consensus Power Sampling (CCPS), a training-free inference-time method for Large Language Models (LLMs) that improves reasoning accuracy by preserving genealogical diversity. CCPS replaces standard equal-weight resampling in Sequential Monte Carlo (SMC) with the Chopthin resampler, which bounds the ratio between largest and smallest particle weights to prevent the loss of potentially correct low-weight trajectories. It further employs a semantic-majority selection mechanism to identify the correct answer from the enriched population. Experiments show CCPS matches or exceeds the accuracy of Power-SMC baselines in 14 of 15 settings.
Entities (8)
Relation Signals (7)
Chopthin-Consensus Power Sampling → uses → Chopthin Resampler
confidence 95% · Our method applies the Chopthin resampler to LLM decoding
Chopthin-Consensus Power Sampling → uses → Semantic-Majority Selection
confidence 93% · Combined with semantic-majority selection, CCPS matches or exceeds the final-answer accuracy
Sequential Monte Carlo → isusedby → Power-SMC
confidence 90% · Power-SMC (Azizi et al., 2026) reformulates the problem using Sequential Monte Carlo (SMC) sampling.
Chopthin-Consensus Power Sampling → outperforms → Power-SMC
confidence 90% · CCPS matches or exceeds the final-answer accuracy of the Power-SMC baseline in 14 of 15 settings
Chopthin Resampler → preserves → Genealogical Diversity
confidence 90% · This targeted intervention preserves a richer set of distinct reasoning paths... mitigating genealogical diversity loss
Chopthin-Consensus Power Sampling → improves → Oracle Coverage
confidence 88% · Chopthin increases oracle coverage in 13 of 15 settings.
Chopthin Resampler → guarantees → Effective Sample Size
confidence 85% · guarantees a lower bound on the post-resampling effective sample size (ESS)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Inference-time power sampling via Sequential Monte Carlo (SMC) can substantially improve large language model (LLM) reasoning without requiring post-training. However, many existing SMC approaches rely on equal-weight resampling, which can aggressively prune low-weight trajectories, discarding potentially correct reasoning paths and degrading the genealogical diversity of the search space. To address this, we introduce Chopthin-Consensus Power Sampling (CCPS). Our method applies the Chopthin resampler to LLM decoding: rather than equalizing weights and forcing unnecessary particle duplication, it enforces an upper bound on the ratio between the largest and smallest weights and carries the unequal weights forward. This targeted intervention preserves a richer set of distinct reasoning paths, keeps the weighted SMC approximation unchanged in conditional expectation, and guarantees a lower bound on the post-resampling effective sample size (ESS). To fully exploit this enriched population, we employ a semantic-majority selection mechanism that merges token-identical final trajectories, clusters semantically equivalent answers, and returns the answer supported by the largest number of distinct trajectories. Evaluating across three open-weight models and five reasoning benchmarks, we show that Chopthin increases oracle coverage in 13 of 15 settings. Combined with semantic-majority selection, CCPS matches or exceeds the final-answer accuracy of the Power-SMC baseline in 14 of 15 settings, delivering absolute gains of up to 10.6 percentage points. These findings demonstrate that diversity-preserving resampling and diversity-aware selection are complementary mechanisms for training-free LLM reasoning. Code is available at this http URL.
Tags
Links
- Source: https://arxiv.org/abs/2609.12243v1
- Canonical: https://arxiv.org/abs/2609.12243v1
Trouble viewing inline? Open PDF directly →
Full Text
73,292 characters extracted from source content.
Expand or collapse full text
Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding Minoo Ahmadi Affiliation: Mehdi Kamal Massoud Pedram Seyedarmin Azizi Affiliation: University of Southern California, Los Angeles, USA Erfan Baghaei Potraghloo Affiliation: minooahm, seyedarm, baghaeip, mehdi.kamal, pedram@usc.edu Abstract Inference-time power sampling via Sequential Monte Carlo (SMC) can substantially improve large language model (LLM) reasoning without requiring post-training. However, many existing SMC approaches rely on equal-weight resampling, which can aggressively prune low-weight trajectories, discarding potentially correct reasoning paths and degrading the genealogical diversity of the search space. To address this, we introduce Chopthin-Consensus Power Sampling (CCPS). Our method applies the Chopthin resampler to LLM decoding: rather than equalizing weights and forcing unnecessary particle duplication, it enforces an upper bound on the ratio between the largest and smallest weights and carries the unequal weights forward. This targeted intervention preserves a richer set of distinct reasoning paths, keeps the weighted SMC approximation unchanged in conditional expectation, and guarantees a lower bound on the post-resampling effective sample size (ESS). To fully exploit this enriched population, we employ a semantic-majority selection mechanism that merges token-identical final trajectories, clusters semantically equivalent answers, and returns the answer supported by the largest number of distinct trajectories. Evaluating across three open-weight models and five reasoning benchmarks, we show that Chopthin increases oracle coverage in 13 of 15 settings. Combined with semantic-majority selection, CCPS matches or exceeds the final-answer accuracy of the Power-SMC baseline in 14 of 15 settings, delivering absolute gains of up to 10.6 percentage points. These findings demonstrate that diversity-preserving resampling and diversity-aware selection are complementary mechanisms for training-free LLM reasoning. Code is available at github.com/MinooAhmadii/chopthin-consensus-power-sampling. 1 Introduction Reinforcement-learning (RL) post-training has driven much of the recent progress in the reasoning ability of large language models (Shao et al., 2024). However, these performance gains come at a high cost: the post-training process requires a dedicated training pipeline, complex reward signals or verifiers, and substantial computational resources, ultimately freezing the improvements into a static set of new model parameters. Fortunately, emerging research suggests that these resource-intensive training steps may not be strictly necessary to achieve similar reasoning capabilities. The improvements observed from RL post-training actually result, in large part, from increasing the probability of successful reasoning trajectories that already exist within the base model’s output distribution (Yue et al., 2025). This pivotal observation has motivated the development of power sampling, an inference-time technique that completely bypasses training by amplifying these valid paths (Karan and Du, 2026). By targeting a distribution in which each complete output sequence has a probability proportional to its base-model probability raised to an exponent greater than 1, power sampling recovers much of the gain of RL post-training. We refer to this resulting distribution over complete sequences as the power distribution. Exact power sampling is computationally intractable because it requires calculating the probabilities of all possible sequence completions (Karan and Du, 2026; Azizi et al., 2026). To approximate this target distribution, initial approaches relied on Metropolis–Hastings, a Markov Chain Monte Carlo method. Metropolis–Hastings operates by iteratively proposing full sequences and probabilistically accepting or rejecting them based on a target ratio; however, this repeated propose-and-reject cycle creates a severe serialization bottleneck, resulting in prohibitive inference latency (Karan and Du, 2026). To avoid this inefficiency, Power-SMC (Azizi et al., 2026) reformulates the problem using Sequential Monte Carlo (SMC) sampling. Rather than generating and evaluating complete sequences one by one, SMC builds solutions incrementally by maintaining a population of N parallel particles, each representing a partial token trajectory. At each decoding step, the algorithm samples tokens from a proposal distribution and assigns an importance weight to each particle. By dynamically evaluating and weighing these partial paths in parallel, SMC successfully targets the sequence-level power distribution, eliminating the latency bottlenecks of Metropolis–Hastings while maintaining competitive reasoning performance. As Power-SMC progresses through the decoding steps, it encounters a major challenge: weight degeneracy (Liu and Chen, 1998). Some particles inevitably accumulate the vast majority of the importance weight, while the rest drop near zero. To counteract this and restore the effective sample size (ESS), standard SMC employs equal-weight resampling, which removes low-weight particles, duplicating high-weight ones, and resetting all surviving weights to equality. However, mitigating weight concentration in this manner introduces a distinct problem: the severe loss of genealogical diversity. By aggressively cloning the highest-weighted particles and dropping the rest, standard equal-weight resampling drastically reduces the number of unique ancestral paths that survive to the end of the decoding process. While the weight-based ESS is restored, this increase can mask a substantial loss of genealogical diversity. This loss of path diversity is particularly detrimental in language model decoding because importance weights are a poor proxy for eventual correctness. The weight of a particle reflects its prefix’s relative mass under the power target and the proposal distribution; it does not evaluate whether that unfinished trajectory is logically sound or will yield the right answer. Consequently, standard weight-based resampling risks permanently deleting low-weight particles that actually contain the correct reasoning path, simply because they currently carry little importance mass. To address the destructive nature of standard resampling, we replace it with Chopthin (Gandy and Lau, 2016). Rather than strictly equalizing the weights in the population, Chopthin enforces a bounded ratio between the largest and smallest output weights, allowing the particles to carry these unequal weights forward in subsequent decoding steps (Figure 1). By categorizing particles by their current mass, Chopthin thus applies a much gentler, targeted intervention: light particles are probabilistically thinned, middle-weight particles pass through entirely unperturbed, and only excessively heavy particles are chopped into equal-weight subdivisions. This approach is designed to preserve more low-weight (yet potentially correct) trajectories, mitigating genealogical diversity loss while retaining unbiasedness, exact particle count, and weight conservation (Section 4.1). Preserving this richer ancestral population can increase the probability that at least one complete trajectory successfully navigates to a correct answer. We define this metric as oracle coverage, which denotes the fraction of problems for which the final population contains at least one correct answer. Although maximizing oracle coverage establishes theoretical headroom for model accuracy, this potential can only be realized if the final selection mechanism reliably identifies the correct answer among the surviving candidates. Unfortunately, standard selection rules often fail in this regard (Brown et al., 2024). For instance, Power-SMC relies by default on a weight draw strategy, which selects a final particle with a probability that is proportional to its accumulated weight. Under this rule, the low-weight, highly diverse trajectories that Chopthin preserves are the least likely to be chosen, highlighting a critical mismatch between trajectory generation and final answer selection (Section 4.3). To bridge this critical gap between trajectory generation and final answer selection, we introduce a novel decoding framework: Chopthin-Consensus Power Sampling (CCPS). CCPS fundamentally upgrades Power-SMC with two interrelated innovations that help uncover the full potential of diverse reasoning paths. First, it replaces standard equal-weight resampling with Chopthin to preserve genealogical diversity. Second, to overcome the limitations of weight-based selection, CCPS introduces a semantic majority voting mechanism to extract the correct answer from the enriched population. This mechanism first merges token-identical final trajectories so that each exact duplicate contributes one count vote, while its pooled weight is retained for tie-breaking. It then clusters semantically equivalent answers by comparing candidate solutions only with one another, without consulting a gold reference. Finally, CCPS returns the answer supported by the largest coalition of distinct reasoning trajectories. This design extends self-consistency (Wang et al., 2023) to particle-based decoding. Furthermore, the framework is highly adaptable: for code generation tasks, CCPS seamlessly clusters answers based on their functional execution on self-generated test inputs (Chen et al., 2023a), providing a robust and generalizable solution to the LLM reasoning bottleneck. ABCDE⋆ ⋆ 1a_1a1a_1a1a_1a2a_2a⋆a a2a_2R=6R=6R=4R=4R=2R=2a⋆×1a \!× 1initial populationresampling 1resampling 2answers(a) Systematic resampling (weights reset to 1/N1/N)chopkeepthinABCDE⋆ ⋆ ⋆ 1a_1a⋆a a2a_2a3a_3a⋆a a⋆a R=6R=6R=5R=5R=4R=4a⋆×3a \!× 3initial populationresampling 1resampling 2answers(b) Chopthin resampling (weights carried, ratio ≤η≤η) Figure 1: One illustrative example of systematic and Chopthin resampling on the same toy population of N=6N=6 weighted particles (circle area ∝ weight; color/letter == founding ancestor; × == killed; R == surviving root lineages). The ⋆ particle is low-weight but would reach the correct answer a⋆a ; token-identical copies are merged before voting. (a) In this realization, systematic resampling resets all weights to 1/N1/N: ⋆ dies at the first event, R collapses 6→4→26→ 4→ 2 (particle impoverishment), and the majority vote fails even though the population contains a⋆a . (b) In this realization, Chopthin carries unequal weights forward: ⋆ survives, R falls only to 44, and the vote is correct. Neither resampler guarantees preservation of a trajectory that would eventually reach the correct answer; the figure illustrates a possible mechanism, not an expected outcome. Here η bounds the ratio between the largest and smallest output weight (Section 4.1), and a1,a2,a3a_1,a_2,a_3 are distinct incorrect answers, with equal tags denoting equal answers. Our contributions are as follows. • We identify equal-weight resampling as a source of genealogical diversity loss in SMC-based reasoning and distinguish this loss from the weight concentration summarized by the ESS. • We introduce the Chopthin resampler to language-model decoding, bounding the ratio between the largest and smallest output weights and carrying the unequal weights forward, while preserving the SMC approximation in conditional expectation and guaranteeing a lower bound on post-resampling ESS. • We pair it with a deduplicate-then-cluster semantic-majority selector that prevents exact duplicate final trajectories from receiving repeated count votes and clusters programs by execution behavior for code. • We evaluate across three open models and five reasoning benchmarks: Chopthin raises oracle coverage in 1313 of 1515 settings, and CCPS matches or improves final-answer accuracy over Power-SMC in 14 of 1515, with ablations attributing the coverage gain to preserved trajectory diversity and the accuracy gain to the semantic-majority selector. 2 Background: Power sampling as sequential Monte Carlo 2.1 The sequence-level power distribution Let pθp_θ be a pretrained autoregressive language model over a vocabulary V. Given a prompt x, it assigns to a sequence y=(y1,…,yT)y=(y_1,…,y_T) terminating in an end-of-sequence (EOS) token the probability pθ(y∣x)=∏t=1Tpθ(yt∣x,y<t)p_θ(y x)= _t=1^Tp_θ(y_t x,y_<t), where y<t:=(y1,…,yt−1)y_<t:=(y_1,…,y_t-1) and “:=:=” means “is defined as”. For a sharpening exponent α>1α>1, the sequence-level power distribution is πα(y∣x)=pθ(y∣x)αZα(x),Zα(x)=∑ypθ(y∣x)α. _α(y x)= p_θ(y x)^αZ_α(x), Z_α(x)= _yp_θ(y x)^α. (1) The exponent acts on the probability of the complete sequence, moving mass toward high-likelihood sequences without changing the model parameters. Recent analyses attribute much of the reasoning gain from RL post-training to this kind of sharpening (Yue et al., 2025). Exact sampling is intractable: the normalizer in (1) sums over an exponentially large space of complete sequences. Sampling each token at temperature 1/α1/α renormalizes at every step, so the sequence distribution it induces is not πα _α (Karan and Du, 2026). 2.2 Sequential Monte Carlo over prefixes Power sampling can instead be written as Sequential Monte Carlo (SMC) (Azizi et al., 2026), using the standard construction of (Del Moral et al., 2006). The sampler keeps N particles. Particle i holds a prefix y(i)1:ty^(i)_1:t and a weight, and all N prefixes are decoded together as one batch. After t tokens, the unnormalized intermediate target is γt(y1:t∣x)=pθ(y1:t∣x)αt,αt=1+(α−1)min(t/Tramp, 1),α0:=1. _t(y_1:t x)=p_θ(y_1:t x) _t, _t=1+(α-1) \! (t/T_ramp,\,1 ), _0:=1. (2) Here γt _t is the intermediate target after t tokens: the same power construction on the first t tokens, with partial exponent αt _t. That exponent is the α-ramp: it grows linearly from 11 to α over the first TrampT_ramp tokens, then stays at α. The ramp mitigates weight degeneracy (the weight concentrating on a small number of particles) early in decoding (Azizi et al., 2026). At each step, every particle samples its next token from a proposal q(⋅∣x,y<t)q(· x,y_<t): any distribution over the next token that depends only on the prompt and the tokens generated so far. Each particle carries a cumulative unnormalized weight Ut(i)U^(i)_t, starting from U0(i)=1U^(i)_0=1. Let ωt(i)ω^(i)_t be the incremental importance weight of particle i at step t: U(i)t=U(i)t−1ω(i)t,ω(i)t=γt(y(i)1:t∣x)γt−1(y(i)1:t−1∣x)q(y(i)t∣x,y(i)<t).U^(i)_t=U^(i)_t-1\,ω^(i)_t, ω^(i)_t= _t\! (y^(i)_1:t x ) _t-1\! (y^(i)_1:t-1 x )\,q\! (y^(i)_t x,y^(i)_<t ). (3) Each increment connects one intermediate target to the next, so once the ramp is over, the weighted population targets the power distribution over reachable sequences. Appendix C gives the expanded form of (3), explains the weight updates during the ramp, and describes particles that finish early. A particle that emits EOS stops generating but remains in the population; its token-level increments become one, while the prefix-exponent correction continues during the ramp, so the population always contains N particles. 2.3 Effective sample size and the resampling interface We monitor weight degeneracy using normalized weights and the effective sample size (ESS): Wt(i)=Ut(i)∑j=1NUt(j),ESSt=(∑i=1N(Wt(i))2)−1∈[1,N],W^(i)_t= U^(i)_t _j=1^NU^(j)_t, _t= ( _i=1^N(W^(i)_t)^2 )^-1∈[1,N], (4) where i indexes the particle whose normalized weight is being computed and j is a summation index running over the whole population. ESS is evaluated at block boundaries every B decoded tokens, and the sampler resamples when ESSt<κNESS_t<κ N; κ is the trigger fraction. At a resampling event, write wi:=Wt(i)w_i:=W^(i)_t for the normalized weight of input particle i. The resampler returns N output particles, indexed by k, together with ancestor indices A1:NA_1:N, where Ak∈1,…,NA_k∈\1,…,N\, and output weights w+1:Nw^+_1:N, where ++ denotes immediately after resampling. Output particle k continues the state of input particle AkA_k and takes cumulative weight wk+w^+_k. The offspring count of input particle i is Ci=∑k=1NAk=iC_i= _k=1^N1\A_k=i\, so Ci=0C_i=0 means particle i is deleted. Multinomial, stratified (Kitagawa, 1996), residual (Liu and Chen, 1998), and systematic resampling (Whitley, 1994; Carpenter et al., 1999) differ in resampling variance (Douc and Cappé, 2005), but all equalize the output weights: wk+=1/Nw^+_k=1/N. Our method changes this resampling rule while leaving the rest of the SMC construction unchanged. 3 Motivation: weight balance is not trajectory diversity 3.1 Weight balance and trajectory diversity are different objectives The ESS (4) measures weight balance, not how many distinct trajectories survive: an equal-weight population maximizes ESS yet may consist of many copies of a few ancestors. We track this second axis with two quantities. The first is RtR_t, the number of surviving root lineages: founding ancestors that still have a descendant at time t (the genealogical diversity of the population). The second is D, the number of token-distinct complete trajectories left after exact duplicates are merged at the end of decoding, so D≤ND≤ N. The two are related but not interchangeable: separate lineages can converge to the same final trajectory, and descendants of one lineage can diverge. Token-distinct trajectories need not give distinct answers, but they preserve the opportunity for different solutions to emerge; answer-level effects are measured by our selector’s clusters and the coverage metric of Section 5. 3.2 Stochastic deletion and lineage collapse Equal-weight resampling solves weight degeneracy, and in classical filtering it works well (Douc and Cappé, 2005). For preserving reasoning trajectories it has two costs. First, stochastic deletion. Under systematic resampling, a particle with Nwi<1Nw_i<1 receives either zero or one offspring, with average NwiNw_i, so it is deleted with probability 1−Nwi1-Nw_i. Deleting by weight suits cases where current importance mass predicts future usefulness; here it does not reliably predict whether the completed trajectory will produce a correct answer, so resampling can delete trajectories that would otherwise have reached a correct solution. Second, lineage collapse. A high-weight particle receives several offspring, all starting from the same prefix, so the number of distinct prefixes can only shrink at an event; the copies begin to differ only as new tokens are sampled. Repeated events can therefore leave many particles but few independent lineages (particle impoverishment; Doucet et al., 2001): RtR_t can fall at every event, as illustrated by one possible realization in Figure 1a, and one equalizing event typically leaves far fewer unique ancestors than particles (Azizi et al., 2026). Underlying both costs is a single move: equal-weight resampling converts each particle’s continuous weight into an integer offspring count, discarding the weight information instead of carrying it forward. 3.3 Design objective We therefore seek a resampling rule that preserves the weighted target and the particle budget, guarantees control of post-resampling weight concentration, and avoids perturbing moderate particles or erasing accumulated weight information. Equal-weight resampling meets the first two demands, but it controls weight concentration in the strongest possible way (exact equality) and perturbs the whole population to get there. Exact equality is stronger than necessary: a bound on the ratio between the largest and smallest output weights already guarantees a floor on the ESS. This weaker condition defines our method. Selection needs the same care: the standard rule draws one particle in proportion to its final weight, so a correct trajectory that survives at a low weight is rarely returned. The second component of our method is a selector that uses the whole population instead (Section 4.3). 4 Chopthin-Consensus Power Sampling 4.1 Chopthin preliminaries: bounded-weight resampling Chopthin (Gandy and Lau, 2016) is an existing resampler from the bounded-weight family: instead of forcing equal output weights, it bounds how uneven they may be, max1≤k≤Nwk+/min1≤k≤Nwk+≤η, _1≤ k≤ Nw^+_k\; /\; _1≤ k≤ Nw^+_k\;≤\;η, (5) where η is a user-chosen ratio bound (for this resampler η≥4η≥ 4; Appendix A explains the constraint). A small η forces the weights closer together; a large η leaves them more uneven and perturbs the population less. All results in this subsection are from (Gandy and Lau, 2016). A bounded ratio is enough to control weight degeneracy, because it forces a floor on the ESS: Proposition 1 (ESS floor; Gandy and Lau, 2016, Lemma 2) If the output weights w+1:N>0w^+_1:N>0 satisfy (5), then ESS(w+):=(∑k=1Nwk+)2∑k=1N(wk+)2≥4(ηN+1−η2)(η+1)2.ESS(w^+)\;:=\; ( _k=1^Nw^+_k )^2 _k=1^N(w^+_k)^2\;≥\; 4 (η N+1-η^2 )(η+1)^2. (6) Chopthin takes the input weights w1:Nw_1:N and η, computes a threshold a>0a>0 from the current weights at every event (a is not a user choice), and gives particle i the expected offspring count haη(wi)h^η_a(w_i), acting on each particle as shown: haη(wi)=wi/a,wi<athin: survive with probability wi/a,at weight a,1,a≤wi<ηa/2keep: one offspring, weight unchanged,2wi/(ηa),wi≥ηa/2chop: split into equal-weight pieces.h^η_a(w_i)\;=\; \ array[]lw_i/a,&w_i<a& thin: survive with probability w_i/a,\ at weight a,\\[2.0pt] 1,&a≤ w_i<η a/2& keep: one offspring, weight unchanged,\\[2.0pt] 2w_i/(η a),&w_i≥η a/2& chop: split into equal-weight pieces. array . (7) The threshold is set so that the expected total matches the budget: ∑i=1Nhaη(wi)=N; _i=1^Nh^η_a(w_i)\;=\;N; (8) a solution exists because the total decreases as a grows. Every output weight lies in [a,ηa][a,η a], which gives (5). Particles in the keep band pass through unchanged, and if all input weights are equal, Chopthin returns every particle unchanged: the resampler perturbs the population only where the weights are uneven. The exact integer offspring counts and the weight-conservation correction are in Appendix A. The thin band is where Chopthin departs most from an equalizing scheme. The resampler returns N particles, each of weight at least a, and conserves the total weight, so Na≤∑kwk+=∑iwi=1Na≤ _kw^+_k= _iw_i=1 and hence a≤1/Na≤ 1/N. A light particle therefore survives with probability wi/a≥Nwiw_i/a≥ Nw_i, exactly its survival probability under systematic resampling (Section 3.2), so none is more likely to be deleted under Chopthin than under the equalizing baseline. Chopthin keeps the statistical validity that Section 3.3 asks for: it is unbiased (on average, the offspring of particle i carry exactly its weight wiw_i), returns exactly N output particles, and conserves the total weight (Proposition 2, Appendix A.5). Unbiasedness means the weighted target is unchanged in expectation, so Chopthin alters the resampling variance and genealogy, not the target (Proposition 3). 4.2 Carried weights and integration CCPS carries Chopthin’s output weights forward, Ut(k)←wk+U^(k)_t← w^+_k for k=1,…,Nk=1,…,N, instead of any uniform reset. SMC weights are defined only up to a common constant, so adopting the normalized outputs as the new cumulative weights is valid. Later increments (3) multiply the carried values, so the weights at the end of decoding keep information from before the last event. Decoding runs as in Section 2. At a triggered event, Chopthin is called on W(1:N)tW^(1:N)_t, the particle states (sequences, termination flags, and cached model state) are reordered by A1:NA_1:N, and the weights are carried forward as above. Algorithm 2 (Appendix B) lists the complete procedure. We set η=3+8≈5.83η=3+ 8≈ 5.83, chosen so that the asymptotic ESS floor is N/2N/2, asymptotically matching the trigger fraction κ=0.5κ=0.5 (Section 5.1); at our experimental N=32N=32 the exact guaranteed floor is 13.1713.17 (0.412N0.412N). The derivation and finite-N values are in Appendix A.6. 4.3 Semantic-majority selection A standard SMC rule is the weight draw: pick one particle with probability proportional to its normalized final weight Wfinal(i)W^(i)_final (4), and return the answer parsed from it (Doucet et al., 2001). Particle-based decoding methods for language models inherit this rule (Azizi et al., 2026; Nguyen et al., 2026). CCPS instead aggregates over the whole population. Selection proceeds in three steps: merge, cluster, and vote. Step 1: Merge. Combine token-identical final trajectories (for code, those with identical extracted programs) into a set T of distinct trajectories, so that each exact duplicate contributes one count vote; pool their final weights for tie-breaking. Each u∈u has an answer aua_u parsed from its trajectory (parsing details in Appendix D). Step 2: Cluster. Group answers the task grader judges equivalent, comparing candidates only with one another, never with the gold answer. For code, we cluster programs by their exact behavioral signatures on a fixed ordered set of test inputs generated once per problem and base model and shared across both resampling arms. Call the resulting set of clusters C. Step 3: Vote. Return the answer of the cluster supported by the largest number of distinct trajectories (distinct programs, for code), C∗=argmaxC∈|u∈:au∈C|,a^maj=rep(C∗),C \;=\; _C \; \u :a_u∈ C\ , a_maj\;=\;rep(C ), (9) where rep(C)rep(C) is the cluster representative of Appendix D, with ties between clusters broken by the sum of the pooled weights of their supporting trajectories, ∑u∈:au∈CW¯u _u :\,a_u∈ C W_u, where W¯u=∑i:y(i)=uW(i)final W_u= _i:\,y^(i)=uW^(i)_final. The output is a consensus point estimate, not a sample from the power distribution. 5 Experiments 5.1 Setup Models and benchmarks. We evaluate Qwen2.5-Math-7B (Yang et al., 2024), Qwen2.5-7B (Qwen et al., 2024), and Qwen3-4B (Yang et al., 2025) on MATH500 (Lightman et al., 2024), GSM8K (Cobbe et al., 2021), AIME 2022–2024 (AI-MO, 2024), GPQA Diamond (Rein et al., 2023), and HumanEval (Chen et al., 2021) without additional fine-tuning. Appendix A.7 gives model, benchmark, and prompting details. Method settings. The Power-SMC and Chopthin generation arms use N=32N=32, α=2α=2, proposal temperature τ=0.5τ=0.5, and ESS trigger κ=0.5κ=0.5. They use identical decoding configurations and differ only in the resampler; the Chopthin arm uses η=3+8η=3+ 8. For final selection, Power-SMC uses a weight draw, whereas CCPS uses semantic-majority selection. Complete settings appear in Appendix A.7. Correctness and coverage. An answer is correct if it is equivalent to the gold answer under the task-specific grader of Appendix D. To measure what the final population contains, independently of any selector, we report for each problem the coverage indicator c=maxig(a(i),a⋆)c= _ig (a^(i),a ), where a⋆a is the gold answer and g∈0,1g∈\0,1\ is the task grader; the oracle coverage is the average of c over problems. Oracle coverage is the at-least-one-correct rate over the correlated population of a single fixed-compute run. The final trajectories share ancestry through resampling and can include exact duplicates, so this is not an i.i.d. pass@N. It is a ceiling for any selector that must return one of the final answers. For HumanEval, which has no symbolic grader, we score a program correct when it passes the held-out tests; the selector never sees these tests (Section 4.3). 5.2 Comparison study Model Resampler ++ selector MATH500 GSM8K AIME GPQA HumanEval Qwen2.5-Math-7B Systematic ++ weight draw (Power-SMC) 77.0 89.5 15.6 30.3 58.5 Systematic ++ majority 79.4 91.5 15.6 31.3 61.0 Chopthin ++ weight draw 78.0 88.8 15.6 31.8 58.5 Chopthin ++ majority (CCPS, ours) 81.4 90.8 16.7 33.8 61.6 Qwen2.5-7B Systematic ++ weight draw (Power-SMC) 74.2 91.0 10.4 29.3 73.2 Systematic ++ majority 76.6 91.5 11.1 31.3 76.2 Chopthin ++ weight draw 73.2 90.7 11.5 27.8 75.0 Chopthin ++ majority (CCPS, ours) 76.0 91.4 12.2 30.3 76.8 Qwen3-4B Systematic ++ weight draw (Power-SMC) 79.0 90.1 15.6 29.8 71.3 Systematic ++ majority 81.0 91.7 15.6 34.3 70.7 Chopthin ++ weight draw 81.8 89.8 16.7 34.3 70.7 Chopthin ++ majority (CCPS, ours) 81.8 92.1 15.6 40.4 70.7 Table 1: Selector ablation: final-answer accuracy (%) for each resampler × selector combination. Green rows reproduce the corresponding entries of Table 2; bold marks the best value per column within each model block. −20246Δ oracle coverage (p)MATH500GSM8KAIMEGPQAHumanEval +2.4+1.8+1.0+0.4+0.7+0.2+4.40.0+2.2+4.0−1.5+5.6+4.3+1.3+1.2 Figure 2: Oracle-coverage difference between Chopthin and systematic resampling across all settings. Color encodes the model: Qwen2.5-Math-7B, Qwen2.5-7B, Qwen3-4B. Table 2: Final-answer accuracy (%) for CCPS against Power-SMC and training-free decoding baselines. Model Method MATH500 GSM8K AIME GPQA HumanEval Qwen2.5-Math-7B Baseline decoding 65.0 80.0 11.1 20.7 20.7 Low-temperature decoding (τ=1/ατ=1/α) 69.8 88.2 8.9 32.8 30.5 MH power sampling (Karan and Du, 2026) 74.8†74.8 81.5‡81.5 9.5§9.5 @sectionsign 38.9†38.9 57.3†57.3 Power-SMC (systematic) 77.0 89.5 15.6 30.3 58.5 CCPS (ours) 81.4 90.8 16.7 33.8 61.6 Qwen2.5-7B Baseline decoding 50.8 82.2 1.1 30.3 28.0 Low-temperature decoding (τ=1/ατ=1/α) 61.2 88.4 6.7 29.3 7.9 MH power sampling (Karan and Du, 2026) 70.6†70.6 84.5‡84.5 8.2§8.2 @sectionsign 31.8†31.8 62.2†62.2 Power-SMC (systematic) 74.2 91.0 10.4 29.3 73.2 CCPS (ours) 76.0 91.4 12.2 30.3 76.8 Qwen3-4B Baseline decoding 82.0 91.8 18.9 39.9 63.4 Low-temperature decoding (τ=1/ατ=1/α) 80.4 91.2 21.1 36.4 55.5 MH power sampling (Karan and Du, 2026) 76.2‡76.2 90.6‡90.6 9.59.5 36.4‡36.4 45.7‡45.7 Power-SMC (systematic) 79.0 90.1 15.6 29.8 71.3 CCPS (ours) 81.8 92.1 15.6 40.4 70.7 Bold numeric entries mark the highest reported value in each model–benchmark block. Green rows indicate our method. †Reported by (Karan and Du, 2026) (α=4α=4). ‡Reported by (Azizi et al., 2026) (α=4α=4). § @sectionsignReported by (Zhou et al., 2026) on AIME. Chopthin raises oracle coverage: Figure 2 shows it lifts the ceiling over Power-SMC in thirteen of the fifteen cells, ties in one, and dips in only one, by at most 1.51.5 percentage points (absolute values in Table 3, Appendix E.1). The semantic-majority selector then converts that headroom into accuracy: together they match or improve final-answer accuracy over the full Power-SMC pipeline (systematic resampling ++ weight draw) in 1414 of 1515 cells, by up to 10.610.6 points (Qwen3-4B on GPQA); the single exception is Qwen3-4B on HumanEval (−0.6-0.6). Table 2 reports these results alongside Power-SMC and training-free decoding baselines. Table 2 reports the full resampler × selector grid. No single selector is best in every cell, but the pattern is selector-robust: semantic majority matches or improves on the weight draw under both resamplers in 2828 of 3030 comparisons (the exceptions are Qwen3-4B on HumanEval under systematic resampling and on AIME under Chopthin). Chopthin’s contribution, by contrast, is the ceiling: it places a correct answer among the N trajectories more often (Figure 2), and with the selector held at majority it wins 99 of 1515 cells, ties 22, and loses 44. Baseline decoding (temperature 11) and low-temperature decoding (τ=1/ατ=1/α) are single-sample references with no resampling. The Metropolis–Hastings (MH) row collects reference values reported at α=4α=4 in the literature, marked per source in the table footnote. We do not tabulate methods available only at other sampling configurations or without public code (Ji et al., 2026; Nguyen et al., 2026). Reported GRPO reference values are collected in Appendix E.2 (Shao et al., 2024; Nguyen et al., 2026). 5.3 Discussion Resampling and selection play distinct roles. A higher ceiling is not automatically a better answer. Under the weight draw, Chopthin’s final-answer accuracy tracks Power-SMC’s and sometimes falls below it despite higher oracle coverage: the weights the draw follows do not, on these benchmarks, favor the correct trajectories. The semantic-majority selector counts distinct trajectories instead of trusting the weights, which is what turns the higher coverage into the gains of Table 2. Chopthin’s more consistent effect is on oracle coverage. Qwen3-4B on MATH500 shows the limit: baseline decoding scores 82.082.0, above Power-SMC’s 79.079.0 and CCPS’s 81.881.8 (Table 2), yet CCPS oracle coverage is 84.884.8, the final population holds correct trajectories the current selector does not return. Arzhantsev et al. (2026) report a similar pattern for Metropolis–Hastings. 6 Ablations Chopthin has two degrees of freedom: the ratio bound η, and the choice to carry the unequal output weights forward instead of resetting them. We ablate both on the primary cell (Qwen2.5-Math-7B on MATH500), using the configuration of Section 5.1. 6.1 Sensitivity to the ratio bound η The bound is set by inverting the ESS floor of Proposition 1 at a target fraction ρ (Appendix A.6), so we sweep ρ and report the induced η (Figure 3), with the trigger fixed at κ=0.5κ=0.5. All arms are identical to CCPS except η. 788286Accuracy /coverage (%)87.085.481.4%80.081.2Majority acc.Oracle coverage121518D14.715.215.416.017.244ρ=0.64ρ=0.643+83+ 8ρ=0.50ρ=0.507.877.87ρ=0.40ρ=0.4011.2411.24ρ=0.30ρ=0.3024.9924.99ρ=0.148ρ=0.148ratio bound η (a) sensitivity to η 788286Accuracy /coverage (%)84.685.887.081.4%79.480.6Majority acc.Oracle coverage121518D13.015.215.2SystematicChopthin++ resetChopthincarried (ours)resampler (b) carried weights vs. uniform reset Figure 3: Ablations on the primary cell (Qwen2.5-Math-7B, MATH500). (a) The green marker is the default η=3+8η=3+ 8. Top: final-answer accuracy of CCPS (blue) and oracle coverage (dark squares); bottom: distinct final trajectories D out of N. Accuracy varies within 1.41.4 points and coverage within 1.61.6 across the range. (b) The hybrid applies Chopthin’s allocation (7) then resets every weight to 1/N1/N: a diagnostic, not a valid resampler. Bold marks the best value per column. The method is robust to its one free parameter: every value of η keeps oracle coverage above the systematic baseline of Figure 3(b) (84.684.6, a margin of +0.8+0.8 to +2.4+2.4). A looser bound perturbs the population less and preserves more distinct trajectories, which is the mechanism the method relies on. The trigger-matched default ρ=0.5ρ=0.5 attains the best majority accuracy, so the theoretically motivated choice of Section 4.1 is also the empirical optimum; we tuned η no further. 6.2 Carried weights versus uniform reset Chopthin changes both who survives a resampling event (probabilistic survival of low-weight particles and the untouched keep band) and what the survivors remember (the carried weights). A hybrid that applies Chopthin’s offspring allocation (7) but resets every output weight to 1/N1/N disentangles the two. Chopthin’s unbiasedness holds for the product of offspring count and output weight (Eq. 10), so resetting the weights to 1/N1/N without adjusting the counts breaks that unbiasedness. Figure 3(b) reports the comparison. In this diagnostic, roughly half of the observed gain over the equalizing baseline comes from the offspring allocation alone: the hybrid preserves the same mean number of distinct trajectories as carried-weight Chopthin (15.215.2), compared with 13.013.0 under systematic resampling, and lifts both final-answer accuracy and oracle coverage. Carrying the unequal weights adds a further increment (+0.8+0.8 majority accuracy and +1.2+1.2 oracle coverage). 7 Related work Particle-based inference-time methods for LLMs. Twisted SMC (Zhao et al., 2024), SMC steering and controlled generation (Lew et al., 2023; Loula et al., 2025), reward-guided particle filtering (Puri et al., 2025), and SMC power sampling (Azizi et al., 2026) all re-equalize the particle weights at every resampling event.11 1 The nearest exception is the without-replacement scheme of (Lew et al., 2023), related to (Fearnhead and Clifford, 2003). To our knowledge, no existing particle-based inference-time method for language models bounds post-resampling weight unevenness or guarantees a post-resampling effective sample size. Recent diversity-preserving efforts act around an equalizing resampler rather than replacing it, by tempering the weights before resampling (Giannone et al., 2025), adding rollout- or value-based auxiliary weights (Nguyen et al., 2026), weighting by intrinsic uncertainty (Giannone et al., 2026), enlarging the eligible pool with reward-scored historical prefixes (Tran et al., 2026), or restoring diversity after an equalizing resample with Metropolis–Hastings rejuvenation moves (Markovic-Voronov et al., 2026). These approaches change which particles are favored; we change how much the population is perturbed, with no extra rollouts, learned potentials, or reward models. Answer selection. Self-consistency returns the most common answer among independent samples (Wang et al., 2023); universal self-consistency and semantic-uncertainty methods cluster free-form answers by meaning before aggregating (Chen et al., 2023b; Kuhn et al., 2023). For code, AlphaCode clusters programs by their behavior on generated inputs (Li et al., 2022), and CodeT selects candidates by agreement on model-generated tests (Chen et al., 2023a). (Brown et al., 2024) show that the coverage of repeated sampling grows with the number of samples while common selectors often fail to realize it. Most recently, marginal sharpening (Arzhantsev et al., 2026) makes the sharpened marginal distribution over answers itself the inference-time target, with self-consistency as its limiting case. We keep Power-SMC’s sequence-level target and act on the resampler and selector instead, so the approaches are complementary; our selector adapts these ideas to a resampled population, where duplicates must be merged before any vote is meaningful (Section 4.3). 8 Conclusion We identified systematic resampling as an obstacle to reasoning performance in SMC-based power sampling. It equalizes particle weights and drops low-weight trajectories that, though unlikely early, can still reach the correct answer. Replacing it with Chopthin gives low-weight trajectories a greater chance of remaining represented and increases observed oracle coverage in 1313 of 1515 settings across three open models and five benchmarks. Pairing Chopthin with the semantic-majority selector matches or improves final-answer accuracy over the Power-SMC baseline in 1414 of 1515 settings, by up to 10.610.6 percentage points. Because the resampling change is independent of how tokens are proposed and operates at the standard resampling interface, it may be combined with other SMC-based decoding methods. Evaluating such combinations is future work. LLM usage LLMs were used only for editing assistance; the authors take full responsibility for all content. References AI-MO (2024) AI-MO AIMO validation AIME. Note: https://huggingface.co/datasets/AI-MO/aimo-validation-aimeHugging Face dataset; AIME 2022–2024 problems, sourced from the Art of Problem Solving Cited by: §5.1. Arzhantsev et al. (2026) A. Arzhantsev, O. Sakhi, and N. Chopin Self-consistency via marginal sharpening. External Links: 2605.28142 Cited by: §5.3, §7. Azizi et al. (2026) S. Azizi, E. Baghaei Potraghloo, M. Ahmadi, S. Kundu, and M. Pedram Power-SMC: low-latency sequence-level power sampling for training-free LLM reasoning. External Links: 2602.10273 Cited by: §A.7, Appendix C, Appendix C, §1, §2.2, §2.2, §3.2, §4.3, Table 2, §7. Brown et al. (2024) B. Brown, J. Juravsky, R. Ehrlich, R. Clark, Q. V. Le, C. Ré, and A. Mirhoseini Large language monkeys: scaling inference compute with repeated sampling. External Links: 2407.21787 Cited by: §1, §7. Carpenter et al. (1999) J. Carpenter, P. Clifford, and P. Fearnhead Improved particle filter for nonlinear problems. IEE Proceedings - Radar, Sonar and Navigation 146 (1), p. 2–7. Cited by: §2.3. Chen et al. (2023a) B. Chen, F. Zhang, A. Nguyen, D. Zan, Z. Lin, J. Lou, and W. Chen CodeT: code generation with generated tests. Note: ICLR 2023 External Links: 2207.10397 Cited by: Appendix D, §1, §7. Chen et al. (2021) M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, et al. Evaluating large language models trained on code. External Links: 2107.03374 Cited by: §5.1. Chen et al. (2023b) X. Chen, R. Aksitov, U. Alon, J. Ren, K. Xiao, P. Yin, S. Prakash, C. Sutton, X. Wang, and D. Zhou Universal self-consistency for large language model generation. arXiv preprint arXiv:2311.17311. Cited by: §7. Cobbe et al. (2021) K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman Training verifiers to solve math word problems. External Links: 2110.14168 Cited by: §5.1. Del Moral et al. (2006) P. Del Moral, A. Doucet, and A. Jasra Sequential Monte Carlo samplers. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 68 (3), p. 411–436. Cited by: Appendix C, §2.2. Del Moral (2004) P. Del Moral Feynman-Kac formulae: genealogical and interacting particle systems with applications. Springer. Cited by: §A.5. Douc and Cappé (2005) R. Douc and O. Cappé Comparison of resampling schemes for particle filtering. In Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis (ISPA), p. 64–69. Cited by: §A.5, §2.3, §3.2. A. Doucet, N. de Freitas, and N. Gordon (Eds.) (2001) A. Doucet, N. de Freitas, and N. Gordon (Eds.) Sequential Monte Carlo methods in practice. Springer. Cited by: §3.2, §4.3. Fearnhead and Clifford (2003) P. Fearnhead and P. Clifford On-line inference for hidden Markov models via particle filters. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 65 (4), p. 887–899. Cited by: footnote 1. Gandy and Lau (2016) A. Gandy and F. D. Lau The chopthin algorithm for resampling. Note: IEEE Transactions on Signal Processing 64(16):4273–4281 External Links: 1502.07532 Cited by: §A.3, §A.4, §A.7, §1, §4.1, §4.1, Proposition 1, Algorithm 1. Giannone et al. (2026) G. Giannone, M. Eyceoz, S. Baig, S. Sudalairaj, A. C. Doris, F. Ahmed, A. Srivastava, and K. Xu Intrinsic selection and particle resampling for inference-time scaling beyond domain verifiability. External Links: 2606.08850 Cited by: §7. Giannone et al. (2025) G. Giannone, G. Xu, N. S. Nayak, R. M. Awhad, S. Sudalairaj, K. Xu, and A. Srivastava Mitigating premature exploitation in particle-based monte carlo for inference-time scaling. arXiv preprint arXiv:2510.05825. Cited by: §7. Hendrycks et al. (2021) D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt Measuring mathematical problem solving with the MATH dataset. External Links: 2103.03874 Cited by: Appendix D. Holtzman et al. (2020) A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi The curious case of neural text degeneration. In International Conference on Learning Representations (ICLR), Cited by: §A.7. Ji et al. (2026) X. Ji, R. Tutunov, M. Zimmer, and H. Bou Ammar Scalable power sampling: unlocking efficient, training-free reasoning for LLMs via distribution sharpening. External Links: 2601.21590 Cited by: §5.2. Karan and Du (2026) A. Karan and Y. Du Reasoning with sampling: your base model is smarter than you think. In The Fourteenth International Conference on Learning Representations (ICLR), Note: arXiv:2510.14901 Cited by: Table 4, §1, §1, §2.1, Table 2, Table 2, Table 2, Table 2. Kitagawa (1996) G. Kitagawa Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. Journal of Computational and Graphical Statistics 5 (1), p. 1–25. Cited by: §2.3. Kuhn et al. (2023) L. Kuhn, Y. Gal, and S. Farquhar Semantic uncertainty: linguistic invariances for uncertainty estimation in natural language generation. In The Eleventh International Conference on Learning Representations (ICLR), Cited by: §7. Lew et al. (2023) A. K. Lew, T. Zhi-Xuan, G. Grand, and V. K. Mansinghka Sequential Monte Carlo steering of large language models using probabilistic programs. External Links: 2306.03081 Cited by: §7, footnote 1. Lewkowycz et al. (2022) A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, Y. Wu, B. Neyshabur, G. Gur-Ari, and V. Misra Solving quantitative reasoning problems with language models. In Advances in Neural Information Processing Systems, Vol. 35. Cited by: Appendix D. Li et al. (2022) Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. Dal Lago, T. Hubert, P. Choy, C. de Masson d’Autume, I. Babuschkin, X. Chen, P. Huang, J. Welbl, S. Gowal, A. Cherepanov, J. Molloy, D. J. Mankowitz, E. Sutherland Robson, P. Kohli, N. de Freitas, K. Kavukcuoglu, and O. Vinyals Competition-level code generation with AlphaCode. Science 378 (6624), p. 1092–1097. Cited by: Appendix D, §7. Lightman et al. (2024) H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe Let’s verify step by step. In The Twelfth International Conference on Learning Representations (ICLR), Note: arXiv:2305.20050 Cited by: Appendix D, §5.1. Liu and Chen (1998) J. S. Liu and R. Chen Sequential Monte Carlo methods for dynamic systems. Journal of the American Statistical Association 93 (443), p. 1032–1044. Cited by: §1, §2.3. Loula et al. (2025) J. Loula, B. LeBrun, L. Du, B. Lipkin, C. Pasti, G. Grand, T. Liu, Y. Emara, M. Freedman, J. Eisner, R. Cotterell, V. Mansinghka, A. K. Lew, T. Vieira, and T. J. O’Donnell Syntactic and semantic control of large language models via sequential Monte Carlo. In The Thirteenth International Conference on Learning Representations (ICLR), Note: arXiv:2504.13139 Cited by: §7. Markovic-Voronov et al. (2026) J. Markovic-Voronov, W. Zhu, B. Long, Z. Wang, S. Gupta, K. Behdin, B. Chen, and D. Agarwal Sampling for quality: training-free reward-guided LLM decoding via sequential Monte Carlo. External Links: 2604.16453 Cited by: §7. Nguyen et al. (2026) T. Nguyen, M. Zimmer, R. Tutunov, X. Ji, and H. Bou Ammar The model knows, the decoder finds: future value guided particle power sampling. External Links: 2605.02427 Cited by: Table 4, §4.3, §5.2, §7. Puri et al. (2025) I. Puri, S. Sudalairaj, G. Xu, K. Xu, and A. Srivastava A probabilistic inference approach to inference-time scaling of LLMs using particle-based Monte Carlo methods. In The Thirty-ninth Annual Conference on Neural Information Processing Systems (NeurIPS), Note: arXiv:2502.01618 Cited by: §7. Qwen et al. (2024) Qwen, A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, et al. Qwen2.5 technical report. External Links: 2412.15115 Cited by: §5.1. Rein et al. (2023) D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman GPQA: a graduate-level google-proof Q&A benchmark. External Links: 2311.12022 Cited by: §5.1. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo DeepSeekMath: pushing the limits of mathematical reasoning in open language models. External Links: 2402.03300 Cited by: Table 4, §1, §5.2. Tran et al. (2026) D. Tran, D. A. Le, N. Luu, Q. Pham, T. Pham, and H. Bui Beyond the frontier: stochastic backtracking for efficient test-time scaling. External Links: 2605.25143 Cited by: §7. Wang et al. (2023) X. Wang, J. Wei, D. Schuurmans, Q. V. Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations (ICLR), Cited by: §1, §7. Whitley (1994) D. Whitley A genetic algorithm tutorial. Statistics and Computing 4 (2), p. 65–85. Cited by: §2.3. Yang et al. (2025) A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. Qwen3 technical report. External Links: 2505.09388 Cited by: §5.1. Yang et al. (2024) A. Yang, B. Zhang, B. Hui, B. Gao, B. Yu, C. Li, D. Liu, J. Tu, J. Zhou, J. Lin, K. Dang, et al. Qwen2.5-Math technical report: toward mathematical expert model via self-improvement. External Links: 2409.12122 Cited by: §5.1. Yue et al. (2025) Y. Yue, Z. Chen, R. Lu, A. Zhao, Z. Wang, Y. Yue, S. Song, and G. Huang Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?. Note: NeurIPS 2025 External Links: 2504.13837 Cited by: §1, §2.1. Zhao et al. (2024) S. Zhao, R. Brekelmans, A. Makhzani, and R. Grosse Probabilistic inference in language models via twisted sequential Monte Carlo. In Proceedings of the 41st International Conference on Machine Learning (ICML), Cited by: §7. Zhou et al. (2026) F. Zhou, A. Mehrotra, and Q. C. Liu Reasoning with sampling: cutting at decision points. External Links: 2605.30327 Cited by: Table 2. Appendix A Details of the Chopthin resampler The main text uses only Chopthin’s interface (ancestor indices and output weights), the three-band behavior (7), and the guarantees of Propositions 2 and 1; the details here are what realize them. A.1 Expected-offspring function and threshold The expected-offspring function haηh^η_a of (7) is valid for η≥4η≥ 4; the map a↦∑ihaη(wi)a _ih^η_a(w_i) is continuous and non-increasing, diverges as a→0+a→ 0^+, and vanishes as a→∞a→∞, so a solution of the threshold equation (8) exists. The threshold partitions the particles into light particles L=i:wi<aL=\i:w_i<a\, which are thinned, and heavy particles H=i:wi≥aH=\i:w_i≥ a\, which are kept or chopped; particles with a≤wi<ηa/2a≤ w_i<η a/2 have haη(wi)=1h^η_a(w_i)=1 and pass through unchanged. When the solution set of (8) is an interval, our implementation takes its upper endpoint, so boundary particles with wi=aw_i=a fall on the heavy side; the equal-weights property of Section 4.1 relies on this convention. In particular, for equal weights the upper endpoint is a=wia=w_i: every particle falls in the keep band with expected offspring count one, deterministically, which yields the property. A.2 Thinning and chopping Thinning. A light particle has haη(wi)=wi/a<1h^η_a(w_i)=w_i/a<1 and receives zero or one offspring. The survivors are drawn by a single systematic pass over the expected counts wi/ai∈L\w_i/a\_i∈ L, so particle i survives with probability wi/aw_i/a, and a survivor is assigned the weight a. Let NLN_L be the number of survivors. Chopping. A heavy particle receives Ci=⌊haη(wi)⌋+miC_i= h^η_a(w_i) +m_i offspring: the integer parts are guaranteed, and the residual NU=N−NL−∑i∈H⌊haη(wi)⌋N_U=N-N_L- _i∈ H h^η_a(w_i) offspring are allocated by a second systematic draw over the fractional parts haη(wi)\h^η_a(w_i)\, giving mi∈0,1m_i∈\0,1\ (here x:=x−⌊x⌋\x\:=x- x denotes the fractional part). Because NLN_L is random, the weight transferred to the light side deviates from its expectation; the constant ζ=∑i∈Lwi−aNL∑i∈Hhaη(wi)ζ= _i∈ Lw_i-aN_L _i∈ H\h^η_a(w_i)\ redistributes this deviation over the heavy particles. A heavy particle with Ci=c>0C_i=c>0 is split into c equal pieces, each of weight w^i/c w_i/c with w^i=wi+ζhaη(wi) w_i=w_i+ζ\h^η_a(w_i)\. Since [ζ]=0E[ζ]=0 the step stays unbiased, and the correction makes the total weight exactly conserved in exact arithmetic (weight conservation, Proposition 2); in implementation, conservation holds to numerical tolerance. Every output weight lies in [a,ηa][a,η a], which gives the bounded ratio (5). A.3 Pseudocode Algorithm 1 assembles the steps above; it is adapted from (Gandy and Lau, 2016), in our notation and with an explicit guard for the case in which every heavy expected count is an integer (then NU=0N_U=0 and ζ is set to zero). Algorithm 1 The Chopthin resampler, adapted from (Gandy and Lau, 2016). 1: weights w1:Nw_1:N (sum >0>0), ratio bound η≥4η≥ 4, target count N 2: ancestors A1:NA_1:N, weights w+1:Nw^+_1:N with maxkwk+/minkwk+≤η _kw^+_k/ _kw^+_k≤η 3: solve ∑ihaη(wi)=N _ih^η_a(w_i)=N for a ⊳ upper endpoint 4: L←i:wi<aL←\i:w_i<a\; H←i:wi≥aH←\i:w_i≥ a\ 5: draw u∼(0,1)u (0,1) ⊳ thin: systematic over wi/ai∈L\w_i/a\_i∈ L 6: for i∈Li∈ L do 7: u←u+wi/au← u+w_i/a 8: if u≥1u≥ 1 then 9: emit (A=i,w+=a)(A=i,\ w^+=a); u←u−1u← u-1 10: end if 11: end for 12: NL←N_L← number of survivors 13: NU←N−NL−∑i∈H⌊haη(wi)⌋N_U← N-N_L- _i∈ H h^η_a(w_i) 14: if ∑i∈Hhaη(wi)=0 _i∈ H\h^η_a(w_i)\=0 then⊳ all heavy counts integral; then NU=0N_U=0 15: ζ←0ζ← 0; mi←0m_i← 0 for all i∈Hi∈ H 16: else 17: ζ←(∑i∈Lwi−aNL)/∑i∈Hhaη(wi)ζ← ( _i∈ Lw_i-aN_L ) / _i∈ H\h^η_a(w_i)\ 18: allocate NUN_U residual offspring mi∈0,1m_i∈\0,1\ by a systematic draw over haη(wi)\h^η_a(w_i)\, i∈Hi∈ H 19: end if 20: for i∈Hi∈ H do 21: c←⌊haη(wi)⌋+mic← h^η_a(w_i) +m_i 22: if c>0c>0 then 23: emit c copies of (A=i,w+=(wi+ζhaη(wi))/c) (A=i,\ w^+=(w_i+ζ\h^η_a(w_i)\)/c ) 24: end if 25: end for A.4 Why η≥4η≥ 4 The condition η≥4η≥ 4 arises from requiring every admissible split to keep its pieces inside [a,ηa][a,η a]: the constraint 2w/(ηa)≤⌊w/a⌋2w/(η a)≤ w/a for all w≥aw≥ a, evaluated for w just below 2a2a, forces η≥4η≥ 4 (Gandy and Lau, 2016). A.5 Target preservation Proposition 2 (Chopthin guarantees) Given the input weights: unbiasedness, [∑k:Ak=iwk+|w1:N]=wifor every iE [ _k:\,A_k=iw^+_k\; |\;w_1:N ]\;=\;w_i every i (10) (on average, the offspring of particle i carry exactly its weight); exact count, N output particles; and weight conservation, ∑kwk+=∑iwi _kw^+_k= _iw_i. Unbiasedness (10) implies that replacing the resampler does not change what the weighted population estimates. Proposition 3 (Target preservation) Let w1:Nw_1:N be the input weights of particles X1:NX_1:N, let ℱ=σ(X1:N,w1:N)F=σ(X_1:N,w_1:N) be the pre-resampling σ-field, and let (A1:N,w1:N+)(A_1:N,w^+_1:N) be Chopthin’s output. For any bounded test function φ , [∑k=1Nwk+φ(XAk)|ℱ]=∑i=1Nwiφ(Xi).E [ _k=1^Nw^+_k\, (X_A_k)\; |\;F ]= _i=1^Nw_i\, (X_i). (11) Proof. All offspring of parent i carry the same weight, say viv_i, and there are CiC_i of them. Grouping the output sum by parent gives ∑kwk+φ(XAk)=∑iCiviφ(Xi) _kw^+_k\, (X_A_k)= _iC_iv_i\, (X_i); each φ(Xi) (X_i) is ℱF-measurable. The resampler uses only the weights and external randomness, so unbiasedness holds conditional on ℱF: [Civi∣ℱ]=wiE[C_iv_i ]=w_i, which yields (11). The same argument applies to any unbiased resampler. □ Thus Chopthin leaves the weighted empirical approximation of the intermediate targets unchanged in conditional expectation; it changes the resampling variance and the genealogy, not the target (Del Moral, 2004; Douc and Cappé, 2005). A finite population remains an approximation of πα _α, not an exact draw from it. A.6 Our choice of η The floor of Proposition 1 has leading term 4ηN/(η+1)24η N/(η+1)^2 for large N. Setting this equal to a target fraction ρNρ N of the population and solving for η gives η=(2−ρ+21−ρ)/ρη=(2-ρ+2 1-ρ)/ρ. We target ρ=0.5ρ=0.5, which yields η=3+8≈5.83η=3+ 8≈ 5.83 and satisfies the validity condition η≥4η≥ 4. Setting η=4η=4 in the inversion gives ρ=16/25ρ=16/25, so target fractions ρ>16/25ρ>16/25 would require η<4η<4 and are not attainable under (7). The finite-N floor sits a constant 22≈2.832 2≈ 2.83 particles below the asymptote: at η=3+8η=3+ 8 the bound is exactly N/2−22N/2-2 2, so at N=32N=32 it is ESS≥13.17ESS≥ 13.17 (0.412N0.412N), approaching 0.5N0.5N as N grows. A.7 Solver and integration into Power-SMC Models, benchmarks, and prompting. We use the Hugging Face releases Qwen/Qwen2.5-Math-7B and Qwen/Qwen2.5-7B (base variants) and Qwen/Qwen3-4B (the post-trained release; the separate -Base checkpoint is not used), in bfloat16; Qwen3-4B is prompted through its chat template with the thinking mode disabled. The five benchmarks are MATH500 (500500 problems) and GSM8K (13191319) for mathematics, AIME (competition mathematics; the 9090 problems of AIME 2022–2024), GPQA (graduate-level science, four-way multiple choice; the Diamond split, 198198 problems), and HumanEval (code, 164164 problems). Decoding configuration. The proposal is q(⋅∣x,y<t)∝pθ(⋅∣x,y<t)1/τq(· x,y_<t) p_θ(· x,y_<t)^1/τ with τ=1/α=0.5τ=1/α=0.5, under nucleus truncation (Holtzman et al., 2020) at top-p=0.9top-p=0.9. The exponent is ramped from 11 to 22 over the first 100100 tokens, decoding runs for up to 40964096 new tokens, and the ESS trigger κ=0.5κ=0.5 is evaluated at block boundaries every B=64B=64 tokens. The Power-SMC (systematic) and Chopthin arms are identical except for the resampler; Chopthin uses η=3+8η=3+ 8. We use α=2α=2 rather than a larger exponent because at N=32N=32 it sharpens the target while leaving the weight dispersion that resampling acts on; all method components are otherwise independent of α. We solve the threshold equation (8) by bisection on the predicate ∑ihaη(wi)≥N _ih^η_a(w_i)≥ N: an enclosing bracket is obtained by doubling and halving from maxiwi _iw_i, and the bisection converges to a tolerance of 10−1210^-12 to the supremum of the solution set, i.e., to the upper endpoint when the set is an interval; the weights are processed in double precision on the host. (Gandy and Lau, 2016) give an algorithm that solves (8) in expected linear time, but at our population sizes (N≤128N≤ 128) the simpler bisection suffices. Measured at N=32N=32 across three weight regimes, the full Chopthin call (root solve, thinning, and chopping) takes 0.190.19–0.260.26 ms per event, against roughly 1010–1515 ms per batched token step and at least B=64B=64 token steps between events; the resampling step therefore costs about 2%2\% of one forward step and under 5×10−45× 10^-4 of the wall-clock time of the block it serves (systematic resampling is simpler still). Both resamplers expose the same ancestor-index interface, so all downstream state management is shared verbatim: particle sequences, termination flags, and the Transformer KV caches are reordered by A1:NA_1:N, and the copy-on-write cache handling of (Azizi et al., 2026) applies to Chopthin without modification. Two further decoding details apply to all runs: the end-of-sequence token is masked in the proposal for the first 100100 generated tokens, and the boxed-answer stopping rule is evaluated at every step over a trailing window of 256256 tokens, so a particle may stop before the length cap. Nucleus truncation, EOS masking, and the stopping rule restrict the sampler’s support, so it targets πα _α over the reachable trajectories rather than all sequences; both arms share these choices, so the comparison is unaffected. All runs use single NVIDIA RTX A6000 GPUs (48 GB), PyTorch 2.5.1, and Transformers 4.45.0. Paired randomness. The token-sampling and resampling streams (the latter covering the resampler’s internal draws and the final weight draw) are initialized identically per problem in both arms, so the two arms produce identical token streams until their first divergent resampling event and comparisons are paired at the problem level. Appendix B CCPS pseudocode Algorithm 2 gives the complete sampler. It differs from Power-SMC in exactly two places: the resampling step (lines 11–13), which invokes Chopthin and carries its output weights, and the selection stage (lines 17–20). Algorithm 2 Chopthin-Consensus Power Sampling (CCPS, ours). 1: prompt x; model pθp_θ; exponent α; particles N; ratio bound η; trigger κ; block size B; ramp TrampT_ramp; max tokens TmaxT_ ; proposal q(⋅∣x,y<t)∝pθ(⋅∣x,y<t)1/τq(· x,y_<t) p_θ(· x,y_<t)^1/τ, τ=1/ατ=1/α, nucleus-truncated 2: answer a^maj a_maj 3: y(i)←∅y^(i)← ; U(i)←1U^(i)← 1; done(i)←falsedone^(i) for i=1,…,Ni=1,…,N 4: for t=1t=1 to TmaxT_ do⊳ decode N particles as one batch 5: while the ramp is active, multiply each particle absorbed before step t by the prefix factor pθ(y(i)∣x)αt−αt−1p_θ(y^(i) x) _t- _t-1 ⊳ Appendix C 6: for each active particle i do 7: sample yt(i)∼q(⋅∣x,y<t(i))y^(i)_t q(· x,y^(i)_<t); U(i)←U(i)⋅ωt(i)U^(i)← U^(i)·ω^(i)_t ⊳ Eq. (3) 8: if EOS or stopping rule then done(i)←truedone^(i) ⊳ stops generating; stays in the population 9: end for 10: if tmodB=0t B=0 and ESSt<κNESS_t<κ N then ⊳ Eq. (4) 11: (A1:N,w1:N+)←Chopthin(Wt(1:N),η,N)(A_1:N,w^+_1:N)← Chopthin (W^(1:N)_t,η,N ) ⊳ Alg. 1 12: reorder sequences, flags, and KV caches by A1:NA_1:N 13: U(k)←wk+U^(k)← w^+_k for all k ⊳ carry the weights; no uniform reset 14: end if 15: end for 16: Wfinal(i)←U(i)/∑jU(j)W^(i)_final← U^(i)/ _jU^(j) for all i ⊳ normalize final weights 17: ←T← deduplicate token-identical trajectories; pool weights W¯u W_u 18: ←C← cluster answers by grader or behavioral equivalence 19: C∗←argmaxC∈|u∈:au∈C|C ← _C \u :a_u∈ C\ ⊳ ties by ∑u∈:au∈CW¯u _u :\,a_u∈ C W_u; Sec. 4.3 20: return a^maj←rep(C∗) a_maj (C ) Appendix C Power-SMC incremental weights and the α-ramp This appendix restates the weight construction of (Azizi et al., 2026) in our notation; nothing here is new. The incremental weight (3) expands, using γt(y1:t∣x)=pθ(y1:t∣x)αt _t(y_1:t x)=p_θ(y_1:t x) _t, as ωt=pθ(y1:t−1∣x)αt−αt−1pθ(yt∣x,y<t)αtq(yt∣x,y<t). _t=p_θ(y_1:t-1 x) _t- _t-1\, p_θ(y_t x,y_<t) _tq(y_t x,y_<t). (12) The first factor reweights the whole prefix from exponent αt−1 _t-1 to αt _t; it is computable from the accumulated base log-probability of the prefix. In the absence of resampling, the accumulated weight telescopes by induction to the sequential-importance-sampling identity U(i)t=∏s≤tω(i)s=pθ(y(i)1:t∣x)αt∏s≤tq(ys(i)∣x,y<s(i))=γt(y(i)1:t∣x)∏s≤tq(ys(i)∣x,y<s(i)),U^(i)_t= _s≤ tω^(i)_s= p_θ(y^(i)_1:t x) _t _s≤ tq(y^(i)_s x,y^(i)_<s)= _t(y^(i)_1:t x) _s≤ tq(y^(i)_s x,y^(i)_<s), the exact importance weight for the intermediate target γt _t. With resampling, this identity holds between consecutive resampling events; at the population level, the weighted empirical approximation of the intermediate targets is preserved by the resampler’s unbiasedness (Proposition 3). Once the ramp completes (αt=α _t=α), the population targets πα _α over the reachable sequences (exactly πα _α only under a full-support proposal and EOS-only termination) in the stopping-policy sense of Section 2 (Del Moral et al., 2006; Azizi et al., 2026), and the incremental weight reduces to ωt=pθ(yt∣x,y<t)α/q(yt∣x,y<t) _t=p_θ(y_t x,y_<t)^α/q(y_t x,y_<t). Absorbed particles are handled by the same identity. A particle that stops at step t0t_0 contributes unit token-level increments for t>t0t>t_0, but the prefix factor pθ(y1:t0∣x)αt−αt−1p_θ(y_1:t_0 x) _t- _t-1 continues to be applied while the ramp is active, with a final catch-up to the full exponent at the end of decoding. An absorbed particle’s trajectory therefore also carries weight pθ(y1:t0∣x)αp_θ(y_1:t_0 x)^α divided by its proposal mass, and stopping during the ramp does not distort the final target. Appendix D Selector definitions All selectors operate post hoc on the same saved final population of N weighted particles; no selector invokes the model after the final populations and behavioral test inputs have been generated. The behavioral test inputs used on HumanEval are synthesized once per problem and per base model, before selection, and are shared verbatim by both resampling arms and all selectors, so any accuracy difference between selectors is attributable to selection alone. Let a(i)a^(i) be the parsed answer of particle i, Wfinal(i)W^(i)_final its normalized final weight, and g(a,a⋆)∈0,1g(a,a )∈\0,1\ the task grader against the gold answer a⋆a . Power-SMC’s default rule and two baselines are the weight draw (Section 4.3), which returns a(I)a^(I) with I∼Categorical(Wfinal(1:N))I (W^(1:N)_final); argmax, which returns the answer of the single highest-weight particle; and oracle, maxig(a(i),a⋆) _ig(a^(i),a ), the population ceiling used in the main text. The semantic-majority selector maj, defined in Section 4.3, aggregates over the distinct trajectories T, each carrying a pooled weight W¯u=∑i:y(i)=uW(i)final W_u= _i:\,y^(i)=uW^(i)_final used only to break ties; it is the selector we report throughout. For the symbolic-answer benchmarks, rep(C)rep(C) is the first parsed answer that creates cluster C under the greedy clustering procedure; for HumanEval, it is the distinct program in C with the largest pooled final weight. Other aggregations of the same clusters are possible, for example weighting each vote by its pooled weight or by an entropy or confidence score, or scoring with a process-reward model, but we use plain majority. A single equivalence check e(a,b)e(a,b) underlies both scoring and clustering: scoring compares a candidate to the gold answer, g(a,a⋆)=e(a,a⋆)g(a,a )=e(a,a ), while clustering compares candidates to one another and never consults the gold answer. Per benchmark, e combines, for MATH500, Hendrycks-style normalization (Hendrycks et al., 2021) with sympy simplification of the difference in the manner of Minerva (Lewkowycz et al., 2022) and the PRM800K grader (Lightman et al., 2024), with strict matching required when the reference argument is an integer; absolute difference below 10−610^-6 after comma stripping for GSM8K; strict integer equality for AIME; and single-letter comparison after uppercasing for GPQA. Answers are extracted by a three-tier ladder: a boxed expression; otherwise an “answer is” pattern in the final 300 characters; otherwise a tail fallback, which for GPQA is a standalone capital letter A–D in the final 200 characters. Clustering is greedy match-to-representative: each answer joins the first cluster whose representative it grades equivalent to, with a string-equality shortcut, so no transitive closure is computed. Particles whose answers fail to parse are excluded from the vote; if no particle parses, the problem is scored incorrect for the majority selector. For HumanEval, programs are clustered by exact behavioral signatures on a fixed ordered set of model-generated inputs (Section 4.3), adapting the execution-based clustering of AlphaCode (Li et al., 2022) and the generated-test agreement of CodeT (Chen et al., 2023a). The signature records the serialized output on each input; when evaluation of an individual input raises an exception, the exception type is recorded as part of the signature. Programs whose sandboxed execution fails or times out before producing a signature are excluded. The selector falls back to the final-weight draw if no generated inputs are available or if no program produces a usable behavioral signature (this fallback exists only on HumanEval). Table 2 details. On HumanEval the “majority” selector is behavioral majority (Section 4.3); exact counts out of 164164, per (Qwen2.5-Math-7B, Qwen2.5-7B, Qwen3-4B): systematic ++ weight draw (96,120,117)(96,120,117); systematic ++ behavioral (100,125,116)(100,125,116); Chopthin ++ weight draw (96,123,116)(96,123,116); Chopthin ++ behavioral (101,126,116)(101,126,116). Appendix E Additional results E.1 Oracle coverage values Table 3 reports the absolute oracle coverage underlying the differences of Figure 2. Comparing each resampler’s coverage against its majority-selector accuracy in Table 2, coverage exceeds accuracy in every cell (median gap 4.24.2 points; smallest gap +1.5+1.5, Qwen2.5-7B on AIME under Chopthin), so selector headroom remains even after majority voting. Table 3: Oracle coverage (%): fraction of problems whose final population contains at least one correct trajectory, per resampler. Bold marks the higher value per column within each block; green rows mark the Chopthin arm (ours). Coverage is measured over the correlated population of a single run and is a ceiling for any selector, not i.i.d. pass@N. Model Resampler MATH500 GSM8K AIME GPQA HumanEval Systematic 84.6 94.4 17.8 50.5 64.0 Qwen2.5-Math-7B Chopthin 87.0 94.8 22.2 54.5 68.3 Systematic 81.2 94.1 13.7 50.5 83.5 Qwen2.5-7B Chopthin 83.0 94.8 13.7 49.0 84.8 Systematic 83.8 93.7 17.8 39.9 72.6 Qwen3-4B Chopthin 84.8 93.9 20.0 45.5 73.8 E.2 Reported GRPO reference values Table 4 situates CCPS against reported GRPO post-training results under unmatched evaluation protocols. Training-free CCPS is competitive on MATH500 and stronger on HumanEval, while GRPO retains a clear advantage on GPQA on both models. Table 4: Reported post-training reference values for GRPO (Shao et al., 2024), as reported by (Karan and Du, 2026) and tabulated by (Nguyen et al., 2026), alongside CCPS. The evaluation protocol (prompts, checkpoints, graders, budgets) is not matched to ours; these numbers are context, not a controlled comparison. Model Method MATH500 GPQA HumanEval Qwen2.5-Math-7B GRPO (post-training, reported) 78.5 39.9 53.7 CCPS (training-free, ours) 81.4 33.8 61.6 Qwen2.5-7B GRPO (post-training, reported) 74.0 35.4 56.1 CCPS (training-free, ours) 76.0 30.3 76.8 Bold marks the higher value per model and benchmark; green rows are our method.