Paper deep dive
Claim-Level Reliability Assessment for Efficient Test-Time Reasoning
Sen Xu, Wei Wang, Shixi Liu, Jixin Min, Yingwei Dai, Zhibin Yin, Yirong Chen, Junlin Zhang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We propose claim-level falsification as a principle for test-time scaling and instantiate it through Claim-Level Reliability Assessment (CLR), a training-free framework that reallocates test-time compute from additional solution sampling to targeted verification. Since whole-trace evaluation often obscures decisive errors due to signal dilution from routine tokens, CLR condenses each reasoning trace into a compact set of decision-critical claims, thereby isolating its logical anchors. Furthermore, recognizing the inherent difficulty of generating entirely correct solutions under fixed model capabilities, CLR shifts the focus to semantic falsification. This approach exploits a fundamental asymmetry between solution construction and claim refutation. Constructing a valid solution requires a flawless reasoning path, whereas refuting an incorrect claim requires identifying only a single decisive flaw. This targeted search for negative evidence systematically compresses the survival space of high-confidence incorrect traces, effectively suppressing erroneous consensus via nonlinear reliability scoring. Across four LLMs and four reasoning benchmarks under matched budgets, CLR generally improves upon pass@1 and self-consistency. On GPT-OSS-20B/CMIMC25, for instance, CLR exceeds pass@1 by 27.15 percentage-points and raises self-consistency accuracy from 77.50\% to 82.19\% with 37.0\% fewer tokens.
Tags
Links
- Source: https://arxiv.org/abs/2608.11994v1
- Canonical: https://arxiv.org/abs/2608.11994v1
Trouble viewing inline? Open PDF directly →
Full Text
34,770 characters extracted from source content.
Expand or collapse full text
Published as a workshop paper at COLM 2026 Claim-Level Reliability Assessment for Efficient Test-Time Reasoning ∗ Sen Xu Wei Wang Shixi Liu Jixin Min Yingwei Dai Zhibin Yin Yirong Chen Junlin Zhang Sina Weibo Inc. xusen1,junlin6@staff.weibo.com Abstract We propose claim-level falsification as a principle for test-time scaling and instantiate it through Claim-Level Reliability Assessment (CLR), a training-free framework that reallocates test-time compute from additional solution sampling to targeted verification. Since whole-trace evaluation of- ten obscures decisive errors due to signal dilution from routine tokens, CLR condenses each reasoning trace into a compact set of decision-critical claims, thereby isolating its logical anchors. Furthermore, recognizing the inher- ent difficulty of generating entirely correct solutions under fixed model capabilities, CLR shifts the focus to semantic falsification. This approach ex- ploits a fundamental asymmetry between solution construction and claim refutation. Constructing a valid solution requires a flawless reasoning path, whereas refuting an incorrect claim requires identifying only a single decisive flaw. This targeted search for negative evidence systematically compresses the survival space of high-confidence incorrect traces, effec- tively suppressing erroneous consensus via nonlinear reliability scoring. Across four LLMs and four reasoning benchmarks under matched budgets, CLR generally improves upon pass@1 and self-consistency. On GPT-OSS- 20B/CMIMC25, for instance, CLR exceeds pass@1 by 27.15 percentage- points and raises self-consistency accuracy from 77.50% to 82.19% with 37.0% fewer tokens. 1 Introduction Test-time scaling has emerged as an effective paradigm for enhancing the reasoning ca- pabilities of large language models (LLMs) without parameter updates (Snell et al., 2025; Muennighoff et al., 2025; Wang et al., 2023). Existing methods can be broadly categorized based on where reliability signals are applied during the reasoning process. The first cat- egory derives intrinsic signals, such as token probabilities, entropy, or hidden states (Fu et al., 2025; Wang et al., 2025), to estimate the overall uncertainty of a single completed trace. The second operates across multiple parallel traces, producing final predictions via answer aggregation or Best-of-N selection (Cobbe et al., 2021; Toshniwal et al., 2025; Weng et al., 2023; Zhao et al., 2025; Singhi et al.; Chen et al.). The third intervenes during generation by using explicit evaluation signals to guide structured searches like branching, pruning, or backtracking (Yao et al., 2023). Despite their differences, these approaches share a fundamental bottleneck. As inference budgets grow, extracting accurate and highly discriminative reliability signals remains challenging, which ultimately limits the extent to which additional compute translates into trustworthy answers. Crucially, existing reliability signals often fail to capture the decision-critical semantic content that dictates answer correctness. Methods relying on token probabilities or internal states implicitly treat statistical confidence as a proxy for logical reliability. However, high statistical confidence does not guarantee logical soundness, and a highly confident trace may still harbor a decisive flaw (Xiong et al., 2024). Furthermore, whole-trace evaluation suffers ∗ Code is available at https://github.com/WeiboAI/CLR. 1 arXiv:2608.11994v1 [cs.AI] 12 Aug 2026 Published as a workshop paper at COLM 2026 from severe signal dilution. Because most tokens correspond to routine reasoning steps, they create a weakly discriminative background that degrades the signal-to-noise ratio. This background often obscures localized but fatal mistakes within an otherwise plausible trace. While step-level verification can isolate such errors, it is computationally exhaustive and typically requires process-level supervision or separately trained verifiers (Lightman et al., 2024). These limitations highlight the critical need for an intermediate granularity that bridges the gap between whole-trace and step-by-step evaluation by directly targeting the semantic content anchoring a reasoning trace. Building on these observations, we formulate claim-level falsification as a principle for test-time scaling and instantiate it through Claim-Level Reliability Assessment (CLR), a training-free framework for consensus-based answer aggregation. CLR extracts more discriminative reliability signals through two complementary mechanisms: (1) Claim-level evaluation. Instead of evaluating the full trace, CLR condenses it into a compact set of decision-critical claims. By filtering out routine tokens that dilute the reliability signal, this representation directs test-time compute exclusively toward the logical anchors that determine answer correctness. (2) Falsification-based verification. Treating each claim as a falsifiable proposition, CLR prompts the model to actively search for disconfirming evidence. This establishes a fundamental asymmetry: while constructing a correct solution demands a completely valid reasoning path, refuting an incorrect claim requires identifying only a single decisive flaw. By replacing open-ended re-solving with targeted refutation, CLR effectively repurposes the base model into a rigorous verifier. Crucially, this focused, one- sided check can expose critical errors overlooked during generation, even within erroneous traces produced with high statistical confidence. Operationally, CLR follows a two-stage inference pipeline. In the first stage, it samples Ksolution traces, each accompanied by a compact set of decision-critical claims. In the second stage, the same model independently verifies the claims associated with each trace using only the original problem and the extracted claims. We use Cons@Kto denote self-consistency (Wang et al., 2023) withKsampled traces. CLR@Ktherefore requiresK calls for solution generation andKcalls for claim verification, matching the 2Ksolution generation calls used by Cons@2K. Finally, CLR maps claim-level verdicts into nonlinear trace-level reliability scores for reliability-weighted aggregation, allowing a reliable minority to overturn an incorrect consensus formed by a majority of flawed traces. We evaluate CLR on four LLMs across multiple challenging reasoning benchmarks. Under matched model call budgets, CLR improves accuracy, token efficiency, or both across the evaluated model regimes. On HMMT25 (HMMT, 2025), CLR@32 raises the accuracy of Gemma-4-12B-it (Gemma Team, 2026) from 76.67% with Cons@64 to 88.75%. On CMIMC25 (CMIMC, 2025), CLR@32 improves GPT-OSS-20B (Agarwal et al., 2025) from 77.50% with Cons@64 to 82.19% while using 37.0% fewer generated tokens, outperforming Pass@1 by 27.15 percentage points. 2 Method Given a problemqand a solution-sampling countK, CLR independently samplesKtraces under a fixed decoding configuration. For each trace, a Stage-1 request generates a complete reasoning tracet k , its final predictiony k , and an ordered list ofMdecision-critical claims C k = (c k,1 ,. . .,c k,M ). A Stage-2 request then reuses the same model to search for refutations using onlyqandC k . CLR converts the resulting verdicts into a trace-level reliability score and aggregates equivalent predictions by their total reliability. The second stage therefore reweights sampled candidates without generating new predictions. Fig. 1 illustrates how claim-level reliability can suppress an erroneous five-trace consensus and select a smaller, better-supported answer group from the same eight samples. Critical Claim Extraction. Stage 1 outputsy k in a task-specific structured format and appends exactlyMconcise intermediate claims whose failure would undermine the pre- diction, excluding generic summaries and prediction restatements. Typical claims encode intermediate conclusions, constraints, decision points, transformations, or evidence linking 2 Published as a workshop paper at COLM 2026 Problem Find the largest prime factor of 45 5 − 1. Rollout π θ (· | q) K = 8 T0: The expression ... 4193821 ✗ T1: To find the largest ... 4193821 ✗ T2: We can factor ... 4193821 ✗ T3: The given expression ... 2851 ✓ T4: To find the largest prime ... 2851 ✓ T5: The expression to factor ... 2851 ✓ T6: The number is ... 4193821 ✗ T7: The number given ... 4193821 ✗ Cons@K prediction 4,193,821 5 vs. 3 votes ✗ incorrect CLR prediction Trace / answerclaim verdicts (5 claims)score Trace T0 4,193,821 ✗ C1C2C3C4C5 ✓✗✓ (4/5) 5 = 0.328 Trace T1 4,193,821 ✗ C1C2C3C4C5 ✓✗ (3/5) 5 = 0.078 Trace T2 4,193,821 ✗ C1C2C3C4C5 ✓ (5/5) 5 = 1.000 T3–T5 2,851 ✓ (3 traces) C1C2C3C4C5 ✓ 3 × (5/5) 5 = 3.000 Trace T6 4,193,821 ✗ C1C2C3C4C5 ✓ (5/5) 5 = 1.000 Trace T7 4,193,821 ✗ C1C2C3C4C5 ✓✗ (3/5) 5 = 0.078 Example Claims (Trace T1) C1 45 5 −1 can be factored as (45−1)(45 4 +45³+45²+45+1) by the identity a 5 −1=(a−1)(a 4 +a³+a²+a+1). VALID C245−1 = 44 = 2²·11, contributing the prime factors 2 and 11.VALID C3Evaluating 45 4 +45³+45²+45+1 yields 4,193,821.VALID C4 4,193,821 has no prime divisors ≤ 2046 (its square root), hence it is prime. REFUTED C5Since 4,193,821 > 44, it is the largest prime factor of 45 5 −1.REFUTED 2,851 3.000 vs. 2.483 scores ✓ correct prediction reversal Figure 1: Overview of CLR through an illustrative example. Given the sameK =8 sampled traces, count-based self-consistency selects the incorrect answer 4,193,821 by a 5-to-3 majority. CLR instead applies falsification-based checks toM =5 decision-critical claims per trace and uses the resulting claim-level outcomes to perform reliability-weighted aggregation, recovering the correct answer 2,851. qtoy k , as illustrated by the factorization, primality, and maximality statements in Fig. 1. This fixed-size representation compresses each trace into focused semantic assertions. The full prompt appears in Appendix B.2. Falsification-Based Claim Assessment. For each trace, CLR makes one joint Stage-2 re- quest containingqand the ordered claim listC k , without the original trace or final prediction as separate inputs. The same model searches each claim for a decisive contradiction, coun- terexample, factual or logical error, missing condition, or unsupported inference, while also checking for conflicts across claims. We encode the outcome as v k,m = 0, refuted, if a decisive flaw is found, 1, not refuted, otherwise. (1) TheVALIDoutput token denotes only not refuted by this assessment, rather than formal proof of correctness. This one-sided objective focuses compute on negative-evidence search, where one decisive witness can refute a claim without constructing an alternative complete solution. We treat this asymmetry as an inductive bias rather than a guarantee that falsification is uniformly easier than generation. The complete prompt appears in Appendix B.3. Reliability Scoring and Aggregation.Lets k be the fraction of claims that survive falsifica- tion. We map this fraction to the trace score r k = s M k = 1 M M ∑ m=1 v k,m ! M .(2) ForM>1, the exponent makes a refuted decision-critical claim more consequential than under linear averaging, increasingly suppressing a trace as more of its decision-critical claims are refuted. Compared to linear averaging, this nonlinear penalty reduces the aggregate influence of error-prone traces even when they are numerically dominant, allowing a smaller but more reliable answer group to overturn an incorrect majority, as illustrated in Fig. 1. This monotone transformation is a heuristic, not a joint correctness probability, and does not assume claim independence. We use the claim countMas the exponent in all experiments. CLR extractsy k with a task-specific output parser and omits traces without a parsed pre- diction. It then partitions the predictions into equivalence groupsGaccording to a task- 3 Published as a workshop paper at COLM 2026 appropriate equivalence criterion. For each group G ∈G, the reliability support is R(G) = ∑ k:y k ∈G r k , b y = canon arg max G∈G R(G) ! ,(3) wherecanon(G)denotes the canonical candidate answer for groupG. Eq. 3 reduces to ordinary self-consistency when all traces receive the same positive score; otherwise, refuted claims reduce a trace’s influence on the final consensus. Ties, including the all-zero-score case, are resolved by the earliest equivalence group in sampling order. Budget and Scope. One CLR@Kflow usesKsolution-generation requests andKclaim- assessment requests, so it matches the request count of Cons@2K. Request parity does not imply token parity because claim assessment operates onqandMclaims rather than generat- ing another complete solution; we therefore report both matched-request and realized-token comparisons. CLR only reweights parsed candidates, so b y∈y k : y k is parsed and cannot recover a correct prediction absent from the Stage-1 samples. Its role is to convert candidate coverage into a more reliable selection under a fixed request or token budget. 3 Experiments 3.1 Experimental Setup Models and benchmarks. We evaluate CLR with Gemma-4-12B-it (Gemma Team, 2026), GPT-OSS-20B, GPT-OSS-120B (Agarwal et al., 2025), and Qwen3.5-27B (Qwen Team, 2026) on multiple reasoning benchmarks: HMMT25 (HMMT, 2025), HMMT26 (HMMT, 2026), CMIMC25 (CMIMC, 2025), and Apex-shortlist (Dekoninck et al., 2026). Baselines and test-time settings. Our primary baseline is self-consistency (Wang et al., 2023), which samplesKindependent solutions and aggregates their final answers. We evaluate Cons@KwithK ∈ 8, 16, 32, 64and CLR@KwithK ∈ 4, 8, 16, 32. Following the request accounting defined in Section 2, our primary matched-request comparison is CLR@32 versus Cons@64. Regular sampling uses a minimal solution prompt consisting of the problem and a single step-by-step instruction. CLR Stage 1 extends this base prompt by appending the claim-generation instructions; both templates are provided in Appendix B. Unless otherwise stated, CLR extractsM =5 claims per trace, and its accuracy is averaged over N = 8 independent complete flows. Metrics.For regular sampling, single-rolloutpass@1is the accuracy averaged over the 64 individual solution rollouts, whereas Cons@Kreports the accuracy obtained by aggregating Ksampled solutions. For CLR, CLR@Kdenotes a complete two-stage TTS flow comprising Ksolution-generation calls andKclaim-assessment calls. We reportpass@1as the mean accuracy acrossN =8 independent executions of the full flow. We measure efficiency using the average number of generated tokens per problem. CLR token counts include both solution generation and claim-level assessment. To account for their different output lengths, we report matched-request results together with accuracy as a function of realized token consumption. Model-specific decoding configurations are summarized in Appendix A. 3.2 Main Results At the primary operating point reported in Tab. 1, accuracy deltas are given in percentage points, while token deltas are relative changes; both are measured against Cons@64. The accuracy–efficiency profile varies across base models. For Gemma-4-12B-it, CLR improves accuracy on all four benchmarks by 7.12–12.08 points, including an increase from 76.67% to 88.75% on HMMT25. These gains, however, come with 22.2–47.8% more generated tokens. The results for GPT-OSS-20B follow a different pattern. CLR uses 36.3–39.8% fewer tokens on every benchmark while improving accuracy in three of four cases. On CMIMC25, it raises accuracy from 77.50% to 82.19% with 37.0% fewer tokens. GPT-OSS-120B similarly improves 4 Published as a workshop paper at COLM 2026 ModelBenchmarkPass@1 Cons@64CLR@32∆ vs. Cons@64 Acc.Tok. (×10 3 )Acc.Tok. (×10 3 )Acc. (p)Tok. (%) Gemma-4-12B-it HMMT2564.5876.671342.388.751791.0+12.08+33.4 HMMT2657.7269.701543.577.272203.8+7.57+42.8 CMIMC2555.2368.751521.380.621858.9+11.87+22.2 Apex-shortlist21.3532.981829.440.102704.2+7.12+47.8 GPT-OSS-20B HMMT2558.2380.00968.179.58616.4-0.42-36.3 HMMT2657.7272.731037.173.48624.5+0.75-39.8 CMIMC2555.0477.50926.382.19583.3+4.69-37.0 Apex-shortlist15.8220.831292.424.22795.0+3.39-38.5 Qwen3.5-27B HMMT2591.4193.332012.995.002004.8+1.67-0.4 HMMT2682.5390.912134.890.912083.30.00-2.4 CMIMC2585.9095.002150.097.502096.9+2.50-2.5 Apex-shortlist54.7272.923625.175.523099.3+2.60-14.5 GPT-OSS-120B HMMT2561.4686.67459.187.08350.1+0.41-23.7 HMMT2659.1972.73457.375.38350.7+2.65-23.3 CMIMC2559.1875.00423.680.00332.1+5.00-21.6 Apex-shortlist18.3923.96489.727.86382.4+3.90-21.9 Table 1: Token–accuracy trade-off across four models and four reasoning benchmarks under matched model-call budgets. CLR@32 and Cons@64 both use 64 model calls. Benchmark Regular SamplingCLR Stage 1 OnlyFull CLR Stage-2 Gain Pass@1 Cons@64 Pass@1 Cons@32CLR@32∆ (p) HMMT2558.2380.0053.6775.1079.58+4.48 HMMT2657.7272.7354.8866.4773.48+7.01 CMIMC2555.0477.5054.3977.1982.19+5.00 Apex-shortlist15.8220.8314.6817.9324.22+6.29 Table 2: Decomposition of CLR on GPT-OSS-20B. Stage-2 gain denotes the accuracy differ- ence between full CLR@32 and unweighted Stage-1 Cons@32 on the same candidates. accuracy by up to 5.00 percentage points while reducing token consumption by 21.6–23.7%. For Qwen3.5-27B, whose Cons@64 accuracy already exceeds 90% on three benchmarks, the remaining headroom is smaller. CLR matches or improves accuracy by up to 2.60 percentage points, with its largest token reduction of 14.5% occurring on Apex-shortlist. The matched-call comparison indicates that test-time compute need not be spent exclusively on generating more solutions. Reallocating half of the calls to claim-level falsification can either improve accuracy over count-based consensus or retain competitive accuracy with substantially fewer generated tokens, with the dominant benefit depending on the base model’s operating regime. 3.3 Decomposing the Gains of CLR Tab. 2 separates Stage-1 claim prompting from falsification-based reliability weighting. Unweighted aggregation over the same 32 candidates isolates Stage 2, while 64-sample self-consistency provides the matched-call reference in Tab. 1 because full CLR uses 32 calls per stage. The claim prompt alone reduces single-rollout accuracy by 0.65–4.56 percentage points relative to regular sampling. After claim-level assessment and reweighting, however, accuracy improves by 4.48–7.01 points over the same unweighted candidates. The gains therefore arise from falsification-based reliability weighting rather than claim prompting itself, complementing the matched-call comparison in Tab. 1. 5 Published as a workshop paper at COLM 2026 481632 Number of Stage-1 traces (K) 0 10 20 30 40 50 Rescue rate (%) HMMT25HMMT26CMIMC25Apex-shortlist Figure 2: CLR rescue rates on GPT-OSS-20B across Stage-1 sampling counts, conditioned on Cons@K being incorrect despite a correct candidate among the same K traces. 3.4 Rescuing Incorrect Consensus Fig. 1 illustrates CLR overturning an incorrect count-based consensus when the correct answer is already present among the candidates. To quantify how often this occurs, we report the rescue rate in Fig. 2. For a single TTS flow, the metric is defined as RescueRate@K = ∑ q 1 h ∃k : ˆ y q,k = y q , ˆ y Cons q ̸= y q , ˆ y CLR q = y q i ∑ q 1 h ∃k : ˆ y q,k = y q , ˆ y Cons q ̸= y q i .(4) The denominator counts recoverable consensus failures, in which Cons@Kis incorrect de- spite the presence of a correct Stage-1 candidate. The numerator counts the subset corrected by CLR, so RescueRate@K measures the fraction of such failures that CLR overturns. Across the 16 benchmark–budget settings, pooled rescue rates span roughly 16–48% and average about 37%, with raw counts in Appendix A.2. Without changing the underlying model or adding a separate verifier, CLR overturns a substantial fraction of erroneous consensus outcomes while preserving reliable minority traces. 3.5 Effect of the Number of Claims Benchmark Number of Claims M = 1M = 3M = 5 Acc. (%)↑ Tok. (×10 3 )↓ Acc. (%)↑ Tok. (×10 3 )↓ Acc. (%)↑ Tok. (×10 3 )↓ HMMT2572.50533.375.83623.079.58616.4 HMMT2670.83547.274.62644.273.48624.5 CMIMC2577.19511.380.62591.782.19583.3 Apex-shortlist20.83682.523.96792.624.22795.0 Table 3: Effect of the number of decision-critical claims on GPT-OSS-20B withK =32. Accuracy averages eight CLR flows; tokens are mean per-problem totals in thousands. In Tab. 3, we examine the claim-set size. AtM =1, Eq. 2 reduces tor k = v k,1 ∈ 0, 1, yielding binary filtering, while multiple claims provide broader decision-critical coverage and graded nonlinear weights. IncreasingMfrom 1 to 3 improves accuracy by 3.13–3.79 percentage points across all benchmarks. IncreasingMto 5 yields further gains on three, while HMMT26 peaks atM =3. The main benefit therefore comes from moving beyond a single claim, with task-dependent returns thereafter. The multi-claim settings use 14.1–17.7% more generated tokens thanM =1, whileM =3 andM =5 differ by at most 3.1% with no consistent direction. BecauseMcontrols both the number of claims and the exponent in Eq. 2, this ablation jointly varies semantic coverage, score resolution, and penalty sharpness rather than isolating claim count alone. 6 Published as a workshop paper at COLM 2026 1.0M 75 80 85 90 Accuracy Gemma-4-12B-it 8 16 32 64 4 8 16 32 HMMT25 1.0M 65 70 75 8 16 32 64 4 8 16 32 HMMT26 1.0M 70 75 80 816 32 64 4 8 16 32 CMIMC25 1.0M 25 30 35 40 8 16 32 64 4 8 16 32 Apex-shortlist 100k1.0M 65 70 75 80 85 Accuracy GPT-OSS-20B 8 16 32 64 4 8 16 32 100k1.0M 66 68 70 72 74 8 16 32 64 4 8 16 32 100k1.0M 65 70 75 80 8 16 32 64 4 8 16 32 100k1.0M 16 18 20 22 24 8 16 32 64 4 8 1632 1.0M 94 95 96 Accuracy Qwen3.5-27B 8 163264 48 16 32 1.0M 86 88 90 8 16 32 64 4 8 16 32 1.0M 92 94 96 98 8 1632 64 4 8 16 32 1.0M 65.0 67.5 70.0 72.5 75.0 8 16 32 64 4 8 16 32 100k Average tokens per problem 60 70 80 90 Accuracy GPT-OSS-120B 8 16 32 64 4 8 16 32 100k Average tokens per problem 67.5 70.0 72.5 75.0 77.5 8 16 3264 4 8 16 32 100k Average tokens per problem 65 70 75 80 8 16 32 64 4 8 16 32 100k Average tokens per problem 15 20 25 8 16 32 64 4 8 16 32 Self-consistencyCLR Figure 3: Token–accuracy trade-off across four models and four reasoning benchmarks as test-time budgets increase. Compared to count-based self-consistency, CLR avoids early performance saturation and demonstrates a more steady scaling behavior. 3.6 Cross-Budget Accuracy–Efficiency Scaling Fig. 3 compares CLR with self-consistency over increasing test-time budgets for all four evaluated models and benchmarks. The methods allocate requests differently, so the hori- zontal axis reports realized token consumption rather than treating equally sizedKvalues as equivalent budgets. Unlike the matched-call comparison in Tab. 1, these curves show how each method responds as its test-time budget grows. In several settings, self-consistency saturates or fluctuates as more solutions are added, while CLR improves more steadily, yielding a more favorable accuracy–compute scaling trajectory than count-based aggrega- tion. This pattern is especially visible across the four Gemma-4-12B-it benchmarks and on CMIMC25 and Apex-shortlist for both GPT-OSS models, suggesting that the gains are not confined to the primary operating point. The curves can nevertheless cross at intermediate budgets, and the two methods remain closer on the near-saturated Qwen3.5-27B settings; CLR is therefore not uniformly dominant at every operating point. Across budgets, the value of additional compute depends on the reliability signal it produces. Claim-level falsification can move the accuracy–token frontier outward when additional count-based samples provide noisy or weakly informative support, while offering less headroom when the base consensus is already reliable. 7 Published as a workshop paper at COLM 2026 4 Conclusion In this work, we explore claim-level falsification as a principle for test-time scaling (TTS). Unlike common forward-search approaches that rely on additional sampling or iterative revision to find a correct reasoning trace, claim-level falsification uses decision-critical semantic anchors to systematically constrain erroneous reasoning paths, allowing reliable paths to exert greater influence. CLR provides an initial instantiation and validation of this idea within consensus-based aggregation framework. Our experiments show that reallocating part of the reasoning budget from additional forward generation to targeted falsification can substantially improve the performance frontier at test time. Future work will explore broader applications of this principle across different TTS paradigms. References Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925, 2025. Jiefeng Chen, Jie Ren, Xinyun Chen, Chengrun Yang, Ruoxi Sun, Jinsung Yoon, and Sercan O Arik. Sets: Leveraging self-verification and self-correction for improved test-time scaling. Transactions on Machine Learning Research. CMIMC. Carnegie Mellon Informatics and Mathematics Competition 2025: Problems and solutions, 2025. URL https://cmimc.math.cmu.edu/math/past-problems/2025. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. Jasper Dekoninck, Nikola Jovanovi ́ c, Tim Gehrunger, K ́ ari R ̈ ognvaldsson, Ivo Petrov, Chenhao Sun, and Martin Vechev. Beyond benchmarks: MathArena as an evalua- tion platform for mathematics with LLMs. arXiv preprint arXiv:2605.00674, 2026. doi: 10.48550/arXiv.2605.00674. Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. Deep think with confidence. 2025. Gemma Team. Gemma 4 technical report. 2026. HMMT. Harvard–MIT Mathematics Tournament February 2025: Problems and solutions, 2025. URL https://w.hmmt.org/w/archive/282. HMMT. Harvard–MIT Mathematics Tournament February 2026: Problems and solutions, 2026. URL https://w.hmmt.org/w/archive/292. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In International Conference on Learning Representations, volume 2024, p. 39578–39601, 2024. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand ` es, and Tatsunori B Hashimoto. s1: Simple test-time scaling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 20286–20332, 2025. Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URLhttps: //qwen.ai/blog?id=qwen3.5. Nishad Singhi, Hritik Bansal, Arian Hosseini, Aditya Grover, Kai-Wei Chang, Marcus Rohrbach, and Anna Rohrbach. When to solve, when to verify: Compute-optimal problem solving and generative verification for llm reasoning. In Second Conference on Language Modeling. 8 Published as a workshop paper at COLM 2026 Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. In International Conference on Learning Representations, 2025. Shubham Toshniwal, Ivan Sorokin, Aleksander Ficek, Ivan Moshkov, and Igor Gitman. Genselect: A generative approach to best-of-n. In 2nd AI for Math Workshop@ ICML 2025, 2025. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations, 2023. Yiming Wang, Pei Zhang, Baosong Yang, Derek Wong, and Rui Wang. Latent space chain-of- embedding enables output-free llm self-evaluation. In International Conference on Learning Representations, volume 2025, p. 70938–70970, 2025. Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. Large language models are better reasoners with self-verification. In Findings of the Association for Computational Linguistics: EMNLP 2023, p. 2550–2575, 2023. Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms. In International Conference on Learning Representations, volume 2024, p. 23650–23678, 2024. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822, 2023. Eric Zhao, Pranjal Awasthi, and Sreenivas Gollapudi. Sample, scrutinize and scale: Effec- tive inference-time search by scaling verification. In International Conference on Machine Learning, p. 77272–77309. PMLR, 2025. 9 Published as a workshop paper at COLM 2026 A Experimental Details A.1 Decoding Configuration We use the default inference settings of each model unless explicitly stated otherwise. For all models, we enable thinking mode and keep the thinking effort at the model default. The same sampling configuration is used for solution generation and claim-level reliability assessment. Tab. 4 summarizes the model-specific sampling hyperparameters. ModelTemperatureTop-pTop-kPresence penalty Gemma-4-12B-it1.01.00400.0 GPT-OSS-20B1.01.00400.0 GPT-OSS-120B1.01.00400.0 Qwen3.5-27B1.00.95201.5 Table 4: Model-specific decoding configurations used in the experiments. All other model- specific runtime settings are left at their defaults. A.2 Raw Counts for Rescue Rate Tab. 5 reports the raw counts underlying Fig. 2. Because the reported rates pool results from N =8 independent flows, each denominator counts recoverable question–flow pairs rather than unique benchmark questions. Specifically, it counts pairs in which at least one Stage-1 candidate has the correct final answer but unweighted Cons@Kis incorrect. The numerator counts the subset corrected by CLR using the same candidates. KHMMT25HMMT26CMIMC25Apex-shortlist 415/43 (34.88%)15/35 (42.86%)33/72 (45.83%)26/63 (41.27%) 818/43 (41.86%)16/41 (39.02%)28/69 (40.58%)24/113 (21.24%) 1615/49 (30.61%)19/44 (43.18%)32/70 (45.71%)42/156 (26.92%) 3214/40 (35.00%)20/55 (36.36%)29/61 (47.54%)32/199 (16.08%) Table 5: Raw counts for the CLR rescue rate. Each entry reports rescued cases over recover- able consensus errors, followed by the rescue rate in parentheses. B Prompt Templates and Output Formats This appendix reports the base prompt used for regular solution sampling and the two prompt templates used by CLR. The placeholderquestionis replaced by the benchmark problem, andnum_claimscorresponds toMin the main text and specifies the number of claims and verdicts. The XML tag sequences andclaims_blockare assembled dynami- cally according to this hyperparameter. Both stages use the same sampling parameters. B.1 Regular Solution Sampling Regular sampling uses the following base prompt. CLR Stage 1 extends this template by appending the claim-generation instructions shown in the next subsection. question Let's think step by step and output the final answer within . 10 Published as a workshop paper at COLM 2026 B.2 Stage 1: Solution and Claim Generation Stage 1 asks the model to generate a step-by-step solution, place the final answer in , and append a fixed number of verification claims. The template, including its runtime placeholders, is shown below. question Let's think step by step and output the final answer within . After the final answer, append exactly **num_claims** verification claims selected from your completed solution. ### Claim Rules - **Relevance:** Each claim should be an intermediate mathematical statement used to reach the final answer. - **Diagnostic Value:** Prefer claims whose failure would strongly undermine the final answer. - **Non-triviality:** Do not use claims that merely restate the problem, repeat the final answer, duplicate another claim, or only describe a trivial final arithmetic step. - **Clarity:** Use precise mathematical objects, variables, equations, constraints, transformations, case distinctions, or counting formulas. The final answer must appear before the claims and must be written as: final answer Then output exactly: <claims> claim_tags </claims> For compactness,claim_tagsdenotes exactlynum_claimsentries of the form <claim>CLAIMCONTENT</claim>, assembled at runtime. B.3 Stage 2: Claim-Level Reliability Assessment Stage 2 stress-tests the extracted claims and returns an ordered binary verdict for each claim. The placeholderclaims_blockcontains the claims extracted in Stage 1. The template is shown below. You are a rigorous mathematical verifier. ## Problem question ## Claims to Verify The following num_claims claims were made during a solution attempt: claims_block ## Your Task Treat every claim as a suspicious assertion rather than a fact. For each claim, try to FALSIFY it by checking whether it: - contradicts the problem conditions, - conflicts with another claim in the list, - relies on an unjustified inference or hidden assumption, - fails on a simple test case or counterexample, - contains a calculation or logical error. Do NOT assume that a claim is true just because you cannot immediately find a counterexample. Actively search for flaws. After your analysis of each claim, give a final verdict. ## Output Format After your analysis, you MUST end your response with verdicts in this format: <verdicts> verdict_tags </verdicts> Output exactly num_claims verdict tags in order. Replace each VERDICT with exactly one token: VALID or REFUTED. Do not output VERDICT literally. Do not output "VALID or REFUTED" literally. Do not put any text after </verdicts>. 11 Published as a workshop paper at COLM 2026 For compactness,verdict_tagsdenotes exactlynum_claimsentries of the form <verdict>VERDICT</verdict>, assembled at runtime. Claim rendering.Before insertion into the prompt, line breaks within each extracted claim are collapsed to spaces. The dynamically assembledclaims_blockhas the following form: 1. <claim text 1> 2. <claim text 2> ... num_claims. <claim text num_claims> Output parsing and fallbacks. Malformed XML tags or truncated traces may prevent complete claim extraction. If fewer thanMclaims are successfully parsed, the remaining slots are padded with empty claims, while missing or unparseable verdicts default to not refuted. 12