Paper deep dive
CAP-CoT: Cycle Adversarial Prompt for Improving Chain of Thoughts in LLM Reasoning
Shuxu Chen, Yitian Zhou, Jiaquan Zhang, Haoyu Bian, Aming Wu, Sungyoung Lee, Chaoning Zhang, Hyundong Shin
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 99%
Last extracted: 6/21/2026, 6:31:35 AM
Summary
CAP-CoT (Cycle Adversarial Prompt) is a novel optimization framework designed to improve the accuracy and stability of Chain-of-Thought (CoT) reasoning in Large Language Models (LLMs). The framework employs a three-agent cycle consisting of a Solver (generates correct reasoning), an Adversarial Challenger (constructs plausible but flawed reasoning using a specific error taxonomy), and a Feedback Agent (provides structured, step-aligned feedback). This iterative process uses Structured Feedback Prompt Refinement (SFPR) to simultaneously update the solver's prompt to repair weaknesses and the challenger's prompt to generate more targeted, difficult errors. Experimental results across six benchmarks (including MATH and GSM8K) and four LLM backbones (GPT-4o, GPT-4o-mini, Qwen-turbo, Deepseek-V3) demonstrate that CAP-CoT consistently outperforms standard CoT and other advanced reasoning methods like ToT, GoT, and CCoT.
Entities (9)
Relation Signals (7)
CAP-CoT โ comprises โ Solver Agent
confidence 100% ยท CAP-CoT consists of three role agents: a solver... an adversarial challenger... and a feedback agent
CAP-CoT โ comprises โ Adversarial Challenger Agent
confidence 100% ยท CAP-CoT consists of three role agents: a solver... an adversarial challenger... and a feedback agent
CAP-CoT โ comprises โ Feedback Agent
confidence 100% ยท CAP-CoT consists of three role agents: a solver... an adversarial challenger... and a feedback agent
Adversarial Challenger Agent โ generates โ Adversarial Chain
confidence 100% ยท the output of ํบ ํถ is an adversarial chain ํถ ํถ
Feedback Agent โ updates โ Solver Prompt
confidence 100% ยท It not only updates the solver prompt based on errors exposed by the challenger
Feedback Agent โ updates โ Challenger Prompt
confidence 100% ยท and updating the challenger prompt to generate increasingly targeted errors
Structured Feedback Prompt Refinement โ usedby โ CAP-CoT
confidence 100% ยท We adopt an adaptive strategy to refine role LLM prompts through a structured feedback prompt refinement (SFPR) step.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Chain-of-Thought (CoT) prompting has emerged as a simple and effective way to elicit step-by-step solutions from large language models (LLMs). However, CoT reasoning can be unstable across runs on long, multi-step problems, leading to inconsistent answers for unchanged task. Most prior work focuses on improving the forward reasoning chain within a single pass, with less attention to iterative and contrastive correction. To address this gap, we propose CAP-CoT, a Cycle Adversarial Prompt optimization framework designed to improve both CoT reasoning accuracy and stability of a single deployed solver. In each cycle, a forward solver generates candidate reasoning chains, an adversarial challenger constructs plausible but deliberately flawed chains using targeted error strategies, and a feedback agent contrasts the two chains and produces step-aligned structured feedback. This feedback closes the optimization loop in two directions, including updating the solver prompt based on errors exposed by the challenger, and updating the challenger prompt to generate increasingly targeted errors in subsequent cycles. Unlike safety-oriented adversarial prompting such as jailbreak or prompt-injection attacks, our adversarial component is task-semantic and aims to expose logical vulnerabilities in reasoning chains. Experiments across six benchmarks and four LLM backbones demonstrate that within two to three adversarial prompt optimization cycles, CAP-CoT consistently reduces variability across runs while improving reasoning accuracy and robustness to prompt perturbations.
Tags
Links
- Source: https://arxiv.org/abs/2604.23270v1
- Canonical: https://arxiv.org/abs/2604.23270v1
Trouble viewing inline? Open PDF directly โ
Full Text
60,708 characters extracted from source content.
Expand or collapse full text
CAP-CoT: Cycle Adversarial Prompt for Improving Chain of Thoughts in LLM Reasoning Shuxu Chen a , Yitian Zhou b , Jiaquan Zhang b , Haoyu Bian b , Aming Wu c , Sungyoung Lee d , Chaoning Zhang b and Hyundong Shin a,โ a Department of Electronic Engineering, Kyung Hee University, Yongin-si, 17104, Gyeonggi-do, Korea b School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, Sichuan, China c Henan Polytechnic University, Jiaozuo, 454099, Henan, China d School of Computing, Kyung Hee University, Yongin-si, 17104, Gyeonggi-do, Korea A R T I C L E I N F O Keywords: Adversarial Prompt Optimization Chain-of-Thought Contrastive Optimization Iterative Refinement LLM Reasoning A B S T R A C T Chain-of-Thought (CoT) prompting has emerged as a simple and effective way to elicit step-by-step solutions from large language models (LLMs). However, CoT reasoning can be unstable across runs on long, multi-step problems, leading to inconsistent answers for unchanged task. Most prior work focuses on improving the forward reasoning chain within a single pass, with less attention to iterative and contrastive correction. To address this gap, we propose CAP-CoT, a Cycle Adversarial Prompt optimization framework designed to improve both CoT reasoning accuracy and stability of a single deployed solver. In each cycle, a forward solver generates candidate reasoning chains, an adversarial challenger constructs plausible but deliberately flawed chains using targeted error strategies, and a feedback agent contrasts the two chains and produces step-aligned structured feedback. This feedback closes the optimization loop in two directions, including updating the solver prompt based on errors exposed by the challenger, and updating the challenger prompt to generate increasingly targeted errors in subsequent cycles. Unlike safety-oriented adversarial prompting such as jailbreak or prompt- injection attacks, our adversarial component is task-semantic and aims to expose logical vulnerabilities in reasoning chains. Experiments across six benchmarks and four LLM backbones demonstrate that within two to three adversarial prompt optimization cycles, CAP-CoT consistently reduces variability across runs while improving reasoning accuracy and robustness to prompt perturbations. 1. Introduction Large language models (LLMs) have shown strong per- formance across a wide range of reasoning tasks, particu- larly with prompt-based methods such as Chain-of-Thought (CoT) prompting, which encourages step-by-step solutions by structuring intermediate reasoning before arriving at a final answer [36, 14]. Despite this progress, higher CoT accuracy does not always translate into robust reasoning behavior. As problems grow longer and more complex, per- formance degrades substantially even for strong models [17], revealing that the apparent capability measured on standard benchmarks may not generalize to harder or more varied instances. Moreover, CoT-based approaches remain sensi- tive to prompt-level variation and contextual perturbations [44]. For instance, reordering premises without changing the underlying task can lead to large performance drops [4], and irrelevant context can distract the modelโs reasoning and reduce arithmetic accuracy [40]. This sensitivity makes it difficult to distinguish which reasoning steps are truly robust from those that are only accidentally correct, especially on multi-step problems where early deviations propagate and compound throughout the chain. โ Corresponding author chenshuxu@khu.ac.kr (S. Chen); ytzhouuu@gmail.com (Y. Zhou); jiaquanzhang2005@gmail.com (J. Zhang); haoyubian04@gmail.com (H. Bian); amingwu@hpu.edu.cn (A. Wu); sylee@oslab.khu.ac.kr (S. Lee); chaoningzhang1990@gmail.com (C. Zhang); hshin@khu.ac.kr (H. Shin) ORCID(s): 0009-0002-5017-9515 (S. Chen) Most prior work addresses these limitations by improv- ing the forward reasoning trace, for example, by refining prompts, sampling multiple trajectories, or searching over structured reasoning spaces [34, 43, 2, 30, 52, 48]. While effective, these methods primarily optimize what the model should do next along a single forward direction and generally lack a systematic mechanism for exposing latent failure modes and converting them into targeted corrections. A complementary perspective is that CoT demonstrations may not reliably teach models how to reason [32]. When a model performs multi-step reasoning, errors within an intermediate step can derail subsequent steps and undermine final accu- racy and reliability, which motivates explicit mechanisms to detect and repair such fragile reasoning [19, 31]. Recent studies on efficient representation and retrieval also suggest that improving intermediate structure alone is insufficient without robustness-aware modeling [8, 23]. Therefore, a natural direction emerging from these ob- servations is to learn from comparisons rather than exclu- sively from correct solutions, that is, to exploit the signal contained in both good and bad reasoning chains simultane- ously. This intuition aligns with preference-based learning, in which human comparisons of better and worse outputs are used to train a reward model that drives subsequent improve- ment [24]. A related principle appears in representation learning, where supervised contrastive learning improves generalization by pulling positive examples together while Shuxu Chen et al.: Preprint submitted to ElsevierPage 1 of 13 arXiv:2604.23270v1 [cs.AI] 25 Apr 2026 CAP-CoT repelling negative ones [13, 16]. Together, these ideas sug- gest that explicit contrast between correct and incorrect can- didates can provide a strong learning signal beyond purely positive demonstrations, motivating contrastive prompting methods that show the model what to avoid alongside what to do in the prompt [5]. While static contrasts of this kind are effective, they can be limited because the negative examples are fixed and may become less informative as the reasoning model improves, particularly in long and complex chains where the space of meaningful errors shifts as earlier weaknesses are resolved. Thus, an iterative cycle is proposed to address this, which continually generates targeted hard negatives calibrated to the modelโs current behavior while converting newly ex- posed weaknesses into explicit and actionable prompt up- dates. Iterative refinement of this form has proven broadly effective for improving LLM behaviors at the prompt level [21, 28, 39, 49, 50], reinforcing the case that closes the loop between error generation and prompt correction. Inspired by these learning patterns, we propose CAP- CoT, a Cycle Adversarial Prompt optimization framework that strengthens CoT reasoning through repeated contrast between a solver and an evolving challenger. CAP-CoT consists of three role agents: a solver that produces a forward reasoning chain, an adversarial challenger that constructs plausible but flawed chains using controlled error strategies, and a feedback agent that contrasts the two chains and outputs step-aligned, actionable instructions. Crucially, the feedback closes the cycle in both directions. It not only updates the solver prompt to repair fragile steps exposed by the challenger, but also updates the challenger prompt based on the solverโs current behavior so that subsequent cycles generate increasingly targeted hard errors. We emphasize that the term adversarial here is task-semantic, referring to the construction of deceptive but plausible reasoning errors, which is distinct from adversarial prompting in LLM safety that typically denotes jailbreak prompts or prompt-injection attacks aimed at eliciting prohibited behaviors [26, 20, 22]. By making the contrast evolve cycle by cycle in response to the modelโs current reasoning, CAP-CoT systematically dis- covers and repairs reasoning weaknesses that static demon- strations may fail to surface, improving both accuracy and robustness. Our contributions are summarized as follows: (1) We propose CAP-CoT, a cycle-based adversarial prompt optimization framework that strengthens CoT rea- soning through adaptive contrast between correct and erro- neous chains. (2) We design an adaptive challenger that starts from a lightweight error taxonomy and evolves over cycles to gen- erate targeted hard negative reasoning chains for adversarial optimization. (3) We introduce a feedback agent generating structured, step-aligned feedback that explicitly updates both the solver and challenger prompts, forming a clear and interpretable optimization cycle. (4) We conduct extensive evaluations of CAP-CoT on multiple reasoning benchmarks and LLM backbones, demon- strating improvements in accuracy and robustness. 2. Related Work Our work builds on three main lines of research: Chain- of-Thought (CoT) reasoning, contrastive and adversarial prompting for robustness and stability, and feedback-driven prompt optimization. 2.1. Chain-of-Thought Reasoning With the development of LLMs with greater scale and stronger ability, they demonstrate extreme capability for complex reasoning tasks with suitable prompts [33, 56]. Chain-of-Thought prompting improves LLM reasoning by encouraging explicit, step-by-step solutions and has become a standard prompting primitive for arithmetic and symbolic tasks [36, 14]. Beyond single-chain CoT, many methods strengthen test-time reasoning by aggregating multiple can- didate traces or expanding the search space with structured exploration [48]. Self-Consistency samples diverse CoT traces and selects the most consistent answer, often outper- forming greedy decoding with modest changes to inference [34]. Tree of Thoughts (ToT) treats intermediate reasoning steps as search states and performs deliberate exploration with evaluation and backtracking [43]. Graph of Thoughts (GoT) generalizes linear or tree-structured traces to a graph of intermediate thoughts, enabling flexible composition and refinement through dependency edges [2]. More recently, Forest-of-Thought (FoT) further scales test-time compute by integrating multiple reasoning trees and using consensus- style selection to revisit and correct flawed branches [3]. Atom of Thoughts (AoT) decomposes complex problems into self-contained atomic subquestions with a Markov- style progression, reducing long-horizon interference and serving as a plug-in for test-time scaling [30]. In parallel, long-context benchmarks such as LongBench highlight that reasoning quality can degrade when models must integrate evidence over very long inputs, motivating methods that are robust to noisy or lengthy contexts [1]. In multimodal set- tings, recent work further extends CoT to visual reasoning, emphasizing alignment between modalities and intermediate reasoning faithfulness [55, 35, 15] 2.2. Contrastive and Adversarial Prompting Adversarial training has long been employed to im- prove model robustness and generalization by constructing challenging counterexamples and encouraging models to handle harder cases. Existing research has included models across modalities, including CycleGAN [57] with unpaired translation, StyleGAN3 [11] with frequency-domain reg- ularization, PAIRED [7] for environment-based opponent modeling, and AdvGAN [38] for sample-level adversarial attacks. For LLM reasoning, a closely related idea is to make errors explicit in the prompt, as CoT demonstrations are typ- ically positive-only. Contrastive Chain-of-Thought (CCoT) addresses this by pairing correct and incorrect rationales in Shuxu Chen et al.: Preprint submitted to ElsevierPage 2 of 13 CAP-CoT the prompt, making errors explicit and improving robust- ness over vanilla CoT [5]. More interactive settings explore stronger counterexamples through debate-style protocols. Khan et al. [12] finds that more persuasive debaters can lead to more truthful answers, while debate-style adversarial collaboration improves factuality and reduces hallucination [41]. Yi et al. [46] studies belief-driven debate/coordination via Bayesian Nash equilibrium to stabilize outcomes. Also, analogical prompting treats LLMs as analogical reasoners and improves performance by retrieving and reusing solution patterns rather than explicit negatives [45]. These lines of work motivate our cycle-based view. Instead of relying on fixed negatives, the bad reasoning should evolve with the solver so that newly exposed weaknesses can be turned into targeted corrections. We also note that our use of adversarial refers to generating plausible-but-wrong reasoning traps and encouraging models to optimize with counterexamples. It should be distinguished from adversarial prompting in LLM safety, which typically studies jailbreaks, prompt injection, and harmful instruction elicitation [20, 25, 27, 9]. 2.3. Feedback-Driven Prompt Optimization A growing line of work improves LLM reasoning by using natural-language feedback to refine outputs or in- structions, often without parameter updates. These methods rely on critiques, checks, or structured comments to re- fine instructions [51]. Self-Refine [21] iteratively generates, critiques, and revises model outputs, achieving consistent gains without additional training. Reflexion [28] converts task feedback into short reflective notes that are stored and reused, enabling better future attempts through explicit tex- tual memory rather than weight updates. From the input side, Self-Polish [37] highlights that refining the problem state- ment itself can improve multi-step reasoning and robustness, complementing answer-side prompting methods. In addi- tion, prompt-level optimization can be either automated by using LLMs to search for better instructions (e.g., OPRO [39]) or guided by human preference feedback when numeric scores are unavailable [18]. Also, workflow-level methods can generate and refine multi-step prompting pipelines with feedback signals [49, 53]. These approaches support the broader view that iterative, feedback-driven prompt editing can be an effective mechanism for improving reasoning reliability. CAP-CoT builds on this view by closing the cycle with an evolving challenger so that feedback improves both the solverโs repairs and the challengerโs next hard negatives. 3. Method We propose CAP-CoT, a cycle-based prompt optimiza- tion framework for CoT reasoning. As shown in Fig. 1, CAP-CoT uses three role agents, including a solver ํบ ํ , a challenger ํบ ํถ , and a feedback agent ํน, all sharing the same LLM backbone but following different role prompts. The core design is a short optimization cycle that repeat- edly (i) generates a correct candidate chain, (i) generates a targeted flawed counter-chain, and (i) turns their contrast into prompt edits. Each cycle updates the solver prompt to repair fragile reasoning steps and the challenger prompt to produce more diagnostic hard negatives in the next cycle, so the contrast remains informative as the solver improves. After a few cycles (typically 2โ3), we keep the refined solver prompt and use only the solver at inference time. 3.1. Solver Agent The solver ํบ ํ , which is the main reasoning model in our framework, aims to produce a reasonable, coherent, and logically consistent CoT reasoning sequence given an input query. Let the query be denoted asํ, the solverํบ ํ produces a forward reasoning chain ํถ ํ under a solver prompt ํ ํ , which can be formulated as: ํถ ํ = ํบ ํ (ํ ํ + ํ,ํ ํ ),(1) where ํ ํ represents the parameters of ํบ ํ . To ensure the quality of the reasoning chain, ํบ ํ is instructed to generate clearly numbered and interpretable reasoning steps, state important assumptions when needed, and keep a consistent output format across runs. This makes it easier for the feed- back module to locate weak steps and allows the subsequent cycle to apply precise prompt edits to ํ ํ . 3.2. Adversarial Challenger Agent The challenger ํบ ํถ is used to construct a plausible but incorrect reasoning chain that stands in contrast to the for- ward reasoning chain ํถ ํ , stress-testing ํบ ํ . For the same query ํ and a challenger prompt ํ ํถ , the output of ํบ ํถ is an adversarial chain ํถ ํถ : ํถ ํถ = ํบ ํถ (ํ ํถ + ํ,ํ ํถ ),(2) where ํ ํถ corresponds to the parameters of ํบ ํถ . ํถ ํถ fol- lows the surface structure of the problem while injecting controlled reasoning flows such as logical leaps, conceptual confusions, and unsupported assumptions. In CAP-CoT, ad- versarial refers to task-semantic counter-chains that expose logical vulnerabilities in ํถ ํ , rather than safety-oriented attacks such as jailbreak or prompt injection. Error Strategy for the Challenger. To effectively challenge the reasoning capability of the solver and encourage more rigorous reasoning chains, we define an error strategy for erroneous chain construction. We use a lightweight error taxonomy to cold-start the challenger in the first cycle. For each query ํ, ํบ ํถ samples one or two error families from jump, confusion, fuzzy, wrapper: jump omits key steps, confusion mixes related concepts, fuzzy hides gaps behind vague language, and wrapper embeds a wrong core step in a fluent explanation. These error types capture common deficiencies observed in reasoning processes, serving as a structured basis for improving robustness and accuracy when facing diverse reasoning tasks. The challenger prompt ํ ํถ specifies which error types to use and how to inject them when generating the adversarial chainํถ ํถ , so thatํถ ํถ follows the problem structure but contains targeted flaws. In later cycles, ํ ํถ is no longer driven only by this fixed taxonomy, but is refined according to feedback so that ํบ ํถ gradually shifts toward error patterns that are most effective against Shuxu Chen et al.: Preprint submitted to ElsevierPage 3 of 13 CAP-CoT Figure 1: Overview of the proposed framework CAP-CoT. the current solver. This error strategy makes the contrast be- tween the correct chainํถ ํ and the erroneous chainํถ ํถ more informative, and helps ํบ ํถ maintain challenging attacks as the solver becomes stronger over cycles. 3.3. Feedback Agent The feedback agent ํน is responsible for evaluating the reasoning chains generated by ํบ ํ and ํบ ํถ and providing structured feedback for both agents accordingly. Formally, given ํ, prompts (ํ ํ ,ํ ํถ ), and reasoning chains (ํถ ํ ,ํถ ํถ ), ํน compares the two, diagnoses issues such as missing as- sumptions, incorrect inferences, or unclear transitions, and outputs feedback: ํฝ ํ = ํน(ํ ํ +ํ,ํถ ํ ,ํ ํน ),ํฝ ํถ = ํน(ํ ํถ +ํ,ํถ ํถ ,ํ ํน ), (3) where ํ ํน denotes the parameters of the feedback agent, and ํฝ ํ andํฝ ํถ represent the structured feedback for ํบ ํ and ํบ ํถ , respectively. We use a concise format such as: Chain: ํถ ํ or ํถ ํถ ; Step: index ํก; Issue type: missing assumption, incorrect inference, unclear step, etc.; Suggestion: a short, actionable fix. This step-aligned format allows us to track the fragile parts of the solver chain and to observe which adversarial patterns in the challenger are most effective. 3.4. Prompt Refinement We adopt an adaptive strategy to refine role LLM prompts through a structured feedback prompt refinement (SFPR) step. In each optimization round, the solver agent ํบ ํ and the adversarial challenger agent ํบ ํถ generate ํถ ํ and ํถ ํถ , and the feedback agent ํน analyzes them jointly. From the erroneous chain ํถ ํถ , ํน extracts how the solver should repair its reasoning (e.g., which missing checks or assumptions to add) and produces feedbackํฝ ํ to update the solver prompt: ํ ํ โ SFPR(ํ ํ ,ํฝ ํ ).(4) From the solver chain ํถ ํ , ํน identifies its current blind spots and derives feedbackํฝ ํถ that tells the challenger which error types and attack patterns to prioritize, so that ํ ํถ is refine with both how to inject errors and which errors to focus on to produce more targeted adversarial chains: ํ ํ โ SFPR(ํ ํ ,ํฝ ํ ), ํ ํถ โ SFPR(ํ ํถ ,ํฝ ํถ ),(5) In addition, ํน keeps its previous records of the latest optimization round termed asํฝ ํน . It could refines its own prompt through ํ ํท โ SFPR(ํ ํท ,ํฝ ํน ). By converting step- level critiques on ํถ ํถ into updates for ํ ํ and using ํถ ํ to guide updates of ํ ํถ , SFPR maintains a transparent refine- ment process and supports an iterative cycle whereํบ ํ learns to verify its logic more carefully while ํบ ํถ learns to generate increasingly targeted adversarial chains. 4. Experiments In this section, we assess the effectiveness and generaliz- ability of our proposed CAP-CoT framework. We mainly ad- dress the following questions: (1) How does this framework compare with existing LLM reasoning methods in terms of question answering (QA) performance and efficiency, especially with chain-of-thought (CoT)-like methods? (2) Shuxu Chen et al.: Preprint submitted to ElsevierPage 4 of 13 CAP-CoT Table 1 Performance comparison between our proposed method and representative baseline methods across six reasoning benchmarks under four backbones, with percent symbol (%) omitted in all the accuracy results. MethodsBackboneMATH GSM8K BBH MMLU-CF HotpotQA LongBench Avg. CoT GPT-4o-mini 78.390.978.369.667.257.673.6 CoT-SC 81.892.083.471.166.258.675.5 Self-Refine78.791.780.069.768.358.274.4 PromptAgent 80.692.580.170.569.858.775.4 AP65.487.272.565.864.752.968.1 CCoT 80.192.480.570.471.558.875.6 AFlow83.093.576.069.573.561.076.1 ToT 83.094.585.970.274.263.178.5 GoT82.394.984.171.676.863.878.9 MAD82.994.685.271.075.564.278.9 ECON 83.595.086.071.577.065.079.7 FoT82.594.082.470.666.759.176.1 AoT 83.695.086.070.980.668.580.8 CAP-CoT (Ours)87.296.187.972.583.169.382.7 CoT Qwen-turbo 78.190.778.169.466.857.373.4 CoT-SC 81.491.583.270.865.958.475.2 Self-Refine78.591.479.869.568.258.074.2 PromptAgent80.192.280.470.168.958.575.0 AP65.287.072.265.264.552.767.8 CCoT79.992.180.270.171.258.575.3 AFlow82.493.175.769.373.260.475.7 ToT81.994.283.771.376.462.478.3 GoT82.793.884.970.174.062.878.1 MAD83.293.883.570.874.563.278.2 ECON83.994.284.471.275.864.078.9 FoT82.293.982.370.466.459.075.7 AoT 83.594.785.470.577.568.179.9 CAP-CoT (Ours)87.295.687.171.680.468.981.8 CoT Deepseek-V3 78.591.378.569.967.457.773.9 CoT-SC82.092.183.671.566.658.975.8 Self-Refine78.991.980.470.169.158.474.8 PromptAgent80.892.881.270.669.759.275.7 AP65.687.672.866.164.953.468.4 CCoT 80.492.180.270.171.258.575.3 AFlow83.493.676.469.874.061.476.4 ToT82.595.084.472.076.963.279.0 GoT83.294.586.270.374.263.478.6 MAD 84.194.584.871.276.864.579.3 ECON84.894.985.571.878.065.280.0 FoT 82.794.282.670.566.859.376.0 AoT84.095.186.170.880.668.780.9 CAP-CoT (Ours)87.596.087.872.282.869.582.6 CoT GPT-4o 79.592.579.871.569.559.575.4 CoT-SC82.893.284.572.868.260.877.1 Self-Refine80.292.881.671.871.560.276.4 PromptAgent 82.193.982.572.372.261.077.3 AP67.588.574.267.566.855.270.0 CCoT81.593.881.972.173.860.577.3 AFlow84.594.877.571.275.563.577.8 ToT83.595.585.573.578.564.580.2 GoT84.295.287.072.276.265.280.0 MAD85.895.586.273.078.565.280.7 ECON86.495.986.973.579.866.081.4 FoT 83.894.983.572.468.561.577.4 AoT85.595.887.273.081.569.582.1 CAP-CoT (Ours)89.296.689.174.584.070.484.0 Shuxu Chen et al.: Preprint submitted to ElsevierPage 5 of 13 CAP-CoT How does the frameworkโs performance change over opti- mization rounds across different datasets, LLM backbones, and training settings? (3) How do individual agents and key components in this framework, including the adversar- ial challenger, the feedback agent, and structured feedback prompt refinement, as well as hyperparameter configuration, contribute to the overall performance? 4.1. Experimental Setup Datasets and Evaluation: We evaluate our method on six QA benchmarks: MATH [10], GSM8K [6], BBH [29], MMLU-CF [54], HotpotQA [42], and LongBench [1]. Col- lectively, these datasets cover advanced and grade-school math, logical and algorithmic reasoning, commonsense and factual knowledge, multi-hop QA, and very long-context understanding. For all experiments, we adopt standard ac- curacy as the evaluation metric. Baselines: We compare CAP-CoT with a broad set of strong reasoning baselines, including standard Chain-of- Thought (CoT) [14]; its self-consistency variant CoT-SC [34], which samples multiple reasoning paths (ํ = 5) and selects the most consistent answer; Contrastive CoT (CCoT) [5] that enhance reasoning demonstration; enhanced CoT- like methods including Tree-of-Thought (ToT) [43], Graph- of-Thought (GoT) [2], Forest-of-Thought (FoT) [3] (with reasoning branches ํ = 8), and Atom of Thoughts (AoT) [30]; Self-Refine [21], where an LLM iteratively generates, critiques, and revises its own outputs; Analogical Prompting (AP) [45] that uses analogy-based prompt structures to transfer solution patterns to new problems; feedback-driven approaches such as Analogical Flow (AFlow) [49]; and debate- or committee-style methods such as MAD [12] and ECON [47]. Implementation Details: Our framework uses three role agents on a shared LLM backbone: the solver ํบ ํ , the adver- sarial challenger ํบ ํถ , and the feedback agent ํน. For back- bone comparison, we instantiate all three roles with GPT- 4o-mini, Qwen-turbo, DeepSeek-V3, and GPT-4o, while in all other experiments we use GPT-4o-mini as the default backbone for its good trade-off between reasoning ability and latency. We implement the system in LangChain, which lets us modularize the three roles and call the underlying LLMs via official APIs. Unless otherwise noted, we disable nucleus sampling, set the maximum generation length to 2048 tokens, and set both frequency and presence penal- ties to 0.0. Except for robustness-to-rounds experiments, all reported results are averaged over three independent runs; ablation studies use five runs with the same averaging protocol. In accuracy-focused ablations, the temperature is fixed at 0.0, while in robustness-to-rounds experiments, we sweep the temperature from 0.0 to 1.0 on the MATH dataset to study its effect on accuracy and stability. 4.2. Performance Evaluation Table 1 compares CAP-CoT with all baselines on six benchmarks under four LLM backbones, using results of three rounds of the optimization cycle for our method. Table 2 Ablation results of different components in our proposed framework on the MATH dataset. Model Components Accuracy(%) Mean Variation ํบ ํ 82.54.15 ํบ ํ + ํน83.83.80 ํบ ํ + ํบ ํถ + ํน87.22.62 CAP-CoT consistently outperforms all baselines across all datasets under four LLM backbones, and achieves its strongest results on the most capable backbone, GPT-4o. It shows clear gains on math-heavy and multi-step reasoning tasks (MATH, GSM8K, BBH) and on more challenging settings like HotpotQA and LongBench, which require multi-hop reasoning and long-context understanding. While CoT and its variants perform reasonably well on simpler problems, they lag behind on multi-hop and long-context QA, and AP is generally weaker, suggesting that analogy alone is not sufficient for complex reasoning. In contrast, our proposed framework yields more accurate, stable, and backbone- agnostic solvers. 4.3. Ablation Studies 4.3.1. Effect of Framework Components Table 2 shows the ablation results assessing the contri- bution of each module in our framework on MATH dataset. Using only the solver ํบ ํ gives a strong baseline, but adding the feedback agentํน already improves accuracy and slightly reduces mean variation, showing that step-level feedback alone helps tighten the solverโs reasoning. Introducing the adversarial challengerํบ ํถ on top of this (ํบ ํ +ํบ ํถ +ํน) brings the largest gain, with the highest accuracy and the lowest variation. This indicates that targeted erroneous chains from ํบ ํถ , together with structured feedback from ํน, are crucial for improving both accuracy and stability of the solver. 4.3.2. Performance over Optimization Rounds We further present the performance of our framework across different optimization rounds in Table 2. Note that the results for round 1 are obtained after one entire optimization cycle with one step of adversarial chain generation and feedback-based refinement, rather than from the raw forward reasoning without optimization. Across all backbones and benchmarks, reasoning accuracy consistently improves from round 1 to round 5, with the largest gains typically occurring between the first and second, and second and third rounds, after which performance quickly saturates. Following this trend, we use the model after three rounds of optimization to generate all main results. This choice provides a good trade-off between performance and additional optimization cost, where a small number of rounds already captures most of the benefit of cycle adversarial prompt optimization and structured feedback. 4.3.3. Effect of Hyperparameter Fig. 3 examines how temperature affects reasoning sta- bility on MATH for AoT, MAD, CCoT, and our method Shuxu Chen et al.: Preprint submitted to ElsevierPage 6 of 13 CAP-CoT Figure 2: Accuracy results (%) of our framework over optimization rounds on four LLM backbones. Figure 3: Effect of LLM temperature on reasoning stability on the MATH dataset, comparing the variation in accuracy between the three baselines and our method over multiple optimization rounds. Shuxu Chen et al.: Preprint submitted to ElsevierPage 7 of 13 CAP-CoT Figure 4: Average token consumption and token efficiency per question across six datasets for different reasoning methods. across optimization rounds, with temperatures ranging from 0 to 1 in increments of 0.1. For all baselines, variation grows quickly as temperature increases, with relatively high mean variation. Our method starts at a similar level in round 1, but successive optimization rounds substantially reduce sensi- tivity to temperature: the mean variation drops below 1.0 by round 3 and continues to decrease slightly thereafter. At higher temperatures (e.g., 0.7-1.0), our optimized solver remains much more stable than AoT, MAD, and CCoT, indi- cating that adversarial optimization with structured feedback not only improves accuracy but also makes the model more robust to sampling noise. 4.4. Computational Efficiency Analysis Fig. 4 presents the token usage of CAP-CoT compared with all baselines across six datasets. As expected, single- pass methods such as CoT and AP are the most cost-efficient, since they generate only one reasoning trace per query. In contrast, methods that rely on explicit test-time scaling, either by sampling and voting such as CoT-SC and FoT, structured search over many branches (ToT, GoT), or multi- turn debating and coordination (MAD, ECON, AoT), require substantially higher token costs, often exceeding CoT by a large margin due to repeated generation, evaluation, and aggregation. CAP-CoT sits in the middle of this spectrum, as its cost is higher than simple CoT-style baselines because each optimization cycle involves additional generations from the challenger and feedback roles. But, CAP-CoT remains markedly cheaper than heavy tree-, graph-, and debate-based methods, since it does not expand into a large branching space at inference time, nor does it require long multi-round deliberation for every test query. This indicates that our cycle-based adversarial optimization provides a favorable trade-off between performance and token consumption. 5. Conclusion In this work, we propose CAP-CoT, a cycle adversarial prompt optimization framework for improving the accuracy and stability of Chain-of-Thought reasoning. CAP-CoT pairs a solver with a challenger agent that generates targeted erroneous reasoning chains to contrast the solverโs reasoning chain, and a feedback agent that compares the two chains to produce step-level feedback. This feedback refines the solver prompt to strengthen fragile reasoning steps and adapts the challenger prompt to generate more targeted future attacks, forming an iterative optimization cycle while keeping in- ference as a single-model setup. Extensive experiments on six benchmarks under four LLM backbones show that CAP- CoT consistently outperforms strong reasoning baselines. Ablations further confirm the contributions of the challenger and feedback agent, and demonstrate improved robustness and stability with a favorable token-cost trade-off. A. Appendix A: Examples Prompts for Role Agents This appendix provides example prompts used in CAP- CoT for the three roles in one optimization cycle: the Solver (ํบ ํ ), the Challenger (ํบ ํถ ), and the Feedback Agent (ํน). All roles may share the same LLM backbone but use different role instructions. For clarity and reproducibil- ity, we keep unified placeholders (e.g., ํผํํํขํก ํํขํํ ํกํํํ, Shuxu Chen et al.: Preprint submitted to ElsevierPage 8 of 13 CAP-CoT ํํกํํํกํํํฆํทํํํํํํกํํํ) that are filled with instance-specific content at runtime. A.1. Solver Prompt (ํบ ํ ) Role Definition. You are an expert reasoning engine de- signed to solve complex problems with high accuracy and stability. Your goal is to derive the correct answer through a rigorous, step-by-step Chain-of-Thought (CoT) process. Base Instructions. (1) Analyze the request: identify the core question, key variables, and constraints. (2) Step-by- step derivation: break the problem into logical sub-steps; for each step, explicitly state the premise and conclusion. (3) Self-verification: briefly check the logic of each step before moving to the next to prevent error propagation. (4) Final answer: conclude with a clear and concise final answer. Dynamic Guidelines (Iteratively Updated). This section is empty in the initial cycle. In later cycles, it is populated with specific, context-aware constraints produced by the feedback agent to address weaknesses revealed by the contrast. A placeholder example is: โDefine all variables before use, and verify unit consistency in the final computation.โ Input Task. Question: ํผํํํขํก ํํขํํ ํกํํํ. Instruction: Pro- vide your reasoning chain and final answer. A.2. Challenger Prompt (ํบ ํถ ) Role Definition. You are an adaptive adversarial challenger. Your goal is not to solve the problem correctly. Instead, generate a plausible but incorrect reasoning chain that serves as a hard negative sample to test the solverโs robustness. Core Objective. Construct a reasoning chain that matches the style and tone of a correct solution but contains a specific flaw dictated by the adversarial instruction. The error should be subtle enough to mislead a careless solver, yet logically fatal to the final answer. Adversarial Instruction (Input). This instruction specifies the error type you must inject. It can be a predefined category or a context-aware directive. Strategy Definition (Context). Detailed constraints for the current strategy are provided below. You must follow them when constructing the error. Definition: ํํกํํํกํํํฆํทํํํํํํกํํํ. Execution Guidelines. (1) Plausibility is key: avoid obvi- ous nonsense; keep a high-quality step-by-step structure. (2) Targeted sabotage: inject the error only as required by ํํกํํํกํํํฆ ํทํํํํํํกํํํ; keep the rest coherent to make the flaw hard to spot. (3) Incorrect conclusion: ensure the reasoning leads to a final answer that is wrong and distinct from the ground truth. Input Task. Question: ํผํํํขํก ํํขํํ ํกํํํ. Instruction: Gen- erate the adversarial reasoning chain following the strategy definition above. A.3. Feedback Agent Prompt (ํน) Role Definition. You are the meta-optimization controller. Your objective is to drive the solver (ํบ ํ ) toward a logically flawless Chain-of-Thought. You do this by running an evo- lutionary loop with the challenger (ํบ ํถ ). You must execute a strict two-step process: first, extract high-level reasoning principles to strengthen the solver; second, design a new adversarial strategy to stress-test the solver from a different angle in the next cycle. Input Context. (1) Question: the problem statement. (2) Challenger output (ํถ ํถ ): the adversarial reasoning chain (negative sample). (3) Solver output (ํถ ํ ): the solverโs rea- soning chain (target sample). Task Instructions (Two-Step Process). Step 1: Comparative Analysis and Solver Improvement (Optimizing ํบ ํ ). Dissect ํถ ํถ to identify the fundamental logical flaw or structural gap it exploits. Assess whether ํถ ํ is robust enough to prevent this kind of flaw; even if ํถ ํ is correct, identify weaknesses in rigor, clarity, or verification. Then synthesize a high-level improvement principle for ํบ ํ . The goal is to raise the solverโs overall reasoning standard rather than patching a single case. Step 2: Strategy Diversification (Directing ํบ ํถ ). Assume the solver will adapt to the previous error type. Choose a distinct, unexplored dimension of reasoning to test next, and formulate a new adversarial strategy that targets a potential weakness in ํถ ํ . This directive will be used to populate the challengerโs โStrategy Definitionโ in the next cycle. Output Format. [Step 1: Comparative Logic Analysis] Adversarial Logic Flaw: [technical description of the flaw in ํถ ํถ ]. Solver Logic Assessment: [evaluation of ํถ ํ โs robustness regarding this flaw]. [Step 1 Output: Logical Enhancement Directive for ํบ ํ ] Imperative principle to elevate reasoning rigor: [directive for ํบ ํ , e.g., โEnforce a strict premiseโdeductionโconclusion structure for each step, and add minimal checks for key assumptions.โ] [Step 2: Strategic Rationale] Evolutionary Direction: [why the new adversarial focus is distinct and necessary]. [Step 2 Output: Next-Step Adversarial Strategy for ํบ ํถ ] Strategy Name: [formal strategy designation]. Strategy Def- inition: [precise directive for the next negative sample, e.g., โKeep arithmetic correct but misinterpret the direction of a key constraint, and ensure the mistake appears at an early step.โ] Structured Feedback Prompt Refinement. CAP-CoT uses a unified Structured Feedback Prompt Refinement (SFPR) mechanism to convert the Feedback Agentโs outputs into prompt updates for both the Solver and the Challenger. SFPR works as follows: it first extracts the high-level reasoning principle from the feedback, removes instance-specific de- tails while keeping transferable constraints, rewrites them into concise imperative natural language, de-duplicates re- peated items and enforces a maximum number of updates, and finally appends the resulting instructions to the corre- sponding role prompt. B. Appendix B: Output Examples In this section, we provide qualitative examples (Exam- ple B.1, B.2, B.3) with typical error types to demonstrate the optimization process of CAP-CoT, showing a solver output, Shuxu Chen et al.: Preprint submitted to ElsevierPage 9 of 13 CAP-CoT Example B.1: Jump-type hard negative for enforcing constraint checks (MATH-style). Question. Solve โ ํฅ โ 1 = ํฅ โ 3 over the reals. Solver output ํถ (0) ํ . (1) Domain: require ํฅโ1 โฅ 0 and ํฅโ3 โฅ 0, so ํฅ โฅ 3. (2) Square both sides: ํฅโ1 = (ํฅโ3) 2 = ํฅ 2 โ6ํฅ+9. (3) Rearrange: 0 = ํฅ 2 โ 7ํฅ + 10 = (ํฅ โ 5)(ํฅ โ 2). (4) Candidates: ํฅ = 2 or ํฅ = 5. (5) Answer: 2, 5. Challenger output ํถ (0) ํถ (Jump error: skip the domain filter). (1) Square both sides to remove the root: ํฅ โ 1 = (ํฅ โ 3) 2 . (2) Solve (ํฅ โ 5)(ํฅ โ 2) = 0, so ํฅ = 2 or ํฅ = 5. (3) Answer: 2, 5. Feedback (step-aligned). For ํถ (0) ํ : Step (4)โ(5) retains candidates without applying the domain constraint ํฅ โฅ 3. Require an explicit candidateโ constraint checkโ substitution check pattern before concluding. For ํถ (0) ํถ : The chain hides the key validity condition by omitting domain filtering, making the extra root appear legitimate. Solver output ํถ (1) ํ (after one CAP-CoT cycle). (1) Domain: โ ํฅ โ 1 requires ํฅ โฅ 1, and equality to ํฅ โ 3 also requires ํฅ โ 3 โฅ 0, so ํฅ โฅ 3. (2) Square: ํฅ โ 1 = (ํฅ โ 3) 2 โ ํฅ 2 โ 7ํฅ + 10 = 0โ ํฅ โ 2, 5. (3) Constraint check: only ํฅ = 5 satisfies ํฅ โฅ 3; discard ํฅ = 2. (4) Substitute ํฅ = 5: โ 5 โ 1 = 2 and 5 โ 3 = 2, so it holds. (5) Final answer: 5. Example B.2: Confusion-type hard negative for unit/quantity separation (GSM8K-style). Question. Alice drives 100 miles at 50 mph and then 100 miles at 25 mph. What is her average speed for the whole trip? Solver output ํถ (0) ํ . (1) Average speed is total distance divided by total time. (2) Time: 100โ50 = 2 hours, and 100โ25 = 4 hours. (3) Total distance: 200 miles; total time: 6 hours. (4) Average speed: 200โ6 โ 33.3 mph. Challenger output ํถ (0) ํถ (Confusion error: average of rates). (1) The trip has two speeds: 50 mph and 25 mph. (2) Take their average: (50 + 25)โ2 = 37.5 mph. (3) Answer: 37.5 mph. Feedback (step-aligned). For ํถ (0) ํ : The computation is correct, but the justification in Step (1) is brief. Add an explicit reminder that averaging rates requires weighting by time or distance; otherwise it becomes the confusion exploited by ํถ (0) ํถ . For ํถ (0) ํถ : The chain confuses โmean of two speedsโ with โdistance-weighted average speed,โ ignoring that time differs across legs. Solver output ํถ (1) ํ (after one CAP-CoT cycle). (1) Average speed is total distance total time ; we cannot average mph directly unless the legs have equal time (or we apply proper weighting). (2) Compute time per leg: 100โ50 = 2 hours and 100โ25 = 4 hours. (3) Total distance = 200 miles, total time = 6 hours. (4) Average speed = 200โ6 โ 33.3 mph. a challenger-generated hard negative, step-aligned feedback, and the improved solver output after one cycle. C. Appendix C: Error Strategy Coverage and Minimality Analysis This appendix analyzes the error strategy design of the Adaptive Adversarial Challenger (ํบ ํถ ) in CAP-CoT in detail, focusing on whether the cold-start error taxonomy is crit- ical for the gains, whether CAP-CoT depends on carefully engineered combinations of error types, and whether the feedback cycle can still start and evolve with only a minimal error signal. C.1. Cold-start Error Taxonomy As described in Section 3.2, the challenger is cold- started with a lightweight error taxonomy covering four common reasoning failure modes jump, confusion, fuzzy, wrapper. The purpose of this taxonomy is not to enu- merate all possible errors, but to provide a minimal and operational bootstrap signal that makes the contrastive cycle start reliably. A key concern is whether CAP-CoT relies on this multi-type taxonomy, or whether it mainly serves Example B.3: Wrapper-type hard negative for guarding against hidden assumptions (BBH-style syllogism). Question. Consider the argument: โAll florps are glibs. Some glibs are drams. Therefore, some florps are drams.โ Is the conclusion necessarily true? Solver output ํถ (0) ํ . (1) We know florps โ glibs. (2) We also know โํฅ โถ ํฅ โ glibs โฉ drams. (3) Answer: Not necessarily true. Challenger output ํถ (0) ํถ (Wrapper error: fluent but assumes overlap). (1) Since all florps are glibs, florps live inside the set of glibs. (2) Some glibs are drams, so the โdram partโ exists inside glibs. (3) Therefore florps must include at least one element in that dram part. (4) Answer: Yes, the conclusion must be true. Feedback (step-aligned). For ํถ (0) ํ : The final judgment is correct, but Step (3) lacks a concrete refutation. Add a counterexample to show why the conclusion does not follow (to prevent the wrapper-style assumption in ํถ (0) ํถ from sounding persuasive). For ํถ (0) ํถ : The argument quietly assumes florps โฉ drams โ โ from florps โ glibs and glibs โฉ drams โ โ , which is invalid without an additional premise. Solver output ํถ (1) ํ (after one CAP-CoT cycle). (1) Premises: florps โ glibs and glibs โฉ drams โ โ . (2) These do not imply florps โฉ drams โ โ . (3) Counterexample: let glibs = ํ,ํ, florps = ํ, drams = ํ. Then all florps are glibs and some glibs are drams, but no florps are drams. (4) Final answer: the conclusion is not necessarily true. Shuxu Chen et al.: Preprint submitted to ElsevierPage 10 of 13 CAP-CoT Table C.1 Single-error cold-start ablation on three datasets (meanยฑstd over three runs). The challenger is restricted to one error type in the cold-start stage, while all other settings are identical. Cold-start Error MATH (%) GSM8K (%) BBH (%) Jump only86.8ยฑ0.32 95.9ยฑ0.21 87.4ยฑ0.28 Confusion only 86.5ยฑ0.35 96.0ยฑ0.18 87.2ยฑ0.31 Fuzzy only86.6ยฑ0.37 95.7ยฑ0.24 87.3ยฑ0.34 Wrapper only86.9ยฑ0.30 95.8ยฑ0.22 87.6ยฑ0.27 Full taxonomy 87.2ยฑ0.29 96.1ยฑ0.17 87.9ยฑ0.25 as a bootstrap mechanism. To test minimality, we run a single-error cold-start ablation. We use GPT-4o-mini as the backbone and evaluate on MATH, GSM8K, and BBH for three optimization cycles. The only change is that, in the first one to two cycles, the challenger is restricted to a single error type; all other settings (solver, feedback agent, decoding configuration) are kept the same. For inference, we use the final solver prompt with a single forward pass, without the challenger or feedback roles. Table C.1 reports the final accuracies under different single-error cold starts. It can be observed that a single error type is sufficient to start CAP-CoT, as each single-error setting reaches performance close to the full-taxonomy set- ting after two to three cycles across all three datasets. Also, differences among single-error settings are small (within about one point) and remain within the run-to-run standard deviation, suggesting that the method does not depend on any particular error type. The full taxonomy is slightly more stable on average, but with a limited margin, indicating that the primary role of the cold-start taxonomy is to make the early cycles more reliable, rather than to determine the final performance. These findings support the view that CAP- CoT does not require a finely designed or comprehensive taxonomy. Instead, the cold-start strategy mainly triggers the adversarial feedback cycle, while later improvements are driven by feedback-conditioned strategy adaptation based on the solverโs current behavior. C.2. Strategy Evolution Beyond the Cold-start Although the challenger starts with a fixed set of error types, its strategy is not constrained by that taxonomy in later cycles. In cycles two to three, we frequently observe derived error patterns that are not explicitly listed in the cold- start set, such as: omitting candidate validation (producing candidate answers but failing to check constraints), inject- ing implicit assumptions (using an unstated premise that is required for the conclusion), misinterpreting constraint direction (e.g., reversing an inequality or logical condition), and causal reversal (keeping local steps plausible while flipping causal direction). These patterns emerge as the feedback agent updates the challenger prompt to better match the solverโs current blind spots, rather than being manually specified. C.3. Case Studies The qualitative examples in Appendix B illustrate this evolution. For MATH, errors can shift from a simple Jump to a more specific failure, such as producing candidates without filtering them by domain constraints. For BBH-style logical tasks, errors can evolve from a generic Wrapper into a concrete invalid inference, such as incorrectly concluding that an intersection is non-empty from a subset relation and a separate non-empty intersection. These cases align with the ablation results above, where CAP-CoT can start from a minimal error signal, and then progressively discovers more fine-grained and more diagnostic failure modes through cycle-by-cycle feedback and challenger adaptation. command is used after appendix sections to list author credit taxonomy contribution roles tagged using in frontmatter. References [1] Bai, Y., Lv, X., Zhang, J., Lyu, H., Tang, J., Huang, Z., Du, Z., Liu, X., Zeng, A., Hou, L., et al., 2024. Longbench: A bilingual, multitask benchmark for long context understanding, in: ACL (1), p. 3119โ 3137. [2] Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al., 2024. Graph of thoughts: Solving elaborate problems with large language models, in: Proceedings of the AAAI conference on artificial intelligence, p. 17682โ17690. [3] Bi, Z., Han, K., Liu, C., Tang, Y., Wang, Y., 2025. Forest-of-thought: Scaling test-time compute for enhancing llm reasoning, in: Forty- second International Conference on Machine Learning. [4] Chen, X., Chi, R.A., Wang, X., Zhou, D., 2024. Premise order matters in reasoning with large language models, in: Proceedings of the 41st International Conference on Machine Learning, p. 6596โ6620. [5] Chia, Y.K., Chen, G., Tuan, L.A., Poria, S., Bing, L., 2023. Contrastive chain-of-thought prompting.arXiv preprint arXiv:2311.09277 . [6] Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al., 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 . [7] Dennis, M., Jaques, N., Vinitsky, E., Bayen, A., Russell, S., Critch, A., Levine, S., 2020. Emergent complexity and zero-shot transfer via unsupervised environment design. Advances in neural information processing systems 33, 13049โ13061. [8] Guo, J., Chen, X., Xia, Q., Wang, Z., Ou, J., Qin, L., Yao, S., Tian, W., 2025. Hash-rag: bridging deep hashing with retriever for efficient, fine retrieval and augmented generation, in: Findings of the Association for Computational Linguistics: ACL 2025, p. 26847โ26858. [9] Hayase, J., Borevkoviฤ, E., Carlini, N., Tramรจr, F., Nasr, M., 2024. Query-based adversarial prompt generation. Advances in Neural Information Processing Systems 37, 128260โ128279. [10] Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., Steinhardt, J., 2021. Measuring mathematical problem solving with the math dataset, in: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). [11] Karras, T., Aittala, M., Laine, S., Hรคrkรถnen, E., Hellsten, J., Lehtinen, J., Aila, T., 2021. Alias-free generative adversarial networks, in: Proc. NeurIPS. [12] Khan, A., Hughes, J., Valentine, D., Ruis, L., Sachan, K., Radhakr- ishnan, A., Grefenstette, E., Bowman, S.R., Rocktรคschel, T., Perez, E., 2024. Debating with more persuasive llms leads to more truthful answers, in: International Conference on Machine Learning, PMLR. p. 23662โ23733. Shuxu Chen et al.: Preprint submitted to ElsevierPage 11 of 13 CAP-CoT [13] Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., Krishnan, D., 2020. Supervised contrastive learning. Advances in neural information processing systems 33, 18661โ18673. [14] Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y., 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems 35, 22199โ22213. [15] Li, C., Liu, J., Zhang, S., Jian, H., Ni, H., Lee, L.H., Bae, S.H., Wang, G., Yang, Y., Zhang, C., 2026a. Experience transfer for multimodal llm agents in minecraft game. arXiv preprint arXiv:2604.05533 . [16] Li, Y., Lan, T., Qi, Z., 2026b. When right meets wrong: Bilateral context conditioning with reward-confidence correction for grpo. arXiv preprint arXiv:2603.13134 . [17] Lin, B.Y., Le Bras, R., Richardson, K., Sabharwal, A., Poovendran, R., Clark, P., Choi, Y., 2025. Zebralogic: On the scaling limits of llms for logical reasoning, in: Forty-second International Conference on Machine Learning. [18] Lin, X., Dai, Z., Verma, A., Ng, S.K., Jaillet, P., Low, B.K.H., 2024. Prompt optimization with human feedback, in: ICML 2024 Workshop on Models of Human Feedback for AI Alignment. [19] Ling, Z., Fang, Y., Li, X., Huang, Z., Lee, M., Memisevic, R., Su, H., 2023. Deductive verification of chain-of-thought reasoning. Ad- vances in Neural Information Processing Systems 36, 36407โ36433. [20] Liu, Y., Deng, G., Li, Y., Wang, K., Wang, Z., Wang, X., Zhang, T., Liu, Y., Wang, H., Zheng, Y., et al., 2023. Prompt injection attack against llm-integrated applications. arXiv preprint arXiv:2306.05499 . [21] Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al., 2023. Self- refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems 36, 46534โ46594. [22] Mazeika, M., Phan, L., Yin, X., Zou, A., Wang, Z., Mu, N., Sakhaee, E., Li, N., Basart, S., Li, B., et al., 2024. Harmbench: A stan- dardized evaluation framework for automated red teaming and robust refusal, in: International Conference on Machine Learning, PMLR. p. 35181โ35224. [23] Ou, J., Guo, J., Jiang, S., Wang, Z., Qin, L., Yao, S., Tian, W., 2025. Accelerating adaptive retrieval augmented generation via instruction- driven representation reduction of retrieval overlaps, in: Findings of the Association for Computational Linguistics: ACL 2025, p. 26983โ27000. [24] Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al., 2022. Training language models to follow instructions with human feedback. Ad- vances in neural information processing systems 35, 27730โ27744. [25] Paulus, A., Zharmagambetov, A., Guo, C., Amos, B., Tian, Y., 2025. Advprompter: Fast adaptive adversarial prompting for llms, in: Forty- second International Conference on Machine Learning. [26] Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., Irving, G., 2022. Red teaming language models with language models, in: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 3419โ3448. [27] Samvelyan, M., Raparthy, S.C., Lupu, A., Hambro, E., Markosyan, A.H., Bhatt, M., Mao, Y., Jiang, M., Parker-Holder, J., Foerster, J., et al., 2024. Rainbow teaming: Open-ended generation of diverse adversarial prompts. Advances in Neural Information Processing Systems 37, 69747โ69786. [28] Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., Yao, S., 2023. Reflexion: Language agents with verbal reinforcement learning. Ad- vances in Neural Information Processing Systems 36, 8634โ8652. [29] Suzgun, M., Scales, N., Schรคrli, N., Gehrmann, S., Tay, Y., Chung, H.W., Chowdhery, A., Le, Q.V., Chi, E.H., Zhou, D., et al., 2023. Challenging big-bench tasks and whether chain-of-thought can solve them, in: ACL (Findings). [30] Teng, F., Yu, Z., Shi, Q., Zhang, J., Wu, C., Luo, Y., 2025. Atom of thoughts for markov llm test-time scaling. CoRR . [31] Turpin, M., Michael, J., Perez, E., Bowman, S., 2023. Language models donโt always say what they think: Unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems 36, 74952โ74965. [32] Wang, B., Min, S., Deng, X., Shen, J., Wu, Y., Zettlemoyer, L., Sun, H., 2023a. Towards understanding chain-of-thought prompting: An empirical study of what matters, in: Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers), p. 2717โ2739. [33] Wang, C., Zhang, Y., Wang, W., Zhao, X., Feng, F., He, X., Chua, T.S., 2025. Think-while-generating: On-the-fly reasoning for personalized long-form generation. arXiv preprint arXiv:2512.06690 . [34] Wang, X., Wei, J., Schuurmans, D., Le, Q.V., Chi, E.H., Narang, S., Chowdhery, A., Zhou, D., 2023b. Self-consistency improves chain of thought reasoning in language models, in: The Eleventh International Conference on Learning Representations. [35] Wang, X., Zhang, C., Sun, Q., Huang, Z., Lu, C., Zheng, S., Ma, Z., Qin, C., Yang, Y., Shen, H., 2026. Transforming external knowledge into triplets for enhanced retrieval in rag of llms. arXiv preprint arXiv:2604.12610 . [36] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al., 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35, 24824โ24837. [37] Xi, Z., Jin, S., Zhou, Y., Zheng, R., Gao, S., Liu, J., Gui, T., Zhang, Q., Huang, X.J., 2023. Self-polish: Enhance reasoning in large language models via problem refinement, in: Findings of the Association for Computational Linguistics: EMNLP 2023, p. 11383โ11406. [38] Xiao, C., Li, B., Zhu, J.Y., He, W., Liu, M., Song, D., 2018. Generat- ing adversarial examples with adversarial networks, in: Proceedings of the 27th International Joint Conference on Artificial Intelligence, p. 3905โ3911. [39] Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q.V., Zhou, D., Chen, X., 2024. Large language models as optimizers, in: The Twelfth International Conference on Learning Representations. [40] Yang, M., Huang, E., Zhang, L., Surdeanu, M., Wang, W.Y., Pan, L., 2025a. How is llm reasoning distracted by irrelevant context? an analysis using a controlled benchmark, in: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 13340โ13358. [41] Yang, Y., Ma, Y., Feng, H., Cheng, Y., Han, Z., 2025b. Minimiz- ing hallucinations and communication costs: Adversarial debate and voting mechanisms in llm-based multi-agents. Applied Sciences 15, 3676. [42] Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., Manning, C.D., 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering, in: Proceedings of the 2018 Confer- ence on Empirical Methods in Natural Language Processing, p. 2369โ2380. [43] Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., Narasimhan, K., 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information process- ing systems 36, 11809โ11822. [44] Yao, Y., Cen, Z., Li, M., Han, W., Zhang, Y., Liu, E., Liu, Z., Gan, C., Zhao, D., 2025. Your language model may think too rigidly: Achieving reasoning consistency with symmetry-enhanced training. arXiv preprint arXiv:2502.17800 . [45] Yasunaga, M., Chen, X., Li, Y., Pasupat, P., Leskovec, J., Liang, P., Chi, E.H., Zhou, D., 2024. Large language models as analogical reasoners, in: The Twelfth International Conference on Learning Representations. [46] Yi, X., Zhou, Z., Cao, C., Niu, Q., Liu, T., Han, B., 2025a. From debate to equilibrium: Belief-driven multi-agent llm reasoning via bayesian nash equilibrium, in: Forty-second International Conference on Machine Learning. [47] Yi, X., Zhou, Z., Cao, C., Niu, Q., Liu, T., Han, B., 2025b. From debate to equilibrium: Belief-driven multi-agent llm reasoning via bayesian nash equilibrium, in: Forty-second International Conference on Machine Learning. Shuxu Chen et al.: Preprint submitted to ElsevierPage 12 of 13 CAP-CoT [48] Zhang, J., Sun, Q., Zhang, C., Wang, X., Huang, Z., Zhou, Y., Zheng, P., lok Andy Tai, C., Bae, S.H., Ma, Z., Qin, C., Guo, J., Yang, Y., Shen, H., 2026a. Tda-rc: Task-driven alignment for knowledge- based reasoning chains in large language models. arXiv preprint arXiv:2604.04942 . [49] Zhang, J., Xiang, J., Yu, Z., Teng, F., Chen, X.H., Chen, J., Zhuge, M., Cheng, X., Hong, S., Wang, J., et al., 2025a. Aflow: Automating agen- tic workflow generation, in: The Thirteenth International Conference on Learning Representations. [50] Zhang, J., Zhang, C., Chen, S., Huang, Z., Zheng, P., Wang, Z., Guo, P., Mo, F., Bae, S.H., Zou, J., Wei, J., Yang, Y., 2026b. Lightweight llm agent memory with small language models. arXiv preprint arXiv:2604.07798 . [51] Zhang, J., Zhang, C., Chen, S., Liu, Y., Li, C., Sun, Q., Yuan, S., Puspitasari, F.D., Han, D., Wang, G., et al., 2026c. Text summariza- tion via global structure awareness, in: The Fourteenth International Conference on Learning Representations. [52] Zhang, J., Zhang, C., Chen, S., Wang, X., Huang, Z., Zheng, P., Yuan, S., Zheng, S., Sun, Q., Zou, J., et al., 2026d. Learning global hypothesis space for enhancing synergistic reasoning chain, in: The Fourteenth International Conference on Learning Representations. [53] Zhang, M., Wei, W., Zhou, Z., Liu, W., Zhang, J., Belatreche, A., Yang, Y., 2025b. Spike-driven lightweight large language model with evolutionary computation. IEEE Transactions on Evolutionary Computation , 1โ1doi:10.1109/TEVC.2025.3606613. [54] Zhao, Q., Huang, Y., Lv, T., Cui, L., Sun, Q., Mao, S., Zhang, X., Xin, Y., Yin, Q., Li, S., et al., 2024. Mmlu-cf: A contamination-free multi-task language understanding benchmark. CoRR . [55] Zheng, P., Zhang, C., Mo, J.H., Li, G., Zhang, J., Zhang, J., Cao, S., Zheng, S., Qin, C., Wang, G., Yang, Y., 2026. Llava-fa: Learning fourier approximation for compressing large multimodal models. arXiv preprint arXiv:2602.00135 . [56] Zhou, Y., Li, Y., Cheng, D., Fan, H., Cheng, Y., 2026. Look inward to explore outward: Learning temperature policy from llm internal states via hierarchical rl. arXiv preprint arXiv:2602.13035 . [57] Zhu, J.Y., Park, T., Isola, P., Efros, A.A., 2017. Unpaired image-to- image translation using cycle-consistent adversarial networks, in: Pro- ceedings of the IEEE International Conference on Computer Vision, p. 2223โ2232. Shuxu Chen et al.: Preprint submitted to ElsevierPage 13 of 13