Paper deep dive
CAS: Conformalized Agentic Search via Adaptive Retrieval and Policy Weighting
Zixi Zhu, Jiayuan Su, Jian Zhang, Yu Lin, Hongwei Wang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/24/2026, 5:17:18 AM
Summary
The paper introduces Conformalized Agentic Search (CAS), a framework designed to enhance the reliability of Search Agents during reinforcement learning (RL) fine-tuning. CAS addresses issues like evidence loss, noise inclusion, and hallucination by integrating Conformal Prediction (CP) into both the retrieval and training phases. On the retrieval side, Adaptive Prediction Set (APS) dynamically truncates documents based on query difficulty to ensure statistical coverage. On the training side, Adaptive Conformal Inference (ACI) quantifies answer confidence and penalizes low-confidence trajectories within the Group Relative Policy Optimization (GRPO) objective. Experiments on single-hop and multi-hop QA datasets show that CAS improves reasoning accuracy and reduces redundant tool invocations compared to baselines like Search-R1 and Search-R2.
Entities (18)
Relation Signals (13)
Jiayuan Su → affiliatedwith → Zhejiang University
confidence 95% · Jiayuan Su ... Affiliation: State Key Laboratory of CAD&CG, Zhejiang University.
Zixi Zhu → affiliatedwith → Zhejiang University
confidence 95% · Zixi Zhu ... Affiliation: State Key Laboratory of CAD&CG, Zhejiang University.
Hongwei Wang → affiliatedwith → Zhejiang University
confidence 95% · Hongwei Wang ... Affiliation: State Key Laboratory of CAD&CG, Zhejiang University.
Yu Lin → affiliatedwith → Zhejiang University
confidence 95% · Yu Lin ... Affiliation: State Key Laboratory of CAD&CG, Zhejiang University.
Jian Zhang → affiliatedwith → ZJU-UIUC Institute
confidence 95% · Jian Zhang Affiliation: ZJU-UIUC Institute
CAS → uses → Adaptive Prediction Set
confidence 95% · on the retrieval side, an Adaptive Prediction Set (APS), a specific CP realization, translates statistical coverage into dynamic document truncation
CAS → uses → Adaptive Conformal Inference
confidence 95% · on the training side, Adaptive Conformal Inference (ACI), a dynamic CP algorithm, dynamically constructs prediction sets with controllable coverage
CAS → evaluatedon → HotpotQA
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Search Agents face a severe reliability crisis during reinforcement learning (RL) fine-tuning. Heuristic Top-K retrieval often causes critical evidence loss or noise inclusion, while over-confidence induced by progressive RL leads to hallucinated answers and redundant searches. To build highly reliable agents, we introduce Conformal Prediction (CP) and propose Conformalized Agentic Search (CAS). This framework establishes reliability guarantees on both the retrieval and training sides: on the retrieval side, an Adaptive Prediction Set (APS), a specific CP realization, translates statistical coverage into dynamic document truncation to construct prediction sets that are adaptive in size; on the training side, Adaptive Conformal Inference (ACI), a dynamic CP algorithm, dynamically constructs prediction sets with controllable coverage to quantify answer confidence, which is then used to penalize low-confidence trajectories within the Group Relative Policy Optimization (GRPO) objective, ensuring the model learns only from reliable ones. Experiments across single-hop and multi-hop QA datasets demonstrate that our framework significantly improves reasoning accuracy while drastically reducing redundant tool invocations, establishing a highly reliable and efficient agent paradigm. Our code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.20771v1
- Canonical: https://arxiv.org/abs/2608.20771v1
Trouble viewing inline? Open PDF directly →
Full Text
82,838 characters extracted from source content.
Expand or collapse full text
CAS: Conformalized Agentic Search via Adaptive Retrieval and Policy Weighting Zixi Zhu Jiayuan Su Affiliation: State Key Laboratory of CAD&CG, Zhejiang University. Jian Zhang Affiliation: ZJU-UIUC Institute, Zhejiang University. Affiliation: Tencent Inc.zixizhu, 12221038, hongweiwang@zju.edu.cnyulin@intl.zju.edu.cnmatt.jiayuan.su@gmail.com Yu Lin Hongwei Wang Abstract Search Agents face a severe reliability crisis during reinforcement learning (RL) fine-tuning. Heuristic Top-K retrieval often causes critical evidence loss or noise inclusion, while overconfidence induced by progressive RL leads to hallucinated answers and redundant searches. To build highly reliable agents, we introduce Conformal Prediction (CP) and propose Conformalized Agentic Search (CAS). This framework establishes reliability guarantees on both the retrieval and training sides: on the retrieval side, an Adaptive Prediction Set (APS), a specific CP realization, translates statistical coverage into dynamic document truncation to construct prediction sets that are adaptive in size; on the training side, Adaptive Conformal Inference (ACI), a dynamic CP algorithm, dynamically constructs prediction sets with controllable coverage to quantify answer confidence, which is then used to penalize low-confidence trajectories within the Group Relative Policy Optimization (GRPO) objective, ensuring the model learns only from reliable ones. Experiments across single-hop and multi-hop QA datasets demonstrate that our framework significantly improves reasoning accuracy while drastically reducing redundant tool invocations, establishing a highly reliable and efficient agent paradigm. Our code is available at https://github.com/S1llyBird/CAS. 22footnotetext: Corresponding authors. Figure 1: The CAS framework. (A) APS dynamically sizes the retrieved document set based on local query difficulty. (B) ACI modulates the GRPO policy loss by penalizing low-confidence trajectories (NLL>q1−αtNLL>q_1- _t) to enforce reliable policy optimization. 1 Introduction Large Language Models (LLMs) have significantly advanced complex problem-solving by integrating external knowledge 34; 26. While traditional Retrieval-Augmented Generation (RAG) employs a static "retrieve-then-generate" paradigm 24, the recent emergence of Agentic Search offers a more dynamic and autonomous approach 17; 14; 51. By interleaving internal reasoning steps with external information-gathering actions within a continuous generation trajectory, the agent autonomously plans when to retrieve and how to integrate newly acquired knowledge 48. However, fine-tuning these agents via reinforcement learning (RL) carries inherent risks of unreliability. First, on the retrieval side, heuristic Top-K truncation is inherently unreliable. Given varying query difficulties, a fixed K inevitably leads to the omission of critical facts or the inclusion of distracting noise 27. Second, on the training side, LLMs often exhibit overconfidence as RL progresses 23, leading the model to generate hallucinated responses. Without effective confidence constraints, Search Agents are easily trapped in a cycle of highly inefficient, redundant tool invocations 49. To address these risks, we adopt Conformal Prediction (CP) 3, a principled statistical framework that quantifies model uncertainty with rigorous theoretical guarantees. Unlike heuristic methods, CP provides strong finite-sample coverage guarantees: given a user-specified error rate α, it constructs a prediction set that satisfies a target coverage level of at least 1−α1-α. These properties make CP a theoretically grounded choice for establishing reliability in autonomous systems. Building on this, we propose Conformalized Agentic Search (CAS), which simultaneously applies CP to both the retrieval and training sides to provide rigorous statistical guarantees 20; 37. On the retrieval side, we implement CP via the Adaptive Prediction Set (APS) method 33. While ensuring a strict marginal coverage guarantee, APS dynamically adjusts the size of the set of retrieved items based on the model’s "uncertainty" regarding the current input 4. On the training side, to mitigate the severe calibration degradation and overconfidence inherent in standard RL, we utilize Adaptive Conformal Inference (ACI) 11, a dynamic CP algorithm. ACI dynamically constructs prediction sets with controllable coverage to quantify answer reliability during training. We optimize the Group Relative Policy Optimization (GRPO) 35 process by penalizing low-confidence trajectories (both blind overconfidence and erratic underconfidence), ensuring the model learns only from highly reliable reasoning paths. In summary, our core contributions are threefold: • A Reliable Theoretical Framework: We propose CAS, the first framework to introduce CP into the RL fine-tuning of search agents, providing rigorous statistical guarantees for reasoning and retrieval reliability. • CP Constraints on Both Sides: We implement APS on the retrieval side to construct prediction sets that are adaptive in size, ensuring the inclusion of correct evidence. Concurrently, we apply ACI on the training side to optimize the GRPO process, mitigating low-confidence outputs. • Superior Accuracy and Efficiency: Extensive experiments across single-hop and multi-hop QA datasets demonstrate that our framework significantly improves reasoning accuracy and training stability while drastically reducing redundant tool invocations, yielding a highly reliable and efficient agent paradigm. 2 Conformal Prediction CP 3 is a principled statistical framework that quantifies uncertainty with rigorous coverage guarantees, regardless of the underlying model or data distribution. Central to CP is a non-conformity score function s(x,y)s(x,y), which measures the “unusualness” of a candidate output y given an input x. Let (X,Y)(X,Y) be a sample, where X represents the input and Y represents the output. Suppose we have a calibration set of n samples, denoted as (Xi,Yi)i=1n(X_i,Y_i)_i=1^n, and a test sample (Xtest,Ytest)(X_test,Y_test) drawn independently and identically (i.i.d.) from the same underlying distribution. Given a user-specified target error rate α∈(0,1)α∈(0,1), CP computes a quantile threshold q q corresponding to the ⌈(n+1)(1−α)⌉n (n+1)(1-α) n empirical quantile of the calibration scores. It then constructs a prediction set 1−α(Xtest)C_1-α(X_test) defined as: 1−α(Xtest)=y∈:s(Xtest,y)≤q^C_1-α(X_test)=\y :s(X_test,y)≤ q\ (1) Under the i.i.d. assumption, this procedure formally guarantees marginal coverage: P(Ytest∈1−α(Xtest))≥1−αP(Y_test _1-α(X_test))≥ 1-α. See Appendix B.1 for the formal proof. 2.1 Adaptive Prediction Set As a specific realization of the CP framework for classification and generative tasks, APS 33 constructs prediction sets by defining a specialized non-conformity score. Specifically, given sorted predictive probabilities π(1)(x)≥⋯≥π(n)(x) _(1)(x)≥·≥ _(n)(x), APS defines the non-conformity score s(x,y)s(x,y) as the cumulative mass up to the true label y: s=∑j=1L(y)π(j)(x)s= _j=1^L(y) _(j)(x), where L(y)L(y) denotes the rank of y. During the CP inference phase, APS identifies the minimum index k to form the prediction set such that the cumulative probability exceeds the calibrated threshold q q: ∑i=1kπ(i)(x)≥q _i=1^k _(i)(x)≥ q (2) This mechanism adaptively yields compact sets for confident inputs and expanded sets for ambiguous ones, strictly maintaining the 1−α1-α CP coverage guarantee while asymptotically approximating conditional coverage. See Appendix B.2 for the formal proof and discussions on its conditional coverage. 2.2 Adaptive Conformal Inference While the standard CP framework fundamentally relies on the exchangeability (i.i.d.) assumption, ACI 11 is a dynamic extension designed to handle data streams where the underlying data distribution may change over time. Instead of maintaining a static target error rate, ACI introduces a time-varying error parameter αt _t. At each time step t, we observe a test point (Xt,Yt)(X_t,Y_t), where XtX_t is the input and YtY_t is the true response. The algorithm evaluates whether YtY_t was contained within the previous prediction set via the empirical miscoverage indicator: errt:=1,if Yt∉t(αt)0,otherwiseerr_t:= cases1,&if Y_t _t( _t)\\ 0,&otherwise cases (3) where t(αt):=y∈:s(Xt,y)≤Q^t(1−αt)C_t( _t):=\y :s(X_t,y)≤ Q_t(1- _t)\ is the dynamic prediction set, and Q^t(⋅) Q_t(·) is the empirical quantile function. Given a long-term target error rate ρ and a step size γ>0γ>0, ACI updates the error parameter via a simple online rule: αt+1=αt+γ(ρ−errt) _t+1= _t+γ(ρ-err_t) (4) This recursive mechanism acts as a feedback loop: miscoverage (errt=1err_t=1) decreases αt _t, thereby expanding the subsequent prediction set to be more conservative. Conversely, success (errt=0err_t=0) increases αt _t, tightening the set. By continuously adapting αt _t, ACI preserves valid uncertainty quantification even when the data distribution changes over time. The formal proof of this dynamic guarantee is provided in Appendix B.3. 3 Methodology We present the CAS framework (Figure 1) to reliably bridge LLMs’ internal reasoning with external retrieval. We introduce two synergistic modules for statistical reliability: APS to bound dynamic retrieval uncertainty, and ACI to penalize low-confidence trajectories during policy optimization. 3.1 Overview of Agentic Search Our policy model πθ(y|x) _θ(y|x) employs a strict generative grammar to interleave internal reasoning with external actions. Given an input x, the model initiates reasoning within <think>...</think> tags.Upon reaching a knowledge boundary, it emits a query q enclosed in <search>...</search> tags,, halting generation to invoke an external search engine S. Crucially, rather than using a fixed top-k, the raw retrieved evidence D=(q)D=S(q) is dynamically truncated into a reliable subset DAPSD_APS via an APS. This subset is then wrapped in <information>...</information> tags and appended to the context. This generation-retrieval cycle repeats until the model outputs its final prediction apreda_pred inside <answer>...</answer> tags. The complete prompt template is provided in Table 5. 3.2 Retrieval Side via APS Traditional tool-use frameworks typically append a fixed number of top-k results to the context, which often injects redundant noise or truncates critical information. To rigorously bound the uncertainty of external evidence, we frame our dynamic retrieval mechanism within the general CP paradigm (Eq. (1)). However, while standard CP methods guarantee marginal coverage across the data distribution, they fail to guarantee conditional coverage—often failing to adapt to the specific difficulty of a given input. To heuristically approximate conditional coverage, we specify αAPS _APS and construct a reliable document subset by implementing the APS detailed in Section 2.1, thereby obtaining the calibrated threshold q^APS q_APS. Given a query q, the search engine S returns an initial candidate set D=d1,d2,…,dnD=\d_1,d_2,…,d_n\ accompanied by their raw retrieval scores. We normalize these scores using a softmax operation to obtain a relevance probability p(di|q)p(d_i|q) for each document. Following the rigorous APS inference procedure, our system identifies the truncation index k by accumulating these probabilities until the mass first exceeds the calibrated threshold q^APS q_APS. By dynamically adjusting to the conditional probability distribution of q, this process yields a statistically guaranteed subset DAPS=d1,…,dkD_APS=\d_1,…,d_k\. Crucially, this provides a statistical guarantee that ensures a 1−αAPS1- _APS coverage rate while adapting the context length to the difficulty of q. 3.3 Reward Design In our RL framework, we train the policy πθ _θ using a rule-based reward r(x,y)=racc+rfmtr(x,y)=r_acc+r_fmt. We define racc=EM(apred,agold)∈0,1r_acc=EM(a_pred,a_gold)∈\0,1\ as the exact match indicator. To enforce structural integrity, rfmt(y)r_fmt(y) incorporates two Boolean indicators, validI_valid and ansI_ans, representing strict grammatical correctness and the successful generation of the <answer> boundary, respectively. With a scaling factor γ=0.2γ=0.2, the format reward is formulated as: rfmt(y)=γ⋅[−racc(1−valid)+(1−racc)(valid+12ans(1−valid))]r_fmt(y)=γ· [-r_acc(1-I_valid)\\ +(1-r_acc) (I_valid+ 12I_ans(1-I_valid) ) ] (5) This formulation rewards the correct format: when the answer is correct (racc=1r_acc=1), it strictly applies a −γ-γ penalty for format violations to prevent reward hacking. Conversely, when the answer is incorrect (racc=0r_acc=0), it provides a dense intermediate signal (γ for full validity, or 12γ 12γ for partial structural effort) to guide the model toward using the correct format. 3.4 Training Side via ACI Standard CP relies on the strict exchangeability (i.i.d.) assumption. However, during RL, the policy πθ _θ continuously evolves, and the prevalent use of binary rewards often leads to model overconfidence. To mitigate the redundant invocations and hallucinated outputs caused by this overconfidence, we employ ACI. In our framework, at each RL iteration t, for the i-th sampled trajectory yiy_i given input xtx_t, we define its non-conformity score s(xt,yi)s(x_t,y_i) as the Negative Log-Likelihood (NLL) 31 of the generated tokens strictly within the <answer>...</answer> tags. Let ℳiM_i denote the set of token indices corresponding to these final answer tokens. The score is formulated as: s(xt,yi)=∑j∈ℳi(−logπθ(yj|y<j,xt))|ℳi|s(x_t,y_i)= _j _i(- _θ(y_j|y_<j,x_t))|M_i| (6) Given a pre-specified target error rate ρ, instead of using the standard update rule, we employ a smoothed ACI update mechanism over previously observed data to mitigate local variations in the error rate 11. Specifically, we update the error rate αt _t by evaluating the recent empirical miscoverage frequency using an exponentially weighted moving average of past errors: αt+1=αt+γ(ρ−∑s=1tvserrs) _t+1= _t+γ (ρ- _s=1^tv_serr_s ) (7) where vs1≤s≤t\v_s\_1≤ s≤ t is a sequence of increasing weights such that ∑s=1tvs=1 _s=1^tv_s=1. In practice, we define the temporal weights with a smoothing factor of 0.950.95 as vs:=0.95t−s∑s′=1t0.95t−s′.v_s:= 0.95^t-s _s =1^t0.95^t-s . This approach effectively produces smoother trajectories for αt _t with less local variation. The updated αt _t is then used to dynamically adjust the quantile threshold q^t=Q^t(1−αt) q_t= Q_t(1- _t). Specifically, the empirical quantile function Q^t(⋅) Q_t(·) is evaluated over a rolling calibration window of past scores from iteration r=max(1,t−2000)r= (1,t-2000) to t−1t-1. This threshold q^t q_t is subsequently employed to partition low-confidence trajectories. We employ GRPO 35 for the RL training. For a given input xtx_t at iteration t, the policy samples a group of G trajectories y1,y2,…,yG\y_1,y_2,…,y_G\. GRPO optimizes the policy by computing the relative advantage AiA_i for each trajectory yiy_i, which is obtained by normalizing its reward RiR_i within the group: Ai=Ri−mean()std()A_i= R_i-mean(R)std(R). Concurrently, to penalize low-confidence sequences generated during training, we introduce a discount factor η∈(0,1)η∈(0,1) for samples falling into the low-confidence set. Accordingly, we minimize the ACI-guided GRPO loss function as follows: ℒGRPO(θ)=[1G∑i=1GωiLiclip(θ)−βDKL(πθ||πref)]L_GRPO(θ)=E [ 1G _i=1^G _iL_i^clip(θ)-β D_KL( _θ|| _ref) ] (8) where βDKLβ D_KL is the KL divergence penalty against the reference model πref _ref, and Liclip(θ)L_i^clip(θ) is the standard GRPO clipped objective function driven by the advantage AiA_i. The dynamic confidence-based weight ωi _i is defined as: ωi=1,if s(xt,yi)≤q^tη,otherwise _i= cases1,&if s(x_t,y_i)≤ q_t\\ η,&otherwise cases (9) Crucially, this weighting mechanism synergizes with the RL advantage AiA_i to provide a dual-constraint on model reliability. For unconfident lucky guesses (s(xt,yi)>q^ts(x_t,y_i)> q_t with Ai>0A_i>0), the positive reinforcement is discounted by η, preventing the model from learning to guess. Conversely, if the model is confidently incorrect (s(xt,yi)≤q^ts(x_t,y_i)≤ q_t but yielding Ai<0A_i<0), the full weight (ωi=1 _i=1) ensures the model receives the maximum penalty. To effectively penalize low-confidence samples during early training while preventing over-penalization of relatively high-confidence trajectories classified as low-confidence after the model converges, we set the discount factor to η=0.5η=0.5. The complete procedure of CAS is formally presented in Algorithm 1. 4 Experiments Table 1: The main results on seven datasets. †/⋆ / represents in-domain/out-of-domain datasets. The best and second best performances are set as bold and underlined, respectively. Methods General QA Multi-Hop QA Average NQ† TriviaQA⋆ PopQA⋆ HotpotQA† 2WikiMultiHopQA⋆ Musique⋆ Bamboogle⋆ Direct Inference 0.106 0.288 0.108 0.149 0.244 0.020 0.024 0.134 CoT 0.023 0.032 0.005 0.021 0.021 0.002 0.000 0.015 RAG 0.348 0.544 0.387 0.255 0.226 0.047 0.080 0.270 IRCoT 0.111 0.312 0.200 0.164 0.171 0.067 0.240 0.181 Search-o1 0.238 0.472 0.262 0.221 0.218 0.054 0.320 0.255 SFT 0.249 0.292 0.104 0.186 0.248 0.044 0.112 0.176 R1-base 0.226 0.455 0.173 0.201 0.268 0.055 0.224 0.229 R1-instruct 0.210 0.449 0.171 0.208 0.275 0.060 0.192 0.224 Rejection Sampling 0.294 0.488 0.332 0.240 0.233 0.059 0.210 0.265 Search-R1 (Qwen2.5-3B-Instruct) 0.397 0.565 0.391 0.331 0.310 0.124 0.232 0.336 Search-R1 (Qwen3-8B) 0.440 0.631 0.418 0.372 0.355 0.157 0.430 0.400 Search-R2 (Qwen3-8B)11 1 Equal contribution. 0.477 0.676 0.466 0.412 0.405 0.172 0.512 0.446 Ours (Qwen2.5-3B-Instruct) 0.441 0.607 0.447 0.407 0.385 0.169 0.352 0.401 Ours (Qwen3-8B) 0.490 0.668 0.465 0.463 0.431 0.213 0.520 0.464 Table 2: Performance improvements of our method compared to Search-R1 and Search-R2 on General QA (single-hop) and Multi-Hop QA. Δ denotes the absolute performance gain. Methods General QA Multi-Hop QA Overall Qwen2.5-3B-Instruct Search-R1 0.451 0.249 0.336 Ours 0.498 0.328 0.401 Improvement (Δ ) +0.047 +0.079 +0.065 Qwen3-8B Search-R1 0.496 0.329 0.400 Search-R2 0.540 0.375 0.446 Ours 0.541 0.407 0.464 Improvement (Δ ) vs. Search-R1 +0.045 +0.078 +0.064 Improvement (Δ ) vs. Search-R2 +0.001 +0.032 +0.018 4.1 Experimental Setup Datasets. To comprehensively evaluate CAS, we conduct experiments on seven diverse open-domain Question Answering (QA) datasets covering both single-hop and complex reasoning capabilities: NQ 21, TriviaQA 18, PopQA 29, HotpotQA 47, 2WikiMultihopQA (2Wiki) 15, MuSiQue 39, and Bamboogle 30. For training our RL framework, we construct a mixed training corpus using the training splits of NQ and HotpotQA. Baselines. We compare our method against a comprehensive suite of competitive baselines, which can be logically categorized into four groups: inference without retrieval, including Direct Inference and CoT reasoning 43; inference with retrieval, comprising standard RAG 24, IRCoT 40, and Search-o1 25; fine-tuning based methods, which involve Supervised Fine-Tuning (SFT) 6, RL-based fine-tuning without search (R1) 12, and rejection sampling with a search engine 1; and finally, our reference backbone, Search-R1 17, along with Search-R2 14. Implementation Details. We initialize our policy model using Qwen2.5-3B 32 and Qwen3-8B 46. For the retrieval module, we utilize the dense retriever E5 42, paired with the 2018 Wikipedia dump 19 as the external knowledge base. During the GRPO training phase, we set the group rollout size to G=5G=5 and sample 512 prompts per training step. To prevent infinite generation, the maximum number of assistant-search interaction rounds is capped at 4. The learning rate is set to 1×10−61× 10^-6. For the ACI module, the step size is empirically set to γ=0.005γ=0.005. All models are evaluated using the Exact Match (EM) metric. We provide more details in Appendix C. Conformal Settings. We set αAPS=0.2 _APS=0.2 and the target error rate ρ=0.25ρ=0.25 for ACI. The ACI calibration set C comprises 150 randomly sampled queries from a mixture of NQ and HotpotQA training splits. For ACI, we evaluate the untrained backbone on C to compute the initial non-conformity scores (Eq. (6)), establishing the base empirical quantile qα0q_ _0 to bootstrap the dynamic tracking process. For APS calibration, we construct a distinct calibration set APSC_APS. We utilize DeepSeek-V3.2 8 to decompose multi-hop queries from the original 150 samples. These decomposed sub-queries, alongside the original single-hop questions, form APSC_APS, which totals 239 queries. Given the target αAPS=0.2 _APS=0.2, this sample size provides a statistical error margin of ϵ=0.026ε=0.026 for the actual coverage 3. DeepSeek-V3.2 then acts as the judge to locate the ground-truth documents within APSC_APS, yielding the calibrated threshold q^APS q_APS as formulated in Section 3.2. 4.2 Main Results Table 1 and Table 2 present the comprehensive evaluation results of our method against all baselines across the seven datasets. We summarize the key findings as follows: CAS achieves superior performance across all evaluated settings. Specifically, on the Qwen3-8B backbone, our method achieves the highest overall average score of 0.464, outperforming the strong baseline Search-R2 (0.446) and substantially surpassing Search-R1 (0.400). A similar trend is observed on the Qwen2.5-3B-Instruct backbone, where our method achieves an average score of 0.401, improving upon Search-R1 by an absolute margin of +0.065. This consistent superiority across different model scales highlights the generalizability and robustness of CAS. 11footnotetext: As Search-R2 is closed-source, we are unable to evaluate it on Qwen2.5-3B-Instruct. Our approach demonstrates exceptional performance across both complex multi-step and straightforward single-hop scenarios. As shown in Table 2, on Multi-Hop QA datasets, our method yields massive gains, outperforming Search-R1 by +0.079 (3B) and +0.078 (8B), and surpassing the highly optimized Search-R2 by +0.032. On General QA (single-hop) tasks, our models also achieve highly competitive accuracy, with the 8B backbone significantly outperforming Search-R1 (+0.045) and slightly edging out Search-R2 (+0.001). This demonstrates the improvements of our framework: APS provides a retrieval set with marginal coverage and approximate conditional coverage to prevent the model from encountering hallucinations due to excessive context in simple queries or missing answers in complex ones; meanwhile, ACI effectively ensures high-confidence model outputs, preventing hallucinations and redundant tool invocations. Further experimental analysis regarding the Qwen3-8B backbone is deferred to Appendix E. Table 3: Ablation study on Qwen2.5-3B-Instruct. The table presents the unablated framework, component-wise ablations, and sensitivity analyses for ρ and αAPS _APS.Full results across all individual datasets are detailed in Table 8. Methods General QA Multi-Hop QA Overall Ours (Default) 0.498 0.328 0.401 -ACI 0.481 0.311 0.384 -APS 0.490 0.313 0.389 ρ=0.1ρ=0.1 0.497 0.302 0.386 ρ=0.4ρ=0.4 0.495 0.273 0.368 αAPS=0.35 _APS=0.35 0.476 0.294 0.372 αAPS=0.05 _APS=0.05 0.504 0.234 0.350 (a) Average Search Calls (b) Ratio of Low-Confidence Samples (c) Adaptive Target Error Rate (αt _t) Figure 2: Training dynamics and sensitivity analyses on the Qwen2.5-3B-Instruct backbone. (a) Evolution of average search calls during training across different ablation configurations. (b) The ratio of low-confidence samples penalized by the ACI weight under varying target error rates (ρ). (c) The dynamic adaptation of the target error rate (αt _t), demonstrating stable convergence to the preset ρ values. 4.3 Ablation Study To evaluate the individual contributions of our proposed modules, we conducted a component-wise ablation study on the Qwen2.5-3B-Instruct backbone. For the configuration where the APS is disabled (-APS), the retrieval mechanism falls back to a fixed top-k (k=3k=3) setting. As shown in Table 3, removing either component leads to a notable degradation in both General QA and Multi-Hop QA tasks. Impact of the ACI Weight. Removing the ACI weight (-ACI) decreases the overall score from 0.401 to 0.384. This decline is intrinsically linked to the model’s search behavior. As illustrated in the tool usage trajectories (Figure 2(a)), the baseline and the -ACI variant exhibit significantly higher and more fluctuating tool calls. Without confidence constraints, blind overconfidence causes the model to hallucinate, initiating searches that deviate from the target question. Notably, some trajectories, due to a lack of confidence, conversely resort to secondary searches to verify answers. In summary, through the ACI constraint, the model avoids not only overconfidence but also blind underconfidence, thereby maintaining a stable and efficient search frequency (as denoted by the default trajectory). Impact of the Adaptive Prediction Set. Disabling APS (-APS) drops accuracy to 0.389 by restricting retrieval to a fixed-length context. This rigid setup degrades performance via two paths: in simple queries, fixed top-k retrieval introduces noise through redundant documents; in complex multi-hop queries, the static window often misses critical facts. Notably, the resulting information scarcity in complex scenarios forces the model to issue additional tool calls to compensate, even with the ACI weight active. Consequently, its tool usage frequency falls between our full framework and the baseline (Figure 2(a)). This dynamic corroborates the complementarity of the two modules: APS provides an adaptive, noise-free context in a single step, while the ACI weight suppresses unnecessary exploratory searches. 4.4 Sensitivity Analysis To verify the robustness and controllability of our framework, we conduct a sensitivity analysis on the ACI target error rate ρ∈0.1,0.25,0.4ρ∈\0.1,0.25,0.4\ and the APS significance level αAPS∈0.05,0.2,0.35 _APS∈\0.05,0.2,0.35\. All experiments in this section are performed on the Qwen2.5-3B-Instruct backbone, with calibration set configurations consistent with the Conformal Settings. The performance results are summarized in the bottom sections of Table 3. ACI Weight under Different ρ. Figures 2(b) and 2(c) illustrate the dynamic characteristics of the ACI mechanism during RL fine-tuning. The ACI weight effectively maintains the ratio of low-confidence samples within expected ranges and ensures that the dynamically adjusted αt _t closely tracks the target error rate ρ. Notably, a pronounced spike is observed in the ratio of low-confidence samples at the very first step (figure 2(b)). This phenomenon is directly attributed to the surge of trajectories as the policy πθ _θ begins to update, introducing highly non-i.i.d. data into the stream. The rapid stabilization of αt _t following this shock demonstrates ACI’s robust adaptability, highlighting the fundamental inadequacy of Static CP in dynamic RL environments. Furthermore, as shown in Table 3, a strict target (ρ=0.4ρ=0.4) classifies nearly 40% of the reasoning trajectories as low-confidence. While enforcing rigorous quality constraints, this over-penalization severely dilutes the RL reward signals, diminishing training efficiency. Conversely, a relaxed target (ρ=0.1ρ=0.1) applies the ACI weight to only 10% of the samples. With such lenient filtering, the performance degenerates toward the unconstrained baseline due to insufficient confidence guidance. Table 4: Average number of retrieved documents under different APS significance levels (αAPS _APS). αAPS _APS Avg. Retrieved Documents 0.200.20 (Default) 3.4 0.350.35 2.4 0.050.05 4.8 APS Retrieval under Different αAPS _APS. The significance level αAPS _APS dictates the aggressiveness of the dynamic context truncation. Table 4 presents the average number of retrieved documents under different αAPS _APS settings. A high-guarantee setting (αAPS=0.05 _APS=0.05) yields an average of 4.8 documents, ensuring a 95% marginal coverage. While this extensive context significantly benefits single-hop General QA by minimizing the risk of omitting critical evidence, the excessive information introduces substantial noise, which severely impairs the reasoning quality in complex Multi-Hop QA (see Table 3). In contrast, a low-guarantee setting (αAPS=0.35 _APS=0.35) returns only 2.4 documents on average. This aggressive truncation fails to provide sufficient supporting facts, resulting in suboptimal performance across both tasks. Consequently, our default configuration (αAPS=0.20 _APS=0.20) strikes the optimal balance between comprehensive information retrieval and effective noise reduction. 5 Related Works 5.1 Retrieval in LLMs Traditional RAG 24; 10 significantly expands the knowledge boundaries of LLMs by prepending retrieved external documents to the input context. With the continuous evolution of RAG, the emergence of frameworks such as Adaptive RAG 16, Search-o1 25, and SAKI-RAG 38 has highlighted the inherent challenges of determining when to trigger retrieval in static paradigms. Concurrently, approaches that integrate retrieval with Reinforcement Learning 17; 14; 51; 36, have been introduced. However, these methods universally rely on fixed Top-K truncation. This static constraint fails to guarantee the marginal coverage of the retrieved knowledge. 5.2 Reinforcement Learning for Agents Reinforcement Learning has fundamentally transformed the capabilities of LLMs, evolving them from passive generators into autonomous agents 28, such as SWE-agents 50; 44, Web Agents 9; 13, and Search Agents 17; 51; 14; 36. However, they universally face the problem of overconfidence driven by sparse, binary rewards, which subsequently leads to hallucination issues 23. Recent work proposes training models to explicitly verbalize their confidence scores alongside their answers 7. Yet, this approach struggles in Search Agents, as the discontinuous generation process caused by continuous tool invocations prevents the model from explicitly expressing its confidence. 5.3 Conformal Prediction and Uncertainty Quantification CP 41 offers highly reliable marginal coverage guarantees without distributional assumptions. Due to its theoretical rigor, CP has been widely applied in traditional classification and detection tasks 45; 2. Recent works have integrated CP into LLMs 20; 31; 37 and further into RL fine-tuning for robust alignment 5. However, while CP guarantees marginal coverage, it fails to guarantee conditional coverage, exhibiting a lack of adaptability when faced with complex queries 33. Furthermore, CP methods fundamentally rely on a strict independent and identically distributed (i.i.d.) assumption, which is inherently violated in reinforcement learning where the model and policy are continuously updating. Therefore, breaking these constraints is essential for reliable agentic search. 6 Conclusion We propose CAS, a framework that integrates Conformal Prediction to resolve the reliability crisis in RL-trained search agents. The synergy between APS and ACI ensures reliable document retrieval and mitigates model overconfidence. Empirically, CAS significantly enhances reasoning accuracy and reduces redundant tool invocations. By balancing theoretical rigor with practical performance, this work establishes a principled foundation for future reliable autonomous agents. 7 Limitations Although CAS demonstrates significant potential in improving the accuracy and efficiency of Search Agents, several limitations remain. First, our empirical validation is primarily focused on general open-domain Question Answering (QA) tasks. While CAS provides robust statistical guarantees within these general information-seeking contexts, its applicability in highly specialized professional domains remains unexplored. Second, the framework relies heavily on a strong external teacher model (e.g., DeepSeek-V3.2) to construct the calibration set for the APS by decomposing queries and acting as a relevance judge. Finally, CAS primarily focuses on outcome reliability without extending statistical guarantees to the intermediate reasoning process. Future research should explore verifiable process reliability. Acknowledgments This work was supported by the National Key Research and Development Program of China (2024YFF0907802 and 2024YFF0907803) and the National Natural Science Foundation of China (62276230). References Ahn et al. (2024) J. Ahn, R. Verma, R. Lou, D. Liu, R. Zhang, and W. Yin Large language models for mathematical reasoning: progresses and challenges. External Links: 2402.00157, Link Cited by: §4.1. andéol et al. (2025) L. andéol, L. Mossina, A. Mazoyer, and S. Gerchinovitz Conformal object detection by sequential risk control. External Links: 2505.24038, Link Cited by: §5.3. Angelopoulos and Bates (2022) A. N. Angelopoulos and S. Bates A gentle introduction to conformal prediction and distribution-free uncertainty quantification. External Links: 2107.07511, Link Cited by: §B.1, §1, §2, §4.1. Chakraborty et al. (2026) D. Chakraborty, E. Yang, D. Khashabi, D. Lawrie, and K. Duh Principled context engineering for rag: statistical guarantees via conformal prediction. In Advances in Information Retrieval, p. 537–546. External Links: ISBN 9783032213006, ISSN 1611-3349, Link, Document Cited by: §1. Chen et al. (2026) T. Chen, X. Liu, V. Nandam, K. Liou, and H. Wei Conformal feedback alignment: quantifying answer-level reliability for robust llm alignment. External Links: 2601.17329, Link Cited by: §5.3. Chung et al. (2022) H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, Y. Li, X. Wang, M. Dehghani, S. Brahma, A. Webson, S. S. Gu, Z. Dai, M. Suzgun, X. Chen, A. Chowdhery, A. Castro-Ros, M. Pellat, K. Robinson, D. Valter, S. Narang, G. Mishra, A. Yu, V. Zhao, Y. Huang, A. Dai, H. Yu, S. Petrov, E. H. Chi, J. Dean, J. Devlin, A. Roberts, D. Zhou, Q. V. Le, and J. Wei Scaling instruction-finetuned language models. External Links: 2210.11416, Link Cited by: §4.1. Damani et al. (2025) M. Damani, I. Puri, S. Slocum, I. Shenfeld, L. Choshen, Y. Kim, and J. Andreas Beyond binary rewards: training lms to reason about their uncertainty. External Links: 2507.16806, Link Cited by: §5.2. DeepSeek-AI et al. (2025) DeepSeek-AI, A. Liu, A. Mei, B. Lin, B. Xue, B. Wang, B. Xu, B. Wu, B. Zhang, C. Lin, C. Dong, C. Lu, C. Zhao, C. Deng, C. Xu, C. Ruan, D. Dai, D. Guo, D. Yang, D. Chen, E. Li, F. Zhou, F. Lin, F. Dai, G. Hao, G. Chen, G. Li, H. Zhang, H. Xu, H. Li, H. Liang, H. Wei, H. Zhang, H. Luo, H. Ji, H. Ding, H. Tang, H. Cao, H. Gao, H. Qu, H. Zeng, J. Huang, J. Li, J. Xu, J. Hu, J. Chen, J. Xiang, J. Yuan, J. Cheng, J. Zhu, J. Ran, J. Jiang, J. Qiu, J. Li, J. Song, K. Dong, K. Gao, K. Guan, K. Huang, K. Zhou, K. Huang, K. Yu, L. Wang, L. Zhang, L. Wang, L. Zhao, L. Yin, L. Guo, L. Luo, L. Ma, L. Wang, L. Zhang, M. S. Di, M. Y. Xu, M. Zhang, M. Zhang, M. Tang, M. Zhou, P. Huang, P. Cong, P. Wang, Q. Wang, Q. Zhu, Q. Li, Q. Chen, Q. Du, R. Xu, R. Ge, R. Zhang, R. Pan, R. Wang, R. Yin, R. Xu, R. Shen, R. Zhang, S. H. Liu, S. Lu, S. Zhou, S. Chen, S. Cai, S. Chen, S. Hu, S. Liu, S. Hu, S. Ma, S. Wang, S. Yu, S. Zhou, S. Pan, S. Zhou, T. Ni, T. Yun, T. Pei, T. Ye, T. Yue, W. Zeng, W. Liu, W. Liang, W. Pang, W. Luo, W. Gao, W. Zhang, X. Gao, X. Wang, X. Bi, X. Liu, X. Wang, X. Chen, X. Zhang, X. Nie, X. Cheng, X. Liu, X. Xie, X. Liu, X. Yu, X. Li, X. Yang, X. Li, X. Chen, X. Su, X. Pan, X. Lin, X. Fu, Y. Q. Wang, Y. Zhang, Y. Xu, Y. Ma, Y. Li, Y. Li, Y. Zhao, Y. Sun, Y. Wang, Y. Qian, Y. Yu, Y. Zhang, Y. Ding, Y. Shi, Y. Xiong, Y. He, Y. Zhou, Y. Zhong, Y. Piao, Y. Wang, Y. Chen, Y. Tan, Y. Wei, Y. Ma, Y. Liu, Y. Yang, Y. Guo, Y. Wu, Y. Wu, Y. Cheng, Y. Ou, Y. Xu, Y. Wang, Y. Gong, Y. Wu, Y. Zou, Y. Li, Y. Xiong, Y. Luo, Y. You, Y. Liu, Y. Zhou, Z. F. Wu, Z. Z. Ren, Z. Zhao, Z. Ren, Z. Sha, Z. Fu, Z. Xu, Z. Xie, Z. Zhang, Z. Hao, Z. Gou, Z. Ma, Z. Yan, Z. Shao, Z. Huang, Z. Wu, Z. Li, Z. Zhang, Z. Xu, Z. Wang, Z. Gu, Z. Zhu, Z. Li, Z. Zhang, Z. Xie, Z. Gao, Z. Pan, Z. Yao, B. Feng, H. Li, J. L. Cai, J. Ni, L. Xu, M. Li, N. Tian, R. J. Chen, R. L. Jin, S. S. Li, S. Zhou, T. Sun, X. Q. Li, X. Jin, X. Shen, X. Chen, X. Song, X. Zhou, Y. X. Zhu, Y. Huang, Y. Li, Y. Zheng, Y. Zhu, Y. Ma, Z. Huang, Z. Xu, Z. Zhang, D. Ji, J. Liang, J. Guo, J. Chen, L. Xia, M. Wang, M. Li, P. Zhang, R. Chen, S. Sun, S. Wu, S. Ye, T. Wang, W. L. Xiao, W. An, X. Wang, X. Sun, X. Wang, Y. Tang, Y. Zha, Z. Zhang, Z. Ju, Z. Zhang, and Z. Qu DeepSeek-v3.2: pushing the frontier of open large language models. External Links: 2512.02556, Link Cited by: §4.1. Ding et al. (2026) H. Ding, P. Liu, J. Wang, Z. Ji, M. Cao, R. Zhang, L. Ai, E. Yang, T. Shi, and L. Yu DynaWeb: model-based reinforcement learning of web agents. External Links: 2601.22149, Link Cited by: §5.2. Gao et al. (2024) Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, and H. Wang Retrieval-augmented generation for large language models: a survey. External Links: 2312.10997, Link Cited by: §5.1. Gibbs and Candès (2021) I. Gibbs and E. Candès Adaptive conformal inference under distribution shift. External Links: 2106.00170, Link Cited by: §B.3, §B.3, §B.3, §B.3, §1, §2.2, §3.4. Guo et al. (2025) D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, Q. Zhu, R. Xu, R. Zhang, S. Ma, X. Bi, X. Zhang, X. Yu, Y. Wu, Z. F. Wu, Z. Gou, Z. Shao, Z. Li, Z. Gao, A. Liu, B. Xue, B. Wang, B. Wu, B. Feng, C. Lu, C. Zhao, C. Deng, C. Ruan, D. Dai, D. Chen, D. Ji, E. Li, F. Lin, F. Dai, F. Luo, G. Hao, G. Chen, G. Li, H. Zhang, H. Xu, H. Ding, H. Gao, H. Qu, H. Li, J. Guo, J. Li, J. Chen, J. Yuan, J. Tu, J. Qiu, J. Li, J. L. Cai, J. Ni, J. Liang, J. Chen, K. Dong, K. Hu, K. You, K. Gao, K. Guan, K. Huang, K. Yu, L. Wang, L. Zhang, L. Zhao, L. Wang, L. Zhang, L. Xu, L. Xia, M. Zhang, M. Zhang, M. Tang, M. Zhou, M. Li, M. Wang, M. Li, N. Tian, P. Huang, P. Zhang, Q. Wang, Q. Chen, Q. Du, R. Ge, R. Zhang, R. Pan, R. Wang, R. J. Chen, R. L. Jin, R. Chen, S. Lu, S. Zhou, S. Chen, S. Ye, S. Wang, S. Yu, S. Zhou, S. Pan, S. S. Li, S. Zhou, S. Wu, T. Yun, T. Pei, T. Sun, T. Wang, W. Zeng, W. Liu, W. Liang, W. Gao, W. Yu, W. Zhang, W. L. Xiao, W. An, X. Liu, X. Wang, X. Chen, X. Nie, X. Cheng, X. Liu, X. Xie, X. Liu, X. Yang, X. Li, X. Su, X. Lin, X. Q. Li, X. Jin, X. Shen, X. Chen, X. Sun, X. Wang, X. Song, X. Zhou, X. Wang, X. Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Y. Zhang, Y. Xu, Y. Li, Y. Zhao, Y. Sun, Y. Wang, Y. Yu, Y. Zhang, Y. Shi, Y. Xiong, Y. He, Y. Piao, Y. Wang, Y. Tan, Y. Ma, Y. Liu, Y. Guo, Y. Ou, Y. Wang, Y. Gong, Y. Zou, Y. He, Y. Xiong, Y. Luo, Y. You, Y. Liu, Y. Zhou, Y. X. Zhu, Y. Huang, Y. Li, Y. Zheng, Y. Zhu, Y. Ma, Y. Tang, Y. Zha, Y. Yan, Z. Z. Ren, Z. Ren, Z. Sha, Z. Fu, Z. Xu, Z. Xie, Z. Zhang, Z. Hao, Z. Ma, Z. Yan, Z. Wu, Z. Gu, Z. Zhu, Z. Liu, Z. Li, Z. Xie, Z. Song, Z. Pan, Z. Huang, Z. Xu, Z. Zhang, and Z. Zhang DeepSeek-r1 incentivizes reasoning in llms through reinforcement learning. Nature 645 (8081), p. 633–638. External Links: ISSN 1476-4687, Link, Document Cited by: §4.1. Guo et al. (2026) Y. Guo, W. Yang, S. Yang, Z. Liu, C. Chen, Y. Wei, Y. Hu, Y. Huang, G. Hao, D. Yuan, J. Wang, X. Chen, H. Yu, L. Lei, and P. Di OpAgent: operator agent for web navigation. External Links: 2602.13559, Link Cited by: §5.2. He et al. (2026) B. He, M. Hu, Z. Xu, H. Wang, L. Zong, Y. Chen, C. Ma, X. Liu, P. Zhou, and I. King Search-r2: enhancing search-integrated reasoning via actor-refiner collaboration. External Links: 2602.03647, Link Cited by: §1, §4.1, §5.1, §5.2. Ho et al. (2020) X. Ho, A. D. Nguyen, S. Sugawara, and A. Aizawa Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. External Links: 2011.01060, Link Cited by: §4.1. Jeong et al. (2024) S. Jeong, J. Baek, S. Cho, S. J. Hwang, and J. C. Park Adaptive-rag: learning to adapt retrieval-augmented large language models through question complexity. External Links: 2403.14403, Link Cited by: §5.1. Jin et al. (2025) B. Jin, H. Zeng, Z. Yue, J. Yoon, S. Arik, D. Wang, H. Zamani, and J. Han Search-r1: training llms to reason and leverage search engines with reinforcement learning. External Links: 2503.09516, Link Cited by: Appendix D, §1, §4.1, §5.1, §5.2. Joshi et al. (2017) M. Joshi, E. Choi, D. S. Weld, and L. Zettlemoyer TriviaQA: a large scale distantly supervised challenge dataset for reading comprehension. External Links: 1705.03551, Link Cited by: §4.1. Karpukhin et al. (2020) V. Karpukhin, B. Oğuz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih Dense passage retrieval for open-domain question answering. External Links: 2004.04906, Link Cited by: §4.1. Kumar et al. (2023) B. Kumar, C. Lu, G. Gupta, A. Palepu, D. Bellamy, R. Raskar, and A. Beam Conformal prediction with large language models for multi-choice question answering. External Links: 2305.18404, Link Cited by: §1, §5.3. Kwiatkowski et al. (2019) T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee, K. Toutanova, L. Jones, M. Kelcey, M. Chang, A. M. Dai, J. Uszkoreit, Q. Le, and S. Petrov Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics 7, p. 452–466. External Links: Link, Document Cited by: §4.1. Lei et al. (2017) J. Lei, M. G’Sell, A. Rinaldo, R. J. Tibshirani, and L. Wasserman Distribution-free predictive inference for regression. External Links: 1604.04173, Link Cited by: §B.1. Leng et al. (2025) J. Leng, C. Huang, B. Zhu, and J. Huang Taming overconfidence in llms: reward calibration in rlhf. External Links: 2410.09724, Link Cited by: §1, §5.2. Lewis et al. (2021) P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela Retrieval-augmented generation for knowledge-intensive nlp tasks. External Links: 2005.11401, Link Cited by: §1, §4.1, §5.1. Li et al. (2025) X. Li, G. Dong, J. Jin, Y. Zhang, Y. Zhou, Y. Zhu, P. Zhang, and Z. Dou Search-o1: agentic search-enhanced large reasoning models. External Links: 2501.05366, Link Cited by: §4.1, §5.1. Liang et al. (2026) T. Liang, Y. Chen, S. Ji, Y. Chen, Z. Jia, J. Lu, F. Zhuo, X. Pu, Y. Li, and Z. Zhao VoxMind: an end-to-end agentic spoken dialogue system. External Links: 2604.15710, Link Cited by: §1. Liu et al. (2023) N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang Lost in the middle: how language models use long contexts. External Links: 2307.03172, Link Cited by: §1. Luo et al. (2025) J. Luo, W. Zhang, Y. Yuan, Y. Zhao, J. Yang, Y. Gu, B. Wu, B. Chen, Z. Qiao, Q. Long, R. Tu, X. Luo, W. Ju, Z. Xiao, Y. Wang, M. Xiao, C. Liu, J. Yuan, S. Zhang, Y. Jin, F. Zhang, X. Wu, H. Zhao, D. Tao, P. S. Yu, and M. Zhang Large language model agent: a survey on methodology, applications and challenges. External Links: 2503.21460, Link Cited by: §5.2. Mallen et al. (2023) A. Mallen, A. Asai, V. Zhong, R. Das, D. Khashabi, and H. Hajishirzi When not to trust language models: investigating effectiveness of parametric and non-parametric memories. External Links: 2212.10511, Link Cited by: §4.1. Press et al. (2023) O. Press, M. Zhang, S. Min, L. Schmidt, N. A. Smith, and M. Lewis Measuring and narrowing the compositionality gap in language models. External Links: 2210.03350, Link Cited by: §4.1. Quach et al. (2024) V. Quach, A. Fisch, T. Schuster, A. Yala, J. H. Sohn, T. S. Jaakkola, and R. Barzilay Conformal language modeling. External Links: 2306.10193, Link Cited by: §3.4, §5.3. Qwen et al. (2025) Qwen, :, A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, X. Ren, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Wan, Y. Liu, Z. Cui, Z. Zhang, and Z. Qiu Qwen2.5 technical report. External Links: 2412.15115, Link Cited by: §4.1. Romano et al. (2020) Y. Romano, M. Sesia, and E. J. Candès Classification with valid and adaptive coverage. External Links: 2006.02544, Link Cited by: §B.2, §B.2, §1, §2.1, §5.3. Schick et al. (2023) T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom Toolformer: language models can teach themselves to use tools. External Links: 2302.04761, Link Cited by: §1. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo DeepSeekMath: pushing the limits of mathematical reasoning in open language models. External Links: 2402.03300, Link Cited by: §1, §3.4. Singh et al. (2026) A. Singh, A. Ehtesham, S. Kumar, T. T. Khoei, and A. V. Vasilakos Agentic retrieval-augmented generation: a survey on agentic rag. External Links: 2501.09136, Link Cited by: §5.1, §5.2. Su et al. (2025) J. Su, F. Lin, Z. Feng, H. Zheng, T. Wang, Z. Xiao, X. Zhao, Z. Liu, L. Cheng, and H. Wang CP-router: an uncertainty-aware router between llm and lrm. External Links: 2505.19970, Link Cited by: §1, §5.3. Tao et al. (2025) W. Tao, X. Xing, Z. Li, and X. Xu SAKI-RAG: mitigating context fragmentation in long-document RAG via sentence-level attention knowledge integration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), Suzhou, China, p. 1195–1213. External Links: Link, Document, ISBN 979-8-89176-332-6 Cited by: §5.1. Trivedi et al. (2022) H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal MuSiQue: multihop questions via single-hop question composition. External Links: 2108.00573, Link Cited by: §4.1. Trivedi et al. (2023) H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. External Links: 2212.10509, Link Cited by: §4.1. Vovk et al. (2005) V. Vovk, A. Gammerman, and G. Shafer Algorithmic learning in a random world. Springer-Verlag, Berlin, Heidelberg. External Links: ISBN 0387001522 Cited by: §5.3. Wang et al. (2024) L. Wang, N. Yang, X. Huang, B. Jiao, L. Yang, D. Jiang, R. Majumder, and F. Wei Text embeddings by weakly-supervised contrastive pre-training. External Links: 2212.03533, Link Cited by: §4.1. Wei et al. (2023) J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou Chain-of-thought prompting elicits reasoning in large language models. External Links: 2201.11903, Link Cited by: §4.1. Wei et al. (2025) Y. Wei, O. Duchenne, J. Copet, Q. Carbonneaux, L. Zhang, D. Fried, G. Synnaeve, R. Singh, and S. I. Wang SWE-rl: advancing llm reasoning via reinforcement learning on open software evolution. External Links: 2502.18449, Link Cited by: §5.2. Wu et al. (2026) Z. Wu, S. W. Jeong, Y. Liu, Y. J. Jung, and C. Donnat Filtering with confidence: when data augmentation meets conformal prediction. External Links: 2509.21479, Link Cited by: §5.3. Yang et al. (2025) A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu Qwen3 technical report. External Links: 2505.09388, Link Cited by: §4.1. Yang et al. (2018) Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning HotpotQA: a dataset for diverse, explainable multi-hop question answering. External Links: 1809.09600, Link Cited by: §4.1. Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. External Links: 2210.03629, Link Cited by: §1. Yin et al. (2026) C. Yin, Z. Sha, S. Cui, C. Meng, and Z. Li The reasoning trap: how enhancing llm reasoning amplifies tool hallucination. External Links: 2510.22977, Link Cited by: §1. Zhang et al. (2026) Z. Zhang, Y. Duan, Y. Zhang, Y. Xu, Z. Wang, K. Liang, Y. Li, J. Liang, D. Xia, J. Huang, J. He, and Y. Wu One tool is enough: reinforcement learning for repository-level llm agents. External Links: 2512.20957, Link Cited by: §5.2. Zhao et al. (2025) S. Zhao, T. Yu, A. Xu, J. Singh, A. Shukla, and R. Akkiraju ParallelSearch: train your llms to decompose query and search sub-queries in parallel with reinforcement learning. External Links: 2508.09303, Link Cited by: §1, §5.1, §5.2. Appendix A Pseudocode We provide the pseudocode of CAS in Algorithm 1 Input: Dataset D, Initial policy πθ _θ, Search engine S Parameters: Group size G, ACI step size γ, Target error rate ρ, Initial calibration set size NcalN_cal Output: Optimized policy πθ _θ 1 Initialize ACI threshold α1←ρ _1←ρ; 2 Initialize calibration set C with NcalN_cal scores from untrained policy; 3 4 for each RL training iteration t=1,2,…t=1,2,… do 5 Sample a prompt xt∼x_t ; 6 7 for trajectory i=1i=1 to G do 8 while trajectory yiy_i not terminated do 9 Generate tokens ynext∼πθ(⋅∣xt,yi)y_next _θ(· x_t,y_i); 10 if search query q generated then 11 DAPS←APS((q))D_APS (S(q)) (Sec. 3.2); 12 yi←yi⊕DAPSy_i← y_i D_APS; 13 end if 14 end while 15 end for 16 17 for trajectory i=1i=1 to G do 18 Compute reward Ri=racc+rfmtR_i=r_acc+r_fmt; 19 Compute NLL score s(xt,yi)s(x_t,y_i) for answer tokens (Eq. (6)); 20 end for 21 Compute advantages A1,…,AGA_1,…,A_G from rewards Ri\R_i\; 22 23 Compute quantile threshold q^t←Q^t(1−αt) q_t← Q_t(1- _t) over C; 24 25 for trajectory i=1i=1 to G do 26 Determine ACI confidence weight ωi _i using q^t q_t (Eq. (9)); 27 end for 28 ←∪s(xt,y1),…,s(xt,yG)C ∪\s(x_t,y_1),…,s(x_t,y_G)\ ; 29 Update πθ _θ by minimizing the ACI-guided objective (Eq. (8)); 30 Update ACI threshold αt+1 _t+1 (Eq. (7)); 31 end for 32 Algorithm 1 CAS Appendix B Mathematical Proofs In this section, we provide the formal mathematical proofs for the statistical guarantees of the CP methods used in CAS. We begin with the fundamental marginal coverage guarantee of standard Split CP. B.1 Marginal Coverage Guarantee of Conformal Prediction Following the standard proof of validity for split-conformal prediction (3), we demonstrate that the prediction sets constructed via CP possess a strict, finite-sample marginal coverage guarantee. Theorem 1 (Conformal calibration coverage guarantee). Suppose the calibration data (Xi,Yi)i=1,…,n(X_i,Y_i)_i=1,…,n and the test point (Xtest,Ytest)(X_test,Y_test) are independent and identically distributed (i.i.d.). Define the conformal quantile q q as: q^=infq:|i:s(Xi,Yi)≤q|n≥⌈(n+1)(1−α)⌉n q= \q: |\i:s(X_i,Y_i)≤ q\|n≥ (n+1)(1-α) n \ (10) and the resulting prediction sets as: (X)=y:s(X,y)≤q^C(X)=\y:s(X,y)≤ q\ (11) Then, the marginal coverage satisfies: ℙ(Ytest∈(Xtest))≥1−αP(Y_test (X_test))≥ 1-α (12) Proof of Theorem 1. Let si=s(Xi,Yi)s_i=s(X_i,Y_i) for i=1,…,ni=1,…,n and stest=s(Xtest,Ytest)s_test=s(X_test,Y_test). To avoid handling ties, we consider the case where the non-conformity scores sis_i are distinct with probability 1. Without loss of generality, we assume the calibration scores are sorted such that s1<s2<⋯<sns_1<s_2<…<s_n. In this case, the quantile q q can be explicitly written as: q^=s⌈(n+1)(1−α)⌉ q=s_ (n+1)(1-α) (13) when α≥1n+1α≥ 1n+1, and q^=∞ q=∞ otherwise. Note that in the case where q^=∞ q=∞, the prediction set includes the entire label space, i.e., (Xtest)=C(X_test)=Y, so the coverage property is trivially satisfied. Thus, we only need to handle the case when α≥1n+1α≥ 1n+1. We proceed by noticing the strict equality of the two following events: Ytest∈(Xtest)=stest≤q^\Y_test (X_test)\=\s_test≤ q\ (14) Combining this with our definition of the sorted quantile q q yields: Ytest∈(Xtest)=stest≤s⌈(n+1)(1−α)⌉\Y_test (X_test)\=\s_test≤ s_ (n+1)(1-α) \ (15) Now comes the crucial insight: by the exchangeability of the random variables (X1,Y1),…,(Xtest,Ytest)(X_1,Y_1),…,(X_test,Y_test), their corresponding non-conformity scores s1,…,sn,stests_1,…,s_n,s_test are also exchangeable. Because they are exchangeable, stests_test is equally likely to fall anywhere between the sorted calibration points s1,…,sns_1,…,s_n. Therefore, the probability that stests_test is less than or equal to the k-th sorted score is exactly: ℙ(stest≤sk)=kn+1P(s_test≤ s_k)= kn+1 (16) for any integer k. (Note that here, the randomness is over all variables s1,…,sn,stests_1,…,s_n,s_test). From this property, we substitute k=⌈(n+1)(1−α)⌉k= (n+1)(1-α) to conclude: ℙ(stest≤s⌈(n+1)(1−α)⌉)=⌈(n+1)(1−α)⌉n+1≥1−αP (s_test≤ s_ (n+1)(1-α) )= (n+1)(1-α) n+1≥ 1-α (17) which implies the desired result: ℙ(Ytest∈(Xtest))≥1−αP(Y_test (X_test))≥ 1-α. Theorem 2 (Conformal calibration upper bound). Additionally, if the scores s1,…,sns_1,…,s_n have a continuous joint distribution (i.e., avoiding ties), the coverage is tightly bounded from above: ℙ(Ytest∈(Xtest))≤1−α+1n+1P(Y_test (X_test))≤ 1-α+ 1n+1 (18) (Proof deferred to Theorem 2.2 of 22). B.2 Statistical Guarantees of Adaptive Prediction Sets As established in the foundational literature, achieving exact finite-sample conditional coverage is theoretically impossible without strong distributional assumptions. However, the APS framework 33 effectively circumvents this limitation. It provides a rigorous marginal coverage guarantee while sensibly approximating conditional coverage by adapting the prediction set size to the local uncertainty of the input. To construct the adaptive sets, APS introduces a generalized inverse quantile conformity score. Given a base model’s probability estimate π π and a uniform random variable U∼Uniform(0,1)U (0,1) for tie-breaking, the conformity score function E is defined as: E(x,y,u,π^)=minτ∈[0,1]:y∈(x,u,π^,τ)E(x,y,u; π)= \τ∈[0,1]:y (x,u; π,τ)\ (19) where S is the generalized conditional quantile function that includes classes in descending order of their estimated probabilities until the cumulative mass reaches τ. Using this conformity score, APS achieves the following rigorous marginal guarantee: Theorem 3 (Marginal coverage of APS). If the calibration samples (Xi,Yi)i∈ℐ2(X_i,Y_i)_i _2 and the test sample (Xtest,Ytest)(X_test,Y_test) are exchangeable, and the conformity scores are calculated using a model trained on a disjoint split ℐ1I_1, the APS prediction set ^APS C_APS satisfies: ℙ(Ytest∈^APS(Xtest))≥1−αP (Y_test∈ C_APS(X_test) )≥ 1-α (20) Furthermore, if the scores EiE_i are almost surely distinct, the coverage is bounded tightly from above by 1−α+1/(|ℐ2|+1)1-α+1/(|I_2|+1). Proof of Theorem 3. Let Ei=E(Xi,Yi,Ui,π^)E_i=E(X_i,Y_i,U_i; π) denote the conformity score for the i-th calibration sample in ℐ2I_2, and Etest=E(Xtest,Ytest,Utest,π^)E_test=E(X_test,Y_test,U_test; π) for the test point. By the construction of the APS prediction set, a label y is included in ^APS(Xtest) C_APS(X_test) if and only if its requisite cumulative mass τ is less than or equal to the calibrated threshold Q^1−α Q_1-α. Mathematically, we know that: Ytest∈^APS(Xtest)⇔Etest≤Q^1−α(Eii∈ℐ2)Y_test∈ C_APS(X_test) E_test≤ Q_1-α(\E_i\_i _2) (21) where Q^1−α(Eii∈ℐ2) Q_1-α(\E_i\_i _2) is defined as the ⌈(1−α)(1+|ℐ2|)⌉ (1-α)(1+|I_2|) -th smallest value in the calibration score set Eii∈ℐ2\E_i\_i _2. Because the data points (X,Y)(X,Y) are exchangeable and the uniform variables U are i.i.d., all the evaluated conformity scores EtestE_test and Eii∈ℐ2\E_i\_i _2 are completely exchangeable. Under the property of exchangeability, the rank of EtestE_test is uniformly distributed among the |ℐ2|+1|I_2|+1 scores. Therefore, the probability of the event that EtestE_test falls below the empirical (1−α)(1-α)-quantile is bounded from below by the nominal level: ℙ(Etest≤Q^1−α(Eii∈ℐ2))≥1−αP (E_test≤ Q_1-α(\E_i\_i _2) )≥ 1-α (22) which immediately establishes ℙ(Ytest∈^APS(Xtest))≥1−αP(Y_test∈ C_APS(X_test))≥ 1-α. Asymptotic Conditional Coverage. While Theorem 3 guarantees marginal coverage, the structural design of APS provides an asymptotic approximation of conditional coverage. Consider an Oracle model with perfect knowledge of the true conditional distribution πy(x)=ℙ(Y=y|X=x) _y(x)=P(Y=y|X=x). The Oracle’s prediction set αoracle(x)C_α^oracle(x) naturally attains exact conditional coverage. According to 33, as the sample size increases and if the base predictive model is consistent (i.e., π^y(x)≈πy(x) π_y(x)≈ _y(x)), the constructed sets (X,U,π^,τ)S(X,U; π,τ) will converge to contain the true labels for exactly a fraction τ of the points. In this limit, the threshold Q^1−α≈1−α Q_1-α≈ 1-α, and the decision rule approaches: ^APS(Xtest)≈y∈:E(Xtest,y,Utest,π)≤1−α C_APS(X_test)≈\y :E(X_test,y,U_test;π)≤ 1-α\ (23) which mathematically equates to the exact output of the Oracle procedure, thereby closely approximating optimal conditional coverage in complex data scenarios. B.3 Statistical Guarantees of Adaptive Conformal Inference Standard CP fundamentally relies on the exchangeability of the data. In online settings, the policy continuously evolves, leading to severe distribution shifts that violate the i.i.d. assumption. To maintain rigorous coverage, we employ ACI 11. ACI guarantees the target coverage frequency over long-time intervals irrespective of the true data-generating process by dynamically adjusting the nominal error level. Following 11, let ρ∈(0,1)ρ∈(0,1) be the target miscoverage rate. At each time step t, the algorithm uses a parameter αt _t to construct the prediction set ^t(αt) C_t( _t), and records the miscoverage event: errt:=1,if Yt∉^t(αt)0,otherwiseerr_t:= cases1,&if Y_t∉ C_t( _t)\\ 0,&otherwise cases (24) The parameter αt _t is recursively updated using a step size γ>0γ>0: αt+1:=αt+γ(ρ−errt) _t+1:= _t+γ(ρ-err_t) (25) To establish the distribution-free guarantee, we assume that with probability one, α1∈[0,1] _1∈[0,1] and the quantile function Q^t(x) Q_t(x) is non-decreasing with Q^t(x)=−∞ Q_t(x)=-∞ for x<0x<0 and Q^t(x)=∞ Q_t(x)=∞ for x>1x>1. Lemma 4 (Boundedness of αt _t, Lemma 4.1 in 11). With probability one, we have that for all t∈ℕt , αt∈[−γ,1+γ] _t∈[-γ,1+γ]. Proof of Lemma 4. Assume by contradiction that with positive probability, the sequence αtt∈ℕ\ _t\_t is such that inftαt<−γ _t _t<-γ (the case for suptαt>1+γ _t _t>1+γ is symmetric). Notice that the maximum change in one step is bounded: supt|αt+1−αt|=suptγ|ρ−errt|<γ _t| _t+1- _t|= _tγ|ρ-err_t|<γ. Thus, with positive probability, we may find a specific time step t∈ℕt such that αt<0 _t<0 and αt+1<αt _t+1< _t. However, by the boundary definition of the quantile function: αt<0⟹Q^t(1−αt)=∞⟹errt=0 _t<0 Q_t(1- _t)=∞ _t=0 (26) Substituting errt=0err_t=0 into the update rule gives: αt+1=αt+γ(α−0)≥αt _t+1= _t+γ(α-0)≥ _t (27) This contradicts the assumption that αt+1<αt _t+1< _t. Thus, ℙ(∃t such that αt+1<αt<0)=0P(∃ t such that _t+1< _t<0)=0, establishing the lower bound. Theorem 5 (Distribution-free asymptotic coverage, Proposition 4.1 in 11). With probability one, for all horizon lengths T∈ℕT , the empirical miscoverage rate satisfies: |1T∑t=1Terrt−ρ|≤maxα1,1−α1+γTγ | 1T _t=1^Terr_t-ρ |≤ \ _1,1- _1\+γTγ (28) In particular, as T→∞T→∞, the average miscoverage converges almost surely to the target rate α: limT→∞1T∑t=1Terrt=a.s.ρ _T→∞ 1T _t=1^Terr_t a.s.=ρ (29) Proof of Theorem 5. By recursively expanding the update rule αt+1=αt+γ(α−errt) _t+1= _t+γ(α-err_t) from t=1t=1 to T, we obtain the telescoping sum: αT+1=α1+∑t=1Tγ(ρ−errt) _T+1= _1+ _t=1^Tγ(ρ-err_t) (30) Rearranging the terms to isolate the empirical average of errterr_t, we get: 1T∑t=1T(errt−ρ)=α1−αT+1Tγ 1T _t=1^T(err_t-ρ)= _1- _T+1Tγ (31) Taking the absolute value on both sides yields: |1T∑t=1Terrt−ρ|=|α1−αT+1|Tγ | 1T _t=1^Terr_t-ρ |= | _1- _T+1|Tγ (32) From Lemma 4, we know that αT+1∈[−γ,1+γ] _T+1∈[-γ,1+γ]. Given that the initialization α1∈[0,1] _1∈[0,1], the maximum possible distance between α1 _1 and αT+1 _T+1 is bounded by: |α1−αT+1|≤maxα1−(−γ),1+γ−α1| _1- _T+1|≤ \ _1-(-γ),1+γ- _1\ (33) which simplifies to maxα1,1−α1+γ \ _1,1- _1\+γ. Substituting this upper bound into the absolute difference completes the proof for Equation 28. Taking the limit as T→∞T→∞, the right-hand side of Equation 28 diminishes to zero (since γ is a fixed positive constant), proving that ACI flawlessly achieves the exact marginal coverage frequency over time, without making any assumptions on the nature of the data distribution shift. Appendix C Supplementary Implementation Details Environment Our framework operates on a dual-service architecture developed based on the VeRL distributed reinforcement learning framework. The Training Service executes GRPO using Python 3.12, PyTorch 2.8.0 (CUDA 12.9), and is distributed across 4 GPUs via Ray (v2.49.2). To accelerate asynchronous multi-turn rollouts, it leverages sglang (v0.5.3rc0) equipped with the flashinfer backend and flash-attn (v2.8.3). The Retrieval Service operates independently as a FastAPI-based REST endpoint using Python 3.10 and PyTorch 2.4.0 (CUDA 12.1). It utilizes faiss-gpu (v1.8.0) and the e5-base-v2 embedding model, performing high-throughput dense retrieval via mean pooling on 256-token inputs with FP16 precision. The retrieval backend is configured to handle a peak rate of 120 queries per second (QPS) with a 30-second timeout. Configurations This encompasses our data processing, optimization, and CP settings. Data & Rollout: Models are trained on a unified search-integrated reasoning dataset in Parquet format. We set the maximum prompt, response, and context lengths to 4096, 3000, and 15,000 tokens, respectively, filtering out prompts that exceed the limit. During the GRPO step, we sample G=5G=5 trajectories per prompt with a maximum of 4 assistant turns. Optimization: The Actor is optimized with a learning rate of 1×10−61× 10^-6 and a warmup ratio of 0.285 (100 steps), while the Critic uses 1×10−51× 10^-5. Training employs a global batch size of 512, a low-variance KL penalty coefficient of 0.001, and Fully Sharded Data Parallel (FSDP) with tensor model parallelism set to 1. Reward Design: The rule-based reward comprises an EM accuracy score (weight 1.0) and format rewards (0.2 for structural integrity, 0.1 for the final answer boundary). CAS: On the retrieval side, APS are applied with a significance level αAPS=0.20 _APS=0.20 and a temperature of 0.01, dynamically restricting the retrieved subset to between 1 and 5 documents. On the training side, ACI is initialized with a target error rate ρ=0.25ρ=0.25 and an update step size γ=0.005γ=0.005. We apply a discount factor η=0.5η=0.5 to penalize low-confidence trajectories (si>q^ts_i> q_t), while empirical error tracking utilizes an Exponential Moving Average (EMA) ratio of 0.05 to maintain quantile stability. Hardware All experiments were conducted on a single server node. The server is configured with dual-socket AMD EPYC 9454 48-Core processors, providing a total of 96 physical cores and 192 threads, organized into two NUMA nodes. The server is equipped with four NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs and 755 GiB of system memory. Storage infrastructure includes a 446.6 GB drive for the OS and environment, alongside a 14.6 TB enterprise-grade drive for high-throughput data caching. The software environment is built on Ubuntu 24.04.3 LTS. Appendix D Prompts In this section, we present the detailed prompt templates utilized across different stages of CAS. The configuration of the reasoning template in Table 5 is adapted from Search-R1 17 to maintain consistency in agentic behavior. Additionally, the query decomposition prompt in Table 6 and the retrieval relevance judge prompt in Table 7 are specifically employed to construct the calibration set for the retrieval-side APS. Appendix E Additional Experimental Analysis on Qwen3-8B In Table 1, it is observed that our method’s performance on Qwen3-8B consistently outperforms that on Qwen2.5-3B-Instruct. This improvement is primarily attributed to the inherent model capacity of Qwen3-8B, which exhibits a significant advantage over the 3B-Instruct variant, as illustrated in Figure 3(a). Regarding the search behavior shown in Figure 3(b), we note that the average search calls for Qwen3-8B remain lower than those of Qwen2.5-3B-Instruct during approximately the first 80 training steps. This phenomenon occurs because Qwen2.5-3B-Instruct, as a smaller model, tends to exhibit erratic and indiscriminate tool invocation during the early stages of training. In contrast, the larger parameter scale of Qwen3-8B ensures more efficient search calls from the beginning. This efficiency is further evidenced by comparing Figure 3(a) and Figure 3(b), where Qwen3-8B achieves substantially higher EM scores despite a noticeably lower frequency of search invocations. Furthermore, as depicted in Figure 3(a), although the number of search calls for Qwen3-8B increases slightly relative to Qwen2.5-3B-Instruct after convergence, it remains significantly more efficient than baseline methods lacking ACI constraints. This demonstrates that the ACI mechanism effectively modulates low-confidence trajectories even when applied to the 8B model. For the experiments involving Qwen3-8B, the thinking mode is disabled by default, as enabling this feature leads to a drastic reduction in training effectiveness, as shown in Figure 3(c). The underlying cause is revealed in Figure 3(d): after enabling the thinking mode, the model initially tends towards aggressive search calls due to the interleaving of internal reasoning with our prescribed reasoning grammar. However, the model rapidly discovers that many single-hop problems can be resolved solely through internal reasoning. Consequently, it gradually ceases to invoke the search tool, leading to a complete cessation of active information gathering and rendering the training process ineffective for the intended search-integrated tasks. Prompt 1: Template for CAS You are Qwen, created by Alibaba Cloud. You are a helpful assistant. Answer the given question. You must conduct reasoning inside <think> and </think> first every time you get new information. After reasoning, if you find you lack some knowledge, you can call a search engine by <search>query</search> and it will return the top searched results between <information> and </information>. You can search as many times as you want. If you find no further external knowledge needed, you can directly provide the answer inside <answer> and </answer>, without detailed illustrations. For example, <answer> Beijing </answer>. Question: [Question] Table 5: Template for CAS reasoning process. Prompt 2: Query Decomposition You decompose QA tasks into hop-level search query-answer pairs. Return strict JSON. Given a QA sample, split it into single-hop query-answer pairs. Rules: 1) If single-hop, return one pair. 2) If multi-hop, return one pair per hop. 3) answer should be concise and factual. 4) Output JSON only in schema: "pairs":["query":"...","answer":"..."] question: question ground_truth_answers: gt_answers metadata: metadata extra_info: extra_info Table 6: Prompt for decomposing multi-hop queries into single-hop sub-queries. Prompt 3: Retrieval Relevance Judge You are a precise retrieval relevance judge for QA reasoning. You are given a retrieval query, its target answer, and retrieved documents. Decide which document(s) can support reasoning to the target answer. Return strict JSON with schema: "golden_doc_indices":[0,1], "best_golden_doc_index":0, "reason":"..." Rules: 1) If none can support the answer, return empty golden_doc_indices and -1 as best index. 2) best_golden_doc_index must be one item in golden_doc_indices, or -1. 3) Never output markdown. query: query answer: answer retrieved_docs: docs_for_llm Table 7: Judge prompt for locating the most relevant documents. (a) EM score during training. (b) Average search calls. (c) EM (Thinking mode). (d) Calls (Thinking mode). Figure 3: Visualization of training metrics and the impact of the thinking mode. Panels (a) and (b) illustrate the progression of accuracy and tool invocation frequency during training. Panels (c) and (d) demonstrate how the thinking mode influences the final EM scores and the efficiency of the search process. Appendix F Detailed Results for Ablation and Sensitivity Analysis Corresponding to Table 3 in the main text, we provide the complete results across all individual datasets in Table 8. Table 8: Detailed ablation study and sensitivity analysis on Qwen2.5-3B-Instruct. The table presents component-wise ablations and sensitivity analyses for ρ and αAPS _APS. Methods General QA Multi-Hop QA Average NQ† TriviaQA⋆ PopQA⋆ HotpotQA† 2WikiMultiHopQA⋆ Musique⋆ Bamboogle⋆ - ACI 0.417 0.598 0.430 0.378 0.372 0.143 0.352 0.384 - APS 0.435 0.603 0.433 0.393 0.374 0.143 0.344 0.389 ρ=0.1ρ=0.1 0.441 0.601 0.449 0.382 0.375 0.132 0.320 0.386 ρ=0.4ρ=0.4 0.438 0.596 0.452 0.356 0.351 0.113 0.272 0.368 αAPS=0.35 _APS=0.35 0.420 0.584 0.432 0.372 0.370 0.146 0.288 0.372 αAPS=0.05 _APS=0.05 0.442 0.613 0.459 0.328 0.285 0.076 0.248 0.350 Appendix G Case Study In this section, we present representative qualitative cases to illustrate the core behavioral patterns of CAS. Table 10 illustrates a straightforward single-hop scenario. The model directly addresses the factual query by formulating a precise search action. Because the retrieved documents are clean and highly relevant, the agent swiftly concludes its reasoning and extracts the correct answer without unnecessary actions. Table 9 presents a more challenging single-hop case characterized by high retrieval noise. Although the search results contain highly distracting entities with similar names, the agent successfully evaluates the contextual relevance of each document, filters out the irrelevant distractors, and accurately grounds its final answer on the correct source. Table 11 demonstrates the framework’s capability to handle multi-hop queries through interleaved reasoning and search. The agent dynamically decomposes the complex task in its initial <think> block, retrieves the missing bridge entity in the first hop, and uses this intermediate information to construct a targeted query for the subsequent hop. This iterative process highlights the effectiveness of allowing the model to flexibly transition between internal deliberation and external tool interaction. Question: What BBC television drama serial starring Jim McManus was based on Sarah Waters’ debut novel of the same name? Golden Answer: [’Tipping the Velvet’] <think> The question asks for a BBC television drama serial based on Sarah Waters’ debut novel of the same name. I recall that Sarah Waters’ debut novel is probably "Tipping the Velvet", but I need to confirm it, especially because Jim McManus may create ambiguity in search results. </think> <search> Jim McManus Tipping the Velvet </search> <information> Doc 1 (Title: "Jack McManus") Jack McManus Jack McManus (born Thomas McManus, March 1862 – May 1905), also known as Eat ’Em Up, was a noted New York City gangster around the turn of the 20th century. Born in Boston, he was considered one of the premier boxers of the underworld, rivaled only by Monk Eastman. Doc 2 (Title: "Jimmy Velvet") toured in 1973-1978 with the Jimmy Velvet Show band. Band members included Steve Morgan (guitar), John Collins (bassist), Dennis Moog (drums), and Kathy Slinkard Velvet. Velvet was championed by TV host Dick Clark, who had him on his "American Bandstand" show more than once. Doc 3 (Title: "Tipping the Velvet") Tipping the Velvet is a historical novel published as Sarah Waters’ debut novel in 1998. Set in England during the 1890s, it tells a coming of age story about a young woman named Nan who falls in love with a male impersonator, follows her to London, and finds various ways to support herself as she journeys through the city. Doc 4 (Title: "James McManus") James "Jim" McManus (born March 22, 1951) is an American teacher, writer and poker player living in Kenilworth, Illinois. McManus is best known as the author of the book "Positively Fifth". </information> <think> The search results contain several noisy results about other people named McManus or Velvet, but Doc 3 directly confirms that "Tipping the Velvet" is Sarah Waters’ debut novel. Since the question asks for the BBC drama serial based on that debut novel of the same name, the answer is "Tipping the Velvet". </think> <answer> Tipping the Velvet </answer> Table 9: Case Study: Complex Single-Hop QA. Question: Who won the Men’s Singles French Open Tennis Tournament in 2018? Golden Answer: [’Rafael Nadal’] <think> I need to find out who won the Men’s Singles French Open Tennis Tournament in 2018. I remember Rafael Nadal was dominant at the French Open around that time, but I should verify the exact 2018 result. </think> <search> 2018 French Open Men’s Singles winner </search> <information> Doc 1 (Title: "2018 French Open – Men’s Singles") 2018 French Open – Men’s Singles Rafael Nadal was the defending champion and successfully defended his title, defeating Dominic Thiem in the final, 6–4, 6–3, 6–2. Nadal equaled Margaret Court’s all-time record of 11 Grand Slam singles titles won at one major and became the first player to achieve that feat in the Open era. He dropped only one set during the entire tournament (losing the first set of his quarterfinal match against Diego Schwartzman). Nadal also retained the ATP No. 1 singles ranking by defending the title. Doc 2 (Title: "2018 French Open – Men’s Singles") 72nd in the ATP rankings, became the first unseeded men’s singles semifinalist at the French Open since Gaël Monfils in 2008, the lowest-ranked male singles player to reach the French Open semifinals since Andriy Medvedev in 1999, and the first Italian man to reach a Grand Slam singles semifinal since Corrado Barazzutti at the 1978 French Open. 2018 French Open – Men’s Singles Rafael Nadal was the defending champion and successfully defended his title, defeating Dominic Thiem in the final, 6–4, 6–3, 6–2. </information> <think> The search results clearly state that Rafael Nadal won the 2018 French Open men’s singles title by defeating Dominic Thiem in the final. </think> <answer> Rafael Nadal </answer> Table 10: Case Study: Simple Single-Hop QA. Question: Which hockey team does the younger brother of Sean James Pronger currently work for? Golden Answer: [’the Florida Panthers’] <think> I need to identify Sean James Pronger’s younger brother first, then determine which hockey team that person currently works for. This requires at least two steps: family relation lookup, then current team lookup. </think> <search> Sean James Pronger younger brother </search> <information> Doc 1 (Title: "Jordan Reaves") with Saskatchewan for 2018 training camp. Jordan is the youngest son of former Winnipeg Blue Bomber running back Willard Reaves, who played with the team for five years and was a member of the 72nd Grey Cup winning team. Jordan is also the younger brother of Ryan Reaves, a professional hockey player and right wing for the Vegas Golden Knights of the National Hockey League. Doc 2 (Title: "Chris Pronger") the 1990–91 season. One of his defence partners in Stratford was future NHL player Greg de Vries. In May 1991, Pronger indicated he was going to join his older brother Sean at Bowling Green State University (NCAA) instead of opting for the OHL. Regardless of his pre-draft indications, Pronger was selected in the 6th round by the Peterborough Petes in the OHL Priority Selection. Doc 3 (Title: "Luke Schenn") hits. He believes he is at his best when he is playing a tough game, while staying responsible. Growing up, Schenn patterned his play after fellow defencemen Chris Pronger and Rob Blake. Schenn was born on November 2, 1989, in Saskatoon, Saskatchewan. His younger brother Brayden was picked fifth overall in the 2009 NHL Entry Draft by the Los Angeles Kings. Doc 4 (Title: "Jordan Eberle") two sisters, Ashley and Whitney, and a younger brother, Dustin. He attended high school at the Athol Murray College of Notre Dame in Wilcox, Saskatchewan. His brother Dustin was drafted. </information> <think> Doc 2 says Chris Pronger planned to join his older brother Sean at Bowling Green State University. Therefore, Sean James Pronger’s younger brother is Chris Pronger. Now I need to find which hockey team Chris Pronger currently works for. </think> <search> Chris Pronger current team </search> <information> Doc 1 (Title: "Chris Pronger") Chris Pronger Christopher Robert Pronger (born October 10, 1974) is a Canadian former professional ice hockey defenceman who is currently the senior advisor of hockey operations for the Florida Panthers. He had not played since November 2011 due to post-concussion syndrome. Doc 2 (Title: "Chris Pronger") Chris Pronger Christopher Robert Pronger is a Canadian former professional ice hockey defenceman who is currently the senior advisor of hockey operations for the Florida Panthers. </information> <think> The second search confirms that Chris Pronger currently works as senior advisor of hockey operations for the Florida Panthers. Therefore, the hockey team is the Florida Panthers. </think> <answer> Florida Panthers </answer> Table 11: Case Study: Multi-Hop QA.