Paper deep dive
Risk-Aware Online Conformal State Probing
Pietro Talli, Petar Popovski, Osvaldo Simeone
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/23/2026, 5:16:48 AM
Summary
The paper introduces Online Conformal State Probing (OCSP), a framework for AI agents to manage uncertainty in safety-critical environments by jointly deciding on control actions and when to probe for state information. OCSP uses conformal prediction to provably control the Missed Query Error (MQE) without distributional assumptions, wrapping around pre-trained value-based policies. Simulations on a Markov Decision Process demonstrate that OCSP achieves worst-case reliability guarantees while minimizing probing rates, outperforming or matching baseline methods like 'augment-and-train' across different belief calibration levels.
Entities (5)
Relation Signals (5)
OCSP → controls → Missed Query Error
confidence 95% · OCSP is designed to provably control the missed query error (MQE)...
OCSP → basedon → Conformal Prediction
confidence 93% · OCSP builds on online risk control, which provides deterministic, distribution-free coverage and risk guarantees...
OCSP → appliedto → Markov Decision Process
confidence 92% · As a use case, OCSP is applied to an existing value-based control policy in a Markov decision process (MDP)...
OCSP → wrapsaround → pre-trained value-based control policies
confidence 90% · OCSP can be applied to existing pre-trained value-based control policies without requiring retraining or fine-tuning.
OCSP → outperformsormatches → Augment-and-train
confidence 85% · OCSP, at lower query rates, is seen to almost match the performance of the ideal augment-and-train scheme.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:AI-based autonomous agents, typically hosted at data centers, must acquire state information from robots or edge devices in order to issue informed control decisions. Managing uncertainty about the state is particularly consequential in safety-critical settings, in which average-case guarantees are insufficient. In this context, we study a sequential decision maker process that jointly decides which actions to take and when to probe given access to an arbitrary state prediction model. We propose online conformal state probing (OCSP), an action and probing policy that certifies worst-case reliability levels without relying on distributional assumptions. OCSP is designed to provably control the missed query error (MQE), i.e., the fraction of instances where probing would have been beneficial, while minimizing the probing rate. OCSP can be applied to existing pre-trained value-based control policies without requiring retraining or fine-tuning. We validate OCSP through numerical simulations to verify theoretical guarantees and to assess performance trade-offs as a function of the calibration of the state predictor.
Tags
Links
- Source: https://arxiv.org/abs/2609.25889v1
- Canonical: https://arxiv.org/abs/2609.25889v1
Trouble viewing inline? Open PDF directly →
Full Text
26,723 characters extracted from source content.
Expand or collapse full text
Risk-Aware Online Conformal State Probing Pietro Talli⋆Petar Popovski† Osvaldo Simeone †thanks: The work of O. Simeone was supported by an Open Fellowship of the EPSRC (EP/W024101/1) and by the EPSRC (EP/X011852/1). The work of P. Talli and O. Simeone was supported by the ERC (No. 101198347). Abstract AI-based autonomous agents, typically hosted at data centers, must acquire state information from robots or edge devices in order to issue informed control decisions. Managing uncertainty about the state is particularly consequential in safety-critical settings, in which average-case guarantees are insufficient. In this context, we study a sequential decision maker process that jointly decides which actions to take and when to probe given access to an arbitrary state prediction model. We propose online conformal state probing (OCSP), an action and probing policy that certifies worst-case reliability levels without relying on distributional assumptions. OCSP is designed to provably control the missed query error (MQE), i.e., the fraction of instances where probing would have been beneficial, while minimizing the probing rate. OCSP can be applied to existing pre-trained value-based control policies without requiring retraining or fine-tuning. We validate OCSP through numerical simulations to verify theoretical guarantees and to assess performance trade-offs as a function of the calibration of the state predictor. Index Terms: Conformal risk control, optimal control, active data acquisition †address: ⋆Institute for Intelligent Networked Systems, Northeastern University London †Connectivity Section, Department of Electronic Systems, Aalborg University Email: \p.talli, o.simeone\@northeastern.edu, petarp@es.aau.dk 1 Introduction Recent advances in AI, including large language models (LLMs) and vision language action models (VLAs) [2], have equipped autonomous agents, typically hosted at data centers, with greater capabilities and generalization skills. However, the deployment of AI-based autonomous agents increasingly demands strict reliability guarantees on performance [13, 14, 14]. A central difficulty in ensuring reliable operation is that agents residing on the cloud must act on system states that are not promptly available, as telemetry from robots or edge devices can be too costly to query continuously and in a timely fashion (see Fig. 1). As a result, the agent must act under partial observability, actively deciding when to probe the state in order to reduce its uncertainty about the environment before committing to an action [7]. state probingcontrol actions uncertainty for current belief Figure 1: Example of an agent that navigates the environment according to an estimated belief on the current state, actively probing the state to control the outage rate. Management of uncertainty is particularly consequential for reliability-critical applications, in which standard average-case guarantees are insufficient. In fact, a policy that performs well in expectation, but occasionally fails catastrophically, may be unacceptable. Ideally, reliability constraints must hold even in the worst case, without distributional assumptions on the environment or observation noise. In this work, we study a general sequential decision-making problem in which an intelligent agent jointly decides which control actions to execute and when to probe the state (Sec. 2). Fig. 1 illustrates the example of a robot controlled by a cloud based agent. At each time step, a state-estimation model at the agent produces an uncertainty estimate for the current state, which is used by the agent to determine control and probing actions. Reliability requirements are imposed by constraining the missed query error (MQE) rate, i.e., the fraction of steps in which probing would have been beneficial but the state was not probed [11]. To ensure provable control of the MQE, we introduce online conformal state probing (OCSP) , an action and probing policy that wraps around any pre-trained state predictor (Sec. 3). OCSP builds on online risk control, which provides deterministic, distribution-free coverage and risk guarantees for sequentially observed data [1, 6, 12, 18]. Unlike this line of work, however, OCSP does not aim to calibrate a predictor [15] or to defer from cloud to edge [11], but to certify and control the reliability of a closed-loop decision-making mechanism that jointly governs which actions to take and when to probe. As a use case, OCSP is applied to an existing value-based control policy in a Markov decision process (MDP), providing worst-case performance guarantees for a pre-trained agent (Sec. 4). 2 System Model We consider a stateful sequential decision process whose state at discrete time t is given by st∈s_t , where S is a discrete finite set. As in the example of Fig. 1, at each time step t, the agent does not have direct access to state sts_t, but it can acquire state sts_t through a probing action Pt∈0,1P_t∈\0,1\, where Pt=1P_t=1 indicates that the state is probed. The sequence of past probed states is used to maintain an estimate of the current state in the form of a probability bt∈Δb_t∈ _S given by bt=ft((t′,st′):Pt′=1t′<t),b_t=f_t(\(t ,s_t ):P_t =1\_t <t), (1) where Δ _S is the simplex of probability distributions over the set S and ftf_t is a sequence of functions. Functions ftf_t may be implemented as a pre-trained predictor, and are not subject to optimization. As discussed in Sec. 4, the belief may be evaluated using a probabilistic transition model for the state. At every time step t, the agent interacts with the environment, taking an action at∈a_t as a function of the current belief btb_t, where A is a discrete finite set. This yields the control action at=πt(bt),a_t= _t^A(b_t), with action policy πtπ^A_t to be optimized. In general, the next state st+1s_t+1 can be affected by the previous actions at′a_t with t′≤t ≤ t, and we do not make any assumption on the functional or probabilistic form of this dependence. Upon taking action ata_t, the agent collects the utility Ut(st,at)∈[Umin,Umax]U_t(s_t,a_t)∈[U_min,U_max], where Ut(⋅,⋅)U_t(·,·) is an arbitrary known function, and we have bounds Umin≥0U_min≥ 0 and Umax<∞U_max<∞. Thus, given the correct state sts_t, the optimal action is at∗∈argmaxa∈Ut(st,a),a_t^*∈ _a U_t(s_t,a), (2) which corresponds to the optimal utility value Ut∗=Ut(st,a∗)U_t^*=U_t(s_t,a^*). As further discussed in Sec. 4, the utility Ut(s,a)U_t(s,a) may be obtained as the state-action value function of a pre-trained value-based agent. For all times t with Pt=1P_t=1, the state sts_t is known and the action can be optimally set to at=at∗a_t=a^*_t. In contrast, at times with no probing, the action ata_t taken by the agent generally incurs a regret compared to the optimal action at∗a^*_t in (2), which is given by the difference Rt(st,a^t)=Ut∗−Ut(st,a^t).R_t(s_t, a_t)=U^*_t-U_t(s_t, a_t). (3) To quantify the acceptable performance loss caused by incomplete state probing, we introduce a tolerance threshold Rmax≤Umax−UminR_max≤ U_max-U_min, representing the maximum admissible regret. Violations of this threshold are captured by the binary outage variable Ot=[Rt(st,a^t)>Rmax].O_t= 1\! [R_t(s_t, a_t)>R_max ]. (4) Importantly, since the agent does not observe the state unless it actively probes it, the outage signal OtO_t is itself observed only when a probe is issued, i.e., when Pt=1P_t=1. We model the decision of whether to probe as a stochastic probing policy πt(bt)∈[0,1] _t^P(b_t)∈[0,1], with Pt∼Bernoulli(πt(bt))P_t ( _t^P(b_t)) denoting the decision to probe at time t. Overall, given fixed belief functions in (1), we aim at designing the action policy πt _t^A and the probing policy πt _t^P. The design goal is to control the MQE [11], i.e., the fraction of instances in which probing would have been beneficial, since Ot=1O_t=1, but was not performed, i.e., Pt=0P_t=0. Formally, the MQE is defined as MQET=1NT∑t=1TOt(1−Pt),MQE_T= 1N_T _t=1^TO_t\,(1-P_t), (5) where NT=∑t=1TOtN_T= _t=1^TO_t is the number of outages in the sequence, with the convention MQET=0MQE_T=0 when NT=0N_T=0. A zero MQE can be obtained by a trivial policy that always probes the state, setting Pt=1P_t=1 for all times t. Therefore, our design goal is to control the MQE, while making a best effort at minimizing the probing rate ∑t=1TPt/T _t=1^TP_t/T. Formally, we aim to ensure that, for any sequence of states stt=1T\s_t\_t=1^T, given any fixed belief functions ftt=1T\f_t\_t=1^T, the MQE does not exceed a target level α∈[0,1]α∈[0,1] when T is sufficiently large. This requirement is expressed by the inequality MQET≤α+oNT(1),MQE_T≤α+o_N_T(1), (6) where the additive term oNT(1)o_N_T(1) vanishes as the number of outages NTN_T grows. 3 Online Conformal State Probing In this section we present OCSP, an online action-probing policy that provides worst-case guarantees on the MQE as per the requirement (6), while attempting to minimize the probing rate. OCSP. At each time step t, OCSP first evaluates a candidate action a^t a_t by minimizing the regret under the current belief btb_t, i.e., a^t=argmina∈∑s∈bt(s)Rt(s,a). a_t= _a _s b_t(s)R_t(s,a). (7) We emphasize that, since the belief is an arbitrary estimate of the state, the action in (7) does not provide any optimality guarantee. The action policy πt _t^A then sets at=a^ta_t= a_t when Pt=0P_t=0, while setting at=at∗a_t=a^*_t when Pt=1P_t=1. Algorithm 1 Online Conformal State Probing (OCSP) 1: target α∈[0,1]α∈[0,1]; tolerance Rmax≥0R_ ≥ 0; step sizes η>0η>0; exploration ρ∈(0,1−α]ρ∈(0,1-α]; utility functions Utt=1T\U_t\_t=1^T; estimation functions ftt=1T\f_t\_t=1^T; initial μ1 _1 2: for t=1,2,…t=1,2,… do 3: bt←ft(st′:Pt′=1t′<t)b_t← f_t(\s_t :P_t =1\_t <t) ⊳ δs1 _s_1 if t=1 4: a^t←argmina∈∑s∈bt(s)Rt(s,a) a_t← _a _s b_t(s)R_t(s,a) 5: O^t←∑s∈bt(s)[Rt(s,a^t)>Rmax] O_t← _s b_t(s) 1[R_t(s, a_t)>R_max] ⊳ outage score 6: pt←ρ+(1−ρ) 1[O^t≥μt]p_t←ρ+(1-ρ)\, 1[ O_t≥ _t] 7: draw Pt∼Bernoulli(pt)P_t (p_t) 8: if Pt=0P_t=0 then 9: at←a^ta_t← a_t; μt+1←μt _t+1← _t 10: else⊳ state probing (rule-driven or exploratory) 11: observe sts_t; compute RtR_t and OtO_t 12: at←at∗;a_t← a^*_t; 13: update μt _t according to (10) OCSP decides whether to probe the state, setting Pt=1P_t=1, or not, Pt=0P_t=0, depending on a local estimate of the probability of an outage OtO_t in (4) for the candidate action a^t a_t. The outage probability is estimated as the probability of the outage event Ot=[Rt(s,a^t)>Rmax]O_t= 1[R_t(s, a_t)>R_max] under the belief btb_t, i.e., O^t=∑s∈bt(s)[Rt(s,a^t)>Rmax]. O_t= _s b_t(s) 1[R_t(s, a_t)>R_max]. (8) Specifically, OCSP follows a probabilistic query policy πt _t^P whereby probing is carried out with probability pt=πt(bt)=ρ+(1−ρ)[O^t≥μt],p_t= _t^P(b_t)=ρ+(1-ρ) 1[ O_t≥ _t], (9) i.e., Pt∼Bernoulli(pt)P_t (p_t), where ρ∈(0,1−α]ρ∈(0,1-α] is an exploration probability and μt _t is a threshold to be calibrated online. The rationale behind the probing probability (9) is twofold: (i) if the estimate O^t O_t is larger than a well-designed threshold μt _t, it is plausible, although not guaranteed, that the candidate action a^t a_t would lead to an outage event; and (i) in order to ensure that the true outage variable OtO_t is available at the agent sufficiently often, the agent queries the state with probability no smaller than the exploration probability ρ. The threshold in (9) is potentially updated whenever the agent decides to probe the state, i.e., all times t with Pt=1P_t=1. Based on state sts_t and outage indicator OtO_t in (4), the update of the threshold μt _t follows the rule μt+1←μt−ηOtPtpt[(1−pt)−α], _t+1← _t-η\, O_tP_tp_t [(1-p_t)\,-α ], (10) where η>0η>0 is an update rate and μ1∈[0,1] _1∈[0,1]. By (10), when Pt=0P_t=0 (no query) or Ot=0O_t=0 (no outage), the threshold is not modified and we recover μt+1←μt _t+1← _t. When Pt=1P_t=1 and Ot=1O_t=1, i.e., in the case of a query and of an outage event, the threshold μt _t is modified as follows: ∙ If O^t<μt O_t< _t, the query is prompted by exploration, since the probing probability in (9) is pt=ρp_t=ρ. This implies that the estimation O^t O_t failed to predict the outage event Ot=1O_t=1 under the current threshold μt _t. Accordingly, the threshold is decreased by the amount η(1−ρ−α)/ρ>0η(1-ρ-α)/ρ>0 to facilitate the prediction of future outage events. Recall the design constraint ρ<1−αρ<1-α. The intuition behind the magnitude of the update is that, since exploration already inherently identifies a fraction ρ of the outages, we wish the threshold mechanism to identify the remaining fraction 1−ρ1-ρ. The scale factor 1/ρ1/ρ plays the role of an inverse probability weighting [3], compensating the fact that OtO_t is selectively observed. ∙ If O^t≥μt O_t≥ _t, the query is caused by a correct thresholding of the outage probability estimate O^t O_t. In this case, the threshold is increased by ηαηα with the goal of reducing the probing rate. Missed Query Error Guarantees. We now show that OCSP guarantees condition (6) on the MQE. Proposition 1 (Performance guarantees). For any δ∈(0,1)δ∈(0,1), any state sequence stt=1T\s_t\_t=1^T, any estimation functions ftt=1T\f_t\_t=1^T and utility functions Ut∈[Umin,Umax]t=1T\U_t∈[U_min,U_max]\_t=1^T, with probability at least 1−δ1-δ over the algorithm’s exploration mechanism in (9), OCSP satisfies the upper bound MQET≤α+Δ(NT,δ)MQE_T\;≤\;α+ (N_T,δ) (11) where, for NT≥1N_T≥ 1, we have defined Δ(NT,δ)=1+2η/ρηNT+2(1−ρ)ρNTlog2δ+2(1−ρ)3ρNTlog2δ, (N_T,δ)= 1+2η/ρη N_T+ 2(1-ρ)ρ N_T 2δ+ 2(1-ρ)3ρ N_T 2δ, and Δ(0,δ)=0 (0,δ)=0. Proof. We start by rewriting (5) as MQET=α+1NT[∑t=1Tet+∑t=1TOt1−αpt(pt−Pt)].MQE_T=α+ 1N_T [ _t=1^Te_t+ _t=1^TO_t 1-αp_t(p_t-P_t) ]. (12) The first sum in (12) is deterministically controlled by the update (10) in a manner similar to online conformal prediction [6], and we omit details here due to space constraints, while the second sum can be upper bounded with high probability with respect to the sequence of probing decisions Pt\P_t\, concluding the proof. For the second sum, writing Pt−1=[P1,…,Pt−1]P^t-1=[P_1,...,P_t-1] and =t:Ot=1t≤TT^O=\t:O_t=1\_t≤ T, we have [Pt∣Pt−1]=ptE[P_t P^t-1]=p_t, and thus the sum ∑t∈(pt−Pt)/pt _t ^O(p_t-P_t)/p_t is a martingale difference sequence. Moreover, we have Var((pt−Pt)/pt∣Pt−1)=(1−pt)/pt≤(1−ρ)/ρVar((p_t-P_t)/p_t P^t-1)=(1-p_t)/p_t≤(1-ρ)/ρ. Therefore, using Freedman’s inequality [5] on the sequence (pt−Pt)/pt\(p_t-P_t)/p_t\, we obtain the inequality |∑t∈pt−Ptpt|≤2(1−ρ)NTρlog2δ+2(1−ρ)3ρlog2δ | _t ^O p_t-P_tp_t |≤ 2 (1-ρ)N_Tρ 2δ+ 2(1-ρ)3ρ \ 2δ (13) with probability no smaller than 1−δ1-δ. ∎ 4 Experiments 002⋅10−22· 10^-24⋅10−24· 10^-26⋅10−26· 10^-28⋅10−28· 10^-20.10.1Augment-and-trainOCSP w/ underconf. beliefOCSP w/ exact beliefOCSP w/ overconf. beliefUpper bound (δ=0.05δ=0.05) ((a)) 05×1045× 10^410510^50.20.20.30.30.40.40.50.5time ttMQE(a) ((b)) 05×1045× 10^410510^50.50.50.550.550.60.60.650.650.70.7time ttProbing Rate(b) ((c)) 05×1045× 10^410510^52.52.52.62.62.72.72.82.82.92.9time ttReturn(c) ((d)) Figure 1: MQE, average query rate and average reward over time t for 10510^5 steps. Results are averaged over 100 different seeds. We validate OCSP and its theoretical guarantees by investigating the MQE, utility and query rate of OCSP under different belief models, while comparing with the ideal benchmark in [17], which requires state augmentation and retraining. Setting. Even though our framework applies to generic, even deterministic and adversarial, dynamic systems, we consider here an MDP, since this allows us comparisons with established benchmarks. The considered MDP has ||=30|S|=30 states, ||=4|A|=4 actions, and a transition model M obtained as follows. For each pair (s,a)(s,a) we randomly select a target state sa∗∈s_a^* and define the transition probability as M(s,a,s′)∝exp(−d(s′,sa∗)2/σ2)M(s,a,s ) (-d(s ,s_a^*)^2/σ^2), where d(s,s∗)=min(|s−s∗|,−|s−s∗|)d(s,s^*)= (|s-s^*|,S-|s-s^*|) is a circular distance between discrete states and σ2σ^2 = 0.67. The state sequence is drawn from this model given the control actions. The utility is derived from a reward given by the distance from a randomly selected target state s∗s^* as r(s,a,s′)=10exp(−0.5⋅d(s′,s∗))r(s,a,s )=10 (-0.5· d(s ,s^*)). Specifically, the utility Ut(s,a)U_t(s,a) is obtained from a pre-trained state-action value function Q(s,a)Q(s,a) optimized by running Q-learning [10] using the implementation provided in [4] over the given MDP. Implementation. When Pt−1=0P_t-1=0, the belief btb_t is obtained from an estimated transition model M~ M by updating the last belief bt−1b_t-1 given action at−1a_t-1 as ft(bt−1,at−1)=bt−1⊺M~at−1,f_t(b_t-1,a_t-1)=b_t-1 M_a_t-1, where M~at−1 M_a_t-1 is the ||×|||S|×|S| transition matrix associated with action at−1a_t-1. The estimated transition model M~ M is defined using the same moder as for M, but with a generally different dispersion parameter σ2σ^2. Specifically we considered three settings corresponding to predictive models with different calibration levels [8, 16, 9]: (i) exact belief, i.e., σ2=0.67σ^2=0.67; (i) overconfident belief with a lower spread of σ2=0.37σ^2=0.37; and (i) underconfident belief with the larger spread σ2=2σ^2=2. Baseline. As a baseline, we consider the “augment-and-train” methodology in [17], which trains from scratch in the MDP a policy that simultaneously learns how to act and probe. This policy operates on an augmented state space, describing also the time elapsed from the last probed state, and it is trained to meet an average probing rate. Augment-and-train serves as an upper bound on the performance of OCSP for a given probing rate, since OCSP operates as a wrapper around a pre-trained value function. In Fig. 2 we plot MQETMQE_T, the average probing rate and the average normalized cumulative return, as a function of time t. Fig. 2 demonstrates that OCSP can control the MQE to meet the target rate α=0.3α=0.3, irrespective of the quality of the prediction without requiring training. In contrast, the augment-and-train scheme, which is trained for the given target probing rate, which is set to approximately match that of OCSP with an exact belief (see Fig. 3(b)), yields an MQE as high as 0.4. The figure also reports the bound from Proposition 1 with δ=0.05δ=0.05 to further validate the theoretical guarantees of OCSP. In Fig. 3(b) we observe that OCSP tends to concentrate state probings in the first time steps, while stabilizing to a lower probing rate as time goes on. Specifically, an underconfident belief induces candidate actions (7) that cater to a larger set of possible states, thus requiring OCSP to issue fewer probing actions to compensate for regret events. In contrast, with an overconfident belief, the candidate actions (7) may make riskier choices tailored to a particular state, and OCSP compensates for resulting outage events by increasing the probing rate. The different probing rates translate into different cumulative returns, as shown in Fig. 3(c). In particular, Fig. 3(c) shows that OCSP with an overconfident belief obtains a slightly higher average return than OCSP with an exact belief but with higher average probing rate. 0.20.20.30.30.40.40.50.50.60.60.70.70.80.82.02.22.42.62.83.0α=0.2α=0.2α=0.3α=0.3α=0.4α=0.4α=0.5α=0.5Probing RateCumulative ReturnAugment-and-trainOCSP w/ exact beliefOCSP w/ overconf. beliefOCSP w/ underconf. beliefrandom Figure 2: Cumulative reward vs. probing rate for different values of α. The trade-off between probing rate and cumulative return is further investigated in Fig. 2, which shows the Pareto front on the plane with axes given by cumulative return and probing rate, both evaluated at time t=105t=10^5. The Pareto front is obtained for OCSP by varying the MQE target α, while for augment-and-train we varied the target probing rate. OCSP, at lower query rates, is seen to almost match the performance of the ideal augment-and-train scheme. Using an exact belief is observed to be instrumental for achieving a better performance trade-off with OCSP. Moreover, an overconfident belief model is preferable to an underconfident one at low query rates, while, when the query rate is higher than 0.7, the underconfident belief obtains a higher trade-off. 5 Conclusions In this work, we developed a sequential decision-making framework with adaptive state probing and formal performance guarantees. Future work may focus on larger-scale evaluations and on applications to AI-based policy models. References [1] A. N. Angelopoulos and S. Bates (2023) Conformal prediction: a gentle introduction. Foundations and Trends in Machine Learning 16 (4), p. 494–591. Cited by: §1. [2] K. Black, M. Galliker, and S. Levine (2025) Real-time execution of action chunking flow policies. In Advances in Neural Information Processing Systems, Vol. 38, p. 33383–33407. External Links: Document Cited by: §1. [3] E. Candès, L. Lei, and Z. Ren (2023) Conformalized survival analysis. Journal of the Royal Statistical Society Series B: Statistical Methodology 85 (1), p. 24–45. Cited by: §3. [4] I. Chadès, G. Chapron, M. Cros, F. Garcia, and R. Sabbadin (2014) MDPtoolbox: a multi-platform toolbox to solve stochastic dynamic programming problems. Ecography 37 (9), p. 916–920. Cited by: §4. [5] D. A. Freedman (1975) On tail probabilities for martingales. Annals of Probability, p. 100–118. Cited by: §3. [6] I. Gibbs and E. J. Candès (2024) Conformal inference for online prediction with arbitrary distribution shifts. Journal of Machine Learning Research 25 (162), p. 1–36. Cited by: §1, §3. [7] D. Gündüz, F. Chiariotti, K. Huang, A. E. Kalør, S. Kobus, and P. Popovski (2023) Timely and massive communication in 6G: pragmatics, learning, and inference. IEEE BITS the Information Theory Magazine 3 (1), p. 27–40. Cited by: §1. [8] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger (2017) On calibration of modern neural networks. In International conference on machine learning, p. 1321–1330. Cited by: §4. [9] J. Huang, S. Park, N. Paoletti, and O. Simeone (2025) Distilling calibration via conformalized credal inference. In 2025 International Joint Conference on Neural Networks (IJCNN), p. 1–10. Cited by: §4. [10] T. Jaakkola, M. I. Jordan, and S. P. Singh (1993) Convergence of stochastic iterative dynamic programming algorithms. In Proceedings of the 7th International Conference on Neural Information Processing Systems, NIPS’93, p. 703–710. Cited by: §4. [11] S. Kiyani, S. Noorani, G. Pappas, and H. Hassani (2026) Strategic decision support for ai agents. arXiv preprint arXiv:2606.12587. Cited by: §1, §1, §2. [12] J. Lekeufack, A. N. Angelopoulos, A. Bajcsy, M. I. Jordan, and J. Malik (2024) Conformal decision theory: safe autonomous decisions from imperfect predictions. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 11668–11675. Cited by: §1. [13] C. Li, R. Zhang, J. Wong, C. Gokmen, S. Srivastava, R. Martín-Martín, C. Wang, G. Levine, M. Lingelbach, J. Sun, et al. (2023) Behavior-1k: a benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In Conference on Robot Learning, p. 80–93. Cited by: §1. [14] S. Rabanser, S. Kapoor, P. Kirgis, K. Liu, S. Utpala, and A. Narayanan (2026) Towards a science of AI agent reliability. arXiv preprint arXiv:2602.16666. Cited by: §1. [15] G. Shafer and V. Vovk (2008) A tutorial on conformal prediction.. Journal of machine learning research 9 (3). Cited by: §1. [16] O. Simeone (2022) Machine learning for engineers. Cambridge university press. Cited by: §4. [17] P. Talli, E. D. Santi, F. Chiariotti, T. Soleymani, F. Mason, A. Zanella, and D. Gündüz (2025) Pragmatic communication for remote control of finite-state markov processes. IEEE Journal on Selected Areas in Communications 43 (7), p. 2589–2603. Cited by: §4, §4. [18] M. Zecchin, U. K. Ganesan, G. Durisi, P. Popovski, and O. Simeone (2026) Prediction-powered communication with distortion guarantees. IEEE Journal on Selected Areas in Information Theory 7, p. 33–45. Cited by: §1.