Paper deep dive
Evaluation Metrics for Safe Reinforcement Learning
Lindsay Spoor, Aske Plaat, Thomas Moerland
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Safe reinforcement learning (RL) is commonly formalized as a Constrained Markov Decision Process (CMDP), in which an agent maximizes expected reward while keeping its expected cumulative cost below a specified safety bound. Existing safe RL benchmarks predominantly report whether an algorithm is safe on average, following this expectation-based guarantee. We argue that this convention is insufficient to reliably characterize an algorithm's true safety: it fails to capture how often and how severely the safety bound is violated, whether this holds consistently across tasks and safety bounds, and whether training-time behavior is representative of behavior of the final converged policy. Therefore, we introduce (i) evaluation metrics for safe RL that address each of these concerns and in addition allow for aggregation across tasks and safety bounds. We furthermore define (ii) a safety tier system to systematically categorize and compare algorithms in terms of safety and reliability at both training and for a final policy. Using this framework, we provide (iii) an empirical safety evaluation across multiple safety navigation tasks. Our results show that aggregate metrics, distributional reporting, and task- and safety bound-specific results each reveal information the other metrics cannot. We therefore recommend reporting all three jointly, rather than compressing this information into a single value, as is common practice. We provide SafeRLEval, an open-source evaluation suite to support the reliable characterization of safety in future safe RL research.
Tags
Links
- Source: https://arxiv.org/abs/2609.15315v1
- Canonical: https://arxiv.org/abs/2609.15315v1
Trouble viewing inline? Open PDF directly →
Full Text
71,851 characters extracted from source content.
Expand or collapse full text
Evaluation Metrics for Safe Reinforcement Learning Lindsay Spoor †thanks: corresponding author: l.j.spoor@liacs.leidenuniv.nl†thanks: Leiden Institute of Advanced Computer Science, Leiden University, Leiden, the Netherlands Aske Plaat22footnotemark: 2 Thomas Moerland22footnotemark: 2 Abstract Safe reinforcement learning (RL) is commonly formalized as a Constrained Markov Decision Process (CMDP), in which an agent maximizes expected reward while keeping its expected cumulative cost below a specified safety bound. Existing safe RL benchmarks predominantly report whether an algorithm is safe on average, following this expectation-based guarantee. We argue that this convention is insufficient to reliably characterize an algorithm’s true safety: it fails to capture how often and how severely the safety bound is violated, whether this holds consistently across tasks and safety bounds, and whether training-time behavior is representative of behavior of the final converged policy. Therefore, we introduce (i) evaluation metrics for safe RL that address each of these concerns and in addition allow for aggregation across tasks and safety bounds. We furthermore define (i) a safety tier system to systematically categorize and compare algorithms in terms of safety and reliability at both training and for a final policy. Using this framework, we provide (i) an empirical safety evaluation across multiple safety navigation tasks. Our results show that aggregate metrics, distributional reporting, and task- and safety bound-specific results each reveal information the other metrics cannot. We therefore recommend reporting all three jointly, rather than compressing this information into a single value, as is common practice. We provide SafeRLEval11 1 https://github.com/lindsayspoor/SafeRLEval, an open-source evaluation suite to support the reliable characterization of safety in future safe RL research. 1 Introduction Reinforcement learning (RL) addresses sequential decision-making problems by enabling agents to learn from feedback in the form of rewards, with the goal of maximizing their long-term cumulative reward (Sutton et al., 2018). Despite their success on tasks without critical safety concerns (Mnih et al., 2015; Schulman et al., 2017; Haarnoja et al., 2018), agents deployed in safety-critical domains, such as autonomous driving, robotics, and power systems (Ramanujam et al., 2025), must often satisfy restrictive constraints on their behavior, in addition to maximizing task performance. Safe RL provides a framework for such safety-critical tasks, in which the learning objective is extended to explicitly incorporate constraints. This is commonly formalized as a Constrained Markov Decision Process (CMDP) (Altman, 1999), in which the agent’s objective is to maximize expected reward subject to [C]≤dE[C]≤ d, where C denotes the cumulative episodic cost and d the safety bound. Safe RL has been extensively studied over the last decade, leading to a variety of approaches (Garcıa and Fernández, 2015), including Lagrangian (primal-dual) methods (Ray et al., 2019; Tessler et al., 2019; Stooke et al., 2020), trust-region methods (Zhang et al., 2020; Achiam et al., 2017), and increasing cost penalty coefficients in the objective (Zhang et al., 2022). Existing safe RL benchmarks generally report whether an algorithm is safe on average (Ray et al., 2019; Ji et al., 2024). This convention follows directly from the CMDP formulation itself, with the standard formal guarantee, [C]≤dE[C]≤ d, being expectation-based. Ray et al. (2019) proposed an evaluation protocol that distinguishes average performance during training from that of the final policy, alongside an average training regret capturing cumulative unsafe behavior incurred while learning. However, this protocol does not indicate how often, or how severely, a policy violates its constraints, nor does it distinguish between evaluating the final policy with exploration noise enabled and evaluating it as a greedy policy, i.e., with exploration noise disabled. Even though more recent benchmark suites (Ji et al., 2023; Ji et al., 2024; Tomilin et al., 2026; Ramanujam et al., 2025) have substantially expanded the range of available tasks, algorithms, and constraint types, the standard evaluation protocol has changed comparatively little. We argue that an expectation-based guarantee, with a measure of safety collapsed into a single aggregate value, remains insufficient to reliably characterize a method’s true safety. Additionally, behavior observed during training is often subject to, and optimized under, exploration noise. However, the performance can differ for the behavior of a final, converged policy evaluated without exploration noise, since such a policy is commonly deployed deterministically in simulation. Figure 1 illustrates this on a representative single-seed training run of Lagrangian PPO (Tomilin et al., 2026): although all evaluated episodes of the converged final policy lie below the bound d when exploration noise is enabled, a substantial fraction of episodes violate d when the same policy is instead evaluated deterministically, showing a distributional shift between the two settings that, in this case, results in an even higher violation rate. Such shifts can occur because Lagrangian methods only constrain the expected cost under the stochastic policy used during training, while the deterministic policy, which simply takes the most likely action at each step, was never directly subject to this constraint. Figure 1: An example illustrating that the formal guarantee [C]≤dE[C]≤ d in the CMDP framework falls short of reliably ensuring safety, from a single-seed run of Lagrangian-based PPO, task: Safe Circle Point (Level 1), d=25d=25 (Tomilin et al., 2026). (left) The Monte Carlo estimate of [C]E[C] is shown by the smoothed mean. All individual data points show single-episode costs. Even though the final policy with exploration noise shows that the full distribution of episodes lies below d (middle), greedy policy rollouts of the final policy show a distribution with a substantially higher violation rate (right). This motivates the need to unpack the aggregate value of safety, typically reported as cost on average, into more informative measures, such as violation rate, violation magnitude, and cost deviation from the safety bound, both with and without exploration noise in the final policy. Moreover, current evaluation practices in safe RL have yet to adopt statistically rigorous aggregation of performance across a range of tasks, as well as across a range of safety bounds (Spoor et al., 2025). Agarwal et al. (2021) showed in the general RL literature that reporting single point estimates of performance, without accounting for variability across runs and tasks, can produce misleading comparisons, and proposed more statistically robust aggregate measures. Such aggregation across tasks, and in particular across safety bounds, remains largely absent from the safe RL literature. Moreover, Safe RL yet has to adopt an evaluation protocol that jointly reports these distributional properties alongside the single-value metrics of evaluated algorithms. We therefore propose the following contributions: 1. Evaluation metrics for safe RL We introduce evaluation metrics for safe RL that go beyond reporting average reward and cost alone, focusing on violation frequency, violation magnitude, and cost deviation, each normalized to allow for aggregation across both tasks and safety bounds. We provide an open-source evaluation suite at https://github.com/lindsayspoor/SafeRLEval. 2. Safety tier system We define a set of safety tiers, based on conditions tied to the proposed evaluation metrics, that allow existing safe RL algorithms to be systematically categorized and compared by their safety and reliability, both for any policy during training and for the final converged policy, rather than by average performance alone. 3. Empirical safety evaluation We provide a robust empirical evaluation across a variety of tasks, multiple safety bounds, a variety of popular safe RL algorithms, and 30 seeds. Alongside the standard practice of reporting reward and cost training curves, our evaluation additionally reports the episodic cost distribution over the entire course of training, for the final policy under exploration noise, and for the final greedy policy, both for individual tasks and safety bounds and aggregated across all such conditions. 2 Safe reinforcement learning Safe RL commonly uses the formal framework of a Constrained Markov Decision Process (CMDP) (Altman, 1999), an extension of the Markov Decision Process (MDP) (Sutton et al., 2018; Bellman, 1957), in which we not only want to maximize desired behavior, but also want to minimize undesired, unsafe behavior. A CMDP is defined as the tuple ℳ=(,,P,R,P0,γ,)M=(S,A,P,R,P_0,γ,C), where S and A are the set of all possible states and actions, respectively, P is the transition dynamics distribution P:×→Δ()P:S×A→ (S), R is the reward function R:×→ℝR:S× A× S , P0∈Δ()P_0∈ (S) is the initial state distribution, and γ∈[0,1)γ∈[0,1) is the discount factor. A set of m cost functions :=C1,…,CmC:=\C_1,...,C_m\ with associated safety bounds :=d1,…,dmD:=\d_1,...,d_m\ maps transition tuples to costs, :×→ℝmC:S× A× S ^m. Actions are selected from a policy π:→Δ()π:S→ (A), s↦π(⋅|s)s π(·|s). For a trajectory τ=(st,at,st+1,…)τ=(s_t,a_t,s_t+1,...), the discounted return and discounted cost are given by R(τ)=∑t=0TγtR(st,at,st+1)R(τ)= _t=0^Tγ^tR(s_t,a_t,s_t+1) and Ci(τ)=∑t=0TγtCi(st,at,st+1)C_i(τ)= _t=0^Tγ^tC_i(s_t,a_t,s_t+1), respectively. The objective of a CMDP is to find a policy that maximizes expected discounted return while keeping each expected discounted cost within its safety bound: maxπ∈Πτ∼π,P[R(τ)]s.t.τ∼π,P[Ci(τ)]≤di,i=1,…,m, aligned & _π∈ \;E_τ π,P [R(τ) ]\\ &s.t. _τ π,P [C_i(τ) ]≤ d_i, i=1,...,m, aligned (1) where Π denotes the set of policies. Unlike standard MDPs (Bellman, 1957), policy search under a CMDP must additionally ensure that each policy update remains feasible with respect to its constraints, i.e., that it stays within the feasible set Π:=π∈Π:τ∼π[Ci(τ)]≤di∀i _C:=\π∈ :E_τ π[C_i(τ)]≤ d_i\;∀ i\, rather than optimizing over Π alone. For a single cost function (m=1m=1), we write this constraint as [C]≤dE[C]≤ d, with C the cumulative episodic cost and d the safety bound. In this study, we evaluate several widely used safe RL algorithms, introduced below. A large family of safe RL algorithms address the constrained optimization problem in Eq. 1 by modifying the underlying objective function itself. Lagrangian relaxation (Ray et al., 2019; Tessler et al., 2019; Stooke et al., 2020) penalizes the reward objective in proportion to the constraint violation, with the Lagrange multiplier learned jointly with the policy via dual gradient ascent. Under convexity conditions, this guarantees feasibility only asymptotically, offering no guarantee for any individual policy iterate. P3O (Zhang et al., 2022) similarly penalizes the objective, but instead progressively increases a fixed-form penalty coefficient in response to observed violations, avoiding the instability of a learned Lagrange multiplier at the loss of a formal convergence guarantee. An alternative family of methods instead restricts the policy update itself. FOCOPS (Zhang et al., 2020) solves this constrained update via a first-order approximation. This approach, in approximation, enforces feasibility at every policy iteration. 3 Evaluation metrics We first characterize an algorithm’s safety within a single fixed condition: a single task m, safety bound d, algorithm a, and seed k. Each of the introduced metrics can be used to evaluate a fixed policy πj _j at policy iterate j∈0,…,Jj∈\0,...,J\, where J denotes the final policy iterate, resulting in the converged policy πfinal _final. Instance i denotes a single evaluation episode out of N episodes evaluated under policy πj _j, with N episodes forming a Monte Carlo estimate of the expectation in the CMDP definition from Eq. 1. We then describe how these metrics are aggregated across all policy iterates, tasks, and safety bounds, and furthermore introduce a safety tier system based on the resulting metrics, enabling a generalized comparison of algorithms. Exploration noise Exploration noise is an inherent part of the training process itself, making it the relevant condition for characterizing safety while the policy is still being learned, whereas a converged policy is typically deployed deterministically in practice, making the greedy evaluation the relevant condition for characterizing deployment-time safety. All of our proposed metrics can be evaluated under two distinct policy conditions: with exploration noise enabled, or with a greedy policy, with exploration noise disabled. Comparing them directly isolates the effect that exploration noise itself has on the safety of an algorithm. Throughout this paper, we report metrics evaluated with exploration noise enabled when characterizing behavior over the course of training, and metrics evaluated under both a greedy and an exploration-based policy when characterizing the final policy, for direct comparison. 3.1 Metrics for a fixed policy As a standard performance metric already reported by prior benchmarks, the average reward and cost are described as in Eq. 2. R¯(m,d,a,k)j R^j_(m,d,a,k) =1N∑i=1Nri,(m,d,a,k),C¯(m,d,a,k)j=1N∑i=1Nci,(m,d,a,k), = 1N _i=1^Nr_i,(m,d,a,k), C^j_(m,d,a,k)= 1N _i=1^Nc_i,(m,d,a,k), (2) where ri,(m,d,a,k)r_i,(m,d,a,k) and ci,(m,d,a,k)c_i,(m,d,a,k) are the reward and cost of rollout episode i under policy iterate j, for seed k∈1,…,Kk∈\1,…,K\, task m, safety bound d, and algorithm a. We propose the violation rate as a metric that indicates how often an algorithm violates constraints, which is the fraction of episodes on task m with bound d in which algorithm a exceeded the safety bound d, as described by Eq. 3. V(m,d,a,k)j=1N∑i=1N[ci,(m,d,a,k)>d], V^j_(m,d,a,k)= 1N _i=1^N 1[c_i,(m,d,a,k)>d], (3) where [⋅] 1[·] is the indicator function. Additionally, we introduce the mean normalized cost deviation as the metric that captures how far algorithm a’s average cost sits above (>0>0, unsafe) or below (<0<0, safe) the bound, scaled by the bound itself, as in Eq. 4. Dnorm,(m,d,a,k)j=C¯(m,d,a,k)j−d. D^j_norm,(m,d,a,k)= C^j_(m,d,a,k)-dd. (4) Unlike Ray et al. (2019), who normalize deviation from the bound relative to a naive, unconstrained baseline algorithm like PPO (Schulman et al., 2017), we normalize directly by the safety bound d itself. Because an unconstrained baseline typically incurs costs far above d, the deviation term would be large, and the normalized scores of safe RL algorithms would vanish. We instead treat d as the natural unit of measurement for cost deviation, centering DnormD_norm around 00 for C¯=d C=d. Among the episodes in which algorithm a violated safety bound d, the normalized violation magnitude then captures how much they overshot the bound on average, relative to the bound itself, following Eq. 5. Dnorm+,(m,d,a,k)j=1Na,k+d∑i=1N(ci,(m,d,a,k)−d)+if Na,k+>0,0if Na,k+=0, D^j_norm^+,(m,d,a,k)= cases 1N^+_a,k\,d _i=1^N (c_i,(m,d,a,k)-d )^+&if N^+_a,k>0,\\[6.45831pt] 0&if N^+_a,k=0, cases (5) where Na,k+=∑i=1N[ci,(m,d,a,k)>d]N^+_a,k= _i=1^N 1[c_i,(m,d,a,k)>d] is the number of violating rollout episodes for algorithm a under seed k, and (x)+:=max(x,0)(x)^+:= (x,0) denotes the positive part of x. Along with DnormD_norm from Eq. 4, which reflects how far the algorithm’s cost sits from the bound on average, over all episodes, Dnorm+D_norm^+ in Eq. 5 is restricted to the set of violating episodes only. This distinction matters because two algorithms with an identical average cost can exhibit very different violation profiles. An algorithm may violate the bound frequently but only slightly on each occasion, or violate rarely but catastrophically when it does. 3.2 Aggregation across policy iterates Each metric X(m,d,a,k)j∈R¯(m,d,a,k)j,C¯(m,d,a,k)j,V(m,d,a,k)j,Dnorm,(m,d,a,k)j,Dnorm+,(m,d,a,k)jX^j_(m,d,a,k)∈\ R^j_(m,d,a,k), C^j_(m,d,a,k),V^j_(m,d,a,k),D^j_norm,(m,d,a,k),D^j_norm^+,(m,d,a,k)\ is defined for a fixed policy πj _j at policy iterate j. To evaluate an algorithm’s performance across the entire course of training, i.e., aggregated over all policy iterates, we take the average across all iterates, as shown in Eq. 6. Since this is meant to capture behavior during the training process, we evaluate policy πj _j with exploration noise enabled for this aggregation. X(m,d,a,k)=1J∑j=0JX(m,a,d,k)j. X_(m,d,a,k)= 1J _j=0^JX^j_(m,a,d,k). (6) 3.3 Aggregation across tasks and safety bounds To characterize an algorithm’s overall safety performance, each per-seed metric is first averaged over seeds k∈1,…,Kk∈\1,...,K\, and then aggregated over the full set of task–bound pairs (m,d)∈(m,d) . Following Agarwal et al. (2021), we report the interquartile mean (IQM) across P, taking the mean of the values that fall between the 25th and 75th percentile. We denote this as IQM(Xaj)IQM (X^j_a ) for a fixed policy πj _j, and as IQM(Xa)IQM (X_a ) for the metric aggregated over the entire course of training, as defined in Eq. 6. Safety tiers While the level of detail in the introduced metrics is essential for a reliable evaluation, it can also make it difficult to directly compare algorithms. Therefore, we define thresholds on each X∈Va,Dnorm,a,Dnorm+,aX∈\V_a,D_norm,a,D_norm^+,a\, separating algorithms into safety tiers from level 0 to 4, with increasingly stringent safety requirements, following Table 1. We argue that any algorithm should be assessed on these tiers under two settings: over the course of training, using IQM(Xaexpl)IQM(X^expl_a), where the superscript expl denotes evaluation with exploration noise enabled; and at the final, greedy policy, using IQM(Xafinal,greedy)IQM(X^final,greedy_a). This means, for example, that an algorithm may achieve tier 1 during training, but tier 2 at the final greedy policy. Table 1: Safety tiers based on threshholds for the aggregated metrics IQM(Dnorm)IQM(D_norm), IQM(V)IQM(V), and IQM(Dnorm+)IQM(D_norm^+), for a single algorithm a. Tier Label Conditions 0 Unsafe – 1 Minimal IQM(Dnorm)≤0IQM(D_norm)≤ 0 2 Moderate IQM(Dnorm)≤0IQM(D_norm)≤ 0 and IQM(V)≤0.5IQM(V)≤ 0.5 3 Substantial IQM(Dnorm)≤0IQM(D_norm)≤ 0 and IQM(V)≤0.1IQM(V)≤ 0.1 and IQM(Dnorm+)≤0.1IQM(D_norm^+)≤ 0.1 4 Strict IQM(Dnorm)≤0IQM(D_norm)≤ 0 and IQM(V)=0IQM(V)=0 and IQM(Dnorm+)=0IQM(D_norm^+)=0 IQM(Dnorm)≤0IQM(D_norm)≤ 0 is required across all tiers from 1 upward, since it directly reflects an algorithm being safe on average. We therefore treat this as the minimal requirement for tier 1. However, IQM(Dnorm)≤0IQM(D_norm)≤ 0 alone does not guarantee infrequent violations. It is possible for the average cost to sit below the safety bound while the majority of episodes still violate it, with only large undershoots in a minority of episodes pushing the mean below the bound. We therefore impose an additional threshold for tier 2 at IQM(V)≤0.5IQM(V)≤ 0.5, such that the majority of episodes are not allowed to violate the safety bound. Because moving directly from tier 2 to strict safety would leave too large a gap between a moderately-safe algorithm and one that is fully constraint-satisfying, we introduce an intermediate tier 3, for which we set IQM(V)≤0.1IQM(V)≤ 0.1, and additionally set IQM(Dnorm+)≤0.1IQM(D_norm^+)≤ 0.1, so that, even under this low violation rate, the average overshoot on the episodes that do violate remains within 10%10\% of the safety bound. Tier 4 characterizes strict safety: the average cost is not allowed to exceed the safety bound, and no constraint violations are permitted at all. From IQM(V)=0IQM(V)=0, it then naturally follows that IQM(Dnorm+)IQM(D_norm^+) is 00 as well, since there are no violating episodes over which to compute an overshoot. Aggregate cost distribution The metrics introduced in this section summarize an algorithm’s safety only at the single threshold where cost equals the bound. To capture an algorithm’s performance across the entire range of possible deviations from the safety bound, we recommend reporting the empirical cumulative distribution function (CDF) of the normalized episodic cost deviation. This provides an informative visualization of constraint adherence as a performance profile of an algorithm, and offers a visual way to compare algorithms across the entire set of episodes, rather than compressing this information into a single value, as recommended by Agarwal et al. (2021). For algorithm a, the aggregate empirical CDF evaluated at threshold κ is given by Eq. 7. CDFaj(κ)=∑(m,d)∈∑k=1K∑i=1N[ci,(m,d,a,k)−d≤κ]||⋅K⋅N, ^j_a(κ)= _(m,d) _k=1^K _i=1^N 1\! [ c_i,(m,d,a,k)-dd≤κ ]|P|· K· N, (7) where ci,(m,d,a,k)c_i,(m,d,a,k) is the cost of the i-th episode, evaluated under policy πj _j, N is the number of evaluated episodes per seed, and [⋅] 1[·] is the indicator function. As with the other metrics introduced in this section, the CDF can be computed for a fixed policy πj _j at any training iterate j, for the final policy evaluated with exploration noise enabled or under a greedy policy, or aggregated across the entire course of training. This allows the effect of exploration noise on the full distribution of constraint adherence to be visualized directly, both over the course of training and at the final policy. 4 Results For a detailed description of our experimental setup, we refer to Appendix Section A.1. The main focus of this paper is on the results of each metric aggregated over all tasks and bounds, shown in Table 2. Based on these aggregated metrics, we classify each algorithm into one of the safety tiers from Table 1, both over the entire course of training and at the final greedy policy. We furthermore present results on the aggregated metrics evaluated on final policies with exploration noise on and off (greedy) in Table A.3 in Appendix A.2.1. Results for single tasks and bounds are reported in Appendix A.2.2, along with their training curves and episodic cost distributions. The results in Table 2 show that all three safe RL algorithms reach a higher safety tier for a final greedy policy than over the entire course of training with exploration noise, which is expected for the selection of studied algorithms, since all of them only guarantee constraint satisfaction asymptotically or within a local trust region, rather than at every individual policy iterate encountered during training. These results also show that FOCOPS is marginally safer than PPO-Lag in terms of violation rate over training, yet FOCOPS is assigned tier 0 for training while PPO-Lag reaches tier 2. This is because FOCOPS’s IQM(Dnorm)IQM(D_norm) over training is 0.060.06, narrowly above the threshold of IQM(Dnorm)≤0IQM(D_norm)≤ 0 required for tier 1 and above. Nevertheless, FOCOPS achieves the highest IQM(R¯)IQM( R) at both training and for a final greedy policy, while simultaneously achieving the lowest IQM(Dnorm)IQM(D_norm), IQM(V)IQM(V), and IQM(Dnorm+)IQM(D_norm^+) for a final greedy policy. PPO-Lag, by contrast, is the most conservative algorithm during training, with IQM(Dnorm+)=0.25IQM(D_norm^+)=0.25, but achieves a substantially lower IQM(R¯)IQM( R) than FOCOPS. Table 2: Safety and performance evaluation across algorithms, aggregated over all K=30K=30 seeds for all (m,d)∈(m,d) pairs, IQM reported on each metric, for both over the course of all policy iterates (Train) and final greedy policy (Final) settings. CIs for this table, computed via stratified bootstrapping, are reported in Appendix Table A.3. Bold text indicates best result in that specific column, excluding PPO, as this is not a safe RL algorithm. Lower is better for each metric except for IQM(R¯)IQM( R). PPO-Lag reaches tier 2 for training, even though its IQM(V)IQM(V) is slightly higher than for FOCOPS, which sits in tier 0 due to IQM(Dnorm)=0.06IQM(D_norm)=0.06 for training. Algorithm IQM(R¯)IQM( R) IQM(C¯)IQM( C) IQM(Dnorm)IQM(D_norm) IQM(V)IQM(V) IQM(Dnorm+)IQM(D_norm^+) Tier Train Final Train Final Train Final Train Final Train Final Train Final PPO 115.63115.63 68.1768.17 145.98145.98 83.6083.60 4.774.77 2.202.20 1.001.00 0.940.94 4.794.79 2.292.29 0 0 PPO-Lag 77.7277.72 54.1854.18 26.62 26.62 18.1418.14 0.00 0.00 −0.13-0.13 0.430.43 0.340.34 0.25 0.25 0.500.50 2 2 P3O 90.4190.41 57.8057.80 45.6945.69 23.7623.76 0.550.55 −0.03-0.03 0.700.70 0.420.42 0.810.81 0.530.53 0 2 FOCOPS 108.76 108.76 74.26 74.26 30.3930.39 17.09 17.09 0.060.06 −0.19 -0.19 0.41 0.41 0.29 0.29 0.340.34 0.48 0.48 0 2 Figure 2 shows the empirical CDFs computed by Eq. 7, for three settings: over the entire course of training, for the final policy evaluated with exploration noise enabled, and for the final, greedy policy. The ordering of the curves at κ=0κ=0 matches the ordering of the violation rate reported in Table 2. Note, however, that the values do not correspond one-to-one: Table 2 reports IQM(V)IQM(V), whereas the CDF is computed over all instances. The overall shape of each curve to the right of κ=0κ=0 reflects the severity of constraint violations across all conditions, rather than a single-value statistic. A curve that rises steeply immediately to the right of κ=0κ=0 indicates that, on the episodes where the safety bound is exceeded, overshoot tends to be small, whereas a curve with a longer tail extending further to the right instead reflects episodes with more severe violations, consistent with a higher Dnorm+D_norm^+. The curves for all three safe RL algorithms shift visibly to the left when moving from training to the final policy, i.e., toward κ=0κ=0 and below, consistent with the improvement in safety tier from training to the final policy observed in Table 2. Notably, within the final policy, the curve for the policy with exploration noise consistently lies to the left of the curve for the greedy policy, across all four algorithms, indicating fewer and less severe constraint violations under exploration noise than under a greedy policy. This is consistent with the results shown in Table A.3 in Appendix A.2.1. The CMDP guarantee [C]≤dE[C]≤ d constrains the expected cost under the policy with exploration noise during training, but the greedy policy was never itself directly subject to this constraint. Because this pattern holds consistently across all tasks and safety bounds, these results point to a general limitation of relying on evaluations under a final policy with exploration noise as a good indication of the performance of a final greedy policy. Figure 2: CDFs following Eq. 7, over the entire range of training (left), for a final policy with exploration noise on (middle), and for a final greedy policy (right). Threshold κ=0κ=0 is indicated by the dashed vertical line at Dnorm=0D_norm=0. Shaded regions indicate the 95% CIs computed via stratified bootstrap. The x-axis is cut off at Dnorm=2D_norm=2 for visual purposes. For a visualization across the entire range of the x-axis, see Figure A.1 in Appendix A.2.1. Unlike the aggregate metrics in Table 2, the CDFs reveal the full shape of the underlying violation distribution across all thresholds, not just a single summary value. The CDF curves for the final policy with exploration noise enabled cross κ=0κ=0 at a higher density value than those for the greedy final policy. This indicates that, for the algorithms shown, lower costs and fewer violations occur under exploration noise than under a greedy policy. 5 Discussion While our proposed evaluation protocol provides a robust set of metrics that allow for a broadened view of safety and reliability, as well as aggregation across multiple tasks and bounds, we would like to point out that the classification into safety tiers can, in some cases, be sharply determined by a single condition. A small, consistent overshoot of bound d can shift an algorithm by multiple tiers: for instance, in Table 2, FOCOPS’s training IQM(Dnorm)IQM(D_norm) is only marginally above zero, yet this alone places it two tiers below PPO-Lag, despite FOCOPS achieving a comparable and even lower training IQM(V)IQM(V). This illustrates that the thresholds introduced in Table 1 can be sensitive to small differences near a boundary, and that the assigned safety tiers should be interpreted alongside the underlying metric values rather than as a standalone summary. Additionally, aggregating results across tasks and safety bounds using the IQM is only meaningful for metrics that are directly comparable across tasks and safety bounds with potentially very different scales. V is a rate bounded in [0,1][0,1] by construction, and DnormD_norm and Dnorm+D_norm^+ are explicitly normalized by the safety bound d, making all three comparable across tasks and bounds. By contrast, R¯ R and C¯ C are not normalized and can vary substantially in scale across tasks, which is why our safety tiers are conditioned only on V, DnormD_norm and Dnorm+D_norm^+. Limitations Our empirical evaluation is limited to four tasks and three safety bounds. While a broader evaluation across additional task domains (Tomilin et al., 2025; Ramanujam et al., 2025) and bounds would further strengthen the generality of our findings, the selected conditions were chosen to capture a broad range of diversity in constraint types and task complexity, providing a clear empirical basis for the results presented in this study, with approximately 2,880 GPU-hours on NVIDIA L4 GPUs with 24GB of memory. However, our results are limited to algorithms that permit constraint violations during training and only guarantee constraint adherence asymptotically or approximately. Stricter safe RL methods that disallow constraint violations even during training (Alshiekh et al., 2018; Dalal et al., 2018; As et al., 2025; Cheng et al., 2019) are yet to be evaluated using our proposed framework. We therefore provide an open-source evaluation suite that enables the community to adopt our proposed evaluation metrics and apply them to other safe RL algorithms and task domains, which would help further extending the broadened view of safety and reliability that this work provides. Future Work For safe real-world deployment, the gap between simulation-based and real-world deployment performance should ideally be negligible. Our results, however, show a substantial gap between a final policy with and without exploration noise for the algorithms evaluated in this work, even within simulation alone. Future work could compare the evaluation of simulated greedy policy rollouts against real-world deployment performance, using the distributions of V, DnormD_norm, and Dnorm+D_norm^+ introduced here to quantify this gap directly. The results in Appendix A.2.2 reveal that the gap between exploration and greedy final-policy performance, observed in the aggregate results in Section 4, is not uniform across tasks: the episodic cost distributions in Figures A.7-A.10 show a substantial difference between exploration and greedy policies for Safe Circle Point and Safe Goal Point, but considerable overlap for Safe Button Point and Safe Push Point. This suggests that the gap between final policies with exploration noise on and off may itself be task-dependent, and could be further explored in future studies. We argue that a natural extension of this work is to incorporate the proposed evaluation metrics directly into algorithm design, rather than using them only for evaluation purposes. For example, an algorithm’s training objective could explicitly minimize V, DnormD_norm or Dnorm+D_norm^+. This would allow safe RL algorithms to be optimized directly against the same safety and reliability criteria used to evaluate them. 6 Conclusion In this work, we introduced a comprehensive safe RL evaluation framework, including the violation rate (V), mean normalized cost deviation (DnormD_norm), and normalized violation magnitude (Dnorm+D_norm^+), together with a safety tier system and a distributional view of constraint adherence, directly addressing the insufficiency of the expectation-based guarantee [C]≤dE[C]≤ d illustrated in Figure 1. Our evaluation study showed that no algorithm dominates uniformly across all conditions and both training and for a final policy: FOCOPS achieves the strongest overall balance of reward and safety at a final greedy policy, while PPO-Lag trades off reward for a lower violation magnitude over the course of training. We further found that, across all evaluated algorithms, the final policy is consistently less safe when evaluated greedily than under exploration noise, pointing to a general limitation of relying on evaluations under a final policy with exploration noise as a good indication of the performance of a final greedy policy. Additionally, our results show that aggregate safety tiers, the CDF, and results per task and safety bound each reveal information that is crucial to a complete evaluation of an algorithm’s safety and reliability. We therefore recommend reporting all three jointly, rather than mean cost or mean reward alone, and provide an open-source evaluation suite to provide the tools to assess safe RL algorithms by their reliability, not just their average performance. References Achiam et al. (2017) J. Achiam, D. Held, A. Tamar, and P. Abbeel Constrained Policy Optimization. arXiv. Note: arXiv:1705.10528Comment: Accepted to ICML 2017 External Links: Document Cited by: §1. Agarwal et al. (2021) R. Agarwal, M. Schwarzer, P. S. Castro, A. C. Courville, and M. Bellemare Deep reinforcement learning at the edge of the statistical precipice. Advances in neural information processing systems 34, p. 29304–29320. Cited by: Appendix A.1, §1, §3.3, §3.3. Alshiekh et al. (2018) M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, and U. Topcu Safe reinforcement learning via shielding. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32. Cited by: §5. Altman (1999) E. Altman Constrained Markov Decision Processes. Routledge, New York. External Links: ISBN 978-1-315-14022-3, Document Cited by: §1, §2. As et al. (2025) Y. As, L. Treven, C. Sferrazza, S. Coros, and A. Krause Actsafe: active exploration with safety constraints for reinforcement learning. In International Conference on Learning Representations, Vol. 2025, p. 29752–29776. Cited by: §5. Bellman (1957) R. Bellman A Markovian Decision Process. Journal of Mathematics and Mechanics 6 (5), p. 679–684. Note: Publisher: Indiana University Mathematics Department External Links: ISSN 0095-9057 Cited by: §2, §2. Cheng et al. (2019) R. Cheng, G. Orosz, R. M. Murray, and J. W. Burdick End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33, p. 3387–3395. Cited by: §5. Dalal et al. (2018) G. Dalal, K. Dvijotham, M. Vecerik, T. Hester, C. Paduraru, and Y. Tassa Safe exploration in continuous action spaces. arXiv preprint arXiv:1801.08757. Cited by: §5. Efron (1992) B. Efron Bootstrap methods: another look at the jackknife. In Breakthroughs in statistics: Methodology and distribution, p. 569–593. Cited by: Appendix A.1. Garcıa and Fernández (2015) J. Garcıa and F. Fernández A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research 16 (1), p. 1437–1480. Cited by: §1. Haarnoja et al. (2018) T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, p. 1861–1870. Cited by: §1. Ji et al. (2023) J. Ji, B. Zhang, J. Zhou, X. Pan, W. Huang, R. Sun, Y. Geng, Y. Zhong, J. Dai, and Y. Yang Safety gymnasium: a unified safe reinforcement learning benchmark. Advances in Neural Information Processing Systems 36, p. 18964–18993. Cited by: §1. Ji et al. (2024) J. Ji, J. Zhou, B. Zhang, J. Dai, X. Pan, R. Sun, W. Huang, Y. Geng, M. Liu, and Y. Yang Omnisafe: an infrastructure for accelerating safe reinforcement learning research. Journal of Machine Learning Research 25 (285), p. 1–6. Cited by: §1. Mnih et al. (2015) V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. Human-level control through deep reinforcement learning. nature 518 (7540), p. 529–533. Cited by: §1. Ramanujam et al. (2025) A. Ramanujam, A. Elyoumi, H. Chen, S. M. Kompalli, A. S. Ahluwalia, S. Pal, D. J. Papageorgiou, and C. Li Safeor-gym: a benchmark suite for safe reinforcement learning algorithms on practical operations research problems. arXiv preprint arXiv:2506.02255. Cited by: §1, §1, §5. Ray et al. (2019) A. Ray, J. Achiam, and D. Amodei Benchmarking safe exploration in deep reinforcement learning. arXiv preprint arXiv:1910.01708 7 (1), p. 2. Cited by: Appendix A.1, §1, §1, §2, §3.1. Schulman et al. (2017) J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. Cited by: Appendix A.1, §1, §3.1. Spoor et al. (2025) L. Spoor, Á. Serra-Gómez, A. Plaat, and T. Moerland Towards a practical understanding of lagrangian methods in safe reinforcement learning. arXiv preprint arXiv:2510.17564. Cited by: §1. Stooke et al. (2020) A. Stooke, J. Achiam, and P. Abbeel Responsive Safety in Reinforcement Learning by PID Lagrangian Methods. In Proceedings of the 37th International Conference on Machine Learning, p. 9133–9143 (en). Note: ISSN: 2640-3498 Cited by: §1, §2. Sutton et al. (2018) R. S. Sutton A. G. Barto et al. Reinforcement learning: an introduction. Vol. 2. Cited by: §1, §2. Tessler et al. (2019) C. Tessler, D. J. Mankowitz, and S. Mannor Reward constrained policy optimization. In International Conference on Learning Representations, Cited by: §1, §2. Tomilin et al. (2026) T. Tomilin, M. Boustani, M. Beurskens, and T. D. Simão CRAX: fast safe reinforcement learning benchmarking. arXiv preprint arXiv:2606.20376. Cited by: Appendix A.1, §A.2.2, Figure 1, Figure 1, §1, §1. Tomilin et al. (2025) T. Tomilin, M. Fang, and M. Pechenizkiy HASARD: a benchmark for vision-based safe reinforcement learning in embodied agents. External Links: 2503.08241 Cited by: §5. Zhang et al. (2022) L. Zhang, L. Shen, L. Yang, S. Chen, B. Yuan, X. Wang, and D. Tao Penalized proximal policy optimization for safe reinforcement learning. arXiv preprint arXiv:2205.11814. Cited by: Appendix A.1, §1, §2. Zhang et al. (2020) Y. Zhang, Q. Vuong, and K. Ross First Order Constrained Optimization in Policy Space. In Advances in Neural Information Processing Systems, Vol. 33, p. 15338–15349. External Links: Link Cited by: Appendix A.1, §1, §2. Appendix A.1 Experimental setup We train algorithms PPO-Lag (Ray et al., 2019), P3O (Zhang et al., 2022), FOCOPS (Zhang et al., 2020) and, as an unconstrained baseline, PPO (Schulman et al., 2017) using the CRAX benchmark suite (Tomilin et al., 2026), for 30 seeds each. The set of tasks is ℳ=Safe Goal Point, Safe Circle Point, Safe Push Point, Safe Button PointM=\Safe Goal Point, Safe Circle Point, Safe Push Point, Safe Button Point\, all Level 1, and the set of safety bounds is =15,25,50D=\15,25,50\. This set of tasks focuses on safe navigation and consists of different types of constraints and rewards, selected carefully to span a set of tasks with sufficient variety in their cost functions. We train each run for 5⋅1085· 10^8 total environment steps, with an episode length of T=2000T=2000 steps. During training, exploration noise is turned on, and training metrics are recorded once per complete batch: after every cycle in which all N=2048N=2048 parallel environments finish their episode (episode length T=2000T=2000 steps, or equivalently every ∼4.2⋅106 4.2· 10^6 environment steps), yielding J=122J=122 data points per seed over 5⋅1085· 10^8 total environment steps. Each data point is averaged over the 20482048 episodes in that batch, for a total of approximately 2.5⋅1052.5· 10^5 episodes per seed. At the final policy evaluation we turn off exploration noise and use a deterministic trained (greedy) policy, and we report the average over N=100N=100 evaluation episodes per seed. For metric X∈R¯,C¯,Dnorm,V,Dnorm+X∈\ R, C,D_norm,V,D^+_norm\, we report the task- and bound-specific results as 1K∑k=1KX(m,d,a,k)±σk 1K _k=1^KX_(m,d,a,k)± _k, where σk _k is the sample standard deviation for seed k, for K=30K=30 seeds. This standard deviation reflects the variability across individual training runs under identical conditions, and does not shrink with additional seeds. For aggregation over all tasks and safety bounds, we use the IQM as explained in Section 3.3. Since the IQM is a nonlinear statistic of the distribution across conditions, we report uncertainty as a 95% confidence interval (CI) computed via a stratified bootstrap with B=2000B=2000 replicates (Efron, 1992; Agarwal et al., 2021). Unlike the standard deviation in a single task or bound, this confidence interval reflects the standard error of the IQM estimate itself, i.e., how much the aggregated IQM value would be expected to change under a different sample of seeds, and would narrow with additional seeds or conditions. We therefore report both: the standard deviation characterizes the inherent variability of individual training runs, while the IQM confidence interval characterizes the reliability of our aggregate statistics. Likewise, the CDF curve following Eq. 7 is estimated from finite data (K seeds × N episodes per condition), so we used the same stratified bootstrap procedure to report the CI band showing how much the curve would shift under different seeds. Compute resources All experiments were run on NVIDIA L4 GPUs (24 GB memory), using a single GPU and 4 CPU cores per run. A single training run of 5⋅·108 environment steps took approximately 2 hours of wall-clock time. Our full experimental suite consists of 4 tasks × 3 safety bounds × 4 algorithms (PPO, PPO-Lag, P3O, FOCOPS) × 30 seeds, totaling 1,440 runs, for an estimated total of approximately 2,880 GPU-hours, or equivalently, approximately 11,520 CPU-core-hours, given 4 cores allocated per run. Table A.1: Hyperparameters used across all experiments. Shared (all algorithms) Hyperparameter Value Hyperparameter Value Total environment steps 5×1085× 10^8 Discount factor γ 0.990.99 Episode length 20002000 GAE λ 0.950.95 Parallel environments 20482048 PPO clipping ε 0.30.3 Unroll length 88 Entropy coefficient 5×10−35× 10^-3 Batch size 10241024 Reward scaling 0.10.1 Number of minibatches 3232 Observation normalisation ✓ SGD epochs per batch 66 Deterministic eval. episodes 100100 Learning rate 5×10−45× 10^-4 Seeds 3030 Algorithm-specific PPO-Lag FOCOPS Lagrange multiplier LR 3.03.0 Initial ν 0.10.1 Initial λ 0.00.0 ν learning rate 1.01.0 νmax _ 100.0100.0 KL penalty λ 1.51.5 Advantage temperature η 0.020.02 P3O Initial κ 0.010.01 κ increase factor 1.11.1 κmax _ 50.050.0 Appendix A.2 Additional results A.2.1 Aggregate metrics Figure A.1 shows the empirical CDFs computed by Eq. 7, aggregated over all tasks, safety bounds, and seeds. Table A.3 presents algorithm performance on the aggregated metrics evaluated on final policies with exploration noise on (expl) and off (greedy), together with their stratified bootstrapped CIs. The results show that, across all four algorithms, every metric is significantly lower under exploration noise than under the greedy policy, consistent with the distributional shift illustrated in Figure A.1 (which is better visible in Figure 2 in the main paper). This holds even for the unconstrained PPO baseline, indicating that a final greedy policy systematically increases constraint violation across all evaluated algorithms, rather than being specific to safe RL algorithms only. Figure A.1: CDFs following Eq. 7, over the entire range of training (left), for a final policy with exploration noise on (middle), and for a final greedy policy (right). Threshold κ=0κ=0 is indicated by the dashed vertical line at Dnorm=0D_norm=0. Shaded regions indicate the 95% CIs computed via stratified bootstrap. Table A.2: Safety and performance evaluation across algorithms, aggregated over all (m,d)(m,d) pairs. IQM reported with 95% stratified bootstrap CI [lower,upper][lower,upper] (B=2000B=2000), comparing the final policy evaluated with exploration noise enabled against the final, greedy policy. Bold indicates best result per column, excluding PPO. Algorithm IQM(R¯)IQM( R) IQM(C¯)IQM( C) IQM(Dnorm)IQM(D_norm) IQM(V)IQM(V) IQM(Dnorm+)IQM(D_norm^+) Tier Final (expl.) Final (greedy) Final (expl.) Final (greedy) Final (expl.) Final (greedy) Final (expl.) Final (greedy) Final (expl.) Final (greedy) Final (expl.) Final (greedy) PPO 50.37[49.32,51.43]50.37_ [49.32,51.43] 68.17[67.07,69.22]68.17_ [67.07,69.22] 60.63[59.07,62.13]60.63_ [59.07,62.13] 83.60[81.24,85.73]83.60_ [81.24,85.73] 1.055[1.002,1.101]1.055_ [1.002,1.101] 2.196[2.111,2.280]2.196_ [2.111,2.280] 0.821[0.805,0.835]0.821_ [0.805,0.835] 0.941[0.923,0.958]0.941_ [0.923,0.958] 1.285[1.238,1.321]1.285_ [1.238,1.321] 2.288[2.213,2.366]2.288_ [2.213,2.366] 0 0 PPO-Lag 38.80[35.91,40.60]38.80_ [35.91,40.60] 54.18[51.99,56.39]54.18_ [51.99,56.39] 14.26[13.17,15.47]14.26_ [13.17,15.47] 18.14[16.50,20.69]18.14_ [16.50,20.69] −0.477[−0.509,−0.418]-0.477_ [-0.509,-0.418] −0.126[−0.204,−0.089]-0.126_ [-0.204,-0.089] 0.110[0.090,0.136]0.110_ [0.090,0.136] 0.341[0.312,0.373]0.341_ [0.312,0.373] 0.425[0.244,0.348]0.425_ [0.244,0.348] 0.497[0.429,0.514]0.497_ [0.429,0.514] 2 2 P3O 42.68[41.01,44.23]42.68_ [41.01,44.23] 57.80[55.95,59.65]57.80_ [55.95,59.65] 15.71[14.70,16.78]15.71_ [14.70,16.78] 23.76[21.80,25.81]23.76_ [21.80,25.81] −0.357[−0.401,−0.325]-0.357_ [-0.401,-0.325] −0.025[−0.046,−0.006]-0.025_ [-0.046,-0.006] 0.175[0.142,0.207]0.175_ [0.142,0.207] 0.421[0.400,0.432]0.421_ [0.400,0.432] 0.433[0.304,0.392]0.433_ [0.304,0.392] 0.531[0.478,0.557]0.531_ [0.478,0.557] 2 2 FOCOPS 52.36[50.79,53.63] 52.36_ [50.79,53.63] 74.26[72.60,75.76] 74.26_ [72.60,75.76] 14.21[12.99,15.29] 14.21_ [12.99,15.29] 17.09[15.53,19.51] 17.09_ [15.53,19.51] −0.490[−0.510,−0.453] -0.490_ [-0.510,-0.453] −0.193[−0.265,−0.131] -0.193_ [-0.265,-0.131] 0.109[0.089,0.137] 0.109_ [0.089,0.137] 0.291[0.258,0.319] 0.291_ [0.258,0.319] 0.305[0.202,0.244] 0.305_ [0.202,0.244] 0.479[0.306,0.414] 0.479_ [0.306,0.414] 2 2 Table A.3: Safety and performance evaluation across algorithms, aggregated over all (m,d)(m,d) pairs. IQM reported with 95% stratified bootstrap CI [lower,upper][lower,upper] (B=2000B=2000), for both over the course of all policy iterates (Train) and final greedy policy (Final) settings. Bold indicates best result per column, excluding PPO. Algorithm IQM(R¯)IQM( R) IQM(C¯)IQM( C) IQM(Dnorm)IQM(D_norm) IQM(V)IQM(V) IQM(Dnorm+)IQM(D_norm^+) Tier Train Final Train Final Train Final Train Final Train Final Train Final PPO 115.63[114.87,116.38]115.63_ [114.87,116.38] 68.17[67.08,69.21]68.17_ [67.08,69.21] 145.98[145.18,146.83]145.98_ [145.18,146.83] 83.60[81.24,85.79]83.60_ [81.24,85.79] 4.77[4.70,4.85]4.77_ [4.70,4.85] 2.20[2.11,2.28]2.20_ [2.11,2.28] 1.00[1.00,1.00]1.00_ [1.00,1.00] 0.94[0.92,0.96]0.94_ [0.92,0.96] 4.79[4.72,4.87]4.79_ [4.72,4.87] 2.29[2.21,2.36]2.29_ [2.21,2.36] 0 0 PPO-Lag 77.72[73.31,80.50]77.72_ [73.31,80.50] 54.18[51.92,56.40]54.18_ [51.92,56.40] 26.62[25.69,27.77] 26.62_ [25.69,27.77] 18.14[16.48,20.79]18.14_ [16.48,20.79] 0.00[−0.01,0.00] 0.00_ [-0.01,0.00] −0.13[−0.21,−0.09]-0.13_ [-0.21,-0.09] 0.43[0.42,0.44]0.43_ [0.42,0.44] 0.34[0.31,0.37]0.34_ [0.31,0.37] 0.25[0.23,0.28] 0.25_ [0.23,0.28] 0.50[0.46,0.56]0.50_ [0.46,0.56] 2 2 P3O 90.41[89.32,91.55]90.41_ [89.32,91.55] 57.80[56.04,59.67]57.80_ [56.04,59.67] 45.69[44.70,46.75]45.69_ [44.70,46.75] 23.76[21.79,25.82]23.76_ [21.79,25.82] 0.55[0.52,0.57]0.55_ [0.52,0.57] −0.03[−0.05,−0.01]-0.03_ [-0.05,-0.01] 0.70[0.69,0.72]0.70_ [0.69,0.72] 0.42[0.40,0.43]0.42_ [0.40,0.43] 0.81[0.78,0.83]0.81_ [0.78,0.83] 0.53[0.49,0.57]0.53_ [0.49,0.57] 0 0 FOCOPS 108.76[107.77,109.78] 108.76_ [107.77,109.78] 74.26[72.63,75.83] 74.26_ [72.63,75.83] 30.39[30.26,30.52]30.39_ [30.26,30.52] 17.09[15.58,19.51] 17.09_ [15.58,19.51] 0.06[0.06,0.06]0.06_ [0.06,0.06] −0.19[−0.26,−0.13] -0.19_ [-0.26,-0.13] 0.41[0.40,0.41] 0.41_ [0.40,0.41] 0.29[0.26,0.32] 0.29_ [0.26,0.32] 0.34[0.33,0.34]0.34_ [0.33,0.34] 0.48[0.43,0.52] 0.48_ [0.43,0.52] 0 2 A.2.2 Task- and bound-specific results All training curves plotted in Figures A.3-A.6 are smoothed using a moving average filter with a window of 5. The episodic distributions for final policies with exploration noise on or off (greedy) for each individual task and safety bound are shown in Figures A.7-A.10, over N=100N=100 rollouts. All metrics X∈R¯,C¯,Dnorm,V,Dnorm+X∈\ R, C,D_norm,V,D^+_norm\ are reported in Tables A.4-A.7 as 1K∑k=1KX(m,d,a,k)±σk 1K _k=1^KX_(m,d,a,k)± _k, where σk _k is the sample standard deviation for seed k, for K=30K=30 seeds, for both over the entire course of training, and for the final greedy policy. On the results for Safe Button Point The results for PPO-Lag on Safe Button Point in Table A.7 show that, for d=25d=25, the average reward R¯ R is negative at both training and for the final policy, and drops substantially further at the final policy performance (−8.84±41.09-8.84± 41.09) compared to training (−0.17±9.63-0.17± 9.63). We suspect this artifact is caused by a collapse in the cost, in which the Lagrange multiplier λ grew too large during training, effectively freezing the policy into doing nothing as shown in Figure 6(b). We confirm this by inspecting the corresponding Lagrange multiplier trajectory, shown in Figure A.2. Furthermore, comparing the training and final policy columns for R¯ R and DnormD_norm across all algorithms on Safe Button Point in Table A.7, both metrics show a large drop from training to final policy performance. Rewards in this task come from actively pressing goal buttons (Tomilin et al., 2026), which requires exploration or directed behavior. The final (greedy) policies of all evaluated algorithms appear to converge to a passive strategy resulting in low reward and low cost, possibly navigating the environment without pressing any buttons at all. This is consistent with the drop in V observed at the final policy performance across all algorithms: pressing fewer buttons means fewer opportunities to trigger a hazard and violate the safety constraint. Figure A.2: Lagrange multiplier λ for PPO-Lag on Safe Button Point (Level 1), for safety bounds d∈15,25,50d∈\15,25,50\, averaged across 30 seeds. The shaded regions denote the 1σ1σ over all seeds. The curve for d=25d=25 shows that λ grows very large compared to d=15d=15 and d=50d=50, suggesting that the policy for d=25d=25 is frozen throughout the course of training. (a) d=15d=15 (b) d=25d=25 (c) d=50d=50 Figure A.3: Smoothed training curves for Safe Goal Point (Level 1), for safety bounds d∈15,25,50d∈\15,25,50\, averaged over 30 seeds. The shaded regions denote the 1σ1σ over 30 seeds. (a) d=15d=15 (b) d=25d=25 (c) d=50d=50 Figure A.4: Smoothed training curves for Safe Circle Point (Level 1), for safety bounds d∈15,25,50d∈\15,25,50\, averaged over 30 seeds. The shaded regions denote the 1σ1σ over 30 seeds. (a) d=15d=15 (b) d=25d=25 (c) d=50d=50 Figure A.5: Smoothed training curves for Safe Push Point (Level 1), for safety bounds d∈15,25,50d∈\15,25,50\, averaged over 30 seeds. The shaded regions denote the 1σ1σ over 30 seeds. (a) d=15d=15 (b) d=25d=25 (c) d=50d=50 Figure A.6: Smoothed training curves for Safe Button Point (Level 1), for safety bounds d∈15,25,50d∈\15,25,50\, averaged over 30 seeds. The shaded regions denote the 1σ1σ over 30 seeds. Figure A.7: Episodic cost distributions for final policies with exploration noise on and off (greedy). Safe Goal Point (Level 1). Averaged over 30 seeds. Error bars denote 1σ1σ over 30 seeds. Table A.4: Safety and performance evaluation per metric for Safe Goal Point (Level 1), for each individual d∈15,25,50d∈\15,25,50\. The numbers indicate mean ±1σ± 1σ over 30 seeds. Bold text indicates best result in that specific column, excluding PPO, as this is not a safe RL algorithm. Algorithm R¯ R C¯ C DnormD_norm V Dnorm+D_norm^+ Tier Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) d=15d=15 PPO 72.45±1.5072.45± 1.50 72.05±1.5972.05± 1.59 199.05±3.47199.05± 3.47 200.39±5.23200.39± 5.23 12.27±0.2312.27± 0.23 12.36±0.3512.36± 0.35 1.00±0.001.00± 0.00 1.00±0.001.00± 0.00 12.27±0.2312.27± 0.23 12.36±0.3512.36± 0.35 T0 F0 PPO-Lag 62.46±2.3462.46± 2.34 63.82±1.7963.82± 1.79 14.84±0.2514.84± 0.25 14.78±1.3214.78± 1.32 −0.01±0.02 -0.01± 0.02 −0.01±0.09-0.01± 0.09 0.44±0.030.44± 0.03 0.43±0.080.43± 0.08 0.22±0.07 0.22± 0.07 0.41±0.070.41± 0.07 T2 F2 P3O 58.12±4.6658.12± 4.66 63.15±1.9663.15± 1.96 26.68±1.7626.68± 1.76 14.80±1.2114.80± 1.21 0.78±0.120.78± 0.12 −0.01±0.08-0.01± 0.08 0.62±0.030.62± 0.03 0.44±0.070.44± 0.07 1.29±0.161.29± 0.16 0.41±0.070.41± 0.07 T0 F2 FOCOPS 68.86±0.41 68.86± 0.41 73.73±0.67 73.73± 0.67 15.79±0.05 15.79± 0.05 12.65±2.19 12.65± 2.19 0.05±0.000.05± 0.00 −0.16±0.15 -0.16± 0.15 0.32±0.03 0.32± 0.03 0.29±0.14 0.29± 0.14 0.44±0.070.44± 0.07 0.28±0.08 0.28± 0.08 T0 F2 d=25d=25 PPO 72.92±1.2172.92± 1.21 72.35±1.3372.35± 1.33 199.35±3.56199.35± 3.56 201.53±5.87201.53± 5.87 6.97±0.146.97± 0.14 7.06±0.237.06± 0.23 1.00±0.001.00± 0.00 1.00±0.001.00± 0.00 6.97±0.146.97± 0.14 7.06±0.247.06± 0.24 T0 F0 PPO-Lag 64.70±1.6564.70± 1.65 65.40±1.4065.40± 1.40 24.79±0.3624.79± 0.36 23.76±1.2723.76± 1.27 −0.01±0.01 -0.01± 0.01 −0.05±0.05-0.05± 0.05 0.46±0.030.46± 0.03 0.40±0.060.40± 0.06 0.12±0.02 0.12± 0.02 0.32±0.040.32± 0.04 T2 F2 P3O 60.79±3.1460.79± 3.14 64.00±1.9164.00± 1.91 34.75±1.0834.75± 1.08 24.01±2.4024.01± 2.40 0.39±0.040.39± 0.04 −0.04±0.10-0.04± 0.10 0.59±0.020.59± 0.02 0.43±0.110.43± 0.11 0.70±0.080.70± 0.08 0.31±0.050.31± 0.05 T0 F2 FOCOPS 70.44±0.35 70.44± 0.35 74.17±2.12 74.17± 2.12 25.49±0.04 25.49± 0.04 23.53±6.57 23.53± 6.57 0.02±0.000.02± 0.00 −0.06±0.26 -0.06± 0.26 0.37±0.03 0.37± 0.03 0.36±0.21 0.36± 0.21 0.22±0.020.22± 0.02 0.28±0.18 0.28± 0.18 T0 F2 d=50d=50 PPO 72.66±1.4872.66± 1.48 72.34±1.3472.34± 1.34 200.85±3.50200.85± 3.50 202.53±5.50202.53± 5.50 3.02±0.073.02± 0.07 3.05±0.113.05± 0.11 1.00±0.001.00± 0.00 1.00±0.001.00± 0.00 3.02±0.073.02± 0.07 3.05±0.113.05± 0.11 T0 F0 PPO-Lag 67.32±1.4667.32± 1.46 67.29±1.6167.29± 1.61 49.87±0.2249.87± 0.22 48.60±1.8048.60± 1.80 0.00±0.00 0.00± 0.00 −0.03±0.04-0.03± 0.04 0.49±0.040.49± 0.04 0.44±0.060.44± 0.06 0.06±0.01 0.06± 0.01 0.23±0.030.23± 0.03 T2 F2 P3O 64.86±2.7164.86± 2.71 64.89±4.1064.89± 4.10 57.01±0.9257.01± 0.92 48.99±2.3048.99± 2.30 0.14±0.020.14± 0.02 −0.02±0.05-0.02± 0.05 0.56±0.030.56± 0.03 0.45±0.070.45± 0.07 0.28±0.030.28± 0.03 0.24±0.040.24± 0.04 T0 F2 FOCOPS 72.39±0.27 72.39± 0.27 75.65±0.60 75.65± 0.60 50.27±0.02 50.27± 0.02 46.39±3.35 46.39± 3.35 0.01±0.000.01± 0.00 −0.07±0.07 -0.07± 0.07 0.43±0.03 0.43± 0.03 0.37±0.10 0.37± 0.10 0.08±0.010.08± 0.01 0.19±0.04 0.19± 0.04 T0 F2 Figure A.8: Episodic cost distributions for final policies with exploration noise on and off (greedy). Safe Circle Point (Level 1). Averaged over 30 seeds. Error bars denote 1σ1σ over 30 seeds. Table A.5: Safety and performance evaluation per metric for Safe Circle Point (Level 1), for each individual d∈15,25,50d∈\15,25,50\. The numbers indicate mean ±1σ± 1σ over 30 seeds. Bold text indicates best result in that specific column, excluding PPO, as this is not a safe RL algorithm. Algorithm R¯ R C¯ C DnormD_norm V Dnorm+D_norm^+ Tier Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) d=15d=15 PPO 234.91±17.41234.91± 17.41 240.14±37.48240.14± 37.48 93.02±5.4093.02± 5.40 91.52±12.3191.52± 12.31 5.20±0.365.20± 0.36 5.10±0.825.10± 0.82 0.99±0.010.99± 0.01 0.98±0.110.98± 0.11 5.24±0.355.24± 0.35 5.24±0.455.24± 0.45 T0 F0 PPO-Lag 111.81±52.03111.81± 52.03 100.74±80.28100.74± 80.28 17.55±8.36 17.55± 8.36 12.37±15.73 12.37± 15.73 0.17±0.56 0.17± 0.56 −0.18±1.05-0.18± 1.05 0.35±0.12 0.35± 0.12 0.34±0.430.34± 0.43 2.02±1.102.02± 1.10 2.89±3.662.89± 3.66 T0 F2 P3O 161.95±36.26161.95± 36.26 151.60±69.61151.60± 69.61 41.46±12.1441.46± 12.14 29.86±25.4829.86± 25.48 1.76±0.811.76± 0.81 0.99±1.700.99± 1.70 0.76±0.090.76± 0.09 0.55±0.380.55± 0.38 2.38±0.832.38± 0.83 1.83±1.831.83± 1.83 T0 F0 FOCOPS 162.69±9.35 162.69± 9.35 179.63±45.95 179.63± 45.95 24.42±1.2524.42± 1.25 11.55±21.0711.55± 21.07 0.63±0.080.63± 0.08 −0.23±1.40 -0.23± 1.40 0.48±0.110.48± 0.11 0.24±0.42 0.24± 0.42 1.65±0.50 1.65± 0.50 1.78±1.54 1.78± 1.54 T0 F2 d=25d=25 PPO 230.53±20.00230.53± 20.00 244.86±22.52244.86± 22.52 92.04±7.7292.04± 7.72 88.55±16.1888.55± 16.18 2.68±0.312.68± 0.31 2.54±0.652.54± 0.65 0.97±0.050.97± 0.05 0.96±0.170.96± 0.17 2.78±0.232.78± 0.23 2.62±0.492.62± 0.49 T0 F0 PPO-Lag 117.27±49.88117.27± 49.88 128.72±75.68128.72± 75.68 28.77±15.65 28.77± 15.65 23.61±34.47 23.61± 34.47 0.15±0.63 0.15± 0.63 −0.06±1.38-0.06± 1.38 0.39±0.110.39± 0.11 0.31±0.400.31± 0.40 1.35±0.691.35± 0.69 1.89±2.141.89± 2.14 T0 F2 P3O 152.18±40.39152.18± 40.39 157.25±53.58157.25± 53.58 47.50±9.0447.50± 9.04 35.41±23.0635.41± 23.06 0.90±0.360.90± 0.36 0.42±0.920.42± 0.92 0.74±0.090.74± 0.09 0.55±0.400.55± 0.40 1.32±0.341.32± 0.34 1.13±1.25 1.13± 1.25 T0 F0 FOCOPS 174.84±10.24 174.84± 10.24 198.84±35.64 198.84± 35.64 31.87±1.3731.87± 1.37 19.19±30.2019.19± 30.20 0.27±0.050.27± 0.05 −0.23±1.21 -0.23± 1.21 0.37±0.07 0.37± 0.07 0.22±0.40 0.22± 0.40 1.22±0.38 1.22± 0.38 1.44±1.451.44± 1.45 T0 F2 d=50d=50 PPO 234.20±16.73234.20± 16.73 228.97±55.88228.97± 55.88 91.60±8.9091.60± 8.90 88.75±17.9588.75± 17.95 0.83±0.180.83± 0.18 0.78±0.360.78± 0.36 0.96±0.080.96± 0.08 0.94±0.160.94± 0.16 0.89±0.100.89± 0.10 0.86±0.170.86± 0.17 T0 F0 PPO-Lag 159.31±35.70159.31± 35.70 172.76±69.93172.76± 69.93 48.52±5.2048.52± 5.20 48.39±27.4248.39± 27.42 −0.03±0.10 -0.03± 0.10 −0.03±0.55-0.03± 0.55 0.48±0.050.48± 0.05 0.53±0.460.53± 0.46 0.53±0.33 0.53± 0.33 0.47±0.54 0.47± 0.54 T2 F1 P3O 190.05±29.27190.05± 29.27 166.43±70.37166.43± 70.37 66.74±10.6566.74± 10.65 56.41±31.5456.41± 31.54 0.33±0.210.33± 0.21 0.13±0.630.13± 0.63 0.74±0.120.74± 0.12 0.52±0.430.52± 0.43 0.55±0.150.55± 0.15 0.64±0.590.64± 0.59 T0 F0 FOCOPS 196.17±6.95 196.17± 6.95 198.88±64.22 198.88± 64.22 57.18±1.81 57.18± 1.81 38.57±39.78 38.57± 39.78 0.14±0.040.14± 0.04 −0.23±0.80 -0.23± 0.80 0.47±0.04 0.47± 0.04 0.33±0.47 0.33± 0.47 0.80±0.080.80± 0.08 0.87±0.410.87± 0.41 T0 F2 Figure A.9: Episodic cost distributions for final policies with exploration noise on and off (greedy). Safe Push Point (Level 1). Averaged over 30 seeds. Error bars denote 1σ1σ over 30 seeds. Table A.6: Safety and performance evaluation per metric for Safe Push Point (Level 1), for each individual d∈15,25,50d∈\15,25,50\. The numbers indicate mean ±1σ± 1σ over 30 seeds. Bold text indicates best result in that specific column, excluding PPO, as this is not a safe RL algorithm. Algorithm R¯ R C¯ C DnormD_norm V Dnorm+D_norm^+ Tier Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) d=15d=15 PPO 57.07±9.9957.07± 9.99 64.79±10.4164.79± 10.41 85.43±15.0285.43± 15.02 77.26±15.0577.26± 15.05 4.70±1.004.70± 1.00 4.15±1.004.15± 1.00 1.00±0.001.00± 0.00 0.99±0.010.99± 0.01 4.70±1.004.70± 1.00 4.19±1.004.19± 1.00 T0 F0 PPO-Lag 27.13±15.8227.13± 15.82 33.84±22.9833.84± 22.98 15.23±0.15 15.23± 0.15 10.69±3.45 10.69± 3.45 0.02±0.01 0.02± 0.01 −0.29±0.23 -0.29± 0.23 0.44±0.040.44± 0.04 0.25±0.12 0.25± 0.12 0.15±0.07 0.15± 0.07 1.00±0.831.00± 0.83 T0 F2 P3O 38.51±17.1538.51± 17.15 52.49±18.4952.49± 18.49 20.44±1.1620.44± 1.16 14.93±3.4414.93± 3.44 0.36±0.080.36± 0.08 −0.00±0.23-0.00± 0.23 0.71±0.080.71± 0.08 0.37±0.090.37± 0.09 0.54±0.110.54± 0.11 1.02±0.641.02± 0.64 T0 F2 FOCOPS 56.53±11.59 56.53± 11.59 63.78±22.21 63.78± 22.21 16.18±0.2716.18± 0.27 12.52±4.8712.52± 4.87 0.08±0.020.08± 0.02 −0.17±0.32-0.17± 0.32 0.42±0.05 0.42± 0.05 0.30±0.150.30± 0.15 0.37±0.040.37± 0.04 0.78±0.88 0.78± 0.88 T0 F2 d=25d=25 PPO 57.13±9.6057.13± 9.60 64.15±10.4064.15± 10.40 85.11±14.4185.11± 14.41 77.52±14.2877.52± 14.28 2.40±0.582.40± 0.58 2.10±0.572.10± 0.57 1.00±0.001.00± 0.00 0.98±0.030.98± 0.03 2.40±0.582.40± 0.58 2.15±0.562.15± 0.56 T0 F0 PPO-Lag 33.45±17.3633.45± 17.36 42.98±22.8042.98± 22.80 25.18±0.29 25.18± 0.29 20.49±4.85 20.49± 4.85 0.01±0.01 0.01± 0.01 −0.18±0.19 -0.18± 0.19 0.45±0.040.45± 0.04 0.31±0.11 0.31± 0.11 0.12±0.10 0.12± 0.10 0.69±0.430.69± 0.43 T0 F2 P3O 39.50±17.3039.50± 17.30 50.25±18.5650.25± 18.56 29.66±2.1029.66± 2.10 23.55±4.5123.55± 4.51 0.19±0.080.19± 0.08 −0.06±0.18-0.06± 0.18 0.64±0.120.64± 0.12 0.41±0.100.41± 0.10 0.34±0.150.34± 0.15 0.57±0.120.57± 0.12 T0 F2 FOCOPS 63.27±12.00 63.27± 12.00 76.30±12.88 76.30± 12.88 25.49±0.3425.49± 0.34 21.43±6.0621.43± 6.06 0.02±0.010.02± 0.01 −0.14±0.24-0.14± 0.24 0.39±0.03 0.39± 0.03 0.33±0.130.33± 0.13 0.23±0.040.23± 0.04 0.47±0.42 0.47± 0.42 T0 F2 d=50d=50 PPO 55.57±10.0155.57± 10.01 63.36±10.1563.36± 10.15 86.52±16.0486.52± 16.04 77.98±16.4977.98± 16.49 0.73±0.320.73± 0.32 0.56±0.330.56± 0.33 0.99±0.050.99± 0.05 0.80±0.150.80± 0.15 0.73±0.320.73± 0.32 0.73±0.260.73± 0.26 T0 F0 PPO-Lag 42.73±15.0642.73± 15.06 51.73±17.1751.73± 17.17 48.78±3.17 48.78± 3.17 46.31±5.81 46.31± 5.81 −0.02±0.06 -0.02± 0.06 −0.07±0.12-0.07± 0.12 0.39±0.170.39± 0.17 0.39±0.090.39± 0.09 0.10±0.080.10± 0.08 0.38±0.090.38± 0.09 T2 F2 P3O 43.69±14.5243.69± 14.52 52.02±15.0152.02± 15.01 51.35±2.4051.35± 2.40 49.26±3.0349.26± 3.03 0.03±0.050.03± 0.05 −0.01±0.06-0.01± 0.06 0.48±0.160.48± 0.16 0.43±0.060.43± 0.06 0.17±0.140.17± 0.14 0.43±0.120.43± 0.12 T0 F2 FOCOPS 71.08±8.84 71.08± 8.84 81.92±7.79 81.92± 7.79 48.83±1.0848.83± 1.08 44.14±4.8644.14± 4.86 −0.02±0.02-0.02± 0.02 −0.12±0.10 -0.12± 0.10 0.33±0.09 0.33± 0.09 0.35±0.10 0.35± 0.10 0.09±0.01 0.09± 0.01 0.31±0.06 0.31± 0.06 T2 F2 Figure A.10: Episodic cost distributions for final policies with exploration noise on and off (greedy). Safe Button Point (Level 1). Averaged over 30 seeds. Error bars denote 1σ1σ over 30 seeds. Table A.7: Safety and performance evaluation per metric for Safe Button Point (Level 1), for each individual d∈15,25,50d∈\15,25,50\. The numbers indicate mean ±1σ± 1σ over 30 seeds. Bold text indicates best result in that specific column, excluding PPO, as this is not a safe RL algorithm. Algorithm R¯ R C¯ C DnormD_norm V Dnorm+D_norm^+ Tier Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) Train Final (greedy) d=15d=15 PPO 158.70±8.70158.70± 8.70 2.47±0.022.47± 0.02 351.56±30.21351.56± 30.21 10.77±1.8810.77± 1.88 22.44±2.0122.44± 2.01 −0.28±0.13-0.28± 0.13 1.00±0.001.00± 0.00 0.24±0.060.24± 0.06 22.44±2.0122.44± 2.01 0.86±0.260.86± 0.26 T0 F2 PPO-Lag 121.04±13.19121.04± 13.19 2.44±0.112.44± 0.11 16.02±1.27 16.02± 1.27 11.66±4.8211.66± 4.82 0.07±0.08 0.07± 0.08 −0.22±0.32-0.22± 0.32 0.47±0.11 0.47± 0.11 0.29±0.150.29± 0.15 0.34±0.11 0.34± 0.11 0.69±0.190.69± 0.19 T0 F2 P3O 119.25±9.73119.25± 9.73 2.47±0.02 2.47± 0.02 44.49±5.5344.49± 5.53 9.63±2.17 9.63± 2.17 1.97±0.371.97± 0.37 −0.36±0.14 -0.36± 0.14 1.00±0.001.00± 0.00 0.23±0.08 0.23± 0.08 1.97±0.371.97± 0.37 0.68±0.200.68± 0.20 T0 F2 FOCOPS 126.48±7.90 126.48± 7.90 2.46±0.052.46± 0.05 20.51±0.5620.51± 0.56 10.58±2.3510.58± 2.35 0.37±0.040.37± 0.04 −0.29±0.16-0.29± 0.16 0.80±0.060.80± 0.06 0.25±0.070.25± 0.07 0.48±0.020.48± 0.02 0.63±0.27 0.63± 0.27 T0 F2 d=25d=25 PPO 159.05±6.10159.05± 6.10 2.47±0.022.47± 0.02 349.05±36.43349.05± 36.43 10.55±2.0810.55± 2.08 12.96±1.4612.96± 1.46 −0.58±0.08-0.58± 0.08 1.00±0.001.00± 0.00 0.11±0.040.11± 0.04 12.96±1.4612.96± 1.46 0.58±0.220.58± 0.22 T0 F2 PPO-Lag −0.17±9.63-0.17± 9.63 −8.84±41.09-8.84± 41.09 2.33±0.46 2.33± 0.46 2.11±1.45 2.11± 1.45 −0.91±0.02 -0.91± 0.02 −0.92±0.06 -0.92± 0.06 0.00±0.00 0.00± 0.00 0.01±0.01 0.01± 0.01 0.77±0.320.77± 0.32 0.33±0.27 0.33± 0.27 T2 F2 P3O 116.64±10.90116.64± 10.90 2.47±0.02 2.47± 0.02 54.62±5.2354.62± 5.23 10.42±2.3110.42± 2.31 1.18±0.211.18± 0.21 −0.58±0.09-0.58± 0.09 0.99±0.010.99± 0.01 0.10±0.040.10± 0.04 1.20±0.211.20± 0.21 0.46±0.180.46± 0.18 T0 F2 FOCOPS 149.51±7.69 149.51± 7.69 2.46±0.022.46± 0.02 26.24±0.3526.24± 0.35 10.81±2.2510.81± 2.25 0.05±0.010.05± 0.01 −0.57±0.09-0.57± 0.09 0.41±0.050.41± 0.05 0.09±0.050.09± 0.05 0.28±0.05 0.28± 0.05 0.40±0.180.40± 0.18 T0 F2 d=50d=50 PPO 158.01±5.82158.01± 5.82 2.47±0.022.47± 0.02 353.01±26.43353.01± 26.43 11.06±2.1511.06± 2.15 6.06±0.536.06± 0.53 −0.78±0.04-0.78± 0.04 1.00±0.001.00± 0.00 0.03±0.020.03± 0.02 6.06±0.536.06± 0.53 0.20±0.130.20± 0.13 T0 F0 PPO-Lag 148.02±12.79148.02± 12.79 2.46±0.022.46± 0.02 47.39±1.97 47.39± 1.97 13.82±5.1113.82± 5.11 −0.05±0.04 -0.05± 0.04 −0.72±0.10-0.72± 0.10 0.30±0.11 0.30± 0.11 0.03±0.040.03± 0.04 0.14±0.080.14± 0.08 0.20±0.160.20± 0.16 T2 F2 P3O 122.83±10.21122.83± 10.21 2.47±0.02 2.47± 0.02 75.26±4.4175.26± 4.41 9.91±2.21 9.91± 2.21 0.51±0.090.51± 0.09 −0.80±0.04 -0.80± 0.04 0.93±0.030.93± 0.03 0.01±0.01 0.01± 0.01 0.56±0.090.56± 0.09 0.19±0.19 0.19± 0.19 T0 F2 FOCOPS 174.33±6.08 174.33± 6.08 2.45±0.072.45± 0.07 50.02±0.1250.02± 0.12 13.23±9.5513.23± 9.55 0.00±0.000.00± 0.00 −0.74±0.19-0.74± 0.19 0.41±0.030.41± 0.03 0.03±0.070.03± 0.07 0.08±0.01 0.08± 0.01 0.24±0.350.24± 0.35 T2 F2