Paper deep dive
How Much Evidence Should a Coding Agent's Self-Correction Carry? Adaptive Dirichlet Evidence for Self-Distillation
Yunbo Long, Guangya Hao, Yuhan Liu, Yiting Duan, Longyan Tan, Yunchen Long, Hao Wu
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Execution feedback lets coding agents revise programs and learn from their own corrections. A correction's learning weight should reflect both the transitions supported by its executions and the amount of evidence behind that support. We introduce Effective-Evidence Self-Distillation (EESD), which represents these quantities separately. Normalized execution relevance determines relative transition support and an effective pseudo-count mass; a Dirichlet posterior then produces an uncertainty-penalized weight for KL-anchored correction learning. Under a symmetric prior, changing mass preserves category ordering, and effective mass yields a supervised coefficient bounded by its matched fixed-mass counterpart. Across four model-domain history sweeps, increasing visible observations from one to eight reduces future-outcome NLL by 55.0-59.3%. At eight observations, effective mass achieves lower NLL than fixed mass in all four comparisons. In the primary matched DeepSeek/RunBugRun study, argmax predictions agree on all 3,000 examples, with the largest NLL gain under concentrated relevance. After one correction-learning round, DeepSeek/CodeARC all-tests Pass@1 increases from 15.0% to 20.4%, with a paired 95% source-bootstrap interval of [+2.8, +8.0] percentage points. The twelve-setting downstream evaluation establishes the model-domain scope of this update. These results show how separating evidence support from evidence mass changes probability estimation and correction learning in coding agents.
Tags
Links
- Source: https://arxiv.org/abs/2610.08514v1
- Canonical: https://arxiv.org/abs/2610.08514v1
Trouble viewing inline? Open PDF directly →
Full Text
49,261 characters extracted from source content.
Expand or collapse full text
How Much Evidence Should a Coding Agent’s Self-Correction Carry? Adaptive Dirichlet Evidence for Self-Distillation Yunbo Long, Guangya Hao, Yuhan Liu, Yiting Duan Affiliation: University of Cambridge Western Sydney University Longyan Tan, Yunchen Long, Hao Wu Affiliation: University of Cambridge Western Sydney University Affiliation: Guangdong University of Technology Abstract Execution feedback lets coding agents revise programs and learn from their own corrections. A correction’s learning weight should reflect both the transitions supported by its executions and the amount of evidence behind that support. We introduce Effective-Evidence Self-Distillation (EESD), which represents these quantities separately. Normalized execution relevance determines relative transition support and an effective pseudo-count mass; a Dirichlet posterior then produces an uncertainty-penalized weight for KL-anchored correction learning. Under a symmetric prior, changing mass preserves category ordering, and effective mass yields a supervised coefficient bounded by its matched fixed-mass counterpart. Across four model–domain history sweeps, increasing visible observations from one to eight reduces future-outcome NLL by 55.055.0–59.3%59.3\%. At eight observations, effective mass achieves lower NLL than fixed mass in all four comparisons. In the primary matched DeepSeek/RunBugRun study, argmax predictions agree on all 3,000 examples, with the largest NLL gain under concentrated relevance. After one correction-learning round, DeepSeek/CodeARC all-tests Pass@1 increases from 15.0%15.0\% to 20.4%20.4\%, with a paired 95% source-bootstrap interval of [+2.8,+8.0][+2.8,+8.0] percentage points. The twelve-setting downstream evaluation establishes the model–domain scope of this update. These results show how separating evidence support from evidence mass changes probability estimation and correction learning in coding agents. 1 Introduction Coding agents use language models to generate and revise programs from execution feedback, including compiler errors and unit-test outcomes. This supports self-correction in code generation and program repair. Self-Refine and Reflexion reuse model-generated feedback or reflection at inference time (Madaan et al., 2023; Shinn et al., 2023); Self-Debugging revises programs after observing execution behavior (Chen et al., 2024); and execution-guided repair, search, and verification systems use tests to select or refine candidate programs (Ni et al., 2023; Tang et al., 2024; Li et al., 2025b; Li et al., 2025a; Tao et al., 2026). These methods use feedback to improve the next candidate program. Self-training also requires a decision about how strongly to learn from each trajectory. Correctness filtering, reward weighting, and confidence-aware training assess the quality of generated behavior (Jiang et al., 2025; Jang et al., 2025; Yang et al., 2025). Execution evidence supplies two further quantities: which before/after transitions the observations support and how much evidence supports those transitions. Representing both quantities makes the correction weight sensitive to evidence concentration even when relative support stays fixed. Figure 1: Direction and strength in a coding agent’s self-correction. (a) Fixing an off-by-one range error changes four public executions from fail to pass. (b) Aggregating execution relevance by transition gives r=(1,0,0,0)r=(1,0,0,0): all support favors Fix. (c) The same transitions under uniform and concentrated relevance give E(p)=4.000E(p)=4.000 and 1.0621.062; at fixed prior and utility settings, they yield different supervised weights. The p bars index executions, whereas the r bars index transition categories. The relevance vectors define an analytical example at fixed transition support. Figure 1 illustrates the distinction through an off-by-one repair. Extending a Python range endpoint makes all four public executions pass, so relative support is r=(1,0,0,0)r=(1,0,0,0) under either relevance pattern. Uniform relevance assigns comparable weight to every execution and gives E(p)=4E(p)=4; concentrated relevance p=(0.97,0.01,0.01,0.01)p=(0.97,0.01,0.01,0.01) gives E(p)≈1.062E(p)≈ 1.062. A fixed-mass control restores total mass to four in both cases. EESD instead lets relevance concentration change the supervised weight while retaining the same correction target. The analytical relevance vectors isolate the effect of concentration at fixed transition support. The problem becomes more consequential when corrections are written back into model parameters. A mistaken inference-time correction affects one answer; an over-weighted training correction changes the policy that will generate future answers and future training data, a feedback pattern relevant to recursive self-improvement (RSI) and self-evolving agents (Yin et al., 2025; Ranaldi, 2026; Xiao et al., 2026; Li et al., 2026). We study the correction-to-update step within this feedback process: given a self-generated correction, how much should the next policy learn from its supporting execution evidence? We introduce Effective-Evidence Self-Distillation (EESD), a posterior-based rule for weighting correction learning. Given nonnegative relevance scores sis_i, EESD normalizes them into p. Relative support r describes what transition categories the observations favor, while E(p)=(∑ipi2)−1E(p)=( _ip_i^2)^-1 describes how much pseudo-count mass that support contributes. A Dirichlet posterior produces an uncertainty-penalized correction utility; its positive part scales correction cross-entropy while a KL term anchors the student to the entering policy. The upstream correction generator and relevance estimator remain modular: the contribution is how their execution record is converted into a learning coefficient. Our contributions are: 1. We formulate correction-level direction–strength separation: distinguishing relative execution support from the total evidence mass assigned to that support. 2. We introduce an effective-evidence Dirichlet estimator (EED), which converts relevance concentration into adaptive Dirichlet pseudo-count mass, and connect its posterior utility to KL-anchored LoRA self-distillation. We characterize count recovery, ordering invariance, and the resulting control of the supervised coefficient. 3. We evaluate the construction through visible-history scaling, a matched-mass mechanism study with fixed argmax, and one-round correction-based self-training followed by fresh code generation across multiple models and domains. 2 Related Work Table 1 positions EESD relative to representative self-correction and self-training approaches. The central distinction is between improving an answer, assessing its suitability for learning, and determining how much execution evidence supports the resulting update. Table 1: Positioning by the role of feedback. ✓ : explicit use; –: absent in the cited formulation; ∘ : varies within a grouped family. Generator updates refer to the code-generating policy. Adaptive training includes filtering, preferences, and reward weighting. The final three columns describe correction-level support–mass construction. Representative methods are discussed in Section 2. Method Inference revision / selection Generator update Execution feedback Adaptive training Transition support r Effective mass E(p)E(p) Support / mass separation Self-Refine ✓ – – – – – – Reflexion (coding) ✓ – ✓ – – – – Self-Debugging ✓ – ✓ – – – – Execution-guided verification / reranking ✓ – ✓ – – – – Importance-weighted self-training – ✓ – ✓ – – – Confidence-aware self-training – ✓ – ✓ – – – Reward / process-weighted learning ∘ ∘ ✓ ∘ – – – EESD (ours) – ✓ ✓ ✓ ✓ ✓ ✓ Self-correction and execution-guided repair. Self-Refine iteratively revises outputs using model-generated critiques, and Reflexion reuses verbal feedback to guide later attempts (Madaan et al., 2023; Shinn et al., 2023). Reflexion’s coding setting and Self-Debugging use execution feedback to revise programs (Shinn et al., 2023; Chen et al., 2024). Execution-guided verification and reranking evaluate candidate programs with test outcomes (Ni et al., 2023; Li et al., 2025a). REx uses Bayesian beliefs to allocate repair-search effort, while ExecCritic couples testing with repair-agent learning (Tang et al., 2024; Tao et al., 2026). LEVER trains an auxiliary verifier for candidate selection (Ni et al., 2023). EESD operates on the correction record produced by such upstream procedures and determines its contribution to a generator update. Learning from self-generated corrections. Correctness filtering, importance weighting, and confidence-aware self-training select or weight generated trajectories using outcome and model signals (Jiang et al., 2025; Jang et al., 2025). CodePRM learns a process evaluator from execution feedback, and iterative preference learning updates models from self-generated data (Li et al., 2025b; Yang et al., 2025). Self-evolving systems also reuse experience across updates or skill construction (Yin et al., 2025; Ranaldi, 2026; Xiao et al., 2026; Li et al., 2026). EESD adds a correction-level evidence representation: relative transition support and the pseudo-count mass assigned to that support jointly determine the training weight. Evidential uncertainty and effective sample size. Dirichlet models separate categorical probabilities from distributional concentration and support evidential uncertainty modeling (Sensoy et al., 2018). The inverse squared concentration of normalized weights is an established effective-sample-size functional (Martino et al., 2017). EESD combines these tools at the level of an execution record. Relative transition support r allocates pseudo-counts across categories, effective mass E(p)E(p) sets their total, and posterior utility sets the supervised coefficient in KL-anchored self-distillation. This execution-to-learning construction links evidence concentration directly to the strength of a correction update. 3 Effective-Evidence Self-Distillation EESD converts a coding agent’s execution feedback into a correction-specific training weight. The key is to separate relative transition support from total evidence mass, then use both to determine how strongly to learn from a correction. Figure 2: Effective-Evidence Self-Distillation. The public correction record yields p and transition labels ziz_i. Parallel support and mass summaries form a pseudo-count posterior and weight w. Teacher and student consume the same prompt and past correction tokens; the current target aj1a^1_j enters CE only. Teacher probabilities supply the forward-KL reference. The weight w scales CE. The dashed coral arrow updates LoRA parameters only. Code and decoder blocks schematically represent the correction record and frozen model bases. 3.1 Correction evidence For a programming task x, a coding agent with policy πt _t produces an initial program a0a^0, and an upstream repair procedure returns a correction a1a^1. Both programs are evaluated on the same n≥1n≥ 1 public executions qii=1n\q_i\_i=1^n, yielding before/after outcomes (oi0,oi1)(o_i^0,o_i^1). Let h denote this correction record. Each outcome pair defines a transition zi∈Fix,Regress,Preserve,Unresolved,z_i∈\ Fix, Regress, Preserve, Unresolved\, (1) corresponding to fail→ , pass→ , pass→ , and fail→ , respectively. Each execution receives a relevance score si≥0s_i≥ 0, with ∑isi>0 _is_i>0. Define pi p_i =si∑jsj, = s_i _js_j, (2) rk r_k =∑ipi[zi=k]. = _ip_iI[z_i=k]. (3) The four-vector r describes which transitions the observations support. Direction denotes the ordering of transition categories. Our implementation obtains sis_i from a frozen lexical kernel comparing the code change with public execution descriptors; Appendix B gives the details. 3.2 From effective evidence to correction utility Our fixed-mass control uses ckfixed=nrkc_k^fixed=nr_k, preserving relative support while assigning every record of length n the same total mass. EED replaces this nominal mass with E(p) E(p) =(∑ipi2)−1, = ( _ip_i^2 )^-1, (4) αkpost _k^post =α+E(p)rk,α>0, =α+E(p)r_k, α>0, (5) and defines θ|h∼Dir(post)θ h ( α^post). Thus r allocates pseudo-counts across transitions, while E(p)E(p) sets their total mass. We have 1≤E(p)≤n1≤ E(p)≤ n; uniform relevance gives E(p)pi=1E(p)p_i=1 and recovers ordinary counts. The posterior is conditional on the relevance weights, and its pseudo-count mass summarizes their concentration. To reward fixes and penalize regressions, set v=(1,−1,0,0)v=(1,-1,0,0). Writing α0=∑kαkpost _0= _k _k^post and μk=αkpost/α0 _k= _k^post/ _0, the posterior utility moments are u¯ u =[v⊤θ]=μFix−μRegress, =E[v θ]= _ Fix- _ Regress, (6) σu2 _u^2 =μFix+μRegress−u¯2α0+1. = _ Fix+ _ Regress- u^2 _0+1. (7) The correction weight is w(h)=[u¯−λσu]+,λ≥0,w(h)= [ u-λ _u ]_+, λ≥ 0, (8) where [a]+=max(a,0)[a]_+= (a,0). The uncertainty penalty reduces the supervised contribution of corrections with less decisive posterior utility; nonpositive utility yields zero supervised weight. The score combines expected utility with a posterior standard-deviation penalty. For the all-Fix example in Figure 1, α=0.1α=0.1 and λ=0.5λ=0.5 give weights 0.8320.832 and 0.5420.542 for uniform and concentrated relevance. The matched fixed-mass control assigns 0.8320.832 to both. Appendix G.1 gives the calculation. The example holds correction targets and KL coefficients fixed. 3.3 KL-anchored self-distillation The student learns from a1a^1 while a frozen copy of πt _t supplies a forward-KL anchor: ℒEESD(h)=w(h)ℒCE(h)+βDKL(h)(πt∥πt+1).L_ EESD(h)=w(h)L_CE(h)+β D_KL^(h)( _t\| _t+1). (9) Here ℒCEL_CE is correction cross-entropy, and DKL(h)D_KL^(h) is forward KL averaged over the same response-token prefixes. Only the supervised term is weighted by w(h)w(h). We train LoRA parameters with the quantized base frozen; token-level definitions and fixed hyperparameters are in Appendix B. Proposition 1 (Outcome ordering and correction weight). Fix r, a symmetric prior α>0α>0, λ≥0λ≥ 0, and v=(1,−1,0,0)v=(1,-1,0,0). Replace E(p)E(p) by any mass τ>0τ>0 in Equations 5–8. The posterior mean satisfies μk(τ)=γ(τ)rk+1−γ(τ)4,γ(τ)=τ4α+τ. _k(τ)=γ(τ)r_k+ 1-γ(τ)4, γ(τ)= τ4α+τ. (10) Consequently, category ordering and the argmax set are invariant to τ. Moreover, the correction weight wτw_τ is nondecreasing in τ, so wE(p)≤wn.w_E(p)≤ w_n. (11) At fixed support, effective mass attenuates the supervised coefficient relative to nominal mass while preserving category ordering. Its effect on log loss depends on the observed category. Appendix A gives the proofs and the outcome-dependent log-loss condition. 4 Experiments We evaluate three questions. RQ1 examines whether additional visible executions improve outcome prediction. RQ2 isolates how effective mass changes predictive probabilities at fixed relative support. RQ3 tests the complete correction-learning update through fresh program generation. Figures 3 and 4 organize these measurements, with full numerical reports in the appendix. 4.1 Experimental setup Tasks and models. We use RunBugRun and CodeARC, together with APPS Replay and CodeContests Replay (Prenner & Robbes, 2023; Wei et al., 2025). RunBugRun contains executable buggy programs; CodeARC tests program synthesis from behavioral observations. The replay domains are controlled synthesis extensions evaluated on the recorded source populations and execution budgets. CodeARC uses the fixed-source execution protocol specified in Appendix B; the original benchmark uses interactive differential testing (Wei et al., 2025). The downstream matrix covers Qwen2.5-Coder-7B-Instruct, DeepSeek-Coder-6.7B-Instruct, and Gemma-3-4B-it, with 500 primary sources per model–domain cell. Measurements and comparisons. The prediction studies measure future-outcome negative log-likelihood (NLL) and categorical accuracy. The history sweep compares fixed mass n with effective mass E(p)E(p) as visible history grows. A separate matched comparison holds relative support and predictor parameters fixed, changing only the pseudo-count mass. Downstream evaluation generates one fresh candidate per source after one update; all-tests Pass@1 requires passing all ten evaluator tests. It compares full EESD (eesd_full) with the unchanged entering policy (no_update). The paired effect, in percentage points, is Δp=100[S(eesd_full)−S(no_update)], _p=100 [S( eesd\_full)-S( no\_update) ], (12) where S is the fraction of solved sources. Paired 95% intervals use 10,000 whole-source bootstrap draws and quantify source-level uncertainty conditional on the evaluated policies. Evidence scoring uses public executions; hidden outcomes are reserved for evaluation. Fresh generation produces a new completion for each recorded source. Protocol and replication details are in Appendix B. Training and hardware. We adapt LoRA parameters on a frozen 4-bit NF4 base with bfloat16 computation. The correction scorer uses α=0.1α=0.1 and λ=0.5λ=0.5; the forward-KL coefficient is β=0.03β=0.03. Experiments use NVIDIA RTX A6000 hardware with 48 GB nominal memory per GPU. Evidence weights are computed from public records before student fitting, and the frozen entering policy supplies the KL reference. Appendix B specifies the relevance kernel, response-token objective, and optimization settings. 4.2 RQ1: Does additional visible evidence improve prediction? We first ask whether the available executions carry useful predictive information. Figure 3 varies visible history size over n∈1,2,4,8n∈\1,2,4,8\ for DeepSeek and Qwen on RunBugRun and CodeARC. Here n is the number of visible observations. The sweep measures the predictive value of a longer execution history. Prediction improves with longer histories. EED NLL decreases at every measured step in all four model–domain cells. On RunBugRun, the endpoint changes are 0.4696→0.21140.4696→ 0.2114 for DeepSeek and 0.4737→0.21130.4737→ 0.2113 for Qwen. On CodeARC, they are 0.7103→0.28920.7103→ 0.2892 and 0.7785→0.34150.7785→ 0.3415. Table 2 expresses these changes as relative NLL reductions of 55.055.0–59.3%59.3\%. Categorical accuracy improves as well: by 2.52.5 and 1.91.9 percentage points on RunBugRun, and by 5.55.5 and 6.16.1 points on CodeARC, respectively. Additional history therefore benefits both the probability assigned to the observed outcome and the categorical prediction. RQ3 evaluates the resulting policy update. Effective mass adds a distinct comparison. Fixed-mass prediction also benefits from additional observations. At n=8n=8, EED further reduces NLL relative to Fixed in all four cells, as summarized in Figure 4(a): 0.00460.0046 and 0.00750.0075 on RunBugRun, and 0.01130.0113 and 0.00240.0024 on CodeARC, for DeepSeek and Qwen respectively. All four values are absolute NLL differences. At n=1n=1, the two constructions coincide because p1=1p_1=1 and E(p)=n=1E(p)=n=1; differences at shorter histories are generally small. Figure 3 and Appendix E retain the complete trajectories. The history sweep varies the available observations; RQ2 holds relative support fixed to isolate the mass intervention. Table 2: Endpoint summary of the EED history sweep. NLL reduction and accuracy change compare n=1n=1 with n=8n=8; ΔNLL8=NLLFixed−NLLEED _8=NLL_Fixed-NLL_EED at n=8n=8. The reductions are calculated from the reported rounded measurements. Benchmark Backbone EED NLL →81\!→\!8 Reduction (%) Acc. change (p) ΔNLL8 _8 ↑ RunBugRun DeepSeek 6.7B →0.21140.4696\!→\!0.2114 55.0 +2.5 +0.0046 RunBugRun Qwen2.5 7B →0.21130.4737\!→\!0.2113 55.4 +1.9 +0.0075 CodeARC DeepSeek 6.7B →0.28920.7103\!→\!0.2892 59.3 +5.5 +0.0113 CodeARC Qwen2.5 7B →0.34150.7785\!→\!0.3415 56.1 +6.1 +0.0024 Figure 3: History size and effective evidence answer different questions. (a,b) EED future-outcome NLL: lower is better. (c,d) ΔNLL=NLLFixed−NLLEED =NLL_Fixed-NLL_EED: higher is better, in units of 10−310^-3. All panels use n=1,2,4,8n=1,2,4,8 visible observations. Differences are calculated from the four-decimal measurements in Table 8. The matched-parameter comparison is reported separately in Section 4.3. 4.3 RQ2: Does effective evidence change strength without changing direction? The primary matched comparison uses DeepSeek/RunBugRun with the same relative support and effective-arm hyperparameters for both predictors. Only total pseudo-count mass changes. Their categorical argmax predictions agree on all 3,000 future-outcome examples, while effective mass reduces aggregate NLL by 0.006090.00609 with a 95% source-bootstrap interval of [0.00211,0.01075][0.00211,0.01075]. Thus better probability estimates do not require changing the selected category. The ordering argument in Proposition 1 applies to any categorical predictor with a symmetric prior, as detailed in Appendix A.4; the four-transition utility is the separate training construction. The largest gain occurs under concentrated relevance. Table 3 and Figure 4(b) divide the matched comparison into three effective-mass bins. The NLL reduction is 0.01590.0159 for 1≤E(p)<21≤ E(p)<2, 0.00770.0077 for 2≤E(p)<32≤ E(p)<3, and 0.00180.0018 for 3≤E(p)≤43≤ E(p)≤ 4. The corresponding mean masses retain 37.8%37.8\%, 63.0%63.0\%, and 92.0%92.0\% of the nominal count of four (Appendix G). In this comparison, the largest predictive benefit is associated with the largest reduction in pseudo-count mass. When support is already broadly distributed, effective and nominal mass are closer and the observed difference is smaller. Changing mass adjusts shrinkage toward the symmetric prior at fixed observations and relative support. The favored category remains fixed while its assigned probability changes. The reported mass fractions quantify this pseudo-count intervention. Table 3: Concentration-stratified matched-mass comparison. Primary DeepSeek/RunBugRun comparison. Gain is fixed-mass NLL minus EED NLL. Both predictors agree in argmax on all 3,000 examples. Effective-mass bin Examples Mean E(p)E(p) NLL gain ↑ [1,2)[1,2) 568 1.511 +0.0159 [2,3)[2,3) 815 2.521 +0.0077 [3,4][3,4] 1617 3.681 +0.0018 From probability estimates to learning. The matched study isolates the evidence estimator. For training, the posterior is further mapped to a correction-specific utility and supervised coefficient. Proposition 1 characterizes this mapping; RQ3 examines the complete resulting update. The other recorded mechanism repetitions and their intervals are provided in Appendix F, separately from the history sweep. The log-loss analysis in Appendix A.4 explains why a change in mass need not have the same benefit for every outcome. 4.4 RQ3: Does correction learning improve fresh generation? We now move from outcome prediction to the coding agent’s code-generating policy. Each evaluated program is freshly sampled after learning. Figure 4(c) presents four selected model–domain contrasts against the unchanged policy; Appendix D reports the complete twelve-cell evaluation. Figure 4: Prediction and fresh-generation performance. (a) Fixed-minus-EED NLL at n=8n=8 in the four history-sweep cells. (b) The matched DeepSeek/RunBugRun comparison by effective-mass bin; argmax agrees on all 3,000 examples. Both NLL panels use units of 10−310^-3 and positive values favor EED. (c) Four selected one-round contrasts against no_update, with paired 95% source-bootstrap intervals and solved counts out of 500. The complete model–domain evaluation is in Table 7. The clearest gain is on DeepSeek/CodeARC. All-tests Pass@1 rises from 75/50075/500 (15.0%15.0\%) to 102/500102/500 (20.4%20.4\%), a net increase of 27 solved sources. The paired improvement is +5.4+5.4 percentage points, with a 95% source-bootstrap interval of [+2.8,+8.0][+2.8,+8.0]. Each solved program passes all ten evaluator tests. The comparison measures the effect of the complete correction-learning update. Further gains appear on RunBugRun and CodeContests Replay. DeepSeek increases from 184/500184/500 to 198/500198/500 on RunBugRun (+2.8+2.8 points), and Qwen increases from 228/500228/500 to 240/500240/500 (+2.4+2.4 points). DeepSeek also increases from 18/50018/500 to 24/50024/500 on CodeContests Replay (+1.2+1.2 points). These correspond to 14, 12, and 6 additional solved sources in the respective aggregate counts. The paired intervals for these three contrasts span zero. Across all twelve settings, changes range from −6.8-6.8 to +5.4+5.4 points (Table 7), placing the DeepSeek/CodeARC gain within a model–domain-dependent response. Together, the studies examine successive parts of the evidence-to-learning path: additional observations improve prediction, matched mass changes probability estimates without changing category ordering, and the complete weighted update yields positive fresh-generation point estimates in several model–domain settings. The downstream comparison evaluates the complete update; the matched-mass comparison isolates the probability estimator. 5 Limitations Our evaluation covers executable coding tasks and one-round adaptation with a frozen lexical relevance kernel. Effective mass measures normalized relevance concentration; accounting for duplicated or dependent executions requires an additional dependence model. The uncertainty-penalized utility is a training score with no asserted confidence-coverage level. Downstream results compare the full update with the entering policy, so isolating the contributions of weighting and KL anchoring requires matched training controls. The reported source-bootstrap intervals condition on one fitting run per setting. Source-disjoint transfer and repeated RSI updates are further evaluation settings beyond the present protocol. 6 Conclusion EESD represents a correction’s execution evidence through relative transition support and effective pseudo-count mass. A Dirichlet posterior maps this representation to an uncertainty-penalized supervised weight within KL-anchored self-distillation. The analysis establishes category-order invariance and the ordering of effective- and fixed-mass coefficients. Experiments show lower outcome NLL with longer histories, lower NLL than fixed mass in all four eight-observation comparisons, and a primary matched-mass gain at identical argmax predictions. One correction-learning round increases DeepSeek/CodeARC all-tests Pass@1 by 5.45.4 percentage points. The full model–domain matrix defines the scope of this result. Together, these findings support explicit evidence-mass modeling when learning from a coding agent’s self-corrections. References Chen et al. (2024) Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. In International Conference on Learning Representations, 2024. URL https://arxiv.org/abs/2304.05128. Jang et al. (2025) Hyosoon Jang, Yunhui Jang, Sungjae Lee, Jungseul Ok, and Sungsoo Ahn. Self-training large language models with confident reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2025, p. 14925–14939, 2025. doi: 10.18653/v1/2025.findings-emnlp.806. Jiang et al. (2025) Chunyang Jiang, Chi-Min Chan, Wei Xue, Qifeng Liu, and Yike Guo. Importance weighting can help large language models self-improve. Proceedings of the AAAI Conference on Artificial Intelligence, 39(23):24257–24265, 2025. doi: 10.1609/aaai.v39i23.34602. Li et al. (2025a) Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph E. Gonzalez, and Ion Stoica. S*: Test time scaling for code generation. In Findings of the Association for Computational Linguistics: EMNLP 2025, p. 15964–15978, 2025a. doi: 10.18653/v1/2025.findings-emnlp.865. Li et al. (2025b) Qingyao Li, Xinyi Dai, Xiangyang Li, Weinan Zhang, Yasheng Wang, Ruiming Tang, and Yong Yu. CodePRM: Execution feedback-enhanced process reward model for code generation. In Findings of the Association for Computational Linguistics: ACL 2025, p. 8169–8182, 2025b. doi: 10.18653/v1/2025.findings-acl.428. Li et al. (2026) Yanzhou Li, Yiran Zhang, Xiaoyu Zhang, Xiaoxia Liu, and Yang Liu. CODESKILL: Learning self-evolving skills for coding agents, 2026. URL https://arxiv.org/abs/2605.25430. Madaan et al. (2023) Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, 2023. Martino et al. (2017) Luca Martino, Víctor Elvira, and Francisco Louzada. Effective sample size for importance sampling based on discrepancy measures. Signal Processing, 131:386–401, 2017. doi: 10.1016/j.sigpro.2016.08.025. Ni et al. (2023) Ansong Ni, Srini Iyer, Dragomir Radev, Veselin Stoyanov, Wen-Tau Yih, Sida Wang, and Xi Victoria Lin. LEVER: Learning to verify language-to-code generation with execution. In Proceedings of the 40th International Conference on Machine Learning, volume 202, p. 26106–26128. PMLR, 2023. Prenner & Robbes (2023) Julian Aron Prenner and Romain Robbes. Runbugrun—an executable dataset for automated program repair, 2023. URL https://arxiv.org/abs/2304.01102. Ranaldi (2026) Leonardo Ranaldi. Evolving agents. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 16561–16569, 2026. doi: 10.18653/v1/2026.acl-long.754. Sensoy et al. (2018) Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. In Advances in Neural Information Processing Systems, 2018. Shinn et al. (2023) Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, 2023. Tang et al. (2024) Hao Tang, Keya Hu, Jin Peng Zhou, Sicheng Zhong, Wei-Long Zheng, Xujie Si, and Kevin Ellis. Code repair with llms gives an exploration-exploitation tradeoff. In Advances in Neural Information Processing Systems, 2024. doi: 10.52202/079017-3746. Tao et al. (2026) Leitian Tao, Baolin Peng, Haorui Wang, Hang Wang, Hao Cheng, Wenlin Yao, Qianhui Wu, Tao Ge, Sharon Li, and Jianfeng Gao. ExecCritic: Learn to test, test to improve for coding agents, 2026. URL https://arxiv.org/abs/2609.09133. Wei et al. (2025) Anjiang Wei, Tarun Suresh, Jiannan Cao, Naveen Kannan, Yuheng Wu, Kai Yan, Thiago S. F. X. Teixeira, Ke Wang, and Alex Aiken. CodeARC: Benchmarking reasoning capabilities of LLM agents for inductive program synthesis, 2025. URL https://arxiv.org/abs/2503.23145. Xiao et al. (2026) Chuan Xiao, Zhengbo Jiao, Shaobo Wang, Wei Wang, Bing Zhao, Hu Wei, Linfeng Zhang, and Lin Qu. Socratic-swe: Self-evolving coding agents via trace-derived agent skills, 2026. URL https://arxiv.org/abs/2606.07412. Yang et al. (2025) Haoyan Yang, Khiem Le, Ting Hua, Shangqian Gao, Binfeng Xu, Zheng Tang, Jie Xu, Nitesh V. Chawla, Hongxia Jin, and Vijay Srinivasan. Dynamic noise preference optimization: Self-improvement of large language models with self-synthetic data, 2025. URL https://arxiv.org/abs/2502.05400. Yin et al. (2025) Xunjian Yin, Xinyi Wang, Liangming Pan, Li Lin, Xiaojun Wan, and William Yang Wang. Gödel agent: A self-referential agent framework for recursively self-improvement. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 27890–27913, 2025. doi: 10.18653/v1/2025.acl-long.1354. Appendix A Proofs and Interpretation of Effective Evidence A.1 Pseudo-count construction and elementary properties Weighted-likelihood representation. Condition on the observed relevance weights p. Starting from a symmetric Dirichlet prior, the weighted likelihood L~(θ,h)=∏i=1nθziE(p)pi=∏k=14θkE(p)rk L(θ;h)= _i=1^n _z_i^E(p)p_i= _k=1^4 _k^E(p)r_k (13) gives p~(θ∣h)∝∏k=14θkα−1+E(p)rk. p(θ h) _k=1^4 _k^α-1+E(p)r_k. (14) This is the density of Dir(α+E(p)r)Dir( 1+E(p)r). This weighted likelihood provides an algebraic interpretation of the relevance-conditioned pseudo-count update. Bounds and count recovery. Since ∑ipi=1 _ip_i=1 and pi≥0p_i≥ 0, Cauchy–Schwarz yields 1=(∑ipi)2≤n∑ipi2,1= ( _ip_i )^2≤ n _ip_i^2, while ∑ipi2≤1 _ip_i^2≤ 1. Therefore 1≤E(p)≤n1≤ E(p)≤ n. Equality at the upper bound requires uniform weights, for which E(p)pi=1E(p)p_i=1 and E(p)rk=∑i[zi=k].E(p)r_k= _iI[z_i=k]. Equality at the lower bound requires a point mass. Multiplying every sis_i by the same positive constant leaves p, r, E(p)E(p), and the pseudo-count posterior unchanged. A.2 Posterior utility moments For a Dirichlet vector with mean μ and total concentration α0 _0, Cov(θ)=diag(μ)−μμ⊤α0+1.Cov(θ)= diag(μ)-μ _0+1. Thus [v⊤θ]=v⊤μE[v θ]=v μ and Var(v⊤θ)=∑kμkvk2−(v⊤μ)2α0+1.Var(v θ)= _k _kv_k^2-(v μ)^2 _0+1. Taking v=(1,−1,0,0)v=(1,-1,0,0) gives Equations 6 and 7. The training score u¯−λσu u-λ _u combines these utility moments. A.3 Proof of Proposition 1 Proof. Set A=4αA=4α. The posterior mean is μk(τ)=α+τrkA+τ=τA+τrk+A+τ14. _k(τ)= α+τ r_kA+τ= τA+τr_k+ AA+τ 14. For any categories j,kj,k, μj(τ)−μk(τ)=τ(rj−rk)A+τ. _j(τ)- _k(τ)= τ(r_j-r_k)A+τ. Its sign is independent of τ>0τ>0, proving invariance of category ordering, including ties. To establish monotonicity of the training weight, write d=rFix−rRegress,b=rFix+rRegress.d=r_ Fix-r_ Regress, b=r_ Fix+r_ Regress. Then u¯τ u_τ =τdA+τ, = τ dA+τ, (15) στ2 _τ^2 =(2α+τb)(A+τ)−τ2d2(A+τ)2(A+τ+1). = (2α+τ b)(A+τ)-τ^2d^2(A+τ)^2(A+τ+1). (16) If d≤0d≤ 0, then u¯τ≤0 u_τ≤ 0 and wτ=0w_τ=0 for every τ. Suppose d>0d>0. All Dirichlet parameters are positive, so στ>0 _τ>0. The ratio Tτ=u¯τ/στT_τ= u_τ/ _τ satisfies Tτ2=d2(A+τ+1)b−d2+(2α+Ab)/τ+2αA/τ2.T_τ^2= d^2(A+τ+1)b-d^2+(2α+Ab)/τ+2α A/τ^2. (17) Because |d|≤b≤1|d|≤ b≤ 1, we have b−d2≥0b-d^2≥ 0. The numerator is strictly increasing in τ, and the positive denominator is strictly decreasing. Therefore TτT_τ is strictly increasing. Since u¯τ u_τ is also positive and increasing, wτ=u¯τ[1−λTτ]+w_τ= u_τ [1- λT_τ ]_+ is nondecreasing for every λ≥0λ≥ 0. Finally, E(p)≤nE(p)≤ n gives wE(p)≤wnw_E(p)≤ w_n. ∎ The proposition characterizes the supervised coefficient under fixed r, α, λ, and utility vector. A.4 When reducing evidence mass changes log loss For a predictive label set of size K, the symmetric-prior construction is μk(τ)=(α+τrk)/(Kα+τ) _k(τ)=(α+τ r_k)/(Kα+τ). The difference μj−μk=τ(rj−rk)/(Kα+τ) _j- _k=τ(r_j-r_k)/(Kα+τ) preserves category ordering for every K. The category-order statement holds for any predictive label set. The correction-utility result uses K=4K=4 and v=(1,−1,0,0)v=(1,-1,0,0). For an observed transition y, define ℓτ(y)=−logμy(τ)=−logα+τry4α+τ. _τ(y)=- _y(τ)=- α+τ r_y4α+τ. Differentiation gives ∂ℓτ(y)∂τ=α(1−4ry)(α+τry)(4α+τ). ∂ _τ(y)∂τ= α(1-4r_y)(α+τ r_y)(4α+τ). (18) Consequently, when E(p)<nE(p)<n, reducing mass from n to E(p)E(p) lowers log loss exactly when ry<1/4r_y<1/4, increases it when ry>1/4r_y>1/4, and leaves it unchanged when ry=1/4r_y=1/4. The log-loss effect is therefore outcome-dependent: empirical gains reflect how relevance concentration aligns with errors in relative support. A.5 Interpretive limits of effective mass Consider replacing every observation by m identical copies. Each copy then has normalized weight pi/mp_i/m, so E(p′)=(∑i∑ℓ=1m(pi/m)2)−1=mE(p),r′=r.E(p )= ( _i _ =1^m(p_i/m)^2 )^-1=mE(p), r =r. Duplicating observations increases mass while leaving relative support fixed. Effective mass consequently summarizes normalized relevance concentration, while dependence and absolute score reliability are separate properties of the execution record. Appendix B Execution Protocol and Implementation B.1 Measurement protocols Table 4: Evaluation protocols and outcomes. Candidate selection, outcome prediction, and post-update generation use the protocols below. Study Recorded protocol Reported outcome Candidate-bank audit 500 source groups per benchmark; eight candidates and four public executions per group Visible-selected and retrospective hidden-oracle success rates History-size prediction n∈1,2,4,8n∈\1,2,4,8\ visible observations; RunBugRun and CodeARC; DeepSeek and Qwen; primary run Future-outcome NLL and categorical accuracy Matched-mass prediction DeepSeek/RunBugRun; same relative support and effective-arm settings within each contrast NLL difference and argmax agreement; three repetitions One-round learning Three backbones and four domains; 500 primary sources per cell; one fitting run and one fresh candidate per source Pass@1 requiring all ten evaluator tests; paired source-bootstrap interval The retrospective oracle in Appendix C selects within a fixed candidate bank using evaluator outcomes. The downstream reference is the unchanged entering policy. Outcome prediction is scored with NLL and categorical accuracy; post-update generation is scored with all-tests Pass@1. Evaluation uses the recorded source populations. The supplied aggregate reports leave the overlap between training and evaluator source populations unspecified. B.2 Randomness and uncertainty The recorded random-number-generator seeds are 1701 for the primary history sweep, matched-mass comparison, and downstream evaluation, and 1702 and 1703 for the additional matched-mass repetitions. Each downstream model–domain cell reports one fitting run. Its interval is a nominal per-cell 95% paired source-bootstrap interval conditional on the evaluated policies. We use the recorded 10,000-draw intervals and report fitting-run replication separately from source-level uncertainty. B.3 Relevance scoring and token-level objective Let Δa a be the unified diff between a0a^0 and a1a^1, and let f be the frozen 256-dimensional signed-hash text encoder. Execution descriptor eie_i contains the input, expected output, actual output, stderr, and categorical outcome. The one-round correction scorer uses si=exp(κcos(f(Δa),f(ei))),κ=16.s_i= \! (κ (f( a),f(e_i) ) ), κ=16. (19) The scorer is frozen and uses public execution descriptors. Equation 19 specifies the one-round correction scorer; the prediction studies retain their recorded matched-comparison settings. For scored correction-response length LhL_h, let ch,jc_h,j be the actual training prompt followed by a<j1a^1_<j. The two token-mean losses are ℒCE(h) _CE(h) =−1Lh∑j=1Lhlogπt+1(aj1∣ch,j), =- 1L_h _j=1^L_h _t+1(a^1_j c_h,j), (20) DKL(h)(πt∥πt+1) D_KL^(h)( _t\| _t+1) =1Lh∑j=1LhDKL(πt(⋅∣ch,j)∥πt+1(⋅∣ch,j)). = 1L_h _j=1^L_hD_KL\! ( _t(· c_h,j)\,\|\, _t+1(· c_h,j) ). (21) Teacher and student use the same response prefixes. The execution record determines the scalar w(h)w(h), and the training prompt supplies the token context. At w(h)=0w(h)=0, the objective retains its KL term. B.4 Optimization settings Table 5: Recorded one-round training configuration. GPU memory denotes nominal capacity. Setting Value Hardware NVIDIA RTX A6000; 48 GB per GPU Student adaptation / base LoRA only / frozen Quantization 4-bit NF4, double quantization Compute dtype bfloat16 LoRA rank / scaling alpha / dropout 16 / 32 / 0 Optimizer / learning rate AdamW / 2×10−42× 10^-4 Gradient accumulation / clipping 16 / 1.0 Maximum sequence length 4608 Relevance encoder / κ 256-dimensional signed hash / 16 Dirichlet prior α 0.1 Utility / uncertainty coefficient (1,−1,0,0)(1,-1,0,0) / λ=0.5λ=0.5 Forward-KL coefficient β 0.03 Loss normalization Response tokens Training budget policy Response-token accounting B.5 One-correction algorithm Given (a0,a1)(a^0,a^1) and their public before/after execution outcomes: 1. Compute sis_i using Equation 19 and normalize pi=si/∑jsjp_i=s_i/ _js_j. 2. Assign each outcome pair its transition ziz_i and aggregate rk=∑ipi[zi=k]r_k= _ip_iI[z_i=k]. 3. Set E(p)=1/∑ipi2E(p)=1/ _ip_i^2 and αkpost=α+E(p)rk _k^post=α+E(p)r_k. 4. Calculate u¯ u and σu _u under v=(1,−1,0,0)v=(1,-1,0,0), then set w=[u¯−λσu]+w=[ u-λ _u]_+. 5. Fit the LoRA student with wℒCE+βDKL(h)(πt∥πt+1)wL_CE+β D_KL^(h)( _t\| _t+1) while the entering-policy teacher is frozen. Appendix C Candidate-Bank Selection Diagnostic For each of 500 source groups per benchmark, visible-score selection chooses among eight fixed candidates using four public executions. A retrospective hidden oracle selects the candidate with the highest additional evaluator score in that bank. This evaluation-only diagnostic measures selection headroom among the generated candidates. Table 6: Selection headroom within a fixed candidate bank. Both selectors operate on the same eight generated candidates; oracle selection uses evaluator outcomes. Benchmark Visible selected (%) Hidden oracle (%) Gap (p) CodeARC 27.0 31.2 +4.2 RunBugRun 71.4 75.4 +4.0 The gaps quantify the selection headroom available within each fixed candidate bank. Appendix D Complete One-Round Downstream Results Table 7: Complete one-round downstream matrix for the primary run. Each model–domain setting contains 500 sources and reports fresh all-tests Pass@1. Intervals are paired source-bootstrap 95% intervals. Benchmark Backbone no_update ↑ EESD ↑ Δ (p) ↑ 95% CI (p) RunBugRun Qwen2.5 Coder 7B 228/500 240/500 +2.4 [-2.0,+6.8] DeepSeek Coder 6.7B 184/500 198/500 +2.8 [-1.8,+7.4] Gemma 3 4B 142/500 132/500 -2.0 [-5.6,+1.4] CodeARC Qwen2.5 Coder 7B 85/500 83/500 -0.4 [-2.8,+2.0] DeepSeek Coder 6.7B 75/500 102/500 +5.4 [+2.8,+8.0] Gemma 3 4B 59/500 56/500 -0.6 [-2.6,+1.4] APPS Replay Qwen2.5 Coder 7B 57/500 57/500 +0.0 [-1.8,+1.8] DeepSeek Coder 6.7B 23/500 17/500 -1.2 [-3.2,+0.8] Gemma 3 4B 36/500 2/500 -6.8 [-9.0,-4.8] CodeContests Replay Qwen2.5 Coder 7B 47/500 39/500 -1.6 [-3.2,+0.0] DeepSeek Coder 6.7B 18/500 24/500 +1.2 [-0.2,+2.8] Gemma 3 4B 31/500 4/500 -5.4 [-7.6,-3.2] Table 7 reports all twelve model–domain settings. DeepSeek has positive point estimates in three domains; Qwen improves on RunBugRun and matches its entering-policy solved count on APPS Replay. The full range of paired differences is [−6.8,+5.4][-6.8,+5.4] percentage points. Intervals quantify source-level uncertainty for the primary fitting run. Appendix E History-Size Mechanism Details Table 8: Primary history-size sweep with both fixed and effective mass. Values are future-outcome NLL; lower is better. Displayed precision is four decimals. n=1n=1 n=2n=2 n=4n=4 n=8n=8 Benchmark Backbone Fixed EED Fixed EED Fixed EED Fixed EED RunBugRun DeepSeek .4696 .4696 .3656 .3646 .2703 .2693 .2160 .2114 RunBugRun Qwen .4737 .4737 .3292 .3291 .2704 .2704 .2188 .2113 CodeARC DeepSeek .7103 .7103 .5440 .5440 .4149 .4149 .3005 .2892 CodeARC Qwen .7785 .7785 .6176 .6176 .4725 .4727 .3439 .3415 Table 8 supplies Figure 3. The history-size sweep and the matched-mass study in Appendix F retain their respective recorded predictor settings. The DeepSeek/RunBugRun difference at n=4n=4 is 0.00100.0010 in the sweep; the primary matched-mass contrast is 0.006090.00609. Displayed ties refer to the four-decimal reporting precision. Table 9: Categorical accuracy (%) in the reported history-size sweep. Values are reported to one decimal place. Accuracy is measured on future categorical outcomes. Benchmark Backbone n=1n=1 n=2n=2 n=4n=4 n=8n=8 RunBugRun DeepSeek 6.7B 90.7 90.9 92.2 93.2 RunBugRun Qwen2.5 7B 90.6 91.4 91.5 92.5 CodeARC DeepSeek 6.7B 83.3 83.9 84.7 88.8 CodeARC Qwen2.5 7B 80.5 81.0 83.7 86.6 Appendix F Matched-Mass Mechanism and Replication Concentration bins. Table 3 and Figure 4(b) report the same primary DeepSeek/RunBugRun comparison at matched effective-arm settings. The bins contain 3,000 examples in total, with identical argmax predictions between estimators. Positive NLL differences favor effective mass. The table reports the recorded bin-level point estimates. Recorded repetitions. Table 10 gives all three recorded repetitions. Each contrast matches the effective-arm parameters between predictors. The table records the initialization-specific contrasts under the outcome-dependent log-loss characterization in Appendix A.4. Initialization identifiers are defined in Appendix B.2. Table 10: DeepSeek/RunBugRun mechanism sensitivity across recorded initializations at the effective-arm parameters. Positive NLL gain favors effective evidence over fixed mass. Initialization NLL gain ↑ 95% CI 1701 +0.00609 [+0.00211,+0.01075] 1702 -0.00367 [-0.00447,-0.00287] 1703 +0.00067 [+0.00023,+0.00118] Appendix G Derived Arithmetic and Paired-Count Interpretation G.1 Coding illustration and supervision weights In Figure 1, sum(range(1, n)) gives 0,1,3,60,1,3,6 on inputs 1,2,3,41,2,3,4, while sum(range(1, n+1)) gives 1,3,6,101,3,6,10. All transitions are Fix, so r=(1,0,0,0)r=(1,0,0,0) for either stipulated relevance vector: (0.25,0.25,0.25,0.25)(0.25,0.25,0.25,0.25) or (0.97,0.01,0.01,0.01)(0.97,0.01,0.01,0.01). The relevance vectors define the analytical example independently of the lexical kernel. Their masses are 44 and 1/0.9412≈1.0624731/0.9412≈ 1.062473. With α=0.1α=0.1, λ=0.5λ=0.5, and v=(1,−1,0,0)v=(1,-1,0,0), Equations 5–8 give w≈0.832081w≈ 0.832081 and 0.5419460.541946. Fixed mass assigns 0.8320810.832081 to both. Targets and KL coefficients are held fixed while the supervised coefficients change with effective mass. G.2 Reported mass fractions and paired counts For the four-observation matched comparison in Table 3, the retained pseudo-count fraction is E(p)/4E(p)/4. The three bin means therefore retain 100(1.511/4)=37.775%100(1.511/4)=37.775\%, 100(2.521/4)=63.025%100(2.521/4)=63.025\%, and 100(3.681/4)=92.025%100(3.681/4)=92.025\% of fixed mass. The main text rounds these to one decimal place. These ratios summarize retained pseudo-count mass. For paired downstream outcomes, let N01N_01 count sources that fail under no_update and pass under EESD, and N10N_10 count the reverse. The difference in solved-source counts equals N01−N10N_01-N_10, giving Δp=100N01−N10N,N=500. _p=100 N_01-N_10N, N=500. Aggregate solved counts identify the net difference N01−N10N_01-N_10. The paired uncertainty intervals are taken from the recorded whole-source bootstrap comparisons.