Paper deep dive
The Harness as the Only Mutable Surface: Compliance-Bounded Self-Evolution of LLM Agents in Credit Pipelines, with a Measured Admission Gate
Ravil Akhtyamov
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 10/9/2026, 7:42:50 AM
Summary
This paper proposes a compliance-bounded self-evolution framework for LLM agents in credit pipelines, restricting modifications to the runtime harness (instruction text, tool logic, primitives) while keeping model weights fixed. A dual-loop engine with a measured admission gate ensures every adaptation is auditable via hash-chained records. Simulations show the gate admits only 144 of 7,449 candidates, preventing harmful changes and maintaining false-positive rates, unlike unbounded systems which admitted 309 harmful changes. The work maps these mechanisms to EU AI Act provisions and highlights gaps in US model-risk guidance.
Entities (8)
Relation Signals (7)
Runtime Harness → ismutablesurfacefor → Self-Evolving Agents
confidence 95% · We argue that self-evolution is reviewable only if it is confined to the runtime harness... while model weights stay fixed
Admission Gate → prevents → Harmful Changes
confidence 94% · the gated loop admitted 144 of 7,449 candidate changes, none of which worsened error... With the gate replaced... the same loops admitted 309 harmful changes
Admission Gate → enforces → Compliance Bounds
confidence 93% · A bound is only as good as the control that enforces it... the admission gate... was specified but not measured
EU AI Act → regulates → High-Risk Credit Scoring
confidence 92% · The EU AI Act classifies creditworthiness assessment of natural persons as high-risk
CACE-Bench → evaluates → Credit Risk Pipelines
confidence 90% · A released benchmark (Section 9), CACE-Bench, whose reference run on decision availability we now report
Self-Harness → influences → Local Loop
confidence 85% · Our local loop follows the weakness-mining, proposal and validation cycle of Self-Harness [5]
HarnessX → influences → Global Loop
confidence 85% · our global loop follows the primitive replacement of HarnessX [6]
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Self-improving LLM agents can adapt a credit pipeline to a changed rule, but an agent that rewrites itself destroys the artefact a supervisor reviews: a named change, a recorded test, an approval. We argue that self-evolution is reviewable only if it is confined to the runtime harness (instruction text, tool-call logic and primitive composition) while model weights stay fixed, so that every adaptation is a diff with a cause and a test attached. We give a dual-loop engine built on that bound, with one admission gate that writes a hash-chained record before deployment, and we measure the gate in simulation, with a simulated agent and a seeded-search proposer rather than language models. Across three families of supervisory re-interpretation at three severities, 10 seeds each, the gated loop admitted 144 of 7,449 candidate changes, none of which worsened error on held-out history, and restored the false-positive rate to the oracle level without raising missed flags in every low- and mid-severity cell. With the gate replaced by the check an unbounded system applies (fewer errors visible in recent traces), the same loops admitted 309 harmful changes and left missed flags above 10% in 49 of 90 runs: false positives fell because the screen was loosened. Evaluated on pre-shift labels, the gate rejected every candidate, so a re-interpretation must be encoded as a rule that relabels history. Parametric and scope shifts were repaired locally, a structural one only by primitive replacement; at the highest structural severity the gate's fixed tolerance blocked the correct replacement in half the seeds. We map the mechanisms to the EU AI Act's provisions for high-risk credit scoring and note that the April 2026 US model-risk guidance excludes agentic AI from its scope.
Tags
Links
- Source: https://arxiv.org/abs/2610.10629v1
- Canonical: https://arxiv.org/abs/2610.10629v1
Trouble viewing inline? Open PDF directly →
Full Text
48,933 characters extracted from source content.
Expand or collapse full text
The Harness as the Only Mutable Surface: Compliance-Bounded Self-Evolution of LLM Agents in Credit Pipelines, with a Measured Admission Gate Ravil Akhtyamov Affiliation: Digital Economy Lab Email: latam@digitaleconomylab.org ORCID 0000-0002-0783-8573 Abstract Self-improving large language model (LLM) agents can adapt a credit pipeline to a changed rule, but an agent that rewrites itself destroys the artefact a supervisor reviews: a named change, a recorded test, an approval. We argue that self-evolution is reviewable only if it is confined to the runtime harness—instruction text, tool-call logic and primitive composition—while model weights stay fixed, so that every adaptation is a diff with a cause and a test attached. We give a dual-loop engine built on that bound, in which a local loop edits one primitive and a global loop replaces primitives from a typed library, and both submit to one admission gate that writes a hash-chained record before deployment. We then measure the gate in simulation, with a simulated agent and a seeded-search proposer rather than language models. Across three families of supervisory re-interpretation at three severities, 10 seeds each, the gated dual loop admitted 144 of 7,449 candidate changes, none of which worsened error on held-out history, and restored the false-positive rate to the oracle level without raising missed flags in all low- and mid-severity cells. With the gate replaced by the check an unbounded system applies—fewer errors visible in recent traces—the same loops admitted 309 harmful changes and left missed flags above 10% in 49 of 90 runs: false positives fell because the screen was loosened. Evaluated on pre-shift labels, the gate rejected every candidate, so a re-interpretation must be encoded as a rule that relabels history. Parametric and scope shifts were repaired locally, a structural one only by primitive replacement; at the highest structural severity the gate’s fixed tolerance blocked the correct replacement in half the seeds. We map the mechanisms to the EU AI Act’s provisions for high-risk credit scoring and note that the April 2026 US model-risk guidance excludes agentic AI from its scope. Keywords: LLM agents; self-evolving agents; credit risk; model risk management; EU AI Act; auditability; RegTech; benchmark 1 Introduction Credit underwriting is moving from monolithic scoring models to pipelines of specialised LLM agents that query registries, screen counterparties and draft decisions. When a statute or a supervisory interpretation changes, such a pipeline must change too. Work on self-evolving agents promises to automate that adaptation [14, 15], but it was built to raise a task metric on open benchmarks, not to leave behind what a supervisor reviews. The difficulty is not that self-evolution is unsafe in general. It is that model-risk governance presumes a versioned object with an owner, a test result and an approval [3, 1], and continuous self-modification dissolves all three. The question is therefore not whether to bound self-evolution but which bound preserves the reviewable artefact. Our answer is that the runtime harness is the only legitimate mutable surface, because it is the auditable one. Weights cannot be diffed into named changes; a harness can. If the engine may rewrite instruction text, tool-call logic and the composition of typed primitives, and nothing else, then “the agent changed” stops being an event that requires re-validating a model and becomes a text diff with a cause and a test attached. The self-evolution literature treats the harness as one surface among several to optimise over [5, 6, 7]; we treat it as the only one an institution may let evolve. A bound is only as good as the control that enforces it. In an earlier presentation of this design the admission gate—the mechanism that distinguishes it from unbounded methods—was specified but not measured, and three referees said so. This paper is the measurement. We claim: 1. A position and an architecture (Sections 4–5). Harness-only evolution with fixed, digest-pinned weights (a design commitment: no model is pinned in our simulation); a local and a global loop sharing one retrospective pool and one gate; an admission rule with a value for every threshold (Table 2); a hash-chained log written before deployment. We separate what the log makes auditable (the configuration change) from what it does not make reproducible (downstream behaviour of a hosted model). 2. A measured gate (Sections 6–7). A regulatory-shift track with three families and three severities of re-interpretation, 10 seeds per cell and ten arms, including a no-gate arm, a gate on stale labels, oracle and manual updates and a compute-matched local-only arm. We report missed-flag rates on every row, candidate counts, false admissions audited on held-out data, one worked rejection, and the gate’s own failure mode. 3. An answer to where the labels come from (Section 6.2). After a re-interpretation the retrospective pool must be relabelled by the new rule, or a correct adaptation is indistinguishable from a regression. This makes the adaptation supervised at the level of the rule and autonomous only at the level of the harness, and we position the autonomy claim accordingly. 4. A regulatory mapping (Section 8) onto the EU AI Act’s provisions on predetermined changes, logging, human oversight, robustness and change management, and an account of the gap the April 2026 US guidance leaves for agentic systems. 5. A released benchmark (Section 9), CACE-Bench, whose reference run on decision availability we now report with missed-flag rates and 5 seeds. What we do not claim is equally specific. The proposer in our experiments is a seeded search over a stated edit space, not a language model, and the typed library contains the correct replacement primitive by construction. We therefore measure whether the bound and the gate select correct changes and reject harmful ones, not whether an LLM can invent the changes. All data are synthetic. No supervisor has reviewed the mapping. 2 Related work Self-evolving agents. Agents that improve themselves now span memory [9], prompts [8, 7], harnesses [5], typed harness primitives co-evolved with model parameters [6], workflows [13], and code or architecture [10, 11, 12]; two surveys organise the field by what evolves and where the loop closes [14, 15]. Our local loop follows the weakness-mining, proposal and validation cycle of Self-Harness [5]; our global loop follows the primitive replacement of HarnessX [6], without HarnessX’s co-evolution of model parameters, which is precisely what our bound excludes. We do not claim either loop as new. Table 1 places the work on the axes a regulated deployment cares about. The field closes its loops on a task metric and produces no change record intended for a third party; that is a governance gap, not a capability gap, and we make no claim that our engine optimises better than any of these systems. Table 1: Positioning against representative self-evolution systems. Gate: is a change admitted only after an explicit, recorded test? Record: is a change artefact produced for an external reviewer? Gate measured: are admission and rejection statistics reported? ∼ : a validation step exists but is neither recorded for review nor reported as a control. System What evolves Gate Record Gate measured Reflexion [9] memory – – – PromptBreeder [8] prompts – – – GEPA [7] prompts ∼ – – Self-Harness [5] harness ∼ – – HarnessX [6] primitives, weights ∼ – – EvoAgentX [13] workflows ∼ – – ADAS [10], DGM [12], Gödel Agent [11] code / architecture – – – This work harness only yes yes, hash-chained yes Evaluation of agents in finance. We build on a three-level quality-metric system for LLM agents in the credit pipeline introduced in prior work [16]; it is not a contribution of this paper. Every number reported here is computed against deterministic labels; no LLM judge is involved anywhere in the results. Model-risk governance. US supervisory model-risk guidance was revised on 17 April 2026 (SR 26-2; OCC Bulletin 2026-13; FDIC FIL-15-2026), rescinding SR 11-7 and stating that generative and agentic AI models are outside its scope [3, 4]. The EU AI Act classifies creditworthiness assessment of natural persons as high-risk [1, Annex I, point 5(b)] and regulates change in systems that continue to learn [1, Art. 43(4)]. Section 8 develops both. 3 Problem, threat model and requirements We consider a multi-agent credit pipeline—client assistant, underwriter, compliance and legal agents—whose agents execute typed primitives: named, individually testable units of behaviour such as registration verification or the beneficial-owner check. A primitive is defined by its instruction text, its tool-call contract and its output schema. The set of primitives available to an agent, with their instruction texts and parameters, is that agent’s runtime harness. It is the only object the engine may modify and the granularity at which every audit record is written. The benchmark is jurisdiction-parametrised; its reference run (Section 9) covers seven Latin American jurisdictions, and nothing in the engine depends on a particular national statute. Threat model. We design against five failure modes. F1 Silent drift: a change improves the target metric while degrading an unmeasured one; mitigated by the non-regression clause, and only to the extent that the degraded dimension is measured. F2 Retro-test overfitting: a candidate is tuned until it passes the pool; mitigated by a pool split the gate never sees (HvalH_val) and a held-out post-shift window. F3 Untraceable change: mitigated by making the log write a precondition of deployment. F4 Objective capture: evolution objective and audit evidence from the same judge; mitigated by computing both on deterministic labels only. F5 Exogenous model change: the provider updates a hosted model beneath a fixed harness; not mitigated by our design (Section 4). Requirements. (R1) Measurability: every step yields signals localisable to a primitive. (R2) Auditability: each change carries a cause, a test result and a deployment record in an append-only log. (R3) Bounded autonomy: adaptation must not degrade previously correct behaviour beyond a stated tolerance, and weights must not change. (R4) Regulatory correspondence: controls must correspond to identifiable obligations (Section 8). 4 The harness as the governance primitive The engine may rewrite instruction text, tool-call logic and primitive composition; it may not fine-tune, and the served model is pinned by digest. This preserves the object a supervisor knows how to review. A change under model-risk governance is expected to have a cause, a test, an approver and a rollback path. A weight update supplies none of these naturally: it is continuous, not decomposable into named changes, and cannot be diffed. A harness edit supplies all of them by construction: it is a diff against a named primitive, reproducible from the log, and revertible by writing the previous version back. Two properties must be kept apart, because a referee rightly observed that we had conflated them. Auditability of the change is delivered: every harness version is a canonical serialisation with a SHA-256 digest, every gate decision names the digests before and after, and the log is hash-chained so that removing or reordering a record breaks verification (all 540 of the 540 logs written in our experiments, 43,814 records in all, verify). The record names a deploying identity; in the simulation it is the gate’s service identity, and a human approval step before deployment is a deployment choice we do not simulate. Reproducibility of behaviour is not delivered by the same mechanism: identical harnesses on a hosted model can behave differently if the provider changes the model, the decoding settings drift, a tool returns different data, or retrieval state moves. Weight attestation binds our changes and not the provider’s (F5). For a deterministic pipeline—as in our benchmark—the two coincide; for a hosted LLM, behaviour reproducibility additionally requires pinned model versions and decoding parameters, recorded tool responses, and canary probes on a fixed prompt set. The cost of the bound is real: the system cannot learn anything that instruction text and primitive composition cannot express. A task the base model cannot do falls outside the loop and remains engineering work. We regard that as the right trade for regulated deployment, not as a free one. 5 The engine 5.1 Admission rule Let h be the deployed harness, h′h a candidate, HselH_sel the half of the retrospective pool the gate may use, labelled under the current rule (Section 6.2), C⊆HselC H_sel the cases h answers correctly, and k∈FP,missk∈\FP,miss\ the error kind of the cluster that caused the proposal. Over the cases in HselH_sel on which an error of kind k is possible (true clear for FP, true flag for misses), let b count cases where h errs and h′h does not, c the reverse, and nkn_k their number. The paired reduction is g^=(b−c)/nk g=(b-c)/n_k with variance estimate var^=(b+c)/nk2−(b−c)2/nk3 var=(b+c)/n_k^2-(b-c)^2/n_k^3. The candidate is admitted iff g^−z0.975var^>0⏟significant improvement∧|x∈C:h′(x)≠y(x)||C|≤ϵ⏟non-regression on C. g-z_0.975 var>0_significant improvement |\x∈ C:h (x)≠ y(x)\||C|≤ε_non-regression on C. (1) The improvement margin is statistical rather than a fixed δ, so 0.1 and 5 percentage points are no longer alike to the gate. The tolerance ϵ=0.005ε=0.005 is not zero because execution noise makes strict non-regression unsatisfiable: any change that alters the rule’s output on a case can meet that case’s noise draw and turn a correct answer wrong. Section 7.6 measures what this choice costs. Admission is a write, not only a decision. Before a candidate reaches the runtime the gate appends ⟨ , stream position, loop, cause, diff, digest of h, digest of h′h , rendered text of h′h , b, c, nkn_k, g g and its interval, |C||C|, regressions on C, decision, deploying identity, hash of the previous record⟩ and the record’s own hash. Rejections are written with the same schema. 5.2 Loops Operational feedback arrives in windows of 500 cases. Every flagged case is reviewed by an analyst, so its false positive becomes visible; cleared cases are audited at 5%, so most misses stay invisible. This asymmetry is deliberate and realistic, and it is what makes an ungated loop dangerous: a proposer guided by visible failures sees false positives and almost no misses. Algorithm 1 Local loop, per window 1: window W, deployed harness h, gate pool HselH_sel, budget Bℓ=3B_ =3, clusters K=3K=3, escalation threshold N=3N=3 2: F←F← visible errors in W clustered by (hit type, error kind); keep the K most frequent 3: for each cluster f∈Ff∈ F do 4: P←P← one-step edits to the deployed primitive (Table 2), shuffled; keep BℓB_ 5: for each h′∈Ph ∈ P do 6: write record; if h′h satisfies Eq. (1) on HselH_sel then h←h′h← h ; break 7: end for 8: [f]←0stall[f]← 0 if admitted, else [f]+1stall[f]+1 9: if [f]≥Nstall[f]≥ N then Escalate(f)(f) to Algorithm 2; [f]←0stall[f]← 0 10: end if 11: end for Algorithm 2 Global loop, on escalation (dual loop) or on threshold breach (global-only arm) 1: cluster f, deployed harness h, typed library ℒL, budget Bg=6B_g=6 2: P←(p,θ):p∈ℒ∖ph,θ∈ΘP←\(p,θ):p \p_h\,\ θ∈ \ with h’s scope, shuffled; keep BgB_g 3: for each h′∈Ph ∈ P do 4: write record; if h′h satisfies Eq. (1) on HselH_sel then h←h′h← h ; return 5: end for The local loop (Algorithm 1) may change one parameter or one clause of the deployed primitive per candidate. The global loop (Algorithm 2) may replace the primitive with another from a typed library—here, a check on the stake held through the first-layer vehicle versus a check on the effective stake along the ownership chain—so a replacement is still a discrete, nameable change. It has no privileged path to deployment: the same gate applies. In the global-only arm the loop is triggered when the visible false-positive share of reviewed flags exceeds its pre-shift level by τ=5τ=5 percentage points for two consecutive windows. Table 2 lists every constant. Table 2: Every free parameter of the engine and the experiment, with the value used. All are constants in evolve.py. Parameter Value Stream 23,000 screening alerts; shift at T=70%T=70\%; adaptation on [T,85%)[T,85\%); held-out evaluation on [85%,100%][85\%,100\%] Retrospective pool pre-T stream; even-indexed cases HselH_sel (gate), odd-indexed HvalH_val (audit only) Window / cycle 500 cases; flags reviewed 100%; clears audited 5% Execution noise outcome flipped with probability 0.02 per case, draw shared across harness versions Local loop Bℓ=3B_ =3 candidates per cluster, K=3K=3 clusters per cycle; for a false-positive cluster: remove its hit type from scope, or θ+0.05,0.10,0.15,0.20,0.25θ+\0.05,0.10,0.15,0.20,0.25\; for a miss cluster: add its hit type, or θ−0.05,0.10,0.15θ-\0.05,0.10,0.15\ Escalation N=3N=3 consecutive cycles without admission for a cluster Global loop Bg=6B_g=6 candidates; library \first-layer stake, effective stake\; Θ=0.15,0.20,0.25,0.30,0.35,0.40,0.50 =\0.15,0.20,0.25,0.30,0.35,0.40,0.50\ Gate paired 95% lower bound of reduction >0>0; ϵ=0.005ε=0.005; sensitivity at ϵ=0.010ε=0.010 (structural family) Global-only trigger visible flag-FP share above pre-T level by τ=0.05τ=0.05 for 2 windows Manual update correct harness deployed 2,000 cases after T (an assumption, not a measurement) Agent a deterministic rule applying the harness, plus the execution noise above; no language model Proposer seeded random choice from the edit space above; no language model Seeds 10 per cell; 3 families × 3 severities == 9 cells, 90 runs 6 The regulatory-shift track 6.1 World and rules The population is the screening-alert queue: each case carries a name hit of one of five types (formal designation, high- and low-grade adverse media, foreign and domestic politically exposed person) and a beneficial-ownership chain of depth one to three. The compliance primitive returns flag or clear. Ground truth is a versioned rule. Before T (rule v1): flag when the hit type is in scope and the stake held through the first-layer vehicle is at least 25%. From T (rule v2) one of three re-interpretations applies: • Thr (parametric): the ownership threshold rises to 30, 40 or 50%. • Scope (clause): low-grade adverse media; then all adverse media; then also domestic PEP status no longer qualify on their own. • Struct (structural): the threshold applies to the effective stake, the product of stakes along the chain, with multi-layer chains in 30, 50 or 70% of cases. No threshold or scope edit to the first-layer check can express this rule. All three make the v1 harness over-flag, so the observable symptom is the same—a rise in false positives—while the correct repair differs in kind. 6.2 Where the labels come from after a re-interpretation A referee observed that, as we had described it, the retrospective test was circular: the pre-shift pool carries labels under the interpretation the shift replaced, so a correct adaptation must look like a regression on it. The objection is correct, and the answer is a design commitment rather than a detail. At T the pool is relabelled by applying rule v2 to the stored case facts. In a deployment this is the compliance function encoding the supervisor’s re-interpretation as an executable rule—nobody relabels cases by hand, but a person authors the rule. The engine therefore searches for a harness edit autonomously while the target of the search is set by people: the adaptation is supervised at the level of the rule and autonomous at the level of the harness. We position the autonomy claim accordingly. The “gate on stale labels” arm, which keeps the v1 labels, shows empirically what happens otherwise (Section 7.2). The relabelling assumes the rule encodes the re-interpretation correctly; a mis-encoded rule would be faithfully optimised towards. That is the right place for the risk to sit—the rule is a short, reviewable artefact authored by an accountable function—but it is a risk. 6.3 Arms All arms see the same cases, the same noise draws and the same feedback model; what each observes depends on its own harness. Healthy: no shift, v1 harness. Degraded: shift, no adaptation. Oracle: the correct v2 harness from T. Manual: the correct harness after a 2,000-case lag, standing in for an engineer who rewrites the prompt after an alert. No gate: the dual loop with Eq. (1) replaced by the check an unbounded self-evolution system applies—admit if the candidate makes fewer errors visible in the current window. Gate on stale labels: the dual loop with HselH_sel under v1 labels. Global only, local only, and local only, compute-matched, whose per-cycle budget is multiplied, per run, by the ratio of the dual loop’s proposals to the local-only arm’s (rounded up; between 2 and 2 in our runs). Dual loop: the full engine. 6.4 Metrics On the held-out window, with flag as the positive class: false-positive rate =FP/(FP+TN)=FP/(FP+TN); missed-flag rate =FN/(FN+TP)=FN/(FN+TP) and recall =1−=1-missed-flag rate; regression rate == the share of held-out cases answered correctly by the degraded harness that the final harness answers wrongly (a paired, case-level measure). During adaptation: excess errors == true errors in [T,85%)[T,85\%) minus those of the oracle arm; recovery window == the first window from which the arm’s error rate stays within one percentage point of the oracle’s (7 means never). For the gate: candidates proposed, admitted and rejected by reason; false admissions, admitted changes that increased true error on HvalH_val, which the gate never sees; and oracle rejections, rejections of the exactly correct harness. Means are over seeds with 95% t-intervals, clipped to [0,100][0,100]. The held-out window holds 3,450 alerts per run; its numbers of true clear and true flag cases under the post-shift rule, which are the denominators of the two rates, are given per cell in Table 4. 7 Results Table 3 reports the mid-severity cell of each family for all ten arms; Table 4 the whole grid for six arms; Table 5 the gate statistics over all 90 runs. Table 3: Mid severity, held-out window, mean over seeds with 95% t-intervals. FP and missed-flag rates in %; intervals clipped to [0,100][0,100]. For the no-gate arm the missed-flag interval is replaced by the range min–max across seeds because its distribution is bimodal. Regression on held-out cases previously answered correctly, in %. Gate: candidates proposed / admitted / false admissions audited on HvalH_val, summed over seeds. Arm FP Missed flag Regr. Excess err. Recov. win. Gate: prop. / adm. / false Thr shift, mid severity, 10 seeds Healthy (no shift) 2.0 [1.6, 2.3] 1.9 [1.6, 2.1] 0.0 – – – Degraded (no adaptation) 28.0 [27.3, 28.7] 1.9 [1.6, 2.2] 0.0 498 7.0 – Oracle update 1.9 [1.6, 2.2] 1.9 [1.6, 2.2] 0.3 0 0.0 – Manual update (lag 2,000) 1.9 [1.6, 2.2] 1.9 [1.6, 2.2] 0.3 295 4.0 – No gate (trace check only) 1.9 [1.6, 2.2] 1.9 1.2–2.5 0.3 139 2.5 784 / 30 / 6 Gate on stale labels 28.0 [27.3, 28.7] 1.9 [1.6, 2.2] 0.0 498 7.0 948 / 0 / 0 Global loop only 28.0 [27.3, 28.7] 1.9 [1.6, 2.2] 0.0 498 7.0 360 / 0 / 0 Local loop only 1.9 [1.6, 2.2] 1.9 [1.6, 2.2] 0.3 81 1.3 566 / 18 / 0 Local only, compute-matched 1.9 [1.6, 2.2] 1.9 [1.6, 2.2] 0.3 74 1.0 1519 / 20 / 0 Dual loop (gated) 1.9 [1.6, 2.2] 1.9 [1.6, 2.2] 0.3 82 1.3 834 / 21 / 0 Scope shift, mid severity, 10 seeds Healthy (no shift) 2.0 [1.6, 2.3] 1.9 [1.6, 2.1] 0.0 – – – Degraded (no adaptation) 36.9 [36.1, 37.7] 1.8 [1.6, 2.1] 0.0 780 7.0 – Oracle update 2.0 [1.7, 2.2] 1.8 [1.6, 2.1] 0.6 0 0.0 – Manual update (lag 2,000) 2.0 [1.7, 2.2] 1.8 [1.6, 2.1] 0.6 455 4.0 – No gate (trace check only) 2.0 [1.7, 2.2] 79.7 2.1–98.7 38.1 940 6.9 536 / 68 / 50 Gate on stale labels 36.9 [36.1, 37.7] 1.8 [1.6, 2.1] 0.0 780 7.0 921 / 0 / 0 Global loop only 36.9 [36.1, 37.7] 1.8 [1.6, 2.1] 0.0 780 7.0 360 / 0 / 0 Local loop only 3.7 [0.0, 7.7] 1.8 [1.6, 2.1] 0.6 266 3.0 590 / 19 / 0 Local only, compute-matched 2.0 [1.7, 2.2] 1.8 [1.6, 2.1] 0.6 111 1.0 1306 / 20 / 0 Dual loop (gated) 2.0 [1.7, 2.2] 1.8 [1.6, 2.1] 0.6 236 2.8 789 / 20 / 0 Struct shift, mid severity, 10 seeds Healthy (no shift) 2.1 [1.7, 2.5] 1.9 [1.7, 2.2] 0.0 – – – Degraded (no adaptation) 27.4 [26.8, 28.0] 1.9 [1.6, 2.2] 0.0 498 7.0 – Oracle update 2.0 [1.7, 2.3] 1.9 [1.6, 2.2] 0.3 0 0.0 – Manual update (lag 2,000) 2.0 [1.7, 2.3] 1.9 [1.6, 2.2] 0.3 293 4.0 – No gate (trace check only) 2.0 [1.7, 2.3] 32.7 1.2–98.1 16.9 1126 6.8 478 / 88 / 63 Gate on stale labels 27.4 [26.8, 28.0] 1.9 [1.6, 2.2] 0.0 498 7.0 954 / 0 / 0 Global loop only 2.0 [1.7, 2.3] 1.9 [1.6, 2.2] 0.3 156 2.1 43 / 10 / 0 Local loop only 27.4 [26.8, 28.0] 1.9 [1.6, 2.2] 0.0 498 7.0 630 / 0 / 0 Local only, compute-matched 27.4 [26.8, 28.0] 1.9 [1.6, 2.2] 0.0 498 7.0 2133 / 0 / 0 Dual loop (gated) 2.0 [1.7, 2.3] 1.9 [1.6, 2.2] 0.3 228 3.1 878 / 10 / 0 Table 4: Full grid, held-out window: FP / missed-flag rate in %, mean over 10 seeds. n: true clear / true flag cases per seed under the post-shift rule, the denominators of the two rates. Family Severity n clear / flag Degraded No gate Global only Local only Dual Oracle Thr low 1,600 / 1,850 13.1 / 1.9 2.0 / 1.9 13.1 / 1.9 2.0 / 1.9 2.0 / 1.9 2.0 / 1.9 Thr mid 1,940 / 1,510 28.0 / 1.9 1.9 / 1.9 28.0 / 1.9 1.9 / 1.9 1.9 / 1.9 1.9 / 1.9 Thr high 2,261 / 1,189 38.0 / 2.0 1.9 / 3.1 38.0 / 2.0 1.9 / 2.0 1.9 / 2.0 1.9 / 2.0 Scope low 1,815 / 1,635 23.1 / 1.8 2.0 / 69.2 23.1 / 1.8 2.0 / 1.8 2.0 / 1.8 2.0 / 1.8 Scope mid 2,223 / 1,227 36.9 / 1.8 2.0 / 79.7 36.9 / 1.8 3.7 / 1.8 2.0 / 1.8 2.0 / 1.8 Scope high 2,627 / 823 46.3 / 1.9 1.9 / 78.8 46.3 / 1.9 1.9 / 1.9 1.9 / 1.9 1.9 / 1.9 Struct low 1,717 / 1,733 18.9 / 1.9 2.1 / 36.1 2.0 / 1.9 18.9 / 1.9 2.0 / 1.9 2.0 / 1.9 Struct mid 1,923 / 1,527 27.4 / 1.9 2.0 / 32.7 2.0 / 1.9 27.4 / 1.9 2.0 / 1.9 2.0 / 1.9 Struct high 2,134 / 1,316 34.5 / 1.8 2.1 / 48.0 18.1 / 1.8 34.5 / 1.8 18.1 / 1.8 2.1 / 1.8 Table 5: Gate statistics over all 90 runs (9 cells × 10 seeds). Rejections are counted by the first clause of Eq. (1) that fails (improvement is checked first): no significant improvement / regression on C. †No-gate rejections are by the trace check. False adm.: admitted changes that increased true error on HvalH_val. Oracle rej.: rejections of the exactly correct harness. Miss>>10%: runs whose final missed-flag rate exceeds 10%. Max adm.: largest number of admissions in one run. Arm Prop. Adm. Rej.: no gain Rej.: regr. False adm. Oracle rej. Miss>>10% Max adm. Dual loop (gated) 7449 144 5640 1665 0 25 0 4 Local loop only 5315 111 2745 2459 0 2 0 3 Global loop only 2443 25 500 1918 0 28 0 1 Gate on stale labels 8490 0 8490 0 0 470 0 0 No gate (trace check only) 5353 550 4803† – 309 0 49 13 7.1 The gate is what keeps the screen from being loosened Every arm that recovers brings the false-positive rate down to the oracle’s. What separates them is the missed-flag rate, which the earlier presentation did not report. With the gate, the dual loop ends at the oracle’s missed-flag rate in all three families (Table 3; 1.9%, 1.8% and 1.9% against the oracle’s 1.9%, 1.8% and 1.9%), with 0 false admissions over 144 admissions in 90 runs (Table 5). Without it, the same proposers, fed the same windows, admitted 309 changes that increased true error on HvalH_val; the final missed-flag rate exceeded 10% in 49 of 90 runs, reaching a mean of 79.7% in the mid scope cell while its false-positive rate matched the oracle’s. This is the failure referees anticipated: under scope and structural re-interpretations, false positives fall because flagging stops. Under the threshold family the visible-failure check happens to point in the right direction—raising the threshold is the repair—and the no-gate arm’s missed-flag rate stays near the oracle’s. It is not hypothetical here; it is what a loop closed on visible failures does when misses are mostly invisible, and Eq. (1), evaluated on fully labelled history, is what stops it. The no-gate arm also churns: up to 13 admissions in a run against at most 4 for the dual loop, and more excess errors during adaptation than leaving the pipeline degraded (940 against 780 in the mid scope cell). 7.2 Stale labels make the gate refuse every correct change The gate on v1 labels rejected all 8,490 of its candidates, every one for lack of significant improvement—including the exactly correct harness 470 times; every one of the 8,490 also failed the non-regression clause— and left the pipeline at the degraded false-positive rate in every cell. Under the old interpretation a correct repair does not reduce errors; it creates them. The circularity the referee identified is therefore real, and relabelling the pool by the new rule (Section 6.2) is a necessary part of the design, not an implementation choice. 7.3 Division of labour between the loops The ablation separates the loops cleanly by the kind of re-interpretation (Table 4). Parametric (thr) and clause (scope) shifts are repaired by the local loop alone, and the global-only arm, whose library offers only primitive replacements, cannot repair them. The structural shift is the reverse: the local-only arm proposed 1,890 candidates across the three structural cells and admitted 0; 1,890 were rejected for regression, because no threshold on the first-layer stake reproduces a rule on the effective stake; the local-only arm stays at the degraded rate, and the compute-matched local-only arm, with its budget scaled to the dual loop’s proposal count, does no better (27.4% against 27.4% in the mid cell). Repair comes only through primitive replacement. Local-only repair of the scope family is not perfect either: in the mid cell it recovered in 9/10 seeds. The dual loop repairs all three families because escalation routes each problem to the loop that can express its fix. This division of labour is, in part, true by construction—the local edit space cannot express a rule on the effective stake, and the library contains one—so what the experiment adds is that escalation finds the right loop without being told the family, and that the gate rejects the inexpressible local candidates rather than admitting the least-bad of them. The dual loop pays for routing in two ways. It recovers more slowly than the right single loop on the structural family, because the local loop must stall for N cycles before escalating (excess errors 228 against 156 for global-only in the mid cell). And it makes extra proposals after recovery: residual noise still forms clusters, their candidates are proposed and rejected, and some escalate; none of these was admitted. The earlier version of this paper reported this division of labour from a single run on different metrics. A referee suggested it was the cleaner finding; the measured version confirms the structure and replaces the anecdote. 7.4 Recovery speed against a human engineer Under our assumed 2,000-case lag, the manual update accumulates more excess errors during adaptation than the dual loop in every family at mid severity (Table 3), but not everywhere: in the high-severity structural cell, where the dual loop recovers in only half the seeds, it accumulates 500 against the manual update’s 408. The comparison depends entirely on that lag, which is an assumption; its value is to show where a gated automatic loop helps—time to repair—not to claim the automatic loop is better than a person. 7.5 Worked rejection In the mid structural cell, seed 0, the largest visible error cluster after the shift is false positives on formal designations, and at case 16099 the local loop’s first candidate is scope -designation: stop treating a designation as a qualifying hit. It is the kind of edit a loop guided only by visible failures finds attractive, since flags it no longer raises can no longer be reviewed as false. On HselH_sel under v2 labels it removes 287 false positives and introduces 4 among 4,561 true clear cases, a reduction of 6.2 points (95% interval [5.5, 6.9]), so the improvement clause passes. But 864 of the 6,696 cases the deployed harness answered correctly (12.9%) would turn wrong—designated owners above the threshold, now cleared. The record, sequence 0 with hash prefix 0f6978c25079 chained to the genesis value, is written as rejected: regression on C. Threshold edits to the first-layer check fail the same clause, because none expresses a rule on the effective stake. After 21 local rejections the cluster escalates; the global loop’s first 4 candidates, effective-stake checks at thresholds 0.20, 0.40, 0.15, 0.50, are rejected, and at case 17099 it proposes primitive direct_stake_check → effective_stake_check, which reduces false positives by 26.3 points (lower bound 25.0) with 21 regressions on C (0.31%, attributable to execution noise), and is admitted as record 25. Both records, and every other, are in the released results. 7.6 What the tolerance costs The gate has a second failure mode, and we report it. In the high-severity structural cell, replacing the primitive changes the rule’s output on a large share of cases, and the shared execution noise then turns enough of them wrong that the correct replacement exceeds ϵ=0.005ε=0.005. The dual loop recovered in 5/10 seeds, rejecting the exactly correct harness 24 times, and its mean false-positive rate stayed at 18.1% (Table 6). Doubling the tolerance to ϵ=0.010ε=0.010, which we tested on the structural family only, recovers 10/10 seeds without any false admission. A fixed ϵε conflates noise with regression; a tolerance scaled to the expected noise on the cases a candidate changes would separate them, and is the next revision of the rule. Until then, ϵε is a policy parameter: lower values guard more strictly against regression and, as measured here, reject more large correct changes. No harmful change was admitted at either value, so the protective side of the trade-off is not measured by this track. Table 6: Sensitivity to the non-regression tolerance, structural family, dual loop, held-out window. Recovered: seeds whose final harness is exactly the correct one. Oracle rej.: rejections of the exactly correct harness, summed over seeds. Severity ϵε FP % Missed flag % Recovered Oracle rej. False adm. low 0.005 2.0 [1.7, 2.3] 1.9 [1.6, 2.2] 10/10 0 0 low 0.010 2.0 [1.7, 2.3] 1.9 [1.6, 2.2] 10/10 0 0 mid 0.005 2.0 [1.7, 2.3] 1.9 [1.6, 2.2] 10/10 0 0 mid 0.010 2.0 [1.7, 2.3] 1.9 [1.6, 2.2] 10/10 0 0 high 0.005 18.1 [6.0, 30.2] 1.8 [1.5, 2.1] 5/10 24 0 high 0.010 2.1 [1.8, 2.3] 1.8 [1.5, 2.1] 10/10 0 0 7.7 Harness growth A referee asked whether repeated repair accumulates special-case rules. In this track the edit space is parametric, so the harness’s length is bounded (its rendered text ranged from 151 to 253 characters across all runs); what can grow is churn. The gated dual loop made at most 4 admissions in a run, the no-gate arm up to 13. With an LLM proposer that writes free text, length and rule count become the quantities to bound, and the log already records each version’s rendered text for that purpose. 8 Regulatory correspondence The EU AI Act. AI systems intended to evaluate the creditworthiness of natural persons or establish their credit score are high-risk [1, Annex I, point 5(b)]; after the Digital Omnibus on AI the corresponding obligations apply from 2 December 2027 [2]. The provision that matters most for self-evolution is Article 43(4): for high-risk systems that continue to learn, changes pre-determined by the provider at the initial conformity assessment and described in the technical documentation do not constitute a substantial modification, whereas a change not foreseen in that assessment that affects compliance does [1, Arts. 3(23), 43(4)]. A harness-only engine with a fixed edit space, a fixed library and a fixed admission rule is a way of making change pre-determinable: the set of reachable harnesses and the rule that admits them can be written into the technical documentation before deployment. Table 7 maps the remaining obligations. Table 7: Obligation, mechanism, artefact, and the status of the evidence in this paper. This is a design argument, not a compliance opinion. Obligation (EU AI Act) Mechanism Artefact handed to an examiner Evidence here Predetermined changes, Art. 43(4); substantial modification, Art. 3(23) Harness-only evolution over a fixed edit space and library Edit space, library and Eq. (1) in technical documentation Implemented; bound enforced in code Record-keeping, Art. 12 Gate writes before deployment; hash chain Append-only log of admissions and rejections Implemented; 90 logs verified Accuracy and robustness; feedback loops, Art. 15(1),(4) Non-regression on C; held-out HvalH_val; missed-flag rate tracked Per-candidate test result Measured: 0 false admissions (a weak test; see Section 10) Traceability of the model in force Weights pinned by digest Model digest per decision Design only: no model in the simulation Change management in the QMS, Art. 17(1)(a) Gate as the only path to deployment Rejected-candidate records Measured: rejections by reason Human oversight, Art. 14 Rule authored by the compliance function; escalation to people Rule version per decision; escalation records Design only Post-market monitoring, Art. 72 Windowed metrics on reviewed decisions Per-window error series Implemented in simulation Data governance and bias, Art. 10(2)(f)–(g) None None Gap: no protected attributes The US gap. The interagency guidance of 17 April 2026 replaced SR 11-7 and OCC Bulletin 2011-12, the framework our earlier presentation leaned on, and states that generative and agentic AI models are not within its scope; the agencies announced a forthcoming request for information on AI use, including generative and agentic AI [3, 4]. We read this as strengthening rather than weakening the case for the design. There is, as of writing, no US supervisory template for change control in agentic systems, and the guidance that does exist addresses models whose changes are versioned objects. An architecture that turns agentic adaptation back into versioned, tested, recorded changes is a candidate answer to the question the agencies have left open. We claim no more than that: the guidance does not endorse the design, and our mapping is to the EU text. Two further limits belong in this section. Bias monitoring under Article 10 has no mechanism in our design and no evidence in our experiments, because the benchmark carries no protected attributes. And no examiner has reviewed any artefact; “a gate a regulator could accept” is a design goal, not an obtained acceptance. 9 The reference run: deciding without evidence CACE-Bench also measures a failure mode that accuracy-style metrics cannot see. Each case requires several classes of evidence, each resolved through a per-jurisdiction provider chain in which a provider may not respond; when the sources needed for the compliance conclusion did not all answer, escalation is the correct outcome. The silent-decision rate is the share of such undecidable cases on which the pipeline decided anyway. Table 8 reports the reference run (seed 0, 23,000 cases, seven Latin American jurisdictions, provider registry 2026-08-03) with the missed-flag rate now reported alongside false positives, and the range over 5 seeds. Table 8: Reference run, CACE-Bench v0.6.0. Blind: availability-blind single pass. Judge: with the judge-and-recovery layer. Seed 0 with 95% Wilson intervals [17]; denominator n; range over seeds 0–4. Metric (%) Blind Judge n Blind, seeds Judge, seeds Silent decision 55.1 [53.5, 56.7] 6.5 [5.8, 7.4] 3,582 54.3–56.1 6.5–7.4 Compliance FP 22.5 [21.8, 23.2] 5.0 [4.6, 5.3] 14,948 22.2–22.7 4.9–5.1 Missed flag 15.3 [14.3, 16.4] 3.5 [3.0, 4.0] 4,470 14.6–15.3 3.1–3.5 Hallucination 2.6 [2.4, 2.8] 0.6 [0.5, 0.7] 23,000 2.5–2.8 0.4–0.7 Provenance compl. 93.4 [93.1, 93.7] 99.3 [99.2, 99.4] 23,000 93.3–93.8 99.3–99.4 Undecidable cases are 15.6% of the stream (15.6–16.2% across seeds), and an availability-blind pipeline decides 55.1% of them. The judge-and-recovery layer brings that to 6.5%. False positives fall from 22.5% to 5.0%, and missed flags fall too, from 15.3% to 3.5%, so the reduction is not bought by flagging less. The recovery layer, however, is a documented parametric model—each first-pass error is detected with probability 0.85 and corrected with probability 0.92—so this run establishes where the failure concentrates, not how well any particular implementation repairs it. 10 Threats to validity Neither the agent nor the proposer is a language model. The agent is a deterministic rule that applies the harness, with a 2% outcome flip standing in for execution error; the proposer is a seeded search over a stated edit space, and the library contains the correct primitive. The track measures the bound and the gate, not proposal quality; an LLM proposer will produce candidates outside this space, including free-text edits whose regression surface is larger. The interface accepts one, and that is the next experiment. Synthetic throughout. Distributions, shift families and the feedback model (all flags reviewed, 5% of clears audited) are ours. The no-gate failure depends on misses being less visible than false positives; we believe that asymmetry is realistic, but its size is an assumption. Execution noise is modelled as common random numbers. Sharing each case’s noise draw across harness versions makes paired comparison clean and isolates the gate’s response to rule changes. Real LLM noise is not shared across versions, which would raise the noise floor of the regression clause and strengthen the case for a noise-scaled ϵε (Section 7.6). The false-admission audit is weak by construction. HselH_sel and HvalH_val are interleaved halves of the same pre-shift stream labelled by the same deterministic rule, so a rule-level edit that passes on one half will almost always pass on the other. Zero false admissions on HvalH_val is therefore expected for any gated arm; the informative contrasts are against the no-gate arm and on the post-shift held-out window. Baselines not run. We did not run a static prompt-optimisation baseline or direct implementations of the cited self-evolution methods; the no-gate arm stands in for the latter’s admission logic only. One primitive, one shift at a time. Interacting primitives and concurrent shifts are not tested. Label source. The relabelled pool is only as correct as the rule that relabels it. Fairness. No protected attributes; no fairness claim. 11 Conclusion Self-evolution in regulated lending is acceptable only if what evolves is something a supervisor can review. We argued that the runtime harness is that thing and the only thing, built an engine that respects the bound, and measured the control that enforces it. On a synthetic regulatory-shift track the gate admitted 144 of 7,449 candidates with 0 harmful admissions and recovered every low- and mid-severity cell, while the same loops without it admitted 550 changes, 309 of them harmful, and under scope and structural re-interpretations loosened the screen until missed flags exceeded 10% in 49 of 90 runs. The gate needs labels under the new rule, which makes adaptation supervised at the level of the rule; it divides work between a local and a global loop by the kind of re-interpretation; and its fixed tolerance rejects some large correct changes, which a noise-scaled tolerance should fix. The proposer in these experiments is not a language model, and replacing it with one is the experiment that will show whether the bound survives contact with free text. Declarations Availability of data and materials. All code, configuration and per-run outputs, including every gate log, are available in the CACE-Bench repository at https://github.com/rav11l/cace-bench, release v0.6.0, archived on Zenodo under concept DOI 10.5281/zenodo.21394049. The data are fully synthetic; no personal or production data were used. Use of generative AI. A generative AI assistant (Anthropic Claude) was used in preparing this work: to write and debug parts of the simulation and analysis code, to run the experiments, to draft and edit text, and to check the manuscript against the result files. All research questions, the position argued, the experimental design decisions and the final text are the author’s responsibility. Every number in the paper is generated by a script from the released result files rather than typed, and the numbers were independently recomputed from the raw outputs before submission. No generative AI system is a component of the experiments reported: the agent and the proposer in the simulation are deterministic or seeded programs. Competing interests. The author declares no competing interests. Funding. This research received no external funding. Authors’ contributions. R.A. conceived the study, wrote the software, ran the experiments and wrote the manuscript. Preprint. This preprint has not been peer reviewed. References [1] European Parliament and Council. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, L, 12 July 2024. [2] European Parliament and Council. Regulation (EU) 2026/1744 amending Regulation (EU) 2024/1689 as regards the simplification of the implementation of harmonised rules on artificial intelligence (Digital Omnibus on AI). Official Journal of the European Union, L, 24 July 2026. [3] Board of Governors of the Federal Reserve System. SR 26-2: Revised Guidance on Model Risk Management, with attachment “Supervisory Guidance on Model Risk Management” (jointly with the FDIC and OCC), 17 April 2026. [4] Office of the Comptroller of the Currency. Bulletin 2026-13, Model Risk Management: Revised Interagency Guidance, and news release NR-OCC-2026-29, 17 April 2026; Federal Deposit Insurance Corporation, FIL-15-2026, 17 April 2026. [5] H. Zhang et al. Self-Harness: Harnesses that improve themselves. arXiv:2606.09498, 2026. [6] Darwin Agent Team. HarnessX: A composable, adaptive, and evolvable agent harness foundry. arXiv:2606.14249, 2026. [7] L. A. Agrawal et al. GEPA: Reflective prompt evolution can outperform reinforcement learning. arXiv:2507.19457, 2025. [8] C. Fernando, D. Banarse, H. Michalewski, S. Osindero, T. Rocktäschel. Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv:2309.16797, 2023. [9] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, S. Yao. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, 2023. [10] S. Hu, C. Lu, J. Clune. Automated design of agentic systems. In International Conference on Learning Representations, 2025. [11] X. Yin et al. Gödel Agent: A self-referential agent framework for recursive self-improvement. arXiv:2410.04444, 2024. [12] J. Zhang et al. Darwin Gödel Machine: Open-ended evolution of self-improving agents. arXiv:2505.22954, 2025. [13] Y. Wang et al. EvoAgentX: An automated framework for evolving agentic workflows. In Proceedings of EMNLP 2025: System Demonstrations, 2025. arXiv:2507.03616. [14] J. Fang et al. A comprehensive survey of self-evolving AI agents: A new paradigm bridging foundation models and lifelong agentic systems. arXiv:2508.07407, 2025. [15] H. Gao et al. A survey of self-evolving agents: What, when, how, and where to evolve on the path to artificial super intelligence. arXiv:2507.21046, 2025. [16] R. Akhtyamov. A three-level quality-metric system for LLM agents in the credit pipeline. Vestnik Finansovykh Tekhnologiy, 2026. [17] E. B. Wilson. Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22(158):209–212, 1927.