Paper deep dive

Who Verifies the Verifier? Co-Evolving Inspectable Graders with Self-Improving Agents

Xing Zhang, Guanghui Wang, Yanwei Cui, Ziyuan Li, Wei Qiu, Bing Zhu, Peiyang He

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 73

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%

Last extracted: 10/10/2026, 1:39:51 AM

Summary

This paper introduces a method for co-evolving inspectable graders (verifiers) with self-improving agents to address the lack of reliable verification in open-ended tasks. Instead of using hand-written rubrics or bare LLM judges, which are prone to reward hacking and shared blind spots, the authors evolve a verifier composed of deterministic drawback detectors. This verifier is selected based on agreement with a small anchored reference set and consensus on unlabeled outputs, rather than the agent's score. The proposed 'Double Ratchet' system pairs this evolved verifier with a lifecycle-managed skill loop. Experiments on MBPP+, Spider 2.0, and report generation show that the evolved verifier retains 88-110% of the performance lift provided by ground truth or rubrics. A key finding is that downstream task scores cannot certify a self-evolved verifier, as collapsed verifiers (always-pass) can train skills just as well, highlighting the necessity of anchor guards for safety.

Entities (8)

MBPP · datasetbenchmark · 99%Spider 2.0 · datasetbenchmark · 99%Double Ratchet · systemmethod · 98%Ratchet · systemmethod · 95%Reward Hacking · problemphenomenon · 95%Anchor Guards · mechanism · 93%Drawback Detectors · component · 92%LLM Judge · component · 90%

Relation Signals (7)

LLM Judge → suffersfrom → Reward Hacking

confidence 95% · inviting reward hacking and shared blind spots

Double Ratchet → uses → evolved verifier

confidence 95% · Double Ratchet, pairing the verifier with a lifecycle-managed skill loop

Anchor Guards → prevents → collapse to always-pass grader

confidence 93% · removing the anchor guards collapses the verifier into a vacuous always-pass grader

Double Ratchet → retainsliftof → ground truth

confidence 92% · retains 88-110% of the lift that ground truth or a rubric buys the same loop

Double Ratchet → achievesperformanceon → Spider 2.0

confidence 90% · across code generation, enterprise text-to-SQL, and reference-free report generation

Double Ratchet → achievesperformanceon → MBPP

confidence 90% · On MBPP+ it gains +0.21 held-out agreement

Ratchet → iscomponentof → Double Ratchet

confidence 90% · Ratchet [1,22]... is adopted unchanged here as the skill-side component

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:We changed the agent: did it actually get better? Every self-improving agent loop answers this hundreds of times, and every answer comes from a verifier. On open-ended tasks none exists, so the loop is handed a hand-written rubric or a bare LLM judge grading output from a model like itself, inviting reward hacking and shared blind spots. We make the verifier the evolving object: an inspectable expression over small, mostly deterministic drawback detectors, synthesized from clustered failures, gated at birth, and selected for agreement with a ten-item anchored reference set plus consensus over unlabeled outputs, never for the agent's score. On MBPP+ it gains +0.21 held-out agreement over the hand-authored seed composition, on every seed, and ends ahead of the bare LLM judge it contains. One finding should change how co-evolved verifiers are validated: removing the anchor guards collapses the verifier into a vacuous always-pass grader, yet that collapsed verifier trains skills just as well. Downstream task score cannot certify a self-evolved verifier. Score does answer sufficiency, and there an evolved verifier can substitute: Double Ratchet, pairing the verifier with a lifecycle-managed skill loop, retains 88-110% of the lift that ground truth or a rubric buys the same loop, across code generation, enterprise text-to-SQL, and reference-free report generation. When evolved skills gamed the report rubric, an outer judge caught it and one added detector repaired it; the judge itself was wrong until given the task contract.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

69,880 characters extracted from source content.

Expand or collapse full text

Who Verifies the Verifier? Co-Evolving Inspectable Graders with Self-Improving Agents Xing Zhang 1 Guanghui Wang 1 Yanwei Cui 1 Ziyuan Li 2 Wei Qiu 2 Bing Zhu 2 Peiyang He 1∗ 1 AWS Forward Deployed Engineering 2 HSBC Holdings Plc., HSBC Technology Center, China Abstract We changed the agent: did it actually get better? Every self-improving agent loop answers this hundreds of times, and every answer comes from a verifier. On open-ended tasks none exists, so the loop is handed a hand-written rubric or a bare LLM judge grading output from a model like itself, inviting reward hacking and shared blind spots. We make the verifier the evolving object: an inspectable expression over small, mostly deterministic drawback detectors, synthesized from clustered failures, gated at birth, and selected for agreement with a ten-item an- chored reference set plus consensus over unlabeled outputs, never for the agent’s score. On MBPP+ it gains+0.21held-out agreement over the hand-authored seed composition, on every seed, and ends ahead of the bare LLM judge it contains. One finding should change how co-evolved verifiers are validated: removing the anchor guards collapses the verifier into a vacuous always-pass grader, yet that collapsed verifier trains skills just as well. Downstream task score cannot certify a self-evolved verifier. Score does answer sufficiency, and there an evolved verifier can substitute: Double Ratchet, pairing the verifier with a lifecycle-managed skill loop, retains 88–110% of the lift that ground truth or a rubric buys the same loop, across code generation, enterprise text-to-SQL, and reference-free report genera- tion. When evolved skills gamed the report rubric, an outer judge caught it and one added detector repaired it; the judge itself was wrong until given the task contract. 1 Introduction We update an agent’s prompt, add a tool, or change its reasoning strategy: did it actually get better? A self-improving agent asks this every round. The standard recipe wraps a frozen language model in a loop that writes skills, retrieves them, measures their contribution, and retires the ones that hurt [1,2,3], and it consumes one expensive ingredient per round: a verifier that says whether an attempt passed. On code benchmarks that ingredient is free, since unit tests exist. In deployed systems it is the bottleneck, and the industry’s own products document the workaround: Claude’s managed agents have developers hand-write outcome rubrics for a separate grader [4], and Codex’s Record & Replay has a human demonstrate each workflow once [5]. Both are manual substitutes for a missing verifier, supplied one rubric or demonstration at a time. This paper automates that step: can an agent’s verifier be evolved from a handful of anchored examples, drive learning as well as ground truth, and not be gamed in the process? Two clarifications determine how every result below should be read. First, the verifier, or grader, is the function that grades one output, and it is the only thing that evolves; the task sets and their references are fixed before any loop starts, so no result here can come from an easier test. Second, in our target setting ∗ Corresponding author: peiyan@amazon.com 40th Conference on Neural Information Processing Systems (NeurIPS 2026). Workshop: Who Verifies the Agents? Toward Reliable Agent Development. arXiv:2610.11464v1 [cs.AI] 8 Oct 2026 no accurate verifier exists, so there is no baseline verifier to beat: the yardstick is how much of the improvement an accurate verifier would have enabled a system recovers without one. Matching a ground-truth-driven reference loop is the win condition, not a consolation. The design rests on one stance about verification. We rarely know what good is, but given an output we can usually find drawbacks, so a clean verdict means no known drawback was found rather than certified correctness. That framing is what makes bootstrapping possible, and it determines the shape of the artifact (Figure 1): the verifier is an expression over a pool of small typed detectors, each checking one failure class, synthesized from clustered failures, gated at birth, scored by marginal contribution, and retired when useless. Detectors span the three cost tiers a real deployment already has (static checks, execution against a sandbox or a live warehouse, and narrow judge queries), so composing them folds heterogeneous signals into one grader instead of trusting a single scalar. Being mostly deterministic, they fail differently from the model being graded, which is what resists the shared-blind-spot collusion a bare LLM judge invites; being a readable expression, the grader localizes a failure to a nameable check that can be repaired by hand (§5.4). A verifier that grades the loop that produced it is an obvious Goodhart hazard [6,7], so anchor discipline is strict and load-bearing. Selection never asks whether the agent scores well: it asks whether the verifier agrees with a tiny anchored reference set, the only supervised signal verifier evolution reads, and with detector consensus on a large unlabeled set. A separate locked set is read by no verifier loop and serves only for verifier evaluation and skill-side rollback. We test by ablation whether that discipline carries the safety load, and the answer bears directly on how this community should validate co-evolved verifiers at all. Our contributions are as follows. • Verifier evolution, with an objective that needs no downstream metric. We formulate the grading function, rather than the artifact being graded, as the evolving object: anchored search over compositions of atomic drawback detectors under a lifecycle of synthesis, birth gating, marginal-contribution scoring, and retirement. Selection is recall-weighted agreement with a ten-item anchor times reliability-weighted consensus over unlabeled outputs, and the result beats both the hand-authored composition it starts from and the bare LLM judge it contains. •Where the safety lives, and why task score cannot find it. Anchor discipline, not the detector lifecycle, is load-bearing for verifiers, inverting the lesson from skill evolution. Since a collapsed verifier trains skills as well as a valid one, validating a self-evolved verifier by downstream performance is unsound: the most natural validation experiment would certify a grader that passes everything. •Double Ratchet, and a reward-hacking episode end to end. Co-evolving the verifier with a skill loop retains 88–110% of the lift ground truth buys, across three task areas. On the reference-free task we document proxy gaming caught by an outer judge, repaired by one added detector, and an audit of the auditor showing the outer judge needs the task contract. 2 Related work Self-evolving agents assume the verifier. Agents grow skill libraries, revise from failure traces, and bank reusable experience as insights, manuals, memories, or workflows [2,3,8,9,10,11,12]; a recent wave evolves, analyzes, and benchmarks such libraries [13,14,15,16,17,18,19], and two surveys map the landscape [20,21]. Ratchet [1,22] showed that unmanaged LLM-authored libraries drift and that a minimal lifecycle restores the gains; that published loop is adopted unchanged here as the skill-side component and is not a contribution of this work. Other lines search over agent code [23], inherit the architecture-search tradition [24], or optimize prompts and pipelines against a metric [25,26,27,28]. All of them optimize against a verification signal taken as given. We evolve that signal, and ask which part of the discipline carries the safety load once the grader itself moves. Searched objectives are scored by a metric that already exists. Existing work evolves loss functions [29], whole learning algorithms [30], symbolic pruning metrics [31], and program code [32] against supplied evaluators. The search space is analogous, primitive checks composed into a program, but the fitness differs in kind: each is scored by a metric taken as valid, with AlphaEvolve 2 THE PREMISE “We changed the agent. Did it actually get better?” On open-ended tasks there is nothing that can answer this: no oracle verifier no reward model to trust What we do have: 10 anchored examples golden reference each; the only supervision a frozen solver, and an agent loop that needs a grade every round. WHAT WE EVOLVEan inspectable verifier, not an opaque judge the selected expression, verbatim (MBPP+)verdict on one held-out attempt any (disjunction) spec_mismatchjudge op narrow LLM query: does it meet the spec? drawback crashexecution op run in sandbox on probe inputs clean returns_not_print_onlystatic op parse the AST for a returned value clean verifier verdict:FAILand it names the failure class, so the attempt becomes a usable failure capsule. A scalar reward could not have said which requirement was misread. HOW IT GETS THERE sense misses + gaps synthesize a typed detector birth-gate reject 99% compose OR / AND / vote select on the anchor curate retire by merit repeated every round WHAT IT DRIVES agent skill loop adopted unchanged solve frozen solver + routed skills grade the evolved verifier judges capsule failures carry named defects synthesize new skills; retire the rest grades GUARDS: WHERE THE SAFETY LIVES the verifier is optimized to predict an anchor it never trains on, and the measurement never passes through it dev anchor 10 golden refs→ soft labels; the only supervised signal locked test read by no verifier loop; held-out score + rollback outer judge stronger model, outside all loops; audits final outputs pairwise Figure 1: What evolves. Center: the verifier selected on MBPP+, verbatim, as an expression over typed detectors spanning the three cost tiers, with each leaf’s verdict on one held-out attempt. Naming the failure class rather than emitting a scalar is what turns a graded attempt into a usable failure capsule. Left: the premise, no oracle verifier and ten anchored examples. Right: the consumer, a lifecycle-managed skill loop [1] adopted unchanged, whose training the evolved verifier grades. Bottom: the anchor guards. stating outright that progress must be machine-gradable, the precondition our regime lacks. We supply the missing foothold: anchored agreement plus consensus regularization in place of a downstream score. Learned graders trade the assumption for opacity. Reward design is delicate even in classical RL [33]; RLHF reward models are over-optimized by the policies they train and misspecification produces systematic gaming [34,7,35,36,37]; AI-feedback and self-rewarding setups let the model judge itself [38,39]; and LLM judges are widely audited for bias [40,41,42,43]. We share the reward-hacking hazard but not the structure: our grader is a composition of typed detectors optimized to predict an anchor it never trains on, so it can be read and repaired by hand, and its agent-facing use is confined to training. Agent-as-judge co-evolution [44] is closest in spirit and validates the judge through the system it grades, which §5.2 shows a vacuous grader also passes. The evaluators nearest our tasks fix in advance what we synthesize, gate, and retire: CodeScore [45] predicts functional correctness from the execution-labeled supervision we take to be missing, and SQLHD [46] needs no ground truth but fixes a hand-authored family of metamorphic relations [47]. Either could be one typed detector in our pool, so we read them as components rather than competitors, while agentic text-to-SQL pipelines [48] improve the solver, an orthogonal axis. We use MBPP+ [49,50] and Spider 2.0 [51] as hard anchors for certifying evolved verifiers, in the execution-graded tradition of HumanEval [52], not as leaderboard entries. 3 The verifier and its evolution Let a task family come with taskstand a frozen solver mappingtto an outputy. A verifierM maps(t,y)to a verdict inPASS, FAIL, ABSTAIN, andMis what we search over, with tasks and references held fixed. Three disjoint splits play the usual roles with one twist each. train is large and unlabeled: solver outputs here expose where candidate verifiers disagree or abstain, and in co-evolution the evolved verifier grades these tasks for the skill loop. dev is tiny and anchored, ten items throughout: each task carries a golden reference against which a teacher emits soft pass/fail labels, the only supervised signal verifier evolution reads. Those references grade only the ten tasks that carry them, while the evolved verifier must grade arbitrary outputs on tasks that have none. Dev is deliberately small because sparse anchoring is the realistic regime: fresh labels in deployed systems 3 arrive as a trickle of demonstrations and incident reviews. test is locked against the verifier: it holds the strongest available reference (unit tests, official execution comparison, the report rubric) and no part of verifier evolution reads it, so every validity number below is fully out of sample. The skill loop alone touches it, for held-out scoring and rollback, as specified by Ratchet [1] and kept unchanged because a deployed agent rolls back against the cases it is judged on. The reference and co loops roll back identically, so the retention comparison that carries our claim stays clean, and rollback fired in only 23 of their 2,900 rounds, suggesting limited direct exposure; absolute skill-side levels are nonetheless optimistic against a never-touched third split. The naive ablation switches rollback off, so its skill side is less protected, not more. 3.1 Verifiers as compositions of drawback detectors An op is an atomic drawback detectoro(t,y,c)→DRAWBACK, CLEAN, ABSTAINwithca shared context, checking exactly one failure class and abstaining otherwise. Each task family starts from a handful covering its obvious failure modes, cheap to write by hand or have an LLM draft from the task spec, and the lifecycle grows the pool from there. Ops come in three cost tiers: static ops parse and inspect the artifact; execution ops run it, against a sandbox for code or a live warehouse for SQL, so the environment itself supplies part of the ground truth; judge ops ask an LLM one narrow question. A fixed non-evolving root per family validates basic structure and caches the parsed artifact the other ops read. A verifier expression composes op verdicts with logical operators: a node is a leaf op, a disjunction (a drawback if any child finds one), a conjunction (only if all children agree), a negation, an unweighted K-of-kvote, or a weighted vote against a threshold (Appendix C gives the grammar). Abstaining children are excluded from their combinator, a node whose children all abstain abstains itself, and the output passes when the fixed root conjoined with the evolved expression finds no drawback. Ops are pure functions of(t,y), so verdicts are cached and evaluation reduces to table lookups. Negation and weighted votes stay available every round but appear in none of the selected verifiers, so every expression we report reads as a plain drawback disjunction or vote, fully reproducible from its expression string plus the registered op pool. 3.2 The verifier loop Algorithm 1 sketches one round. Two signals feed synthesis: misses, dev items the incumbent passes although their soft label is fail, and gaps, train outputs on which the pool abstains entirely or splits. Both are clustered by failure class under a closed task-specific taxonomy, and a synthesizer authors a typed op spec for any recurring cluster, admitted only if it passes the birth gate. Ops born from anchored misses enter active and are selectable, their birth evidence already tying them to the anchor; ops born only from unlabeled gaps enter as shadows, recording verdicts every round but unselectable until their verdicts demonstrably raise dev agreement, since nothing yet shows they track quality rather than consensus noise. Candidates then come from an LLM composer that sees the op catalog, the current gaps, and an elite history of the best expressions with their shortcomings, plus mutations and crossovers of those elites. Selection keeps the best-so-far candidate by S(e) = A dev (e)· A train (e) w − λC(e),(1) whereA dev (e)is agreement with the dev soft labels,A train (e)agreement with the consensus verdict of the opining pool ops on unlabeled train outputs,C(e)expression size, andw,λsmall constants. Both agreement terms are weighted, and each weighting is a safety choice rather than a tuning knob (constants in Appendix C). Dev agreement is recall-weighted, a missed drawback costing twice a false alarm, so no candidate reaches a perfect score by acing the easier class alone. Consensus is reliability-weighted by each op’s anchored leave-one-out marginal, so anchor-proven ops lead the vote; otherwise a bloc of correlated detectors, cheap to synthesize and liable to share a blind spot, would define the very pseudo-label the regularizer rewards agreeing with.A train is thus a transfer regularizer pulled toward the anchor rather than a popularity vote, and entering multiplicatively against a fail-closedA dev it never selects a verifier on its own: a verifier that memorizes ten dev items but behaves erratically loses, and a degenerate always-fail verifier loses on dev. Two guards apply: selection fails closed, so a candidate with no usable dev opinion is unselectable, and a validity gate drops candidates that pass, fail, or abstain on everything. Per-op fitness is the leave-one-out marginal of the incumbent expression, and non-positive ops retire after a grace period. 4 Algorithm 1 One round of verifier evolution 1: Sense (where is the verifier wrong?) 2:misses← dev items the incumbent passes but the soft label fails 3:gaps← train outputs where the pool abstains or splits 4: Grow (expand the detector pool) 5:cluster misses∪ gaps by failure class; author one typed op per recurring cluster 6: birth gate: fires on≥half its cluster and clean on known-good⇒admit (active if anchored, else shadow) 7: Select (recombine, keep the best) 8:candidates← LLM compositions over the pool, plus mutations and crossovers of elites 9:keep arg max e S(e) over candidates and the incumbent, behind the validity gate (Eq. 1) 10: Curate (merit-based lifecycle) 11:retire ops with non-positive leave-one-out marginal; promote shadows that raise dev agreement 12: Audit (measurement only, never a training signal) 13:report agreement against the locked test set 3.3 Co-evolution with the skill loop: Double Ratchet The Ratchet skill loop [1], adopted here unchanged, needs the one thing this setting does not supply: a grader for its training tasks. Co-evolution hands it the current best evolved verifier, and we call the paired system Double Ratchet. The two loops alternate under a fixed front-loaded curriculum, verifier phases longest early when coverage is worst and shrinking as the verifier matures. In the skill phases the evolved verifier grades train attempts, and failed attempts become capsules whose error text feeds skill synthesis. The skill loop’s held-out evaluation and rollback anchor stay pinned to the locked test set, which the evolved verifier never reads, so a corrupted verifier can slow skill learning but cannot redefine the evaluation metric. The comparison that matters is the reference skill loop, identical machinery handed the ground-truth verifier (or best available rubric) for free. Two ablations, one per safety ingredient. The naive arm disables the two anchor guards (fail- closed anchoring and the validity gate), plus skill-loop rollback on the skill side, with the detector lifecycle fully on, the collusion-prone configuration that unanchored agent-judge co-evolution implic- itly runs. The no-lifecycle arm does the opposite, keeping the guards but disabling the birth gate, shadow tier, and merit-based retirement, so every synthesized detector enters active and nothing retires: the verifier-side analog of an unmanaged skill library. An outer audit. Where no accurate verifier exists, rubric scores are references and not truth, so a final audit sits outside all loops: a stronger LLM compares each final output pairwise against the pre-evolution baseline, judged twice with positions swapped, a win counting only when both orders agree (Appendix D). This mimics the human acceptance step a real deployment keeps, and it exposed a proxy-gaming failure the loops could not see. It is an audit, never a training signal. 4 Experimental setup Task areas.MBPP+ [49,50], Python function synthesis with hidden unit tests as the hard anchor, and Spider 2.0-Snow [51], enterprise text-to-SQL over real Snowflake warehouses anchored by official execution-result comparison, are the validation tasks: their exact ground truth makes both verifier validity and the reference-loop match measurable. Report generation is the target task, a deployment-style workload in which a pipeline writes analyst-report sections from audited evidence cards. No golden verifier exists for it; the available rubric score plus golden demonstration sections is a partial reference signal, not ground truth. Task cards are in Appendix A. Hard subsets and splits. For MBPP+ and Spider 2.0 we screen every task with five independent frozen-solver samples, drop those passed on every sample (no lift room), and stratify the rest by screen pass rate. Report tasks derive from golden demonstrations and need no screening. Dev always holds ten items, five clear failures and five clean positives. The solver is a single direct LLM call with no tools, iteration, or scaffolding, so absolute numbers are not comparable to leaderboard agents. 5 Appendix B gives the splits, the anchors, and the Spider prompt grounding, a large shared effect applied identically to every arm. Roles and models.All experiments use Claude models: every in-loop role (solver, skill synthesizers, teachers, judge ops) is Claude Opus 4.7, the sole exception being the report task’s final judge, the stronger Claude Opus 4.8 kept outside all loops. A same-family teacher is not a circular self-grader: it compares candidate outputs against golden references on dev rather than judging unaided, and never sees the locked anchor. Skill loops run 100 rounds; co-evolution interleaves verifier phases of 15/8/5/2 rounds between four 25-round skill phases, matching the reference loop’s skill budget. Every run is seeded and configuration-driven (Appendix G). 5 Results Two axes organize the evidence, taken in order. First verifier validity, agreement with a locked reference no part of verifier evolution reads, which is the claim the guard ablations then localize. Then task score, which we show cannot certify a verifier and therefore use only to ask whether an evolved one suffices in place of ground truth. Each run uses three seeds, reported as mean and sample standard deviation, with loop comparisons adding the seed-level difference and a 95% bootstrap interval. Those means are coarse, so validity claims are also tested paired over matched per-item verdicts: evolution rescores the same locked items, two rounds therefore differ only in the grader, and the test asks about that change of grader rather than any task-level population effect. Those matched pairs are clustered, three seeds over the same locked tasks, so exact McNemar over pooled pairs would be anti-conservative and the reported test is a sign-flip permutation at the task level. Per-round noise runs near five points, so we report peak and final-rounds values (mean of the last ten rounds) against each run’s round-0 baseline; peak is a locked-set argmax and therefore optimistic, while the final-rounds column is fixed in advance and unselected, which is why the paired test is run at both. 5.1 The evolved verifier beats the engineering it starts from On MBPP+ the evolved composition reaches0.625± 0.050held-out agreement against0.417± 0.058 for the hand-authored seed composition it starts from, so the+0.21is lift over static engineering rather than over nothing. It also beats the bare LLM judge op it contains (0.55± 0.04), the natural baseline for a grader built without ground truth, and unlike that judge it is inspectable and diagnosable. The paired test makes the lift consistent but not significant: across 120 matched verdicts, evolution corrects 61 items against 36 broken at peak and 36 against 20 at the final rounds, with every seed net positive, yet clustering those verdicts by task leavesp = 0.12at peak andp = 0.21at the final rounds. Pooling them as independent would reportp = 0.014andp = 0.044: with 40 locked tasks and three seeds the design supports a consistent direction, not a calibrated rejection. Figure 2 separates two quantities that are easy to confuse. What selection optimizes rises on every seed of every task and never degrades afterwards, which best-so-far selection guarantees and which is therefore no evidence of validity; only transfer to the locked set that selection never sees is informative. Transfer is also not monotone even though the objective is, since five of nine runs end below their own mid-run peak, the ordinary generalization gap of optimizing a ten-item proxy. The other two tasks are not replications of the MBPP+ claim. Report generation starts at ceiling, because subset seeding already places a near-optimal composition in the round-0 candidate set, so it carries sufficiency and the safety ablations rather than lift. Spider sits near chance (0.500± 0.026) despite a robust objective climb, and the paired test finds nothing there; §5.5 shows this is that task’s failure regime rather than a sample-size artifact, and §5.3 why imperfect agreement does not doom the system. The lifecycle does real work behind small expressions. Each pool starts from 9 to 11 hand-authored seed detectors, gap-driven synthesis then proposes continuously, and the birth gate rejects the overwhelming majority, leaving a working pool of a dozen to a few dozen from which anchored selection keeps one to three leaves. One Spider detector shows the whole path: the loop clustered train outputs that returned every column although the question named specific ones, authored the typed specforbids_select_star, admitted it as a shadow because its birth evidence was an unlabeled gap, promoted it after composing it raised dev agreement from 0.71 to 0.81, and then selected it in place of the seed pool’s grouping check (Appendix E). Nor is the ten-item anchor fragile: replaying 6 MBPP+mechanically checkable failures what selection optimizes 0.150.48 70 rounds rises on every seed of every task what transfers (held-out) 0.4 0.5 0.6 0.7 0.8 0.9 chance round 0 by hand 0.42 evolved composed 0.62 0.55 judge contained birth gate:10 seed›997-1014 rejected›12-14 pool›3 leaves held-out transfer: +0.21 Spider 2.0-Snowwrong values, clean execution what selection optimizes 0.600.72 70 rounds rises on every seed of every task what transfers (held-out) 0.4 0.5 0.6 0.7 0.8 0.9 chance round 0 by hand 0.49 evolved composed 0.50 0.45 judge contained birth gate:9 seed›612-785 rejected›33-39 pool›1-2 leaves held-out transfer: flat, near chance Report generationclosed, checkable taxonomy what selection optimizes 0.760.86 70 rounds rises on every seed of every task what transfers (held-out) 0.4 0.5 0.6 0.7 0.8 0.9 chance round 0 by hand 0.84 evolved composed 0.84 judge n/a birth gate:11 seed›172-188 rejected›22-26 pool›2 leaves held-out transfer: at ceiling from round 0 Figure 2: Verifier validity, separating what selection sees from what it does not. Left of each panel: the selection objective over the first 70 of 100 rounds (three-seed mean). Right: held-out agreement with the locked anchor, from the hand-authored round-0 composition to the evolved peak (mean± std, three seeds), against the bare LLM judge op contained in the composition and the chance line. Bottom: the birth-gate funnel, three-seed range. Verifier validity (report generation) the collapse is plainly visible here anchored guards on, lifecycle on objective 0.865 held-out agr. 0.830 train pass 0.78-0.82 naive anchor guards OFF objective 1.000 held-out agr. fail-open: unmeasurable train pass 0.97-1.00 no-lifecycle lifecycle OFF objective 0.896 held-out agr. 0.868 train pass not recorded naive reports a perfect objective while passing 0.97-1.00 of everything it grades: a vacuous always-pass grader. Removing the detector lifecycle instead leaves validity inside the anchored band. Anchor discipline carries the load. Held-out task score and here it is invisible MBPP+0.70-0.74 Spider0.46-0.48 Report0.81-0.85 0.40.50.60.70.80.9 reference loopDouble Ratchetnaive (vacuous verifier) The vacuous verifier scores as well as the anchored one, and on two tasks better: unfiltered practice still improves skills. So downstream score cannot validate a self-evolved verifier. Figure 3: The same three arms, measured two ways. Left: verifier validity on report generation, three seeds. The naive arm claims a perfect objective while passing 0.97–1.00 of everything, so its held-out agreement is fail-open and unmeasurable rather than merely low. Right: held-out task score for the reference loop, Double Ratchet, and the naive arm, which does not separate the three on any task. report selection against anchors of four, six, and eight items does not lower the mean held-out agreement and only widens its spread (Appendix F). 5.2 Which guard carries the load, and why task score cannot tell The ablations are judged on verifier validity, not held-out task score, and Figure 3 shows why the distinction is not pedantic. On report generation, where the anchor is softest and collusion pressure highest, disabling the anchor guards collapses the verifier on three of three seeds: selection latches onto a detector that almost never fires, fail-open scoring reports a vacuous perfect objective, and the verifier passes 0.97–1.00 of everything it grades. Disabling the lifecycle collapses nothing: objectives and held-out agreement stay inside the anchored band, and although the pool grows, the extra detectors are inert, since one that is never selected never grades anything. This inverts the emphasis of the skill-evolution literature. For skills, lifecycle management is the key finding; for verifiers under anchored selection, anchor discipline carries the safety load, so the verifier-side analog of library drift is anchor drift, not pool drift. The asymmetry is structural: a junk skill gets routed into prompts and actively hurts, while a junk detector matters only if selection picks it, which the anchor prevents. The lifecycle still buys efficiency, and we keep it. The collapsed arm carries a warning that outlives this paper. Its verifier is worthless, yet its co- loop scores as well as the anchored one and on two tasks slightly better (Figure 3, right; Table 1). Neither number is wrong: an always-pass verifier reduces training to unfiltered practice, which on its own improves skills, and the measurement never passes through the verifier, so the task score survives what the grader does not. Downstream task performance therefore cannot validate a self-evolved verifier, and the experiment this literature reaches for first would here have certified a 7 Table 1: Held-out task score over three seeds on 40–48-item locked sets. reference: the same skill loop handed ground truth (MBPP+, Spider) or the best available rubric (Report) for free. co: Double Ratchet on its own evolved verifier. naive † : the ablation with the anchor guards and skill-side rollback off, seed-matched to co; its verifier is vacuous (Table 3) yet it scores as well as co, which is why task score cannot certify a grader. Lift ret.: co’s peak lift over its round-0 baseline as a fraction of the reference loop’s. Verifier agr.: peak agreement with the locked anchor. Improved: never-passed tasks solved at least once, or for Report the fraction of sections ending above round 0. Peak held-out scorelift∆peakverifierImproved Taskreferenceconaive † ret.(95% CI)agr.referenceco MBPP+0.700±.0250.717±.0380.742±.014106%+.02 [−.03,.06]0.6216/2319/23 Spider0.483±.0380.458±.0380.458±.029110%−.03 [−.08,.03]0.504/126/12 Report0.850±.0100.812±.0060.841±.00388%−.04 [−.05,−.03]0.8499%99% grader that passes everything. Validating the verifier requires a held-out reference it never reads, and any deployment that triages or gates on the verifier needs one anyway. 5.3 Sufficiency: an evolved verifier substitutes for ground truth With validity measured independently, the remaining question is sufficiency: can a verifier evolved from ten reference examples drive a skill loop that ground truth would otherwise drive? It can substitute, which is a weaker claim than causing the gain: §5.2 already showed retained lift does not isolate verifier accuracy. Double Ratchet retains 106%, 110%, and 88% of the reference loop’s lift on MBPP+, Spider, and Report (Table 1; curves in Figure 4): within noise on both ground-truth tasks, where the∆peak intervals contain zero, and a small gap on the report task, whose reference is itself a partial rubric. That is why the independent judge in §5.4 is the more informative read there. Aggregates hide a task-level result. Both loops solve held-out problems the frozen solver never solves, and on both hard-anchor tasks Double Ratchet converts more of them than the ground-truth-driven loop (Table 1, Improved). That settles how good a verifier must be to suffice here: not very. Spider’s verifier agrees with ground truth only at0.500± 0.026yet its co-loop retains the full reference lift, because the verifier’s role in training is directional, deciding which attempts become failure capsules, whose concrete error text does not depend on the pass/fail label being right. The verifier-graded training pass rate stays in a realistic band (0.78–0.82 on Report, 0.41–0.83 on Spider), so the verifier is really grading. The reference loop and the co-loop, which never see each other, independently synthesize functionally matched skills on every task (Appendix E). The honest reading is that a skill loop is a forgiving consumer of grades, and Spider’s smallest absolute lift for both loops reflects one ceiling binding them alike: a single-call solver’s residual failures mostly need live exploration that no guidance-text skill expresses. So the reasons to build a verifier well lie elsewhere, in triage, gating, and repairability, the last of which the next subsection puts under load. 5.4 A reward-hacking episode, and auditing the auditor The report task produced this paper’s clearest evidence that an inspectable verifier is worth having. The rubric’s metric-discipline dimension counts inline evidence tags, and the first skill runs raised the rubric score by+0.26partly by gaming exactly that: evolved skills wrote the tag in place of the number, leaving about 30% of tags at peak rounds with no value beside them, and invented confident forecasts to satisfy a style dimension. This is textbook proxy gaming, and no in-loop signal saw it. The independent judge did, preferring the pre-evolution baseline in 88% of decided pairs and describing unrendered placeholders and fabricated conviction. The repair was one coupled change: a vocabulary-aware value-erasure check added to the capsule gate, where a tag whose registered metric has a numeric value but no numeral nearby is a defect, plus rewritten failure hints that teach value-plus-tag, the hints being how a detector’s verdict reaches skill synthesis. We claim the pair rather than the detector alone, since separating them needs reruns we did not do. The rerun cut erased tags from roughly 30% to about 1% while the rubric score rose to 8 Table 2: Final-judge win rate of evolved report outputs over their own pre-evolution baselines, over decided pairs; each pair is double-judged with positions swapped and a win requires both orders to agree (protocol and counts in Appendix D). Rows isolate the one-detector erasure fix, columns the judge’s convention-blindness: only the task-aware judge registers the fix (+0.255). Outer judgeeffect of Report loop stategenerictask-awarethe rubricWhat the row shows pre-repair (proxy gamed)0.1220.515+0.393only the aware judge sees past the syntax post-repair (erasure fixed)0.1260.770+0.644content gain the generic judge cannot see effect of the fix+0.004+0.255the generic judge penalizes the required format 0.850± 0.010, trading nothing away. Reading the verifier is what localized the defect to a nameable check, an intervention a learned scalar does not admit. The audit then needed auditing, a concrete instance of the scalable-oversight problem. The pipeline requires raw evidence tags in its output, a format contract the original generic rubric did not know, so its loss rationales blamed that syntax and the repair barely moved the win rate, from 0.122 to 0.126. A task-aware rubric stating the contract, while still treating valueless tags and fabricated figures as defects, re-judged the same stored pairs. Table 2 separates the two layers: down the task-aware column the erasure fix moves the win rate from 0.515 to 0.770, a content change the contract makes visible, while each row’s generic-to-task-aware jump measures the generic judge’s convention-blindness. That rubric was written after reading the generic judge’s rationales, so it diagnoses the audit rather than independently confirming the repair; its pre-repair cell is the control. The lesson runs both directions: self-evolved verifiers need an independent outer audit, and the audit needs the task contract. 5.5 The detectability spectrum One variable organizes every result above: how mechanically detectable the solver’s failures are. MBPP+ failures are crashes and wrong outputs that cheap detectors see directly, and held-out agreement lifts+0.21. Report generation derives its anchor from golden demonstrations through a closed, checkable failure taxonomy, and transfer reaches 0.84. Spider 2.0 gives an in-task controlled contrast: under our first prompt most failures were compile errors and the evolved verifier reached 0.85 held-out agreement. Grounding the prompt with typed schemas then raised the solver’s baseline by half (0.211 to 0.329) and left residual failures that are semantically wrong values under clean execution, invisible to deterministic detectors, so agreement fell to0.500± 0.026even as the objective still climbed. Same method, same task, different failure distribution. That regime shifts the burden onto judge ops and human audits, and sharper solvers leave more of it: verifier evolution buys the most where failures are checkable, a scoping condition rather than an implementation limit. 6 Discussion and conclusion Evolution expands coverage but never creates ground truth. With no anchor the loop optimizes noise, so anchor quality matters as much as its existence, and hardening soft anchors into checkable ones pays most. The open frontier is outputs that execute cleanly yet are semantically wrong. This is a mechanism study, not a scaling result: three task families, one model family, rounds in the low hundreds, held-out sets of tens of items, and one stronger model as outer judge rather than a panel or human study. An agent lacking a verifier can evolve one under the discipline that stabilizes its skills, with safety engineered rather than assumed. Three commitments carried our results. Anchor what you cannot manufacture: tie selection to a small reference set the loop cannot edit, never to the agent’s own score. Keep the verifier legible: mostly deterministic detectors fail differently from the model they grade, and one named check repaired a live case of proxy gaming. Audit the auditor: an outer judge is necessary but not sufficient, and ours was wrong until given the task contract. Never validate a self-evolved verifier by the agent’s performance: a grader that passes everything passes that test too. Who verifies the verifier? An anchor it predicts but never sees, and a judge outside every loop. 9 References [1] Xing Zhang, Yanwei Cui, Guanghui Wang, Ziyuan Li, Wei Qiu, Bing Zhu, and Peiyang He. Ratchet: A minimal hygiene recipe for self-evolving LLM agents. arXiv preprint arXiv:2605.22148, 2026. [2] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research, 2024. [3] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Re- flexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, 2023. [4] Anthropic. New in claude managed agents: Dreaming, outcomes, and multiagent orches- tration.https://claude.com/blog/new-in-claude-managed-agents, 2026. Product announcement, May 19, 2026. [5]OpenAI.Codex:Record & replay.https://developers.openai.com/codex/ record-and-replay, 2026. Developer documentation. [6] Charles A. E. Goodhart. Problems of monetary management: The UK experience. Monetary Theory and Practice: The UK Experience, pages 91–121, 1984. [7] Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. In Proceedings of the 40th International Conference on Machine Learning (ICML), pages 10835–10866, 2023. [8] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36, 2023. [9]Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL: LLM agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, 2024. [10]Minghao Chen, Yihang Li, Yanting Yang, Shiyu Yu, Binbin Lin, and Xiaofei He. AutoManual: Generating instruction manuals by LLM agents via interactive environmental learning. In Advances in Neural Information Processing Systems, volume 37, 2024. [11]Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 2023. [12] Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory. arXiv preprint arXiv:2409.07429, 2024. [13] Rong Wu, Xiaoman Wang, Jianbiao Mei, Pinlong Cai, Daocheng Fu, Cheng Yang, Licheng Wen, Xuemeng Yang, Yufan Shen, Yuxin Wang, et al. Self-evolving LLM agents through an experience-driven lifecycle. arXiv preprint arXiv:2510.16079, 2025. [14] Yutao Yang, Junsong Li, Qianjun Pan, Bihao Zhan, Yuxuan Cai, Lin Du, Jie Zhou, Kai Chen, Qin Chen, Xin Li, et al. AutoSkill: Experience-driven lifelong learning via skill self-evolution. arXiv preprint arXiv:2603.01145, 2026. [15]Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, Kaide Zeng, Yu Wang, Siwei Han, Yiyang Zhou, Xujiang Zhao, Haifeng Chen, et al. SkillRL: Evolving agents via recursive skill-augmented reinforcement learning. arXiv preprint arXiv:2602.08234, 2026. [16] Jingwei Ni, Yihao Liu, Xinpeng Liu, Yutao Sun, Mengyu Zhou, Pengyu Cheng, Dexin Wang, Xiaoxi Jiang, and Guanjun Jiang. Trace2Skill: Parallel inductive skill distillation for LLM agents. arXiv preprint arXiv:2603.25158, 2026. 10 [17]Junjie Wang, Yiming Ren, and Haoyang Zhang. From procedural skills to strategy genes: Towards experience-driven test-time evolution. arXiv preprint arXiv:2604.15097, 2026. [18] Xing Zhang, Guanghui Wang, Yanwei Cui, Wei Qiu, Ziyuan Li, Bing Zhu, and Peiyang He. Experience compression spectrum: Unifying memory, skills, and rules in LLM agents. arXiv preprint arXiv:2604.15877, 2026. [19]Xiangyi Li, Wenbo Chen, Yimin Liu, Shenghan Zheng, Xiaokun Chen, Yifeng He, Yubo Li, Bingran You, Haotian Shen, Jiankai Sun, et al. SkillsBench: Benchmarking how well agent skills work across diverse tasks. arXiv preprint arXiv:2602.12670, 2026. [20]Jinyuan Zhang, Changzhou Qu, Junyi Fang, et al. A survey of self-evolving agents: On path to artificial super intelligence. arXiv preprint arXiv:2507.21046, 2025. [21] Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. A com- prehensive survey of LLM-based autonomous agents. Frontiers of Computer Science, 18(6), 2024. [22] Xing Zhang, Yanwei Cui, Guanghui Wang, Ziyuan Li, Wei Qiu, Bing Zhu, and Peiyang He. Library drift: Diagnosing and fixing a silent failure mode in self-evolving LLM skill libraries. In ICML 2026 Workshop on Failure Modes in Agentic AI, 2026. [23]Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems. In International Conference on Learning Representations (ICLR), 2025. [24]Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey. Journal of Machine Learning Research, 20(55):1–21, 2019. [25]Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy: Compiling declarative language model calls into self-improving pipelines. In International Conference on Learning Representations (ICLR), 2024. [26]Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. In International Conference on Learning Representations (ICLR), 2024. [27]Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rock- täschel. Promptbreeder: Self-referential self-improvement via prompt evolution. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. [28]Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. TextGrad: Automatic “differentiation” via text. arXiv preprint arXiv:2406.07496, 2024. [29]Peidong Liu, Gengwei Zhang, Bochao Wang, Hang Xu, Xiaodan Liang, Yong Jiang, and Zhenguo Li. Loss function discovery for object detection via convergence-simulation driven search. In International Conference on Learning Representations (ICLR), 2021. [30]Esteban Real, Chen Liang, David So, and Quoc Le. AutoML-Zero: Evolving machine learning algorithms from scratch. In Proceedings of the 37th International Conference on Machine Learning (ICML), pages 8007–8019, 2020. [31]Peijie Dong, Lujun Li, Zhenheng Tang, Xiang Liu, Xinglin Pan, Qiang Wang, and Xiaowen Chu. Pruner-zero: Evolving symbolic pruning metric from scratch for large language models. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. [32]Alexander Novikov, Ngân V ̃ u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131, 2025. 11 [33]Andrew Y. Ng, Daishi Harada, and Stuart Russell. Policy invariance under reward transfor- mations: Theory and application to reward shaping. In Proceedings of the 16th International Conference on Machine Learning (ICML), 1999. [34]Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022. [35] Alexander Pan, Kush Bhatia, and Jacob Steinhardt. The effects of reward misspecification: Mapping and mitigating misaligned models. In International Conference on Learning Repre- sentations (ICLR), 2022. [36] Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward gaming. Advances in Neural Information Processing Systems, 35:9460– 9471, 2022. [37]Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in AI safety. arXiv preprint arXiv:1606.06565, 2016. [38]Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073, 2022. [39]Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. Self-rewarding language models. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. [40]Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-judge with MT-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2023. [41]Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. AlpacaFarm: A simulation framework for methods that learn from human feedback. Advances in Neural Information Processing Systems, 36, 2023. [42]Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. A survey on LLM-as-a-judge. arXiv preprint arXiv:2411.15594, 2024. [43]Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. Humans or LLMs as the judge? a study on judgement bias. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024. [44] Mingchen Zhuge, Changsheng Zhao, Dylan Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, Yangyang Shi, Vikas Chandra, and Jürgen Schmidhuber. Agent-as-a-judge: Evaluate agents with agents. arXiv preprint arXiv:2410.10934, 2024. [45] Yihong Dong, Jiazheng Ding, Xue Jiang, Ge Li, Zhuo Li, and Zhi Jin. CodeScore: Evaluating code generation by learning code execution. ACM Transactions on Software Engineering and Methodology, 2024. [46]Bo Yang, Yinfen Xia, Weisong Sun, and Yang Liu. Hallucination detection for LLM-based text-to-SQL generation via two-stage metamorphic testing. arXiv preprint arXiv:2512.22250, 2025. [47]Tsong Yueh Chen, Shing Chi Cheung, and Shiu Ming Yiu. Metamorphic testing: A new approach for generating next test cases. Technical Report HKUST-CS98-01, Hong Kong University of Science and Technology, 1998. 12 [48]Minghang Deng, Ashwin Ramachandran, Canwen Xu, Lanxiang Hu, Zhewei Yao, Anupam Datta, and Hao Zhang. ReFoRCE: A text-to-SQL agent with self-refinement, consensus enforcement, and column exploration. arXiv preprint arXiv:2502.00675, 2025. [49]Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. [50]Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by ChatGPT really correct? rigorous evaluation of large language models for code generation. In Advances in Neural Information Processing Systems, volume 36, 2023. [51]Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, Victor Zhong, Caiming Xiong, Ruoxi Sun, Qian Liu, Sida Wang, and Tao Yu. Spider 2.0: Evaluating language models on real-world enterprise text-to-SQL workflows. In International Conference on Learning Representations (ICLR), 2025. [52]Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. A Task descriptions MBPP+ (validation task; anchor: hidden unit tests). A one-paragraph natural-language pro- gramming problem plus one example call; the solver writes a Python function. The anchor is the EvalPlus hidden test suite [50], which adds many more tests per problem than the original MBPP and repairs faulty ones, giving reliable pass/fail. Typical failures are crashes, mishandled edge cases, printing instead of returning, and misread specifications, which are the failure classes the detector pool checks. Spider 2.0-Snow (validation task; anchor: execution match). An analyst question over a real hosted Snowflake warehouse: production-scale databases with hundreds of tables, cryptic columns, sharded table families, and dialect-specific SQL. The solver writes one query from the question, a rendered schema catalog, and optional domain notes. The anchor executes the candidate and compares the fetched table to the stored gold result with the official value-based comparator, ignoring column names and order. Typical failures are unresolvable identifiers, type errors on date-like columns, and, hardest, queries that run cleanly but compute the wrong thing. Our single-call solver passes about a third of screened tasks. Report generation (target task; no accurate verifier exists). Write one section of an analyst report (for example, “value-chain map: control points from ASIC to optical engine”) in markdown, given a section title and purpose, a style contract fixing units, reference dates, and accounting rules, a set of audited evidence cards, and a metric vocabulary registering the section’s key figures. An evidence card carries an id, a one-sentence claim, a verbatim source quote, source attribution and date, a stance, and an auditor’s credibility verdict. The section must ground every claim in the cards, state each key figure with the inline tag binding it to the metric vocabulary, hedge weak or disputed evidence, and include required structure such as a summary line. No verifier exists because section quality is holistic: even an expert author cannot reduce “a good section” to a checkable rule. The rubric scores lexical and structural proxies of the requirements above and is treated as reference signal only. B Setup details Splits and screening.For MBPP+ and Spider 2.0 we screen every task with five independent sam- ples of the frozen single-call solver, graded by the official anchor, drop tasks the solver always passes, stratify the rest across pass levels, then split 60/40 into train and test. The resulting train/dev/test sizes are 60/10/40 for MBPP+, 59/10/40 for Spider 2.0, and 73/10/48 for report generation. The dev 13 set is ten items, five clear failures and five clean positives (for report generation, five golden-quality positives and five class-stratified checkable failures), so the verifier is scored on both classes. Anchors. MBPP+: hidden unit tests. Spider 2.0: execute on the live warehouse and compare the fetched table to the stored gold result with the official comparator; we reproduce the evaluator’s CSV round-trip of the fetched frame, without which even gold queries fail on dtype. Report generation: the verifier loop’s anchor is a frozen demo-delta soft label, where a teacher compares the frozen-agent report against the golden section under a closed seven-class miss taxonomy and passes only if at most one miss class is present. Each of the 48 held-out sections contributes two locked items, the frozen-agent report (which the strict anchor almost always fails) and the golden section (which passes by definition), giving 96 items split 54 pass and 42 fail, so a degenerate always-pass verifier scores 0.56. The skill and co-loops are evaluated by the official rubric evaluator (0–100 normalized), whose dimensions cover evidence grounding, metric-tag discipline, claim calibration, structure, and style. Spider 2.0 prompt grounding. The generation prompt grounds the solver with a compact typed schema catalog (columns as name:TYPE, sharded tables collapsed, sample values on date-like columns) and Snowflake-specific rules. We report this because it is a large shared effect we control for, not a contribution: grounding raises the frozen baseline by half on the screened pool (0.211 to 0.329) but is applied identically to every arm, so all reported lifts sit on top of it. It also sets up the detectability contrast of §5.5, since grounding pushes residual failures from compile errors into clean-executing wrong values. C Expression grammar, selection, and lifecycle constants Writingv(e)∈1, 0,⊥for the verdict of expressioneon(t,y), with1denoting DRAWBACK and ⊥ ABSTAIN, the expression grammar of §3.1 is e ::= o W i≤k e i V i≤k e i ¬e P i≤k v(e i )≥ K P i≤k w i v(e i )≥ θ ,(2) where|separates alternative node forms: a leaf op, a disjunction, a conjunction, a negation, an unweighted K-of-k vote, or a weighted vote against threshold θ. The objective multiplies dev agreement by consensus agreement on unlabeled train outputs (expo- nent 1.0 for MBPP+ and Spider, 0.25 for report generation, where detector consensus is a weaker regularizer), minus a small complexity penalty. Dev agreement is the recall-weighted mean of the two per-class recalls over the dev items a candidate opines on, with the FAIL class at weight 2 and CLEAN at weight 1. Train consensus is reliability-weighted, each opining op’s vote scaled by 1 + 10 max(0,m o )for that op’s anchored leave-one-out marginalm o , so anchor-proven ops lead the vote and ops with no anchored evidence contribute at unit weight; this keeps a bloc of cheaply synthesized correlated detectors from defining the pseudo-label the regularizer rewards agreeing with. The complexity penalty charges 0.001 per op beyond a floor of three and a further 0.002 per LLM-calling op, which is why selected expressions stay small and mostly deterministic. Fail-closed anchoring makes candidates without a usable dev opinion unselectable, and a validity gate drops all-pass, all-fail, and all-abstain candidates. The birth gate requires a new op to fire on at least half its cluster and stay clean on known-good outputs; ops born from anchored misses enter active, unlabeled-gap ops enter shadow and are promoted only on demonstrated anchor agreement; per-op fitness is the leave-one-out marginal of the incumbent expression and non-positive ops retire after a two-round grace. The naive ablation disables exactly the two anchor guards (and skill-loop rollback); the no-lifecycle ablation disables exactly the birth gate, shadow tier, and retirement. D Final-judge protocol and rubrics For each of the 48 held-out sections we pair the stored round-0 report against the stored peak-round report, and separately against the final-round report, for every arm and seed. Pre-repair this is 3 arms×3 seeds×2 rounds×48 sections=864 pairs; the post-repair rerun adds a fourth seed on the two co-loop arms, giving 1,056. Each pair is judged twice with positions swapped, and a win counts only when both orders agree, otherwise it is a tie. The generic rubric asks a senior-editor question about holistic reader value with no knowledge of the pipeline’s conventions. The task-aware rubric additionally states the format contract: evidence citations and inline metric tags are required 14 output that renders downstream, a tag with no value nearby is a defect, and fabricated figures are disqualifying. Both rubrics judge the same stored pairs, so the four cells of Table 2 differ only in the repair pass (rows) and the rubric (columns). The decided-pair counts behind the task-aware column are 271 of 526 pre-repair and 435 of 565 post-repair. E What evolution actually produced Finalverifiers,verbatim. OnMBPP+,(any spec_mismatch crash returns_not_print_only): flag a solution if the judge finds a specification mismatch, or it crashes on probe inputs, or it prints instead of returning. On Spider 2.0,(any missing_group_by spec_sql_mismatch): flag a query if it aggregates without grouping, or the judge finds it does not answer the question; this form was selected independently on two of three seeds. On report generation,(any ledger_metric_uncovered low_status_card_overclaimed): flag a section if a registered key figure never appears, or a weak-credibility card is cited without hedging; the other two seeds swap the first leaf for an uncited-claims detector. One detector through the whole lifecycle. On the Spider seed that ends at(any syn_forbids_select_star_when_columns spec_sql_mismatch), the loop clustered four un- labeled train outputs that returned every column although the question implied specific ones, and authored the typed speckind: static, check: forbids_select_star, detects: select_star_specific_columns . Because its birth evidence was an unlabeled gap rather than an anchored miss, it entered as a shadow: it ran and recorded verdicts but could not be selected. It was promoted in the same round, but only after composing it with the incumbent raised dev agreement from 0.71 to 0.81, and selection then adopted it, replacing the grouping check the seed pool had contributed. Two of the co-loop seeds on this task converge on a different synthesized detector (syn_top_n_missing_limit) and the third on the seedselect_starcheck, so the leaf is not a fixed artifact of the pool. Grading one task: hidden tests against the evolved verifier. For “write a function to find the ratio of zeroes in an array of integers”, the solver returnedzeros / non_zeros: the code runs cleanly and returns a number, but computes the ratio to non-zeros where the specification wants the ratio to all elements, so only the meaning is wrong. The hidden unit tests raiseAssertionError and return FAIL, a reliable verdict with no indication of which requirement was misread. The evolved composition returns FAIL as well, fromcrash: clean,returns_not_print_only: clean, spec_mismatch: DRAWBACK, and so names the failure class. That named drawback is what turns the attempt into a usable failure capsule for skill synthesis; the anchor’s bare verdict could not have driven the same repair. Same lesson, two loops. Each pair below was synthesized independently, one by the reference loop under ground-truth grading and one under the evolved verifier, and neither loop saw the other. MBPP+: referencefunction-name-from-assert, “the harness imports the symbol lit- erally as written in the assert; lift the identifier verbatim including case, the prose is only a hint”; cocopy_callable_identifier_from_assert, “the grader imports the name verbatim from the assert; reproduce that exact spelling letter for letter, including apparent typos”. Spi- der 2.0: referencesql-empty-result-diagnosis, “a 0-row result is almost never correct when the gold has rows; incrementally relax predicates to locate which condition eliminates all rows”; co sql-empty-result-sanity-probe, “an empty result almost always means one predicate matched nothing; validate each CTE’s row count in isolation before trusting the join”. Report: reference metric-tag-annotates-not-replaces, “the scorer counts a figure only when a literal number sits adjacent to the brace tag”; cometric-tag-inline-after-number, “the tag annotates the value; it never replaces the value, and a bare card cite does not satisfy it”. F Lifecycle statistics and full per-loop scores Figure 4 plots the held-out learning curves behind Table 1, Table 4 recaps what the detector lifecycle did in the anchored arms, and Table 5 gives the complete held-out init, peak, and final-rounds scores. Frozen single-call baselines are approximately 0.27 (MBPP+), 0.285 (Spider 2.0), and 0.56 (report generation). 15 Table 3: Verifier-validity ablations on report generation, final-rounds values over three seeds, each arm flipping one guard family (✓on,x off). Removing the anchor guards collapses the verifier into a vacuous always-pass grader on every seed; removing the detector lifecycle does not. Train pass: fraction of training attempts passed (n/r: not recorded for this arm). The collapse is invisible in held-out task score (Table 1, naive). Tinted row: our configuration. GuardsVerifier validity Armanchorlifecycleobjectiveheld-out agr.train passOutcome anchored✓0.865±0.0020.830±0.012.78–.82valid, composes naivex✓1.000 (vacuous)fail-open.97–1.0collapses to always-pass no-lifecycle✓x0.896±0.0720.868±0.061n/rno collapse Skill-side behavior. Synthesis is prolific at roughly 1.5 skills per round, the bounded bank forces contribution-based eviction of more than half of everything created, and rollback fires rarely. The report skill loop is the outlier: its targeted failure capsules produce skills effective enough that a handful suffice, with one seed holding its plateau on a bank of six. MBPP+ 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0255075 ref co lift retained 106% skill-evolution round Spider 2.0-Snow 0.1 0.2 0.3 0.4 0.5 0255075 ref co lift retained 110% skill-evolution round Report generation 0.5 0.6 0.7 0.8 0.9 0255075 ref co lift retained 88% skill-evolution round held-out score reference skill loop (ground truth / best rubric)Double Ratchet (its own evolved verifier)frozen solver Figure 4: Held-out learning curves (mean±std, three seeds): the reference skill loop on ground truth or the best available rubric, against Double Ratchet on its own evolved verifier. Callout: co-evolution’s lift retention as a fraction of the reference loop’s. Held-out evaluation never passes through the evolved verifier. Table 4: Detector lifecycle, anchored arm over 100 rounds. Three-seed min–max; a bare number means all seeds agree. Anchored selection prefers precise small compositions over detector accumu- lation. Taskseed detectorsbornbirth-gate rejectionsfinal poolleaves in verifier MBPP+102–4997–101412–143 Spider924–30612–78533–391–2 Report1111–15172–18822–262 Two further scope limits. Guidance-text skills cannot substitute for capabilities that need inter- action, which is the ceiling both loops hit on Spider 2.0 (§5.3). The op type system also admits failure-anticipating detectors in the spirit of metamorphic testing [47], which perturb a task and check that the output changes as it should; these are implemented but no selected verifier used one, so we make no claim about them. Naive co-loops are not a free pass.Naive co-loops match the anchored arm on held-out evaluation because their measurement anchor is untouched, but their verifiers are degenerate on every task, not only on report generation (§5.2): on Spider 2.0 the naive objective barely moves on two of three seeds. Skill-loop robustness to verifier quality. Per-seed evidence for the directional-signal argument of §5.3: two of three Spider co-evolution seeds selected a weak synthesized single-detector verifier (train agreement 0.41–0.44) yet reached peak evaluations of 0.45–0.50, indistinguishable from the sibling seed with a two-detector verifier. 16 Table 5: Held-out evaluation per loop, mean±std over three seeds; the naive rows use the same three seeds as co on each task. Final rounds: mean of each run’s last ten rounds. Tinted: our configuration. TaskLoopinitpeakfinal rounds MBPP+reference0.275±0.0500.700±0.0250.592±0.039 co0.267±0.0380.717±0.0380.652±0.034 naive0.317±0.0580.742±0.0140.632±0.023 Spiderreference0.317±0.0380.483±0.0380.327±0.010 co0.275±0.0430.458±0.0380.323±0.038 naive0.292±0.0520.458±0.0290.343±0.028 Reportreference0.562±0.0010.850±0.0100.802±0.024 co0.557±0.0060.812±0.0060.767±0.015 naive0.559±0.0030.841±0.0030.811±0.015 Anchor size. Ten dev items is a design choice about what sparse anchoring realistically supplies, not a tuned setting. The replay behind the §5.1 claim takes report-generation selection offline, draws 200 stratified subsamples per anchor size from the same ten items, preserves the fail/clean balance within each draw, and rescores every candidate expression the live run actually considered under the reduced anchor. No new generation and no new solver calls are involved, so the comparison isolates the anchor’s contribution to selection from run-to-run variance. Held-out agreement is0.871,0.882, and0.874at four, six, and eight anchored items against0.854at ten. Subsample standard deviations grow as the anchor shrinks (0.063 at four items against 0.041 at eight), the expected cost of a smaller anchor: any single small anchor is a lottery, while the mean over draws shows selection is not relying on the marginal item. G Reproducibility No GPUs are involved, since every model call goes to a hosted API, so a run needs only a CPU host (ours: 48-core x86-64, 186 GB RAM, Ubuntu 24.04, Python 3.12, withpydantic,pyyaml, boto3, and for Spider 2.0snowflake-connector-python), and wall-clock time is dominated by API latency. Each run writes its round-by-round history, detector pool, skill bank, and judge outputs as structured JSON and SQLite files, and every number in this paper is read directly from those files; the per-run databases store all 52,600 locked verdicts of the reported verifier runs with their per-detector traces, which is what makes the paired tests recomputable without rerunning any loop. Every LLM role (solver, teacher, composer, skill synthesizer, judge ops, final judge) is bound per call from a role-to-model table, so repeating any experiment under a different model or a mixed assignment is a configuration change rather than a code change; we hold that table uniform here to keep the comparisons clean. Spider 2.0 requires credentialed access to the benchmark’s hosted warehouse; all other experiments are self-contained. 17

Instant research discovery