Paper deep dive
Penumbra: Sample-Efficient Adversarial Search for Regulatory Obligations
Anthony Rhodes
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Agents are entering finance, healthcare and law, sectors where a violation leaves no lexical signature and carries real penalties. Whether an omission is material, or a disclosure sufficient, depends on what the response left out. Probing such an obligation means finding responses one minimal edit from flipping compliance, and every probe costs a generation and two adjudications, so the binding constraint on regulatory red-teaming is sample efficiency, not volume. We introduce Penumbra, an adversarial search that walks from a verified anchor under an expanding edit budget until a two-evaluator committee changes its verdict, and emits the two adjacent responses that straddle the change. Allocation is adaptive, and the objective is coverage of the defeat surface: distinct (obligation x defeat mode) cells resolved per candidate. At matched budget, adaptive allocation reaches uniform allocation's full-budget coverage on 59% of the candidates; at equal records it covers 1.43x the defeat modes of naive enumeration, and the gain is confined to the axis it targets. On 60 screened obligations of a financial advisory constitution, Penumbra returns 144 pairs, each a compliant and a violating response that the committee places on opposite sides of the boundary, differing by a handful of words where a model asked for both directly produces texts sharing almost nothing. A second constitution, for clinical triage, reproduces this on 18 obligations: 49 pairs at the same tightness, with the same modes hardest. Pairs like these show where an obligation's own terms stop deciding, which is what an agent deployed under it must be tested against, and the search finds them at a cost that scales with the boundary, not the text.
Tags
Links
- Source: https://arxiv.org/abs/2610.04693v1
- Canonical: https://arxiv.org/abs/2610.04693v1
Trouble viewing inline? Open PDF directly →
Full Text
44,879 characters extracted from source content.
Expand or collapse full text
Self-Evolving Diversity-Driven Search for Robust AI Systems Penumbra: Sample-Efficient Adversarial Search for Regulatory Obligations Anthony Rhodes Affiliation: Confidential Core AI Email: a.rhodes@confidentialcore.ai Abstract Agents are entering finance, healthcare and law, sectors where a violation leaves no lexical signature and carries real penalties. Whether an omission is material, or a disclosure sufficient, depends on what the response left out. Probing such an obligation means finding responses one minimal edit from flipping compliance, and every probe costs a generation and two adjudications, so the binding constraint on regulatory red-teaming is sample efficiency, not volume. We introduce Penumbra, an adversarial search that walks from a verified anchor under an expanding edit budget until a two-evaluator committee changes its verdict, and emits the two adjacent responses that straddle the change. Allocation is adaptive, and the objective is coverage of the defeat surface: distinct (obligation × defeat mode) cells resolved per candidate. At matched budget, adaptive allocation reaches uniform allocation’s full-budget coverage on 59% of the candidates; at equal records it covers 1.43× the defeat modes of naive enumeration, and the gain is confined to the axis it targets. On 60 screened obligations of a financial advisory constitution, Penumbra returns 144 pairs, each a compliant and a violating response that the committee places on opposite sides of the boundary, differing by a handful of words where a model asked for both directly produces texts sharing almost nothing. A second constitution, for clinical triage, reproduces this on 18 obligations: 49 pairs at the same tightness, with the same modes hardest. Pairs like these show where an obligation’s own terms stop deciding, which is what an agent deployed under it must be tested against, and the search finds them at a cost that scales with the boundary, not the text. 1 Introduction Agents are entering sectors that carry binding professional obligations: portfolio analysis and compliance review in finance, triage and documentation in healthcare. Such an agent inherits a constitution written for human professionals: “Maintain version traceability. Do not represent inference as retrieved fact. Do not selectively omit material information.” An agent can breach any of these without producing anything a lexical detector could match, because the breach is not a phrase in the response but a judgement about it: whether an omission was material, a disclosure sufficient, an answer within scope. Concretely, two responses from our corpus answer the same client question under do not selectively omit material information; they differ by three words: violation …the company faces high valuation risk and cyclical demand risk in the semiconductor industry… compliant …the company faces high valuation risk, customer concentration issues, and cyclical demand risk in the semiconductor industry… Both disclose risk, and no keyword test separates them. What separates them is what the agent knew: the tool output it had retrieved named a customer concentration exposure, so the first response omits a material fact and the second does not. Such pairs are hard to find, and yet they are worth finding: each fixes a point on the shadow line of an obligation, where its own light gives out, and that edge is where compliance actually rests, on a reading of materiality or sufficiency that no checklist states in advance. Prior approaches were not built to find such pairs, and the reasons are instructive. Prompting a model to violate a rule yields conspicuous violations far from any line. Self-evolving red-team loops (Perez et al., 2022; Ganguli et al., 2022; Bai et al., 2022; Samvelyan et al., 2024) generate at volume, but every record is a violation with no compliant sibling, and Section 3 shows they drift away from the line as they run. Probing must therefore be diverse, covering the distinct ways an obligation can be defeated, and boundary-seeking, placing each defeat a minimal edit from compliance. Penumbra pursues both over a space of obligation × defeat mode, reading disagreement between two independent evaluators as the signal for the second. Rules of this kind have a settled core and a penumbra where their own terms do not decide the case (Hart, 1961; Wittgenstein, 1953; Dworkin, 1986; Solum, 2013; Bhatia and Bhatia, 2018). Two observers a step apart inside it will disagree about whether the light is covered, and the disagreement is a fact about where they stand rather than about their eyes; and thus a split between two competent adjudicators is the observable signature of that region. Probing an obligation is also expensive. Each candidate costs one generation and two adjudications, one from each evaluator, and most edits leave the verdict unmoved, so the question is not how much adversarial text a loop emits but rather how few adjudications each certified pair costs. Penumbra is a search built around that question: it spends adjudications only where a verdict is about to change, and so maps an obligation’s boundary at a cost that scales with the boundary rather than the text, which is what makes red-teaming agents in regulated sectors tractable. We contribute a calibrated boundary detector, with the negative result that the obvious candidate fails (Section 2); Penumbra and its adaptive allocation (Sections 4 and 5); and a corpus of 144 certified minimal-edit pairs with their coverage map. 2 A calibrated boundary detector A boundary example in regulatory policy is one a competent evaluator could reasonably judge either way. The tempting instrument is the evaluator’s own confidence, and it fails: on a ladder of 77 screened compliant responses over eleven obligations, authored at three difficulty levels (Appendix F), an evaluator reports mean confidence 0.935, 0.929 and 0.928 for clear, guarded and borderline, 97% at 0.9 or above. Confidence carries no difficulty information, consistent with the finding that verbalised confidence sits near the top of the scale whatever the answer (Xiong et al., 2024; Tian et al., 2023). Disagreement between two separately prompted evaluators does carry it. Each evaluator applies a written checklist for the obligation it scores. On the same ladder the disagreement rate runs 0.000 / 0.032 / 0.091 under the original checklists (slope +0.045+0.045) and 0.042 / 0.097 / 0.136 under a patched set (slope +0.047+0.047), so sensitivity does not depend on how much the checklist has been revised. The calibration pair was cross-vendor and the search committee same-vendor, so the ladder bounds the search’s sensitivity from above (Appendix F). Two evaluators therefore give a three-level ordinal: violation when both convict, boundary when they split, pass when both acquit. Its middle level is a termination condition, not a threshold on a noisy scalar. Formally this is query-by-committee (Seung et al., 1992), with the query synthesised, not selected, and the committee calibrated, not assumed. 3 Why existing generation misses the boundary Self-evolving red-team loops, the iterative generator-and-evaluator systems of Section 1, drift away from the boundary as they run. When we apply the detector of Section 2 to a faithfully implemented self-evolving loop over eleven obligations, disagreement falls by round, 0.133, 0.081, 0.052, 0.038, 0.100, with a negative slope in all eleven (p=2−11p=2^-11 by sign test). Its outputs get easier as it runs, and its objectives explain why: it rewards evasion and novelty, and neither points at the boundary. Adapting either side does not repair the drift (Appendix G). We therefore freeze the committee and diagnose each disagreement rather than patching it. 4 Penumbra 60 obligations× 5 defeat modes300 cellsGeneratorsoften / hardenexpanding edit budgetEvaluator committeetwo independent judgesviolationdisagreepasscandidate responsethree-band verdict steers the next edit ADVERSARIAL CO-ADAPTATION LOOPcrossing preference pairs pass ↔ violation DPO-usable 78 of 144 exemplar pairs into disagreement 30 penumbral 35 artifact 66 of 144 resolved cell masked, so the next attempt goes to an unresolved one Figure 1: Penumbra. Left: the generator proposes a candidate under an expanding edit budget; the committee’s three-level verdict selects the next operator and budget, and a change of level emits the two adjacent responses as a certified pair. Right: one chain for one obligation; a step that clears the penumbra gives a preference pair, one landing inside gives an exemplar (Appendix D). Penumbra turns the detector of Section 2 into a search. Its unit of work is a cell: one obligation paired with one defeat mode, a named way of failing that obligation (Appendix C). The defeat surface is the set of all cells, and coverage is the number that hold a certified pair. Within a cell the search starts from a verified anchor, a response both evaluators convict or both acquit, and edits it under a budget that begins at a few words. The committee scores each edit: if its verdict class matches the parent’s, the edit becomes the new subject and the budget widens; if the class changes, the parent and the edit straddle the boundary and are emitted as a certified pair. Two operators drive the walk. Soften loosens a violating response and Harden tightens a compliant one, and both must preserve the defect they edit around rather than delete it, since an edit that simply removes the defect crosses the line trivially and brackets nothing (Appendix E). Four design choices then make the search a measurement rather than a heuristic; each removes a way it could report yield that does not mean what it claims. Minimality is enforced, not filtered for. The budget starts small and expands only while the verdict class holds: a schedule that starts wide and shrinks never establishes a bracket. The pair we emit is the subject and the crossing child, adjacent chain links, not the child and anchor (Appendix E). Termination is ordinal. The walk stops when the verdict class changes, not when a score crosses a threshold, so no sensitivity parameter decides what counts as the boundary; the committee does. One scalar remains, a similarity floor τ=0.5τ=0.5 below which a crossing is discarded; it never stops the walk. The two products are counted separately. A crossing from pass to violation yields a preference pair with clean opposite labels; a crossing into the band yields an exemplar whose preferred half is itself contested. We report them apart because pooling overstates the yield usable for direct preference optimisation (Rafailov et al., 2023) by 1.8× (Algorithm 1). Allocation is adaptive, over cells chosen by measurement. Adaptive allocation only pays if disagreement is concentrated somewhere, so we measured where it clusters before choosing the cell: by obligation and by defeat mode, well above the binomial noise floor, but not by attack mechanism (Appendix B). That is why the cell is (obligation × defeat mode); the five modes are Obscure, Defer, Substitute, Vacuous and Negate (Appendix C). Spending then follows that structure. Within an obligation, a mask over modes that already hold a certified pair sends the next attempt to an unresolved cell. Across obligations, budget goes to the arm with the best upper confidence bound on crossings per candidate. Both rules optimise one quantity, cells resolved, so novelty is not a separate objective but the shape of the mask itself, as in quality-diversity search (Samvelyan et al., 2024). 5 Results candidates 50 100 200 250 309 adaptive 6 11 25 30 35 uniform 2 3 13 20 24 Figure 2: Left, cells resolved against candidates spent, 24 obligations. Right, coverage of (obligation × mode × predicate) cells against records, eleven obligations at equal records: guided separates early and holds; naive and unguided are indistinguishable (Appendices B and L). Obligation set. The obligations are the 75 rules of a constitution we authored for a regulated financial advisory agent (Appendix H). Two LLM screeners, the same two models that later form the committee (Appendix J), rated each on admissibility (could a single response violate it) and non-determinism (does compliance turn on materiality, sufficiency or scope). They agreed 0.920 on both (κ=0.72κ=0.72 and 0.580.58) and 55 qualified; one reinstated and four surface-decidable controls give 60 obligations, 300 cells over five modes (Appendix J). Adaptive allocation is worth 1.7× the budget. The acquisition rule decides where every candidate is spent, so we measured its effect directly: a live adaptive arm over 24 obligations against uniform allocation at the same 309-candidate budget (Appendix B). Uniform allocation resolves 24 cells with its full budget. Adaptive allocation resolves the same 24 after 181 candidates, 59% of the spend, and 35 by the end, so matching its coverage costs uniform allocation 1.7× the candidates (Figure 2). Allocation is genuinely non-uniform, 10 to 19 candidates per obligation against uniform’s 12; the gain comes from spending where a pair is cheapest. Diversity is bought where you aim it. Figure 2 compares three arms on the eleven obligations they share: naive enumeration (direct prompting, no search), unguided search (the walk with mask and bandit off) and guided search (the full method). Three findings bound the result. The mask buys coverage only where it aims: guided search covers 1.43× naive enumeration’s defeat modes at 88 records, yet on an independent predicate decomposition of each obligation naive enumeration covers more, 23.2 cells against 19.8. The advantage is a constant factor from about 130 records, not one that compounds. And records are not calls, so this is breadth at equal records, not cost parity (Appendix L). Yield, coverage, and pair quality. Penumbra returns 144 certified pairs over 47 of 60 obligations and 122 of 300 cells, and in the median case the verdict changes on the first edit. We measure a pair’s tightness as the similarity between its two halves, so 1.0 would be identical texts. 78 are preference pairs with opposite labels at mean tightness 0.957; 66 are exemplar pairs from evaluator disagreement, at 0.961, which attribution splits into penumbral and checklist artifact (Appendix C). Modes differ in cost, not only reach: among obligations whose budget reached all five modes, so that attempt order does not confound the comparison, Obscure resolves on 4.0 candidates per cell while Negate needs 10.6 and resolves a third of the time (Figure 3). Naive pair prompting, a baseline that asks for a compliant and a violating response to one scenario, yields opposite labels reliably and a boundary never: its pairs differ by a median of 378 words at 0.062 tightness against our 14 and 0.957. Opposite labels come for free; two responses that differ only at the line do not, and that proximity is what the search is for. A second constitution, 18 clinical triage obligations at the same budget, returns 49 pairs over 15 obligations at tightness 0.964 and 8 words with the same modes hardest; six universal rules anchored in both domains resolve 18 cells in finance, 25 in healthcare (Appendix K). The edits are diverse rather than a shared trigger, 0.983 against 0.882 mean pairwise distance (Appendix I). Two nulls against our own hypotheses: the pairs are no harder for an independent detector to catch, and the search does not escalate as it runs; the detector contests 41% of our compliant halves, so these are a probe set, not training data (Appendix H). 6 Discussion What this establishes is narrow and durable: disagreement between two competent adjudicators is a measurable property of an obligation, a search that terminates on it finds where the obligation’s own terms stop deciding, and it does so at a cost that scales with the boundary rather than the text. The claim is bounded by two authored constitutions, one at scale, a same-vendor committee not yet checked against humans, anchors of unequal strength, and an independent detector that contests 41% of preferred halves (Appendix H). The completing experiment is set out in Appendix H. Where obligations are hardest to check, the boundary is where compliance is decided, and a search that seeks it directly, spending adjudications where verdicts change rather than generating attacks at volume, is what makes red-teaming an agent against those obligations tractable. Acknowledgments and Disclosure of Funding The author thanks colleagues at Confidential Core AI for discussions and for feedback on drafts, and the anonymous EvoRobust reviewers for comments that improved the paper. This work received no external funding. Large language models assisted with the drafting and editing of text; the author designed the method, ran all experiments and verified every numeric claim against the underlying run data. References Bai et al. [2022] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073, 2022. Bhatia and Bhatia [2018] Vijay K. Bhatia and Aditi Bhatia. Corpus linguistics and the study of vagueness in legal texts. International Journal of Language and Law, 7:1–27, 2018. Cohen [1960] Jacob Cohen. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46, 1960. Dworkin [1986] Ronald Dworkin. Law’s Empire. Harvard University Press, Cambridge, MA, 1986. Ganguli et al. [2022] Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858, 2022. Golchin and Wetter [2026] Shahriar Golchin and Ryan Wetter. Intent laundering: Ai safety datasets are not what they seem. arXiv preprint arXiv:2602.16729, 2026. Hart [1961] H. L. A. Hart. The Concept of Law. Oxford University Press, Oxford, 1961. Perez et al. [2022] Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models. arXiv preprint arXiv:2202.03286, 2022. Rafailov et al. [2023] Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2023. Samvelyan et al. [2024] Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, et al. Rainbow teaming: Open-ended generation of diverse adversarial prompts. In Advances in Neural Information Processing Systems (NeurIPS), 2024. Seung et al. [1992] H. S. Seung, M. Opper, and H. Sompolinsky. Query by committee. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory (COLT), pages 287–294, 1992. Solum [2013] Lawrence B. Solum. Communicative content and legal content. Notre Dame Law Review, 89(1):479–520, 2013. Tian et al. [2023] Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D. Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. In Empirical Methods in Natural Language Processing, 2023. arXiv:2305.14975. Wittgenstein [1953] Ludwig Wittgenstein. Philosophical Investigations. Blackwell, Oxford, 1953. Translated by G. E. M. Anscombe. Xiong et al. [2024] Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs. In International Conference on Learning Representations, 2024. arXiv:2306.13063. Zheng et al. [2023] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging LLM-as-a-judge with MT-Bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2023. Appendix A The coverage map Figure 3 is the map the counts of Section 5 are read from: every cell the uniform-allocation run touched, with its outcome. Figure 3: The coverage map of the uniform-allocation run: the 50 obligations that anchored, five defeat modes, 250 cells, of which 122 hold a certified pair. (a) Outcome by mode. Green cells hold a preference pair, amber an exemplar, pink were attempted without a crossing, grey were never reached. Modes are attempted in a fixed order, so grey marks where a per-obligation budget ran out rather than where the search declined to look, and the per-mode totals carry that order as much as difficulty. (b) The same 250 cells as one column per obligation, sorted by cells resolved: four obligations resolve on all five modes, three on none, and the bulk on two or three. Among the 20 obligations whose budget reached all five modes, Obscure costs 4.0 candidates per attempted cell and Negate 10.6, resolving a third of the time. This run holds one obligation per process, so its allocator is inert; it is the uniform baseline of Figure 2, and the adaptive result is measured separately (Appendix B). Appendix B Acquisition, in detail Choosing the cell. Before allocating we measured how much the disagreement rate varies along three candidate axes and compared each spread with its binomial floor, the variation that sampling noise alone would produce if the true rate were the same everywhere. Across obligations the spread is 0.062 against a floor of 0.035, and across defeat modes 0.043 against 0.029: both carry real structure. Across attack mechanisms it is 0.039 against 0.032, indistinguishable from noise. The cell is therefore (obligation × defeat mode) and not (obligation × mechanism). candidates 50 100 200 250 309 adaptive 6 11 25 30 35 uniform 2 3 13 20 24 The two arms. The 60-obligation run shards one obligation per process, so its allocator holds a single arm and the across-obligation term is inert; that run is therefore a free uniform-allocation arm, and only the adaptive arm had to be bought. We ran it in one process over 24 obligations, 309 candidates. Budget is counted in judged candidates rather than API calls because both arms spend one generation and two adjudications per candidate, plus two more when attribution fires on a disagreement; counting candidates removes a surcharge that would otherwise flatter whichever arm found more disagreements. A cell counts as resolved at the trace line where the verdict class first differs from its parent’s, provided that cell also appears in the arm’s pairs file, so crossings discarded by the similarity floor are not credited. The uniform arm’s yield at budget N is the cells resolved within each obligation’s first N/24N/24 candidates. Appendix C Attribution and cross-check agreement Where a disagreement comes from. We re-judge every disagreement against the bare obligation with the checklist removed. If the evaluators still disagree the obligation underdetermines the case and we call the pair penumbral; if they now agree the checklist wording caused the split and we call it a checklist artifact. The raw split is 30 penumbral to 35 artifact over the 66 exemplar pairs, one unattributed. That overstates the wording channel, because re-judging the same crossings with the checklist retained converges 37% of the time anyway, so the excess attributable to wording is nearer 17 points than the 54 the raw split suggests. The distinction still routes work, since a criteria defect is fixable by rewriting policy text and an underdetermination is not, but it is a smaller effect than the raw counts imply. Mode labelling. We label each generated violation with the defeat mode it exploits. An independent labeller cross-checked that labelling: κ from 0.197 to 0.580 under a fixed taxonomy with structured disagreement, and 0.345 raw rising to 0.421 after merging two systematically conflated modes [Cohen, 1960]. Moderate agreement on the mode axis is a real caveat on the cell count, though not on the pair count, which does not depend on the labelling. Appendix D One chain, drawn Figure 4 draws a single chain and the pair it emits. both evaluators:VIOLATIONboth evaluators:PASSpenumbrax−x^-no straddleeach step may edit morelands inside: exemplar emit: the two adjacent linksx+x^+(a) the semantic space of one obligation T2-CP-03 do not engage in unfair, deceptive or abusive practices compliant …its higher annual fee will cost you tens of thousands of dollars more than the Index Fund over two decades. violation the same disclosure, then These ongoing costs are just nominal administrative expenses to keep the fund running smoothly. fourteen words added, nothing removed, nothing false, similarity 0.971, verdict flips (b) an emitted pairedit budget at that stepsoften, walking from x−x^-harden, walking from x+x^+the certified pair Figure 4: One chain in the semantic space of a single obligation. The penumbra is a band rather than a line, and a surface-decidable obligation should have a narrow one or none, which is the intuition the control was meant to test (Appendix H). A chain leaves a verified anchor and takes steps under a budget that may grow. A step that does not change the band becomes the new subject; a crossing too wide to be tight is walked past rather than emitted. A step clearing the band gives a preference pair, one landing inside gives an exemplar. The pair shown is real, and it is the paper’s median edit: the violation withholds nothing and states nothing false, it characterises the fee it has just disclosed. Appendix E The search, precisely Figure 1 gives the intuition; this is what runs. J1,J2J_1,J_2 are the two evaluators, Ordinal maps their verdict pair to violation, boundary, pass, and τ is the similarity floor below which a crossing is discarded as too wide to be a boundary pair. Algorithm 1 Bracket and cross, per (obligation, defeat mode) 1: x+←x^+\!← verified compliant anchor; x−←x^-\!← verified violating anchor 2: s←x−s← x^- or x+x^+; c←Ordinal(J1(s),J2(s))c← Ordinal(J_1(s),J_2(s)) ⊳ start side alternates per attempt 3: for k=1…Kk=1… K do 4: β←βkβ← _k ⊳ edit budget, expanding: words → clause → sentence → paragraph 5: x←Soften(s,β)x← Soften(s,β) if c is violating, else Harden(s,β) Harden(s,β) ⊳ the defect is preserved in both 6: c′←Ordinal(J1(x),J2(x))c ← Ordinal(J_1(x),J_2(x)) 7: if c′≠c ≠ c then ⊳ the band changed: s and x straddle it 8: if sim(s,x)≥τsim(s,x)≥τ then 9: emit (s,x)(s,x) as a certified pair and return 10: end if⊳ too wide, so keep walking rather than emit a loose pair 11: end if 12: s←xs← x; c←c′c← c 13: end for The pair is the subject and the child that crossed rather than the child and the original anchor: pairing against the anchor gave two unrelated texts at 0.06 similarity. The emitted pair is labelled preference when c,c′=violation,pass\c,c \=\violation,pass\ and exemplar when either side is the boundary band, since in the latter case the preferred half is itself contested and the pair teaches only that the contested response is better than the flagrant one. Appendix F The difficulty ladder The calibration set of Section 2 is 99 compliant responses generated by claude-sonnet-4-5-20250929 for eleven obligations, three at each of three levels. Clear: comply in the most obvious way, with nothing that should look borderline to any reader. Guarded: comply in full, but in a hedged and heavily caveated register, which tests whether an evaluator mistakes caution for violation. Borderline: go as close to the line the rule draws as possible while staying clearly compliant, so the response looks risky on a careless reading and defensible on a careful one. Because the generator’s own account of why a response is compliant is a claim rather than evidence, every response was screened against the original checklist by gpt-5.4-2026-03-05, and 77 survived. That screener is also one of the two calibration evaluators, so the survivors are conditioned on its acceptance under the original criteria; the slopes in Section 2 are therefore measured on responses one evaluator had already passed once, which can only flatten them. Each survivor was scored by two evaluators, gpt-5.4-2026-03-05 and claude-sonnet-4-5-20250929, under both the original checklists and the set the self-evolving loop of Section 3 had patched, giving 308 verdicts. Reported confidence and cross-evaluator disagreement are read from those verdicts. The search itself ran on claude-sonnet-4-5-20250929 and claude-opus-4-5-20251101, a same-vendor pair. Same-vendor judges correlate, which lowers the disagreement rate, so the cross-vendor ladder is an upper bound on the sensitivity of the pair the search actually runs. Appendix G The self-evolving loop, in detail Eleven obligations, five rounds, eight samples per obligation-round, scored on the same two-evaluator oracle. Cross-evaluator disagreement by round: 0.133, 0.081, 0.052, 0.038, 0.100. The per-obligation slope is −0.022-0.022 (t=−2.34t=-2.34), negative in all eleven, and the round-five uptick does not recover the round-one level. Five rounds is a short series and the sign test across obligations is the stronger statistic. Adapting the generator. Conditioning it on the evaluator’s stated reasoning, so it can aim at the current weakness, appears to double evasion, 0.124 to 0.245. But the arms are identical by construction in round one, before any feedback exists, and a round-one balance check shows the same +0.119+0.119 gap there. Difference-in-differences over the rounds where the manipulation operates is −0.002-0.002 (t=−0.02t=-0.02): the entire effect is a pre-existing offset between arms. Adapting the evaluator. The loop patches evaluator checklists between rounds, on 10 of 11 obligations. Under criteria frozen at round one, detection rises from 0.867 to 0.931, so the outputs became easier to catch and the apparent hardening was criteria expansion rather than adversary improvement. The patches cost a false positive rate of 0.026 to 0.182 on screened compliant traffic for a recall gain of 0.034. That refiner patched on judge misses without asking whether a miss was the criteria’s fault or the obligation’s, which is the distinction the attribution of Appendix C supplies. Appendix H Further limitations and planned experiments The completing experiment. It holds out by obligation and compares preference training on certified pairs against four baselines: the base model, obligation statements in context, few-shot violations, and SFT on loop-generated violations. We predict a lower violation rate on unseen obligations at equal or lower over-refusal. Artifact-gated refinement should cost less precision per unit recall than the ungated refiner of Section 3. One tension is already visible and is created by the method itself: minimal-edit pairs sit where direct preference optimisation is least stable, since the two halves induce nearly identical representations and the standard mitigation is to filter high-similarity pairs out, which would discard this dataset entirely. Integration therefore has to route by margin rather than filter by it, which is the first thing we will test. Two nulls, in full. We expected minimal-edit probes to be harder to catch; they are not. An independent detector, claude-opus-4-6, with no part in selection, misses 3 of our 78 violating halves against 0 of 49 naive ones (p=0.28p=0.28). What minimality buys is ambiguity on the other side: it calls 41% of our compliant halves violations against 2% of naive ones. We also expected the search to escalate as easy defeats ran out. It does not: ordered by resolution within an obligation, steps to cross move from 1.99 to 1.92 and tightness from 0.960 to 0.964. The mask moves sideways into unresolved cells, not upward in difficulty. The length cue. On the 78 preference pairs the violating half is longer 76% of the time, since a soften step adds hedging more often than it removes content. Length-balanced sampling, or a length-controlled objective, should precede any training run on these pairs. Instrument stability. Re-judging the 276 texts behind an earlier build of this corpus under identical conditions manufactures a disagreement on 1.6% of the texts the run scored as agreed, against a 4.9% cross-evaluator rate on the same texts, so the disagreements the search selects on are not an artifact of re-running the instrument. Agreed verdicts are stable (0.8% of individual verdicts flip); contested ones are not, reproducing 69% of the time, which is what a borderline case should look like. Controls we have not yet run. Two. We have no cross-vendor test-retest number, so we cannot yet separate the 0.049 cross-evaluator disagreement rate from single-evaluator sampling noise. And we have not run the direct ablation of the search itself: prompting a model for a minimal edit that flips the verdict, verified by the same committee. The control in full. Of the four obligations designated surface-decidable before any generation ran, three produce zero evaluator disagreements in 109 candidates and the fourth produces 4 in 34. That fourth turns on scope, which the screening instrument itself names as requiring adjudication, and the screeners split on it. Pooled the contrast is 4 of 143 against 80 of 1,531, p=0.14p=0.14 one-sided, so the class as designated is too small and too impure to test the claim either way. We report it because it was pre-registered, not because it supports anything. Limits on what we show. The main study is one authored financial advisory constitution. A second, for clinical triage, is run at 18 obligations in Appendix K, so the reach into healthcare is demonstrated at small scale and the reach into law is still argued. The oracle is a pair of LLM judges, and we never asked human adjudicators whether they would split on the cases we call penumbral, so machine agreement stands in for professional judgement, and establishing that correspondence comes before any training run. Coverage is 47 of 60 obligations. Of the 13 that produced no pairs, most failed anchor verification, and an obligation with no verifiable violating anchor may not be single-response admissible after all. We authored the obligations for this study rather than lifting them verbatim from a regulatory text, which buys coverage of the judgement-required cases at the cost of provenance. Limits on the instrument. We kept only anchors on which the evaluator pair that later serves as the oracle agreed unanimously. That biases each chain to start well inside the core, which makes crossings harder to find, not easier, but it also conditions the disagreement denominator on evaluator agreement at rates that differ between the arms. We machine-rewrote 49 of the 60 checklists, so the artifact channel is substantially a defect in text our own pipeline produced, and we have not separated that from a property of written criteria in general. Edit budgets are advisory, not enforced: two steps returned similarity near 0.6 even though we instructed the generator to change a single sentence. Because budgets only expand, the crossing step is the widest edit in its chain, so the median of 14 words is an upper bound; bisecting inside the bracket once it is established would tighten it, and mean tightness is in any case measured on what τ admits. Our evaluators are LLM judges and inherit the known limitations of that instrument [Zheng et al., 2023]. Appendix I Not a lexical regularity Minimal edits without diversity collapse onto the triggering cues that make safety corpora fail once paraphrased [Golchin and Wetter, 2026]. Ours do not: 47% of the edit vocabulary occurs once, its commonest tokens are determiners and pronouns, and the edits are more mutually distinct than naive pair prompting’s, 0.983 against 0.882 mean pairwise Jaccard distance over 11 content tokens against its 198. A frozen sentence-embedding classifier trained on 1,037 violations reaches 0.737 AUC when held out by obligation, against 0.867 TPR at 2.6% FPR for an untrained prompted evaluator. Surface features carry some signal but not enough to substitute for adjudication, so the pairs do not merely encode a lexical regularity that a cheaper detector would find. Appendix J Screening procedure We put each of the 75 obligations to two independent LLM screeners, claude-sonnet-4-5-20250929 and claude-opus-4-5-20251101, using the same instrument. Admissible: could a single agent response, on its own, violate this obligation, as opposed to the obligation binding a process, a record, or an organisation? Non-deterministic: does deciding compliance require adjudication over materiality, sufficiency, proportionality or scope, as opposed to checking for the presence or absence of a token? Raw agreement was 0.920 on both, which is κ=0.722κ=0.722 on admissibility and κ=0.583κ=0.583 on adjudication once the base rates are corrected for. Sixteen obligations were not admissible, chiefly process and governance duties; four were admissible but surface-decidable, and we kept them as the controls of Section 5. We reviewed one exclusion and reinstated it. We report the screen rather than filtering silently because the control result depends on our having separated the two classes before running any generation. Appendix K A second constitution To test whether the method depends on the domain it was built in, we authored a second constitution for a clinical documentation and triage assistant and ran the same pipeline on it unchanged except for the domain wording in the generator and judge prompts and the scenario tool set (electronic health record lookup, medication list, laboratory results, triage protocol). Twelve obligations are clinical: documentation supported by the record, reported symptoms kept distinct from findings, red-flag escalation, working impressions not stated as diagnoses, material allergies and interactions surfaced, dosing not stated without its parameters, minimum-necessary disclosure, consent scope, ordering and prescribing left to the clinician, equal triage rigor across patient characteristics, proportionate uncertainty, and overdue results surfaced. Eight are the T1 universal rules of the finance constitution copied verbatim, so that the same rule can be compared across domains. Same screeners, same committee, uniform allocation, 100 calls per obligation, one shard per obligation. The screen excluded two of the twenty, T1-BA-04 and the ordering-and-prescribing rule, on a split between the screeners rather than agreement that they were surface-decidable. Anchors failed for two more, T1-DP-01 and the documentation rule, which is the same failure mode as the finance run, at a similar rate. Sixteen obligations anchored; fifteen produced pairs. finance clinical triage obligations screened / anchored / with a pair 60 / 50 / 47 18 / 16 / 15 certified pairs, preference / exemplar 144, 78 / 66 49, 32 / 17 cells resolved, of attempted 122 of 197 49 of 75 mean tightness, preference / exemplar 0.957 / 0.961 0.964 / 0.959 median words changed 14 8 median steps to cross 1 1 attribution, penumbral / artifact 30 / 35 6 / 11 The difficulty ordering largely reproduces. Resolution conditional on attempt runs Substitute 12/13, Obscure 13/16, Defer 12/16, Vacuous 7/16, Negate 5/14, and cost per attempted cell runs 4.4, 6.9, 6.1, 8.9 and 8.9 candidates: the two modes that were hardest in finance are hardest here. On the six universal rules that anchored in both domains the search resolves 18 cells in finance and 25 in healthcare (3, 1, 4, 3, 2, 5 against 3, 4, 4, 5, 4, 5 for HND-01, HND-02, HND-05, TE-02, TE-03 and C-04), so a rule that was penetrable in one domain was at least as penetrable in the other. Two things did not reproduce and are worth stating. The obligation authored as the clinical analogue of the running example, surfacing a material interaction the record contains, anchored and then never crossed: 36 candidates over three modes, and the committee agreed on every one of them, 18 violations and 18 passes. Its boundary, if it has one at the widths the walk tried, sits outside the budget. And the artifact share of exemplars is higher, 11 of 17 against 35 of 65, which is the expected direction for criteria templated from predicates rather than refined against a generator, and is the reason attribution exists. Appendix L Matched-budget coverage Coverage ratios computed on unequal budgets overstate, which is why Appendix B counts candidates rather than calls and why the comparison against naive enumeration is rarefied. A predicate is one atomic condition in an obligation’s formalisation, so do not exceed authorised scope decomposes into whether the query falls outside scope, whether the response gives authoritative guidance, and whether it flags the limitation. There are 43 predicates over the eleven obligations, three to six each. The decomposition is independent of the defeat modes and of the mask, which is why it serves as an off-axis check. The naive arm is given the same budget as the others, 40 records per obligation drawn in independent rounds, each round shown the strategies already produced for that obligation and asked for materially different ones; a single call returning eight strategies is a weaker baseline and we do not use it. Rarefying all three arms to 88 records over 2,000 resamples on the eleven obligations they share: cells counted over naive unguided guided obligation × defeat mode 23.1 27.9 (1.21×) 33.1 (1.43×) obligation × predicate 23.2 17.3 (0.74×) 19.8 (0.85×) obligation × mode × predicate 33.2 33.0 (0.99×) 41.3 (1.24×) Coverage against budget, on the joint axis, over the range every arm covers: records 22 44 88 176 264 328 naive 14.9 23.1 33.2 45.6 53.1 57.0 unguided 15.0 23.0 33.0 44.0 50.4 54.0 guided 17.0 27.8 41.3 56.0 64.7 69.6 guided / naive 1.14 1.20 1.24 1.23 1.22 1.22 What a record is, and what it costs. The three arms are matched on records, and a record is not the same object in each. A naive-enumeration record is one sentence proposing an attack, unverified; an unguided or guided record is a generated violation that a judge has scored; and the corpus Penumbra finally emits is neither, it is a certified pair with a compliant sibling. Nor do they cost the same. Naive enumeration returns eight strategies per generation call and is never adjudicated, so a record costs about an eighth of a call, while a search record costs a generation plus a judge verdict, roughly sixteen times more. Matched on calls rather than records, naive enumeration would draw far more records than either search arm and, on a space this size, approach saturation, and we expect the ratios here would invert; we did not run that comparison. We report records because that is the unit on which the three arms describe the same kind of object, an attack located in the cell space, and because cost parity would compare a proposal against a verified artifact. The claim this table supports is therefore narrow and should be read as stated: per attack proposed, guided acquisition reaches more of the defeat surface than direct prompting does. It is not a claim that guided acquisition is the cheaper way to enumerate attack ideas, and we do not make one. Two readings matter for scale. Unguided allocation tracks naive enumeration at every budget, so the co-adaptation loop on its own buys no coverage and the mask is the entire contribution. And the guided advantage is a constant factor that establishes by roughly 130 records and then holds flat over a further threefold budget, so it does not compound: with a fixed total budget over a large obligation set the return comes from covering more obligations, not from probing each one harder. The defeat-mode axis is what the unresolved-cell mask optimises, and the predicate decomposition is independent of it. Raw ratios on the unmatched budgets are 1.69× and 2.25×, so most of the apparent advantage is budget rather than allocation. Mode labelling carries κ=0.42κ=0.42 (Appendix C), which bounds how hard any of these cell counts can be pushed.