Paper deep dive
Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness
Fouad Bousetouane
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/1/2026, 1:58:54 AM
Summary
This paper introduces the ProofAgent Index (PAI), a governance readiness framework for AI agents that distinguishes production readiness from mere capability. PAI evaluates agents across four dimensions: Evaluation (behavior), Context (operating environment), Compliance (rule alignment), and Governance (organizational control). Implemented in ProofAgent Harness, PAI uses a weighted geometric mean and hard block rules to prevent unsafe deployments, validated in healthcare and finance domains.
Entities (11)
Relation Signals (9)
ProofAgent Index → hascomponent → Evaluation
confidence 95% · PAI combines four dimensions of deployment evidence: Evaluation, Context, Compliance, and Governance.
ProofAgent Index → hascomponent → Context
confidence 95% · PAI combines four dimensions of deployment evidence: Evaluation, Context, Compliance, and Governance.
ProofAgent Index → hascomponent → Compliance
confidence 95% · PAI combines four dimensions of deployment evidence: Evaluation, Context, Compliance, and Governance.
ProofAgent Index → hascomponent → Governance
confidence 95% · PAI combines four dimensions of deployment evidence: Evaluation, Context, Compliance, and Governance.
ProofAgent Index → implementedin → ProofAgent Harness
confidence 95% · PAI is implemented inside ProofAgent Harness, an open source infrastructure for auditable AI agent evaluation and governance.
Capability → doesnotdetermine → Readiness
confidence 92% · capability improves behavior but does not determine readiness by itself
ProofAgent Index → validatedindomain → Healthcare
confidence 90% · Validation across two heavily regulated domains, healthcare and finance, shows that PAI carries held out readiness signal...
ProofAgent Index → validatedindomain →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:AI agents are moving into production workflows where they retrieve information, call tools, maintain state, and act on behalf of users or organizations, but many release decisions still rely on capability signals, demos, or behavioral tests that do not show whether an agent is ready to operate under production constraints. Capability is therefore not production readiness. This paper introduces the ProofAgent Index (PAI), a governance readiness index for AI agents. PAI combines four dimensions of deployment evidence: Evaluation, Context, Compliance, and Governance. Evaluation measures observed behavior, Context measures the operating environment that shapes that behavior, Compliance measures alignment with applicable rules and controls, and Governance measures whether the organization can authorize, monitor, audit, and control the agent during operation. PAI is implemented inside ProofAgent Harness, an open source infrastructure for auditable AI agent evaluation and governance. Validation across two heavily regulated domains, healthcare and finance, shows that PAI carries held out readiness signal and separates higher risk from lower risk configurations. The results show that context engineering strongly changes reliability, capability improves behavior but does not determine readiness, and governance evidence must remain visible rather than averaged away. PAI reframes agent release from a faith based deployment decision into an auditable readiness decision.
Tags
Links
- Source: https://arxiv.org/abs/2607.27677v1
- Canonical: https://arxiv.org/abs/2607.27677v1
Trouble viewing inline? Open PDF directly →
Full Text
53,686 characters extracted from source content.
Expand or collapse full text
Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness Fouad Bousetouane 1,2 1 ProofAgent.ai 2 The University of Chicago, USA bousetouane@uchicago.edu Abstract AI agents are moving into production workflows where they retrieve information, call tools, maintain state, and act on behalf of users or organizations. Yet many release decisions still rely on capability signals, demos, or behavioral tests that do not show whether an agent is ready to operate under production constraints. A capable agent can still be unsafe to deploy if its context is weak, its tool use is poorly controlled, its compliance evidence is incomplete, or its governance controls are missing. Capability is therefore not production readiness. This paper introduces the ProofAgent Index (PAI), a governance readiness in- dex for AI agents. PAI combines deployment evidence across Evaluation, Context, Compliance, and Governance. Evaluation measures observed behavior. Context mea- sures the operating environment that shapes that behavior. Compliance measures alignment with applicable rules and controls. Governance measures whether the orga- nization can authorize, monitor, audit, and control the agent during operation. PAI is implemented inside ProofAgent Harness 1 , an open source infrastructure for auditable AI agent evaluation and governance. The implementation produces de- composed axis scores, readiness bands, hard block decisions, traceable evidence, and release reports. Validation across heavily regulated domains, healthcare and finance, shows that PAI carries held out readiness signal and separates higher risk from lower risk config- urations. The results show that context engineering strongly changes agent reliability, that capability improves behavior but does not determine readiness by itself, and that governance evidence must remain visible rather than averaged away. PAI reframes agent release from a faith based deployment decision into an auditable readiness decision. AI agents should not be shipped because they appear capable; they should be shipped only when there is evidence that they are ready to operate inside a governed production environment. 1 https://github.com/ProofAgent-ai/proofagent-harness 1 arXiv:2607.27677v1 [cs.MA] 30 Jul 2026 Contents 1 Introduction3 1.1 From Capability to Readiness . . . . . . . . . . . . . . . . . . . . . . . . . .3 1.2 The ProofAgent Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4 2 Related Work: From Agent Evaluation to Governance4 2.1 AI agent evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 2.2 Context engineering and context quality . . . . . . . . . . . . . . . . . . . .6 2.3 Safety and adversarial evaluation . . . . . . . . . . . . . . . . . . . . . . . .6 2.4 Compliance and governance . . . . . . . . . . . . . . . . . . . . . . . . . . .7 3 The ProofAgent Index7 3.1 Readiness dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 3.2 Aggregation and admissibility gate . . . . . . . . . . . . . . . . . . . . . . .8 3.3 Hard block rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 3.4 Readiness bands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 3.5 Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 4 Empirical Validation of PAI as an AI Readiness Risk Signal10 4.1 Evaluation goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10 4.2 Evaluation setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10 4.3 Held out validation strategy . . . . . . . . . . . . . . . . . . . . . . . . . . .12 4.4 AUC calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12 4.5 Configuration level PAI values . . . . . . . . . . . . . . . . . . . . . . . . . .13 4.6 Ablation analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14 4.7 Large scale turn validation . . . . . . . . . . . . . . . . . . . . . . . . . . . .15 4.8 Context effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16 4.9 Capability effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16 4.10 Joint context and capability effect . . . . . . . . . . . . . . . . . . . . . . . .16 4.11 Domain stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17 4.12 Failure mode coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17 4.13 Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18 5 Implementation in ProofAgent Harness18 5.1 CLI workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 5.2 Mapping the CLI to PAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 5.3 Governance profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 5.4 Reports and release review . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 6 Conclusion21 2 1 Introduction AI agents are moving from controlled demonstrations into production systems where they retrieve information, call tools, maintain state, interact across multiple turns, and take actions on behalf of users or organizations. This shift changes the nature of risk. A traditional language model may produce an incorrect answer. An agent may produce an incorrect answer, use the wrong tool, follow a malicious instruction, expose sensitive information, violate a business policy, or execute an action outside its approved scope. In regulated domains such as healthcare and finance, these failures are not merely performance defects. They become compliance, policy, safety, and governance failures. Current evaluation practice does not fully capture this transition. Agent evaluations have made important progress in measuring reasoning, planning, tool use, and multi turn task performance [17, 13, 11, 19]. LLM as a judge methods have also made scalable evaluation possible by allowing models to assess outputs against task specific rubrics [12, 18]. These tools are necessary, but they remain insufficient for production release decisions. They measure how an agent behaves under test. They do not determine whether the agent has enough evidence to be deployed, monitored, audited, and governed under a defined operating scope. 1.1 From Capability to Readiness The gap matters because production readiness is not the same as behavioral performance. Capability measures what an agent can do under test. Readiness measures whether that agent can be safely, lawfully, and accountably deployed under a defined production scope. This distinction matters because capable agents can still be unready. An agent may solve tasks, call tools, and produce fluent responses while lacking the context, compliance evidence, monitoring, ownership, approval, or rollback mechanisms required for production deployment. Conversely, governance artifacts alone cannot make an incapable or unsafe agent ready. Readiness is therefore a system property. It depends on the agent, the model, the en- gineered context, the policy environment, the compliance obligations, and the governance regime under which the agent is allowed to operate. A high capability signal is not permission to deploy. It is one component of a broader readiness decision. Regulatory and governance frameworks already recognize that AI systems require more than technical performance. The EU AI Act establishes obligations for AI systems based on risk and intended use [5]. The NIST AI Risk Management Framework emphasizes map- ping, measuring, managing, and governing AI risks across the system lifecycle [14]. ISO/IEC 42001 defines requirements for an organizational AI management system [8]. However, these frameworks are often applied as documentation, review, or compliance processes separate from agent evaluation. The result is a practical gap: technical teams can produce behav- ioral scores, while governance teams maintain control checklists, but deployment owners still lack a unified readiness signal that connects behavior, context, compliance, and governance evidence. 3 1.2 The ProofAgent Index This paper introduces the ProofAgent Index (PAI) to address that gap. PAI is a gover- nance readiness index for AI agent production deployment. It integrates four dimensions of deployment evidence: behavioral Evaluation (E), Context quality (Q), Compliance evidence (C), and Governance control (G). The purpose of PAI is not to replace behavioral evalua- tion, but to place it inside a broader readiness framework. A high behavioral score becomes one input to the release decision, not the release decision itself. The central claim is simple: AI agents should not be shipped on faith. A deployment decision requires decomposable evidence, limited compensation across critical dimensions, and the ability to block release when mandatory controls fail. A score that averages away unsafe tool behavior, missing compliance evidence, or absent ownership is not a readiness score. It is a false sense of readiness. The contributions of this paper are fourfold: 1. We define AI agent production readiness as a governance problem that extends beyond behavioral evaluation. 2. We introduce PAI as a four dimension readiness index integrating Evaluation, Context, Compliance, and Governance. 3. We describe an implementation of PAI inside ProofAgent Harness, an open source infras- tructure for auditable AI agent evaluation and governance. 4. We provide validation evidence across healthcare and finance showing that PAI carries held out readiness signal and that context, capability, and failure mode coverage matter for deployment risk. The paper is organized as follows. Section 2 positions PAI relative to agent evaluation, context engineering, safety testing, compliance, and governance. Section 3 defines the PAI model, its readiness dimensions, aggregation rule, hard block logic, and readiness bands. Section 4 presents the empirical validation, including the held out strategy, configuration level AUC, and 10,000 turn validation across healthcare and finance. Section 5 describes the implementation of PAI inside ProofAgent Harness. Section 6 concludes. 2 Related Work: From Agent Evaluation to Gover- nance AI agent production readiness sits at the intersection of several lines of work that have largely evolved in parallel: agent evaluation, context engineering, safety and adversarial testing, regulatory compliance, and organizational governance. Each tradition contributes an important part of the readiness question, but none by itself answers whether an agent should be released into production. In this paper, evaluation means structured measurement of behavior under test. Com- pliance means whether the agent and its use satisfy applicable legal, regulatory, policy, 4 and control obligations. Governance means whether the organization has the ownership, approval, monitoring, audit, and accountability mechanisms required to operate the agent responsibly. PAI integrates these layers into one readiness decision. Figure 1 illustrates the readiness stack used in this paper. The lower layers generate technical evidence through agent infrastructure and ProofAgent Harness evaluation. The upper layers translate that evidence into compliance assurance and governance decisions. Figure 1: Multi level AI agent readiness framework. The lower layers generate tech- nical evidence through agent infrastructure and ProofAgent Harness evaluation, while the upper layers translate that evidence into compliance assurance and governance based de- ployment decisions. 2.1 AI agent evaluation Traditional language model benchmarks focus on reasoning, factuality, and task accuracy. As language models became more agentic, evaluation expanded to planning, action, and tool use. ReAct showed that language models can interleave reasoning and acting [17]. Toolformer demonstrated that models can learn to use tools [13]. AgentBench introduced a systematic framework for evaluating LLMs as agents across decision environments [11], and WebArena provided realistic web environments for autonomous agents [19]. Scalable evaluation methods have also advanced. G Eval showed that LLM based evalua- tors can align with human judgment in some generation settings [12]. MT Bench and Chatbot 5 Arena popularized LLM as a judge evaluation for conversational systems [18]. These meth- ods are important because production agents generate complex multi turn behavior that cannot always be scored through exact match rules. ProofAgent Harness extends this evaluation tradition into deployment governance by providing open infrastructure for adversarial, multi turn, and auditable AI agent evaluation [3]. Rather than evaluating isolated outputs, it evaluates agents across scenario sequences, traps, tool behavior, instruction following, manipulation resistance, safety, and hallucination resistance, while preserving turn evidence and structured reports. Evaluation remains necessary but incomplete. It answers how the agent behaved under test. It does not fully answer whether the agent is admissible for production under a de- fined risk tier, compliance scope, and governance regime. That gap motivates the readiness framework developed in Section 3. 2.2 Context engineering and context quality A growing body of work shows that agent behavior is shaped not only by the base model but also by the information environment in which the model operates. Retrieval Augmented Generation demonstrated that external retrieval can improve knowledge intensive generation by grounding model outputs in evidence [9]. Long context studies show that models do not use all context uniformly and may fail to recover relevant information depending on where and how it appears in the context window [10]. For agents, context extends beyond documents. It includes system instructions, role def- initions, memory state, tool schemas, grounding artifacts, guardrails, policies, and untrusted inputs. These elements collectively define the operating envelope of the agent. A weak or inconsistent context can cause failures even when the underlying model is capable. Prior work formalizes context quality as an independent readiness signal for AI agents rather than as a secondary prompt engineering concern [2]. In this view, the context sur- rounding an agent is evaluated before and separately from the agent’s downstream behavior. The context is assessed across role clarity, guardrail coverage, instruction consistency, tool schema quality, grounding sufficiency, injection hardening, and token efficiency. This makes context quality measurable as part of the agent’s operating environment, not merely inferred after a failure occurs. PAI adopts this view through the Context dimension, where context becomes one of the required forms of evidence for production readiness. 2.3 Safety and adversarial evaluation AI agent deployment introduces safety and robustness risks beyond ordinary task accuracy. Agents can hallucinate unsupported facts, follow malicious instructions, misuse tools, dis- close sensitive information, or act outside their approved scope. Prompt injection attacks are especially relevant for tool using and retrieval based agents because untrusted content can enter the agent context and conflict with higher priority instructions [7]. Adversarial evaluation and red teaming attempt to expose these failures before deployment [6, 11, 19]. Safety testing is essential, but it is not the same as readiness. A red team report can reveal unsafe behavior, yet deployment still requires a decision about whether the risk is acceptable, whether mitigations exist, whether compliance evidence is complete, and whether 6 governance controls can enforce the release decision. PAI uses adversarial evaluation as part of the Evaluation dimension, then connects that evidence to Context, Compliance, and Governance. 2.4 Compliance and governance Compliance asks whether the agent, its behavior, and its intended use satisfy applicable rules. This includes legal requirements, regulatory obligations, internal policies, and control frameworks. In healthcare, privacy and safety obligations may dominate. In finance, fairness, disclosure, consumer protection, and record keeping may dominate. Frameworks such as GDPR, HIPAA, GLBA, the EU AI Act, NIST AI RMF, and ISO/IEC 42001 formalize many of these obligations [4, 15, 16, 5, 14, 8]. Governance asks a different question: who owns the agent, who approved it, what scope is allowed, what monitoring is required, how incidents are handled, how rollbacks work, and when the agent must be reassessed. Compliance can be satisfied on paper while governance remains weak. Governance can be documented while behavior remains unsafe. Readiness requires both. PAI contributes a unified structure for these layers. It does not replace regulation, com- pliance review, or safety testing. It binds their evidence into a decomposable readiness signal that can support release decisions. 3 The ProofAgent Index The ProofAgent Index (PAI) is a governance readiness index for AI agents. Its purpose is to answer a deployment question that ordinary behavioral scores do not answer: Is this agent ready to operate in production under its intended scope, risk tier, compliance obligations, and governance controls? PAI is built on the premise that readiness is multi causal. An agent can fail because its behavior is unsafe, its context is weak, its compliance evidence is missing, or its governance controls are insufficient. A single behavioral score cannot represent all of these risks. PAI therefore combines four independently measured readiness dimensions. 3.1 Readiness dimensions PAI is computed from four normalized dimensions on a [0, 100] scale: Evaluation (E). Measures observed agent behavior under adversarial and expected con- ditions. In ProofAgent Harness this includes task success, hallucination resistance, safety, instruction following, manipulation resistance, and tool use. Context (Q). Measures the quality of the agent operating context, including role clarity, instructions, grounding, memory, tool schemas, guardrails, injection hardening, and token efficiency. 7 Compliance (C). Measures whether observed behavior and supporting artifacts satisfy applicable controls. Missing evidence is not treated as satisfied evidence. Governance (G). Measures ownership, scope definition, approval, monitoring, incident re- sponse, rollback, evidence retention, and lifecycle control. Together, the dimensions separate technical behavior from the conditions required for responsible deployment. Evaluation asks what the agent did. Context asks whether the information environment supports safe action. Compliance asks whether the agent satisfies applicable obligations. Governance asks whether the organization can control the agent over time. 3.2 Aggregation and admissibility gate PAI is computed in two stages. The first stage measures the balance of readiness evidence across the evaluated dimensions. The second stage applies release blocking rules that deter- mine whether the agent is admissible for production. Let P =E,Q,C,G denote the required readiness dimensions for a complete PAI report. Each dimension a∈ P is normalized to [0, 100] and assigned a weight w a . Unless otherwise specified, all dimensions receive equal weight. The raw PAI score is computed as a weighted geometric mean: PAI raw = Y a∈P max(a,ε) w a ! 1 P a∈P w a .(1) The parameter ε > 0 is a log safe floor used when computing the score in logarithmic form: PAI raw = exp P a∈P w a log(max(a,ε)) P a∈P w a .(2) The geometric mean provides limited compensation across dimensions. A low dimension pulls the score down more sharply than an arithmetic mean would, so strong performance on one dimension can only partially offset weakness on another. Compensation is therefore limited, not eliminated. The second stage applies the admissibility gate. Let τ review denote the minimum score for review level readiness, let τ ready denote the score required for full readiness, and let δ > 0 denote a margin below the review threshold. The release cap is defined as: cap = ( τ review − δ, if a hard block condition is present, 100,otherwise. (3) The final PAI score is then: PAI = min (PAI raw , cap),(4) with the final score clamped to [0, 100]. 8 Eq. 1 defines the measurement layer. Eq. 4, together with the cap in Eq. 3 and the hard block rules, defines the admissibility gate. Critical deficiencies are not handled by the geometric mean. They are handled by the cap. For example, if a configuration has strong behavioral performance and strong context quality but triggers a prohibited use condition, the raw geometric score may still be high. The final PAI is capped below the review threshold, preventing the configuration from receiving a positive release verdict. If a required dimension lacks sufficient evidence, the configuration is marked as incomplete and assigned a release blocking cap until the missing evidence is supplied. This fail closed rule prevents missing Compliance or Governance evidence from being treated as readiness evidence. 3.3 Hard block rules A hard block caps the readiness score and prevents a positive release verdict even if the aggregate score would otherwise appear acceptable. Hard block conditions include prohibited use, critical safety failure, hallucination resistance below a required floor, tool use breach, critical technical finding, missing mandatory compliance evidence, unresolved governance finding, or insufficient capability for the deployment risk tier. The hard block layer prevents the most dangerous failure mode of composite scoring: averaging away a non negotiable defect. In a production release gate, some conditions must block deployment regardless of the average score. 3.4 Readiness bands PAI maps the numerical score and hard block status into operational readiness bands: Blocked. A hard block condition fired. The agent should not be released until the blocking issue is resolved. Not ready. The agent does not meet the minimum readiness bar for its intended scope or risk tier. Ready with caveats. The agent meets a review threshold, but remaining issues require monitoring, restrictions, or remediation. Ready. The agent satisfies the full readiness threshold and has no unresolved blocking condition. These bands make PAI usable for release review, continuous integration gates, audit reporting, regression testing, and executive governance. 3.5 Interpretation PAI should not be interpreted as a claim that an agent will never fail. A high PAI means that the agent has satisfied measured readiness requirements under the tested scope and evidence package. A low PAI means that the agent lacks sufficient readiness evidence, has 9 weak dimension performance, or has triggered blocking conditions. The final score is useful because it is decomposable. Its value is not only the number, but the map of what must improve before deployment. 4 Empirical Validation of PAI as an AI Readiness Risk Signal 4.1 Evaluation goal This section evaluates whether PAI provides a meaningful AI readiness risk signal for pro- duction deployment. The central question is: Do lower PAI scores correspond to higher risk of failure on unseen ProofAgent Harness traps? The evaluation uses one configuration grid across two regulated domains, three agent capability tiers, and two context conditions. The configuration level study tests whether PAI orders unseen PASS or FAIL outcomes. The 10,000 turn validation analyzes the same factor structure at scale and shows how context, capability, domain, and failure mode patterns explain readiness risk. The validation also tests the central thesis of the paper: capability is not readiness. Capability affects failure risk, but readiness depends on capability operating inside a governed context with sufficient compliance and governance evidence. Table 1: Evaluation goals for validating PAI. Goal QuestionEvidence used G1Does PAI order unseen failure risk?Held out AUC G2Does context engineering reduce defects? Defect rate by context G3Does capability affect failure risk?Defect rate by capability G4Does capability imply readiness?Capability and context interaction G5Does the signal hold across domains?Defect rate by domain G6Does PAI remain auditable?Decomposed readiness evidence 4.2 Evaluation setup The validation covers two heavily regulated domains: healthcare and finance. The configu- ration grid crosses three factors: domain, capability tier, and context condition. This yields twelve configurations. Capability tier refers only to the evaluated agent backbone. It is not derived from any ProofAgent Harness score. This preserves the tier as an external ordinal factor based on model class and generation. The weak tier uses an 8B model, the mid tier uses a 70B model, and the strong tier uses a newer mixture of experts model. 10 The context condition is the second experimental factor. B0 represents a baseline oper- ating context with weak instructions, weak grounding, weak tool schema constraints, limited guardrails, and no complete governance profile. B5 represents a fuller operating context with stronger instructions, grounding, tool structure, guardrails, and governance evidence. B0 should not be read as a no tool condition. Tool interfaces remain available where required by the scenario, but they are weakly specified and weakly governed. Table 2: Evaluation setup. DimensionValuesPurpose DomainHealthcare, finance Regulated domain coverage CapabilityWeak, mid, strongAgent backbone capability tier ContextB0, B5Operating context quality Configurations12Domain, capability, and context grid Held out study turns 420Development and exam evidence Validation turns10,000Deployment scale turn evidence Sessions1,000Multi turn validation sessions Table 3: Agent backbones used in the capability tiers. Tier IDModelCapability definition 0Weakamazon/meta.llama3-8b-instruct-v1:08B backbone 1Midscaleway/llama-3.3-70b-instruct70B backbone 2Strong databricks/databricks-llama-4-maverick Llama 4 Maverick MoE All evaluated agents were accessed through Eden AI’s EU endpoint 6. This provides an all EU agent execution pipeline for the evaluated systems. Eden AI is acknowledged separately for infrastructure support. Table 4: Harness LLMs used for scoring. RoleModelLocation Primary harness LLMgemma-3-27b-itLocal LM Studio Fallback harness LLM scaleway/qwen3-235b-a22b-instruct-2507 Eden AI EU endpoint The harness uses three scoring personas with Debate consensus. Compliance checks use compliancepasses: 3, corresponding to a per control majority vote. The primary harness LLM is deliberately a small local model. This tests whether a low cost, on premises harness LLM can screen readiness before deployment. The fallback invocation rate should be reported from run artifacts because fallback use changes the scoring model used on a turn. 11 Table 5: Regulatory and governance frameworks used during evaluation. FrameworkJurisdiction Relevance EU AI ActEUAI system risk and deployment obligations GDPREUPrivacy and personal data protection CCPA / CPRA US (CA)Consumer privacy obligations NIST AI RMFUSAI risk management and governance HIPAAUSHealthcare privacy and protected health information SOC 2USSecurity, availability, and control assurance GLBAUSFinancial privacy and safeguards FINRA / SECUSFinancial conduct, supervision, and records 4.3 Held out validation strategy The main validation uses disjoint development and exam evidence. For each configuration, ProofAgent Harness creates two separate trap packs. The development pack contains 25 turns and is used to compute Evaluation, Context, Compliance, Governance, and the final PAI score. The held out exam pack contains 10 turns and uses different traps. It is used only to assign the PASS or FAIL outcome. This separation prevents direct circular validation. PAI is computed from one set of turns, while failure is measured on a different set of unseen turns. The predictor and outcome therefore come from non overlapping trap evidence. In this experiment, a configuration is marked FAIL if the held out exam pack contains a release blocking failure or violates the failure tolerance defined for the deployment risk tier. Otherwise, the configuration is marked PASS. The PASS or FAIL rule is applied only to the held out exam pack and is not used to compute PAI. Table 6: Held out validation protocol. ComponentDevelopment packHeld out exam pack PurposeCompute E, Q, C, G and PAI Measure unseen failure Turns per configuration 2510 Trap sourceProofAgent Harness trapsDifferent traps OverlapNoneNone OutputDimension scores and PAIPASS or FAIL Used as predictorYesNo Used as outcomeNoYes 4.4 AUC calculation PAI is a configuration readiness index, not a per turn classifier. Each configuration con- tributes one PAI score and one held out outcome. Lower PAI is interpreted as higher readi- ness risk. The AUC measures whether configurations that fail the held out exam receive lower PAI scores than configurations that pass. 12 Across the twelve configurations, PAI achieves AUC = 0.98. The result reflects strong ordering of held out readiness risk, with one tied boundary case between the highest scoring failing configuration and the lowest scoring passing configuration. This makes the separation strong but not perfect, and avoids overstating robustness in a twelve configuration study. Table 7: Primary held out AUC result for PAI. QuantityValueMeaning Unit of analysisConfigurationOne PAI and one outcome per configuration Configurations12Fully crossed readiness units Failed configurations8Positive class for AUC Passing configurations 4Negative class for AUC AUC0.98Strong ranking of unseen failure risk Boundary caseOne tied pairHighest FAIL and lowest PASS at the score boundary PredictorPAIComputed from development pack OutcomePASS or FAIL Measured on held out exam pack Figure 2: Configuration level ROC for PAI. PAI is computed from development trap evidence, while failure is measured on unseen ProofAgent Harness traps. Lower PAI is interpreted as higher readiness risk. Across the twelve configurations, PAI achieves AUC = 0.98, with one tied boundary case at the score boundary. 4.5 Configuration level PAI values Table 8 reports the configuration level PAI values used for the held out AUC calculation. Each row corresponds to one configuration in the same domain, capability, and context grid. PAI is computed from the development pack. The PASS or FAIL outcome is measured on the disjoint held out exam pack. 13 The table uses the default experimental policy values ε = 1, τ review = 65, τ ready = 85, and δ = 1. A hard block caps the final score below the review threshold when the raw score would otherwise exceed the release boundary. The band is derived from the final PAI score and hard block status. All PAI values are reported to two decimal places. Table 8: Configuration level PAI values and held out outcomes. DomainCap.Ctx. E Q C G Raw Block Final BandOutcome FinanceWeak B0228 55 50 19.81 Yes 19.81 Blocked FAIL Healthcare Weak B0330 56 51 22.52 Yes 22.52 Blocked FAIL FinanceMidB05038 68 61 52.98 Yes 52.98 Blocked FAIL Healthcare MidB05140 69 62 54.35 Yes 54.35 Blocked FAIL FinanceStrong B05142 70 63 55.44 Yes 55.44 Blocked FAIL Healthcare Strong B05244 71 64 56.78 Yes 56.78 Blocked FAIL FinanceWeak B54878 82 78 69.95No69.95 Caveats FAIL Healthcare Weak B552.04 92 92 93 80.00No80.00 Caveats FAIL FinanceMidB59672 75 79 80.00No80.00 Caveats PASS Healthcare MidB59988 89 87 90.63No90.63 ReadyPASS FinanceStrong B59987 90 88 90.88No90.88 ReadyPASS Healthcare Strong B59989 91 90 92.17No92.17 ReadyPASS The table makes the AUC reproducible without treating the score as more robust than it is. The highest failing configuration and the lowest passing configuration meet at the score boundary. This tied boundary case is why the held out AUC is high but not perfect. The table also shows why PAI must remain decomposable. Mid and strong agents un- der B0 receive higher Evaluation scores than weak agents, but weak context and blocking conditions prevent release. This is the central distinction between capability and readiness: higher behavioral capability can coexist with low production readiness. The Healthcare Weak B5 row is especially important. It has strong context, compliance, and governance evidence, but the Evaluation score remains low because the weak backbone still produces high defect risk. The configuration reaches the score boundary but fails the held out exam. This is the clearest capability floor case in the study: governance and context can improve an agent, but they cannot fully rescue insufficient capability. 4.6 Ablation analysis Table 9 compares PAI against simpler readiness predictors on the same held out outcomes. The behavioral Evaluation dimension alone captures capability related behavior, but it does not fully capture production readiness. The full PAI score adds context, compliance, gover- nance, and release blocking logic. 14 Table 9: Ablation analysis on the held out configuration outcomes. PredictorHeld out AUC E only0.80 Q only0.88 Geometric mean of E and Q0.94 Arithmetic mean of E, Q, C, G0.94 PAI without hard block cap0.97 PAI with hard block cap0.98 The ablation supports the claim that capability is not readiness. The Evaluation di- mension alone achieves AUC = 0.80, while the final PAI achieves AUC = 0.98. Capability therefore provides signal, but it does not fully explain held out readiness outcomes. Context improves discrimination, and the full PAI further improves the signal by adding compliance, governance, and hard block logic. The comparison between arithmetic aggregation and the final PAI also supports the need for a readiness gate. Arithmetic aggregation and uncapped scores can still allow strong values on some dimensions to offset weak readiness evidence on another dimension. The capped PAI prevents release blocking conditions from being averaged away. 4.7 Large scale turn validation The 10,000 turn validation uses the same configuration grid and agent backbone assignments. It tests whether the readiness factors represented by PAI produce coherent defect patterns at deployment scale. The validation set contains 10,000 evaluated turns across 1,000 sessions, two regulated domains, three capability tiers, and two context conditions. Across all turns, 4,179 defects are observed, corresponding to an overall defect rate of 41.79%. The reported defect rates are trap conditional rates. They should not be interpreted as expected production failure rates. ProofAgent Harness traps are adversarial by design and are intended to expose failure modes before deployment. Each defective turn is assigned one primary defect type, so defect type counts are mutually exclusive and sum to the total number of defective turns. Table 10: Summary of the 10,000 turn validation set. QuantityValueDescription Evaluated turns10,000Total validation turns Sessions1,000Multi turn evaluation sessions Configurations12Domain, capability, and context combinations Domains2Healthcare and finance Capability tiers3Weak, mid, strong Context conditions 2B0 and B5 Observed defects4,179Deployment relevant failures Overall defect rate41.79% Trap conditional defect rate 15 4.8 Context effect Context engineering has the strongest observed effect. The B0 condition produces 3,287 defects across 5,000 turns, for a defect rate of 65.74%. The B5 condition produces 892 defects across 5,000 turns, for a defect rate of 17.84%. This is a 47.90 percentage point absolute reduction and a 72.9% relative reduction. Table 11: Observed defect rate by context condition. Context conditionTurns Defects Defect rate B05,0003,28765.74% B55,00089217.84% Absolute reduction–47.90 p Relative reduction–72.9% 4.9 Capability effect Capability also matters, but its effect saturates. Weak agents produce 2,510 defects across 3,334 turns, for a defect rate of 75.28%. Mid agents produce 839 defects across 3,334 turns, for a defect rate of 25.16%. Strong agents produce 830 defects across 3,332 turns, for a defect rate of 24.91%. The major improvement occurs from weak to mid. The mid and strong tiers are nearly identical in aggregate defect rate. This supports the paper’s core claim: capability matters, but capability alone is not readiness. Table 12: Observed defect rate by capability tier. Capability tier Turns Defects Defect rate Weak3,3342,51075.28% Mid3,33483925.16% Strong3,33283024.91% 4.10 Joint context and capability effect The interaction between context and capability is the clearest evidence that capability is not readiness. Engineered context sharply reduces defects for mid and strong agents. Mid agents fall from 49.79% under B0 to 0.54% under B5. Strong agents fall from 48.98% under B0 to 0.84% under B5. Weak agents also improve, but remain materially risky, falling from 98.44% under B0 to 52.13% under B5. This pattern shows that capability changes the baseline risk, but context determines whether that capability can be expressed safely and consistently. A mid capability agent under B5 is far more reliable than a strong capability agent under B0. Conversely, a weak agent remains risky even under B5, supporting the need for a minimum capability floor in high risk deployments. 16 Table 13: Observed defect rate by capability and context. Capability Context Turns Defects Defect rate WeakB01,6671,64198.44% WeakB51,66786952.13% MidB01,66783049.79% MidB51,66790.54% StrongB01,66681648.98% StrongB51,666140.84% 4.11 Domain stability The validation results are balanced across the two regulated domains. Finance produces 2,083 defects across 5,002 turns, for a defect rate of 41.64%. Healthcare produces 2,096 defects across 4,998 turns, for a defect rate of 41.94%. Table 14: Observed defect rate by regulated domain. DomainTurns Defects Defect rate Finance5,0022,08341.64% Healthcare4,9982,09641.94% 4.12 Failure mode coverage The observed defects span the major failure modes that matter for AI agent deployment. Safety defects account for 1,385 cases. Tool use defects account for 1,381 cases. Hallucination resistance defects account for 1,314 cases. Phantom tool call claims account for 99 cases. These failure modes reinforce why readiness must remain broader than capability. A ca- pable agent may still fail through unsafe action, tool misuse, unsupported claims, or phantom tool call behavior. These are deployment relevant failures, not merely task performance er- rors. Table 15: Observed defect types across the 10,000 turn validation set. Each defective turn is assigned one primary defect type. Defect typeCount Share of defects Safety1,38533.14% Tool use1,38133.05% Hallucination resistance1,31431.44% Phantom tool call claimed992.37% Total4,179100.00% 17 4.13 Interpretation The validation supports six conclusions. First, PAI provides strong held out readiness dis- crimination, with AUC = 0.98 across twelve configurations. The result is reproducible from the configuration table and includes one tied boundary case, which avoids overstating ro- bustness. Second, the validation is not circular with respect to trap exposure because PAI is com- puted from the development pack and the PASS or FAIL outcome is measured on a disjoint held out exam pack. The harness LLM and scoring protocol are fixed across all configura- tions, although fallback use should be reported from the run artifacts for full reproducibility. Third, the 10,000 turn validation shows that context engineering has the largest observed effect, reducing trap conditional defects from 65.74% under B0 to 17.84% under B5. Fourth, the results support the distinction between capability and readiness. The weak to mid capability jump reduces failures sharply, but the mid to strong jump does not. In contrast, context changes the reliability of mid and strong agents by nearly two orders of magnitude. Capability is therefore necessary, but not sufficient. Fifth, weak agents remain risky even under B5. This supports treating minimum capa- bility as a possible release condition for high risk deployments. Sixth, PAI must remain decomposable. The final score provides the release signal, but the dimension evidence, hard block status, and failure traces explain why a configuration is ready, blocked, or risky. Taken together, these results support PAI as a holistic AI readiness risk signal for agent deployment in regulated environments. 5 Implementation in ProofAgent Harness PAI is implemented inside ProofAgent Harness [1], an open source evaluation and gov- ernance infrastructure for AI agents. The harness turns an agent run into a structured readiness artifact containing behavioral evidence, context evidence, compliance evidence, governance metadata, hard block findings, readiness bands, and exportable reports. The package is available as a PyPI package. The source code is available in the GitHub repository. Additional usage details are provided in the ProofAgent Harness documentation. 18 5.1 CLI workflow ProofAgent Harness is installed as a Python package and exposes the proof command line interface: pip install proofagent-harness proof version A representative governed evaluation run is shown below: proof run examples/credit_agent/agent.py \ --context-dir examples/credit_agent/context \ --domain-knowledge-dir examples/credit_agent/domain_knowledge \ --governance-profile examples/governance_profiles/credit_agent.yaml \ --assess-context \ --assess-compliance \ --agent Agent_name \ --profile default_multi_turn \ --llm gemma-3-27b-it \ --fallback-llm scaleway/qwen3-235b-a22b-instruct-2507 \ --turns 8 \ --agent-version "$(git rev-parse --short HEAD)" \ --source manual \ --environment development \ --json run.json \ --seed 42 \ --markdown run.md This command creates a governed evaluation run. It evaluates the agent under adver- sarial multi turn scenarios, assesses context and compliance evidence, attaches governance metadata, and exports the evidence bundle used for PAI computation and release review. 5.2 Mapping the CLI to PAI The command maps directly to the four PAI dimensions. The agent run produces the Evaluation evidence. The context directory and context assessment produce the Context evidence. The compliance assessment produces the Compliance evidence. The governance profile, agent identity, version, source, and environment metadata produce the Governance evidence. 19 InputRole in PAI --context-dirLoads the operating context used to assess context quality. --domain-knowledge-dirLoads grounding material for domain specific evaluation. --governance-profileLoads governance as code for the evaluated agent. --assess-contextProduces the Context axis and context findings. --assess-complianceProduces the Compliance axis and control findings. --llm and --fallback-llmSelect the primary and fallback harness LLMs. --turnsSets the number of multi turn evaluation turns. --agent-versionRecords the evaluated code version. --sourceand --environment Record run provenance and deployment context. --json and --markdownExport machine readable and human readable reports. --uploadSends the completed run to the governance platform for review. Table 16: Mapping from ProofAgent Harness CLI inputs to PAI evidence generation. 5.3 Governance profile The governance profile connects technical evaluation to production readiness. It defines the agent’s intended use, autonomy level, data sensitivity, operating region, oversight require- ments, and release policy. A simplified profile is: agent_governance_profile: name: "CreditLine Concierge" fail_on: block intake: use_case: creditworthiness autonomy_level: L3 data_sensitivity: pii region: eu human_oversight: false takes_consequential_actions: true This profile gives the harness the risk context needed to interpret the evaluation. A cred- itworthiness agent, for example, requires stricter treatment of personal data, consequential actions, fairness risk, and human oversight than a low risk internal assistant. 5.4 Reports and release review Each run can produce a JSON report for reproducibility and a Markdown report for human review. When upload is enabled, the evidence bundle can also be sent to the ProofAgent governance platform for release review. The report carries two readiness figures rather than one. The gate is the score after the hard block cap has been applied, and is the value a release decision is made against. The gauge is the same weighted aggregate before the cap. The distinction matters in practice: once an agent is hard blocked the gate stops moving, so only the gauge reveals whether successive versions are improving. A blocked report also names 20 which finding produced the cap, so the decision is auditable rather than asserted. This is the operational form of PAI. The readiness score is tied to an agent version, context package, domain knowledge source, governance profile, evaluation trace, compliance evidence, and release decision 6 Conclusion AI agents should not be shipped on faith. Behavioral capability is necessary, but it is not sufficient for production readiness. A production ready agent must show evidence that it can operate safely, lawfully, and accountably inside a defined production environment. That evidence must span behavior, context, compliance, and governance. This paper introduced the ProofAgent Index (PAI) as a governance readiness index for AI agent deployment. PAI combines four dimensions of deployment evidence, applies limited compensation through geometric aggregation, enforces hard block conditions, and preserves decomposable evidence for audit, review, and remediation. By implementing PAI inside ProofAgent Harness, the index becomes operational: it can be executed through an evaluation workflow, tied to an agent version, connected to a governance profile, and used to support release review. The validation shows that PAI carries meaningful held out readiness signal. It achieves AUC = 0.98 across twelve configuration level readiness units, with one tied boundary case at the readiness threshold. The 10,000 turn validation further shows that deployment risk is shaped by context engineering, capability, domain stability, and failure mode coverage. The results support the central claim of the paper: capability improves agent behavior, but capability alone does not make an agent production ready. The practical implication is direct. AI engineers need more than behavioral scores. Risk teams need more than policy checklists. Business leaders need more than impressive model demonstrations. PAI provides a shared readiness language for all three groups: measurable enough for engineering, auditable enough for risk and compliance, and interpretable enough for executive release decisions. As AI agents become more autonomous and more deeply embedded in enterprise work- flows, governance cannot remain a document produced after deployment. It must become part of the release mechanism itself. PAI is a step in that direction: a way to convert agent evaluation from an act of confidence into an auditable readiness decision. Conflict of Interest The author is affiliated with ProofAI LLC and ProofAgent.ai, which develop ProofAgent Harness and the ProofAgent Index described in this paper. The methods, analysis, conclu- sions, and readiness framework presented here are solely those of the author. 21 Acknowledgments This work was developed with the support of ProofAI LLC as part of the ProofAgent.ai open source initiative (https://w.proofagent.ai). The author thanks the ProofAgent.ai com- munity and early users for feedback on AI agent evaluation, adversarial testing, governance workflows, and evidence linked reporting. The author also thanks Eden AI (https://w w.edenai.co) for providing evaluation credits and access to large language models hosted in the European Union region. This infrastructure support enabled large scale validation of the ProofAgent evaluation pipeline across approximately 120 million tokens of agent evaluation workloads. References [1] Fouad Bousetouane. Human-on-the-bridge: Scalable evaluation for ai agents, 2026. URL https://arxiv.org/abs/2606.16871. [2] Fouad Bousetouane. Ai agents do not fail alone: The context fails first. arXiv preprint, 2026. URL https://arxiv.org/abs/2607.14275. [3] Fouad Bousetouane. Proofagent harness: Open infrastructure for adversarial evaluation of ai agents. arXiv preprint arXiv:2605.24134, 2026. doi: 10.48550/arXiv.2605.24134. URL https://arxiv.org/abs/2605.24134. [4] European Union. Regulation (eu) 2016/679 (general data protection regulation), 2016. URL https://eur-lex.europa.eu/eli/reg/2016/679/oj. Official Journal of the European Union. [5] European Union. Regulation (eu) 2024/1689 laying down harmonised rules on artificial intelligence, 2024. URL https://eur- lex.europa.eu/eli/reg/2024/1689/oj. Official Journal of the European Union. [6] Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Samuel R. Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan, Danny Hernandez, Tristan Hume, Josh Jacobson, Scott Johnston, Shauna Kravec, Catherine Olsson, Sam Ringer, Eli Tran-Johnson, Dario Amodei, Tom Brown, Nicholas Joseph, Sam McCandlish, Chris Olah, Jared Kaplan, and Jack Clark. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858, 2022. doi: 10.48550/arXiv.2209.07858. URL https://arxiv.org/abs/ 2209.07858. [7] Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. arXiv preprint arXiv:2302.12173, 2023. doi: 10.48550/arXiv.2302.12173. URL https://arxiv.org/abs/2302.12173. 22 [8] ISO/IEC.Iso/iec 42001:2023:Information technology—artificial intelligence— management system, 2023. URL https://w.iso.org/standard/42001. International Organization for Standardization and International Electrotechnical Commission. [9] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K ̈uttler, Mike Lewis, Wen-tau Yih, Tim Rockt ̈aschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Processing Systems, 2020. doi: 10.48550/a rXiv.2005.11401. URL https://arxiv.org/abs/2005.11401. [10] Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173, 2024. doi: 10.1162/tacl a00638. URL https://aclanthology.org/2024.tacl-1.9/. [11] Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. Agentbench: Evaluating llms as agents. In International Conference on Learning Representations, 2024. doi: 10.48550/arXiv.2308.03688. URL https://arxiv.org/abs/2308.03688. [12] Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu.G-eval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303.16634, 2023.doi: 1 0 . 4 8 5 5 0 / a r X i v . 2 3 0 3 . 1 6 6 3 4.URL https://arxiv.org/abs/2303.16634. [13] Timo Schick, Jane Dwivedi-Yu, Roberto Dess`ı, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Lan- guage models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023. doi: 10.48550/arXiv.2302.04761. URL https://arxiv.org/abs/2302.04761. [14] Elham Tabassi. Artificial intelligence risk management framework (ai rmf 1.0). Technical Report NIST AI 100-1, National Institute of Standards and Technology, 2023. URL https://doi.org/10.6028/NIST.AI.100-1. [15] U.S. Congress. Health insurance portability and accountability act of 1996, 1996. URL https://w.congress.gov/bill/104th-congress/house-bill/3103. Public Law 104-191. [16] U.S. Congress. Gramm-leach-bliley act, 1999. URL https://w.congress.gov/bill/10 6th-congress/senate-bill/900. Public Law 106-102. [17] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations, 2023. doi: 10.48550/arXiv.2210.03629. URL https://arxiv.org/abs/2210.03629. 23 [18] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems Datasets and Benchmarks Track, 2023. doi: 10.48550/arXiv.2306.05685. URL https://arxiv.org/abs/2306.05685. [19] Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xi- anyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neu- big. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854, 2023.doi: 1 0 . 4 8 5 5 0 / a r X i v . 2 3 0 7 . 1 3 8 5 4.URL https://arxiv.org/abs/2307.13854. 24