Paper deep dive

Agent-Integrated Software: Interaction Contracts and Continuous Assurance

Shengcheng Yu, Chunrong Fang, Zhenyu Chen

Year: 2026Venue: arXiv preprintArea: cs.SEType: PreprintEmbeddings: 96

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%

Last extracted: 9/11/2026, 5:12:03 AM

Summary

The paper introduces Agent-Integrated Software (AIS) as a software pattern that embeds intelligent agents into existing applications while maintaining a conventional core. It proposes Intent-Level Interaction Abstraction (IIA) to manage task semantics and interaction contracts to ensure dependable integration by linking task-level interactions with application behavior. The framework uses an open transition-system model to relate execution to task states, addressing coordination problems where users revise goals while delegated execution continues.

Entities (8)

Chunrong Fang · author · 99%Shengcheng Yu · author · 99%Zhenyu Chen · author · 99%Agent-Integrated Software · software-pattern · 98%Intent-Level Interaction Abstraction · abstraction-model · 97%Interaction Contract · specification · 96%Open Transition System · mathematical-model · 94%Continuous Assurance · process · 93%

Relation Signals (7)

Zhenyu Chen → affiliatedwith → Nanjing University

confidence 99% · Zhenyu Chen Affiliation: State Key Laboratory for Novel Software Technology, Nanjing University

Shengcheng Yu → affiliatedwith → Technical University of Munich

confidence 99% · Shengcheng Yu Affiliation: Technical University of Munich

Chunrong Fang → affiliatedwith → Nanjing University

confidence 99% · Chunrong Fang... Affiliation: State Key Laboratory for Novel Software Technology, Nanjing University

Agent-Integrated Software → includes → Intent-Level Interaction Abstraction

confidence 95% · AIS integrates a conventional application core with a built-in intelligent agent... Within AIS, the Intent-Level Interaction Abstraction (IIA) gives delegated work a user-facing semantics

Interaction Contract → constrains → AIS execution

confidence 94% · Interaction contracts constrain this relation through task bindings, role-specific authority, control transitions, and outcome evidence

Open Transition System → models → AIS execution

confidence 93% · An open transition-system model relates AIS execution to IIA states and events

Continuous Assurance → maintains → scoped claims

confidence 92% · continuous assurance maintains scoped claims as their dependencies change

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Embedding an intelligent agent in an existing application creates a persistent coordination problem: users can revise goals and manipulate shared objects while delegated execution continues. We argue that dependable integration requires an explicit correspondence between task-level interaction and application behavior. We introduce Agent-Integrated Software (AIS) as a software pattern combining a conventional core, direct interaction, and a built-in agent, and Intent-Level Interaction Abstraction (IIA) as the task semantics through which users inspect and control delegated work. An open transition-system model relates AIS execution to IIA states and events. Interaction contracts constrain this relation through task bindings, role-specific authority, control transitions, and outcome evidence; continuous assurance maintains scoped claims as their dependencies change. A compact disclosure contract and conditional propositions illustrate why local component validity is insufficient and how selected admission invariants can be separated from planning. Contrasting software domains expose the framework's assumptions and limits. This perspective develops a research agenda spanning application abstraction, development support, controlled execution, quality assessment, and human supervision, with the aim of making agent integration a maintainable software engineering discipline.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

92,681 characters extracted from source content.

Expand or collapse full text

Engineering Agent-Integrated Software: Interaction Contracts and Continuous Assurance CCS: Software and its engineering Software design engineeringCCS: Software and its engineering Software testing and debuggingCCS: Software and its engineering Software system modelsCCS: Computing methodologies Intelligent agents Shengcheng Yu Affiliation: Technical University of Munich, Heilbronn, Germany, 74076 email: shengcheng.yu@tum.de , Chunrong Fang Note: Chunrong Fang is the corresponding author. Affiliation: State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China, 210093 email: fangchunrong@nju.edu.cn and Zhenyu Chen Affiliation: State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China, 210093 email: zychen@nju.edu.cn Date: 10 September 2026 Abstract. Embedding an intelligent agent in an existing application creates a persistent coordination problem: users can revise goals and manipulate shared objects while delegated execution continues. We argue that dependable integration requires an explicit correspondence between task-level interaction and application behavior. We introduce Agent-Integrated Software (AIS) as a software pattern combining a conventional core, direct interaction, and a built-in agent, and Intent-Level Interaction Abstraction (IIA) as the task semantics through which users inspect and control delegated work. An open transition-system model relates AIS execution to IIA states and events. Interaction contracts constrain this relation through task bindings, role-specific authority, control transitions, and outcome evidence; continuous assurance maintains scoped claims as their dependencies change. A compact disclosure contract and conditional propositions illustrate why local component validity is insufficient and how selected admission invariants can be separated from planning. Contrasting software domains expose the framework’s assumptions and limits. This perspective develops a research agenda spanning application abstraction, development support, controlled execution, quality assessment, and human supervision, with the aim of making agent integration a maintainable software engineering discipline. Keywords: Agent-Integrated Software, Intent-Level Interaction Abstraction, Software Engineering, Human–AI Interaction, Application Interfaces, Quality Evaluation 1. Introduction Embedding a language-model agent in an application changes the relationship between interaction and execution. A user can delegate a goal while continuing to edit the very objects on which the agent is acting. The application must then interpret two kinds of input: direct operations on its existing interface and instructions whose realization may require a sequence of inferred operations. Their coexistence creates a software engineering problem that cannot be assessed through the quality of the generated response alone. A convincing proposal, an authorized application programming interface (API) call, and a correct database update can still compose into an effect that no longer corresponds to the user’s instruction. Consider a collaboration application in which an organizer asks an agent to prepare meeting materials and send public versions to confirmed participants. The organizer inspects the proposal, replaces an attachment, and removes a recipient through the conventional graphical user interface (GUI). A delayed approval for the earlier proposal may remain authentic even though its scope is obsolete. If a delivery subsequently times out, the absence of a response does not establish whether disclosure occurred. This example brings the central difficulty into focus: the meaning of delegated work can change before, during, and after execution, while different people retain authority over the resources and consequences involved. We introduce Agent-Integrated Software (AIS) as a software-pattern concept for this setting. AIS integrates a conventional application core with a built-in intelligent agent, supporting both direct user interaction and goal-directed task execution. The core continues to own the application’s domain objects, business behavior, and durable state. The agent is integrated into the application’s functionality and lifecycle, although its model or runtime may execute remotely. The integrated agent uses a large language model or multimodal foundation model. Existing applications can adopt the pattern incrementally. Within AIS, the Intent-Level Interaction Abstraction (IIA) gives delegated work a user-facing semantics: goals, contextual references, proposals, endorsements, interventions, and outcomes. It is the abstraction through which users inspect and redirect a task, i.e., the meaning of the interaction rather than the component that performs inference. Its realization can span a conversational panel, an editable preview, existing GUI controls, and runtime services. The agent plans and executes; the IIA defines how that activity is presented and controlled; AIS identifies the application in which both interaction paths coexist. Our contribution is to make the relationship between interaction and application effects an explicit object of specification. We call this relationship an interaction–effect obligation. It binds an effect to the task revision, referenced objects, role-specific authority, operational controller, and evidence that justifies its reported outcome. Mixed-initiative interaction and co-planning already address the distribution of initiative and revision of shared plans (Horvitz, 1999; Feng et al., 2026); intent-oriented software and agent–UI protocols address complementary software and communication concerns (Xie et al., 2026; AG-UI Contributors, 2026). AIS and IIA introduce an engineering boundary and semantic abstraction for the continuing application. Their contribution is the joint treatment of obligations across that boundary, independently of a particular protocol, access-control scheme, or transaction mechanism. Our position is that dependable agent integration requires a maintained correspondence between revisable task intent and application behavior. This correspondence should guide development support and quality assessment from the outset. We formulate AIS as an open transition system and IIA as an abstraction of its task-relevant behavior. Interaction contracts constrain the correspondence; continuous assurance records which parts remain supported as tasks, policies, and implementations change. The formulation separates architectural membership from dependability: an application can instantiate AIS and still violate its interaction contract. This perspective develops the argument from the semantic framework in Section 2, through its engineering implications in Section 3, to a compact contract and assurance model in Section 4. The resulting research agenda connects application analysis, interface design, controlled execution, and quality assessment. Contrasting examples in collaboration, spreadsheets, integrated development environments (IDEs), and service consoles examine the scope of the argument. The contribution is a conceptual framework with conditional reasoning and worked examples; its practical benefits remain questions for comparative engineering and human studies. 2. AIS and IIA: A Semantic Framework An architectural diagram identifies components, but it does not determine whether a user’s intervention changes the behavior that those components can produce. We therefore describe AIS at two levels: application execution and task interaction. The distinction lets us ask which implementation details can remain hidden and which must remain meaningful at the interaction boundary. The notation uses established state-machine refinement and contract reasoning (Abadi and Lamport, 1991; Meyer, 1992); the proposed contribution is their application to the relationship between shared application state and revisable delegated tasks. 2.1. AIS as an Open Application System Definition 0 (AIS execution model). An AIS realization is represented by an open labeled transition system (1) ℳ=⟨X,X0,Σ,⟶⟩,Σ=ΣG⊎ΣI⊎ΣA⊎ΣE.M= X,X_0, , , = _G _I _A _E. Here X is the global state space, X0X_0 contains initial states, and x→x′x ax is a possible transition. The label classes distinguish direct GUI operations (G), intent-level commands and feedback (I), agent/runtime steps (A), and environmental events (E). Labels carry the relevant principal, task, and operation identifiers. The realization retains a conventional core and direct interaction path, integrates a goal-directed agent, and supports an IIA realization over their shared application objects. For analysis, write a state as x=(s,t,c,j,w)x=(s,t,c,j,w). The conventional core state s includes domain objects, versions, and applicable policies. The task store t records goals, revisions, contextual bindings, delegated scope, and unresolved decisions. The control state c identifies who may coordinate each task. The journal j associates admitted effects with their payloads and observed outcomes. Admission is the host’s acceptance of a specific effect for execution; it does not by itself establish that the effect has occurred. The remaining state w represents agent internals, pending messages, and relevant environmental state. This is a logical decomposition; it neither requires five services nor assumes that the agent observes the entire state. In particular, an external effect can have occurred while its outcome remains unknown in j. The open-system view matters because task execution does not suspend the application. A GUI edit, another user’s action, a policy revocation, or a delayed provider response can interleave with an agent step. These events are part of the behavior to be reasoned about, rather than exceptions to a sequential dialogue. Labels are tagged by their role in the model: a user-facing agent report belongs to ΣI _I, whereas internal planning belongs to ΣA _A. The tag does not establish trustworthiness. An agent-originated operation still requires authorization, and a direct operation still obeys the application’s business rules. Architectural membership requires the coexistence of these responsibilities, not satisfaction of a safety property. A defective integration remains an AIS instance. Similarly, neither local model execution nor a particular chat interface is a membership condition. These distinctions keep the model applicable to incremental adoption and distributed deployments without making quality claims true by definition. 2.2. IIA as a Task-Level Abstraction The IIA abstracts how delegated work develops over time. A task record can be written as tκ=(κ,r,g,b,d,U)t_κ=(κ,r,g,b,d,U): task identity κ, revision r, declared goal g, contextual bindings b, delegated scope d, and unresolved decisions U. A binding refers to a stable object and the relevant version or view, rather than only its current screen position. The goal supplies task-specific acceptance conditions, which may remain partial until decisions in U are resolved. This representation makes explicit what has been agreed; it does not assume that unrestricted natural-language meaning can be converted into a complete logical predicate. Definition 0 (Intent-Level Interaction Abstraction). An IIA specification is an abstract transition system ℐ=⟨Y,Y0,Λ,⟹,V⟩I= Y,Y_0, , ,V . Here Y contains task-level states, Y0Y_0 their initial states, Λ the event labels, and ⟹ the permitted abstract transitions. States summarize tasks, decisions, control status, and effect knowledge; events express goal submission, revision, endorsement, intervention, admission, and outcome reporting. Vu(y)V_u(y) is the view permitted for principal u. A candidate realization supplies a state abstraction π:X→Yπ:X→ Y and an event abstraction α:Σ→Λ∪εα: → ∪\ \, where ε denotes an unobservable step. The central requirement is that a concrete step have a valid task-level interpretation: (2) π⁡(X0)⊆Y0,x→x′⟹π⁡(x)⟹α⁡(a)π(x′).π(X_0) Y_0, x ax \ \ π(x) α(a) π(x ). For α⁡(a)=εα(a)= , the right-hand side means π⁡(x)=π⁡(x′)π(x)=π(x ). Internal reasoning can therefore remain hidden when it changes no task-relevant fact. A changed recipient, an admitted disclosure, or an acknowledged cancellation cannot be hidden in this way when it changes the contract’s task state. A GUI event affecting a delegated task must have an abstract interpretation even though it did not originate in the IIA interface. Where a useful abstraction needs history, the analysis state may be augmented with records of earlier endorsements and effects (Abadi and Lamport, 1991). The abstraction describes a specification relationship, not an assertion that a runtime can automatically recover π. Implementations must supply the object, task, and outcome information on which it depends. Furthermore, y represents shared task semantics while Vu(y)V_u(y) deliberately restricts what each role can inspect. An affected recipient need not see an organizer’s private context. User-facing feedback is faithful when its factual claims are justified by the corresponding authorized view; completeness of explanation and usability require additional judgment. Control requires more than observation. A request to stop and an acknowledgement that stopping took effect are distinct abstract events. For a task κ, the controller epoch k identifies a generation of control authority. An acknowledged stop invalidates that generation for subsequent admissions under (κ,k)(κ,k), while permitting reconciliation of earlier admissions. An interface that displays “stopped” while the old controller can continue admitting work violates the abstraction’s control semantics. Eventual acknowledgement is a separate progress requirement, conditional on service availability and communication assumptions. Fig. 1 brings the two levels together: the conventional core and agent constitute the execution system, while the IIA exposes its task-level meaning through the abstraction π. The lower band uses the state spaces X and Y defined above. A boundary labeled AIS contains a conventional GUI on the left, a layer realizing the Intent-Level Interaction Abstraction (IIA) above a built-in agent on the right, and a shared application core below. Users, potentially occupying different requester, owner, approver, and affected-party roles, exchange input and feedback through the two paths. Dashed arrows connect GUI context to the IIA layer and external knowledge to the agent. The agent cycles through retrieval, planning, invocation, and outcome reconciliation. The core contains permission checks, business services and versioned objects, and durable state with outcome records. A lower semantic band maps the AIS state space X to the IIA state space Y through an abstraction, with contracts constraining traces and evidence supporting claims. Figure 1. AIS with continuing direct and intent-level interaction over a shared core. The IIA realization presents and controls task semantics; the built-in agent plans and invokes capabilities. The lower band relates the execution state space X to the task state space Y. Control and effect records connect both levels to interaction contracts and assurance.A boundary labeled AIS contains a conventional GUI on the left, a layer realizing the Intent-Level Interaction Abstraction (IIA) above a built-in agent on the right, and a shared application core below. Users, potentially occupying different requester, owner, approver, and affected-party roles, exchange input and feedback through the two paths. Dashed arrows connect GUI context to the IIA layer and external knowledge to the agent. The agent cycles through retrieval, planning, invocation, and outcome reconciliation. The core contains permission checks, business services and versioned objects, and durable state with outcome records. A lower semantic band maps the AIS state space X to the IIA state space Y through an abstraction, with contracts constraining traces and evidence supporting claims. 2.3. Interaction Contracts and Behavioral Conformance The same abstract event can have different obligations across applications. A spreadsheet commit changes shared cells; a delivery commit may disclose information outside the application. We capture these choices in an interaction contract (3) =⟨,,,,⟩.K= Pre, Step, Inv, Post, Dep . Pre gives operation preconditions, including authority; Step constrains task and control transitions; Inv states consistency obligations; Post defines what effect or report counts as fulfilling an operation; and Dep identifies the assumptions and versions on which these clauses depend. An implementation may realize these clauses through APIs, runtime guards, ordinary application code, and interaction design. The contract is an application-level semantic specification, independent of that choice. For an effect e, its interaction–effect obligation is the relation (4) Ψe=⟨κ,r,βe,γe,ℓe,ϵe⟩, _e= κ,r, _e, _e, _e, _e , linking the task and revision to reviewed payload/object bindings βe _e, role-specific authority γe _e, controller identity and epoch ℓe _e, and available outcome evidence ϵe _e. The relation is temporal. Authority is checked at admission; later evidence determines which outcome can be reported. Revising a task can invalidate future admissions without erasing the attribution of an earlier committed effect. Consequently, consistency cannot be reduced to equality between the latest task state and every historical record. Let Tr denote finite execution traces and let Π be the trace projection induced by π and α, with unobservable repetitions removed. For stated environmental assumptions H, the proposed conformance obligation is (5) Π⁡(⁡(ℳ∣H))⊆⁡(ℐ∣). ( Tr(M H) ) Tr(I ). The right-hand side contains abstract traces permitted by the contract. This finite-trace condition addresses valid transitions and claims about observed effects. It does not establish eventual task completion: a runtime that remains silent can satisfy some safety clauses while failing a progress requirement. Such requirements need explicit fairness, availability, and deadline assumptions. Nor does conformance establish that the declared goal adequately represents the user’s needs. Proposition 2.3 (Local validity does not establish interaction conformance). Valid component calls and authenticated permissions are insufficient to establish Equation (5) under a contract requiring current endorsement bindings when task-relevant state can change before effect admission. Proof. Take a proposal for recipients a,b\a,b\ and an authentic endorsement of that proposal. Before admission, a GUI revision removes b. A send operation for b can still satisfy its API schema and the caller’s ordinary resource permission. Its projected trace nevertheless contains an admission inconsistent with the current task binding. Thus component validity permits a trace excluded by the interaction contract. ∎ The proposition locates the contribution of the framework. The missing relation crosses the boundaries of interface state, task interpretation, authority, and effects. Access control and transaction mechanisms remain essential, but their successful composition must be established with respect to that relation. A different mechanism that already preserves it is an alternative realization of the obligation. 2.4. Foundations and Derivation of the Engineering Agenda The development summarized in Fig. 2 explains why this relation is now exposed more broadly. Direct manipulation and mixed initiative preserve user participation (Shneiderman, 1983; Horvitz, 1999); demonstration and task shortcuts connect requests to existing application operations (Li et al., 2017; Arsan et al., 2021); model-based planning and tool use support less predetermined procedures (Yao et al., 2023; Schick et al., 2023; Li et al., 2023; Wen et al., 2024; Zhang et al., 2025). Studies of product copilots and software built with foundation models (FMware) identify the resulting integration and lifecycle demands (Parnin et al., 2025; Rajbahadur et al., 2026). The framework concentrates these developments into one question: which task-level relationships must survive changes in the underlying execution? Five groups of publications run from direct manipulation in 1983 and mixed-initiative interaction in 1999 through demonstration in 2017, capability exposure in 2021, model-driven execution in 2023 and 2024, and lifecycle engineering in 2025 and 2026. Each group has a mechanism icon and representative publications. User control, object meaning, authority, and evolution remain common concerns. A final band defines AIS through a conventional core, built-in agent, and direct and IIA-mediated interaction. Figure 2. Selected foundations of AIS: interaction (Shneiderman, 1983; Horvitz, 1999), demonstration (Li et al., 2017), capability exposure (Arsan et al., 2021), model-driven execution (Yao et al., 2023; Li et al., 2023; Wen et al., 2024), and lifecycle engineering (Parnin et al., 2025; Rajbahadur et al., 2026). Publication groups indicate overlapping foundations, not replacement or a time scale.Five groups of publications run from direct manipulation in 1983 and mixed-initiative interaction in 1999 through demonstration in 2017, capability exposure in 2021, model-driven execution in 2023 and 2024, and lifecycle engineering in 2025 and 2026. Each group has a mechanism icon and representative publications. User control, object meaning, authority, and evolution remain common concerns. A final band defines AIS through a conventional core, built-in agent, and direct and IIA-mediated interaction. The challenge areas follow from perturbing the terms in this framework. Changes to g, b, and d expose task interpretation, interface, and context obligations; changes to c, authority in s, and effects in j expose continuity, permission, and recovery obligations. Uncertainty about their observation or evolution exposes traceability and maintenance obligations. The next section develops these implications, followed by design principles and a research agenda. Their connections are synthesized at the end of the agenda. This organization is a retrospective design rationale, not an empirical taxonomy or an exhaustive derivation. Overlap is expected because a single object change can affect several relations. 3. Engineering Implications of the Framework The semantic framework shifts the engineering question from whether an agent can invoke an operation to whether its evolving execution remains a valid realization of the task. This section develops eight engineering implications. Each concerns information or control that crosses a component boundary; their separation identifies where development support and assurance are needed. 3.1. C1: Task Abstraction and Degrees of Autonomy A task becomes actionable through the relationship between its declared goal g, delegated scope d, and unresolved decisions U. “Prepare the materials” can authorize retrieval and proposal construction while leaving disclosure undecided. Substituting a public version for a private attachment may satisfy a release policy but alter the user’s intended result. The IIA must preserve that distinction throughout the task, rather than allowing an inferred plan to stand in for an endorsement. Autonomy is consequently a property of permitted transitions for a task and role, not a fixed level assigned to an agent. Participation in planning can improve the opportunity for correction (He et al., 2025), but a review is consequential only if subsequent admissions remain bound to it. Requiring approval for every internal step can increase effort without narrowing the relevant effects. The design problem is to expose decisions that change the task’s acceptance conditions or authority while allowing factual retrieval and reversible preparation within the established scope. This connects formal control semantics to the supervision question developed in Section 6.2. 3.2. C2: Bidirectional Interfaces The two abstraction levels require complementary interfaces. The application supplies capabilities through which the agent observes and changes domain state; the agent runtime supplies task-control services through which the application realizes the IIA. Their contracts meet at task identity, contextual bindings, and effect evidence. Table 1 summarizes the division. An API can be callable without exposing enough information to relate its effects to a reviewed task, and an event stream can be well formed without establishing whether an operation committed. Table 1. Complementary interfaces and the semantics needed to relate task interaction to application behavior. Interface provider Services available to the other side Semantics to make explicit Application core to agent Discover capabilities; read context and objects; subscribe to changes; validate authority; execute and verify operations. Object identity and version, scope, prerequisites, side effects, errors, and retry behavior. Agent runtime via the IIA layer Accept goals and context updates; expose progress and proposals; request input; return outcomes; suspend, cancel, or resume tasks. Task identity and revision, pending user decisions, confirmed effects, and cancellation limits. API selection and structured invocation address the first part of this problem (Qin et al., 2024; Xie et al., 2025). Semantic compatibility additionally requires stable meanings for preconditions, effects, cancellation, and outcomes. A schema-preserving change to a tool can therefore break the conformance relation. Conversely, an implementation of the Agent–User Interaction (AG-UI) protocol can transport state, edited approvals, interrupts, and resume events (AG-UI Contributors, 2026), while the host supplies the admission checks and authoritative outcomes. The framework makes this division explicit so that protocol conformance is not mistaken for application-level conformance. 3.3. C3: Context, Knowledge, and Memory Context connects a task to an application state that may outlive the utterance that referred to it. The binding b must distinguish object identity from position, a current value from an observed version, and an explicit instruction from an inferred preference. A screenshot can locate a document yet omit the identity needed to detect that the document was replaced. Cross-platform replay work by Yu et al. (Yu et al., 2021) illustrates the difficulty of preserving targets as interfaces change; preserving the intended business effect requires the additional task relation. Personalization and retrieval extend these dependencies beyond the current screen (Cai et al., 2025; Shi et al., 2026). A remembered preference can conflict with the current request, and an updated source can invalidate a derived summary. The challenge is to retain enough provenance for selective refresh and correction without turning memory into unrestricted retention. Read authority also differs from disclosure authority. The IIA view VuV_u and any transferred task summary must respect purpose and recipient scope even when the agent was permitted to read the underlying source. 3.4. C4: Interaction Continuity and Shared State Continuity requires a stable task meaning across changes in interface, controller, and execution location. Fig. 3 illustrates how a GUI edit changes the admissible continuation of a delegated task. The important event is the change in the reviewed binding, not whether the correction arrived through chat or direct manipulation. Treating the two paths as separate sessions would lose precisely the relationship that the IIA abstracts. The user selects materials and requests a sending plan. The agent retrieves evidence and presents a proposal. User corrections and GUI edits feed task revision and updated application state. The user then approves or cancels. Approved calls go to core scope and state checks before execution; changed conditions return to task revision. Non-approval and acknowledged cancellation lead to fencing new admissions and tracking admitted work. The core records or queries actual effects, the agent reconciles confirmed, partial, or unknown outcomes, and the IIA presents results and handover options. Cancellation does not automatically undo committed effects. Figure 3. Task revision and guarded execution across IIA/GUI interaction, agent coordination, and core operations. GUI edits revise the pending binding; changed conditions require refresh and, where necessary, renewed endorsement. An acknowledged stop fences new admissions while earlier admissions remain subject to outcome reconciliation. Each responsibility lane can involve multiple principals.The user selects materials and requests a sending plan. The agent retrieves evidence and presents a proposal. User corrections and GUI edits feed task revision and updated application state. The user then approves or cancels. Approved calls go to core scope and state checks before execution; changed conditions return to task revision. Non-approval and acknowledged cancellation lead to fencing new admissions and tracking admitted work. The core records or queries actual effects, the agent reconciles confirmed, partial, or unknown outcomes, and the IIA presents results and handover options. Cancellation does not automatically undo committed effects. The term handover covers two distinct changes: operational takeover by the same user (H1) and transfer of responsibility to another person (H2). Remote continuation or relocation (H3) and recovery after failure (H4) may accompany either, but do not themselves imply a change of responsible person. Table 2 compares these four cases. A controller lease records which runtime holds task-control authority and the conditions under which that authority expires. The four cases share a continuity record containing task revision, unresolved decisions, relevant object versions, grants, controller lease/epoch, and admitted effects with known outcomes. The record supports continuity only when the host can establish which controller may next admit work. Table 2. Four continuity cases sharing a task record but requiring different authority and recovery semantics. Case What changes Required special handling H1: user takes over Operational control moves from agent to the same user’s direct interaction. Fence agent admissions; show admitted/unknown effects and a continuation point. User edits invalidate affected plans, not the whole application. H2: task transfers to another person The accountable controller or task requester changes. Revalidate the recipient’s own rights and required endorsements; minimize transferred context; preserve attribution. Credentials and approvals are not transferred implicitly. H3: remote continuation or relocation The same task continues after UI disconnection, or moves between runtimes. Persist task/effect identities. Reconnection reads authoritative state. Relocation establishes a new executor lease; client loss alone does not imply cancellation. H4: recovery after failure A runtime restarts with potentially incomplete observations. Fence stale workers, rebuild from the journal, query unknown effects before resubmission, and disclose irrecoverable evidence gaps. A transfer therefore combines a semantic obligation with an enforcement obligation. The successor needs a rights-filtered account of what remains to be done, while the old controller must lose the ability to admit further effects under its former authority. Lost acknowledgements leave ownership unresolved; late provider responses leave effect knowledge incomplete. Neither uncertainty can be removed by copying a conversation. A closed client window is similarly compatible with an active remote task. These cases require different continuations even though they share a task identifier. 3.5. C5: Permission, Safety, Security, and Privacy Delegation crosses a relation among principals, rather than a single user–agent permission boundary. The initiator requests an outcome; resource owners govern the objects; approvers endorse particular effects; affected parties receive their consequences; and an executor/controller coordinates the work. A person may occupy several roles, but their authorities do not merge automatically. In the document scenario, an organizer’s request does not supersede a team’s release policy or a disclosure officer’s approval. For an effect e in state x, the authority condition is conjunctive: (6) permit⁡(e,x)=taskGrant⁡(e,x)∧resourcePolicy⁡(e,x)∧requiredEndorsements⁡(e,x)∧executorScope⁡(e,x). splitpermit(e,x)=&taskGrant(e,x) (e,x)\\ & (e,x) (e,x). split Attribute-based access control supplies established mechanisms for subject, resource, action, and environment conditions (Hu et al., 2014). The framework additionally relates these conditions to the current task binding. Owner consent may be represented by a standing grant; affected-party consent and separation of duties become required predicates where the domain policy demands them. Changes to ownership, recipients, account, or controller require revalidation of the dependencies concerned. An endorsement must also have an origin distinct from agent inference. Execution delegation cannot include the ability to write the approval ledger or activate an indistinguishable approval input. Otherwise, an authenticated session establishes an account identity but not a person’s endorsement of the particular effect. The distinction between data and authority is equally important for retrieved material. Prompt-injection defenses and runtime constraints provide relevant enforcement techniques (Debenedetti et al., 2024; Debenedetti et al., 2025; Wang et al., 2026); their scope depends on complete mediation across reachable execution paths (Saltzer and Schroeder, 1975). An unrestricted alternative shell or database connection defeats a claim covering only guarded business APIs. Safety, security, and privacy impose related but different obligations. A permitted action can still be mistaken; an adversarial action can redirect control; an otherwise legitimate result can disclose excessive information. Role-specific views, logs, and derived memory therefore need their own disclosure and retention conditions. Preserving attribution during a handover does not imply transferring the previous controller’s credentials or private context. 3.6. C6: Execution Semantics and Failure Recovery The distinction between application state s and effect knowledge in j is essential under failure. A timeout can leave the runtime unable to distinguish non-execution from an already committed effect. Treating both as failure permits duplicate execution; treating both as success produces misleading feedback. Stable effect identifiers and authoritative status queries make that uncertainty manageable, while the contract determines whether replay is safe. Cancellation changes future admissibility at a particular boundary. It does not reverse an effect already admitted to an external provider. Compensation introduces a new domain operation with its own authority and failure conditions, as in sagas (Garcia-Molina and Salem, 1987). A calendar invitation can be withdrawn, but its notification may already have been read. A spreadsheet undo can restore prior values yet incorrectly overwrite a collaborator’s later edit. Recovery must therefore preserve the relation among original effects, subsequent changes, and new compensating actions; a generated inverse command is insufficient. 3.7. C7: Observability, Debugging, and Responsibility The abstraction provides a natural basis for diagnosis: a failure occurs where a concrete trace no longer has the task-level interpretation promised by the contract. Agent debugging and lifecycle observability offer useful inspection mechanisms (Epperson et al., 2025; Dong et al., 2024). For AIS, traces must additionally connect task revisions and endorsements to host admissions and outcome records. Otherwise, an explanation can describe why the agent acted without establishing whether that action was authorized or what it changed. Different readers need different projections of this evidence. Users need the effects that occurred and the decisions that remain. Developers need the dependency or transition responsible for a mismatch. The evidence includes approvals, admission witnesses, receipts, etc. A model-generated explanation can help interpret those records but cannot replace them. Replay also needs controlled versions and external state before it can support causal diagnosis. This separation makes evidence useful for accountability while leaving institutional responsibility to the relevant organizational arrangements. 3.8. C8: Architecture, Joint Evolution, and Operating Costs The correspondence between ℳM and ℐI can change even when visible API types do not. A model replacement changes possible plans; a host update changes object meaning; an adapter revision changes admission or reporting behavior. Experience with learned-system engineering and technical debt motivates explicit management of such dependencies (Amershi et al., 2019a; Sculley et al., 2015; Rajbahadur et al., 2026). The framework makes their consequence precise: some previously supported traces or assurance claims may no longer describe the revised implementation. This raises an architectural choice for existing applications. Stable references, guarded writes, task-control hooks, and outcome records have development and maintenance costs. Integrating an agent without those services may still offer useful preparation or retrieval, but supports weaker effect guarantees. Deployment further changes latency, information exposure, and continuity assumptions. A viable integration must therefore be judged against total development, computation, supervision, and recovery effort, including the conventional workflow it complements. 4. Interaction Contracts and Continuous Assurance The framework makes a distinction that conventional task-success measures can obscure. An execution may reach the requested final state through an unauthorized intermediate effect, or preserve every authorization condition while failing to achieve a useful outcome. Dependability therefore requires both a valid interaction history and an adequate task result. Interaction contracts state selected obligations over that history; assurance identifies the evidence for believing they hold in a particular realization. 4.1. Quality Across the Abstraction Boundary At the IIA level, quality concerns the fidelity and usability of task expression, proposals, interventions, and feedback. At the AIS level, it additionally concerns planning, domain effects, authority, availability, and evolution. Existing software and AI-system quality models supply broader terminology (ISO/IEC, 2023a; ISO/IEC, 2023b); Table 3 specializes the concerns to the abstraction boundary. A correct update with misleading feedback and a clear preview followed by an incorrect update are distinct failures of the same relationship. Table 3. Quality concerns across the IIA abstraction and its AIS realization. Primary loci identify the information or mechanisms needed to assess each concern. Quality concern Primary locus Example of a failure Intent and user control IIA layer and agent Ignore a recipient corrected through the GUI. Grounded planning Agent and context services Use an obsolete conversation as the basis for a plan. Business-effect correctness Core and agent runtime Repeat a write after an ambiguous timeout. Feedback fidelity IIA layer and core evidence Report delivery before the core confirms it. Consistency and reliability AIS across both paths Resume from task state that disagrees with application state. Safety, security, and privacy AIS across trust boundaries Treat retrieved text as authority to disclose data. Availability and efficiency AIS deployment Lose the direct-operation fallback when the model is unavailable. Maintainability and observability AIS lifecycle Silently lose a capability after an interface update. Task acceptance must also depend on the authorized stage. A preparation request can succeed with an inspectable proposal; an execution request can appropriately lead to clarification or justified non-execution. Such outcomes cannot be evaluated by counting completed writes. Conversely, conformance to permission and control clauses does not establish that a summary is accurate or a refactoring useful. The framework separates these judgments so that strong evidence for one property cannot silently stand in for another. The placement of responsibility follows the location of the required knowledge. Object identity and commit evidence belong at the host boundary; task revision and endorsement must remain connected to them; user-facing explanations must draw on the resulting records. This follows the spirit of end-to-end reasoning about application-specific correctness (Saltzer et al., 1984). It also explains why a single aggregate quality score is inadequate: improved response fluency cannot compensate for an unauthorized disclosure, and fewer interruptions do not necessarily improve user control. 4.2. A Compact Contract for Reviewed Disclosure Contract 1 instantiates K for the meeting-materials scenario. Its compact form separates the scenario’s binding and authority from reusable admission, control, and reporting clauses. The values are symbolic: task T42T42 at revision 7, controller epoch 3, a public document view at version 12, and recipients a and b. H7H7 identifies the reviewed view and D7D7 its canonical proposal digest. The contract gives semantic obligations rather than prescribing a configuration language or a runtime architecture. The contract’s invariants are expressed by four clauses. K1, endorsement binding, relates an admitted payload to the current task revision, reviewed objects/views, recipients, and effect scope. K2, current authority, requires the conjunction in Equation (6) at admission. K3, controlled admission, requires running task state, an authenticated controller lease, and the active epoch, with the acceptance decision recorded atomically. K4, outcome fidelity, permits a committed report only when authoritative evidence establishes the specified effect. These clauses specialize Pre, Inv, and Post; the revision and control rules specialize Step. Contract 1: Reviewed document disclosure, share-materials/v1. Binding κ=T42κ=T42, r=7r=7, k=3k=3; document 7, version 12, public view H7H7; recipients a,b\a,b\; frozen proposal D7D7. Only reviewed materials and recipients may be delivered. Authority Organizer initiates; team owns the document; event chair and disclosure officer approve; recipients are affected; agent service acts for the organizer under lease L3L3. Owner grant G18G18 and policy policy-v4 apply. Endorsement A host ledger records endorsements through a non-delegated channel, bound to (κ,r,D7,G18,policy-v4)(κ,r,D7,G18, policy-v4) and required roles. Recheck scope, revocation, and expiry; task endorsement expires no later than ten minutes after recording. Preparation Read authorized context and public views; preview without delivery. Material revision increments r and invalidates prior task endorsements. Complete, valid approval enables running; unspecified operations or transitions are rejected. Admission Atomically check K1–K3 and journal a stable effect ID for each recipient. Submit only its frozen payload, using the same provider idempotency key when safe to retry. Status queries obey read permissions. Control Cancel atomically stops the task, revokes task endorsement, and advances k. H1/H2 fence the old lease and require revalidation and acceptance; H3/H4 revalidate delegation and reconcile before resuming affected writes. Preserve unresolved effect IDs. Outcome K4 governs every report. Distinguish not admitted, admitted, submitted, committed, failed, and unknown. Commitment means provider-recorded delivery to the named inbox; submission acceptance alone is insufficient. Dependencies Contract, host, adapter, policy, role grants, and controller state. An unknown submission is queried before replay; cancellation fences new admissions while earlier admitted effects may still commit. Standing owner grants remain usable after revalidation; revising a task does not itself revoke them. Let BeB_e be the reviewed binding for effect e, AeA_e its required endorsements, and ℓe _e its authenticated controller lease with epoch kek_e. For state x immediately before admission, (7) admit⁡(e,x)⟹current⁡(Be,x)∧endorsed⁡(Ae,Be,x)∧permit⁡(e,x)∧running⁡(κ,x)∧validLease⁡(ℓe,x)∧ke=kactive(κ,x). splitadmit(e,x) &current(B_e,x) (A_e,B_e,x)\\ & (e,x) (κ,x)\\ & ( _e,x) k_e=k_active(κ,x). split The gate evaluates this predicate and records the admitted payload within one host-controlled atomic boundary. Merely supplying the latest epoch number does not authenticate a controller. Likewise, retaining a conversational thread does not establish a current endorsement. A stable effect ID cannot be rebound to a different payload, and an unresolved submission cannot become a supposedly new action by receiving a fresh ID after a task revision. Proposition 4.1 (Planner-independent admission safety). Suppose all admissions in the contract’s scope pass through a trusted gate that atomically enforces Equation (7), the bindings and authority records are authoritative, and only this gate can append immutable admission records to an initially valid journal. Every admitted effect then satisfies K1–K3 at its admission point, independently of how the planner selected it. Proof. The initial journal satisfies the property by assumption. A transition that appends no admission preserves it. A transition that appends one first establishes the admission predicate and binds the accepted payload to that record atomically. The property therefore holds for every finite sequence of admissions. Subsequent policy changes affect later decisions without changing what held at an earlier admission point. ∎ This limited result explains a useful architectural separation. The model can propose actions nondeterministically while selected admission invariants remain properties of the host boundary. The result establishes neither K4 nor goal suitability, progress, or absence of unmediated effects. It also exposes the assumptions that an integration must justify before claiming such a separation. A gate that reads stale policy state or an adapter that bypasses the gate does not satisfy the premises. Task control and effect state remain independent. In this contract, commitment means delivery recorded by the provider, not that a recipient has read or understood the material. A stopped task can contain a committed delivery. If the provider exposes only submission acceptance, the promised postcondition must be narrowed or delivery knowledge remains unknown. Provider-level duplicate prevention similarly requires an idempotency guarantee or an equivalent queryable operation identity; the host journal alone cannot create exactly-once remote execution. 4.3. Revision, Interruption, and the Meaning of a Valid Trace The earlier counterexample becomes precise under the contract. Revision 7 proposes a,b\a,b\; a GUI edit removes b and produces revision 8. A delayed endorsement for revision 7 fails K1. Once the revised proposal receives the required approval, the gate may admit delivery EaE_a. A lost provider response leaves its outcome unknown under K4. Cancellation then advances the epoch and excludes new admissions, but a late receipt can still establish that EaE_a committed. A faithful final account includes both the stopped remainder and the completed delivery. Four design principles follow from the trace. P1, preserve referential integrity, keeps revised goals attached to the right objects and effects. P2, separate interpretation from authorization, makes endorsement depend on accountable roles. P3, make control operational, gives interventions consequences for admissible behavior. P4, ground feedback in outcomes, keeps the user’s understanding aligned with evidence. In the compact contract these principles become K1–K4, rather than an additional independent checklist. The same trace can guide development without pretending to be an experiment. It identifies the records an adapter must expose and the distinctions a test oracle must observe. Injecting a stale approval, timeout, or delayed receipt would exercise a different clause. More permissive contracts could reuse unaffected step-level endorsements after revision; doing so would require evidence that the dependency analysis preserved their scope. The conservative invalidation in Contract 1 makes that trade-off explicit. 4.4. Assurance as a Versioned Argument Continuous assurance concerns the standing of a claim about the realization, rather than the mere presence of a monitor. For each claim, define a record (8) =⟨q,Ω,D,E,R,S⟩,S∈,,.A= q, ,D,E,R,S , S∈\ supported, refuted, insufficient\. Here Ω identifies the task, interval, or release scope; D records assumptions and versioned dependencies; E contains evidence with producer and observation time; and R is the rule used to assess it. Support is conditional on those premises. A witnessed violation refutes a claim, whereas an unavailable receipt, expired premise, or coverage gap leaves insufficient support. The distinction prevents missing evidence from being interpreted as a successful check. The assurance argument separates five claims: endorsement and authority at admission (Q1), effective stopping of new admissions (Q2), evidence-grounded outcome reporting (Q3), continuity without silent replay (Q4), and release behavior on a declared assessment scope (Q5). Table 4 states their scope, evidence, and checking rules. Table 4. Five scoped assurance claims with evidence, checking rules, and conditions requiring reassessment. Claim and scope Evidence producer and artifact Check, invalidation, and response Q1: every admitted effect in the observed interval matches endorsement and authority at admission. Host admission journal; authenticated approval/role ledger; coverage of write gates. Check K1/K2 and the admission witness. A mismatching record refutes Q1. Lost coverage removes support. Changed policy/ownership or stale grants require a new scope and revalidation for subsequent admissions. Q2: after an acknowledged stop, the old controller admits no new effects. Task store: stop acknowledgement, lease/epoch transition; admission journal. Check runnable state, controller lease, and K3 ordering at the gate. A later stale admission refutes Q2. Missing acknowledgement/order evidence is insufficient; fence and reconcile. Q3: every reported committed effect has authoritative matching outcome evidence. Core commit record or provider receipt; effect journal; user-facing report. Match effect ID and frozen payload under K4. A premature completion report refutes Q3. A timeout leaves the effect outcome unknown; query before reporting or resubmitting. Q4: a control transition preserves unresolved work without silent replay. Transfer/recovery record; old/new controller acknowledgements; stable effect IDs. Check H1–H4 obligations. Missing transfer acknowledgement or an unreconciled submission prevents resumption of affected writes; changed recipient rights invalidate transferred context. Q5: a release meets specified task/interaction properties on its declared regression scope. Version-bound scenario tests, negative traces, and human-reviewed task outcomes. Check stated acceptance rules and record coverage. A relevant model, adapter, host, or specification change invalidates affected evidence; rerun or narrow the claim. Passing samples do not establish universal correctness. The timeout trace can support Q1 while leaving delivery unknown. Reporting that uncertainty preserves Q3, whereas asserting delivery without evidence violates it. A late receipt does not refute Q2 when it concerns an admission preceding the stop. The claims therefore express separable obligations that a generic task-success indicator would collapse. Changes determine which arguments must be reconsidered. Let Dep⁡(q,Ω)Dep(q, ) denote the dependencies needed to support claim q over scope Ω , and let Δ be the set of changed dependencies. A candidate invalidation rule is (9) Dep⁡(q,Ω)∩Δ≠∅⟹reassess⁡(q,Ω′),Dep(q, )∩ ≠ (q, ), where Ω′ is the proposed scope after the change. An argument remains applicable only if its relevant premises remain valid or are re-established. Historical evidence retains its original attribution; it does not automatically support the new scope. Dependency analysis is itself an assurance obligation: an omitted dependency makes selective invalidation unsound. The conditional separation in Proposition 4.1 suggests that not every change needs the same response. Replacing a planner may invalidate behavioral evidence for Q5 while preserving an independently established invariant of an unchanged gate. Adding an unmediated write capability invalidates that gate’s coverage premise and affects Q1/Q2. Changing an outcome adapter can undermine Q3 even if planning remains unchanged. This is the practical meaning of continuous assurance: a maintained argument across relevant changes, rather than universal retesting after every token. Design analysis, release evaluation, and operational records offer different evidence. Production-readiness and lifecycle-observability work provides useful foundations (Breck et al., 2017; Dong et al., 2024); the framework binds their artifacts to the same interaction–effect relation. Its research challenge is to make that binding economical and trustworthy enough to guide maintenance. Evidence provenance, retention cost, monitor completeness, and acceptable uncertainty will influence whether the approach remains usable beyond a small example. 4.5. What Quality Assessment Must Observe Assessment must observe task-relevant transitions as well as final outcomes. Executable web, desktop, and mobile environments provide foundations for outcome-based evaluation (Zhou et al., 2024; Xie et al., 2024; Rawles et al., 2025). Stateful tools, business tasks, and side-effect checks broaden that view (Lu et al., 2025; Yao et al., 2025; Huang et al., 2025; Trivedi et al., 2024). For AIS, the distinguishing need is to exercise the correspondence across direct and delegated interaction: revisions, role changes, interrupted admissions, and unresolved effects. Scenario-guided GUI testing (Yu et al., 2026c) and dual-control environments (Barres et al., 2025) suggest ways to construct such cases. Equivalent intentions need not produce identical click sequences, and a justified refusal need not be a failed task. The relevant oracle compares permissible business behavior under equivalent initial conditions. Real users remain necessary for evaluating whether the IIA makes consequential differences inspectable; user simulation alone can misrepresent human interaction (Seshadri et al., 2026). The next generation of assessment should connect behavioral validity with the effort and decision quality of the people supervising it. 5. A Research Agenda from Development to Assurance The framework suggests a research program organized around constructing and maintaining the correspondence in Equation (5). Application analysis supplies the concrete semantics; interaction design supplies the abstract task model; contracts relate them; evidence supports their continued use. The six directions below develop this program from application analysis to the maintenance of evidence. Their shared artifacts create an opportunity for development assistance and quality assessment to reinforce one another. 5.1. R1: Recovering Application Abstractions and Interface Contracts R1 concerns the construction of π and the two interface contracts from an existing application. The central difficulty is recovering semantic obligations rather than merely enumerating callable operations. A tool should distinguish an object’s stable identity, the conditions that authorize its modification, and the evidence that establishes the resulting effect. GUI analysis, testing, and intent inference offer candidate sources for this information (Yu et al., 2026b; Yu et al., 2024; Yu et al., 2026a). Their outputs would need explicit links to host implementations and reviewable uncertainty. This direction also concerns abstraction adequacy. A representation that omits a recipient or an intermediate disclosure may admit a false correspondence even when every represented transition is valid. Development support should therefore help identify distinctions that the domain considers consequential, expose unsupported clauses, and maintain the resulting schema as the application evolves. The relevant benefit is a reduction in total specification and maintenance effort while retaining those distinctions. 5.2. R2: Maintaining Task Meaning under Context Change R2 concerns how g, b, d, and U evolve together. Rebuilding the entire task after every GUI edit is disruptive, while preserving every prior endorsement is unsafe. A useful task representation would identify which decisions depend on a changed source, object version, preference, or recipient. It could then distinguish a harmless presentation change from one that requires a new endorsement or invalidates an earlier plan. The difficult cases involve dependencies absent from a formal schema. A renamed document can retain its meaning, whereas an unchanged filename can conceal different disclosure implications. Research is needed on combining explicit domain relations, inferred dependencies, and user correction without treating model confidence as proof of equivalence. Such representations must also support rights-filtered continuity in H1–H4, so that preserving task meaning does not imply preserving all private context. 5.3. R3: Enforcing Delegation across Heterogeneous Effect Boundaries R3 concerns the realization of Pre, Step, and Inv where operations cross APIs, GUI automation, and remote services. Runtime enforcement offers mechanisms for selected rules (Wang et al., 2026); the larger challenge is ensuring that they refer to the current task and the actual admission boundary. A capability description should reveal whether the provider supports atomic checks, duplicate prevention, status queries, and compensation. This suggests conformance profiles with explicitly different guarantees. A local versioned edit can support stronger interference checks than an external delivery service with opaque commitment. Comparing these profiles through adversarial traces would clarify which guarantees survive retries, revocation, and controller transfer. The research objective is a principled basis for accepting, restricting, or declining delegated effects under the provider’s actual semantics. 5.4. R4: Assisting Development through Shared Semantic Artifacts R4 concerns development tools that use the same contract artifacts as assurance. An admission predicate can guide adapter generation; a prohibited transition can generate a negative test; a failed assurance claim can locate a missing record or an invalid assumption. Existing work on test generation and migration, declarative language-model programs, and prompt comparison provides complementary techniques (Yu et al., 2023; Khattab et al., 2024; Arawjo et al., 2024). The distinctive opportunity is to connect these activities through maintained task and effect semantics. Generated specifications cannot serve as independent oracles merely because they are expressed formally. A tool that infers a contract from faulty code may reproduce the fault. Research should distinguish recovered implementation behavior, intended requirements, and reviewed contract clauses, while measuring the effort of resolving disagreements. Developer experience should include subsequent host changes and evidence maintenance, not only the speed of producing an initial integration. 5.5. R5: Assessing Conformance and Maintaining Evidence R5 concerns which evidence justifies a scoped conformance claim. Tests can explore traces, static or runtime analysis can establish selected invariants, and human assessment can judge task adequacy and inspectability. Their scopes differ. Metamorphic relations (Chen et al., 2018), for example, could test whether meaning-preserving request variants retain permitted business effects, but identifying such variants requires domain judgment beyond the tested model. Evidence dependencies introduce a second research problem. Selective reassessment is useful only if the dependency model catches relevant changes without overwhelming developers with unnecessary work. Longitudinal evaluations should therefore consider missed invalidations, unnecessary reassessments, diagnostic value, and maintenance cost. A system that produces frequent passing checks while silently losing the premises of its claims would fail the purpose of continuous assurance. 5.6. R6: Preserving Human Authority and Sustainable Integration R6 concerns the allocation of authority and burden across the people affected by delegated work. The requester, owner, approver, and operator may have different interests; a common protocol does not resolve them. Portable role and consent descriptions could help preserve obligations across organizational boundaries, provided that changes of context trigger explicit revalidation rather than implicit inheritance. The same perspective applies to deployment and model substitution. Interchangeable interfaces can conceal different data exposure, latency, operating cost, and recovery behavior. Sustainable integration requires assessing those changes together with the effort of supervision and maintenance. The long-term objective is software in which delegation remains useful and accountable as both the host and its intelligent components evolve. Taken together, the directions connect the implications developed in Section 3 with the principles derived in Section 4.3. Table 5 records these connections alongside their literature premises and motivating perturbations. Fig. 4 projects the same challenge–direction links into a visual overview. The links are a design synthesis: they identify shared research work without assigning empirical weights or excluding additional relationships. Table 5. Synthesis of the engineering implications (C1–C8, Section 3), design principles (P1–P4, Section 4.3), and research directions (R1–R6, Section 5). The literature premises and perturbations explain the proposed connections. Area Literature premise Perturbation and derived obligation Principles; directions C1 Task abstraction Goal uncertainty and user participation (Horvitz, 1999; He et al., 2025). Change “prepare” to “send”: distinguish inferred intent from delegated effects. P1, P2; R1, R2, R3, R6 C2 Interfaces Tool discovery/calling and product integration (Li et al., 2023; Parnin et al., 2025). Change a tool’s effect without its schema: specify capability and task-control contracts. P1–P4; R1, R4, R5 C3 Context and memory Personalized and knowledge-dependent actions (Cai et al., 2025; Shi et al., 2026). Replace a document or preference: retain identity, provenance, and invalidation dependencies. P1, P2; R2, R3, R5 C4 Continuity Shared plans and stateful interaction (Feng et al., 2026; Lu et al., 2025). Edit in the GUI or change controller: reconcile task state and ownership. P1, P3; R2, R3, R5 C5 Authority and safety Untrusted tool content and runtime policies (Debenedetti et al., 2024; Wang et al., 2026). Introduce an external recipient or revoke an owner grant: revalidate authority at admission. P2; R3, R5, R6 C6 Execution and recovery Outcome-based evaluation and compensation (Trivedi et al., 2024; Garcia-Molina and Salem, 1987). Lose a delivery response: distinguish unknown, failed, committed, and compensable effects. P3, P4; R1, R3, R5 C7 Observability Agent debugging and lifecycle traces (Epperson et al., 2025; Dong et al., 2024). Present a success message without a receipt: preserve evidence lineage for diagnosis and feedback. P4; R4, R5, R6 C8 Architecture and evolution Learned-system dependencies and production maintenance (Sculley et al., 2015; Rajbahadur et al., 2026). Replace a model, adapter, or host version: invalidate affected behavioral evidence. P1, P4; R1, R4, R5, R6 An eight-row, six-column matrix uses labeled icons and 26 dots to connect challenges with interfaces, context and state, controlled delegation, developer support, quality assurance, and user interests and ecosystem. The dots have no weights or empirical interpretation. Four tiles below identify shared task context, capability contracts, outcome records, and version dependencies. Figure 4. Connections between engineering implications C1–C8 and research directions R1–R6, grounded in Table 5. The 26 unweighted links identify shared research concerns; labels abbreviate the subsection topics. Task context, contracts, outcome records, and version dependencies connect development assistance with assurance; blank cells do not exclude further relationships.An eight-row, six-column matrix uses labeled icons and 26 dots to connect challenges with interfaces, context and state, controlled delegation, developer support, quality assurance, and user interests and ecosystem. The dots have no weights or empirical interpretation. Four tiles below identify shared task context, capability contracts, outcome records, and version dependencies. 6. Discussion 6.1. Scope across Software Domains The framework’s scope follows from the relationships it represents, rather than the surface form of a task. Table 6 instantiates the same questions for four domains: what constitutes a binding, whose authority matters, which event changes the permissible continuation, and what evidence establishes an outcome. The examples are hypothetical design probes. Their value is to expose different interpretations of the shared model, including cases in which a host cannot support a desired contract. Table 6. Domain interpretations of task bindings, authority, and continuity in four hypothetical design probes. Application and task Binding and authority Intervention/continuity probe and limitation Collaboration: share meeting materials Exact views, recipients, release-policy version; organizer, document owners, and disclosure approver. Remove a recipient after preview. Rebind approval; an admitted delivery can still commit after cancellation. A received disclosure is not undone by deleting a sent item. Spreadsheet: clean a shared data range Workbook version, stable row identities, formulas/protected ranges; task requester and workspace policy. The user sorts or edits while the agent prepares a patch. Reconcile by row identity and pre-values; undo must not overwrite a later collaborator edit. Positional cell coordinates alone are insufficient. IDE: apply a refactoring Worktree/branch, file versions, proposed diff, command scope; developer rights and repository policy. The developer edits a file or takes over while a remote worker runs. Fence that worker and reconcile its patch/processes. Local edit approval does not authorize merge, deployment, or arbitrary shell effects. Service console: refund a payment Payment/order state, amount, beneficiary, policy threshold; operator, account/resource authority, finance approver, affected customer. Transfer the case to another employee after a timeout. Recheck the recipient employee’s rights and query the stable refund ID. Without provider duplicate prevention, blind replay risks a second refund. The differences are substantial. Spreadsheet correctness depends on stable row identity and interference with later edits; an IDE must distinguish a local patch from authority to run commands or deploy; a refund depends on financial policy and the provider’s duplicate-prevention semantics. Thus Post and the assumptions in Dep are domain-specific, even when task revision and endorsement have a common representation. In each case, the abstraction must preserve consequential intermediate effects as well as the final state. A workflow that briefly discloses confidential information and then deletes it is not equivalent to one that never disclosed it. The H1–H4 distinction follows the same logic. An editable spreadsheet continuation supports operational takeover, while transferring a service case changes accountable principals. Remote IDE continuation requires durable task and process identity; recovery after a refund timeout requires provider evidence. A common record enables analysis across these cases, but their authority and continuation rules remain different. This is the intended generality of the framework: shared questions and semantic relations with explicit domain interpretations. 6.2. Human Authority and the Cost of Supervision An inspectable abstraction does not ensure that people can supervise it effectively. A long, technically complete preview can conceal the one changed recipient that matters. The choice of mandatory review must therefore have an accountable source. Application and domain owners define required classes, e.g., cross-boundary disclosure, financial commitment, or destructive changes to shared resources. Resource policies add further conditions. A requester may narrow delegation or demand additional review, but cannot waive another principal’s restriction. A model can flag uncertainty; it cannot independently downgrade a required gate. Reversibility, affected people, disclosure scope, commitment cost, unfamiliar authority, and unresolved intent provide candidate dimensions for these decisions. Their interpretation depends on the domain. Formatting a private draft and refunding a payment need different thresholds, while ambiguity about what the user wants differs from uncertainty about a retrievable fact. Human–AI interaction guidance supports correction and usable scoping (Amershi et al., 2019b); the framework locates the corresponding decisions in the task and authority model. When a required classification cannot be established, preparation can continue only within the remaining non-effectful scope. Supervisory burden is consequently multidimensional. Active inspection time, intervention frequency, context switches, and recovery effort capture different costs; latency and computation are separate measures. Their interpretation requires task outcomes and decision quality, including consequential mismatches approved without detection and interruptions unnecessary under the applicable policy. Reducing the number of confirmation dialogs alone can improve convenience while weakening control. Future human studies should test whether the IIA makes meaningful differences easier to recognize and act upon for a specified user population. 6.3. Architectural Scope and Theoretical Limits AIS and IIA operate at different levels of description. The former identifies a software pattern with a continuing core and dual interaction paths; the latter specifies the task semantics through which delegated activity becomes inspectable and controllable. Neither implies a single software module. Their overlap with mixed-initiative or intent-oriented systems does not diminish the need to define the application-level relation, but it does place the burden of contribution on the explanatory and engineering value of that relation. GUI-agent engineering already motivates attention to safety, recovery, and maintenance (Yu et al., 2026d); the framework gives those concerns a shared semantic object within the application. The formalization makes selected obligations precise while leaving three limits visible. First, the abstraction must be adequate: a relevant effect omitted from Y cannot be assessed through its traces. Second, its realization must expose trustworthy bindings and evidence; stronger models cannot reconstruct unavailable commit receipts or authoritative permission changes. Third, safety conformance does not imply progress, semantic usefulness, fairness, or usability. Those properties require further assumptions and evaluation. The propositions establish conditional consequences of the definitions and gate assumptions, not verified properties of a deployed system. These limits also determine adoption choices. An application with inaccessible third-party state or unmediated write paths may support a restricted contract. A deterministic workflow may offer lower cost for a stable goal and procedure. The strongest role for AIS is therefore not universal replacement of conventional interaction, but selective delegation whose assumptions and consequences remain available to the people who depend on the application. 6.4. What Would Establish the Value of This Perspective? The central hypothesis is that maintaining interaction–effect obligations as shared artifacts can expose defects and dependencies that isolated UI, agent, and API descriptions leave implicit. Its evaluation should compare the effort and outcomes of building and evolving representative integrations with and without those artifacts. Relevant evidence would concern missed binding changes, authority violations, recovery errors, evidence invalidation, and the quality of user interventions. The existing examples identify such observations without supplying empirical results. The hypothesis could fail in several informative ways. Existing application contracts may already maintain the same relationships at comparable effort. Annotation and evidence costs may outweigh the defects prevented. Users may overlook consequential changes despite a more faithful abstraction. Such findings would narrow the useful scope of the framework and guide lighter realizations. The present perspective provides definitions, conditional arguments, and an agenda for investigating these possibilities; it does not claim an implemented runtime, demonstrated productivity gains, or cross-domain validation. 7. Conclusion Integrating an intelligent agent into an existing application introduces an enduring relationship between direct operations, delegated tasks, and shared business state. We propose AIS as the software pattern that contains this relationship and IIA as its task-level interaction abstraction. Modeling their correspondence makes a central obligation explicit: task revisions, role-specific authority, operational control, and outcome evidence must remain connected throughout execution. Interaction contracts specify that obligation, while continuous assurance maintains the standing of claims about its realization as dependencies change. The resulting research agenda joins application analysis and development assistance with control semantics, quality assessment, and human supervision. Progress should be judged by whether this shared abstraction helps developers build and maintain useful delegation at acceptable cost, while preserving the user’s ability to understand and influence what the application does. References Abadi and Lamport (1991) M. Abadi and L. Lamport The existence of refinement mappings. Theoretical Computer Science 82 (2), p. 253–284. External Links: Document Cited by: §2.2, §2. AG-UI Contributors (2026) AG-UI Contributors AG-UI: agent–user interaction protocol. events and interrupts. Note: Living protocol documentationDocumentation version consulted 10 September 2026; the year identifies the consulted snapshot External Links: Link Cited by: §1, §3.2. Amershi et al. (2019a) S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Nagappan, B. Nushi, and T. Zimmermann Software engineering for machine learning: a case study. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), p. 291–300. External Links: Document Cited by: §3.8. Amershi et al. (2019b) S. Amershi, D. Weld, M. Vorvoreanu, A. Fourney, B. Nushi, P. Collisson, J. Suh, S. Iqbal, P. N. Bennett, K. Inkpen, J. Teevan, R. Kikin-Gil, and E. Horvitz Guidelines for human-AI interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, New York, NY, USA. External Links: Document Cited by: §6.2. Arawjo et al. (2024) I. Arawjo, C. Swoopes, P. Vaithilingam, M. Wattenberg, and E. L. Glassman ChainForge: a visual toolkit for prompt engineering and LLM hypothesis testing. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, External Links: Document Cited by: §5.4. Arsan et al. (2021) D. Arsan, A. Zaidi, A. Sagar, and R. Kumar App-based task shortcuts for virtual assistants. In The 34th Annual ACM Symposium on User Interface Software and Technology, p. 1089–1099. External Links: Document Cited by: Figure 2, §2.4. Barres et al. (2025) V. Barres, H. Dong, S. Ray, X. Si, and K. Narasimhan τ^2τ 2-bench: evaluating conversational agents in a dual-control environment. Note: arXiv preprint arXiv:2506.07982, version 1 External Links: Document, Link Cited by: §4.5. Breck et al. (2017) E. Breck, S. Cai, E. Nielsen, M. Salib, and D. Sculley The ML test score: a rubric for ML production readiness and technical debt reduction. In 2017 IEEE International Conference on Big Data (Big Data), p. 1123–1132. External Links: Document Cited by: §4.4. Cai et al. (2025) H. Cai, Y. Li, W. Wang, F. Zhu, X. Shen, W. Li, and T. Chua Large language models empowered personalized web agents. In Proceedings of the ACM on Web Conference 2025, p. 198–215. External Links: Document Cited by: §3.3, Table 5. Chen et al. (2018) T. Y. Chen, F. Kuo, H. Liu, P. Poon, D. Towey, T. H. Tse, and Z. Q. Zhou Metamorphic testing: a review of challenges and opportunities. ACM Computing Surveys 51 (1). External Links: Document Cited by: §5.5. Debenedetti et al. (2025) E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Carlini, D. Fabian, C. Kern, C. Shi, A. Terzis, and F. Tramèr Defeating prompt injections by design. Note: arXiv preprint arXiv:2503.18813, version 2 External Links: Document, Link Cited by: §3.5. Debenedetti et al. (2024) E. Debenedetti, J. Zhang, M. Balunović, L. Beurer-Kellner, M. Fischer, and F. Tramèr AgentDojo: a dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. In Advances in Neural Information Processing Systems, Vol. 37, p. 82895–82920. External Links: Document, Link Cited by: §3.5, Table 5. Dong et al. (2024) L. Dong, Q. Lu, and L. Zhu AgentOps: enabling observability of LLM agents. Note: arXiv preprint arXiv:2411.05285, version 2 External Links: Document, Link Cited by: §3.7, §4.4, Table 5. Epperson et al. (2025) W. Epperson, G. Bansal, V. Dibia, A. Fourney, J. Gerrits, E. Zhu, and S. Amershi Interactive debugging and steering of multi-agent AI systems. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, External Links: Document Cited by: §3.7, Table 5. Feng et al. (2026) K. J. K. Feng, K. Pu, M. Latzke, T. August, P. Siangliulue, J. Bragg, D. S. Weld, A. X. Zhang, and J. C. Chang Cocoa: co-planning and co-execution with AI agents. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, External Links: Document Cited by: §1, Table 5. Garcia-Molina and Salem (1987) H. Garcia-Molina and K. Salem Sagas. In Proceedings of the 1987 ACM SIGMOD International Conference on Management of Data, New York, NY, USA, p. 249–259. External Links: Document Cited by: §3.6, Table 5. He et al. (2025) G. He, G. Demartini, and U. Gadiraju Plan-then-execute: an empirical study of user trust and team performance when using LLM agents as a daily assistant. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, External Links: Document Cited by: §3.1, Table 5. Horvitz (1999) E. Horvitz Principles of mixed-initiative user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, p. 159–166. External Links: Document Cited by: §1, Figure 2, §2.4, Table 5. Hu et al. (2014) V. C. Hu, D. Ferraiolo, R. Kuhn, A. Schnitzer, K. Sandlin, R. Miller, and K. Scarfone Guide to attribute based access control (ABAC) definition and considerations. NIST Special Publication Technical Report 800-162, National Institute of Standards and Technology. Note: Includes updates as of 2 August 2019 External Links: Document, Link Cited by: §3.5. Huang et al. (2025) K. Huang, A. Prabhakar, S. Dhawan, Y. Mao, H. Wang, S. Savarese, C. Xiong, P. Laban, and C. Wu CRMArena: understanding the capacity of LLM agents to perform professional CRM tasks in realistic environments. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 3830–3850. External Links: Document, Link Cited by: §4.5. ISO/IEC (2023a) ISO/IEC ISO/IEC 25010:2023: systems and software engineering. systems and software quality requirements and evaluation (SQuaRE). product quality model. International Organization for Standardization. External Links: Link Cited by: §4.1. ISO/IEC (2023b) ISO/IEC ISO/IEC 25059:2023: software engineering. systems and software quality requirements and evaluation (SQuaRE). quality model for AI systems. International Organization for Standardization. External Links: Link Cited by: §4.1. Khattab et al. (2024) O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts DSPy: compiling declarative language model calls into state-of-the-art pipelines. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §5.4. Li et al. (2023) M. Li, Y. Zhao, B. Yu, F. Song, H. Li, H. Yu, Z. Li, F. Huang, and Y. Li API-Bank: a comprehensive benchmark for tool-augmented LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 3102–3116. External Links: Document, Link Cited by: Figure 2, §2.4, Table 5. Li et al. (2017) T. J. Li, A. Azaria, and B. A. Myers SUGILITE: creating multimodal smartphone automation by demonstration. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, p. 6038–6049. External Links: Document Cited by: Figure 2, §2.4. Lu et al. (2025) J. Lu, T. Holleis, Y. Zhang, B. Aumayer, F. Nan, H. Bai, S. Ma, S. Ma, M. Li, G. Yin, Z. Wang, and R. Pang ToolSandbox: a stateful, conversational, interactive evaluation benchmark for LLM tool use capabilities. In Findings of the Association for Computational Linguistics: NAACL 2025, p. 1160–1183. External Links: Document, Link Cited by: §4.5, Table 5. Meyer (1992) B. Meyer Applying “Design by Contract”. Computer 25 (10), p. 40–51. External Links: Document Cited by: §2. Parnin et al. (2025) C. Parnin, G. Soares, R. Pandita, S. Gulwani, J. Rich, and A. Z. Henley Building your own product copilot: challenges, opportunities, and needs. In 2025 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), p. 338–348. External Links: Document Cited by: Figure 2, §2.4, Table 5. Qin et al. (2024) Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qian, S. Zhao, L. Hong, R. Tian, R. Xie, J. Zhou, M. Gerstein, D. Li, Z. Liu, and M. Sun ToolLLM: facilitating large language models to master 16000+ real-world APIs. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §3.2. Rajbahadur et al. (2026) G. K. Rajbahadur, G. A. Oliva, D. Lin, J. Shin, and A. E. Hassan From cool demos to production-ready FMware: core challenges and a technology roadmap. ACM Transactions on Software Engineering and Methodology. Note: Online accepted manuscript External Links: Document Cited by: Figure 2, §2.4, §3.8, Table 5. Rawles et al. (2025) C. Rawles, S. Clinckemaillie, Y. Chang, J. Waltz, G. Lau, M. Fair, A. Li, W. E. Bishop, W. Li, F. Campbell-Ajala, D. K. Toyama, R. J. Berry, D. Tyamagundlu, T. P. Lillicrap, and O. Riva AndroidWorld: a dynamic benchmarking environment for autonomous agents. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §4.5. Saltzer et al. (1984) J. H. Saltzer, D. P. Reed, and D. D. Clark End-to-end arguments in system design. ACM Transactions on Computer Systems 2 (4), p. 277–288. External Links: Document Cited by: §4.1. Saltzer and Schroeder (1975) J. H. Saltzer and M. D. Schroeder The protection of information in computer systems. Proceedings of the IEEE 63 (9), p. 1278–1308. External Links: Document Cited by: §3.5. Schick et al. (2023) T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom Toolformer: language models can teach themselves to use tools. In Advances in Neural Information Processing Systems, Vol. 36, p. 68539–68551. External Links: Document, Link Cited by: §2.4. Sculley et al. (2015) D. Sculley, G. Holt, D. Golovin, E. Davydov, T. Phillips, D. Ebner, V. Chaudhary, M. Young, J. Crespo, and D. Dennison Hidden technical debt in machine learning systems. In Advances in Neural Information Processing Systems, Vol. 28, p. 2503–2511. External Links: Link Cited by: §3.8, Table 5. Seshadri et al. (2026) P. Seshadri, S. Cahyawijaya, A. Odumakinde, S. Singh, and S. Goldfarb-Tarrant Lost in simulation: LLM-simulated users are unreliable proxies for human users in agentic evaluations. Note: arXiv preprint arXiv:2601.17087, version 2 External Links: Document, Link Cited by: §4.5. Shi et al. (2026) Q. Shi, A. Zytek, P. Razavi, K. Narasimhan, and V. Barres τ-knowledge: evaluating conversational agents over unstructured knowledge. Note: arXiv preprint arXiv:2603.04370, version 1 External Links: Document, Link Cited by: §3.3, Table 5. Shneiderman (1983) B. Shneiderman Direct manipulation: a step beyond programming languages. Computer 16 (8), p. 57–69. External Links: Document Cited by: Figure 2, §2.4. Trivedi et al. (2024) H. Trivedi, T. Khot, M. Hartmann, R. Manku, V. Dong, E. Li, S. Gupta, A. Sabharwal, and N. Balasubramanian AppWorld: a controllable world of apps and people for benchmarking interactive coding agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 16022–16076. External Links: Document, Link Cited by: §4.5, Table 5. Wang et al. (2026) H. Wang, C. M. Poskitt, and J. Sun AgentSpec: customizable runtime enforcement for safe and reliable LLM agents. In Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering, External Links: Link Cited by: §3.5, §5.3, Table 5. Wen et al. (2024) H. Wen, Y. Li, G. Liu, S. Zhao, T. Yu, T. J. Li, S. Jiang, Y. Liu, Y. Zhang, and Y. Liu AutoDroid: LLM-powered task automation in Android. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking, p. 543–557. External Links: Document Cited by: Figure 2, §2.4. Xie et al. (2026) T. Xie, D. Ran, Y. Cao, M. Wu, Y. Guo, and W. Yang From user operations to agentic automation: toward intent-oriented software in the LLM era. Journal of Computer Science and Technology 41 (1), p. 245–257. External Links: Document Cited by: §1. Xie et al. (2024) T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Lei, Y. Liu, Y. Xu, S. Zhou, S. Savarese, C. Xiong, V. Zhong, and T. Yu OSWorld: benchmarking multimodal agents for open-ended tasks in real computer environments. In Advances in Neural Information Processing Systems, Vol. 37, p. 52040–52094. External Links: Document, Link Cited by: §4.5. Xie et al. (2025) W. Xie, L. Zhang, S. Wang, R. Yi, and M. Xu DroidCall: a dataset for LLM-powered Android intent invocation. In Findings of the Association for Computational Linguistics: EMNLP 2025, p. 9116–9134. External Links: Document, Link Cited by: §3.2. Yao et al. (2025) S. Yao, N. Shinn, P. Razavi, and K. Narasimhan τ-bench: a benchmark for tool-agent-user interaction in real-world domains. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §4.5. Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: Figure 2, §2.4. Yu et al. (2024) S. Yu, C. Fang, X. Li, Y. Ling, Z. Chen, and Z. Su Effective, platform-independent GUI testing via image embedding and reinforcement learning. ACM Transactions on Software Engineering and Methodology 33 (7). External Links: Document Cited by: §5.1. Yu et al. (2023) S. Yu, C. Fang, Y. Ling, C. Wu, and Z. Chen LLM for test script generation and migration: challenges, capabilities, and opportunities. In 2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security (QRS), p. 206–217. External Links: Document Cited by: §5.4. Yu et al. (2026a) S. Yu, C. Fang, J. Liu, and Z. Chen Test script intention generation for mobile application via GUI image and code understanding. ACM Transactions on Software Engineering and Methodology 35 (1). External Links: Document Cited by: §5.1. Yu et al. (2026b) S. Yu, C. Fang, Z. Tuo, Q. Zhang, C. Chen, Z. Chen, and Z. Su Vision-based mobile app GUI testing: a survey. ACM Computing Surveys 58 (6). External Links: Document Cited by: §5.1. Yu et al. (2021) S. Yu, C. Fang, Y. Yun, and Y. Feng Layout and image recognition driving cross-platform automated mobile testing. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), p. 1561–1571. External Links: Document Cited by: §3.3. Yu et al. (2026c) S. Yu, Y. Ling, C. Fang, Q. Zhou, Y. Zhao, C. Chen, S. Zhu, and Z. Chen Scenario-guided LLM-based mobile app GUI testing. ACM Transactions on Software Engineering and Methodology. Note: Online accepted manuscript External Links: Document Cited by: §4.5. Yu et al. (2026d) S. Yu, Y. Ling, J. Xing, Q. Zhou, C. Fang, and Z. Chen Software engineering for and with GUI agent. Note: arXiv preprint arXiv:2608.09278, version 1 External Links: Document, Link Cited by: §6.3. Zhang et al. (2025) C. Zhang, H. Huang, C. Ni, J. Mu, S. Qin, S. He, L. Wang, F. Yang, P. Zhao, C. Du, L. Li, Y. Kang, Z. Jiang, S. Zheng, R. Wang, J. Qian, M. Ma, J. Lou, Q. Lin, S. Rajmohan, and D. Zhang UFO2: the desktop AgentOS. Note: arXiv preprint arXiv:2504.14603 External Links: Document, Link Cited by: §2.4. Zhou et al. (2024) S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, T. Ou, Y. Bisk, D. Fried, U. Alon, and G. Neubig WebArena: a realistic web environment for building autonomous agents. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §4.5.

Instant research discovery