Paper deep dive
Harness Continual Learning: Continual Adaptation Beyond Model Parameters
Borui Kang, Jinrui Gu, Junhan Lv, Wenbin Li, Lei Wang, Yang Gao
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/20/2026, 5:28:28 AM
Summary
The paper introduces Harness Continual Learning (HCL), a paradigm where an agent adapts by updating an external 'harness' (prompts, memory, tools, routing) around a frozen foundation model, rather than updating model parameters. This approach addresses 'harness-level forgetting,' where updates to the harness disrupt previously reliable behaviors. The framework includes four components: Task Interface, Experience Memory, Capability Map, and Adaptive Router, managed by a Guarded Harness Evolution process involving a Continual Optimizer and Evaluator to balance stability and plasticity.
Entities (11)
Relation Signals (9)
Harness Update → causes → Harness-level Forgetting
confidence 96% · a harness update can disrupt previously reliable behavior... define the resulting loss of earlier behavior as harness-level forgetting.
Harness Continual Learning → uses → Frozen Foundation Model
confidence 95% · HCL is a new continual learning paradigm in which the harness evolves around a frozen foundation model
Guarded Harness Evolution → includes → Continual Optimizer
confidence 93% · A Continual Optimizer proposes candidate harnesses from post-execution feedback
Guarded Harness Evolution → includes → Continual Evaluator
confidence 93% · a Continual Evaluator commits the resulting candidate harness only after checking current improvement, historical retention, and validity.
Harness Continual Learning → comprises → Adaptive Router
confidence 92% · We instantiate HCL with four execution-facing components: ... and Adaptive Router.
Harness Continual Learning → comprises → Task Interface
confidence 92% · We instantiate HCL with four execution-facing components: the Task Interface...
Harness Continual Learning → comprises → Experience Memory
confidence 92% · We instantiate HCL with four execution-facing components: ... Experience Memory ...
Harness Continual Learning → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Continual learning has largely been model-centric, treating model parameters as the state that changes with sequential experience. Modern agents can also adapt through a harness of prompts, memories, tools, skills, and routing rules. Because these contents jointly shape later execution, a harness update can disrupt previously reliable behavior even when the model is frozen. This raises a new question: how can an agent continually improve its state outside the model while retaining behavior acquired earlier? We formulate Harness Continual Learning (HCL), a new continual learning paradigm in which the harness evolves around a frozen foundation model, and define the resulting loss of earlier behavior as harness-level forgetting. We instantiate HCL with four execution-facing components: the Task Interface, Experience Memory, Capability Map, and Adaptive Router. We further introduce guarded harness evolution to separate update generation from state commitment. A Continual Optimizer proposes candidate harnesses from post-execution feedback, and a Continual Evaluator commits the resulting candidate harness only after checking current improvement, historical retention, and validity. Experiments on textual reasoning, multimodal perception, and open-world interaction demonstrate capability accumulation and failure recovery, with relative gains exceeding 10% over corresponding baselines in multiple settings. Component ablations assess the contribution of each harness component, while controlled retention sweeps reveal measurable harness-level forgetting and show that the stability--plasticity trade-off can be explicitly adjusted.
Tags
Links
- Source: https://arxiv.org/abs/2608.19013v1
- Canonical: https://arxiv.org/abs/2608.19013v1
Trouble viewing inline? Open PDF directly →
Full Text
75,697 characters extracted from source content.
Expand or collapse full text
Harness Continual Learning: Continual Adaptation Beyond Model Parameters Borui Kang Jinrui Gu Junhan Lv Wenbin Li Thanks: Corresponding author Lei Wang Affiliation: University of Wollongong, Australia Yang Gao [0.5em] State Key Laboratory for Novel Software Technology Nanjing University China Abstract Continual learning has largely been model-centric, treating model parameters as the state that changes with sequential experience. Modern agents can also adapt through a harness of prompts, memories, tools, skills, and routing rules. Because these contents jointly shape later execution, a harness update can disrupt previously reliable behavior even when the model is frozen. This raises a new question: how can an agent continually improve its state outside the model while retaining behavior acquired earlier? We formulate Harness Continual Learning (HCL), a new continual learning paradigm in which the harness evolves around a frozen foundation model, and define the resulting loss of earlier behavior as harness-level forgetting. We instantiate HCL with four execution-facing components: the Task Interface, Experience Memory, Capability Map, and Adaptive Router. We further introduce guarded harness evolution to separate update generation from state commitment. A Continual Optimizer proposes candidate harnesses from post-execution feedback, and a Continual Evaluator commits the resulting candidate harness only after checking current improvement, historical retention, and validity. Experiments on textual reasoning, multimodal perception, and open-world interaction demonstrate capability accumulation and failure recovery, with relative gains exceeding 10% over corresponding baselines in multiple settings. Component ablations assess the contribution of each harness component, while controlled retention sweeps reveal measurable harness-level forgetting and show that the stability–plasticity trade-off can be explicitly adjusted. 1 Introduction Continual learning studies how a system acquires capabilities from sequential experience while retaining previously learned behavior (8; 42; 35). Existing formulations realize this process mainly by changing model parameters, representations, or architectural components. We refer to this established view as model-centric continual learning. The rise of agentic AI introduces another source of adaptation: an external harness that determines how a foundation model receives information, retrieves experience, and acts (12; 47; 48; 4; 20; 29). Prompts, memories, tool and skill specifications, and routing policies can persist and evolve across interactions even when the foundation model remains frozen. Agent adaptation is therefore no longer confined to model state: harness state can also accumulate experience and reshape future behavior. This makes the harness a new object of continual learning research, extending the study of continual adaptation beyond model parameters, as illustrated in Figure 1. Figure 1: The shift in the object of continual learning. Model-centric methods update model parameters θ over sequential experience. HCL instead updates harness state around a frozen foundation model. In both settings, adaptation can improve new behavior while interfering with behavior acquired earlier. We formalize this new direction as Harness Continual Learning (HCL), a continual learning paradigm that acquires and retains capabilities by sequentially updating harness state around a frozen foundation model. Conventional harness optimization typically searches for prompts, functions, or workflows that improve a current objective (56; 55). HCL instead studies a sequence of updates. Its concern is not only whether the next update helps the current interaction, but also whether the evolving harness retains behavior that earlier updates made reliable. This setting introduces a distinct retention problem. Harness components are coupled in execution: a memory update can change the evidence retrieved for an earlier query; a skill revision can alter tool use; and a routing edit can break a previously successful workflow. An update that helps recent cases can therefore turn an earlier correct answer, valid tool call, or successful action trajectory into a failure without changing the foundation model. We call this phenomenon harness-level forgetting. It extends the classical stability–plasticity problem from model state to harness state. To study continual adaptation under this retention requirement, we develop an HCL framework with two parts. First, we define the Task Interface, Experience Memory, Capability Map, and Adaptive Router as the harness state and learning object of HCL. These components are jointly versioned and determine how the agent processes information, reuses experience and capabilities, and organizes execution. Second, guarded harness evolution governs state transitions through two modules: a Continual Optimizer that proposes candidate harnesses from post-execution feedback, and a Continual Evaluator that determines whether those candidates can be committed. The two parts jointly operationalize HCL: the former defines what is learned, while the latter controls how the harness is updated over time. Only a candidate harness that improves current validation performance while satisfying the historical-retention budget and validity constraints is committed as the deployed state. This proposal–evaluation–commitment process makes retention an explicit condition of harness adaptation, mitigating harness-level forgetting while controlling the stability–plasticity trade-off. We evaluate HCL across textual reasoning, multimodal perception, and open-world interaction. The results show that harness evolution can accumulate capabilities and support failure recovery, while also producing measurable harness-level forgetting. Historical-retention budgets shift the operating point between adaptation and retention, and more permissive updates do not necessarily produce a stronger final harness. In this work, our contributions are as follows: • We propose and formalize Harness Continual Learning as a new continual learning paradigm, shifting the learning object from model state to harness state around a frozen foundation model. • We identify harness-level forgetting and develop guarded harness evolution, in which a Continual Optimizer proposes candidate harnesses and a Continual Evaluator controls commitment through current, historical, and validity checks. • We show across textual reasoning, multimodal perception, and open-world interaction that harness evolution supports capability accumulation and failure recovery while exhibiting measurable forgetting and a controllable stability–plasticity trade-off. 2 Related Work 2.1 Harness Engineering Contemporary agent systems place a runtime harness around a foundation model to turn inference into task-directed execution (20; 29; 11; 61). Across implementations, persistent runtime contents commonly serve four functions. An interface converts raw instructions, observations, documents, or multimodal inputs into a form the agent can use. Memory stores interaction records, summaries, and reusable guidance. A capability registry describes tools, APIs, environment actions, and learned skills together with their invocation conditions. A router or workflow controller selects relevant memories and capabilities, orders their use, and assembles the execution context. Environment adapters execute actions, and task-specific validators check outcomes at the boundary of the pipeline (10; 6). Existing systems develop different parts of this structure. ReAct couples reasoning with environment interaction (50). Toolformer, MRKL, and HuggingGPT expose and coordinate external capabilities (32; 15; 34). MemGPT, Reflexion, and Voyager retain experience as memory, feedback, or executable skills (30; 36; 41). Together, these components form a coupled execution pipeline. The interface shapes what the router sees. Memory and capability descriptions determine what it can select. The resulting workflow determines how the model acts. Harness engineering also uses execution feedback to revise prompts, declarative programs, memories, tool-use policies, skills, and workflows (62; 16; 2; 32; 36; 41; 59; 58). Recent work broadens this process to configuration search, cross-layer failure diagnosis, and sustained agent improvement (53; 5; 51; 25). These systems show that a harness is editable and can improve with experience. Their main objective, however, is usually the quality of a component or the next configuration on a current task or target distribution. Repeated improvement alone does not provide a general retention criterion for the full harness state (23). Our work differs by treating the entire mutable harness as a unified continual learning state and by making retention across committed updates an explicit objective. 2.2 Model-Centric Continual Learning Model-centric continual learning adapts a model to a non-stationary stream of tasks or data while seeking to retain capabilities acquired from earlier experience. Its central challenge is catastrophic forgetting, which arises when learning new knowledge disrupts knowledge encoded by the model (18; 8; 42; 13). Representation-based approaches learn features or prompts that remain useful across tasks (45; 44). Recent analysis also examines how these internal representations shift across a learning sequence (17). Architecture-based approaches isolate, expand, or select model components to reduce interference between tasks (24; 28). Optimization-based approaches alter the update trajectory or constrain gradients using information from earlier tasks (27; 1; 33). Regularization-based approaches penalize changes to parameters or functions that support old behavior (18; 19). Replay-based approaches retain or reconstruct earlier examples and mix them with new data (40; 43; 52; 3). Recent work extends these families to large language models and broader knowledge streams, but the state being learned remains model knowledge, representations, architectures, or parameters (22; 54). Our work moves the continual learning object outside the model. The foundation model parameters remain frozen, while the harness state evolves under explicit acquisition and retention constraints. 3 Harness Continual Learning Table 1: Execution functions and updatable contents of the four jointly versioned components in the deployed harness HnH_n. Component Function during execution Contents updated in HCL Task Interface InI_n Transforms raw interactions into structured representations. Prompts, task templates, and parsing and normalization rules. Experience Memory MnM_n Provides concrete interactions and abstract guidance for reuse. Raw interaction records and LLM-generated Abstract Memory entries. Capability Map CnC_n Provides external operations and reusable inner skills. Inner skills extracted from Abstract Memory. Adaptive Router RnR_n Selects and organizes memory and capabilities. Routing prompts, selection criteria, and workflow templates. 3.1 Definition and Problem Setting Consider a fixed foundation model FθF_θ and a harness HnH_n deployed at interaction step n. The model parameters θ remain unchanged, whereas a committed harness update affects subsequent interactions. We define Harness Continual Learning as the problem of sequentially updating the deployed harness to acquire new behavior while retaining behavior that was reliable before the update. Previously reliable behavior may be a correct response, a valid tool call, or an action trajectory that satisfies an environment goal. Retention requires such behavior to remain successful after later harness updates when evaluated under the same input and execution conditions. This setting differs from conventional harness engineering, which typically optimizes a prompt, tool configuration, or workflow for a current objective. HCL instead studies a sequence of deployed harnesses. At interaction step n, nu_n denotes the raw interaction, such as an instruction, an observation, or a multimodal input. The harness transforms nu_n into the structured interaction ni_n. Guided by the frozen foundation model, it then combines ni_n with selected memory and capabilities to assemble the execution context nz_n. The model and external runtime execute nz_n to produce the outcome ny_n. Post-execution feedback is denoted by nf_n. We collect these interaction-level objects as n=(n,n,n,n,n).e_n= (u_n,i_n,z_n,y_n,f_n ). (1) The Optimizer provides the foundation model FθF_θ with an update rule, the deployed harness, and the available interaction evidence as context for generating a candidate harness:: H~n+1=Fθ(Hn,n). H_n+1=O_F_θ (H_n,e_n ). (2) The candidate remains separate from the deployed harness until a commitment decision is made. Let Gn∈0,1G_n∈\0,1\ denote this decision. The deployed harness evolves as Hn+1=H~n+1,Gn=1,Hn,Gn=0.H_n+1= cases H_n+1,&G_n=1,\\ H_n,&G_n=0. cases (3) Therefore, a candidate affects later interactions only when it is committed. Our framework realizes HCL in two parts. First, it defines the deployed harness state HnH_n by specifying its mutable contents and versioning them jointly. Second, it controls the update from HnH_n to Hn+1H_n+1 by checking current improvement, historical retention, and validity before commitment. 3.2 Harness State for Continual Learning The design of the HCL state builds on established mechanisms from prior harness and agent systems, including prompt-based task interfaces, persistent memory, tool and skill registries, and routing or workflow controllers (20; 11). Rather than inheriting the architecture of any single system, HCL organizes these recurring execution functions into four jointly versioned components, whose mutable contents evolve from sequential experience under explicit acquisition and retention constraints. Accordingly, HCL organizes the mutable harness state as Hn=(In,Mn,Cn,Rn),H_n= (I_n,M_n,C_n,R_n ), (4) where InI_n, MnM_n, CnC_n, and RnR_n denote the Task Interface, Experience Memory, Capability Map, and Adaptive Router, respectively. At interaction step n, HnH_n represents the complete harness currently deployed. Its prompts and processing rules, stored experience, reusable skills, and routing specifications persist across interactions and jointly determine how the agent handles future tasks. Although these four execution functions are common in agent harnesses, HCL differs in how their mutable contents are learned and deployed. Because a change to one component may interact with the others and affect both new and previously learned behavior, HCL treats all proposed changes as one complete candidate harness. The candidate replaces HnH_n only after it satisfies current improvement, historical retention, and validity requirements. Otherwise, none of its changes enters the deployed harness. HCL therefore turns harness contents into a coordinated mechanism for continual learning rather than a collection of independently edited artifacts. Table 1 summarizes the execution function of each component and the contents that can be updated through continual interaction. Figure 2 shows how these components support execution and how post-execution feedback initiates a candidate harness. Figure 2: Overview of the HCL framework. The deployed harness HnH_n supports the execution path from raw interaction nu_n to outcome ny_n. When post-execution feedback is available, the Continual Optimizer proposes a candidate harness H~n+1 H_n+1, and the Continual Evaluator accepts or rejects it based on current improvement, historical retention, and validity. 3.2.1 Task Interface The Task Interface is the input-processing layer of the harness. It transforms a raw task interaction nu_n into a structured representation of the available input, task objective, and execution constraints: n=In(n)=(n,n,n),i_n=I_n (u_n )= (x_n,g_n,k_n ), (5) where nx_n contains the available input, ng_n specifies what the task aims to accomplish, and nk_n records constraints such as output format, legal tool use, and environment restrictions. Internally, InI_n specifies the prompts, task templates, and parsing and normalization rules used by an LLM-based parser to perform this transformation. In HCL, the Task Interface maps heterogeneous task data into a unified representation, making the relevant input, objective, and constraints explicit. This helps the agent focus on task requirements and process different task forms within the same continual learning pipeline. Since interface updates may change how tasks are interpreted, InI_n is versioned with the harness. 3.2.2 Experience Memory Agent memory can take many forms, including episodic records, summaries, and reflections (31; 30; 60; 36; 46). From a continual learning perspective, HCL organizes accumulated experience into two complementary forms: Mn=(Mnraw,Mnabs),M_n= (M_n^raw,M_n^abs ), (6) where MnrawM_n^raw and MnabsM_n^abs denote Raw Memory and Abstract Memory, respectively. Raw Memory preserves concrete interactions, whereas Abstract Memory extracts reusable knowledge from them. Raw Memory MnrawM_n^raw stores the raw task input nu_n, the resulting response or action trajectory ny_n, and the subsequent environment or verifier feedback nf_n. To keep memory collection simple and storage bounded, it retains a fixed number of interactions from each task in arrival order. These records preserve task-specific evidence about successful behavior and encountered failures, helping the agent reuse earlier solutions and avoid repeating previous errors. Abstract Memory MnabsM_n^abs is produced by using an LLM to summarize the contents of Raw Memory. The LLM consolidates recurring patterns into scoped guidance, such as output conventions, reliable reasoning patterns, and common errors to avoid. As new raw interactions are stored, the summarization process can produce new or updated abstract entries for related future tasks. Raw Memory retains concrete experience for replay and behavioral recovery, while Abstract Memory generalizes that experience for transfer across tasks. Together, they support adaptation to new tasks while preserving useful knowledge acquired earlier. 3.2.3 Capability Map The Capability Map defines the operations and skills that the agent can invoke during execution. HCL organizes these capabilities by their origin: Cn=(Cnouter,Cninner),C_n= (C_n^outer,C_n^inner ), (7) where CnouterC_n^outer contains capabilities provided by the external runtime, and CninnerC_n^inner contains skills acquired through continual interaction. Outer capabilities connect the frozen model to external resources, such as APIs, retrieval services, perception models, calculators, and environment actions. Each entry specifies its function, expected inputs and outputs, invocation protocol, availability conditions, and known limitations. These capabilities provide the basic operations needed to access information and act in different environments. Inner capabilities are reusable skills further abstracted from MnabsM_n^abs. An LLM can consolidate related abstract memories into more general skills with explicit inputs, outputs, execution steps, and applicable scopes. This turns knowledge accumulated from earlier interactions into procedures that can be directly invoked across tasks. As Abstract Memory evolves, new inner skills can be added and existing skills can be revised. Unlike a static capability map limited to a predefined library of external operations, CnC_n can expand its executable skill set through experience. This dynamic connection between accumulated knowledge and inner capabilities allows the frozen-model agent to continually acquire, refine, and transfer skills across tasks. 3.2.4 Adaptive Router The Adaptive Router connects the Task Interface, Experience Memory, and Capability Map to task execution. Given the structured interaction ni_n, it retrieves relevant experience from MnM_n, selects capabilities from CnC_n, and organizes them into an execution context: n=Rn(n,Mn,Cn).z_n=R_n (i_n,M_n,C_n ). (8) The resulting nz_n contains the structured task representation, selected experience and capabilities, and the workflow used for execution. As MnM_n and CnC_n evolve, which experience and capabilities are useful for a task and how they should be organized may also change. At each interaction, RnR_n uses an LLM together with its routing prompts, selection criteria, and workflow templates to adapt the execution strategy to the current task and available contents. These routing specifications can also be revised across interactions, allowing the Router to evolve alongside Memory and the Capability Map. The frozen model and external runtime then use nz_n to produce the response or action ny_n. 3.3 Guarded Harness Evolution A harness update may improve current behavior while degrading previously reliable behavior on earlier tasks. We therefore introduce guarded harness evolution, which separates update generation from deployment through a proposal–evaluation–commitment process. Given feedback, the Continual Optimizer produces an isolated candidate harness. The Continual Evaluator commits it only if it satisfies current-improvement, historical-retention, and validity requirements. Otherwise, HnH_n remains deployed. This process makes retention an explicit condition for harness evolution rather than assuming that a useful update on the current task is safe for earlier tasks. 3.3.1 Continual Optimizer: Candidate Generation Interaction feedback indicates whether the current execution is successful, but does not specify how the harness should change. The Continual Optimizer implements the update operator O in Eq. (2) using a prompt template for the foundation model FθF_θ. It provides the deployed harness HnH_n and the interaction evidence ne_n to the model and asks it to propose a candidate harness H~n+1 H_n+1. FθF_θ analyzes the execution outcome in light of the feedback and examines the execution context to identify which harness components require revision. It may modify prompts or parsing rules in the Task Interface, record or summarize experience in Memory, add or revise skills in the Capability Map, or adjust selection and workflow rules in the Adaptive Router. To provide alternative update directions while limiting repeated LLM calls, we use a simple sequential strategy when multiple components require revision. The selected components are considered in a predefined order. For each component, the Optimizer generates up to K alternatives one at a time. Each alternative is evaluated by replacing only the selected component in the current candidate harness while keeping all other components fixed. For each selected component, the Continual Optimizer generates up to K alternatives, each of which is evaluated while all other components remain fixed. The highest-scoring admissible alternative is retained as the basis for revising the next component. If no alternative passes the gate, that component remains unchanged. The deployed harness HnH_n remains unchanged until the resulting candidate completes evaluation and is committed. 3.3.2 Continual Evaluator: Historical Evaluation and Commitment To align harness updates with the objective of continual learning, we introduce a retention-aware evaluation standard rather than judging candidates only by current-task gains. The Continual Evaluator E examines three complementary aspects: current improvement measures whether the candidate better solves the current task, historical retention checks whether previously reliable behavior is preserved, and validity ensures that the updated harness and its outputs remain usable. The deployed harness HnH_n and candidate H~n+1 H_n+1 are evaluated under the same model, decoding, tool, environment, and seed conditions to provide a controlled comparison. A candidate can replace HnH_n only when all three requirements are satisfied, allowing the harness to acquire new behavior without ignoring what it has already learned. Current Improvement. Let VnV_n denote the validation cases for the current task, and let P(H,Vn)P(H,V_n) denote the performance of harness H on these cases. The improvement produced by the candidate is Δn=P(H~n+1,Vn)−P(Hn,Vn). _n=P ( H_n+1,V_n )-P (H_n,V_n ). (9) The candidate satisfies this criterion when Δn≥δn _n≥ _n, where δn _n is the predefined minimum improvement. Depending on the task, P may measure answer accuracy, tool-use success, or environment completion. Historical Retention. Current-task improvement does not indicate whether a candidate preserves behavior acquired earlier. The Evaluator therefore maintains a compact anchor set AnA_n for historical evaluation. Each anchor contains the raw input and success criterion of a previously observed case, allowing that case to be rerun under both the deployed and candidate harnesses. At the end of each task, anchors are selected using a predefined ratio of previously successful and failed cases. If either group contains too few cases to meet its target, the remaining slots are filled from the other group. The anchors are used only for evaluation and are unavailable during candidate generation. For each anchor a∈Ana∈ A_n, define the binary success indicator q(H,a)∈0,1,q(H,a)∈\0,1\, (10) where q(H,a)=1q(H,a)=1 if harness H satisfies the corresponding success criterion and 0 otherwise. The historical loss introduced by the candidate is Dn=∑a∈An[q(Hn,a)=1∧q(H~n+1,a)=0],D_n= _a∈ A_n1 [q(H_n,a)=1 q ( H_n+1,a )=0 ], (11) where [⋅]1[·] is the indicator function, equal to 1 when the enclosed condition holds and 0 otherwise. Therefore, DnD_n counts previously solved anchors that fail under the candidate. The candidate satisfies the historical-retention criterion when Dn≤BnD_n≤ B_n, where BnB_n is the predefined tolerance for historical loss. Setting Bn=0B_n=0 requires the candidate to preserve every anchor currently solved by HnH_n. Appendix C specifies the success criterion q(H,a)q(H,a) used for each experimental task. Validity Check. The candidate must also be executable and comply with the task and runtime requirements. Let ℒnL_n denote the set of validity checks applied at interaction step n. For each ℓ∈ℒn _n, define vn,ℓ(H~n+1)∈0,1,v_n, ( H_n+1 )∈\0,1\, (12) where vn,ℓ(H~n+1)=1v_n, ( H_n+1 )=1 indicates that the candidate satisfies validity check ℓ , and 00 otherwise. These checks may cover artifact syntax, output-schema compliance, legal tool use, task constraints, and environment consistency. The three criteria are combined into a candidate-specific commitment decision: Gn(k)=[(Δn(k)≥δn)∧(Dn(k)≤Bn)∧(∀ℓ,vn,ℓ(H~n+1(k))=1)].G_n^(k)=1 [( _n^(k)≥ _n) (D_n^(k)≤ B_n) (∀ ,\;v_n, ( H_n+1^(k) )=1 ) ]. (13) The decision rule in Eq. (13) serves as a hard admissibility gate. When multiple candidates pass the gate, the Continual Evaluator ranks them using a composite score that aggregates their current-performance, validity, and historical-retention scores. The highest-scoring candidate is committed as Hn+1H_n+1, with ties broken randomly. If no candidate passes the gate, HnH_n remains deployed. By making historical retention a necessary condition for commitment, the admissibility gate supports the acquisition of new behavior while explicitly controlling the loss of previously reliable behavior. The tolerance BnB_n further adjusts the balance between stability and plasticity. 3.4 Connections to Model-Centric Continual Learning HCL draws on several complementary principles from model-centric continual learning, but realizes them through harness mechanisms rather than model-parameter updates (8; 42). Replay-based methods retain earlier examples to preserve acquired knowledge. Experience Memory follows this principle by storing concrete interactions for later reuse. Representation-based methods learn abstractions that support transfer across tasks. The Capability Map similarly transforms accumulated experience into reusable skills and combines them with external capabilities. Architecture-based methods organize reusable modules and routines to reduce interference. HCL represents these routines as invocable capabilities and uses the Adaptive Router to select and compose them for each interaction. Optimization- and regularization-based methods control parameter updates using information from earlier tasks, allowing new knowledge to be acquired while limiting interference with previous knowledge. HCL applies the same principle to harness updates through the Continual Optimizer and Continual Evaluator. The Optimizer proposes candidate changes from current feedback, while the Evaluator tests them on current validation cases and historical anchors. Only candidates that improve current performance while satisfying historical retention and validity requirements are committed. This proposal–evaluation–commitment process integrates adaptation and protection into continual harness evolution. These relationships are conceptual rather than one-to-one implementations. More importantly, HCL brings the complementary principles of model-centric continual learning into a unified system-level formulation. Traditional approaches (14; 26) often treat replay, representation, architecture, optimization, and regularization as separate solution families for adapting model parameters. HCL coordinates their functions within a single evolving harness under the same acquisition–retention objective. It therefore extends continual learning from parameter adaptation to the coordinated evolution of agent infrastructure, providing a unified framework for continual learning beyond the model itself. 4 Experiments We evaluate HCL in two regimes. ALFWorld (37) and Minecraft (41) examine capability accumulation, reuse, and failure recovery during open-world interaction. Textual reasoning and multimodal perception use controlled task streams with repeated evaluation of previously observed tasks, making harness-level forgetting and the stability–plasticity trade-off directly measurable. We also evaluate the control of this trade-off and ablate the four editable harness components. We use different foundation models across the experimental settings to examine whether HCL generalizes across model families and scales rather than depending on a particular model. ALFWorld uses Qwen3.5-9B; Minecraft and the main multimodal experiments use Qwen3.6-27B; textual reasoning uses DeepSeek-V4-Flash; and the component ablation uses Qwen3.5-4B. Within each setting, the same foundation model is used for all comparisons and remains frozen throughout the continual-learning stream. Any adaptation therefore comes from harness updates rather than model training. 4.1 Evaluation Protocol For each task stream, a single harness evolves sequentially around the same foundation model. Let H(s)H^(s) denote the deployed harness after learning task sD_s, where s indexes the evaluation stage. At the end of each stage, we evaluate H(s)H^(s) on the current task and every previously observed task: Rs,j=Eval(H(s),jtest),j≤s,R_s,j=Eval (H^(s),D^test_j ), j≤ s, (14) where Rs,jR_s,j is the benchmark score or episode success rate on task j. Current-task validation cases and historical anchors are used only by the Continual Evaluator to determine whether a candidate can be committed. The final test sets are disjoint from both and are used only for reporting. For task streams with metrics on a common scale, we report final average performance and average old-task forgetting: AvgT=1T∑j=1TRT,j,FgtT=1T−1∑j=1T−1(maxr∈j,…,TRr,j−RT,j).Avg_T= 1T _j=1^TR_T,j, _T= 1T-1 _j=1^T-1 ( _r∈\j,…,T\R_r,j-R_T,j ). (15) AvgTAvg_T measures final performance across the complete stream, while FgtTFgt_T measures the average decline of earlier tasks from their best observed performance. Forgetting is marked as “–” for Zero-shot and Static Harness because they make no sequential updates. Stability-HCL and Plasticity-HCL are two configurations of the framework, differing only in the historical-loss tolerance BnB_n. Stability-HCL sets Bn=0B_n=0 and rejects any candidate that causes a currently solved anchor to fail. Plasticity-HCL sets Bn=∞B_n=∞, so historical anchor losses do not block a candidate as long as it satisfies the current-improvement and validity requirements. We evaluate both configurations in ALFWorld and the controlled streams, while Minecraft uses the retention-oriented configuration. Detailed settings are provided in Appendix A. 4.2 Open-World Capability Accumulation We study long-horizon harness evolution in ALFWorld and Minecraft. ALFWorld supports stage-wise evaluation across previously observed task categories, while Minecraft provides a longer interaction curriculum for examining capability accumulation, failure recovery, and skill revision. 4.2.1 ALFWorld We use the text-based ALFWorld environment with a maximum of 50 interaction steps per episode. The continual stream contains six task categories in the order of Pick-and-Place, Look-in-Light, Clean, Heat, Cool, and Two-object manipulation. For each category, 10 training episodes are used for sequential adaptation. After each stage, the harness is evaluated on all observed categories, with final performance reported on the 134 official evaluation episodes. Table 2: Final performance and harness-level forgetting on ALFWorld with Qwen3.5-9B as the frozen foundation model. The best and second-best results in each metric column are marked in bold and underlined, respectively. Method Pick Look Clean Heat Cool Two-object Final Avg. ↑ Avg. Fgt. ↓ Static Harness 95.80 66.70 25.80 26.10 9.50 58.80 47.12 – RAG Baseline 95.80 83.30 41.90 39.10 14.30 58.80 55.56 1.74 MemP (9) 95.80 83.30 48.40 34.80 9.50 47.10 53.15 5.18 MemRL (57) 87.50 66.70 29.00 60.90 23.80 41.20 51.51 5.64 Stability-HCL (Ours) 100.00 83.30 51.60 30.40 28.60 76.50 61.74 2.64 Plasticity-HCL (Ours) 100.00 77.80 41.90 39.10 19.00 100.00 62.98 10.94 We compare HCL with a Static Harness, a RAG baseline, MemP (9), and MemRL (57). For fairness, MemP and MemRL are reimplemented within our framework with unified data processing and action selection, while their algorithms remain unchanged. Table 2 shows that reusing past experience improves the Static Harness but is insufficient for broad continual adaptation. RAG increases the final average from 47.12% to 55.56% and achieves the lowest average forgetting among the adaptive baselines. However, retrieval alone cannot revise reusable procedures or routing rules. MemP and MemRL also improve individual categories, but their performance varies considerably across the stream. These results show that memory-based adaptation supports experience reuse, but does not consistently balance capability acquisition and retention. Both HCL profiles achieve stronger overall performance by evolving the complete harness. Plasticity-HCL obtains the highest final average of 62.98% and solves all Two-object episodes, showing the strongest adaptation to the latest task but also greater forgetting. Stability-HCL reaches a comparable 61.74% and performs best on four of the six categories while substantially reducing average forgetting. Plasticity-HCL therefore favors capability acquisition, whereas Stability-HCL provides a better balance between adaptation and retention. Since the foundation model is frozen and the two profiles differ only in BnB_n, this comparison shows that the Continual Evaluator can explicitly control the stability–plasticity trade-off. 4.2.2 Minecraft We evaluate HCL with Qwen3.6-27B on a 50-task Minecraft curriculum that spans resource collection, crafting, mining, tool use, object placement, smelting, and tasks with multiple dependent operations. After each interaction, environment feedback is stored in Experience Memory and can be used to refine reusable capabilities and execution workflows. Previously validated skill tests are retained as historical anchors. A capability addition or revision is committed only when it improves the current objective and continues to pass all applicable retained tests. For comparison, the Static Harness follows the same curriculum without evolution. MemRL and MemP are reproduced within our harness as memory-management baselines, rather than run from their official repositories. Figure 3: Curriculum progression and execution efficiency. (a) HCL completes all 50 tasks, while the Static Harness plateaus at 15. (b) Cumulative environment actions over the 50-task curriculum: HCL uses 83, versus 88 for MemRL and 91 for MemP; lower is more efficient. Figure 3 shows differences in progression and execution efficiency. The Static Harness follows HCL for 15 tasks and then plateaus; HCL completes all 50, progressing from collection and crafting to persistent assets and coordinated multi-step execution. HCL uses 83 environment actions, compared with 88 for MemRL and 91 for MemP, indicating less redundant execution. Across later multi-step tasks, HCL avoids repeated diagnosis, crafting, and recovery actions, so its lower curve reflects more efficient reuse of accumulated experience while retaining progression across the full curriculum. Reproducing both baselines in our harness keeps the task interface, capability library, and environment stack common while varying memory management. These results show that HCL supports efficient continual adaptation without updating the foundation model. 4.3 Controlled Harness Continual Learning We next evaluate HCL on task sequences. Within each stream, all HCL profiles share the same foundation model, task order, data allocation, editable artifacts, and candidate generator. 4.3.1 Textual Reasoning The textual stream follows the order MuSiQue (39), ProofWriter (38), GSM8K (7), and HotpotQA (49). These tasks cover multi-hop question answering, logical deduction, mathematical reasoning, and knowledge-intensive question answering. For each task, we use 250 examples for adaptation, 50 for validation, and 500 for testing. The foundation model remains frozen throughout the stream. HCL updates only the Task Interface, Experience Memory, Capability Map, and Adaptive Router. Table 3: Final performance after the four-task textual-reasoning stream with DeepSeek-V4-Flash as the frozen foundation model. The Zero-shot baseline evaluates each task independently without sequential harness updates. The best and second-best results in each metric column are marked in bold and underlined, respectively. Method MuSiQue ProofWriter GSM8K HotpotQA Final Avg. ↑ Avg. Fgt. ↓ DeepSeek-V4-Flash Zero-shot 35.00 42.80 49.40 54.80 45.50 – Stability-HCL (Ours) 27.60 73.00 50.40 57.80 52.20 0.00 Plasticity-HCL (Ours) 29.00 77.00 92.00 60.80 64.70 0.07 Table 3 shows how different historical-loss tolerances shift HCL between stronger retention and stronger adaptation. Stability-HCL requires accepted updates to preserve performance on the historical anchor set, reducing average forgetting to zero. This strict constraint substantially limits adaptation, resulting in a final average of 52.20%, compared with 64.70% for Plasticity-HCL. Nevertheless, Stability-HCL still outperforms the 45.50% zero-shot baseline, showing that it can acquire new behavior while fully retaining the previously measured behavior. Plasticity-HCL relaxes the historical-retention requirement and therefore permits more aggressive harness updates. This increases the final average from 52.20% to 64.70%, while introducing only 0.07 average forgetting. With DeepSeek-V4-Flash frozen throughout the stream, these results show that the Continual Evaluator can shift HCL between stronger retention and stronger adaptation solely through the historical-loss tolerance. 4.3.2 Multimodal Perception The multimodal stream follows the order of COCO object detection, COCO image captioning, RefCOCO visual grounding, and VQAv2. Qwen3.6-27B remains frozen throughout the stream. For each task, we use 250 examples for adaptation, 50 for validation, and 500 for testing. We additionally compare with DGG (21), a recent adaptive method for sequential multi-task continual learning whose setting aligns with this controlled multimodal stream. Table 4: Final performance after the four-task multimodal-perception stream with Qwen3.6-27B as the frozen foundation model. The Zero-shot baseline evaluates each task independently without sequential harness updates. The best and second-best results in each metric column are marked in bold and underlined, respectively. Method Detection Caption Grounding VQAv2 Final Avg. ↑ Avg. Fgt. ↓ Qwen3.6-27B Zero-shot 4.27 25.47 43.00 84.87 39.40 – DGG (21) 29.58 29.77 48.96 62.60 42.73 0.26 Plasticity-HCL (Ours) 64.14 37.31 90.60 79.80 67.96 0.81 Stability-HCL (Ours) 65.34 39.41 91.60 79.33 68.92 0.22 Table 4 shows that both HCL profiles substantially outperform Zero-shot and DGG in final average. The largest gains occur in detection and grounding, where the harness must organize spatial information into task-specific outputs. HCL also improves captioning, indicating that its evolving components can support different multimodal objectives and output formats within one task stream. VQAv2 is the only task on which Zero-shot remains stronger, as the frozen model already performs well on direct image–question answering. Nevertheless, both HCL profiles retain substantially higher VQAv2 performance than DGG. Stability-HCL achieves the highest final average of 68.92% and the lowest forgetting of 0.22, while Plasticity-HCL reaches a similar final average of 67.96%. Overall, HCL enables a single frozen model to continually handle heterogeneous multimodal tasks while maintaining a stronger stability–plasticity balance. 4.4 Stability–Plasticity Trade-off Following Eq. (13), we vary only the historical-loss tolerance BnB_n in Dn≤BnD_n≤ B_n, while holding the current-improvement and validity criteria fixed. Specifically, δn _n in Eq. (9) requires an improvement of at least two correct validation cases. Under the validity criterion in Eq. (12), each candidate must achieve at least 90.00% output-format compliance and introduce no syntax, tool-use, or environment violations. These thresholds are chosen heuristically to balance current-task improvement with candidate reliability and remain identical across all settings. The historical loss DnD_n in Eq. (11) counts anchors that are solved by HnH_n but fail under H~n+1 H_n+1. Within each run, we fix Bn≡bB_n≡ b for all candidate decisions and compare b∈0,1,3,∞b∈\0,1,3,∞\. The settings b=0b=0 and b=∞b=∞ correspond to Stability-HCL and Plasticity-HCL, respectively. The intermediate settings b=1b=1 and b=3b=3 allow each candidate to introduce at most one and three newly failed anchors across AnA_n. Each run uses 300 adaptation, 80 validation, and 600 test examples per task, with 80 anchors for every earlier task. A predefined parameter controls the composition of previously successful and failed examples in each anchor set. If either group contains too few examples to meet its target, the remaining slots are filled from the other group. All other experimental conditions remain fixed. Table 5: Performance under different fixed values of b, where Bn≡bB_n≡ b within each run. All other experimental conditions are held constant. The best and second-best results in each metric column are marked in bold and underlined, respectively. Historical-loss tolerance b MuSiQue ProofWriter GSM8K HotpotQA Final Avg. ↑ Avg. Fgt. ↓ b=0b=0 27.83 73.33 84.33 59.50 61.25 0.39 b=1b=1 24.83 77.50 92.33 59.17 63.46 1.22 b=3b=3 26.83 79.83 83.00 58.50 62.04 2.00 b=∞b=∞ 28.33 71.00 82.00 59.17 60.13 3.45 (a) Textual reasoning under different fixed values of b. (b) Multimodal perception under b=0b=0 and b=∞b=∞. Figure 4: Stage-wise forgetting under different fixed historical-loss tolerances. Table 5 shows that increasing b weakens retention. Average forgetting rises from 0.39 at b=0b=0 to 3.45 at b=∞b=∞. Final performance does not increase accordingly: the highest final average of 63.46% occurs at b=1b=1, while the unrestricted setting reaches 60.13%. One possible explanation is that each committed update changes the subsequent evolution trajectory: without historical constraints, locally beneficial updates may overwrite reusable harness contents, weakening both retention and the experience or capabilities available for later tasks. A moderate value of b therefore provides additional flexibility for adaptation without allowing excessive historical loss. The remaining forgetting at b=0b=0 occurs because the constraint covers a finite anchor set, whereas forgetting is evaluated on separate historical test cases. Preserving all anchors currently solved by HnH_n cannot guarantee unchanged behavior on historical cases not represented by AnA_n. Figure 4 complements these final results by showing how forgetting develops across the task sequence. In the textual stream shown in Figure 4(a), smaller values of b generally maintain lower forgetting, with final forgetting increasing consistently from 0.39 at b=0b=0 to 3.45 at b=∞b=∞. In the multimodal stream shown in Figure 4(b), Stability-HCL remains below Plasticity-HCL at every stage after T1T_1 and finishes with forgetting of 0.22 rather than 0.81. This pattern reflects the role of b in the commitment gate: smaller values reject more candidates that improve the current task at the expense of historical behavior, thereby constraining the harness to more retention-preserving update trajectories. Larger values permit greater adaptation flexibility but expose earlier tasks to more regression. Together, the two trajectories illustrate that a stricter historical-loss tolerance suppresses forgetting throughout harness evolution. 4.5 Ablation Study We conduct component ablations on the controlled multimodal stream using Qwen3.5-4B with the balanced HCL configuration. The stream follows COCO object detection → COCO image captioning → RefCOCO visual grounding → VQAv2, with 250 adaptation, 50 validation, and 500 test examples for each task. Starting from Full HCL, we disable updates to one harness component at a time while keeping the other three components adaptive. All variants use the same foundation model, task order, evaluation criteria, and update schedule. Table 6 summarizes the resulting component-wise ablation results. Table 6: Component ablation on the controlled multimodal stream. I, M, C, and R denote the Task Interface, Experience Memory, Capability Map, and Adaptive Router. A check mark indicates that the component is updated, while a cross indicates that its update is disabled. The best and second-best results in each metric column are marked in bold and underlined, respectively. Component I M C R Final Avg. ↑ Avg. Fgt. ↓ Zero-shot – – – – 34.84 – w/o Interface update × ✓ ✓ ✓ 62.37 0.11 w/o Memory update ✓ × ✓ ✓ 62.28 0.83 w/o Capability update ✓ ✓ × ✓ 63.12 0.06 w/o Router update ✓ ✓ ✓ × 62.77 0.14 Full HCL ✓ ✓ ✓ ✓ 63.41 0.45 Full HCL achieves the highest final average of 63.41%, showing that the four components contribute complementarily to continual adaptation. Disabling Experience Memory or the Task Interface produces the largest decrease in final performance. In particular, removing Memory updates also increases forgetting to 0.83, indicating that evolving memory supports both the acquisition and retention of behavior. Disabling Capability Map or Adaptive Router updates causes smaller but consistent performance reductions. The small effect of Capability updates may reflect that this multimodal stream relies less on reusable executable procedures than the Minecraft curriculum. Several ablations show lower forgetting than Full HCL because restricting the editable components also limits the extent of adaptation. Lower forgetting alone therefore does not necessarily indicate a better evolving harness and should be considered together with final performance. Exact interventions and full per-task results are reported in Appendix B. 5 Conclusion We formulate Harness Continual Learning (HCL) as a new continual learning paradigm in which the agent harness, rather than model parameters, evolves through sequential experience. Our framework treats the mutable harness components as a unified evolving state and separates candidate generation from evaluation and commitment, making historical retention an explicit condition for deployment. Experiments show that harness evolution can accumulate capabilities and recover from failures, while also causing measurable forgetting under a frozen foundation model. Explicitly controlling historical loss enables HCL to balance stability and plasticity. These findings demonstrate the potential of continual learning at the harness level, while highlighting unresolved challenges in efficient retention evaluation, harness-content consolidation, and evaluation over longer interaction streams. We hope HCL provides a foundation for addressing these challenges and encourages broader research on reliable agent continual learning. References Abbes et al. (2026) I. Abbes, G. Subbaraj, M. Riemer, N. Islah, T. Tabaru, H. Kingetsu, S. Chandar, and I. Rish Revisiting replay and gradient alignment for continual pre-training of large language models. In Proceedings of the 4th Conference on Lifelong Learning Agents, p. 465–486. Cited by: §2.2. Abuzakuk et al. (2026) S. Abuzakuk, A. Kermarrec, R. Sharma, R. M. Veski, and M. de Vos Optimizing Agentic Workflows using Meta-tools. External Links: 2601.22037 Cited by: §2.1. Bellitto et al. (2024) G. Bellitto, F. P. Salanitri, M. Pennisi, M. Boschini, L. Bonicelli, A. Porrello, S. Calderara, S. Palazzo, and C. Spampinato Saliency-driven Experience Replay for Continual Learning. In Advances in Neural Information Processing Systems, Vol. 37. Cited by: §2.2. Chen et al. (2025) J. Chen, J. Ye, and G. Wang From Standalone LLMs to Integrated Intelligence: A Survey of Compound AI Systems. External Links: 2506.04565 Cited by: §1. Chen et al. (2026a) M. Chen, J. Wang, Z. Liu, Y. Wang, and Q. Wang From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws. External Links: 2606.06324 Cited by: §2.1. Chen et al. (2026b) T. Chen, S. Lu, K. Zhao, W. Meng, H. Teng, T. Li, C. Li, X. Liu, J. Liang, Z. Zhang, Y. Xie, H. Qu, K. Shao, and J. Luan HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry. External Links: 2606.14249 Cited by: §2.1. Cobbe et al. (2021) K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman Training Verifiers to Solve Math Word Problems. External Links: 2110.14168 Cited by: §4.3.1. Delange et al. (2022) M. Delange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuytelaars A Continual Learning Survey: Defying Forgetting in Classification Tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (7), p. 3366–3385. External Links: Document Cited by: §1, §2.2, §3.4. Fang et al. (2026) R. Fang, Y. Liang, X. Wang, J. Wu, S. Qiao, P. Xie, F. Huang, H. Chen, and N. Zhang MemP: Exploring Agent Procedural Memory. In Findings of the Association for Computational Linguistics: ACL 2026, p. 17490–17502. External Links: Document Cited by: §4.2.1, Table 2. Gu (2026) S. Gu From Model Scaling to System Scaling: Scaling the Harness in Agentic AI. External Links: 2605.26112 Cited by: §2.1. He et al. (2026) C. He, X. Zhou, D. Wang, H. Xu, W. Liu, and C. Miao Harness Engineering for Language Agents: The Harness Layer as Control, Agency, and Runtime. Preprints. External Links: Document Cited by: §2.1, §3.2. Jimenez et al. (2024) C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan SWE-bench: Can Language Models Resolve Real-World GitHub Issues?. Cited by: §1. Kang et al. (2026) B. Kang, J. Gu, T. Feng, Q. Fan, Y. Shi, L. Wang, W. Li, and Y. Gao Don’t forget why you started: tackling dual forgetting in vision-language continual learning. In Forty-third International Conference on Machine Learning, Cited by: §2.2. Kang et al. (2025) B. Kang, L. Wang, Z. Wu, T. Feng, Y. Li, Y. Gao, and W. Li Dynamic multi-layer null space projection for vision-language continual learning. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 2077–2086. Cited by: §3.4. Karpas et al. (2022) E. Karpas, O. Abend, Y. Belinkov, B. Lenz, O. Lieber, N. Ratner, Y. Shoham, H. Bata, Y. Levine, K. Leyton-Brown, D. Muhlgay, N. Rozen, E. Schwartz, G. Shachaf, S. Shalev-Shwartz, A. Shashua, and M. Tenenholtz MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning. External Links: 2205.00445 Cited by: §2.1. Khattab et al. (2024) O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. Cited by: §2.1. Kim et al. (2025) J. Kim, Y. Kim, and J. Sohn Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective. In Proceedings of the 42nd International Conference on Machine Learning, Cited by: §2.2. Kirkpatrick et al. (2017) J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114 (13), p. 3521–3526. Cited by: §2.2. Lewandowski et al. (2025) A. Lewandowski, M. Bortkiewicz, S. Kumar, A. György, D. Schuurmans, M. Ostaszewski, and M. C. Machado Learning Continually by Spectral Regularization. In The Thirteenth International Conference on Learning Representations, Cited by: §2.2. Li et al. (2026a) J. Li, X. Xiao, Y. Zhang, C. Liu, L. Zhao, X. Liao, Y. Ji, J. Wang, J. Gu, Y. Ge, W. Xu, X. Fang, X. Xu, T. Zhao, Y. Kim, T. Wang, J. Hamm, S. Krishnaswamy, J. Huan, and C. Reddy Agent Harness Engineering: A Survey. Note: Withdrawn TMLR submission Cited by: §1, §2.1, §3.2. Li et al. (2026b) S. Li, M. Gao, T. Su, X. Zhang, and Z. Wang Multimodal continual instruction tuning with dynamic gradient guidance. External Links: 2511.15164 Cited by: §4.3.2, Table 4. Liang et al. (2025) Y. Liang, J. Chen, and W. Li Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models. Advances in Neural Information Processing Systems 38, p. 76577–76607. External Links: Document Cited by: §2.2. Lin et al. (2026) M. Lin, J. Wu, Z. Wang, Z. Shi, Y. Sang, B. He, Z. Liu, T. Wei, Z. Wu, Z. Zhang, D. Wang, X. Zhang, B. Dumoulin, C. Xie, Y. Zhou, S. Wang, and H. Lu Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents. External Links: 2605.30621 Cited by: §2.1. Liu et al. (2026a) Y. Liu, T. Nguyen, and F. D. Salim CP-moe: consistency-preserving mixture-of-experts for continual learning. arXiv preprint arXiv:2605.20247. Cited by: §2.2. Liu et al. (2026b) Z. Liu, Z. Shi, Y. Sang, B. He, M. Lin, T. Wei, D. Wang, B. Dumoulin, W. Jin, and H. Lu Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams. External Links: 2606.01770 Cited by: §2.1. Liu et al. (2026c) Z. Liu, B. Kang, W. Li, H. Yuan, Y. Yang, W. Li, Y. Zhu, T. Feng, and J. Luo Branch, or layer? zeroth-order optimization for continual learning of vision-language models. In Proceedings of the AAAI Conference on Artificial Intelligence, p. 24026–24034. Cited by: §3.4. Lopez-Paz and Ranzato (2017) D. Lopez-Paz and M. Ranzato Gradient Episodic Memory for Continual Learning. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Cited by: §2.2. Lu et al. (2024) A. Lu, T. Feng, H. Yuan, X. Song, and Y. Sun Revisiting Neural Networks for Continual Learning: An Architectural Perspective. External Links: 2404.14829, Document Cited by: §2.2. Meng et al. (2026) Q. Meng, Y. Wang, L. Chen, Y. Li, W. Wu, W. Jiang, Q. Wang, C. Lu, Y. Gao, Y. Wu, and Y. Hu Agent Harness for Large Language Model Agents: A Survey. Preprints. External Links: Document Cited by: §1, §2.1. Packer et al. (2023) C. Packer, S. Wooders, K. Lin, V. Fang, S. G. Patil, I. Stoica, and J. E. Gonzalez MemGPT: Towards LLMs as Operating Systems. External Links: 2310.08560 Cited by: §2.1, §3.2.2. Park et al. (2023) J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein Generative Agents: Interactive Simulacra of Human Behavior. External Links: Document Cited by: §3.2.2. Schick et al. (2023) T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom Toolformer: Language Models Can Teach Themselves to Use Tools. Vol. 36. External Links: Document Cited by: §2.1, §2.1. Shang et al. (2025) J. Shang, S. Shao, T. Tong, F. Yang, Y. Chen, Y. Jiao, J. Liu, and Y. Gao Divide and Orthogonalize: Efficient Continual Learning with Local Model Space Projection. In Proceedings of the Forty-First Conference on Uncertainty in Artificial Intelligence, Cited by: §2.2. Shen et al. (2023) Y. Shen, K. Song, X. Tan, D. Li, W. Lu, and Y. Zhuang HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face. Vol. 36. External Links: Document Cited by: §2.1. Shi et al. (2025) H. Shi, Z. Xu, H. Wang, W. Qin, W. Wang, Y. Wang, Z. Wang, S. Ebrahimi, and H. Wang Continual Learning of Large Language Models: A Comprehensive Survey. Vol. 58. External Links: Document Cited by: §1. Shinn et al. (2023) N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao Reflexion: Language Agents with Verbal Reinforcement Learning. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36, p. 8634–8652. Cited by: §2.1, §2.1, §3.2.2. Shridhar et al. (2021) M. Shridhar, X. Yuan, M. Côté, Y. Bisk, A. Trischler, and M. Hausknecht ALFWorld: Aligning Text and Embodied Environments for Interactive Learning. International Conference on Learning Representations. External Links: 2010.03768 Cited by: §4. Tafjord et al. (2021) O. Tafjord, B. Dalvi, and P. Clark ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, C. Zong, F. Xia, W. Li, and R. Navigli (Eds.), Online, p. 3621–3634. Cited by: §4.3.1. Trivedi et al. (2022) H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal MuSiQue: Multihop Questions via Single-hop Question Composition. Transactions of the Association for Computational Linguistics 10, p. 539–554. Cited by: §4.3.1. Urettini and Carta (2025) E. Urettini and A. Carta Online curvature-aware replay: leveraging second-order information for online continual learning. In Proceedings of the 42nd International Conference on Machine Learning, p. 60590–60609. Cited by: §2.2. Wang et al. (2024a) G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar Voyager: An Open-Ended Embodied Agent with Large Language Models. Transactions on Machine Learning Research. Cited by: §2.1, §2.1, §4. Wang et al. (2024b) L. Wang, X. Zhang, H. Su, and J. Zhu A comprehensive survey of continual learning: Theory, method and application. IEEE transactions on pattern analysis and machine intelligence 46 (8), p. 5362–5383. Cited by: §1, §2.2, §3.4. Wang et al. (2025a) X. Wang, S. Li, J. Zhang, and S. Chen Cut out and replay: a simple yet versatile strategy for multi-label online continual learning. In Proceedings of the 42nd International Conference on Machine Learning, p. 63530–63548. Cited by: §2.2. Wang et al. (2022a) Z. Wang, Z. Zhang, S. Ebrahimi, R. Sun, H. Zhang, C. Lee, X. Ren, G. Su, V. Perot, J. Dy, and T. Pfister DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning. Cited by: §2.2. Wang et al. (2022b) Z. Wang, Z. Zhang, C. Lee, H. Zhang, R. Sun, X. Ren, G. Su, V. Perot, J. Dy, and T. Pfister Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 139–149. Cited by: §2.2. Wang et al. (2025b) Z. Z. Wang, J. Mao, D. Fried, and G. Neubig Agent Workflow Memory. Proceedings of Machine Learning Research, Vol. 267, PMLR. Cited by: §3.2.2. Xie et al. (2024) T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Lei, Y. Liu, Y. Xu, S. Zhou, S. Savarese, C. Xiong, V. Zhong, and T. Yu OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. Vol. 37. Cited by: §1. Xu et al. (2025) F. F. Xu, Y. Song, B. Li, Y. Tang, K. Jain, M. Bao, Z. Z. Wang, X. Zhou, Z. Guo, M. Cao, M. Yang, H. Y. Lu, A. Martin, Z. Su, L. Maben, R. Mehta, W. Chi, L. Jang, Y. Xie, S. Zhou, and G. Neubig TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks. Vol. 38. Cited by: §1. Yang et al. (2018) Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. Cohen, R. Salakhutdinov, and C. D. Manning HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii (Eds.), Brussels, Belgium, p. 2369–2380. Cited by: §4.3.1. Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao ReAct: Synergizing Reasoning and Acting in Language Models. Cited by: §2.1. Yao et al. (2026) Y. Yao, X. Tan, C. Liu, Y. Li, Z. Wang, W. Yu, Z. Tan, Y. Tian, G. Zhao, L. Sun, X. Zhang, and T. Yang Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows. External Links: 2605.27922 Cited by: §2.1. Yue et al. (2025) W. Yue, B. Liu, and P. Stone T-dgr: a trajectory-based deep generative replay method for continual learning in decision making. In Proceedings of the 3rd Conference on Lifelong Learning Agents, p. 481–497. Cited by: §2.2. Zhang et al. (2026a) H. Zhang, S. Zhang, K. Li, C. Zhang, Y. Chen, Y. Zhang, L. Bai, and S. Hu Self-Harness: Harnesses That Improve Themselves. External Links: 2606.09498 Cited by: §2.1. Zhang et al. (2026b) H. Zhang, Z. Ji, J. Liu, Y. Pang, and J. Han Multi-stage knowledge integration of vision-language models for continual learning. IEEE Transactions on Image Processing 35, p. 615–628. External Links: 2411.06764, Document Cited by: §2.2. Zhang et al. (2025) J. Zhang, J. Xiang, Z. Yu, F. Teng, X. Chen, J. Chen, M. Zhuge, X. Cheng, S. Hong, J. Wang, B. Zheng, B. Liu, Y. Luo, and C. Wu AFlow: Automating Agentic Workflow Generation. In International Conference on Learning Representations, Cited by: §1. Zhang et al. (2024) S. Zhang, J. Zhang, J. Liu, L. Song, C. Wang, R. Krishna, and Q. Wu Offline Training of Language Model Agents with Functions as Learnable Weights. In Proceedings of the 41st International Conference on Machine Learning, R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp (Eds.), Proceedings of Machine Learning Research, Vol. 235, p. 60315–60335. Cited by: §1. Zhang et al. (2026c) S. Zhang, J. Wang, R. Zhou, J. Liao, Y. Feng, Z. Li, Y. Zheng, W. Zhang, Y. Wen, Z. Li, et al. Memrl: self-evolving agents via runtime reinforcement learning on episodic memory. arXiv preprint arXiv:2601.03192. Cited by: §4.2.1, Table 2. Zhang et al. (2026d) Z. Zhang, K. Shi, S. Huang, A. Nie, Y. Zeng, Y. Zhao, Z. Fang, Q. Su, H. Qiu, W. Yang, Q. Ren, S. Zou, W. Huang, L. Chen, Z. Chen, and F. Zhao SkillFlow: Benchmarking Lifelong Skill Discovery and Evolution for Autonomous Agents. External Links: 2604.17308 Cited by: §2.1. Zhong et al. (2026) S. Zhong, Y. Lu, J. Ning, Y. Wan, L. Feng, Y. Ao, L. F. R. Ribeiro, M. Dreyer, S. Ammirati, and C. Xiong SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Tasks. External Links: 2604.20087 Cited by: §2.1. Zhong et al. (2024) W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang MemoryBank: Enhancing Large Language Models with Long-Term Memory. Proceedings of the AAAI Conference on Artificial Intelligence 38, p. 19724–19731. Cited by: §3.2.2. Zhou et al. (2026) C. Zhou, H. Chai, W. Chen, Z. Guo, R. Shan, Y. Song, T. Xu, Y. Yang, A. Yu, W. Zhang, C. Zheng, J. Zhu, Z. Zheng, Z. Zhang, X. Lou, C. Zhang, Z. Fu, J. Wang, W. Liu, J. Lin, and W. Zhang Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering. External Links: 2604.08224 Cited by: §2.1. Zhou et al. (2023) Y. Zhou, A. I. Muresanu, Z. Han, K. Paster, S. Pitis, H. Chan, and J. Ba Large Language Models Are Human-Level Prompt Engineers. Cited by: §2.1. Supplementary Material The supplementary material provides implementation details, full ablation results, and task-specific anchor criteria. Appendix A Implementation and Experimental Settings A.1 Harness and Evaluator Boundaries Table 7 summarizes the access and update boundaries of the harness components and the evaluation-only anchor set. Table 7: Access and update boundaries of the deployed harness and anchor set. Artifact Execution and candidate-generation access Update boundary Task Interface InI_n Constructs ni_n; the Optimizer may revise prompts, templates, and parsing or normalization rules. Changes enter HnH_n only with a committed candidate. Raw and Abstract Memory Mnraw,MnabsM_n^raw,M_n^abs Supplies records and guidance to the Router; the Optimizer may add raw records or revise abstract entries. Changes enter HnH_n only with a committed candidate. Capability Map CnC_n Supplies capabilities to the Router; the Optimizer may add or revise internal skills. Changes enter HnH_n only with a committed candidate. Adaptive Router RnR_n Constructs nz_n; the Optimizer may revise routing prompts, selection criteria, or workflow templates. Changes enter HnH_n only with a committed candidate. Anchor Set AnA_n Used only by the Evaluator; unavailable to execution and candidate generation. Updated at the end of each task and then fixed during candidate generation and evaluation for the next task. Thus, HnH_n contains only persistent execution-time contents; ni_n, nz_n, and ny_n are transient, and AnA_n remains evaluation-only. Component-level alternatives are evaluated sequentially, and only committed changes enter the deployed harness. A.2 Experimental Settings Table 8 summarizes the experimental settings. Table 8: Experimental settings. Counts are per task or category unless a stream total is stated. Experiment Stream and frozen model Adaptation/evaluator data Final reporting ALFWorld main Six categories in the order Pick-and-Place, Look-in-Light, Clean, Heat, Cool, and Two-object; frozen Qwen3.5-9B. 10 training episodes per category, with at most 50 interaction steps per episode. Evaluation on all observed categories after each stage. Final success on 134 official evaluation episodes, category macro-average, and average forgetting over the first five categories. Minecraft main 50 tasks covering collection, crafting, mining, tool use, placement, smelting, and multi-step dependencies; frozen Qwen3.6-27B. Sequential environment feedback, with retained skill tests as historical anchors. Cumulative task completion, recovery events, and validated skill changes. Completed tasks are not systematically replayed after every update. Textual main MuSiQue → ProofWriter → GSM8K → HotpotQA; frozen DeepSeek-V4-Flash. 250 adaptation and 50 validation examples per task. 500 test examples per task. Final task scores, average performance, and forgetting. Multimodal main COCO detection → COCO captioning → RefCOCO grounding → VQAv2; frozen Qwen3.6-27B. 250 adaptation and 50 validation examples per task. 500 test examples per task. Final task scores, average performance, and forgetting. Textual budget sweep The same textual order; frozen DeepSeek-V4-Flash. 300 adaptation and 80 validation examples per task, with 80 anchors retained for each earlier task. 600 test examples per task. Each profile receives 40 proposals, with ten at each task stage. Across all experiments, validation cases and historical anchors are restricted to the Evaluator, and final test cases are used only for reporting. The main Stability-HCL and Plasticity-HCL profiles use Bn=0B_n=0 and Bn=∞B_n=∞, respectively. A main-profile candidate must improve by at least one validation case for discrete metrics or strictly improve the designated continuous score, without introducing an invalid outcome. Minecraft applies Bn=0B_n=0 to retained skill tests and therefore evaluates skill-level rather than full task-level retention. The independent textual sweep uses 40 proposal opportunities, requires two additional correct predictions among 80 validation cases and at least 90% format compliance, and varies only Bn≡bB_n≡ b for b∈0,1,3,∞b∈\0,1,3,∞\. Appendix B Component Ablation Details All ablation variants use frozen Qwen3.5-4B and share the task order, data allocation, evaluation criteria, and update schedule in Section 4.5. Table 9 specifies their permitted persistent updates. A disabled component remains available during execution but retains its initialized contents throughout the stream. Zero-shot evaluates the frozen model without the structured HCL harness or sequential updates. B.1 Ablation Configurations Table 9: Update scope of the component-ablation variants. A ✓ permits persistent updates, while × keeps the component fixed. Method I M C R Fixed contents Zero-shot – – – – No structured HCL harness or persistent updates. Full HCL ✓ ✓ ✓ ✓ None. w/o Interface update × ✓ ✓ ✓ Prompts, templates, parsing, and normalization rules. w/o Memory update ✓ × ✓ ✓ Raw and Abstract Memory entries. w/o Capability update ✓ ✓ × ✓ Reusable skills. w/o Router update ✓ ✓ ✓ × Routing prompts, selection criteria, and workflow templates. Because reusable skills may be distilled from Abstract Memory, disabling Memory updates also removes this source of new skills. This variant therefore measures both direct memory adaptation and its downstream effects. B.2 Full Per-Task Results Table 10: Full component-ablation results on the controlled multimodal stream. “Committed” counts candidate updates entering the persistent harness. Method Detection Caption Grounding VQAv2 Final Avg. ↑ Avg. Fgt. ↓ Committed Zero-shot 35.11 22.98 0.00 81.27 34.84 – – Full HCL 53.07 36.09 87.60 76.87 63.41 0.45 18 w/o Interface update 53.45 33.56 87.80 74.67 62.37 0.11 24 w/o Memory update 55.50 28.95 88.00 76.67 62.28 0.83 46 w/o Capability update 55.11 34.16 86.40 76.80 63.12 0.06 16 w/o Router update 53.59 36.68 87.40 73.40 62.77 0.14 4 Interface updates contribute most visibly to Caption and VQAv2, while disabling Memory updates primarily degrades Caption. Fixing the Router causes its largest decline on VQAv2. Capability updates have a smaller effect in this multimodal stream, whose tasks rely less on long-horizon executable skills than the Minecraft curriculum. Commit counts are trajectory-specific: each commitment changes the deployed harness and may affect subsequent feedback and proposals. Because variants do not necessarily share a proposal sequence, these counts are not directly comparable acceptance rates or measures of update efficiency. Appendix C Anchor Success Criteria Tables 11–13 define the fixed task-specific criterion q(H,a)q(H,a) in Eq. (10), applied to the same raw input under HnH_n and H~n+1 H_n+1. C.1 Textual Reasoning Table 11: Anchor success criteria for textual reasoning. Task q(H,a)=1q(H,a)=1 when MuSiQue / HotpotQA The normalized predicted short answer exactly matches an accepted reference answer. ProofWriter The parsed entailment label exactly matches the gold label and the output schema is valid. GSM8K The parsed final numeric value equals the gold value after comma and unit normalization. C.2 Multimodal Perception Table 12: Anchor success criteria for multimodal perception. Task q(H,a)=1q(H,a)=1 when COCO detection For the queried annotated instance, the predicted category is correct, the matched bounding box has IoU ≥0.5≥ 0.5, and the box schema is valid. COCO captioning Sentence-level CIDEr against the reference captions is at least 0.5 on the normalized [0,1][0,1] scale, and the caption schema is valid. RefCOCO grounding The predicted box is valid and has IoU ≥0.5≥ 0.5 with the referred-object box. VQAv2 The standard VQA consensus score is 1.0 after answer normalization. C.3 Interactive Environments Table 13: Anchor success criteria for interactive environments. Environment q(H,a)=1q(H,a)=1 when ALFWorld The environment’s specified goal predicate is true within the 50-step limit under a valid action sequence. Minecraft The retained test for the corresponding skill reaches its predefined inventory or world-state predicate through a valid action sequence. Historical-loss counting. Eq. (11) counts an anchor only when it succeeds under HnH_n but fails under H~n+1 H_n+1. For example, a RefCOCO IoU drop from 0.68 to 0.41 contributes one loss by crossing the 0.5 threshold; improvement on another anchor does not offset it.