Paper deep dive

Teach and Grow: An Agent-Centered Architecture for General Robot Learning

Chang Nie, Zhe Liu, Hesheng Wang

Year: 2026Venue: arXiv preprintArea: cs.ROType: PreprintEmbeddings: 94

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 8/19/2026, 4:27:02 AM

Summary

The paper introduces Teach-and-Grow Learning (TGL), an agent-centered architecture for general robot learning that avoids the 'retraining tax' of end-to-end models. TGL uses few-shot demonstrations to create reusable 'Skill Blocks' (closed-loop behaviors) stored in a Skill Library, while a structured Experience Memory records successes, failures, and repairs. An AI agent composes these blocks, executes them, and adapts based on physical feedback, allowing new tasks to be acquired without retraining the underlying policy. The authors propose a scaling-law hypothesis relating effective reusable experience to future-task error and teaching demand.

Entities (9)

Teach-and-Grow Learning · method · 98%Skill Block · concept · 96%Experience Memory · component · 95%Skill Library · component · 95%Retraining Tax · problem · 92%LIBERO · benchmark · 90%Vision-Language-Action · model-type · 88%RT-1 · model · 85%RT-2 · model · 85%

Relation Signals (8)

Teach-and-Grow Learning → storesdatain → Skill Library

confidence 95% · A Skill Library stores executable behavior

Teach-and-Grow Learning → storesdatain → Experience Memory

confidence 95% · structured Experience Memory carries forward success, failure, and repair

Teach-and-Grow Learning → uses → Skill Block

confidence 95% · a multimodal agent turns a few successful demonstrations into reusable Skill Blocks

Teach-and-Grow Learning → solves → Retraining Tax

confidence 92% · This recurring burden is the retraining tax... We present Teach-and-Grow Learning (TGL)

Teach-and-Grow Learning → evaluatedon → LIBERO

confidence 90% · Our LIBERO evaluation attains state-of-the-art performance

Vision-Language-Action → suffersfrom → Retraining Tax

confidence 88% · End-to-end vision-language-action (VLA)... reliability is bounded by validated physical coverage... correcting the failure requires new robot data... This recurring burden is the retraining tax.

RT-1 → istypeof → Vision-Language-Action

confidence 85% · A VLA or world-action model... RT-1 and RT-2 established the promise

RT-2 → istypeof → Vision-Language-Action

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:End-to-end vision-language-action (VLA) and world-action models offer an elegant route to general-purpose robotics, but their reliability is bounded by validated physical coverage. When an unfamiliar object, sensor, embodiment, or contact falls outside that coverage and no validated fallback exists, correcting the failure requires new robot data, a policy update, and regression testing. This recurring burden is the retraining tax. Unlike text, embodied data must often be created by operating machines. We present Teach-and-Grow Learning (TGL), an agent-centered architecture for general robot learning. In its general form, a multimodal agent turns a few successful demonstrations into reusable Skill Blocks: closed-loop behaviors for meaningful subgoals. In a new scene, the agent grounds and composes these blocks, selects learned or geometric tools, observes the physical outcome, and revises the route when execution departs from intent. A Skill Library stores executable behavior, while structured Experience Memory carries forward success, failure, and repair. New tasks are acquired without task-specific policy retraining. Our LIBERO evaluation attains state-of-the-art performance; controlled studies expose skill induction, persistent reuse, and agent-directed adaptation. Finally, we propose the Teach-and-Grow scaling-law hypothesis: if X denotes effective reusable experience, future-task error and teaching demand should approach irreducible floors as power laws in X. The architecture therefore treats deployment as a period of continued learning, in which one task can make the next easier.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

90,830 characters extracted from source content.

Expand or collapse full text

Teach and Grow: An Agent-Centered Architecture for General Robot Learning Chang Nie Zhe Liu Hesheng Wang Thanks: Chang Nie, Zhe Liu, and Hesheng Wang are with the School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, and the Shanghai Key Laboratory of Navigation and Location Based Services, Shanghai 200240, China. Corresponding author: Hesheng Wang (e-mail: wanghesheng@sjtu.edu.cn). Abstract End-to-end vision-language-action (VLA) and world-action models offer an elegant route to general-purpose robotics, but their reliability is bounded by validated physical coverage. When an unfamiliar object, sensor, embodiment, or contact falls outside that coverage and no validated fallback exists, correcting the failure requires new robot data, a policy update, and regression testing. This recurring burden is the retraining tax. Unlike text, embodied data must often be created by operating machines. We present Teach-and-Grow Learning (TGL), an agent-centered architecture for general robot learning. In its general form, a multimodal agent turns a few successful demonstrations into reusable Skill Blocks: closed-loop behaviors for meaningful subgoals. In a new scene, the agent grounds and composes these blocks, selects learned or geometric tools, observes the physical outcome, and revises the route when execution departs from intent. A Skill Library stores executable behavior, while structured Experience Memory carries forward success, failure, and repair. New tasks are acquired without task-specific policy retraining. Our LIBERO evaluation attains state-of-the-art performance; controlled studies expose skill induction, persistent reuse, and agent-directed adaptation. Finally, we propose the Teach-and-Grow scaling-law hypothesis: if X denotes effective reusable experience, future-task error and teaching demand should approach irreducible floors as power laws in X. The architecture therefore treats deployment as a period of continued learning, in which one task can make the next easier. Index Terms: Agentic robotics, few-shot robot learning, vision-language-action models, lifelong learning, skill composition, embodied intelligence. I Introduction End-to-end robot foundation models now define the dominant vision of general-purpose robotics. A VLA or world-action model maps observations and language directly to actions, replacing a collection of task-specific systems with one scalable policy. RT-1 and RT-2 established the promise of this route [6, 48]; later generalist policies and world-action models broadened its reach [25, 15, 5, 40]. Their success has encouraged a simple extrapolation: keep enlarging the model and the dataset, and general robot intelligence will eventually emerge. Robotics lacks the pre-existing data substrate that made language-model scaling possible. Text and code existed before modern language models; embodiment-aligned interaction usually does not. When a new task, sensor modality, camera, gripper, or robot body falls outside validated coverage, the system needs new data that align observation, action, contact, and physical outcome. Producing those data requires operating robots or simulators. Updating a monolithic policy then requires another round of optimization and regression testing. Web-scale semantics may identify the intended object or instruction, but they do not by themselves supply the geometry, dynamics, or contact behavior needed for reliable execution [26, 14, 36, 1, 8]. A failure may concern one object or scene, but repairing it in a monolithic policy forces the system through new data collection, model updating, and regression testing across previously supported behavior. This mismatch between a local gap and a global update is the retraining tax. As deployment expands across tasks, sensors, and embodiments, repeated global updates become a practical bottleneck. General robotics therefore needs an architecture that can add and validate local competence without rewriting unrelated behavior. Fig. 1: Two architectures facing the same OOD pick-and-place task. Left: an end-to-end VLA/WAM is brittle to the unfamiliar plush toy, returns to a retraining loop, and remains closed to incremental growth. Right: TGL learns from few-shot teaching, reuses Skill Blocks through an AI agent, adapts from physical feedback, and carries the acquired capability into later deployment. Fig. 2: Teach-and-Grow for one unfamiliar task. Teaching reveals shared state changes and new Skill Blocks. The AI Agent composes a route, runs one block, and checks its effect after the robot acts. Verified capabilities enter the Skill Library; outcomes, failures, and repairs enter Experience Memory. The complete Grow state returns to composition on the next task. Modern coding agents suggest what this organization can look like. Systems such as Codex and Claude Code inspect a changing workspace, form a plan, call tools, read the result, and revise their strategy until the goal is complete [27, 4]. Robot autonomy has the same essential shape: observe the world, decide what evidence or capability is needed, act, inspect the physical consequence, and continue. We study whether a general AI agent can supply this organizing intelligence for a robot. Semantic task understanding alone does not supply metric geometry, collision checking, contact handling, or continuous control. A robot agent therefore needs robot-native tools and short feedback cycles: after each meaningful action, it sees what changed and revises the remaining route. This closes the gap between a plausible plan and grounded physical execution. ASPIRE and related systems show that zero-shot agent exploration can assemble and repair robot behavior without updating a task-specific policy [22, 43]. On a physical robot, however, learning mainly through trial requires many sequential decisions and can trigger unsafe exploratory actions. Teach-and-Grow therefore uses sparse successful teaching to seed task structure and reserves exploration for the remaining gaps. We therefore propose Teach-and-Grow Learning (TGL). A few successful demonstrations give the agent evidence about what the task means and how success is organized. By analyzing them together, the agent identifies semantic subgoals and turns the shared strategy into reusable Skill Blocks. A block is not a trajectory fragment to replay. It is a closed-loop behavior whose objects, geometry, and control targets are re-instantiated from the current scene. Demonstration is evidence; the executable skill is the abstraction built from it. The idea behind Teach-and-Grow is also inspired by the classical Chinese principle (jiào xué xiāng zhǎng), which we translate as teaching and learning promote one another. Teaching gives the robot a productive first structure; the robot’s attempts then expose missing knowledge, sharpen the next intervention, and gradually reduce how much must be taught. The teacher does not merely deposit trajectories: human guidance and robot experience improve the curriculum together. At task time, the agent retrieves and composes blocks, gathers the evidence each block needs, and executes one meaningful stage before looking again. The physical result determines what follows. The agent may continue, repair the remaining composition, or explore a missing piece; it can also request targeted teaching when another demonstration would introduce a new behavior or shorten acquisition. Teaching gives exploration a productive starting point, and exploration lets the system grow beyond the teacher’s demonstrations. Execution does not disappear when the episode ends. Validated behaviors enter a persistent Skill Library. A structured textual Experience Memory records the context that made them succeed or fail and the repair that changed the outcome. Together, these stores give the robot an explicit, editable history of what it can do and what it has learned about doing it. New competence can therefore be added locally, while earlier skills remain available as separate objects for future composition. The architecture is deliberately asymmetric. The Agent is responsible for questions that can change the task route: which subgoal matters now, what evidence is missing, and whether the last action achieved its intended effect. Robot-native executors remain responsible for metric motion, contact, and control. This separation prevents a language-level plan from being treated as motor authority, while still allowing physical evidence to alter the plan. It also localizes improvement. A better grasp detector can replace one component inside a block; a new recovery can be attached to the effect that failed; and a useful block can be admitted without changing unrelated skills. Growth is therefore not the accumulation of ever longer plans. It is the gradual conversion of uncertain tasks into verified, addressable capabilities. Figure 1 places the two routes in the same scene. The difference is architectural: the end-to-end route asks model weights to absorb each new piece of the physical world, whereas TGL turns sparse evidence into executable knowledge that can be reused and extended. It is not an adapter around a fixed policy. It is a different organization of robot intelligence, with learned policies, planners, perception models, and controllers serving as capabilities under an agent that decides what the task now requires. TGL separates the acquisition of unfamiliar behavior from the fast execution of familiar behavior. The Agent handles unfamiliar conditions because it can inspect outcomes, change the remaining plan, and write validated behavior to persistent state. Once a behavior is stable, a VLA, WAM, or other fast policy can execute that block directly or learn from its verified trajectories. The Agent expands the boundary of competence; the fast policy provides low-latency execution within familiar conditions. We are not aware of prior work that combines this exact cycle of few-shot teaching, closed-loop composition, and persistent skill growth without task-specific policy retraining. This lifecycle also changes the quantity that should scale. Neural scaling laws relate loss to resources such as parameters, data, and compute [13, 9]. Let X>0X>0 denote the robot’s effective reusable experience: accumulated interaction that has survived validation and can still be retrieved, grounded, and composed in a new scene. We propose the Teach-and-Grow scaling-law hypothesis ℰfuture(X) _future(X) =ℰ∞+AX−α, =E_∞+AX^-α, (1) Dteach(X) D_teach(X) =D∞+BX−β, =D_∞+BX^-β, where ℰfutureE_future is expected error on related future tasks, DteachD_teach is the teaching needed to acquire one, ℰ∞,D∞≥0E_∞,D_∞≥ 0, and A,B,α,β>0A,B,α,β>0. Unlike model size or a frozen offline dataset, X can grow throughout the robot’s life. The law predicts a simple consequence: the more the robot has reliably learned to reuse, the less it should need to be taught next time. Our contributions are: • We formulate Teach-and-Grow Learning, an agent-centered architecture that learns new manipulation tasks from sparse teaching, acts through dynamic capability composition and physical feedback, and grows without task-specific policy retraining. • We introduce reusable closed-loop Skill Blocks that turn several demonstrations into shared subgoal strategies while recomputing physical realization from the current scene. • We couple a persistent Skill Library with structured Experience Memory, allowing successful behavior, failure knowledge, and repair to shape later tasks and future fast-policy students. • We propose a Teach-and-Grow scaling-law hypothesis for lifelong robot intelligence, relating effective reusable experience to future-task error and the marginal amount of teaching required for new tasks. I Related Work I-A Generalist VLA and World-Action Policies VLA models learn a direct mapping from visual-language observations to robot actions. RT-1 and RT-2 established the promise of large-scale behavior cloning and web-to-robot transfer [6, 48]. Subsequent generalist policies broadened the route across model families [25, 15, 5], while larger robot datasets expanded its empirical foundation [26, 14, 36]. Across these efforts, scaling data and model resources produces systematic gains [31, 18, 42]. These results support end-to-end scaling, but they also expose its binding resource: physical coverage. Semantic knowledge from web-scale pretraining does not by itself supply action-conditioned geometry, contact, embodiment dynamics, or recovery. Probing studies report a gap between instruction understanding and reliable execution under distribution shift [8]. Online reinforcement learning and few-shot policy modules reduce the adaptation burden [21, 7, 34], while world-action models such as DreamZero learn visual dynamics and actions jointly [40]. The remaining systems question is how a robot should absorb uncovered tasks, modalities, and failures when another global training cycle is too costly or too slow. TGL takes a different architectural route. The generalist policy becomes one physical capability inside a larger learning system rather than the sole container of robot intelligence. New behavior is first acquired as explicit, reusable competence and can later be distilled into the end-to-end fast path. I-B Demonstrations, Few-Shot Adaptation, and Reusable Skills Learning from demonstration supplies successful structure without unrestricted physical exploration. Existing systems use demonstrations, play, and language to generate behavior that can be transferred or recombined [23, 39, 47]. A closely related line makes skill structure explicit for continual learning, symbolic composition, or few-shot adaptation [44, 30, 7]. The learning target in TGL differs from direct imitation: demonstrations are used to infer semantic subgoals, reusable relations, ordering, expected effects, and the scope in which a strategy applies. A Skill Block can contain several perception-action iterations and can be realized by a learned policy, planner, servo, controller, or tool composition. The original action sequence is not the final policy; the block is re-instantiated from current observations. I-C Agentic Tool Use, Memory, and Lifelong Robots Language-conditioned planners and tool-using agents have shown that high-level reasoning can organize robot programs, constraints, and feedback [2, 17, 10]. Recent robot operating layers extend this idea to skill graphs, memory, and cross-embodiment interfaces [20, 29, 41]. Other systems retain deployment experience or grow and repair skill libraries over time [37, 33, 22]. Memory, tools, and executable skills are related but not interchangeable. Episodic memory can recall what happened and why a previous attempt failed, yet a record cannot move the robot by itself. A tool exposes a physical capability, but does not say when that capability should be used or what result should count as success. An executable skill connects those pieces: it names an intended change, grounds the current scene, invokes an appropriate executor, and checks the outcome. A system may therefore remember many episodes without gaining a reusable behavior, or possess a broad policy without retaining the lesson behind a repair. TGL keeps these roles separate so that experience can guide choice while verified blocks change what the robot can actually do. These works provide many of the ingredients for agentic robotics. TGL joins them into a persistent learning cycle: few-shot demonstrations reveal task structure; the agent creates closed-loop Skill Blocks; physical feedback reshapes their composition; and the resulting success or failure changes what the robot can do next time. The agent remains active at semantic state transitions rather than generating a workflow once and stepping aside. Our contribution is how acquisition, execution, reflection, and growth operate as one persistent cycle. I Teach-and-Grow Learning I-A Learning with Fixed Model Weights TGL learns a new task by expanding explicit skills and memory, while its pretrained models remain unchanged. Consider a sequence of tasks nn=1N\T_n\_n=1^N. When task nT_n arrives, the robot receives a small teaching set n=dn(1),…,dn(mn),mn≪|pretrain|,D_n=\d_n^(1),…,d_n^(m_n)\, m_n |D_pretrain|, (2) where pretrainD_pretrain denotes the corpus used to train the fixed foundation stack. The robot begins with a Skill Library ℬnB_n, Experience Memory ℳnM_n, and interaction history ℋnH_n. Let Δℋn _n contain the useful lessons from executing this task. The update is θn+1 _n+1 =θn=θ, = _n=θ, (3) (ℬn+1,ℳn+1) (B_n+1,M_n+1) =⁡(ℬn,ℳn,n,Δℋn), =U(B_n,M_n,D_n, _n), ℋn+1 _n+1 =ℋn∪Δℋn. =H_n∪ _n. New behaviors enter ℬB; their context, failures, and repairs enter ℳM; and the pretrained parameters θ stay fixed. The robot learns by changing what it can retrieve and execute, not by retraining a task-specific policy. Equation 3 separates objects that are often treated as the same thing in lifelong robot learning. The fixed parameters θ provide pretrained semantic and physical priors. The Skill Library ℬB contains executable behavior and changes only when a block passes its admission checks. Experience Memory ℳM contains task context, outcomes, diagnoses, and repairs. It can guide a later decision, but it cannot execute an action by itself. The history ℋH preserves the evidence from which both stores were updated, and X later summarizes the portion that remains verified and reusable. A future fast student has separate parameters ϕφ and is trained only after the agentic system has produced verified trajectories. Task acquisition, external memory, and policy distillation are therefore different operations. TGL can acquire a task while θ remains fixed, and a later update to ϕφ does not turn that earlier acquisition into task-time policy retraining. Behavior cloning turns demonstrations into supervised action targets, while reinforcement learning uses interaction to change a policy or value function. TGL treats demonstrations as evidence about task structure and makes executable capability itself the object of learning. I-B Skill Blocks A Skill Block is represented as bi:⁣=⟨gi,i,ρi,γi,Πi,vi,ℛi⟩.b_i := g_i,S_i, _i, _i, _i,v_i,R_i . (4) Each field answers one practical question: gig_i says what should change; iS_i says when the block applies; ρi _i stores the reusable strategy; γi:→i _i:O _i maps the observation space O to the block’s grounded variables iZ_i; Πi _i lists compatible executors; viv_i maps pre-action observations, post-action observations, and executor evidence to pass, fail, or inconclusive; and ℛiR_i lists bounded recovery choices. At decision step t, the Agent selects an executor πi,t∈Πi _i,t∈ _i. The word block emphasizes composition rather than a low-level motor primitive. A block for acquiring the requested container turns a semantic request into a grounded grasp and closes its loop by verifying retention. The next block only needs the achieved effect: the target object is held. It does not depend on the backend that produced that effect. A complete task is represented at each semantic decision step by an ordered working plan τt=(bit,1,bit,2,…,bit,Lt), _t= (b_i_t,1,b_i_t,2,…,b_i_t,L_t ), (5) where LtL_t is the current route length. After a block returns an outcome, the agent may keep, shorten, or replace the remaining route. The tuple is therefore a plan for what comes next, not a script fixed at episode start. I-C Structured Experience Memory A Skill Block stores a behavior; Experience Memory stores the lesson learned from using it. Each record is a structured textual item μj=⟨ _j= task,context,blocks,outcome, ,context,blocks,outcome, (6) diagnosis,repair,evidence⟩. ,repair,evidence . It tells a later run what happened, why it happened, and what worked next. The record stays linked to its observations and provenance, but is retrieved as compact guidance that a person can inspect or revise. The library remembers how to act; memory remembers what experience taught. I-D The Teach-and-Grow Scaling-Law Hypothesis The scaling resource in TGL is not the number of stored episodes. It measures how much of the robot’s past can still help it act in a new scene. At a checkpoint with at least one positively weighted experience, define Xn=∑h∈ℋnω⁡(h,ℬn,ℳn),X_n= _h _nω(h;B_n,M_n), (7) where ω⁡(h,ℬn,ℳn)∈[0,1]ω(h;B_n,M_n)∈[0,1] is a dimensionless score fixed before evaluation; the positive-weight condition above ensures Xn>0X_n>0. The score uses only evidence available at checkpoint n: evidence reliability, added coverage, retrievability, grounding validity, and compatibility with admitted blocks. Future-task outcomes never enter it. Thus, two robots can store the same number of episodes yet possess very different XnX_n. In Eq. 1, X denotes this same checkpoint quantity under the fixed scoring rule. Equation 1 states the central prediction. As effective experience grows, error on related future tasks and the teaching required to acquire them should fall toward task-dependent floors. The exponents α and β characterize how efficiently a system turns lived experience into future competence. A robot that records many episodes but cannot retrieve or re-ground them will scale poorly; one that turns a small number of verified episodes into broadly reusable blocks may scale well. The law can be tested over sequential acquisition experiments: hold the foundation models and tools fixed, grow X, and measure future-task error and marginal teaching at each checkpoint. We define DteachD_teach as teacher intervention time under a fixed protocol until the preregistered success criterion is reached. If the criterion is not reached before the preregistered teaching budget is exhausted, the observation is recorded at that cap rather than omitted. Because Skill Blocks remain explicit objects, the same study can also track whether new growth preserves earlier behavior. IV General Agentic Robot Architecture Figure 2 follows one unfamiliar task from teaching to growth. Demonstrations reveal reusable structure; the agent turns that structure into feedback-conditioned action; and the resulting experience changes what the next task begins with. Perception, planning, control, and learned policies therefore appear as capabilities selected inside Skill Blocks, not as one fixed global pipeline. The figure also separates a proposed plan from authority to act. The Agent interprets the task, retrieves candidate blocks, and decides what remains to be done. A selected block must bind its contract to current observations, and an executor must produce the local physical change. The robot moves on only when fresh evidence satisfies the block’s declared effect. One successful action is not enough for persistent admission: the block also needs a bounded scope, compatible executors, a usable success test, recovery behavior, and provenance. These boundaries prevent a fluent plan from being mistaken for a valid grasp, path, or task success. They also keep improvement local. A stronger detector, planner, controller, or learned policy can replace one backend while the block contract and the rest of the library remain explicit. IV-A Few-Shot Teaching and Cross-Demonstration Abstraction For a new task, a few successful demonstrations show how the desired physical change unfolds. The learner looks for meaningful state changes, such as an object being acquired, a drawer opening, or a relation being established, rather than copying the original motion. The AI agent reads the instruction together with synchronized observations. Each demonstration is then viewed as an ordered sequence of segments, divided around meaningful changes in task state: d(j)=(s1(j),s2(j),…,sKj(j)).d^(j)= (s_1^(j),s_2^(j),…,s_K_j^(j) ). (8) Segments are aligned by what they accomplish, even when their timing and motion differ. Let kA_k collect all segments aligned to semantic effect k. The agent then synthesizes the shared strategy ρk=Synthesize⁡(k). _k=Synthesize(A_k). (9) The agent keeps the shared roles, relations, ordering, and effects, then recomputes scene-specific details such as pose, grasp, path, and control. Differences among demonstrations show how widely the block can be reused. A pattern seen with one instrument stays narrow; a strategy repeated across interchangeable objects can support a broader scope. IV-B Skill Blocks in the Current Scene Fig. 3: A Skill Block is a reusable contract for one meaningful change. It carries a goal, scope, strategy, grounding function, executor set, success test, and recovery. At runtime it observes the scene, binds current objects and geometry, chooses an executor, acts, and verifies the result. Figure 3 shows the division that makes reuse possible: the block keeps its meaning, but rebuilds its physical realization from the scene in front of the robot. A new block is tried in scenes beyond the teaching demonstrations. Success shows where it can be reused; failure narrows that range. Once reliable, it joins the library. Different backends may realize the same block as long as they achieve and verify the same result. An acquisition block may use geometric grasping, a VLA, or a tactile policy. Adding a new sensor or executor therefore changes only the blocks that need it. IV-C Dynamic Agentic Composition and Tool Selection Given a task instruction, current observations, and persistent state (ℬn,ℳn)(B_n,M_n), the Agent retrieves a compact set of blocks and forms the working plan in Eq. 5. The retrieved sequence is a plan, not a script. A familiar scene may route directly to a learned policy; an uncertain contact may call for geometry, touch, or a more deliberate controller. The choice can change whenever the physical state changes meaningfully. The agent does not emit high-frequency motor commands. At semantic decision step t, a block runs its own physical loop and returns the next observation and outcome evidence: (ot+1,εt)=Run⁡(bit,1,πit,1,t,ot).(o_t+1, _t)=Run(b_i_t,1, _i_t,1,t,o_t). (10) Here εt _t records executor and verifier evidence, including whether the intended effect passed, failed, or remained inconclusive. The agent uses it to continue, look again, choose another route, ask for teaching, or stop. What remains to be done depends on what physically happened. A conventional VLA or WAM can react to new observations while still operating inside a workflow fixed before the episode. Its observation interface, policy invocation, action decoder, success check, and fallback are predetermined. Feedback changes the next action, but it does not normally decide that the robot now needs a different sensor, a geometric planner, a newly retrieved behavior, or a revised task-level route. TGL adds this semantic control layer. After each verified effect, the Agent may continue, acquire more evidence, select another executor, replace the remaining blocks, request targeted teaching, or stop. The distinction is action adaptation inside a fixed execution path versus explicit task-level recomposition of that path. IV-D Physical Feedback, Bounded Exploration, and Targeted Teaching After a block executes, the agent compares the observed effect with the intended change. If they agree, the task continues. If they differ, the agent looks again, selects another executor or route, or asks the teacher for a focused demonstration. Once a mismatch is detected, the plan is corrected before the next block runs. Targeted teaching broadens this loop. The agent can ask for another demonstration when it would introduce a missing behavior, clarify an ambiguous subgoal, or move acquisition forward faster than unguided trial. Teaching seeds productive exploration; interaction broadens the teaching set; later guidance becomes increasingly selective. IV-E Structured Growth and Fast-Policy Distillation Every attempt can teach the next one. Reusable behavior enters the Skill Library; Experience Memory records why an attempt worked, failed, or recovered. Both stores remain searchable, versioned, and human-editable. Verified trajectories can also train a fast student. A VLA or WAM may execute a mature block directly, while the agentic path continues to handle novelty, diagnosis, and capability expansion. As robot foundation models become stronger, several specialist tools may collapse into a single executor. The organizing loop remains: teach, compose, act and check, then grow. V Experiments and Results We evaluate TGL on LIBERO [19] in two complementary ways. The standard benchmark measures the complete system. Controlled studies then look inside the learning cycle: whether demonstrations become usable blocks, whether those blocks persist, and whether physical feedback changes the plan. Full run details are in Appendix I. On the completed standard LIBERO evaluation, TGL attains state-of-the-art task success rates. The studies below explain how the architecture grows. TABLE I: Controlled Teach-and-Grow studies on LIBERO. Detailed splits and statistics appear in Appendix I. Study Observed result Main lesson Task-specific learning cycle Two learned blocks solve 3/3 states, survive save-and-reload at 3/3, and stop at the first unmet semantic effect outside their learned scope Demonstrations become persistent, scope-aware behavior Feedback-driven execution Two representative successful traces replan or reobserve after physical feedback Outcomes change what the Agent does next Fixed-executor library pilot Six-block library: 0/6; eight-block library: 4/6 Local library growth changes what the same executor can do From demonstrations to persistent blocks. Ten visual demonstrations decompose into acquisition and release stages, and all 20 observable effects are confirmed. In a separate learning cycle, three teacher trajectories produce two blocks: acquire the requested object and release it in the required relation. Together they solve all three evaluation states, are saved, reloaded, and solve the same states again. Save-and-reload shows that the taught route has joined the library as a reusable alternative; the existing six-block route remains available on the same states. Beyond its learned range, the route stops when the expected effect is absent, preventing one mistake from propagating through the task. The reported visual study uses a deterministic decomposition, while the general formulation can place a multimodal Agent in this role. Feedback changes the workflow. For the bowl-on-plate task, the Agent begins with an acquisition block followed by a release block. The pick succeeds, but the post-grasp evidence is not yet sufficient for placement. The Agent rebuilds the remaining route and completes the task. For the drawer-opening task, an inconclusive outcome prompts a fresh observation. In both traces, evidence from the physical world changes the Agent’s next action. Local growth under a fixed executor. Local library growth changes what the same executor can do. With six blocks it solves 0/6 evaluations; after two learned blocks are added, it solves 4/6. No model weights, runtime, or evaluation budget change. This small two-task pilot shows the mechanism in action: adding reusable capability can change future behavior without retraining the whole policy. VI Discussion VI-A Learning as Capability Growth Most learning systems store new ability inside a policy, value function, or world model. TGL also stores it as reusable behavior and the experience that gives that behavior meaning. A Skill Block may be realized by a learned policy, planner, controller, or conventional program. It becomes learned knowledge when the robot can recognize where it applies and produce the intended result in a new scene. Zero-shot exploration asks the robot to discover task structure and physical realization at the same time. Few-shot teaching separates those burdens. A handful of successful demonstrations supplies the basic task structure: what must change, in what order, and what success looks like. The robot still recomputes objects, geometry, contact, and control from the current scene. This narrows the search without limiting the robot to the demonstrations. Execution can still reveal variations, failures, and repairs that the teacher never showed. VI-B Persistent Memory and Non-Destructive Growth The Skill Library and Experience Memory make growth explicit. A new task leaves behind a named behavior and the context that makes it useful. When a related task arrives, the Agent can start from what the robot already learned instead of asking a global policy to reconstruct it. This knowledge is also legible. Structured text explains why a behavior was chosen, while a versioned block states what it should achieve and where it applies. A person can inspect and revise both. Saving a block is only the first step: it remains useful when the robot can retrieve, ground, and execute it in the current world. Consider what the next task inherits. A robot may first be taught to place a particular container on a support. In a later scene, the object, pose, camera, or executor can change. Replaying the original trajectory would tie the lesson to its first setting. TGL instead carries forward the verified subgoal structure: what relation should be established, which evidence is needed before acting, and what outcome closes the loop. The next task is not solved in advance, but it begins with a behavior that can be grounded again and a record of the conditions under which that behavior was useful. Growth therefore occurs on two timescales. During task acquisition, the pretrained parameters θ stay fixed while the Skill Library and Experience Memory change. The former adds something the robot can execute; the latter adds context that guides later choice but cannot act on its own. Effective reusable experience X summarizes the part of this inheritance that remains retrievable, groundable, and composable, rather than the number of episodes saved. A future fast student with parameters ϕφ may later absorb mature behavior for lower-latency execution. That is compression after acquisition, not the mechanism by which the new task first became available. Keeping these timescales separate makes the central claim simple: the robot grows when a local lesson becomes a reusable capability without requiring every lesson to rewrite the foundation model. VI-C The Agent as Organizing Intelligence Many earlier VLM and VLA systems operate inside a fixed workflow. They may react to observations, but the available sensors, policy calls, success checks, and fallback steps are chosen before execution. TGL places the Agent above that workflow. At each meaningful boundary, it can decide which evidence is missing, which executor should act, and whether the remaining task route still makes sense. In the LIBERO traces, that decision leads either to a better view or to a rebuilt route. Perception models, geometric planners, servos, controllers, and VLA policies provide the physical capabilities on which this reasoning acts. They will become broader and more integrated as robot foundation models improve. That progress strengthens the architecture: the toolbox may shrink, but the need to organize behavior, interpret outcomes, and retain new competence remains. Placing the Agent above the workflow does not mean asking a language model to issue every motor command. Most control should remain inside fast, tested executors. The Agent becomes active where the meaning of the task can change: after a subgoal is reached, when evidence is inconclusive, when an executor cannot satisfy its contract, or when the remaining route no longer fits the scene. These are sparse but consequential decisions. Treating them explicitly gives the system a readable boundary between reasoning and control. It also makes failure more useful. Instead of returning only a task-level zero, execution can identify which intended effect was missing, which observation was uncertain, and which local choice should be reconsidered. That information can guide the next attempt and later tasks without turning the entire episode into a new global training problem. VI-D The Teach-and-Grow Scaling-Law Hypothesis Fig. 4: Experience-driven robot scaling. The two aligned plots use separate schematic vertical scales: as effective reusable experience X grows, future-task error and teaching demand are predicted to approach their own irreducible floors. The Agent learns unfamiliar tasks and keeps verified experience; a future compression path moves mature behavior into a fast policy, with uncertain cases returning to the Agent. Model scaling improves the prior before deployment. Test-time scaling spends more computation on one episode. Figure 4 adds lifetime scaling: useful experience from one task changes how the next task begins. A single bridge block can open many new compositions; a large archive of redundant trajectories may add almost nothing. Equations 1 and 7 make that idea testable. With the foundation models and tools fixed, increasing X should lower future-task error and marginal teaching in a predictable way. The present studies establish block creation, persistence, and reuse; Appendix G specifies the longitudinal study that will measure the lifetime curve. VI-E A Slow Teacher and a Fast Student In the proposed two-speed extension, the agentic route learns unfamiliar tasks while a distilled fast policy executes familiar ones. Verified agentic trajectories can train a VLA, WAM, diffusion policy, or block-specific controller. When the fast path is uncertain, control returns to the Agent, which diagnoses the gap and expands the library. This division is more than a speed optimization. It gives unfamiliar behavior a place to be reasoned about before it is compressed. The Agent can ask what failed, change tools, and preserve the reason for a repair; a fast policy can then absorb behavior that has become stable enough to repeat. Exploration and routine execution no longer have to compete inside one opaque update. Over a robot’s lifetime, the boundary moves: today’s slow, newly taught task can become tomorrow’s familiar fast behavior, while the Agent remains available for the next unfamiliar case. The same division can later extend to a fleet: robots may share candidate strategies and failure signatures, while each receiver re-grounds and verifies them locally. VI-F Limitations The main practical cost today is time: an unfamiliar task may require several rounds of Agent reasoning, tool use, and observation. This gap should narrow as multimodal agents become faster and mature behaviors are distilled into fast policies. VII Conclusion A robot should be able to learn a local lesson locally. In TGL, sparse teaching becomes reusable Skill Blocks, physical outcomes change the plan, and each success or failure can improve the next task. Our LIBERO studies show that blocks can be induced, saved, reloaded, and recomposed, while the Agent adapts when execution departs from intent. Each part has a clear role: foundation models provide the prior, the Agent handles new tasks and organizes capability, the Skill Library keeps executable behavior, Experience Memory keeps its lessons, and a future fast policy can execute mature behavior. The Teach-and-Grow scaling-law hypothesis states the larger idea: as reusable experience grows, future error and teaching demand should fall. For a lifelong robot, learning means changing what it can reliably do. Deployment therefore remains a period of learning. The goal is not to preserve every interaction, but to keep the small part of experience that turns today’s unfamiliar task into tomorrow’s reusable capability. References [1] AgiBot-World-Contributors et al. (2025) AgiBot World Colosseo: a large-scale manipulation platform for scalable and intelligent embodied systems. External Links: 2503.06669, Document, Link Cited by: §A-A, §I. [2] M. Ahn et al. (2022) Do as i can, not as i say: grounding language in robotic affordances. External Links: 2204.01691, Document, Link Cited by: §B-B, §I-C. [3] B. Ai et al. (2025) Towards embodiment scaling laws in robot locomotion. External Links: 2505.05753, Document, Link Cited by: §G-B. [4] Anthropic (2025) Claude code overview. Note: Official documentation, accessed August 2026 External Links: Link Cited by: §I. [5] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky (2024) π0 _0: a vision-language-action flow model for general robot control. External Links: 2410.24164, Document, Link Cited by: §I, §I-A. [6] A. Brohan et al. (2022) RT-1: robotics transformer for real-world control at scale. External Links: 2212.06817, Document, Link Cited by: §I, §I-A. [7] Z. Ding and Z. Wang (2026) SkillPlug: unsupervised skill mining for few-shot adaptation in robotic manipulation. External Links: 2607.08354, Document, Link Cited by: §I-A, §I-B. [8] I. Fang, J. Zhang, S. Tong, and C. Feng (2025) From intention to execution: probing the generalization boundaries of vision-language-action models. External Links: 2506.09930, Document, Link Cited by: §A-A, §I, §I-A. [9] J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, O. Vinyals, J. W. Rae, and L. Sifre (2022) Training compute-optimal large language models. arXiv preprint arXiv:2203.15556. External Links: Document, Link Cited by: §G-A, §I. [10] W. Huang, C. Wang, Y. Li, R. Zhang, and L. Fei-Fei (2024) ReKep: spatio-temporal reasoning of relational keypoint constraints for robotic manipulation. External Links: 2409.01652, Document, Link Cited by: §B-B, §I-C. [11] W. Huang, C. Wang, R. Zhang, Y. Li, J. Wu, and L. Fei-Fei (2023) VoxPoser: composable 3d value maps for robotic manipulation with language models. External Links: 2307.05973, Document, Link Cited by: §B-B. [12] W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar, P. Sermanet, N. Brown, T. Jackson, L. Luu, S. Levine, K. Hausman, and B. Ichter (2022) Inner monologue: embodied reasoning through planning with language models. External Links: 2207.05608, Document, Link Cited by: §B-B. [13] J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei (2020) Scaling laws for neural language models. arXiv preprint arXiv:2001.08361. External Links: Document, Link Cited by: §G-A, §I. [14] A. Khazatsky et al. (2024) DROID: a large-scale in-the-wild robot manipulation dataset. External Links: 2403.12945, Document, Link Cited by: §A-A, §H-A, §I, §I-A. [15] M. J. Kim et al. (2024) OpenVLA: an open-source vision-language-action model. External Links: 2406.09246, Document, Link Cited by: §I, §I-A. [16] J. Kwok, C. Agia, R. Sinha, M. Foutter, S. Li, I. Stoica, A. Mirhoseini, and M. Pavone (2025) RoboMonkey: scaling test-time sampling and verification for vision-language-action models. External Links: 2506.17811, Document, Link Cited by: §G-B. [17] J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng (2022) Code as policies: language model programs for embodied control. External Links: 2209.07753, Document, Link Cited by: §B-B, §I-C. [18] F. Lin et al. (2024) Data scaling laws in imitation learning for robotic manipulation. External Links: 2410.18647, Document, Link Cited by: §G-B, §I-A. [19] B. Liu et al. (2023) LIBERO: benchmarking knowledge transfer for lifelong robot learning. External Links: 2306.03310, Document, Link Cited by: §V. [20] Y. Liu, W. Chen, X. Song, T. Pu, S. Mo, Y. Bai, Z. Chen, Q. Sun, L. Zhong, Y. Shen, and L. Lin (2026) PhyAgentOS: a self-evolving operating system for embodied agents with decoupled cognitive planning and physical execution. External Links: 2607.16636, Document, Link Cited by: §I-C. [21] G. Lu, W. Guo, C. Zhang, Y. Zhou, H. Jiang, Z. Gao, Y. Tang, and Z. Wang (2025) VLA-RL: towards masterful and general robotic manipulation with scalable reinforcement learning. External Links: 2505.18719, Document, Link Cited by: §I-A. [22] R. Lu, Y. Wu, E. Kou, L. Fu, W. Xiao, A. Mandlekar, Y. Xu, G. Shi, K. Goldberg, A. Chen, M. Chowdhury, Y. Zhu, L. J. Fan, and G. Wang (2026) ASPIRE: agentic skills discovery for robotics. External Links: 2607.00272, Document, Link Cited by: §B-B, §I, §I-C. [23] A. Mandlekar et al. (2023) MimicGen: a data generation system for scalable robot learning using human demonstrations. External Links: 2310.17596, Document, Link Cited by: §H-A, §I-B. [24] S. Nasiriany et al. (2024) RoboCasa: large-scale simulation of everyday tasks for generalist robots. External Links: 2406.02523, Document, Link Cited by: §H-A. [25] Octo Model Team et al. (2024) Octo: an open-source generalist robot policy. External Links: 2405.12213, Document, Link Cited by: §I, §I-A. [26] Open X-Embodiment Collaboration et al. (2023) Open x-embodiment: robotic learning datasets and RT-X models. External Links: 2310.08864, Document, Link Cited by: §A-A, §I, §I-A. [27] OpenAI (2025) Introducing codex. Note: Official product description, accessed August 2026 External Links: Link Cited by: §I. [28] T. Pearce et al. (2024) Scaling laws for pre-training agents and world models. External Links: 2411.04434, Document, Link Cited by: §G-B. [29] X. Qin, S. Luan, J. See, C. Yang, and Z. Li (2026) AEROS: a single-agent operating architecture with embodied capability modules. Note: Submitted to Engineering Applications of Artificial Intelligence External Links: 2604.07039, Document, Link Cited by: §I-C. [30] B. Quartey, S. Castro, E. Rosen, W. Thomason, G. Konidaris, and S. Tellex (2026) Jointly learning predicates and actions enables zero-shot skill composition. External Links: 2605.20648, Document, Link Cited by: §B-B, §I-B. [31] S. Sartor and N. Thompson (2024) Neural scaling laws in robotics. External Links: 2405.14005, Document, Link Cited by: §G-B, §I-A. [32] C. Tie, S. Sun, J. Zhu, Y. Liu, J. Guo, Y. Hu, H. Chen, J. Chen, R. Wu, and L. Shao (2025) Manual2Skill: learning to read manuals and acquire robotic skills for furniture assembly using vision-language models. External Links: 2502.10090, Document, Link Cited by: §H-A. [33] G. Tziafas and H. Kasaei (2024) Lifelong robot library learning: bootstrapping composable and generalizable skills for embodied control with language models. Note: ICRA 2024 External Links: 2406.18746, Document, Link Cited by: §B-B, §I-C. [34] C. Wang, C. Zhang, Z. Wu, R. Li, A. Ma, K. Chao, Y. Liang, X. Xu, Z. Wang, Y. Tang, and J. Lu (2026) SkillMemo: expert-guided skill memory for compositional embodied manipulation. External Links: 2608.05970, Document, Link Cited by: §B-B, §I-A. [35] Z. Wang, B. He, K. Yu, S. Lee, R. Gao, F. Huang, and Y. Aloimonos (2026) HumanEgo: zero-shot robot learning from minutes of human egocentric videos. External Links: 2605.24934, Document, Link Cited by: §H-A. [36] K. Wu et al. (2024) RoboMIND: benchmark on multi-embodiment intelligence normative data for robot manipulation. External Links: 2412.13877, Document, Link Cited by: §A-A, §I, §I-A. [37] S. Wu, Y. Ji, Q. Li, Z. Zhang, Q. He, W. Xie, G. Zhang, B. Bayramli, Y. Ding, and H. Lu (2025) Dejavu: towards experience feedback learning for embodied intelligence. External Links: 2510.10181, Document, Link Cited by: §I-C. [38] C. Xu, Y. Xu, M. Xue, J. Lin, J. Wang, X. Zhang, Y. Wang, and J. Yu (2026) The curse of precision: a data scaling law for high-precision robotic manipulation. Note: Accepted to ICRA 2026 External Links: 2607.23108, Document, Link Cited by: §A-B, §G-C. [39] M. Xu, Z. Xu, C. Chi, M. Veloso, and S. Song (2023) XSkill: cross embodiment skill discovery. External Links: 2307.09955, Document, Link Cited by: §H-A, §I-B. [40] S. Ye, Y. Ge, K. Zheng, S. Gao, S. Yu, G. Kurian, S. Indupuru, Y. L. Tan, C. Zhu, J. Xiang, A. Malik, K. Lee, W. Liang, N. Ranawaka, J. Gu, Y. Xu, G. Wang, F. Hu, A. Narayan, J. Bjorck, J. Wang, G. Kim, D. Niu, R. Zheng, Y. Xie, J. Wu, Q. Wang, R. Julian, D. Xu, Y. Du, Y. Chebotar, S. Reed, J. Kautz, Y. Zhu, L. J. Fan, and J. Jang (2026) World action models are zero-shot policies. External Links: 2602.15922, Document, Link Cited by: §I, §I-A. [41] S. Yoon, M. Yoo, S. Ahn, S. Choi, and H. Woo (2026) RoboBRIDGE: a modular framework for bridging policies to robust real-world robotic agents. Note: Accepted to IROS 2026 External Links: 2607.27881, Document, Link Cited by: §I-C. [42] L. Zha et al. (2025) Guiding data collection via factored scaling curves. External Links: 2505.07728, Document, Link Cited by: §A-B, §G-B, §I-A. [43] J. Zhang, J. Ge, H. Yoo, L. Fu, Z. Yang, Y. Liu, R. Saravanan, S. Yin, J. Yu, D. Niu, Z. Wang, R. Herzig, K. Goldberg, Y. Bai, D. M. Chan, I. Stoica, A. Kanazawa, J. Lei, H. Feng, and T. Darrell (2026) Playful agentic robot learning. External Links: 2606.19419, Document, Link Cited by: §I. [44] S. Zhang, S. Wang, H. Tang, J. Wu, and L. Nie (2026) Learning new tasks via reusable skills: skill-compositional experts for embodied continual learning. External Links: 2606.15685, Document, Link Cited by: §B-B, §I-B. [45] Z. Zhao, M. Cho, H. Shen, B. Zheng, K. Gao, Y. Cao, and Z. M. Mao (2026) Test-time scaling for world action models via zero-shot geometric evaluation. Note: Extended version of a CVPR 2026 EAI workshop paper External Links: 2607.17454, Document, Link Cited by: §G-B. [46] K. Zhu et al. (2025) Scaling test-time compute for LLM agents. External Links: 2506.12928, Document, Link Cited by: §G-B. [47] Y. Zhu, W. Wang, S. Wu, Y. Shi, and J. Wang (2025) Learning semantic atomic skills for multi-task robotic manipulation. External Links: 2512.18368, Document, Link Cited by: §I-B. [48] B. Zitkovich et al. (2023) RT-2: vision-language-action models transfer web knowledge to robotic control. External Links: 2307.15818, Document, Link Cited by: §I, §I-A. Appendix A Why End-to-End Scaling Remains Costly A-A The Missing Data Substrate Large language models began with an unusual historical advantage: vast quantities of text, code, books, and webpages had already been produced for human purposes. Model builders still pay heavily to curate and train on that material, but they do not need to recreate every sentence by operating a physical machine. General robot learning faces a different starting condition. Much of its valuable evidence consists of synchronized observations, actions, embodiment state, contact, calibration, failure, recovery, and final physical effect. This evidence does not exist until a robot or simulator is run. Open X-Embodiment, DROID, RoboMIND, and AgiBot World are major advances [26, 14, 36, 1]; their scale also makes visible how much new infrastructure and interaction must be created before robot data can resemble a mature web-scale substrate. The most consequential evidence is often the least likely to appear in a successful demonstration set: near collisions, transparent or deformable objects, unstable grasps, partial insertion, sensor disagreement, recovery after an interrupted sequence, and the chained states produced by earlier mistakes. These events determine whether a robot is useful outside a curated distribution. A broad visual backbone can contribute language, object semantics, and strong priors, yet it does not automatically supply embodiment-specific geometry, contact dynamics, or action-conditioned consequences [8]. Robot generality is therefore constrained not only by how much a model can absorb, but by how much of the physical world can be economically produced as training evidence. A-B Coverage Grows by Interacting Factors The difficulty is not captured by a single count of demonstrations. Let a deployment domain contain F physically relevant factors, including object identity, pose, clutter, illumination, camera, gripper, material, contact mode, and task relation. If njn_j denotes the meaningfully different regimes for factor j, a dense Cartesian design contains Ndense=∏j=1FnjN_dense= _j=1^Fn_j (11) coverage cells, where nj≥1n_j≥ 1. This idealized product shows why physical coverage can grow much faster than any one dataset coordinate: adding another independent factor multiplies the joint space. Even pairwise screening must consider F⁡(F−1)/2F(F-1)/2 factor pairs, with ∑j<knjnk _j<kn_jn_k joint pairwise cells. Factored scaling curves were introduced precisely because exhaustive environmental variation is prohibitively expensive [42]. Precision creates a second pressure. Let P denote a target tolerance, where a smaller value is stricter and c is the smallest attainable tolerance. One recent analysis reports a regime of the form log⁡Ndemo∝1/(P−c) N_demo 1/(P-c) for P>cP>c as P↓cP c [38]. This relation shows why producing the final increments of physical precision can require sharply more data. A-C The Retraining Tax Is Global Even When Failure Is Local Suppose a policy fails only for one unfamiliar container under one camera and grasp configuration. The missing competence is local, but repairing a monolithic policy commonly reopens a global process: collect representative interaction, optimize a new version, and recheck previously supported behavior. Adapters, replay, modular heads, online reinforcement learning, and continual-learning methods can reduce this burden. They do not remove the systems fact that a change distributed through model parameters is difficult to inspect, delimit, and certify locally. As the policy’s coverage expands, the cost of deciding what else may have changed expands with it. The retraining tax is therefore more than another training bill. It repeatedly turns a small uncovered part of the physical world into a new version of the whole policy. Teach-and-Grow changes the unit of update: a missing behavior can first be added as a separately addressable Skill Block, tested where it applies, linked to existing blocks, and retained without rewriting unrelated behavior. Appendix G develops the scaling and cost consequences of this change. A-D The Modality and Embodiment Tax A VLA can be redesigned to accept touch, audio, force, point clouds, or a new camera. A new gripper or robot can likewise be represented through additional data and an adapted action interface. The issue is not theoretical impossibility; it is that each change requires aligned interaction, fusion, calibration, optimization, and broad regression. In TGL, a modality is introduced where its evidence is consumed, and an embodiment is introduced through the blocks and executors that realize the relevant effects. Local integration still requires validation, but it need not force every unrelated behavior through the same update. A-E The Long Tail Is Where Explicit Knowledge Matters Most End-to-end policies are strongest when deployment resembles their training data. Long-tail failures are different: they are sparse, varied, and often useful only after diagnosis. A failed grasp matters more when the robot records that a transparent wall confused depth, a side approach caused collision, or retention must be checked before transport. Experience Memory keeps these lessons; Skill Blocks keep the response. When retrieval, grounding, or control fails, the robot can point to what it believed, what it expected, and what local object should change. Appendix B Teach-and-Grow Learning versus Existing Learning Forms TABLE I: Different learning objects and update mechanisms. Property Behavior cloning / imitation Reinforcement learning Teach-and-Grow Learning Primary evidence Expert observation–action pairs Interaction, reward, transitions Demonstrations, instructions, success, failure, diagnosis, recovery Learned object Parametric policy Policy, value function, or world model Explicit Skill Library and task compositions Demonstration role Target behavior to imitate Initialization, prior, or guidance Seed evidence for semantic structure and reusable strategy Exploration Usually limited Central Targeted after few-shot teaching; bounded by safety and missing knowledge Failure signal Distribution mismatch or supervised error Reward / return Structured cause, negative condition, recovery, or missing block Update Gradient optimization Gradient or search-based optimization Create, narrow, merge, replace, compose, or retire Skill Blocks Persistence Distributed in parameters Distributed in parameters / replay Explicit, versioned, human-readable behavior objects B-A The Learned Object and the Update Operator Table I shows why TGL is not behavior cloning plus an exploration wrapper. The demonstrations do not determine the final action policy. They establish an initial hypothesis about task structure: which semantic effects matter, how they are ordered, and which relations survive across demonstrations. Current objects, geometry, grasps, paths, and commands remain runtime variables. The robot can therefore realize the same taught structure with different backends, discover a recovery absent from the teacher, and update its explicit behavior state without optimizing the foundation policy. The difference can be expressed through the update operator. Parametric learning changes a model, θn+1=Optimize⁡(θn,n), _n+1=Optimize( _n,D_n), (12) whereas Teach-and-Grow applies Eq. 3 to the explicit skill, memory, and history state while the foundation parameters θ stay fixed. The two operations are compatible. A learned policy may implement a block, and verified behavior may later be distilled into separate student parameters ϕφ. Changing the foundation weights is simply not the only meaningful way for a lifelong robot to learn. B-B From a Model in the Workflow to an Agent That Owns the Workflow Language and vision-language models have already entered robotics as skill selectors, program generators, spatial reasoners, and constraint builders. SayCan ranks available skills, Code as Policies writes programs against robot APIs, VoxPoser and ReKep turn language and images into spatial objectives or constraints, and Inner Monologue adds environmental feedback to language-model planning [2, 17, 11, 10, 12]. These systems are important ancestors of agentic robotics. The distinction is not that earlier work has no tools or feedback. The architectural step in TGL is that the agent owns the evolving workflow rather than filling one slot inside it. It can decide which evidence to acquire, which block or backend to invoke, whether the last effect is sufficient, how the remaining task should change, what failure should be remembered, and whether the experience justifies a new persistent block. This resembles the difference between asking a language model for one answer and giving a coding agent a workspace: the latter repeatedly inspects state, acts through tools, reads the consequence, and changes both its plan and the artifacts available to future runs. In robotics, the workspace is physical and the artifacts are executable skills and grounded experience. Recent systems such as LRLL, ASPIRE, SkillMemo, SCE, and PACTS move toward lifelong skill acquisition, agentic discovery, memory, and compositional reuse [33, 22, 34, 44, 30]. TGL is closest to this family. Its defining conjunction is a continuous cycle in which few-shot teaching seeds cross-demonstration abstraction, the agent executes and revises a closed-loop composition, success and failure alter persistent skill and memory state, and mature behavior can be distilled into a fast policy. B-C Teaching and Learning Promote One Another The phrase comes from the classical Chinese educational text Xue Ji and is commonly translated as teaching and learning promote one another. It expresses more than the sequence teach, then grow. In TGL, the teacher provides demonstrations that reveal a productive structure. The robot’s execution then returns evidence about what was underspecified: an ambiguous object role, a missing recovery, an unsupported embodiment, or an outcome test that was too weak. That evidence improves the next act of teaching. The resulting loop is teachn→composen→act+checkn→grown→teachn+1.teach_n _n +check_n _n _n+1. (13) Early teaching may introduce entire behaviors. Later teaching can focus on the frontier: rare failures, new tools, or missing transitions between familiar blocks. The robot grows because it is taught; teaching becomes more precise because the robot exposes what it does and does not yet understand. This mutual refinement is the educational principle behind the architecture and the practical basis for the prediction that teaching demand should fall as reusable experience accumulates. Appendix C From Zero-Shot Agent Control to Teach and Grow C-A The Minimal Agent Route The project began by asking a multimodal Agent to plan directly from robot-view images. It understood the goal, but understanding the task was not enough to execute it: depth, collision, contact, and high-rate control still demanded robot-native competence. C-B Tool Use and Agentic Visual Feedback We then gave the Agent perception, grasping, planning, and control tools, and returned a fresh observation after each short execution. It could now change its route when the world disagreed with the plan. This made zero-shot agent control technically feasible, but learning a physical task could still require many trials and model calls. Structured memory made those trials useful beyond the current episode: successes supplied strategies, while failures supplied conditions and repairs. The update changed explicit memory rather than model parameters. C-C Why Few-Shot Teaching Is the Practical Default Unrestricted exploration is slow and can be unsafe on a physical robot. Few-shot teaching provides a successful starting structure, so autonomous behavior can focus on the variation, correction, and recovery that the demonstrations did not cover. This became the default Teach-and-Grow route. The operating cost of the current Agent is higher than feed-forward policy inference because a task can require multiple sequential model calls. This cost is expected to fall with model and caching improvements, but it also motivates the slow-teacher/fast-student design. The Agentic system is most valuable for acquiring novelty and generating structured training data; mature tasks should migrate to fast policies. Appendix D Skill Blocks and Atomic Skills D-A Why the Term Skill Block Atomic skills are an established and useful concept. The word atomic, however, often suggests the smallest indivisible controller, action primitive, or short policy. Our decomposition operates at a broader semantic level. A block is one decision-level building element of a task and may contain a substantial internal feedback loop. The block metaphor also makes the scaling argument intuitive: new tasks are built by recombining old blocks and learning only the missing pieces. A short gripper close can be a block when it is independently meaningful. So can a longer acquire the requested container behavior that detects, approaches, closes, lifts, and checks retention. The granularity is determined by semantic closure and reuse, not by duration or number of motor commands. D-B Taxonomy Skill Blocks can be organized by the kind of effect they produce: • acquisition blocks: establish stable possession or contact; • transport blocks: preserve a state while changing a spatial relation; • release and placement blocks: establish support, containment, alignment, or insertion; • articulation and activation blocks: open, close, rotate, press, toggle, or actuate; • observation blocks: gather evidence needed by later decisions; • recovery blocks: restore a precondition after a diagnosed failure. These families guide retrieval but do not impose implementation. A learned policy, planner, servo, or human intervention can realize the same semantic block. D-C A Populated Block from the Experiment TABLE I: A populated acquisition Skill Block induced from three teacher trajectories. Field Stored content Subgoal g Acquire and retain the requested alphabet-soup can. Scope S Object-specific in the reviewed LIBERO-Object task; it is not exposed as a generic can skill. Reusable strategy ρ Bind the current object frame; approach the visible graspable body; align; close with bounded local motion; verify that the object is held; lift while preserving retention. Grounding γi(ot) _i(o_t) Estimate current object identity, frame, confidence, obstacles, reachability, grasp candidates, and local motion parameters from fresh observations. Executors Πi _i Current-scene grasp selection, servo acquisition, and a newly planned collision-aware transition. Outcome test viv_i Pass only when fresh observations and action feedback verify object retention and displacement. Recovery ℛR Reobserve, adjust local parameters, invoke an allowed grasp repair, choose an alternative block, recompose the task, or abort. Excluded Demonstration world coordinates, pixels, old path, joint trajectory, and low-level action replay. The second induced block represents release the held object through the visible opening. It retains the relation and release strategy; the opening frame, release band, held-object offset, path, and controller commands are computed in the current scene. Appendix E Detailed Agent-Led Skill Induction E-A Source Normalization Teaching is first converted into observable event sequences. Robot demonstrations provide synchronized observations and actions; human video provides visible state changes and object relations; written instructions provide intent and order. The source type stays attached because each source supports a different part of a block. E-B Semantic Decomposition The Agent splits each trajectory at meaningful state changes using task language, object relations, gripper state, and motion. A segment receives the semantic signature σ⁡(s)=⟨gs,us,as,ℓs,ps,qs⟩,σ(s)= g_s,u_s,a_s, _s,p_s,q_s , (14) where gsg_s is the subgoal, usu_s the manipulated entity, asa_s its affordance, ℓs _s the target relation, psp_s the preceding effect, and qsq_s the ordering context. E-C Cross-Demonstration and Cross-Embodiment Alignment Alignment follows semantic effect rather than motor coordinates. Acquire the requested container can preserve the object role and the result object is stably held whether a robot uses a parallel gripper, suction, or a dexterous hand. The receiving robot supplies its own executor, grounding, and local verification. This makes cross-embodiment reuse a natural next test; the present study uses one embodiment. The alignment procedure follows four steps: 1. normalize each source into observable events while keeping its type; 2. align segments that achieve the same effect and have compatible entities or relations; 3. separate shared task structure from pose, path, timing, and embodiment-specific control; 4. validate the resulting block on fresh cases and narrow its scope when needed. Variable-length demonstrations therefore need not have the same number of raw segments. They must agree on the semantic effects used for the final task composition. E-D Invariant Synthesis For each aligned group, the Agent separates: • semantic invariants: subgoal, object role, relation, and expected effect; • behavioral invariants: approach family, contact region, ordering, and permitted executor family; • instance variables: pose, pixels, point cloud, grasp, path, timing, and robot configuration. The first two categories define reusable strategy; instance variables are recomputed at runtime. A segment that still depends on the original coordinates remains an episode rather than a block. E-E Scope Selection Scope expands only with supporting variation. Repeated success across poses supports pose invariance. Multiple instances support object-level abstraction. Cross-category success with a common affordance may support affordance-level reuse. Contradictory demonstrations create alternative blocks or narrower conditions. A single unusual instrument remains object-specific until evidence justifies more. E-F Validation and Library Update A candidate block is tested on separated cases before normal retrieval. The test asks whether it produces the intended effect from the current scene, preserves matched old behavior, and stays within its claimed scope. A weak block is narrowed or repaired; a validated one enters a new library version. Appendix F Agentic Execution, Memory, and Failure Learning F-A The Agent’s Decision Surface At task time, the agent decides: • which observations and modalities are needed; • which Skill Blocks are relevant; • how they should be ordered or conditioned; • which backend should instantiate a block; • whether an observed effect is sufficient to continue; • whether to retry, substitute, recompose, request teaching, or stop; • which parts of an execution are valuable for future learning. The agent does not need to produce high-rate control. It delegates continuous behavior to the block’s executor and reasons at meaningful state transitions. F-B Success and Failure as Memory A successful trace can establish a new block, broaden evidence, or confirm a composition. A failure can be more informative than another success. It may reveal that a visual condition is ambiguous, a grasp family is unsuitable, a planner bridge fails under clutter, or a block’s outcome test is too weak. The memory update records a structured explanation and links it to observations and outcomes. Because the memory is explicit, humans can inspect and edit it. An operator can correct an overgeneralized rule, forbid a recovery, or mark a tool version incompatible. This editability is difficult to achieve when every experience is absorbed only into distributed parameters. F-C Four Levels of Retention Persistent storage avoids direct parametric overwriting, yet behavior can still degrade. We distinguish: 1. storage retention: the block remains present; 2. retrieval retention: the correct block is still selected; 3. grounding retention: its strategy still binds to current sensing and embodiment; 4. behavioral retention: execution success is maintained. A longitudinal evaluation must measure all four rather than equating a saved file with preserved competence. Appendix G Scaling Laws, Cost, and the Teach-and-Grow Hypothesis G-A What a Scaling Law Means The influential language-model scaling laws did more than state that larger models perform better. They identified explicit resources such as model parameters, training tokens, and compute, then showed that held-out loss follows approximately predictable power-law trends over broad ranges [13]. Chinchilla further showed that the allocation of a fixed compute budget matters: model size and data should grow together near the compute-optimal frontier [9]. A useful scaling law needs four things: a measurable resource, an outcome that matters, a stable relationship between them, and a prediction that holds at scales not used to fit the curve. The claim that more experience should help is not yet a scaling law. A robot scaling law must say what kind of experience counts, what remains fixed, what performance is predicted, and where the relationship should fail or saturate. G-B What Existing Robot Scaling Laws Measure Robot learning already exhibits several valuable scaling regularities. A broad meta-analysis finds power-law-like improvement with data, model size, and compute across many robot-learning studies [31]. Large imitation-learning experiments report approximately power-law gains with demonstrations, environments, and object diversity, with diversity often more valuable than repeatedly sampling the same condition [18]. Factored scaling curves estimate which environmental factors most deserve additional data [42]; embodiment scaling studies ask how performance changes as the diversity of bodies grows [3]; and agent/world-model work studies pretraining resources and architecture-dependent scaling [28]. A parallel line spends more computation at inference through sampling, verification, or world-action evaluation [46, 16, 45]. These laws answer important questions, but they mostly concern one of three regimes: how a policy improves before deployment as offline resources grow; how a family of policies changes with training diversity; or how one episode improves as test-time computation increases. They do not directly describe a deployed robot that repeatedly turns new experience into persistent, separately reusable competence and then faces another task with a different acquisition burden. G-C Why Performance Scaling and Cost Scaling Diverge A benign performance curve can hide a steep data-production curve. Suppose a manipulation domain is broadened along F interacting factors, each with roughly r regimes. Dense coverage scales as rFr^F in the idealized Cartesian case of Eq. 11. A learner may generalize between cells and need far fewer demonstrations, but newly discovered interactions still require fresh robot operation. Near a tolerance limit, log⁡Ndemo∝1/(P−c) N_demo 1/(P-c) with P>cP>c predicts sharply increasing demand as P↓cP c [38]. For an end-to-end route, a useful cost decomposition is CE2E(K)= C_E2E(K)= Ccollect(Ncover(K))+Ctrain(Ncover(K)) C_collect(N_cover(K))+C_train(N_cover(K)) (15) +Cregress(θnew(K),old), +C_regress( _new(K),T_old), Here K counts matched semantic capability additions, Ncover(K)N_cover(K) is the interaction needed to cover them, θnew(K) _new(K) is the resulting updated policy, and oldT_old is the earlier task set. All cost terms in this section use one preregistered scalar unit, so collection, compute, human time, validation, and regression are converted before they are added. Equation 15 identifies the regime in which dense joint coverage, strict tolerance, and repeated global regression make the next useful increment increasingly costly. Teach-and-Grow is designed around a different cost structure. In TGL, each counted semantic capability addition is one admitted reusable Skill Block. For block k, define κk= _k= κkteach+κkground+κkvalidate+κklink, _k^teach+ _k^ground+ _k^validate+ _k^link, (16) CTGL(K)= C_TGL(K)= C0+∑k=1Kκk+Cretrievecum(K). C_0+ _k=1^K _k+C_retrieve^cum(K). Here C0C_0 is the fixed infrastructure cost, and Cretrievecum(K)C_retrieve^cum(K) is the retrieval cost accumulated while admitting the first K semantic capability additions, each realized by one block in TGL. If block scope and validation stay local, each κk _k remains bounded. With hierarchical retrieval satisfying Cretrievecum(K)=O⁡(K)C_retrieve^cum(K)=O(K), cumulative cost is at most linear in K. The robot then pays mainly for the missing behavior and its local interfaces. All-to-all compatibility testing would instead make linking quadratic; block contracts and scopes are what keep the update local. Figure 5 contrasts the two acquisition regimes as a conceptual prediction: broadly coupled updates grow convexly, while local additions can remain near-additive. Fig. 5: Conceptual acquisition-cost regimes. When each addition reopens joint coverage and regression, globally coupled updates can grow convexly with capability breadth. Local Skill-Block additions can remain near-additive when interfaces stay bounded and retrieval is at most linear. Inset: for target tolerance P>cP>c, the cited precision law diverges as P↓cP c. G-D The New Scaling Resource: Effective Reusable Experience The core Teach-and-Grow law concerns benefit rather than storage volume. Equation 7 gives each stored experience a score in [0,1][0,1] based on five practical factors: evidence reliability, added coverage, retrievability, grounding validity, and compatibility with admitted blocks. The factors and aggregation rule are fixed before the scaling study, and future-task outcomes are hidden from the score. With the foundation agent, tool versions, test-time budget, future-task distribution, and scoring rule fixed, Eq. 1 predicts that future-task error and teaching demand decay toward irreducible floors as power laws in XnX_n. Equation 16 adds the cost-side prediction: explicit capability can grow close to additively when updates remain local. The resource grows after deployment, and one task can make the next task easier without retraining the full policy. G-E A Longitudinal Test A scaling study freezes the foundation agent, tools, inference budget, success criteria, future-task distribution, and scoring rule while the robot learns tasks in sequence. Each library checkpoint receives an XnX_n value before the next outcomes are observed. The evaluation mixes old tasks, new compositions, new instances of known affordances, and tasks that require a genuinely new behavior. These groups show whether progress comes from remembering an episode, recombining a known skill, broadening its scope, or acquiring a missing block. At every checkpoint, the primary variables are future-task error ℰfuture(Xn)E_future(X_n) and teacher intervention time Dteach(Xn)D_teach(X_n). The latter runs until the fixed success rule or the preregistered teaching-budget cap; budget-exhausted acquisitions remain in the analysis at that cap. Retention, reuse, harmful retrieval, latency, and cumulative acquisition cost explain why a checkpoint departs from the main curve. Raw episodic memory, a fixed tool pipeline, sequential VLA adaptation, and an Agent–VLA hybrid provide matched points of comparison. The power law is fitted on early checkpoints and tested on larger held-out checkpoints alongside standard alternative curves. Total cost includes teaching, robot interaction, inference, verification, storage, retraining, and regression. The prediction is useful only if it reaches unseen checkpoints and reveals where improvement saturates. Appendix H A Whole-System View of the Robot Learning Ecosystem Fig. 6: Whole-system ecology of Teach-and-Grow Learning. TGL provides a semantic routing and verification layer through which human knowledge, simulation, learned policies, learning methods, perception, planning, control, and safety mechanisms contribute evidence or executable change. The Agent composes these specialized roles around a common learning cycle; verified capability and experience persist separately for later tasks. Dashed paths denote future distillation and fleet sharing. H-A Sources of Teaching and Experience The outer network in Fig. 6 accepts more than robot demonstrations. Teleoperation and robot trajectories provide the richest embodiment-aligned evidence and can seed both blocks and fast policies [14]. Simulation, procedural generation, and sim-to-real pipelines can produce controlled variations, failures, and rare conditions before physical deployment [23, 24]. Human demonstration and egocentric video provide inexpensive semantic and interaction structure, although embodiment conversion must recover robot-executable geometry and action [39, 35]. Manuals, natural-language instructions, diagrams, and CAD can provide ordering, constraints, affordances, and object-specific procedures [32]. Autonomous exploration contributes the complementary evidence that teachers rarely provide: failures, repairs, and unexpected but successful alternatives. These sources differ in what they can justify. A manual may reveal that a latch must be released before a panel moves, but it cannot by itself certify a grasp controller. Human video may reveal contact sequence and object relation while leaving robot kinematics unresolved. Simulation may offer exact state and broad variation yet still require real-world calibration. TGL preserves source type and uncertainty so that semantic structure can transfer while physical claims are validated by an appropriate executor and observation channel. H-B Existing Robot Methods Become Roles, Not Rivals A VLA, WAM, diffusion policy, or task-specific policy can occupy at least three places in the architecture. It may implement one Skill Block, such as a dexterous insertion behavior; it may execute an entire mature composition as a fast path; or it may be the student distilled from verified agentic trajectories. Imitation learning and reinforcement learning remain natural ways to build or improve these executors. Teach-and-Grow does not compete with their ability to produce fast continuous control; it supplies the surrounding system that decides when a policy is applicable, what evidence it needs, how its effect is checked, and what should happen when it leaves its trained distribution. Classical robotics is equally native to the system. Detection, segmentation, RGB-D geometry, grasp synthesis, motion planning, impedance control, visual servoing, and tactile feedback can each provide evidence or realize a block. Traditional controllers are often the most reliable backend for a well-understood local effect. World models can predict consequences or rank alternatives; test-time verifiers can reject weak candidates; safety filters and humans can constrain the action set. The top-level task is no longer forced into a universal perception–planning–control pipeline. Each block assembles the physical loop appropriate to its own semantic effect. H-C The Agent Supplies the Missing Organizational Layer The common interface is a semantic contract: what effect a block is intended to establish, where it applies, what current evidence it requires, which backends can realize it, how success is observed, and what recovery is available. The agent uses those contracts to connect methods that were previously evaluated as separate systems. A visual servo can follow a VLA proposal; a planner can bridge two learned policies; a world model can screen candidate blocks; a human can teach the one transition that simulation failed to discover. The relevant distinction is whether a component contributes reliable evidence or executable change, not whether it is classified as classical or learned. Integration remains local: a new module needs its own adapter, calibration, and validation, while unrelated blocks remain unchanged. A tactile sensor can strengthen the blocks that need contact, and a new VLA can replace one executor without erasing a proven visual-servo block. The system can adopt stronger components while preserving the learning cycle around them. H-D Verified Data Generation and Distillation A slow agentic learner produces richer supervision than a raw successful trajectory. Its records can include semantic boundaries, selected blocks, current bindings, outcomes, verifier decisions, failure classes, recoveries, and final task success. These signals can train a student to execute one block, choose among blocks, or directly map observations to actions for a mature task family. A practical robot can alternate between acquisition and compression: novel task task →few-shot agent solution→verified experience -shot agent solution experience (17) →distilled fast student→routine deployment. fast student deployment. Uncertainty, distribution shift, or failure returns control to the agentic route, which expands both the library and the next student dataset. The slow system learns at the frontier; the fast system serves conditions that have become familiar. H-E Personal, Fleet, and Ecosystem Growth At the personal level, one robot accumulates skills adapted to its own tools, calibration, and environment. At the fleet level, robots can share candidate semantic strategies, failure signatures, outcome tests, and distilled students; each receiving robot re-grounds and revalidates them with its own embodiment. At the ecosystem level, third parties may publish interaction blocks, specialist perception modules, VLA executors, recovery methods, embodiment adapters, simulators, and validation suites. Teach-and-Grow does not replace robotics with one model. Foundation models provide broad priors. Demonstration, simulation, and human knowledge provide teaching. Learned and classical methods provide physical competence. The Agent organizes these capabilities, while persistent skills and memory carry useful knowledge forward. Distillation turns mature behavior into speed. The architecture allows progress in each component to improve the same learning cycle. Appendix I Experimental Details I-A Agent Configuration and Trace Provenance The online traces use gpt-5.6-sol with seed 0 and structured decisions. The Agent receives the task, current agent-view and wrist evidence, registered Skill Blocks, allowed tool routes, and verifier or planner evidence. Each trace allows at most one evidence refresh and two replans; no human changes an in-episode decision. The two representative successful traces come from a frozen LIBERO-GOAL archive. Together they show the two decisions studied here: rebuilding the route for bowl-on-plate and gathering a new observation for drawer opening. I-B Visual Demonstration Decomposition The visual study uses ten successful demonstrations from two related LIBERO-Object condiment-transfer tasks, five per task. Sampled agent-view and wrist images are processed without reward, task-success labels, hidden simulator object state, demonstration actions, or reference boundaries as production inputs. Independent annotations are used only after prediction. The corpus contains 40 predicted and 40 reference stages. Ordered role/type accuracy is 1.000. Boundary F1 is 0.100 at exact matching, 0.633 within one sampled frame, and 0.900 within two sampled frames. Observable tests verify ten acquisition and ten release effects, producing 20/20 effect confirmations. I-C Task-Specific Skill Learning Three teacher trajectories from states 0–2 produce an object-specific acquisition block and a relation-aware release block. On separated states 3–5, the pair succeeds on 3/3 evaluation initial states; the existing six-block route also succeeds on 3/3. After scope checks, the two blocks enter an eight-block library, survive save-and-reload, and again succeed on 3/3. On farther states 6–8, the learned route stops at the missing semantic effect, showing that its scope is enforced during execution. I-D Related-Task Fixed-Executor Pilot The pilot compares six-block and eight-block libraries on two related LIBERO-Object tasks. Executor, runtime, seed 0, success criteria, and the three-attempt budget stay fixed; only the learned acquisition and release blocks are added. Library contents and scopes are frozen before evaluation. Onboarding uses states 12–14, and both libraries are evaluated on the same disjoint states 15–17. With unsuccessful acquisitions retained at the three-attempt cap, the observed median is 2.5 attempts with six blocks and 1.5 with eight. Evaluation success is 0/6 versus 4/6; the corresponding 95% Wilson intervals are [0.00,0.39][0.00,0.39] and [0.30,0.90][0.30,0.90], with two-sided Fisher exact p=0.061p=0.061. These cell-level statistics summarize a compact two-task, single-seed mechanism study. I-E Failure-Localization Cohort A separate eight-attempt acquisition cohort records 0/8 task success and localizes the remaining failures: two occur before motion planning, four arise from path consistency or calibration, and two from gripper closure. These categories identify where the feedback loop should improve next. Appendix J Current System The current stack reflects the limits of today’s general agents. A multimodal coding agent reasons over language, images, tools, and Skill Blocks. Image changes support demonstration analysis; detection, segmentation, RGB-D, and robot state bind objects and relations. Contact-GraspNet and MPLib supply grasps and collision-aware motion, while bounded controllers carry out local action. Fresh observations test physical effects; a versioned Skill Library and structured failure memory preserve what should matter later. Each role is replaceable. A temporal model can analyze demonstrations; tactile or 3D representations can strengthen grounding; a learned policy, WAM, or dynamic planner can realize behavior; and a reward model or multimodal verifier can judge effects. As these models absorb better geometry, dynamics, and feedback, several roles may collapse into one block executor. The persistent loop still records what was acquired, where it applies, and how it composes. Distillation is the proposed next step for the fast path.

Instant research discovery