Paper deep dive
MIND: Instruction-Conditioned Incremental Grounding for Robots
Mohamed Abouagour, Junhyun Nam, Wonse Jo, Byung-Cheol Min
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 10/15/2026, 5:31:48 AM
Summary
The paper introduces MIND (Maintenance via INstruction-conditioned Dependencies), a framework for robots to maintain target-room decisions under persistent language instructions without re-evaluating every room after environmental changes. MIND compiles instructions into semantic qualification and target selection dependencies, intersecting them with event footprints to refresh only affected judgments. Experiments on ProcTHOR, AI2-THOR, and Unitree G1 hardware demonstrate significant reductions in semantic evaluations while preserving decision correctness compared to full recomputation.
Entities (9)
Relation Signals (8)
Byung-Cheol Min → affiliatedwith → Indiana University Bloomington
confidence 95% · Byung-Cheol Min Affiliation: Indiana University Bloomington, USA.
Mohamed Abouagour → affiliatedwith → Indiana University Bloomington
confidence 95% · Mohamed Abouagour Affiliation: Indiana University Bloomington, USA.
Junhyun Nam → affiliatedwith → Incheon National University
confidence 95% · Junhyun Nam Affiliation: Incheon National University, Republic of Korea.
Wonse Jo → affiliatedwith → Incheon National University
confidence 95% · Wonse Jo Affiliation: Incheon National University, Republic of Korea.
MIND → reduces → semantic evaluations
confidence 95% · MIND reduces mean semantic evaluations from 30.6 to 11.2 while preserving all 1,200 post-event decisions.
MIND → usedon → ProcTHOR
confidence 95% · On 240 five-event ProcTHOR sequences... MIND reduces mean semantic evaluations
MIND → usedon → AI2-THOR
confidence 95% · In 32 closed-loop AI2-THOR episodes... MIND matches the final target decision
MIND → usedon → Unitree G1
confidence 95% · Unitree G1 experiments demonstrate target reuse and reselection... under MIND
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Robots following persistent language instructions must decide which cached room judgments remain valid as their environment changes. We present MIND (Maintenance via INstruction-conditioned Dependencies), a framework for maintaining target-room decisions without re-evaluating every room after each event. A language model compiles each instruction into dependencies for semantic qualification and target selection. The controller intersects semantic dependencies with event footprints, refreshes affected judgments, and updates reachability and selection separately. For fixed deterministic evaluation, planning, and selection, conservative dependency and footprint coverage preserves the decisions of full recomputation. On 240 five-event ProcTHOR sequences with an exact benchmark predicate, MIND reduces mean semantic evaluations from 30.6 to 11.2 while preserving all 1,200 post-event decisions. Frozen-evaluator experiments with Phi-4 and Qwen3-14B also preserve all 1,200 decisions, without improving the evaluators' underlying task accuracy. In 32 closed-loop AI2-THOR episodes using exact-predicate qualification and metadata-derived footprints, MIND matches the final target decision of full recomputation in every episode. Unitree G1 experiments demonstrate target reuse and reselection across room transitions without additional semantic evaluations under MIND, and abstention after a barrier and an object removal at 4 evaluations against 15 to 18. Additional materials are available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2610.19262v1
- Canonical: https://arxiv.org/abs/2610.19262v1
Trouble viewing inline? Open PDF directly →
Full Text
51,149 characters extracted from source content.
Expand or collapse full text
MIND: Instruction-Conditioned Incremental Grounding for Robots Mohamed Abouagour Affiliation: Indiana University Bloomington, USA. Junhyun Nam Affiliation: Incheon National University, Republic of Korea. Wonse Jo Affiliation: Incheon National University, Republic of Korea. Byung-Cheol Min Affiliation: Indiana University Bloomington, USA. Abstract Robots following persistent language instructions must decide which cached room judgments remain valid as their environment changes. We present MIND (Maintenance via INstruction-conditioned Dependencies), a framework for maintaining target-room decisions without re-evaluating every room after each event. A language model compiles each instruction into dependencies for semantic qualification and target selection. The controller intersects semantic dependencies with event footprints, refreshes affected judgments, and updates reachability and selection separately. For fixed deterministic evaluation, planning, and selection, conservative dependency and footprint coverage preserves the decisions of full recomputation. On 240240 five-event ProcTHOR sequences with an exact benchmark predicate, MIND reduces mean semantic evaluations from 30.630.6 to 11.211.2 while preserving all 1,2001,200 post-event decisions. Frozen-evaluator experiments with Phi-4 and Qwen3-14B also preserve all 1,2001,200 decisions, without improving the evaluators’ underlying task accuracy. In 3232 closed-loop AI2-THOR episodes using exact-predicate qualification and metadata-derived footprints, MIND matches the final target decision of full recomputation in every episode. Unitree G1 experiments demonstrate target reuse and reselection across room transitions without additional semantic evaluations under MIND, and abstention after a barrier and an object removal at 44 evaluations against 1515 to 1818. Additional materials are available at https://mind-icra27.github.io/. I Introduction A robot may continue following the same instruction while its environment changes. For “bring me the nearest mug,” entering another room, closing a doorway, or removing an object can alter the appropriate target in different ways. The robot has already evaluated which rooms satisfy the instruction, yet those cached judgments need not all become invalid together. Re-evaluating every room discards reusable computation; retaining every judgment can preserve a stale target. Each room judgment is a language-model call, which on a robot shares the GPU with perception and on a hosted model is billed, so the calls avoided are the resource at stake. We ask: after a world change, which room judgments must the robot check again? The event alone does not determine the required update. For the nearest chair, robot motion can change the selected room without changing which rooms contain chairs. For a chair in the room I am in, the same motion changes which rooms qualify. The same variable thus enters qualification or selection depending on the instruction, and its stage decides whether semantic re-evaluation is needed. Fig. 1: An overview of “Bring me the nearest mug.” At t0t_0 every room is judged and cached. Moving (t1t_1) changes no room: judgments are reused, only the target is re-chosen. Removing the kitchen mug (t2t_2) refreshes one judgment. Full recomputation rejudges all four each time. Fig. 1 illustrates the distinction within one persistent instruction. Moving to another room changes the ranking of candidates but not their inventory-based qualification, so no room judgment needs refreshing. Removing the kitchen mug instead requires rechecking the kitchen while retaining the other judgments. Dependency-based reuse is well established in incremental computation, where dependencies may be declared in program or query structure or discovered during execution [1, 2, 3, 4]. In our setting, the state dependencies relevant to maintenance are implicit in a language instruction and differ between qualification and selection. MIND infers these instruction-specific dependencies from language, separately for qualification and selection, to determine which cached judgments require updating. We therefore study decision-preserving maintenance of a given grounding pipeline, rather than replacing its semantic evaluator or replanning the entire task after every event. We present MIND (Maintenance via INstruction-conditioned Dependencies), a framework for instruction-conditioned incremental grounding. Before execution, a language model compiles the instruction into separate dependency sets for room qualification and target selection. After an event, the controller intersects the semantic dependencies with a footprint of reported state changes to identify which cached judgments to refresh. Reachability and deterministic selection are maintained separately, allowing a target to change without unnecessary semantic evaluation. Under the fixed-evaluator and conservative-coverage assumptions of Section IV, MIND preserves the decisions of full recomputation. Over-approximation may increase work, whereas missing coverage can leave stale judgments; the guarantee does not certify the compiler’s predictions or the evaluator’s accuracy. We contribute: • Instruction-conditioned, stage-specific dependency signatures combined with region-scoped event footprints, separating semantic refresh, reselection, and reuse. • Sufficient conditions under which this maintenance preserves full-recomputation decisions, with semantic cost as initialization plus later regional refreshes. • On 240240 five-event ProcTHOR sequences, mean semantic evaluations fall from 30.630.6 to 11.211.2 with all 1,2001,200 decisions unchanged; compiler, footprint, and interface tests examine failures outside the guarantee, 3232 closed-loop AI2-THOR episodes test maintenance during execution, and 2626 Unitree G1 runs over four scenarios exercise reuse, reselection, global refresh, and abstention on hardware. I Related Work Grounded planning and dynamic revision. Language-conditioned robots ground instructions in affordances and scene graphs [5, 6, 7], revise plans from feedback [8, 9, 10], and abstain or relax infeasible goals [11, 12, 13]. Other systems maintain contextual or task memory for replanning [14, 15, 16, 17]; Event-Grounding Graph represents spatial structure and events [18], while LINGO-Space updates location distributions as referring expressions arrive [19]. These approaches motivate maintaining useful world knowledge, but stored information can become stale [20]. MIND addresses the validity of cached room-qualification judgments under a persistent instruction: which judgments a world change requires the controller to refresh, and when selection alone must be reconsidered. Incremental computation and embodied reuse. Self-adjusting computation, view maintenance, build systems, and truth maintenance reuse outputs through dependencies on changed inputs [1, 2, 3, 4]. Dependencies may be declared in formal descriptions or tracked during execution. KEEP manages model-state reuse, whereas AgenticCache caches and validates planning transitions [21, 22]. MIND shares the dependency-based reuse principle but caches regional semantic verdicts rather than model states or plan transitions: instruction-conditioned signatures separate qualification inputs from selection inputs, regional footprints scope the invalidation, and the rule preserves a fixed pipeline’s decisions when both conservatively cover the required updates. Dependencies derived from language. LiP-LLM, DeComplex, and DART-LLM infer precedence among subtasks [23, 24, 25]; incremental parsing repairs grammar or lexical knowledge after parsing or planning failures [26]. LG-TAP constructs temporally compact world models through backward observation search and uses efficient graph updates to reuse grounding computations by selectively recomputing world-dependent factor features [27]. MIND applies dependency-based reuse to a different update unit: cached room-qualification verdicts supporting a persistent target decision. It infers separate qualification and selection dependencies and combines them with event footprints, yielding explicit conditions for decision-preserving regional updates under conservative estimates. Fig. 2: Architecture for “bring me the nearest mug.” (A) Once per instruction, the compiler separates variables needed to judge each room from those needed only to choose among qualifying rooms. (B) After each event, the detector reports what changed and which rooms may be affected. (C) Their intersection fixes which room judgments to recheck; position and topology changes also update reachability. (D) The controller reuses untouched judgments and reselects a reachable target whenever the feasible set is recomputed or a reported change intersects either compiled dependency slot. I Problem Formulation System and event stream. An episode fixes a natural-language instruction q and a building with regions ℛR; in our indoor experiments each region is one room. The episode then evolves through events e1:Te_1:T. Event ete_t changes the world from Xt−1X_t-1 to XtX_t, where XtX_t holds region inventories, semantic region labels, the robot’s region, interaction history, and the region-adjacency graph. We denote these state categories by =inventory,region label,position,history,topology.V=\inventory,region label,position,history,topology\. (1) We instantiate V with five state categories common to indoor grounding. Concretely, inventory is the multiset of object types present in a region, so counts matter and removing one of several chairs is a change, region label is its semantic name, position is the region the robot currently occupies rather than a metric pose, history is the set of regions already used to satisfy the instruction, and topology is the region-adjacency graph with per-doorway passability. History is therefore region-level: a request such as another chair asks for a chair in a region not yet used, not for a different chair instance within a used region, which the state does not represent. Grounding remains region-level and does not represent object coordinates or instance identities. In the graph benchmarks, nearest means fewest doorway hops in GtG_t; the closed-loop AI2-THOR experiment instead uses navigation-grid path length. The rule requires only a variable vocabulary shared by compiler, evaluator, and detector; further categories can be added without changing it. We assume their current values are available to the controller; perception is outside the present formulation. The system contains a semantic evaluator, a reachability planner, a deterministic selector with fixed tie-breaking, and a controller that maintains their results across events. A fresh semantic evaluation, the room-level semantic check, asks whether region r satisfies q in the current state, sq(r,Xt)∈0,1.s_q(r;X_t)∈\0,1\. (2) Let GtG_t denote the region-adjacency graph at time t, let ct∈ℛc_t be the robot’s current region, and let T be the number of events. The planner returns the reachable-region set Rt=Reach(Gt,ct)R_t=Reach(G_t,c_t). For any binary regional support vector z, define Feas(z,Rt):=r∈ℛ:z(r)=1∩Rt.Feas(z,R_t):=\r :z(r)=1\∩ R_t. (3) Full recomputation evaluates every region and returns the feasible set and the selected target region, or none, Ftfull(q) F_t^full(q) =Feas(sq(⋅,Xt),Rt), =Feas (s_q(·;X_t),R_t ), (4) atfull a_t^full =πq(Ftfull(q),Xt), = _q (F_t^full(q);X_t ), where the request-specific selector πq _q may rank feasible candidates using current state and returns none when none remain. Cache of room judgments and objective. The incremental controller stores S^q,t(r) S_q,t(r), its most recent semantic verdict for region r, initialized exactly as S^q,0(r)=sq(r,X0) S_q,0(r)=s_q(r;X_0) for every region. After event ete_t, a maintenance policy chooses a refresh set Ut(q)⊆ℛU_t(q) and updates S^q,t(r)=sq(r,Xt),r∈Ut(q),S^q,t−1(r),r∉Ut(q). S_q,t(r)= casess_q(r;X_t),&r∈ U_t(q),\\ S_q,t-1(r),&r∉ U_t(q). cases (5) It maintains F^t(q)=Feas(S^q,t,Rt) F_t(q)=Feas( S_q,t,R_t) and reruns πq _q whenever the feasible set or a selection input may have changed; a^t a_t is the resulting decision. For n=|ℛ|n=|R|, the semantic-evaluation cost is Csem=n+∑t=1T|Ut(q)|,C_sem=n+ _t=1^T|U_t(q)|, (6) against n(T+1)n(T+1) for full recomputation. The problem is minCsemsubject toa^t=atfullfor every t. C_sem to a_t=a_t^full\ for every t. (7) Planner and deterministic-selection work is accounted for separately. Scope. Semantic qualification is unary in the candidate region, and its inventory dependence is restricted to that region’s contents. Relations to a distinguished state-dependent region, such as the robot’s current region, remain unary in the candidate, but predicates that inspect other regions’ inventories are outside the locality assumption used below. Predicates over multiple free candidate regions, cross-candidate cardinality constraints, and future-state prediction are outside scope. IV Methodology MIND compiles dependencies once per instruction and uses event footprints to maintain cached room judgments, reachability, and target selection (see Fig. 2). IV-A Instruction dependency map The compiler (Fig. 2) maps q to D^(q)=(D^sem(q),D^sel(q)), D(q)= ( D_sem(q), D_sel(q) ), (8) The true read sets are defined counterfactually for the fixed semantic function sqs_q and selector πq _q used in the comparison. A variable v belongs to Dsem∗(q)D^*_sem(q) if changing only state category v can change sq(r,X)s_q(r;X) for some candidate region. It belongs to Dsel∗(q)D^*_sel(q) if, with the feasible set fixed, changing only v can change the selected region. The two sets need not be disjoint: a variable may affect both qualification and selection. A selection-only change requires deterministic reselection but no fresh semantic verdict. These true read sets are unavailable to the controller. Reachability is maintained separately, is refreshed whenever topology or robot position changes, and is always applied through (3). The compiler prompt defines the five categories, explains the two slots, and supplies one development example per request family. It emits two variable lists, parsed into (8), with inventory included in the semantic slot by construction. Decoding is greedy. The frozen downstream configuration compiles under K cyclic orders of the six examples and unions each slot, with K=2K=2 unless stated; these are K pre-execution consultations per instruction, never repeated after world events. Two repairs expand a signature monotonically. Sample union unions each slot over the K orders. Stage widening treats reported variables as possibly belonging to either stage. With W=D^sem∪D^selW= D_sem∪ D_sel, define widen(D^sem,D^sel)=(W,W∖inventory).widen( D_sem, D_sel)= (W,\,W \inventory\ ). (9) The selector never reads inventory. Widening repairs semantic-to-selection demotions and preserves (12) whenever the unwidened signature satisfies it, at the cost of additional semantic evaluations. When widening is evaluated, it is applied after sample union. Unless explicitly labeled as widened, the main configuration uses slot-wise union only (K=2K=2); widened configurations are reported separately. Neither operation recovers a variable absent from every compiler output. IV-B Event footprint The detector represents an observed event by E^t=(v,A^t,v), E_t=\(v, A_t,v)\, (10) where v∈v is a reported changed variable and A^t,v⊆ℛ A_t,v is its estimated invalidation scope. Reference footprints use changed rooms for inventory: one for a removal, source and destination for a relocation. Position, topology, history, and region label use ℛR whenever they change, conservatively allowing nonlocal effects on qualification. This is an invalidation convention, not a claim that every room changes physically. Detector estimates may miss or add variables or regions; Proposition 1 needs coverage of the reference footprint. IV-C Scoped update and robot target selection Intersecting (8) and (10) yields Isem(et,q)=r∈ℛ: I_sem(e_t,q)= \r : ∃(v,A^t,v)∈E^t, ∃(v, A_t,v)∈ E_t, (11) v∈D^sem(q),r∈A^t,v. v∈ D_sem(q),\ r∈ A_t,v \. The policy sets Ut(q)=Isem(et,q)U_t(q)=I_sem(e_t,q) and then: 1. refreshes planner outputs if topology or robot position was reported changed; 2. updates (5) only for r∈Ut(q)r∈ U_t(q); 3. recomputes F^t(q) F_t(q) if reachability changed or Ut(q)≠∅U_t(q)≠ ; and 4. reruns πq _q if F^t(q) F_t(q) was recomputed or a reported changed variable lies in D^sem(q)∪D^sel(q) D_sem(q)∪ D_sel(q). Only step 2 invokes the semantic evaluator. The downstream steps trigger on a report rather than only on a changed verdict: a selection dependency that the compiler promoted into the semantic slot may refresh no region and reproduce every support value, and step 4 still reruns selection because the variable was reported changed. IV-D Conservative estimates and correctness The controller receives D^(q) D(q) and E^t E_t, and either may be inexact. We say D^(q) D(q) conservatively contains D∗(q)D^*(q) when Dsem∗⊆D^semandDsel∗⊆D^sem∪D^sel.D^*_sem D_sem D^*_sel D_sem∪ D_sel. (12) Let Et∗E_t^* contain all actually changed categories, using the preceding footprint convention: changed rooms for inventory and ℛR for the other categories. Define E^t⪰Et∗ E_t E_t^* when every (v,At,v∗)∈Et∗(v,A^*_t,v)∈ E_t^* has a corresponding (v,A^t,v)∈E^t(v, A_t,v)∈ E_t with At,v∗⊆A^t,vA^*_t,v A_t,v. With the semantic evaluator and its input interface fixed, approximation errors are asymmetric. Promoting a selection dependency into the semantic stage causes unnecessary re-evaluation but still triggers reselection; demoting or omitting one can leave stale support and voids the guarantee, though it need not change the target on every event. Adding dependencies or regions preserves deterministic outputs. Proposition 1 (Decision equivalence under dual conservative containment) Assume S^q,0=sq(⋅,X0) S_q,0=s_q(·;X_0); all task-relevant changes are in V; inventory dependencies are local to the candidate region; the semantic function sqs_q is a fixed function of the state, chosen independently of D D (the reference-field interface of Section V); (12) holds; every observed footprint satisfies E^t⪰Et∗ E_t E_t^* under the stated footprint convention; and semantic evaluation, planning, and selection are deterministic. Then a^t=atfull a_t=a_t^full after every event. Proof sketch. Induct on events. The caches agree initially. If a fresh verdict sq(r,Xt)s_q(r;X_t) can differ from S^q,t−1(r) S_q,t-1(r), some variable in Dsem∗D^*_sem changed with r in its true footprint. Signature and footprint containment place r in (11), so it is refreshed; thus no stale semantic value survives and S^q,t=sq(⋅,Xt) S_q,t=s_q(·;X_t). Observed topology and position changes also refresh planner output, so F^t=Ftfull F_t=F_t^full. Finally, any changed variable in Dsel∗D^*_sel lies in D^sem∪D^sel D_sem∪ D_sel, so step 4 reruns selection whenever it is reported changed, whether or not its footprint refreshed a region. Extra reported dependencies or regions add work but cannot alter a deterministic result. □ Proposition 1 preserves a fixed evaluator’s decisions, not their task correctness. Compiled prompt fields change the evaluator itself, so agreement within that interface need not imply agreement with the reference-field evaluator; Section V defines both interfaces and Table I reports task accuracy separately. Semantic-evaluation cost. For mt=|Ut(q)|m_t=|U_t(q)|, the cost is n+∑tmtn+ _tm_t in (6); savings vanish when global refreshes give mt≈nm_t≈ n. Maintaining all cached verdicts is stronger than preserving only the target, so this is not a minimum-cost solution to (7). V Benchmarks and Experimental Setup The six-family dependency and request benchmarks use automatically checked manifests frozen by hash before model evaluation. A dependency manifest is refused if a sequence has no verdict-changing event or abstains throughout, a history sequence is not moved by its use event, a move or door event changes a verdict for a nearest request, or no event exposes the inventory-only rule; a request manifest is refused if families are unbalanced or share a signature, a minimal pair adds an edge other than the one claimed, a held-out template leaks into development, or a signature disagrees with the event instrument. These are consistency checks, not independent certification of natural-language semantics. Generators, prompts, and model configurations are in the repository11 1 https://anonymous.4open.science/r/mind-icra27. Dependency benchmark. The primary benchmark contains 240240 five-event sequences over houses from the ProcTHOR [28] test split, using 1111 distinct instructions across six request families. Inventory-only reads inventory only; named-region adds a semantic region label; current-region adds robot position to qualification; history-dependent adds the set of previously used regions; nearest-candidate places position and topology in selection; and nearest+history combines the last two. Events shut doorways, empty or relocate objects, move the robot, and consume a referent. After each event, the manifest records the reference target and the regional verdicts that actually change value. The latter define invalidation recall and precision, distinguishing missed changed verdicts from conservative extra refreshes. The 240240 sequences span 5757 houses, with 5555, 99, 2727, 5454, 4040, and 5555 sequences for the six families in the order listed. The 1111 instructions are held-out-template phrasings from the request benchmark and are disjoint from the six compiler examples; two also appear among the minimal pairs. Request benchmark. The request benchmark has 216 requests from 36 templates, six per family, and holds out whole templates rather than sentences, since a sentence split measures rewording; the frozen compiler evaluation uses the 69 requests of the held-out templates. It also includes 144 minimal pairs in which a modifier changes a specified set of stage-specific dependency edges while leaving unrelated edges unchanged. Unseen-placement benchmark (Adjacent). A seventh family tests a dependency placement absent from all six compiler demonstrations. The benchmark interprets a chair in a room next to the one I am in as a chair in a room connected to the robot’s current room by one currently passable doorway, so closing that doorway changes qualification although the rooms remain physically adjacent. The semantic dependencies are inventory, position, and topology. The judge receives passability as a state field; the compiler sees only the phrasing and the variable definitions, in which topology is reachability given doors. The benchmark contains 8080 sequences in 8080 houses, six held-out phrasings, three of which name a door or connection explicitly, and all five event types. Backbones. We evaluate Phi-4, Qwen3-8B and Qwen3-14B (microsoft/phi-4, Qwen/Qwen3-8B, Qwen/Qwen3-14B) as locally deployable models through HuggingFace Transformers, and gpt-oss-120b through a hosted OpenAI-compatible endpoint. Decoding is greedy (temperature 00 at the endpoint), Qwen3 thinking mode is disabled, compiler outputs are capped at 512512 new tokens, and judge verdicts at 88. Local weights load in bfloat16 on one L40S GPU (Transformers 5.8, PyTorch 2.10), and judge prompts run in chunks of four per forward pass. The fallible-evaluator experiments use frozen Phi-4 and Qwen3-14B verdicts; the timing study uses Phi-4. Policies. Within each evaluator-interface condition, policies share the semantic evaluator, planner, deterministic selector, and composition rule (3). They differ in the maintenance schedule, including semantic refreshes and reselection triggers. Table I compares full recomputation, a static inventory-only signature, a universal semantic signature, reference variables with and without stage placement, and the compiled signature. Reference signatures are oracle controls, not inferred baselines. The exact predicate, the affordance sets it scores, and the selector are benchmark components fixed per family: the selector returns the fewest-hop feasible region for nearest-candidate requests, breaking ties on the display name, and the alphabetically first feasible region otherwise. Every policy reruns this selector after every event, so the inventory-only row fails only through stale semantic verdicts and never through suppressed reselection. Maintenance policies receive the compiled signature and the footprint, not the family label. Semantic evaluators and prompt interfaces. An exact benchmark predicate isolates maintenance errors from semantic-model errors. Fallible-judge experiments use a frozen verdict tape: each distinct regional question is evaluated once per interface, and the resulting verdict is replayed to every policy; an unparsable generation aborts the run. The reference-field interface constructs prompts from Dsem∗(q)D^*_sem(q) and holds these fields fixed independently of the predicted signature. A prompt carries exactly one line per declared variable: the candidate region’s contents for inventory, and position, history, and adjacency as yes/no relations to that region; the tape key is the full prompt string. The compiled-field interface instead uses D^sem(q) D_sem(q) without reference fields. An omitted field can make both policies agree on an incorrect predicate, so task accuracy is evaluated separately from agreement. Phi-4 recovers the semantic slots exactly on the 1111 main dynamic instructions, making the two interfaces coincide there; the unsafe-signature and Adjacent experiments test their divergence. Metrics. For five-event sequences, episode accuracy requires all five post-event targets to match the benchmark reference. Incremental agreement measures target agreement with full recomputation under the same evaluator and interface. Semantic evaluations count regional predicate evaluations including initialization, as in (6); under a frozen tape these are logical refresh costs, not repeated inference calls. Conservative containment is (12), and semantic inflation is |D^sem|/|Dsem∗|| D_sem|/|D^*_sem|. VI Results Every number below is measured on the frozen evaluation units of Section V with the metrics defined there. Semantic-evaluation counts are reported per five-event sequence. Table I reports episode-accuracy proportions with 95%95\% Wilson intervals and semantic-evaluation means with standard deviations. Decision-agreement rates are descriptive proportions over the evaluated decisions; the observation-derived footprint experiment reports a one-sided Clopper–Pearson bound. VI-A Ablation: the same targets, fewer rooms rechecked Conservative coverage and update scope. Table I separates target accuracy from semantic cost using the exact predicate. The static inventory-only policy attains 0.4790.479 episode accuracy and 0.7030.703 invalidation recall, whereas all containing signatures preserve the 1,2001,200 decisions. Along the ablation sequence, regional footprints reduce mean evaluations from 30.630.6 to 23.423.4, request-specific variables to 14.114.1, and stage separation to 11.211.2; these are sequential differences, not order-independent contributions. Stage separation matters for the families with selection dependencies: 18.8→8.318.8→ 8.3 (nearest-candidate) and 23.5→13.223.5→ 13.2 (nearest+history). TABLE I: Maintenance ablation on 240 five-event robot episodes with the exact benchmark predicate. Episode accuracy with 95%95\% Wilson intervals; semantic evaluations per sequence, mean ± SD. policy episode acc. sem. evals inv. recall full recomputation 1.000 [.984,1][.984,1] 30.6±9.730.6± 9.7 1.000 static inventory-only 0.479 [.417,.542][.417,.542] 8.1±1.68.1± 1.6 0.703 universal semantic signature 1.000 [.984,1][.984,1] 23.4±6.423.4± 6.4 1.000 reference variables, no stage split 1.000 [.984,1][.984,1] 14.1±6.614.1± 6.6 1.000 reference stage-specific 1.000 [.984,1][.984,1] 11.2±3.611.2± 3.6 1.000 MIND, Phi-4 (K=2K=2) 1.000 [.984,1][.984,1] 11.2±3.611.2± 3.6 1.000 Compile-once signatures outperform event-wise relevance decisions. The compiler-free alternative shows the instruction and each event to a language model and asks whether recomputation is needed (none, selection, or semantic), with the compiler’s six examples and stage definitions in the prompt; everything else is shared. Compile-once signatures reach 1.0001.000 episode accuracy at 11.211.2 evaluations from 22 compiler consultations with no non-containing decision. Event-wise relevance reaches 0.6580.658 at 13.113.1 evaluations: of 1200 decisions, 184 omit a required semantic dependency and 165 add a spurious one. The Phi-4 decider is asked once per event (12001200 logical calls, 434434 distinct prompts since identical prompts are replayed) and returns different verdicts for 1818 of the 5555 instruction and event-type pairs recurring with other room names. VI-B Instruction-conditioned compilation for new requests Containment versus exact recovery. On the 6969 held-out requests (Table I), Phi-4 and Qwen3-14B produce only conservative errors across six demonstration orders. Qwen3-14B attains containment 1.000±0.0001.000± 0.000 despite exact recovery 0.454±0.0970.454± 0.097; Qwen3-8B attains 0.988±0.0190.988± 0.019 containment. gpt-oss-120b has the highest exact recovery (0.9350.935), without inflation or non-containing outputs. Sample union. For Qwen3-8B, increasing K from 11 to 33 raises containment from 0.9880.988 to 1.0001.000 and removes all non-containing outputs, but lowers exact recovery from 0.3720.372 to 0.2100.210 at 21%21\% inflation. Means are over six demonstration orders. TABLE I: Dependency-compiler evaluation on the frozen 69-instruction held-out set. compiler, K exact sig. containment inflation non-containing (total) gpt-oss-120b, K=1K=1 0.935 1.000 1.00 0 Phi-4, K=1K=1 0.727 1.000 1.04 0 Phi-4, K=2K=2 (deployed) 0.667 1.000 1.06 0 Qwen3-14B, K=1K=1 0.454 1.000 1.08 0 Qwen3-14B, K=2K=2 0.353 1.000 1.11 0 Qwen3-8B, K=1K=1 0.372 0.988 1.12 5 Qwen3-8B, K=2K=2 0.273 0.998 1.17 1 Qwen3-8B, K=3K=3 0.210 1.000 1.21 0 On the 144 minimal pairs, scored by whether the variant’s signature gains exactly the modifier’s variables in the right slot (edge-correct) and nothing in the other slot (clean), Phi-4 reaches 0.8540.854 and 0.7990.799, gpt-oss-120b 0.7430.743 and 0.7290.729, Qwen3-14B 0.6810.681 and 0.4650.465, and Qwen3-8B 0.6390.639 and 0.5690.569. The hard case is the modifier that should add position and topology to selection: edge-correct for 00 of 3636 pairs under both Qwen models, which place it in the semantic slot (0.580.58 for Phi-4, 0.470.47 for gpt-oss-120b), a conservative promotion matching the no-stage-split ablation. Conservative repair on Adjacent. For Adjacent’s passable one-hop rule (Section V), only gpt-oss-120b places topology in the semantic slot across all six demonstration orders (raw containment 1.0001.000); Qwen3-14B, Phi-4, and Qwen3-8B place it in selection (0.7220.722, 0.5560.556, 0.4440.444). Stage widening (9) repairs the demotion to 1.0001.000 containment for the first two, and Qwen3-8B additionally needs a K=3K=3 union to recover omitted variables. With the exact benchmark predicate, every repaired signature reaches 1.0001.000 episode accuracy at 18.318.3 semantic evaluations, compared with 1.0001.000 at 30.430.4 for full recomputation. With a Phi-4 judge over 400400 decisions, the raw Phi-4 signature yields agreement of 0.9600.960 under the reference-field interface and 0.9530.953 under the compiled-field interface; widening restores agreement on all 400400 decisions under both interfaces. The 1.66×1.66× reduction is smaller than in the primary benchmark because position and topology affect qualification and trigger global refreshes. Full-recomputation episode accuracy remains 0.060.06 under the compiled-field interface, whose prompt fields come from the raw, unwidened Phi-4 signature, and 0.000.00 under the reference-field interface, with final-step accuracy of 0.380.38 and 0.310.31, respectively. Widening, adopted after these demotions were observed, reads no labels; on the primary benchmark it keeps every decision at 15.315.3 evaluations. TABLE I: Interface ablation: a missed dependency and its conservative repair (Phi-4 judge, 12001200 decisions). compiler signature interface agreement episode acc. evals Qwen3-8B, one sample reference 0.992 (1190/12001190/1200) 0.329 10.4 Qwen3-8B, one sample compiled 1.000 0.329 10.4 Qwen3-8B, K=2K=2 union reference 1.000 0.342 11.5 Phi-4, K=2K=2 (main) both 1.000 0.342 11.2 VI-C Effects of missed dependencies and footprints Missed dependencies and repair. A Qwen3-8B signature omits semantic history (Table I). Under the reference-field interface, stale reuse changes 10/120010/1200 decisions, reducing agreement to 0.9920.992 and episode accuracy from 0.3420.342 to 0.3290.329, while evaluations fall from 11.211.2 to 10.410.4. Under the compiled-field interface, both policies lack history and agree on all 12001200 decisions, but both score 0.3290.329 episode accuracy. The same three history episodes fail through different mechanisms: stale reuse versus an incomplete evaluator input. A K=2K=2 union restores reference-field agreement on all decisions and accuracy 0.3420.342 at 11.511.5 evaluations. Approximate event footprints. Perturbing the footprint alone over 240240 sequences with the exact predicate, over-approximation never costs correctness (agreement, accuracy, and recall stay 1.0001.000 while evaluations rise from 11.211.2 to 18.518.5 when every untouched region is added); under-approximation degrades monotonically as evaluations fall to 8.28.2, its stale-step rate tracking the fraction of partial footprints (Pearson 0.9970.997). Mislabeling the changed variable behaves the same way: acting on the detector’s top guess falls to 0.6830.683 accuracy at a 40%40\% error rate, whereas invalidating over the whole candidate set holds 1.0001.000 at 27.527.5 evaluations versus 30.630.6. VI-D Closed-loop and observation-derived robot updates Inferred maintenance matches full recomputation under fallible evaluators. With frozen Phi-4 and Qwen3-14B evaluators, inferred maintenance matches full recomputation on all 1200 post-event decisions across 240 sequences at 11.211.2 semantic evaluations versus 30.630.6. The static inventory-only rule reaches agreement 0.9580.958 and 0.9830.983, respectively. Full-recomputation episode accuracy is 0.3420.342 with Phi-4 and 0.2750.275 with Qwen3-14B. That statistic compounds evaluator error over five decisions (final-step accuracy is 0.7040.704 and 0.6380.638); inferred maintenance reproduces both figures exactly, so agreement measures maintenance fidelity, not semantic-model competence. Selective maintenance reduces semantic-evaluator runtime. On the separate 59-sequence timing subset, full recomputation averages 29.229.2 semantic evaluations and 1.44s1.44\,s of Phi-4 evaluator time per five-event sequence, with 17221722 regional consultations and 354354 batched forward passes in total. Inferred maintenance averages 10.210.2 evaluations and 0.58s0.58\,s, with 602602 consultations and 207207 batches. The measured evaluator-time reduction is 2.49×2.49× before compilation. Charging one 0.56s0.56\,s instruction compilation to each sequence gives 1.14s1.14\,s of accounted model time versus 1.44s1.44\,s for full recomputation, a 0.30s0.30\,s saving. Savings depend on how many room judgments each event invalidates; compilation is amortized only when accumulated inference-time savings exceed its cost. The wall-clock figures are specific to one local model and one benchmark horizon. Timings are fresh inference with the tape disabled, wall clock around the generation calls on one L40S in bfloat16, one run without warm-up exclusion; the 0.56s0.56\,s covers both K=2K=2 compiler consultations. These figures exclude observation acquisition, change detection, planning, and motion. Online footprints inferred from simulator state. The preceding experiments replay supplied footprints. In the closed-loop AI2-THOR [29] study, the controller instead obtains footprints by differencing consecutive simulator states. Qualification uses the exact benchmark predicate. Each episode applies five physical events in a fixed order (door closure, relocation, robot move, use, second relocation) and ends after the fifth; agreement is on the final target decision. Reachability is a breadth-first search over the live navigation grid, which a closed door splits, and nearest here is navigation-grid path length rather than doorway hops. This evaluates maintenance during execution with metadata-derived changes, not perception-driven grounding. Across 3232 episodes spanning all six families, inferred maintenance matches the full-recomputation target decisions in all 3232 episodes at 10.4710.47 semantic evaluations versus 26.0026.00, a 2.48×2.48× reduction, with no stale support entries. Event-wise relevance matches the reference on 30/3230/32 episodes at 11.9711.97 evaluations. Fig. 3: Removal localization in ProcTHOR validation house 571. (A, B) Matched views before and after mug removal. (C) The candidate footprint contains the affected bedroom, the only room refreshed here. (D) Reselection identifies the kitchen mug; the living room remains unreachable. Observation-derived removal footprints. OWLv2 [30] (google/owlv2-large-patch14-ensemble on 896×672896× 672 renders) processes matched before/after room scans (Fig. 3) for known one-object removals, one standpoint per room at four headings, every room rescanned after the event from the same standpoints. Given the removed object type T, the candidate footprint includes rooms whose detection count for T changes and rooms whose prior count reaches the conservative multiplicity threshold. The operating point (detection threshold 0.050.05, multiplicity 66), fixed on 4141 calibration events in 4141 houses, contains the affected room for all 5454 held-out houses, a disjoint index range of the same split, with a mean footprint of 2.192.19 of 4.614.61 rooms and a one-sided 95%95\% Clopper–Pearson lower bound of 0.9460.946. This evaluates affected-room localization given matched observations and a supplied event type, not event discovery; scan and detector cost is excluded. Fig. 4: Unitree G1 hardware validation. Left: the robot, with the 3D lidar and the RGB-D camera used for localization and detection. Centre: ceiling view of the four-zone laboratory, chairs in A and C and a backpack in the start zone D, with the path taken in one run of scenario 2, the nearest thing to sit on. Insets: the robot’s camera in each zone; the three moments of the run carry the target and the evaluation count from the module log. Three initial evaluations select C, one hop away, and the robot walks there and arrives. When the operator then drives it into A, the target re-ranks to A on position alone; position is a selection dependency for this request, so no room is re-judged. VI-E Physical-robot demonstration Fig. 4 shows MIND on a Unitree G1 humanoid in a four-zone laboratory: ROS 2 with lidar localization on a frozen map, Nav2 planning, a YOLO-World detector, and a 4-bit on-board judge (Qwen3-8B in scenarios 1 and 2, Phi-4 in 3 and 4). Signatures came from the deployed Phi-4 compiler, declared to the controller. Provided: zone polygons, candidate regions, and a scoring-only ground-truth inventory; observed: the robot’s region, planner reachability, and the detector’s inventory. Four scenarios were run, 2626 scored runs and 88 excluded with reasons. (1) Something to sit on, chairs in A and C, a backpack in the start room D: three evaluations selected A, and the walk through C, a transient loss and recovery of the route to A, and the arrival re-judged no room. MIND used 33 semantic evaluations and approximately 0.33s0.33\,s of judge time per run over five runs, compared with 99 evaluations and approximately 1.0s1.0\,s for full recomputation over three. (2) The nearest thing to sit on, same layout and move: the same move re-ranked the target from C to A without additional semantic evaluation under MIND, 33 against 99; in both scenarios all scored runs reached the expected target. (3) A chair in the room I am in, chairs in all three rooms: position is semantic, so every move re-judged every room under both policies and nothing was saved, as (6) predicts for a global refresh. The judge read the inventory line but not the position line, so every decision in the six runs inherited the same error while navigation and perception passed; the two policies agreed on every decision, as a fixed evaluator requires. (4) Something to sit on, then a barrier closing the route to A and the chair removed from C: the barrier refreshed reachability only and the removal re-judged one room, 44 evaluations against 1515 to 1818, and both policies abstained; one floor span was declared as wall so the barrier closed the only route. Evaluation counts per event were identical across repetitions; one full-recomputation run was excluded for a detector miss, a perception failure scored apart from the decision. The physical-robot demonstrations are also shown in the supplementary video. VII Conclusion MIND combines instruction-conditioned qualification and selection dependencies with event footprints to reduce semantic re-evaluation. Under Proposition 1’s assumptions, selective maintenance preserves full-recomputation target decisions. The evaluations separate savings from spatial scope, request-specific variables, and stage placement, and expose the consequences of missed dependencies and footprints. The controlled benchmarks assume populated semantic maps and a restricted state vocabulary; dependency labels are not independently annotated. Metadata-derived closed-loop updates and known-removal image localization test different components. Frozen verdict tapes isolate maintenance fidelity, not task correctness or online determinism. The hardware evidence is four scenarios on one layout. In the current-region case both policies inherited the same verdicts from an evaluator that misread the position field; semantic judgment is confined to one replaceable evaluator rather than special-cased in code, so a stronger judge changes the maintenance rule by nothing. Broader instruction coverage and robustness under changing evaluator outputs remain open. Acknowledgment Claude was used to assist with development and debugging of experimental software and data processing. All AI-assisted content was reviewed and validated by the authors. This research was supported by the NVIDIA Academic Grant Program using NVIDIA RTX PRO 6000 Blackwell GPUs and by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2026S1A5A8012989).. References [1] U. A. Acar, G. E. Blelloch, and R. Harper (2006) Adaptive functional programming. ACM Transactions on Programming Languages and Systems 28 (6), p. 990–1034. Cited by: §I, §I. [2] A. Gupta and I. S. Mumick (1995) Maintenance of materialized views: problems, techniques, and applications. IEEE Data Engineering Bulletin 18 (2), p. 3–18. Cited by: §I, §I. [3] A. Mokhov, N. Mitchell, and S. Peyton Jones (2018) Build systems à la carte. In Proceedings of the ACM on Programming Languages (ICFP), Vol. 2, p. 79:1–79:29. Cited by: §I, §I. [4] J. Doyle (1979) A truth maintenance system. Artificial Intelligence 12 (3), p. 231–272. Cited by: §I, §I. [5] M. Ahn et al. (2022) Do as i can, not as i say: grounding language in robotic affordances. In Proceedings of the 6th Conference on Robot Learning (CoRL), Proceedings of Machine Learning Research, Vol. 205, p. 287–318. Note: arXiv:2204.01691 Cited by: §I. [6] W. Huang et al. (2023) Grounded decoding: guiding text generation with grounded models for embodied agents. In Advances in Neural Information Processing Systems (NeurIPS), Note: arXiv:2303.00855 Cited by: §I. [7] K. Lin, C. Agia, T. Migimatsu, M. Pavone, and J. Bohg (2023) Text2Motion: from natural language instructions to feasible plans. Autonomous Robots 47 (8), p. 1345–1365. Note: Special issue on Large Language Models in Robotics; arXiv:2303.12153 External Links: Document Cited by: §I. [8] W. Huang et al. (2022) Inner monologue: embodied reasoning through planning with language models. In Proceedings of the 6th Conference on Robot Learning (CoRL), Proceedings of Machine Learning Research, Vol. 205, p. 1769–1782. Note: arXiv:2207.05608 Cited by: §I. [9] S. Yao et al. (2023) ReAct: synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), Note: arXiv:2210.03629 Cited by: §I. [10] K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. Reid, and N. Sünderhauf (2023) SayPlan: grounding large language models using 3D scene graphs for scalable robot task planning. In Proceedings of the 7th Conference on Robot Learning (CoRL), Proceedings of Machine Learning Research, Vol. 229, p. 23–72. Note: Oral presentation; arXiv:2307.06135 Cited by: §I. [11] A. Z. Ren et al. (2023) Robots that ask for help: uncertainty alignment for large language model planners. In Proceedings of the 7th Conference on Robot Learning (CoRL), Proceedings of Machine Learning Research, Vol. 229, p. 661–682. Note: arXiv:2307.01928 Cited by: §I. [12] J. Park et al. (2024) CLARA: classifying and disambiguating user commands for reliable interactive robotic agents. IEEE Robotics and Automation Letters 9 (2), p. 1059–1066. Note: arXiv:2306.10376 External Links: Document Cited by: §I. [13] E. Musumeci et al. (2025) Context matters! relaxing goals with LLMs for feasible 3D scene planning. arXiv preprint arXiv:2506.15828. Cited by: §I. [14] H. Yang, Z. Jiao, S. Wang, Y. Niu, S. Liu, and H. Liu (2026) Integrated exploration and sequential manipulation on scene graph with LLM-based situated replanning. In arXiv preprint, Note: arXiv:2602.04419 Cited by: §I. [15] G. Kim, T. Kim, S. S. Kannan, V. L. N. Venkatesh, D. Kim, and B. Min (2023) DynaCon: Dynamic Robot Planner with Contextual Awareness via LLMs. arXiv preprint. Note: arXiv:2309.16031 Cited by: §I. [16] M. Lei et al. (2025) STMA: A Spatio-Temporal Memory Agent for Long-Horizon Embodied Task Planning. arXiv preprint. Note: arXiv:2502.10177 Cited by: §I. [17] H. Xu, Z. He, H. Wang, J. Xu, and H. Dong (2026) Mimir: a neuro-symbolic memory system with dynamic grounding for embodied agents in interactive environments. In arXiv preprint, Note: arXiv:2608.04933 Cited by: §I. [18] P. Nguyen, F. Verdoja, and V. Kyrki (2026) Event-grounding graph: unified spatio-temporal scene graph from robotic observations. IEEE Robotics and Automation Letters 11 (5), p. 5286–5293. Cited by: §I. [19] D. Kim, N. Oh, D. Hwang, and D. Park (2024) LINGO-Space: language-conditioned incremental grounding for space. In Proc. AAAI Conf. Artificial Intelligence, Cited by: §I. [20] Y. Sun and Y. Zhang (2026) When memory lies: an empirical study of spatial memory staleness in VLM agents. arXiv preprint. Note: arXiv:2608.04574 Cited by: §I. [21] Z. Yang, T. Xie, B. Lu, S. Liu, B. Yu, and M. Li (2026) KEEP: a KV-cache-centric memory management system for efficient embodied planning. In arXiv preprint, Note: arXiv:2602.23592 Cited by: §I. [22] H. Kim, Y. Wu, and T. Tambe (2026) AgenticCache: cache-driven asynchronous planning for embodied AI agents. In Proceedings of Machine Learning and Systems (MLSys), Note: code at github.com/hojoonleokim/MLSys26_AgenticCache Cited by: §I. [23] K. Obata, T. Aoki, T. Horii, T. Taniguchi, and T. Nagai (2024) LiP-LLM: integrating linear programming and dependency graph with large language models for multi-robot task planning. In arXiv preprint, Note: arXiv:2410.21040 Cited by: §I. [24] P. Pramanick, H. B. Barua, and C. Sarkar (2020) DeComplex: task planning from complex natural instructions by a collocating robot. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Note: arXiv:2008.10084 Cited by: §I. [25] Y. Wang et al. (2024) DART-LLM: Dependency-Aware Multi-Robot Task Decomposition and Execution using Large Language Models. arXiv preprint. Note: arXiv:2411.09022 Cited by: §I. [26] M. Abrams, T. Oelerich, and M. Scheutz (2025) LLM-Augmented Incremental Language Parsing for Robotic Agents. In Proceedings of the Twelfth Annual Conference on Advances in Cognitive Systems, p. 194–209. External Links: Document Cited by: §I. [27] S. Patki, J. Arkin, N. Raicevic, and T. M. Howard (2023) Language guided temporally adaptive perception for efficient natural language grounding in cluttered dynamic worlds. In IROS, Cited by: §I. [28] M. Deitke et al. (2022) ProcTHOR: large-scale embodied AI using procedural generation. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 35, p. 5982–5994. Note: arXiv:2206.06994 Cited by: §V. [29] E. Kolve et al. (2017) AI2-THOR: an interactive 3D environment for visual AI. arXiv preprint arXiv:1712.05474. Cited by: §VI-D. [30] M. Minderer, A. Gritsenko, and N. Houlsby (2023) Scaling open-vocabulary object detection. In Advances in Neural Information Processing Systems, Vol. 36. Cited by: §VI-D.