Paper deep dive
MemBodied: Recurrent Associative Memory for Vision-Language-Action Models
Tej Deep Pala, Navonil Majumder, Bryce Goh, Raphael Yee, Jianfei Yang, Liming Chen, Soujanya Poria
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/26/2026, 3:41:04 AM
Summary
The paper introduces MemBodied, a fixed-size episodic memory mechanism for Vision-Language-Action (VLA) models to address history-dependent manipulation tasks. It utilizes an associative state to record interactions via a gated delta rule and an episode anchor to preserve initial scene information. MemBodied significantly outperforms stateless and vanilla recurrent baselines on RMBench and LIBERO-Long benchmarks while reducing inference latency and parameter count.
Entities (8)
Relation Signals (7)
MemBodied → evaluatedon → RMBench
confidence 95% · Across five evaluated RMBench tasks requiring memory
MemBodied → evaluatedon → LIBERO-LONG
confidence 95% · On the fully observable LIBERO-Long suite, it reached 90.6%
MemBodied → usescomponent → Episode Anchor
confidence 95% · MemBodied... with two complementary components: an associative state... and an episode anchor
MemBodied → usescomponent → Associative State
confidence 95% · MemBodied... with two complementary components: an associative state... and an episode anchor
MemBodied → outperforms → NativeMEM
confidence 90% · outperforming the strongest memory-augmented baseline [NativeMEM] by 1.3x
MemBodied → outperforms → π 0
confidence 90% · MemBodied achieves 7.81x the mean success rate of a stateless policy [π0]
Associative State → updatedby → Gated Delta Rule
confidence 90% · The associative key-value state is then updated using a gated delta rule
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Vision-Language-Action models provide a strong foundation for general-purpose robot control, yet a vast majority of policies do not preserve and leverage episode-level information beyond the current observation. This limitation is consequential in history-dependent manipulation tasks that depend on information available only in past observations. Retaining past observations in context can aid in recovering this information, but at the significant cost of ever-growing, bloated context and inference latency. We thus introduce MemBodied, a fixed-size episodic memory with two complementary components: an associative state that records interactions across policy calls and an episode anchor that preserves a compact representation of the initial scene as a reference. At each policy call, the model conditions action generation on the current input and the memory components, rather than directly using past observations. Across five evaluated RMBench tasks requiring memory, MemBodied achieves $7.81\times$ the mean success rate of a stateless policy and $2.98\times$ of vanilla recurrent memory, while outperforming the strongest memory-augmented baseline by $1.3\times$ with $10\times$ fewer added parameters. On the fully observable LIBERO-Long suite, it reached 90.6%, a 5.4% improvement over the stateless $\pi_0$ policy. These findings support MemBodied as a practical alternative to expanding the policy context for history-dependent manipulation.
Tags
Links
- Source: https://arxiv.org/abs/2609.28256v1
- Canonical: https://arxiv.org/abs/2609.28256v1
Trouble viewing inline? Open PDF directly →
Full Text
73,904 characters extracted from source content.
Expand or collapse full text
MEMBODIED: RECURRENT ASSOCIATIVE MEMORY FOR VISION-LANGUAGE-ACTION MODELS Tej Deep Pala 1∗ Navonil Majumder 1 Bryce Goh 2 Raphael Yee 2 Jianfei Yang 1 Liming Chen 3 Soujanya Poria 1 1 Nanyang Technological University 2 Griffin Labs 3 ́ Ecole Centrale de Lyon § Code: https://github.com/declare-lab/MemBodied Project Website: https://declare-lab.github.io/MemBodied ABSTRACT Vision-Language-Action models provide a strong foundation for general-purpose robot control, yet a vast majority of policies do not preserve and leverage episode- level information beyond current observation. This limitation is consequential in history-dependent manipulation tasks that depend on information available only in past observations. Retaining past observations in context can aid in recovering this information, but at the significant cost of ever-growing bloated context and inference latency. We thus introduce MEMBODIED, a fixed-size episodic memory with two complementary components: an associative state that records interac- tions across policy calls and an episode anchor that preserves a compact represen- tation of the initial scene as a reference. At each policy call, the model conditions action generation on the current input and the memory components, rather than directly using past observations. Across five evaluated RMBench tasks requir- ing memory, MEMBODIED achieves 7.81× the mean success rate of a stateless policy and 2.98× of vanilla recurrent memory, while outperforming the strongest memory-augmented baseline by 1.3× with 10× fewer added parameters. On the fully observable LIBERO-Long suite, it reached 90.6%, a 5.4% improvement over the stateless π 0 policy. These findings support MEMBODIED as a practical alter- native to expanding the policy context for history-dependent manipulation. 1INTRODUCTION Vision-language-action (VLA) models are increasingly a mainstay for general robot control as they map semantic representations learned from large vision-language datasets to continuous or token- based actions. RT-2 and OpenVLA formulate actions within vision-language backbones (Brohan et al., 2023; Kim et al., 2025b); Octo (Team et al., 2024) learns a diffusion-based generalist policy from heterogeneous robot data; and π 0 couples a pretrained vision-language model to a continuous flow-matching action expert (Black et al., 2024). More recent models, such as OpenVLA-OFT (Kim et al., 2025a) and π 0.5 (Intelligence et al., 2025), have improved control efficiency, action generation, and open-world generalisation. Together, these developments have made VLA policies increasingly capable at complex manipulation tasks. However, many existing VLA policies condition each policy call on the current observation and the language instruction to predict future action chunks. These approaches discard past information, limiting the policy’s ability to complete memory-dependent manipulation tasks. Although action chunking promotes consistency within a predicted horizon, it does not preserve observational infor- mation across subsequent policy calls. For example, if a robot moves an object away from its initial position and must later return the object, the current observation may no longer reveal the correct destination. This is a temporal state aliasing problem, where identical or similar current observations may require different actions depending on the past observations. ∗ Corresponding author: tejdeep001@e.ntu.edu.sg 1 arXiv:2609.28256v1 [cs.RO] 23 Sep 2026 Although preserving history is desirable, placing past observations directly into the policy context is costly. Retaining the complete history causes the context length, memory consumption, and infer- ence cost to grow throughout an episode. A fixed observation window bounds these costs but may discard task-relevant evidence once it falls outside the window. Moreover, the relevant evidence may be a short event separated from the current decision by many intermediate observations. An effective memory must therefore preserve relevant information without growing in size or inference cost over the episode. Recent work addresses this problem by compressing historical observations (Koo et al., 2026; Wang et al., 2026), maintaining recurrent latent representations (Cherepanov et al., 2026b; Li et al., 2026; Guo et al., 2026), or explicitly storing and retrieving historical information (Fang et al., 2025; Shi et al., 2026; Li et al., 2025; Haresh et al., 2026). Methods that retain compressed observations as history tokens can still enlarge the policy context as new information is added. Recurrent-token methods keep the recurrent state fixed in size, but rely on carried embeddings to implicitly pre- serve and expose relevant information through repeated transformations. Memory-bank, retrieval, and language-based approaches may instead introduce auxiliary objectives or additional processing pipelines. These considerations motivate a memory that is updated online, remains fixed in size, and retrieves information according to the policy’s current state. Associative memory meets these requirements through an explicit read/write structure in which the delta rule specifies how the state is updated while the policy learns what to write, retain, and retrieve using only its action objective. In this paper, we introduce MEMBODIED, an episodic memory for VLA control designed to preserve both interaction history and initial-scene information. Its associative memory stores interactions in layer-wise matrices through learned read and write interfaces. After each policy call, the model combines the executed action with its observed visual consequence and writes this interaction using a gated delta update. Because repeated updates may overwrite fine-grained details from early in the episode, the anchor pathway provides persistent access to a compact representation of the initial scene. At subsequent calls, information retrieved from both memory mechanisms conditions the action network. This division of roles allows past interactions and initial-scene evidence to inform action prediction while keeping the memory footprint and access cost independent of episode length. We address three research questions in this paper: RQ1: How effective is MEMBODIED for memory-dependent manipulation compared with existing approaches? RQ2: How computation- ally efficient is MEMBODIED during inference? RQ3: Does MEMBODIED preserve performance on fully observable or Markovian manipulation where memory is not required? Across five memory-dependent RMBench tasks, MEMBODIED achieves 7.81× the mean success rate of a stateless π 0 policy and 2.98× that of vanilla recurrent memory, outperforming Native- MEM’s compressed-history approach by 1.30×. Evaluated on a second VLA backbone (π 0.5 ), MEMBODIED yields a 3.87× performance increase over the stateless policy. On physical robot experiments across three tasks, MEMBODIED demonstrates an 8.0× improvement in success rate. Crucially, MEMBODIED achieves these gains while reducing inference latency by 91.9% relative to NativeMEM. On LIBERO, MEMBODIED achieved 95.1% mean success rate, comparable to π 0 ’s result of 94.2%. Notably, on the long-horizon LIBERO-Long suite, MEMBODIED reached 90.6%, exceeding the π 0 result of 85.2% by 5.4 percentage points. These results demonstrate that MEMBODIED improves history-dependent control while remaining competitive on fully observable manipulation tasks. 2RELATED WORK Memory representations for manipulation. HAMLET (Koo et al., 2026) and Native- MEM (Wang et al., 2026) compress observations into historical tokens. μVLA (Cherepanov et al., 2026b), ReMem-VLA (Li et al., 2026), and AVA-VLA (Xiao et al., 2026) carry recurrent latent states across control steps. SAM2Act (Fang et al., 2025), MemoryVLA (Shi et al., 2026), MAP-VLA (Li et al., 2025), and Notes-to-Self (Haresh et al., 2026) use memory banks, retrieval, or language-based records. These approaches trade off explicit access to past evidence against the cost of retaining history or introducing additional memory components. MEMBODIED instead represents episode history through an evolving associative state and an initial-scene anchor, whose readouts condition the action expert. It therefore neither appends a growing sequence of observations nor retrieves discrete past observations. We discuss these and other related work in more detail in Appendix A. 2 Associative readVLA action expertEpisode anchor Transition-conditioned write Is ℓ t ,, h t W q W g tanh → L₂ norm σ q t g t × S t − 1 r t W m αr mem / × g m t rSq t = − 1 h t x mem,t e t a h, + f cond statememoryH actions Self-attention a t I 1 16×16 → 4×4 / camera frozen A fixed zI t = pool( ) W q A W k A W v A rQKV t A = At t n ( ,,) bWzWr tAt o A t A =(+) after execution h t,out y t [;]hy t,out W k W v W α β , tanh → L₂ norm σ k t v t α β t , (,,)a t Is + 1 + 1 state-query pooling yea t t h th = [;] v + isi 1 on , ∑ S t − 1 Δ S t next call One layer shown · Fixed r×r stateEpisode start: learned S₀ · A rebuilt from I₁ Figure 1: MEMBODIED architecture. An associative state records interactions, while the episode anchor preserves a reference to the initial scene. 3METHODOLOGY MEMBODIED comprises two complementary memory pathways: a recurrent associative state and a fixed visual reference to the initial scene. We first describe the associative update and memory- token readout, followed by the anchor pathway, read/write schedule, and sequence training. We also evaluate attention-steering and hierarchical memory variants for comparison. 3.1RECURRENT VLA FORMULATION We treat manipulation as a history-dependent sequential decision problem where information from prior observations might be necessary to complete the task. We denote the complete episodic mem- ory byE t = (M t ,A), whereM t is the recurrent associative state and A is the episode anchor. At interaction step t, the policy receives current camera observations I t , robot state s t , and language instruction ℓ to predict an action chunka t up to horizon H : a t = [a t,1 ,...,a t,H ]∼ p θ (·| I t ,s t ,ℓ,E t−1 ). The associative state M t−1 persists across policy calls and is reset to a learned initial state at the start of each episode. The anchor A is constructed from the episode’s first observation and remains unchanged until the episode ends. 3.2ASSOCIATIVE-MEMORY MECHANISM The memory state comprises one associative matrix S (l) t for each of the L action-network layers. For batch size B and memory rank r, M t =S (l) t L l=1 , S (l) t ∈R B×r×r . In our experiments, we used r = 128. The state size remains fixed through the episode, i.e., writing new information in memory updates the existing associative matrices rather than appending obser- vations to a growing history. 3.2.1WRITE-VALUE CONSTRUCTION For an interaction event, we construct the value to be written from the action chunka t and its corresponding observed visual consequence, post-execution, in I t+1 . For each camera c, a query derived from the projected robot state s t+1 attends to the corresponding visual patch tokens X (c) t+1 : e (c) t+1 = softmax (W pq φ s (s t+1 ))(W pk X (c) t+1 ) ⊤ √ D ! W pv X (c) t+1 3 The camera-specific observation encodings e (c) t+1 are pooled independently and then averaged: e vision t+1 = 1 C C X c=1 e (c) t+1 . Note that, the gradients from the memory-value path are stopped at the visual patch tokens. The pooled next-observation encoding is concatenated with the aggregate of its preceding causal action chunk: y t = " e vision t+1 ; H X h=1 a t,h # , v (l) t = W (l) v y t . The resulting representation y t combines the action summary and observed visual consequence, and each layer-specific projection v (l) t determines the value written to that layer’s associative state. This choice allows the memory value to represent a dynamic interactive transition rather than an isolated observation. 3.2.2GATED DELTA-RULE WRITE To construct the write key at layer l, we transform the post-layer state representation h (l) t,out : k (l) t = L 2 Norm tanh(W (l) k h (l) t,out ) . The write and retention gate values are then computed from the layer output and write value: β (l) t = σ(W (l) β [h (l) t,out ;y t ]), α (l) t = σ(W (l) α [h (l) t,out ;y t ]). The associative key-value state is then updated using a gated delta rule (Yang et al., 2025) S (l) t = Diag(α (l) t )S (l) t−1 + Diag(β (l) t ) v (l) t − Diag(α (l) t )S (l) t−1 k (l) t (k (l) t ) ⊤ . The first term retains the existing state, while the second term applies a delta correction by com- paring the new value with the value already associated with the write key. The retention gate α (l) t controls how much of the existing state remains, while the write gate β (l) t controls the strength of the correction. The update could therefore revise an existing association without indiscriminately accumulating every value. 3.3ASSOCIATIVE RETRIEVAL AND MEMORY VARIANTS At each layer l, the relevant memory readout r (l) t is retrieved from the respective associative matrix S (l) t−1 using a query q (l) t constructed from the state token representation entering the layer, h (l) t,in : q (l) t = L 2 Norm tanh(W (l) q h (l) t,in ) , r (l) t = S (l) t−1 q (l) t . We compare the following three variants of conditioning the action network on readout r (l) t : 1. Attention steering (MEMBODIED-AS): Projections of the associative readout produce additive corrections to the attention query and output representations of the action network. 2. Hierarchical memory (MEMBODIED-H): An LSTM-like recurrent cell is updated from the associative matrices controlled by a learned forget gate and added back, augmenting the base gated-delta update. 3. Memory-token readout (MEMBODIED): The associative readout is projected to the width of the action network and added to a dedicated contextual token before self-attention. 4 3.3.1MEMBODIED: MEMORY-TOKEN INJECTION In MEMBODIED, the action-network suffix is arranged as [state token, memory token, H noisy action tokens]. The memory token is a learned vector that is inserted into the suffix at every policy call. The memory token does not persist across policy calls. Instead, the associative matrices carry the recurrent state. At layer l, the retrieved vector is linearly transformed and scaled by α mem /r, where α mem is the memory scaling factor and r is the memory rank. A scalar gate derived from the current state representation then controls its contribution: x (l) mem,t ← x (l) mem,t + g (l) t m (l) t , where m (l) t = α mem r W (l) m r (l) t , g (l) t = σ(W (l) g h (l) t,in ) The modified memory token, x (l) mem,t , then participates in self-attention, allowing the action tokens to use the retrieved content. Unlike attention steering, this approach makes the retrieved vector available as contextual content rather than using it only to alter existing attention computations. 3.4FIXED EPISODE-ANCHOR MEMORY The associative matrices record how an episode evolves, but repeated delta-rule updates can over- write fine-grained details of the initial scene. We therefore use a separate anchor pathway to preserve a compact, fixed visual reference derived from the episode’s first observation. To construct this an- chor, we take the frozen vision tokens used in the policy prefix, average-pool each camera’s 16× 16 patch grid into a 4×4 grid, and concatenate the pooled tokens across cameras. The resulting anchor A∈R N A ×D provides access to the initial scene without retaining the raw observation. At policy call t, the current robot state s t attends to each camera’s visual tokens X (c) t using the pooling operation defined in the write-value construction. We average the camera-specific outputs to obtain z t , which queries the anchor through rank-r A cross-attention: c t = z t + W A o " softmax (W A q z t )(W A k A) ⊤ √ r A ! W A v A # . c t is then projected to the action-network width and repeated across the action horizon: b t = W A c t , b t,h = b t , h = 1,...,H. Let e a t,h denote the action representation at horizon position h. The anchor-conditioned representa- tion is computed as e a t,h = f cond e a t,h ; b t,h , h = 1,...,H, where f cond denotes the action network’s input-conditioning function. For our experiments, we used r A = 64. The anchor remained fixed within an episode and reset at the start of a new episode. The anchor does not append the first image or its complete token sequence to the policy prefix. Instead, the current observation selectively retrieves a compact spatial reference through cross- attention, leaving the prefix length, token positions, and attention-mask structure unchanged. 3.5CAUSAL COORDINATION OF READING AND WRITING At step t, the policy first reads M t−1 and generatesa t . After the action chunk is executed, the environment supplies the resultant visual observation, I t+1 . The model then associates the hidden state and action chunk from step t with I t+1 and writes the association toM t : M t−1 read with (I t ,s t ,ℓ) −→a t execute −→ I t+1 write −→M t . This delayed-write schedule allows the stored value to include the observed consequence of an action without exposing that future observation to the action that caused it. During inference, the policy caches the preceding layer states and the sampled action chunk from the final denoising step, and performs the write when the next observation arrives. Thus, training and inference use the same causal interpretation of an interaction event. 5 3.6SEQUENCE TRAINING The delayed write makes sequence-level training necessary. An action loss at a later policy call must propagate through memory operations performed at earlier calls to train the memory parame- ters. Each training sample comprises a sequence of N observation-action pairs. Consecutive pairs are separated by one action horizon of H environment steps, approximating successive policy calls during execution. Image encoding and policy-input construction are parallelised across the batch and sequence dimensions, while the memory state is propagated sequentially through the sequence. Gradients pass through the full sequence, allowing later action losses to optimise earlier memory operations. The memory parameters are thus learned jointly through the policy’s native action ob- jective, without a separate memory-prediction loss. 4EXPERIMENTAL SETUP We evaluate MEMBODIED on memory-dependent manipulation using RMBench (Chen et al., 2026) with the π 0 and π 0.5 backbones, and on three real-robot tasks. We also use LIBERO (Liu et al., 2023) to assess whether MEMBODIED preserves performance on fully observable manipulation tasks. 4.1BENCHMARKS AND EVALUATION PROTOCOL RMBench (Chen et al., 2026) is a bimanual simulation benchmark that organises tasks by mem- ory complexity. Its M(1)-category of tasks require retention of task-relevant observations from a single past event, whereas the M(n) tasks require information from multiple past events. Our five-task evaluation covered three M(1) tasks: put backblock, rearrangeblocks, and swapblocks; and two M(n) tasks: batterytry and blockrankingtry. We conducted 50 rollouts per task and report each task’s success rate and the mean across the five tasks. As a complementary general manipulation evaluation, we use the four standard LIBERO suites: LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, and LIBERO-Long. Each suite contains ten tasks, and we conduct 50 rollouts per task, yielding 500 rollouts per suite and 2,000 rollouts per checkpoint. We report the mean success rate for each suite and the unweighted mean across all four suites. On both benchmarks, memory is reset at the start of each rollout and persists only for the duration of that episode. 4.2BASELINES Our RMBench-centered comparison includes π 0 -based baselines, published benchmark results, and variants of our method. The locally trained policies share the same π 0 backbone, LoRA adaptation settings, and number of optimiser steps. These controls align the adaptation recipe and training-step budget, but do not imply identical architectures or compute costs. The full-fine-tuning LIBERO comparisons are reported separately. π 0 -Stateless takes only the current observation, robot state, and instruction as input, essentially a memory-free baseline. π 0 -FrameStack additionally receives a sequence of past observations and serves as an in-context history baseline. π 0 -Hint augments the stateless policy with simulator- derived textual hints on task progression at inference time, without any training on those hints; it is therefore a privileged-information reference. π 0 -Vanilla Recurrent Memory adapts the associative memory mechanism of Delta-Mem (Lei et al., 2026), whileπ 0 -μ-VLA adapts the recurrent memory- token mechanism of μVLA (Cherepanov et al., 2026b). We report the Diffusion Policy, ACT, π 0.5 , and X-VLA results from Chen et al. (2026), and use the π 0.5 scores as a full-fine-tuned π 0.5 baseline. With reference to our method, we com- pare the attention-steering (MEMBODIED-AS), hierarchical (MEMBODIED-H), and memory-token (MEMBODIED) variants of the associative mechanism.Finally, for comparison with Native- MEM (Wang et al., 2026), we integrate its video-history encoding into our setup and evaluate it both independently and in combination with MEMBODIED. The full implementation and evaluation details are provided in Appendix B and C. 6 5RESULTS 5.1PERFORMANCE ON MEMORY-DEPENDENT MANIPULATION Table 1: Success rates (%) on the five evaluated RMBench tasks. Published policy values are taken from Chen et al. (2026). The bold figures indicate the best result for each task/column. Model Put back block Rearrange blocks Swap blocks Battery try Block ranking Mean Published policies (Chen et al., 2026) Diffusion Policy0.00.011.010.010.06.2 ACT0.029.02.019.00.010.0 X-VLA18.013.016.026.01.014.8 π 0.5 11.013.016.016.06.012.4 Baselines in our setup (π 0 backbone) π 0 -Stateless6.02.08.016.00.06.4 π 0 -Hint10.00.00.00.00.02.0 π 0 -μ-VLA0.00.00.00.00.00.0 π 0 -FrameStack14.030.00.04.026.014.8 π 0 -Vanilla Recurrent Memory18.022.014.022.08.016.8 MEMBODIED-AS32.026.012.020.014.020.8 MEMBODIED-H16.050.014.028.08.023.2 MEMBODIED w/o anchor memory34.076.016.030.032.037.6 MEMBODIED40.092.056.040.022.050.0 Recurrent memory and its formulation both affect performance. To assess the effectiveness of MEMBODIED for memory-dependent manipulation (RQ1), we compare it with baselines and MEM- BODIED variants across five RMBench tasks. As shown in Table 1, MEMBODIED reaches 50.0% mean success, exceeding π 0 -Stateless by 43.6 percentage points and π 0 -FrameStack by 35.2 points; its advantage over the stateless policy is positive across all five tasks. These comparisons support the value of maintaining a recurrent memory across policy calls rather than ignoring history or pre- senting a bounded history directly in context. MEMBODIED also surpasses π 0 -Vanilla Recurrent Memory by 33.2 points; as both policies maintain a recurrent state, this comparison indicates the merit of our proposed memory formulation beyond recurrence alone. 020406080100 Success rate (%) Put back block Rearrange blocks Swap blocks Battery try Block ranking 52 38 86 16 0 70 40 82 34 0 40 92 56 40 22 NativeMEM NativeMEM + MemBodied MemBodied (a) Success on each RMBench task. 08001600 NativeMEM MemBodied 1594 129 Inference latency (ms) 01020 NativeMEM MemBodied 20.6 18.6 Peak GPU memory (GiB) (b) Inference Efficiency Comparison. Figure 2: NativeMEM and MEMBODIED in the same evaluation setup. We evaluate NativeMEM’s video- history encoding alone and combined with MEMBODIED, alongside standalone MEMBODIED. Standalone MEMBODIED exceeds NativeMEM by 11.6 percentage points on average. Comparison with compressed history methods. Following Fig. 2, even with this more granular access to history, NativeMEM reaches 38.4% mean success, compared with 50.0% for standalone MEMBODIED. Adding MEMBODIED to NativeMEM improves its mean to 45.2%, but remains 4.8% below standalone MEMBODIED. The gain from adding MEMBODIED to NativeMEM could reflect more focused memory retrieval by MEMBODIED. However, NativeMEM’s history tokens could also draw attention away from the information retrieved by MEMBODIED, potentially explaining why the combined model performs worse than standalone MEMBODIED. Exploring how to combine explicit video history with associative memory more effectively is a direction for future work. However, as seen in Fig. 2a, no single method dominates at the task level. NativeMEM is stronger than standalone MEMBODIED on Put Back Block and Swap Blocks, whereas MEMBODIED is 7 stronger on Rearrange Blocks, Battery Try, and Block Ranking. The combined model obtains the highest Put Back Block rate, but does not surpass the strongest standalone methods on the other four tasks. These results favour MEMBODIED in the five-task mean success rate while showing that compressed history and associative memory have complementary, task-dependent strengths. 020406080100 Success rate (%) Put back block Rearrange blocks Swap blocks Battery try Block ranking Mean 11 13 16 16 6 12.4 38 94 50 38 20 48.0 π 0.5 FFT MemBodied (π 0.5 ) (a) π 0.5 FFT baseline and MEMBODIED. Put back block Rearrange blocks Swap blocks 0 20 40 60 80 100 Success rate (%) 1/201/20 0/20 7/207/20 2/20 StatelessMemBodied (b) Real robot Evaluation. Whiskers are marginal 95% Wilson binomial confidence intervals. Figure 3: Additional backbone and real-robot memory tasks. The gains transfer to π 0.5 backbone. With π 0.5 backbone (Fig. 3a), MEMBODIED reaches 48.0% mean success as compared to 12.4% for the π 0.5 baseline, improving on all five evaluated tasks. The largest gain is achieved on Rearrange Blocks, from 13.0% to 94.0%; Put Back Block improves from 11.0% to 38.0%. MEMBODIED improves real-robot performance. As shown in Fig. 3b, MEMBODIED increases mean success across three real-robot tasks from 3.33% to 26.67%, with improvements on every task. 5.2INFERENCE EFFICIENCY As shown in Fig. 2b, MEMBODIED achieves 91.9% lower inference latency while using 9.5% less peak GPU memory, as compared to NativeMEM. MEMBODIED also introduces substantially fewer 40M parameters (only 1.26% of π 0 ), roughly one-tenth of NativeMEM’s 415M (12.81% of π 0 ). During inference, NativeMEM encodes the incoming observations with its video-history encoder and adds the resulting tokens to the VLA context, introducing an additional encoding step and a growing context size. In contrast, MEMBODIED maintains a fixed-size recurrent state that is read during action generation and updated at the start of a new policy call. Together, these results show that MEMBODIED achieves higher mean success than NativeMEM with substantially lower inference latency, parameter overhead, and memory footprint. 5.3PERFORMANCE ON FULLY-OBSERVABLE MANIPULATION Table 2: LIBERO success rates (%). MethodSpatialObjectGoalLongMean Diffusion Policy (Chi et al., 2025)78.392.568.350.572.4 Octo (Team et al., 2024)78.985.784.651.175.1 OpenVLA (Kim et al., 2025b)84.788.479.253.775.9 SpatialVLA (Qu et al., 2025)88.289.978.655.578.1 UniACT (Zheng et al., 2025)77.087.077.070.076.8 π 0 (Black et al., 2024)96.898.895.885.294.2 π 0 -FAST (Pertsch et al., 2025)96.496.888.660.285.5 MEMBODIED96.297.895.890.695.1 The largest improvement is skewed toward long-horizon LIBERO tasks. LIBERO pro- vides a complementary setting for RQ3 by test- ing MEMBODIED on fully observable tasks where episodic recall is not required to com- plete the task. Table 2 shows that MEMBODIED reaches 90.6% success on LIBERO-Long, ex- ceeding π 0 ’s score of 85.2% by 5.4 percentage points. In contrast, Goal is unchanged, while Spatial and Object decrease by 0.6 and 1.0 points, respectively. The four-suite mean remains com- parable at 95.1% against π 0 ’s 94.2%. The skewness of the improvement toward LIBERO-Long sup- ports the view that MEMBODIED is particularly useful for improving task performance on longer fully-observable manipulation, without degrading aggregate performance. 8 6ANALYSES 020406080100 Success rate (%) Put back block Rearrange blocks Swap blocks Battery try Block ranking Mean 34 40 76 92 16 56 30 40 32 22 37.6 50 Without anchorWith anchor (a) Fixed episode anchor. Vision + action VisionAction Write-value content 0 20 40 60 80 100 Success rate (%) 34 32 18 76 72 64 30 24 22 Put back block Rearrange blocks Battery try (b) Write-value composition. Figure 4: Memory design ablations on RMBench. Memory architecture ablation. Figure 4a and Table 1 compare the associative-state variants and the contribution of the episode anchor pathway. Among the anchor-free variants, memory-token injection reaches 37.6% mean success, compared with 23.2% for hierarchical memory and 20.8% for attention steering. This indicates that feeding the retrieved vector as contextual content is more effective, in our setting, than the two alternative memory designs. The combined two-pathway memory reaches 50.0%, compared with 37.6% when the anchor pathway is removed. Relative to the anchor-free variant, the MEMBODIEDimproves on four out of five tasks, with the largest gain on Swap Blocks, from 16.0% to 56.0%. However, the anchor reduces performance on Block Ranking from 32.0% to 22.0%; this task depends more on tracking evolving task progress than recalling the initial scene, making the fixed first-frame reference potentially less useful. Overall, both the memory-token design and episode anchor contribute to the complete memory formulation, although the anchor’s effect is task-dependent. Vision and action play complementary roles through memory. Across the three evaluation tasks, Fig. 4b shows that combining the subsequent visual observation with the action summary achieves 46.7% mean success, compared with 42.7% for vision-only and 34.7% for action-only memory. On Put Back Block and Rearrange Blocks, visual information accounts for most of the measured performance, with the action summary adding 2 and 4 percentage points. On Battery Try, the combined value reaches 30.0%, compared with 24.0% for vision alone and 22.0% for ac- tion alone; removing either modality therefore reduces success by 6 or 8 points. The combined representation is the strongest on every task, indicating the complementarity of these two channels. Qualitative analysis on the impact of memory. Manual annotation of 50 seed-matched rollout pairs per task shows that MEMBODIED and π 0 -Stateless behave similarly when the current ob- servation is sufficient, but diverge at decisions requiring past information. For example, on Put Back Block, MEMBODIED reduces wrong-pad returns from 37/50 to 13/50, although both policies complete the preceding block movement and button press in every rollout (see more details in Ap- pendix E). Across tasks, MEMBODIED more often selects the episode-specific target, tracks task progress, and uses earlier outcomes to guide subsequent attempts. Its remaining failures largely involve motor execution or longer-term tracking. These observations localise MEMBODIED’s gains to decisions that require information no longer visible in the current observation. 7CONCLUSION In this work, we introduced MEMBODIED, a fixed-capacity episodic memory that combines an as- sociative memory state with an initial-scene anchor without expanding the policy context. Across five memory-dependent RMBench tasks, it achieved 50.0% mean success, outperforming stateless, frame-history, vanilla recurrent, and compressed-history baselines. The improvements also extended 9 to the π 0.5 backbone and three real-robot tasks. Importantly, the gains in memory-dependent manip- ulation do not come at the expense of inference efficiency. Compared with NativeMEM, MEMBOD- IED pairs higher mean success with substantially lower latency and memory footprint. MEMBOD- IED also preserves general manipulation ability, maintaining comparable mean success on LIBERO while improving on the long-horizon suite. Together, these results support MEMBODIED as a prac- tical alternative to expanding observation histories in VLA control. AI USE STATEMENT In this work, we used generative AI tools to provide feedback on research methodology and exper- iments, assist with method implementation, create or modify scientific figures, summarise existing literature, and draft or edit portions of the manuscript for clarity. We did not use these tools to develop theoretical models, formulate or prove mathematical claims, propose or refine hypotheses, clean or reformat datasets, generate synthetic data, or translate the manuscript. We manually checked LLM-generated code for correctness, verified literature summaries against the cited sources, and re- viewed all AI-generated text and figures. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI. REFERENCES Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π 0 : A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024. Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022. Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choroman- ski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023. Tianxing Chen, Yuran Wang, Mingleyang Li, Yan Qin, Hao Shi, Zixuan Li, Yifan Hu, Yingsheng Zhang, Kaixuan Wang, Yue Chen, et al. Rmbench: Memory-dependent robotic manipulation benchmark with insights into policy design. arXiv preprint arXiv:2603.01229, 2026. Egor Cherepanov, Nikita Kachaev, Alexey Kovalev, and Aleksandr I. Panov. Memory, benchmark & robots: A benchmark for solving complex tasks with reinforcement learning. In The Fourteenth International Conference on Learning Representations, 2026a. URL https://openreview. net/forum?id=9cLPurIZMj. Egor Cherepanov, Nikita Kachaev, Daniil Zelezetsky, Aydar Bulatov, Artem Pshenitsyn, Yuri Kura- tov, Alexey Skrynnik, Aleksandr I Panov, and Alexey K Kovalev. μ vla: On recurrent memory for partially observable manipulation in vla models. arXiv preprint arXiv:2606.12497, 2026b. Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The Inter- national Journal of Robotics Research, 44(10-11):1684–1704, 2025. Haoquan Fang, Markus Grotz, Wilbert Pumacay, Yi Ru Wang, Dieter Fox, Ranjay Krishna, and Jiafei Duan. Sam2act: Integrating visual foundation model with a memory architecture for robotic manipulation. arXiv preprint arXiv:2501.18564, 2025. Xinying Guo, Chenxi Jiang, Hyun Bin Kim, Yuhang Han, Ying Sun, Yang Xiao, and Jianfei Yang. Chameleon: Control-indexed prospective memory for visuomotor manipulation. arXiv preprint arXiv:2603.24576, 2026. Sanjay Haresh, Daniel Dijkman, Apratim Bhattacharyya, and Roland Memisevic. to-self: Scratch- pad augmented vlas for memory dependent manipulation tasks. arXiv preprint arXiv:2602.21013, 2026. 10 Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. π 0.5 : a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025. Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Opti- mizing speed and success. arXiv preprint arXiv:2502.19645, 2025a. Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Open- vla: An open-source vision-language-action model. In Pulkit Agrawal, Oliver Kroemer, and Wolfram Burgard (eds.), Proceedings of The 8th Conference on Robot Learning, volume 270 of Proceedings of Machine Learning Research, p. 2679–2713. PMLR, 06–09 Nov 2025b. URL https://proceedings.mlr.press/v270/kim25c.html. Myungkyu Koo, Daewon Choi, Taeyoung Kim, Kyungmin Lee, Changyeon Kim, Younggyo Seo, and Jinwoo Shin. Hamlet: Switch your vision-language-action model into a history-aware policy. In International Conference on Learning Representations, volume 2026, p. 101537–101558, 2026. Jingdi Lei, Di Zhang, Junxian Li, Weida Wang, Kaixuan Fan, Xiang Liu, Qihan Liu, Xiaoteng Ma, Baian Chen, and Soujanya Poria. δ-mem: Efficient online memory for large language models. arXiv preprint arXiv:2605.12357, 2026. Hang Li, Fengyi Shen, Dong Chen, Liudi Yang, Xudong Wang, Jinkui Shi, Zhenshan Bing, Ziyuan Liu, and Alois Knoll. Remem-vla: Empowering vision-language-action model with memory via dual-level recurrent queries. arXiv preprint arXiv:2603.12942, 2026. Runhao Li, Wenkai Guo, Zhenyu Wu, Changyuan Wang, Haoyuan Deng, Zhenyu Weng, Yap-Peng Tan, and Ziwei Wang. Map-vla: Memory-augmented prompting for vision-language-action model in robotic manipulation. arXiv preprint arXiv:2511.09516, 2025. Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36:44776–44791, 2023. Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025. Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, JiaYuan Gu, Bin Zhao, Dong Wang, et al. Spatialvla: Exploring spatial representations for visual- language-action model. arXiv preprint arXiv:2501.15830, 2025. Imanol Schlag, Kazuki Irie, and J ̈ urgen Schmidhuber. Linear transformers are secretly fast weight programmers. In International conference on machine learning, p. 9355–9366. PMLR, 2021. J ̈ urgen Schmidhuber. Learning to control fast-weight memories: An alternative to dynamic recurrent networks. Neural Computation, 4(1):131–139, 1992. doi: 10.1162/neco.1992.4.1.131. Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. Memoryvla: Perceptual-cognitive memory in vision-language- action models for robotic manipulation. In International Conference on Learning Representa- tions, volume 2026, p. 18567–18602, 2026. Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213, 2024. Ziye Wang, Modi Shi, Chaojun Ni, Jiazhi Yang, Mengdi Li, Zhizhong Su, Tianwei Lin, and Hongyang Li. Nativemem: Native memory compression for long-horizon robotic manipulation. arXiv preprint arXiv:2607.06678, 2026. 11 Lei Xiao, Jifeng Li, Juntao Gao, Feiyang Ye, Yan Jin, Jingjing Qian, Jing Zhang, Yong Wu, and Xiaoyuan Yu. Ava-vla: Improving vision-language-action models with active visual attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 13453–13463, 2026. Songlin Yang, Jan Kautz, and Ali Hatamizadeh. Gated delta networks: Improving mamba2 with delta rule. In International Conference on Learning Representations, volume 2025, p. 29687– 29707, 2025. Jinliang Zheng, Jianxiong Li, Dongxiu Liu, Yinan Zheng, Zhihao Wang, Zhonghong Ou, Yu Liu, Jingjing Liu, Ya-Qin Zhang, and Xianyuan Zhan. Universal actions for enhanced embodied foun- dation models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 22508–22519. IEEE, 2025. AEXTENDED RELATED WORK This section expands the review in Section 2, providing additional context on the broader literature surrounding generalist VLA policies, memory-augmented manipulation, associative memory, and memory-focused evaluation. A.1GENERALIST VLA POLICIES Vision-language-action models transfer semantic representations from pretrained vision-language models into robot control. RT-1 consumes a bounded image history and compresses visual tokens before causal processing (Brohan et al., 2022). RT-2 and OpenVLA represent robot actions as to- kens within vision-language backbones (Brohan et al., 2023; Kim et al., 2025b). Octo instead trains a diffusion-based generalist policy from heterogeneous robot trajectories (Team et al., 2024). π 0 combines a pretrained PaliGemma vision-language model with a flow-matching action expert for continuous action chunk prediction (Black et al., 2024). OpenVLA-OFT shows that continuous ac- tion representations, parallel decoding, and action chunking can improve control speed and success, while π 0.5 combines low-level control with semantic prediction and heterogeneous co-training for open-world tasks (Kim et al., 2025a; Intelligence et al., 2025). Across these architectures, temporal context is limited to a bounded observation history or an action chunk. Such local context can preserve recently visible evidence, but it does not provide a persistent episodic state for information that must be retained beyond the input window or selectively carried across an episode. A.2MEMORY REPRESENTATIONS FOR MANIPULATION Recent memory-aware policies can be distinguished by the representation they preserve. Compressed-history methods retain selected observations as tokens. HAMLET (Koo et al., 2026) encodes observations into moment tokens and aggregates them through a lightweight history mod- ule. NativeMEM (Wang et al., 2026) reuses the VLA vision encoder to compress each historical camera frame into a native memory token. These methods preserve explicit temporal evidence more efficiently than raw frame histories, although their context can still grow with the number of retained observations. Fixed-size recurrent methods compress history into a bounded latent state. μVLA carries learnable memory tokens across OpenVLA-OFT control steps and trains them with truncated backpropagation through time (Cherepanov et al., 2026b). ReMem-VLA propagates separate frame and chunk level recurrent queries (Li et al., 2026), while AVA-VLA uses recurrent state to modulate current visual features (Xiao et al., 2026). Memory-bank and retrieval methods provide more structured storage. SAM2Act (Fang et al., 2025) integrates a visual memory architecture for manipulation, and MemoryVLA combines working-memory tokens with a bank that preserves visual detail and higher-level semantics (Shi et al., 2026). MAP-VLA (Li et al., 2025) retrieves task-stage prompts learned from demonstrations. Notes-to-Self (Haresh et al., 2026) externalises object locations, plans, and subgoal progress in a 12 language scratchpad. These systems make memory content more explicit, but they introduce a bank, retrieval rule, or language-generation channel in addition to the recurrent policy state. MEMBODIED occupies a different position in this design space. It neither retains a growing se- quence of observations nor retrieves discrete memory entries. Instead, it represents episode history through a set of layer-wise associative matrices and an initial-scene anchor, both with storage inde- pendent of episode length, and conditions the action expert on their readouts. A.3FAST-WEIGHT ASSOCIATIVE MEMORY Fast-weight models separate slowly optimised parameters from rapidly updated episode state. Schmidhuber (1992) introduced learned control of fast-weight memories. Linear attention was later interpreted as an associative fast-weight programmer (Schlag et al., 2021). DeltaNet replaced purely additive key-value writes with a delta rule that corrects the association already recalled for a key, and Gated DeltaNet added data-dependent retention (Yang et al., 2025). Most directly related, δ- mem augments a language model with a fixed-size online matrix whose readout produces low-rank attention corrections (Lei et al., 2026). MEMBODIED transfers the associative-state principle to continuous robot control. Rather than per- turbing attention queries or outputs, it injects a projected memory vector into a dedicated suffix token before attention. We use this interface comparison to analyse how an associative state is most effectively exposed to the action network. A.4MEMORY-FOCUSED EVALUATION Memory-specific manipulation benchmarks have recently proliferated. MemoryBench evaluates spatial memory and action recall, requiring policies to recover scene information or actions from earlier in an episode (Fang et al., 2025). MIKASA-Robo broadens this scope through a diverse set of partially observable tabletop tasks designed to test memory-intensive control (Cherepanov et al., 2026a). RMBench comprises nine dual-arm manipulation tasks organised by memory complex- ity, ranging from retaining information from a single earlier stage to accumulating evidence across multiple observations or attempts (Chen et al., 2026). We use RMBench as our primary mem- ory evaluation benchmark because its controlled M(1) and M(n) categories distinguish tasks that require retaining a single earlier event from those that require accumulating information across sev- eral events within a common bimanual manipulation setting. In contrast, LIBERO is not designed specifically around episodic recall; its spatial, object, goal, and long-horizon suites provide a com- plementary test of whether a memory-augmented policy remains effective on general manipulation (Liu et al., 2023). BADDITIONAL EXPERIMENTAL DETAILS This section describes the evaluated tasks, training configurations, baseline implementations, and inference-efficiency profiling. Table 3 summarises the shared training configurations and hyperpa- rameters. Table 3: Training configuration and Hyperparameters. SettingConfiguration Visual inputThree camera streams, resized with padding to 224× 224 Action generation50-step action horizon; 10 flow-matching denoising steps Numerical settingsbfloat16; seed 42; EMA 0.99; global gradient clipping at 1 OptimiserAdamW, (β 1 ,β 2 ) = (0.9, 0.95), ε = 10 −8 , weight decay 10 −10 Learning-rate scheduleCosine decay with 1,000 warm-up steps, peak rate 2.5× 10 −5 , and ter- minal rate 2.5× 10 −6 13 B.1EVALUATED RMBENCH TASKS Our evaluation uses five RMBench tasks that cover both memory-complexity categories (Chen et al., 2026): three M(1) tasks and two M(n) tasks. M(1) tasks. These tasks require information from a single prior event in the observations. In Put Back Block, the robot moves a block from one of four pads to the centre, presses a button, and returns it to its original pad, requiring it to remember which pad was initially occupied. In Rearrange Blocks, the robot moves the block between two pads onto the empty pad, presses a button, and then moves the block that originally occupied a pad into the middle, requiring it to retain the initial layout. In Swap Blocks, the robot exchanges two blocks placed on two of three trays using the empty tray as temporary space, requiring it to remember the initial block-tray assignment and block position throughout the transfers. M(n) tasks. These tasks require accumulating information across multiple prior events which are specifically attempts in this case. In Battery Try, the robot inserts two batteries in different orienta- tions until the dashboard indicates the correct combination, requiring it to remember which combi- nations have already been tested. In Block Ranking, the robot rearranges three coloured blocks and presses a button to test each ordering until the correct one is found, requiring it to remember which orderings have already failed. B.2RMBENCH CONFIGURATION The RMBench models adapted the PaliGemma backbone using LoRA rank 16 and alpha 16, and the action expert using rank 32 and alpha 32. Models were trained for 10,000 steps with batch size 8. Gradients propagated through the complete sampled sequence. Consecutive sequence elements were separated by 50 environment steps, matching one action horizon. The sequence length was 8 for Put Back Block, Rearrange Blocks, and Swap Blocks; 14 for Battery Try; and 18 for Block Ranking, determined based on average episode lengths in the task. he main MEMBODIED configuration used associative rank r = 128 and α mem = 256, giving an explicit retrieval multiplier of α mem /r = 2. B.3LIBERO CONFIGURATION We trained the LIBERO experiment differently from the RMBench models using full fine-tuning to match the published π 0 baseline setup. Training used a batch size of 32, 30,000 steps, and a sequence length of six. Evaluation used five-step replanning, following the official π 0 LIBERO setup. To match this replanning cadence while retaining the 50-step memory-update interval used during training, we maintained ten recurrent memory slots in a round-robin schedule. Each five-step policy call reads one slot, and that slot was revisited after 50 environment steps. At the revisit, the delayed write used the new observation together with a summary of the complete 50-step action chunk predicted on the slot’s previous visit. This summary therefore includes predicted actions beyond the five steps executed before the next replan. B.4BASELINE IMPLEMENTATIONS Table 4 records how each evaluated baseline was implemented. Diffusion Policy, ACT, X-VLA, and the π 0.5 baseline in Table 1 are published RMBench results (Chen et al., 2026). The π 0 -Hint annotations exposed only task progress relevant to the evaluated memory demands. For Put Back Block, they identified the block’s original mat and whether it had been moved, the button pressed, and the block returned. For Rearrange Blocks, they recorded whether the first block had reached the empty mat, the button had been pressed, and the second block had been moved between the mats. For Swap Blocks, they described completion of the three block-transfer stages and the button press. Battery Try listed previously attempted polarity combinations and whether the dashboard was on, while Block Ranking listed colour orders attempted before earlier button presses. Because the stateless policy was not trained to consume these annotations, π 0 -Hint is a privileged-information reference rather than an upper bound. 14 Table 4: Implementation details for locally evaluated baselines. “Local adaptation” indicates that the method was implemented and trained within our π 0 evaluation setup. MethodTypeImplementation π 0 -StatelessLocal baselineCurrent observation only; batch size 32; 10,000 steps. π 0 -FrameStackLocal baselineFour observations separated by 50 environment steps and concate- nated in the policy context; samples with insufficient history were padded to four frames. π 0 -μ-VLALocal adaptation64 recurrent memory tokens; sequence length 8; stride 1; gradients truncated every two recurrent steps. π 0 -Vanilla Recurrent Memory Local adaptationRank 128 and scale 256; query/output attention corrections; recur- rent hidden-state writes. NativeMEMLocal adaptationMemory-tokenizer training for 50,000 steps, offline token caching, then policy training for 20,000 steps with queue stride 1. NativeMEM + MEMBODIED CombinationNativeMEM pipeline above with MEMBODIED sequence length 8 and sequence stride 50. π 0 -HintInference-only reference Simulator-derived textual task-progress hints appended at infer- ence, without retraining. The internal variants changed how the associative state affected action generation. MEMBODIED- AS projected each retrieval into additive corrections to the attention query and output representa- tions. MEMBODIED-H augmented the gated-delta state with a slower LSTM-like recurrent cell controlled by a learned forget gate. The latter therefore changed the state dynamics as well as its use by the action network, so these results compare complete memory designs rather than isolated readout substitutions. Unlike the other locally trained RMBench baselines, the NativeMEM baseline followed a separate staged training procedure. A memory tokenizer was first trained on the RMBench mixture for 50,000 steps, after which its per-frame, per-view tokens were cached offline for each target task. The tokenizer was then frozen while the queue-conditioned policy was trained for 20,000 steps. The standalone model used a queue stride of one; the combined NativeMEM + MEMBODIED model retained this queue while training the associative state with sequence length 8 and stride 50. B.5MEMORY VARIANTS The reported attention-steering and hierarchical variants use the same associative memory configu- ration as MEMBODIED. Attention steering (MEMBODIED-AS). At each action-expert layer, the state token queries the associative matrix. Separate learned projections turn the readout into additive corrections to the action tokens’ attention queries and projected attention outputs. The readout is scaled by α mem /r, and each correction is scaled by the root-mean-square magnitude of the representation it modifies. Hierarchical memory (MEMBODIED-H). This variant retains the attention-steering readout but adds a second, slower matrix C (l) t to each layer’s recurrent state. The gated-delta write produces an updated associative matrix for each value slot. A sigmoid forget gate derived from the post- layer state token blends their average with the previous slow matrix C (l) t−1 to obtain C (l) t . The final associative state is a weighted average of each gated-delta matrix and C (l) t . Thus, MEMBODIED- H changes the state update as well as the readout relative to the main memory-token model; the variants are complete designs, not isolated readout substitutions. B.6 π 0.5 ADAPTATION Unlike π 0 , π 0.5 represents robot state in the language prefix and has no dedicated state token in the action-expert suffix. We therefore use the memory token’s hidden representation for both associative-memory operations. At each action-expert layer, the token’s pre-attention representation is projected to form the read query, and the gated retrieval is then added to that token before at- tention. Its post-layer representation is projected to form the write key for the subsequent memory 15 update, whose value combines the action summary with the following visual observation. The token is recreated on each call, whereas the associative matrices persist across calls within an episode. The initial-scene anchor is retained by fusing its readout into the action tokens. B.7INFERENCE-EFFICIENCY PROFILING We profiled MEMBODIED and NativeMEM in bfloat16 on NVIDIA A100 GPUs with 80 GB of VRAM, using one GPU per evaluation run. We measured model-side computation only. For MEM- BODIED, timing covered policy calls from invocation until the action array was returned. This includes policy-side input preprocessing, current-observation vision encoding, associative-memory reads and updates, and generation of a 50-step action chunk using 10 flow-matching denoising steps. For NativeMEM, per-cycle latency was the sum of the model-side time for all video history encod- ing calls since the preceding policy query and the subsequent policy inference call. This includes per-frame video-history encoding, updates to the expanding history queue and policy context, and action generation. We evaluated Put Back Block, Rearrange Blocks, Swap Blocks, and Battery Try for 50 episodes per task and policy, yielding 200 episodes per policy. The first three policy queries of each task-policy run were treated as warm-up and excluded. The reported steady-state latency is the arithmetic mean over all retained policy cycles pooled across the four tasks, comprising 2,548 NativeMEM cycles and 2,563 MEMBODIED cycles. Peak memory is the maximum JAX allocator peak bytesinuse observed across all calls and tasks. NativeMEM required 1,593.7 ms per policy cycle and 20.57 GiB of peak GPU memory, whereas MEMBODIED required 129.2 ms and 18.61 GiB, corresponding to 91.9% lower latency and 9.5% lower peak memory. CREAL-WORLD DATA AND ROBOT EVALUATION C.1ROBOT DETAILS The robot comprised two AgileX PiPER arms, an Orbbec Gemini 336L top camera, and two Intel RealSense D405 wrist cameras (Figure 5). The robot state and action vectors each contained 14 values: six joint values and one gripper value for each arm. (a) Robot(b) Put Back Block.(c) Rearrange Blocks.(d) Swap Blocks. Figure 5: Real-robot platform and representative task configurations. The task panels show top-camera observations approximately one second after the start of a cleaned demonstration. They illustrate the initial workspace configurations and are not evaluation rollouts. C.2DATASET COLLECTION AND PREPROCESSING We manually collected 50 demonstrations for each of the three tasks. Each demonstration contained synchronised top, left-wrist, and right-wrist views recorded at 30 Hz, together with robot states and actions. Table 5 summarises the data retained after cleaning. The cleaned top-camera videos have resolution 330× 240, while both wrist-camera streams have resolution 424× 240. Demonstrations affected by recording faults were removed before training. The retained trajecto- ries were temporally trimmed to the demonstrated interaction, and the recorded joint states were denoised. The top-camera stream was cropped to the task workspace during preprocessing. Task- specific real-robot policies were trained on the corresponding cleaned datasets using a training recipe similar to the RMBench experiments. 16 Table 5: Real-world demonstration data after cleaning. Mean duration is computed from the retained 30 Hz trajectories. TaskCollectedRetainedFramesMean duration (s) Put Back Block504154,22844.1 Rearrange Blocks505048,91432.6 Swap Blocks504986,81859.1 C.3EVALUATION PROTOCOL We evaluated each policy for 20 physical rollouts on each task. A rollout was counted as successful only when the complete instructed task was achieved, and partial completion received no credit. The recurrent associative state and initial-scene anchor were reset at the start of every rollout. The resulting success rates are reported in Figure 3b. DADDITIONAL ANALYSIS D.1ADDITIONAL BASELINE CLARIFICATION Table 1 shows that our π 0 -μVLA adaptation achieved 0.0% success on all five RMBench tasks under the shared training budget. This result shows only that the adaptation did not learn an effective recur- rent policy under our training recipe; it should not be interpreted as a general comparison with μVLA under its original setting. One possible explanation for the poor performance in our adaptation is that learning recurrence through carried tokens may require longer training. The inference-time π 0 -Hint policy reached 2.0% mean success. Because the policy was not trained to consume its simulator- derived textual hints, this condition is a privileged-information reference rather than a trained oracle or an upper bound. D.2MEMORY RANK 3264128 Memory rank, r 0 20 40 60 80 100 Success rate (%) 22 26 34 32 58 76 Put back block Rearrange blocks Figure 6: Associative memory rank comparison in the anchor-free memory-token variant. In the anchor-free memory-token variant, success increases monotonically as the rank grows from 32 to 64 to 128 on both Put Back Block and Rearrange Blocks (Figure 6). Their two-task mean rises from 27.0% to 42.0% and 55.0%, respectively. The policy’s sensitivity to rank is consistent with associative-state capacity influencing action selection: a larger state can preserve a richer set of episode-dependent associations for later retrieval. D.3INITIAL-FRAME ANCHOR REPRESENTATION We also tested a simpler way to provide the initial scene: adding the episode’s first frame to the policy prefix alongside the current input, rather than using the fixed episode-anchor pathway. On Put Back Block task, this first-frame-prefix variant achieved 22.0% success, compared with 34.0% 17 for the memory-token model without an anchor and 40.0% with the proposed anchor. This could be because adding the full first frame introduces distracting visual tokens into the policy context, whereas the anchor selectively retrieves a compact reference to the initial scene. D.4CROSS-EPISODE MEMORY CARRYOVER Table 6: RMBench success rates (%) with associative memory reset or carried across episodes. The initial-frame anchor is reset at the start of every episode in both conditions. TaskMemory resetMemory carried over Put Back Block40.018.0 Rearrange Blocks92.078.0 Swap Blocks56.036.0 Battery Try40.024.0 Block Ranking22.028.0 Mean50.036.8 As an exploratory test, we carry the associative state across rollout boundaries, although MEMBOD- IED was trained with the state reset at the start of each episode. The initial-frame anchor is still reset for each new rollout. Table 6 shows that this reduces mean success from 50.0% to 36.8%, with lower success on four of the five tasks. The largest declines occur on Put Back Block and Swap Blocks (22 and 20 percentage points), where the relevant location or progress must be established within the current episode. Block Ranking is the exception, improving by 6 points. Its trial-and-error structure may respond differently to carried state, although this increase does not establish that information from earlier episodes was useful. Overall, the results do not support a general benefit from carry- ing over memory under this evaluation, as the declines are consistent with information from earlier episodes interfering with episode-specific decisions. EQUALITATIVE ANALYSIS The mean success rates show that MEMBODIED improves memory-dependent control, but do not identify whether the gains arise at the decisions that require history. We therefore manually anno- tate all 50 paired rollouts of π 0 -Stateless and MEMBODIED, using identical seeds, and identify the stages at which their behaviours diverge. The analysis separates RMBench tasks that require recall- ing one earlier observation, denoted M(1), from tasks that require accumulating the outcomes of several attempts, denoted M(n). Across both groups, the policies behave similarly when the current observation is sufficient, but diverge once the correct action depends on information that is no longer visible. E.1RECALLING AN EARLIER OBSERVATION E.1.1PUT BACK BLOCK Put Back Block isolates spatial recall: both policies move the block to the centre and press the button in every rollout, but they differ when deciding where to return it. The stateless policy returns to the wrong pad in 37/50 rollouts. Its 13 correct choices are close to the 13.8 expected from its own pad preference, indicating that its return location does not track the episode’s initial pad. MEMBODIED reduces wrong-pad returns to 13/50 and corrects 24 stateless errors without introducing the reverse error on any paired seed. Its 37 correct choices also exceed the 14.5 expected by chance. Both comparisons are significant at p<10 −4 . The benefit of MEMBODIED is therefore concentrated on the history-dependent choice of return location, rather than the preceding pick-and-place sequence. Aside from the recall errors, the remaining failures expose two execution constraints. Of the 17 correct returns towards the front pad, 14 reach the pad but stall instead of releasing the block. The stateless policy exhibits the same tendency in 8/9 relevant rollouts, suggesting a shared placement difficulty rather than a memory-specific failure. In addition, six of MEMBODIED’s eight failures classified as residual recall errors occur when the block starts on the right: the policy repeats its 18 0.0 s20.0 s 30.0 s34.0 s ✓ 31.5 s49.9 s ✗ 050100 13/50 37/50 0.0 s26.0 s 33.0 s38.7 s ✓ 33.0 s69.9 s ✗ 050100 0/50 20/40 0.0 s35.5 s 50.0 s60.5 s ✓ 39.0 s99.9 s ✗ 050100 19/50 41/50 M(1) tasks: recall one earlier observation 1 Information shown 2 Information hidden 3 Decision 4 Outcome identical for both policies (same seed)policies diverge Failure-mode rate (95% Wilson CI) (a) Put Back Block memory needed: where the block started · seed 100025 block starts on the back pad block at centre; back pad empty MemBodied carries it to the back pad returned to its start π 0 -Stateless carries it to the left pad left on the wrong pad Wrong-pad return % of all 50 rollouts p Holm = 1.1 × 10 −6 McNemar, paired seeds (b) Rearrange Blocks memory needed: which block started on a mat · seed 100000 right mat occupied at the start both mats full; start is hidden MemBodied takes the right-mat block original block moved to gap π 0 -Stateless takes the left-mat block wrong block moved to gap Wrong-block selection % of rollouts that moved a block p Holm = 1.6 × 10 −8 Fisher exact (c) Swap Blocks memory needed: how many of the 3 moves are done · seed 100023 start: 0 of 3 moves 2 of 3 moves done MemBodied makes the 3rd move presses done after 3 moves π 0 -Stateless presses done after 2 moves times out; swap incomplete Premature completion % of all 50 rollouts p Holm = 6.3 × 10 −5 Fisher exact Figure 7: Failure modes on the three M(1) tasks. Representative paired rollouts show the same seed as task-relevant information becomes hidden and the policies reach the ambiguous decision. The blue trajectory corresponds to MEMBODIED and the grey trajectory to π 0 -Stateless. Bars summarise the annotated failure mode across rollouts, with marginal 95% Wilson confidence intervals. Put Back Block and Swap Blocks use all 50 rollouts per policy; Rearrange Blocks is conditioned on rollouts in which a block is moved (50 for MEMBODIED, 40 for π 0 -Stateless). The p-values shown in the figure are Holm-corrected. dominant left-arm motion towards the front pad instead of switching arms. These two patterns account for 20 of the 30 MEMBODIED failures. Thus, memory largely resolves the target location recall problem, but not the motor strategy needed to reach every target reliably. E.1.2REARRANGE BLOCKS Rearrange Blocks tests recall through object selection rather than target selection. After the first placement and button press, both mats contain identical blocks, so only the initial layout identifies which block should be returned to the gap. Both policies complete the shared early stages in every rollout, but their subsequent choices differ. MEMBODIED selects the correct block specified by the initial layout in all 50 rollouts. The stateless policy instead exhibits a strong left-block preference: 19 it chooses the left block in 37/40 rollouts in which it moves a block, selects the correct block only 20 times, and moves no block in the remaining ten rollouts. This shows that memory changes the episode-specific decision made after the two scenes become visually indistinguishable, rather than merely improving the manipulation stages. Object selection does not explain the entire performance gap. Of the 20 stateless rollouts that select the correct block and bring it to the target pad, only one succeeds, whereas MEMBODIED succeeds in 46/50 rollouts overall. The stateless policy also does not press the button fully, leading to task failures. These execution errors occur after the history-dependent object selection and therefore compound, rather than explain, the recall gap. E.1.3SWAP BLOCKS Swap Blocks differs from the previous tasks because the hidden variable is progress rather than an initial location. Exchanging two identical blocks requires three moves through an empty pad, after which the visible occupancy matches the start of the episode. Intermediate occupancy patterns can also be valid starting layouts. Consequently, the current image does not reveal whether the policy should move another block or press the completion button. Both policies can perform the required manipulation: MEMBODIED makes at least one move in 48/50 rollouts and the stateless policy in 41/50. The main difference is in deciding when the swap is complete. The stateless policy presses the completion button prematurely in 41/50 rollouts, com- pared with 19/50 for MEMBODIED, and completes all three moves in only eight rollouts, compared to 31 for MEMBODIED. Among rollouts in which the button is pressed, MEMBODIED waits until the swap is complete in 28/47 cases, whereas the stateless policy does so in only 6/47. All three progress-related comparisons remain significant (p<10 −4 ). This pattern is consistent with mem- ory tracking progress through the move sequence. However, the tracking is not perfect: 12 of the 22 residual MEMBODIED failures press after two moves, making the final transition the dominant remaining error. E.2ACCUMULATING OUTCOMES ACROSS ATTEMPTS E.2.1BATTERY TRY Battery Try requires the policy to change its next action after transient failure feedback. The two policies set both batteries at similar rates (23/50 for MEMBODIED and 20/50 for π 0 -Stateless), indicating comparable grasp-and-insert behaviour before memory is needed. They diverge after an unsuccessful orientation test, once the meter has returned to rest and the current image no longer reveals what was tried. The stateless policy makes a median of four further attempts and completes the task in 8/20 rollouts; MEMBODIED usually succeeds within one further attempt and resolves it in 20/23. This difference remains significant after Holm correction (p=0.028), localising the benefit of MEMBODIED to the point where an earlier outcome must alter the next action. The dominant remaining limitation is grasp stability: 25 of the 30 MEMBODIED failures involve dropping a battery during pickup or insertion, a common failure mode in both policies. E.2.2BLOCK RANKING TRY Block Ranking Try extends the same dependence on earlier outcomes to a longer search. Both policies complete an initial block arrangement at similar rates (38/50 for MEMBODIED and 40/50 for π 0 -Stateless), but differ in whether they verify it. MEMBODIED presses the test button after 35/38 completed arrangements, compared with 14/40 for the stateless policy. On four paired seeds, the stateless policy even constructs the same arrangement that MEMBODIED successfully verifies but dismantles it without testing it. Because a completed arrangement looks identical before and after testing, this difference shows that MEMBODIED helps in remembering whether a candidate arrangement still needs to be verified, rather than in the preceding pick-and-place operation. The remaining errors separate immediate verification from longer-term search. Six of the 14 state- less button presses miss the button, compared with none of the 35 MEMBODIED presses, so motor precision further compounds the stateless failure rate. After a rejected arrangement, however, MEM- BODIED builds and tests a new order in only 10/28 rollouts; in other cases, it fails in rearranging the blocks or retests the rejected arrangement. These longer-search errors do not differ from the stateless 20 25.0 s34.0 s 40.0 s45.6 s ✓ 60.0 s99.9 s ✗ 050100 3/23 12/20 4.5 s63.0 s 65.5 s65.9 s ✓ 70.0 s349.9 s ✗ 050100 3/38 26/40 M(n) tasks: accumulate the outcome of earlier attempts 1 Information shown 2 Information hidden 3 Decision 4 Outcome identical for both policies (same seed)policies diverge Failure-mode rate (95% Wilson CI) (d) Battery Try memory needed: that the last insertion failed · seed 100084 inserts both batteries meter silent; view unchanged MemBodied removes and re-inserts meter responds after 1 retry π 0 -Stateless removes and re-inserts 4th retry; meter still silent Unresolved retrial % of rollouts with both batteries seated p Holm = 0.028 Fisher exact (e) Block Ranking Try memory needed: whether the new order was checked · seed 100025 presses to check the first order new order built, not yet checked MemBodied presses to check it order accepted π 0 -Stateless dismantles it unchecked never checked; times out Unverified rearrangement % of rollouts that rearranged p Holm = 6.6 × 10 −7 Fisher exact Figure 8: Failure modes on the two M(n) tasks. Representative paired rollouts show decisions that depend on the outcome of an earlier attempt. The blue trajectory corresponds to MEMBODIED and the grey trajectory to π 0 -Stateless. Bars report the conditional failure rate with marginal 95% Wilson confidence intervals: unre- solved retries among rollouts that seat both batteries (23 for MEMBODIED, 20 for π 0 -Stateless), and unverified arrangements among rollouts that complete a rearrangement (38 and 40, respectively). The p-values shown in the figure are Holm-corrected. policy. The differences in immediate verification and button execution both remain significant af- ter Holm correction (p≤0.001), but the broader pattern shows that memory supports the immediate transition from rearranging to verification more clearly than the full multi-attempt search. Across all five tasks, the rollout analysis localises the performance gains shown in Section 5 to the decisions that require past information in an episode. MEMBODIED does not consistently change the early motor stages shared by both policies. Instead, it reduces the failure specific to each aliased decision: returning to the wrong pad, selecting the wrong block, declaring a swap complete too early, repeating an unsuccessful battery orientation, or dismantling an arrangement before testing it. The remaining failures separate into execution errors, such as stalling, missed presses, and dropped objects, and longer-term memory errors that arise when several updates must be retained. These be- havioural patterns are consistent with the recurrent state retaining task-relevant episode information while preserving general robot control ability. 21