Paper deep dive
rMuscle: Robotic Muscle Memory for Efficient Vision-Language-Action Model Inference
Kaijun Zhou, Zhiyang Li, Le Chen, Jinyu Gu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/19/2026, 2:42:30 AM
Summary
The paper introduces rMuscle, a real-time Vision-Language-Action (VLA) inference framework designed to accelerate robotic manipulation by exploiting cross-execution similarity. Inspired by human muscle memory, rMuscle utilizes a dual-phase cache: a Context Cache to reuse visual-token outputs in the compute-bound VLM prefill stage, and an Action Cache to reuse neuron activation patterns in the memory-bound action denoising stage. The system employs online recomputation, sliding-window retrieval, and mask sharing to maintain low memory overhead. Evaluations on RTX 4090 and Jetson Thor across LIBERO, RoboTwin, and physical tasks show speedups of 1.29x to 1.42x over state-of-the-art engines like FlashRT, while maintaining original success rates.
Entities (15)
Relation Signals (15)
rMuscle → achievesspeedupon → RTX 4090
confidence 95% · rMuscle achieves 1.29-1.42X speedup on RTX 4090
rMuscle → achievesspeedupon → Jetson Thor
confidence 95% · rMuscle achieves 1.29-1.42X speedup on ... Jetson Thor
rMuscle → uses → Action Cache
confidence 95% · the Action Cache reuses neuron activation patterns to reduce weight accesses.
rMuscle → uses → Context Cache
confidence 95% · The Context Cache reuses visual-token outputs to reduce computation
rMuscle → evaluatedon → LIBERO
confidence 90% · We evaluate rMuscle on ... across 40 LIBERO tasks
rMuscle → evaluatedon → RoboTwin
confidence 90% · We evaluate rMuscle on ... 50 RoboTwin tasks
rMuscle → outperforms → FlashRT
confidence 90% · reaching up to 1.29× and 1.42× speedup over FlashRT
Context Cache → targets → VLM prefill
confidence 90% · The Context Cache accelerates visual context encoding... targeting the two major stages of VLA inference.
Action Cache → targets →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Factory work is a promising early scenario for embodied AI: assigning repetitive manual jobs to robots has clear economic payoff, and a structured station keeps the jobs tractable for current policies. Vision-Language-Action (VLA) models now dominate as the policy paradigm for these robots. The inference latency of VLA models directly affects robot responsiveness and motion smoothness. However, existing VLA inference frameworks do not fully exploit the characteristics of embodied workloads or account for the distinct bottlenecks across different stages of VLA inference. In this paper, we first characterize embodied workloads and identify substantial task similarity across repeated robot executions. We further find that such similarity extends beyond observations and action trajectories to internal model states. Drawing on these observations, we present rMuscle, a real-time VLA inference framework inspired by human muscle memory. It exploits cross-execution similarity through a dual-phase muscle-memory cache. The Context Cache reuses visual-token outputs to reduce computation, while the Action Cache reuses neuron activation patterns to reduce weight accesses. We keep both the cache memory footprint and access overhead low through online cache recomputation, sliding-window cache retrieval, and mask sharing across consecutive denoising steps. rMuscle achieves 1.29-1.42X speedup on RTX 4090 and Jetson Thor across LIBERO, RoboTwin, and physical manipulation tasks, while maintaining the original success rates on real-world robots.
Tags
Links
- Source: https://arxiv.org/abs/2609.19104v1
- Canonical: https://arxiv.org/abs/2609.19104v1
PDF not stored locally. Use the link above to view on the source site.
Full Text
70,485 characters extracted from source content.
Expand or collapse full text
rMuscle: Robotic Muscle Memory for Efficient Vision-Language-Action Model Inference Kaijun Zhou, Zhiyang Li, Le Chen, and Jinyu Gu Note: Corresponding author: gujinyu@sjtu.edu.cn. Affiliation: Institute of Parallel and Distributed Systems, Shanghai Jiao Tong University 2026 Abstract. Factory work is a promising early scenario for embodied AI: assigning repetitive manual jobs to robots has clear economic payoff, and a structured station keeps the jobs tractable for current policies. Vision-Language-Action (VLA) models now dominate as the policy paradigm for these robots. The inference latency of VLA models directly affects robot responsiveness and motion smoothness. However, existing VLA inference frameworks do not fully exploit the characteristics of embodied workloads or account for the distinct bottlenecks across different stages of VLA inference. In this paper, we first characterize embodied workloads and identify substantial task similarity across repeated robot executions. We further find that such similarity extends beyond observations and action trajectories to internal model states. Drawing on these observations, we present rMuscle, a real-time VLA inference framework inspired by human muscle memory. It exploits cross-execution similarity through a dual-phase muscle-memory cache. The Context Cache reuses visual-token outputs to reduce computation, while the Action Cache reuses neuron activation patterns to reduce weight accesses. We keep both the cache memory footprint and access overhead low through online cache recomputation, sliding-window cache retrieval, and mask sharing across consecutive denoising steps. rMuscle achieves 1.291.29–1.42×1.42× speedup on RTX 4090 and Jetson Thor across LIBERO, RoboTwin, and physical manipulation tasks, while maintaining the original success rates on real-world robots. 1. Introduction Figure 1. rMuscle operates in a manner analogous to human muscle memory: similar contexts trigger relevant prior experience, allowing cached results to be retrieved and reused to avoid redundant computation. Two stages compared side by side for human motor control and . Stage one: a pianist's sheet music, keyboard, and hand position recall one learned memory from several similar candidates, while a robot's instruction, observation, and motion state retrieve one reference episode from several similar candidates. Stage two: two schematics show that expected repetitions evoke a lower neural response and that brain activity localizes with learning, while reuses reference states through its Context and Action Caches and recomputes only the parts that changed. Embodied AI extends learned intelligence from digital content to physical interaction, and factory floors are a promising entry point: robots that take over repetitive manual tasks offer clear economic value, and a factory workstation is far more structured than an open-world environment, with a recurring set of tasks, a finite set of objects, and controlled lighting. Leading embodied AI companies are therefore bringing robots into factories, including Figure AI’s humanoids on a BMW production line, Tesla’s Optimus, and AgiBot’s G2 on a consumer-electronics line (Figure AI, 2025; Vaziri, 2026; AgiBot, 2026). Robot execution follows a closed-loop observe–infer–execute cycle, in which sensors capture the current state, a policy predicts an action chunk, and the robot executes it. Inference latency of the policy therefore limits how frequently the robot can update its actions: slow inference delays responses to environment changes and can disrupt motion continuity. The policy typically uses a Vision-Language-Action (VLA) model (Black et al., 2024; Physical Intelligence et al., 2025; NVIDIA, 2025; Zheng et al., 2025; Li et al., 2024; Cheang et al., 2025; Bu et al., 2025; Shukor et al., 2025), which combines multimodal context encoding with diffusion-based generative action prediction. Prior works (Su, 2026; Nguyen et al., 2026) develop inference engines to reduce VLA inference latency through kernel fusion, quantization, and static graph execution. Other studies (Zhou et al., 2026; Yang et al., 2025; Xu et al., 2025) accelerate VLA inference by adapting techniques originally developed for VLMs or text-to-image models, such as cross-frame visual-token reuse and denoising-step skipping. However, these approaches fall short of fully exploiting the acceleration opportunities in VLA inference, primarily due to two limitations. First, they fail to exploit similarity across embodied tasks. In industrial manipulation, a robot typically executes a recurring set of tasks at a fixed workstation, and executions of similar tasks have similar observations and action trajectories. Such similarity persists across the robot’s entire operational workflow, while existing works focus only on the immediately preceding frame or within a single denoising phase. Second, existing approaches do not account for the distinct performance bottlenecks across different stages of VLA inference. In representative models such as π0.5 _0.5 (Physical Intelligence et al., 2025), multimodal context encoding is predominantly compute-bound, whereas action denoising is memory-bound, with both stages contributing substantially to end-to-end latency. Optimizing only one stage results in limited end-to-end speedup. After an in-depth analysis of VLA workloads and inference behavior, we find that similarity across task executions extends beyond observations and action trajectories to internal model states, including FFN inputs and outputs as well as neuron activation patterns. Inspired by human motor control and muscle memory (Heald et al., 2021; Summerfield et al., 2008; Imamizu et al., 2000) (Figure 1), we present rMuscle, a VLA inference framework that exploits this similarity through a dual-phase cache design tailored to the distinct bottlenecks of multimodal context encoding and action denoising. The dual-phase cache consists of a Context Cache and an Action Cache, targeting the two major stages of VLA inference. The Context Cache accelerates visual context encoding by exploiting similarity in the inputs to latency-critical MLP layers. Instead of fully recomputing similar inputs, rMuscle reuses their cached outputs and performs computation only for the parts that deviate substantially. The newly computed outputs are then merged with the cached results to reconstruct the complete layer output, thereby reducing computation while preserving model accuracy. The Action Cache targets memory-bound action denoising, where short sequences make each step dominated by weight loading rather than computation. Its design builds on two observations: neurons contribute unequally to the result, and the same denoising step across similar tasks tends to activate similar sets of neurons. rMuscle thus uses the activation pattern of a similar past execution to predict the important neurons, loads and computes only their weights, and approximates the remaining neurons with the outputs of an earlier fully computed step, substantially reducing memory traffic while retaining the effect of every neuron. We further introduce three techniques to efficiently manage the dual-phase cache. First, leveraging the alternating pattern of inference and physical execution in robotic workflows, rMuscle stores only the input context for caches that are not currently referenced and reconstructs the complete cache on demand through online recomputation, which is overlapped with action execution. Second, rMuscle exploits the temporal continuity of robotic workflows. During the execution of a complete task, the caches that are likely to be referenced in the near future are predictable. We therefore maintain only a sliding window of relevant caches in GPU memory and update this window dynamically as the robot executes actions. Third, rMuscle divides consecutive denoising steps into groups, with each group sharing an important-neuron mask. As a result, rMuscle achieves a minimal cache memory footprint and access overhead. We evaluate rMuscle on π0.5 _0.5, GR00T N1.6, and X-VLA across 92 embodied tasks, spanning 40 LIBERO tasks (Liu et al., 2023), 50 RoboTwin tasks (Mu et al., 2024), and two physical-robot tasks: dual-arm pick-and-place on ALOHA (Fu et al., 2024) and single-arm assembly-line packing with DOBOT and Franka (DOBOT Robotics, 2026; Franka Robotics, 2026). It reaches 1.201.20–1.29×1.29× end-to-end speedup on RTX 4090 and 1.231.23–1.42×1.42× on Jetson Thor over the state-of-the-art (SOTA) inference engine (Su, 2026). Our method matches the original success rates across simulation and real-world tasks, including dual-arm ALOHA manipulation and single-arm assembly-line packing. This paper makes the following contributions: • We find that repeated executions of the same task have similar FFN inputs and neuron activation patterns, providing opportunities for computation reuse. • We design an end-to-end dual-cache inference system for SOTA VLA models, with a Context Cache for compute-bound VLM prefill and an Action Cache for memory-bound denoising. • We evaluate rMuscle on three models, two simulation benchmarks, and 92 tasks on RTX 4090 and Jetson Thor, reaching up to 1.29×1.29× and 1.42×1.42× speedup over FlashRT, respectively, while matching the average accuracy of the original policies. 2. Background 2.1. Embodied Robots for Industrial Manipulation Factories are a promising entry point for embodied robots, and leading embodied AI companies are bringing robots onto factory floors. Figure AI deployed humanoids on a BMW production line, Tesla is developing Optimus for manufacturing, and AgiBot reports that its G2 humanoid transports and sorts tablets on a consumer-electronics line (Figure AI, 2025; Vaziri, 2026; AgiBot, 2026). A factory-floor case study at Siemens adapts π0.5 _0.5 to pack accessory bags into cardboard boxes (Zhu et al., 2026). Amazon’s Vulcan Pick picks targeted items from fabric storage pods in a warehouse (Park et al., 2025). Despite differences in task objectives, these applications share a common execution pattern: a robot operates at a fixed workstation and repeatedly performs similar tasks, with variations mainly in the target objects and their locations. Such settings also impose stringent requirements on the responsiveness and reliability of the policy. 2.2. Embodied Intelligence Serving Loop An embodied robot operates in a closed observe–infer–execute loop. At each inference step, the robot collects visual observations, a language instruction, and its current joint state, which consists of the joint angles of the robot’s degrees of freedom. Given these inputs, the policy produces an action chunk containing S future actions, which the robot executes before the next observation and inference. This loop repeats until the task is completed, forming an episode. In our evaluation, completing a task typically requires 4–40 inferences per episode. If an episode contains N inference-and-execution iterations, the total number of executed actions required to complete the task lies in [(N−1)S+1,NS] [(N-1)S+1,\,NS ]. 2.3. Visual-Language-Action Models VLA models map images, language instructions, and robot state to executable actions. Representative models include RT-2, OpenVLA, the π family, and GR00T. RT-2 and OpenVLA represent actions as text tokens in an autoregressive formulation that transfers web-scale vision-language pretraining to robot control (Brohan et al., 2023; Kim et al., 2024). More recent models generate continuous actions through a diffusion process: π0.5 _0.5 (Physical Intelligence et al., 2025) combines a VLM with a denoising action expert for broadly generalizable manipulation, while GR00T N1.6 pairs a Cosmos-based VLM with a diffusion transformer for humanoid and bimanual control (NVIDIA GEAR Team, 2025). X-VLA uses a soft-prompted transformer as a scalable cross-embodiment VLA model (Zheng et al., 2025). Those VLA policies are commonly evaluated by rollout success rate on manipulation benchmarks and real-world tasks that emphasize different capabilities. 2.4. Dual-Phase Inference in VLAs We focus on the VLA models that couple a vision-language model (VLM) with a diffusion action expert, an architecture adopted by the majority of recent VLA models (Physical Intelligence et al., 2025; NVIDIA, 2025; Zheng et al., 2025; Community, 2026). As shown in Figure 2, the vision transformer (ViT) embedding stage converts the observations and language instruction into input embeddings. The visual observations typically consist of images captured by the robot’s cameras. Inference then proceeds through two core phases that account for most of the latency. First, the VLM performs a prefill pass over the ViT embeddings to compute visual features for action generation. Second, the action expert starts from a noisy action sample and iteratively refines it through multiple denoising steps to produce an action chunk. A dual-phase VLA architecture with a VLM stack on the left and a denoising action expert on the right. The VLM processes text and image tokens through attention and MLP blocks and passes visual features to the action expert. The denoising action expert refines noisy actions through cross-attention and MLP blocks over multiple diffusion steps. Figure 2. Representative dual-phase VLA architecture π0.5 _0.5. A VLM processes multimodal inputs and passes visual features to an action expert that iteratively denoises the action chunk.A dual-phase VLA architecture with a VLM stack on the left and a denoising action expert on the right. The VLM processes text and image tokens through attention and MLP blocks and passes visual features to the action expert. The denoising action expert refines noisy actions through cross-attention and MLP blocks over multiple diffusion steps. 2.5. Denoising-Step Skipping and Its Limits in VLAs Prior work on accelerating diffusion models for text-to-image and text-to-video generation has used caching to exploit redundancy at two levels: across denoising steps within a request and across related requests. DP-Cache and PAB reuse the output of one denoising step as the output of several subsequent steps within a request (Zhao et al., 2025; Zhou et al., 2026), while NIRVANA and MoDM maintain references across requests for text-to-image diffusion serving (Agarwal et al., 2024; Xia et al., 2026). NIRVANA stores intermediate latents from similar prompts and resumes generation from that state, skipping several denoising steps for new requests (Agarwal et al., 2024). However, these methods do not transfer well to embodied model inference, because both forms of reuse operate at the granularity of denoising steps. Whole-step reuse relies on consecutive steps producing nearly identical outputs, as is often the case for text-to-image and text-to-video models that typically run 20–50 steps (Labs, 2024; Wan et al., 2025). VLA models use far fewer steps. For example, GR00T uses only four steps (NVIDIA GEAR Team, 2025). Each step therefore makes a substantial change to the action chunk, so skipping any step discards refinement that subsequent steps do not recover. 3. Characterizing Embodied Workloads To identify opportunities for accelerating embodied inference, we characterize VLA workloads from both the execution and model perspectives. We first locate the performance bottlenecks in VLM prefill and action denoising, then examine observation and trajectory similarity across executions. We next examine how this similarity extends to internal model states, specifically in VLM states and neuron contributions during denoising. 3.1. Heterogeneous Bottlenecks in VLA Inference Three panels. Panel (a) shows stacked horizontal bars of end-to-end latency for the official PyTorch implementation, torch.compile, and FlashRT, split into prefix embedding, VLM prefill, and action denoising. Panel (b) shows stacked vertical bars of FlashRT VLM prefill and action denoising latency split into MLP, attention, and other. Panel (c) shows per-operator latency aggregated over the action-denoising loop. Figure 3. Latency breakdown of π0.5 _0.5 inference on RTX 4090. (a) End-to-end latency with three execution engines. (b) FlashRT latency of VLM prefill and action denoising by component. (c) Per-operator latency aggregated over the action-denoising loop, colored by component as in (b).Three panels. Panel (a) shows stacked horizontal bars of end-to-end latency for the official PyTorch implementation, torch.compile, and FlashRT, split into prefix embedding, VLM prefill, and action denoising. Panel (b) shows stacked vertical bars of FlashRT VLM prefill and action denoising latency split into MLP, attention, and other. Panel (c) shows per-operator latency aggregated over the action-denoising loop. Figure 3(a) compares the end-to-end latency across execution engines. Through kernel fusion and operator optimizations tailored to low-batch workloads, FlashRT reduces the latency from 194.3 ms with the official PyTorch implementation to 46.0 ms. We therefore use FlashRT for the following analysis. The latency breakdown of FlashRT shows distinct computational profiles across VLM prefill and action denoising phases, although both spend a substantial fraction of their execution time in feed-forward networks (FFNs). As shown in Figure 3(b), the MLP accounts for 83.5% of VLM prefill latency and 51.1% of action denoising latency. The operator-level breakdown in Figure 3(c) shows that action denoising spends most of its MLP time in the gate, up, and down projections. We therefore focus on the MLP in both phases and characterize its bottlenecks next. Table 1. Roofline analysis of the π0.5 _0.5 MLPs. AI (arithmetic intensity) is in FLOP/B. Each device lists theoretical compute / memory latency in μ derived from BF16 peaks (165 TFLOP/s, 1000 GB/s on 4090; 250 TFLOP/s, 270 GB/s on Thor). MLP GFLOP MB AI RTX 4090 Jetson Thor TcompT_comp/TmemT_mem TcompT_comp/TmemT_mem VLM prefill 57.04 64 891 345.3 / 64 228.2 / 237 Action denoise 0.27 8 34 1.6 / 8.0 1.1 / 29.6 Roofline analysis in Table 1 shows that the MLP faces different bottlenecks in the two phases. On RTX 4090, the VLM MLP has substantially higher arithmetic intensity and is compute-bound, whereas the action-denoising MLP has a much smaller workload and is memory-bound (Jiang et al., 2026; Zhou et al., 2026). On Jetson Thor, the VLM MLP lies near the ridge point: it is theoretically memory-bound, but the actual bottleneck varies depending on measured hardware performance and implementation details. 3.2. Trajectory Similarity Across Repeated Tasks In industrial assembly-line settings, robots repeatedly perform the same task using a limited set of objects in a relatively stable workspace. However, variations in lighting, tabletop texture, and object pose can change both observations and required motions. We therefore examine whether action trajectories remain similar across executions despite these variations. Trajectory Similarity. An action trajectory is the VLA’s observable output: a temporally ordered sequence of robot actions that traces a continuous path in 3D space. To test whether useful repetition survives environmental variation, we model these factors in simulation. In the representative task shown in Figure 4, a dual-arm robot lifts two beverage bottles and moves them to designated positions above the tabletop. We vary the initial bottle positions and lighting conditions across executions, then compare a current execution with a reference execution of the same task. Figure 4. A dual-arm robot moves two soda bottles, one Coke and one Sprite, to the same positions above the tabletop. Both the current and reference runs finish in four inference steps and reach their target positions successfully and stably. Although the initial bottle positions and lighting differ, the end-effector trajectories and the camera views of both arms remain closely aligned, with path cosine similarities of 0.995 for the left arm and 0.994 for the right arm. 3.3. VLM-State Similarity and Visual-Token Coverage Comparing MLP inputs and outputs from the current and reference executions at every VLM layer, we find cosine similarities as high as 98.5% for inputs and 81.0% for outputs. This similarity motivates examining how the remaining output changes are distributed across visual tokens. We further investigate whether MLP input differences can identify the tokens that account for most of these changes. Visual-Token Coverage. At each layer, we rank visual tokens by RMS FFN input change relative to the current input magnitude and select the top fraction ρp _p. Coverage is the selected tokens’ share of the total squared L2L_2 change in FFN outputs between current and reference executions. Figure 5 shows that selecting ρp=0.4 _p=0.4 of the visual tokens captures approximately 80% of the output change. Output-change coverage versus the fraction of selected visual patches, averaged across VLM layers and shown for layers 0 and 16. Input-change ranking is compared with output-change optimal and random selection. Figure 5. Coverage of π0.5 _0.5 VLM FFN output changes under input-based patch selection and optimal selection by actual output changes.Output-change coverage versus the fraction of selected visual patches, averaged across VLM layers and shown for layers 0 and 16. Input-change ranking is compared with output-change optimal and random selection. Figure 6. Spatial distribution of visual patches selected for recomputation across four action chunks.Four main-camera views show the frequency with which each visual patch is selected across 17 VLM FFN layers. Selected regions move with the bottles, grippers, and arms across action chunks. Figure 6 shows the selected regions for the bottle-moving task. Important patches concentrate around the bottles, grippers, and arms, with a smaller number on background regions that encode scene context. Their locations shift across action chunks as manipulation progresses, motivating selection for each policy call rather than a fixed spatial mask. 3.4. Denoising-Neuron Contribution and Overlap The similar trajectories of current and reference executions motivate examining whether their action-denoising processes also exhibit similar internal activations. Following our analysis of VLM states, we first compare the inputs and outputs of action denoising. Although the initial noise is identical under the same random seed, the generated action chunks exhibit only 69% similarity. This difference motivates exploring finer-grained reuse within denoising instead of directly reusing reference actions. We therefore examine how neuron contributions are distributed within each denoising FFN and whether these contribution patterns are similar between the current and reference executions. Neuron Contribution and Coverage. For each denoising step t and layer l, we quantify each neuron’s contribution to the FFN output using the following score (Sun et al., 2024): (1) ct,l,n=(∑r=1S|ht,l,r,n|)∥Wl,n∥2.c_t,l,n= ( _r=1^S h_t,l,r,n ) W_l,n _2. Here, ht,l,r,nh_t,l,r,n is the post-GeGLU activation of neuron n for action r, and Wl,nW_l,n is its down-projection weight. Within each (t,l)(t,l), we rank all neurons by c in descending order. We call the highest-contribution neurons important neurons, and write ρn _n for the fraction of hidden neurons retained at each layer. Coverage is the share of the total contribution score captured by the top ρn _n fraction of neurons. Each layer curve in Figure 7 averages this coverage across 100 inferences and denoising steps. Figure 7. Cumulative contribution-score coverage of denoising MLP neurons in π0.5 _0.5. As shown in Figure 7, retaining ρn=0.5 _n=0.5 of the neurons captures 84.4% of the total contribution score. Neuron Overlap Across Executions. We then compare these important-neuron sets across executions. For repeated executions of the same task, the overlap averages 86% (ranging from 82% to 91%), whereas for executions of different tasks it drops to 71% (70%–72%). The gap between the two settings is larger than the within-setting variation, indicating that important neurons are task-dependent rather than fixed and motivating the use of a same-task reference to predict important neurons. 4. Design Overview. rMuscle combines two complementary mechanisms: dual-phase caching for selective recomputation and cache management for low-overhead access and storage. Figure 8. Dual-phase cache design. Context Cache reduces visual-token FFN computation during VLM prefill. Action Cache reduces weight memory accesses during denoising.Dual-phase cache architecture. Intermediate vision features retrieve a reference for VLM FFN input and output reuse. During action denoising, selected neuron updates are added to outputs from dense anchors in the current policy call. Figure 8 illustrates how the dual-phase cache works. It uses a Context Cache to accelerate compute-bound VLM prefill, and an Action Cache for memory-bound action denoising. The Context Cache reuses FFN outputs for similar visual tokens and recomputes those whose inputs have changed substantially. During action denoising, similar executions tend to activate overlapping sets of important neurons. The Action Cache uses these reference activation patterns to select which neurons to evaluate and which weights to load. Cache management supplies the dual-phase cache with reference FFN inputs and outputs and neuron masks from a shared library. To reduce memory overhead, the library retains compact reference inputs and neuron masks, and expanded FFN states are only reconstructed when a reference is likely to be retrieved. References are selected using visual context and robot-state history. A sliding GPU working set is further used to bound memory residency, combined with a reconstruction budget that limits reconstruction work. 4.1. Dual-Phase Cache Given a reference from the cache library, Context Cache uses its FFN inputs and outputs for visual-token recomputation. Action Cache uses neuron masks to select which important neurons to update relative to the current call’s dense anchors. The following mechanisms specify these computations. Visual-Token Recomputation. Our visual-token selective recomputation is inspired by prior work on token-level and similarity-based feature reuse (Hwang et al., 2025; Wei et al., 2025; Zou et al., 2025). We adapt these reuse principles to VLM FFNs across task executions: the Context Cache recomputes selected visual-token rows and reuses the remaining outputs from a retrieved reference execution. Input drift predicts which visual tokens account for most of the FFN output change, as measured in §3.3. For visual token p at layer l, let xl,px_l,p be the current FFN input after RMSNorm and let (x¯l,p,y¯l,p)( x_l,p, y_l,p) be the reference input and output. The runtime ranks visual tokens by (2) dl,p=RMS(xl,p−x¯l,p)max(RMS(xl,p),ε),ε=10−6.d_l,p= RMS(x_l,p- x_l,p) (RMS(x_l,p), ),\; =10^-6. For P visual tokens, the set l(ρp)P_l( _p) contains the ⌈ρpP⌉ _pP tokens with the largest scores. This selection uses only FFN inputs, so it does not require evaluating the current dense FFN to decide what to reuse. Selected visual rows are recomputed, and the remaining rows reuse their reference outputs: (3) y^l,p=Fl(xl,p),p∈l(ρp),y¯l,p,p∉l(ρp). y_l,p= casesF_l(x_l,p),&p _l( _p),\\ y_l,p,&p _l( _p). cases Instruction and robot-state rows are always recomputed. The selected rows pass through the FFN’s gate, up, activation, and down operations and are then scattered into a full output buffer initialized with the reference visual outputs. The profiled ratio ρp _p bounds FFN computation while allowing selected positions to vary by layer and policy call. Attention remains dense over the complete sequence. Group-Shared Neuron Masks. Identifying the exact important neurons for the current step through dense execution would defeat the reduction in weight traffic. A task-independent mask would instead ignore the task-dependent activation patterns observed in §3.4. rMuscle therefore predicts important neurons from a related reference and stores their indices with that reference’s inputs. A separate mask for every denoising step requires repeated weight gathering. Consecutive denoising steps can instead form a group that shares one mask, trading some step-specific coverage for less gather traffic. Figure 9 compares these choices for π0.5 _0.5. Two group-specific masks track step-specific important neurons more closely than one mask shared across all ten steps. The first and middle steps run densely, so mask coverage does not limit computation at those anchors. Offline mask-construction is described in §5. Important-neuron coverage across denoising steps for step-specific reference masks and masks shared within one or two groups. Figure 9. Important-neuron coverage with step-specific and group-shared masks, using top-k=2048k=2048 neurons.Important-neuron coverage across denoising steps for step-specific reference masks and masks shared within one or two groups. Anchors with Mask Recomputation. Each group begins with a dense anchor that establishes the current policy call’s activations. Let aga_g be the anchor step of group g. At each action-expert layer l, the anchor executes the dense FFN, producing hidden activations hag,lh_a_g,l and output yag,ly_a_g,l. The Action Cache retains the output and the activations selected by the group’s mask. In the ten-step π0.5 _0.5 configuration, step 0 anchors for steps 1–4, and step 5 anchors for steps 6–9. At a non-anchor step t, let M=ℳl(g)M=M_l^(g) be the retrieved reference mask. We compute only the selected neurons’ hidden activations ht,lMh_t,l^M and update the output using their down-projection weights Wl,MdownW_l,M^down: (4) y^t,l=yag,l+(ht,lM−hag,lM)Wl,Mdown. y_t,l=y_a_g,l+ (h_t,l^M-h_a_g,l^M )W_l,M^down. Subtracting the selected anchor contribution before adding its current value avoids double counting and preserves the anchor contribution of neurons outside M. The output follows the normal adaptive gate and residual addition, and attention remains dense. The reference determines which neurons to update, while all reused anchor values come from the current policy call. Selected weights are prepared in shared compact buffers and reused across each group’s steps, with gathering overlapped with model execution (§5). 4.2. Cache Management Cache management supplies the dual-phase cache with relevant reference data while bounding GPU residency and reconstruction work. It combines asynchronous online reconstruction, context-aware retrieval, and sliding-window prefetching guided by temporal locality. Direct storage retains expanded FFN states for every reference. Asynchronous reconstruction stores compact reference inputs and prepares selected FFN states while the robot executes actions. Figure 10. Direct storage versus asynchronous online reconstruction. Robot execution provides time to expand compact reference inputs before the next policy call.Direct storage retains expanded FFN states for every reference. Asynchronous reconstruction stores compact reference inputs and prepares selected FFN states while the robot executes actions. Asynchronous Online Reconstruction. Expanded VLM states are too large to store for every reference across all tasks. For three-view π0.5 _0.5, each of the 17 FFN layers retains a pair of [768,2048][768,2048] BF16 input and output tensors, approximately 107 MB per policy call. An episode of 40 policy calls therefore requires about 4.28 GB of intermediate states. Storing only compact inputs avoids this expansion, but reconstructing the states synchronously would put dense VLM computation back on the inference critical path. rMuscle instead reconstructs references during physical action execution, as shown in Figure 10. The repository retains reference model inputs and offline neuron masks. During an execution window, the GPU fetches inputs for upcoming candidate positions and runs VLM prefill densely to prepare their FFN states. Since neuron masks are computed offline, reconstruction does not rerun action denoising. Only the active working set holds expanded FFN states on the GPU. This organization reduces persistent host and disk storage, but does not reduce the GPU footprint of an individual reconstructed entry. The candidate positions are determined by the prefetching policy described below. Retrieving a Relevant Reference. The instruction first restricts retrieval to a compatible task pack. Visual matching uses the policy’s intermediate ViT features. For three-view π0.5 _0.5, the query is a BF16 tensor of shape [768,1152][768,1152] from the 15th of 27 vision layers. GPU search ranks reference positions by cosine similarity between flattened current and reference representations, retaining the top T candidates. It then compares their aligned robot-state histories of length L, represented as [L,14][L,14] tensors, and selects the candidate with the smallest motion distance to distinguish motions through similar scenes in different directions. §5 specifies the motion-distance calculation and fallback policy. Search is limited to references whose FFN states are already prepared on the GPU, so retrieval requires no transfer of expanded states on the critical path. The intermediate ViT query allows retrieval to overlap with the remaining vision encoding. §5 describes the stream schedule. Temporal Locality. Figure 11 compares reference and current episode progress. Matches concentrate along y=xy=x: as the current episode proceeds, observations change and retrieval advances to later references. In the measured traces, 96.6% of matches remain within approximately ten percentage points of aligned progress. This locality suggests that a small window of reference positions can cover the references likely to be needed next. Figure 11. Temporal locality of retrieved references. Matches track current progress along the diagonal.Normalized reference progress versus normalized current episode progress. Matches concentrate near the diagonal, with 96.6\% within approximately ten percentage points of aligned progress. Sliding Window Prefetching. We use this locality to maintain a sliding GPU working set across candidate episodes and nearby inference positions (Figure 12). The first policy call has neither a preceding execution window nor a previous match. Before an episode starts, rMuscle therefore loads the initial-position FFN states of the current task’s reference episodes. With the robot at its known initial pose, inference 0 selects the top T episodes by visual similarity. Only their upcoming positions are then reconstructed during the first action chunk. The initial visual matches select candidate episodes, and subsequent matched positions guide the window’s progress. In the representative configuration, the episode-axis width is T=4T=4. For each candidate episode, the window retains the current matched position and the next L=4L=4 positions, yielding T(L+1)=20T(L+1)=20 entries in total. The four-position lookahead corresponds to 10% of the maximum observed episode length of about 40 policy calls. The robot-state history length is also L=4L=4. rMuscle retains entries shared by successive windows, releases entries that leave the window, and reconstructs only newly entering positions. Thus, the persistent repository can grow without making every reference’s intermediate states GPU-resident. rMuscle also handles exceptional cases during sliding-window execution. If no suitable reference is available, it invokes the fallback policy. If the same reference is retrieved more often than the task’s maximum number of policy calls, rMuscle identifies the robot as stalled, resets the episode, and marks the episode as a failure. Figure 12. Mechanism of the sliding GPU working set.Instruction filtering selects a task pack. Initial-position states support the first policy call. A two-dimensional window then retains selected reference episodes and nearby positions, preparing only new positions as execution advances. Cache Entry Resident Count Budget. The cache-entry reconstruction count is bounded by candidate availability, free GPU memory, and the time before the next policy call. With the parameters in Table 2, the number of new entries is (5) Nprep=minNcand,⌊MfreeMentry⌋,⌊Tavl−TtransT¯rec⌋.N_prep= \! \N_cand, M_freeM_entry , T_avl-T_trans T_rec \. This expression estimates capacity from average reconstruction time rather than guaranteeing a hard deadline for each entry. Asynchronous inference may start the next policy call before the current action chunk finishes (Black et al., 2025), shortening the reconstruction window (TavlT_avl). rMuscle could adjust the number of reference entries prepared to fit the available time. Table 2. Parameters for the online reconstruction budget. Symbol Description NcandN_cand New candidate positions entering the window. MfreeM_free Free GPU memory after reserving model and workspaces. MentryM_entry Expanded intermediate-state footprint per entry. TavlT_avl Available reconstruction interval. TtransT_trans Input-transfer time for the proposed batch. T¯rec T_rec Average dense reconstruction time per reference. NprepN_prep Number of new entries reconstructed. 5. Implementation This section describes how rMuscle builds the cache offline and prepares and retrieves cached references to accelerate online inference. Profiling and Constructing the Cache. rMuscle uses two recomputation ratios: ρp _p for visual tokens and ρn _n for denoising neurons, both profiled offline for each model. Cache construction proceeds in two rounds. First, we sample task variants across object categories and partition the workspace by object pose, e.g., into four quadrants. We profile 2–8 episodes per region, depending on task difficulty, to estimate the recomputation ratios and populate the cache. Second, we cluster observations from these episodes and collect additional episodes for underrepresented clusters to improve scene coverage. This process yields 200–800 inference calls per task in our evaluation. Constructing Group Neuron Masks. To construct neuron masks, we run the cached inference offline and compute ct,l,nc_t,l,n using Equation 1. For each layer l, we select the top Kl=⌈ρnHl⌉K_l= _nH_l neurons at each denoising step. Within each denoising group, we take the union of these candidates, rank them by summed contribution, and retain the top KlK_l. The resulting mask is shared across non-anchor steps in the group. Cache Capacity and Reconstruction Budget. Online reconstruction is bounded by both execution time and GPU memory. In the three-view π0.5 _0.5 configuration, reconstructing one VLM cache entry takes about 25 ms, while the available GPU memory on an RTX 4090 can hold over one hundred reconstructed entries. A 15-action chunk executed at 30 Hz provides roughly 500 ms during which reconstruction can overlap with robot execution, allowing about 20 entries to be prepared before the next inference request. Since only compact reconstruction inputs are transferred, at about 1.7 MB per entry, data movement contributes little additional overhead. We therefore determine the reconstruction budget from the available execution interval, memory capacity, and measured transfer time. Joint-History Retrieval. Subsequent calls retrieve candidate positions from the top T episodes selected at inference 0. These positions are reranked by the RMS joint-angle difference between query and candidate histories, computed over aligned history steps and each arm’s joints (e.g., six per arm for ALOHA). This distance distinguishes execution states that may share similar end-effector poses but differ in joint configurations. Single-arm tasks use the active arm’s distance, while dual-arm tasks use the larger of the two arm distances. The closest candidate provides the cache payload. Fallback policy. If the visual cosine similarity is below 0.8, rMuscle switches to dense execution for the remainder of the episode and retries cache reuse in the next episode. Overlapping Retrieval and Weight Gathering. Once intermediate ViT features are available, a side CUDA stream retrieves a prepared reference while the main stream completes vision encoding. The side stream then gathers selected neuron weights, preparing the first denoising group before the second, while the main stream executes VLM prefill. CUDA events ensure that retrieval completes before VLM FFN reuse and that each denoising FFN waits until its required weights are ready. 6. Evaluation We evaluate rMuscle to answer three major questions: • How much speedup does it provide, and how much does each optimization contribute? (§6.2 & §6.5) • Does it preserve task success rates across diverse tasks, and how do cache configurations affect this? (§6.3 & §6.5) • What additional memory overhead does it incur? (§6.4) 6.1. Experimental Setup Hardware and Models. We use a desktop with an RTX 4090 (24 GB GPU memory), an Intel Core i7-11700, and 32 GB RAM, as well as a Jetson Thor with 128 GB unified memory. They run PyTorch 2.6.0/CUDA 12.4 and PyTorch 2.8.0/CUDA 13.0, respectively. We evaluate π0.5 _0.5, GR00T N1.6, and X-VLA (Physical Intelligence et al., 2025; NVIDIA GEAR Team, 2025; Zheng et al., 2025) on LIBERO, and π0.5 _0.5 on RoboTwin, following checkpoint availability. All methods use BF16 or FP16 without quantization. All π0.5 _0.5 latencies use three camera inputs, masking the unused view on two-camera LIBERO. These three models represent distinct VLA architectures. π0.5 _0.5 is VLM-heavy, so Context Cache reuse has the larger opportunity. GR00T N1.6 is denoising-heavy, so Action Cache reuse and denoising-step skipping have the larger opportunity. X-VLA has a lightweight VLM, but its visual features are concatenated with the denoising actions and processed by self-attention. We therefore apply visual-token sparsity and neuron sparsity together inside the denoising loop. Simulation Benchmarks. LIBERO (Liu et al., 2023) covers 10 tasks in each of its Spatial, Object, Goal, and Long suites, with 100 seeded rollouts per task under the standard initialization protocol. For RoboTwin (Mu et al., 2024), we use the clean setting across 50 tasks, with 100 seeded rollouts per task varying object instances and poses. Physical Tasks. Two physical tasks are included. In cooperative pick-and-place, a dual-arm ALOHA (Fu et al., 2024) places two beverage bottles into a box, varying lighting as well as bottle and box positions across trials (Figure 13). In assembly-line packing, Franka and DOBOT arms (Franka Robotics, 2026; DOBOT Robotics, 2026) pack boards and instruction booklets for six furniture types into boxes on an 8 m/min conveyor, with randomized lighting and box poses (Figure 14). Both tasks run at 30 Hz with 30-action chunks and 50 trials per method. Figure 13. Cooperative task with the dual-arm ALOHA.Three views of the cooperative task show the initial bottle positions, the dual-arm grasp, and the bottles moved toward the storage box. (a) Conveyor setup and packing in progress. (b) Box before and after packing. The top panel shows the robot and moving conveyor during furniture packing. The bottom panel compares an empty box with a box containing aligned furniture boards and an instruction booklet. Figure 14. Packing furniture components. The robot aligns boards with a moving box and adds a manual on top. Baselines. torch.compile, vla.cpp (Nguyen et al., 2026), and FlashRT (Su, 2026) are included as inference-acceleration baselines that preserve model accuracy. vla.cpp is unsupported on Jetson Thor and omitted there. NIRVANA (Agarwal et al., 2024), DP-Cache (Zhou et al., 2026), and rMuscle are all implemented on FlashRT for π0.5 _0.5 and GR00T N1.6, and on torch.compile for X-VLA. We also evaluate a variant that uses the previous inference as its reference, following VLA-Cache (Xu et al., 2025). We refer to this variant as rMuscle w/o Cache Repo. Cache Settings. Table 3 shows the π0.5 _0.5–RoboTwin settings for the different methods. DP-Cache uses default settings, with dense VLM prefill and the latest denoising output reused at skipped steps. Our NIRVANA adaptation keeps VLM prefill dense and reuses a prefix of reference denoising outputs. It shares rMuscle’s retrieval method and reference repository of about 8–32 episodes per task. Profiling reduces NIRVANA’s reused prefix to one step for GR00T N1.6 and four for X-VLA, as half-schedule reuse degrades success. Table 3. Settings of different methods for π0.5 _0.5 on RoboTwin. Method Setting Value Shared Denoising steps / chunk size 1010 / 3232 DP-Cache Cached steps 0,1,2,6,8,9\0,1,2,6,8,9\ NIRVANA Reused steps first 55 rMuscle (ρp,ρn)( _p,\, _n) (0.4, 0.5)(0.4,\,0.5) Metrics. Latency spans from ready inputs to returned action chunks, including online retrieval and weight gathering, after 20 warmups. We report success rate (SR, %) across all episodes. Mean policy calls include only successful episodes, since failures run to a step limit far beyond typical completion in both simulation and physical experiments. Vanilla denotes dense execution. Methods share initializations and checkpoints. Two stacked panels of grouped bars show mean inference latency measured during benchmark evaluation on RTX 4090 (top) and Jetson Thor (bottom) for pi0.5, GR00T N1.6, and X-VLA, with speedups relative to FlashRT for pi0.5 and GR00T, and torch.compile for X-VLA. Figure 15. Mean inference latency measured during benchmark evaluation on (a) RTX 4090 and (b) Jetson Thor.Two stacked panels of grouped bars show mean inference latency measured during benchmark evaluation on RTX 4090 (top) and Jetson Thor (bottom) for pi0.5, GR00T N1.6, and X-VLA, with speedups relative to FlashRT for pi0.5 and GR00T, and torch.compile for X-VLA. 6.2. Inference Performance Figure 15 shows that rMuscle reduces latency relative to each model’s fastest exact baseline on both platforms. Specifically, on RTX 4090, rMuscle reaches inference rates of 28.1, 51.0, and 35.7 Hz for π0.5 _0.5, GR00T N1.6, and X-VLA, respectively. Speedups over the corresponding SOTA inference engine are 1.29×1.29×, 1.20×1.20×, and 1.50×1.50×. On Jetson Thor, the corresponding inference rates are 13.4, 17.5, and 19.7 Hz, with speedups of 1.42×1.42×, 1.23×1.23×, and 1.43×1.43×, respectively. Compared with denoising step-skipping methods, rMuscle also achieves inference speedup by accelerating both VLM prefill and action denoising. Although step-skipping methods can achieve greater speedup in the denoising stage by aggressively skipping denoising steps, this comes at the cost of reduced task success rates. Scaling Model Sizes. Scaling up VLA models can improve task performance (Generalist Team, 2025), but also increases inference cost. We therefore examine how inference latency and the speedup provided by rMuscle change with model size. Keeping the visual tokenizer fixed, we scale both components within the Gemma family: (VLM, denoise) sizes of (2B, 0.3B), (2B, 2B), (7B, 2B), and (7B, 7B) correspond to π0.5 _0.5, π0.5 _0.5-L, π0.5 _0.5-XL, and π0.5 _0.5-XXL, respectively. For Gemma-7B, we use only the first 18 layers for visual-feature extraction. End-to-end latency for four VLM and denoiser size pairs on RTX 4090 and Jetson Thor. Speedups over the corresponding dense baseline are approximately 1.3, 1.4, 1.6, and 1.6 on RTX 4090, and 1.4, 1.4, 1.7, and 1.9 on Thor. Figure 16. Inference latency of rMuscle across different model sizes on (a) RTX 4090 and (b) Jetson Thor.End-to-end latency for four VLM and denoiser size pairs on RTX 4090 and Jetson Thor. Speedups over the corresponding dense baseline are approximately 1.3, 1.4, 1.6, and 1.6 on RTX 4090, and 1.4, 1.4, 1.7, and 1.9 on Thor. Figure 16 shows how speedup over the FlashRT baseline changes with model size. From the smallest to the largest configuration, speedup rises from 1.3×1.3× to 1.6×1.6× on RTX 4090 and from 1.4×1.4× to 1.9×1.9× on Thor. These results suggest that the dual-phase cache can provide greater relative benefits as VLA models scale up. 6.3. Policy Quality rMuscle achieves the lowest latency among the compared methods. We next evaluate its policy quality in simulation and on physical robots. Simulation. On LIBERO, Figure 17 shows that rMuscle matches the average success rates of vanilla and DP-Cache across π0.5 _0.5, GR00T N1.6, and X-VLA. It exceeds NIRVANA by 0.3, 1.1, and 0.3 points on the three models, respectively. Figure 17. LIBERO results over four methods. SR axes span 85–100%. Calls axes span 15–16, 21–23, and 5–6 in panels (a)–(c), respectively. Table 4. RoboTwin results over all 50 tasks, with blue/pink shading for the best/worst values per row. Task Vanilla DP-Cache NIRVANA rMuscle Click Alarmclock 36 34 35 42 Move Can Pot 22 22 23 28 Adjust Bottle 99 98 86 99 Handover Block 14 4 9 12 Handover Mic 45 31 30 47 … (50 tasks) Move Pillbottle Pad 24 20 21 28 Place Empty Cup 39 39 28 36 Dump Bin Bigbin 58 59 53 60 Stamp Seal 17 17 8 19 Turn Switch 24 26 20 28 Average SR (%) ↑ 35.2 33.9 26.9 35.2 Calls ↓ 7.0 7.6 7.5 7.1 Table 5. π0.5 _0.5 on physical tasks. Blue/pink shading marks the best/worst values per row. Metric Vanilla DP-Cache NIRVANA rMuscle ALOHA: Cooperative bottle pick-and-place SR (%) ↑ 76 74 70 76 Calls ↓ 38.7 39.5 40.0 39.1 DOBOT + FRANKA: Conveyor furniture packing SR (%) ↑ 84 80 66 84 Calls ↓ 10.1 10.5 10.7 10.2 On RoboTwin, Table 4 shows that rMuscle matches vanilla’s average success rate of 35.2%, exceeding DP-Cache and NIRVANA by 1.3 and 8.3 points, respectively. It trails vanilla by 2 and 3 points on Handover Block and Place Empty Cup due to corner cases with observation similarity near 0.8, just above the fallback threshold. In follow-up tests, raising this threshold or adding two relevant reference episodes resolves these failures, while NIRVANA still fails. These results highlight the importance of fallback and reference coverage in preserving policy quality. Physical Robots. Table 5 shows that rMuscle matches vanilla execution on both physical tasks: 76% on ALOHA cooperative bottle pick-and-place and 84% on DOBOT and Franka conveyor packing. It exceeds DP-Cache by 2 and 4 points, and NIRVANA by 6 and 18 points on the two tasks, respectively. 6.4. Memory Overhead We quantify the CPU cache footprint of rMuscle and compare its GPU memory use with vanilla execution. To represent increasing reference requirements for more difficult tasks, we increase the per-task cache capacity from 200 entries across 8 reference episodes to 800 entries across 32 reference episodes, while expanding the candidate window from T=L=1T=L=1 to T=L=4T=L=4. Figure 18. Memory use as a share of device memory on (a) RTX 4090 and (b) Jetson Thor. T and L determine the GPU candidate working-set size, T(L+1)T(L+1). C denotes the total number of cached inference entries for the current task.Memory use as a share of device memory across four cache configurations. On RTX 4090, GPU memory rises from 39.4\% to 47.4\% of 24 GB, compared with 35.2\% for vanilla, an increase of 4.2 to 12.2 percentage points, while the CPU cache occupies 3.7\% to 14.9\% of 32 GB. On Jetson Thor, unified memory rises from 8.3\% to 12.6\% of 128 GB, compared with 6.6\% for vanilla, an increase of 1.7 to 6.0 percentage points. Figure 18 shows that on RTX 4090, the CPU cache accounts for 3.7–14.9% of 32 GB host memory. GPU memory increases from 8.4 GB for vanilla to 9.4–11.4 GB, raising occupancy by less than 13%. On Jetson Thor, where CPU and GPU share 128 GB unified memory, occupancy increases from 6.6% to 8.3–12.6%. Overall, rMuscle supports up to 800 cached inference entries with less than 3 GB additional GPU memory on RTX 4090 and at most 6.0 percentage points additional unified-memory occupancy on Jetson Thor. 6.5. Ablation Studies We evaluate cache contributions and quality–latency tradeoffs using π0.5 _0.5 on RoboTwin. Cache Contributions. We compare FlashRT, Context Cache alone, Action Cache alone, and the full system in Figure 19 to isolate the latency contribution of each cache on RTX 4090 and Jetson Thor. Context Cache alone gives the larger end-to-end speedup: 1.25×1.25× on RTX 4090 and 1.22×1.22× on Jetson Thor versus FlashRT. With Context Cache, the reduction in VLM prefill latency accounts for 87% of the total latency reduction on RTX 4090, compared with 60% on Jetson Thor. Action Cache alone speeds up action denoising by 1.11×1.11× on RTX 4090 and 1.49×1.49× on Jetson Thor, consistent with the more memory-bound Thor profile in Table 1. Figure 19. Latency breakdown for VLA inference phases. Reference Repository. The rMuscle w/o Cache Repo variant uses the previous policy call within the same episode as its reference instead of retrieving one from the repository. As shown in Figure 20, Default achieves a success rate of 35.2%, compared with 25.9% without the reference repository, at similar latency. These results highlight the value of references from prior executions in preserving policy quality, while the similar latency suggests that cache management adds negligible overhead to the critical path. Quality–Performance Tradeoff. We vary the visual-token recomputation ratio and the neuron fraction of rMuscle. Figure 20 compares the Fast, Default, and Slow configurations on RTX 4090 alongside Vanilla, DP-Cache, NIRVANA, and the repository ablation discussed above. Figure 20. Quality–latency tradeoff on RTX 4090. Numeric labels report average calls. (ρp,ρn)( _p, _n) configurations: Fast (0.2,0.25)(0.2,0.25), Default (0.4,0.50)(0.4,0.50), Slow (0.6,0.75)(0.6,0.75). Default increases success from 23.0% to 35.2% compared with Fast, at an additional latency of 4.9 ms. Default matches Vanilla’s success rate, while Slow achieves a slightly higher success rate. They reduce latency relative to Vanilla by 10.5 ms and 4.5 ms, respectively. These results meet our design goal of preserving policy quality while reducing inference latency. Consistent with our profiling results, Default offers the lowest latency without sacrificing success. 7. Related Work VLA Profiling and Inference Engines. VLA-Perf analytically explores how architecture, hardware placement, and serving choices affect inference latency (Jiang et al., 2026), while a cross-XPU study measures energy and latency across heterogeneous accelerators and validates DP-Cache on LIBERO and a physical Franka robot (Zhou et al., 2026). Inference engines optimize how model operations execute on the target hardware. The vla.cpp runtime provides a portable C++ implementation for multiple VLA architectures and hardware tiers, including native support for iterative diffusion and flow-matching action heads (Nguyen et al., 2026). RealVLA removes inference overhead and integrates the optimized model path with streaming robot control (Ma et al., 2025). FlashRT uses specialized kernels, operator fusion, and whole-forward CUDA Graph replay, together with precision-specific execution modes, for latency-sensitive VLA serving (Su, 2026). These systems reduce framework, kernel, and scheduling overhead without determining which visual tokens or denoising computations may be reused. Approaches to VLA Acceleration. VLA accelerators reduce multimodal or action-generation work through pruning, approximate speculative execution, and efficient model design (Guan et al., 2025). VLA-Cache accelerates attention computation by reusing cached KV states of static visual tokens from the preceding frame (Xu et al., 2025). However, attention-only reuse does not target the dominant FFN bottlenecks in the state-of-the-art VLA models studied here. SpecPrune-VLA uses recent history and action-aware control to prune visual tokens, while Realtime-VLA FLASH combines preceding visual context with draft-model speculation and action-expert verification (Wang et al., 2026; Niu et al., 2026). Neither searches a persistent repository. For action generation, TS-DP couples a distilled diffusion drafter with a learned scheduler (Li et al., 2025), whereas RoboMamba builds an efficient policy around a Mamba state-space backbone and a lightweight policy head (Liu et al., 2024). 8. Discussion Why Inference Latency Matters. Policy latency is the gap between the last action of one chunk and the first action of the next, while actions within a chunk are executed at 20–30 Hz. As policy-call frequency approaches the execution rate, chunk boundaries become smoother, and the robot can re-observe and react to environmental changes more frequently. Lower inference latency therefore improves responsiveness, motion smoothness, and task throughput. Since each episode requires 4–40 policy calls, reducing per-call latency shortens the overall task cycle and enables a fixed-workstation robot to complete more tasks per hour on the same hardware. Dynamic Cache Update. Unseen tasks initially run densely, and successful episodes naturally provide new cache references at no additional computation cost. Successful episodes from underrepresented observation clusters could be admitted. A new task variant therefore transitions from dense execution to cached acceleration as its reference coverage improves. We leave online cache updates for future work. 9. Conclusion This paper introduces rMuscle, the fastest VLA inference engine optimized for embodied workloads. Through an in-depth analysis of robot execution patterns, we find that repeated task executions exhibit similarity not only in observations and action trajectories, but also in internal model states throughout inference. rMuscle exploits these similarities through a dual-cache design that reduces VLM computation and denoising weight traffic. It further combines online cache reconstruction, sliding-window cache retrieval, and mask sharing across denoising steps to keep the memory footprint and cache-access overhead negligible. rMuscle achieves a 1.291.29–1.42×1.42× end-to-end speedup on RTX 4090 and Jetson Thor over the state-of-the-art inference engine, while preserving the original success rates on both simulation benchmarks and physical-robot tasks. References Agarwal et al. (2024) S. Agarwal, S. Mitra, S. Chakraborty, et al. Approximate caching for efficiently serving text-to-image diffusion models. In USENIX Symposium on Networked Systems Design and Implementation (NSDI), Cited by: §2.5, §6.1. AgiBot (2026) AgiBot AGIBOT and Longcheer Technology achieve world’s first embodied AI deployment in consumer electronics precision manufacturing mass-production line. Note: https://w.agibot.com/article/231/detail/60.html Cited by: §1, §2.1. Black et al. (2024) K. Black, N. Brown, D. Driess, et al. π0 _0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164. Cited by: §1. Black et al. (2025) K. Black, M. Y. Galliker, and S. Levine Real-time execution of action chunking flow policies. External Links: 2506.07339, Link Cited by: §4.2. Brohan et al. (2023) A. Brohan, N. Brown, J. Carbajal, et al. RT-2: vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818. Cited by: §2.3. Bu et al. (2025) Q. Bu, J. Cai, L. Chen, et al. AgiBot World Colosseo: a large-scale manipulation platform for scalable and intelligent embodied systems. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Cited by: §1. Cheang et al. (2025) C. Cheang, S. Chen, Z. Cui, et al. GR-3 technical report. arXiv preprint arXiv:2507.15493. External Links: Link Cited by: §1. Community (2026) S. Community StarVLA: a lego-like codebase for vision-language-action model developing. External Links: 2604.05014, Link Cited by: §2.4. DOBOT Robotics (2026) DOBOT Robotics DOBOT Robotics: collaborative robot arms. Note: https://w.dobot-robots.com/Official website for DOBOT collaborative and industrial robotic arms Cited by: §1, §6.1. Figure AI (2025) Figure AI F.02 contributed to the production of 30,000 cars at BMW. Note: https://w.figure.ai/news/production-at-bmw Cited by: §1, §2.1. Franka Robotics (2026) Franka Robotics Franka research 3. Note: https://franka.de/franka-research-3Seven-degree-of-freedom, force-sensitive robot system designed for robotics and AI research Cited by: §1, §6.1. Fu et al. (2024) Z. Fu, T. Z. Zhao, and C. Finn Mobile aloha: learning bimanual mobile manipulation with low-cost whole-body teleoperation. In Conference on Robot Learning (CoRL), Cited by: §1, §6.1. Generalist Team (2025) Generalist TeamGEN-0: embodied foundation models that scale with physical interaction(Website) Generalist AI. External Links: Link Cited by: §6.2. Guan et al. (2025) W. Guan, Q. Hu, A. Li, and J. Cheng Efficient vision-language-action models for embodied manipulation: a systematic survey. External Links: 2510.17111, Link Cited by: §7. Heald et al. (2021) J. B. Heald, M. Lengyel, and D. M. Wolpert Contextual inference underlies the learning of sensorimotor repertoires. Nature 600, p. 489–493. Cited by: §1. Hwang et al. (2025) J. Hwang, D. Kim, S. Lee, Y. Kim, G. Heo, H. Kim, Y. Jeong, T. Meaza, E. Park, J. Ahn, and J. Park Déjà vu: efficient video-language query engine with learning-based inter-frame computation reuse. External Links: 2506.14107, Link Cited by: §4.1. Imamizu et al. (2000) H. Imamizu, S. Miyauchi, T. Tamada, Y. Sasaki, R. Takino, B. Pütz, T. Yoshioka, and M. Kawato Human cerebellar activity reflecting an acquired internal model of a new tool. Nature 403, p. 192–195. Cited by: §1. Jiang et al. (2026) W. Jiang, J. Clemons, K. Sankaralingam, and C. Kozyrakis How fast can i run my vla? demystifying vla inference performance with vla-perf. External Links: 2602.18397, Link Cited by: §3.1, §7. Kim et al. (2024) M. J. Kim, K. Pertsch, S. Karamcheti, et al. OpenVLA: an open-source vision-language-action model. In Conference on Robot Learning (CoRL), Cited by: §2.3. Labs (2024) B. F. Labs FLUX. Note: https://github.com/black-forest-labs/flux Cited by: §2.5. Li et al. (2024) Q. Li, Y. Liang, Z. Wang, et al. CogACT: a foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650. Cited by: §1. Li et al. (2025) Y. Li, J. Feng, Y. Meng, K. Ji, C. Tang, X. Wen, S. Xia, Z. Wang, and W. Zhu TS-dp: reinforcement speculative decoding for temporal adaptive diffusion policy acceleration. External Links: 2512.15773, Link Cited by: §7. Liu et al. (2023) B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone LIBERO: benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems (NeurIPS), External Links: Link Cited by: §1, §6.1. Liu et al. (2024) J. Liu, M. Liu, Z. Wang, P. An, X. Li, K. Zhou, S. Yang, R. Zhang, Y. Guo, and S. Zhang RoboMamba: efficient vision-language-action model for robotic reasoning and manipulation. External Links: 2406.04339, Link Cited by: §7. Ma et al. (2025) Y. Ma, Y. Zhou, Y. Yang, T. Wang, and H. Fan Running vlas at real-time speed. External Links: 2510.26742, Link Cited by: §7. Mu et al. (2024) Y. Mu, T. Chen, S. Peng, Z. Chen, Z. Gao, Y. Zou, L. Lin, Z. Xie, and P. Luo RoboTwin: dual-arm robot benchmark with generative digital twins. External Links: 2409.02920, Link Cited by: §1, §6.1. Nguyen et al. (2026) K. D. Nguyen, H. T. Ho, C. T. Nguyen, T. Q. Duong, L. D. Le, D. M. H. Nguyen, V. A. Ngo, and A. T. Le Vla.cpp: a unified inference runtime for vision-language-action models. External Links: 2606.08094, Link Cited by: §1, §6.1, §7. Niu et al. (2026) J. Niu, K. Gu, Y. Zhao, S. Liang, T. Wang, X. Hu, Y. Wang, and H. Li Realtime-vla flash: speculative inference framework for diffusion-based vlas. External Links: 2605.13778, Link Cited by: §7. NVIDIA GEAR Team (2025) NVIDIA GEAR Team GR00T N1.6: an improved open foundation model for generalist humanoid robots. Note: NVIDIA Research External Links: Link Cited by: §2.3, §2.5, §6.1. NVIDIA (2025) NVIDIA GR00T N1: an open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734. Cited by: §1, §2.4. Park et al. (2025) K. Park, J. Kulick, A. Melkozerov, R. A. Vilagrasa, T. S. Lembono, V. Neubauer, A. Minichev, K. Turner, O. Agrigoroaiei, P. Klink, J. Lee, K. Dwivedi, M. Eltayeb, I. Posner, A. Parness, and C. Erdogan Vulcan Pick: a robotic system for picking targeted objects from fabric pods. External Links: Link Cited by: §2.1. Physical Intelligence et al. (2025) Physical Intelligence, K. Black, N. Brown, et al. π0.5 _0.5: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054. Cited by: §1, §1, §2.3, §2.4, §6.1. Shukor et al. (2025) M. Shukor, D. Aubakirova, F. Capuano, P. Kooijmans, S. Palma, A. Zouitine, M. Aractingi, C. Pascal, M. Russi, A. Marafioti, S. Alibert, M. Cord, T. Wolf, and R. Cadene SmolVLA: a vision-language-action model for affordable and efficient robotics. External Links: 2506.01844, Link Cited by: §1. Su (2026) L. Su Execution-state capsules: graph-bound execution-state checkpoint and restore for low-latency, small-batch, on-device physical-ai serving. External Links: 2606.20537, Document, Link Cited by: §1, §1, §6.1, §7. Summerfield et al. (2008) C. Summerfield, E. H. Trittschuh, J. M. Monti, M. Mesulam, and T. Egner Neural repetition suppression reflects fulfilled perceptual expectations. Nature Neuroscience 11 (9), p. 1004–1006. Cited by: §1. Sun et al. (2024) M. Sun, Z. Liu, A. Bair, and J. Z. Kolter A simple and effective pruning approach for large language models. In International Conference on Learning Representations (ICLR), External Links: Link Cited by: §3.4. Vaziri (2026) A. Vaziri Tesla to kill off model S and X vehicles, convert Fremont factory to build robots. Note: https://w.sfchronicle.com/tech/article/tesla-end-model-s-x-21320796.php Cited by: §1, §2.1. Wan et al. (2025) T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, J. Zeng, J. Wang, J. Zhang, J. Zhou, J. Wang, J. Chen, K. Zhu, K. Zhao, K. Yan, L. Huang, M. Feng, N. Zhang, P. Li, P. Wu, R. Chu, R. Feng, S. Zhang, S. Sun, T. Fang, T. Wang, T. Gui, T. Weng, T. Shen, W. Lin, W. Wang, W. Wang, W. Zhou, W. Wang, W. Shen, W. Yu, X. Shi, X. Huang, X. Xu, Y. Kou, Y. Lv, Y. Li, Y. Liu, Y. Wang, Y. Zhang, Y. Huang, Y. Li, Y. Wu, Y. Liu, Y. Pan, Y. Zheng, Y. Hong, Y. Shi, Y. Feng, Z. Jiang, Z. Han, Z. Wu, and Z. Liu Wan: open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. Cited by: §2.5. Wang et al. (2026) H. Wang, J. Xu, Y. Xiang, J. Pan, Y. Zhou, Y. Li, and G. Dai SpecPrune-vla: accelerating vision-language-action models via action-aware self-speculative pruning. In International Conference on Machine Learning (ICML), External Links: Link Cited by: §7. Wei et al. (2025) C. Wei, C. Guo, J. Zhang, H. Shan, Y. Xu, Z. Zhang, Y. Liu, Q. Wang, C. Zhou, H. ". Li, and Y. Chen Focus: a streaming concentration architecture for efficient vision-language models. External Links: 2512.14661, Link Cited by: §4.1. Xia et al. (2026) Y. Xia, D. Sharma, Y. Yuan, S. Kundu, and N. Talati MoDM: efficient serving for image generation via mixture-of-diffusion models. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), ASPLOS ’26, Vol. 1, New York, NY, USA, p. 163–182. External Links: ISBN 9798400721656, Link, Document Cited by: §2.5. Xu et al. (2025) S. Xu, Y. Wang, C. Xia, et al. VLA-Cache: efficient vision-language-action manipulation via adaptive token caching. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §6.1, §7. Yang et al. (2025) Y. Yang, Y. Wang, Z. Wen, et al. EfficientVLA: training-free acceleration and compression for vision-language-action models. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1. Zhao et al. (2025) X. Zhao, X. Jin, K. Wang, and Y. You Real-time video generation with pyramid attention broadcast. In International Conference on Learning Representations (ICLR), External Links: Link Cited by: §2.5. Zheng et al. (2025) J. Zheng, J. Li, Z. Wang, et al. X-VLA: soft-prompted transformer as scalable cross-embodiment vision-language-action model. arXiv preprint arXiv:2510.10274. Cited by: §1, §2.3, §2.4, §6.1. Zhou et al. (2026) K. Zhou, Q. Chen, D. Peng, Z. Li, X. Li, and J. Gu Characterizing vision-language-action models across XPUs: constraints and acceleration for on-robot deployment. In International Conference on Machine Learning (ICML), External Links: Link Cited by: §1, §2.5, §3.1, §6.1, §7. Zhu et al. (2026) B. Zhu, P. Schmitt, P. Meister, L. Gensler, M. Khalil, E. Poggi, J. Hechtl, C. Braunroth, K. Wurm, G. Narayanan, E. Solowjow, G. von Wichert, A. Scholz, F. Albrecht, and M. Metzner A factory-floor deployment case study of VLA pipelines for industrial packaging task: workflow, failures, and lessons. External Links: 2605.27461, Link Cited by: §2.1. Zou et al. (2025) C. Zou, X. Liu, T. Liu, S. Huang, and L. Zhang Accelerating diffusion transformers with token-wise feature caching. In International Conference on Learning Representations (ICLR), External Links: Link Cited by: §4.1.