Paper deep dive
Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures
Manoj Vishwanathan, Suvinay Subramanian, Anand Raghunathan
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 7/20/2026, 4:10:12 AM
Summary
This paper characterizes Vision-Language-Action (VLA) models for edge AI, identifying the memory-bound action generation phase as the primary bottleneck, consuming up to 75% of end-to-end latency. Using MolmoAct-7B on Nvidia Jetson Orin and Thor platforms, the authors demonstrate that current hardware is insufficient for real-time 10-20 Hz control frequencies. They project hardware requirements for scaling to 100B parameter models, suggesting high-bandwidth memory (GDDR7) and Processing-in-Memory (PIM) as critical pathways to bridge the latency gap.
Entities (8)
Relation Signals (7)
MolmoAct-7B → runson → Nvidia Jetson Orin
confidence 95% · We profile the MolmoAct-7B model on contemporary Nvidia Jetson Orin and Thor hardware platforms
MolmoAct-7B → runson → Nvidia Jetson Thor
confidence 95% · We profile the MolmoAct-7B model on contemporary Nvidia Jetson Orin and Thor hardware platforms
Action Generation → isbottleneckfor → Vision-Language-Action (VLA) models
confidence 92% · we identify a primary execution bottleneck: up to 75% of end-to-end latency is consumed by the memory-bound action-generation phase.
Nvidia Jetson Thor → hasmemorybandwidth → 273 GB/s
confidence 90% · Table 1. Commercial edge platforms... Thor LPDDR5X 273
Nvidia Jetson Orin → hasmemorybandwidth → 203 GB/s
confidence 90% · Table 1. Commercial edge platforms... Orin LPDDR5 203
Processing-in-Memory (PIM) → improvesperformanceof → Vision-Language-Action (VLA) models
confidence 85% · explore the impact of high-bandwidth memory technologies and processing-in-memory (PIM) as promising future pathways
GDDR7 → improvesperformanceof → Vision-Language-Action (VLA) models
confidence 85% · improved bandwidth from GDDR7 and PIM memories substantially improves performance
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Vision-Language-Action (VLA) models are an emerging class of workloads critical for robotics and embodied AI at the edge. As these models scale, they demonstrate significant capability gains, yet they must be deployed locally to meet the strict latency requirements of real-time applications. This paper characterizes VLA performance on two generations of edge hardware, viz. the Nvidia Jetson Orin and Thor platforms. Using MolmoAct-7B, a state-of-the-art VLA model, we identify a primary execution bottleneck: up to 75% of end-to-end latency is consumed by the memory-bound action-generation phase. Through analytical modeling and simulations, we project the hardware requirements for scaling to 100B parameter models. We also explore the impact of high-bandwidth memory technologies and processing-in-memory (PIM) as promising future pathways in edge systems for embodied AI.
Tags
Links
- Source: https://arxiv.org/abs/2603.02271v1
- Canonical: https://arxiv.org/abs/2603.02271v1
Trouble viewing inline? Open PDF directly →
Full Text
11,130 characters extracted from source content.
Expand or collapse full text
Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures Manoj Vishwanathan 1,2 , Suvinay Subramanian 1 , and Anand Raghunathan 2 1 Google, Mountain View, CA USA 2 Purdue University, West Lafayette, IN USA Abstract Vision-Language-Action (VLA) [1] models are an emerg- ing class of workloads critical for robotics and embodied AI at the edge. As these models scale, they demonstrate significant capability gains, yet they must be deployed locally to meet the strict latency requirements of real-time applications. This paper characterizes VLA performance on two generations of edge hardware, viz. the Nvidia Jet- son Orin [6] and Thor [7] platforms. Using MolmoAct-7B [4], a state-of-the-art VLA model, we identify a primary execution bottleneck: up to 75% of end-to-end latency is consumed by the memory-bound action-generation phase. Through analytical modeling and simulations, we project the hardware requirements for scaling to 100B parameter models. We also explore the impact of high-bandwidth memory technologies and processing-in-memory (PIM) [3] as promising future pathways in edge systems for embodied AI. 1 Problem Statement Vision-Language-Action (VLA) models represent a trans- formative shift in embodied AI, enabling robots to move from rigid pre-programmed routines to generalized se- mantic reasoning. As with Large Language Models (LLMs), the advancement of these models is increasingly gov- erned by neural scaling laws; recent research suggests that robotic task performance improves at a power-law rate [8]. To achieve true general-purpose utility in com- plex, real-world environments and tasks, models must scale to 10—100B parameters to effectively synthesize world knowledge with precise sensorimotor control. This imperative for scaling, however, faces a critical hardware barrier in real-time deployment. Safe, dynamic manipulation in physical environments requires a con- sistent control frequency of at least 10–20 Hz. Contem- porary edge accelerators are structurally ill-equipped for the sparse, memory-bound autoregressive processing re- quired for VLA action generation. While state-of-the-art models like Gemini Robotics 1.5 [2] have demonstrated impressive reasoning through "two-brain" architectures, these rely on offloading capable models to powerful off- board servers while maintaining only rudimentary control loops locally. Such hybrid configurations are effective for demonstrations, but they remain far from feasible in the context of low-latency, fully autonomous edge systems. Figure 1. VLA System Architecture. The work- load consists of a Vision Encoder, a Generation En- gine (autoregressive decoding), and an Action Trans- former. This paper provides a systematic characterization of VLA workloads to guide the next generation of edge sys- tem designs. We profile the MolmoAct-7B model on con- temporary Nvidia Jetson Orin and Thor hardware plat- forms to establish a baseline and identify key execution bottlenecks. Building on these measurements, we lever- age an in-house simulator to project the performance of future VLA models across current and hypothetical future hardware configurations. 2 Background and Related Work: Vision-Language-Action (VLA) Models Vision-Language-Action (VLA) models are a class of multi- modal foundation models that integrate visual perception, natural language understanding, and physical action. As illustrated in Figure 1 , the computational architecture of a VLA is structured into three primary subsystems: •Vision Encoder (Perception Core): This stage pro- cesses raw pixel data into structured feature em- beddings. To capture both semantic context and spatial-geometric detail, modern VLAs often uti- lize fused backbones such as SigLIP and DINOv2. These high-dimensional features are mapped into the embedding space of the reasoning engine via a projector module, typically a multi-layer MLP. •Generation (Reasoning Engine): The core of the model is a decoder-only Transformer that processes a concatenated sequence of visual and textual to- kens. During this phase, the model performs cross- modal reasoning and may generate intermediate outputs, such as "Chain-of-Thought" (CoT) reason- ing or spatial waypoints, to decompose high-level instructions into executable plans. •Action Transformer: The final stage translates the model’s internal representations into motor com- mands. In discrete action tokenization, the robot’s arXiv:2603.02271v1 [cs.PF] 1 Mar 2026 Vishwanathan, Subramanian, and Raghunathan continuous action space is quantized into bins, al- lowing the model to predict actions as discrete to- kens within the existing vocabulary. In continu- ous action generation, specialized decoders such as Diffusion Transformers (DiT) are used to out- put smooth, high-frequency joint or end-effector trajectories. 3 Methodology Our evaluation employs a hybrid approach that combines empirical hardware profiling with high-fidelity simulated projections. This methodology allows us to establish a baseline using contemporary edge platforms and then ex- tend those findings to evaluate the performance of larger models on hypothetical future hardware architectures. 3.1 Real Hardware Characterization To establish an empirical baseline, we characterize the performance of MolmoAct-7B on NVIDIA Jetson AGX Orin (64GB) and NVIDIA Jetson Thor (128GB) platforms. We instrument the PyTorch runtime using NVIDIA Nsight compute [5] to capture kernel-level execution traces. This profiling allows us to decompose end-to-end latency into specific phases: vision encoding, generation (autoregres- sive decoding), and action transformer. 3.2 Performance Projection and Simulation For modeling future architectures and larger model scales, we utilize an in-house, high-fidelity XPU simulator. This simulator incorporates detailed analytical performance models validated against several production-grade accel- erators (GPUs, TPUs), achieving an accuracy of 70% to 90% across several production-grade models (x−x B parameter LLMs). The simulator decomposes the VLA model into its constituent stages: vision encoding, autore- gressive decoding, and action generation. Each stage is modeled as a multi-layer Transformer backbone, where each layer is further resolved into a sequence of operators, primarily high-dimensional einsums. Key features of the modeling framework include: •Micro-architectural fidelity: The cost models in- corporate specific hardware details, including the number of Streaming Multiprocessors (SMs), tiling strategies, and asymmetric bandwidth characteris- tics across different dimensions of the XPU’s matrix engine. • Analytical roofline: The performance of individual operators is calculated using a roofline model that accounts for both compute and memory bandwidth constraints. •Cross-operator optimization: The framework per- forms optimization across operator boundaries to model effective prefetching. This is particularly crit- ical for memory-bound operations, as it allows for early movement of operands through the memory hierarchy to minimize stalls. Figure 2. Performance on current edge platforms. Latency of MolmoAct-7B on Jetson Orin and Jetson Thor. Commercial hardware MemoryBW (GB/s) BF16 TFLOPS OrinLPDDR5203100 ThorLPDDR5X273500 Hypothetical variants MemoryBW (GB/s) BF16 TFLOPS Orin+LPDDR5XLPDDR5X273100 Orin+GDDR7GDDR71000100 Orin+PIMLPDDR6X PIM21801074 Thor+GDDR7GDDR71000500 Thor+PIMLPDDR6X PIM21803993 Table 1. Commercial edge platforms and hypotheti- cal hardware systems used in our experiments. For sys- tems with PIM, the compute throughput (BF16 TFLOPS) includes both the SoC and PIM. 4 Evaluations and Analysis This section presents and analyzes the results of our mea- surements and simulations. 4.1 Real Hardware Characterization Figure 2 summarizes our profiling results and illustrates that (i) the latencies are∼200−300×higher than those needed for real-time (10Hz) operation, (i) the generation phase (auto-regressive decode with reasoning) is the pri- mary bottleneck in VLA models, accounting for≈ 75% of the (full-model) step latency, and (i) The generation phase is predominantly memory bandwidth bound–while Thor provides 5x the compute of Orin, the end-to-end latency only improves by 1.4x. 4.2 Scaled Models on Future Architectures We scale VLA models upto 100B parameters, following scaling laws in [1,8]. We also evaluate these models in our in-house simulator on a range of current and hypothetical systems. The hypothetical systems build on a standard off-the-shelf edge hardware, but augment them with more capable memory systems (specs listed in Table 1). Figure 3 illustrates that the improved bandwidth from GDDR7 and PIM memories substantially improves performance; yet, they are remain substantially lower than the target 10–20 Hz needed for real-world deployments. 5 Conclusion The primary bottleneck in Spatial reasoning VLA work- loads is the memory-bound Action Generation phase. Standard memory scaling is insufficient for handling 10– 100B parameter models at interactive rates. Future re- search must explore holistic system optimizations—both hardware and software to bridge the latency gap for em- bodied intelligence. Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures Figure 3. Control Frequency for various edge system configurations Higher memory bandwidth and PIM in- crease control frequency, but achieving the 10 Hz target for long horizon action generation at larger model sizes re- quires new innovations and algorithm-system co-design. AI Use Statement The authors used Gemini (Google) to draft the Problem Statement and Related Work sections, debug PyTorch pro- filing scripts for the MolmoAct-7B model, and provide structural feedback on the manuscript. All AI-generated suggestions were reviewed and verified by the authors, who maintain full accountability for the technical integrity of the results. References [1]Anthony Brohan et al.2023. RT-2: Vision Language Action Mod- els Transfer Web Knowledge to Robotic Control. arXiv preprint arXiv:2307.15818 abs/2307.15818 (2023), 1–15. [2]Google DeepMind. 2025. Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfer. arXiv preprint arXiv:2510.03342 abs/2510.03342 (2025), 1–1. [3]S. Lee et al.2021. Hardware Architecture and Software Stack for PIM Based on Commercial DRAM Technology. In ISCA. IEEE, Valencia, Spain, 43–56. [4]S. Lee et al.2025. MolmoAct: Action Reasoning Models that can Reason in Space. arXiv preprint arXiv:2508.07917 (2025), 1–15. [5] NVIDIA Corporation. [n. d.]. NVIDIA Nsight Compute. https: //developer.nvidia.com/nsight-compute. Accessed: 2026-02-04. [6]NVIDIA Corporation. 2022. NVIDIA Jetson AGX Orin Technical Brief. NVIDIA, Santa Clara, CA. [7]NVIDIA Corporation. 2024. NVIDIA Drive Thor: The Next Generation of Centralized Compute. GTC Keynote, San Jose, CA. [8]Neil Thompson Sebastian Sartor. 2024. Neural Scaling Laws in Robotics. arXiv preprint arXiv:2405.14005 abs/2405.14005 (2024), 1–1.