Paper deep dive
vla.simd: Efficient CPU Inference for Language-Conditioned Manipulation
Khanh D. Nguyen, Hoang M. Truong, An T. Le
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Deploying language-conditioned manipulation without a dedicated GPU requires efficient inference and action chunks that cover the delay between policy queries. We present this http URL, a CPU inference engine that combines shared SIMD micro-kernels, reusable computation, and target-specific optimization. We relate query latency and execution horizon to action availability under lagged and time-aligned execution, distinguishing action supply from feedback frequency. Across six policies and four CPUs, this http URL achieves approximately $1.4\times$ median speedup over compiled PyTorch references while preserving fp32 numerical fidelity. We also introduce IMPACT, an ACT-based policy with cached text representations and language-modulated visual features. IMPACT is the only language-conditioned policy in our evaluated set that supplies at least 30 actions/s on the Raspberry Pi 5: after a 90 s thermal soak, it supplies 33.5 actions/s in fp32 and 81.2 with int8. Separate GPU evaluations yield $76.4\%$ mean success across four LIBERO suites without robot pretraining; instruction-shuffling tests demonstrate selection among familiar goals. Trials with IMPACT on an SO-101 arm and SmolVLA on a UR10e with a Robotiq gripper demonstrate CPU deployment on two robot embodiments.
Tags
Links
- Source: https://arxiv.org/abs/2609.24274v1
- Canonical: https://arxiv.org/abs/2609.24274v1
Trouble viewing inline? Open PDF directly →
Full Text
47,736 characters extracted from source content.
Expand or collapse full text
vla.simd: Efficient CPU Inference for Language-Conditioned Manipulation Khanh D. Nguyen Affiliation: VinRobotics, Vietnam. Hoang M. Truong Affiliation: VinRobotics, Vietnam. An T. Le †thanks: Corresponding author: An T. Le. Affiliation: VinRobotics, Vietnam. Affiliation: Center for AI Research, VinUniversity, Vietnam. Affiliation: Intelligent Autonomous Systems, TU Darmstadt, Germany. Abstract Deploying language-conditioned manipulation without a dedicated GPU requires efficient inference and action chunks that cover the delay between policy queries. We present vla.simd, a CPU inference engine that combines shared SIMD micro-kernels, reusable computation, and target-specific optimization. We relate query latency and execution horizon to action availability under lagged and time-aligned execution, distinguishing action supply from feedback frequency. Across six policies and four CPUs, vla.simd achieves approximately 1.4×1.4× median speedup over compiled PyTorch references while preserving fp32 numerical fidelity. We also introduce IMPACT, an ACT-based policy with cached text representations and language-modulated visual features. IMPACT is the only language-conditioned policy in our evaluated set that supplies at least 30 actions/s on the Raspberry Pi 5: after a 90 s thermal soak, it supplies 33.5 actions/s in fp32 and 81.2 with int8. Separate GPU evaluations yield 76.4% mean success across four LIBERO suites without robot pretraining; instruction-shuffling tests demonstrate selection among familiar goals. Trials with IMPACT on an SO-101 arm and SmolVLA on a UR10e with a Robotiq gripper demonstrate CPU deployment on two robot embodiments. Project page: https://vla-simd.github.io/. I Introduction Deploying a language-conditioned manipulation policy [1, 2, 3, 4] on a robot without a dedicated GPU raises a practical question: can the available CPU deliver actions fast enough to keep the robot moving? Answering this requires more than comparing model sizes or query latencies. A manipulation policy must provide actions within the timing constraints of its controller while retaining the ability to select the goal specified by the user. C2: H= 2fcTqH\,=\,2f_cT_qC1: H=fcTqH\,=\,f_cT_qmeets C1 & C2below C1int8ACTIMPACTSmolVLADP(100 steps)DP (10 steps)TurboVLAOcto-Small30301001003003001 0001\,0003 0003\,00010 00010\,00030 00030\,00044121232325050100100median query latency TqT_q (ms)available action window HHM4i9RyzenPi 5Pi 5 int8 Fig. 1: Nominal action-supply budgets of chunked policies. Markers show each policy’s available action window H and median query latency TqT_q on each CPU. Rows sharing H are offset by a factor of 1.12; no marker crosses a line. The lines omit control-step rounding: above C1 a policy supplies actions at least as fast as a fc=30f_c=30 Hz controller consumes them; above C2 it still does when stale actions are discarded (Section I). On the Pi 5, ACT meets C1 with 100 actions per query, whereas TurboVLA fails with 12; IMPACT meets C1 in fp32 and C2 with int8. Action chunking creates an opportunity for CPU deployment. By predicting several future actions in each query [5, 6], a policy can overlap inference with execution and operate below the controller’s update frequency [7, 4]. The relevant constraint is therefore whether the next usable action chunk arrives before the current one is exhausted. This depends jointly on inference latency, execution horizon, and how delayed actions are handled (Section I, Fig. 1). A faster query does not necessarily provide a higher action-supply rate, and meeting a nominal compute budget does not by itself guarantee responsive or successful manipulation. CPU deployment must therefore meet the action-supply budget while preserving language-based goal selection. vla.simd combines CPU optimization with action-supply analysis to identify which policies can sustain execution on each processor. Linear layers, convolutions, and attention projections share matrix computations that can exploit SIMD instructions through packed-panel micro-kernels [8, 9]. Data layout, register use, memory reuse, and threading must suit the policy and processor: a setting that helps one CPU can hurt another. Our engine, vla.simd, combines shared SIMD micro-kernels with target-specific choices and separates computation lifetimes: weights are packed per process, instruction-only tensors are cached per episode, and observation-dependent features are computed per query. Efficient execution must also preserve the information needed to choose the task. ACT [5] provides an efficient starting point, meeting the basic 30 Hz action-supply budget on the Pi 5, but it does not condition on language instructions. In a multi-task setting, the same scene can correspond to different goals, such as placing the tape into either the box or the cup; the observation alone cannot specify the user’s intention. We therefore introduce IMPACT (Instruction-Modulated Perception and Action Chunking with Transformers), an action-chunking policy that extends ACT with language conditioning through cached text representations, instruction tokens, and visual modulation. Our contributions are: • vla.simd, a CPU engine with a shared SIMD micro-kernel for linear layers, convolutions, and attention projections, plus target-specific tiling and quantization. • IMPACT, a language-conditioned action-chunking policy whose language conditioning accounts for approximately 2.6% of per-query compute in the SO-101 configuration, evaluated through LIBERO, instruction shuffling and real SO-101 manipulation. • A study of six policies on four CPUs and two embodiments, showing where CPU optimization meets action-supply budgets, while assessing thermal effects, PyTorch fidelity, and robot deployment. I Related Work Efficient vision-language-action policies. Billion-parameter vision-language-action (VLA) models [1, 2, 10] motivated compact designs that shorten or remove the language model [4, 11, 12], reduce the overall parameter count [13, 14], or quantize the policy [15]; see [16] for a survey. Recent studies characterize VLA latency on GPUs and embedded accelerators [17, 18], with broader hardware comparisons also including CPUs [19]. Lite VLA reports CPU deployment on a Pi 4, with inference taking seconds to minutes [20]. We compare six policies on four CPUs using the same timing boundary and account for the actions supplied by each query. Chunked execution in real time. ACT [5] and Diffusion Policy [6] predict action chunks. Real-time chunking [7] overlaps inference with motion and guides new chunks for continuity; LeRobot supports asynchronous serving [4, 21]. Bidirectional decoding [22] addresses the consistency–reactivity trade-off through guided sampling. We apply the action-availability condition of [7] to sequential CPU queries and distinguish lagged from time-aligned execution. Language conditioning. Feature-wise linear modulation (FiLM) [23] uses language to scale and shift visual features in BC-Z [24], RT-1 [25], and BAKU [26]. MT-ACT [27] combines FiLM in the backbone with a language token in an ACT policy, and Octo [3] reads frozen T5 tokens. Concurrent work, DEM [28], also caches instruction embeddings from a separate encoder and evaluates decoupled policies on a GPU. IMPACT combines these established mechanisms while caching instruction-only features to limit the language-processing cost of each CPU query. CPU inference. Our micro-kernel follows GotoBLAS and BLIS [8, 9], with register constraints based on Low et al. [29] and empirical validation on each target. Production runtimes such as XNNPACK [30], ONNX Runtime [31], OpenVINO [32], ExecuTorch [33], and llama.cpp [34] select kernels for each instruction set architecture (ISA). vla.cpp [35] extends llama.cpp with cached multimodal prefixes and iterative action heads; its reported evaluation uses discrete and embedded GPUs. SIMD also accelerates motion planning on CPUs [36, 37]. vla.simd focuses on CPU execution across policy families, testing how kernel, threading, and precision choices affect action-supply budgets. I Action-Supply Budgets We first determine when a policy can supply actions continuously despite inference delay. A controller running at frequency fcf_c consumes one action every Δ=1/fc =1/f_c seconds. A policy predicts N actions from the latest observation, of which a window of H≤NH≤ N actions is available for execution. For example, our Diffusion Policy predicts 64 actions but exposes a 32-action window; the other settings use their full predicted chunk (Table I). We index this window from the observation’s control step t and denote query latency by TqT_q, or d=⌈fcTq⌉d= f_cT_q control steps. The analysis assumes fixed delay, steady-state execution, and sequential queries starting on control ticks s≥ds≥ d steps apart. The ratio feff=H/Tqf_eff=H/T_q, termed effective control rate in [35], is an action-supply rate; it does not measure feedback frequency. We consider two execution modes. In lagged execution, the robot starts each chunk from its first action upon arrival. To avoid running out of actions, the next chunk must arrive before the current one is exhausted, requiring s≤Hs≤ H. In time-aligned execution, actions retain their intended control steps, and stale actions are discarded, as in real-time chunking [7] and the LeRobot client for our robot [4, 21]. At arrival, max(0,H−d) (0,H-d) actions remain, so covering the next arrival requires s+d≤Hs+d≤ H, equivalent to d≤H−sd≤ H-s in [7]. Combining these requirements with s≥ds≥ d, a feasible query interval exists when (C1)d≤H,(C2)2d≤H,(C1) d≤ H, (C2) 2d≤ H, (1) for lagged (C1) and time-aligned (C2) execution, respectively; both can use s=ds=d. C1 is equivalent to feff≥fcf_eff≥ f_c; ignoring rounding to control steps gives the approximation feff≥2fcf_eff≥ 2f_c for C2. Thus, time-aligned execution needs enough actions to cover both the current inference delay and the wait for the next chunk. These conditions ensure action availability under the stated model; smooth chunk transitions and task success require separate evaluation. Each executed action uses an observation at least d steps old. We evaluate these nominal budgets using median engine latency, excluding camera capture and transport. Reliable deployment must also account for these additional delays and occasional slow queries. We use fc=30f_c=30 Hz, matching our SO-101 controller [38] and the frame rate of its training datasets. IV The vla.simd Engine Fig. 2: SIMD execution in vla.simd. Linear layers, convolutions, and attention projections share a packed-panel SIMD micro-kernel (SMK). Convolution supplies image patches to this kernel one panel at a time. The labels x, y, and A denote X, Y, and packed W⊤W , respectively. The mr×nrm_r× n_r output tile remains in registers during accumulation (right). The attention block uses a separate kernel for QK⊤QK and SVSV, where S is the row-wise softmax of the scaled scores; the key projection writes K⊤K directly. IV-A Computation Reuse vla.simd maps raw camera frames, joint state, and an instruction to an action chunk in robot units, without a deep learning framework at run time. It handles resizing, normalization, and tokenization; query timing excludes camera capture, transport, and episode initialization. Weights are converted offline to contiguous fp32 arrays and packed once per process; instruction-only tensors are cached per episode. Each query computes observation-dependent features, including image-dependent language features. Shared operators dispatch to Intel AVX2, AMD AVX2, ARMv8 NEON, or Apple silicon backends. Apple uses Accelerate [39] for dense matrix products and convolutions by default; the others use the shared micro-kernel (SMK). IV-B Selecting a Register Tile for Each Target A linear layer computes Y=XW⊤+mb⊤∈ℝm×nY=XW +1_mb ^m× n, where X∈ℝm×kX ^m× k, W∈ℝn×kW ^n× k, b∈ℝnb ^n, and m1_m is the all-ones vector. Following GotoBLAS and BLIS [8, 9], W⊤W is packed into panels of nrn_r output columns. The micro-kernel accumulates an mr×nrm_r× n_r output tile in vector registers over depth k, then adds bias on storage. Fused multiply-add (FMA) instructions follow increasing depth order, preserving each output’s accumulation order across tile dimensions. With L fp32 values per vector register and nv=nr/Ln_v=n_r/L registers per tile row, a tile requires mrnvm_rn_v accumulator registers, nvn_v weight registers, and q activation registers. AVX2 broadcasts each input value into one register, reusing that register across rows, so q=1q=1 and each depth step issues Λ=nv+mr =n_v+m_r loads. NEON instead keeps one vector per row and reads its lanes over L consecutive depth steps, giving q=mrq=m_r and an average of Λ=nv+mr/L =n_v+m_r/L loads per step for unpacked activations. For a target with RmaxR_ vector registers, throughputs of PfmaP_fma vector FMAs and PldP_ld vector loads per cycle, and FMA latency τ cycles, we require mrnv+nv+q m_rn_v+n_v+q ≤Rmax, ≤ R_ , (2) mrnv m_rn_v ≥Pfmaτ, ≥ P_fma\,τ, (3) mrnvPld m_rn_v\,P_ld ≥ΛPfma, ≥ \,P_fma, (4) Equation (2) limits register use, (3) provides enough independent accumulators to hide FMA latency [29], and (4) requires loads to keep pace with arithmetic. Selection runs offline: Spills compiles a candidate and checks its inner loop for vector reloads from the stack. Algorithm 1 Offline register-tile selection for one target 1: L,Rmax,Pfma,Pld,τL,R_ ,P_fma,P_ld,τ, operand mode, output widths N of the packed layers 2: g←gcdg← ; best ←⊥← ⊳ ⊥ : unpacked fallback 3: for each nrn_r with L|nrL n_r and nr|gn_r g, ascending do 4: nv←nr/Ln_v← n_r/L 5: for mr←Rmaxm_r← R_ down to 11 do 6: (q,Λ)←(1,nv+mr),AVX2(mr,nv+mr/L),NEON(q, )← cases(1,n_v+m_r),&AVX2\\ (m_r,n_v+m_r/L),&NEON cases 7: if (2)–(4) hold and not Spills(mr,nrm_r,n_r) then 8: if best =⊥= or ϕ(mr,nr)>ϕ(best)φ(m_r,n_r)>φ(best) then 9: best ←(mr,nr)←(m_r,n_r) 10: break ⊳ largest feasible mrm_r for this nrn_r 11: return best Algorithm 1 considers widths nrn_r divisible by L and dividing the packed layers’ greatest common output-width divisor g. The score ϕ(mr,nr)=mrnr/(nr+Lmr)φ(m_r,n_r)=m_rn_r/(n_r+Lm_r) increases with mrm_r at fixed nrn_r, justifying the early break; the highest-scoring feasible tile wins, with unpacked weights as fallback. The score is the AVX2 broadcast kernel’s FMA-to-load ratio; its use for NEON is an empirical heuristic. The NEON load model alone favors 9×89×8 and 8×88×8, yet the selected 4×164×16 tile is 12% and 19% faster on the Cortex-A76. For our policy widths (g=64g=64), this procedure selects 6×166×16 on AVX2, using 15 of 16 registers, and 4×164×16 on NEON, using 24 of 32. The AVX2 choice matches the single-precision tile used by BLIS for Haswell and Zen. On the Pi 5, the selected tile reaches 106.9 GFLOP/s with four threads and 96% of peak arithmetic throughput on one core with data in L2 cache. On both ISAs, the selected tile is the fastest tested candidate across all six benchmarked layer shapes. IV-C Convolution and Attention Convolution reuses the micro-kernel through im2col [40]: rows contain image patches and packed weights contain flattened filters. Constructing patches one panel at a time respects stride and padding without storing the full patch matrix. Attention’s query (Q), key (K), value (V), and output projections use the same linear operator; the key projection writes K⊤K directly by transposing tiles on storage. The attention products QK⊤QK and softmax(QK⊤/dh)Vsoftmax(QK / d_h)V use a separate per-backend kernel; dhd_h is the head width, and softmax operates row-wise over keys. Both operands change with each query, and the key count need not be a multiple of nrn_r. Linear layers whose output width is not a multiple of nrn_r, such as action heads, use a row-blocked kernel. IV-D Runtime Settings and Quantization The backend and register tile are fixed at build time. Runtime settings control weight packing, convolution panels, matrix-multiplication blocking, attention kernels, and threading (Table IV). Optional quantization represents both weights and activations as signed 8-bit integers (W8A8). This path uses symmetric quantization, 32-bit integer accumulation, and fp32 rescaling, with Arm dotprod or x86 AVX-VNNI instructions for integer products. Weight scales are per output channel; activation scales are computed dynamically per token for linear layers and per input tensor for convolutions, using the maximum absolute value divided by 127 (scale 1 if zero). Selected operator groups use this path; normalization, nonlinearities, and unsupported output widths remain fp32. IMPACT’s cached text encoder remains fp32. Section VI-D evaluates these choices per target. V IMPACT Once per instructionInstructionFrozen T5-smallProjectedtext tokensMean pool + FiLMcoefficientsTwo viewsResNet-18 + FiLM6-layer encoderVisual + text tokens4-layer decoder50 learned queriesState + latent z=0z=0Linear projectionsAction projection50-step chunkEach query: new views and state; cached language features Fig. 3: IMPACT. A frozen T5-small encodes the instruction once per episode, providing transformer tokens and FiLM coefficients for each ResNet-18 stage. Each query runs the modulated backbone and encoder-decoder to predict a 50-step action chunk. IMPACT (Instruction-Modulated Perception and Action Chunking with Transformers) adds language-based task selection to an action-chunking policy while keeping most language computation outside the repeated inference loop. Following ACT [5], it maps two camera views, robot state, and an instruction to a 50-action chunk (Fig. 3). Visual and state inputs The views share an ImageNet-initialized ResNet-18 [41]; batch normalization is folded into convolutions for inference. The SO-101 configuration uses 480×640480×640 images, producing 600 visual tokens across both views, and predicts six-dimensional joint and gripper commands. The LIBERO configuration uses 256×256256×256 images, producing 128 visual tokens, with an eight-dimensional end-effector state and seven-dimensional relative actions. Linear projections map visual features and robot state to width 512. Language conditioning A frozen T5-small text encoder [42] maps the instruction to E∈ℝnℓ×512E ^n_ × 512, where 1≤nℓ≤321≤ n_ ≤ 32 counts non-padding tokens. Longer inputs are truncated, and padding is masked within T5. These states condition the policy in two ways. First, projected text tokens are appended to the transformer encoder sequence, with padding excluded. Second, the mean state, e¯=nℓ−1∑j=1nℓEj,:⊤ e=n_ ^-1 _j=1^n_ E_j,: , modulates the feature map FiF_i after the residual addition and ReLU of each backbone stage i: [γi;βi]=Gie¯+bi,Fi←(1+γi)⊙Fi+βi,[ _i; _i]=G_i e+b_i, F_i←(1+ _i) F_i+ _i, (5) For cic_i channels, Gi∈ℝ2ci×512G_i ^2c_i× 512 and bi∈ℝ2cib_i ^2c_i are learned; [γi;βi][ _i; _i] concatenates the channel scales and shifts, broadcast over pixels. The symbol ⊙ denotes element-wise multiplication. Initializing GiG_i and bib_i to zero makes FiLM an identity operation at the start of training. The engine caches the projected text tokens and FiLM coefficients per instruction and applies the modulation to each query’s features. Action prediction and training A six-layer transformer encoder and four-layer decoder, with eight attention heads and feed-forward width 3200, predict the chunk from 50 learned action queries in one pass. The encoder receives the visual and text tokens, one state token, and one latent token. Visual tokens use two-dimensional sinusoidal positions; state, latent, and text positions are learned. Training follows ACT’s conditional variational autoencoder (CVAE) formulation: a four-layer encoder receives the demonstrated actions, state, and instruction and predicts a distribution over a 32-dimensional latent z. The objective combines L1 action reconstruction with a Kullback–Leibler penalty of weight 10 toward a standard normal latent prior; inference sets z=0z=0. State and actions are normalized by the training-set mean and standard deviation; predictions are converted back to robot units before execution. IMPACT has 60 M parameters in its per-query policy and 35.3 M in the frozen T5-small encoder, whose outputs are cached per episode. Computational cost For the SO-101 configuration, counting a multiply-add as two floating-point operations gives approximately 88 GFLOP per query. Text encoding, projection, and FiLM coefficient generation add about 1.2 GFLOP once per episode. Relative to the same six-layer encoder and four-layer decoder without language, up to 32 text tokens add about 2.3 GFLOP per query and FiLM adds 9 MFLOP, approximately 2.6% of the total operation count. Compared with the shallower ACT in Table I, IMPACT takes 1.201.20–1.33×1.33× as long per query across the four CPUs. VI Experiments VI-A Setup TABLE I: CPU platforms and benchmark settings. P/E: performance/efficiency cores; C/T: cores/hardware threads. Device CPU core ISA Cores Kernel Thr. Apple M4 Apple NEON 4P+6E Accelerate 8 Intel i9-14900HX Raptor Lake AVX2 8P+16E SMK 6×166×16 16 AMD Ryzen 5 5500 Zen 3 AVX2 6C/12T SMK 6×166×16 12 Raspberry Pi 5 Cortex-A76 NEON 4C SMK 4×164×16 4 TABLE I: Evaluated policies and available action windows H. “+T5” denotes a separate frozen text encoder. Policy Params Vision / language Action model H ACT [5] 34 M ResNet-18 / - CVAE transformer 100 DP [6] 278 M ResNet-18 / - U-Net, 100/10 steps 32 Octo-Small [3] 27 M+T5 conv stem / T5-base diffusion, 20 steps 4 TurboVLA [11] 0.2 B DINOv3 / BERT ACT-style decoder 12 SmolVLA [4] 450 M SmolVLM2 flow matching 50 IMPACT (ours) 60 M+T5 ResNet-18 / T5-small CVAE transformer 50 Hardware and policies We evaluate six policies on four CPUs, including a passively cooled Pi 5 (Tables I and I). CPU benchmarks use checkpoints fine-tuned on 30 Hz SO-101 data, except for TurboVLA, whose LIBERO checkpoint is evaluated against the same action-supply budget. Diffusion Policy uses one checkpoint with 100-step DDPM [43] or 10-step DDIM [44] sampling. Measurement protocol We time each query from observation preprocessing through conversion of predicted actions to robot units, as defined in Section IV. Nominal latency is the median of 15 queries after warmup, measured in a fresh process with the CPU initially at its idle temperature. To compare settings A and B, we interleave measurements in A–B–B–A order within one session to limit thermal drift. Before timing fp32 implementations, we check their outputs against the corresponding reference policy; quantization is evaluated separately. VI-B Action Supply and Thermal Effects TABLE I: Action-supply rate fefff_eff (actions/s). Gray: fails C1; italic: meets C1 only; upright black: meets C2, at fc=30f_c=30 Hz. Nominal: fp32; last two columns: after a 90 s Pi 5 soak; dashes: unmeasured. Raspberry Pi 5 Nominal 90 s soak Policy H M4 i9 Ryzen fefff_eff TqT_q (s) fp32 int8 ACT 100 1150 892 627 109 0.92 89.9 242 DP, 100 steps 32 7.2 7.2 3.6 0.8 40.1 - - DP, 10 steps 32 58 55 29.3 6.3 5.08 - - Octo-Small 4 76 48 48 7.8 0.51 5.8 11.3 TurboVLA 12 87 58 57 9.3 1.28 7.2 - SmolVLA 50 73 42 38 6.1 8.19 - - IMPACT 50 453 334 262 41 1.21 33.5 81.2 We test whether each policy supplies enough actions for lagged execution (C1) or for execution that discards stale actions (C2). To assess the effect of heating, we also measure latency after 90 s of continuous inference on the passively cooled Pi 5. Table I and Fig. 1 show that most evaluated policies meet C1 on desktop CPUs, while only ACT and IMPACT qualify on the Pi 5. IMPACT is the only evaluated language-conditioned policy to supply at least 30 actions/s on the Pi 5. Octo-Small queries are shorter, but their four-action window is insufficient for either budget. After this workload, IMPACT supplies 33.5 actions/s in fp32 and 81.2 with int8, satisfying C1 and C2, respectively. VI-C Speed and Numerical Fidelity M4i9RyzenPi00252550507575100100action-supply rate (actions/s)TurboVLAM4i9RyzenPi002020404060608080Octo-SmallM4i9RyzenPi002020404060608080SmolVLAM4i9RyzenPi002502505005007507501,0001,0001,2501,250ACTM4i9RyzenPi0015153030454560607575DP (10 steps)M4i9RyzenPi00100100200200300300400400500500IMPACT lower rate engine gain engine loss – – 30 actions/s Fig. 4: Paired fp32 action-supply rates for six configurations on four CPUs; panel scales differ. Gray bars show the lower of the two rates; blue/red extensions show an engine gain/loss relative to torch.compile. The dashed line marks 30 actions/s; it does not denote feedback frequency. On the Ryzen, Octo-Small and SmolVLA cross this line with vla.simd. Diffusion Policy with 100 steps is omitted from the figure. We compare vla.simd with compiled PyTorch using default-mode torch.compile [45] and the Inductor CPU backend. We also measure uncompiled (eager) execution, using PyTorch 2.11 for five policies and 2.13 for TurboVLA. All implementations use fp32 and the same query-timing boundary over six interleaved blocks after warmup; compilation time is excluded. Both implementations cache instruction-only encodings where supported and use the same thread count, with one PyTorch inter-op thread. We check numerical fidelity against each policy’s PyTorch reference using identical observations and stochastic inputs, with preset tolerances of 10−210^-2 absolute error and 10−410^-4 relative to action magnitude. Paired timings are collected in the same session and may differ from the nominal measurements in Table I. For each configuration–CPU pair, speedup is the median within-block ratio Tcompile/TengineT_compile/T_engine. The median across 28 pairs (six policies, with two DP samplers, on four CPUs) is approximately 1.4×1.4×, spanning 0.830.83–3.37×3.37×. Eager and compiled execution have similar latency in most settings; Octo-Small on the M4 shows the largest gain from compilation. On the Ryzen, Octo-Small and SmolVLA cross C1 (Fig. 4). The M4 regressions do not change C1 feasibility, although 10-step Diffusion Policy loses its C2 margin. A separate numerical comparison on 2–12 observations per reported policy gives maximum absolute action differences of at most 8.8×10−58.8×10^-5 in robot units. These small discrepancies are consistent with changes in floating-point accumulation order. The Intel and AMD backends produce bitwise-identical fp32 actions on the tested inputs. For IMPACT, the reference is our PyTorch implementation. Task performance is evaluated separately below. VI-D Processor-Specific Implementation Choices TABLE IV: Effects of individual runtime settings on ACT. A/B: consistently faster setting; =: no consistent difference. Values are median 100(TB/TA−1)100(T_B/T_A-1) (%); positive favors A. Bottom: Tfp32/TW8A8T_fp32/T_W8A8; above 1 is faster; n/a: unsupported; –: unmeasured. Setting A / B M4 i9 Ryzen Pi 5 Attention kernel library / custom A ++9.9 = = = bf16 expansion off / on = = = = Min. panel rows 16 / 4 = = B −-4.2 = Convolution tiled / flat = A ++156 A ++260 A ++16.9 Block rows auto / 64 = A ++8.3 A ++5.0 = Thread schedule static / dyn. = = = = Parallel cutoff 8192 / 1024 = = = = Weight packing on / off = A ++31.0 A ++30.7 A ++14.8 Thread count more / fewer = A ++5.4 = A ++32.0 Threads per view auto / 2 B −-6.2 A ++104 A ++75.6 A ++17.8 Thread waiting passive / active A ++72.9 A ++1.7 = = W8A8, ACT 1.13 1.24 n/a 2.68 W8A8, IMPACT 1.06 1.26 n/a 2.39 W8A8, Octo-Small 0.66 1.03 n/a 1.73 W8A8, SmolVLA 0.71 1.14 n/a - We vary eleven settings individually on ACT using the paired protocol above. We report a consistent improvement only when a setting is faster in all but at most one pair and its median effect exceeds baseline drift. In Table IV, bf16 expansion converts bfloat16 weights to fp32; the parallel cutoff is the minimum element count for parallel execution. The higher/lower thread counts are 10/4 on the M4, 32/8 on the i9, 12/6 on the Ryzen, and 4/2 on the Pi 5. We also compare W8A8 with fp32 on supported policy–CPU pairs and examine selective quantization of convolution and attention layers. The W8A8 comparisons quantize all supported operator groups: for IMPACT, these are ResNet convolutions, token projections, and encoder/decoder linear layers, with the text encoder kept in fp32. Varying one setting at a time measures its individual effect; interactions between settings are not evaluated. Table IV shows that useful settings depend on the target CPU: tiled convolution benefits the non-Apple backends, while extra camera-view threads help the M4 but hurt the other devices. Quantization is similarly uneven, yielding substantial gains on the Pi 5 but slowing some policies on the M4. For IMPACT on the M4, quantizing only convolutions is faster than quantizing all layers. These results support selecting settings and quantized layers through measurements on each target. VI-E Manipulation and Instruction Following TABLE V: LIBERO success rates (%); protocols differ across published baselines. IMPACT: final checkpoint versus selection on the same evaluation episodes. Policy Spatial Object Goal Long Avg No robot pretraining Diffusion Policy 78.3 92.5 68.3 50.5 72.4 TurboVLA 99.2 99.8 97.4 94.2 97.7 SmolVLA 90.0 96.0 92.0 71.0 87.3 IMPACT, final 83.5 83.5 84.5 54.0 76.4 IMPACT, selected 83.5 83.5 89.0 58.5 78.6 Pretrained on robot data Octo-Base 78.9 85.7 84.6 51.1 75.1 OpenVLA 84.7 88.4 79.2 53.7 76.5 LIBERO We evaluate IMPACT on four LIBERO simulation suites [46], each containing ten manipulation tasks. We train one policy per suite without robot pretraining and evaluate each task from 20 initial states on a GPU host. Training uses AdamW with batch size 64, seed 1000, constant learning rate 10−410^-4, and weight decay 10−410^-4. Evaluation uses two 256×256256×256 views and executes all 50 relative end-effector actions before querying the policy again. The final 30,000-step checkpoints achieve 76.4% mean success (Table V). Selecting checkpoints on these same evaluation rollouts yields 78.6%; we report this exploratory result separately because selection can inflate the estimate. Published DP, Octo-Base, and OpenVLA results average three seeds of 500 rollouts per suite; TurboVLA uses 500 and SmolVLA 100 [47, 11, 4]. These published results use different observation inputs and execution horizons and provide context rather than a controlled comparison. Instruction following To test whether IMPACT uses language to select a task, we keep the LIBERO-Goal environments fixed and change only the instruction, using the selected 25,000-step checkpoint. All ten tasks share the same scene and objects. We evaluate three instruction permutations, each replacing every original instruction, for 600 rollouts in total. We score both the original and newly instructed goals using LIBERO’s success conditions. IMPACT achieves the newly instructed goal in 91% of trials and the original goal in none, against an 88.5% correct-instruction control. This intervention supports language-based selection among familiar goals; generalization to unseen wording and the individual contributions of the two conditioning paths remain untested. SO-101 On the SO-101 arm (Fig. 5), we use 44 demonstrations for three instructions in a shared scene and 20 for a separate drawer task. LeRobot’s asynchronous interface [21] connects to vla.simd on the M4, Pi 5, and i5-12400F, or to PyTorch on a GPU. The trials use one scene and one operator; IMPACT results pool trials across these CPU and GPU servers. Each row of Table VI reports 20 trials with a Wilson 95% confidence interval. IMPACT performs better on destination selection than on changing the grasp target (Table VI). The fp32 and int8 success counts are similar, although 20 trials per condition are insufficient to establish equivalent performance. As a no-language control, we train ACT with IMPACT’s chunk length, transformer depth, and shared ResNet-18, removing text and FiLM. This control is deeper than the ACT in the CPU benchmarks. We train one policy on all three tasks and three specialist policies on one task each. All train for 19,000 steps with batch size 8 and execute synchronously, so inference and motion do not overlap. Each specialist therefore receives more passes over its smaller dataset. The multi-task control succeeds in 4 of 20 trials, compared with 46 of 60 across specialists. The operator attributes 9 of the 20 multi-task trials to target ambiguity and 7 to grasp failures. These observations suggest task ambiguity as a failure mode, but differences in data exposure and execution protocol prevent attributing the success gap to language conditioning alone. Fig. 5: Real-robot tasks on two embodiments. Top: three instructions in a shared SO-101 [38] scene vary the destination or grasp target. Middle: the SO-101 opens a drawer, places the tape inside, and closes it. Bottom: a UR10e with a Robotiq gripper picks up the cup and places it in the box. The middle and bottom rows show successive stages from left to right. UR10e SmolVLA, served by vla.simd from the M4 and Ryzen, places a cup into a box using a UR10e with a Robotiq gripper (Table VII). Both hosts achieve 12 successes in 20 trials, with a 2.0×2.0× difference in mean query round-trip time. The small samples leave the effect of host latency unresolved. Round-trip times include transport and serialization, exceeding nominal engine latency by approximately 0.5 s on the M4 and 1.0 s on the Ryzen. These additional delays show why deployment must be assessed using the full observation-to-action path. TABLE VI: SO-101 success rates (SR, %): 20 trials/row, Wilson 95% confidence intervals (CI). IMPACT pools servers; ACT controls execute synchronously. Model Goal Training SR (%, 95% CI) ACT all three multi-task 20 (8–42) ACT tape → box single-task 60 (39–78) ACT tape → cup single-task 90 (70–97) ACT cup → box single-task 80 (58–92) IMPACT tape → box multi-task 90 (70–97) IMPACT tape → cup multi-task 85 (64–95) IMPACT cup → box multi-task 60 (39–78) IMPACT int8 tape → box multi-task 85 (64–95) IMPACT int8 tape → cup multi-task 95 (76–99) IMPACT int8 cup → box multi-task 50 (30–70) IMPACT tape → drawer drawer 65 (43–82) TABLE VII: SmolVLA on UR10e: 20 trials/host, Wilson 95% CI. Mean query round-trip times include transport. Host Round-trip (ms) SR (%, 95% CI) Apple M4 1177 60 (39–78) Ryzen 5 5500 2343 60 (39–78) VII Conclusion We presented vla.simd, an inference engine that combines shared SIMD kernels with target-specific optimization, and IMPACT, an ACT-based policy that adds language-guided task selection with little repeated computation. CPU deployment becomes feasible when the execution horizon covers inference delay and the implementation uses the processor efficiently. On the Ryzen, vla.simd pushes Octo-Small and SmolVLA across the 30 Hz threshold. On the Pi 5, IMPACT is the only language-conditioned policy in our evaluated set that meets this budget, and its int8 path also meets the stricter time-aligned budget after a thermal soak. Instruction shuffling and real-robot trials demonstrate language-based goal selection, extending ACT to instruction-guided manipulation. VIII Limitations Our evaluation covers policies up to 450 M parameters and compares CPU inference with eager and compiled PyTorch; it does not benchmark GPUs, embedded accelerators, or dedicated runtimes such as ONNX Runtime and OpenVINO. The budgets use median engine latency, whereas uninterrupted deployment depends on the tail of the full observation-to-action delay. Thermal behavior is measured over one 90 s soak. The int8 path is validated on the SO-101 multi-task bench, but its effect on simulation success and on the long-horizon task is unmeasured. LIBERO evaluation uses one seed, and the SO-101 trials pool servers, limiting conclusions about behavior on individual CPUs. References [1] M. J. Kim, K. Pertsch, S. Karamcheti, et al. (2024) OpenVLA: an open-source vision-language-action model. In Conference on Robot Learning (CoRL), Proceedings of Machine Learning Research, Vol. 270, p. 2679–2713. Cited by: §I, §I. [2] K. Black, N. Brown, D. Driess, et al. (2025) π0 _0: A vision-language-action flow model for general robot control. In Robotics: Science and Systems (RSS), External Links: Document Cited by: §I, §I. [3] Octo Model Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, J. Luo, Y. L. Tan, L. Y. Chen, Q. Vuong, T. Xiao, P. Sanketi, D. Sadigh, C. Finn, and S. Levine (2024) Octo: an open-source generalist robot policy. In Robotics: Science and Systems (RSS), External Links: Document Cited by: §I, §I, TABLE I. [4] M. Shukor et al. (2025) SmolVLA: a vision-language-action model for affordable and efficient robotics. arXiv preprint arXiv:2506.01844. Cited by: §I, §I, §I, §I, §I, §VI-E, TABLE I. [5] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn (2023) Learning fine-grained bimanual manipulation with low-cost hardware. In Robotics: Science and Systems (RSS), Cited by: §I, §I, §I, §V, TABLE I. [6] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song (2023) Diffusion policy: visuomotor policy learning via action diffusion. In Robotics: Science and Systems (RSS), Cited by: §I, §I, TABLE I. [7] K. Black, M. Y. Galliker, and S. Levine (2025) Real-time execution of action chunking flow policies. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 38, p. 33383–33407. External Links: Document Cited by: §I, §I, §I. [8] K. Goto and R. A. van de Geijn (2008) Anatomy of high-performance matrix multiplication. ACM Transactions on Mathematical Software 34 (3), p. 1–25. External Links: Document Cited by: §I, §I, §IV-B. [9] F. G. Van Zee and R. A. van de Geijn (2015) BLIS: a framework for rapidly instantiating BLAS functionality. ACM Transactions on Mathematical Software 41 (3), p. 1–33. External Links: Document Cited by: §I, §I, §IV-B. [10] K. Black, N. Brown, J. Darpinian, et al. (2025) π0.5 _0.5: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054. Cited by: §I. [11] H. Xie, C. Yao, X. Wu, Y. Zhu, D. Liang, X. Bai, and H. Ding (2026) TurboVLA: real-time vision-language-action model at 32 Hz on an RTX 4090 with <<1 GB VRAM. arXiv preprint arXiv:2607.27205. Cited by: §I, §VI-E, TABLE I. [12] Y. Yang, Y. Wang, Z. Wen, Z. Luo, C. Zou, Z. Zhang, C. Wen, and L. Zhang (2025) EfficientVLA: training-free acceleration and compression for vision-language-action models. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 38. Cited by: §I. [13] J. Wen et al. (2025) TinyVLA: toward fast, data-efficient vision-language-action models for robotic manipulation. IEEE Robotics and Automation Letters 10 (4), p. 3988–3995. External Links: Document Cited by: §I. [14] J. Chen, J. Wang, L. Chen, C. Cai, and J. Lu (2025) NanoVLA: routing decoupled vision-language understanding for nano-sized generalist robotic policies. arXiv preprint arXiv:2510.25122. Cited by: §I. [15] H. Wang et al. (2025) BitVLA: 1-bit vision-language-action models for robotics manipulation. arXiv preprint arXiv:2506.07530. Cited by: §I. [16] Z. Yu, B. Wang, P. Zeng, et al. (2025) A survey on efficient vision-language-action models. arXiv preprint arXiv:2510.24795. Cited by: §I. [17] W. Jiang, J. Clemons, K. Sankaralingam, and C. Kozyrakis (2026) How fast can I run my VLA? demystifying VLA inference performance with VLA-Perf. arXiv preprint arXiv:2602.18397. Cited by: §I. [18] M. Vishwanathan, S. Subramanian, and A. Raghunathan (2026) Characterizing VLA models: identifying the action generation bottleneck for edge AI architectures. arXiv preprint arXiv:2603.02271. Cited by: §I. [19] K. Zhou, Q. Chen, D. Peng, et al. (2026) Characterizing vision-language-action models across XPUs: constraints and acceleration for on-robot deployment. arXiv preprint arXiv:2604.24447. Cited by: §I. [20] J. Williams, K. D. Gupta, R. George, and M. Sarkar (2025) Lite VLA: efficient vision-language-action control on CPU-bound edge robots. arXiv preprint arXiv:2511.05642. Cited by: §I. [21] R. Cadene, S. Alibert, F. Capuano, et al. (2026) LeRobot: an open-source library for end-to-end robot learning. In International Conference on Learning Representations (ICLR), Cited by: §I, §I, §VI-E. [22] Y. Liu, J. I. Hamid, A. Xie, Y. Lee, M. Du, and C. Finn (2025) Bidirectional decoding: improving action chunking via guided test-time sampling. In International Conference on Learning Representations (ICLR), Cited by: §I. [23] E. Perez, F. Strub, H. de Vries, V. Dumoulin, and A. Courville (2018) FiLM: visual reasoning with a general conditioning layer. In AAAI Conference on Artificial Intelligence, Cited by: §I. [24] E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn (2021) BC-Z: zero-shot task generalization with robotic imitation learning. In Conference on Robot Learning (CoRL), Proceedings of Machine Learning Research, Vol. 164, p. 991–1002. Cited by: §I. [25] A. Brohan, N. Brown, J. Carbajal, et al. (2023) RT-1: robotics transformer for real-world control at scale. In Robotics: Science and Systems (RSS), External Links: Document Cited by: §I. [26] S. Haldar, Z. Peng, and L. Pinto (2024) BAKU: an efficient transformer for multi-task policy learning. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 37, p. 141208–141239. External Links: Document Cited by: §I. [27] H. Bharadhwaj, J. Vakil, M. Sharma, A. Gupta, S. Tulsiani, and V. Kumar (2024) RoboAgent: generalization and efficiency in robot manipulation via semantic augmentations and action chunking. In IEEE International Conference on Robotics and Automation (ICRA), p. 4788–4795. External Links: Document Cited by: §I. [28] X. Sun, C. Liang, Z. Zeng, Q. Wang, H. Zhang, Y. Sun, Q. Li, and D. Rakita (2026) Decoupling vision, language, and action for efficient multi-task robot policies. arXiv preprint arXiv:2609.18374. Cited by: §I. [29] T. M. Low, F. D. Igual, T. M. Smith, and E. S. Quintana-Ortí (2016) Analytical modeling is enough for high-performance BLIS. ACM Transactions on Mathematical Software 43 (2), p. 12:1–12:18. External Links: Document Cited by: §I, §IV-B. [30] Google (2026) XNNPACK: high-efficiency floating-point neural network inference operators for mobile, server, and web. Note: https://github.com/google/XNNPACK Cited by: §I. [31] ONNX Runtime developers (2026) ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator. Note: https://github.com/microsoft/onnxruntime Cited by: §I. [32] Intel Corporation (2026) OpenVINO toolkit. Note: https://github.com/openvinotoolkit/openvino Cited by: §I. [33] M. Nachin, D. Desai, S. S. Jia, et al. (2026) ExecuTorch – a unified PyTorch solution to run AI models on-device. arXiv preprint arXiv:2605.08195. Cited by: §I. [34] G. Gerganov et al. (2023) Llama.cpp. Note: https://github.com/ggml-org/llama.cpp Cited by: §I. [35] K. D. Nguyen, H. T. Ho, C. T. Nguyen, T. Q. Duong, L. D. Le, D. M. H. Nguyen, V. A. Ngo, and A. T. Le (2026) Vla.cpp: a unified inference runtime for vision-language-action models. arXiv preprint arXiv:2606.08094. Cited by: §I, §I. [36] W. Thomason, Z. Kingston, and L. E. Kavraki (2024) Motions in microseconds via vectorized sampling-based planning. In IEEE International Conference on Robotics and Automation (ICRA), p. 8749–8756. Cited by: §I. [37] C. W. Ramsey, Z. Kingston, W. Thomason, and L. E. Kavraki (2024) Collision-affording point trees: SIMD-amenable nearest neighbors for fast collision checking. In Robotics: Science and Systems (RSS), Cited by: §I. [38] R. Knight, P. Kooijmans, R. Cadene, S. Alibert, M. Aractingi, D. Aubakirova, A. Zouitine, R. Martino, S. Palma, C. Pascal, and T. Wolf (2024) Standard open SO-100 & SO-101 arms. Note: https://github.com/TheRobotStudio/SO-ARM100 Cited by: §I, Fig. 5. [39] Apple Inc. (2026) Accelerate. Note: https://developer.apple.com/documentation/accelerate Cited by: §IV-A. [40] K. Chellapilla, S. Puri, and P. Simard (2006) High performance convolutional neural networks for document processing. In International Workshop on Frontiers in Handwriting Recognition (IWFHR), La Baule, France. External Links: Link Cited by: §IV-C. [41] K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p. 770–778. Cited by: §V. [42] C. Raffel, N. Shazeer, A. Roberts, et al. (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21 (140), p. 1–67. Cited by: §V. [43] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 33, p. 6840–6851. Cited by: §VI-A. [44] J. Song, C. Meng, and S. Ermon (2021) Denoising diffusion implicit models. In International Conference on Learning Representations (ICLR), Cited by: §VI-A. [45] J. Ansel, E. Yang, H. He, et al. (2024) PyTorch 2: faster machine learning through dynamic Python bytecode transformation and graph compilation. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), p. 929–947. External Links: Document Cited by: §VI-C. [46] B. Liu et al. (2023) LIBERO: benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §VI-E. [47] M. J. Kim, C. Finn, and P. Liang (2025) Fine-tuning vision-language-action models: optimizing speed and success. In Robotics: Science and Systems (RSS), External Links: Document Cited by: §VI-E.