Paper deep dive

FOCAL-VLA: Subtask-Guided Geometry Distillation and Implicit World Modeling for Vision-Language-Action Models

Zhiyuan Gao, Di Wen, Yanxiang Zhan, Mohammad Khoshnazar, Jeroen Schäfer, Kunyu Peng, Michael Beetz

Year: 2026Venue: arXiv preprintArea: cs.ROType: PreprintEmbeddings: 48

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%

Last extracted: 9/21/2026, 4:09:28 AM

Summary

The paper introduces FOCAL-VLA, a Vision-Language-Action model that enhances robotic manipulation by combining subtask-guided geometry distillation from VGGT and implicit world modeling using Track4World. This approach focuses on relevant spatial and temporal features for current interactions, outperforming baselines in simulation and real-world tasks.

Entities (8)

FOCAL-VLA · model · 100%LIBERO · dataset · 95%RoboCasa · dataset · 95%Track4World · model · 95%VGGT · model · 95%$\\pi_{0.5}$ · model · 90%University of Bremen · organization · 90%Zhiyuan Gao · person · 90%

Relation Signals (6)

FOCAL-VLA → evaluatedon → LIBERO

confidence 95% · evaluate FOCAL-VLA on LIBERO

FOCAL-VLA → evaluatedon → RoboCasa

confidence 95% · evaluate FOCAL-VLA on ... RoboCasa

FOCAL-VLA → usesteachermodel → VGGT

confidence 95% · transfer geometric knowledge from VGGT to the VLA model

FOCAL-VLA → usesteachermodel → Track4World

confidence 95% · incorporate implicit world modeling using Track4World features

Zhiyuan Gao → affiliatedwith → University of Bremen

confidence 90% · Zhiyuan Gao ... University of Bremen

FOCAL-VLA → buildson → $\\pi_{0.5}$

confidence 90% · FOCAL-VLA, built on pi0.5

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Vision-language-action (VLA) models built on pretrained vision-language models have demonstrated strong performance across diverse robotic manipulation tasks. However, VLA models that directly map current 2D observations to actions often lack sufficient spatial and temporal understanding, limiting their performance in precise and long-horizon manipulation. Recent methods enhance VLA models through geometric supervision and future-state prediction across the entire scene. However, these methods can suffer from redundant scene information, distracting the model from learning the geometry and dynamics relevant to the current interaction. To address this issue, we propose FOCAL-VLA, a framework that combines subtask-guided geometry distillation with implicit world modeling to learn representations of current spatial structure and future interaction dynamics. To focus geometric learning on the current subtask, we transfer geometric knowledge from VGGT to the VLA model by aligning geometry latents with features from subtask-relevant image regions. To capture the future 3D evolution of the current interaction, we incorporate implicit world modeling using Track4World features from current and future demonstration frames. The two complementary representations jointly guide action generation without running VGGT or Track4World at inference time. Experiments show that FOCAL-VLA outperforms baselines on both simulation benchmarks and real-world manipulation tasks. Project website: this https URL.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

44,740 characters extracted from source content.

Expand or collapse full text

FOCAL-VLA: Subtask-Guided Geometry Distillation and Implicit World Modeling for Vision–Language–Action Models Zhiyuan Gao Di Wen Yanxiang Zhan Mohammad Khoshnazar Jeroen Schäfer Kunyu Peng Michael Beetz †thanks: $ˆ1$University of Bremen, Germany. Emails: gao@uni-bremen.de, yanxiang@uni-bremen.de, khoshnam@uni-bremen.de, jeroen.schaefer@uni-bremen.de, beetz@informatik.uni-bremen.de.†thanks: $ˆ2$Karlsruhe Institute of Technology (KIT), Germany. Emails: di.wen@kit.edu, kunyu.peng@kit.edu.†thanks: $ˆ3$Robotics Institute Germany (RIG). Abstract Vision–language–action (VLA) models built on pretrained vision–language models have demonstrated strong performance across diverse robotic manipulation tasks. However, VLA models that directly map current 2D observations to actions often lack sufficient spatial and temporal understanding, limiting their performance in precise and long-horizon manipulation. Recent methods enhance VLA models through geometric supervision and future-state prediction across the entire scene. However, these methods can suffer from redundant scene information, distracting the model from learning the geometry and dynamics relevant to the current interaction. To address this issue, we propose FOCAL-VLA, a framework that combines subtask-guided geometry distillation with implicit world modeling to learn representations of current spatial structure and future interaction dynamics. To focus geometric learning on the current subtask, we transfer geometric knowledge from VGGT to the VLA model by aligning geometry latents with features from subtask-relevant image regions. To capture the future 3D evolution of the current interaction, we incorporate implicit world modeling using Track4World features from current and future demonstration frames. The two complementary representations jointly guide action generation without running VGGT or Track4World at inference time. Experiments show that FOCAL-VLA outperforms baselines on both simulation benchmarks and real-world manipulation tasks. Project website: https://zhiyuan-gao.github.io/FOCAL-VLA/. Fig. 1: Overview of FOCAL-VLA. The π0.5 _0.5 backbone processes current observations, the task instruction, proprioceptive state, and two groups of learnable queries. The output hidden states at the query positions form geometry and future motion latents, which jointly condition the action expert alongside the native vision–language context. Geometry distillation and implicit world modeling use cached, subtask-relevant targets constructed from frozen VGGT and Track4World features, respectively (Fig. 2). An auxiliary language-modeling branch predicts subtask text through the native LM head. Dashed borders indicate training-only branches; distillation prediction heads are omitted for clarity. The motion thumbnails show enlarged regions at t and t+Ht+H, where H is the action horizon; Track4World receives complete frames. I Introduction Vision–language–action (VLA) models extend pretrained vision–language models to robotic control, enabling robots to perform diverse tasks and generalize to new objects and environments [1, 2, 3, 4]. Advances in large-scale robot learning [5, 6] and training across embodiments [7] have further expanded these capabilities. However, precise and long-horizon manipulation requires more than semantic understanding: robots must perceive the spatial relationships between interacting entities and anticipate how an interaction will evolve [8, 9, 10]. To strengthen spatial understanding, recent methods incorporate 3D representations and spatial encodings into VLA models [11, 12, 13], or transfer geometric knowledge from pretrained 3D foundation models through representation alignment [8, 14, 15]. The latter provides geometric supervision during training without requiring the teacher at inference time. A complementary direction improves temporal understanding by incorporating motion history or predicting future observations [16, 17, 18]. These approaches range from visual trace prompting [16] and joint image–action prediction [18] to learning future visual features, 3D point tracks, and scene flow [9, 19, 20, 21]. Together, these developments motivate combining representations of current spatial structure with those of future interaction dynamics. The usefulness of spatial and temporal information, however, depends on its relevance to the current interaction. Recent studies address this requirement through visual attention conditioned on instructions and execution history [22], or through structured affordance representations and progress estimation [23]. For geometry and future-motion distillation, the spatial scope of the supervisory targets is therefore an important design choice. Existing methods that align all visual tokens [8, 14] or pool motion features across the entire scene [21] also supervise regions unrelated to the current interaction. Such supervision can introduce redundant scene information, distracting the model from learning the geometry and dynamics relevant to the current operation. Moreover, the information needed for action generation changes as the interaction progresses. For example, placing an object into a drawer requires understanding the object–drawer spatial relationship, whereas closing the drawer requires anticipating the drawer’s motion. These observations motivate constructing geometric and future-motion supervision around the entities involved in the current interaction. To address these limitations, we propose FOCAL-VLA, built on π0.5 _0.5 [4], which combines subtask-guided geometry distillation with implicit world modeling to guide action generation (Fig. 1). To focus geometric learning on the current subtask, we distill features from frozen VGGT [24] into geometry latents by aggregating teacher features only from regions corresponding to the entities involved in that subtask. The selected regions change with the interacting entities across subtasks. To anticipate how the current interaction will unfold, we incorporate implicit world modeling through future motion latents learned from current observations. Frozen Track4World [25] extracts features from current and future demonstration frames. We aggregate features only from regions relevant to the current subtask to supervise the future motion latents. The two complementary representations jointly condition the action expert, without running either teacher at inference time. The subtask annotations and relevance masks used to construct distillation targets are not required at inference time. We evaluate FOCAL-VLA on LIBERO [26], RoboCasa [27], and four real-world manipulation tasks. FOCAL-VLA performs competitively with existing VLA baselines on both simulation benchmarks and outperforms the π0.5 _0.5 baseline on real-world manipulation tasks. Ablation studies further demonstrate the complementary benefits of geometry and future motion supervision, while showing that restricting supervision to subtask-relevant regions outperforms full-image supervision. In summary, our main contributions are: • To focus geometric learning on the current subtask, we transfer VGGT’s geometric knowledge to the VLA model through distillation over subtask-relevant regions, adapting the selected regions as the interacting entities change. • To anticipate how the current interaction will unfold, we introduce implicit world modeling through future motion latents, using future demonstration context while restricting spatial supervision to regions relevant to the current subtask. • To evaluate the effectiveness of FOCAL-VLA, we conduct experiments on simulation benchmarks and real-world manipulation tasks, complemented by ablation studies examining the contributions of the distillation branches, supervision scope, and action conditioning. I Related Work I-A Vision–Language–Action Models VLA models adapt pretrained vision–language representations to robot action generation [1, 2, 3]. Representative designs include autoregressive action tokenization in OpenVLA [2] and continuous flow-matching action generation in π0 _0 [3]. Recent generalist models further expand capabilities for instruction following and transfer across tasks and embodiments: π0.7 _0.7 [28] uses diverse context conditioning to learn from heterogeneous data, while LingBot-VLA [29] scales pretraining across multiple dual-arm robot configurations. Our base policy, π0.5 _0.5 [4], combines a continuous action expert with heterogeneous co-training that includes semantic subtask prediction. FOCAL-VLA builds on this architecture and augments its policy representations through subtask-guided geometry distillation and implicit world modeling. I-B Geometry-Enhanced VLA Models One approach to improving spatial understanding is to provide the policy with explicit geometric information, including point-cloud embeddings [30, 31], robot-centric pointmaps [32], and egocentric 3D position encodings [12]. Rather than supplying explicit geometric inputs to the policy, feature-distillation methods transfer geometric knowledge from pretrained 3D foundation models into policy representations without requiring the teacher at inference time. Spatial Forcing [8] and GLaD [14] align a single VLA backbone layer with VGGT features, while ROCKET [33] extends alignment to multiple layers through a shared projector. 3DThinkVLA [15] incorporates both geometric and spatial reasoning features into action-query states to guide action prediction. These methods align geometric features across the entire image rather than focusing the alignment on regions relevant to the current subtask. We align the policy’s geometry latents with teacher features from subtask-relevant regions, adapting these regions as the interacting entities change across subtasks. The resulting geometry latents directly condition the action expert. I-C Future Prediction for VLA Models One line of work guides VLA action generation by predicting future images or visual features. CoT-VLA [34] generates a future goal image before generating its action chunk, while DreamVLA [9] predicts future dynamic regions, depth, and semantic features. Related world-action models couple future video generation with action prediction or planning [35, 36]. Other methods represent future changes through motion features or 3D trajectories. CoWVLA [37] learns a factorized motion representation from video, while GeoPredict [38] combines future robot-track prediction with predictive 3D Gaussian geometry. Pri4R [19] uses auxiliary future 3D point-track prediction during training while retaining the original VLA architecture at deployment. LaMP [20] learns future 3D scene flow with a motion expert and supplies its denoising features to the action expert. Track4Action [21] pools scene and motion features from Track4World over an action-aligned clip into a tracker target that supervises query states. These query states condition action generation through gated fusion. Our method uses the same teacher but restricts feature distillation to regions relevant to the current subtask. This supervises the policy’s future motion latents using features that capture upcoming 3D changes in the interacting entities. I Method In this section, we present FOCAL-VLA. We begin with the preliminaries (Sec. I-A), followed by subtask-guided geometry distillation (Sec. I-B) and implicit world modeling through future-informed motion distillation (Sec. I-C). We then describe action generation and training (Sec. I-D). An overview of FOCAL-VLA is shown in Fig. 1. I-A Preliminaries We first review the VLA formulation using π0.5 _0.5 [4], which serves as our base policy. It combines a pretrained PaliGemma vision–language backbone with a continuous action expert. At time t, the backbone encodes the current multi-view observation oto_t, task instruction ℓ , and tokenized proprioceptive state into context tokens CtC_t. Conditioned on CtC_t, the action expert predicts an action chunk at:t+H−1=(at,…,at+H−1)a_t:t+H-1=(a_t,…,a_t+H-1), where ata_t denotes a robot action and H is the prediction horizon. The action expert is trained on expert demonstrations through conditional flow matching and generates actions by integrating a learned velocity field from Gaussian noise. Building on this architecture, FOCAL-VLA introduces two groups of learnable queries, QgQ_g and QmQ_m, as additional input tokens to the vision–language backbone. Their output hidden states form the geometry latents ZgZ_g and future motion latents ZmZ_m, respectively, with the attention pattern defined in Sec. I-D. The geometry latents are aligned with subtask-relevant VGGT features, while the future motion latents are aligned with future-informed Track4World features. Both groups provide additional conditioning to the action expert alongside CtC_t. I-B Subtask-Guided Geometry Distillation Fig. 2: Construction of subtask-relevant geometry and future motion targets. (a–b) PICK and CLOSE from the same demonstration select different entities: the black bowl and the bottom drawer, respectively. (c) VGGT features are pooled within the selected regions in each current view and averaged across valid views to form the geometry target. (d) Track4World processes current and future frames over the action horizon H. The current subtask mask selects features at time t, enriched by future context, for motion-target pooling. Mask overlays indicate pooling regions; both teachers receive complete images. Color blocks show standardized cached target vectors on a shared [−4,4][-4,4] scale, not spatial maps; feature stacks are schematic. Existing geometry-distillation methods align VLA visual embeddings with 3D foundation-model features across the entire image. This alignment includes scene regions unrelated to the current operation and does not account for changes in the relevant entities across subtasks. To focus geometric learning on the current interaction, we propose subtask-guided geometry distillation, which aligns the policy’s geometry latents with features from our geometry teacher, VGGT [24], aggregated over entities involved in the current subtask (Fig. 2). The geometry query group QgQ_g consists of eight learnable tokens appended to CtC_t. VGGT is a feed-forward multi-view model for camera and scene geometry estimation. The frozen model jointly processes the current-view images, yielding final-layer patch features Gtv∈ℝPg×dgG_t^v ^P_g× d_g for each view v∈v , where PgP_g is the number of patches per view and dg=2048d_g=2048. To select features relevant to the current subtask, we use an offline vision–language model to align manually specified subtask descriptions with demonstration frames and identify the entities involved. A binary relevance mask MtvM_t^v is formed by taking the union of the whole-entity masks for entities involved in the current subtask. Let wt,pv∈[0,1]w_t,p^v∈[0,1] denote the coverage of the transformed relevance mask on patch p. We aggregate features only within these subtask-relevant regions, pooling within each view and averaging over views with nonzero mask coverage: ytg=1|t|∑v∈t∑p=1Pgwt,pvGt,pv∑p=1Pgwt,pv,y_t^g= 1|V_t| _v _t _p=1^P_gw_t,p^vG_t,p^v _p=1^P_gw_t,p^v, (1) where t=v:∑pwt,pv>0V_t=\v: _pw_t,p^v>0\ is the set of valid views. A prediction head applies per-query normalization, mean pooling, and a linear projection to produce y^tg=hg(Zg)∈ℝdg y_t^g=h_g(Z_g) ^d_g. For each branch b∈g,mb∈\g,m\, we standardize targets and predictions using x~=(x−μb)/σb x=(x- _b)/ _b, where μb _b and σb _b are the per-dimension target mean and standard deviation computed on the training split. We align the geometry latents with the pooled VGGT target using a masked mean-squared error. Let ℬgB_g contain the batch samples with at least one valid view: ℒgeo=1|ℬg|dg∑t∈ℬg‖y^~tg−y~tg‖22.L_geo= 1|B_g|d_g _t _g \| y_t^g- y_t^g \|_2^2. (2) I-C Implicit World Modeling We implement implicit world modeling through future-informed motion distillation. Conditioned on the current context CtC_t, the model learns to infer a latent representation of upcoming 3D interaction dynamics, using future demonstration frames only to construct training targets. Recent approaches incorporate future motion representations into VLA models. However, full-scene feature pooling [21] combines motion cues from interacting entities with features from regions unrelated to the current subtask. Our future-informed motion distillation addresses this issue by restricting feature aggregation to the currently relevant entities while retaining temporal context from future demonstration frames. Aggregated features from our motion teacher, Track4World [25], provide targets for the policy’s future motion latents (Fig. 2). We implement this branch with eight learnable query tokens QmQ_m appended to CtC_t. Track4World is a feed-forward model that estimates dense 3D scene flow between video frames and reconstructs point trajectories in a shared world coordinate system. We use its frozen motion features, rather than the reconstructed trajectories, as distillation targets. By processing current and future demonstration frames together, the teacher provides motion features informed by the subsequent interaction. For a training anchor at time t, let t=τkk=1KT_t=\ _k\_k=1^K denote future timestamps sampled uniformly over the demonstration interval corresponding to the next action chunk, excluding the current frame. Sampling spans the full chunk interval without truncation at subtask boundaries. The teacher processes the current agent-view frame otagento_t^agent together with oτkagentk=1K\o_ _k^agent\_k=1^K, for K+1K+1 frames in total. The current agent-view mask MtagentM_t^agent selects the features used for distillation. Let Dt,p∈ℝdmD_t,p ^d_m denote the final output of Track4World’s temporally global 3D flow-feature aggregator at source time index zero and source patch p, after the teacher has processed the full clip, where dm=256d_m=256. Mapping the current relevance mask to the teacher’s 45×4545× 45 source grid (Pm=2025P_m=2025) gives coverage weights ut,pu_t,p. The motion target is ytm=∑p=1Pmut,pDt,p∑p=1Pmut,p.y_t^m= _p=1^P_mu_t,pD_t,p _p=1^P_mu_t,p. (3) Each Dt,pD_t,p incorporates temporal context from the full clip, while the current mask determines which entities contribute to ytmy_t^m. Aligning ZmZ_m with this target trains the policy to infer future motion features of the current subtask’s entities from its current context. Future evolution is thus modeled in latent feature space, without explicitly reconstructing future images or 3D trajectories. A separate head with the same normalize–mean–project structure predicts y^tm=hm(Zm)∈ℝ256 y_t^m=h_m(Z_m) ^256, supervised by a masked Smooth L1 loss with transition parameter β=1β=1 in the standardized feature space: ℒmotion=1|ℬm|∑t∈ℬmSmoothL1β=1⁡(y^~tm,y~tm).L_motion= 1|B_m| _t _mSmoothL1_β=1 ( y_t^m, y_t^m ). (4) Here Smooth L1 is averaged over the feature dimensions, and ℬmB_m contains batch samples with a valid current agent mask and all K sampled future frames. I-D Action Generation and Training The geometry and future motion query groups attend independently to CtC_t and to tokens within their own groups. The action expert attends to both groups alongside CtC_t, while the context tokens retain their native pathway. The attention pattern is Ct C_t ←Ct, ← C_t, (5) Zg Z_g ←Ct+Zg, ← C_t+Z_g, Zm Z_m ←Ct+Zm, ← C_t+Z_m, Action ←Ct+Zg+Zm. ← C_t+Z_g+Z_m. Here each left-hand side denotes the token group being updated, the right-hand side lists the groups it may attend to, and ++ denotes their union. For the action expert, only external conditioning groups are shown; its native internal attention pattern is retained. To provide the shared context with subtask-level semantic guidance, we also train the model’s native language-modeling branch to predict the subtask text obtained in Sec. I-B. This branch reuses the PaliGemma tokenizer, token embeddings, transformer, and tied language-model head to generate the text autoregressively from CtC_t. Given target tokens s1:Lss_1:L_s of length LsL_s, the semantic objective is ℒsem=−1Ls∑k=1Lslogpθ(sk∣Ct,s<k),L_sem=- 1L_s _k=1^L_s p_θ(s_k C_t,s_<k), (6) including the end-of-sequence token and excluding padded positions. The native autoregressive branch requires no additional semantic query or semantic embedding head. Its attention mask allows target tokens to attend to CtC_t and their causal prefix, but prevents the context, geometry, future motion, and action tokens from attending to semantic target tokens. The semantic loss therefore trains the shared vision–language context without providing ground-truth subtask text to the action pathway. The overall objective is ℒ=ℒaction+λgℒgeo+λsℒsem+λmℒmotion,L=L_action+ _gL_geo+ _sL_sem+ _mL_motion, (7) where ℒactionL_action is the native π0.5 _0.5 action objective, and λg _g, λs _s, and λm _m weight the three auxiliary objectives. We jointly optimize the policy parameters, query embeddings, and prediction heads using the cached distillation targets. At inference time, the policy computes ZgZ_g and ZmZ_m from the current observation, instruction, and proprioceptive state and uses them to condition the action expert. The teachers, future demonstration frames, relevance masks, and semantic targets are not used. IV Experiments We evaluate FOCAL-VLA on the LIBERO and RoboCasa simulation benchmarks and on real-world robotic manipulation tasks. We further conduct ablation studies to examine the contributions of individual components and the effects of our design choices. TABLE I: Comparison on LIBERO (success rate, %). Bold indicates the best result in each column. Method Spatial Object Goal Long Avg. OpenVLA [2] 84.7 88.4 79.2 53.7 76.5 π0 _0 [3] 96.8 98.8 95.8 85.2 94.2 π0.5 _0.5 [4] 98.8 98.2 98.0 92.4 96.9 OpenVLA-OFT [39] 97.6 98.4 97.9 94.5 97.1 GR00T N1.7 [40] 95.0 100.0 98.0 93.0 96.5 SpatialVLA [12] 88.2 89.9 78.6 55.5 78.1 DreamVLA [9] 97.5 94.0 89.5 89.5 92.6 GeoPredict [38] 98.0 98.2 95.7 94.0 96.5 Track4Action [21] 95.0 99.6 97.6 95.8 97.0 FOCAL-VLA 100.0 98.6 97.8 95.0 97.9 IV-A Experimental Setup Simulation benchmarks We train one policy per benchmark. LIBERO [26] comprises Spatial, Object, Goal, and Long (LIBERO-10), each with ten tasks; the policy uses agent and wrist views and predicts and executes 10-step action chunks. For RoboCasa [27], we use Atomic-24 under Human-50 (50 demonstrations per task), with left agent, wrist, and right agent views; the policy predicts 50-step chunks and executes 25 steps before replanning. We evaluate 50 episodes per task, reporting LIBERO’s per-suite success rates and their unweighted average, and RoboCasa’s unweighted average task success rate. Real-world experiments We use a Franka Emika Panda with one fixed external Orbbec Astra Pro RGB camera (Fig. 3). Fig. 4 illustrates four tasks: sequentially placing a banana and mandarin into instructed plates (possibly the same plate), fully opening a specified upper or lower drawer, storing a banana in a specified initially open upper or lower drawer and closing it, and closing the microwave. We collect 50–150 successful demonstrations per task using SpaceMouse teleoperation, totaling 350 demonstrations recorded at 10 Hz with 640×480640× 480 RGB observations. FOCAL-VLA and π0.5 _0.5 each learn one policy across all four tasks, predicting and executing 10-step chunks. We report per-task and overall success rates over 20 trials per task, using the same initial-layout distribution for both methods. Success requires completing all instructed actions in order and reaching the specified final configuration. Fig. 3: Real-world platform with a Franka Emika Panda and a fixed external camera. The workspace contains the objects used across the four manipulation tasks. Implementation details We fine-tune the official OpenPI π0.5 _0.5 implementation end to end. Offline annotation (Sec. I-B) uses Qwen3-VL-8B-Instruct [41] to identify subtask starts and entities from sampled demonstration frames, task instructions, and manually specified subtask descriptions. Entities include manipulated objects and associated containers, supports, or articulated objects, excluding the gripper. Masks come from simulator annotations or entity-prompted Grounded-SAM-2 [42, 43] for real-world data. Frozen VGGT and Track4World targets are cached offline; RoboCasa motion targets use the current frame and ten future frames uniformly sampled over the 50-step chunk. Training uses eight NVIDIA A100 80GB GPUs and AdamW (β1=0.9 _1=0.9, β2=0.95 _2=0.95, ϵ=10−8ε=10^-8, weight decay 10−1010^-10, gradient-norm clipping at 1.0). Simulation training uses 30,000 updates with global batch sizes of 256 for LIBERO and 128 for RoboCasa; real-world training uses 16,000 updates with a batch size of 256. We linearly warm up to 5×10−55× 10^-5 over 10,000 updates in simulation and 5,200 in real-world experiments. The learning rate then remains constant, except on RoboCasa, where it decays to 5×10−65× 10^-6 with a cosine schedule. For the full model, we use λg=0.05 _g=0.05, λs=0.01 _s=0.01, and λm=0.05 _m=0.05, with ablation-specific changes described below. We use the official π0.5 _0.5 training and evaluation seeds. Fig. 4: Illustration of the four real-world tasks. Representative frames from each task are shown in temporal order from left to right, viewed from the fixed external policy camera. Task descriptions appear below each sequence. A uniform color correction is applied for display. Baselines We evaluate π0.5 _0.5 on RoboCasa using the same training and evaluation settings as FOCAL-VLA. Results for the other simulation baselines are taken from the cited sources. GR00T N1.7 results follow the preliminary official LeRobot evaluation [40]. RoboCasa results for KYC, GeoVLA, and PointVLA use Pointmap’s π0.5 _0.5-based reimplementations [32]; Being-H0.5 uses its RoboCasa-specific model. IV-B Evaluation Results LIBERO Table I summarizes results on LIBERO. FOCAL-VLA achieves an average success rate of 97.9%, improving over π0.5 _0.5 by 1.0 percentage point. It reaches 100.0% on Spatial and obtains its largest gain on Long, improving from 92.4% to 95.0%. These results are consistent with our joint modeling of current geometry and future interaction dynamics: geometry distillation emphasizes the spatial relationships needed for the current operation, while implicit world modeling complements this information with anticipated changes in the interacting entities. As a task progresses, the supervision follows the entities involved in each subtask rather than treating all image regions equally. RoboCasa Table I reports results on RoboCasa. FOCAL-VLA achieves 63.4% average success, improving over π0.5 _0.5 by 8.2 percentage points and outperforming the other listed methods. Kitchen manipulation requires selecting the objects and fixtures involved in the current operation from a cluttered scene. Subtask-guided geometry distillation targets the spatial structure of these entities, while implicit world modeling captures their future interaction dynamics using features from current and future demonstration frames. The two branches therefore provide complementary information about the same interaction: its current configuration and its future evolution. This design offers an explanation for the improvement over the base policy, with the ablations below examining the roles of both distillation branches, supervision scope, and action conditioning. TABLE I: Comparison on RoboCasa. The best success rate is shown in bold. Method Avg. SR (%) Base VLAs π0 _0 [19] 38.8 π0.5 _0.5 [4] 55.2 OpenVLA-OFT [19] 33.1 Being-H0.5 [44] 53.9 3D inputs required at inference KYC [45, 32] 59.1 GeoVLA [31, 32] 57.1 PointVLA [30, 32] 57.3 Robot-Centric Pointmap [32] 62.9 3D inputs not required at inference GeoPredict [38] 52.4 GaussianDream [46] 54.8 Pri4R [19] 57.0 FOCAL-VLA 63.4 Real-world experiments Table I reports the real-world results on the four tasks illustrated in Fig. 4. FOCAL-VLA improves success on all four tasks, raising the overall success rate from 47.5% to 63.8%. Two-fruit rearrangement shows the largest gain and requires switching between instructed objects and destinations. Its improvement provides further evidence of the method’s effectiveness in sequential manipulation. Drawer storage similarly requires the robot to place the banana inside the drawer and then switch to closing it. Completing this sequence involves a change in both the interacting entity and the required motion. The gain is consistent with our motivation to associate geometric and future motion information with the current subtask: the policy must use the banana–drawer spatial relationship during placement and the drawer’s subsequent motion during closing. Gains on Drawer opening and Close the microwave indicate that the benefits also extend to individual articulated-object interactions, rather than being limited to tasks with multiple subtasks. Some real-world grasping failures occur when the target object is occluded in the single fixed camera view. TABLE I: Real-world task success rates (%). Each method is evaluated in 20 trials per task. Task π0.5 _0.5 FOCAL-VLA Two-fruit rearrangement 25.0 50.0 Drawer opening 70.0 80.0 Drawer storage 20.0 40.0 Close the microwave 75.0 85.0 Overall 47.5 63.8 IV-C Ablation Studies We ablate the two distillation branches, supervision scope, and action conditioning on RoboCasa under the same training and evaluation protocol (Table IV). TABLE IV: Ablations on RoboCasa. Variant Success (%) Full method 63.4 Without geometry 60.0 Without future motion 59.6 Task-level entity supervision 61.9 Full-image supervision 58.7 Supervision-only 58.5 Geometry and future motion To evaluate each teacher’s contribution, we separately remove the geometry or future motion queries together with the corresponding distillation loss. Removing geometry reduces success from 63.4% to 60.0%, while removing future motion reduces it to 59.6%. The drop without future motion supports learning anticipated interaction changes in addition to current geometry. Conversely, the drop without geometry shows the benefit of retaining a dedicated representation of the current spatial configuration. Together, these results support combining subtask-relevant geometry distillation with implicit world modeling, rather than relying on either learning signal alone. Supervision scope To evaluate the scope of supervision, we compare subtask-relevant supervision with task-level entity supervision and full-image supervision. Different subtasks within a task may involve different entities. Task-level entity supervision uses the union of entities relevant to all subtasks throughout the task, whereas subtask-level supervision selects only those relevant to the current subtask. In both cases, masks follow the entities in each frame. Both geometry and future motion distillation use these masks, with all other settings unchanged. Full-image supervision instead pools VGGT and Track4World features over the entire image [8, 14, 21]. Task-level entity supervision achieves 61.9% success, exceeding full-image supervision at 58.7%, while the full model achieves 63.4%. These results suggest that selecting task-relevant entities improves performance, with a further 1.5-percentage-point gain from adapting the supervised entities to the current subtask. Action conditioning To evaluate the benefit of direct action conditioning, we retain both distillation branches but disconnect their output latents from the action expert, yielding a supervision-only variant. The supervision-only variant achieves 58.5% success, compared with 63.4% for the full model. This 4.9-percentage-point gain supports directly conditioning the action expert on the learned latents, rather than using them solely as auxiliary training signals. V Limitations FOCAL-VLA relies on offline subtask annotation and teacher-target construction, which add data-preparation and computational costs. Errors in subtask boundaries or entity masks can affect the quality of the distillation targets. Our evaluation covers two simulation benchmarks and a single real-world robot platform, leaving generalization to other embodiments and task families unexplored. Future work will investigate reducing the annotation effort and extending evaluation to more diverse robotic settings. VI Conclusion We presented FOCAL-VLA, a framework for enhancing VLA models through subtask-guided geometry distillation and implicit world modeling. Geometry distillation aligns the policy’s geometry latents with VGGT features from entities relevant to each subtask. Implicit world modeling is implemented through future-informed motion distillation, using Track4World features from current and future demonstration frames. These features are pooled over subtask-relevant regions to construct the distillation targets. The two groups of latents jointly guide action generation without requiring teacher models or subtask annotations at inference time. Experiments on LIBERO, RoboCasa, and real-world manipulation tasks show higher average success rates than the π0.5 _0.5 baseline. These results highlight the value of subtask-relevant geometric and future motion knowledge for improving VLA manipulation performance. References [1] B. Zitkovich et al. (2023) RT-2: vision–language–action models transfer web knowledge to robotic control. In Proceedings of the Conference on Robot Learning, p. 2165–2183. Cited by: §I, §I-A. [2] M. J. Kim et al. (2025) OpenVLA: an open-source vision–language–action model. In Proceedings of the Conference on Robot Learning, p. 2679–2713. Cited by: §I, §I-A, TABLE I. [3] K. Black et al. (2025) π0 _0: A vision–language–action flow model for general robot control. In Robotics: Science and Systems, Cited by: §I, §I-A, TABLE I. [4] K. Black et al. (2025) π0.5 _0.5: A vision–language–action model with open-world generalization. In Proceedings of the Conference on Robot Learning, p. 17–40. Cited by: §I, §I, §I-A, §I-A, TABLE I, TABLE I. [5] A. Brohan et al. (2023) RT-1: robotics transformer for real-world control at scale. In Robotics: Science and Systems, Cited by: §I. [6] Octo Model Team et al. (2024) Octo: an open-source generalist robot policy. In Robotics: Science and Systems, Cited by: §I. [7] A. O’Neill et al. (2024) Open X-Embodiment: robotic learning datasets and RT-X models. In Proceedings of the IEEE International Conference on Robotics and Automation, p. 6892–6903. Cited by: §I. [8] F. Li, W. Song, H. Zhao, J. Wang, P. Ding, D. Wang, L. Zeng, and H. Li (2026) Spatial Forcing: implicit spatial representation alignment for vision–language–action model. In International Conference on Learning Representations, Cited by: §I, §I, §I-B, §IV-C. [9] W. Zhang et al. (2025) DreamVLA: a vision–language–action model dreamed with comprehensive world knowledge. In Advances in Neural Information Processing Systems, Vol. 38, p. 24195–24228. Cited by: §I, §I-C, TABLE I. [10] M. Lin et al. (2025) HiF-VLA: hindsight, insight and foresight through motion representation for vision–language–action models. arXiv preprint arXiv:2512.09928. Cited by: §I. [11] H. Zhen, X. Qiu, P. Chen, J. Yang, X. Yan, Y. Du, Y. Hong, and C. Gan (2024) 3D-VLA: a 3D vision–language–action generative world model. In Proceedings of the International Conference on Machine Learning, p. 61229–61245. Cited by: §I. [12] D. Qu et al. (2025) SpatialVLA: exploring spatial representations for visual–language–action model. arXiv preprint arXiv:2501.15830. Cited by: §I, §I-B, TABLE I. [13] Z. Zhang, H. Li, Y. Dai, Z. Zhu, L. Zhou, C. Liu, D. Wang, F. E. H. Tay, S. Chen, Z. Liu, Y. Liu, X. Li, and P. Zhou (2026) From spatial to actions: grounding vision–language–action model in spatial foundation priors. In International Conference on Learning Representations, Cited by: §I. [14] M. Guo, M. Cao, J. Tao, R. Xu, Y. Yan, X. Liang, I. Laptev, and X. Chang (2025) GLaD: geometric latent distillation for vision–language–action models. arXiv preprint arXiv:2512.09619. Cited by: §I, §I, §I-B, §IV-C. [15] J. Shi, X. Zhang, F. Zhu, Z. Li, S. Zhu, and W. Yuan (2026) 3DThinkVLA: endowing vision–language–action models with latent 3D priors via 3D-thinking-guided co-training. arXiv preprint arXiv:2606.04436. Cited by: §I, §I-B. [16] R. Zheng, Y. Liang, S. Huang, J. Gao, H. Daumé I, A. Kolobov, F. Huang, and J. Yang (2025) TraceVLA: visual trace prompting enhances spatial-temporal awareness for generalist robotic policies. In International Conference on Learning Representations, Cited by: §I. [17] Y. Du, S. Yang, B. Dai, H. Dai, O. Nachum, J. Tenenbaum, D. Schuurmans, and P. Abbeel (2023) Learning universal policies via text-guided video generation. In Advances in Neural Information Processing Systems, Vol. 36, p. 9156–9172. Cited by: §I. [18] H. Wu et al. (2024) Unleashing large-scale video generative pre-training for visual robot manipulation. In International Conference on Learning Representations, Cited by: §I. [19] J. Kim, J. Cho, S. Chu, A. Bal, J. Kim, G. Lee, S. Lee, S. H. Kim, B. Han, H. Lee, L. A. Jeni, and S. Kim (2026) Pri4R: learning world dynamics for vision–language–action models with privileged 4D representation. arXiv preprint arXiv:2603.01549. Cited by: §I, §I-C, TABLE I, TABLE I, TABLE I. [20] X. Wang, C. Wang, Y. Xu, M. Ye, F. Zhang, J. Tian, X. Zhan, L. Zhu, C. Lu, and L. Yang (2026) LaMP: learning vision–language–action policy with 3D scene flow as latent motion prior. arXiv preprint arXiv:2603.25399. Cited by: §I, §I-C. [21] C. Wang, X. Wang, B. Lin, J. Tian, F. Zhang, C. Lu, and L. Yang (2026) Track4Action: distilling world-centric 3D tracker into vision–language–action policies. arXiv preprint arXiv:2608.03727. Cited by: §I, §I, §I-C, §I-C, §IV-C, TABLE I. [22] L. Xiao, J. Li, J. Gao, F. Ye, Y. Jin, J. Qian, J. Zhang, Y. Wu, and X. Yu (2026) AVA-VLA: improving vision–language–action models with active visual attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 13453–13463. Cited by: §I. [23] Y. Liu, J. Zhu, Y. Mo, G. Li, X. Cao, J. Jin, Y. Shen, Z. Li, T. Yu, W. Yuan, F. Ding, and I. Lourentzou (2026) PALM: progress-aware policy learning via affordance reasoning for long-horizon robotic manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 28096–28110. Cited by: §I. [24] J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny (2025) VGGT: visual geometry grounded transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 5294–5306. Cited by: §I, §I-B. [25] J. Lu, J. Xu, W. Hu, R. Zhu, C. Zhao, S. Yeung, Y. Shan, and Y. Liu (2026) Track4World: feedforward world-centric dense 3D tracking of all pixels. arXiv preprint arXiv:2603.02573. Cited by: §I, §I-C. [26] B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone (2023) LIBERO: benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems, Vol. 36, p. 44776–44791. Cited by: §I, §IV-A. [27] S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, A. Joshi, A. Mandlekar, and Y. Zhu (2024) RoboCasa: large-scale simulation of everyday tasks for generalist robots. In Robotics: Science and Systems, Cited by: §I, §IV-A. [28] Physical Intelligence et al. (2026) π0.7 _0.7: A steerable generalist robotic foundation model with emergent capabilities. arXiv preprint arXiv:2604.15483. Cited by: §I-A. [29] W. Wu et al. (2026) A pragmatic VLA foundation model. arXiv preprint arXiv:2601.18692. Cited by: §I-A. [30] C. Li, J. Wen, Y. Peng, Y. Peng, and Y. Zhu (2026) PointVLA: injecting the 3D world into vision–language–action models. IEEE Robotics and Automation Letters 11 (3), p. 2506–2513. Note: doi: 10.1109/LRA.2026.3653303 Cited by: §I-B, TABLE I. [31] L. Sun, B. Xie, Y. Liu, H. Shi, T. Wang, and J. Cao (2025) GeoVLA: empowering 3D representations in vision–language–action models. arXiv preprint arXiv:2508.09071. Cited by: §I-B, TABLE I. [32] B. Lee, D. Hwang, D. Kim, H. Lee, M. Park, and J. Choo (2026) See like a robot: robot-centric pointmaps for vision–language–action models. arXiv preprint arXiv:2607.11498. Cited by: §I-B, §IV-A, TABLE I, TABLE I, TABLE I, TABLE I. [33] G. Sun, T. Du, K. Feng, C. Luo, X. Ding, Z. Shen, Z. Wang, Y. He, and A. Li (2026) ROCKET: residual-oriented multi-layer alignment for spatially-aware vision–language–action models. arXiv preprint arXiv:2602.17951. Cited by: §I-B. [34] Q. Zhao, Y. Lu, M. J. Kim, Z. Fu, Z. Zhang, Y. Wu, Z. Li, Q. Ma, S. Han, C. Finn, A. Handa, T. Lin, G. Wetzstein, M. Liu, and D. Xiang (2025) CoT-VLA: visual chain-of-thought reasoning for vision–language–action models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 1702–1713. Cited by: §I-C. [35] M. J. Kim, Y. Gao, T. Lin, Y. Lin, Y. Ge, G. Lam, P. Liang, S. Song, M. Liu, C. Finn, and J. Gu (2026) Cosmos Policy: fine-tuning video models for visuomotor control and planning. In International Conference on Learning Representations, Cited by: §I-C. [36] J. A. Lopetegui Gonzalez, P. Pacaud, and C. Schmid (2026) Spatially aware world action model via geometric latent diffusion. arXiv preprint arXiv:2609.02531. Cited by: §I-C. [37] F. Yang, D. Di, L. Tang, X. Zhang, L. Fan, H. Li, W. Chen, T. Su, and B. Ma (2026) Chain of world: world model thinking in latent motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 6675–6684. Cited by: §I-C. [38] J. Qian, B. Han, C. Shi, L. Xiao, L. Yang, S. Shi, and L. Jiang (2026) GeoPredict: leveraging predictive kinematics and 3D gaussian geometry for precise VLA manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 13529–13539. Cited by: §I-C, TABLE I, TABLE I. [39] M. J. Kim, C. Finn, and P. Liang (2025) Fine-tuning vision–language–action models: optimizing speed and success. In Robotics: Science and Systems, Cited by: TABLE I. [40] Hugging Face LeRobot (2026) GR00T policy: GR00T N1.7 LIBERO results. Note: Preliminary LeRobot integration results. Accessed: Sep. 10, 2026 External Links: Link Cited by: §IV-A, TABLE I. [41] Qwen Team (2025) Qwen3-VL technical report. arXiv preprint arXiv:2511.21631. Cited by: §IV-A. [42] S. Liu et al. (2024) Grounding DINO: marrying DINO with grounded pre-training for open-set object detection. In Proceedings of the European Conference on Computer Vision, p. 38–55. Cited by: §IV-A. [43] N. Ravi et al. (2025) SAM 2: segment anything in images and videos. In International Conference on Learning Representations, Cited by: §IV-A. [44] H. Luo et al. (2026) Being-H0.5: scaling human-centric robot learning for cross-embodiment generalization. arXiv preprint arXiv:2601.12993. Cited by: TABLE I. [45] T. Jiang, J. Ji, X. Tan, J. Fang, A. Bhattad, V. Guizilini, and M. R. Walter (2026) Do you know where your camera is? View-invariant policy learning with camera conditioning. In Proceedings of the IEEE International Conference on Robotics and Automation, Cited by: TABLE I. [46] Z. Zhang et al. (2026) GaussianDream: a feed-forward 3D gaussian world model for robotic manipulation. arXiv preprint arXiv:2605.20752. Cited by: TABLE I.

Instant research discovery