Paper deep dive

AtomEgo: Exploring Ego-Robot Integration for Embodied Foundation Model Pretraining

Di Wu, Dongchen Zheng, Junhe Sheng, Zhongxing Wei, Songxin Zhang, Zejian Xie, Xiaoquan Sun, Junyang Zheng, Zhuoyang Song, Jiaxing Zhang, Jiayu Chen

Year: 2026Venue: arXiv preprintArea: cs.ROType: PreprintEmbeddings: 60

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 9/21/2026, 4:25:20 AM

Summary

The paper introduces AtomEgo, a systematic study on integrating large-scale egocentric human interaction data with robot demonstrations for pretraining embodied foundation models. It addresses the embodiment and action-space gaps between humans and robots by curating a 2,659-hour corpus and investigating three co-training paradigms: joint co-training with domain-specific action heads (Atom-DH), progressive ego-to-robot transfer via embodiment alignment (Atom-CL), and joint video-action modeling (Atom-WAM). The study establishes a key principle: Data Scale × Alignment Quality → Capability Gain, demonstrating that effective alignment is crucial for leveraging egocentric data to improve robot generalization.

Entities (15)

AtomEgo · methodframework · 98%Data Scale * Alignment Quality -> Capability Gain · principle · 95%EgoVerse · dataset · 95%Atom-CL · model-variant · 92%Atom-DH · model-variant · 92%Atom-WAM · model-variant · 92%Embodiment Alignment · technique · 90%Fast-WAM · modelarchitecture · 90%PaliGemma · modelarchitecture · 90%Vision-Language-Action (VLA) · model-class · 90%World-Action Model (WAM) · model-class · 90%AgiBot World Beta · dataset · 85%DROID · dataset · 85%RoboCOIN · dataset · 85%RoboMIND · dataset · 85%

Relation Signals (11)

AtomEgo → curates → 2,659-hour corpus

confidence 95% · supported by a curated corpus of approximately 2,659 hours

AtomEgo → discovers → Data Scale * Alignment Quality -> Capability Gain

confidence 95% · Our results reveal a simple principle: Data Scale * Alignment Quality --> Capability Gain

AtomEgo → uses → EgoVerse

confidence 95% · We use EgoVerse [21] as the source of approximately 1,079.5 hours of large-scale egocentric interaction data.

AtomEgo → includes → Atom-DH

confidence 92% · We investigate three representative paradigms: joint co-training with domain-specific action heads... (Atom-DH)

AtomEgo → includes → Atom-CL

confidence 92% · progressive ego-to-robot transfer through embodiment alignment... (Atom-CL)

AtomEgo → includes → Atom-WAM

confidence 92% · joint video–action modeling... (Atom-WAM)

AtomEgo → combines → Ego Data

confidence 90% · We use EgoVerse... as the source of approximately 1,079.5 hours of large-scale egocentric interaction data.

AtomEgo → combines → Robot Data

confidence 90% · We combine open-source robot demonstrations from AgiBot World Beta, DROID, RoboCOIN, and RoboMIND.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Embodied foundation models are constrained by the limited scale and diversity of robot demonstrations, motivating the use of large-scale egocentric human interaction data. However, how to effectively incorporate such data into embodied-model pre-training remains unclear because of substantial embodiment and action-space gaps between humans and robots. We present AtomEgo, a systematic study of ego--robot co-training supported by a curated corpus of approximately 2,659 hours and a scalable data processing pipeline. Across vision--language--action and world--action model architectures, we investigate three representative paradigms: joint co-training with domain-specific action heads, progressive ego-to-robot transfer through embodiment alignment, and joint video--action modeling. We evaluate these paradigms through multi-task real-robot experiments and language-conditioned cross-embodiment representation analysis. Our results reveal a simple principle: Data Scale * Alignment Quality --> Capability Gain; egocentric data can improve generalization, but their value depends on how effectively they are aligned and utilized. This principle can provide practical guidance for scalable ego--robot pre-training.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

56,229 characters extracted from source content.

Expand or collapse full text

Agentic Intelligence Lab | HKU AtomEgo: Exploring Ego–Robot Integration for Embodied Foundation Model Pretraining Di Wu 1,4 Dongchen Zheng 1,3 Junhe Sheng 1 Zhongxing Wei 1 Songxin Zhang 2 Zejian Xie 2 Xiaoquan Sun 5 Junyang Zheng 1 Zhuoyang Song 2 Jiaxing Zhang 2 Jiayu Chen 1,3 1 INFIFORCE 2 Lionrock Artificial Intelligence Laboratory 3 HKU 4 Tongji University 5 Huazhong University of Science and Technology Abstract Embodied foundation models are constrained by the limited scale and diversity of robot demonstrations, motivating the use of large-scale egocentric human interaction data. However, how to effectively incorpo- rate such data into embodied-model pre-training remains unclear because of substantial embodiment and action-space gaps between humans and robots. We present AtomEgo, a systematic study of ego–robot co-training supported by a curated corpus of approximately 2,659 hours and a scalable data processing pipeline. Across vision–language–action and world–action model architectures, we investigate three representative paradigms: joint co-training with domain-specific action heads, progressive ego-to-robot transfer through embodiment alignment, and joint video–action modeling. We evaluate these paradigms through multi-task real-robot experiments and language-conditioned cross-embodiment representation analysis. Our results reveal a simple principle: Data Scale×Alignment Quality→Capability Gain; egocentric data can improve generalization, but their value depends on how effectively they are aligned and utilized. This principle can provide practical guidance for scalable ego–robot pre-training. Code: https://github.com/Agentic-Intelligence-Lab/Atom-0 Correspondence: Jiayu Chen (jiayuc@hku.hk)Sep, 2026 1 Introduction Recent vision–language–action (VLA) models combine pretrained vision–language representations with action generation, enabling instruction-conditioned manipulation and transfer across tasks and embodiments [9,14]. World– action models (WAMs) [6,37] further learn predictive physical dynamics from visual observations, offering a complementary route toward general-purpose embodied intelligence. These developments suggest that embodied foundation models may benefit from the same scaling principles that have driven progress in language and multimodal modeling [10]. However, scaling these models requires large and diverse datasets, while robot demonstrations are costly to collect, constrained by specific hardware, and limited in task, environment, object, and interaction diversity [19, 35]. Consequently, data has become a central bottleneck to scaling embodied foundation models. Egocentric human interaction data provide a compelling source of scalable supervision. First-person videos capture rich object interactions in diverse, naturally occurring environments and can be collected with substantially less hardware and operational overhead than robot trajectories [18]. Yet their scale does not translate automatically into useful policy supervision. Most egocentric videos lack executable action labels, while inferred hand motions differ from robot commands in morphology, kinematics, coordinate frames, and visual appearance. Existing studies have demonstrated several promising mechanisms for bridging this gap, including action retargeting, embodiment alignment, and latent-action or world-model objectives [11,26,27,38]. Nevertheless, these methods have largely been studied in isolation, and many operate in post-training rather than large-scale pre-training. This naturally raises the question: Technical Paper © 2026 Agentic Intelligence Lab arXiv:2609.21461v1 [cs.RO] 18 Sep 2026 How can egocentric data be incorporated simply and effectively into ego–robot co-training during pre-training to improve model performance? To answer this question, we present AtomEgo, a systematic study of ego-robot co-training for embodied foundation models. Rather than committing to a single mechanism for exploiting human data, we study both VLA and WAM architectures and investigate three complementary pre-training paradigms: joint co-training with domain-specific action heads (Section 4.2), progressive ego-to-robot transfer through embodiment alignment (Section 4.3), and joint video–action modeling for cross-embodiment transfer (Section 4.4). Together, these paradigms test whether simply isolating domain-specific supervision is sufficient to bridge the human–robot gap, or whether effective transfer requires a stronger mechanism through explicit embodiment alignment or a unified visual-prediction objective. To support this study, we curate an ego-robot corpus at the scale of 3,033 hours and develop a scalable data processing pipeline. We then conduct systematic pre-training experiments with matched data and computational settings, integrating each ego-data paradigm into a unified embodied foundation model. Finally, we build an evaluation pipeline that measures policy performance and generalization across tasks and conditions, enabling objective comparison among the three paradigms and yielding practical guidance for future ego-robot pre-training. This guidance reveals a key principle: scaling egocentric data alone is insufficient; high-quality alignment is equally essential for converting human experience into robot capability. Our main contributions are summarized as follows: •We construct a curated 3,033-hour ego–robot pre-training corpus with a scalable processing and quality- control pipeline for heterogeneous interaction data. • We formulate and implement three paradigms for incorporating egocentric data into embodied foundation- model pre-training, spanning domain isolation, explicit embodiment alignment, and joint video–action modeling. •We conduct controlled pre-training and comprehensive evaluation under a unified data and experimental framework, enabling a systematic comparison of alternative ego-robot co-training strategies. •We derive empirical takeaways for aligning and utilizing egocentric data and reveal a simple principle: Data Scale× Alignment Quality→ Capability Gain, linking capability gains to both scale and alignment. 2 Related Work 2.1 Embodied Foundation Models Vision–language–action (VLA) models extend pretrained vision–language models with action generation, enabling instruction-conditioned policies trained on large, heterogeneous robot corpora [4,5,8,14,25]. Recent work advances cross-embodiment transfer, long-horizon execution, temporal memory, and tactile grounding [7,9,15,17,22,23,30]. In parallel, world–action models (WAMs) jointly predict actions and future visual states [6], using video dynamics to complement VLA semantic priors [1,3,24,34,37]. However, embodied foundation models remain limited by scarce, costly robot demonstrations [19,35,39], and naively mixing incompatible embodiments and action spaces can cause negative transfer [26,36]. We therefore study how to align large-scale egocentric experience with robot data for effective embodied foundation-model pre-training. 2.2 Leveraging Egocentric Data for Embodied Pre-Training Egocentric data are video recordings collected from a human wearer’s viewpoint that capture first-person human interactions at scale [18,31], but their human-centric motions are not directly executable by robots [11,21]. Prior work bridges human observations and motions to robot representations and control, demonstrating the value of egocentric data across VLA and WAM formulations [11,13,16,17,32,33,38]. However, how to jointly use ego and robot data during VLA/WAM pre-training stage remains a challenge. For example, Ego2Robot [27] converts human videos into robot-format visual–action trajectories, whereas JoyAI-RA 0.5 [26] aligns heterogeneous data through implicit latent actions and explicit canonical actions. However, existing studies primarily validate individual design choices, rather than systematically comparing different ego–robot pre-training strategies under controlled settings [16, 26, 27]. Our work systematically examines multiple co-training paradigms within a unified framework. 2 Open-Source Robot DataOpen-Source Ego Data In-House Human–Robot Data (Task-matched) HumanRobotHumanRobotHumanRobot Figure 1: Qualitative overview of the three data pools used in AtomEgo pre-training: open-source robot data, open-source egocentric data, and in-house task-matched human–robot alignment data. 3 Data Preprocessing and Curation This section describes our data recipe, data processing, and the quality-control procedure. Starting from approximately 3,033 hours of robot and egocentric data, our quality filtering pipeline retains about 2,659 hours of trajectories. 3.1 Data Recipe Our final expanded co-training recipe contains 334,054 training episodes, corresponding to approximately 2,659 hours. We organize the corpus into three groups according to their roles in ego-robot co-training; Table 1 reports the contribution of each source. Robot Data. We combine open-source robot demonstrations from AgiBot World Beta [2], DROID [12], Robo- COIN [29], and RoboMIND [28]. We also collect in-house demonstrations using the Piper bimanual robot. Together, these datasets span single-arm, bimanual, mobile, and humanoid robots, providing broad cross-embodiment action supervision over approximately 1,560 hours. Ego Data. We use EgoVerse [21] as the source of approximately 1,079.5 hours of large-scale egocentric interaction data. Its first-person human and robot trajectories provide diverse manipulation experience beyond the environments and tasks covered by the robot datasets. Alignment Data. Beyond the in-house robot demonstrations, we collect egocentric human demonstrations using wearable devices, and record robot executions of the same task set. We refer to these human and robot trajectories as alignment data: although they differ in embodiment and action space, they share task semantics and interaction objectives, providing explicit supervision for bridging the human–robot embodiment gap. This collection contains 1,296 episodes, corresponding to approximately 20 hours. 3.2 Data quality control Inspired by Qwen-Manip [36], we apply three complementary state–action checks to identify corrupted or inconsistent episodes before training. Applying this quality-control procedure to the original 3,033 hours of data yields the final 2,659-hour training recipe. 3 Table 1: Composition of the expanded co-training recipe. Proportions are calculated by training episode count. GroupSourceEpisodesHoursProportion Robot data Piper (in-house)5,82925.51.745% AgiBot World Beta [2]21,837314.06.537% DROID [12]64,124343.019.196% RoboCOIN [29]93,352655.727.945% RoboMIND [28]83,152221.724.892% Ego dataEgoVerse [21]64,4641,079.519.297% Alignment dataIn-house aligned ego–robot data1,296 ∼20.00.388% Total334,054 ∼2,659.5100% Table 2: Semantic layout of the unified 80-dimensional state–action space. Indices are zero-based and end-exclusive. RangeWidthPhysical meaningAction semantics 0:77Left arm jointsRelative 7:136Left end-effector position and Euler rotationAbsolute 13:163ReservedMasked 161Left gripperAbsolute 17:2912Left hand jointsAbsolute 29:367Right arm jointsRelative 36:426Right end-effector position and Euler rotationAbsolute 42:453ReservedMasked 451Right gripperAbsolute 46:5812Right hand jointsAbsolute 58:7012Left and right leg jointsAbsolute 70:744Head and waist jointsAbsolute 74:751Other body jointAbsolute 75:805ReservedMasked Sudden-change detection (S1). We smooth each state and action dimension and detect abrupt changes using the residual, acceleration, and jerk relative to robust split-level thresholds. A frame is flagged only when a large residual coincides with abnormal acceleration or jerk, reducing false positives from normal rapid motion. State–action trend alignment (S2). For physically comparable state–action dimensions, we measure temporal lag through cross-correlation and evaluate whether their motion directions agree. This check identifies mismatched trajectories, timing offsets, dropped actions, and cases in which the observed state incorrectly leads the command. Extreme-value filtering (S3). We estimate per-dimension 1st and 99th percentiles over each dataset and flag values outside an expanded quantile interval, as well as any NaN or infinite values. This removes severe long-tail outliers that could distort normalization and destabilize training. 3.3 Data processing Format unification. We convert each filtered source dataset into a validated episode-level RLDS representation. Distinct embodiment, state-action space, and camera configurations are retained as separate builders, producing 45 builders in total. At loading time, all builders are mapped to a common model interface, with unavailable camera views replaced by masked placeholders. Unified state–action space. The shared 80-dimensional space is organized by physical semantics rather than by simply padding each native vector. The principal slots are listed in Table 2. Each dataset defines a mapping from its native state and action fields to these slots. Dimensions not used by a sample are set to zero and excluded from training through the action mask. Single-arm embodiments are consistently mapped to the right arm and gripper slots. For egocentric demonstrations, we use a 12-dimensional bimanual action representation in which each hand is encoded as an absolute end-effector pose comprising 3D position and Euler angles. For arm slots configured as relative joint actions, the target at horizon stephis computed against the current observation state, ∆a t,h,j = a t+h,j − s t,j .(1) 4 QKV FFN QKV FFN + adaRMS Shared Multi-Head Attention 18 Layers️ 18 Layers️ VLM (gemma-2B) Action Expert (gemma-300M) Human DataRobot Data SigLIP (1) Vis️ion(2) Language Text Tokenizer (3) Nois️ed Actions️ action_hidden [B，50，1024] times️tep t Prompt_prefix、 Task、 Discretized state 퐿 푓푙표푤 푟표푏표푡 +퐿 푓푙표푤 푒푔표 Total Los️s️ Robot Head Projection MLP Ego Head Projection MLP Robot Data Samples Ego Data Samples Figure 2: Overview of joint ego–robot co-training with domain-specific action heads (Atom-DH). See Section 4.2 for details. Other valid slots, including gripper commands, dexterous-hand joints, body joints, and end-effector poses, retain their absolute semantics. For every observation at time t, we construct a 50-step target action chunk. Dataset-specific normalization. As embodiments differ in physical ranges and control units, we compute statistics independently for each dataset after applying the same dimension mapping, 50-step chunk construction, and relative- action conversion. For every valid state and action dimension, we estimate the 1st and 99th percentiles and apply ̃x = 2 x− q 0.01 q 0.99 − q 0.01 + 10 −6 − 1,(2) During training, these dataset-specific statistics place heterogeneous state and action values on a common numerical scale, enabling stable joint optimization across embodiments. 4 Exploring Ego-Robot Co-Training Paradigms Rather than committing to a single recipe for leveraging egocentric data, we investigate three complementary co- training paradigms. First, direct utilization (Section 4.2) incorporates ego data into joint training while separate action heads isolate domain-specific control outputs. Second, explicit alignment (Section 4.3) uses a progressive curriculum based on aligned demonstrations to transfer egocentric knowledge to robot control. Third, world modeling (Section 4.4) exploits future-state prediction, an objective naturally suited to the visual interaction structure of egocentric data. Evaluating these paradigms under a shared framework enables a systematic comparison of alternative mechanisms for ego-data pre-training. 4.1 Main Architecture AtomEgo investigates ego–robot co-training across both vision–language–action (VLA; Sections 4.2–4.3) and world–action model (WAM; Section 4.4) architectures. Model instantiation. The two VLA paradigms adopt theπ 0.5 policy [8] architecture and are initialized from the base PaliGemma VLM weights rather than a pretrainedπ 0.5 policy checkpoint. The WAM paradigm instead retains the Fast-WAM architecture, as detailed in Section 4.4. Although their backbones and learning objectives differ, all paradigms use the same processed ego–robot corpus and standardized control interface. Model inputs and outputs. LetO t denote the current visual observation, organized into one base-view and two wrist- view slots; letℓdenote the task instruction,μthe action-representation metadata, ands t the unified 80-dimensional robot state. At the policy interface, the two model families are expressed as ˆ A VLA = f VLA (O t ,ℓ,μ,s t ), ˆ A WAM = f WAM (O t ,ℓ),(3) Both outputs are normalized action chunks in R B×50×80 , representing 50 future control steps in the action space. 5 (a) VLA policy architecture (b) Progressive ego-to-robot curriculum Vision input Languageinput Action hidden [B, 50, 1024] Flow-matching loss 1 2 3 HUMAN DATA Ego pre-training Learn interaction priors from scalable egocentric video HUMANROBOT Robot–ego alignment Bridge embodiment gaps with task- aligned pairs ROBOT DATA Robot pre-training Specialize the aligned policy for robot control VLA checkpoint 1 VLA checkpoint 2 FINAL VLA CHECKPOINT SigLIP Text Tokenizer Noised Actions QKV FFN QKV FFN + adaRMS Shared Multi-Head Attention 18 Layers 18 Layers VLM (gemma-2B) Action Expert (gemma-300M) Figure 3: Overview of progressive ego-to-robot transfer through curriculum-based embodiment alignment (Atom- CL). See Section 4.3 for details. Table 3: Three-stage pre-training curriculum for progressive ego-to-robot transfer. Stage Objective and training dataInitializationUpdates Trainable components 1Egocentric prior learning on Ego dataBase PaliGemma VLM100,000 Full model 2Embodiment alignment on task-aligned human and robot data Stage 1 checkpoint50,000 Vision encoder and action expert 3Robot pre-training on robot datasetsStage 2 checkpoint97,728 Full model 4.2 Atom-DH: Joint Co-Training with Domain-Specific Action Heads Egocentric and robot trajectories provide complementary interaction experience, but their control targets follow distinct embodiment- and domain-specific distributions. A fully shared policy must therefore fit heterogeneous action mappings through the same output projection, which can introduce conflicting supervision at the control interface. We address this issue with a minimal architectural modification: the perception, language, and action representations are shared across domains, while the final action projection is separated into ego-specific and robot-specific heads. Leth i ∈ R 50×D denote the action representation produced by the shared backbone for samplei, and letd i ∈ ego, robot denote its domain. Two independently parameterized projections decode the shared representation, ˆv i = W ego h i , d i = ego, W robot h i , d i = robot. (4) Each sample supervises only its corresponding head, while gradients from both domains continue to update the shared backbone. The ego head is used only to provide auxiliary supervision during co-training; downstream robot control is decoded exclusively through the robot head. We additionally evaluate an optimal-transport (OT) regularizer on the aligned human–robot subset. It derives temporal correspondences from end-effector trajectories and encourages matched action latents to remain close, while a lightweight embodiment prompt distinguishes human and robot inputs. Implementation details and experimental results for this variant are provided in Appendix C. Overall, the dual-head model provides a minimal baseline for exploiting egocentric data through shared representation learning while retaining domain-specific action decoding. It also serves as a controlled reference for the more structured transfer paradigms introduced below. 4.3 Atom-CL: Progressive Ego-to-Robot Transfer via Embodiment Alignment Directly mixing egocentric and robot supervision requires the model to bridge a substantial embodiment gap within a single training stage. Prior work has shown that introducing an intermediate alignment stage can improve the utilization of egocentric data [38]; we further investigate this strategy at pre-training scale. We formulate ego-to-robot transfer as a curriculum with three successive data distributions: The model first learns general interaction priors from large-scale egocentric trajectories, then adapts on a smaller set of task-aligned human and robot demonstrations, 6 Robotic Data & Ego Data Gaussian action noise Action DiT a₁a₂ ⋯ aₕ Action chunk Wan2.2-TI2V-5B Self-Attention Cross-Attention FFN Cross-Attention FFN Self-Attention VAE Encoder VAE Encoder Pick up the yellow bowl on the green bowl Task Instruction Input Figure 4: Overview of joint world–action modeling for ego–robot co-training (Atom-WAM). See Section 4.4 for details. and finally trains on robot pre-training data. The intermediate alignment stage provides a gradual transition from human interaction patterns to robot behavior, enabling the three-stage curriculum to fully leverage the benefits of egocentric data. Aligned human–robot supervision. The alignment set contains human and robot demonstrations that share task semantics and interaction objectives, while differing in morphology, appearance, and native control space. Within this stage, we convert both domains to a common end-effector representation. A tracked human hand pose is treated as a canonical end effector, while robot joint trajectories are converted to Cartesian end-effector poses using embodiment-specific forward kinematics when Cartesian poses are not directly available. Both are expressed in their fixed egocentric camera frame and then transformed into motion relative to the current end-effector frame. Let C T E t denote the end-effector pose at timetin camera frameC. For prediction horizonτ h , the aligned target is the relative transform from the current pose to the future pose, ∆T t,h = C T E t −1 C T E t+τ h .(5) This construction aligns ego and robot trajectories at the representation level, allowing the task-matched dataset to provide an explicit transition from egocentric pre-training to robot learning. This common representation is used specifically for the alignment stage; the subsequent robot-only stage retains each robot dataset’s original joint-space control targets. Consequently, only the small alignment set requires additional conversion, avoiding costly reconstruction of the full pre-training corpus. Progressive training. The pre-training curriculum consists of three sequential stages. Egocentric pre-training first acquires broad interaction priors, alignment fine-tuning then bridges human and robot motion under the shared end-effector representation, and robot pre-training finally adapts the aligned model to heterogeneous executable control. Each stage initializes from a checkpoint of the preceding stage. Table 3 summarizes this schedule. Overall, this design offers two advantages: progressive transfer of egocentric priors and lower processing cost, since alignment-specific post-processing is required only for the second-stage alignment dataset, not the full corpus. 4.4 Atom-WAM: Joint World-Action Modeling for Cross-Embodiment Transfer The preceding paradigms address the embodiment gap at the action interface, either by separating domain-specific outputs or by explicitly aligning human and robot motions. Our third paradigm instead exploits a signal that is naturally shared across embodiments: the visual evolution of an interaction. Although human hands and robot manipulators have different control spaces, both induce temporally structured changes in objects and scenes. We therefore instantiate this route by fine-tuning Fast-WAM [37] on a mixture of egocentric and robot trajectories, using future-video modeling as a shared channel for cross-embodiment transfer. Fast-WAM initialization. Unlike the two VLA-based routes above, this paradigm initializes from Fast-WAM architecture. Fast-WAM couples a video Diffusion Transformer (DiT) [20] with an Action DiT under a shared- attention Mixture-of-Transformers formulation. The Video DiT models future visual latents, while the Action DiT predicts a continuous action chunk from the current observation and language instruction. During training, clean observation tokens provide a common visual anchor for both branches. A structured attention mask allows both branches to access this anchor while preventing action tokens from attending to noisy future-video latent tokens, thereby avoiding future-information leakage. 7 Task-Matched ID and OOD Evaluation ID settings Rigid-Object ManipulationLong-Horizon T1 · Hang Cup T2 · Place Plate T3 · Place Pot T4 · Press Button T5 · Push Box T6 · Cook Bread T7 · Place Fruits OOD settings Physical Heavier cup Spatial Rack shift Spatial Yaw +60° Appearance Low light Spatial Box rotation Spatial Pot yaw 45° Appearance Extra fruit Figure 5: Task-matched ID and OOD evaluation settings for the seven real-robot tasks. Each OOD setting preserves the task objective while changing one primary spatial, appearance, or physical factor. Joint ego–robot fine-tuning. We train the model on the curated robot and egocentric data described in Section 3. We construct the fine-tuning distribution as D mix = ρD ego + (1− ρ)D robot ,(6) whereρis the effective ego-data sampling probability. Future frames from both domains supervise the video flow-matching objective. Following Fast-WAM, we optimize L joint = E D mix [L action + λ video L video ],(7) whereλ video balances action learning and video co-training. In this way, both domains update the world representation through future-video prediction, while action supervision is restricted to the control dimensions available for each sample. The shared visual-dynamics objective therefore provides the primary transfer channel between human and robot embodiments without requiring their native action spaces to be identical. At deployment, we follow the Fast-WAM inference procedure and omit future-video tokens, denoising, and decoding. The Video DiT processes only the current visual context to produce the latent world representation, which then generates the robot action chunk. 4.5 Training pipeline All three paradigms follow a common pre-training–post-training–evaluation pipeline. Our study focuses on pre- training, where the methods differ in how they incorporate egocentric and robot data. Each pre-training resulting checkpoint is subsequently post-trained on the same in-house multi-task robot dataset using an identical recipe, isolating the effect of the pre-training strategy. All pre-training runs use the same training steps, batch size, and other hyperparameters; each uses 3× 8 NVIDIA B200 GPUs and takes approximately four days. 5 Experiments Having introduced three paradigms for incorporating egocentric data during pre-training, we now evaluate their effectiveness and derive practical insights. Real-robot task performance provides the most direct evaluation; in Section 5.1, we evaluate a suite of manipulation tasks under both in-distribution and out-of-distribution conditions. Beyond real-robot evaluation, Section 5.2 examines the latent representations of egocentric and robot data, providing an offline measure of cross-embodiment representation learning. 8 Table 4: Per-task real-robot results under ID and OOD conditions. VLAWAM BaselineAtom-DHAtom-CLBaselineAtom-WAM TaskIDOODIDOODIDOODIDOODIDOOD Overall (7 tasks)30/7016/7032/7029/7045/7030/7021/7019/7015/7014/70 Success rate42.86% 22.86% 45.71% 41.43% 64.29% 42.86% 30.00% 27.14% 21.43% 20.00% Cook Bread10/100/101/100/102/100/102/101/100/100/10 Hang Cup1/101/100/100/1010/1010/100/100/100/100/10 Place Fruits9/108/1010/109/1010/1010/109/1010/1010/1010/10 Place Plate0/100/100/100/101/100/101/100/100/100/10 Place Pot4/102/101/100/102/100/100/100/100/100/10 Press Button1/101/1010/1010/1010/1010/109/108/105/104/10 Push Box5/104/1010/1010/1010/100/100/100/100/100/10 42.9 22.9 32.9 45.7 41.4 43.6 64.3 42.9 53.6 30.0 27.1 28.6 21.4 20.0 20.7 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 IDOODCombined Success Rate (%) VLA BaselineAtom-DHAtom-CLWAM BaselineAtom-WAM Figure 6: Real-robot success rates of the VLA and WAM checkpoints under ID, OOD, and combined evaluation settings. 5.1 Real-Robot Evaluation Experimental setup.We conduct real-world experiments on a dual-arm platform; hardware details are provided in Appendix A.1. We evaluate seven manipulation tasks: Cook Bread,Hang Cup, Place Fruits, Place Plate, Place Pot, Press Button, and Push Box. For each method, we use only its final pre-training checkpoint, obtained after approximately 100,000 updates, for identical joint post-training on all seven tasks; the resulting post-training checkpoint is evaluated under the multi-task setting. For each checkpoint, we evaluate all seven tasks, conducting ten trials per task under the nominal ID condition and ten under the corresponding OOD condition, yielding 140 trials in total. Each OOD variant preserves the task objective while changing one primary spatial, appearance, or physical factor. A rollout is counted as successful only when it reaches the complete predefined task end state. The matched nominal and shifted task settings are summarized in Figure 5. Baselines. Each baseline is pre-trained only on robot demonstrations using the standard objective, without ego- specific mechanisms, then post-trained on the same seven-task robot dataset. In contrast, the Atom variants incorporate both robot and ego data during pre-training through their respective co-training strategies, while following the identical post-training procedure. All methods use the same training steps, learning rate, batch size, and other hyperparameters. Because the VLA and WAM routes use different model architectures, we construct a separate robot-only baseline for each family. The VLA baseline is compared with domain-specific dual-head co-training (Atom-DH) and progressive embodiment alignment (Atom-CL), while the WAM baseline is compared with joint world–action co-training (Atom-WAM). Results.Table 4 reports the aggregate and per-task real-robot results, which are summarized in Figure 6. Within the VLA family, Atom-CL achieves the highest ID and combined success rates of 64.29% and 53.57%, improving over the VLA baseline by 21.43 and 20.71 percentage points, respectively. Atom-DH reaches 43.57% combined success. 9 Table 5: Language-conditioned cross-embodiment representation results on seven tasks. Higher raw matched- task similarity indicates stronger consistency, while lower normalized different-task similarity indicates better task separation. CheckpointRaw matched mean↑Normalized different-task mean↓ Atom-DH 0.9996250.592719 Atom-CL0.994481 0.387240 Atom-WAM0.9973570.480727 Figure 7: Row-normalized language-conditioned Ego-to-Robot task similarity for Atom-DH (top left), Atom-CL (top right), and Atom-WAM (bottom). Rows denote Ego tasks and columns denote Robot tasks. Both methods improve OOD success over the VLA baseline’s 22.86%, reaching 41.43% for Atom-DH and 42.86% for Atom-CL. In contrast, Atom-WAM surprisingly underperforms its WAM baseline across both conditions: ID success decreases from 30.00% to 21.43%, OOD success decreases from 27.14% to 20.00%, and combined success decreases from 28.57% to 20.71%. This counterintuitive result suggests that visual prediction alone may capture domain-specific dynamics without sufficiently grounding them in executable robot actions. Together, the results indicate that egocentric data are not inherently beneficial and that their value depends strongly on the integration strategy, with progressive embodiment alignment providing the most consistent gains under this evaluation. While this discussion focuses on the direct experimental results, we will provide a higher-level synthesis and overall takeaways in Section 5.3. 5.2 Language-Conditioned Cross-Embodiment Representation Evaluation To further examine cross-embodiment representation learning, we evaluate whether ego and robot videos of the same task are encoded similarly while remaining distinguishable from other tasks. We use seven task-matched sets from our self-collected alignment dataset, which contains egocentric and real-robot demonstrations of the same tasks. Protocol and metrics.We evaluate three post-trained models—Atom-DH, Atom-CL, and Atom-WAM—on 62 ego and 62 robot videos. For each video, we uniformly sample 32 front-view frames, pair them with the task instruction, and aggregate the resulting language-conditioned frame features into a clip representationzusing temporal-change weighting. For ego task p and robot task q, we compute S pq = 1 |E p ||R q | X i∈E p X j∈R q cos(z i ,z j ),(8) whereE p andR q denote the corresponding ego and robot video sets. The resulting7×7matrix is visualized in Figure 7 after row-wise Min–Max normalization. Table 5 reports the mean raw diagonal similarity for matched tasks and the mean normalized off-diagonal similarity for different tasks. Together, these metrics measure cross-embodiment task consistency and task discrimination. Results. Atom-DH achieves the highest matched-task similarity but also the highest normalized different-task similarity, indicating strong cross-embodiment consistency alongside relatively weak task separation. Atom-CL 10 exhibits slightly lower matched-task similarity but substantially better task separation, while Atom-WAM lies between the two VLA methods on both metrics. This intermediate position suggests that joint visual prediction captures some shared cross-embodiment structure but does not produce the strongest task selectivity. More broadly, the representation ordering does not directly match the real-robot performance ordering. Useful cross-embodiment alignment must therefore preserve action-relevant task structure rather than simply bringing ego and robot representations closer. 5.3 Result Summary and Takeaways We now return to our initial question: How can we use egocentric data effectively during pre-training? Our experiments yield the following takeaways. Takeaway 1: Effective alignment requires more than direct data mixing. Simply combining egocentric and robot data during pre-training does not reliably improve robot control. Despite providing greater scale and interaction diversity, the substantial embodiment and action-space gaps can introduce conflicting supervision. Consistent with this observation, Atom-DH, which relies only on domain-specific action-head separation, improves ID success only marginally, from 42.86% to 45.71%. Thus, effective transfer requires structured cross-embodiment alignment rather than direct data mixing. Takeaway 2: Ego data particularly benefit OOD generalization. Both ego-pretrained VLA variants improve OOD success over the VLA baseline’s 22.86%, reaching 41.43% for Atom-DH and 42.86% for Atom-CL. These gains suggest that diverse human interactions can provide transferable priors that are particularly valuable under distribution shift, enabling generalization beyond the robot training distribution. Takeaway 3: WAM-based transfer requires stronger alignment mechanisms. Direct ego–robot co-training with Fast-WAM [37] reduces combined success from 28.57% to 20.71%. Although both domains share the same future-video supervision objective, this common visual objective alone does not bridge the embodiment gap. Effective ego utilization in WAMs therefore requires finer-grained cross-embodiment alignment rather than naive data mixing, or a stronger, carefully designed architecture rather than simply reusing Fast-WAM. Overall, our findings suggest a simple principle: Data Scale×Alignment Quality→Capability Gain; the value of egocentric data depends not only on its scale, but also on how effectively it is aligned and utilized. 6 Limitation and future work While this study provides a controlled comparison of representative ego–robot co-training paradigms, several directions remain open. First, due to resource constraints, the scale of our pre-training corpus remains limited. Second, our scope prioritizes a systematic evaluation of alternative paradigms rather than specialized algorithmic development for any single route. In the future, we plan to further investigate the most effective approach and scale its training to substantially larger datasets toward a more capable embodied foundation model. 7 Conclusion We presented AtomEgo, a unified framework for studying scalable ego–robot co-training for embodied foundation models. Using a curated corpus of robot and egocentric interaction data, we compared three complementary paradigms. Our results show that egocentric data can improve policy generalization, but their effectiveness depends critically on the cross-embodiment alignment strategy. These findings provide a practical foundation for scaling embodied models beyond robot-only supervision. 11 Acknowledgements We gratefully acknowledge the computational support provided by the Lionrock Artificial Intelligence Laboratory at the China Merchants Research Institute of Advanced Technology. Author Contributions Data collection and processing: Di Wu. Algorithm and training: Junhe Sheng, Zhongxing Wei, Xiaoquan Sun, Junyang Zheng, Di Wu, Songxin Zhang and Zejian Xie. Evaluation: Dongchen Zheng. Writing: Di Wu and Dongchen Zheng. Advisor: Jiayu Chen, Jiaxing Zhang, Zhuoyang Song. References [1] Niket Agarwal, Arslan Ali, Jon Allen, Martin Antolini, Adeline Aubame, Alisson Azzolini, Junjie Bai, Maciej Bala, Yogesh Balaji, Josh Bapst, et al. Cosmos 3: Omnimodal world models for physical ai. arXiv preprint arXiv:2606.02800, 2026. [2]AgiBot-World-Contributors, Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xu Huang, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.06669, 2025. [3]Hongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang, Shuhe Huang, Haitian Liu, Ruowen Zhao, Yao Feng, Chendong Xiang, Yinze Rong, et al. Motus: A unified latent action world model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 35101–35113, 2026. [4]Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. pi0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024. [5]Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023. [6]Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023. [7] Jialei Huang, Shuo Wang, Fanqi Lin, Yihang Hu, Chuan Wen, and Yang Gao. Tactile-VLA: Unlocking vision-language-action model’s physical knowledge for tactile generalization. arXiv preprint arXiv:2507.09160, 2025. [8]Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. pi05: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025. [9] Physical Intelligence, Bo Ai, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Greg Balke, Kevin Black, George Bokinsky, Shihao Cao, Thomas Charbonnier, et al. pi0.7: a steerable generalist robotic foundation model with emergent capabilities. arXiv preprint arXiv:2604.15483, 2026. [10] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. [11]Simar Kareer, Dhruv Patel, Ryan Punamiya, Pranay Mathur, Shuo Cheng, Chen Wang, Judy Hoffman, and Danfei Xu. Egomimic: Scaling imitation learning via egocentric video. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 13226–13233. IEEE, 2025. 12 [12]Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945, 2024. [13] Ji Woong Kim, Ke Wang, Zipeng Fu, Sirui Chen, Jeff Lai, Chelsea Finn, et al. Ego-pi: Vla fine-tuning for ego-centric human and robot data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1515–1524, 2026. [14]Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024. [15] Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, and Juanzi Li. Rm-bench: Benchmarking reward models of language models with subtlety and style. In International Conference on Learning Representations, volume 2025, pages 44323–44355, 2025. [16]Hao Luo, Yicheng Feng, Wanpeng Zhang, Sipeng Zheng, Ye Wang, Haoqi Yuan, Jiazheng Liu, Chaoyi Xu, Qin Jin, and Zongqing Lu. Being-h0: vision-language-action pretraining from large-scale human videos. arXiv preprint arXiv:2507.15597, 2025. [17]Hao Luo, Wanpeng Zhang, Yicheng Feng, Sipeng Zheng, Haiweng Xu, Chaoyi Xu, Ziheng Xi, Yuhui Fu, and Zongqing Lu. Being-h0. 7: A latent world-action model from egocentric videos. arXiv preprint arXiv:2605.00078, 2026. [18]Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. arXiv preprint arXiv:2203.12601, 2022. [19] Abby O’Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892–6903. IEEE, 2024. [20] William Peebles and Saining Xie. Scalable diffusion models with transformers. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4172–4182. IEEE, 2023. [21]Ryan Punamiya, Simar Kareer, Zeyi Liu, Josh Citron, Ri-Zhao Qiu, Xiongyi Cai, Alexey Gavryushin, Jiaqi Chen, Davide Liconti, Lawrence Y Zhu, et al. Egoverse: An egocentric human dataset for robot learning from around the world. arXiv preprint arXiv:2604.07607, 2026. [22]Wenxuan Song, Jiayi Chen, Xiaoquan Sun, Huashuo Lei, Yikai Qin, Wei Zhao, Pengxiang Ding, Han Zhao, Tongxin Wang, Pengxu Hou, et al. Rethinking the practicality of vision-language-action model: A comprehen- sive benchmark and an improved baseline. arXiv preprint arXiv:2602.22663, 2026. [23]Xiaoquan Sun, Zetian Xu, Chen Cao, Zonghe Liu, Yihan Sun, Jingrui Pang, Ruijian Zhang, Zhen Yang, Kang Pang, Dingxin He, et al. Atomvla: Scalable post-training for robotic manipulation via predictive latent world models. arXiv preprint arXiv:2603.08519, 2026. [24]Xiaoquan Sun, Ruijian Zhang, Chen Cao, Yihan Sun, Jiahui Chen, Zetian Xu, Bo Chen, Haijier Chen, Zhen Yang, Jiarun Zhu, et al. Himem-wam: Hierarchical memory-gated world action models for robotic manipulation. arXiv preprint arXiv:2606.10363, 2026. [25]Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213, 2024. [26] RA Team et al. Joyai-ra 0.5: Scaling robot manipulation learning via dual action alignment. arXiv preprint arXiv:2608.05674, 2026. [27]Ye Wang, Pei Lin, Xiong-Hui Chen, Haoqi Yuan, Zhixuan Liang, Yiyang Huang, Anzhe Chen, Zixing Lei, Jie Zhang, Tao Zhang, et al. Ego2robot: Scalable robot data synthesis from egocentric human data. arXiv preprint arXiv:2608.02580, 2026. [28]Kun Wu, Chengkai Hou, Jiaming Liu, Zhengping Che, Xiaozhu Ju, Zhuqin Yang, Meng Li, Yinuo Zhao, Zhiyuan Xu, Guang Yang, et al. Robomind: Benchmark on multi-embodiment intelligence normative data for robot manipulation. arXiv preprint arXiv:2412.13877, 2024. 13 [29]Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, et al. Robocoin: An open-sourced bimanual robotic data collection for integrated manipulation. arXiv preprint arXiv:2511.17441, 2025. [30]Wei Wu, Fangjing Wang, Fan Lu, He Sun, Shi Liu, Yunnan Wang, Yibin Yan, Yong Wang, Shuailei Ma, Xinyang Wang, et al. From foundation to application: Improving vla models in practice. arXiv preprint arXiv:2607.06403, 2026. [31]Tete Xiao, Ilija Radosavovic, Trevor Darrell, and Jitendra Malik. Masked visual pre-training for motor control. arXiv preprint arXiv:2203.06173, 2022. [32]Ruihan Yang, Qinxi Yu, Yecheng Wu, Rui Yan, Borui Li, An-Chieh Cheng, Xueyan Zou, Yunhao Fang, Xuxin Cheng, Ri-Zhao Qiu, et al. Egovla: Learning vision-language-action models from egocentric human videos. arXiv preprint arXiv:2507.12440, 2025. [33] Seonghyeon Ye, Joel Jang, Byeongguk Jeon, Se June Joo, Jianwei Yang, Baolin Peng, Ajay Mandlekar, Reuben Tan, Yu-Wei Chao, Bill Yuchen Lin, et al. Latent action pretraining from videos. In International Conference on Learning Representations, volume 2025, pages 28213–28239, 2025. [34]Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, et al. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922, 2026. [35]Yifan Ye, Yankai Fu, Yaoxu Lv, Bohan Hou, Jun Cen, Lingdong Kong, Duo Zheng, Tianxing Chen, Jiaming Liu, Ziang Cao, et al. Data pyramid for embodied manipulation. arXiv preprint arXiv:2607.24744, 2026. [36]Haoqi Yuan, Zhixuan Liang, Anzhe Chen, Ye Wang, Haoyang Li, Pei Lin, Yiyang Huang, Zixing Lei, Tong Zhang, Jiazhao Zhang, et al. Qwen-robotmanip technical report: Alignment unlocks scale for robotic manipula- tion foundation models. arXiv preprint arXiv:2606.17846, 2026. [37] Tianyuan Yuan, Zibin Dong, Yicheng Liu, and Hang Zhao. Fast-wam: Do world action models need test-time future imagination? arXiv preprint arXiv:2603.16666, 2026. [38] Ruijie Zheng, Dantong Niu, Yuqi Xie, Jing Wang, Mengda Xu, Yunfan Jiang, Fernando Castañeda, Fengyuan Hu, You Liang Tan, Letian Fu, et al. Egoscale: Scaling dexterous manipulation with diverse egocentric human data. arXiv preprint arXiv:2602.16710, 2026. [39]Yifan Zhong, Fengshuo Bai, Shaofei Cai, Xuchuan Huang, Zhang Chen, Xiaowei Zhang, Yuanfei Wang, Shaoyang Guo, Tianrui Guan, Ka Nam Lui, et al. A survey on vision-language-action models: An action tokenization perspective. arXiv preprint arXiv:2507.01925, 2025. 14 Appendix Contents A Evaluation Details15 A.1 Hardware Platform15 A.2 Cross-Embodiment Evaluation Instructions15 A.3 Real-Robot Task Definitions15 B Training Hyperparameters16 C Optimal-Transport Alignment Details16 A Evaluation Details A.1 Hardware Platform All real-robot experiments use a dual-arm platform comprising two AgileX Piper 6-DoF arms. The system is equipped with three Intel RealSenseD435i RGB cameras, including two wrist cameras, and one headcamera. Figure 8 shows the hardware setup used throughout the seven-task evaluation. Wrist Camera Front Camera 6-DoF Arms 1-DoF Gripper Objects Workspace Hardware Platform Figure 8: Hardware platform used for real-robot evaluation. A.2 Cross-Embodiment Evaluation Instructions The cross-embodiment representation evaluation uses the following seven fixed task instructions in the same order as the evaluation matrix. The Ego and Robot videos associated with the same task receive exactly the same instruction. 1. pink_mug_to_green_bowl: “Pick up the pink mug and place it in the green bowl.” 2. water_bottle_to_basket: “Pick up the water bottle and place it in the basket.” 3. red_cube_into_blue_tray: “Pick up the red cube and place it in the blue tray.” 4. fold_towel : “Use both hands to grasp the two designated corners along the lower edge of the towel and fold the towel once from bottom to top.” 5. close_box_lid: “Use both hands to grasp the two sides of the box lid and close the lid.” 6. insert_block: “Pick up the wooden block and insert it into the hole of the white foam block.” 7. push_cube_with_stick : “Pick up the wooden stick, then use it to push the red cube completely into the white target area.” A.3 Real-Robot Task Definitions The ID configurations follow the nominal setups used in our seven-task real-robot evaluation. Each paired OOD condition preserves the task objective while perturbing one primary factor, allowing the resulting performance change to be associated with a specific type of distribution shift. Task initializations, perturbation types and execution timing, maximum rollout durations, and complete-task success criteria are fixed before evaluation and shared across checkpoints. Table 6 lists the seven evaluated task configurations and success criteria. 15 Table 6: Seven-task real-robot evaluation protocol. Each OOD condition changes one primary factor while holding other task-relevant variables fixed whenever applicable. Symbolic quantities denote protocol parameters fixed before execution, including illumination, mass, displacement, and stability duration. TaskID setupOOD typeOOD perturbation OOD specification and controls Success criterion Cook BreadStandard pot and bread poses SpatialRotate potPot yaw = 45 ◦ CCW; pot center and other object poses unchanged Bread is placed in the pot, and the pot is placed on the cloth Hang CupStandard cup mass PhysicalIncrease cup mass Mass m ID → m OOD ; cup appearance and initial pose unchanged Handle is inserted and the cup hangs stably for the predefined duration Place FruitsStandard target set Appearance Add visually similar non-target fruit as a distractor Distractor identity and pose fixed; target identities and initial poses unchanged All three target fruits are placed in the basket while the distractor remains outside Place PlateStandard rack pose SpatialTranslate rackRack displacement ∆x rack fixed; plate pose and rack orientation unchanged Plate is aligned, inserted, released, and remains stable in the rack Place PotStandard pot pose SpatialRotate potPot yaw = 60 ◦ ; pot center and cloth pose unchanged Pot is grasped by the handle and placed on the cloth Press Button Standard illumination Appearance Reduce illumination Illuminance L ID → L OOD ; camera and button poses unchanged Button activation is registered and the end effector reaches the predefined retreat region Push BoxStandard box orientation SpatialRotate box Box yaw= 90 ◦ ; box center and box–target distance unchanged Box reaches the predefined target region Figure 9 provides qualitative task-completion progressions for the same seven-task evaluation suite. B Training Hyperparameters Table 7 summarizes the key settings for robot pre-training and subsequent multi-task post-training. Pre-training uses 34 audited in-house and open-source robot datasets, while post-training adapts the model to two in-house Piper multi-task datasets. Under the reported hardware configuration, pre-training takes approximately four days, while post-training takes about 12 hours. Table 7: Key hyperparameters for robot pre-training and multi-task post-training. HyperparameterRobot pre-trainingMulti-task post-training Training data34 robot datasets; 150.1M source frames; EgoVerse excluded Two in-house Piper multi-task datasets Action representationUnified80Unified80 Hardware 3× 8 NVIDIA B200 GPUs 1× 8 NVIDIA B200 GPUs FSDP devices44 Global batch size (train / validation) 1,536 / 96512 / 96 Training steps97,72820,000 Warmup / decay steps5,000 / 97,7281,000 / 30,000 Peak / decay learning rate 1× 10 −6 / 1× 10 −7 2.5× 10 −5 / 2.5× 10 −6 Evaluation / checkpoint interval1,000 / 10,0001,000 / 5,000 Action MSEDisabledEnabled InitializationBase PaliGemma VLMFinal robot pre-training checkpoint C Optimal-Transport Alignment Details The domain-specific heads in Section 4.2 isolate human and robot action decoding, but they do not explicitly constrain the shared action representation across domains. We therefore examine a training-only optimal-transport 16 Figure 9: Uniformly sampled task-completion progressions for the seven real-robot evaluation tasks. Each row pair shows front and head views at ten time points from a complete task execution. 17 (OT) regularizer that brings human and robot latents closer when their end-effector trajectories describe similar interactions. Data pools and alignment supervision. Because data with meaningful human–robot correspondence constitute only a small fraction of the full corpus, unconstrained sampling would rarely place sufficient paired examples in the same batch. We therefore divide the training mixture into a general co-training pool and an alignment pool, and reserve a fixed batch quota for each. The alignment pool consists exclusively of our in-house aligned data: wearable-camera human demonstrations and robot executions collected for the same task set. The general pool contains all remaining egocentric and robot datasets and preserves the scale and diversity of the complete corpus. Only samples from the alignment pool receive the additional OT supervision, while samples from the general pool retain the original flow-matching objective. Trajectory-aware OT alignment. Human and robot demonstrations of the same interaction may progress at different speeds, making direct frame-by-frame latent matching unreliable. We first align their ground-truth Cartesian trajectories using Soft Dynamic Time Warping (Soft-DTW), a smooth sequence-alignment procedure that permits nonlinear temporal correspondence. This operation is applied only to end-effector positions and is detached from gradient computation. It produces a correspondence weight matrixWthat indicates which portions of the human and robot trajectories should be compared. We construct separate bridges for single-arm and bimanual interactions. The former uses the right end-effector position, while the latter jointly uses the left and right end-effectors. For each bridge, valid human–robot pairs are collected from the global batch. We then use the Sinkhorn algorithm, an entropy-regularized solver for soft optimal transport, to match their action-expert latents with pairwise cost C ij = 1 2 ∥h i − r j ∥ 2 2 W ij ,(9) where h i and r j denote the human and robot latent trajectories, and the overline averages over temporal and feature dimensions. The valid single-arm and bimanual transport costs are summed to obtainL OT . The final training objective is L =L FM + αL OT ,(10) whereαcontrols the strength of latent alignment and flow matching remains the primary action objective. OT is used only during training; the output heads and inference procedure are unchanged, so the variant introduces no deployment-time computation. Experiment result.We compare two Atom-DH variants that differ in whether the OT regularizer is enabled, using the same seven-task real-robot evaluation protocol used in the main paper. Each variant is evaluated with ten ID and ten OOD trials per task. Table 8 reports the resulting success rates. Table 8: Ablation of the training-only OT regularizer on the seven-task real-robot evaluation. Each ID or OOD entry aggregates 70 trials; Combined pools all 140 trials. VariantID successOOD successCombined success Atom-DH (+OT)38.57% (27/70)31.43% (22/70)35.00% (49/140) Atom-DH (w/o OT)44.29% (31/70)42.86% (30/70)43.57% (61/140) Disabling OT improves ID, OOD, and combined success by 5.72, 11.43, and 8.57 percentage points, respectively. Under this protocol, explicit latent transport alignment does not improve downstream control and produces the largest degradation under OOD conditions. We therefore treat OT as an auxiliary training variant rather than a default component of Atom-DH. 18

Instant research discovery