Paper deep dive

Thousand-GPU Large-Scale Training and Optimization Recipe for AI-Native Cloud Embodied Intelligence Infrastructure

Chen Zhou, Haoran Sun, Hedan Yang, Jing Long, Junwu Xiong, Luqiao Wang, Mingxi Luo, Qiming Yang, Shuai Di, Song Wang, Tianyun Zhao, Wanting Xu, Wen Huang, Xiaodong Bai, Xiaomeng Tian, Xiaolong Xiang, Yicheng Gong, Yongjian Guo, Yucheng Guo, Yunxuan Ma, Yu Wei, Zhong Guan, Zhen Sun

Year: 2026Venue: arXiv preprintArea: cs.ROType: PreprintEmbeddings: 61

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%

Last extracted: 3/22/2026, 6:15:47 AM

Summary

The paper presents a cloud-native, thousand-GPU distributed training infrastructure for embodied intelligence, built on the LeRobot framework. It introduces optimizations including variable-length FlashAttention, Data Packing, and FP8 quantization to improve training efficiency for VLA models like GR00T-N1.5 and π-0.5. The system features a triple-level asynchronous architecture (RL-VLA3) and an end-to-end evaluation pipeline, achieving significant speedups and throughput improvements in large-scale robotic model training.

Entities (6)

GR00T-N1.5 · model · 99%LeRobot · framework · 98%π-0.5 · model · 98%FlashAttention · optimization-technique · 97%RL-VLA3 · architecture · 95%JoyBuilder · platform · 92%

Relation Signals (3)

GR00T-N1.5 → trainedon → JoyBuilder

confidence 96% · distributed training framework for embodied intelligence based on the JD Cloud JoyBuilder platform

LeRobot → underpins → JoyBuilder

confidence 95% · built upon the widely adopted LeRobot framework

RL-VLA3 → optimizes → Training Pipeline

confidence 94% · propose and implement a fully asynchronous strategy training pipeline for the first time — RL-VLA3

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Embodied intelligence is a key step towards Artificial General Intelligence (AGI), yet its development faces multiple challenges including data, frameworks, infrastructure, and evaluation systems. To address these issues, we have, for the first time in the industry, launched a cloud-based, thousand-GPU distributed training platform for embodied intelligence, built upon the widely adopted LeRobot framework, and have systematically overcome bottlenecks across the entire pipeline. At the data layer, we have restructured the data pipeline to optimize the flow of embodied training data. In terms of training, for the GR00T-N1.5 model, utilizing thousand-GPU clusters and data at the scale of hundreds of millions, the single-round training time has been reduced from 15 hours to just 22 minutes, achieving a 40-fold speedup. At the model layer, by combining variable-length FlashAttention and Data Packing, we have moved from sample redundancy to sequence integration, resulting in a 188% speed increase; {\pi}-0.5 attention optimization has accelerated training by 165%; and FP8 quantization has delivered a 140% speedup. On the infrastructure side, relying on high-performance storage, a 3.2T RDMA network, and a Ray-driven elastic AI data lake, we have achieved deep synergy among data, storage, communication, and computation. We have also built an end-to-end evaluation system, creating a closed loop from training to simulation to assessment. This framework has already been fully validated on thousand-GPU clusters, laying a crucial technical foundation for the development and application of next-generation autonomous intelligent robots, and is expected to accelerate the arrival of the era of human-machine integration.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

60,841 characters extracted from source content.

Expand or collapse full text

Thousand-GPU Large-Scale Training and Optimization Recipe for AI-Native Cloud Embodied Intelligence Infrastructure 1 AI Infra Team at JDT 2 Tsinghua University 3 Peking University 4 Tianjin University 5 Beihang University 6 University of Science and Technology of China Full author list in Contributions Abstract Embodied intelligence is a key step towards Artificial General Intelligence (AGI), yet its de- velopment faces multiple challenges including data, frameworks, infrastructure, and evaluation systems. To address these issues, we have, for the first time in the industry, launched a cloud- based, thousand-GPU distributed training platform for embodied intelligence, built upon the widely adopted LeRobot framework, and have systematically overcome bottlenecks across the en- tire pipeline. At the data layer, we have restructured the data pipeline to optimize the flow of embodied training data. In terms of training, for the GR00T-N1.5 model, utilizing thousand- GPU clusters and data at the scale of hundreds of millions, the single-round training time has been reduced from 15 hours to just 22 minutes, achieving a 40-fold speedup. At the model layer, by combining variable-length FlashAttention and Data Packing, we have moved from sample re- dundancy to sequence integration, resulting in a 188% speed increase;π 0.5 Attention optimization has accelerated training by 165%; and FP8 quantization has delivered a 140% speedup. On the infrastructure side, relying on high-performance storage, a 3.2T RDMA network, and a Ray-driven elastic AI data lake, we have achieved deep synergy among data, storage, communication, and computation. At the training level, we propose and implement a fully asynchronous strategy training pipeline for the first time — RL-VLA 3 , a triple-level asynchronous architecture cover- ing the entire process from environment interaction and trajectory generation to policy network updates, achieving a maximum throughput increase of 126.67%. We have also built an end-to- end evaluation system, creating a closed loop from training to simulation to assessment. This framework has already been fully validated on thousand-GPU clusters, laying a crucial technical foundation for the development and application of next-generation autonomous intelligent robots, and is expected to accelerate the arrival of the era of human-machine integration. Date:March 13, 2026 Correspondence:Junwu Xiong at xiongjunwu.1@jd.com 1Introduction Designing agents with physical-world action capabilities that reach or surpass human-level is one of the key technical paths toward Artificial General Intelligence (AGI) [1], a direction known as “Embodied AI.” Embodied intelligence is currently experiencing a period of market expansion: Driven by global innovation and market demand, the embodied AI industry is expanding rapidly at an annual growth rate achieving 39% [ 2], with the market size expected to surpass$100billion by 2035 [3]. In the field of embodied intelligence, traditional dynamics-based embodied strategies mainly address pre- 1 arXiv:2603.11101v1 [cs.RO] 11 Mar 2026 Figure 1AI-Native Cloud Embodied Intelligence Infrastructure defined task sets in controlled environments, but they have weak generalization abilities and are sensitive to noise in real-world scenarios [4]. In recent years, breakthroughs in multimodal understanding, reasoning, and utilization of vision and language foundation models [5] have propelled the rapid development of Vision- Language-Action (VLA) models. VLA models, with powerful cross-modal sequential modeling capabilities, integrate perception, understanding, reasoning, and action generation end-to-end, enabling direct mapping from visual observation and language instructions to physical actions. Existing research has demonstrated their outstanding versatility, flexibility, and generalization to complex environments [6–8]. Simultaneously, the embodied intelligence ecosystem continues to integrate with world models [9], robotic hardware [10], open-source datasets [11], simulation environments [12,13], training algorithms [14], and computational resource[15] supply, collectively forming the AI-Native Embodied Intelligence Infrastructure (as shown in Figure1), which provides a solid foundation for the continuous optimization and iteration of VLA-based embodied intelligence systems. Success stories such as the game of Go [ 16] and LLMs [17] show that large-scale, general-purpose computa- tion is essential to move embodied intelligence from scientific research to industrial application. However, achieving large-scale training for embodied intelligence models poses higher demands and challenges for AI infrastructure. Training framework challenges:For thousand-GPU scale embodied training, there is a lack of industrial- grade systems that seamlessly connect simulation, training, and evaluation; multi-dimensional parallelism is complex, making communication and load balancing diﬀicult; large batch data loading is easily blocked by I/O, leading to unstable training and low compute utilization, which hinders eﬀicient large-scale training. Data engine challenges:Mixed storage of multimodal files increases system complexity, and high concur- rency puts pressure on metadata processing; frequent file operations lead to increased latency and reduced throughput, affecting GPU data supply; uneven load among nodes may block distributed training; the com- plexity of data preprocessing further burdens storage. Traditional data lakes struggle to dynamically allocate large files, and serial processing can lead to resource idleness and task blockage, lacking elastic scalability and failing to meet cloud-native high-availability requirements. Model computation challenges:Insuﬀicient dynamic computation and memory optimization, padding in traditional attention mechanisms generates invalid tokens, causing wasted compute and memory redundancy; ineﬀicient training data organization leads to resource waste and low hardware utilization due to short data being padded to fixed lengths; model inference and edge deployment are constrained by real-time requirements and compute resources, making eﬀicient compression and acceleration of small-parameter models key for deployment. Based on these challenges, the AI Infra team at JDT has proposed and implemented a thousand-GPU dis- tributed training framework for embodied intelligence based on the JD Cloud JoyBuilder platform, using the open-source LeRobot framework as its foundation, as shown in Figure 2. It is equipped with a 3.2T RDMA backend network supporting up to ten thousand GPUs and a flexible VPC frontend network, sys- 2 tematically overcoming bottlenecks across the entire pipeline. Through Yunhai high-performance storage and a Ray-driven elastic AI data lake, the training data pipeline is collaboratively optimized to meet embod- ied model training needs. For the GR00T-N1.5 [8] model, in thousand-GPU clusters and billion-scale data scenarios, single-round training time is reduced from 15 hours to 22 minutes, a 40-fold speedup; combining variable-length FlashAttention and Data Packing achieves sequence integration from sample redundancy, ac- celerating by 188%; customπ 0.5 architecture and post-training quantization improve training and inference eﬀiciency by 165%; FP8 quantization accelerates by 140%; and an end-to-end evaluation system is built to connect the training-simulation-evaluation pipeline. At the VLA training level, we propose and implement a fully asynchronous strategy training pipeline for the first time — RL-VLA 3 [18], a triple-level asynchronous architecture covering the entire process from environment interaction and trajectory generation to policy network updates. On the LIBERO benchmark, this framework achieves a maximum throughput improve- ment of 59.25% compared to existing synchronous training strategies. After further optimization through a decoupling strategy, the throughput improvement reaches 126.67%. This framework has been validated at scale on 256 GPUs clusters, providing robust infrastructure support for the industrialization of embodied intelligence. 2Core Architecture and Model Optimization 2.1Overall Architecture Design Figure 2The overview of Joybuilder AI-Native Cloud Embodied Intelligence Infrastructure With the continuous growth in the scale of embodied intelligence models and the complexity of data, eﬀicient distributed frameworks are playing an increasingly important role in large-scale training and multi-scenario adaptation. These frameworks integrate VLA models [ 19], standardized data formats, and unified simu- lation interfaces, significantly lowering the R&D threshold. Mainstream open-source frameworks such as Lerobot [ 11] and RLinf [20,21] have their respective strengths in model support, data adaptation, and simu- lation environments, jointly promoting the learning of generalized skills for robots. LeRobot [11] is renowned for its strong usability and active community, natively supporting standard data formats and various simula- tion environments, and is convenient for real robot integration, making it suitable for rapid prototyping and academic exploration. RLinf focuses on reinforcement learning and high-fidelity simulation, offering flexible sampling but requiring more customization. SimpleVLA-RL [ 22] and Dexbotic [23] focus on minimal packag- ing and industrial data pipelines, but have limited functionality and community support. Overall, LeRobot’s usability and ecosystem make it an ideal framework for research and development. Based on the LeRobot framework, we have integrated NVIDIA’s high-fidelity simulation platform to build a new generation of cloud-native embodied intelligence training framework, as shown in Figure 2, accelerating the development 3 of embodied intelligence. 2.1.1LeRobot and Nvidia Open-Source Ecosystem LeRobot is a lightweight embodied intelligence training framework launched by the Hugging Face community, supporting standard Dataset formats and various simulation environments, and can be easily integrated with real robots, making it suitable for rapid experimentation and algorithm verification. However, when faced with high-fidelity simulation and large-scale training requirements, a single open-source framework is still insuﬀicient. NVIDIA has built a full-stack ecosystem covering simulation, training, and deployment, providing high-fidelity simulation platforms such as Isaac Lab [24] and Omniverse, optimizing the training process, and supporting eﬀicient migration from simulation to real-world with Isaac ROS and Jetson hardware. This ecosystem performs excellently in complex tasks but requires higher hardware and engineering capabilities from developers, forming a complementary relationship with community frameworks. 2.1.2Cloud-Native Cloud Embodied Intelligence Training Infrastructure We have integrated NVIDIA high-fidelity simulation and LeRobot standardized data to build a new generation of cloud-native embodied intelligence training framework, systematically integrating data processing, model training, simulation evaluation, and distributed computing power, providing full-stack support for large-scale training and evaluation of VLA models. The framework consists of four layers: the data layer is compatible with mainstream formats (such as LeRobot, RLDS), supports eﬀicient preprocessing and streaming loading, meeting the needs of massive samples; the training layer supports pre-training, fine-tuning, and reinforcement learning, integrates distributed tools such as PyTorch DDP and DeepSpeed [ 25], improves cluster compute utilization, and offers experiment tracking and checkpoint recovery; the simulation evaluation layer uniformly connects to multiple environments (Open Gym [26], Mujoco [13], Isaac Sim [12]), with built-in automated evaluation processes for quantitative model validation; the distributed infrastructure relies on CUDA, NCCL, and Ray to achieve eﬀicient communication, storage acceleration, and resource scheduling, fully adapting to cloud-native environments. The overall framework combines open-source usability with industrial-grade system design, facilitating the rapid deployment of embodied intelligence technologies. This framework outlines a complete technology stack from data processing to resource scheduling. To support training of thousand-GPU clusters and models with tens or even hundreds of billions of parameters, the key lies in eﬀiciently and flexibly organizing and scheduling large-scale computing resources—this is the core mission of distributed parallel technology. Next, we will delve into the design and practice of distributed parallel strategies from an architectural perspective, specifically analyzing how multidimensional parallel combinations can achieve optimal resource allocation and maximize training eﬀiciency in real-world model training scenarios. 2.1.3Distributed Parallel Training In actual training of models with tens or hundreds of billions of parameters, a single parallel strategy is often insuﬀicient, requiring multidimensional combinations: •3D Parallelism:Data parallelism + pipeline parallel parallelism + tensor parallelism. Tensor paral- lelism decomposes large layers within nodes (high-speed interconnect), pipeline parallelism decomposes model depth across nodes, and data parallelism replicates the above mixed parallel groups to handle larger batches. •Expert Parallel :On top of 3D parallelism, for MoE models, expert parallelism is introduced. Expert parallel groups can be nested or separated from tensor parallel groups to optimize communication. •Sequence Parallelism:If sequence length leads to activation memory explosion, sequence parallelism is further introduced, usually closely combined with tensor parallelism. This multidimensional, hierarchical parallel strategy design is the core of modern large model training systems, requiring trade-offs between computational eﬀiciency, memory usage, and communication overhead. 4 Figure 3Schematic diagram of distributed training parallel strategies Table 1Comparison of Distributed Training Parallelism Strategies:Data Parallelism (DP), Pipeline Parallelism (P), Tensor Parallelism (TP), Expert Parallelism (EP) and Sequence Parallelism (SP). Strategy Partitioned Object Communication Pattern Key Problem Solved Scalability Limitation DPTraining dataAll-ReduceData throughput, compute acceleration Per-device memory, global batch size PPModel layersPoint-to-point (adjacent stages) Model depth (layers) Pipeline bubbles, stage load balancing TPIntra-layer weight matrices All-Reduce / All-Gather (within layer) Model width (layer size) Frequent intra-layer communication, bandwidth & latency sensitive EPExpert sub-networks All-to-All (token routing) Total parameter count Load balancing, token-routing communication overhead SPActivation tensors (sequence dim) All-Gather / Reduce-Scatter Activation memory for long sequences Communication overhead of sequence operations 5 Distributed Data Parallel (DDP)is a distributed training paradigm based on data parallelism. Its core goal is to eﬀiciently train large models on single-node or multi-node clusters with multiple computing devices (such as multiple GPUs) by parallelizing data loading and gradient synchronization, achieving near-linear training acceleration. PyTorch DDP maximizes distributed compute resource utilization through three core technologies: a multi-process architecture, peer-to-peer All-Reduce communication, and bucketed gradient synchronization, while maintaining model replica consistency. The core workflow is as follows: 1.Data Sharding:Each process loads a different shard of the global dataset, generating a local mini- batch; 2.Forward Pass:The local model replica performs forward computation on the shard data; 3.Backward Pass:Local gradients are computed based on the loss; 4.Gradient Synchronization:AllReduce collective communication is triggered, summing and averaging the gradients of the same parameters across all processes, synchronizing them to each process; 5.Parameter Update:Each process independently updates its local model parameters with the synchro- nized gradients. Through DDP-based distributed training, we have implemented GR00T and LeRobot training on thousand- GPU clusters. Figure 4Diagram illustrating Distributed Data Parallelism (DDP) 2.2Model-Level Training Optimization for Embodied Models The cloud-native embodied distributed framework provides an eﬀicient foundation for embodied model train- ing, while computational optimization at the model layer is key to further unleashing compute potential. In cloud-native embodied models, VLA is responsible for mapping multimodal perception and high-level in- structions to action sequences; the World Model realizes safer and more eﬀicient autonomous decision-making through environment modeling and simulation reasoning. The two together form the decision-making core. Embodied base models represented by VLA have undergone technical evolution from basic capability explo- ration to eﬀicient, scenario-oriented multimodal fusion architectures. Early models like PaLM-E validated the underlying “vision-language-action” linkage, while models like GR00T andπseries continuously enhance multi-task adaptation and cross-scenario action generalization. In 2025, VLA entered a period of intensive eﬀicient iteration, breaking deployment bottlenecks through lightweight design and structural optimization, and expanding action scenarios with reinforcement learning, becoming a core requirement for large-scale cloud applications. Although current VLA models have distinctive architectures, they generally follow the universal computa- tional paradigm of “multimodal encoding—large model decision-making—action generation”: visual obser- 6 Figure 5Schematic diagram of a typical VLA model architecture vations, text instructions, and embodiment states are encoded by encoders, input into LLMs for planning and decision-making, and then output as continuous action sequences by the action generation module. The πseries (π 0 [7],π 0.5 [6]) and GR00T N1.5 [8] are typical representatives of this paradigm.π 0.5 improves the generalization ability of robotic arms in home environments through hierarchical reasoning and multi- source data training; GR00T N1.5 achieves eﬀicient collaboration and task adaptability with a tightly coupled dual-system architecture and innovative data pyramid strategy, and realizes integrated data, training, and simulation through the NVIDIA Isaac ecosystem [ 12]. Based on the cloud-native embodied training frame- work and typical models, the team has optimized distributed training for GR00T N1.5 at a thousand-GPU scale, significantly improving large-scale training performance. At the same time, eﬀiciency improvements have also been made for theπseries models, mainly in three aspects: 1.Optimizing the attention structure for long-sequence processing with Eﬀicient Attention or Mamba Block to reduce computational complexity; 2.Filtering image and text tokens to reduce redundant data and ineffective computation; 3.Compressing model size using quantization techniques to accelerate computation while maintaining model accuracy. 2.2.1Attention Dynamic Computation and Memory Optimization (1) Variable-Length Flash-Attention: Eliminating Padding Compute WasteAs the “understanding core” of VLA, VLM undertakes cross-modal understanding of images and text, and its computational eﬀiciency directly determines the training and inference performance of the entire VLA model. During model training and inference, multimodal inputs (such as image patch tokens and text sequences) naturally have inconsistent lengths. The traditional approach uses padding to fill all inputs to a fixed length, causing many invalid padding tokens to participate in attention computation, resulting in wasted compute and memory redundancy. This work focuses on the attention computation of the Qwen2.5-VL model LM Decoder, systematically analyzing the current attention paradigms supported by the oﬀicial Transformers library and introducing data packing strategies to further compress redundant computation. Specifically, the visual-side attention module, when Flash-Attention-2 is enabled, directly calls its variable-length (varlen) interface to perform matrix operations only on valid visual tokens; on the language model side, visual features and text embeddings are concatenated into a mixed sequence and then fed into the LM Decoder for autoregressive modeling. When Flash-Attention-2 is effective, the LM Decoder dynamically selects the “variable-length” or “non- variable-length” computation path according to the sequence distribution, achieving optimal matching of computational intensity and memory bandwidth. Based on Flash-Attention, we optimize for variable-length computation, calculating only on valid token 7 Figure 6Key optimization strategies for the VLA model sequences, greatly improving model training and inference eﬀiciency while maintaining stable model perfor- mance: •As sequence length or padding rate increases, the time savings of variable-length computation become more significant; •As sequence length increases, the TFLOPS of variable-length attention gradually approaches that of non-variable-length, and with moderate padding rates, can even surpass non-variable-length; •Variable-length attention maintains stable leaderboard performance compared to non-variable-length. (2) Data Packing: From Sample Redundancy to Sequence IntegrationIn large model training, training data often consists of text sequences of varying lengths. During data preprocessing, the data packing strategy is used to pack different samples into one sequence, so that the data received during training is in a no-padding format, andflash_attnis called for attention computation. The traditional approach fills texts shorter than the fixed length with special tokens, resulting in computational waste and ineﬀicient training. To address the eﬀiciency issue caused by inconsistent data lengths in large model training, this study inno- vatively combines the Data Packing strategy with Flash-Attention. Our proposed optimization intelligently concatenates multiple shorter training samples to construct a sequence close to the model’s maximum con- text length, thereby minimizing or even eliminating the use of padding tokens. This sequence integration method provides a new technical path for eﬀicient training of large-scale language models, especially suitable for handling multimodal training data with significant length differences. Specific experimental results are shown in section 3.2.2. Figure 7Padding and packing processing [27] 8 2.2.2Acceleration Optimization for VLA Model Architecture:π 0.5 Case Study To improve the training eﬀiciency of theπ 0.5 VLA model, this work proposes a customized optimization strategy targeting redundant computation and memory overhead in its attention mechanism. The strategy focuses on the waste of computational resources caused by invalid tokens during the processing of multimodal inputs (images, text, states, and actions), especially redundant information in visual and textual modalities. Specifically, theπ 0.5 model uses a Vision-Language Model (VLM) to process image, text, and state infor- mation during training, and generates action outputs through an independent action expert module. In this process, the attention mechanism establishes interactions between multimodal sequences, causing many invalid tokens (such as irrelevant image regions or redundant text fragments) to participate in computation, resulting in significant memory usage and computational redundancy. To solve these problems, this work optimizes on two levels: •At the sequence modeling level, a dynamic sequence padding mechanism is introduced, which com- putes the maximum sequence length (max_length) for each training batch according to actual input lengths, achieving dynamic alignment of variable-length sequences and avoiding resource waste caused by traditional fixed-length padding (e.g., uniformly padding to 200 tokens). •In the data preprocessing stage, invalid visual tokens are pruned based on prior knowledge. For example, in the LIBERO dataset, right-hand perspective images are verified to have no significant contribution to task execution, so these images are directly removed before input, reducing the number of visual tokens at the source and lowering attention computation complexity and memory consumption. This optimization method significantly improves training eﬀiciency and reduces invalid computation and memory usage without changing the model structure, providing a more scalable training paradigm for large- scale multimodal policy learning. Figure 8π 0.5 Attention Mask Example 2.2.3Quantization Compression: Achieving Model “Slimming” and Acceleration While Maintaining Accuracy Large model quantization is a model compression technique that reduces model size and accelerates compu- tation through low-bit quantization. The finer the granularity, the smaller the accuracy loss, but managing multiple scaling factors increases computational and memory overhead. Quantization is especially critical for deployment on edge devices and is well-suited for small-parameter VLA models, enabling eﬀicient compression while maintaining accuracy. The quantization process selectively uses per-tensor, per-channel, or block-wise quantization according to the original neural network layers and data types to maximize model accuracy during conversion. •Per-tensor quantization:Uses a single scaling factor (scalar) to scale the entire tensor. 9 •Per-channel quantization:Each channel of the tensor has a specific scaling factor; for convolutional neural networks, the output channel at the 0th dimension of the kernel is usually the quantization axis. •Block-wise quantization:The tensor is partitioned into fixed-size blocks along one or more dimensions, with a scaling factor defined for each block. The main difference among the three lies in the granularity of scaling. The finer the granularity, the higher the potential accuracy, but managing multiple scaling factors increases computational and memory overhead. Therefore, choosing the appropriate granularity is important to balance the advantages of quantization (e.g., reduced memory usage) and its potential disadvantages (e.g., accuracy loss). Quantization is crucial for reducing memory usage and improving energy eﬀiciency, making it ideal for de- ployment on resource-constrained edge devices. This is exactly the scenario faced when deploying VLA/VLM models on terminal devices, making exploration of quantization for VLA/VLM models highly significant. Ordinary FP8 quantization (i.e., per-tensor scaling) has a significant impact on model accuracy, especially for small models. Due to real-time requirements and edge compute constraints, current VLA and the VLMs they use are usually small models with parameter sizes of 0.5B, 1B, 3B, etc. We apply fine-grained FP8 quantization (block-wise FP8 quantization), partitioning tensors along the last two dimensions into blocks of size128×128. The vision module (ViT) is not quantized to maintain visual feature quality, while the language module (LLM) undergoes fine-grained FP8 quantization, applied post-training (PTQ), without FP8-aware training (QAT). This compresses model size and improves inference speed while maintaining model accuracy. Figure 9The example of per-tensor, per-channel and per-block 2.3RL-VLA 3 : Reinforcement Learning VLA Accelerating via Full Asynchronism Existing VLA training pipelines are inherently constrained by the synchronous execution paradigm, failing to fully leverage the parallel processing capabilities of computational resources. The serial dependencies among simulator interaction, policy generation, and model training lead to idle computational resources and throughput bottlenecks, limiting further improvements in training eﬀiciency. In current practices of LLM reinforcement learning, asynchronous training mechanisms have been proven to significantly enhance training eﬀiciency and system throughput, with related research giving rise to various mature asynchronous optimiza- tion strategies. However, in the domain of VLA model reinforcement learning training, such asynchronous training methods remain in the exploratory stage and have not been fully utilized. To bridge these gaps, this paper systematically introduces asynchronous training and inference mechanisms based on an existing unified framework. Drawing on the asynchronous optimization concepts from LLM reinforcement learning, we design a triple-level asynchronous execution architecture, RL-VLA3, which en- compasses asynchronous training and inference, asynchronous interaction policy, and streaming generation, 10 as show in Figure10. These mechanisms substantially reduce inter-component waiting times, enabling con- tinuous and saturated utilization of computational resources. Experiments demonstrate that the proposed asynchronous framework significantly boosts training throughput and reduces overall training time while maintaining good training stability and policy performance. b2 GPU GPU GPU GPU GPU GPU GPU GPU Rollout 1 Rollout 2 Rollout 3Rollout 6 Rollout 4 Rollout 5 Training b1-b6 Rollout 1 Rollout 2 Rollout 3Rollout 6 Rollout 4 Rollout 5 b1 , b3b4b5b6 Rollout 7 Rollout 8 Rollout 9 b7-b9 Rollout 10 Rollout 12 Rollout 11 b10 Step n Step n+1 Rollout 7 Rollout 8 Rollout 9 Step n Step n+1 (a) Synchronous Training(b) Asynchronous Training (Ours) Macro Level (Across Rollout Epochs) Micro Level (Within Each Rollout Epoch) Simulation Inference Engine Env 2 Env 3 Env 4 1 Env 1 Env 2 Env 3 (c) Synchronous Rollout (e) Waiting Time-Triggered Asynchronous Rollout (Ours) 2 3 4 3 1 4 (d) Batch Size-Triggered Asynchronous Rollout (Ours) Waiting Queue Env 1 Env 3 Env 4 3Env 4 Env 1 4 2 1 1 3 Queue size reaches 2 3 1 4 2 Env 2 Env 3 Env 4 Env 1 Env 2 Env 3 Env 4 Env 1Env 1 Env 2 Env 3 4 4 Env 1 Env 2 Env 3 4 Env 3 Env 1 4 Env 2 4 Waiting time reaches 20ms Simulation Inference Engine Simulation Inference Engine Waiting Queue Waiting Queue Figure 10An illustration of the proposed asynchronous framework for VLA training. The design operates asyn- chronously on two levels: (1) macroscopically, between the rollout and training pipeline stages, and (2) microscopically, within each rollout epoch. The latter is governed by two trigger strategies: batch size and waiting time. 2.3.1Asynchronous training and inference Rollout workers (responsible for environment interaction and trajectory generation) and actor workers (re- sponsible for policy model updates) are deployed on entirely different GPU devices. After a rollout worker completes a single trajectory, it immediately places that trajectory into the transmission queue of the com- munication pipe without waiting for other rollout processes to finish, and proceeds to generate the next trajectory based on the current policy version. The actor worker no longer waits for all rollout processes to complete; instead, once the accumulated trajectory data in the communication pipe reaches a predefined training batch size, it asynchronously collects data from the queue and initiates policy optimization and parameter updates. Asynchronous training and inference achieve computational masking between the two core stages of rollout and actor: the model updates by the actor and trajectory generation by the rollout workers are heavily overlapped in time, thus successfully resolving the resource idle problem of traditional synchronous training. 2.3.2Asynchronous interaction policy Traditional VLA training frameworks typically employ a synchronous batch interaction method, where mul- tiple environments run in parallel but require all environments to complete their current step before entering model inference as a whole batch, leading to strict synchronization dependencies. RL-VLA 3 adopts a dynamic batching scheduler, which balances throughput and latency through two key parameters:B max (the maxi- mum batch size allowed for a single inference) andT max (the maximum time a request can wait). Dynamic batching avoids prolonged idle spinning caused by waiting to form a batch; under high load, it naturally 11 tends toward larger batches to improve throughput, while under low load or environment jitter, it prioritizes system fluidity. 2.3.3Streaming generation During training, the actor needs to accumulate a suﬀicient number of trajectory samples to form a complete global training batch before it can start forward and backward computations of the model, leading to inter- mittent GPU idle periods. RL-VLA3 splits the global training batch into several independent micro-batches. As soon as the accumulated samples reach the size of a single micro-batch, the actor immediately initiates forward and backward computations for that micro-batch. After all micro-batches are computed sequentially, the gradients generated from all micro-batches are aggregated together, and a single model parameter update is performed, thereby avoiding intermittent idle periods. 3Experiments 3.1Thousand-GPU Scale Framework Validation Based on the aforementioned architecture and cloud-native optimizations, AI Infrastructure has achieved a breakthrough in the scalability and eﬀiciency of embodied intelligence training on JoyBuilder, validating the framework’s advancement and reliability. 3.1.1DDP Distributed Thousand-GPU Training Scaling Law (a)Diagram illustrating the relationship between mini- batch size, training time, and GPU memory usage. (b)Diagram illustrating the relationship between data parallelism and training time. Figure 11The impact of different mini-batch sizes (MBS) and data parallel instances (DP) on training time and GPU memory usage. An important observation metric for the scaling law in DDP training is the training time under different global batch sizes. Therefore, we focus on exploring the impact of two key factors that constitute the global batch size (GBS): mini-batch size (MBS) and data parallel instances (DP), on training time. •Mini-Batch Size (MBS):With DP fixed at 128 nodes (1024 GPUs), we compare cases where MBS is 256 and 512. Experimental results show that with MBS=256, the training time per epoch is 48 minutes and memory utilization is 55.5%. When MBS=512, combined with further storage optimization, the training time per epoch is reduced to 22 minutes and memory utilization rises to 93.98%, as shown in Figure 11. Thus, as MBS increases, training time decreases while memory usage further increases. •Data Parallel Instances (DP):With MBS fixed at 128, we compare training times per epoch for DP at 32 nodes (256 GPUs), 64 nodes (512 GPUs), and 128 nodes (1024 GPUs). Results show that as DP 12 increases, training time per epoch decreases from 2.55 hours to 1.24 hours and 0.73 hours, respectively. Specifically, when DP increases from 32 to 64 nodes, training time is halved, but when DP increases from 64 to 128 nodes, distributed communication overhead also increases, resulting in a final speedup of 1.69x. 3.1.2Thousand-GPU Training of GR00T N1.5 Leveraging the JoyBuilder embodied toolchain, we achieved stable and eﬀicient training of the GR00T N1.5 model on over 100 million frames of embodied data using a 1024-GPU cluster. Prior to optimization, the maximum supported batch size was 256; exceeding this value would cause heavy file reading, leading to I/O blocking of Dataloader worker processes on certain nodes, which in turn triggered NCCL timeouts and interrupted training. Under these conditions, training for one epoch took approximately 15 hours (as shown by the light pink line in Figure12). After end-to-end collaborative optimization of cloud storage, network communication, and platform deployment with JoyBuilder, training with a batch size of 512 required only 22 minutes per epoch (as shown by the purple line in Figure. The training time was reduced by 97.57%, achieving an approximately 40-fold speedup, truly realizing “fast, accurate, and stable” model training on the platform. Meanwhile, we have established a comprehensive simulation evaluation system, providing precise assessment standards for algorithm iteration. Figure 12Comparison of system training speeds before and after optimization Based on the widely adopted LeRobot open-source framework—which supports the largest variety of embod- ied models in the industry—and integrating AI Infra’s proprietary cloud-native embodied optimizations, we have achieved the first stable thousand-GPU scale training of the GR00T model in the industry. Training eﬀiciency has been improved by 3.5 times compared to the open-source LeRobot baseline (reduced from 3 hours to 40 minutes, as shown in Figure 11b), significantly lowering the barriers and costs for large-scale training and enabling stable thousand-GPU training. 3.2Attention Dynamic Computation and Memory Optimization 3.2.1Variable-Length Flash-Attention: Eliminating Compute Waste from Padding As the “understanding core” of VLA, the VLM undertakes cross-modal understanding tasks for images and text, and its computational eﬀiciency directly determines the training and inference performance of the entire VLA model. In this experiment, Qwen2.5-VL was selected as the base model. Leveraging Flash-Attention with variable-length optimization, computation is performed only on valid token sequences, significantly improving training and inference eﬀiciency while maintaining stable model performance. Experimental results are shown in Figure 14and Figure15: •When fixing the sequence length (seq_len) and batch size, examining the impact of different padding rates: In Figure14, for fixed sequence length and batch size, the latency of the variable-length interface 13 (a)Lerobot Industry-Standard Framework (GR00T N1.5) brfore optimization (b)The optimized Lerobot industry-standard framework (GR00T N1.5) has achieved full utilization of both com- puting power and memory, resulting in high eﬀiciency. (c)Model optimization pipeline for NVIDIA framework (GR00T N1.5): Before optimization (d)Model optimization pipeline tailored for the NVIDIA framework (GR00T N1.5): improved utilization and sta- bility after optimization. Figure 13Comparison of model optimization using the Lerobot industry-standard framework (top) and the NVIDIA- oriented framework (GR00T N1.5) (bottom). (varlen) is consistently lower than that of the fixed-length interface (fixlen). As the padding rate increases from 3% to 90%, the time savings from the variable-length interface rise from 2.28% to 89.73% (Figure14a). Furthermore, this advantage becomes more pronounced as the batch size increases. For shorter sequences (2048), when the batch size increases from 8 to 32, the peak TFLOPS increases by 1.2x. However, due to the shorter effective sequence lengths, the variable-length interface has lower computational utilization, and its TFLOPS remains lower than the fixed-length interface. In contrast, for long sequences (32k), the variable-length interface executes faster, and its TFLOPS is on par with or even surpasses the fixed-length interface, as shown in Figure 15b. •When fixing the batch size (e.g., 16), as the sequence length increases from 2k to 32k, the average latency of the variable-length interface is consistently lower than the traditional method. For sequence lengths larger than 8k, the time savings range from 25% to 90%, as shown in Figure15a. The optimization effect of the variable-length interface is particularly significant under higher padding rates. Addition- ally, Figure 15bshows that TFLOPS increases rapidly as the sequence length grows from 1k to 8k, then stabilizes beyond 8k, indicating that the variable-length interface achieves higher computational eﬀiciency in large-scale sequence processing. While enabling faster computation, the performance of the variable-length interface remains consistent with the fixed-length interface across multiple benchmarks, as shown in Figure 15c. These quantitative results provide measurable engineering optimization evidence for the eﬀicient training of deep learning models. 3.2.2Data Packing: From Sample Redundancy to Sequence Integration To comprehensively evaluate the effect of Data Packing, we assessed its impact on Qwen2.5-VL across multiple datasets, including visual question answering (TextVQA_VAL), reading comprehension (MMMU_DEV_- VAL, MUIRBench, RealWorldQA), and multi-domain, multi-language benchmarks (MMStar). Specifically, several shorter samples are concatenated end-to-end to form long sequences close to the model’s maximum context length, effectively eliminating invalid padding tokens. Flash-Attention is then used to eﬀiciently perform self-attention computation on these long sequences. Experiments result is shown in figure 16. This approach achieves a 1.88×increase in training throughput and a 46.87% reduction in total training time, while downstream task accuracy remains the same or slightly improves: on MMMU_DEV_VAL, MUIRBench, 14 (a)(b) (c)(d) Figure 14Different batch size of flash attention. (a)Comparison of processing time under different sequence lengths and padding rates (batch_size=16). (b)TFLOPS comparison under differ- ent sequence lengths and padding rates (batch_size=16) (c)Benchmark Score. Figure 15Comparison of eﬀiciency and model performance between variable-length (varlen) and non-variable-length (fixlen) Attention mechanisms 15 Figure 16Data Packing combined with the Flash-Attention strategy RealWorldQA, TextVQA_VAL, and MMStar, accuracy changes from 41.00% to 41.33%, 44.88% to 44.23%, 61.44% to 64.97%, 72.58% to 76.82%, and 51.60% to 53.27%, respectively. These results indicate that the combination of Data Packing and Flash-Attention not only significantly improves training eﬀiciency and memory utilization but also brings additional performance gains in complex open-domain and visual question answering tasks. With virtually unchanged accuracy, training speed increased by 188% and total training time decreased by 46.87%. 3.3π 0.5 Acceleration Optimization (a)Training results.(b)inference result Figure 17omparison of training duration before and after optimization and verification of simulation results In this study, we systematically optimized model training eﬀiciency and compared performance before and after optimization through 30,000-step fine-tuning experiments on the Libero dataset. Experimental results in Figure 17ashow that, during training, the optimized model’s per-step training time was reduced from 4.71 seconds to 2.85 seconds, a decrease of 39.56%. The total training time shortened from 39 hours 40 minutes to 23 hours 44 minutes, representing a 40.2% improvement in eﬀiciency. Notably, this significant acceleration was achieved while maintaining stable model accuracy: the loss value increased only slightly from 0.0058 to 0.0060, a difference of less than 0.02%, and deployment success rate remained unchanged. These results demonstrate that the customized optimization strategy achieved a breakthrough in training eﬀiciency without 16 Table 2Task Success Rate Comparison inπ 0.5 : Original vs. Optimized No. Task NameOriginalOptimized 1 pick up the black bowl between the plate and the ramekin and place it on the plate 100%100% 2 pick up the black bowl next to the ramekin and place it on the plate 100%100% 3 pick up the black bowl from table center and place it on the plate 100%100% 4 pick up the black bowl on the cookie box and place it on the plate 100%100% 5 pick up the black bowl in the top drawer of the wooden cabinet and place it on the plate 94% (3 failures) 92% (4 failures) 6 pick up the black bowl on the ramekin and place it on the plate 98% (1 failure)100% 7 pick up the black bowl next to the cookie box and place it on the plate 100%100% 8 pick up the black bowl on the stove and place it on the plate 96% (2 failures) 94% (3 failures) 9 pick up the black bowl next to the plate and place it on the plate 96% (2 failures) 96% (2 failures) 10 pick up the black bowl on the wooden cabinet and place it on the plate 100%100% Total success rate0.9840.982 Total episodes500500 compromising model performance. To verify the generalization ability of the optimized model, we conducted systematic inference experiments on the Libero Spatial test set. The test set includes 10 independent tasks, each executed 50 times (for a total of 500 rollouts). Results in Figure 17band Table2show that the pre-optimization model succeeded in 492 out of 500 rollouts (8 failures, 98.4% success rate), while the optimized model succeeded in 491 out of 500 rollouts (9 failures, 98.2%), with only a marginal difference of 0.2%. Notably, in Task 5, which had the most failures, the final actions still completed the intended objectives correctly despite some intermediate deviations. Taken together with the training phase improvements (40% faster training and only a 0.0002 increase in loss) and robust inference performance, these findings confirm that the proposed optimization method significantly improves training eﬀiciency while maintaining model performance ( p> 0 . 05 , statistically insignificant). This provides strong empirical evidence for eﬀicient deep learning model optimization, showing that increased computational eﬀiciency does not necessarily come at the cost of model accuracy. 3.4Post-Training Quantization Taking Qwen2.5-VL-3B as an example, the vision module (ViT) retains high precision, while the language module (LLM) adopts fine-grained FP8 block-wise quantization (128×128 blocks), achieving post-training quantization (PTQ) and effectively improving computational eﬀiciency. We compared various quantization strategies on GSM8K and MMLU tasks, including AWQ, GPTQ-int4, LLM Compressor FP8-dynamic, and our customized FP8 block-wise quantization, as show in Figure 18. We can see that FP8 block-wise quantization not only achieved 36.6% model compression, but also maintained the original model accuracy on both tasks, with post-training quantization speeding up computation by more than 140%. Overall, FP8 block-wise quantization outperformed other methods in both speed and leaderboard accuracy, achieving “faster and better” results. 17 Figure 18Performance of various quantization models on GSM8K and MMLU tasks 3.5Reinforcement Learning VLA Accelerating via Full Asynchronism To demonstrate the performance of RL-VLA3, we conduct experiments measuring the throughput ofπ 0.5 ,π 0 , and GR00T in the LIBERO and ManiSkill environments. Baseline comparisons are made against the RLinf framework using both co-located and distributed placement strategies. Table 3Throughput comparison of training strategies. Our asynchronous methods (Train Async, Rollout Async) outperform the Disaggregated baseline and, in most cases, the Colocated baseline across different models and envi- ronments. The best value for each column isbolded. And the Increase% is compare to colocated stratedies. Configuration LIBERO+π 0.5 LIBERO+GR00T N1.5ManiSkill+π 0 8 GPUs 16 GPUs 32 GPUs 8 GPUs 16 GPUs 32 GPUs 8 GPUs 16 GPUs 32 GPUs Colocated289.23 547.55 703.85 371.80 680.46 1125.62132.56232.23 370.26 Disaggregated (1:1) 162.75 307.84 457.23 220.81 409.60 729.98 60.64 150.59 257.21 Asynchronous improvements on Disaggregated (cumulative): + Train Async229.68 441.49 737.46 243.57 477.20 951.33 126.65244.61 436.32 + Rollout Async369.56 686.80 1041.36 434.20 816.481620.3969.07 139.32 275.36 + Streamer383.40 713.38 1120.91 439.64 816.481592.40 71.61 141.49 280.07 Increase %↑32.5%↑30.29%↑59.25%↑18.25%↑19.99%↑43.96%↓-4.46%↑5.33%↑17.84% As shown in Table3, our asynchronous strategy significantly improves throughput compared to the dis- tributed strategy and surpasses the co-located strategy in most scenarios. Training asynchronously (Train Async) delivers substantial throughput gains across all configurations. However, the effectiveness of the asynchronous interaction strategy (Rollout Async) varies across environments. In the LIBERO+π 0.5 envi- ronment, this strategy further boosts training eﬀiciency by approximately 40%. Conversely, it leads to sig- nificant performance degradation on ManiSkill. This discrepancy stems from ManiSkill’s ability to leverage GPU parallelization for environment computations. Our current Rollout Async implementation partitions the entire batch of environments into mini-batches to achieve genuine parallelism between Environment and Rollout workers, which diminishes the eﬀiciency of batched environment computation in ManiSkill. Never- theless, the throughput loss is only 4.46% on a small 8-GPU setup. As the scale increases, the environmental overhead introduced by ManiSkill is offset. At the 32-GPU scale, our method achieves a 17.84% throughput improvement, validating its effectiveness for large-scale deployment. We further investigate the scaling of different strategies by increasing the number of GPUs. Figure 19shows the throughput for the LIBERO+π 0.5 configuration across different GPU counts. Ideally, throughput should scale linearly with the number of GPUs. Our results indicate that the method exhibits near-optimal scaling performance within the range of 8 to 24 GPUs. Scaling eﬀiciency decreases when scaling from 24 to 128 GPUs and degrades further between 128 and 256 GPUs. The sublinear scaling observed at large scales is attributed 18 8162432128256 Number of GPUs 1000 10000 Throughput Scaling Law of Throughput with Different Strategies Colocated Disaggregated + Train Async + Rollout Async + Streamer Ideal Linear Scaling Figure 19Scaling behavior with increasing GPU resources for the LIBERO+π 0.5 to increased communication overhead with the growing number of workers. Improving the method to achieve more desirable scaling capabilities at extreme scales remains an important direction for future work. 4Conclusion and Future Outlook We have systematically built a thousand-GPU distributed training framework for embodied intelligence tar- geting VLA models, achieving full-stack breakthroughs from data pipelines to infrastructure. The constructed thousand-GPU training framework has realized systematic optimization at the data, model, and infrastruc- ture layers, and has successfully supported large-scale training for multiple models, including GR00T N1.5. We plan to further extend this to broader model families such asπ 0 , continuously validating the framework’s advancement and generality. In the next phase of our research, we will tackle four core challenges inherent to the VLA framework: •Balancing model compactness and expressive power to address inference latency and resource con- straints; •Establishing a complete end-to-end Reinforcement Learning infrastructure, spanning from pre-training to reinforcement learning, and constructing a system that integrates world models to achieve a syner- gistic combination of training, inference, and simulation; •Breaking through the Sim2Real transfer bottleneck by establishing a consistency verification mechanism between simulation and real-world environments; •Improving multimodal evaluation standards to promote the industrialization of embodied intelligence beyond the laboratory. Embodied intelligence is a key area for the deep integration of AI and the physical world. We believe that through continuous system optimization and algorithmic innovation, general embodied intelligence will grad- ually mature, achieving breakthroughs in autonomous execution of complex tasks by robots, and ultimately ushering in a new stage of human-machine collaboration. One of our future research themes is exploring how to leverage reinforcement learning [ 28], multi-agent systems [29], and the reasoning capabilities of large mod- els [30]—combined with data synthesis [31] and economic design [32]—to augment sample data for embodied AI. In addition, whether certain security vulnerabilities inherent in LLMs could be further exacerbated in 19 embodied intelligence is also a concern worthy of our attention [33]. Furthermore, we aim to ensure alignment with human values [34] and mitigate the potential risks posed by embodied AI to humans [35]. 20 Contributions Author list ∗ Chen Zhou 1 , Haoran Sun 1,3 , Hedan yang 1,3 Jing Long 1,3 , Junwu Xiong 1 , Luqiao Wang 1,5 , Mingxi Luo 1 , Qiming Yang 1 , Shuai Di 1 , Song Wang 1 , 5 , Tianyun zhao 1 , 2 , Wanting Xu 1 , Wen Huang 1 , 2 , Xiaodong Bai 1 , Xiaomeng Tian 1,5 , Xiaolong Xiang 1,5 , Yicheng Gong 1 , Yongjian Guo 1,2 , Yucheng Guo 1 , Yunxuan Ma 1,3 , Yu Wei 1,6 Zhong Guan 1,4 , Zhen Sun 1,5 ∗ First-Name in Alphabetical Order Affiliation 1 AI Infra Team at JDT 2 Tsinghua University 3 Peking University 4 Tianjin University 5 Beihang University 6 University of Science and Technology of China References [1]Yann LeCun. A path towards autonomous machine intelligence.OpenReview, 62(1):1–62, 2022. [2]MarketsandMarkets. Embodied ai market size, share, trends and growth. Market Research Report SE 9427, MarketsandMarkets, 2024. [3]Goldman Sachs. Humanoid robots: Sooner than you might think. Technical report, Goldman Sachs, 2022. [4]Francesco Capuano, Caroline Pascal, Adil Zouitine, Thomas Wolf, and Michel Aractingi. Robot learning: A tutorial.arXivpreprintarXiv:2510.12403, 2025. [5]Yichao Shen, Fangyun Wei, Zhiying Du, Yaobo Liang, Yan Lu, Jiaolong Yang, Nanning Zheng, and Baining Guo. Videovla: Video generators can be generalizable robot manipulators. In TheThirty-ninthAnnualConferenceon NeuralInformationProcessingSystems, 2025. [6]Zhongyi Zhou, Yichen Zhu, Junjie Wen, Chaomin Shen, and Yi Xu. Vision-language-action model with open- world embodied reasoning from pretrained knowledge.arXivpreprintarXiv:2505.21906, 2025. [7]Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al.π0: A vision-language-action flow model for general robot control. corr, abs/2410.24164, 2024. doi: 10.48550.arXivpreprintARXIV.2410.24164. [8]Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots. arXivpreprintarXiv:2503.14734, 2025. [9]Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models.arXivpreprintarXiv:2301.04104, 2023. [10]Zipeng Fu, Tony Z Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation.arXivpreprintarXiv:2401.02117, 2024. [11]Remi Cadene, Simon Alibert, Alexander Soare, Quentin Gallouedec, Adil Zouitine, and Thomas Wolf. Lerobot: State-of-the-art machine learning for real-world robotics in pytorch, 2024. [12]NVIDIA. Isaac Sim. [13]Kevin Zakka, Baruch Tabanpour, Qiayuan Liao, Mustafa Haiderbhai, Samuel Holt, Jing Yuan Luo, Arthur Allshire, Erik Frey, Koushil Sreenath, Lueder A Kahrs, et al. Mujoco playground.arXivpreprintarXiv:2502.08844, 2025. 21 [14]John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation, 2018. [15]Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In ConferenceonRobotLearning, pages 2165–2183. PMLR, 2023. [16]David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanc- tot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy P. Lillicrap, Karen Simonyan, and Demis Hassabis. Mastering chess and shogi by self-play with a general reinforcement learning algorithm.CoRR, abs/1712.01815, 2017. [17]Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXivpreprint arXiv:2303.08774, 2023. [18]Zhong Guan, Haoran Sun, Yongjian Guo, Shuai Di, Xiaodong Bai, Jing Long, Tianyun Zhao, Mingxi Luo, Chen Zhou, Yucheng Guo, Qiming Yang, Wanting Xu, Wen Huang, Yunxuan Ma, Hongke Zhao, Likang Wu, Xiaotie Deng, Xi Xiao, Sheng Wen, Yicheng Gong, and Junwu Xiong. Rl-vla 3 : Reinforcement learning vla accelerating via full asynchronism, 2026. [19]Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprintarXiv:2406.09246, 2024. [20]Chao Yu, Yuanqing Wang, Zhen Guo, Hao Lin, Si Xu, Hongzhi Zang, Quanlu Zhang, Yongji Wu, Chunyang Zhu, Junhao Hu, et al. Rlinf: Flexible and eﬀicient large-scale reinforcement learning via macro-to-micro flow transformation.arXivpreprintarXiv:2509.15965, 2025. [21]Hongzhi Zang, Mingjie Wei, Si Xu, Yongji Wu, Zhen Guo, Yuanqing Wang, Hao Lin, Liangzhi Shi, Yuqing Xie, Zhexuan Xu, et al. Rlinf-vla: A unified and eﬀicient framework for vla+ rl training.arXivpreprint arXiv:2510.06710, 2025. [22]Haozhan Li, Yuxin Zuo, Jiale Yu, Yuhao Zhang, Zhaohui Yang, Kaiyan Zhang, Xuekai Zhu, Yuchen Zhang, Tianxing Chen, Ganqu Cui, et al. Simplevla-rl: Scaling vla training via reinforcement learning.arXivpreprint arXiv:2509.09674, 2025. [23]Bin Xie, Erjin Zhou, Fan Jia, Hao Shi, Haoqiang Fan, Haowei Zhang, Hebei Li, Jianjian Sun, Jie Bin, Junwen Huang, et al. Dexbotic: Open-source vision-language-action toolbox.arXivpreprintarXiv:2510.23511, 2025. [24]Mayank Mittal, Pascal Roth, James Tigue, Antoine Richard, Octi Zhang, Peter Du, Antonio Serrano-Muñoz, Xinjie Yao, René Zurbrüg, Nikita Rudin, et al. Isaac lab: A gpu-accelerated simulation framework for multi- modal robot learning. arXivpreprintarXiv:2511.04831, 2025. [25]Zhewei Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari, Xiaoxia Wu, Ammar Ahmad Awan, Jeff Rasley, Minjia Zhang, Conglong Li, Connor Holmes, et al. Deepspeed-chat: Easy, fast and affordable rlhf training of chatgpt-like models at all scales.arXivpreprintarXiv:2308.01320, 2023. [26]Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym.arXivpreprintarXiv:1606.01540, 2016. [27]Siyan Zhao, Daniel Israel, Guy Van den Broeck, and Aditya Grover. Prepacking: A simple method for fast prefilling and increased throughput in large language models.arXivpreprintarXiv:2404.09529, 2024. [28]Xiong Junwu, Xiaoyun Feng, YunZhou Shi, James Zhang, Zhongzhou Zhao, and Wei Zhou. Digital human interactive recommendation decision-making based on reinforcement learning, 2022. [29]Yusen Wu, Junwu Xiong, and Xiaotie Deng. How social is it? a benchmark for llms’ capabilities in multi-user multi-turn social agent tasks, 2025. [30]Ling Team, Bin Hu, Cai Chen, Deng Zhao, Ding Liu, Dingnan Jin, Feng Zhu, Hao Dai, Hongzhi Luan, Jia Guo, Jiaming Liu, Jiewei Wu, Jun Mei, Jun Zhou, Junbo Zhao, Junwu Xiong, Kaihong Zhang, Kuan Xu, Lei Liang, Liang Jiang, Liangcheng Fu, Longfei Zheng, Qiang Gao, Qing Cui, Quan Wan, Shaomian Zheng, Shuaicheng Li, Tongkai Yang, Wang Ren, Xiaodong Yan, Xiaopei Wan, Xiaoyun Feng, Xin Zhao, Xinxing Yang, Xinyu Kong, Xuemin Yang, Yang Li, Yingting Wu, Yongkang Liu, Zhankai Xu, Zhenduo Zhang, Zhenglei Zhou, Zhenyu Huang, 22 Zhiqiang Zhang, Zihao Wang, and Zujie Wen. Ring-lite: Scalable reasoning via c3po-stabilized reinforcement learning for llms, 2025. [31]Xiong Jun Wu, Zhenduo Zhang, ZuJie Wen, Zhiqiang Zhang, Wang Ren, Lei Shi, Cai Chen, Deng Zhao, Qing Wang, Xudong Han, Chengfu Tang, Dingnan Jin, Qing Cui, and Jun Zhou. Sharp: Synthesizing high-quality aligned reasoning problems for large reasoning models reinforcement learning, 2025. [32]Yunxuan Ma, Siqiang Wang, Zhijian Duan, Yukun Cheng, and Xiaotie Deng. Learning truthful mechanisms without discretization.arXivpreprintarXiv:2506.22911, 2025. [33]Zehang Deng, Yongjian Guo, Changzhou Han, Wanlun Ma, Junwu Xiong, Sheng Wen, and Yang Xiang. Ai agents under threat: A survey of key security challenges and future pathways.ACMComputingSurveys, 57(7):1–36, 2025. [34]Li Jiang, Yusen Wu, Junwu Xiong, Jingqing Ruan, Yichuan Daniel Ding, Qingpei Guo, Zujie Wen, Jun Zhou, and Xiaotie Deng. Hummer: Towards limited competitive preference dataset. InFirstConferenceonLanguage Modeling(COLM), 2024. [35]Tianyu Cui, Yanling Wang, Chuanpu Fu, Yong Xiao, Sijia Li, Xinhao Deng, Yunpeng Liu, Qinglin Zhang, Ziyi Qiu, Peiyang Li, Zhixing Tan, Junwu Xiong, Xinyu Kong, Zujie Wen, Ke Xu, and Qi Li. Risk taxonomy, mitigation, and assessment benchmarks of large language model systems, 2024. 23

Instant research discovery