Paper deep dive
NavGen: Visual Generative Models as a Scalable Data Engine for Embodied 3D Navigation
Xijie Huang, Yongyang Wan, Chengbin Dong, Zimo Ding, Mo Zhu, Yijin Wang, Zhiyang Liu, Fei Gao, Yuze Wu, Xin Zhou
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/28/2026, 3:37:21 AM
Summary
This paper introduces NavGen, a scalable data engine for embodied 3D navigation that utilizes high-fidelity visual generative models to create diverse training datasets. The pipeline generates approximately 400K vision-language navigation (VLN) episodes using text-to-video models (Wan2.2) and style diversification techniques to cover long-tail scenarios. The resulting dataset outperforms existing simulated and real-world datasets in diversity and navigation performance, enabling a UAV model to achieve a 75% zero-shot success rate in real-world environments.
Entities (10)
Relation Signals (8)
NavGen → produces → 400K navigation episodes
confidence 98% · The resulting dataset contains approximately 400K navigation episodes.
NavGen → uses → Wan2.2 14B
confidence 95% · We first use the open-source, low-cost Wan2.2 T2V 14B ... to produce 300K episodes
NavGen → uses → Qwen3.6-27B
confidence 95% · we use Qwen3.6-27B to re-annotate the generated videos as training prompts
NavGen → enables → UAV
confidence 90% · deploy the trained model ... to real-world flying experiments. The final model achieves a 75% success rate
NavGen → outperforms → existing datasets
confidence 90% · outperforming those trained on existing datasets
NavGen → uses → NavDreamer
confidence 90% · we adopt the video-prediction and action-decoding pipeline of NavDreamer
NavGen → uses → $\pi^3$
confidence 90% · we first use Pi3 to decode the overall motion direction
NavGen → uses → MoGe-2
confidence 90% · original videos are converted into depth videos using MoGe-2
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:General-purpose robot models increasingly rely on large and diverse datasets. For embodied 3D navigation, however, existing data sources face a fundamental trade-off: simulated data can be generated at scale but often suffer from the visual sim-to-real gap, whereas real-world flight data provide realistic observations but are costly to collect. This paper studies another direction: the use of high-fidelity visual generative models as scalable data engines for embodied 3D navigation. We introduce NavGen, a text-to-video data generation pipeline that produces diverse vision-language navigation (VLN) episodes across indoor and outdoor scenes. We also propose a style-diversification method that scales up long-tail data that are difficult and costly to collect. The resulting dataset contains approximately 400K navigation episodes. We evaluate our dataset against existing UAV navigation datasets across multiple metrics, and find that the model trained on our data generally improves with scale, outperforming those trained on existing datasets. To validate real-world transferability, we deploy the trained model in world-action-model paradigm to real-world flying experiments. The final model achieves a 75\% success rate across different navigation tasks and environments.
Tags
Links
- Source: https://arxiv.org/abs/2609.30770v1
- Canonical: https://arxiv.org/abs/2609.30770v1
Trouble viewing inline? Open PDF directly →
Full Text
43,418 characters extracted from source content.
Expand or collapse full text
NavGen: Visual Generative Models as a Scalable Data Engine for Embodied 3D NavigationThanks: Zhejiang University.Thanks: Differential Robotics.Thanks: Corresponding author: wuyuze000@zju.edu.cn Xijie Huang Yongyang Wan Chengbin Dong Zimo Ding Mo Zhu Yijin Wang Zhiyang Liu Fei Gao Yuze Wu Affiliation: Project page: https://xinjiu612.github.io/NavGen Xin Zhou Abstract General-purpose robot models increasingly rely on large and diverse datasets. For embodied 3D navigation, however, existing data sources face a fundamental trade-off: simulated data can be generated at scale but often suffer from the visual sim-to-real gap, whereas real-world flight data provide realistic observations but are costly to collect. This paper studies another direction: the use of high-fidelity visual generative models as scalable data engines for embodied 3D navigation. We introduce NavGen, a text-to-video data generation pipeline that produces diverse vision-language navigation (VLN) episodes across indoor and outdoor scenes. We also propose a style-diversification method that scales up long-tail data that are difficult and costly to collect. The resulting dataset contains approximately 400K navigation episodes. We evaluate our dataset against existing UAV navigation datasets across multiple metrics, and find that the model trained on our data generally improves with scale, outperforming those trained on existing datasets. To validate real-world transferability, we deploy the trained model in world-action-model paradigm to real-world flying experiments. The final model achieves a 75% success rate across different navigation tasks and environments. Fig. 1: Overview of the NavGen data generation pipeline. The upper left part uses the open-source model to generate 300K common task samples. The upper right part uses the more powerful but closed-source model to generate 10K long-tail task samples with a style-diversification pipeline to scale up to 100K. I INTRODUCTION Embodied 3D navigation requires an agent to interpret language and visual observations while producing temporally coherent motion in complex environments. Its generalization, however, is ultimately constrained by the range of environments and behaviors represented in the training data. Building such data at scale remains difficult because navigation is not merely a problem of collecting more trajectories: the data must also be visually realistic, behaviorally rich, and diverse in scenes, objects, and tasks. But existing data sources force us to trade one of these properties for another. Simulation platforms such as AirSim [1], Matterport3D [2], and Unreal Engine can produce many trajectories, but their appearance and discrete action abstractions introduce visual and dynamic gaps with respect to the physical world [3, 4, 5]. Real-world flight data provide more realistic observations, but collecting them is costly and difficult to scale [6]; challenging navigation tasks can also put the aircraft at risk of crashing. OpenFly [5] proposes an automated pipeline that can continuously collect trajectories at scale. However, when the underlying scenes, objects, and task configurations are limited, simply increasing the number of trajectories can result in highly homogeneous data, providing limited additional supervision for improving the model’s generalization ability. Overall, existing pipelines do not yet provide a practical way to collect navigation data that is simultaneously realistic, diverse, and scalable. Given this tension, we ask whether visual generative models can play a broader role in embodied intelligence: not only serving as policies, but acting as scalable engines for producing navigational experience. Trained on massive real-world data, these models can synthesize high-fidelity indoor and outdoor scenes, vary object appearances and visual styles, and express camera motion through natural-language or visual conditions. In this paper, we present NavGen, a scalable data engine that composes visual generative models to produce navigation data. We show that navigation performance consistently improves as the amount of generated data grows. We first use the open-source, low-cost Wan2.2 T2V 14B [7] with a diverse text-to-video prompting strategy to produce 300K episodes covering common embodied navigation tasks. For long-tail maneuvers beyond the open-source model’s capability, such as orbiting a target or passing through narrow spaces, we instead use a more powerful closed-source model [8], to generate 10K high-quality samples. To enrich these long-tail data, we scale these samples to 100K through a style-diversification module that combines an image-editing model [9] with a reference-conditioned video-to-video model [10]. All episodes then undergo motion rebalancing, video filtering, and prompt re-annotation. The resulting 400K-episode dataset provides scale, diversity, and long-tail coverage at once. As world action models have emerged as a promising paradigm for embodied AI [11], we adopt the video-prediction and action-decoding pipeline of NavDreamer [12] to evaluate how different training datasets affect navigation performance. Specifically, we train the lightweight Wan2.2 5B video model separately on our dataset and existing embodied 3D navigation datasets. Extensive comparisons show that our dataset is 2.8× more diverse than the strongest existing dataset, has the smallest sim-to-real gap among non-real datasets, and yields a 17% higher navigation progress score than the best baseline. Furthermore, we observe that increasing the scale of our dataset consistently improves the model’s navigation performance. We finally build a low-latency inference system for real-world deployment, and the deployed UAV completes diverse tasks in unseen environments with a 75% zero-shot success rate. Our contributions are: • NavGen, a scalable data engine that composes visual generative models (with motion rebalancing, data diversification, video filtering and re-annotation) into a 400K-episode navigation dataset providing scale, diversity, and long-tail coverage. • A controlled benchmark quantifying the data’s value: 2.8× higher diversity, the smallest sim-to-real gap among non-real datasets, and a 17% higher navigation progress score over the best baseline. • Real-world validation of the data engine: a 47-FPS video generation system deploying the trained model on a physical UAV with a 75% zero-shot success rate in unseen environments. • Our dataset, model weights, and code will be made publicly available upon acceptance. I RELATED WORK I-A Embodied 3D Navigation Datasets Embodied 3D navigation has benefited from large simulated environments such as Matterport3D, AirSim, and Unreal Engine [2, 1]. TravelUAV [3] collects trajectories in AirSim and Unreal Engine environments, while IndoorUAV [4] extends this direction to indoor navigation with human-operated trajectories in Matterport3D assets. These datasets rely on simulation or manual piloting, which limits their realism, scalability, and diversity. OpenFly [5] proposes an automated collection pipeline using AirSim, GTA5, 3DGS [13] scenes, and Google Earth, reducing human operation cost but still depending on simulator appearance and manually designed task rules. UAV-Flow [6] instead collects real-world UAV trajectories, but its scenes are concentrated in limited campus environments. VLA-AN [14] uses navigation data collected in 3DGS environments to mitigate the sim-to-real gap. FlyMirage [15] further explores automated collection in generated 3DGS scenes with a world generation model, but it still requires explicit task design and is affected by blur and artifacts of 3DGS reconstructions. I-B Visual Generative Models for Embodied AI Recent progress in image and video generation models has motivated their use as data sources for embodied AI. Early works mainly focus on data augmentation: ROSIE [16] uses a text-to-image diffusion model to paint new objects, backgrounds, and distractors; RoboTransfer extends appearance editing to video and multi-view settings [17]; and RoboCurate [18] couples generation with verification by filtering generated trajectories through simulation replay. These methods improve visual diversity but remain tied to the initial demonstration distribution. Another line of work synthesizes complete embodied trajectories. DreamGen [19] adapts image-to-video models to robot embodiments, but it requires manually provided high-quality and diverse initial frames. X-Humanoid [20] and Lucid-XR [21] generate large-scale robot-centric visual data for humanoid or manipulation policies. Although these works demonstrate the potential of visual generative models for embodied AI, a systematic framework for generating diverse and high-quality embodied datasets through different generation strategies is still lacking. Moreover, how to transfer this data-generation paradigm to embodied navigation remains underexplored. I METHOD We focus on building a balanced data-generation pipeline. In Section I-A, we describe how we maintain sample diversity at the 300K scale. In Section I-B, we propose a motion rebalancing strategy to correct the distribution bias of video generation models, and scale up 10K long-tail samples from a closed-source model to 100K through style diversification. In Section I-C, we use Qwen3.6-27B to re-annotate the generated videos as training prompts. Finally, we design a low-latency inference system that achieves 47 FPS and introduce our training system design in Section I-D. I-A Diverse Embodied 3D Navigation Descriptions for Text-to-Video Generation Automatically generating diverse navigation task descriptions for text-to-video generation is a core component of our pipeline. A straightforward solution is to use a vision-language model (VLM) to generate prompts. However, when the dataset size reaches hundreds of thousands of samples, VLM responses tend to become increasingly homogeneous. To ensure sufficient diversity in our dataset, we start from an object dataset [22] containing 1M objects, filter out uncommon objects, and obtain 300K object targets. To keep the motion distribution balanced, we assign the 300K samples to a predefined set of motion categories. Given the pre-designed motion prompt and the target object, we use Qwen3.6-27B as an embodied 3D navigation expert to imagine detailed navigation events around the object, including the environment style, background objects, spatial layout, and target-related context. This produces an automatically generated embodied 3D navigation text dataset, while ensuring object-level diversity across different samples, as shown in Fig. 1. With the generated navigation descriptions, we use the step-distilled Wan2.2 14B T2V [23] model to generate videos with four diffusion steps. Each sample takes approximately 36 seconds to generate on an A100 GPU. I-B Motion Rebalancing and Style Diversification I-B1 Motion Rebalancing We observe that video generation models may not strictly follow the designed prompts, leading to an imbalanced motion distribution that introduces bias into the trained model. To address this, we propose a motion rebalancing strategy. Specifically, we first use Pi3 [24] to decode the overall motion direction of each generated video and assign a motion label to every sample. We then identify the imbalanced motion categories and divide them into two major types: left-right imbalance and forward-backward imbalance. For left-right imbalance, we use horizontal mirroring to rebalance the distribution. For forward-backward imbalance, we use temporal reversal to increase the underrepresented motion direction. In this way, we improve the balance of the generated motion distribution without requiring additional video generation. I-B2 Style Diversification For challenging long-tail tasks, we use the Seedance API to generate 10K high-quality samples, including difficult behaviors such as passing through narrow spaces and orbiting around targets. These tasks require stronger temporal consistency and more accurate motion control from the video generation model. To scale up these samples, we extract the first frame of each video and use Qwen3.6-27B to generate style-diversification descriptions that preserve the main object shape while changing the surrounding objects, environment style, and scene appearance, as shown in Fig. 1. We then use Qwen-Image-Edit [9] to edit the first frames according to the generated diversification descriptions. Finally, we use the LTX-2.3 [10] ICLoraPipeline to generate new videos conditioned on the original reference videos. Before this step, the original videos are converted into depth videos using MoGe-2 [25], which provides structural guidance for the generation process. This pipeline allows us to automatically scale the original 10K long-tail samples to 100K samples, and we validate its effectiveness in our experiments. I-C Video Filtering and Description Since video generation models may occasionally hallucinate and produce content that violates consistency, we filter the generated videos with an automatic evaluation pipeline (Section IV-B2). Specifically, we discard samples whose VBench [26] score falls below a predefined threshold or whose navigation grade is below L2, and regenerate them from newly sampled navigation descriptions. This step removes low-quality and inconsistent videos and improves the overall reliability of the dataset. Since the generated videos may not perfectly follow the original detailed prompts, generating accurate video-level event descriptions is essential for training. We use Qwen3.6-27B to describe each generated video by uniformly downsampling it to 11 frames and using them as input. Because VLMs often have limited spatial-direction understanding, we additionally provide the Pi3-decoded overall motion direction as a motion cue for VLMs, which improves the quality and consistency of the generated descriptions. TABLE I: Inference latency breakdown of the proposed system. T5, Enc., DiT, and Dec. denote text-encoding, VAE-encoding, diffusion-transformer, and VAE-decoding latency, respectively. Variant T5 Enc. DiT Dec. Total Speedup (s) (s) (s) (s) (s) (×) Original 0.116 0.189 156.583 13.520 171.568 1.00× Sparse 0.165 0.057 9.585 1.421 12.250 14.00× + Distilled 0.058 0.051 0.553 1.040 1.751 97.98× + Distilled + TAE 0.058 0.049 0.510 0.055 0.702 244.40× I-D Training System Design To evaluate our dataset and test its sim-to-real transferability, we adopt the video-prediction and action-decoding pipeline of NavDreamer [12] with the lightweight Wan2.2 5B model as the base model. For real-world deployment, we first use a bidirectional long-video generation paradigm and train the model on our dataset through supervised fine-tuning (Sections I-D1 and I-D2). We then accelerate inference with a distillation pipeline and a lightweight VAE (Section I-D3). I-D1 Bidirectional Long-Video Generation via Clean-Prefix History Conditioning Although the autoregressive paradigm supports unbounded continuous video generation, we find it yields lower task success rates than the bidirectional paradigm. However, most navigation tasks cannot be completed within a single clip, and conditioning each clip independently on only the current anchor frame causes inconsistent motion dynamics across successive clips. We address both issues with a unified image-to-video (I2V) and video-to-video (V2V) training requiring no architectural change as shown in Fig. 2. During training, the first k latent tokens are replaced with their clean ground-truth observations, where k∼Uniform1,…,Nk \1,…,N\ is sampled each step independently. Those latent tokens are excluded from the loss, and the remaining tokens are denoised under standard full bidirectional attention. Setting k=1k=1 recovers the I2V objective, while k>1k>1 trains the model to continue from k−1k-1 clean history latents, i.e., V2V. A small noise augmentation with probability paugp_aug is applied to history frames to prevent over-reliance on a perfectly clean prefix. Because k is randomized, a single model covers both the I2V and V2V. I-D2 Supervised Fine-Tuning Embodied 3D navigation is typically a long-horizon task, where adjacent frames are highly redundant and high-resolution generation is not always necessary. We find that a resolution of 640×384640× 384 is sufficient for most navigation behaviors, while substantially reducing the computational cost compared with 1280×7201280× 720 generation. Furthermore, we find that a video model trained on sparse frames can efficiently represent long-horizon events using fewer frames. Therefore, we first train a Wan2.2-5B TI2V model at 640×384640× 384 resolution with 33 frames. The training videos are resized and temporally downsampled from the 81-frame outputs generated by the Wan2.2-14B T2V model. This design preserves the main task completion process while reducing both spatial and temporal redundancy. The total generation time is reduced from 171.57s to 12.25s. We first perform supervised fine-tuning (SFT) to adapt the student model to the generated navigation data. Following the flow-matching formulation, we construct the noisy latent zt=(1−t)z0+tϵz_t=(1-t)z_0+tε with the flow target ut=ϵ−z0u_t=ε-z_0, where z0z_0 is the clean video latent, c is the text condition, t∈[0,1]t∈[0,1] is the diffusion timestep, and ϵ∼(0,I)ε (0,I) is Gaussian noise. The SFT objective is: ℒSFT=z0,c,t,ϵ[‖fθ(zt,t,c)−ut‖22]L_SFT=E_z_0,c,t,ε [ \|f_θ(z_t,t,c)-u_t \|_2^2 ] (1) where fθf_θ is the student denoising network. This stage provides a strong initialization before step distillation. Fig. 2: Bidirectional long-video generation via clean-prefix history conditioning. I-D3 Step Distillation After SFT, we reduce the number of sampling steps through continuous-time consistency distillation. We warm up the model for 2.5K steps using the simplified continuous-time consistency model objective (sCM) [27]. Under the TrigFlow parameterization zt=cos(t)z0+sin(t)ϵz_t= (t)z_0+ (t)ε with t∈[0,π/2]t∈[0,π/2], the sCM training gradient is: ∇θℒsCM=∇θzt,t,c[w(t)fθ⊤(zt,t,c)dfθ−(zt,t,c)dt] _θL_sCM= _θE_z_t,t,c [w(t)f_θ (z_t,t,c) df_θ^-(z_t,t,c)dt ] (2) where fθf_θ is the consistency function parameterized by the student network, fθ−f_θ^- is the stop-gradient target, and w(t)w(t) is the timestep-dependent weighting function. In practice, this stage serves as a warmup to stabilize the student before stronger distribution-level distillation. We then apply DMD2-style distribution matching distillation [28]. The student generator GθG_θ produces a video latent z^0=Gθ(zT,c) z_0=G_θ(z_T,c) from noise and condition c. The student is trained by matching its generated distribution to the teacher’s via the velocity difference: ∇θℒDMD=t,z^0,c,ϵ[w(t)(vψ(zt,t,c)−vϕ(zt,t,c))∂θz^0] _θL_DMD=E_t, z_0,c,ε [w(t) (v_ψ(z_t,t,c)-v_φ(z_t,t,c) ) _θ z_0 ] (3) where vψv_ψ and vϕv_φ are the teacher and fake score models (trained on student samples), respectively, and ztz_t is the noised latent. The fake model is optimized via flow matching: ℒfake=z^0,t,c,ϵ[‖vϕ(zt,t,c)−(ϵ−z^0)‖22]L_fake=E_ z_0,t,c,ε [ \|v_φ(z_t,t,c)-(ε- z_0) \|_2^2 ] (4) To improve visual quality, we add an adversarial loss. Following DMD2, the discriminator DηD_η operates on diffused samples via the forward process F(⋅,t)F(·,t): ℒD=−z0,t[logDη(F(z0,t))]−z^0,t[log(1−Dη(F(z^0,t)))] _D=-E_z_0,t [ D_η(F(z_0,t)) ]-E_ z_0,t [ (1-D_η(F( z_0,t)) ) ] (5) and the generator-side GAN loss is: ℒGAN=−z^0,t[logDη(F(z^0,t))]L_GAN=-E_ z_0,t [ D_η(F( z_0,t)) ] (6) The final student objective is ℒstudent=ℒDMD+λGANℒGANL_student=L_DMD+ _GANL_GAN. Both the student and fake models are initialized from the sCM-warmed checkpoint, while the teacher uses the SFT checkpoint. As shown in Table I, the Sparse processing at a reduced resolution of 640×384640× 384 with 33 frames instead of the full 1280×7201280× 720 with 121 frames yields a 14×14× speedup. Step distillation then reduces the denoising process from 30 steps to 4 and removes classifier-free guidance (CFG), achieving a 17×17× reduction in DiT (Diffusion Transformer) computation. At this point, the VAE becomes the bottleneck: encoding and decoding together need to take about 1 second, comparable to the entire 0.5-second DiT pass. We address this by replacing the standard VAE with a lightweight alternative [29], reducing the combined encode and decode time to roughly 0.1 seconds. With these three optimizations, the full inference pipeline runs in 0.7 seconds per clip, corresponding to 47 FPS. Training hyperparameters are summarized in Table I. TABLE I: Training hyperparameters across the three-stage pipeline. Parameter Stage 1: SFT Stage 2: sCM Stage 3: DMD2 LoRA rank 64 64 64 LoRA alpha 64 64 64 Global batch size 32 32 32 Optimizer AdamW AdamW AdamW Generator LR 1e-4 1e-5 2e-5 Critic LR – – 4e-6 Training Steps 12500 2500 5000 GPU Usage 16 NVIDIA A100 16 NVIDIA A100 16 NVIDIA A100 Fig. 3: Waypoints decoding from RGB frames. IV EXPERIMENTS IV-A Dataset Analysis To quantitatively analyze the motion characteristics of the 400K generated navigation samples, we employ an action-decoding pipeline [12], which is consistent with our real-world deployment. Given a generated RGB video, we first use Pi3 [24] to estimate the camera trajectory and geometry as shown in Fig 3. However, Pi3 suffers from scale ambiguity, particularly in outdoor environments. To recover precise metric-scale trajectories, we introduce a metric scale correction module using MoGe-2 [25]. Specifically, we compute a scale factor by aligning the metric depth estimated by MoGe-2 with the depth rendered from the Pi3 point cloud at corresponding viewpoints. This scale factor is then applied to the Pi3 trajectory, yielding physically accurate motion sequences. Fig. 4 summarizes the resulting dataset statistics. Unlike prior datasets that often describe navigation with discrete action labels, our video-generated trajectories provide continuous egocentric motion, making them more suitable for modeling fine-grained aerial behaviors. Since video generation models may occasionally deviate from the initially specified motion trends, the resulting dataset can become biased toward certain motion directions. Our motion rebalancing strategy can correct this distributional imbalance efficiently. The dataset is mainly composed of short-range trajectories, similar to UAV-Flow [6], but still exhibits diverse yaw rotations and vertical movements, reflecting broad motion coverage across different navigation scenarios. Fig. 4: Qualitative analysis of our dataset. Figure (a) is the motion distribution. Figure (b) is the word cloud of the text description. Figure (c), (d), (e) are the analyses of the specific dimension. TABLE I: VBench grouped scores at the best-performing checkpoint per dataset (bold = best, underline = second best per row). Aesthetic: Aesthetic Quality; Smoothness: Motion Smoothness and Temporal Flickering; Sem. Cons.: Subject and Background Consistency; I2V Cons.: alignment to the input first frame, reported separately for Subject (S) and Background (B). Metric TravelUAV [3] IndoorUAV [4] OpenFly [5] UAV-Flow [6] Ours Aesthetic 0.471±0.0570.471± 0.057 0.546±0.0530.546± 0.053 0.551±0.0520.551± 0.052 0.549±0.0470.549± 0.047 0.586±0.0370.586± 0.037 Smoothness 0.888±0.0260.888± 0.026 0.897±0.0210.897± 0.021 0.927±0.0220.927± 0.022 0.987±0.0060.987± 0.006 0.952±0.0160.952± 0.016 Sem. Cons. 0.710±0.0480.710± 0.048 0.858±0.0560.858± 0.056 0.883±0.0370.883± 0.037 0.979±0.0080.979± 0.008 0.952±0.0130.952± 0.013 I2V Cons. S 0.776±0.0510.776± 0.051 0.913±0.0440.913± 0.044 0.912±0.0620.912± 0.062 0.985±0.0060.985± 0.006 0.977±0.0100.977± 0.010 I2V Cons. B 0.797±0.0440.797± 0.044 0.930±0.0290.930± 0.029 0.923±0.0410.923± 0.041 0.974±0.0060.974± 0.006 0.973±0.0070.973± 0.007 Avg 0.7284±0.0320.7284± 0.032 0.8288±0.0340.8288± 0.034 0.8392±0.0320.8392± 0.032 0.8948±0.0110.8948± 0.011 0.888±0.0120.888± 0.012 Fig. 5: Dataset diversity analysis. Figure (a) and (b) show the Vendi Score and FD CLIP. Figure (c) is the visualization of the clip feature distribution of the datasets. IV-B Datasets Benchmark IV-B1 Datasets Diversity and Sim-to-real Benchmark We evaluate our dataset against existing UAV-oriented datasets along two complementary axes: semantic diversity and sim-to-real distribution alignment. For diversity, we adopt the Vendi Score [30], which quantifies the effective number of semantically distinct modes via the matrix exponential entropy of the CLIP [31] feature similarity matrix. As shown in Fig. 5, our dataset achieves a Vendi Score of 47.6, surpassing the next highest competitor IndoorUAV (16.8) by a factor of 2.8×2.8×, indicating substantially richer coverage across scene types, lighting conditions, and viewpoints. For sim-to-real alignment, we compute the Fréchet Distance [32] over CLIP features (FDCLIP) between each dataset and a held-out set of 1000 real-world deployment images. Our dataset achieves FDCLIP = 0.806, the lowest among all non-real-world datasets, and approaching the score of the real-flight dataset UAV-Flow (0.691). The Principal Component Analysis (PCA) projection in Fig. 5 further confirms this qualitatively: real-world samples (★ ) are visually embedded within our dataset’s distribution, while competing datasets occupy largely disjoint regions of the feature space. Together, these results demonstrate that our dataset uniquely combines broad semantic diversity with strong real-world distributional alignment. IV-B2 Video Model Trained With Different Datasets Benchmark Visual Quality Evaluation To validate that our dataset can bridge the sim-to-real gap and generalize to unseen real-world environments, we fine-tune a video generation model, Wan2.2 5B, separately on each of the five datasets. Since the datasets have different sizes and each dataset contains variable-length videos, we adopt an autoregressive training paradigm and group videos with similar lengths into separate training buckets before training [33]. We collect 1000 real-world text-image pairs as an evaluation dataset and first evaluate the resulting checkpoints using VBench [26]. As shown in Table I, our dataset achieves the highest VBench average among all non-real-world datasets at 0.8660.866, outperforming IndoorUAV (0.8060.806), OpenFly (0.8200.820), and TravelUAV (0.7140.714). Notably, ours closely approaches UAV-Flow (0.8740.874), which is a purely real-world aerial dataset. This result suggests that our dataset contains sufficient visual realism to produce generated videos of perceptual quality. Navigation Ability Evaluation While VBench measures visual quality, it cannot assess navigation capabilities. Unlike previous simulation-based benchmarks (e.g., R2R [34]) that suffer from sim-to-real gaps, we propose a real-world navigation benchmark with 1,000 authentic text-image pairs. Leveraging the world action model paradigm, we generate videos to predict task success rates, which closely align with real-world execution as validated in Section IV-D2. Following recent practices [35], we employ Qwen3.6-27B to automatically assess navigation performance using a four-level ordinal rubric: • L0: No usable navigation. • L1: Usable navigation, but the prompted motion is not fully followed. • L2: The prompted motion is followed, but the target view is not reached. • L3: The prompted motion is followed and the target view is reached. We compute the average navigation score by mapping L0–L3 to integer values 0–3 and averaging over the 1000 videos. To validate the reliability of this automatic scoring pipeline, we compare its outputs against ratings from a human expert on the same set of videos. The pipeline’s score exactly matches the human expert’s rating on 56.9% of videos. An additional 32.8% differ by exactly one level, bringing the total agreement within one level to 89.7%. These results provide an initial assessment of the automatic evaluator’s agreement with human ratings. As shown in Fig. 6, training on different datasets produces very different results. The progress score is the average over the four levels. UAV-Flow stays below 30%, as its real-flight data are collected only on a university campus with limited diversity. TravelUAV reaches only around 36% due to the severe sim-to-real gap. IndoorUAV improves quickly at first but saturates at around 45% within the first epoch. OpenFly is the largest comparison dataset with 100k videos, but its homogeneous data makes performance fluctuate between 33% and 46% with no clear gain from more training. In contrast, our dataset improves steadily under both paradigms, reaching around 63% (autoregressive) and 79% (bidirectional) at one epoch, far above all comparison datasets. This shows that our pipeline can generate large-scale, high-fidelity, and diverse datasets that generally improve embodied navigation capabilities. Fig. 6: Navigation performance scaling with dataset size. The ar and bi mean autoregressive and bidirectional, respectively. Fig. 7: Ablation study on style diversification. Performance comparison between training on original 10K samples versus 100K diversified samples on unseen real-world tasks. IV-C Ablation Study IV-C1 Motion Rebalancing We conduct an ablation study under two settings: training on the original data without motion rebalancing and training on the motion-rebalanced data. We train separate models under identical conditions and evaluate the generated videos using the evaluation pipeline described above. The results show that motion rebalancing improves the navigation success rate by approximately 15%, showing its effectiveness in reducing motion-distribution bias and improving policy performance. IV-C2 Style Diversification To validate the effectiveness of style diversification, we train on the diversified 100K dataset versus the original 10K dataset. We assess generalization on 100 unseen real-world test tasks. As shown in Fig. 7, training without diversification plateaus at 78.1% progress score after 50K samples and gains nothing further. With diversification, performance continues to climb, reaching 93.75% at 100K samples, a 15.6-point improvement. The result shows that the original 10K data is simply too narrow for the model to generalize. Diversification breaks this ceiling by broadening the training distribution. IV-D Real-World Experiment To evaluate whether our dataset can alleviate the sim-to-real gap and generalize to unseen environments, we conduct several experiments. TABLE IV: Inference time of each stage in our pipeline. Stage Prompt rewrite Video generation Action decode Ego-planner Time 1 s 0.7 s 0.5 s 10 ms IV-D1 Real-World Deployment We run video model inference on one A100 server. The VLM, Qwen3.6-27B, is deployed on two A100 servers and is used for prompt rewriting. Real-time frames are sent to the server via TCP to maintain a history frame buffer. When a human command is triggered, the VLM rewrites the command into the style of the training datasets to reduce prompt distribution mismatch. The rewritten prompt is then sent to the video model to generate a full-pixel video, which is decoded into a list of waypoints by our action decoder. The drone platform is equipped with a DJI Osmo Action 5 Pro for RGB observations and a LiDAR for pose estimation. The decoded discrete waypoints are used to initialize a trajectory, and Ego-Planner [36] optimizes this trajectory using collision information to achieve collision-free execution. The total inference time is shown in Table IV. Fig. 8: Short-horizon navigation experiments in real-world environments. Fig. 9: Long-horizon navigation demonstration. IV-D2 Short Horizon Experiment We evaluate our method on 20 real-world tasks (5 trials each), achieving a 75% overall success rate. Failures primarily occur when targets are too small or lack color contrast with the background. As shown in Fig.8, these trials span diverse indoor and outdoor scenes, demonstrating that our data pipeline effectively bridges the sim-to-real gap and generalizes well. IV-D3 Long Horizon Experiment We evaluate our long-video generation pipeline on long-horizon navigation tasks. As shown in Fig. 9, some tasks exceed the length of a single clip. We address this by using previously observed RGB frames retrieved from the onboard observation buffer using odometry as context to resume generation, ensuring new segments inherit the preceding state and maintain motion continuity. Notably, the success rate remains stable when the target is clearly visible in the first frame, but drops to 50% when the target is ambiguous. These results show our pipeline effectively handles long-horizon tasks with a reasonable success rate under the bidirectional training paradigm. V CONCLUSIONS In this paper, we propose NavGen, an embodied 3D navigation dataset generation pipeline based on visual generative models. We generate 300K diverse navigation episodes using a scalable open-source video model, and produce an additional 10K high-quality long-tail samples with a more powerful but closed-source model, which are then expanded to 100K through style diversification, yielding 400K episodes in total. Motion rebalancing is applied throughout to correct distribution biases in the generated data. Benchmark comparisons against existing embodied 3D navigation datasets show that models trained on our data consistently outperform those trained on prior datasets on both video-quality metrics and navigation success rate. To validate real-world transferability, we carry out zero-shot deployment across diverse indoor and outdoor environments, achieving a task success rate of approximately 75%. Together, these results demonstrate that large visual generative models are effective scalable data engines for embodied navigation. References [1] S. Shah, D. Dey, C. Lovett, and A. Kapoor (2017) Airsim: high-fidelity visual and physical simulation for autonomous vehicles. In Field and service robotics: Results of the 11th international conference, p. 621–635. Cited by: §I, §I-A. [2] A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y. Zhang (2017) Matterport3D: learning from rgb-d data in indoor environments. International Conference on 3D Vision (3DV). Cited by: §I, §I-A. [3] X. Wang, D. Yang, Z. Wang, H. Kwan, J. Chen, H. Li, Y. Liao, S. Liu, et al. (2025) Towards realistic uav vision-language navigation: platform, benchmark, and methodology. In International Conference on Learning Representations, Vol. 2025, p. 7292–7310. Cited by: §I, §I-A, TABLE I. [4] X. Liu, Y. Liu, H. Qiu, Y. Qirong, and Z. Lian (2026) Indooruav: benchmarking vision-language uav navigation in continuous indoor environments. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 23864–23872. Cited by: §I, §I-A, TABLE I. [5] Y. Gao, C. Li, Z. You, J. Liu, L. Zhen, P. Chen, Q. Chen, Z. Tang, L. Wang, Y. Tang, et al. (2026) OpenFly: a comprehensive platform for aerial vision-language navigation. In International Conference on Learning Representations, Vol. 2026, p. 101858–101871. Cited by: §I, §I-A, TABLE I. [6] X. Wang, D. Yang, Y. Liao, W. Zheng, B. Dai, H. Li, S. Liu, et al. (2026) Uav-flow colosseo: a real-world benchmark for flying-on-a-word uav imitation learning. Advances in Neural Information Processing Systems 38. Cited by: §I, §I-A, §IV-A, TABLE I. [7] T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, et al. (2025) Wan: open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. Cited by: §I. [8] ByteDance Seed (2026) Seedance 2.0. Note: https://seed.bytedance.com/en/seedance2_0Accessed: Aug. 31, 2026 Cited by: §I. [9] C. Wu, J. Li, J. Zhou, J. Lin, K. Gao, K. Yan, S. Yin, S. Bai, X. Xu, Y. Chen, et al. (2025) Qwen-image technical report. arXiv preprint arXiv:2508.02324. Cited by: §I, §I-B2. [10] Y. HaCohen, B. Brazowski, N. Chiprut, Y. Bitterman, A. Kvochko, A. Berkowitz, D. Shalem, D. Lifschitz, D. Moshe, E. Porat, E. Richardson, G. Shiran, I. Chachy, J. Chetboun, M. Finkelson, M. Kupchick, N. Zabari, N. Guetta, N. Kotler, O. Bibi, O. Gordon, P. Panet, R. Benita, S. Armon, V. Kulikov, Y. Inger, Y. Shiftan, Z. Melumian, and Z. Farbman (2026) LTX-2: efficient joint audio-visual foundation model. arXiv preprint arXiv:2601.03233. Cited by: §I, §I-B2. [11] L. Li, Q. Zhang, Y. Luo, S. Yang, R. Wang, F. Han, M. Yu, Z. Gao, N. Xue, X. Zhu, et al. (2026) Causal world modeling for robot control. arXiv preprint arXiv:2601.21998. Cited by: §I. [12] X. Huang, W. Gai, T. Wu, C. Wang, Q. Zheng, Z. Liu, X. Zhou, Y. Wu, and F. Gao (2026) Navdreamer: video models as zero-shot 3d navigators. IEEE Robotics and Automation Letters. Cited by: §I, §I-D, §IV-A. [13] B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis (2023) 3d gaussian splatting for real-time radiance field rendering.. ACM Trans. Graph. 42 (4), p. 139–1. Cited by: §I-A. [14] Y. Wu, M. Zhu, X. Li, Y. Du, Y. Fan, W. Li, Z. Han, X. Zhou, and F. Gao (2025) Vla-an: an efficient and onboard vision-language-action framework for aerial navigation in complex environments. arXiv preprint arXiv:2512.15258. Cited by: §I-A. [15] J. Li, X. Huang, Z. Wang, Y. Wang, W. Ge, Q. He, M. Zhu, F. Gao, Y. Wu, and X. Zhou (2026) FlyMirage: a fully automated generation pipeline for diverse and scalable uav flight data via generative world model. arXiv preprint arXiv:2605.19600. Cited by: §I-A. [16] T. Yu, T. Xiao, A. Stone, J. Tompson, A. Brohan, S. Wang, J. Singh, C. Tan, J. Peralta, B. Ichter, et al. (2023) Scaling robot learning with semantically imagined experience. arXiv preprint arXiv:2302.11550. Cited by: §I-B. [17] L. Liu, X. Wang, G. Zhao, K. Li, W. Qin, J. Zhu, J. Qiu, G. Huang, and Z. Su (2026) Robotransfer: controllable geometry-consistent video diffusion for manipulation policy transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 1410–1420. Cited by: §I-B. [18] S. Kim, S. Jang, B. Yoon, D. Kim, J. Won, and J. Shin (2026) RoboCurate: harnessing diversity with action-verified neural trajectory for robot learning. arXiv preprint arXiv:2602.18742. Cited by: §I-B. [19] J. Jang, S. Ye, Z. Lin, J. Xiang, J. Bjorck, Y. Fang, F. Hu, S. Huang, K. Kundalia, Y. Lin, et al. (2025) Dreamgen: unlocking generalization in robot learning through video world models. arXiv preprint arXiv:2505.12705. Cited by: §I-B. [20] P. Yang, H. Ci, Y. Song, and M. Z. Shou (2025) X-humanoid: robotize human videos to generate humanoid videos at scale. arXiv preprint arXiv:2512.04537. Cited by: §I-B. [21] Y. Ravan, A. Rashid, A. Yu, K. McClennen, G. Huh, K. Yang, Z. Yang, Q. Yu, X. Wang, P. Isola, et al. (2026) Lucid-xr: an extended-reality data engine for robotic manipulation. arXiv preprint arXiv:2605.00244. Cited by: §I-B. [22] T. Luo, C. Rockwell, H. Lee, and J. Johnson (2023) Scalable 3d captioning with pretrained models. Advances in Neural Information Processing Systems 36, p. 75307–75337. Cited by: §I-A. [23] L. Contributors (2025) LightX2V: light video generation inference framework. GitHub. Note: https://github.com/ModelTC/lightx2v Cited by: §I-A. [24] Y. Wang, J. Zhou, H. Zhu, W. Chang, Y. Zhou, Z. Li, J. Chen, J. Pang, C. Shen, and T. He (2026) π3π^3: permutation-equivariant visual geometry learning. In International Conference on Learning Representations, Cited by: §I-B1, §IV-A. [25] R. Wang, S. Xu, Y. Dong, Y. Deng, J. Xiang, Z. Lv, G. Sun, X. Tong, and J. Yang (2026) Moge-2: accurate monocular geometry with metric scale and sharp details. Advances in Neural Information Processing Systems 38, p. 35928–35959. Cited by: §I-B2, §IV-A. [26] Z. Huang, Y. He, J. Yu, F. Zhang, C. Si, Y. Jiang, Y. Zhang, T. Wu, Q. Jin, N. Chanpaisit, et al. (2024) Vbench: comprehensive benchmark suite for video generative models. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 21807–21818. Cited by: §I-C, §IV-B2. [27] C. Lu and Y. Song (2025) Simplifying, stabilizing and scaling continuous-time consistency models. In International Conference on Learning Representations, Vol. 2025, p. 50611–50649. Cited by: §I-D3. [28] T. Yin, M. Gharbi, T. Park, R. Zhang, E. Shechtman, F. Durand, and W. T. Freeman (2024) Improved distribution matching distillation for fast image synthesis. Advances in neural information processing systems 37, p. 47455–47487. Cited by: §I-D3. [29] O. Boer Bohan (2025) TAEHV: tiny autoencoder for hunyuan video. Note: https://github.com/madebyollin/taehv Cited by: §I-D3. [30] D. Friedman and A. B. Dieng (2023) The Vendi score: a diversity evaluation metric for machine learning. Transactions on Machine Learning Research. Cited by: §IV-B1. [31] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. (2021) Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, Cited by: §IV-B1. [32] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter (2017) GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems, Cited by: §IV-B1. [33] S. Yuan, Y. Yin, Z. Li, X. Huang, X. Yang, and L. Yuan (2026) Helios: real real-time long video generation model. arXiv preprint arXiv:2603.04379. Cited by: §IV-B2. [34] P. Anderson, Q. Wu, D. Teney, J. Bruce, M. Johnson, N. Sünderhauf, I. Reid, S. Gould, and A. Van Den Hengel (2018) Vision-and-language navigation: interpreting visually-grounded navigation instructions in real environments. In 2018 IEEE/CVF conference on computer vision and pattern recognition, p. 3674–3683. Cited by: §IV-B2. [35] J. Liu, G. Liu, J. Liang, Z. Yuan, X. Liu, M. Zheng, X. Wu, Q. Wang, M. Xia, X. Wang, et al. (2026) Improving video generation with human feedback. Advances in Neural Information Processing Systems 38, p. 82155–82192. Cited by: §IV-B2. [36] X. Zhou, Z. Wang, H. Ye, C. Xu, and F. Gao (2020) Ego-planner: an esdf-free gradient-based local planner for quadrotors. IEEE Robotics and Automation Letters 6 (2), p. 478–485. Cited by: §IV-D1.