Paper deep dive

From Generation to Simulation: How Far Are World Models from Being True Simulators?

Tong Wang, Huan Deng, Mucheng Yang, Yang He, Xiaohui Kuang, Gang Zhao

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 254

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:With the rapid progress of diffusion models and large-scale video generation, generative world models are increasingly expected to replace traditional simulators, including physics engines, game engines, and reinforcement-learning environments. Yet the remaining distance from generation to simulation lacks a systematic assessment. We present a capability-based study using an external yardstick: eight capabilities of a traditional simulator, namely asset construction, physics engine, interaction, controllability, stability, state feedback, diversity, and evaluation metrics. We trace three main technical routes--latent dynamics, video generation, and joint-embedding prediction--and map exactly 200 representative works published from 2018 to June 2026 onto these capabilities. Our analysis shows that world models have achieved functional substitution in interaction and controllability for specific scenarios, but remain short of traditional simulators in formal guarantees of physical laws, structured state feedback, and reproducible long-horizon evolution. State feedback is the most neglected cross-route shortcoming: only 6 of 163 implementation papers expose a runtime interface for querying entity states or physical parameters. We identify six research directions: formalized physics, a unified action interface, first-class state feedback, long-horizon stability, downstream-utility evaluation, and cross-route hybridization. Project page: this https URL

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

250,752 characters extracted from source content.

Expand or collapse full text

FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 20261 From Generation to Simulation: How Far Are World Models from Being True Simulators? Tong Wang, Huan Deng, Mucheng Yang, Yang He, Xiaohui Kuang, and Gang Zhao Abstract—With the rapid progress of diffusion models and large-scale video generation, generative world models are increasingly expected to replace traditional simulators—physics engines, game engines, and reinforcement-learning environments—and to become a new generation of data-driven, general-purpose simulation environments. Yet how far this road from generation to simulation still runs remains without a systematic answer: existing surveys mostly organize the literature along the architecture, function, or application domain of world models themselves, and thus struggle to characterize their true gap as simulators. In this paper, we present a systematic, capability-based study using an external yardstick, namely the eight capabilities of a traditional simulator: asset construction, physics engine, interaction, controllability, stability, state feedback, diversity, and evaluation metrics. Around this route-independent yardstick, we first trace the evolution of three main technical routes—latent dynamics, video generation, and joint-embedding prediction—and their recent trend toward convergence; we then map exactly 200 representative works from 2018 to June 2026 onto the eight capabilities, systematically comparing the coverage and gaps of each route. Our analysis shows that, aided by advances such as autoregressive-diffusion distillation, latent-action learning, and world-foundation-model platforms, world models have achieved functional substitution in interaction and controllability for specific scenarios, yet remain a critical step short of traditional simulators in formally guaranteeing physical law, in the richness of structured state feedback, and in the reproducibility of long-horizon evolution; among these, state feedback is a shortcoming that cuts across all technical routes yet is the most neglected: only 6 of 163 implementation papers expose a runtime interface for queryable entity states or physical parameters. Accordingly, we look ahead to six research directions—formalized physics, a unified action interface, first-class state feedback, long-horizon stability, downstream-utility evaluation, and cross-route hybridization—charting an evidence-grounded path toward the next generation of true simulators. Project page: https://github.com/AtongWang/world-model-simulators. Index Terms—World models, generative simulation, simulator capabilities, physical plausibility, state feedback, long-horizon stability, embodied AI, autonomous driving, evaluation. ✦ 1 INTRODUCTION T He core idea of a world model is to predict the future state of an environment by learning the physical regularities that govern space and time. Since the pioneering work of Ha and Schmidhuber in 2018, which built an interactive latent environment from a VAE and an RNN [1], world models have undergone several paradigm shifts—from latent dy- namics to video-generative, and further to joint-embedding predictive (JEPA) formulations—with a steadily expanding capability boundary and application scope that now spans game engines [2]–[5], autonomous-driving simulation [6]– [9], robotic manipulation [10]–[14], and open-domain explo- ration [15]–[17]. This trend naturally raises a fundamental question: given that the core capability of a world model is to predict future states, can it replace a traditional simula- tor and become a new generation of data-driven, general- purpose simulation environment? Following the perspective of Cosmos Policy [18], world The authors are with the Institute of Systems Engineering, Academy of Military Sciences, Beijing 100101, China. E-mail: tongwss@foxmail.com; DHuan56@foxmail.com; 4349467ymc@gmail.com; heang@stu.xidian.edu.cn; xhkuang@bupt.edu.cn; bisezhaog@163.com. Corresponding author: Gang Zhao. ORCID iDs: Tong Wang, 0000-0001-6981-916X; Huan Deng, 0000-0002- 5116-1766; Mucheng Yang, 0009-0008-4567-4099; and Gang Zhao, 0009- 0006-8668-777X. Manuscript prepared July 2026. This study analyzes a curated corpus of 200 papers published between 2018 and 2026. models can be divided into two paradigms by function. The policy route uses a world model to directly produce executable actions or value functions that support control decisions; for example, Cosmos Policy, V-JEPA 2-AC [11], and τ 0 -WM [12] unify policy learning and video prediction within a single framework. The generative route instead aims to generate future observations—images, video, 3D structure, or structured state—to serve simulation, planning, or data augmentation [19], [20]. The policy route also models world dynamics, but its output is an action vector or a scalar value, which differs fundamentally from the state-feedback interface of a traditional simulator. This survey therefore focuses primarily on the generative route—works that gen- erate future observations as pixels, tokens, or explicit 3D representations—and examines to what extent they can play the role of a traditional simulator. Formally, the world model we consider is a mapping T : S ×A → Π(S) that learns the distribution of future states given the current state and action, thereby approximating the true environment dynamics [18]. Although world models exhibit potential to surpass tra- ditional simulators in visual realism and learnability, their current state remains markedly short of a rigorous simula- tor. Hallucination is a systematic problem shared by almost all generative models, manifesting as objects appearing or disappearing without cause [2], geometry drifting over long arXiv:2608.23070v1 [cs.AI] 24 Aug 2026 FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 20262 C1Asset ConstructionGAP19% Explicit-3D/4D · Diffusion C7Diversity23% Diffusion · Latent-dyn C6State FeedbackGAP22.5% Occupancy · Diffusion (truth-only 8%) C3InteractionSTRENGTH40% Diffusion · Autoregressive C2Physics EngineGAP17% Diffusion · Latent-dyn C4ControllabilitySTRENGTH62.5% Diffusion · Autoregressive C8Evaluation25.5% Benchmark (18 papers) C5StabilitySTRENGTH40% Autoregressive · Diffusion World Model as Simulator Eight capabilities of a simulator, mapped across generative world models action / camera / language → [ state + dynamics ] → RGB + structured feedback strength structural gap neutral coverage % = share of 200-paper corpus 1 Assets as conditional input MAGE · DriveDreamer 2 World model generates assets WonderWorld · Free4D 3 Unified editable platform WorldGen · GigaWorld-0 1 Scale-driven Cosmos · GenAD · UniSim 2 Composition-driven RoboDreamer · DriveDreamer-2 3 Parametric-controllable GigaWorld-0 1 Multi-task auxiliary outputs Tesseract · MUVO · DiST-4D 2 Derived from physics OrbiSim · Kinema4D 3 Latent-space representation V-JEPA2 · Cosmos Policy 1 Camera-level control GameNGen · Matrix-Game 2 Object-level interaction ActWorld · WorldCraft 3 Multi-agent / algo-in-loop DIAMOND · V-JEPA2-AC 1 Hard physics core OrbiSim · PIN-WM · PWTF 2 Soft physics constraints GEM-4D · PEVA · FlowDreamer 3 Physics evaluation only What-If World · PhysBench 1 Explicit control DisCo · Prisma-World 2 Language / multimodal Pandora · DrivingGPT · UniPi 3 Latent action Genie · AdaWorld · DiLA 1 Open-world quality WorldScore · Nano-WM 2 Game / driving benchmarks WBench · driving suites 3 Closed-loop embodied World-in-World · RoboWM-Bench 1 Self-adversarial training Self-Forcing · Matrix-Game2.0 2 Memory (implicit/explicit) VMem · Long-term Spatial Mem 3 Causal consistency + theory WorldCraft TASP · Fine-flow Fig. 1. Overview of the survey. Centred on “world model as simulator”, the eight capabilities of a traditional simulator are arranged around it: asset construction, physics engine, interaction, controllability, stability, state feedback, diversity, and evaluation metrics. Each capability panel reports its coverage as a principal contribution across the 200-paper corpus, a status tag (strength / structural gap / neutral ), and representative works. Controllability (62.5%), interaction (40%), and stability (40%) are tagged as strengths; asset construction (19%), the physics engine (17%), and state feedback (22.5%) as cross-route structural gaps; and diversity (23%) and evaluation metrics (25.5%) remain neutral. rollouts [21], [22], collision and contact behaviour violating physical intuition [23], [24], and identical actions producing inconsistent visual consequences at different times [25]. The root of these problems is that current world models are conditional-distribution samplers rather than physical- evolution solvers: they learn the conditional distribution of visual patterns in the training data, not the invariant phys- ical laws of the real world [26]. In addition, world models still exhibit critical gaps against traditional simulators in the precision of controllability, the richness of state feedback, and the verifiability of their output. Section 4.6 reports a paper-level audit of state-feedback interfaces across all 163 implementation papers. Motivated by these observations, this survey systematically answers three core questions: (a) Can the generative process of a generative world model become a simulator in the strict sense? (b) On each simulator capability, how far has each technical route progressed, and how far does it remain from the target? (c) In which directions should the next steps be taken? To answer them, Section 2 describes the literature-collection process and in- clusion criteria; Section 3 provides the technical background of world models and the capability yardstick of traditional simulators; Section 4, the core of the survey, uses eight key simulator capabilities as a unified framework (asset construction C1, physics engine C2, interaction C3, control- lability C4, stability C5, state feedback C6, diversity C7, eval- uation metrics C8), summarized in Figure 1, and compares, capability by capability, the concrete works, progress, and shortcomings of each route—previewing our central finding that controllability, interaction, and stability stand out as relative strengths, whereas asset construction, the physics engine, and state feedback remain structural gaps; Section 5 distills six future research directions from the comparative analysis; and Section 6 gives our concluding judgement. The main contributions of this capability-based study are fourfold: •A comparative analysis framework. We examine generative world models through the unified yard- stick of a traditional simulator’s capabilities, estab- lishing an eight-dimensional comparison system. •A reproducible full-corpus evidence map. Based on paper-by-paper coding of a curated 200-paper cor- pus, we quantify the progress and key shortcomings of each technical route capability by capability. •Quantification of the state-feedback shortcom- ing. We identify state feedback—a capability so far severely neglected yet indispensable to a simulator— as a structural shortcoming, and quantify this struc- FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 20263 tural absence through a five-level labelling scheme (B1–B5). •An actionable research roadmap. We provide six clear and actionable research directions, each backed by nascent work as evidence of its feasibility. The corpus annotations, evidence map, and anal- ysisscriptssupportingthisstudyaremaintained on the project page: https://github.com/AtongWang/ world-model-simulators. 2 LITERATURE COLLECTION 2.1 Search Strategy and Inclusion Criteria Our literature collection covers four complementary sources—Google Scholar, arXiv, DBLP, and Crossref—with a search cut-off of 30 June 2026. Each source has its own emphasis: arXiv aggregates the vast majority of the field’s latest preprints and is the primary source for frontier work; Google Scholar is used for cross-database broad discov- ery and citation tracing; and DBLP and Crossref respec- tively provide canonical bibliographic records for computer- science conferences/journals and DOI-based publication metadata, used to verify the formal publication status of each paper. The purpose of using multiple sources is to cover the blind spots of any single database—arXiv is timely but holds only preprints, whereas DBLP and Crossref are authoritative but lag in indexing the newest work. In constructing the queries, rather than using a single broad term, we decomposed each topic into several concept terms combined by Boolean AND, striking a balance between recall and precision. For example, for the topic of interactive video world models we used interactive AND world AND model AND simulation AND video AND generation;forthe differentiable-physics direction we used differentiable AND physics AND simulation AND world AND model AND learned. Around the mainstream technical routes (latent dynamics, autoregressive, diffusion, game engines, driving simulation, embodied robotics, evaluation benchmarks) and several angles easily missed by the main lines (JEPA self-supervision, language models as world models, 3D Gaussians and NeRF, long-horizon memory, occupancy modelling, multimodality, physical-plausibility evaluation), we designed more than twenty queries in total; the complete list appears in Appendix A. Search results were sorted by relevance, truncated to the top-ranked entries per query, automatically de-duplicated against the accumulated set, and then screened manually. Because keyword search cannot exhaust a rapidly evolv- ing field, we further expanded the corpus along the citation network. On one hand, we combed the reference lists of six representative surveys to fill gaps, covering world mod- els for robot learning [27], interactive video world models [28], world models for robotic manipulation [29], embodied intelligence fusing physical simulators and world models [30], unified 2D/video/3D/4D multimodal generation [31], and world models for embodied AI [32]. On the other hand, we expanded along the forward and backward citations of four foundational or milestone works—World Models [1], DreamerV3 [33], Genie [15], and Cosmos [18]—to include peripheral but relevant work. In addition, some industrial- grade world models (such as DeepMind’s Genie 2/3) are released only as technical reports or blog posts without a formal paper; such works are cited individually in the text and are not counted in the searchable-corpus statistics. Because arXiv is a preprint platform and cannot reflect the formal publication status of a paper, we cross-verified every included paper in DBLP and Crossref and gener- ated canonical references accordingly: anchored on each pa- per’s authoritative record (title, authors, year), we searched both databases by title similarity and first-author matching, judging a paper formally published only when the title matched closely, the authors agreed, and the venue was not a preprint. Verification showed that, of the 200 papers, 72 have been published at peer-reviewed conferences or journals—22 at ICML, NeurIPS, or ICLR; 20 at CVPR, ICCV, or ECCV; 6 at CoRL, ICRA, or RSS; 6 at AAAI; 3 in Nature; and 15 others at ICASSP, IROS, WACV, TPAMI, TIP, ACM Computing Surveys, and domain-specific journals—while the remaining 128, mostly recent results from late 2025 to 2026, were still arXiv preprints as of the search date. For published work, the references always adopt the metadata of the formal version; verification also corrected two mislabelled identifiers and three publisher-name errors in the text. Inclusion follows one core criterion: whether a paper substantively accepts some action or conditional input (cam- era pose, keyboard/mouse operation, language instruction, latent action, trajectory signal, or 3D geometric condition) and, on that basis, performs a forward rollout to generate future states (video frames, 3D scenes, occupancy, point clouds, or structured state), with results that can serve interaction, control, or planning. On this basis, we exclude four kinds of work: 1)pure text-to-video/image-to-video backbones unre- lated to interaction or conditional control; 2)work targeting pure representation learning without action conditioning; 3)pure prediction models that forecast only object trajectories without environment-state evolution; 4)papers whose title or abstract contains the phrase “world model” but which actually belong to non- generative directions such as language-model en- coding or cognitive modelling. 2.2 Corpus Overview After several rounds of screening, 200 records were finally included, spanning 2018 to 2026. We performed structured classification at the abstract level or above for all records— covering paradigm membership, technical route, action- interface type, state-feedback capability where annotated, and principal contribution dimensions, covering the mile- stone works of all major routes as well as representative benchmarks and surveys. The corpus contains 163 imple- mentation papers assigned to six technical families: latent- dynamics and latent-action models (29), autoregressive gen- eration (37), diffusion models (63), JEPA-predictive mod- els (5), explicit 3D/4D reconstruction (25), and occupancy- centric methods (4). The remaining 37 records provide con- text: evaluation benchmarks (18), surveys (16), and oth- ers (3). Figure 2 profiles the complete 200-record corpus FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 20264 C1 Asset C2 Physics C3 Interaction C4 Control C5 Stability C6 Feedback C7 Diversity C8 Eval 30 60 90 120 150 (a) Cumulative papers per capability, by year Cumulative paper count to year ≤ 2020≤ 2022≤ 2024≤ 2026 38 34 80 125 80 45 46 51 '18'19'20'21'22'23'24'25'26 0 10 20 30 40 50 60 70 80 Papers 2 3 1 2 5 10 30 72 75 (b) Corpus by publication year: published vs preprint Published Preprint / other Fig. 2. Research attention across the eight simulator capabilities and the corpus composition, 2018–2026. (a) Cumulative papers contributing each capability, drawn as nested year slices (to 2020/2022/2024/2026); the radial value is the number of papers for which that capability is a principal contribution, and the number at each vertex is its 2026 total. Controllability, interaction, and stability dominate, while asset construction, the physics engine, and state feedback stay low. (b) The 200-paper corpus by publication year, split into published (72) and preprint / other (128); the corpus is increasingly preprint-dominated in 2025–2026. along two views: (a) the cumulative number of papers contributing each of the eight capabilities, sliced by year (2020/2022/2024/2026); and (b) the corpus by publication year, split into published versus preprint. Two trends stand out. First, research attention is highly uneven: controllability, interaction, and stability accumulate far more papers than asset construction, the physics engine, and state feedback—a gap the capability-by-capability analysis of Section 4 exam- ines in detail. Second, the corpus is increasingly preprint- dominated: 72 of the 200 records are published while 128 remain arXiv preprints or other records, the latter concen- trated in 2025–2026, reflecting how fast the field is moving relative to the review cycle. 2.3 Distinction from Existing Surveys Our corpus contains 16 surveys, grouped by coverage into five categories: general world models [26], [34], [35], autonomous driving [36]–[39], robotics and embodiment [27], [29], [30], [32], [40], interactive video [28], and effi- ciency/multimodality [31], [41], [42]. Their common feature is that they centre on the world model itself, classifying by architecture, function, or application domain. The core difference of this survey is that it uses simulator capability as an external yardstick: the eight capabilities (C1–C8) derive from the capability set of traditional simulators (physics engine G1, game engine G2, RL environment G3) rather than from the internal attributes of world models. This external- ity makes the yardstick independent of any specific route, so heterogeneous routes can be projected into a unified capability space for quantitative comparison. Methodologically, this study is distinguished by two empirical mechanisms. The first is full-corpus evidence mapping: we annotate all 200 records paper by paper with paradigm, route, one to three principal-contribution capabil- ities, and a summary of novelty and evidence. The resulting distributions measure research attention rather than capabil- ity attainment. The second is a five-variable state-interface audit across all 163 implementation papers: ego informa- tion (B1), sensor output (B2), reward and termination (B3), runtime entity/physics annotations (B4), and closed-loop interaction (B5). Present, absent, and unresolved judgements are reported separately so that insufficient evidence is not treated as absence. Relative to the most relevant surveys, this survey forms a complementary division of labour rather than a replace- ment. Reference [27] concerns how world models assist policy learning, whereas this survey concerns whether the simulator attributes of world models themselves are com- plete; Reference [28] concerns the fluency of interaction (controllability, memory, real time), whereas this survey fur- ther examines the breadth and depth of interaction objects (algorithm-in-the-loop and human-in-the-loop); Reference [29] provides a fine-grained functional taxonomy, which this survey projects into a capability-shortcoming space; and the judgement in [26] that Sora possesses preliminary physical understanding rather than strict simulation aligns in direction with our core conclusion, for which our full analysis of 200 papers provides broader empirical support. In short, existing surveys answer what world models exist and what they can do; this survey asks what a world model still cannot do as a simulator, how far it remains, and why— thereby turning the proposition of “world model as simula- tor” from a vision into an actionable research agenda with concrete dimensions, checkable data, and clearly identified FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 20265 1 Latent-dynamics world model observation oₜ VAE encoder RNN / RSSM transition roll forward in latent space decoder predicted frame ôₜ₊₁ controller / policy encode→ predict in compact latent space → act compact latent state z action aₜ zₜ₊₁ 2 Video-generative world model action / text condition “turn right and move forward” diffusion: iterative denoising noise clean frame PIXEL / TOKEN space predict the next observation directly in pixel / token space current frames future frames token 3 token 2token 1 next token ... autoregressive: next-token prediction ... context observation oₜ context encoder predictor predicted embedding êₜ₊ₖ future observation oₜ₊ₖ target encoder compare (e.g., cosine similarity) abstract embedding space no pixel reconstruction predict the future in abstract representation space 3 Joint-embedding predictive world model context embedding cₜ target embedding eₜ₊ₖ Fig. 3. Three principal routes to world modelling, distinguished by prediction space. A latent-dynamics model encodes observations into a compact state and rolls that state forward for decoding and control; a video-generative model predicts future observations directly in pixel or token space through diffusion or autoregressive generation; and a joint-embedding predictive model predicts a future representation against a target embedding without reconstructing pixels. The routes therefore expose different simulator interfaces: compact state for imagination and control, rendered observations for direct inspection, and abstract embeddings for efficient downstream prediction and planning. shortcomings. 3 BACKGROUND 3.1 Concept and Evolution of World Models World models have developed along three main technical routes and have recently shown a clear trend toward con- vergence. This section reviews that evolution along the twin threads of chronology and modelling mechanism: first the latent-dynamics route, which was the earliest to model envi- ronment dynamics in a compact latent space; then the video- generation route, which predicts the future directly in pixel or token space; next the Joint-Embedding Predictive Archi- tecture, which instead predicts in an abstract representation space; and finally the overall trend, since 2024, of these three routes borrowing from one another and moving toward hybrid fusion. This trajectory reflects both a migration of the representation space—from latent state, to pixels, to abstract embeddings—and a shift of research emphasis from policy learning, to high-fidelity generation, to downstream utility. Figure 3 makes the principal architectural distinction explicit: the three routes differ primarily in the space in which future evolution is represented and predicted. 3.1.1 Latent-Dynamics World Models The concept of world models traces back to the pioneering work of Ha and Schmidhuber in 2018 [1], whose core idea models the environment as three components: a VAE that compresses high-dimensional observations into a compact latent state, an MDN-RNN-based dynamics model that pre- dicts future states in the latent space, and a linear controller that performs policy learning within this latent environ- ment. This framework established the basic paradigm of the world model as an internal simulator of the environment, its key being the reduction of complex high-dimensional envi- ronment interaction to two steps—prediction and control— in latent space. Subsequently, PlaNet [43] and the Dreamer series [33], [44], [45] systematically developed this frame- work into a latent-dynamics system based on the Recurrent State-Space Model (RSSM). RSSM contains both determin- istic and stochastic latent-state variables, enabling long- horizon imagination in a compact latent space and policy optimization thereupon. DreamerV2 [45] raised the repre- sentational power of world models to Atari human level via discrete latent representations, while DreamerV3 [33], through symlog prediction and world-model regularization, achieved stable training without manual tuning across more than 150 environments. The core value of this route is learning in imagination: DayDreamer [46] deployed Dreamer on a real robot for online learning, using the world model to accelerate simulation of real interaction in latent space; MoDem-V2 [47] follows the same idea, extending a visuo- motor latent-dynamics world model to online learning for real-robot manipulation and confirming the paradigm’s vi- FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 20266 ability on contact-rich tasks. 3.1.2 Video-Generative World Models In parallel, video-generative world models developed rapidly between 2023 and 2026. Unlike the latent-dynamics route that operates in a compact latent space, the video- generation route models the future directly in pixel or token space; its advantages are high visual realism, direct suitabil- ity for human evaluation, and native support for generating complex visual scenes. Early works such as GAIA-1 [48] fo- cused on neural world models for driving scenes, while Ge- nie [15] inferred latent actions without labels from internet videos and generated interactive 2D game worlds. Sora then prompted broad discussion of whether video-generation models can serve as world simulators [26]. Cosmos [18], [19] packages this route as a World Foundation Model plat- form for Physical AI, including a video-curation pipeline, pretrained models, tokenizers, and fine-tuning tools. At the mechanism level, video-generative world models fall mainly into diffusion and autoregressive classes. SimPLe [49] showed that a video-prediction world model improves the sample efficiency of RL on Atari, while Phenaki [50] achieved variable-length video generation driven by time- varying text prompts. Representative diffusion works in- clude DIAMOND [51], which trains RL agents in a diffusion world model; GameNGen [2], which demonstrates about 20-FPS interaction in a constrained game domain; and the Matrix-Game series [4], [52], [53], which reports up to 720p and about 40 FPS. Representative autoregressive works in- clude IRIS [54], which combines discrete autoencoding with an autoregressive Transformer; Oasis and RELIC [55], with KV-cache camera-aware memory for real-time generation; GameFactory [3], with scene-generalizing keyboard/mouse control; and GameGen-X [5], an interactive, controllable open-world game DiT. The mechanisms are also converging: causal-forcing distillation [56] and few-step autoregressive diffusion [57] combine diffusion-based generation with au- toregressive inference. 3.1.3 Joint-Embedding Predictive Architecture The third route is the Joint-Embedding Predictive Architec- ture (JEPA), represented by V-JEPA [58] and its successors V- JEPA 2 [11] and UWM-JEPA [59]. JEPA’s design philosophy differs sharply from the two generative routes above: it predicts the future not in pixel space but in a learned abstract representation space. Through self-supervised pre- training of a 1.2B-parameter ViT on over one million hours of internet video, combined with fine-tuning on merely 62 hours of unlabelled robot data, V-JEPA 2 achieves zero- shot transfer Franka manipulation, requiring no data from that robot and no task-specific training or reward signal. In planning efficiency, V-JEPA 2-AC requires only about 16 seconds per action for latent-space planning, whereas pixel-generation planning under comparable conditions re- quires about 4 minutes—roughly a 15× speed advantage [11]. UWM-JEPA [59] further extends the JEPA framework to partially observable settings: by introducing a density- matrix latent representation and a unitary-transformation predictor, it maintains the structural integrity of belief space under blind rollouts with occluded observations, reaching 0.77 accuracy in a five-step forward-simulation task, versus 0.53 for a parameter-matched LSTM-JEPA. A cluster of deepening work has recently appeared on the mechanism of JEPA as a world model: a systematic dissection of the success factors of JEPA in physical planning [60] points out that the combination of action conditioning and prediction in representation space (rather than pixel space) is key to JEPA’s support for planning, providing an interpretable design rationale for this route; variational JEPA [61] turns the originally deterministic predictor into a probabilistic latent-space world model that can characterize multimodal uncertainty about the future (so far verified only at the theoretical and toy-experiment level); and, addressing test- time representation collapse in JEPA, one work [62] repairs it by learning invariant visual representations and, on top of them, realizes reward-free latent-space MPC planning. A unified probing study [63] further shows that it is the prediction (rather than reconstruction) objective that drives the action-relevance of video-world-model latents, giving probe-level evidence for the claim that representation-space prediction beats pixel reconstruction. The cost of the JEPA route, however, is that it produces no readable pixel output, and its latent-vector feedback is opaque to non-learning downstream consumers. 3.1.4 The Trend of Route Convergence Between 2024 and 2026, a clear trend of route convergence emerged; noteworthy instances include the hybridization of diffusion and autoregression (causal-forcing distillation [56], few-step autoregressive diffusion [57]), the comple- mentarity of generative and JEPA approaches (V-JEPA 2- AC’s latent-conditioned planning [11]), the synergy of ex- plicit 3D and implicit generation (Lyra 2.0’s [21] geometry routing plus generated appearance, and GWM’s [13] use of a 3D-Gaussian world model as a neural simulator, built on the 3D Gaussian Splatting representation [64]), and the unification of VLA and world models (τ 0 -WM [12] and WorldVLA [65] integrating policy learning, video predic- tion, and action evaluation in a single framework). These trends may foreshadow the architectural direction of the next generation of world models: no longer a pure diffusion or latent-dynamics model, but a hybrid system composed of functional modules. 3.2 Capability Yardstick of Traditional Simulators Before comparing world models, we must first make ex- plicit the capability yardstick of a rigorous simulator. Three classes of traditional simulator baselines jointly define the capability set that a functionally complete simulation en- vironment should possess: physics engines (e.g., MuJoCo [66], Isaac Sim [67], and PyBullet [68]; denoted G1), game engines (e.g., Unreal Engine [69] and Unity [70]; G2), and reinforcement-learning training environments (e.g., the Atari-based Arcade Learning Environment [71], CARLA [72], and DMControl [73]; G3). We formalize the eight ca- pabilities as follows: •Asset construction (C1): traditional simulators rely on artists and modellers to hand-create or proce- durally generate static assets such as maps, char- acters, objects, and materials; the core strength is fully controllable, instance-level decomposition and FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 20267 editability—every object can be independently se- lected, modified, or replaced—at the cost of high labour and limited content generalization. •Physics engine (C2): based on numerical solvers (rigid-body dynamics, collision detection, contact- force computation, elastic deformation, etc.), it pro- vides physically deterministic motion simulation, whose formal guarantees—energy and momentum conservation, collision impenetrability—are the cor- nerstone of scientific computing and safety-critical simulation. •Interaction (C3): it supports any agent interacting with any element of the environment at any moment, with response latency down to the millisecond level, through interfaces such as API calls, physical con- trollers, or scripting languages. •Controllability (C4): the configuration and evolution of the simulated environment are fully controlled by the user through a precise parametric interface, with no approximation or semantic ambiguity. •Stability (C5): based on deterministic solvers, the simulator’s evolution is fully reproducible for the same input sequence. •State feedback (C6): a traditional simulator can extract rich structured information from any inter- mediate state, including ego pose (B1), sensor data (B2), task-evaluation signals (B3), and runtime en- tity/physics annotations (B4), and supports closed- loop interaction (B5). •Diversity (C7): controlled scene diversity through parameter randomization and procedural genera- tion. •Evaluation metrics (C8): a mature evaluation system, including reward-based RL performance metrics and directly accessible ground-truth labels. On this yardstick, we express the working definition of a rigorous simulator as a function T : S ×A → Π(S ×F ), whereS is the state space,A the action space, andF a struc- tured state-feedback space containing at least B1 through B4, subject to four conditions: state evolution is physically consistent and verifiable, temporally consistent without drift, statistically reproducible (deterministic or controllably stochastic), and supports closed-loop interactive evaluation. 4 WORLD-MODEL–SIMULATOR COMPARISON Before the capability-by-capability comparison, we first compare world models and traditional simulators at a high level. World models hold two fundamental advantages over traditional simulators. First, a world model learns envi- ronment dynamics from data, without manual geometric modelling, material editing, or physical-parameter calibra- tion. Cosmos’s [19] WFM is positioned as a digital twin for Physical AI and can generate customized world models for driving, robotic manipulation, indoor navigation, and other scenarios from one pretrained foundation model, whereas a traditional pipeline must re-model each new scene. Sec- ond, generative world models—especially diffusion-based ones—can achieve visual realism that approaches or even surpasses real photographs; in GameNGen’s [2] human study, only 58–60% of subjects could distinguish real from generated DOOM screenshots, whereas a traditional render- ing pipeline, though stronger in physical accuracy, is limited in expressiveness by model complexity and compute. World models, however, exhibit three recurring limita- tions. The first is the absence of formal guarantees. Tra- ditional simulators can verify physical invariants at each step, whereas world models generally provide plausibility evidence rather than proof of physical correctness. The second is a sparse information interface: only 45 of 163 im- plementation papers explicitly report sensor-level outputs beyond RGB, whereas traditional simulators expose pose, depth, LiDAR, semantics, collision events, rewards, and labels through APIs. The third is limited closed-loop repro- ducibility. Traditional simulators commonly expose random seeds and deterministic modes, but the included literature provides little comparable evidence on seed control, deter- ministic execution, tolerance bands, or repeat-run variance. These three recurring limitations constitute the core focus of the capability-by-capability comparison that follows. For each simulator capability, we examine three angles: what the yardstick of a traditional simulator is on that capability, what concrete efforts each world-model route has made (identifying representative works and their methods), and where the gap to a traditional simulator lies and what its essential cause is. As profiled in Figure 2 of Section 2, research atten- tion is severely imbalanced across the eight capabilities. Measured by the cumulative number of papers for which a capability is a principal contribution, controllability C4 alone has reached 125 papers, interaction C3 has reached 80, and stability C5 has reached 80, whereas asset construction C1, physics engine C2, and state feedback C6 have long stalled in the 35–45 range. This gap is no accidental research vacuum but exactly the three foundational capabilities a world model most lacks as a simulator: the construction of the static world, adherence to physical law, and the feedback of structured state. The analysis uses two complementary evidence views. Table 1 lists representative systems that accept an action or conditional input and roll forward to predict future obser- vations or state, together with their principal-contribution topics. Table 2 aggregates all 163 implementation papers by six model families. Benchmarks, surveys, and the three other contextual records are excluded from these family denominators. 4.1 Asset Construction Asset construction is a foundational capability of a tradi- tional simulator, encompassing the creation and manage- ment of static elements such as maps, characters, objects, materials, and environmental lighting. In a traditional pro- duction pipeline, this process depends heavily on human artists and modelling experts; quality is controllable and instances are decomposable and editable, but the cost is high and generalization is hard. Of the 200 papers, only 38 (19.0%) list asset construction as a principal contribution dimension—a low proportion worth noting: without high- quality, editable static assets as a foundation, the subsequent dynamic simulation lacks a reliable carrier. We organize the analysis of asset construction around one core distinction: a world model can accept assets as FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 20268 TABLE 1 Representative simulator-capable world models, grouped by their primary simulator capability. Each work is marked on every capability it substantively contributes (C1–C8), with a Category following the taxonomy of Section 4 and a one-line Highlight. For C8 we list representative evaluation benchmarks. Routes: Latent (latent dynamics), AR (autoregressive), Diff. (diffusion), 3D/4D, Occ. (occupancy), Bench. (benchmark); arXiv venues are preprints as of the search cutoff. MethodVenueRouteC1C2C3C4C5C6C7C8CategoryHighlight C1 – Asset Construction Lyra2 [21]arXiv’263D/4D✓Asset generationMemory routing removes drift DecMem [22]arXiv’26Diff.✓Asset generationDecoupled memory, minute-scale CP4D [74]arXiv’263D/4D✓Asset generationPhysics-aware compositional 4D Prisma-World [75]arXiv’26Diff.✓Asset as conditionMulti-agent cross-view consistency MoVerse [76]arXiv’263D/4D✓Asset generationPanorama-Gaussian navigable world Cosmos [19]arXiv’25Diff.✓Unified platformFine-tunable foundation platform GWM [13]arXiv’253D/4D✓Asset generationGaussian primitives as state Aether [77]arXiv’253D/4D✓Asset generationUnified 4D reconstruct+predict+plan WonderWorld [16]arXiv’243D/4D✓Asset generationSingle image,< 10 s 3D UniScene [78]arXiv’24Occ.✓Asset generationOccupancy-centric multi-sensor generation C2 – Physics Engine RealWonder (PhysBridge) [79]arXiv’26Diff.✓Simulator bridgeSimulator flow, 4-step refine OrbiSim [10]arXiv’26Latent✓Hard physicsDifferentiable physics core RoboWM-Bench [80]arXiv’26Bench.✓Physical evalGeneration-to-execution verification What-If World [23]arXiv’26Bench.✓Physical evalCausal-intervention physics benchmark ActWorld [81]arXiv’26Diff.✓Soft constraintHierarchical action-aware memory PIN-WM [82]RSS’253D/4D✓Hard physicsPhysics-parameter identification PEVA [83]NeurIPS’25Latent✓Kinematic priorWhole-body pose conditioning WorldVLA [65]arXiv’25AR✓Soft constraintUnifies VLA and WM RoboScape [84]arXiv’25Diff.✓Soft constraintDepth + keypoint dynamics ReconDreamer-RL [85]arXiv’25Diff.✓Simulator bridgeKinematics + diffusion for RL C3 – Interaction Matrix-Game3 [4]arXiv’26AR✓Camera-level720p 40FPS streaming PointWorld [86]arXiv’263D/4D✓Algorithm-in-loop3D point flow, MPC-ready DreamDojo [87]arXiv’26Diff.✓Object-level44k-hour human-video latent action WorldCraft [88]arXiv’26AR✓Object-levelObject-level trajectory control GameFactory [3]ICCV’25AR✓Object-levelScene-generalizing key/mouse control DIAMOND [51]NeurIPS’24Diff.✓Algorithm-in-loopDiffusion WM trains RL agent GameNGen [2]ICLR’24Diff.✓Camera-levelReal-time neural game engine GenAD [6]arXiv’24AR✓Camera-levelFirst large driving predictor UniSim [89]ICLR’23Diff.✓Object-levelUniversal interaction simulator DayDreamer [46]CoRL’22Latent✓Algorithm-in-loopOnline learning on robot C4 – Controllability DiLA [90]arXiv’26Latent✓Latent actionContent-structure decoupling UWM-JEPA [59]arXiv’26JEPA✓Latent actionDensity-matrix belief space DisCo [25]arXiv’26AR✓ExplicitDiscrete motion primitives AdaWorld [91]ICML’25AR✓Latent actionTransferable latent actions GAIA-2 [92]arXiv’25Diff.✓ExplicitStructured multi-view driving control V-JEPA2 [11]arXiv’25JEPA✓Latent actionSelf-supervised, 15x planning Genie [15]ICML’24Latent✓Latent actionUnsupervised latent action Pandora [93]arXiv’24AR✓Language/multimodalAnytime free-text control GameGen-X [5]ICLR’24Diff.✓Language/multimodalFirst open-world game DiT RoboDreamer [94]ICML’24Diff.✓Language/multimodalCompositional text-to-video C5 – Stability OpenWM [95]arXiv’26AR✓Implicit memoryMinute-scale open-source WM FastVDM [96]arXiv’26AR✓Drift suppressionTraining-free attention acceleration LiveWorld [97]arXiv’263D/4D✓Causal consistencyOff-sight dynamics synchronized SWM [98]arXiv’26AR✓Drift suppressionRetrieval-anchored to real city VectorWorld [99]arXiv’26Diff.✓Drift suppressionVector-map streaming diffusion LT-SpatialMem [100]arXiv’25AR✓Explicit 3D memoryGeometry-grounded spatial memory RELIC [55]arXiv’25AR✓Implicit memoryKV-cache camera memory GenWorldExplorer [101]arXiv’24Diff.✓Explicit 3D memoryBelief update via exploration IRIS [54]ICLR’22AR✓Implicit memoryDiscrete AR Transformer WM World Models [1]arXiv’18Latent✓Implicit memoryVAE+RNN, train in dream C6 – State Feedback Cosmos Policy [18]arXiv’26Diff.✓Latent feedbackLatent feedback into policy TesserAct [102]ICCV’253D/4D✓Multi-task outputRGB-depth-normal 4D output ChronoDreamer [103]arXiv’25AR✓Physics-derivedExplicit contact-force prediction OccSora [104]TIP’24Occ.✓Multi-task outputDiffusion 4D-occupancy simulator OccLLaMA [105]arXiv’24Occ.✓Multi-task outputOccupancy-language-action vocab DreamerV2 [45]ICLR’20Latent✓Reward/terminationDiscrete latent, human-level SimPLe [49]ICLR’19AR✓Reward/terminationSample-efficient Atari RL MuZero [106]Nature’19Latent✓Reward/terminationLatent model + tree search Dreamer [44]ICLR’19Latent✓Reward/terminationValue gradients in imagination PlaNet [43]ICML’18Latent✓Reward/terminationLatent overshooting planning C7 – Diversity Cosmos-Drive [107]arXiv’25Diff.✓CompositionSynthetic long-tail driving GigaWorld-0 [20]arXiv’25Diff.✓ParametricControllable data-engine diversity AccidentGen [108]arXiv’25Diff.✓CompositionRare accident scene generation DreamerV3 [33]Nature’23Latent✓Scale-drivenOne config, 150+ domains C8 – Evaluation Metrics DrivingGen [109]arXiv’26Bench.✓Unified qualityDriving WM benchmark WorldBench [110]arXiv’26Bench.✓Physical plausibilitySingle-concept physics diagnosis Omni-WorldBench [111]arXiv’26Bench.✓Closed-loop embodiedInteraction-centric 4D benchmark GameWorld [112]arXiv’26Bench.✓Closed-loop embodiedVerifiable game-agent benchmark OpenGame [113]arXiv’26Other✓Generation qualityAgentic playable-game generation WBench [114]arXiv’26Bench.✓Closed-loop embodiedMulti-turn interaction benchmark WorldScore [115]arXiv’25Bench.✓Unified qualityFirst unified WM benchmark World-in-World [116]arXiv’25Bench.✓Closed-loop embodiedVisual quality != task success Genie-Envisioner [117]arXiv’25Diff.✓Unified qualityUnified policy/eval/sim platform 1X-WorldModel [118]arXiv’25Bench.✓Closed-loop embodiedHumanoid interaction benchmark FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 20269 TABLE 2 Model-family× principal-topic counts for the 163 implementation papers. Columns denote C1 asset construction, C2 physics engine, C3 interaction, C4 controllability, C5 stability, C6 state feedback, C7 diversity, and C8 evaluation metrics. Each cell reports n/N , where N is the complete family denominator and n is the number carrying that principal-topic label. Counts measure research attention, not capability attainment. RouteC1C2C3C4C5C6C7C8 Latent dynamics/action2/295/2911/29 18/29 11/29 10/299/293/29 Autoregressive2/373/3721/37 25/37 23/377/378/377/37 Diffusion12/63 11/63 26/63 52/63 27/63 13/63 14/63 9/63 JEPA predictive0/50/51/55/54/50/51/50/5 Explicit 3D/4D20/255/256/2511/258/256/257/251/25 Occupancy1/40/41/43/41/43/41/41/4 Reading guide. Controllability is the most frequently coded topic in five families. Explicit 3D/4D papers concentrate on asset construction, and three of four occupancy papers discuss state feedback. These distributions motivate route-specific questions, but the small JEPA and occupancy denominators and the absence of a shared performance threshold preclude family maturity rankings. a conditional input for generation—given a scene layout, make it move; it can also generate assets as its own output— creating an explorable world by itself; and it can serve as a unified platform integrating both. These three perspectives correspond to different maturity stages of the capability, from using known assets, to creating new assets, to inte- grating asset construction with dynamic simulation. 4.1.1 Assets as Conditional Input Many world models use explicit 2D or 3D geometric in- formation as a generation condition; such geometry is es- sentially the assets of a traditional simulator, including road layouts, building geometry, object positions, and robot structure. In driving scenes this conditioning is especially natural. MAGE [119] injects BEV layout, camera parameters, and 3D bounding boxes as multiple geometric conditions into a street-view generation diffusion model, where the bird’s-eye view defines road topology and lane structure, the 3D boxes fix the spatial positions of traffic participants, and cross-view attention guarantees geometric consistency across front, rear, and surround views, so that the gen- erated street scene strictly obeys the specified geometric constraints rather than freely generating scenes inconsis- tent with the road structure. DriveDreamer [120] precisely generates driving videos that conform to traffic-structure constraints via a controllable diffusion model, incorporating road topology and lane lines as hard geometric constraints; its key value is to guarantee structurally reasonable con- tent, i.e., vehicles driving in lanes rather than floating in the air. DriveDreamer-2 [121] further lengthens the asset- conditioning chain: an LLM first converts a user’s language query (e.g., generate a congestion scene on a multi-lane highway) into agent trajectories, then generates an HDMap conforming to traffic rules from the trajectories, and finally produces a spatio-temporally consistent driving video via a unified multi-view model. This pipeline—from language to trajectory to HDMap to multi-view video—essentially introduces asset conditions at different abstraction levels at different stages of generation: the semantic layer where the LLM understands congestion, the geometric layer where the HDMap defines road structure, and the instance layer where trajectories define specific vehicle positions and time curves. Such layered conditioning lets users control content at the most appropriate abstraction level rather than being forced to intervene at the pixel level. Figure 4 schematizes this conditional-input route: struc- tured geometry fixes the scene configuration, while the world model is responsible for rolling the conditioned scene forward over time. In robotic scenes, asset conditions are even more struc- tured, because the robot’s own kinematic structure is known and precise. VerseCrafter [122] proposes a 4D geometric- control representation: it encodes the scene background as a 3D point cloud (static asset), represents dynamic objects as 3D-Gaussian trajectories (spatio-temporal encoding of dy- namic assets), and encodes occupancy as spatial constraints, jointly driving a video-diffusion model. The essence of this representation is to separate the geometric state of the world from its pixel projection—controlling object motion not on the 2D image plane but by precisely specifying, in the 3D world coordinate system, that an object is at (x 1 ,y 1 ,z 1 ) at t=1 and (x 2 ,y 2 ,z 2 ) at t=2, then rendering video from this 4D geometric state. This separation of geometric state from visual rendering is structurally isomorphic to the world- state-to-rendering-pipeline design of a traditional game en- gine. Kinema4D [14] drives 4D embodied simulation con- ditioned on a URDF (Unified Robot Description Format) kinematic model, outputting RGB and pointmap as dual feedback; URDF is a standard asset format in robotics that describes joint structure, link lengths, mass distribution, and kinematic chains. Driving generation with a standardized robot-description format lets a world model plug directly into existing robotics software ecosystems, such as ROS [123] and Gazebo [124], without users learning a new model-specific interface. The URDF embodied mask [125] further solves the alignment between coordinate actions and pixel video: by introducing an embodied mask into the dif- fusion model’s attention, it precisely maps actions in 3D co- ordinate space (e.g., move the end-effector to (0.3, 0.1, 0.5)) to visual changes in 2D pixel space, while unifying the control architecture across different robot embodiments. Together, these works reveal a regularity: when a world model accepts explicit, structured assets as input conditions, the precision and controllability of generation improve markedly. This is because the model no longer needs to simultaneously guess what the world looks like and predict how it changes; given the world’s appearance, it focuses on FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202610 structured assets (BEV / HDMap / 3D Boxes) BEV Road-Layout Map 3D Bounding Boxes (Vehicles) HDMap (Lane Geometry) inject as condition world model (video diffusion) neural simulator of the driving world EVOLVING STREET-SCENE ROLLOUT FRONT VIEWLEFT VIEWRIGHT VIEW t = 0 t = 1 t = 2 · · · t = T short-horizon multi-view video over time Fig. 4. Structured assets as conditional input to a world model. BEV road layouts, HD maps, and 3D bounding boxes specify the geometry and traffic configuration supplied to a video-diffusion world model, which produces a temporally evolving, multi-view street-scene rollout consistent with those conditions. predicting the world’s evolution, and the precision of the asset condition translates directly into the precision of gen- eration control. In current practice, however, a critical gap remains between asset conditions and a queryable, editable world state: BEV layouts and 3D point clouds are injected as one-off conditions and cannot be queried or modified after generation, so users cannot edit the scene mid-generation (e.g., change the third building to blue), which is a basic operation of a traditional simulator. 4.1.2 Asset Generation by World Models Complementary to accepting assets as conditions is the world model generating assets itself. On the explicit 3D/4D reconstruction route, several works attempt to generate navigable, explorable, geometrically consistent 3D worlds directly from extremely sparse input (a single image or monocular video). WonderWorld [16] uses FLAGS (Fast Layered Gaussian Surfels) as its scene representation, gen- erating an interactively explorable 3D scene from a single image in under 10 seconds and achieving geometrically consistent stitching via guided depth diffusion. Its key in- novation is to avoid the expensive per-view optimization of traditional 3D reconstruction, using geometric initializa- tion to drastically reduce optimization time and make real- time interactive 3D-world construction possible. Lyra 2.0 [21] specifically targets 3D-consistency degradation under long-range camera trajectories—the key bottleneck of all autoregressive scene generation—and proposes a dual so- lution: spatial-memory routing (using the generated per- frame 3D geometry to retrieve relevant historical frames and establish dense correspondences) to counter spatial forgetting, and self-augmented history training (exposing the model to its own imperfect outputs) to counter temporal drift; together they greatly extend the scale of explorable 3D worlds and support geometric consistency upon revis- iting a location. MoVerse [76] adopts a build-the-world- then-render-observations three-stage strategy: the first stage uses topology-aware diffusion to expand a single narrow- FoV image into a 360 ◦ panorama to bridge the missing field of view, the second lifts the panorama into a per- sistent 3D-Gaussian scaffold to establish explicit spatial memory, and the third generates a real-time roaming video stream along a user-specified trajectory via a Gaussian- conditioned video renderer (about 8 FPS on an RTX 4090). Separating world construction and observation rendering in time—completing geometric reconstruction before visual rendering—is architecturally closer to the design paradigm of a traditional game engine than the generate-while- rendering pure-autoregressive scheme. The sparse-input-to-world process is illustrated in Fig- ure 5: the output must support viewpoint changes along a navigation trajectory rather than merely depict a single generated object or frame. In 4D-scene (dynamic 3D) generation, a body of work explores synthesizing explicit, spatio-temporally consistent 4D assets directly from sparse input. Free4D [126] generates spatio-temporally consistent 4D-Gaussian scenes without per-scene tuning, lifting a single image or text prompt into a freely-navigable dynamic world; HoloTime [127] tames a video-diffusion model to generate panoramic 4D Gaussians, supporting 360 ◦ roaming within the generated scene and extending asset generation from narrow FoV to panorama. Comp4D [128] targets multi-object interaction, a difficulty of 4D generation: an LLM first decomposes the scene into objects and their motion trajectories, then compositional score distillation jointly optimizes inter-object interaction so that the generated dynamic scene is more than a sim- ple superposition of independent objects. A unified text- and-image-guided 4D scene generator [129] supports both text and image conditions driving D-NeRF-style dynamic- scene synthesis under one input interface, widening the entry points for 4D assets; a monocular-video route to- FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202611 navigable 3D/4D world single image / text Single image Text prompt “A peaceful medieval village in the mountains with stone houses, a church, trees, and cobblestone paths.” world model neural simulator (world model) learns dynamics, geometry, and appearance move / look aroundmove / look aroundmove / look around • generate world start view 123 Fig. 5. Generation of a navigable 3D/4D world from sparse input. A single image or text description conditions a world model that constructs an environment with spatial extent and renders consistent observations along a user-controlled camera path. ward real data [130] reconstructs an editable, interactive dynamic world from a single monocular video, directly answering the pain point that generated assets are hard to edit afterwards. These works are of a piece with CP4D [74], which decomposes a 4D scene into two subproblems— static 3D environment background and dynamic foreground objects. The significance of this decomposition exceeds en- gineering convenience; it reflects a structural understanding of the world: changes usually involve only a few objects, while most environment elements are static. CP4D com- bines the motion constraints of a physics simulator with the commonsense-motion prior of a video-diffusion model for hybrid motion synthesis, generating physically plausi- ble trajectories while retaining visual richness. FR3D [131] approaches from another angle, explicitly decoupling ego- motion from scene dynamics. In traditional video genera- tion, camera motion and scene motion are mixed on the image plane, breaking geometric consistency because it is hard to judge whether an object’s displacement arises from camera or object motion. FR3D treats ego-motion as a latent proxy action and, via teacher–student distillation, extracts spatial commonsense from a foundation model to achieve future-2-second dynamic 3D reconstruction without ac- tion annotation, with zero-shot generalization. Aether [77], through task-interleaved feature learning, unifies 4D dy- namic reconstruction, action-conditioned video prediction, and goal-conditioned visual planning in one framework, where the three tasks share the same underlying geometric representation and mutually reinforce; its zero-shot sim-to- real generalization—reconstruction performance rivalling or surpassing domain-specific models even without seen real data—demonstrates the feasibility of geometric understand- ing as a transferable foundational capability. The most noteworthy new paradigm in this direction is to simulate the world directly in 3D asset space rather than simulating in 2D pixel space and then reconstructing 3D. GWM (Gaussian World Model) [13] takes 3D-Gaussian primitives—rather than pixels or latent features—as the world model’s primitive state representation; a DiT-based neural simulator propagates these primitives under actions to reconstruct future 3D scenes, thereby supporting model- based RL policy training in native 3D space. Its advantages are built-in geometric consistency (3D Gaussians natively carry spatial coordinates), physical interaction modelled in 3D space to avoid 2D-projection ambiguity, and out- put directly usable for the 3D positions, orientations, and occupancy needed by downstream robotic manipulation. The Gaussian Action Field [132] further couples action and asset on the same 3D-Gaussian representation: it drives the deformation of 3D Gaussians directly by gripper pose, defining robot action as an explicit spatial transformation of the Gaussian field, so that the evolution of the 4D dynamic world representation and the manipulation action align in the same geometric space rather than being conveyed indirectly through pixels or latent vectors. PlayerOne [133] represents another path—reconstructing the world from a first-person view: given a user’s egocentric scene image, it builds a corresponding explorable world through a coarse- to-fine training pipeline (large-scale pretraining plus scene- specific fine-tuning) and generates egocentric video strictly aligned with the user’s real motion, its 4D reconstruction guaranteeing scene consistency across long video sequences. GigaWorld-0 [20] positions the world model as a complete data engine for embodied AI, where GigaWorld-0-Video generates diverse, texture-rich embodied video sequences under fine-grained control of appearance, camera view, and scene layout, and GigaWorld-0-3DGS generates 3D scenes in a physically plausible way; together they provide con- trollable, diverse training data for VLA learning, elevat- ing asset generation from visual display to the level of downstream function. In driving, the Xiaomi Auto World Model [134] unifies reconstruction and generation in one industrial-grade framework: the reconstruction route yields high-fidelity, editable static-scene assets, while the genera- FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202612 world elements terrain buildings vegetation roads dynamic agents edit createrecompose persistent 4D scene workspace dynamic agents vegetation buildings roads terrain instance-level edit (selected building) move recolor delete persistent over time(world model) t+1t+2 ... recompose recomposed world rollout t t+1 t+2 t+3 time Fig. 6. A unified platform for persistent and editable world assets. Terrain, buildings, vegetation, roads, and dynamic agents are maintained as separable layers; instance-level edits update the persistent scene before the world model generates a recomposed rollout. tion route fills in rare operating conditions, demonstrating the fusion of asset construction and dynamic simulation in a real production line. WorldGen [135] organizes scene elements (terrain, buildings, vegetation, roads) in an ed- itable layered structure, supporting instance-level control of generated assets—an important advance on the key dimension of editability of generated assets. MagiCity4D [136] advances the same goal at city scale: an MLLM agent parses language and layout sketches to drive procedural content generation (PCG), producing real-time-editable 4D city scenes and letting users add, remove, or adjust build- ings after generation, directly answering the earlier-noted gap that asset conditions cannot be queried or edited once injected. Playable Environments [137] pioneeringly learns interactive 3D scenes from monocular video, learning a per- frame environment-state representation manipulated by an action module and rendered by a style-modulated NeRF, demonstrating the feasibility of inversely inferring an inter- active 3D environment from video observation. GameGen-X [5], the first DiT designed for open-world game-video gener- ation and interactive control, unifies text, keyboard/mouse, and visual prompts via InstructNet to interactively control new characters, dynamic environments, complex actions, and diverse events. OpenGame [113] approaches from the entirely different angle of code generation, letting an LLM agent generate a playable web game via a template skill library and a debugging protocol library—demonstrating another possibility for asset construction: not generating visual assets, but generating the code that creates assets. Figure 6 shows the stronger platform-level requirement implied by these systems: assets should remain decom- posed, persistent, and editable while subsequent world evolution is recomposed from the modified scene. The gap in asset construction is essentially the chasm between implicit and explicit, operable representations. Almost all 3D scenes generated by world models are stored implicitly or semi-implicitly—as NeRF weights [138], 3D-Gaussian sets [64], MLP latent features, or even the diffusion-model parameters themselves—lacking the instance-level decomposability that downstream editing tools require. Users cannot individually select, edit, or re- place an object as in Unreal Engine [69], cannot export to standard mesh, material, and collision formats, and cannot modify scene parameters mid-generation. This chasm is not merely an engineering format-conversion problem; it re- flects a core design trade-off: current generative models take end-to-end differentiability as the highest principle, whereas editability inherently requires modularity and interruptibil- ity. How to introduce a structured, editable intermediate representation while preserving generation quality—even at the cost of partially sacrificing end-to-end differentiability— is key to a breakthrough in asset construction. 4.2 Physics Engine The physics engine is the core of a traditional simula- tor, precisely and deterministically simulating the motion and interaction of objects under forces through numerical solvers. Formal guarantees such as energy and momen- tum conservation, collision impenetrability, and friction- cone constraints form the unshakeable cornerstone of scien- tific computing and safety-critical simulation. The physical- law simulation capability of world models is one of the most core—and most worrying—dimensions for assessing whether they approach a rigorous simulator. Of the 200 papers, only 34 (17.0%) list the physics engine as a principal contribution dimension. Physical correctness is, in most current research, treated as post-processing, downstream verification, or future work, rather than a first-class design goal of the architecture. The most natural framework for analysing the physics- engine capability is to proceed by the strictness of physical constraints, from strong to weak. The strictest is to build the world model itself as a differentiable physics engine (hard physics); the intermediate level injects physical con- FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202613 state physics computes STATE differentiable physics core object-centric state obj 1 obj 2 obj 3 state dynamics computes world dynamics actions gradients flow through physics (differentiable) 푝 1 푡 푣 1 푡 푓 1 푡 푝 2 푡 푣 2 푡 푓 2 푡 푝 3 푡 푣 3 푡 푓 3 푡 푝 1 푡+1 푣 1 푡+1 푓 1 푡+1 푝 2 푡+1 푣 2 푡+1 푓 2 푡+1 푝 3 푡+1 푣 3 푡+1 푓 3 푡+1 푝 1 푡+2 푣 1 푡+2 푓 1 푡+2 푝 2 푡+2 푣 2 푡+2 푓 2 푡+2 푝 3 푡+2 푣 3 푡+2 푓 3 푡+2 푢 푡 ,푢 푡+1 ,푢 푡+2 ... renderer draws OBSERVATIONS neural renderer (state-guided diffusion) render observationsobserved frames diffusion steps state conditioning (cross-attention) encoder denoiser (UNet) decoder renderer only draws state (no physics, no dynamics) temporal rollout of observations ... noise 푁(0,1) t t +1 t +2 t +3 Fig. 7. A world model with an explicit differentiable physics core. Object-centric position, velocity, and force states are advanced under actions with gradients propagated through the dynamics; a state-guided neural renderer then converts the physical trajectory into observed frames. straints as soft guidance into the generation process (soft constraints); and the outermost only verifies physical plausi- bility post hoc through benchmark evaluation (physical eval- uation). These three levels correspond not only to decreasing physical strictness but also to a shift in view from physical guarantee to physical inspiration. 4.2.1 Hard Physics Figure 7 separates physical state evolution from observation rendering: the differentiable core computes the dynamics, and the neural renderer only visualizes the resulting state. This is the most radical direction: not content to make the model look as if it obeys physics, it makes the model itself take on the role of a physics engine. OrbiSim [10] is representative, with an architecture of two cooperating modules. OrbiSim-Dynamics is an object-centric differen- tiable physics core that computes physical evolution (po- sition, velocity, object configuration) in an explicit state space, where all intermediate physical quantities are acces- sible and differentiable, so gradients can propagate from downstream policy loss all the way through the physics simulation back to action and physical parameters. OrbiSim- Vision renders visual observations from the physical state via state-guided diffusion, where the diffusion model acts as a neural renderer rather than a world simulator, and physical correctness is guaranteed by the Dynamics module, not the Diffusion module. This differentiable-physics-core- plus-neural-renderer architecture strikes a delicate balance between physical fidelity (explicit physical state and veri- fiable physical constraints) and visual quality (the genera- tive power of diffusion), while creating two abilities that traditional physics engines lack: differentiability (support- ing gradient-based policy optimization and system identi- fication) and physical-parameter inference (inferring mass, friction, elasticity, and other unobservable physical quanti- ties from observations). In experiments, OrbiSim markedly surpasses existing world models in both prediction fidelity and control performance, and its sustained responsiveness to asset configuration and physical parameters indicates its potential as a differentiable tool supporting robotic simula- tion and policy training. PIN-WM [82] approaches from the perspective of physics-informed networks (PINNs), a variant of the hard- physics route: rather than building a full differentiable physics engine, it embeds physical equations as constraint terms in neural-network training. Its core design treats the physical parameters of 3D rigid-body dynamics (mass, friction coefficient, elastic modulus) as learnable variables and jointly optimizes two losses during training—the world model’s prediction loss (how close the predicted next state is to the true next state) and the physical-equation resid- ual (how much the predicted state trajectory violates the Newton–Euler equations). The significance is that, even without a full physics solver, using known physical equa- tions as a soft regularizer rather than a hard constraint can pull model predictions toward physically reasonable regions. Its differentiable physical-identification capability, recovering environmental physical parameters from visual observations, has direct value in robotic scenes requiring sim-to-real transfer. Prompting-with-the-Future [139] goes further still: rather than trying to teach a neural network physics, it directly builds an interactive digital twin, com- puting action consequences precisely with a URDF physics engine and having a VLM act as a judge to select actions in a closed-loop MPC. This essentially pushes the hard- physics route to the extreme—physical correctness is fully guaranteed by an external engine, and the world model recedes to providing semantic evaluation and visual priors, at the cost of the open-domain generalization of a pure gen- erative route. The deformable-object manipulation world model [140] pushes hard-physics formal guarantees from rigid to soft bodies: it builds a perception-dynamics model with a Neural ODE, forms closed-loop control through perception gradients, and, for the first time for deformable- object manipulation, provides convergence criteria and a Lyapunov stability proof—stability guarantees exceedingly FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202614 pretrained geometry / physics foundation model physical-plausibility constraint distilled physical prior 3D scene-flow / 4D correspondence field video-diffusion world model noise denoised latent • conditions (HDMap, 3D boxes, etc.) actions (controls) time index physically-plausible evolution t = 0 t = 1 t = 2 time Fig. 8. Soft physical constraints in a generative world model. A pretrained geometry or physics model supplies a dense scene-flow or 4D-correspondence prior as a training constraint, while the conditioned video-diffusion model remains responsible for generating the evolving observations. rare in the generative-world-model literature. 4.2.2 Soft Constraints The alternative in Figure 8 retains generative dynamics but constrains learning with distilled geometric or physical priors, improving plausibility without providing a formal physical guarantee. Unlike building a physics engine from scratch, the vast majority of works inject physical constraints into an ex- isting video-generation model—not changing the genera- tive paradigm of the diffusion or autoregressive model, but adding physical guidance in the training objective or architecture. This sacrifices formal physical guarantees but retains the visual quality and open-domain capability of generative models. GEM-4D [141] represents an important methodology of physical soft constraints: rather than modelling physics from scratch, it distils geometric and physical knowledge from an existing pretrained model. Specifically, GEM-4D distils dense 4D-correspondence supervision—a per-pixel map of where each pixel should move in the next frame— from a pretrained geometric foundation model and explic- itly injects this supervision into training the video-diffusion backbone. The elegance is that dense 4D correspondence en- codes the scene’s 3D geometric structure and motion, forc- ing the diffusion model to maintain geometric consistency while generating appearance. Its inverse-dynamics module further converts geometrically consistent video rollouts into executable robot trajectories, raising the success rate of real- world robotic manipulation from 61% to 81%. This 20- point gain is not merely a performance number; it directly quantifies the causal chain from geometric consistency to physical usability. FlowDreamer [142] shares the idea but goes further, embedding 3D scene flow itself as an explicit motion representation into an RGB-D world model: the model first predicts a 3D displacement field for each point in the scene, then generates the next-frame observation accord- ingly, making the physical continuity of motion an intrinsic constraint of the generation process rather than a post- hoc check. Mediating through explicit geometric motion is closer to the essence of physical evolution than directly regularizing in pixel space. PEVA [83] adopts another physical-soft-constraint strategy—structured kinematics as a prior. PEVA uses whole-body 3D joint pose (a sequence of human 3D skele- tons) rather than an abstract latent vector as the action condition. The human skeleton is itself a physically con- strained system: joint rotations are limited by anatomical ranges (an elbow cannot bend backward), and limb motion obeys kinematic-chain constraints (the hand’s position is determined by the chained rotation of shoulder, elbow, and wrist); these constraints are already implicitly encoded in the skeleton structure, requiring no inference from pixels. An autoregressive conditional-diffusion model trained on the large-scale egocentric real dataset Nymeria can there- fore learn how physical human motion shapes the first- person visual environment—e.g., a hand pushing a door occludes the view, and the view-sway frequency while walking matches the step frequency—rather than merely the statistical correlation of which pixels change given an action label. Using known physical structure as an architectural prior—not making the model cleverer at learning physics from data, but explicitly encoding physical structure into the model’s input representation—is an important method- ological direction for improving physical capability. ChronoDreamer [103] focuses on a specific but crit- ical physical quantity: contact force. In precise robotic- manipulation tasks such as assembly, insertion, grinding, and wiping, precise perception and control of contact force is central to success, yet from RGB pixels alone it is nearly impossible to infer precisely whether the fingertip applies 3.2 N or 4.1 N. ChronoDreamer trains contact force as an explicit prediction target of the world model, so the model can directly output a structured force-feedback signal for downstream controllers without an extra force-estimation network inferring from pixels—because this inference is FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202615 integrated end-to-end within the world model. Elevating a key physical quantity from implicit inference to explicit output has direct, irreplaceable value for tasks requiring precise force control. Motus [143] handles physical soft constraints from the optical-flow angle: by learning pixel-level incremen- tal actions—an optical-flow field of how much each pixel should move from the current to the next frame—it makes the effect of actions precise to the pixel level. Compared with a global latent-vector action, pixel-level flow provides richer physical-motion information (which regions move, and their direction and speed), achieving 15% and 11–48% performance gains in simulation and real robotic scenes, respectively. A particularly noteworthy soft-constraint strategy uses a physics simulator as a bridge. The core idea of RealWon- der’s PhysBridge component [79] is not to let the generative model face unknown physics directly, but to let a traditional physics simulator act as an intermediate translation layer: given a robot action, the physics simulator first computes the physical consequence in physical space (an optical-flow field and an approximate RGB), and the diffusion model then needs only 4 denoising steps to refine the simula- tor’s approximate output into realistic real-time interactive video. This division—physical correctness guaranteed by the physics simulator, visual quality by the generator—is demonstrated across rigid bodies, fluids, and granular me- dia. The idea is consistent with the design in ReconDreamer- RL [85], where a kinematic model handles physical mod- elling and a diffusion prior handles appearance: physical correctness and visual quality are two distinct optimization goals, and assigning them to different subsystems is more effective than a single model juggling both. On physically-soft-constrained generative backbones, several works explore from different angles. The physics- informed embedding strategy [84] enhances the 3D geo- metric consistency of rendering by jointly training tempo- ral depth prediction and keypoint dynamics, where depth provides the scene’s geometric structure and keypoint dy- namics provide a sparse physical constraint on motion. The frame-level action-conditioning module [144] strengthens the physical coherence of robot–object interaction through fine-grained action-frame alignment (keyboard- and VR- controllable), and its policy evaluation on generated video correlates highly with policy evaluation in the real envi- ronment, meaning that physical plausibility in generated video can, to some extent, substitute for real-environment evaluation. The DPO post-training strategy [24] post-trains the decoder of a 14B DiT, using a decoupled discriminator to directly penalize non-physical behaviour such as clipping and anti-gravity, representing a path that directly optimizes physical plausibility at the training-objective level, and its cross-embodiment action-control results indicate that phys- ical constraints can transfer across robot platforms. WoW [145] introduces VLM guidance on a comparably large (14B) generative world model, using an external vision-language model to give feedback-style corrections on the physical plausibility of generated results and improving physical realism from embodied-interaction data—a new form of soft constraint in which a strong model judges a weaker model’s output. 4.2.3 Physical Evaluation When physics is assessed rather than enforced, paired causal interventions provide a more diagnostic test than visual realism alone, as shown in Figure 9. Whereas the hard-physics and soft-constraint routes above try to improve physical plausibility during gener- ation, physical-evaluation benchmarks approach from an- other angle. What-If World [23] uses 319 causal-intervention prompt pairs to test whether changing a physical variable produces the expected change while preserving the shared scene. Its APEO scoring covers adherence, physical consis- tency, environment preservation, and outcome difference. In its evaluated set, none of nine models exceeds a 52% paired score and the reported open-source result reaches 28%. Scores are also lower for visually subtle interventions, such as changing mass (14.2%), than for visually salient changes to colour or size (40.4%). These results indicate that visual salience may confound causal-physics evaluation and motivate more diagnostic testing rather than establishing a universal ranking of physical understanding. ReactSim-Bench [146] introduces a dedicated reactivity evaluation in the autonomous-driving domain; its core de- sign decouples the ego vehicle’s behaviour from the sim- ulation of surrounding agents, using AV behaviour from an external planning model as an independent input to check whether a behaviour world model reacts physically reasonably to ego behaviour that deviates from the training distribution. This directly targets a core blind spot of data- driven behaviour simulators: they may be highly realistic within the training distribution but wholly unreasonable out of distribution. WorldBench [110] further provides a diagnostic evaluation of single physical concepts, isolating physical constants such as friction and viscosity, and intu- itive physics understanding, so researchers can locate on which specific physical concept a model is weakest. The core gap in the physics-engine capability stems from a choice of technical orientation. A differentiable physics engine pursues precision and formal guarantees, but the physical phenomena it can simulate are limited to the forms of known equations—it can only simulate phenom- ena whose equations have been written down; a data- driven generative world model pursues flexibility and open- domain generalization, but what it learns may be a visual shortcut rather than a true physical law. There is tension between the two, and it cannot be eliminated by enlarging models or adding data. At least two fusion strategies are emerging: functional separation, where a physics simulator guarantees physical correctness and a generator guaran- tees visual quality (e.g., PhysBridge [79], ReconDreamer- RL [85]); and physical parameterization, embedding key physical parameters as learnable variables in the generative model (e.g., PIN-WM [82], OrbiSim [10]). Beyond these two empirically demonstrated strategies, there are also pro- posals from a formal-modelling angle, such as building a physics-consistent world model via Schr ̈ odinger-bridge optimal transport [147], which seeks to impose physical- consistency constraints on state evolution through the math- ematical structure of optimal transport, though it remains at the method-proposal stage. Their commonality is turning physics from a learning target into an architectural com- FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202616 1. CONDITIONS & ACTIONS 2. WORLD MODEL (NEURAL SIMULATOR) 3. PAIRED TEST (SAME SCENE, ONE CHANGE) 4. PHYSICS COMPARATOR (MEASURE) A. BASELINE ROLLOUT B. INTERVENED ROLLOUT: INCREASE OBJECT MASS Initial state Scene, object, environment Action sequence ... e.g., push the block World model Takes conditions & actions, outputs evolving world (observations over time) t = 0t = 1t = 2 ... t = T time Object mass: t = 0t = 1t = 2 ... t = T time Object mass: Paired causal intervention: increase object mass is the DIFFERENCE physically correct? PHYSICALLY CORRECT PHYSICALLY INCORRECT vs. PHYSICS SCORE (0–1) 0 0.82 1 Higher score = more physically correct differences m’＞m m Fig. 9. Physics evaluation through paired causal intervention. A baseline and an intervened rollout share the scene and action sequence but differ in one physical variable, such as object mass; evaluation asks whether the resulting difference follows the expected physical consequence. ponent: the model no longer learns physics only implicitly from data, but reserves an explicit place for physics in the architecture. 4.3 Interaction Interaction is the essential feature distinguishing a simulator from a pure content-generation engine: a simulator must accept an agent’s action input at any moment, update the world state in real time or near-real time, and feed the up- dated observation back to the agent, forming a closed loop. World models have made the most striking progress here among the eight capabilities. Of the 200 papers, 80 (40.0%) list interaction as a principal contribution dimension, mak- ing it the second most studied capability after controllability. The analysis of interaction unfolds most naturally along two dimensions—the object of interaction (what can be in- teracted with) and the depth of interaction (the granularity and closed-loop nature)—because these two dimensions are precisely what separate an interactive content browser from a programmable simulation environment. 4.3.1 The Object of Interaction The spectrum of interaction objects extends from the most basic camera control to object manipulation and then to multi-agent coordination, each extension marking the world model’s evolution from a passive observer to an active participant. Camera-level interaction is the most frequently demon- strated interaction form in the included implementation pa- pers. Its basic closed loop is shown in Figure 10: a navigation action updates the model’s temporal state and returns the next observation from the requested viewpoint. GameNGen [2], as a proof of concept, showed that a diffusion model can provide real-time interaction in a constrained game domain: it uses two-stage training—an RL agent first self- plays in VizDoom [148] to collect 900 million frames of state and action, then an autoregressive diffusion model adapted from Stable Diffusion v1.4 [149] learns to predict the next frame—achieving about 20-FPS real-time interaction with 4-step DDIM sampling on a single TPU-v5. GameNGen is essentially an offline policy-distillation system relying on a traditional engine to generate training data and applies to only a single game, but its conceptual significance is proving that neural rendering can functionally substitute for a traditional pipeline within a specific constrained do- main. DIAMOND [51] demonstrates two uses of diffusion world models: on the Atari 100k benchmark, an RL agent trained inside the learned model reaches strong reported performance (mean HNS 1.46), while also scaling to 87 hours of CS:GO data as an interactive neural game engine, showing that interactive generation and functional simu- lation can coexist in one framework. Matrix-Game 3.0 [4] pushes camera interaction to industrial grade: through an industrial-grade data engine (UE5 and A-game capture plus real-world augmentation), residual-prediction long- horizon consistency training, DMD multi-segment autore- gressive distillation, and 75% pruning of the VAE decoder, its 5B model generates at about 40 FPS at 720p resolution while maintaining minute-level memory consistency; its predecessor Matrix-Game 2.0 [53] already reached 720p, 24 FPS via a continuous-plus-discrete dual-action representa- tion and context forcing. miniWM [56] provides a full-stack open-source alternative, converting existing bidirectional video-foundation models (e.g., Wan2.1, HY1.5) into few-step autoregressive generators through causal forcing and its im- proved training pipeline, covering the complete flow from data construction to streaming inference and providing a reproducible research substrate for academic teams without industrial-grade resources. On the real-time front of camera interaction, several systems have made engineering breakthroughs. Hunyuan- GameCraft 2.0 [52] achieves 1080P, 60-FPS real-time stream- ing infinite interaction via a low-latency 3D-VAE and KV cache, and decouples mechanisms such as physics and game logic from appearance rendering for editability—essentially a neural re-creation of the traditional game engine’s game- FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202617 real-time, e.g. 40 FPS user / controller navigation action move turn zoom move WASD turn ←→ zoom QE real-time world model neural simulator navigation action aₜ current observation at time t Encoder (f enc ) Latent state zₜ Decoder (f dec ) predicted next observation at time t+1 maintains temporal state (memory) updates state and predicts the next observation every time step interactive viewpoint rollout next observed frame (observations over time) t+1t+2t+3 time (evolving viewpoint) viewpoint change indicators forward movement yaw (turn) zoom (in / out) action in → next frame out Fig. 10. Camera-level interaction in a real-time world model. Controller inputs such as movement, turning, and zoom update the model state and produce the next observation, forming an action-in–frame-out loop for interactive viewpoint navigation. logic-loop-plus-rendering-pipeline architecture. The few- step autoregressive diffusion model [57] reaches minute- level long-horizon stable generation at 25 FPS through distillation. Block-diffusion semi-autoregressive inference [150] achieves long-video stability via KV-cache chunked decoding and integrates the LV-Bench benchmark. The open-source high-fidelity real-time world simulator [95] achieves minute-level long-term memory and sub-second- latency 16-fps interaction. Training-free attention acceler- ation [96] maintains stable memory footprint over long rollouts through temporal-cache compression and sparse attention. This lineage of real-time infinite interaction also includes several representative systems: The Matrix [151] achieves infinite-horizon, real-time controllable high-fidelity real-scene world simulation; MineWorld [152] runs as a real- time, open-source autoregressive interactive world model on Minecraft, providing a reproducible baseline for the community; Matrix-Game [153] offers precise action and camera control as an interactive game world foundation model; Yume [154] realizes infinite generation via camera quantization plus masked video diffusion plus a memory module; converting a pre-trained video model into a con- trollable world simulator [155] demonstrates the path of reusing large-scale video priors to obtain interaction capa- bility at low cost; and Peekaboo [156] gives a training-free masked-diffusion interactive video-generation scheme that injects interactive control without retraining. A common feature of these works is systematic attention to real-time performance as an engineering constraint—concerned not only with whether interaction is possible, but with whether it is fast and stable enough. Moving from camera navigation to object-level inter- action introduces an additional state-management require- ment. As Figure 11 emphasizes, the action must update a persistent object state rather than only alter the current view. The action vocabulary of many camera-interaction systems in the included corpus is confined to navigation—move, rotate, zoom—while object-level operations such as grasp- ing, opening, pushing, and assembling are less frequently demonstrated. ActWorld [81] systematically identifies the root of this gap as a dual bottleneck. The first is the data bottleneck: human–object interaction videos lack precise, dense annotation, so one cannot know the event timing— that the hand contacts the doorknob at frame 137 and the door begins to rotate at frame 142. ActWorld therefore builds a dataset of 100k human–object interaction videos, each annotated with chain-of-thought-generated per-chunk cap- tions. The second is the memory bottleneck: existing world models’ bias toward recent-history compression discards the event-transition frames that causally determine an ob- ject’s subsequent state, so when a user opens a door, walks away, and returns, the model has forgotten the door is open. ActWorld’s hierarchical action-aware memory architecture routes history compression by interaction importance and maintains event updates and object-identity tokens in a persistent memory bank—designed precisely to address this bottleneck—so the model can both flexibly navigate the camera and perform meaningful interactive manipulation of objects in the scene. WorldCraft [88] solves the same problem via a different path: Normalized World Trajec- tory (NWT) maps a user-drawn object-motion path into a camera-invariant world coordinate system, eliminating the confusion between object motion and camera motion in 2D projection; Spatial-Path LoRA (SP-LoRA) injects object- manipulation capability while preserving the pretrained camera controller; and Trajectory-Anchored State Persis- tence (TASP) treats world trajectories as persistent spatial state, so an object moved out of view can reappear at the correct new position—solving the basic causal-consistency problem that object state should be independent of the observation viewpoint. In game scenes, object-level interaction has also ad- vanced across several works. Genie [15] infers a discrete latent-action space in an unsupervised manner from un- labelled internet videos—letting the model merely watch videos without being told that someone pressed the jump FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202618 real-time, e.g. 40 FPS user / object action open the door action cues select grasp / click open world model neural simulator current observation at time t Encoder Latent state Decoder predicted next observation at time t+1 action a t updates world state and predicts next observation persistent interaction rollout observed frames over time t+1t+2t+3 initial view (door closed) open the door turn camera away turn camera back door remains open! other objects remain stable / unchanged action inupdated world statenext frame out object state persists across time (memory) Fig. 11. Object-level interaction with persistent state. An object action updates the world state, and the modified state must remain valid across subsequent viewpoint changes; in the illustrated sequence, the door remains open after the camera turns away and returns. key—pioneering the paradigm of interactive game-world generation with zero human annotation. Its key is to re- veal that interactivity can emerge from passive observa- tional data: the video sequence itself contains informa- tion about actions causing state changes, only implic- itly encoded in pixel changes. Genie-Envisioner [117] ex- tends this paradigm to robotic manipulation, forming a unified world-foundation platform for robotic manipula- tion from GE-Base (multi-view language-conditioned video diffusion), GE-Act (a lightweight action model), GE-Sim (an action-conditioned neural simulator), and EWMBench (an evaluation benchmark). GameFactory [3], through the GF-Minecraft keyboard/mouse-annotated dataset and a domain-adapter training strategy, decouples the open- domain diffusion prior from game-specific style learn- ing and action control, achieving scene-generalizing game- video generation under precise keyboard/mouse control. Playable game generation [157] verifies stable playability af- ter 1000 frames in an autoregressive DiT diffusion and pro- poses the first playability-evaluation framework. The multi- person video world model [158] extends interaction from single- to multi-person, achieving multi-agent synchronized multi-view interaction and long-term memory consistency via Checkpointed Self Forcing long-horizon training, with direct value for games and crowd-simulation scenes requir- ing multi-character simulation. The factorized latent-action model [159] decomposes the scene into independent factors, inferring latent actions per entity or region, improving the precision of multi-agent interaction modelling. In driving scenes, the objects of interaction are other vehicles, pedestrians, cyclists, and other traffic partici- pants, whose behaviour must react physically reasonably to the ego vehicle’s behaviour. OmniDreams [8], adapted by post-training from a Cosmos diffusion model and trained on 21k hours of driving data, is a real-time closed-loop autonomous-driving generative world model; its key break- through is using the world model not only as a sensor simulator generating realistic camera images but also as a policy backbone—its WAM (World-Action Model) variant surpasses the VLA-based Alpamayo 1.5 policy model on the NuRec dataset with one-fifth the parameters, showing the dual potential of a real-time world model as a policy architecture. The closed-loop large driving world model [160] unifies perception, prediction, and planning via next- token prediction, achieving end-to-end closed-loop driving through a position-aware action tokenizer and modelling the whole flow from perception to prediction to planning. Multi-agent configurable traffic simulation [161] proposes a framework generating configurable multi-agent traffic- interaction behaviour from destination plus implicit per- sonality, giving agents aggressive or conservative driving personalities to generate more diverse, more realistic inter- action patterns. 4.3.2 The Depth of Interaction Figure 12 illustrates the interface required to move from human-in-the-loop interaction to algorithm-in-the- loop training: observations and rewards must return through a callable environment API, with reset, seed, and state-query operations available alongside actions. The depth of interaction is the key dimension dis- tinguishing an interactive content browser from a pro- grammable simulation environment. The interaction mode of most existing systems is human-in-the-loop: a human operates in real time via keyboard/mouse, gamepad, or language instruction, and the model generates the next frame from the human input in real time. Algorithm-in-the- loop—e.g., an RL agent calling the world model through a programmatic API for policy training and evaluation— requires wholly different interface capabilities, including structured state queries (return the positions and velocities of all current objects), environment reset (restore the world to its initial state), random-seed control (reproduce the same interaction trajectory with a fixed seed), and reward-signal FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202619 real-time, e.g. 40 FPS policy / agent policy update programmatic API action observation reward reset random seed state query world model as environment neural simulator latent (world) state EncoderDecoder maintains state across time evolving environment accepts actionupdates latent state outputs observation and reward simulated rollout (environment rollout) observation at t observation at t+1 observation at t+2 observation at t+3 time (within episode) reward signal +1.2 (example: task progress / return) action observation + reward reset / seed / state query train over episodes (closed-loop learning) training signal / policy update ... 푍 푡 Fig. 12. A world model as a programmable environment. A policy interacts through an API carrying actions, observations, rewards, reset, random seed, and state queries, thereby closing the rollout and policy-update loop required for reinforcement-learning training. computation. These interfaces are standard in traditional simulators but are not consistently documented across the included world-model papers. The few explorations in the algorithm-in-the-loop di- rection deserve special attention. DIAMOND [51] predicts reward and termination signals simultaneously via an in- dependent CNN-LSTM network and trains an RL agent inside the diffusion world model, making it one of the few world models with a complete algorithm-in-the-loop closed loop. The heterogeneous masked autoregressive model [162] models cross-embodiment action-video dynamics through heterogeneous pretraining, its 15× inference speed-up mak- ing it computationally feasible to use the world model as a simulator to evaluate policies. The joint video-action latent representation [163] supports, in a unified latent space, for- ward dynamics (state to next state), inverse dynamics (state to action), and policy (state to action) simultaneously, so a single model can serve the three core functions of forward rollout, action inference, and decision-making. The self- supervised discrete-action bottleneck [164] controls video generation with a discrete action as a bottleneck, achiev- ing frame-by-frame playable, action-controllable video gen- eration, its discrete-bottleneck design making the action space compact and composable. iVideoGPT [165] unifies vision, action, and reward into a token sequence to build an interactive, scalable autoregressive world model, and its explicit modelling of reward tokens makes it a natu- ral fit for algorithm-in-the-loop calls from RL agents. The embodied-space world model [166] shows that, in open- ended navigation tasks, a spatial world model supporting decision-making can emerge spontaneously from sparse- reward training alone, offering an example of how the world representation for algorithm-in-the-loop interaction can be acquired from the task itself. A more fundamental interaction capability, hallmark of a traditional simulator, is action injection at any moment—the user can issue a new action command at any time during simulation, not only at the start. Pandora [93] makes an important advance here, allowing language instructions to be injected at any moment of the rollout rather than only at the start of the sequence, enabling interactive mid-course intervention—e.g., opening a door in the scene at frame 100. DreamX-World 1.0 [167] further enables composable run-time event triggering via event-instruction fine-tuning. Imaginative egocentric 3D exploration [101] extends such anytime interaction to long-horizon exploration: the agent updates its belief about the world in continuously gen- erated new observations and uses the updated belief to help an LLM make the next decision, forming a cognition– interaction closed loop. The video-diffusion adaptation [168] converts a pretrained video-diffusion model into an action- conditioned world model via a lightweight adapter without accessing the original model parameters, a plug-and-play adaptation strategy with practical value for lowering the technical threshold of algorithm-in-the-loop interaction. The gap in interaction is not whether interaction is possible—on the camera-navigation dimension the answer is already yes, and real-time performance (40 FPS [4]) even exceeds that of some traditional simulators. The gap con- centrates in three areas: the breadth of interaction objects, where object-level manipulation, physical-contact feedback, and multi-agent coordination remain nascent; the shift of interaction paradigm, where moving from human-in-the- loop to algorithm-in-the-loop requires completing interfaces for structured queries, environment reset, random seeds, and reward computation; and the persistence of interaction, where whether a door opened a thousand steps ago is still open remains a difficulty for world models relying on implicit memory. 4.4 Controllability Controllability is one of the fundamental attributes dis- tinguishing a simulator from a free generative model. FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202620 1 2 3 Explicit control Language & multimodal control Latent actions keyboard keys mouse displacement camera pose 3D trajectory action-conditioned world model high precision limited transfer direct signals with explicit physical meaning image cue trajectory sketch multimodal world model human-readable intent ambiguous physical execution map semantic intent into world evolution unlabeled video (observed) latent-action learner latent code (compact) transfer to different environment learned from passive video transferable but not directly interpretable discover action structure without action labels more precise more abstract less general more general t... t + T language instruction “push the block to the table edge” t... t + T t... t + T ... world model t... t + T Fig. 13. Controllability of world models across action-representation levels. Explicit signals such as keyboard input, camera pose, and 3D trajectories provide precise control but transfer poorly; language and multimodal conditions express human-readable intent while leaving physical execution ambiguous; and latent actions can be learned from passive video and transferred across environments, but are not directly interpretable. Moving from explicit to latent control therefore trades physical precision for abstraction and generality, making cross-level action mapping an open problem. A traditional simulator provides fully deterministic con- trol through a parametric API: set an object’s position to (1.0, 2.0, 3.0) and it sits precisely at that coordinate. The con- trollability of a world model faces a peculiar challenge: the action space is usually not a numerical physical parameter but a camera pose, language instruction, keyboard/mouse operation, or abstract latent variable, which introduces a fundamental trade-off between control precision and control generalization. Of the 200 papers, 125 (62.5%) list control- lability as a principal contribution dimension, making it the most widely studied capability—reflecting both that controllability is a common need across application sce- narios and the fundamental difficulty of making a system that learns from data and inherently relies on stochastic sampling controllable. The core tension of controllability lies between precision and generalization; it runs through the architectural design of all world models and exhibits a systematic trade-off pattern along the axis of the abstraction level of the action representation. We organize the analysis along this axis from most precise to most abstract, because the abstraction level of the action directly determines what granularity a user can control and how widely control generalizes. Figure 13 summarizes this continuum and the corresponding shift from physically interpretable commands to learned but less transparent action codes. 4.4.1 Explicit Control At the most precise end, actions are represented as sig- nals with clear physical meaning, including keyboard keys, mouse displacement, camera pose, and 3D trajectory coor- dinates. This class of control is closest to the API-call mode of a traditional simulator—the user specifies precisely what is desired and the system executes it precisely. DisCo [25] poses a fundamental challenge to the main- stream scheme of using a continuous camera trajectory as the action condition. Its core finding is that continuous rep- resentations make the feature similarity between different motion patterns too high—the trajectories of advance 1.0 m and advance 1.1 m are nearly indistinguishable in con- tinuous space—causing the model to confuse these close- but-different motion commands during denoising. DisCo’s solution is to discretize the action space into a compact set of motion primitives, such as turn left 30 ◦ , advance, stop, which are well separated in feature space, thereby greatly improving the reliability of action-following. The signifi- cance exceeds the specific scheme: it shows that in certain dimensions, coarse-grained discretization may outperform fine-grained continuity, because separability matters more than resolution. Prisma-World [75] extends control from single- to multi-agent: via multi-agent RoPE distinguishing agent identity while keeping synchronized time coordinates, injecting an attention bias from relative camera geometry to steer overlapping viewpoints toward shared scene evidence, and an overlap-decay curriculum, it achieves cross-view consistent generation for any number of agents under com- plex camera trajectories. This has direct value for simulation scenes requiring multi-sensor fusion (e.g., surround-view multi-camera for autonomous driving) and multi-character game simulation requiring per-character behaviour specifi- cation. A new dimension of control extension is multi-subject action binding. The subject-state token plus spatial bias [169] decouples frame rendering from individual action updates, maintaining each player’s action-following pre- cision while controlling 7 players on the same screen— technically non-trivial, because without decoupling the ac- tion signals of 7 players would interfere in the shared pixel space. On the driving-control side, multi-view la- FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202621 tent diffusion [92] generates multi-camera spatio-temporally consistent driving video with fine-grained control of ego, agents, and semantics, covering rare scenes. Pose-memory retrieval plus frame-level action control [170] achieves over- 20-second long-term consistency without real-robot data, with direct value for robot-policy-ranking evaluation. The GPT-style driving world model [171] achieves 40-second stable drift-resistant generation via spatio-temporal fusion and mask reweighting. Drive-WM [172] generates multi- view controllable driving video through view-factorized joint spatio-temporal modelling and applies it to look-ahead planning, embodying the idea of controllable generation serving decision-making directly; EOT-WM [173] argues that controlling the ego trajectory alone is insufficient and further models ego and other-vehicle trajectories jointly, extending controllability to the interactions among traffic participants. ProphetDWM [174] rollingly predicts future actions and video in driving scenes; its closed-loop rolling- prediction mode from action to video to action essentially re-creates the step loop of a traditional simulator inside the world model: given the current state and action, predict the next state, then select a new action based on the new state, and so on. 4.4.2 Language and Multimodal Control Language instruction as an action condition has a natural advantage—humans can express intent intuitively—but also a natural disadvantage: a natural-language instruction such as push the block to the edge of the table cannot precisely specify the end-effector trajectory. The core challenge of language control is the mapping from semantics to physics. An important contribution of Pandora [93] is to solve the problem of when control can happen, allowing language instructions to be injected at any moment of the rollout rather than only at the start; this anytime-controllable in- teraction mode is closer to how a simulator works than one- off video generation. DreamX-World 1.0 [167], via event- instruction fine-tuning and RL alignment, restores cam- era control and visual quality after distillation, and its lightweight projected positional encoding E-PRoPE retains the ability to project camera geometry while reducing com- putation. The instruction-driven interactive game world model [175] controls camera, character, and environment via text injection and is accompanied by the InterBench evalua- tion benchmark. Text-controlled world events plus keyboard interaction [176] combines text-generated events (e.g., a car approaching from the left) with keyboard interaction (e.g., pressing the right-turn key), achieving a continuously explorable world through context compression. Control- lable panoramic video generation [177] achieves trajectory- controllable long-range scene roaming through a preview- and-refine two-stage strategy, first quickly generating a coarse panoramic point cloud at low resolution and then finely rendering at high resolution. On the driving-control side, ADriver-I [7] proposes the concept of interleaved vision-action pairs, unifying visual features and control signals into one format and, through an autoregressive loop of predicting the current-frame con- trol signal, conditionally generating the next frame, and then predicting from the new frame, achieving continuous driving in its self-generated world. DrivingGPT [9] unifies driving world modelling and planning via image-action- token interleaved autoregression, surpassing strong base- lines on nuPlan and NAVSIM through standard next-token prediction on multimodal driving language, proving that world modelling and behaviour planning can be solved in the same sequence-modelling framework. The unified vision-language-world-action model [178] further extends this idea, achieving personalized driving planning (e.g., choosing a scenic route or avoiding all toll booths) and tra- jectory generation in an instruction-driven manner, making the driving world model not only an environment simulator but also a behaviour decision-maker. UniDrive-WM [179] follows a closely related, VLM-driven route, unifying scene understanding, behaviour planning, and video generation in one driving world model so that language-level intent propagates consistently to controllable generation and plan- ning output. On the robotic-control side, UniPi [180] proposes a con- cise framework reformulating the policy problem as text- conditioned video generation: generate a video as a plan- ning blueprint, then extract control actions from it for execu- tion, achieving cross-task, cross-environment compositional generalization; this works because the compositionality of the video-generation space is far higher than that of the action space, and synthesizing an unseen task in video space is more feasible than in action space. AVDC [181], via the entirely different path of dense inter-frame correspondence rather than text guidance, infers actions from only RGB video and a text goal, requiring no action annotation. Its methodological significance is proving that action, as a kind of information, can be inversely inferred from state changes: given the before and after states, what action caused the change is learnable. The earlier Diffuser [182] pioneered the paradigm of flexible behaviour synthesis by iteratively denoising an entire trajectory with a diffusion model, grounding controllability in direct modelling of the trajectory distribution. Building on this, several works push controllability to a finer robotic-manipulation granularity: ManipDreamer [183] organizes a manipulation world model with an action tree plus multimodal visual guidance, so a complex operation can be decomposed into controllable sub-steps; STARRY [184] proposes action-centric spatio- temporal world modelling with a geometry-aware attention (GASAM), improving the geometric accuracy with which actions control scene evolution; and STORM [185] com- bines a diffusion VLA, video prediction, and MCTS search into a search-guided generative world model, raising the controllability and success rate of manipulation actions by searching within the generated imagination space. 4.4.3 Latent Actions At the most abstract end, actions are represented as learned latent variables, with the model itself deciding what consti- tutes a meaningful action unit, and humans unable to spec- ify them directly. The core progress of the latent-action route can be understood as an evolution from finding actions, to understanding actions, to making actions meaningful. Genie [15] pioneered this paradigm, inferring a discrete latent-action space in an unsupervised manner from unla- belled internet videos. Genie’s key contribution is proving that latent actions can emerge from passive video data: FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202622 without anyone telling the model that the user pressed the left key here, the model automatically discovers action boundaries by judging which pixel changes are predictable, consistent, and compressible into a discrete code. But Ge- nie’s limitation is also here: the latent actions it discovers are independent per training environment, and a jump learned in a platformer cannot transfer to a racing game. AdaWorld [91] advances a step, proposing transferable latent-action learning so that latent-action representations learned in one environment can transfer to another. Motus [143] bridges latent action and the physical world, learning pixel-level in- cremental actions through optical flow so that latent actions can be decoded into an interpretable physical-motion field. DiLA [90] represents the deepest theoretical advance of the latent-action route so far, revealing and exploiting a previously under-recognized fact: the decoupling of con- tent and structure co-evolves with latent-action learning. DiLA’s core insight is that the prediction bottleneck in latent-action learning is itself the driver of decoupling. To compress future state changes into a limited latent-action code, the model is forced to separate what changes (struc- ture, i.e., spatial layout, object position) from what stays constant (content, i.e., texture, colour, style) into different pathways, so that the latent-action codes in the structure pathway naturally align to physically meaningful changes. This symbiosis—decoupling drives action learning, and action learning in turn reinforces decoupling—lets DiLA simultaneously achieve a highly abstract semantic action space and high-fidelity visual generation, two goals previ- ously thought mutually constraining. The principle DiLA reveals is of general significance: the quality of latent actions may ultimately depend on how well the model separates scene content from structure, rather than on the engineering design of the action space itself. This suggests the key to improving controllability may lie more in the decoupling mechanism. After DiLA, the latent-action route continues to deepen in several directions. Learning continuous-constrained la- tent actions from in-the-wild video [186] captures the com- plexity of in-the-wild actions, no longer confined to con- trolled game or robot scenes, and uses the learned latent ac- tions as a planning interface. Sequence control-effect align- ment [187] proposes a strategy of aligning action semantics by anchoring temporal feature differences, achieving zero- shot action transfer—a push learned in one environment transfers zero-shot to a push in another. The hierarchical latent-action model [188] further aggregates low-level latent actions into high-level skills, modelling multi-level temporal structure in long-horizon control tasks. Shared latent ac- tion [189] reduces annotation-sample demand by an order of magnitude by aligning control signals with passively observed actions, of significant practical value in robotic applications with scarce annotated data. On the downstream utility of latent actions, Being-H0.7 [190] proposes a concise dual-branch training design: during training, the posterior branch uses embeddings of future observations to provide prescient information, while the prior branch uses only the current context to provide de- ployable information, and the two are aligned in latent space; at inference, the posterior branch is discarded and only the prior branch is used, thereby gaining the future- aware advantage of a world model while preserving VLA deployment efficiency. This train-time-fuse, inference-time- separate design may offer a general paradigm for unifying predictive world models and efficient VLA policies. The unsolved challenges of controllability concentrate on mapping problems at two levels. The first is cross- abstraction-level mapping: how to build a bidirectional mapping among explicit control (precise but poorly gen- eralizing), language instruction (semantically readable but low precision), and latent action (well generalizing but unreadable)? This requires a world model to simultane- ously learn action representations of multiple granulari- ties and their translation relations. The second is temporal compositionality—e.g., press the left key for 5 seconds, then the right key for 3 seconds; current systems remain unstable in supporting such temporally composed actions, because the segments of a composed action may be causally disconnected, with the conditional observation of the sec- ond segment already deviating from the true world state. This is essentially a problem between open-loop control and closed-loop feedback: a traditional simulator natively supports closed-loop feedback where the current state in- fluences the next action, whereas current world models are more often open-loop sequence continuation. So far only V- JEPA 2-AC’s [11] receding-horizon MPC and WorldVLA’s [65] attention-mask strategy have explored true closed-loop control, and the fundamental challenge of temporal compo- sitionality remains unsolved. 4.5 Stability Stability refers to the reliable consistency of generated re- sults along the temporal and spatial dimensions—no un- warranted drift, distortion, or abrupt change—and it is the key dividing line between a demonstration tool and an engi- neering simulator. A traditional simulator’s stability comes at almost no extra cost: being solver-based, it reproduces identical results infinitely for the same input sequence. A world model’s stability is inherently constrained by its gen- erative paradigm: whether autoregressive frame-by-frame generation or diffusion denoising followed by re-encoding, each step’s tiny error accumulates over long horizons, form- ing a vicious cycle of error, deviation, generate-on-deviation, larger-deviation—i.e., autoregressive drift. Of the 200 papers, 80 (40.0%) list stability as a principal contribution dimen- sion, a high proportion reflecting a broad consensus in the research community: autoregressive drift is the number-one engineering obstacle to the practical use of world models. The analysis of stability is organized most naturally by the mechanism that maintains it, because different mech- anisms represent not only different technical schemes but also different answers to the fundamental question of what really guarantees stability. The stability mechanisms in the current literature fall into four levels: self-adversarial train- ing at the training-strategy layer, implicit and explicit mem- ory at the memory-architecture layer, state persistence at the causal-consistency layer, and drift quantification at the theoretical-guarantee layer. 4.5.1 Self-Adversarial Training The self-forcing mechanism in Figure 14 directly targets exposure bias by replacing clean-only conditioning with the FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202623 clean history ground-truth context t-2 t-1 t autoregressive world model (world model / simulator) without self-forcing train on clean history only with self-forcing train with model's own past outputs (slightly imperfect) self-forcing condition on the model's OWN past outputs corrects Accumulatederror consistent Geometryand layout t t+1 t+2 t+3 t+4 t+5 errors compound over time geometry / layout drift and warp autoregressive drift stable long-horizon rollout t t+1 t+2 t+3 t+4 t+5 training / rollout flow (temporal progression) self-forcing feedback (own past outputs) drift region (compounding error) Fig. 14. Self-forcing training against autoregressive drift. Training only on clean history creates a train–test mismatch under which errors compound during rollout; conditioning on the model’s own past outputs teaches recovery from imperfect context and improves long-horizon geometric consistency. model’s own imperfect rollout history during training. This is a stability-enhancement strategy independently verified as effective by multiple systems. The core idea is that, rather than trying to perfectly eliminate every step’s autoregressive error (impossible in a stochastic-optimization framework), it is better to deliberately expose the model during training to its own imperfect outputs, forcing it to learn correction. Matrix-Game 2.0’s [53] context forcing is a representative implementation: during training it replaces the true history frame with the model’s own previously generated history frame with some probability, as the con- dition, forcing the diffusion model to still generate a rea- sonable next frame when the context has slightly deviated from the true world state. Lyra 2.0’s [21] self-augmented history training follows the same logic, training the model to correct rather than propagate drift by exposing it to its own degraded outputs on long trajectories. DreamX-World 1.0’s [167] self-generated long-horizon context training and residual-recycling mechanism further enhance robustness to imperfect conditions. Checkpointed Self Forcing [158] extends this strategy to multi-person scenes, proving that self-adversarial training is equally effective in more complex interaction scenes. The common insight of these methods distils into a design principle: in a system where errors are unavoidable, the ability to recover matters more than the ability to avoid—essentially a mapping of robust-control thinking into generative models. 4.5.2 Implicit and Explicit Memory Figure 15 contrasts the two principal memory designs: in- ternal caches or tokens provide compact contextual recall, whereas an external geometric store preserves spatial in- formation more persistently at greater computational and storage cost. The essential cause of autoregressive drift is that past information is gradually forgotten in the sequence: if the model does not remember what the initial position looked like, it cannot maintain consistency. Therefore, how to re- member the past is the core of stability-architecture design. Implicit memory retains historical information through the model’s internal state (e.g., RNN hidden state, Trans- former KV cache, or learnable memory tokens) and is the choice of the vast majority of systems, its latest advances embodying delicate engineering innovation. The KV-cache camera-aware memory of RELIC and Oasis [55] retrieves the most relevant historical KV pairs from the cache according to the current viewpoint, letting the model recall the past information most relevant to the current view rather than treating all history equally. DecMem [22] takes a step to- ward theorization, systematically analysing two fundamen- tal limitations of naive learnable memory architectures— computational inefficiency (full attention over long history grows as O(n 2 )) and attention dispersion (attention weights diluted over many irrelevant tokens in long sequences)— and proposing a decoupled memory architecture: sparse global memory achieves O(logn)-level fine-grained history access, and anchored local memory maintains dense at- tention within a local window to guarantee extrapolation quality; this decoupled design makes it one of the best current schemes for minute-level long-video consistency. WorldPack [191] compresses history into compact memory tokens; memory-compressed KV cache [192] maintains long- term consistency in a memory-efficient way and establishes the MAG-Bench benchmark. The GLP architecture [193] fuses an LLM’s latent dynamics with a diffusion decoder, achieving long-horizon consistent simulation and reasoning conditioned on natural-language actions, where the LLM provides abstract long-horizon memory and reasoning and the diffusion model provides concrete visual rendering. Explicit 3D memory takes an entirely different approach: rather than relying on the model’s internal learnable pa- rameters to remember the past, it explicitly stores histor- FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202624 IMPLICIT MEMORY (World model with internal memory) EXPLICIT 3D MEMORY (External geometric store) 1) Observe & Act Actions / Controls 2) Implicit memory in the world model World Model (Neural Simulator) Internal KV-Cache / Memory Tokens Retrieve most view-relevant past tokens 3) Rollout (explore the world over time) t+1t+2t+Ktime 4) Return to the place (revisit) Consistent reconstruction ✓ implicit: compact, fades 1) Observe & Act Actions / Controls 2) Store observations to external 3D memory External 3D Memory Encode Observation Project to 3D Fuse into Map Retrieve by geometry 3) Rollout (explore the world over time) t+1t+2t+Ktime 4) Return to the place (revisit) Consistent reconstruction ✓ explicit: persistent, heavier memory strength time/steps memory strength time/steps Fig. 15. Implicit versus explicit memory for consistent world rollouts. Implicit memory retrieves view-relevant internal states but may fade with horizon length; explicit 3D memory stores observations in a persistent geometric map and retrieves them when the camera revisits a location. ical observations as a persistent 3D spatial representation (e.g., point cloud, voxels, 3D Gaussians). Long-term Spatial Memory [100] represents this route, lifting each frame’s observation into an explicit representation in 3D space and, when generating a new view, retrieving relevant informa- tion from this 3D store through geometry-aware feature retrieval. The fundamental advantage of explicit memory is that its memory does not decay with sequence length— a point placed in 3D space stays there, unconstrained by the number of autoregressive steps; but its cost is also clear: storage overhead grows linearly with scene scale, and updating and evicting stale memory in dynamic scenes is a non-trivial problem requiring additional mechanisms. Per- sistent global-state modelling [97] solves the off-view prob- lem of explicit memory in dynamic scenes by synchronously evolving off-view dynamics through a monitor mechanism, accompanied by the LiveBench benchmark for systematic evaluation. Explicit-3D-memory embodied simulation [194] predicts RGB-D and aggregates it into a persistent 3D map as a condition for generating the environment, preliminarily fusing the advantages of implicit generation and explicit memory: the diffusion model predicts RGB-D to guarantee appearance quality and flexible motion modelling, while the explicit 3D map achieves persistence so memory does not decay. VMem [195] offers a lightweight realization of this route: rather than maintaining a dense point cloud or voxels, it organizes historical-view memory indexed by surfels, retrieving the most relevant historical frames by geometric proximity as conditions when generating a new view, thereby maintaining long-term visual consistency un- der free camera roaming. This sparse explicit memory keyed by surfels strikes a more economical compromise between storage overhead and consistency than dense 3D storage. 4.5.3 Causal Consistency The off-screen persistence test in Figure 16 isolates a basic causal requirement: changing the observer’s view must not reset the underlying world state. This is a special but instructive form of stability: an object’s state (position, orientation, on/off) should be an objective fact independent of whether the observer is watch- ing. WorldCraft’s [88] Trajectory-Anchored State Persistence (TASP) specifically solves this, so a moved object reappears at the correct new position after leaving the camera view rather than returning to its original place because the camera did not see it. This causal-consistency problem seems simple but is highly challenging in autoregressive video generation, because the model has no persistent world-state variable and its memory is entirely implicit and attached to re- cent frames. Prisma-World’s [75] multi-agent joint denoising processes all agents’ video in one full-attention sequence, essentially ensuring that multiple observers see the same world: if two agents’ viewpoints overlap, the overlapping region should show identical scene content. This cross- subject consistency is a natural generalization of temporal consistency to spatial consistency. 4.5.4 Quantification of Drift and Theoretical Guarantees The quantitative characterization of autoregressive drift re- mains poorly standardized across model architectures, scene complexities, and action-sequence lengths. Existing state- ments of long-horizon stability—such as DecMem’s minute- level [22], Matrix-Game 3.0’s minute-level memory consis- tency [4], and the GPT-style driving model’s 40 seconds [171]—all lack a unified evaluation protocol: what counts as staying stable? PSNR not below a threshold, FVD [196] not diverging, or humans unable to tell real from fake? Different papers use different standards and cannot be compared di- rectly. In driving, Fine-flow [197] proposes suppressing drift via coarse-to-fine distillation of optical flow: constraining the global motion trend at coarse granularity first, then pro- gressively refining to local detail, keeping long-horizon gen- erated motion coherent without diverging—a concrete drift- suppression mechanism grounded in motion consistency. FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202625 interaction sequence t+1 object moved to new position camera turns away World Model neural simulator observer moves, world state should not reset persistent world state wrong object snapped back t+2 camera returns object snapped back to old position view changes, state is not preserved correct state persists t+2 camera returns object stays at new position state persists even when off-screen Fig. 16. Causal consistency under viewpoint change. After an object is moved and the camera turns away, a model without persistent state may place the object back at its former location; a consistent world model retains the intervened state when the object is observed again. MiLA [198] attacks the long-horizon-consistency problem from the data and generation end, with multi-view high- fidelity long-term driving-video generation; SWM [98] in- stead anchors generation to a real-world metropolis via retrieval augmentation and stabilizes long-horizon rollouts with a Virtual Lookahead Sink mechanism—the two repre- senting complementary drift-suppression paths in the de- manding driving setting. Within the included corpus, we did not identify an analysis of worst-case theoretical stabil- ity bounds: for a given model architecture and action dis- tribution, is there a theoretical upper bound on the growth rate of autoregressive drift? If so, how do model capacity, training-data scale, and distillation strategy systematically raise this bound? NWM [199] demonstrates the feasibility of using a diffusion world model for trajectory simulation and evaluation in navigation scenes—planning a path through simulation in familiar environments and imagining a navi- gation trajectory from a single image in unfamiliar ones— but its long-horizon stability (how far one can simulate before it becomes unreliable) is not systematically charac- terized. PlaNet’s [43] latent overshooting and DreamerV1’s [44] latent-space imagination, as early milestones of the latent-dynamics route, demonstrated the feasibility of long- horizon imagination in latent space (multi-step prediction not collapsing); MuZero [106], with a rule-free latent model plus tree search, repeatedly invokes its learned dynamics in long-horizon planning without diverging, corroborating from a planning perspective that a compact latent space supports long-horizon stability. But these works are five to seven years old, and whether their stability strategies remain effective or need adjustment in today’s larger-scale models lacks systematic comparative study. The second stability subattribute is run reproducibility: whether a fixed input sequence and a controlled source of randomness produce a repeatable rollout. Stochastic genera- tion does not rule out reproducibility in principle, but the in- cluded corpus provides little directly comparable evidence on exposed seeds, deterministic kernels, tolerance bands, or repeat-run variance. Without such a protocol, A/B test- ing, regression testing, and safety analysis cannot separate model changes from sampling variation. Future evaluations should therefore report both long-horizon consistency and repeat-run reproducibility, rather than treating success on either one as evidence for the other. 4.6 State Feedback State feedback is the nervous system of a traditional simu- lator: a simulator must not only correctly evolve the world state but also return key information about the current state to external callers (human or algorithm) in a structured, consumable format. Traditional simulators have a natural advantage here—ego six-DoF pose, depth maps, LiDAR point clouds, semantic and instance segmentation, collision events, object bounding boxes, reward signals, and ground- truth labels are all explicitly modelled in the simulator design and accessible at any time via API. Of the 200 papers, only 45 (22.5%) list state feedback as a principal contribution dimension—a proportion so low that it points precisely to a clear bias in the current research paradigm. To characterize this absence precisely, we perform a five-variable state-interface audit of all 163 implementation papers. B1 is ego information (camera pose, vehicle speed, robot joint angles); B2 is sensor-level output (depth, se- mantics, LiDAR, normals, occupancy); B3 is task-evaluation signals (reward, termination flag); B4 is a runtime inter- face for queryable, named non-ego entity states or physi- cal parameters, such as an agent’s position, velocity, type, and size or an object’s pose, mass, and friction; and B5 is closed-loop interaction (accepting actions and continuously generating). B4 excludes sensor-space predictions assigned to B2, unreadable latent vectors, and structure used only as an input, training target, or evaluation reference. Table 3 FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202626 TABLE 3 State-interface audit for all 163 implementation papers. “Present”, “absent”, and “unresolved” are reported separately; each row sums to 163. B4 requires a runtime, queryable entity/physics annotation interface and excludes sensor-space estimates, latent vectors, and training-only labels. B5 is a cross-cutting closed-loop attribute. VariableOperational meaningPresentAbsentUnresolved B1Ego information: camera/ego pose, vehicle state, robot joint or end-effector state 657424 B2Sensor output: depth, semantics, LiDAR, normals, occupancy, or compara- ble sensor modality 451135 B3Task signal: reward, termination, or an equivalent task-evaluation signal201376 B4Runtime entity/physics annotations: named non-ego entity states or phys- ical parameters 61570 B5Closed-loop interaction: repeated action-conditioned generation or model- mediated control loop 875620 scene rollout context observations over time t − 2 t − 1 t t + 1 ⋮ time advances → rollout context world-model backbone shared representation encoder latent state (neural simulator) decoder z t−2 z t−1 z t z t+1 rollout in latent space (state feedback) shared world state The latent state z t is the common, evolving representation of the world and the single source for all auxiliary outputs. parallel auxiliary heads(aligned state outputs) prediction head prediction head prediction head prediction head prediction head RGB depth semantic segmentation occupancy grid (top-down) LiDAR points Fig. 17. Structured feedback through multi-task auxiliary outputs. Parallel prediction heads decode a shared temporal representation into aligned RGB, depth, semantic segmentation, occupancy, and LiDAR outputs, providing downstream systems with complementary views of the same simulated state. reports present, absent, and unresolved judgements sepa- rately. Confirmed presence is recorded for B1 in 65 papers, B2 in 45, B3 in 20, B4 in six, and B5 in 87. The paper-level evidence nevertheless exposes an impor- tant interface distinction. Many video-diffusion and autore- gressive systems generate RGB and rely on a downstream perception model to recover structure. This pipeline may be useful, but its estimated output is not equivalent to state read directly from an explicit simulator. The distinction matters for information loss, calibration, and auditability, and it should be reported independently of output modality. Among works that confront the state-feedback problem, three technical strategies can be identified, ordered by how closely their outputs approach the interface of a traditional simulator. These strategies differ not only in output modal- ity but also in whether the returned state is human-readable, physically grounded, or intended primarily for machine consumption. 4.6.1 Multi-Task Auxiliary Output As shown in Figure 17, multi-task prediction heads expose several aligned sensor and scene representations from a shared evolving world state instead of returning RGB alone. The most direct strategy is to add multiple prediction heads on an RGB-generation backbone, simultaneously out- putting complementary modalities such as depth, seman- tics, occupancy, normals, and optical flow. Tesseract [102] demonstrates the feasibility of joint 4D generation of RGB, depth, and normals, learning a unified spatio-temporal rep- resentation from multi-view video and simultaneously out- putting three essentially different but geometrically related modalities. The multimodal egocentric world model [200] outputs RGB and depth while finely controlling objects, ego, and human pose, and proposes the COM (Control Over Multi-objects) metric to measure control precision, taking control precision as a quantifiable feedback-quality metric for the first time. The multimodal autoregressive model [201] simultaneously generates RGB, depth, and arm masks, where depth provides 3D geometric information and the arm mask provides direct localization of the manipu- lation region, offering directly usable structured feedback for robotic-manipulation tasks. In driving, this strategy has recently been especially active: MUVO [202] synchronously predicts LiDAR point clouds and 3D occupancy grids in addition to RGB, aligning the world model’s output di- rectly with the native sensor formats of the autonomous- driving perception stack; DiST-4D [203] outputs metric depth while generating multi-view RGB through decoupled spatio-temporal diffusion, serving as a bridge from pixels to geometry; and GaussianDWM [204] uses 3D Gaussians as FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202627 an intermediate representation to jointly render both RGB and depth. Together these works show that sensor-level feedback (B2) is spreading from a few robotic-manipulation scenes to driving, the application with the largest data scale. UniScene [78] adopts a progressive-generation strategy that goes beyond simply adding prediction heads: it first generates semantic occupancy as a meta-scene represen- tation rich in semantics (class labels) and geometry (3D spatial occupancy), then generates video and LiDAR data respectively conditioned on the occupancy. This architec- ture, from a centralized scene representation to downstream multimodal sensor output, essentially rebuilds in the gen- eration pipeline the information flow of a traditional sim- ulator from world state (occupancy) to multiple sensors (camera, LiDAR), rather than end-to-end generating RGB and outputting other information incidentally. OccLLaMA [105] and OccSora [104] integrate occupancy prediction into a language model and a diffusion framework respectively, demonstrating the portability of occupancy as a general scene representation across different generative backbones. Diffusion-flow on vector maps [99] represents a severely underrated feedback paradigm—outputting an ego-lane and agent vector map rather than pixels. Designing struc- tured vectors rather than pixels as the native output format has a unique advantage for driving simulation requiring de- terministic spatial information (e.g., a surrounding vehicle at 12.3 m travelling at 15.2 m/s), because vector information eliminates the information loss of re-extracting structure from pixels. In LiDAR and semantic prediction, UniScene [78] and street-view generation [119] demonstrate the pos- sibility of scene understanding and semantic information as a natural by-product of the world model rather than an independent task requiring an extra model. 4.6.2 Deriving Structured State from Physical Simulation Figure 18 illustrates why a physics-based state is particu- larly useful as feedback: physical quantities can be read directly by downstream controllers without being recon- structed from rendered pixels. The core advantage of this strategy is the physical interpretability of the state: state derived from a physics engine naturally carries clear physical meaning and can be directly consumed by downstream controllers. OrbiSim’s [10] dual-module architecture naturally separates physical- state computation from visual rendering explicitly; OrbiSim- Dynamics runs in an explicit physical-state space where all intermediate physical quantities (position, velocity, ac- celeration, contact force, potential energy, inferred physical parameters) exist in structured format and can be accessed downstream with zero information loss. The value of this separation is that downstream consumers can choose which layer to draw information from as needed: a control pol- icy needs only the low-dimensional state vector of po- sition and velocity, a perception model needs the high- dimensional sensor simulation of RGB and depth, and a physical-debugging tool may need contact forces and col- lision events—all obtainable from the same world model without training a separate model per downstream task. ChronoDreamer [103] takes contact force as an explicit prediction target—e.g., the contact force between a robot gripper and the table is 3.2 N—a form of structured feed- back with direct, irreplaceable practical value in precision manipulation. Kinema4D [14] outputs RGB and pointmap as dual feedback, the pointmap providing per-pixel 3D co- ordinates so that a downstream control system can directly learn the position of a pixel in the world coordinate system without indirect inference through stereo or RGB-D sensors. GEM-4D’s [141] inverse-dynamics module converts a video rollout into an executable robot trajectory—answering what robot action sequence corresponds to this video sequence— a translation from visual state feedback to action state feed- back. The 3D-point-flow world model [86] redefines action as a per-pixel 3D-displacement prediction, unifying the rep- resentation of state and action and plugging directly into an MPC framework for real-time planning, with key practical value for robotic control systems requiring millisecond-level response. MVISTA-4D [205] builds on a view-consistent 4D world model and, via test-time action inference (imagine- then-act), back-solves the action to execute from the gener- ated 4D state, so the structured geometric state output by the world model closes directly onto manipulation decisions— another instance of geometric state feedback driving control. 4.6.3 Latent Representation as Alternative Feedback At the opposite end of the interface spectrum, Figure 19 shows a compact predictive representation consumed di- rectly by a planner: it is efficient for machine decision- making but not directly auditable by humans. The JEPA route gives the most radical answer to the state-feedback problem: rather than providing human- readable state feedback, it provides, in latent space, a representation directly consumable by downstream neural networks. V-JEPA 2’s [11] closed-loop planning capability— planning an action sequence via energy minimization in latent space and zero-shot-controlling a real robot—proves that feedback that outputs no readable state can equally be effective feedback; its 15× speed advantage (16 seconds per action versus Cosmos’s 4 minutes) shows that latent- space feedback has engineering superiority under specific conditions. Cosmos Policy [18] defines the world model’s feedback as a latent representation directly fed into a policy model. UWM-JEPA [59] demonstrates the ability to maintain belief-space structural integrity under the more demand- ing condition of blind rollouts with occluded observations, reaching 0.77 accuracy versus 0.53 for a parameter-matched LSTM-JEPA, indicating that the internal structure of latent space may be more effective than pixel space at resisting information loss. But latent-space feedback is constrained by a fundamen- tal problem: auditability. In safety-critical systems, a hu- man operator needs to understand and verify the system’s state, which a latent-space vector cannot afford. A trainable representation-translator module that decodes latent repre- sentations into human-readable structured information may be key to bridging this gap, but no work in the current literature has systematically explored it. 4.6.4 Runtime Annotations versus Sensor-Space Esti- mates The audit separates two interface designs that are often conflated. Sensor-space estimates—semantics, depth, occu- FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202628 physical rollout context evolving physical scene t t+1 t+2 ⋮ time advances → physical evolution physics-based world model explicit world state neural simulator / physics simulator hybrid explicit physical state (object-centric) state at tstate at t+1state at t+2 position (m) (0.15, 0.02, 0.00) velocity (m/s) (0.40, 0.00, 0.00) force (N) (3.20, 0.00, 0.00) mass (kg) 1.00 position (m) (0.19, 0.02, 0.00) velocity (m/s) (0.36, 0.00, 0.00) force (N) (2.80, 0.00, 0.00) mass (kg) 1.00 position (m) (0.22, 0.02, 0.00) velocity (m/s) (0.31, 0.00, 0.00) force (N) (2.60, 0.00, 0.00) mass (kg) 1.00 ⋯ state update physics rollout integrate equations of motion and contact dynamics x t position / velocity position (m) (0.22, 0.02) velocity (m/s) → (0.31, 0.00) contact force (e.g. 3.2 N) 3.2 N per-pixel 3D pointmap z (m) 1.0 0.5 0.0 direct state access (consumed directly) downstream controller • plans and controls using explicit physical state (no reconstruction needed) Physically interpretablestate is directly readableby downstream modules. Fig. 18. Structured state derived from physical simulation. An explicit object-centric trajectory exposes position, velocity, force, and mass over time, from which a controller can directly consume quantities such as contact force and per-pixel 3D pointmaps. scene rollout context observations over time t t+1 t+2 ⋮ t+T world model latent neural simulator encoder latent dynamics / state zₜzₜ₊₁zₜ₊₂ ... predictor does NOT emit pixels (no direct image generation) policy / planner energy minimization in latent space latent embedding zₜ energy minimization in latent space energy high low fast, plan in latent space not human-auditable planned action aₜ* (e.g., steer right) Fig. 19. Latent representation as machine-facing state feedback. The world model predicts a compact latent trajectory rather than reconstructing pixels, and a policy or planner selects actions by optimizing directly in that representation space; efficiency is gained at the cost of interpretability. pancy, point flow, or contact maps—remain B2 even when they are produced alongside RGB. B4 instead requires named entity or physics fields that are directly available during rollout. Six systems meet this criterion: Playable Environments [137] maintains per-object position and pose state; TrafficBots [161] rolls forward typed agent states; PIN- WM [82] identifies object dynamics and physical parame- ters; GigaWorld-0 [20] exposes joint dynamics and system- identification parameters; VectorWorld [99] produces a lane– agent vector graph with typed agent fields; and OrbiSim [10] evolves explicit object-centric physical state. This distinction measures whether a simulator-like annotation interface ex- ists, without treating a sensor prediction or a training label as such an interface. 4.7 Diversity Diversity refers to the richness and controllable stochastic- ity of a model’s generated results in environment layout, object appearance, character behaviour, future evolution, and so on, and is the dimension where a generative world model holds the greatest potential advantage over a tradi- tional simulator. A traditional simulator, limited by a hand- modelled asset library and hand-written behaviour logic, can only generate content from existing models, whereas a world model learns from large-scale internet data and can in principle cover a broader diversity of scenes and behaviours. Of the 200 papers, 46 (23.0%) list diversity as a principal contribution dimension. The analysis of diversity unfolds most naturally from the source of diversity, because different sources determine the FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202629 t t+1 t+2 t+3 ... large diverse data heterogeneous datasets across domains, environments & conditions Weather Cities Roads Robots Indoor scenes Buildings & exteriors Rooms Natural environments Time of day & lighting Sensors & modalities ... and many more datasets & conditions pretrained world foundation model foundation model (neural world simulator) WORLD MODEL Encoder E z t Latent dynamics (learned) f θ z t+1 Decoder D learns to simulate the world • captures rich 3D structure, objects, dynamics, and physics • models long-horizon temporal evolution • supports interaction and counterfactuals • generalizes across domains and conditions observations (pixels, sensors) latent states (compressed world state) emergent coverage diverse, evolving, explorable world rollouts (time flows left to right) Sunny city Rainy night city Snowy mountain road Robot manipulation Modern living room Industrial warehouse Rural village zero-shot: unseen scene type Underwater reefs Office at night Desert sandstorm Fig. 20. Scale-driven diversity in a world foundation model. Heterogeneous data across environments, conditions, and embodiments are compressed into a pretrained model that produces diverse evolving rollouts, including zero-shot coverage beyond the scene types emphasized during training. type and boundary of diversity and its trade-off pattern with controllability: coverage inherited from large-scale data, novel combinations induced by compositional instructions, and controlled variation along explicit parameters. 4.7.1 Scale-Driven Diversity Figure 20 summarizes the scale-driven route, in which heterogeneous training data broaden the support of a pre- trained world model and can yield coverage of unseen scene types. The most direct source of diversity is large-scale, di- verse training data. Cosmos’s [18], [19] WFM design idea— pretrain a general world foundation model for downstream fine-tuning into scene-specific world models—embodies ex- actly the driving role of data scale on diversity: the large- scale diverse data encountered during pretraining is com- pressed into model parameters, and fine-tuning releases diversity along specific dimensions per downstream need. GenAD [6], trained on over 2000 hours of global multi- weather, multi-traffic driving data, demonstrates zero-shot generalization on unseen driving datasets, proving that diverse training data not only improves performance on known scenes but also lets the model generalize to scene types never directly present in the training data. UniSim [89] learns a universal real-world interaction simulator by orchestrating multi-source data from driving, robotic ma- nipulation, and indoor navigation, mapping cross-domain data diversity directly into cross-domain scene-generation diversity. The 44k-hour general robot world model [87] transfers interaction knowledge by learning continuous la- tent actions on large-scale human video and, via knowledge distillation, raises inference speed to a real-time 10.81 FPS, letting diverse interaction behaviours execute efficiently. VQ-VAE-plus-GPT autoregressive video generation [206], as a representative of early autoregressive video genera- tion, already demonstrated the feasibility of a Transformer autoregressively generating diverse, high-fidelity natural- video content in the discrete latent space of a VQ-VAE. 4.7.2 Composition-Driven Diversity The compositional route in Figure 21 obtains novelty by decomposing instructions into reusable entities, actions, and conditions and recombining them into scenarios not ob- served as complete examples during training. The compositionality of natural language is a clever strategy for controllable diversity, specifying which parts should change and into what via language instruction. RoboDreamer [94] exploits the compositionality of natural language to parse a complex instruction into a set of prim- itives (e.g., red block plus grasp) and conditions multiple generative models to synthesize planning videos for prim- itive combinations never seen in training. This decompose- then-recompose strategy achieves compositional generaliza- tion, letting the model flexibly combine learned primitives in unseen ways. DriveDreamer-2 [121] introduces an LLM as a diversity amplifier, using the LLM’s world knowledge and commonsense reasoning to generate scene descriptions not directly present in the training data, thereby expanding scene diversity in semantic space. Multi-view latent diffu- sion [92] generates multi-view video covering rare driving scenes under fine-grained control conditions. The synthetic- data generation pipeline [107] drives a WFM to generate controllable multi-view long-tail edge scenes, directly used to alleviate the long-tail problem of autonomous-driving perception models. Accident-scene generation [108] pushes this idea to extremely rare events: it drives a UniAD agent in a closed loop within the world model to actively generate accident-type dashcam scenes scarce in real data for data augmentation, its value lying precisely in that such high- risk, high-value samples are nearly impossible to obtain through real collection, so the diversity advantage of a gen- FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202630 language instruction Example 1: rainy street, car from left, pedestrian crossing Example 2: robot grasp red block LLM / language parser decompose into primitives reusable primitives red blockgrasp rainycar from left pedestriannight intersectionslippery road warehousecollision recombine compositional world model (neural simulator) recombined conditions -> diverse rollouts novel & rare scenes rainy street, car from left, pedestrian crossing robot grasp red block night street with pedestrian warehouse robot manipulation rare long-tail: accident scene t t+1 t+2 t+3 t t+1 t+2 t+3 t t+1 t+2 t+3 t t+1 t+2 t+3 t t+1 t+2 t+3 ... never seen together in training Fig. 21. Compositional diversity through language. A language parser decomposes instructions into reusable primitives whose new combina- tions condition a world model to generate novel or rare rollouts, including long-tail interaction and accident scenarios. independent control axes appearance / texture camera pose scene layout world model neural simulator encoderlatent dynamicsdecoder zₜzₜ₊₁zₜ₊₂ time controlled latent dynamics conditioned rollout generation specify what varies, what stays constant sweep: camera pose appearance fixed, layout fixed front (center) left-offset higher view rear-ish view appearance fixed camera pose varies scene layout fixed sweep: scene layout appearance fixed, pose fixed baseline more traffic curved road roundabout appearance fixed camera pose fixed scene layout varies t t+1 t+2 ⋯ sweep: appearance / texture pose fixed, layout fixed appearance varies camera pose fixed scene layout fixed sunny rainy sunset overcast Fig. 22. Parametrically controllable diversity. Independent controls over appearance, camera pose, and scene layout allow a world model to sweep one factor while keeping the others fixed, separating purposeful variation from unconstrained stochastic generation. erative world model here translates into direct completion of the safety-critical long-tail distribution. 4.7.3 Controllable Parametric Diversity Figure 22 presents a more controlled form of diversity in which one parameter axis is varied while the remaining scene factors are held fixed. Thereisatrade-offbetweendiversityand controllability—higher diversity means poorer control, more precise control means a narrower space—and this trade-off can be managed through parametrically controllable diversity. GigaWorld-0 [20] demonstrates such fine-grained diversity control, generating diverse embodied sequences and 3D scenes under independent control of appearance (texture, material, lighting), camera view (pose, trajectory), and scene layout (object placement, spatial configuration), so users can precisely specify along which dimensions the world should be diverse and along which consistent. Adaptive multimodal spatial control [207] supports Sim2Real data augmentation through multimodal spatial-weighted control of segmentation, depth, and edges, where different types of control maps provide precise spatial constraints on which regions should change and which should stay. Pose-free feed-forward 4D reconstruction [208] achieves scalable 4D reconstruction and novel-view-trajectory generation from in-the-wild monocular video, demonstrating the feasibility of diverse scene reconstruction without the constraint of camera-pose annotation. 4.7.4 Theoretical Insight and Missing Evaluation of Diver- sity The challenge diversity faces is highly symmetric to con- trollability: high-diversity models (e.g., Cosmos- and Sora- like) often produce uncontrollable, irrelevant visual content on precise tasks, while high-controllability models degrade FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202631 sharply out of the training distribution. The root cause of this trade-off is that, in current model design, diversity and controllability share the same capacity budget, and the model must simultaneously compress how many ways the world can look and which one the user wants, information that is redundant in some dimensions and adversarial in others. How to design a controllable-diversity mechanism that lets users specify in a structured way which aspects may vary freely and which must precisely match the specification is a core design challenge for the next generation of world models. In addition, the quantitative evaluation of diversity is severely lacking; at present almost all diversity claims are qualitative, and systematic metrics need to be built from an information-theoretic (conditional entropy, mutual informa- tion) or coverage (long-tail coverage, rare-event frequency) angle. 4.8 Evaluation Metrics Evaluation metrics largely determine how fast a field can move: what cannot be measured usually cannot be improved. A traditional simulator has a mature, direct evaluation system, including environment-reward-based RL performance, ground-truth-based perception accuracy, deterministic-reproduction-based A/B testing, and the sim- to-real transfer gap. Evaluating a world model as a sim- ulator faces a dual challenge: as a generative model, its output quality must be evaluated (visual fidelity, temporal consistency); as a simulator, its downstream usability must be evaluated (whether it can replace the real environment to support policy or perception-model training). The two are not always consistent: a visually perfect-looking video may be physically impossible [23], and a model with a high open-loop generation score may drift catastrophically in closed-loop interaction [116]. Of the 200 papers, 51 (25.5%) list evaluation as a contribution dimension, of which 18 are dedicated benchmarks. Evaluation works fall naturally into three evaluation lenses ordered by their proximity to downstream utility— generation quality, closed-loop embodied evaluation, and physical plausibility—together with a fourth category that identifies cross-cutting structural gaps in the evaluation ecosystem. Figure 23 summarizes the three lenses and em- phasizes that simulator evaluation must progress beyond surface similarity toward task-level and causal validity. 4.8.1 Unified Generation-Quality Evaluation WorldScore [115] is a unified benchmark aiming to pro- vide standardized comparison across world models, cov- ering dimensions such as video quality, temporal consis- tency, action-following, and physical plausibility. By anal- ogy with GLUE [209] and SuperGLUE [210] in NLP, a unified benchmark is crucial for the comparability of com- munity progress, but the task span of video world models is too large—from open-loop FVD [196] to closed-loop success rate to physical-causality verification—so the comprehen- siveness of any single metric’s coverage is questioned. Nano World Models’ [211] unified evaluation protocol covers di- mensions such as generation target, model scale, action con- ditioning, latent space, evaluation, and long rollout, provid- ing a consistency substrate for reproducible academic eval- uation. In the vertical domain, DrivingGen [109] establishes a comprehensive benchmark for generative video world models in autonomous driving, systematically covering the generation-quality and controllability dimensions of driving scenes and filling the gap left by general benchmarks in characterizing driving-specific needs. 4.8.2 Closed-Loop Embodied Evaluation Closed-loop embodied evaluation is particularly relevant to simulator claims because it measures downstream utility rather than only output appearance. World-in-World [116] reports that visual quality does not guarantee task suc- cess rate, controllability matters more than visual quality, and it reports a scaling relationship for embodied-world- model data. This finding questions the implicit assumption that better FVD equals a better simulator. RoboWM-Bench [80] realizes an executability-evaluation closed loop from generated video to extracted action to physical-execution verification, directly testing whether the video generated by a world model can translate into a physically feasible robotic manipulation. WorldArena 2.0 [212] systematically extends the evaluation system along three dimensions—modality (visuo-tactile), function (RL training environment), and plat- form (simulation plus real robot). iWorld-Bench [213], at the evaluation scale of 330k video clips, 6 task categories, and 14 models, provides a large standardized evaluation of interaction capability within the included benchmark records. The 1X World Model Challenge [118], in the form of a technical report and a public challenge, establishes a unified task and evaluation protocol for generative world models in humanoid embodied settings, advancing the standardization of closed-loop embodied evaluation in real- robot-related scenarios. WBench [114] highlights the multi- round nature of interaction and cross-paradigm compara- bility: with 289 cases and 1058 interaction rounds covering interactive world models of different technical paradigms, it specifically checks whether a model can maintain con- sistent world evolution under consecutive multi-round ac- tions rather than evaluating only single-step response— which strikes exactly at the essence distinguishing a sim- ulator from single-shot generation, namely that state must stay coherent across repeated interaction. SpatialWorld [214] evaluates spatial reasoning at the scale of 8 heterogeneous simulation backends and 760 human-annotated tasks and reports a 17.4% success rate for GPT-5 in its evaluated set- ting. The Table30 benchmark [215] provides a reproducible, scalable real-robot VLA-policy evaluation method; the 4D interaction-response benchmark [111] evaluates the inter- action quality of a world model by measuring the causal impact of interaction actions on state transitions; and the verifiable game-agent benchmark [112] realizes standard- ized closed-loop interaction evaluation on 34 games and 170 tasks. 4.8.3 Physical-Plausibility Evaluation What-If World’s [23] causal-intervention methodology and APEO scoring use 319 paired interventions to test whether generated consequences change in the expected direction. ReactSim-Bench [146], with decoupled ego and agent con- trol, checks reaction plausibility under out-of-distribution behaviour. WorldBench [110] isolates physical concepts such as friction, viscosity, and intuitive physics. MIND [216] FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202632 surface similarity downstream simulator utility reliable simulator Generation qualityClosed-loop embodied utility Physical plausibility Generated rollout Reference rollout Does the rollout look correct? policy / agent world model next action observation success rate Can an agent succeed inside and beyond the model? causal intervention physically correct outcome implausible outcome change exactly one variable: increase mass of blue cylinder Does the rollout obey causal and physical structure? 123 MAINLY OPEN-LOOP EVALUATION temporal consistencyvisual fidelity vs action following multi-step interaction task success policy transfer causal intervention physical response memory over time ... Fig. 23. Three complementary lenses for evaluating world models as simulators. Generation-quality evaluation measures visual fidelity, temporal consistency, and action following, but remains predominantly open loop. Closed-loop embodied evaluation tests multi-step interaction, task success, and policy transfer. Physical-plausibility evaluation uses causal interventions to assess whether rollouts preserve physical response and state over time. Together, these lenses shift the target from surface similarity toward downstream simulator utility. evaluates memory retention through forward- and inverse- modelling reranking with 8972 question–answer pairs; in its evaluated setting, the performance gap between VLMs and humans widens as the rollout horizon increases. 4.8.4 Four Evaluation Limitations Across the included benchmark literature, four recurring evaluation limitations are visible: •Benchmark fragmentation. There is no recognized central benchmark, and the protocols, dimensions, and datasets of different benchmarks are mutually incompatible. •Open-loop bias. The closed-loop cumulative-error effect is an important attribute of a simulator, yet most evaluations cannot capture it—what truly needs attention is whether the model is still in a reasonable world after 1000 steps, not the next-frame PSNR under teacher forcing. •Systematic absence of steppable-environment at- tributes. Almost no benchmark measures the incre- mental success rate—the success rate of deploying to the real environment a policy trained inside the world model—which is precisely the most direct measure of a simulator’s value. •Missing comparison with traditional simulators. Existing benchmarks only make relative comparisons among world models, lacking an absolute baseline of how traditional simulators perform on these metrics. The root cause of these gaps is that evaluation thinking remains centred on generation quality rather than down- stream utility. World-in-World [116] and RoboWM-Bench [80] are exemplary in the paradigm shift, but their current community influence remains limited. 5 FUTURE DIRECTIONS AND CHALLENGES Based on the comparative analysis in Section 4, the gap between world models and a rigorous simulator is real and structural, not a marginal quantitative issue. Yet the clear identification of these gaps is itself progress, because only by knowing where the gaps lie can we design targeted research to close them. 5.1 Formalized Physics The physical simulation of current world models re- lies entirely on soft constraints implicitly learned from data; a breakthrough requires deep fusion of a differen- tiable physics engine (e.g., OrbiSim [10]) with the gen- erative pipeline, using the physics solver as a condi- tional regularizer and a source of structured state feed- back. PIN-WM’s [82] parameter identification, Chron- oDreamer’s [103] contact-force prediction, and PhysBridge’s [79] physics-simulator bridge already show explicit phys- ical modelling to be feasible; CP4D’s [74] physics-prior- plus-diffusion-commonsense hybrid and ReconDreamer- RL’s [85] kinematics-diffusion separation show modular physical modelling to be practical. The real difficulty is the tension identified in Section 4.2: a differentiable en- gine is exact but can only simulate phenomena whose equations have been written down, whereas a data-driven model is flexible but prone to visual shortcuts, and this conflict cannot be removed by enlarging the model or the data. The open question is therefore not whether physics can be injected, but whether generated output can carry step-wise verifiable conservation constraints while retaining open-domain generalization—What-If World’s [23] causal- intervention protocol offers a way to measure such progress, and whether certification can extend from rigid bodies to fluids and deformation will set the reach of formalized physics. 5.2 Unified Action Interface Explicit control is precise but generalizes poorly, language instruction is semantically readable but imprecise, and la- tent action generalizes well but is unreadable; the paradigm chasm among the three needs bridging in two directions. FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202633 The first is the joint learning of multi-scale action representa- tions: DiLA’s [90] content–structure separation and DisCo’s [25] discrete primitives both indicate that, along certain dimensions, coarse-grained discretization outperforms fine- grained continuity because separability matters more than resolution. The second is incorporating LLM world knowl- edge into the action-translation pipeline, where DreamX- World 1.0 [167] and DriveDreamer-2 [121] show preliminary feasibility. Harder than representation learning is the tem- poral compositionality dissected in Section 4.4: when action segments are concatenated, the conditional observation of a later segment has already drifted from the true world state, so the segments lose causal coupling—at root a gap between open-loop sequence continuation and closed-loop feedback. Only V-JEPA 2’s [11] receding-horizon MPC and World- VLA’s [65] attention mask so far touch genuine closed- loop control, and how to transplant the mature feedback- plus-feedforward framework of control theory into a high- dimensional generative model, so that the action interface stays closed-loop-consistent over millisecond-level rollouts, remains unresolved. 5.3 First-Class State Feedback The state-interface audit motivates three distinct devel- opment targets. Multi-task prediction heads, as used by Tesseract [102], UniScene [78], and multimodal egocen- tric world models [200], can expose estimated structure alongside RGB. Explicit-state systems such as OrbiSim [10] can provide quantities tied to an interpretable dynamics state. Representation translators may make JEPA-style latent state inspectable. The key research question is not simply whether a model can emit semantics, depth, or boxes, but whether it can expose named entity and physical fields that remain consistent with the generated observation through- out a rollout. Benchmarks should therefore score sensor- space estimates separately from runtime entity/physics an- notations. 5.4 Long-Horizon Stability Suppressing autoregressive drift admits two complemen- tary strategies: a periodic reset-and-anchor mechanism, for which Lyra 2.0’s [21] geometry routing and DecMem’s [22] sparse global memory provide a technical basis; and quantitative characterization of how stability changes with scale and horizon, for which World-in-World [116] reports a scaling relationship and MIND’s [216] memory evalu- ation supplies a methodology. As Section 4.5 notes, drift claims across architectures, scene complexities, and action- sequence lengths still lack a common protocol. Two open questions follow. First, for a given architecture and action distribution, can the growth of rollout error be bounded, and how do model capacity and distillation affect it? Second, how should repeat-run reproducibility be reported under controlled randomness? The included corpus provides little comparable evidence on exposed seeds, deterministic ker- nels, tolerance bands, or repeat-run variance. Standardizing these factors is necessary for A/B testing, regression testing, and safety analysis built on world-model rollouts. 5.5 Downstream-Utility Evaluation The core metric should shift from generation quality to the incremental success rate—the success rate, in the real envi- ronment, of an RL policy trained inside the world model. RoboWM-Bench’s [80] generation-and-execution verifica- tion and World-in-World’s [116] practice of training agents inside the world model are early exemplars. The resistance to this shift lies in the four structural gaps summarized in Section 4.8: benchmark fragmentation, open-loop bias, the absence of steppable-environment attributes, and the miss- ing absolute comparison with traditional simulators. The most fundamental is that evaluation thinking remains cen- tred on generation quality—a model with a strong FVD may, after 1000 closed-loop steps, already sit in an implausible world, which current metrics do not catch. A downstream- utility-centred paradigm therefore has to answer two ques- tions: how to measure the incremental success rate cheaply without a full real-robot deployment each time, and how to use a traditional simulator as an absolute baseline that quantifies how far a world model still falls short, rather than only ranking world models against one another. 5.6 Cross-Route Fusion The first five directions each target a single capability, whereas the capability-by-capability comparison of Sec- tion 4 reveals a more holistic opportunity: the routes’ strengths are complementary—diffusion in vision and in- teraction, latent dynamics in control decisions, JEPA in efficiency, explicit 3D/4D in geometric consistency. Several fusion directions therefore stand out: embedding a differ- entiable physics engine into the diffusion pipeline (OrbiSim [10] and PhysBridge [79]); combining explicit 3D memory with implicit video generation (Lyra 2.0 [21] and GWM [13]); unifying VLA and world models (τ 0 -WM [12] and WorldVLA [65]); and the pretrain-plus-fine-tune paradigm of the world model as a holistic data engine (GigaWorld- 0 [20] and Cosmos [19]). The fundamental obstacle is not engineering assembly but that the routes’ optimization ob- jectives are not naturally compatible—pixel-reconstruction loss, latent-space energy, and policy return live in different metric spaces, and naively summing them lets the terms work against one another, which calls for re-examining the routes from the unified framework T : S ×A→ Π(S ×F ). Being-H0.7’s [190] train-time-fuse, inference-time-separate design hints at one way around the objective conflict, but how to compose progress on the six directions into a single system that meets all requirements of a rigorous simulator at once remains the open question with no precedent and the greatest value. 6 CONCLUSION This study asks whether the generative process of a world model can become a simulator in the strict sense. Using the eight capabilities of a traditional simulator as a unified yardstick, it performs a systematic comparative analysis and paper-level evidence mapping of a curated 200-paper corpus. On the core question of whether it can become a rigorous simulator, our judgement is: under restricted FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202634 conditions it can, but in the strict sense it remains a critical step away. Generative world models, with their open-domain generalization learned from data and their visual quality approaching photorealism, are advancing rapidly toward a new generation of intelligent simulation environments. In interaction and controllability, through autoregressive-diffusion distillation [4], [56], latent-action learning [15], [90], 4D geometric-control representation [122], and world-foundation-model platforms [19], world models have demonstrated the feasibility of functional substitution in specific scenarios. However, on the formal guarantees of the physics engine, the richness and struc- turedness of state feedback—the complete implementation audit confirms B2 in 45 of 163 papers and B4 in only six— the theoretical guarantee of long-horizon stability, and the downstream-utility orientation of the evaluation system, world models remain a critical step away from traditional simulators. On the question of how far each route has progressed, the capability-by-capability comparison shows that autore- gressive models have the broadest interaction coverage, while diffusion models remain prominent in high-fidelity visual generation [2], [4]; both families nevertheless have weak coverage of physics and state feedback. Latent dynam- ics is distinguished by its support for control decisions [33] but has limited visual-generation ability; the JEPA route has outstanding efficiency, with a roughly 15× speed advantage [11], but unreadable output; and the explicit 3D/4D route has a structural advantage in geometric consistency [13], [21], [77] but appearance detail inferior to pure diffusion methods. Table 2 provides the corpus-level coverage evi- dence for these route-dependent trade-offs and, in particu- lar, shows that state feedback C6 remains a structural gap across most model families. On where to go next, we distil six research directions— formalized physics, a unified action interface, first-class state feedback, long-horizon stability, downstream-utility evaluation, and cross-route fusion—each supported by ex- isting nascent work as evidence of feasibility, together form- ing an actionable roadmap toward the next generation of true simulators. We hope the eight-capability comparison framework and the full-corpus evidence map established by this study can provide a systematic coordinate system for tracking progress. We also call on the research community to turn the proposition of “world model as simulator” from an aspirational slogan into a research agenda with rigorous definitions of capability, gap, and evaluation. Only when the goal expands from generating good-looking video to outputting consumable, verifiable, reproducible world state can a generative world model truly be used as a world, rather than merely looking like one. APPENDIX A COMPLETE LIST OF SEARCH QUERIES This appendix reports the complete keyword protocol used in Section 2. Each query is assigned a stable identifier and a short scope label so that the coverage of the search can be inspected without parsing an undifferentiated code block. The Boolean expressions are reproduced verbatim apart from line wrapping. Results were ranked by relevance, trun- cated to the top entries for each query, de-duplicated against the accumulated corpus, and then screened manually. The search cut-off was 30 June 2026. A.1 Mainstream Routes and Application Settings These queries cover the principal generative architectures, interaction settings, and application domains used to form the core candidate pool. M1 Interactive video simulation interactive AND world AND model AND simulation AND video AND generation M2 Playable neural game engines world AND model AND game AND engine AND playable AND real-time M3 Robot manipulation action AND conditioned AND world AND model AND robot AND manipulation AND simulation M4 Latent-action models latent AND action AND world AND model AND video M5 Autoregressive video models autoregressive AND world AND model AND video AND tokenizer AND interactive M6 Autonomous-driving simulation driving AND world AND model AND autonomous AND simulation AND generative M7 Embodied physical simulation embodied AND world AND model AND physics AND simulator M8 Diffusion game simulators neural AND game AND simulator AND diffusion AND world AND model M9 Evaluation benchmarks world AND model AND benchmark AND evaluation AND interactive AND simulation M10 General surveys general AND world AND model AND survey AND simulator M11 Real-time streaming real-time AND generative AND world AND model AND streaming M12 Explorable 4D worlds 4D AND world AND model AND scene AND generation AND explorable A.2 Supplementary Representations and Capabilities These queries target representations and capabilities that broad world-model searches tend to miss, including predic- tive embeddings, explicit geometry, memory, multimodality, and physics-oriented evaluation. S1 Joint-embedding prediction joint AND embedding AND predictive AND architecture AND world AND model AND JEPA S2 Self-supervised representation learning self-supervised AND world AND model AND representation AND learning AND prediction S3 Language models for planning large AND language AND model AND as AND world AND model AND planning AND reasoning FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202635 S4 Text-based simulation language AND model AND world AND model AND text AND adventure AND simulation S5 3D Gaussian representations 3D AND gaussian AND splatting AND world AND model AND scene AND generation S6 Neural radiance fields neural AND radiance AND field AND world AND model AND dynamic AND scene S7 Differentiable physics differentiable AND physics AND simulation AND world AND model AND learned S8 Neural rigid-body dynamics physics AND engine AND neural AND world AND model AND rigid AND body AND dynamics S9 Long-horizon consistency long-term AND memory AND consistent AND world AND model AND video AND generation S10 Persistent spatial memory persistent AND world AND model AND spatial AND memory AND exploration S11 Embodied navigation embodied AND navigation AND world AND model AND spatial AND reasoning S12 Occupancy prediction occupancy AND world AND model AND 3D AND scene AND autonomous AND prediction S13 Multimodal prediction multimodal AND world AND model AND audio AND video AND tactile AND prediction S14 Physical-plausibility evaluation physical AND plausibility AND world AND model AND evaluation AND benchmark AND video S15 General-purpose foundation models foundation AND world AND model AND general AND purpose AND interactive AND simulation S16 Camera- and action-controllable video controllable AND video AND generation AND world AND model AND camera AND action Execution note. On arXiv, each expression was applied to the all field; on Google Scholar, DBLP, and Crossref, the same concept-term conjunction was used without a field prefix. Citation-chain expansion and publication-status ver- ification, described in Section 2, were conducted as separate cross-checks and are therefore not represented as keyword queries above. REFERENCES [1]D. Ha and J. Schmidhuber, “World models,” 2018. [Online]. Available: https://arxiv.org/abs/1803.10122 [2]D. Valevski, Y. Leviathan, M. Arar, and S. Fruchter, “Diffusion models are real-time game engines,” in The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025.OpenReview.net, 2025. [Online]. Available: https://openreview.net/forum?id=P8pqeEkn1H [3]J. Yu, Y. Qin, X. Wang, P. Wan, D. Zhang, and X. Liu, “Gamefactory: Creating new games with generative interactive videos,” in 2025 IEEE/CVF International Conference on Computer Vision (ICCV).IEEE, 10 2025, p. 11590–11599. [Online]. Available: http://dx.doi.org/10.1109/iccv51701.2025.01078 [4]Z. Wang, Z. Liu, J. Li, K. Huang, B. Xu, F. Kang, M. An, P. Wang, B. Jiang, Y. Wei, Y. Xietian, J. Pei, L. Hu, B. Jiang, H. Xue, Z. Wang, H. Sun, W. Li, W. Ouyang, X. He, Y. Liu, Y. Li, and Y. Zhou, “Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory,” 2026. [Online]. Available: https://arxiv.org/abs/2604.08995 [5]H. Che, X. He, Q. Liu, C. Jin, and H. Chen, “Gamegen-x: Interactive open-world game video generation,” in The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025.OpenReview.net, 2025. [Online]. Available: https://openreview.net/forum?id=8VG8tpPZhe [6]J. Yang, S. Gao, Y. Qiu, L. Chen, T. Li, B. Dai, K. Chitta, P. Wu, J. Zeng, P. Luo, J. Zhang, A. Geiger, Y. Qiao, and H. Li, “Genad: Generalized predictive model for autonomous driving,” 2024. [Online]. Available: https://arxiv.org/abs/2403.09630 [7]F. Jia, W. Mao, Y. Liu, Y. Zhao, Y. Wen, C. Zhang, X. Zhang, and T. Wang, “Adriver-i: A general world model for autonomous driving,” 2023. [Online]. Available: https://arxiv.org/abs/2311.13549 [8]NVIDIA, :, A. Basant, A. Kar, D. Paschalidou, F. Wei, F. Ferroni, G. G. Cobo, H. Turki, H. Ling, J. Seo, J. Lucas, J. Z. Wu, J. Wang, J. Lorraine, J. Gao, K. He, K. Tothova, K. Xie, M. Tyszkiewicz, Q. Wu, R. de Lutio, R. Li, S. Fidler, S. W. Kim, T. Shen, T. Cao, T. Pfaff, W. Lew, X. Wu, X. Ren, Y. Lu, Y. Zhang, Z. Gojcic, and Z. Wang, “Nvidia omnidreams: Real-time generative world model for closed-loop autonomous vehicle simulation,” 2026. [Online]. Available: https://arxiv.org/abs/2606.03159 [9]Y. Chen, Y. Wang, and Z. Zhang, “Drivinggpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers,” in IEEE/CVF International Conference on Computer Vision, ICCV 2025, Honolulu, HI, USA, October 19-25, 2025. IEEE, 2025, p. 26 890–26 900. [Online]. Available: https: //doi.org/10.1109/ICCV51701.2025.02496 [10]J. Li, J. Huang, J. Gong, Q. Wang, X. Yang, and Y. Wang, “Orbisim: World models as differentiable physics engines for embodied intelligence,” 2026. [Online]. Available: https: //arxiv.org/abs/2605.16395 [11]M. Assran, A. Bardes, D. Fan, Q. Garrido, R. Howes, M. Komeili, M. Muckley, A. Rizvi, C. Roberts, K. Sinha, A. Zholus, S. Arnaud, A. Gejji, A. Martin, F. R. Hogan, D. Dugas, P. Bojanowski, V. Khalidov, P. Labatut, F. Massa, M. Szafraniec, K. Krishnakumar, Y. Li, X. Ma, S. Chandar, F. Meier, Y. LeCun, M. Rabbat, and N. Ballas, “V-jepa 2: Self-supervised video models enable understanding, prediction and planning,” 2025. [Online]. Available: https://arxiv.org/abs/2506.09985 [12]P. Zhou, S. Chen, D. Chen, J. Wang, R. Jin, B. Zhu, Y. Pan, S. Gu, K. Wang, S. Nan, X. Qiu, C. Qiu, P. Yang, Y. Cai, J. Gao, Y. Li, Y. Fu, X. Yue, Z. Chen, and J. Luo, “τ 0 -wm: A unified video-action world model for robotic manipulation,” 2026. [Online]. Available: https://arxiv.org/abs/2606.01027 [13]G. Lu, B. Jia, P. Li, Y. Chen, Z. Wang, Y. Tang, and S. Huang, “GWM: towards scalable gaussian world models for robotic manipulation,” in IEEE/CVF International Conference on Computer Vision, ICCV 2025, Honolulu, HI, USA, October 19-25, 2025.IEEE, 2025, p. 9263–9274. [Online]. Available: https://doi.org/10.1109/ICCV51701.2025.00865 [14]M. Xu, T. Zhang, T. Liu, Z. Chen, X. Han, and Z. Liu, “Kinema4d: Kinematic 4d world modeling for spatiotemporal embodied simulation,” 2026. [Online]. Available: https://arxiv.org/abs/ 2603.16669 [15]J. Bruce, M. D. Dennis, A. Edwards, J. Parker-Holder, Y. Shi, E. Hughes, M. Lai, A. Mavalankar, R. Steigerwald, C. Apps, Y. Aytar, S. Bechtle, F. M. P. Behbahani, S. C. Y. Chan, N. Heess, L. Gonzalez, S. Osindero, S. Ozair, S. E. Reed, J. Zhang, K. Zolna, J. Clune, N. de Freitas, S. Singh, and T. Rockt ̈ aschel, “Genie: Generative interactive environments,” in Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, ser. Proceedings of Machine Learning Research, R. Salakhutdinov, Z. Kolter, K. A. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp, Eds., vol. 235.PMLR / OpenReview.net, 2024, p. 4603–4623. [Online]. Available: https://proceedings.mlr.press/v235/bruce24a.html [16]H. Yu, H. Duan, C. Herrmann, W. T. Freeman, and J. Wu, “Wonderworld: Interactive 3d scene generation from a single image,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025. Computer Vision Foundation / IEEE, 2025, p. 5916–5926. FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202636 [Online].Available:https://openaccess.thecvf.com/content/ CVPR2025/html/YuWonderWorldInteractive3DScene GenerationfromaSingleImageCVPR2025paper.html [17]Y. He, Z. Yuan, Z. Tu, Y. Ye, and L. Sun, “3d4d: An interactive, editable, 4d world model via 3d video generation,” in Fortieth AAAI Conference on Artificial Intelligence, Thirty-Eighth Conference on Innovative Applications of Artificial Intelligence, Sixteenth Symposium on Educational Advances in Artificial Intelligence, AAAI 2026, Singapore, January 20-27, 2026, S. Koenig, C. Jenkins, and M. E. Taylor, Eds. AAAI Press, 2026, p. 41 595–41 597. [Online]. Available: https://doi.org/10.1609/aaai.v40i48.42351 [18]M. J. Kim, Y. Gao, T.-Y. Lin, Y.-C. Lin, Y. Ge, G. Lam, P. Liang, S. Song, M.-Y. Liu, C. Finn, and J. Gu, “Cosmos policy: Fine-tuning video models for visuomotor control and planning,” 2026. [Online]. Available: https://arxiv.org/abs/2601.16163 [19]NVIDIA, :, N. Agarwal, A. Ali, M. Bala, Y. Balaji, E. Barker, T. Cai, P. Chattopadhyay, Y. Chen, Y. Cui, Y. Ding, D. Dworakowski, J. Fan, M. Fenzi, F. Ferroni, S. Fidler, D. Fox, S. Ge, Y. Ge, J. Gu, S. Gururani, E. He, J. Huang, J. Huffman, P. Jannaty, J. Jin, S. W. Kim, G. Kl ́ ar, G. Lam, S. Lan, L. Leal-Taixe, A. Li, Z. Li, C.-H. Lin, T.-Y. Lin, H. Ling, M.-Y. Liu, X. Liu, A. Luo, Q. Ma, H. Mao, K. Mo, A. Mousavian, S. Nah, S. Niverty, D. Page, D. Paschalidou, Z. Patel, L. Pavao, M. Ramezanali, F. Reda, X. Ren, V. R. N. Sabavat, E. Schmerling, S. Shi, B. Stefaniak, S. Tang, L. Tchapmi, P. Tredak, W.-C. Tseng, J. Varghese, H. Wang, H. Wang, H. Wang, T.-C. Wang, F. Wei, X. Wei, J. Z. Wu, J. Xu, W. Yang, L. Yen-Chen, X. Zeng, Y. Zeng, J. Zhang, Q. Zhang, Y. Zhang, Q. Zhao, and A. Zolkowski, “Cosmos world foundation model platform for physical ai,” 2025. [Online]. Available: https://arxiv.org/abs/2501.03575 [20]GigaWorld Team, A. Ye, B. Wang, C. Ni, G. Huang, G. Zhao, H. Li, J. Zhu, K. Li, M. Xu, Q. Deng, S. Wang, W. Qin, X. Chen, X. Wang, Y. Wang, Y. Cao, Y. Chang, Y. Xu, Y. Ye, Y. Wang, Y. Zhou, Z. Zhang, Z. Dong, and Z. Zhu, “Gigaworld-0: World models as data engine to empower embodied ai,” 2025. [Online]. Available: https://arxiv.org/abs/2511.19861 [21]T. Shen, S. Bahmani, K. He, S. G. Srinivasan, T. Cao, J. Ren, R. Li, Z. Wang, N. Sharp, Z. Gojcic, S. Fidler, J. Huang, H. Ling, J. Gao, and X. Ren, “Lyra 2.0: Explorable generative 3d worlds,” 2026. [Online]. Available: https://arxiv.org/abs/2604.13036 [22]Z. Yang, X. Wu, Z. Lv, X. Shi, X. Wang, P. Wan, K. Gai, and K.-Y. K. Wong, “Decmem: Towards minute-long consistent world generation with decoupled memory,” 2026. [Online]. Available: https://arxiv.org/abs/2605.31336 [23]K. Cai, R. Song, J. Zhang, K. Zhang, P. Bodapati, A. Yu, F. Suya, M. Rostami, J. Ma, and Y. Tian, “What-if world: A causal benchmark for general world models in embodied scenarios,” 2026. [Online]. Available: https://arxiv.org/abs/2605.27589 [24]Y. Chen, R. Chen, D. Huo, Y. Yang, D. Qi, H. Liu, T. Lin, S. Zeng, J. Xiao, X. Chang, F. Xiong, X. Wei, Z. Ma, and M. Xu, “Abot-physworld: Interactive world foundation model for robotic manipulation with physics alignment,” 2026. [Online]. Available: https://arxiv.org/abs/2603.23376 [25]H. Huang, J. Wang, Q. Li, Y.-G. Jiang, and Z. Wu, “Disco: World models with discrete camera motion control,” 2026. [Online]. Available: https://arxiv.org/abs/2606.07967 [26]Z. Zhu, X. Wang, W. Zhao, C. Min, B. Li, N. Deng, M. Dou, Y. Wang, B. Shi, K. Wang, C. Zhang, Y. You, Z. Zhang, D. Zhao, L. Xiao, J. Zhao, J. Lu, and G. Huang, “Is sora a world simulator? a comprehensive survey on general world models and beyond,” 2024. [Online]. Available: https://arxiv.org/abs/2405.03520 [27]B. Hou, G. Li, J. Jia, T. An, X. Guo, S. Leng, H. Geng, Y. Ze, T. Harada, P. Torr, O. Mees, M. Pollefeys, Z. Liu, J. Wu, P. Abbeel, J. Malik, Y. Du, and J. Yang, “World model for robot learning: A comprehensive survey,” 2026. [Online]. Available: https://arxiv.org/abs/2605.00080 [28]J. Liu, C. Ni, M. Liu, C. Peng, F. Wang, S. Shen, M. Pollefeys, M. Tomizuka, A. Tewari, and P. O. Kristensson, “Towards interactivevideoworldmodeling:Frontiers,challenges, benchmarks, and future trends,” 2026. [Online]. Available: https://arxiv.org/abs/2606.01164 [29]F. Wang, Z. Wang, G. Pei, M. Zhang, C. Liang, J. Hu, Z. Li, J. Wu, N. Han, Z. Zhang, J. Qi, H. Wu, S. Zhang, P. Zheng, J. Pan, D. Navarro-Alarcon, S. Liu, and P. Zhou, “World models for robotic manipulation: A survey,” 2026. [Online]. Available: https://arxiv.org/abs/2606.00113 [30]X. Long, Q. Zhao, K. Zhang, Z. Zhang, D. Wang, Y. Liu, Z. Shu, Y. Lu, S. Wang, X. Wei, W. Li, W. Yin, Y. Yao, J. Pan, Q. Shen, R. Yang, X. Cao, and Q. Dai, “A survey: Learning embodied intelligence from physical simulators and world models,” 2025. [Online]. Available: https://arxiv.org/abs/2507.00917 [31]Y. Hu, L. Wang, X. Liu, L.-H. Chen, Y. Guo, Y. Shi, C. Liu, A. Rao, Z. Wang, and H. Xiong, “Simulating the real world: A unified survey of multimodal generative models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, p. 1–20, 2026. [Online]. Available: http://dx.doi.org/10.1109/tpami.2026.3690925 [32]X. Li, X. He, L. Zhang, M. Wu, X. Li, and Y. Liu, “A comprehensive survey on world models for embodied ai,” 2025. [Online]. Available: https://arxiv.org/abs/2510.16732 [33]D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap, “Mastering diverse control tasks through world models,” Nature, vol. 640, no. 8059, p. 647–653, 4 2025. [Online]. Available: http://dx.doi.org/10.1038/s41586-025-08744-2 [34]J. Ding, Y. Zhang, Y. Shang, Y. Zhang, Z. Zong, J. Feng, Y. Yuan, H. Su, N. Li, N. Sukiennik, F. Xu, and Y. Li, “Understanding world or predicting future? a comprehensive survey of world models,” ACM Computing Surveys, vol. 58, no. 3, p. 1–38, 9 2025. [Online]. Available: http://dx.doi.org/10.1145/3746449 [35]J. Yue, Z. Huang, Z. Chen, X. Wang, P. Wan, and Z. Liu, “Simulating the visual world with artificial intelligence: A roadmap,” 2025. [Online]. Available: https://arxiv.org/abs/ 2511.08585 [36]T. Feng, W. Wang, and Y. Yang, “A survey of world models for autonomous driving,” 2025. [Online]. Available: https://arxiv.org/abs/2501.11260 [37]S. Tu, X. Zhou, D. Liang, X. Jiang, Y. Zhang, X. Li, and X. Bai, “The role of world models in shaping autonomous driving: A comprehensive survey,” 2025. [Online]. Available: https://arxiv.org/abs/2502.10498 [38]A. Fu, Y. Zhou, T. Zhou, Y. Yang, B. Gao, Q. Li, G. Wu, and L. Shao, “Exploring the interplay between video generation and world models in autonomous driving: A survey,” 2024. [Online]. Available: https://arxiv.org/abs/2411.02914 [39]C. Zhang, B. Wei, Y. Liu, and S. Labi, “World model-based long- tail and scenario-specific generation for autonomous driving,” Journal of Intelligent and Connected Vehicles, vol. 9, no. 2, p. 9210080, 6 2026. [Online]. Available: http://dx.doi.org/10. 26599/jicv.2026.9210080 [40]X. Wang, T. Feng, H. Liu, and W. Zhu, Self-evolving World Model. Springer Nature Singapore, 2026, p. 211–245. [Online]. Available: http://dx.doi.org/10.1007/978-981-95-7750-7 5 [41]M. He, H. Guo, J. Lin, and Y. Yu, “Video generation models as world models: Efficient paradigms, architectures and algorithms,” 2026. [Online]. Available: https://arxiv.org/abs/ 2603.28489 [42]N. Xie, Z. Tian, L. Yang, X.-P. Zhang, M. Guo, and J. Li, “From 2d to 3d cognition: A brief survey of general world models,” 2025. [Online]. Available: https://arxiv.org/abs/2506.20134 [43]D. Hafner, T. P. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson, “Learning latent dynamics for planning from pixels,” in Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97.PMLR, 2019, p. 2555–2565. [Online]. Available: http://proceedings.mlr.press/v97/hafner19a.html [44]D. Hafner, T. P. Lillicrap, J. Ba, and M. Norouzi, “Dream to control: Learning behaviors by latent imagination,” in 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020.OpenReview.net, 2020. [Online]. Available: https://openreview.net/forum?id= S1lOTC4tDS [45]D. Hafner, T. P. Lillicrap, M. Norouzi, and J. Ba, “Mastering atari with discrete world models,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021.OpenReview.net, 2021. [Online]. Available: https://openreview.net/forum?id=0oabwyZbOu [46]P. Wu, A. Escontrela, D. Hafner, P. Abbeel, and K. Goldberg, “Daydreamer: World models for physical robot learning,” in Conference on Robot Learning, CoRL 2022, 14-18 December 2022, Auckland, New Zealand, ser. Proceedings of Machine Learning Research, K. Liu, D. Kulic, and J. Ichnowski, Eds., vol. 205.PMLR, 2022, p. 2226–2240. [Online]. Available: https://proceedings.mlr.press/v205/wu23c.html FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202637 [47]P. Lancaster, N. Hansen, A. Rajeswaran, and V. Kumar, “Modem-v2: Visuo-motor world models for real-world robot manipulation,” in 2024 IEEE International Conference on Robotics and Automation (ICRA).IEEE, 5 2024, p. 7530–7537. [Online]. Available: http://dx.doi.org/10.1109/icra57147.2024.10611121 [48]A. Hu, L. Russell, H. Yeo, Z. Murez, G. Fedoseev, A. Kendall, J. Shotton, and G. Corrado, “Gaia-1: A generative world model for autonomous driving,” 2023. [Online]. Available: https://arxiv.org/abs/2309.17080 [49]L. Kaiser, M. Babaeizadeh, P. Milos, B. Osinski, R. H. Campbell, K. Czechowski, D. Erhan, C. Finn, P. Kozakowski, S. Levine, A. Mohiuddin, R. Sepassi, G. Tucker, and H. Michalewski, “Model based reinforcement learning for atari,” in 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020.OpenReview.net, 2020. [Online]. Available: https://openreview.net/forum?id= S1xCPJHtDB [50]R. Villegas, M. Babaeizadeh, P. Kindermans, H. Moraldo, H. Zhang, M. T. Saffar, S. Castro, J. Kunze, and D. Erhan, “Phenaki: Variable length video generation from open domain textual descriptions,” in The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023.OpenReview.net, 2023. [Online]. Available: https: //openreview.net/forum?id=vOEXS39nOF [51]E. Alonso, A. Jelley, V. Micheli, A. Kanervisto, A. J. Storkey, T. Pearce, and F. Fleuret, “Diffusion for world modeling: Visual details matter in atari,” in Advances in Neural Information Processing Systems 37: Annual Conference onNeuralInformationProcessingSystems2024,NeurIPS 2024,Vancouver,BC,Canada,December10-15,2024, A. Globersons, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. M. Tomczak, and C. Zhang, Eds., 2024. [Online]. Avail- able:http://papers.nips.c/paper files/paper/2024/hash/ 6bdde0373d53d4a501249547084bed43-Abstract-Conference.html [52]D. Ye, F. Zhou, J. Lv, J. Ma, J. Zhang, J. Lv, J. Li, M. Deng, M. Yang, Q. Fu, W. Yang, W. Lv, Y. Yu, Y. Wang, Y. Guan, Z. Hu, Z. Fang, and Z. Sun, “Yan: Foundational interactive video generation,” 2025. [Online]. Available: https://arxiv.org/abs/2508.08601 [53]W. Sun, H. Zhang, H. Wang, J. Wu, Z. Wang, Z. Wang, Y. Wang, J. Zhang, T. Wang, and C. Guo, “Worldplay: Towards long-term geometric consistency for real-time interactive world modeling,” 2025. [Online]. Available: https://arxiv.org/abs/2512.14614 [54]V. Micheli, E. Alonso, and F. Fleuret, “Transformers are sample-efficient world models,” in The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023.OpenReview.net, 2023. [Online]. Available: https://openreview.net/forum?id=vhFu1Acb0xb [55]Y. Hong, Y. Mei, C. Ge, Y. Xu, Y. Zhou, S. Bi, Y. Hold- Geoffroy, M. Roberts, M. Fisher, E. Shechtman, K. Sunkavalli, F. Liu, Z. Li, and H. Tan, “Relic: Interactive video world model with long-horizon memory,” 2025. [Online]. Available: https://arxiv.org/abs/2512.04040 [56]M. Zhao, H. Zhu, B. Yan, Z. Zhou, Y. Chen, W. Sun, K. Zheng, G. He, X. Yang, C. Li, F. Bao, and J. Zhu, “minwm: A full-stack open-source framework for real-time interactive video world models,” 2026. [Online]. Available: https://arxiv.org/abs/2605.30263 [57]X. He, C. Peng, Z. Liu, B. Wang, Y. Zhang, Q. Cui, F. Kang, B. Jiang, M. An, Y. Ren, B. Xu, H.-X. Guo, K. Gong, S. Wu, W. Li, X. Song, Y. Liu, Y. Li, and Y. Zhou, “Matrix-game 2.0: An open-source real-time and streaming interactive world model,” 2025. [Online]. Available: https://arxiv.org/abs/2508.13009 [58]A. Bardes, Q. Garrido, J. Ponce, X. Chen, M. Rabbat, Y. LeCun, M. Assran, and N. Ballas, “Revisiting feature prediction for learning visual representations from video,” Trans. Mach. Learn. Res., vol. 2024, 2024. [Online]. Available: https://openreview.net/forum?id=QaCCuDfBk2 [59]S. K. Radha and O. Goktas, “Uwm-jepa: Predictive world models that imagine in belief space,” 2026. [Online]. Available: https://arxiv.org/abs/2605.25313 [60]B. Terver, T. Yang, J. Ponce, A. Bardes, and Y. LeCun, “What drives success in physical planning with joint-embedding predictive world models?” Trans. Mach. Learn. Res., vol. 2026, 2026. [Online]. Available: https://openreview.net/forum?id= cHZn5Gdh8e [61]Y. Huang, “Vjepa: Variational joint embedding predictive architectures as probabilistic world models,” 2026. [Online]. Available: https://arxiv.org/abs/2601.14354 [62]L. F. Toso, D. Shadunts, Y. Lu, N. Sharma, D. Zhan, N. H. Nguyen, and J. Anderson, “Learning invariant visual representations for planning with joint-embedding predictive world models,” 2026. [Online]. Available: https://arxiv.org/abs/2602.18639 [63]J. Yeom, H. Kim, J. Park, S. Jung, J. Lee, and T. Kim, “What makes video world model latents action-relevant: Prediction over reconstruction,” 2026. [Online]. Available: https://arxiv.org/abs/2606.07687 [64]B. Kerbl, G. Kopanas, T. Leimk ̈ uhler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering,” ACM Transactions on Graphics, vol. 42, no. 4, p. 1–14, 2023. [65]J. Cen, C. Yu, H. Yuan, Y. Jiang, S. Huang, J. Guo, X. Li, Y. Song, H. Luo, F. Wang, D. Zhao, and H. Chen, “Worldvla: Towards autoregressive action world model,” 2025. [Online]. Available: https://arxiv.org/abs/2506.21539 [66]E. Todorov, T. Erez, and Y. Tassa, “MuJoCo: A physics engine for model-based control,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012, p. 5026–5033. [67]NVIDIA, “Isaac Sim,” Software, 2026, official citation metadata; accessed 21 August 2026. [Online]. Available: https://github. com/isaac-sim/IsaacSim [68]E. Coumans and Y. Bai, “PyBullet, a python module for physics simulation for games, robotics and machine learning,” Software, 2016–2021. [Online]. Available: https://pybullet.org [69]EpicGames,“UnrealEnginedocumentation,”Software documentation, 2026, accessed 21 August 2026. [Online]. Available:https://dev.epicgames.com/documentation/en-us/ unreal-engine [70]A. Juliani, V.-P. Berges, E. Teng, A. Cohen, J. Harper, C. Elion, C. Goy, Y. Gao, H. Henry, M. Mattar, and D. Lange, “Unity: A general platform for intelligent agents,” arXiv preprint arXiv:1809.02627, 2020. [Online]. Available: https://arxiv.org/abs/1809.02627 [71]M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling, “The arcade learning environment: An evaluation platform for general agents,” Journal of Artificial Intelligence Research, vol. 47, p. 253– 279, 2013. [72]A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “CARLA: An open urban driving simulator,” in Proceedings of the 1st Annual Conference on Robot Learning, ser. Proceedings of Machine Learning Research, vol. 78.PMLR, 2017, p. 1–16. [Online]. Available: https://proceedings.mlr.press/v78/ dosovitskiy17a.html [73]S. Tunyasuvunakool, A. Muldal, Y. Doron, S. Liu, S. Bohez, J. Merel, T. Erez, T. Lillicrap, N. Heess, and Y. Tassa, “dm control: Software and tasks for continuous control,” Software Impacts, vol. 6, p. 100022, 2020. [74]H. Zhu, C. Wang, T. He, L. Chen, X. Jin, C. Gao, and Z. Chen, “Cp4d: Compositional physics-aware 4d scene generation,” 2026. [Online]. Available: https://arxiv.org/abs/2606.09187 [75]H. Sun, Z. Peng, S. Wu, K. Wang, K. Liao, D. Wang, X. Zeng, S. Jin, Y. Li, Z. Cao, Z. Liu, and W. Li, “Prisma-world: Camera-controllable multi-agent video world model,” 2026. [Online]. Available: https://arxiv.org/abs/2606.09507 [76]Y. Zhou, Z. Wang, Y. Lu, H. Liu, J. Liang, S. He, andJ.Li,“Moverse:Real-timevideoworldmodeling with panoramic gaussian scaffold,” 2026. [Online]. Available: https://arxiv.org/abs/2606.13376 [77]H. Zhu, Y. Wang, J. Zhou, W. Chang, Y. Zhou, Z. Li, J. Chen, C. Shen, J. Pang, and T. He, “AETHER: geometric-aware unified world modeling,” in IEEE/CVF International Conference on Computer Vision, ICCV 2025, Honolulu, HI, USA, October 19-25, 2025.IEEE, 2025, p. 8535–8546. [Online]. Available: https://doi.org/10.1109/ICCV51701.2025.00799 [78]B. Li, J. Guo, H. Liu, Y. Zou, Y. Ding, X. Chen, H. Zhu, F. Tan, C. Zhang, T. Wang, S. Zhou, L. Zhang, X. Qi, H. Zhao, M. Yang, W. Zeng, and X. Jin, “Uniscene: Unified occupancy-centric driving scene generation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025.Computer Vision Foundation / IEEE, 2025, p. 11 971–11 981. [Online]. Available: https://openaccess.thecvf.com/content/CVPR2025/ html/Li UniSceneUnifiedOccupancy-centricDrivingScene GenerationCVPR2025paper.html FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202638 [79]W. Liu, Z. Chen, Z. Li, Y. Wang, H.-X. Yu, and J. Wu, “Realwonder: Real-time physical action-conditioned video generation,” 2026. [Online]. Available: https://arxiv.org/abs/ 2603.05449 [80]F. Jiang, Y. Chen, K. Xu, Y. Liu, H. Wang, Z. Shen, J. Lu, S. Huang, Y. Wang, C. Xie, and R. Wu, “Robowm-bench: A benchmark for evaluating world models in robotic manipulation,” 2026. [Online]. Available: https://arxiv.org/abs/2604.19092 [81]Z. Xiong, Y. Song, H. Kang, Q. Yan, L. Jiang, J. Yang, Z. Fu, S. Fotiadis, A. Wang, Z. Liu, B. Liu, Y. Yang, X. Lu, and N. Jacobs, “Actworld: From explorable to interactive world model via action-aware memory,” 2026. [Online]. Available: https://arxiv.org/abs/2606.17730 [82]W. Li, H. Zhao, Z. Yu, Y. Du, Q. Zou, R. Hu, and K. Xu, “Pin-wm: Learning physics-informed world models for non-prehensile manipulation,” in Robotics: Science and Systems XXI, ser. RSS2025. Robotics: Science and Systems Foundation, 6 2025. [Online]. Available: http://dx.doi.org/10.15607/rss.2025.xxi.153 [83]Y. Bai, D. Tran, A. Bar, Y. LeCun, T. Darrell, and J. Malik, “Whole- body conditioned egocentric video prediction,” in Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diago, CA, USA, December 2-7, 2025 / Mexico City, Mexico, November 30 - December 5, 2025, D. Belgrave, C. Zhang, L. N. Montoya, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, N. Chen, I. V. M. Ru ́ ız, and A. Loaiza-Bonilla, Eds., 2025. [Online]. Avail- able:http://papers.nips.c/paper files/paper/2025/hash/ f0552f14388d95b19740dee809f5cad1-Abstract-Conference.html [84]Y. Shang, X. Zhang, Y. Tang, L. Jin, C. Gao, W. Wu, and Y. Li, “Roboscape: Physics-informed embodied world model,” 2025. [Online]. Available: https://arxiv.org/abs/2506.23135 [85]C. Ni, G. Zhao, X. Wang, Z. Zhu, W. Qin, X. Chen, G. Jia, G. Huang, and W. Mei, “Recondreamer-rl: Enhancing reinforcement learning via diffusion-based scene reconstruction,” 2025. [Online]. Available: https://arxiv.org/abs/2508.08170 [86]W. Huang, Y.-W. Chao, A. Mousavian, M.-Y. Liu, D. Fox, K. Mo, and L. Fei-Fei, “Pointworld: Scaling 3d world models for in-the-wild robotic manipulation,” 2026. [Online]. Available: https://arxiv.org/abs/2601.03782 [87]S. Gao, W. Liang, K. Zheng, A. Malik, S. Ye, S. Yu, W.-C. Tseng, Y. Dong, K. Mo, C.-H. Lin, Q. Ma, S. Nah, L. Magne, J. Xiang, Y. Xie, R. Zheng, D. Niu, Y. L. Tan, K. R. Zentner, G. Kurian, S. Indupuru, P. Jannaty, J. Gu, J. Zhang, J. Malik, P. Abbeel, M.-Y. Liu, Y. Zhu, J. Jang, and L. J. Fan, “Dreamdojo: A generalist robot world model from large-scale human videos,” 2026. [Online]. Available: https://arxiv.org/abs/2602.06949 [88]B. Gu, T. Wu, Y. Yuan, J. Liu, X. Lu, D. Du, J. Zhang, J. Lai, S. Yang, X. Zhao, A. Zhao, and S. Guo, “Worldcraft: From camera navigation to object manipulation in interactive video world models,” 2026. [Online]. Available: https://arxiv.org/abs/2605. 25077 [89]S.Yang,Y.Du,S.K.S.Ghasemipour,J.Tompson, L. P. Kaelbling, D. Schuurmans, and P. Abbeel, “Learning interactive real-world simulators,” in The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024.OpenReview.net, 2024. [Online]. Available: https://openreview.net/forum?id=sFyTZEqmUY [90]T. Zhang, M. Lyu, Y. Zhang, F. Fang, and S. Wu, “Dila: Disentangled latent action world models,” 2026. [Online]. Available: https://arxiv.org/abs/2605.15725 [91]S. Gao, S. Zhou, Y. Du, J. Zhang, and C. Gan, “Adaworld: Learning adaptable world models with latent actions,” in Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, ser. Proceedings of Machine Learning Research, A. Singh, M. Fazel, D. Hsu, S. Lacoste-Julien, F. Berkenkamp, T. Maharaj, K. Wagstaff, and J. Zhu, Eds., vol. 267.PMLR / OpenReview.net, 2025. [Online]. Available: https://proceedings.mlr.press/v267/gao25u.html [92]L. Russell, A. Hu, L. Bertoni, G. Fedoseev, J. Shotton, E. Arani, and G. Corrado, “Gaia-2: A controllable multi-view generative world model for autonomous driving,” 2025. [Online]. Available: https://arxiv.org/abs/2503.20523 [93]J. Xiang, G. Liu, Y. Gu, Q. Gao, Y. Ning, Y. Zha, Z. Feng, T. Tao, S. Hao, Y. Shi, Z. Liu, E. P. Xing, and Z. Hu, “Pandora: Towards general world model with natural language actions and video states,” 2024. [Online]. Available: https://arxiv.org/abs/2406.09455 [94]S. Zhou, Y. Du, J. Chen, Y. Li, D. Yeung, and C. Gan, “Robodreamer: Learning compositional world models for robot imagination,” in Forty-first International Conference on MachineLearning,ICML2024,Vienna,Austria,July21- 27, 2024, ser. Proceedings of Machine Learning Research, R. Salakhutdinov, Z. Kolter, K. A. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp, Eds., vol. 235.PMLR / OpenReview.net, 2024, p. 61 885–61 896. [Online]. Available: https://proceedings.mlr.press/v235/zhou24f.html [95]Robbyant Team, Z. Gao, Q. Wang, Y. Zeng, J. Zhu, K. L. Cheng, Y. Li, H. Wang, Y. Xu, S. Ma, Y. Chen, J. Liu, Y. Cheng, Y. Yao, J. Zhu, Y. Meng, K. Zheng, Q. Bai, J. Chen, Z. Shen, Y. Yu, X. Zhu, Y. Shen, and H. Ouyang, “Advancingopen-sourceworldmodels,”2026.[Online]. Available: https://arxiv.org/abs/2601.20540 [96]D. Samuel, I. Tzachor, M. Levy, M. Green, G. Chechik, and R. Ben-Ari, “Fast autoregressive video diffusion and world models with temporal cache compression and sparse attention,” 2026. [Online]. Available: https://arxiv.org/abs/2602.01801 [97]Z. Duan, J. Xia, Z. Zhang, W. Zhang, G. Zhou, C. Gou, Y. He, F. Chen, X. Zhang, and L. Liu, “Liveworld: Simulating out-of-sight dynamics in generative video world models,” 2026. [Online]. Available: https://arxiv.org/abs/2603.07145 [98]J. Seo, H. Choi, M. Kwon, J. Choi, S. Jin, G. Lee, J. Kim, J. Lee, G. Gu, D. Han, S. Yun, S. Kim, and J.-H. Kim, “Grounding world simulation models in a real-world metropolis,” 2026. [Online]. Available: https://arxiv.org/abs/2603.15583 [99]C. Jiang, D. Zhou, J. Liu, and K. L. Sun, “Vectorworld: Efficient streaming world model via diffusion flow on vector graphs,” 2026. [Online]. Available: https://arxiv.org/abs/2603.17652 [100] T. Wu, S. Yang, R. Po, Y. Xu, Z. Liu, D. Lin, and G. Wetzstein, “Video world models with long-term spatial memory,” 2025. [Online]. Available: https://arxiv.org/abs/2506.05284 [101] T. Lu, T. Shu, A. Yuille, D. Khashabi, and J. Chen, “Generative world explorer,” 2024. [Online]. Available: https: //arxiv.org/abs/2411.11844 [102] H. Zhen, Q. Sun, H. Zhang, J. Li, S. Zhou, Y. Du, and C. Gan, “Tesseract: Learning 4d embodied world models,” in 2025 IEEE/CVF International Conference on Computer Vision (ICCV).IEEE, 10 2025, p. 5337–5347. [Online]. Available: http://dx.doi.org/10.1109/iccv51701.2025.00507 [103] Z. Zhou and D. Negrut, “Chronodreamer: Action-conditioned world model as an online simulator for robotic planning,” 2025. [Online]. Available: https://arxiv.org/abs/2512.18619 [104] L. Wang, W. Zheng, Y. Ren, H. Jiang, Z. Cui, H. Yu, and J. Lu, “Occsora: 4d occupancy generation models as world simulators for autonomous driving,” IEEE Transactions on Image Processing, vol. 35, p. 4947–4960, 2026. [Online]. Available: http://dx.doi.org/10.1109/tip.2026.3687468 [105] J. Wei, S. Yuan, P. Li, Q. Hu, Z. Gan, and W. Ding, “Occllama: An occupancy-language-action generative world model for autonomous driving,” 2024. [Online]. Available: https://arxiv.org/abs/2409.03272 [106] J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel, T. Lillicrap, and D. Silver, “Mastering atari, go, chess and shogi by planning with a learned model,” Nature, vol. 588, no. 7839, p. 604–609, 12 2020. [Online]. Available: http://dx.doi.org/10.1038/s41586-020-03051-4 [107] X. Ren, Y. Lu, T. Cao, R. Gao, S. Huang, A. Sabour, T. Shen, T. Pfaff, J. Z. Wu, R. Chen, S. W. Kim, J. Gao, L. Leal-Taixe, M. Chen, S. Fidler, and H. Ling, “Cosmos- drive-dreams: Scalable synthetic driving data generation with world foundation models,” 2025. [Online]. Available: https://arxiv.org/abs/2506.09042 [108] Y.Guan,H.Liao,C.Wang,X.Liu,J.Zhang,and Z. Li, “World model-based end-to-end scene generation for accident anticipation in autonomous driving,” Communications Engineering, vol. 4, no. 1, 8 2025. [Online]. Available: http://dx.doi.org/10.1038/s44172-025-00474-7 [109] Y. Zhou, H. Shao, L. Wang, Z. Zong, H. Li, and S. L. Waslander, “Drivinggen: A comprehensive benchmark for generative video world models in autonomous driving,” 2026. [Online]. Available: https://arxiv.org/abs/2601.01528 [110] R. Upadhyay, H. Zhang, J. Solomon, A. Agrawal, P. Boreddy, S. S. Narayana, Y. Ba, A. Wong, C. M. de Melo, and A. Kadambi, “Worldbench: Disambiguating physics for diagnostic evaluation FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202639 of world models,” 2026. [Online]. Available: https://arxiv.org/ abs/2601.21282 [111] M. Wu, Z. Cai, F. Zhao, X. Feng, R. Dang, B. Song, R. Tian, J. Zhu, J. Lei, H. Dou, J. Tang, L. Sun, J. Wu, X. Chu, Z. Liu, and K. Huang, “Omni-worldbench: Towards a comprehensive interaction-centric evaluation for world models,” 2026. [Online]. Available: https://arxiv.org/abs/2603.22212 [112] M. Ouyang, S. Hu, K. Q. Lin, H. T. Ng, and M. Z. Shou, “Gameworld: Towards standardized and verifiable evaluation ofmultimodalgameagents,”2026.[Online].Available: https://arxiv.org/abs/2604.07429 [113] Y. Jiang, J. Hu, Q. Xiao, Y. Zheng, R. Ma, K. Feng, J. Han, T. Peng, K. Fan, M. Zhang, and X. Yue, “Opengame: Open agentic coding for games,” 2026. [Online]. Available: https://arxiv.org/abs/2604.18394 [114] K. Ying, H. Hu, S. Ren, J. Li, F. Chen, Z. Wang, X. Cao, X. Cai, and H. Ding, “Wbench: A comprehensive multi-turn benchmark for interactive video world model evaluation,” 2026. [Online]. Available: https://arxiv.org/abs/2605.25874 [115] H. Duan, H. Yu, S. Chen, L. Fei-Fei, and J. Wu, “Worldscore: A unified evaluation benchmark for world generation,” in IEEE/CVF International Conference on Computer Vision, ICCV 2025, Honolulu, HI, USA, October 19-25, 2025.IEEE, 2025, p. 1–12. [Online]. Available: https://doi.org/10.1109/ICCV51701. 2025.02573 [116] J. Zhang, M. Jiang, N. Dai, T. Lu, A. Uzunoglu, S. Zhang, Y. Wei, J. Wang, V. M. Patel, P. P. Liang, D. Khashabi, C. Peng, R. Chellappa, T. Shu, A. Yuille, Y. Du, and J. Chen, “World-in-world: World models in a closed-loop world,” 2025. [Online]. Available: https://arxiv.org/abs/2510.18135 [117] Y. Liao, P. Zhou, S. Huang, D. Yang, S. Chen, Y. Jiang, Y. Hu, J. Cai, S. Liu, J. Luo, L. Chen, S. Yan, M. Yao, and G. Ren, “Genie envisioner: A unified world foundation platform for robotic manipulation,” 2025. [Online]. Available: https://arxiv.org/abs/2508.05635 [118] R. Mereu, A. Scannell, Y. Hou, Y. Zhao, A. Jitta, A. Dominguez, L. Acerbi, A. Storkey, and P. Chang, “Generative world modelling for humanoids: 1x world model challenge technical report,” 2025. [Online]. Available: https://arxiv.org/abs/2510.07092 [119] R. Gao, K. Chen, E. Xie, L. Hong, Z. Li, D. Yeung, andQ.Xu,“Magicdrive:Streetviewgenerationwith diverse 3d geometry control,” in The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024.OpenReview.net, 2024. [Online]. Available: https://openreview.net/forum?id=sBQwvucduK [120] X. Wang, Z. Zhu, G. Huang, X. Chen, J. Zhu, and J. Lu, DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving.Springer Nature Switzerland, 11 2024, p. 55–72. [Online]. Available: http://dx.doi.org/10.1007/ 978-3-031-73195-2 4 [121] G. Zhao, X. Wang, Z. Zhu, X. Chen, G. Huang, X. Bao, and X. Wang, “Drivedreamer-2: Llm-enhanced world models for diverse driving video generation,” in Thirty-Ninth AAAI Conference on Artificial Intelligence, Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence, Fifteenth Symposium on Educational Advances in Artificial Intelligence, AAAI 2025, Philadelphia, PA, USA, February 25 - March 4, 2025, T. Walsh, J. Shah, and Z. Kolter, Eds. AAAI Press, 2025, p. 10 412–10 420. [Online]. Available: https://doi.org/10.1609/aaai.v39i10.33130 [122] S. Zheng, M. Yin, W. Hu, X. Li, Y. Shan, and Y. Fu, “Versecrafter: Dynamic realistic video world model with 4d geometric control,” 2026. [Online]. Available: https://arxiv.org/abs/2601.05138 [123] M. Quigley, B. Gerkey, K. Conley, J. Faust, T. Foote, J. Leibs, E. Berger, R. Wheeler, and A. Y. Ng, “ROS: An open-source robot operating system,” in ICRA Workshop on Open Source Software, 2009. [Online]. Available: https: //robotics.stanford.edu/ ∼ ang/papers/icraoss09-ROS.pdf [124] N. Koenig and A. Howard, “Design and use paradigms for Gazebo, an open-source multi-robot simulator,” in 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 3. IEEE, 2004, p. 2149–2154. [125] Y. Chen, P. Li, J. Yang, K. He, X. Wu, Y. Xu, K. Wang, J. Liu, N. Liu, Y. Huang, and L. Wang, “Bridgev2w: Bridging video generation models to embodied world models via embodiment masks,” 2026. [Online]. Available: https://arxiv.org/abs/2602.03793 [126] T. Liu, Z. Huang, Z. Chen, G. Wang, S. Hu, L. Shen, Z.Cao,W.Li,andZ.Liu,“Free4d:Tuning-free4d scene generation with spatial-temporal consistency,” in 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 10 2025, p. 25571–25582. [Online]. Available: http: //dx.doi.org/10.1109/iccv51701.2025.02372 [127] H.Zhou,W.Yu,J.Guan,X.Cheng,Y.Tian,and L. Yuan, “Holotime: Taming video diffusion models for panoramic 4d scene generation,” in Proceedings of the 33rd ACM International Conference on Multimedia, ser. M ’25. ACM, 10 2025, p. 9763–9772. [Online]. Available: http: //dx.doi.org/10.1145/3746027.3755128 [128] H. Liang, D. Xu, N. P. Bhatt, H. Hu, H. Liang, and K. N. Plataniotis, “Comp4d: Compositional 4d scene generation,” in 2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).IEEE, 3 2026, p. 3567–3577. [Online]. Available: http://dx.doi.org/10.1109/wacv61042.2026.00348 [129] Y. Zheng, X. Li, K. Nagano, S. Liu, O. Hilliges, and S. De Mello, “A unified approach for text-and image-guided 4d scene generation,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 6 2024, p. 7300–7309. [Online]. Available: http://dx.doi.org/10.1109/cvpr52733.2024.00697 [130] K. Li, S. Zhang, Y. Fang, S. Yuan, Y. Zou, and L. Yang, “Towards dynamic world model generation with monocular video,” in ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).IEEE, 5 2026, p. 12012–12016. [Online]. Available: http://dx.doi.org/10.1109/ icassp55912.2026.11463651 [131] N. Morbitzer, J. Evers, A. Savkin, T. Stauner, N. Navab, F. Tombari, and S. Gasperini, “Future dynamic 3d reconstruction: A 3d world model with disentangled ego-motion,” 2026. [Online]. Available: https://arxiv.org/abs/2606.18250 [132] Y. Chai, L. Deng, R. Shao, J. Zhang, K. Lv, L. Xing, X. Li, H. Zhang, and Y. Liu, “Gaf: Gaussian action field as a 4d representation for dynamic world modeling in robotic manipulation,” 2025. [Online]. Available: https://arxiv.org/abs/2506.14135 [133] Y. Tu, H. Luo, X. Chen, X. Bai, F. Wang, and H. Zhao, “Playerone: Egocentric world simulator,” in Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diago, CA, USA, December 2-7, 2025 / Mexico City, Mexico, November 30 - December 5, 2025, D. Belgrave, C. Zhang, L. N. Montoya, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, N. Chen, I. V. M. Ru ́ ız, and A. Loaiza-Bonilla, Eds., 2025. [Online]. Avail- able:http://papers.nips.c/paper files/paper/2025/hash/ d607a260b3bc0b9a704a1a04d64040a-Abstract-Conference.html [134] L. Zhou, H. Luo, Z. Zhu, C. Chi, M. Tu, K. Xiong, L. Gong, Z. Wu, Z. Zhang, F. Li, H. Li, Y. Shen, J. He, H. Zhu, S. Zhao, K. Wang, Z. Zhan, Y. Pu, K. Tan, R. Yang, X. Wang, T. Yan, J. Zhou, L. Zhang, J. Zhao, X. Zhou, C. Sun, C. Wu, J. Deng, H. Xie, M. Lu, K. Ma, L. Chen, G. Chen, H. Ye, B. Wang, and H. Sun, “Xiaomi auto world model: A joint world model integrating reconstruction and generation for autonomous driving,” 2026. [Online]. Available: https://arxiv.org/abs/2605.18137 [135] D. Wang, H. Jung, T. Monnier, K. Sohn, C. Zou, X. Xiang, Y.-Y. Yeh, D. Liu, Z. Huang, T. Nguyen-Phuoc, Y. Fan, S. Oprea, Z. Wang, R. Shapovalov, N. Sarafianos, T. Groueix, A. Toisoul, P. Dhar, X. Chu, M. Chen, G. Y. Park, M. Gupta, Y. Azziz, R. Ranjan, and A. Vedaldi, “Worldgen: From text to traversable and interactive 3d worlds,” 2025. [Online]. Available: https://arxiv.org/abs/2511.16825 [136] X. Zhang, X. Dong, H. Yu, J. Wang, and Q. Li, “Magicity4d: Controllable and editable 4d city scene generation using mllm-enhanced procedural content generation,” in ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).IEEE, 5 2026, p. 20042–20046. [Online]. Available: http://dx.doi.org/10.1109/icassp55912.2026.11464740 [137] W. Menapace, S. Lathuili ` ere, A. Siarohin, C. Theobalt, S. Tulyakov, V. Golyanik, and E. Ricci, “Playable environments: Video manipulation in space and time,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022.IEEE, 2022, p. 3574–3583. [Online]. Available: https://doi.org/10.1109/CVPR52688.2022. 00357 [138] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ra- mamoorthi, and R. Ng, “NeRF: Representing scenes as neural radiance fields for view synthesis,” in Computer Vision – ECCV 2020. Springer, 2020, p. 405–421. [139] C. Ning, K. Fang, and W.-C. Ma, “Prompting with the future: FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202640 Open-world model predictive control with interactive digital twins,” in Robotics: Science and Systems XXI, ser. RSS2025. Robotics: Science and Systems Foundation, 6 2025. [Online]. Available: http://dx.doi.org/10.15607/rss.2025.xxi.145 [140] Z. Chen, J. R ̈ oning, and S. Li, “World model enhanced embodied intelligence for deformable object manipulation of dynamic targets,” in 2025 IEEE 25th International Symposium on Computational Intelligence and Informatics (CINTI).IEEE, 11 2025, p. 679–684. [Online]. Available: http://dx.doi.org/10.1109/ cinti67731.2025.11311737 [141] K. Zhou, Y. Chen, F. Zhan, H. Hua, G. Chen, X. Chang, A. Qu, Y. Du, Z. Liu, P. P. Liang, and M. Wang, “Gem-4d: Geometry- enhanced video world models for robot manipulation,” 2026. [Online]. Available: https://arxiv.org/abs/2605.22882 [142] J. Guo, X. Ma, Y. Wang, M. Yang, H. Liu, and Q. Li, “Flowdreamer: A rgb-d world model with flow-based motion representations for robot manipulation,” IEEE Robotics and Automation Letters, vol. 11, no. 3, p. 2466–2473, 3 2026. [Online]. Available: http://dx.doi.org/10.1109/lra.2026.3653273 [143] H. Bi, H. Tan, S. Xie, Z. Wang, S. Huang, H. Liu, R. Zhao, Y. Feng, C. Xiang, Y. Rong, H. Zhao, H. Liu, Z. Su, L. Ma, H. Su, and J. Zhu, “Motus: A unified latent action world model,” 2025. [Online]. Available: https://arxiv.org/abs/2512.13030 [144] F. Zhu, H. Wu, S. Guo, Y. Liu, C. Cheang, and T. Kong, “Irasim: A fine-grained world model for robot manipulation,” in IEEE/CVF International Conference on Computer Vision, ICCV 2025, Honolulu, HI, USA, October 19-25, 2025.IEEE, 2025, p. 9834–9844. [Online]. Available: https://doi.org/10.1109/ ICCV51701.2025.00917 [145] X. Chi, P. Jia, C.-K. Fan, X. Ju, W. Mi, K. Zhang, Z. Qin, W. Tian, K. Ge, H. Li, Z. Qian, A. Chen, Q. Zhou, Y. Jia, J. Liu, Y. Dai, Q. Wuwu, C. Bai, Y.-K. Wang, Y. Li, L. Chen, Y. Bao, Z. Jiang, J. Zhu, K. Tang, R. An, Y. Luo, Q. Feng, S. Zhou, C.-m. Chan, C. Hou, W. Xue, S. Han, Y. Guo, S. Zhang, and J. Tang, “Wow: Towards a world omniscient world model through embodied interaction,” 2025. [Online]. Available: https://arxiv.org/abs/2509.22642 [146] Z. Zhang, Y. Peng, J. Zhang, X. Guo, Z. Huang, H. Liu, Q. Li, S. Zhang, X. Jia, and J. Yan, “Reactsim-bench: Benchmarking reactive behavior world model simulation in autonomous driving,” 2026. [Online]. Available: https: //arxiv.org/abs/2606.14058 [147] A. Srivatsa Kadaba, “Physics consistent world models via schr ̈ odinger-bridge optimal transport for computational imaging and 3d-consistent video generations,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 48, p. 41516–41518, 3 2026. [Online]. Available: http://dx.doi.org/10.1609/aaai. v40i48.42325 [148] M. Kempka, M. Wydmuch, G. Runc, J. Toczek, and W. Ja ́ skowski, “ViZDoom: A Doom-based AI research platform for visual re- inforcement learning,” in 2016 IEEE Conference on Computational Intelligence and Games. IEEE, 2016, p. 1–8. [149] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, p. 10 684–10 695. [150] Inferix Team, T. Feng, Y. Han, J. He, Y. He, X. Lin, T. Liu, H. Lu, J. Tang, W. Wang, Z. Wang, J. Wu, M. Yang, Y. Yu, Z. Zhang, and B. Zhuang, “Inferix: A block-diffusion based next-generation inference engine for world simulation,” 2025. [Online]. Available: https://arxiv.org/abs/2511.20714 [151] R. Feng, H. Zhang, Z. Shu, Z. Yang, L. Tang, Z. Wang, A. Zheng, J. Xiao, Z. Liu, R. Chu, Y. Huang, Y. Liu, and H. Zhang, “The matrix: Infinite-horizon world generation with real-time moving control,” in Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diago, CA, USA, December 2-7, 2025 / Mexico City, Mexico, November 30 - December 5, 2025, D. Belgrave, C. Zhang, L. N. Montoya, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, N. Chen, I. V. M. Ru ́ ız, and A. Loaiza-Bonilla, Eds., 2025. [Online]. Avail- able:http://papers.nips.c/paper files/paper/2025/hash/ 7e810b2c75d69be186cadd2fe3febeab-Abstract-Conference.html [152] J.Guo,Y.Ye,T.He,H.Wu,Y.Jiang,T.Pearce, and J. Bian, “Mineworld: a real-time and open-source interactive world model on minecraft,” 2025. [Online]. Available: https://arxiv.org/abs/2504.08388 [153] Y. Zhang, C. Peng, B. Wang, P. Wang, Q. Zhu, F. Kang, B. Jiang, Z. Gao, E. Li, Y. Liu, and Y. Zhou, “Matrix-game: Interactive world foundation model,” 2025. [Online]. Available: https://arxiv.org/abs/2506.18701 [154] X. Mao, S. Lin, Z. Li, C. Li, W. Peng, T. He, J. Pang, M. Chi, Y. Qiao, and K. Zhang, “Yume: An interactive world generation model,” 2025. [Online]. Available: https://arxiv.org/abs/2507.17744 [155] H. He, Y. Zhang, L. Lin, Z. Xu, and L. Pan, “Pre-trained video generative models as world simulators,” in Fortieth AAAI Conference on Artificial Intelligence, Thirty-Eighth Conference on Innovative Applications of Artificial Intelligence, Sixteenth Symposium on Educational Advances in Artificial Intelligence, AAAI 2026, Singapore, January 20-27, 2026, S. Koenig, C. Jenkins, and M. E. Taylor, Eds.AAAI Press, 2026, p. 4645–4653. [Online]. Available: https://doi.org/10.1609/aaai.v40i6.42465 [156] Y. Jain, A. Nasery, V. Vineet, and H. Behl, “Peekaboo: Interactive video generation via masked-diffusion,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).IEEE, 6 2024, p. 8079–8088. [Online]. Available: http://dx.doi.org/10.1109/cvpr52733.2024.00772 [157] M. Yang, J. Li, Z. Fang, S. Chen, Y. Yu, Q. Fu, W. Yang, and D. Ye, “Playable game generation,” 2024. [Online]. Available: https://arxiv.org/abs/2412.00887 [158] G. Savva, O. Michel, D. Lu, S. Waiwitlikhit, T. Meehan, D. Mishra, S. Poddar, J. Lu, and S. Xie, “Solaris: Building a multiplayer video world model in minecraft,” 2026. [Online]. Available: https://arxiv.org/abs/2602.22208 [159] Z. Wang, C. Shi, J. Hu, K. Rohling, R. Mart ́ ın-Mart ́ ın, A. Zhang, and P. Stone, “Factored latent action world models,” 2026. [Online]. Available: https://arxiv.org/abs/2602.16229 [160] W. Zheng, Z. Xia, Y. Huang, S. Zuo, J. Zhou, and J. Lu, “Doe-1: Closed-loop autonomous driving with large world model,” 2024. [Online]. Available: https://arxiv.org/abs/2412.09627 [161] Z. Zhang, A. Liniger, D. Dai, F. Yu, and L. V. Gool, “Trafficbots: Towards world models for autonomous driving simulation and motion prediction,” in IEEE International Conference on Robotics and Automation, ICRA 2023, London, UK, May 29 - June 2, 2023.IEEE, 2023, p. 1522–1529. [Online]. Available: https://doi.org/10.1109/ICRA48891.2023.10161243 [162] L. Wang, K. Zhao, C. Liu, and X. Chen, “Learning real- world action-video dynamics with heterogeneous masked autoregression,” 2025. [Online]. Available: https://arxiv.org/ abs/2502.04296 [163] S. Li, Y. Gao, D. Sadigh, and S. Song, “Unified video action model,” in Robotics: Science and Systems XXI, ser. RSS2025. Robotics: Science and Systems Foundation, 6 2025. [Online]. Available: http://dx.doi.org/10.15607/rss.2025.xxi.074 [164] W. Menapace, S. Lathuili ` ere, S. Tulyakov, A. Siarohin, andE.Ricci,“Playablevideogeneration,”inIEEE ConferenceonComputerVisionandPatternRecognition, CVPR 2021, virtual, June 19-25, 2021.Computer Vision Foundation / IEEE, 2021, p. 10 061–10 070. [Online]. Avail- able: https://openaccess.thecvf.com/content/CVPR2021/html/ Menapace PlayableVideoGenerationCVPR2021paper.html [165] J. Wu, S. Yin, N. Feng, X. He, D. Li, J. Hao, and M. Long, “ivideogpt: Interactive videogpts are scalable world models,” in Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, A. Globersons, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. M. Tomczak, and C. Zhang, Eds., 2024. [Online]. Avail- able:http://papers.nips.c/paperfiles/paper/2024/hash/ 7dbb5bfab324e3b86af9bd0df15498d-Abstract-Conference.html [166] L. Jin and L. Jia, “Embodied world models emerge from navigational task in open-ended environments,” 2025. [Online]. Available: https://arxiv.org/abs/2504.11419 [167] DreamX Team, Y. Bai, R. Chen, X. Chu, R. Dang, H. Dou, B. Gao, Q. Gu, S. Hong, J. Lei, G. Li, J. Li, R. Lin, Q. Shi, B. Song, L. Sun, J. Tang, R. Tian, J. Wang, J. Wu, P. Zhang, S. Zhang, and J. Zhu, “Dreamx-world 1.0: A general- purpose interactive world model,” 2026. [Online]. Available: https://arxiv.org/abs/2606.16993 [168] M. Rigter, T. Gupta, A. Hilmkil, and C. Ma, “Avid: Adapting video diffusion models to world models,” 2024. [Online]. Available: https://arxiv.org/abs/2410.12822 [169] A. Pondaven, Z. Wu, I. Gilitschenski, P. Torr, S. Tulyakov, F. Pizzati, and A. Siarohin, “Actionparty: Multi-subject action FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202641 binding in generative video games,” 2026. [Online]. Available: https://arxiv.org/abs/2604.02330 [170] Y. Guo, L. X. Shi, J. Chen, and C. Finn, “Ctrl-world: A controllable generative world model for robot manipulation,” 2025. [Online]. Available: https://arxiv.org/abs/2510.10125 [171] X. Hu, W. Yin, M. Jia, J. Deng, X. Guo, Q. Zhang, X. Long, and P. Tan, “Drivingworld: Constructing world model for autonomous driving via video gpt,” 2024. [Online]. Available: https://arxiv.org/abs/2412.19505 [172] Y. Wang, J. He, L. Fan, H. Li, Y. Chen, and Z. Zhang, “Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024.IEEE, 2024, p. 14 749–14 759. [Online]. Available: https://doi.org/10.1109/CVPR52733.2024.01397 [173] J. Zhu, Z. Jia, T. Gao, J. Deng, S. Li, L. Zhang, F. Liu, P. Jia, and X. Lang, “Other vehicle trajectories are also needed: A driving world model unifies ego-other vehicle trajectories in video latent space,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 16, p. 13934–13942, 3 2026. [Online]. Available: http://dx.doi.org/10.1609/aaai.v40i16.38403 [174] X. Wang and P. Peng, “Prophetdwm: A driving world model for rolling out future actions and videos,” 2025. [Online]. Available: https://arxiv.org/abs/2505.18650 [175] J. Tang, J. Liu, J. Li, L. Wu, H. Yang, P. Zhao, S. Gong, X. Yuan, S. Shao, L. Zhang, and Q. Lu, “Hunyuan-gamecraft-2: Instruction-following interactive game world model,” 2025. [Online]. Available: https://arxiv.org/abs/2511.23429 [176] X. Mao, Z. Li, C. Li, X. Xu, K. Ying, T. He, J. Pang, Y. Qiao, and K. Zhang, “Yume-1.5: A text-controlled interactive world generation model,” 2025. [Online]. Available: https: //arxiv.org/abs/2512.22096 [177] Y. Liu, X. Lin, X. Li, B. Yang, C. Wang, K. Sunkavalli, Y. Hold- Geoffroy, H. Tan, K. Zhang, X. Xie, Z. Shi, and Y. Hu, “Omniroam: World wandering via long-horizon panoramic video generation,” 2026. [Online]. Available: https://arxiv.org/abs/2603.30045 [178] S. Zuo, Y. Li, W. Zheng, Z. Zhu, J. Zhou, and J. Lu, “Vega: Learning to drive with natural language instructions,” 2026. [Online]. Available: https://arxiv.org/abs/2603.25741 [179] Z. Xiong, X. Ye, B. Yaman, S. Cheng, Y. Lu, J. Luo, N. Jacobs, and L. Ren, “Unidrive-wm: Unified understanding, planning and generation world model for autonomous driving,” 2026. [Online]. Available: https://arxiv.org/abs/2601.04453 [180] Y. Du, S. Yang, B. Dai, H. Dai, O. Nachum, J. Tenenbaum, D. Schuurmans, and P. Abbeel, “Learning universal policies via text-guided video generation,” in Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, Eds., 2023. [Online]. Avail- able:http://papers.nips.c/paper files/paper/2023/hash/ 1d5b9233ad716a43be5c0d3023cb82d0-Abstract-Conference.html [181] P. Ko, J. Mao, Y. Du, S. Sun, and J. B. Tenenbaum, “Learning to act from actionless videos through dense correspondences,” in The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7- 11, 2024.OpenReview.net, 2024. [Online]. Available: https: //openreview.net/forum?id=Mhb5fpA1T0 [182] M. Janner, Y. Du, J. B. Tenenbaum, and S. Levine, “Planning with diffusion for flexible behavior synthesis,” in International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesv ́ ari, G. Niu, and S. Sabato, Eds., vol. 162.PMLR, 2022, p. 9902–9915. [Online]. Available: https://proceedings.mlr.press/ v162/janner22a.html [183] Y. Li, X. Wei, X. Chi, Y. Li, Z. Zhao, H. Wang, N. Ma, M. Lu, and S. Zhang, “Manipdreamer: Boosting robotic manipulation world model with action tree and visual guidance,” in ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).IEEE, 5 2026, p. 12027–12031. [Online]. Available: http://dx.doi.org/10.1109/icassp55912.2026.11460526 [184] Y. Tian, Y. Jin, B. Yu, Y. Shi, H. Wu, C. H. Liu, K. Chen, and C. Huang, “Starry: Spatial-temporal action-centric world modeling for robotic manipulation,” 2026. [Online]. Available: https://arxiv.org/abs/2604.26848 [185] W. Lin, J. Zhang, K. Cai, and K. Wang, “Storm: Search-guided generative world models for robotic manipulation,” 2025. [Online]. Available: https://arxiv.org/abs/2512.18477 [186] Q. Garrido, T. Nagarajan, B. Terver, N. Ballas, Y. LeCun, and M. Rabbat, “Learning latent action world models in the wild,” 2026. [Online]. Available: https://arxiv.org/abs/2601.05230 [187] Y. Jiang, Y. Gu, I. W. Tsang, and M. Z. Shou, “Olaf-world: Orienting latent actions for video world modeling,” 2026. [Online]. Available: https://arxiv.org/abs/2602.10104 [188] H. Kim, L. Pinto, and S. J. Kim, “Hierarchical latent action model,” 2026. [Online]. Available: https://arxiv.org/abs/2603. 05815 [189] M. Alles, X. Zhang, P. van der Smagt, and P. Becker-Ehmck, “Latent action world models for control with unlabeled trajectories,” 2025. [Online]. Available: https://arxiv.org/abs/ 2512.10016 [190] H. Luo, W. Zhang, Y. Feng, S. Zheng, H. Xu, C. Xu, Z. Xi, Y. Fu, and Z. Lu, “Being-h0.7: A latent world-action model from egocentric videos,” 2026. [Online]. Available: https://arxiv.org/abs/2605.00078 [191] Y. Oshima, Y. Iwasawa, M. Suzuki, Y. Matsuo, and H. Furuta, “Worldpack: Compressed memory improves spatial consistency in video world modeling,” 2025. [Online]. Available: https: //arxiv.org/abs/2512.02473 [192] T. Zhu, S. Zhang, Z. Sun, J. Tian, and Y. Tang, “Memorize- and-generate: Towards long-term consistency in real-time video generation,” 2025. [Online]. Available: https://arxiv.org/abs/ 2512.18741 [193] PAN Team, J. Xiang, Y. Gu, Z. Liu, Z. Feng, Q. Gao, Y. Hu, B. Huang, G. Liu, Y. Yang, K. Zhou, D. Abrahamyan, A. Ahmad, G. Bannur, J. Chen, K. Chen, M. Deng, R. Han, X. Huang, H. Kang, Z. Liu, E. Ma, H. Ren, Y. Shinde, R. Shingre, R. Tanikella, K. Tao, D. Yang, X. Yu, C. Zeng, B. Zhou, Z. Liu, Z. Hu, and E. P. Xing, “Pan: A world model for general, interactable, and long-horizon world simulation,” 2025. [Online]. Available: https://arxiv.org/abs/2511.09057 [194] S. Zhou, Y. Du, Y. Yang, L. Han, P. Chen, D. Yeung, and C. Gan, “Learning 3d persistent embodied world models,” in Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diago, CA, USA, December 2-7, 2025 / Mexico City, Mexico, November 30 - December 5, 2025, D. Belgrave, C. Zhang, L. N. Montoya, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, N. Chen, I. V. M. Ru ́ ız, and A. Loaiza-Bonilla, Eds., 2025. [Online]. Avail- able:http://papers.nips.c/paper files/paper/2025/hash/ 970f59b22f4c72aec75174aae63c7459-Abstract-Conference.html [195] R. Li, P. Torr, A. Vedaldi, and T. Jakab, “Vmem: Consistent interactive video scene generation with surfel-indexed view memory,” in 2025 IEEE/CVF International Conference on Computer Vision (ICCV).IEEE, 10 2025, p. 25690–25699. [Online]. Available: http://dx.doi.org/10.1109/iccv51701.2025.02383 [196] T. Unterthiner, S. van Steenkiste, K. Kurach, R. Marinier, M. Michalski, and S. Gelly, “Towards accurate generative models of video: A new metric and challenges,” arXiv preprint arXiv:1812.01717, 2018. [Online]. Available: https: //arxiv.org/abs/1812.01717 [197] X. Wang, Z. Wu, and P. Peng, “Fine-flow distilling coarse- flow video generation for long-term driving world model,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 31, p. 26526–26534, 3 2026. [Online]. Available: http://dx.doi.org/10.1609/aaai.v40i31.39860 [198] H. Wang, D. Liu, H. Xie, H. Liu, E. Ma, K. Yu, L. Wang, and B. Wang, “Mila: Multi-view intensive-fidelity long-term video generation world model for autonomous driving,” 2025. [Online]. Available: https://arxiv.org/abs/2503.15875 [199] A. Bar, G. Zhou, D. Tran, T. Darrell, and Y. LeCun, “Navigation world models,” in 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).IEEE, 6 2025, p. 15791–15801. [Online]. Available: http://dx.doi.org/10.1109/cvpr52734.2025. 01472 [200] M. Hassan, S. Stapf, A. Rahimi, P. M. B. Rezende, Y. Haghighi, D. Br ̈ uggemann, I. Katircioglu, L. Zhang, X. Chen, S. Saha, M. Cannici, E. Aljalbout, B. Ye, X. Wang, A. Davtyan, M. Salzmann, D. Scaramuzza, M. Pollefeys, P. Favaro, and A. Alahi, “GEM: A generalizable ego-vision multimodal world model for fine-grained ego-motion, object dynamics, and scene composition control,” in IEEE/CVF Conference on FROM GENERATION TO SIMULATION: A CAPABILITY AUDIT, 202642 Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025.Computer Vision Foundation / IEEE, 2025, p. 22 404–22 415. [Online]. Available: https: //openaccess.thecvf.com/content/CVPR2025/html/Hassan GEMAGeneralizableEgo-VisionMultimodalWorldModel forFine-GrainedEgo-MotionCVPR2025paper.html [201] C. Zhang, Z. Wu, G. Lu, Y. Tang, and Z. Wang, “Rodyn: Taming interactive robot-dynamic 2.5d world model for robotic manipulation,” 2025. [Online]. Available: https://arxiv.org/abs/ 2510.09036 [202] D. Bogdoll, Y. Yang, T. Joseph, M. Yazgan, and J. M. Zollner, “Muvo: A multimodal generative world model for autonomous driving with geometric representations,” in 2025 IEEE Intelligent Vehicles Symposium (IV).IEEE, 6 2025, p. 2243–2250. [Online]. Available: http://dx.doi.org/10.1109/iv64158.2025.11097718 [203] J. Guo, Y. Ding, X. Chen, S. Chen, B. Li, Y. Zou, X. Lyu, F. Tan, X. Qi, Z. Li, and H. Zhao, “Dist-4d: Disentangled spatiotemporal diffusion with metric depth for 4d driving scene generation,” in 2025 IEEE/CVF International Conference on Computer Vision (ICCV).IEEE, 10 2025, p. 27231–27241. [Online]. Available: http://dx.doi.org/10.1109/iccv51701.2025.02528 [204] T. Deng, X. Chen, Y. Chen, Q. Chen, Y. Xu, L. Yang, L. Xu, Y. Zhang, B. Zhang, W. Huang, and H. Wang, “Gaussiandwm: 3d gaussian driving world model for unified scene understanding and multi-modal generation,” 2025. [Online]. Available: https://arxiv.org/abs/2512.23180 [205] J. Wang, Y. Jiang, T. He, J. Sun, Q. Zhang, J. He, J. Cao, Z. Gan, M. Sun, Q. Shao, and X. Yue, “Mvista- 4d: View-consistent 4d world model with test-time action inference for robotic manipulation,” 2026. [Online]. Available: https://arxiv.org/abs/2602.09878 [206] W. Yan, Y. Zhang, P. Abbeel, and A. Srinivas, “Videogpt: Video generation using vq-vae and transformers,” 2021. [Online]. Available: https://arxiv.org/abs/2104.10157 [207] NVIDIA, :, H. A. Alhaija, J. Alvarez, M. Bala, T. Cai, T. Cao, L. Cha, J. Chen, M. Chen, F. Ferroni, S. Fidler, D. Fox, Y. Ge, J. Gu, A. Hassani, M. Isaev, P. Jannaty, S. Lan, T. Lasser, H. Ling, M.-Y. Liu, X. Liu, Y. Lu, A. Luo, Q. Ma, H. Mao, F. Ramos, X. Ren, T. Shen, X. Sun, S. Tang, T.-C. Wang, J. Wu, J. Xu, S. Xu, K. Xie, Y. Ye, X. Yang, X. Zeng, and Y. Zeng, “Cosmos-transfer1: Conditional world generation with adaptive multimodal control,” 2025. [Online]. Available: https://arxiv.org/abs/2503.14492 [208] Y. Yang, L. Fan, Z. Shi, J. Peng, F. Wang, and Z. Zhang, “Neoverse: Enhancing 4d world model with in-the-wild monocular videos,” 2026. [Online]. Available: https://arxiv.org/abs/2601.00393 [209] A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “GLUE: A multi-task benchmark and analysis platform for natural language understanding,” in 7th International Conference on Learning Representations.OpenReview.net, 2019. [Online]. Available: https://openreview.net/forum?id=rJ4km2R5t7 [210] A.Wang,Y.Pruksachatkun,N.Nangia,A.Singh, J.Michael,F.Hill,O.Levy,andS.R.Bowman, “SuperGLUE:Astickierbenchmarkforgeneral-purpose language understanding systems,” in Advances in Neural Information Processing Systems, vol. 32, 2019, p. 3266–3280. [Online].Available:https://proceedings.neurips.c/paper/ 2019/hash/4496bf24afe7fab6f046bf4923da8de6-Abstract.html [211] S. Huang, P. Kaushik, M. Chen, H. Pan, K. Geng, O. Chehab, F. Moreno-Pino, and M. Simchowitz, “Nano world models: A minimalist implementation of future video prediction,” 2026. [Online]. Available: https://arxiv.org/abs/2605.23993 [212] Y. Shang, Y. Tang, Y. Ma, Z. Li, L. Jin, W. Su, X. Jin, Z. Wang, Z. Wang, X. Zhang, H. Su, W. He, W. Wu, H. Duan, G. Wetzstein, X. Liu, D. Shah, Z. Zhang, Z. Chen, J. Zhu, Y. Tian, T.-S. Chua, W. Zhu, C. Gao, and Y. Li, “Worldarena 2.0: Extending embodied world model benchmarking on modality, functionality and platform,” 2026. [Online]. Available: https://arxiv.org/abs/2605.17912 [213] J. Fang, Y. Lei, Q. Wan, Z. Wang, Y. Huang, Y. Xu, B. Zhao, W. Zhang, C. Gao, X. Chen, and Y. Li, “iworld- bench: A benchmark for interactive world models with a unified action generation framework,” 2026. [Online]. Available: https://arxiv.org/abs/2605.03941 [214] H. Gao, H. Qu, J. Tang, J. Wang, Z. Huang, H. Qiao, S. Huang, J. Yang, Y. Li, H. Yuan, W. Li, B. Zeng, W. Li, B. Wang, J. Liu, O. Huang, H. Huang, W. Zhang, G. Huang, N. Duan, and Y. Dong, “Spatialworld: Benchmarking interactive spatial reasoning of multimodal agents in real-world tasks,” 2026. [Online]. Available: https://arxiv.org/abs/2606.09669 [215] A. Yakefu, B. Xie, C. Xu, E. Zhang, E. Zhou, F. Jia, H. Yang, H. Fan, H. Zhang, H. Peng, J. Tan, J. Huang, K. Liu, K. Liu, K. Gu, Q. Zhang, R. Zhang, S. Huang, S. Cheng, S. Liu, T. Wang, T. Wang, W. Sun, W. Tang, Y. Wei, Y. Chen, Y. Gui, Y. Zhao, Y. Ma, Y. Wei, Y. Yang, Y. Guo, Z. Chen, Z. Du, Z. Zhang, Z. Liu, and Z. Yan, “Robochallenge: Large-scale real-robot evaluation of embodied policies,” 2025. [Online]. Available: https://arxiv.org/abs/2510.17950 [216] Q. Wang, W. Huang, Y. Zhou, H. Yin, T. Bao, J. Lyu, W. Liu, R. Zhang, J. Wu, L. Fei-Fei, and M. Li, “Enact: Evaluating embodied cognition with world modeling of egocentric interaction,” 2025. [Online]. Available: https://arxiv.org/abs/ 2511.20937

Instant research discovery