Paper deep dive
OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models
Hao Wang, Tao Yu, Liuzhou Zhang, HeXin Wang, Haopeng Jin, Yuxuan Zhou, Xinming Wang, Hongzhu Yi, Xinye Li, Yuanlei Wang, Ping Nie, Yan Huang, Yuxuan Zhang, Pengfei Zhou, Yanyan Zou, Wei Yang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Video world models must preserve the visual state of the world over time, but existing evaluation protocols often rely on generated histories, video reference, or selected revisit viewpoints that can confound the assessment of a model's true memory capability. To address this, we introduce OPIS, an input-grounded benchmark that strictly anchors the assessment to a fixed set of object instances from the initial observation for evaluating multi-object memory in video world models. The OPIS dataset comprises 500 cases across real-world, embodied-robotic, and game-world domains, providing dense object-level annotations for 12,672 rigid, articulated, and deformable instances. Our object-centric evaluator combines association and explicit visibility reasoning to hierarchically measure Object (O) Presence (P), Identity (I), and Structure (S), utilizing static or dynamic evaluation tracks based on object kinematics. Across eight image-to-video or camera-conditioned world models, our proposed OPIS scores range from 48.65 to 56.01. As the reference inventory grows from less than 20 to more than 40 objects, the Presence, Identity, and Structure scores show an overall decline, with the average Identity score falling from 40.22 to 23.11. The results demonstrate that preserving the particular object instances in the input is considerably harder than generating plausible visual elements.
Tags
Links
- Source: https://arxiv.org/abs/2609.35052v1
- Canonical: https://arxiv.org/abs/2609.35052v1
Trouble viewing inline? Open PDF directly →
Full Text
73,249 characters extracted from source content.
Expand or collapse full text
OPISWang et al. OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models Hao Wang 1,4,∗,‡ , Tao Yu 2,∗,† , Liuzhou Zhang 3,∗ , HeXin Wang 4 , Haopeng Jin 2 , Yuxuan Zhou 5 , Xinming Wang 2 , Hongzhu Yi 6 , Xinye Li 7 , Yuanlei Wang 8 , Ping Nie 9 , Yan Huang 2,12 , Yuxuan Zhang 10 , Pengfei Zhou 4,11,† , Yanyan Zou 13 , Wei Yang 1,† 1 USTC 2 CASIA 3 HKUST 4 Infrec 5 Tsinghua University 6 UCAS 7 The Chinese University of Hong Kong 8 Sun Yat-sen University 9 University of Waterloo 10 Jiangnan University 11 NUS 12 Fiveages 13 SUTD ∗ Equal contribution. † Corresponding authors. ‡ Work done during an internship at Infrec Tech. Abstract Video world models must preserve the visual state of the world over time, but existing evaluation protocols often rely on generated histories, video reference, or selected revisit viewpoints that can confound the assessment of a model’s true memory capability. To address this, we introduce OPIS, an input-grounded benchmark that strictly anchors the assessment to a fixed set of object instances from the initial observation for evaluating multi-object memory in video world models. The OPIS dataset comprises 500 cases across real-world, embodied-robotic, and game-world domains, providing dense object-level annotations for 12,672 rigid, articulated, and deformable instances. Our object-centric evaluator combines association and explicit visibility reasoning to hierarchically measure Object (O) Presence (P), Identity (I), and Structure (S), utilizing static or dynamic evaluation tracks based on object kinematics. Across eight image-to-video or camera-conditioned world models, our proposed OPIS scores range from 48.65 to 56.01. As the reference inventory grows from less than 20 to more than 40 objects, the Presence, Identity, and Structure scores show an overall decline, with the average Identity score falling from 40.22 to 23.11. The results demonstrate that preserving the particular object instances in the input is considerably harder than generating plausible visual elements. z Date: September 26, 2026 § Code: https://github.com/SSStarain/OPIS õ Data: https://huggingface.co/datasets/Kirito-Lab/OPIS-dataset 1 Introduction Video world models are increasingly being used as visual simulators, with systems spanning learned latent dynamics (Ha & Schmidhuber, 2018; Hafner et al., 2023), interactive environment generation (Bruce et al., 2024), video prediction for autonomous driving (Hu et al., 2023), and long-term spatial memory (Wu et al., 2026b). Given an initial observation and an instruction, they must carry the observed world forward over time while preserving the state that makes it the same world. For these models, the initial observation is the only source of ground-truth visual information. It is therefore essential to distinguish the input, the state carried forward from it, and the generated output. Existing protocols offer complementary views of generated worlds through independent- frame quality (Huang et al., 2024), comparison between complete generated and reference videos (Ye et al., 2026), consistency with generated history (Wu et al., 2026c), and recovery at revisits (Zhang et al., 2026a; Gu et al., 2026) (Figure 1, a–d). Although valuable for their intended tasks, these protocols do not fully isolate the preservation of the world observed in the input. Independent-frame evaluation lacks an input-object anchor; video-reference comparison may penalize valid trajectory differences; generated-history comparison can inherit accumulated drift; and revisit tests cover only selected frames and viewpoints. Meanwhile, image-level scores may conceal missing objects, identity changes, or structural degradation 1 arXiv:2609.35052v1 [cs.CV] 28 Sep 2026 OPISWang et al. (a) Frame-level (c) Video-reference (b) Prefix-conditioned (d) Revisit-based Generated video eval evaleval No input-object anchor Generated video Drift enters the reference Valid trajectory changes can be penalized Only selected frames and viewpoints Object-level memory from a ground-truth input Initialframe Nextframe Drift? eval eval · Generated video (e) OPIS fixed input Image I₀ + Generation input (prompt / trajectory) Object Reference Bank geometry Reference EvalGenerate Rollout V′ Eval result ① 0.0 s4.0 s9.8 s Bookshelf partly occluded at 4 s Rollout V 0.0 s4.0 s9.8 s Eval result ② Vase moves out of viewat 9.8s First visit frame revisit frame Reference video Generated video ignore? Eval Independent frames Figure 1: Comparison of evaluation paradigms and the input-grounded approach. Panels (a)–(d) illustrate evaluation using independent frames, generated prefixes, reference videos, and selected revisits. Panel (e) anchors both rollouts to the same object reference bank and geometry. within otherwise plausible scenes. These limitations motivate evaluating individual objects throughout generation against a fixed reference derived solely from the initial observation, with explicit reasoning about their visibility (Figure 1, e). The same input reference can support different valid rollouts, while visibility reasoning distinguishes memory failures from events such as a vase moving out of view or a bookshelf becoming partially occluded. We introduce OPIS, an input-grounded benchmark for multi-object memory in video world generation. Our annotation pipeline constructs dense object-level references with instance masks, appearance descriptors, and mobility labels, together with an auxiliary, precomputed reference geometric representation. The dataset contains 500 cases across 3 scene domains (real worlds, embodied-robotic, and game worlds) and 10 subcategories. It covers 12,672 object instances: 8,432 rigid, 2,046 articulated, and 2,194 deformable objects. The evaluator is deliberately evidence-first. It compares sampled generated frames against the fixed input reference through one-to-one association and explicit visibility reasoning, measuring Presence, whether instances are accounted for when observable; Identity, whether observed instances retain their input identities; and Structure, whether their geometry or structural properties persist. Based on object kinematics and motion expectations, Structure uses two tracks: static geometry for rigid objects expected to remain stationary, and dynamic structure for the remaining objects, assessing input-grounded structural properties while allowing motion, articulation, and deformation. Experiments on eight image-to-video (i2v) and camera-conditioned world models show that object accounting is substantially easier than preserving the particular input instances. Presence remains acceptable across models (85.63–93.49), whereas Identity is much lower (28.12–36.88). Seedance 2.0 (Seedance et al., 2026), one of the leading i2v models, achieves the highest overall OPIS score (56.01), while Echo-WM-Flash (Zhang et al., 2026b), a recently released camera-conditioned world model, leads the evaluated camera-conditioned group (51.42). The difficulty scales with the number of addressable objects: average Identity falls from 40.22 for scenes with at most 20 reference object instances to 23.11 for scenes with more than 40. These findings show that input-grounded, object-centric evaluation exposes memory failures that aggregate video-quality measures can conceal. Our contributions are fourfold: • An input-grounded formulation of multi-object memory. We anchor evaluation to a fixed set of object instances from the initial observation, measuring their persistence under camera motion and partial visibility without requiring a reference continuation. •A cross-domain dataset with fine-grained annotations. OPIS comprises 500 cases across 3 scene domains and 10 subcategories, covering 12,672 rigid, articulated, and deformable instances. It provides dense object-level annotations. 2 OPISWang et al. •An object-centric, hierarchical evaluator. We assess object memory through Presence, Identity, and Structure, combining object-level association and explicit visibility reasoning with static or dynamic evaluation tracks selected according to each object’s kinematics. •Experiments across eight models reveal a persistent gap between object presence and object identity and show that object memory preservation declines as the number of addressable objects increases. 2 Related Work 2.1 Evaluation References and Memory Table 1 compares evaluation references and object-level evidence. Frame-quality compo- nents in VBench (Huang et al., 2024) assess individual images without testing input-instance preservation. Reference-video tests in MIND (Ye et al., 2026) and Latent Spatial Memory (LSM) (Wang et al., 2026) measure reconstruction under prescribed trajectories; their errors can also reflect alternative valid continuations in open-ended generation. Local consistency in WorldScore (Duan et al., 2025) and WorldTrace (Wu et al., 2026c) uses generated history, which may already contain drift. Revisit tests in MBench (Zhang et al., 2026a), LoopBench (Wu et al., 2026c), and R2M-Bench (Gu et al., 2026) probe selected return views; R2M additionally controls for generic temporal stability. Table 1: Evaluation design comparison. Method Eval. protocol Object- level Fixed input set Visib.- aware Object struct. VBenchF, PW △✗ VBench-2.0F, PW △✗ △ △ MINDF, VR, R✗ MBenchPW, R △✗ △ △ R2M-BenchR, PW △✗ △ △ WorldTracePW, R✗ LSMVR, R, IG △✗ WorldScoreF, PW △✗ WBENCHF, PW, R △ △ △ △ T2V-CompBenchF △✗ PDI-BenchPW✓✗ △✓ OPIS (ours)IG✓ Symbols denote explicit (✓), partial (△), or no explicit (✗) support under the definitions below. F: frame-level; VR: video-reference; PW: generated-prefix/window; R: revisit; IG: input-grounded. The initial-frame return comparison in LSM (Wang et al., 2026) and the first-frame subject anchor in WBENCH (Ying et al., 2026) show that input or first-frame anchoring al- ready has precedents. 2.2 Object-Level Evidence Object-level evaluation is also established: T2V-CompBench (Sun et al., 2025) as- sesses text-grounded composition, VBench- 2.0 (Zheng et al., 2025) includes human iden- tity and anatomy, and MBench (Zhang et al., 2026a) and R2M-Bench (Gu et al., 2026) mea- sure entity consistency. Visibility handling ranges from valid-observation filtering (Ying et al., 2026) to occlusion-aware VLM judg- ments (Gu et al., 2026); PDI-Bench (Wu et al., 2026a) directly measures object rigidity from reconstructed trajectories. These mechanisms differ from retaining a fixed input inventory and distinguishing missing instances from unob- servable ones. OPIS combines that inventory with one-to-one association, explicit visibility and unknown states, and object-level Presence, Identity, and Structure. Its distinction is this integrated evaluation contract, rather than object scoring or first-frame anchoring alone. 3 OPIS Dataset 3.1 Task and Scope Each OPIS case consists of an initial imageI 0 , a generation instructionu, and a fixed evaluator- side referenceR 0 : d = (I 0 ,u,R 0 ), V = G θ (I 0 ,u), R 0 = (O 0 ,G 0 ).(1) The instance setO 0 records the objects visible in the input, whileG 0 provides auxiliary geometric evidence derived from the same image. The generator receives only(I 0 ,u); the reference is 3 OPISWang et al. Real-world 200 cases4 subcategories home public indoor natural outdoor urban Embodied / robotic 150 cases3subcategories industriallaboratory simulated embodied Game world 150 cases3 subcategories cartoonpixel-stylerealistic Object formsRigidArticulatedDeformable 500 cases 12,672 object instances 1. Candidate images Heterogeneous image sources Initial image I₀ 2. VLM selection Clear instances + noun phrases (NPs) bicyclebench 3. NP cleaning Remove background and redundant NPs ✓ bicycle ✓ bench ✓ plant Concept-level NPs 4. Instance annotation SAM 3.1 masks + DINOv3 embeddings 5. Geometry and task MoGe-2 scaffold + camera / text task Text-Prompt 6. Human review Inventory, masks, crops and task ✓ ✓ ✓ Fixed input reference Camera Trajectory plant ... Figure 2: OPIS dataset composition and construction pipeline. Top: 500 cases span real-world, embodied- robotic, and game-world domains, with ten subcategories and rigid, articulated, and deformable objects. Bottom: candidate images undergo VLM-assisted selection, noun-phrase (NP) cleaning, instance annotation, geometry and task construction, and human review to produce a fixed input reference. constructed once before generation and is never updated with generated content. OPIS therefore evaluates how well a rollout preserves input-grounded object evidence across time, without requiring a target continuation or access to the model’s internal memory. The instructions are designed to expose memory under changing viewpoints, motion, and partial visibility. They encourage smooth camera movement, parallax, and re-observation of selected objects, including cases in which an object leaves view and later reappears. These instructions define the evaluation challenge rather than a prescribed future trajectory. OPIS scores observable memory while allowing valid changes in camera pose, object motion, articulation, and deformation. 3.2 Data Coverage and Sources The OPIS dataset contains 500 cases spanning three scene domains and ten subcategories (Figure 2). Each subcategory contains 50 cases: home, public indoor, natural outdoor, and urban scenes; industrial, laboratory, and simulated embodied environments; and cartoon, pixel-style, and realistic game worlds. Across the benchmark, 12,672 addressable rigid, articulated, and deformable instances support memory evaluation across varied object densities and motion expectations. Domain-level counts and evaluation-track assignments are detailed in Appendix C (Table 3). OPIS combines real-world imagery, game screenshots, simulator-rendered scenes, and images generated with gpt-image-2.5-sunburst. Real-world sources include Visual Genome (Krishna et al., 2017), ADE20K (Zhou et al., 2017), COCO (Lin et al., 2014) images indexed through RefCOCO (Yu et al., 2016), and embodied or industrial collections such as BridgeData V2 (Walke et al., 2023) and IndustryShapes (Sapoutzoglou et al., 2026). Game imagery comes from gameplay-caption collections, including Minecraft (Fan et al., 2022) and SuperTuxKart 1 scenes; locally rendered AI2-THOR (Kolve et al., 2017) images provide simulated embodied environments. Together with the generated images, these sources provide complementary scene layouts, visual styles, and object configurations. 1 https://supertuxkart.net/ 4 OPISWang et al. 3.3 Data Construction Our construction pipeline turns heterogeneous source images into a common input-grounded reference, combining VLM-assisted selection and annotation, concept-guided instance segmen- tation, and final human review (Figure 2). Image selection and object vocabulary. A VLM first screens candidate images for a sufficient number of distinguishable object instances with clear boundaries, while proposing a list of noun phrases (NPs) describing the visible objects. When source annotations or simulator metadata already provide suitable NPs, we prioritize that vocabulary. Each NP denotes a concept and may correspond to multiple instances. A subsequent VLM-assisted cleaning stage removes background NPs and incomplete object phrases, and consolidates semantically similar or subsuming NPs. Instance segmentation and annotation. Using the cleaned NP list, we perform multiple rounds of perceptual concept segmentation (PCS) with SAM 3.1 (Carion et al., 2025) to recover individual object instances. We retain each instance mask and its corresponding image crop, and compare masks using intersection-over-union (IoU) to remove highly overlapping duplicates across queries and rounds. DINOv3 (Siméoni et al., 2026) extracts an appearance embedding from each instance crop to support identity matching. Each instance is further assigned a persistent identifier, distinguishing its kinematic form. MoGe-2 (Wang et al., 2025) provides a fixed scene-level geometric scaffold from the initial image for viewpoint and occlusion reasoning. Instruction design and review. For each case, we construct a structured task specification including a camera trajectory and a text prompt. These plans define the intended memory challenge while allowing variation in the generated trajectory. A VLM assists with the initial specification, which is then individually reviewed together with the object inventory, masks, and crops to verify annotation quality and task coherence. Finally, each case is verified by humans. 4 OPIS Evaluation OPIS evaluates memory through the persistence of individual objects, using the initial observa- tion as a fixed reference throughout generation. The evaluator first establishes which instances can be associated and observed, then measures their Presence, Identity, and Structure (Figure 3). This separates object accounting from appearance fidelity and structural preservation, while retaining uncertainty when the available evidence is insufficient. Although OPIS evaluates indi- vidual object instances, we assess frames and videos by the number rather than the proportion of failed instances, since quality within a fixed image area should not depend on object density. 4.1 Association and Visibility For each sampled frameI t , we independently extract object observations using concept-guided segmentation and appearance encoding. Each observation contains a mask, bounding box, category, appearance attributes, and embedding. We associate these observations with the fixed input inventoryO 0 through a partial one-to-one assignment constrained by the reference geometric representation, following bipartite matching formulations (Kuhn, 2010) used in object detection and tracking (Carion et al., 2020). Pairwise affinity combines embedding, attribute, and category similarity, with weights renormalized over available evidence. Null assignments allow unmatched instances, and minimum appearance-evidence requirements prevent category agreement alone from establishing identity. The one-to-one constraint prevents a detected object from being matched to multiple reference instances, while small affinity gaps between competing candidates flag ambiguous matches. Visibility reasoning distinguishes an absent object from one that cannot be assessed. For static instances, we project the input geometry into the current frame and test image bounds, projected size, and depth ordering. Camera motion is estimated with MASt3R (Leroy et al., 2024) correspondences supported 5 OPISWang et al. by the non-target background, with all annotated target masks excluded from camera fitting. Held-out background matches and aligned MoGe-2 depth provide reliability checks. A matched observation can also establish visibility directly. Independently moving objects are assessed through direct observations, since a static reference projection cannot determine their current location. Confirmed occlusion and out-of-view states are excluded from presence accounting. 4.2 Presence and Identity OPIS uses a frame-level criterion: a confirmed missing or extra object invalidates Presence for the frame, while a failure on any evaluable instance invalidates Identity or Structure. This prevents well-preserved objects from concealing a single object-memory failure or an additional generated object. Input Generated frame Iₜ Object reference bank AB C Geometry reference A′B SAM 3.1 + DINOv3 Association and visibility Presence AB C matched missing Identity A A A′ Structure MASt3R OPIS result Pose/VLM models B Figure 3: Input-grounded evaluation of object pres- ence, identity, and structure. Generated observa- tions are associated with the fixed input reference using appearance and geometric evidence. The examples distinguish a missing object (C, when ex- pected visible), an appearance change (AtoA ′ ), and a structural change; valid viewpoint and motion changes are allowed. Presence (P). LetE t contain the input in- stances with evidence that they should be visible in framet, and letp i,t ∈ 0, 1in- dicate whether instanceiis matched to an observation. LetX t denote the confirmed un- matched observations after one-to-one associ- ation, with unresolved extra-object attribution excluded. For a frame withE t ∪X t ̸= ∅, P t = ⊮[∀i∈E t , p i,t = 1]⊮[|X t | = 0]. (2) Any confirmed missing instance or confirmed unmatched extra observation makes the frame score zero. Confirmed occluded, out-of-view, or too-small instances are excluded, while unresolved visibility or unattributable extras remain unknown. Presence therefore requires complete one-to-one accounting among the instances and observations supported by suf- ficient evidence. Identity (I). For an observed match, ap- pearance fidelity is the nonnegative cosine similarity between its input and generated DI- NOv3 embeddings,a i,t = max(0, cos(e i ,be i,t )). LetA t contain matches with valid appearance evidence. ForA t ̸= ∅, letτ I denote the identity acceptance threshold, and define I t = ⊮ min i∈A t a i,t ≥ τ I 1 |A t | X i∈A t a i,t .(3) If any valid identity score falls belowτ I , the entire frame receives zero; otherwise, it retains the mean fidelity of its valid matches. Missing objects contribute to P and do not supply an identity measurement. 4.3 Structure Across Static and Dynamic Objects Structure evaluates properties that should persist despite valid changes in viewpoint or motion. Mobility annotations route rigid objects expected to remain stationary to a static-geometry track; articulated, deformable, and other dynamic-track instances are assessed through dynamic structure. The tracks share an input-grounded reference but use evidence appropriate to each object’s physical characteristics. Static geometry. For a matched static instance, the estimated camera transform projects its fixed input geometry into the current frame. We measure the discrepancy between these 6 OPISWang et al. projections and image correspondences within the reference and observed bounding boxes: ρ i,t = median n∈M i,t ∥Π(K t (R t X 0 n + T t ))− x n,t ∥ 2 diag(b i,t ) , s stat i,t = exp(−ρ i,t /λ g ).(4) HereM i,t contains instance-associated correspondences,X 0 n is their input-derived geometry, andb i,t is the observed bounding box. Residuals use all selected correspondences rather than only camera-fit inliers. Reliable camera and visibility evidence and sufficient correspondences are required for scoring. The resulting measure captures projective preservation of the visible input structure. Dynamic structure. The hybrid evaluator first uses specialist pose models, ViTPose (Xu et al., 2022) and ViTPose++ (Xu et al., 2024), for supported articulated categories. Visible landmarks are lifted using monocular geometry, and corresponding segment lengths are compared after removing a single scale factor per instance: r i,t,e = log ℓ i,t,e ℓ i,0,e , s pose i,t = 1 |E i,t | X e∈E i,t exp − |r i,t,e − median e ′ ∈E i,t r i,t,e ′ | τ .(5) This assesses segment proportions without directly penalizing joint angles or global pose; uniform scale changes are removed by normalization. For other objects, a VLM assesses localized structural claims established from the input image. Claims concern visible parts, connectivity, local shape, or material continuity, as appropriate to the object’s kinematic class. Each judgment is supported, contradicted, or unknown. The score averages the supported fraction within each evaluable claim family, then across families. 4.4 Aggregation and Evidence Coverage LetB t contain instances with valid structural measurements in framet, using the static or dynamic track assigned to each instance. The structural score uses the per-instance structure acceptance threshold τ S : S t = ⊮ min i∈B t s i,t ≥ τ S 1 |B t | X i∈B t s i,t , B t ̸= ∅.(6) Thus, any valid structure score belowτ S invalidates the frame; otherwise, the frame retains the mean score over its valid instances. LetFbe all sampled frames, letF P contain frames with expected-visible reference evidence or confirmed extra-observation evidence, and letF I andF S contain frames with valid appearance and structural measurements, respectively. Case scores are P = P t∈F P P t |F P | , I = P t∈F I I t |F I | , S = |F S | |F| |z C S P t∈F S S t |F S | .(7) A structure-valid frame contains at least one valid static or dynamic measurement; a measured score belowτ S still counts toward coverage. Multiplication byC S prevents a few assessable frames from representing an entire rollout. We report coverage separately because a low S can reflect structural failure or insufficient evidence. The case-level OPIS score is a weighted arithmetic mean with nonnegative component weights (w P ,w I ,w S ) satisfying w P + w I + w S = 1: Score OPIS = 100 (w P P + w I I + w S S).(8) The three components are evaluated on their respective evidence sets: confirmed missing instances are penalized through Presence, whereas Identity and Structure characterize the fidelity of instances with valid measurements. Consequently, the weighted OPIS score is a 7 OPISWang et al. composite diagnostic rather than a joint probability that all input objects are preserved. Scores are macro-averaged over cases within each subcategory, then over subcategories within each domain, and finally over domains. This hierarchy prevents densely annotated cases or larger domains from dominating the benchmark. 5 Experiments 5.1 Experimental Setup Evaluator implementation. We use SAM 3.1 (Carion et al., 2025) for instance segmentation, DINOv3 (Siméoni et al., 2026) for appearance embeddings, and MoGe-2 (Wang et al., 2025) with MASt3R (Leroy et al., 2024) for input-grounded geometric evidence. The geometric reference uses up to 128 query points per instance. The association threshold is0.35, the ambiguity margin is0.05, and the normalized geometric tolerance isλ g = 0.05. Dynamic structure uses a hybrid pose–VLM pipeline: human and animal models based on ViTPose (Xu et al., 2022) and ViTPose++ (Xu et al., 2024) measure landmark proportions, with VLM evidence used for other supported cases and pose fallback. The pose tolerance isτ = 0.2; VLM assessment uses at most 12 claims per object and a confidence threshold of0.7. Identity and Structure retain the mean valid object score only when all valid objects meet their respective thresholds. The identity and structure thresholds areτ I = 0.35andτ S = 0.35, respectively, and P/I/S weights are fixed to (w P ,w I ,w S ) = (0.2, 0.4, 0.4). Evaluation setup. We target approximately 10-second rollouts for each case, using the case-specific text instruction for image-to-video models and the corresponding structured camera task for camera-conditioned world models. Generation uses model-specific adapters and supported output resolutions and frame rates. Evaluation samples every eight frames. Temporal diagnostics use timestamps derived from the recorded output frame rate. Results are macro-averaged through the case–subcategory–domain hierarchy. We estimate 95% confidence intervals using 2,000 bootstrap resamples of cases within each subcategory, retaining the same aggregation hierarchy. We evaluate eight systems spanning two interfaces. The image-to-video group comprises Wan 3.0, Seedance 2.0 (Seedance et al., 2026), Gemini Omni Flash v1.1, and MiniMax H3-Max-Turbo. The camera-conditioned group comprises SANA-WM (Zhu et al., 2026), LingBot-World 2.0 (Gao et al., 2026), Matrix-Game 3.5 (Qian et al., 2026), and Echo-WM-Flash (Zhang et al., 2026b). 5.2 Main Results Table 2 reports the formal OPIS results. Seedance 2.0 and H3-Max-Turbo obtain the highest point estimates, at 56.01 and 55.57, respectively. Within the camera-conditioned group, Echo- WM-Flash has the highest point estimate at 51.42. We also report confidence intervals for the formal OPIS results. Identity fidelity is substantially weaker than object accounting. Presence ranges from 85.63 to 93.49, while strict Identity ranges from 28.12 to 36.88. Across all rollouts, roughly half of the frames with appearance evidence contain at least one identity score belowτ I . The gap is consistent across both model interfaces: systems can account for objects that remain observable, yet frequently fail to preserve the identity of the particular instances established in the input. Matrix-Game records the strongest Identity score (36.88), while Gemini-Omni-Flash records the strongest Presence score (93.49); neither dimension alone predicts the overall ranking. This separation is precisely what an object-centric memory benchmark should reveal and what a single scene-level score would hide. Presence is relatively mature, but not uniform across systems. All eight systems obtain high Presence scores (85.63–93.49), indicating that accounting for observable objects is comparatively mature, while still leaving a measurable gap between the strongest and weakest systems. Gemini-Omni-Flash has the highest Presence score (93.49), yet its Identity (30.03), Structure (49.30), and OPIS (50.43) scores are not among the strongest. The contrast shows that high 8 OPISWang et al. Table 2: Formal OPIS results. P, I, S, and structural frame coverageC S are shown on a 0–100 scale. S already includes the coverage multiplier. All columns use the case–subcategory–domain hierarchy. OPIS reports a 95% stratified case-bootstrap interval. ModelP↑I↑S↑ C S (%) OPIS↑ [95% CI] Image-to-video Seedance 2.089.75 35.07 60.08 81.53 56.01 [52.13, 59.69] H3-Max-Turbo93.04 36.77 55.64 76.77 55.57 [52.20, 58.93] Wan-3.088.95 34.47 50.58 75.86 51.81 [48.55, 55.03] Gemini-Omni-Flash 93.49 30.03 49.30 70.94 50.43 [47.13, 53.76] Camera-conditioned Echo-WM-Flash89.65 34.73 49.00 80.22 51.42 [47.19, 55.62] LingBot-World 2.0 89.71 36.01 45.04 67.42 50.36 [46.48, 54.25] Matrix-Game 3.585.63 36.88 45.85 77.01 50.22 [45.73, 54.48] SANA-WM90.46 28.12 48.26 75.97 48.65 [45.14, 52.29] object accounting does not imply faithful preservation of the particular input instances; overall memory quality depends on balancing Presence with Identity and Structure rather than optimizing P alone. Structural preservation requires both fidelity and evidence. Strong models not only preserve object structure more effectively, but also provide verifiable evidence of that preservation across a larger fraction of generated frames. Seedance 2.0 achieves the highest coverage-adjusted Structure score (60.08) and the broadest structural frame coverage (81.53%). Across models,C S ranges from 67.42% to 81.53%, showing that structural memory depends both on preservation quality and evidence availability. Reporting Structure together with its coverage therefore distinguishes robust structural preservation from high scores supported by only a small number of assessable frames. Additional coverage diagnostics are provided in the appendix (Table 10). Per-subcategory results are included in Appendix D. 5.3 Object-Centric Diagnostics Multi-object load exposes a scaling challenge. Figure 4(c) shows a clear scaling effect: scenes with at most 20 reference objects achieve an average strict Identity of 40.22, compared with 33.10 for 21–40 objects and 23.11 for more than 40 objects. The fraction of identity-valid frames containing at least one threshold failure rises from 44.24% to 51.58% and then 67.55%, as shown in Figure 4(e). As the number of addressable instances grows, preserving every identity becomes substantially harder, revealing a central limitation of current world models in dense multi-object scenes. Cross-domain behavior. Figure 4(a) shows a clear advantage on game worlds for seven of the eight systems, while Matrix-Game performs best on embodied scenes. The domain spread is substantial: LingBot-World reaches 58.74 on game worlds versus 44.00 on embodied scenes, whereas Matrix-Game reaches 56.21 on embodied scenes versus 48.65 on game worlds. These shifts show that memory performance depends strongly on scene composition, object configuration, and the type of visual evidence available, motivating evaluation across diverse domains rather than on a single scene family. 5.4 Evaluator Validation and Ablations Aggregation. Figure 4(b) recomputes scores from exactly the same archived object measure- ments. Removing the structural threshold raises scores by 5.05–8.60 points, because strong objects can then offset a weak one within the same frame. Removing the structural coverage multiplier raises scores by 3.62–9.61 points, with the largest increase for LingBot-World, whose 9 OPISWang et al. Real-world Embodied Game Gemini-Omni-Flash H3-Max-Turbo Wan-3.0 Seedance 2.0 SANA-WM LingBot-World 2.0 Matrix-Game 3.5 Echo-WM-Flash 43.953.154.2 52.153.960.6 49.352.253.9 53.156.558.4 44.649.252.1 48.444.058.7 45.856.248.6 47.249.257.8 (a) OPIS by domain No gate No coverage Minimum Uniform +5.0+5.3-3.7+7.2 +5.4+4.3-4.7+6.2 +6.0+4.2-4.8+6.2 +5.1+3.6-5.1+5.6 +7.1+5.3-4.3+7.0 +5.4+9.6-4.8+6.6 +8.1+4.6-5.2+5.9 +8.6+4.4-5.1+6.4 (b) Aggregation change ≤ 20 21--40 > 40 Reference objects 0 20 40 60 80 100 Score (c) Memory components Presence Identity Structure ≤ 20 21--40 > 40 Objects 0 20 40 60 80 100 Coverage (%) 67.1 79.7 91.6 (d) Structural C S ≤ 20 21--40 > 40 Objects 0 20 40 60 80 100 Fail-I (%) 44.2 51.6 67.5 (e) Identity failures 40 45 50 55 60 65 OPIS −10 −5 0 5 10 Δ OPIS Figure 4: Object-centric diagnostic views. Panel (a) reports OPIS across scene domains; each cell averages subcategories equally within a domain. Panel (b) shows the change in OPIS under aggregation variants relative to the strict score. Panels (c)–(e) show how reference-inventory size affects the memory componentsP,I, andS, structural evidence coverageC S , and the fraction of identity-valid frames containing at least one strict-Identity threshold failure, respectively. Density values use the reported common-case diagnostic. structural coverage is lowest. Replacing the mean on passing frames with the minimum lowers scores by 3.71–5.24 points. Together, these ablations show that changing the aggregation rules produces consistent score shifts across models. Evaluator validity. We assess structural-claim repeatability and agreement with manual review of association, visibility, identity, and structure. Table 13 in Appendix G reports 90.5% repeat agreement (181/200 claim pairs). Agreement with manual review is 92.7% for association, 90.7% for visibility, 86.0% for identity, and 83.3% for structure (150 decisions per dimension). Structure has the lowest agreement in this audit. Appendix G describes the evaluation units, agreement metric, and interpretation. 6 Conclusion OPIS evaluates multi-object memory against a fixed initial observation through Presence, Identity, and Structure. Its strict frame-level criterion exposes failures that can be concealed by averaging over many objects, while separate coverage diagnostics identify limits of the available evidence. The current eight-model study reveals a substantial difference between object accounting and identity fidelity, as well as sensitivity to structural coverage. These findings motivate input-grounded object evaluation alongside broader measures of video quality and world-model capability. AI Use Statement Generative AI tools were used to assist with manuscript drafting and editing, dataset con- struction, and evaluation. Specifically,gpt-image-2.5-sunburstgenerated 83 benchmark input images. Vision-language models assisted with image selection, noun-phrase cleaning, task-specification drafting, quality-control review, and selected dynamic-structure judgments. No generative AI was used to replace human responsibility for scientific claims, mathematical definitions, citations, final statistics, or release decisions; other required AI-use categories are not applicable. All AI-assisted outputs were reviewed by the authors. We verified the dataset inventory, masks, task specifications, evaluator implementation, thresholds, aggregation rules, and reported statistics. Ethics Statement This work does not involve human-subject experiments, participant interaction, private data, or sensitive personal information. OPIS uses licensed or attributed real-world, simulator, and game assets, together with a limited set of AI-generated images. The benchmark is intended 10 OPISWang et al. for academic evaluation of video world models and does not provide instructions for harmful activity or surveillance. Its results may reflect biases in source datasets, generated imagery, segmentation models, and vision-language judgments. Reproducibility Statement The paper specifies the input-grounded task, annotation pipeline, evaluator, Presence–Identity– Structure metrics, aggregation hierarchy, thresholds, sampling procedure, and bootstrap analysis. OPIS contains 500 cases and 12,672 annotated object instances across three domains and ten subcategories. The appendices provide dataset composition, detailed results, ablations, generation configurations, AI-image provenance, and evaluator-validity checks. The evaluation code and configuration information are available at https://github.com/SSStarain/OPIS; the dataset is available at https://huggingface.co/datasets/Kirito-Lab/OPIS-dataset, subject to applicable licenses. The reported implementation uses SAM 3.1, DINOv3, MoGe-2, MASt3R, ViTPose/ViTPose++, and the documented association, geometry, pose, identity, structure, and aggregation settings, enabling independent reruns of the evaluator and reported analyses. References Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. In Forty-first international conference on machine learning, 2024. Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers, 2020. URL https: //arxiv.org/abs/2005.12872. Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Roman Rädle, Triantafyllos Afouras, Effrosyni Mavroudi, Katherine Xu, Tsung-Han Wu, Yu Zhou, Liliane Momeni, Rishi Hazra, Shuangrui Ding, Sagar Vaze, Francois Porcher, Feng Li, Siyuan Li, Aishwarya Kamath, Ho Kei Cheng, Piotr Dollár, Nikhila Ravi, Kate Saenko, Pengchuan Zhang, and Christoph Feichtenhofer. SAM 3: Segment anything with concepts. CoRR, abs/2511.16719, 2025. doi: 10.48550/ARXIV.2511.16719. URL https://doi.org/10.48550/arXiv.2511.16719. Haoyi Duan, Hong-Xing Yu, Sirui Chen, Li Fei-Fei, and Jiajun Wu. Worldscore: A unified evaluation benchmark for world generation, 2025. URL https://arxiv.org/abs/2504.00983. Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar. Minedojo: Building open-ended embodied agents with internet-scale knowledge. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022. URL http://papers.nips.c/paper_files/paper/ 2022/hash/74a67268c5c5910f64938cac4526a90-Abstract-Datasets_and_Benchmarks.html. Zelin Gao, Qiuyu Wang, Jiapeng Zhu, Jingye Chen, Zichen Liu, Qingyan Bai, Jiahao Wang, Yufeng Yuan, Hanlin Wang, Yichong Lu, Ka Leong Cheng, Haojie Zhang, Jian Gao, Tianrui Feng, Yuzheng Liu, Yao Yao, Yinghao Xu, Xing Zhu, Yujun Shen, and Hao Ouyang. Infinite worlds with versatile interactions, 2026. URL https://arxiv.org/abs/2607.07534. Qiwen Gu, Bingjie Gao, Rui Chen, Geng Li, Jifan Li, Qishuai Wen, Li Niu, Jing Tang, Xiangxiang Chu, and Junqiao Zhao. R2m-bench: Evaluating revisit memory via relative consistency in interactive video world models. arXiv preprint arXiv:2608.27328, 2026. 11 OPISWang et al. David Ha and Jürgen Schmidhuber. World models. arXiv preprint arXiv:1803.10122, 2(3):440, 2018. Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023. Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gianluca Corrado. Gaia-1: A generative world model for autonomous driving. arXiv preprint arXiv:2309.17080, 2023. Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Vbench: Comprehensive benchmark suite for video generative models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, p. 21807–21818. IEEE, 2024. doi: 10.1109/ CVPR52733.2024.02060. URL https://doi.org/10.1109/CVPR52733.2024.02060. Eric Kolve, Roozbeh Mottaghi, Daniel Gordon, Yuke Zhu, Abhinav Gupta, and Ali Farhadi. AI2-THOR: an interactive 3d environment for visual AI. CoRR, abs/1712.05474, 2017. URL http://arxiv.org/abs/1712.05474. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei- Fei. Visual genome: Connecting language and vision using crowdsourced dense image annotations. Int. J. Comput. Vis., 123(1):32–73, 2017. doi: 10.1007/S11263-016-0981-7. URL https://doi.org/10.1007/s11263-016-0981-7. Harold W. Kuhn. The hungarian method for the assignment problem. In Michael Jünger, Thomas M. Liebling, Denis Naddef, George L. Nemhauser, William R. Pulleyblank, Gerhard Reinelt, Giovanni Rinaldi, and Laurence A. Wolsey (eds.), 50 Years of Integer Programming 1958-2008 - From the Early Years to the State-of-the-Art, p. 29–47. Springer, 2010. doi: 10.1007/ 978-3-540-68279-0\_2. URL https://doi.org/10.1007/978-3-540-68279-0_2. Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. In Ales Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol (eds.), Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part LXXII, volume 15130 of Lecture Notes in Computer Science, p. 71–91. Springer, 2024. doi: 10.1007/978-3-031-73220-1\_5. URL https://doi.org/10.1007/978-3-031-73220-1_5. Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars (eds.), Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V, volume 8693 of Lecture Notes in Computer Science, p. 740–755. Springer, 2014. doi: 10.1007/978-3-319-10602-1\_48. URL https://doi.org/10.1007/978-3-319-10602-1_48. Runjia Qian, Zile Wang, Jihai Zhang, Kai Zou, Wei Yu, Jiaxing Li, Zexiang Liu, Yaokun Li, Fei Kang, Kaichen Huang, Mengyin An, Haobo Zhang, Biao Jiang, Jiahua Wang, Haofeng Sun, Yang Liu, and Yangguang Li. Matrix-game 3.5: Enhancing real-time streaming interactive world models with patch memory, 2026. URL https://arxiv.org/abs/2608.29910. Panagiotis Sapoutzoglou, Orestis Vaggelis, Athina Zacharia, Evangelos Sartinas, and Maria Pateraki. Industryshapes: An rgb-d benchmark dataset for 6d object pose estimation of industrial assembly components and tools, 2026. URL https://arxiv.org/abs/2602.05555. 12 OPISWang et al. Team Seedance, De Chen, Liyang Chen, Xin Chen, Ying Chen, Zhuo Chen, Zhuowei Chen, Feng Cheng, Tianheng Cheng, Yufeng Cheng, Mojie Chi, Xuyan Chi, Jian Cong, Qinpeng Cui, Fei Ding, Qide Dong, Yujiao Du, Haojie Duanmu, Junliang Fan, Jiarui Fang, Jing Fang, Zetao Fang, Chengjian Feng, Yu Gao, Diandian Gu, Dong Guo, Hanzhong Guo, Qiushan Guo, Boyang Hao, Hongxiang Hao, Haoxun He, Jiaao He, Qian He, Tuyen Hoang, Heng Hu, Ruoqing Hu, Yuxiang Hu, Jiancheng Huang, Weilin Huang, Zhaoyang Huang, Zhongyi Huang, Jishuo Jin, Ming Jing, Ashley Kim, Shanshan Lao, Yichong Leng, Bingchuan Li, Gen Li, Haifeng Li, Huixia Li, Jiashi Li, Ming Li, Xiaojie Li, Xingxing Li, Yameng Li, Yiying Li, Yu Li, Yueyan Li, Chao Liang, Han Liang, Jianzhong Liang, Ying Liang, Wang Liao, J. H. Lien, Shanchuan Lin, Xi Lin, Feng Ling, Yue Ling, Fangfang Liu, Jiawei Liu, Jihao Liu, Jingtuo Liu, Shu Liu, Sichao Liu, Wei Liu, Xue Liu, Zuxi Liu, Ruijie Lu, Lecheng Lyu, Jingting Ma, Tianxiang Ma, Xiaonan Nie, Jingzhe Ning, Junjie Pan, Xitong Pan, Ronggui Peng, Xueqiong Qu, Yuxi Ren, Yuchen Shen, Guang Shi, Lei Shi, Yinglong Song, Fan Sun, Li Sun, Renfei Sun, Wenjing Tang, Boyang Tao, Zirui Tao, Dongliang Wang, Feng Wang, Hulin Wang, Ke Wang, Qingyi Wang, Rui Wang, Shuai Wang, Shulei Wang, Weichen Wang, Xuanda Wang, Yanhui Wang, Yue Wang, Yuping Wang, Yuxuan Wang, Zijie Wang, Ziyu Wang, Guoqiang Wei, Meng Wei, Di Wu, Guohong Wu, Hanjie Wu, Huachao Wu, Jian Wu, Jie Wu, Ruolan Wu, Shaojin Wu, Xiaohu Wu, Xinglong Wu, Yonghui Wu, Ruiqi Xia, Xin Xia, Xuefeng Xiao, Shuang Xu, Bangbang Yang, Jiaqi Yang, Runkai Yang, Tao Yang, Yihang Yang, Zhixian Yang, Ziyan Yang, Fulong Ye, Bingqian Yi, Xing Yin, Yongbin You, Linxiao Yuan, Weihong Zeng, Xuejiao Zeng, Yan Zeng, Siyu Zhai, Zhonghua Zhai, Bowen Zhang, Chenlin Zhang, Heng Zhang, Jun Zhang, Manlin Zhang, Peiyuan Zhang, Shuo Zhang, Xiaohe Zhang, Xiaoying Zhang, Xinyan Zhang, Xinyi Zhang, Yichi Zhang, Zixiang Zhang, Haiyu Zhao, Huating Zhao, Liming Zhao, Yian Zhao, Guangcong Zheng, Jianbin Zheng, Xiaozheng Zheng, Zerong Zheng, Kuan Zhu, and Feilong Zuo. Seedance 2.0: Advancing video generation for world complexity, 2026. URL https://arxiv.org/abs/2604.14148. Oriane Siméoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seung Eun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, Julien Mairal, Hervé Jégou, Patrick Labatut, and Piotr Bojanowski. Dinov3. Trans. Mach. Learn. Res., 2026, 2026. URL https://openreview.net/forum?id=2NlGyqNjns. Kaiyue Sun, Kaiyi Huang, Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, and Xihui Liu. T2v-compbench: A comprehensive benchmark for compositional text-to-video gener- ation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025, p. 8406–8416. Computer Vision Foundation / IEEE, 2025. doi: 10.1109/CVPR52734.2025.00787. URL https://openaccess.thecvf.com/ content/CVPR2025/html/Sun_T2V-CompBench_A_Comprehensive_Benchmark_for_ Compositional_Text-to-video_Generation_CVPR_2025_paper.html. Homer Rich Walke, Kevin Black, Tony Z. Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen- Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, Abraham Lee, Kuan Fang, Chelsea Finn, and Sergey Levine. Bridgedata V2: A dataset for robot learning at scale. In Jie Tan, Marc Toussaint, and Kourosh Darvish (eds.), Conference on Robot Learning, CoRL 2023, 6-9 November 2023, Atlanta, GA, USA, volume 229 of Proceedings of Machine Learning Research, p. 1723–1736. PMLR, 2023. URL https://proceedings.mlr.press/v229/walke23a.html. Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with met- ric scale and sharp details. In Danielle Belgrave, Cheng Zhang, Laura N. Montoya, Hsuan-Tien Lin, Razvan Pascanu, Piotr Koniusz, Marzyeh Ghassemi, Nancy Chen, Iván Vladimir Meza Ruíz, and Arturo Loaiza-Bonilla (eds.), Advances in Neural Information 13 OPISWang et al. Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2-7, 2025 / Mexico City, Mexico, November 30 - December 5, 2025, 2025. URL http://papers.nips.c/paper_files/paper/2025/hash/ 336572db3e99930814d6b328d4220cb6-Abstract-Conference.html. Weijie Wang, Haoyu Zhao, Yifan Yang, Feng Chen, Zeyu Zhang, Yefei He, Zicheng Duan, Donny Y Chen, Yuqing Yang, and Bohan Zhuang. Latent spatial memory for video world models. arXiv preprint arXiv:2606.09828, 2026. Jiaxin Wu, Yihao Pi, Yinling Zhang, Yuheng Li, and Xueyan Zou. Quantitative video world model evaluation for geometric-consistency. arXiv preprint arXiv:2605.15185, 2026a. Tong Wu, Shuai Yang, Ryan Po, Yinghao Xu, Ziwei Liu, Dahua Lin, and Gordon Wetzstein. Video world models with long-term spatial memory. Advances in Neural Information Processing Systems, 38:49371–49393, 2026b. Xindi Wu, Sven Elflein, James Lucas, Olga Russakovsky, Laura Leal-Taixé, Despoina Paschalidou, Jonathan Lorraine, and Aljoša Ošep. Addressable memory for video world models. arXiv preprint arXiv:2608.07408, 2026c. Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. Vitpose: Simple vision transformer baselines for human pose estimation. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022. URL http://papers.nips.c/paper_files/ paper/2022/hash/fbb10d319d44f8c3b4720873e4177c65-Abstract-Conference.html. Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. Vitpose++: Vision transformer for generic body pose estimation. IEEE Trans. Pattern Anal. Mach. Intell., 46(2):1212–1230, 2024. doi: 10.1109/TPAMI.2023.3330016. URL https://doi.org/10.1109/TPAMI.2023.3330016. Yixuan Ye, Xuanyu Lu, Yuxin Jiang, Yuchao Gu, Rui Zhao, Qiwei Liang, Jiachun Pan, Fengda Zhang, Weijia Wu, and Alex Jinpeng Wang. Mind: Benchmarking memory consistency and action control in world models. arXiv preprint arXiv:2602.08025, 2026. Kaining Ying, Hengrui Hu, Siyu Ren, Jiamu Li, Fengjiao Chen, Ziwen Wang, Xuezhi Cao, Xunliang Cai, and Henghui Ding. Wbench: A comprehensive multi-turn benchmark for interactive video world model evaluation, 2026. URL https://arxiv.org/abs/2605.25874. Licheng Yu, Patrick Poirson, Shan Yang, Alexander C. Berg, and Tamara L. Berg. Modeling context in referring expressions. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (eds.), Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I, volume 9906 of Lecture Notes in Computer Science, p. 69–85. Springer, 2016. doi: 10.1007/978-3-319-46475-6\_5. URL https://doi.org/10.1007/ 978-3-319-46475-6_5. Shengjun Zhang, Zhang Zhang, Simin Huang, Zhenyu Tang, Hanyang Wang, Chensheng Dai, Min Chen, Yifan Li, Yuxin Li, Yingjie Chen, et al. Mbench: A comprehensive benchmark on memory capability for video world models. arXiv preprint arXiv:2606.00793, 2026a. Songchun Zhang, Yaowei Li, Junhao Zhuang, Weiyang Jin, Haoyu Wang, Xin Lu, Yilang Sun, Shiyi Zhang, Haoran Li, Xiaoxiao Ma, Yuming Li, Yijun Liu, Yaofeng Su, Yanwen Ma, Haoyu Wu, Zihan Su, Yue Ma, Lvmin Zhang, Haoyang Huang, Zeyue Xue, Anyi Rao, and Nan Duan. Echowm: Open and enterable omnimodal world models, 2026b. URL https://arxiv.org/abs/2608.23189. 14 OPISWang et al. Table 3: Composition of OPIS. Static and dynamic columns indicate the evaluation track assigned to each reference instance. DomainScenes Instances Static Dynamic Real-world scenes2008,0823,5654,517 Embodied / robotic 1501,9328641,068 Game worlds1502,6581,699959 Total50012,672 6,1286,544 Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, and Ziwei Liu. Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness. CoRR, abs/2503.21755, 2025. doi: 10.48550/ARXIV.2503.21755. URL https://doi.org/10.48550/arXiv.2503.21755. Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ADE20K dataset. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, p. 5122–5130. IEEE Computer Society, 2017. doi: 10.1109/CVPR.2017.544. URL https://doi.org/10.1109/CVPR.2017.544. Haoyi Zhu, Haozhe Liu, Yuyang Zhao, Tian Ye, Junsong Chen, Jincheng Yu, Tong He, Song Han, and Enze Xie. Sana-wm: Efficient minute-scale world modeling with hybrid linear diffusion transformer, 2026. URL https://arxiv.org/abs/2605.15178. A Code and Data Availability The OPIS evaluation code is available at https://github.com/SSStarain/OPIS, and the dataset is available at https://huggingface.co/datasets/Kirito-Lab/OPIS-dataset. B Limitations OPIS measures observable preservation of input objects, rather than internal memory mech- anisms, physical causality, or complete world-model competence. Its monocular reference provides estimated visible geometry, not complete 3D ground truth. Segmentation, association, camera estimation, and pose or VLM judgments can all limit the evidence. Strict frame scoring amplifies individual failures, including evaluator errors, and is more demanding in frames with more evaluable objects. The maximum video-generation length supported by some models also limits the study, so long-rollout behavior is not evaluated. C Dataset Composition OPIS contains 12,672 object instances across 500 cases, with 6,128 assigned to static-geometry evaluation and 6,544 to dynamic-structure evaluation (Table 3). D Detailed Results and Reproducibility This section provides the numeric tables corresponding to the main-text figures (Table 4, Table 5, and Table 6), and additional per-subcategory scores (Table 7, Table 8, and Table 9). They are retained for reproducibility and detailed lookup. The reported confidence intervals are computed only for the main results in Table 2. For each model, we perform a stratified case-level bootstrap: within each of its ten subcategories, we resample the available cases with replacement, recompute the subcategory–domain–overall 15 OPISWang et al. Table 4: Multi-object load across reference-inventory sizes. All score and rate columns use a 0–100 scale. Reference objects PIS Fail-I (%) C S (%) ≤ 2094.06 40.22 54.65 44.2467.14 21–4088.52 33.10 55.22 51.5879.74 > 4086.63 23.11 49.42 67.5591.60 Table 5: Aggregation sensitivity on the reported evaluation set. No gate removes only theτ S structural threshold; no coverage removes onlyC S ; minimum replaces the within-frame mean for both I and S after thresholding; uniform uses equal P/I/S weights. All perception and association evidence is held fixed. ModelStrict No gate No coverage Minimum Uniform Gemini-Omni-Flash 50.43 55.4855.7546.7257.61 H3-Max-Turbo55.57 60.9959.8750.9061.81 Wan-3.051.81 57.8455.9747.0058.00 Seedance 2.056.01 61.0859.6250.9361.63 SANA-WM48.65 55.7053.9644.3855.62 LingBot-World 2.0 50.36 55.7859.9745.6056.92 Matrix-Game 3.5 50.22 58.2754.7844.9756.12 Echo-WM-Flash 51.42 60.0255.8646.3657.80 hierarchy using the same aggregation rules, and take the 2.5th and 97.5th percentiles over 2,000 replicates. This preserves the intended domain weights while propagating case-level variation through the reported hierarchy. E Temporal Memory Diagnostics The temporal diagnostic averages each model’s hierarchy-weighted scores equally across the matched evaluation set (Figure 5). Strict Identity drops from 43.13 in the 1–4 s interval to 31.60 in 4–7 s, before reaching 33.24 in 7–10 s. Structure is more stable, moving from 54.23 to 50.33 and 51.90, while structural frame coverage changes from 83.79% to 69.89% and then 76.27%. The result identifies identity preservation as the most fragile component of multi-object memory as generation proceeds. F Additional Protocols F.1 Generation and evaluator configurations. Table 12 records the output format and sampling density in the evaluation. Duration is measured from the output file, which can differ slightly from the requested 10 seconds. All models use an eight-frame stride after the first second, so their effective sampling rates differ. Temporal plots use seconds rather than sample indices. Evaluator thresholds are given in Section 5.1; per-run manifests retain generation settings and judge configurations. F.2 AI-Generated Dataset Images and VLM Quality Control To supplement the collected real-world, embodied, and game scenes, we include a small procedurally specified subset of AI-generated initial images. The final subset contains 83 images generated withgpt-image-2.5-sunburst: 21 home indoor, 30 natural outdoor, 17 public indoor, and 15 urban outdoor cases. These images are used as benchmark initial observations. 16 OPISWang et al. Table 6: OPIS by scene domain. Each domain score averages its subcategories equally. ModelReal-world Embodied Game Gemini-Omni-Flash43.9253.1454.23 H3-Max-Turbo52.1353.9260.65 Wan-3.049.3152.1953.93 Seedance 2.053.1356.5458.35 SANA-WM44.6249.2152.11 LingBot-World 2.048.3544.0058.74 Matrix-Game 3.545.7956.2148.65 Echo-WM-Flash47.2249.2357.82 Table 7: Real-world subcategories: OPIS score. ModelHome Natural Public indoor Urban Gemini-Omni-Flash 36.89 54.2332.5751.99 H3-Max-Turbo41.02 70.8845.8150.83 Wan-3.029.17 70.6943.7953.61 Seedance 2.035.33 69.1247.7360.34 SANA-WM23.11 54.6846.8653.83 LingBot-World 2.0 27.55 66.3340.6858.84 Matrix-Game 3.526.12 62.5440.7053.78 Echo-WM-Flash32.24 64.3641.6850.62 Image generation. For each subcategory, the generator instantiates a setting description and a composition variant (for example, a kitchen, a botanical garden path, a library, or a transit plaza). A variation index is included to request a distinct location and layout across cases. The prompt is designed for image-to-video conditioning: it requests a single coherent 16:9 view with a layered foreground and middle ground, at least 12 clearly visible object instances, varied depth and overlap, and surfaces and edges that can support subsequent motion. It also excludes readable text, logos, watermarks, UI elements, borders, duplicated or malformed objects, heavy blur, and scenes dominated by undifferentiated background scenery. The reusable template is shown below. Image-generation prompt template Create <setting>. Scene category: <subcategory>. Scene variation index:<variation-index>. Use this as a distinct location and layout from every other generated case; do not reuse a generic view. Composition variation: <composition-variant>. This is a benchmark initial frame for image-to-video generation. Compose a single coherent 16:9 camera view with a rich, layered foreground and middle ground. Include at least 12 clearly visible, distinct foreground object instances with varied sizes and depths, including several overlapping but individually recognizable objects. Make the scene useful for motion: objects should have clear surfaces, edges, depth ordering, and plausible spatial relationships. 17 OPISWang et al. Table 8: Embodied subcategories: OPIS score. ModelIndustrial Laboratory Simulated Gemini-Omni-Flash43.5065.4150.50 H3-Max-Turbo45.1765.6350.95 Wan-3.049.0564.7942.71 Seedance 2.051.7367.2150.68 SANA-WM44.8362.4440.35 LingBot-World 2.040.7048.7242.57 Matrix-Game 3.551.3568.3848.89 Echo-WM-Flash45.7068.0833.90 Table 9: Game subcategories: OPIS score. ModelCartoon Pixel Realistic Gemini-Omni-Flash 54.68 55.03 52.99 H3-Max-Turbo63.11 58.09 60.76 Wan-3.045.49 56.30 60.00 Seedance 2.054.24 52.10 68.71 SANA-WM46.97 54.72 54.64 LingBot-World 2.056.06 66.83 53.32 Matrix-Game 3.546.67 41.01 58.28 Echo-WM-Flash52.57 61.78 59.12 Use natural photographic lighting, sharp focus on the main scene, realistic materials, balanced composition, and high detail. Keep important objects away from the extreme edges. Do not make a collage, catalog, isolated studio arrangement, empty landscape, or minimalist scene. No text. Exclude readable text, logos, watermarks, UI, borders, subtitles, artificial labels, duplicated objects, malformed objects, extra limbs, and heavy blur. Do not let sky, grass, pavement, walls, or other scenery dominate the frame; they may appear only as supporting context behind many concrete foreground objects. The result must look like one real photograph. Image requests use one sample per prompt (n=1), the high-quality setting, and a 1536×1024 JPEG response. Each response is converted to RGB and normalized to a 1280×720 JPEG before it enters the benchmark. The generation model, prompt revision, variation index, output dimensions, and response metadata are retained with each case for provenance. VLM review and acceptance. Every generated candidate is reviewed withgemini-2.5-flash using the image and a structured instruction. The reviewer returns one JSON object containing (i) the final list of short, lowercase noun phrases for concrete foreground objects, (i) phrases removed from or added to the candidate list, (i) a list with one entry per distinct visible foreground instance, (iv) a quality score from 1 to 5, and (v) quality flags. The review removes background-only surfaces and scenery, as well as text and HUD/UI elements; multiple instances of the same category remain separate in the instance list. The resulting noun-phrase inventory is used for the generated-image metadata, while dense object-level masks and other benchmark annotations are produced by the general annotation pipeline. 18 OPISWang et al. Table 10: Coverage audit. Percentage columns follow hierarchical averaging. No-S is a rollout count. ModelC P C I Static Dynamic Unknown No-S Gemini-Omni-Flash 97.39 97.32 17.3567.3324.273 H3-Max-Turbo97.46 97.46 19.6874.2326.503 Wan-3.098.68 98.52 23.6472.4125.193 Seedance 2.097.20 97.20 18.9979.1928.592 SANA-WM96.88 96.88 23.7375.0223.793 LingBot-World 2.0 91.88 91.88 24.5965.2335.621 Matrix-Game 3.5 90.37 90.22 28.7773.4033.212 Echo-WM-Flash 99.26 99.26 27.6176.8318.782 1--44--77--10 Time after input (s) 0 20 40 60 80 100 Score (0--100) (a) Strict component scores Presence Identity Structure 1--44--77--10 Time after input (s) 0 20 40 60 80 100 Coverage (%) (b) Structural frame coverage Mean across models Figure 5: Temporal memory diagnostics. Each interval is scored independently using the same strict rules. Left: component scores, averaged equally across models after hierarchical aggregation. Right: structural frame coverage; gray curves show individual models and black shows their mean. VLM review prompt template Inspect the image and clean the candidate noun-phrase list. Keep only concrete visible foreground objects, characters, and useful object parts. Add visible foreground objects that are missing from the list, but do not invent objects. Remove background-only scenery and surfaces, readable text, logos, watermarks, HUDs, menus, buttons, icons, counters, and other interface elements. Return one JSON object with the fieldsnoun_phrases,removed_noun_phrases, added_noun_phrases, foreground_object_instances, quality_score, and quality_flags. Use short lowercase English noun phrases without leading articles. List one entry per distinct visible foreground instance in foreground_object_instances. Assignquality_scorefrom 1 (unusable) to 5 (clear), and return an emptyquality_flagslist when no quality issue is present. A candidate is retained only if the review reports at least 10 foreground object instances, a quality score of at least 4/5, at least 5 usable noun phrases, and no quality flags. This gate is intended to enforce scene complexity and visual usability for an image-to-video initial frame; it does not replace the object-level evaluation protocol. Per-case provenance records the generation and review models, prompt versions, extracted object phrases, instance count, quality score, and any review decisions. AI-generated images are not assumed to be redistributable solely because they were generated; any release must separately verify the applicable usage and 19 OPISWang et al. Table 11: AI-generated images in the real-world portion of OPIS. Scene subcategory Number of cases Home indoor21 Natural outdoor30 Public indoor17 Urban outdoor15 Total83 Table 12: Recorded generation formats and evaluation sampling. The Samples column counts evaluated frames per rollout. ModelResolution FPS Duration (s) Stride Samples Wan-3.0854× 480 3010.000834 H3-Max-Turbo864× 496 2410.042828 Gemini-Omni-Flash640× 360 2410.000827 Seedance 2.0832× 480 2410.125828 SANA-WM1280× 704 1610.000818 LingBot-World 2.0832× 464 1610.000818 Matrix-Game 3.51280× 704 1610.000818 Echo-WM-Flash1280× 704 1610.000818 licensing conditions. G Evaluator Validity We report two complementary checks: agreement between repeated structural-claim judgments and agreement between evaluator decisions and manual review. Table 13 gives the number of comparisons and exact matches for each check. The repeatability check contains 200 pairs of judgments on the same structural claims. The manual audit contains 150 object–frame decisions for each of association, visibility, identity, and structure, yielding 600 dimension-specific comparisons. Structural-Judge Repeatability. Structural-claim repeatability concerns the VLM branch of dynamic structure. Agreement requires the same categorical judgment for a claim across the two evaluations: supported, contradicted, or unknown. Exact label agreement measures repeatability. The two calls agree on 181 of 200 claims, giving 90.5% agreement and 19 disagreements. Repeatability alone cannot establish correctness. A judge can reproduce the same error. Object–Frame Audit and Agreement Metric. The audit contains 150 decisions for each of four dimensions, using the following rubric. •Association: choose the generated observation corresponding to the reference instance, or label it null or ambiguous. • Visibility: assign expected-visible, occluded, out-of-view, too-small, or unknown. •Identity: assign preserved, changed, or unassessable using input-specific appearance, including color pattern, texture, and distinctive parts. 20 OPISWang et al. Table 13: Evaluator-validity results. Repeatability uses 200 claim pairs. Each manual-audit dimension contains 150 decisions. Agreement is the number of agreeing judgments divided byN, expressed as a percentage. CheckN Agreeing Agreement (%) Structural-claim repeatability 20018190.5 Association: exact assignment 15013992.7 Visibility: five labels15013690.7 Identity: three labels15012986.0 Structure: three labels15012583.3 •Structure: assign preserved, changed, or unassessable for input-supported properties. Static objects are judged for shape and part-layout preservation under viewpoint change; dynamic objects allow articulation and deformation consistent with their kinematic class while retaining supported proportions, connectivity, and material continuity. We report exact agreement as100× n agree /N, wheren agree is the reported number of matching judgments andNis the total number of comparisons. For repeatability, the comparison is between two judgments of the same claim; for the manual audit, it is between the evaluator decision and manual review. Association uses exact assignment agreement, while visibility, identity, and structure use categorical label agreement. Percentages are computed directly from the counts and rounded to one decimal place. Association, visibility, identity, and structure have 11, 14, 21, and 25 disagreements, respec- tively. Across these four dimensions, 529 of 600 decisions agree with manual review, giving a descriptive pooled agreement of 88.2%. This denominator counts dimension-specific decisions rather than independent samples and excludes the separate repeatability check. Structure has the largest number of disagreements in this audit. The 90.5% repeat agreement measures judge stability rather than correctness against manual review. H Rationale for OPIS scoring. Although object instances are the basic units of evaluation, OPIS aims to assess memory fidelity at the frame and video levels. For a fixed image extent, we argue that the penalty for an object-memory failure should not be diluted merely because the scene contains more correctly preserved instances. We therefore use an absolute failure criterion rather than the proportion of failed instances: a single confirmed failure is sufficient to invalidate the corresponding frame-level component. This design deliberately measures whether all evaluable instances are preserved together, rather than the average success rate of an individual instance. Accordingly, lower scores in denser scenes indicate greater difficulty in preserving the complete observable inventory, but do not by themselves establish a decline in per-instance memory fidelity. I Case Study Figure 6 shows one example for each evaluated model, spanning seven dataset subcategories. We manually compared every displayed frame with its reference and retained only cases with a visible preservation failure; cases whose low score was attributable only to an incorrect evaluator association were excluded. Table 14 reports the corresponding case-level scores. I.1 Four detailed visual comparisons We show four of the eight cases below. The selection includes two image-to-video models and two camera-conditioned world models. Each figure places the reference and generated frame in one row. 21 OPISWang et al. Figure 6: Eight visual case studies. Each tile pairs the reference input (left) with a frame from that model’s evaluated generation (right). The selected examples span seven subcategories and show visible instance-appearance or structural drift. Red boxes in the two archived panels indicate the evaluator observations used in the original comparison. Seedance 2.0 Gemini-Omni-Flash Wan 3.0 H3-Max-Turbo Echo-WM-Flash LingBot-World 2.0 Matrix-Game 3.5 SANA-WM 0 20 40 60 80 100 Strict case-level score 100.0 63.0 94.1 92.6 100.0100.0 84.6 100.0 2.5 5.2 4.2 27.5 11.4 4.9 73.6 25.4 52.4 24.3 14.7 24.4 51.2 60.8 66.2 0.0 41.9 24.4 26.4 39.3 45.0 46.3 72.8 30.2 PISOPIS Figure 7: Strict case-level Presence (P), Identity (I), Structure (S), and OPIS scores for the eight displayed cases. The chart visualizes the same values reported in Table 14. 22 OPISWang et al. Table 14: Eight visual case studies, one per model, spanning seven dataset subcategories. P, I, S, and OPIS are strict case-level scores on a 0–100 scale. ModelSubcategoryReleased caseVisible changePI S OPIS Seedance 2.0Home indoor home_indoor_0020tabletop and lighting details drift100.0 2.5 52.441.9 Gemini-Omni-Flash Urban outdoor urban_outdoor_0005foreground bicycles and furniture drift63.0 5.2 24.324.4 Wan 3.0Home indoor home_indoor_0006cat, coffee table, and sofa area disappear94.1 4.2 14.726.4 H3-Max-TurboSimulated environment simulated_environment_0023cabinetry and fixture layout changes92.6 27.5 24.439.3 Echo-WM-FlashNatural outdoor natural_outdoor_0037inflatable forms multiply and change shape100.0 11.4 51.245.0 LingBot-World 2.0Robot lab robot_lab_setup_0029tabletop objects and equipment are replaced100.0 4.9 60.846.3 Matrix-Game 3.5Realistic 3D world Realistic_style_3D_world_0001 character and background object details change 84.6 73.6 66.272.8 SANA-WMPublic indoor public_indoor_0004foreground furniture and people change100.0 25.4 0.030.2 Seedance 2.0:home_indoor_0020. For Seedance 2.0 onhome_indoor_0020, the scores are P = 100.0,I = 2.5,S = 52.4, and OPIS= 41.9. At frame 56 (2.33 seconds), the dining-room composition remains visible, while the chandelier branches, several chairs, and small tabletop objects differ from the reference. Wan 3.0:home_indoor_0006. For Wan 3.0 onhome_indoor_0006, the scores areP = 94.1, I = 4.2,S = 14.7, and OPIS= 26.4. At frame 294 (9.80 seconds), the cat, coffee table, and sofa area visible in the reference are absent, leaving the dining table and bookcase as the dominant foreground structures. LingBot-World 2.0:robot_lab_setup_0029. LingBot-World 2.0 is camera-conditioned. On robot_lab_setup_0029, its scores areP = 100.0,I = 4.9,S = 60.8, and OPIS= 46.3. At frame 152 (9.50 seconds), after the requested return interval, the original tabletop arrangement is no longer restored: a large dark object fills the foreground and the remaining items differ in color and shape. Matrix-Game 3.5:Realistic_style_3D_world_0001. Matrix-Game 3.5 is camera- conditioned. OnRealistic_style_3D_world_0001, its scores areP = 84.6,I = 73.6,S = 66.2, and OPIS= 72.8. At frame 152 (9.50 seconds), after the requested return interval, the hooded character’s facial and armor details differ, while the standing figure and sacks at the left are replaced by a blue bag and a different container arrangement. 23 OPISWang et al. Figure 8: Seedance 2.0 on home_indoor_0020: full-frame reference and generated-frame comparison. Figure 9: Wan 3.0 on home_indoor_0006: full-frame reference and generated-frame comparison. Figure 10: LingBot-World 2.0 onrobot_lab_setup_0029: full-frame reference and generated-frame comparison. 24 OPISWang et al. Figure 11: Matrix-Game 3.5 onRealistic_style_3D_world_0001: full-frame reference and generated- frame comparison. 25