Paper deep dive
Unifying Video Tasks via Spatiotemporal Analogy
Chia-Hsiang Kao, Belinda Zeng, Bharath Hariharan, Menglin Jia
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 10/2/2026, 8:28:22 AM
Summary
The paper introduces ViGeo, a framework that unifies diverse video tasks by extending visual in-context learning to the video domain through spatiotemporal canvas completion. ViGeo formulates various tasks (camera control, modal-guided generation, video manipulation) as a single completion objective on a composite video grid. It demonstrates strong generalization to unseen tasks and modalities (e.g., event cameras) and identifies a failure mode called 'task internalization' where prompt format overrides demonstrations, which can be mitigated with minimal data.
Entities (10)
Relation Signals (10)
ViGeo → extends → visual in-context learning
confidence 95% · ViGeo, a framework that extends visual in-context learning to the video domain
ViGeo → uses → Spatiotemporal Canvas Completion
confidence 95% · ViGeo ... via spatiotemporal canvas completion
Task Internalization → isidentifiedin → ViGeo
confidence 92% · we identify task internalization ... in ViGeo
ViGeo → generalizesto → Event Cameras
confidence 90% · ViGeo generalizes to unseen modalities (e.g., event cameras)
ViGeo → isbasedon → Wan2.2-TI2V-5B
confidence 90% · We perform full finetuning on Wan2.2-TI2V-5B
Task Internalization → iscausedby → prompt format shortcut
confidence 90% · trace this to a format–target shortcut in the pretraining corpus
ViGeo → isevaluatedon → DL3DV
confidence 85% · We evaluate on DL3DV
ViGeo → isevaluatedon → BridgeData V2
confidence 85% · We evaluate on held-out environments from BridgeData-v2
ViGeo → outperforms →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Adapting video models to new tasks typically requires dedicated data curation and fine-tuning. While visual analogy provides a training-free alternative by specifying tasks in-context, it remains restricted to the image domain. To explore whether analogy-based methods can unify diverse video tasks and generalize to out-of-distribution scenarios, we introduce ViGeo, a framework that extends visual in-context learning to the video domain via spatiotemporal canvas completion. Evaluated on a diverse task taxonomy with a strict train-test split, ViGeo generalizes to unseen video manipulations and zero-shot modalities (e.g., event cameras). Finally, we identify task internalization, where a query format associated with a pretrained task overrides the demonstration, and show that this shortcut can be removed with a small amount of task-unrelated data, highlighting the need to decorrelate prompt format from task identity.
Tags
Links
- Source: https://arxiv.org/abs/2609.33935v2
- Canonical: https://arxiv.org/abs/2609.33935v2
Trouble viewing inline? Open PDF directly →
Full Text
80,227 characters extracted from source content.
Expand or collapse full text
Unifying Video Tasks via Spatiotemporal Analogy Chia-Hsiang Kao †thanks: Corresponding author: ck696@cornell.edu Affiliation: Cornell University Belinda Zeng Affiliation: Meta Bharath Hariharan Affiliation: Cornell University Menglin Jia Affiliation: Meta Abstract Adapting video models to new tasks typically requires dedicated data curation and fine-tuning. While visual analogy provides a training-free alternative by specifying tasks in-context, it remains restricted to the image domain. To explore whether analogy-based methods can unify diverse video tasks and generalize to out-of-distribution scenarios, we introduce ViGeo, a framework that extends visual in-context learning to the video domain via spatiotemporal canvas completion. Evaluated on a diverse task taxonomy with a strict train-test split, ViGeo generalizes to unseen video manipulations and unseen modalities (e.g., event cameras). Finally, we identify task internalization, where a query format associated with a pretrained task overrides the demonstration, and show that this shortcut can be removed with a small amount of task-unrelated data, highlighting the need to decorrelate prompt format from task identity.11 1 Project page: https://iandrover.github.io/video_analogy/ 1 Introduction Traditionally, cultivating a new capability in visual generative models demands dedicated data curation, architectural modifications, and task-specific fine-tuning. Large language models have demonstrated a compelling alternative: models can learn to execute new tasks simply by observing a few demonstrations in context (Brown et al., 2020; Gao et al., 2021; Zhao et al., 2021; Schick and Schütze, 2021; Liu et al., 2022a; Dai et al., 2023). In vision, this principle takes the form of visual analogy (Hertzmann et al., 2001; Bar et al., 2022), where a model infers a mapping from a source-target image pair and applies it to a novel query. Yet, this capability has remained largely confined to still images, leaving the domain of video largely unexplored. Extending visual analogy to video is challenging because video tasks demand strict spatiotemporal consistency and complex motion understanding, requirements that currently drive researchers to build specialized datasets and architectures for every new capability. This paper investigates whether a single, analogy-based formulation can absorb this diversity. We ask: Can a single analogy-based framework effectively unify diverse video tasks and generalize to tasks it has never seen? Figure 1: ViGeo unifies and generalizes diverse video tasks via video analogy. ViGeo achieves video analogy by inferring a transformation from a demonstration pair and applying it to a novel query. We show that this formulation unifies a highly diverse taxonomy of tasks, including camera motion and view transfer. Additionally, it exhibits strong generalization to unseen guided generation and video manipulation tasks. ViGeo predictions are highlighted in green. Video results are provided in the supplementary material. To explore these questions, we must first design a model capable of video analogy without relying on task-specific pipelines. We introduce ViGeo22 2 Short for Video, ergo genero, Latin for “I see, therefore I generate”—an homage to Descartes. , which formulates the problem strictly as a spatiotemporal canvas completion task. Rather than relying on auxiliary task heads or textual descriptions, ViGeo arranges the demonstration input, demonstration target, and query input into a single composite video grid. The model is then trained to generate the missing quadrant, i.e., the query target, based purely on the visual context. This framework reduces diverse generative operations to a single, unified visual-completion objective, allowing the model to learn complex mapping relationships entirely from pixels. While the canvas formulation is task-agnostic in principle, the central question is whether it can support the spatiotemporal complexity of real video operations. We therefore evaluate ViGeo through a structured task taxonomy spanning three categories that demand spatiotemporal understanding: camera control, modal-guided generation (from signals such as optical flow or point tracks), and video manipulation. This taxonomy is designed to test two complementary properties of the framework: whether a single canvas can unify diverse in-distribution video tasks, and whether the learned analogy transfers to tasks that are never seen during training. Concretely, camera control serves as an in-distribution benchmark against dedicated baselines, while modality-guided generation and video manipulation are evaluated under a strict task-level train-test split. This design allows us to measure genuine in-context learning from the demonstration, rather than memorization of trained task formats. Empirically, ViGeo succeeds on both in-distribution tasks and held-out transfer tasks. On camera control, it remains competitive with task-specific baselines, showing that a single canvas can unify structured in-distribution video operations without task-specific modules. More importantly, it also generalizes beyond its training taxonomy. For modal-guided generation, ViGeo matches or surpasses purpose-built methods on several reconstruction metrics and transfers to unseen guidance modalities such as event camera data. For video manipulation, it correctly infers and executes the demonstrated transformation on 7 of 12 strictly held-out tasks, indicating that the model is not merely copying inputs but performing in-context learning over video transformations. Beyond benchmark results, we examine when the model actually relies on its demonstrations. We identify task internalization, a failure in which the task is resolved from the query’s format rather than from the demonstration. On held-out depth-to-flow task, the same demonstration yields either the desired optical flow or a copy of the RGB video, depending only on whether the RGB hint appears once or is repeated across frames. We trace this to a format–target shortcut in the pretraining corpus, where every 2×32× 3 canvas with a frozen RGB video in query has an RGB target, and show that finetuning on a small amount of data containing neither depth nor optical flow removes the shortcut and restores demonstration-following in both held-out directions. This suggests that in-context generalization depends not only on which tasks a model is trained on, but on whether the prompt format alone reveals the task. In summary, our contributions are as follows: • Formulation: We propose ViGeo, extending visual analogy to the video domain via a pure spatiotemporal canvas completion framework. • Rigorous Task-Level Evaluation: We construct a comprehensive video task taxonomy with a strict task-level train-test split, enabling the explicit measurement of emergent generalization rather than memorization. • One-Shot Capabilities: We demonstrate that ViGeo achieves competitive or even superior performance against task-specific baselines, successfully generalizing to unseen modalities (e.g., event cameras) and novel video manipulation tasks. • Analysis: We identify task internalization, in which during evaluation a specific query input format overrides the demonstration. We trace it to a format–target shortcut in the training data and remove it with a small amount of task-unrelated data, showing that in-context generalization requires prompt formats that do not themselves reveal the task. 2 Related Work Visual (image) analogies. The idea of transduction with visual demonstration is first explored in Image Analogies (Hertzmann et al., 2001). In this earliest conception, it defines it as “synthesis of a new filtered target image” by first inferring the filters from a pair of images in the design phase and applying the transformation on a target image in the application phase, opening up many earlier works (Reed et al., 2015; Liao et al., 2017). Recently, Visual Prompting (Bar et al., 2022) fits this idea into a general self-supervised learning scheme by reformulating it as an inpainting/completion pretext task. Since then, numerous works have been proposed, such as (Xu et al., 2023; Šubrtová et al., 2023; Wang et al., 2023a; Wang et al., 2023b; Nguyen et al., 2023; Zhang et al., 2024; Gu et al., 2024; Lai et al., 2025; Srivastava et al., 2025; Kim et al., 2025; Li et al., 2025). However, they come with one of the following shortcomings: (1) Focus on object-centric editing, manipulation, or style transfer tasks (Šubrtová et al., 2023; Nguyen et al., 2023; Srivastava et al., 2025; Kim et al., 2025; Li et al., 2025; Lai et al., 2025), limiting their scope to localized or low-level transformation; (2) Use auxiliary text prompt (Wang et al., 2023b; Gu et al., 2024; Lai et al., 2025), making it hard to disentangle the effect of the text prompt and image demonstration; or (3) Evaluate on training tasks, with a few exceptions like (Li et al., 2025). In comparison, our ViGeo operates in the video regime, focus on scene-level, video-prompt setup, and only evaluates on testing tasks or scenarios. Concept transfer in video generation. In the video domain, one-shot learning has been focusing on concept transfer in specific domains, such as camera pose (Luo et al., 2025; Mitchel et al., 2025), motion (Sun et al., 2024), and editing (Zhang et al., 2025b). In these works, architectural changes and specific objective functions are designed, but are not yet ready to extend towards diverse video tasks. Our ViGeo demonstrates a generic framework by drawing visual analogies. 3 Method 3.1 Canvas Setup Consider a demo input video (A [rgb]1,0.3906,0.3047A ∈ℝT×H×W×3 ^T× H× W× 3), a demo target (B [rgb]0.9961,0.6836,0B), a query input (C [rgb]0.375,0.8516,0.2148C), and a query target (D [rgb]0.1758,0.5664,1D). A video analogy requires the mapping from A to B is the same as the mapping from C to D, i.e., A:B∼C:D [rgb]1,0.3906,0.3047A: [rgb]0.9961,0.6836,0B [rgb]0.375,0.8516,0.2148C: [rgb]0.1758,0.5664,1D. We define a canvas construction operator C. This operator spatially arranges the videos into a large composite video canvas (A,B,C,D)∈ℝT×(2H+m)×(2W+m)×3C(A,B,C,D) ^T×(2H+m)×(2W+m)× 3, forming a 2×22× 2 grid separated by a margin of size m. The demonstrations are placed in the top row and queries in the bottom, yielding [ABCD] bmatrix [rgb]1,0.3906,0.3047A& [rgb]0.9961,0.6836,0B\\ [rgb]0.375,0.8516,0.2148C& [rgb]0.1758,0.5664,1D bmatrix (the margin is omitted for clarity). Table 1: Overview of task taxonomy and train/eval protocol. We train on a diverse set of task families that all share a single video analogy canvas [ABCD] bmatrix [rgb]1,0.3906,0.3047A& [rgb]0.9961,0.6836,0B\\ [rgb]0.375,0.8516,0.2148C& [rgb]0.1758,0.5664,1D bmatrix: a demonstration pair (A,B)( [rgb]1,0.3906,0.3047A, [rgb]0.9961,0.6836,0B) shows the transformation, and the model must generate D [rgb]0.1758,0.5664,1D given the query input C [rgb]0.375,0.8516,0.2148C. Evaluation tasks are strictly held out to ensure the model never sees them during training. For 2×32×3 layouts, the input spans two grid cells (separated by “∣ ”). See Appendix C for full per-task schemas. Category Task Family A/C [rgb]1,0.3906,0.3047A/ [rgb]0.375,0.8516,0.2148C (input) B/D [rgb]0.9961,0.6836,0B/ [rgb]0.1758,0.5664,1D (target) Layout # Split Camera Motion transfer still frame video @ traj. 2×22×2 1 Train + Eval View transfer video @ pose 1 video @ pose 2 2×22×2 2 Train + Eval Modal Modality estimation rgb video modality video 2×22×2 5 Train Multi-modal quad rgb video 4-modal grid 2×22×2 1 Train Segmentation rgb video mask / matted 2×2,32×\2,3\ 2 Train Guided generation frozen rgb vid. ∣ modal rgb video 2×32×3 4 Train Flow-guided gen frozen rgb vid. ∣ flow viz rgb video 2×32×3 1 Eval Point-guided gen frozen rgb vid. ∣ point map rgb video 2×32×3 1 Eval Event-guided gen 2 anchor frames ∣ event viz rgb video 2×32×3 1 Eval Manip. Zoom (in/out) wide view close-up 2×22×2 4 Train Photometric editing original edited 2×22×2 1 Train Spatiotemporal perm. permuted original 2×22×2 4 Train Inpainting / outpainting masked video original 2×22×2 2 Eval Deblurring blurred sharp 2×22×2 3 Eval Temporal interp. sparse frames full video 2×22×2 2 Eval Jigsaw (2D) shuffled tiles original 2×22×2 2 Eval Symmetry / low-light flipped or dark original 2×22×2 3 Eval 3.2 Task Design As the canvas treats each block as a pixel video, a wide range of video tasks fit the same interface: RGB scenes, depth maps, flow visualizations, segmentation overlays, and point trajectories all occupy the same pixel space, enabling a single model to absorb diverse tasks. We exploit this flexibility to unify a diverse set of video tasks under a single framework, spanning three categories: camera control, modal-guided generation, and video manipulation. Table 1 summarizes the full taxonomy. Training tasks. We train on over 20 task families drawn from complementary data sources, including SpatialVID (Wang et al., 2025a), DL3DV (Ling et al., 2024), SynCamVideo (Bai et al., 2024), CameraClone (Luo et al., 2025) BridgeData-v2 (Walke et al., 2023), DROID (Khazatsky et al., 2024), SA-V (Kirillov et al., 2023), and Hypersim (Roberts et al., 2021). These tasks collectively expose the model to modality estimation and generation, geometric camera transformations, and spatiotemporal permutations, all presented through the same canvas interface without any task-specific text. Evaluation protocol. To rigorously test generalization, we enforce a strict task-level train/test split: evaluation tasks are never seen during training. The held-out tasks probe three transfer axes, including (1) unseen conditioning modality: flow, point, and event-camera guidances are never used as conditioning signals during training, and event-camera is the genuine unseen-modality case as it is absent from every training task; (2) unseen operation: 2D jigsaw, deblurring, and flipping are structurally distinct from the 1D strip shuffles, warps, and photometric edits seen during training; (3) unseen environment: camera view transfer tasks are evaluated on physically held-out buildings and rooms. This design ensures that strong evaluation performance reflects genuine in-context performance from the demonstration. Furthermore, we split the source videos to train set and test set, so that any clip used in the test time, whether as a source or target and whether in the demo or query row, is not seen during training. 3.3 Learning Objective We train the model to complete the target quadrant D [rgb]0.1758,0.5664,1D using a Rectified Flows objective (Liu et al., 2022b). Let ℰE denote the video encoder and f be our generative model. During training, our target data point is the encoded query target, x1=ℰ(D)x_1=E(D), and we sample standard Gaussian noise x0∼(0,I)x_0 (0,I). For a given time step t∈[0,1]t∈[0,1], the noisy latent xtx_t is defined as a linear interpolation between the noise and the target: xt=tx1+(1−t)x0x_t=tx_1+(1-t)x_0, whereas the ground-truth velocity vtv_t is defined as vt=dxtdt=x1−x0v_t= dx_tdt=x_1-x_0. The model is trained to predict this velocity, and the loss function is formulated as the mean squared error between the model’s prediction and the true velocity. To condition the prediction on the canvas context, we make a few adjustments. Let M denote a zero-mask video of the same dimensions as D. We first encode the masked canvas, obtaining the contextual latents ℰ((A,B,C,M))E(C(A,B,C,M)). We then construct the full noisy canvas latent, denoted as ztz_t, by replacing the quadrant corresponding to the masked query target M with our noisy state xtx_t. The generative model f takes this composite latent ztz_t and the timestep t as inputs to predict the velocity of the target region. The overall training objective is defined as: ℒ=x0,x1,t[‖vt−f(zt,t)‖22]L=E_x_0,x_1,t [ \|v_t-f(z_t,t) \|_2^2 ] (1) 3.4 Training Setup All the videos (i.e., A [rgb]1,0.3906,0.3047A, B [rgb]0.9961,0.6836,0B, C [rgb]0.375,0.8516,0.2148C, D [rgb]0.1758,0.5664,1D) are cropped and resized to 480×480480× 480, with a length of 8181. With a margin m of 3232, the canvas spatial resolution is 992×992992× 992 or 992×1504992× 1504, corresponding to 2×22× 2 or 2×32× 3 layout. We perform full finetuning on Wan2.2-TI2V-5B (Wan et al., 2025) with a learning rate of 0.000050.00005 and a batch size of 512512 for 10K steps. The video encoder has a pixel patch size of 16×16×416× 16× 4 and a latent patch size of 2×2×12× 2× 1, and thus we select the margin size m to be 3232. We use FastVideo (Zhang et al., 2025a) and train the model across 3232 NVIDIA A100 nodes. 4 Experiments 4.1 Modal Guided Generation Table 2: Quantitative comparison of our method against strong task-specific baselines on flow-, point-, and event-guided generation. Notably, all three tasks are unseen to our model. The best and second best results are bolded and underlined, respectively. Optical Flow Reconstruction Metrics Task Method EPE ↓ PSNR ↑ SSIM ↑ LPIPS ↓ Flow-Guided Generation FloVD (CVPR 2025) 3.771 12.363 0.240 0.539 Go-with-the-flow (CVPR 2025) 0.356 19.214 0.491 0.181 ViGeo 0.185 22.365 0.671 0.106 Point-Guided Generation Tora (CVPR 2025) 0.612 06.923 0.366 0.230 Wan-Move (NeurIPS 2025) 0.289 17.048 0.365 0.157 ViGeo 0.275 19.515 0.543 0.226 Event-Based Interpolation CBMNet (CVPR 2023) 8.020 23.587 0.759 0.284 RE-VDM (CVPR 2025) 3.384 20.892 0.665 0.239 ViGeo 4.539 21.035 0.652 0.195 Setup. We evaluate three guided generation tasks, all strictly held out during training. Each task uses a 2×32×3 canvas layout. Specifically, for flow- and point-guided generation, the input (A [rgb]1,0.3906,0.3047A/C [rgb]0.375,0.8516,0.2148C) consists of the first frame alongside the modality visualization, whereas the target is the full RGB video (B [rgb]0.9961,0.6836,0B/D [rgb]0.1758,0.5664,1D). For event-based interpolation, two anchor RGB frames are provided and the target is the full RGB video. The three evaluation modalities are optical flow, point trajectories (16 uniformly sampled points), and event camera data. We compare against task-specific baselines: FloVD (Jin et al., 2025) and Go-with-the-flow (Burgert et al., 2025) for flow; Tora (Zhang et al., 2025c) and Wan-Move (Chu et al., 2026) for points; CBMNet (Kim et al., 2023) and RE-VDM (Chen et al., 2025a) for event-based interpolation, where our model digests rendered event data visualization. Since these tasks consider dense control signals, we adopt metrics including optical flow end-point error (EPE) and reconstruction quality (PSNR, SSIM, LPIPS). Table 2 presents the quantitative results. For flow-guided generation, ViGeo achieves exceptional performance across all metrics. Despite lacking any prior exposure to optical flow as conditioning signals, it yields the lowest end-point error (EPE) and the highest reconstruction scores (PSNR, SSIM, and LPIPS), outperforming even dedicated flow-conditioned architectures. In the point-guided setting, conditioned on sixteen, uniform points, ViGeo remains highly competitive against purpose-built baselines such as Tora (Zhang et al., 2025c) and Wan-Move (Chu et al., 2026). It secures the lowest optical flow EPE and the highest PSNR and SSIM scores, demonstrating its robust ability to parse and extrapolate from spatial prompts. Finally, we push the model’s out-of-distribution generalization further by evaluating video frame interpolation guided by event camera data, a visual representation entirely absent from the training distribution. ViGeo adapts to this novel sensory input as rendered images and achieves the best LPIPS score among all methods. The remaining metrics reveal a trade-off relative to the specialized baselines: ViGeo achieves lower optical flow EPE than CBMNet (Kim et al., 2023) and higher PSNR than RE-VDM (Chen et al., 2025a), but it trails RE-VDM in EPE and CBMNet in both PSNR and SSIM. We attribute this gap to the fact that both baselines are designed and trained specifically for event-based interpolation, whereas ViGeo handles event inputs without task-specific design. These results suggest that ViGeo produces perceptually faithful interpolations on an unseen modality, while specialized methods retain an advantage in pixel-level fidelity and motion accuracy. 4.2 Camera Control Table 3: Quantitative comparison of our method on the camera_motion_transfer task. The best and second best results are bolded and underlined, respectively. Camera Metrics Task Dataset Method RotErr ↓ TransErr ↓ CamErr ↓ Camera Motion Transfer DL3DV CamCloneMaster (SIGGRAPH Asia 2025) 3.367 13.941 15.408 XFactor (ICLR 2026) 1.066 05.378 05.761 ViGeo 2.173 14.612 15.203 Table 4: Quantitative result on the camera_view_transfer task. The best results are bolded. Camera Metrics Task Dataset Method RotErr ↓ TransErr ↓ CamErr ↓ Camera View Transfer BridgeData-v2 XFactor (ICLR 2026) 36.131 0.857 1.235 ViGeo 2.382 0.093 0.117 DROID XFactor (ICLR 2026) 64.824 1.200 1.959 ViGeo 9.149 0.116 0.261 Setup. We evaluate two camera control tasks using a 2×22×2 canvas. For camera_motion_transfer, the demonstration shows a static frame (A [rgb]1,0.3906,0.3047A) animated along a trajectory (B [rgb]0.9961,0.6836,0B); the model must apply the same camera motion to the query static frame (C [rgb]0.375,0.8516,0.2148C). We evaluate on DL3DV (Ling et al., 2024). For camera_view_transfer, the demonstration shows a scene from one viewpoint (A [rgb]1,0.3906,0.3047A) captured at a different pose (B [rgb]0.9961,0.6836,0B, with synchronized cameras); the model generates D [rgb]0.1758,0.5664,1D at the target pose given query input C [rgb]0.375,0.8516,0.2148C. We evaluate on held-out environments from BridgeData-v2 (Walke et al., 2023) and DROID (Khazatsky et al., 2024). Baselines include XFactor (Mitchel et al., 2025) (frame-level) and CamCloneMaster (Luo et al., 2025) (video-level). Metrics are rotation error (RotErr, in degrees), translation direction error (TransErr), and combined camera error (CamErr); see Appendix A.2 for reconstruction and covisible metrics. In Table 3, we benchmark ViGeo against both frame-level and video-level baselines for the camera_motion_transfer task. While XFactor achieves the highest overall scores on this specific task, ViGeo achieves highly competitive performance and outperforms CamCloneMaster on both rotation and overall camera error. For camera_view_transfer, ViGeo achieves lower camera errors than XFactor on held-out environments from BridgeData-v2 and DROID (Table 4). We note that this comparison may be unfair due to a difference in training domains: ViGeo’s training data includes such dual-camera setup from both datasets, whereas XFactor is trained on human-captured videos. As shown in our analysis in Appendix A.4, XFactor’s degradation may stem from the large roll and pitch angles of the dual-camera setup, which are rare in its training data. Taken together, our results therefore suggest that, when trained on diverse camera distributions, the video analogy formulation allows ViGeo to infer large multi-axis rotations from a single demonstration pair and transfer them to a new query. 4.3 Video Manipulation Figure 2: Generalization to unseen video processing tasks. We evaluate on 12 held-out video processing tasks and report the accuracy between the model output and the target video. Our model generalizes to 7 of 12 unseen tasks, while deblurring and geometric flipping remain challenging, and the model merely copies the input query video. Setup. We evaluate ViGeo on 12 unseen tasks using a 2×22×2 canvas. These tasks span spatiotemporal completion (2), jigsaw (2), temporal synthesis (2), restoration (4), and spatiotemporal transforms (2). We compare against a naive copy-input baseline, Visual Prompting (Bar et al., 2022), and VisualCloze (Li et al., 2025), a recent model demonstrating emergent out-of-domain capabilities. We calculate the pixel-wise mean absolute error (MAE) and report accuracy as 1−MAE1-MAE. As shown in Figure 2, on 7 of the 12 tasks, the output of ViGeo is substantially better than all three baselines, confirming that it has successfully inferred and applied the demonstrated transformation rather than defaulting to a copy. The model handles spatial permutations well, effectively applying and inverting jigsaw shuffles, and performs reliable temporal interpolation. For strongly generative tasks such as 3D outpainting, while the absolute accuracy is comparatively lower than inpainting, it is likely because it requires hallucinating novel content rather than rearranging existing pixels, ViGeo still demonstrates a significant improvement over the copy baseline. Conversely, the model struggles with the remaining 5 tasks (deblurring and flipping), where it fails to generalize and defaults to the copy baseline. This divergence in performance suggests a consistent pattern: operations that preserve local pixel structure transfer more successfully, while those requiring global geometric reorganization (flipping) or the inversion of a destructive process (deblurring) do not. We hypothesize that test-time generalization to unseen tasks is not unconditional; rather, it requires the underlying transformation to share structural primitives with those seen during training. Because deblurring and flipping involve spatial convolution and global permutation operations that are absent from our training taxonomy, the model struggles to infer them. This implies that for future work, increasing the structural diversity of the training taxonomy may be more critical than simply expanding the dataset size for existing tasks. 5 A Deep Dive to Video Analogy The same analogy can be expressed through different query formulations. For example, an unseen depth-to-flow translation can be prompted with a [depth; optical flow] demo alone. But what happens if we add an RGB hint in the query like [first-frame RGB, depth; optical flow]? In this section, we study two sensitivities: (1) task internalization (Section 5.1), where a familiar query format overrides the demonstration and the model executes the trained task instead of the demonstrated analogy. We then break this format–task coupling by finetuning with an additional normal-to-edge task to mitigate the problem. (2) perceptual invariance (Appendix B.2), where the model abstractly interprets modal guidance across different color palettes. Figure 3: Illustration of task internalization. We test an unseen depth-to-flow translation task under three different demo prompts. In prompt 1 (no RGB image provided, 2×22× 2 format) and prompt 2 (RGB hint provided, where the first RGB frame appears once), the model succeeds in outputting optical flow visualization video. In prompt 3, where the RGB hint is the first frame repeated across all frames, the model’s outputs collapse to RGB videos. 5.1 Task Internalization: The Interplay Between Familiar Queries and Novel Demonstrations Probing Setup. As shown in Figure 3, we compare three query formats: (1) depth only, in the 2×22× 2 format; (2) the first RGB frame followed by black frames, in the 2×32× 3 format; and (3) the first RGB frame repeated across all frames, i.e. a frozen video. The task is out-of-distribution, so a well-behaved model should follow the input–target relation given in the demonstration and output an optical-flow visualization in all three cases. To detect the failure mode, we measure the Pearson correlation between the generated block and the query clip’s RGB video; a high value means the model reproduced the RGB video instead of the demanded modality. We run both depth-to-flow and flow-to-depth, with n=55n=55 samples each task. Findings. The left panel of Figure 4 shows the result. Under prompt 3 the model outputs RGB videos, reaching correlations of 0.850.85 (depth-to-flow) and 0.810.81 (flow-to-depth) with the RGB input. Under prompts 1 and 2 the same model produces the correct modality, with correlations near zero (e.g., −0.02-0.02 and −0.01-0.01 respectively for the depth-to-flow task). The results have two implications. First, the failure is format-dependent rather than task-dependent: the outcome flips according to how the RGB hint is given. Second, a single RGB frame does not trigger it — prompt 2 also supplies RGB, yet behaves like the no-hint condition. Hypothesis: a format–target shortcut. Upon inspecting the training task composition, we find a structural pattern that potentially accounts for this phenomenon. In the training tasks, every 2×3 canvas containing a frozen RGB video query input has an RGB video as its target. The implication of a 2×32× 3 layout with a frozen RGB column ⇒ RGB target therefore holds. We term the behavior task internalization: once the query matches a familiar task template, the model collapses to its pretrained prior. The pattern mirrors cue competition (Rescorla, 1972), in which two informative cues are presented together and the more salient one absorbs the prediction error. This also echoes the in-context learning behaviors in large language models: when the query format suffices to recognize a pretrained task, the demonstration is ignored (Min et al., 2022), and genuine task learning occurs only when recognition fails (Pan et al., 2023). Figure 4: Task internalization, and its mitigation. Correlation between the generated output and the query’s RGB video is computed; a higher value means the model outputs RGB instead of the demanded modality. Left: the pretrained model spikes to 0.850.85 and 0.810.81 under the repeated RGB hint, in both held-out tasks, and sits near zero in every other condition. Right: after finetuning on a normal-to-edge task that contain neither depth nor optical flow, the spike disappears. 5.2 Mitigating Task Internalization Insight. Since the shortcut is structural rather than task-specific, simply training on the tasks of interest would invalidate the held-out split. Yet, this same property itself suggests a solution: because the shortcut is not tied to any task, we may be able to break the format–target correlation using unrelated content. We can construct a 2×32× 3-layout task where the target are not RGB videos anymore, forcing the model to check demo targets, and break the format–target shortcut. Experiment. We build a small-scale normal-to-edge transformation dataset (1.1% of the original training corpus) and mix it into the original pool with a 10×10× oversample and continue finetuning the model for 750750 steps. The right panel of Figure 4 shows the effect. Under the repeated RGB hint prompt, correlation with the RGB video falls from 0.850.85 to 0.040.04 for depth-to-flow and from 0.810.81 to −0.06-0.06 for flow-to-depth. See Section B.1 for more finetuned model results. 6 Conclusion We introduced ViGeo, a framework that extends visual analogy to the video domain by reducing diverse video tasks to a single spatiotemporal canvas completion objective. By arranging demonstration and query videos into a unified grid and training the model to generate the missing target quadrant, ViGeo absorbs a broad taxonomy of video operations, including camera control, modal-guided generation, and video manipulation, without task-specific text, architectures, or modules. Evaluated under a strict task-level train-test split, ViGeo matches or is competitive with dedicated baselines on entirely unseen modalities (event cameras) and successfully generalizes to 77 of 1212 held-out video manipulation tasks. Finally, our discovery of task internalization, where familiar query formats can override in-context demonstrations, provides critical insights into the design principles of spatiotemporal prompting for future generalist video models. References Bae and Davison (2024) G. Bae and A. J. Davison Rethinking inductive biases for surface normal estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 9535–9545. Cited by: §C.2.1. Bai et al. (2024) J. Bai, M. Xia, X. Wang, Z. Yuan, X. Fu, Z. Liu, H. Hu, P. Wan, and D. Zhang Syncammaster: synchronizing multi-camera video generation from diverse viewpoints. arXiv preprint arXiv:2412.07760. Cited by: §C.1.1, §3.2. Bar et al. (2022) A. Bar, Y. Gandelsman, T. Darrell, A. Globerson, and A. Efros Visual prompting via image inpainting. NIPS 35, p. 25005–25017. Cited by: §1, §2, §4.3. Brown et al. (2020) T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems 33, p. 1877–1901. Cited by: §1. Burgert et al. (2025) R. Burgert, Y. Xu, W. Xian, O. Pilarski, P. Clausen, M. He, L. Ma, Y. Deng, L. Li, M. Mousavi, et al. Go-with-the-flow: motion-controllable video diffusion models using real-time warped noise. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 13–23. Cited by: §C.2.2, §4.1. Chen et al. (2025a) J. Chen, B. Y. Feng, H. Cai, T. Wang, L. Burner, D. Yuan, C. Fermuller, C. A. Metzler, and Y. Aloimonos Repurposing pre-trained video diffusion models for event-based video interpolation. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 12456–12466. Cited by: §C.2.2, §4.1, §4.1. Chen et al. (2025b) S. Chen, H. Guo, S. Zhu, F. Zhang, Z. Huang, J. Feng, and B. Kang Video depth anything: consistent depth estimation for super-long videos. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 22831–22840. Cited by: §C.2.1. Chu et al. (2026) R. Chu, Y. He, Z. Chen, S. Zhang, X. Xu, D. WANG, H. Yi, X. Liu, H. Zhao, Y. Liu, et al. Wan-move: motion-controllable video generation via latent trajectory guidance. Advances in Neural Information Processing Systems 38, p. 404–432. Cited by: §4.1, §4.1. Dai et al. (2023) D. Dai, Y. Sun, L. Dong, Y. Hao, S. Ma, Z. Sui, and F. Wei Why can gpt learn in-context? language models secretly perform gradient descent as meta-optimizers. In Findings of the Association for Computational Linguistics: ACL 2023, p. 4005–4019. Cited by: §1. Gao et al. (2021) T. Gao, A. Fisch, and D. Chen Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), p. 3816–3830. Cited by: §1. Gu et al. (2024) Z. Gu, S. Yang, J. Liao, J. Huo, and Y. Gao Analogist: out-of-the-box visual in-context learning with image diffusion model. ACM Transactions on Graphics (TOG) 43 (4), p. 1–15. Cited by: §2. Harley et al. (2025) A. W. Harley, Y. You, X. Sun, Y. Zheng, N. Raghuraman, Y. Gu, S. Liang, W. Chu, A. Dave, S. You, et al. Alltracker: efficient dense point tracking at high resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 5253–5262. Cited by: §C.2.1. Hertzmann et al. (2001) A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin Image analogies. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p. 327–340. Cited by: §1, §2. Jain et al. (2023) J. Jain, J. Li, M. T. Chiu, A. Hassani, N. Orlov, and H. Shi Oneformer: one transformer to rule universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 2989–2998. Cited by: §C.2.1. Jin et al. (2025) W. Jin, Q. Dai, C. Luo, S. Baek, and S. Cho Flovd: optical flow meets video diffusion model for enhanced camera-controlled video synthesis. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 2040–2049. Cited by: §C.2.2, §4.1. Khazatsky et al. (2024) A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y. Chen, K. Ellis, et al. Droid: a large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945. Cited by: §A.4, §C.1.1, §3.2, §4.2. Kim et al. (2025) H. Kim, D. Kim, and S. Kim Difference inversion: interpolate and isolate the difference with token consistency for image analogy generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 18250–18259. Cited by: §2. Kim et al. (2023) T. Kim, Y. Chae, H. Jang, and K. Yoon Event-based video frame interpolation with cross-modal asymmetric bidirectional motion fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 18032–18042. Cited by: §C.2.2, §4.1, §4.1. Kirillov et al. (2023) A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W. Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, p. 4015–4026. Cited by: §C.2.1, §3.2. Lai et al. (2025) B. Lai, F. Juefei-Xu, M. Liu, X. Dai, N. Mehta, C. Zhu, Z. Huang, J. M. Rehg, S. Lee, N. Zhang, et al. Unleashing in-context learning of autoregressive models for few-shot image manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 18346–18357. Cited by: §2. Li et al. (2025) Z. Li, R. Du, J. Yan, L. Zhuo, Z. Li, P. Gao, Z. Ma, and M. Cheng Visualcloze: a universal image generation framework via visual in-context learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 18969–18979. Cited by: §2, §4.3. Liao et al. (2017) J. Liao, Y. Yao, L. Yuan, G. Hua, and S. B. Kang Visual attribute transfer through deep image analogy. arXiv preprint arXiv:1705.01088. Cited by: §2. Ling et al. (2024) L. Ling, Y. Sheng, Z. Tu, W. Zhao, C. Xin, K. Wan, L. Yu, Q. Guo, Z. Yu, Y. Lu, et al. Dl3dv-10k: a large-scale scene dataset for deep learning-based 3d vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 22160–22169. Cited by: §C.1.2, §C.3.1, §3.2, §4.2. Liu et al. (2022a) J. Liu, D. Shen, Y. Zhang, W. B. Dolan, L. Carin, and W. Chen What makes good in-context examples for gpt-3?. In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd workshop on knowledge extraction and integration for deep learning architectures, p. 100–114. Cited by: §1. Liu et al. (2022b) X. Liu, C. Gong, and Q. Liu Flow straight and fast: learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003. Cited by: §3.3. Luo et al. (2025) Y. Luo, X. Shi, J. Bai, M. Xia, T. Xue, X. Wang, P. Wan, D. Zhang, and K. Gai Camclonemaster: enabling reference-based camera control for video generation. In Proceedings of the SIGGRAPH Asia 2025 Conference Papers, p. 1–10. Cited by: §C.1.1, §2, §3.2, §4.2. Min et al. (2022) S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer Rethinking the role of demonstrations: what makes in-context learning work?. In Proceedings of the 2022 conference on empirical methods in natural language processing, p. 11048–11064. Cited by: §5.1. Mitchel et al. (2025) T. W. Mitchel, H. Ryu, and V. Sitzmann True self-supervised novel view synthesis is transferable. arXiv preprint arXiv:2510.13063. Cited by: §A.2, §A.4, §2, §4.2. Nguyen et al. (2023) T. Nguyen, Y. Li, U. Ojha, and Y. J. Lee Visual instruction inversion: image editing via image prompting. Advances in Neural Information Processing Systems 36, p. 9598–9613. Cited by: §2. Pan et al. (2023) J. Pan, T. Gao, H. Chen, and D. Chen What in-context learning “learns” in-context: disentangling task recognition and task learning. In Findings of the Association for Computational Linguistics: ACL 2023, p. 8298–8319. Cited by: §5.1. Reed et al. (2015) S. E. Reed, Y. Zhang, Y. Zhang, and H. Lee Deep visual analogy-making. Advances in neural information processing systems 28. Cited by: §2. Rescorla (1972) R. A. Rescorla A theory of pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. Classical conditioning, Current research and theory 2, p. 64–69. Cited by: §5.1. Roberts et al. (2021) M. Roberts, J. Ramapuram, A. Ranjan, A. Kumar, M. A. Bautista, N. Paczan, R. Webb, and J. M. Susskind Hypersim: a photorealistic synthetic dataset for holistic indoor scene understanding. In Proceedings of the IEEE/CVF international conference on computer vision, p. 10912–10922. Cited by: §C.2.1, §3.2. Schick and Schütze (2021) T. Schick and H. Schütze It’s not just size that matters: small language models are also few-shot learners. In Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: Human language technologies, p. 2339–2352. Cited by: §1. Srivastava et al. (2025) A. Srivastava, T. R. Menta, A. Java, A. G. Jadhav, S. Singh, S. Jandial, and B. Krishnamurthy Reedit: multimodal exemplar-based image editing. In Proceedings of the Winter Conference on Applications of Computer Vision, p. 929–939. Cited by: §2. Šubrtová et al. (2023) A. Šubrtová, M. Lukáč, J. Čech, D. Futschik, E. Shechtman, and D. Sỳkora Diffusion image analogies. In ACM SIGGRAPH 2023 Conference Proceedings, p. 1–10. Cited by: §2. Sun et al. (2024) Y. Sun, H. Zhou, L. Yuan, J. J. Sun, Y. Li, X. Jia, H. Adam, B. Hariharan, L. Zhao, and T. Liu Video creation by demonstration. arXiv preprint arXiv:2412.09551. Cited by: §2. Walke et al. (2023) H. R. Walke, K. Black, T. Z. Zhao, Q. Vuong, C. Zheng, P. Hansen-Estruch, A. W. He, V. Myers, M. J. Kim, M. Du, et al. Bridgedata v2: a dataset for robot learning at scale. In Conference on Robot Learning, p. 1723–1736. Cited by: §C.1.1, §3.2, §4.2. Wan et al. (2025) T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, J. Zeng, J. Wang, J. Zhang, J. Zhou, J. Wang, J. Chen, K. Zhu, K. Zhao, K. Yan, L. Huang, M. Feng, N. Zhang, P. Li, P. Wu, R. Chu, R. Feng, S. Zhang, S. Sun, T. Fang, T. Wang, T. Gui, T. Weng, T. Shen, W. Lin, W. Wang, W. Wang, W. Zhou, W. Wang, W. Shen, W. Yu, X. Shi, X. Huang, X. Xu, Y. Kou, Y. Lv, Y. Li, Y. Liu, Y. Wang, Y. Zhang, Y. Huang, Y. Li, Y. Wu, Y. Liu, Y. Pan, Y. Zheng, Y. Hong, Y. Shi, Y. Feng, Z. Jiang, Z. Han, Z. Wu, and Z. Liu Wan: open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. Cited by: §3.4. Wang et al. (2025a) J. Wang, Y. Yuan, R. Zheng, Y. Lin, J. Gao, L. Chen, Y. Bao, Y. Zhang, C. Zeng, Y. Zhou, et al. Spatialvid: a large-scale video dataset with spatial annotations. arXiv preprint arXiv:2509.09676. Cited by: §C.1.1, §C.2.1, §3.2. Wang et al. (2025b) J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny Vggt: visual geometry grounded transformer. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 5294–5306. Cited by: §A.2, §C.4. Wang et al. (2023a) X. Wang, W. Wang, Y. Cao, C. Shen, and T. Huang Images speak in images: a generalist painter for in-context visual learning. In CVPR, p. 6830–6839. Cited by: §2. Wang and Deng (2025) Y. Wang and J. Deng Waft: warping-alone field transforms for optical flow. arXiv preprint arXiv:2506.21526. Cited by: §C.2.1. Wang et al. (2023b) Z. Wang, Y. Jiang, Y. Lu, P. He, W. Chen, Z. Wang, M. Zhou, et al. In-context learning unlocked for diffusion models. Advances in Neural Information Processing Systems 36, p. 8542–8562. Cited by: §2. Xu et al. (2023) J. Xu, Y. Gandelsman, A. Bar, J. Yang, J. Gao, T. Darrell, and X. Wang Improv: inpainting-based multimodal prompting for computer vision tasks. arXiv preprint arXiv:2312.01771. Cited by: §2. Zhang et al. (2024) J. Zhang, B. Wang, L. Li, Y. Nakashima, and H. Nagahara Instruct me more! random prompting for visual in-context learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, p. 2597–2606. Cited by: §2. Zhang et al. (2025a) P. Zhang, Y. Chen, R. Su, H. Ding, I. Stoica, Z. Liu, and H. Zhang Fast video generation with sliding tile attention. arXiv preprint arXiv:2502.04507. Cited by: §3.4. Zhang et al. (2025b) Z. Zhang, Y. Zhou, D. Peng, J. Lim, Z. Tu, D. W. Soh, and L. G. Foo Visual prompting for one-shot controllable video editing without inversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 7784–7794. Cited by: §2. Zhang et al. (2025c) Z. Zhang, J. Liao, M. Li, Z. Dai, B. Qiu, S. Zhu, L. Qin, and W. Wang Tora: trajectory-oriented diffusion transformer for video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 2063–2073. Cited by: §4.1, §4.1. Zhao et al. (2021) Z. Zhao, E. Wallace, S. Feng, D. Klein, and S. Singh Calibrate before use: improving few-shot performance of language models. In International conference on machine learning, p. 12697–12706. Cited by: §1. Zheng et al. (2024) G. Zheng, T. Li, R. Jiang, Y. Lu, T. Wu, and X. Li Cami2v: camera-controlled image-to-video diffusion model. arXiv preprint arXiv:2410.15957. Cited by: §C.4. Appendix A More Experiments A.1 Visualization We provide qualitative rollouts generated by ViGeo across several challenging scenarios: depth-flow dual estimation, flow-guided video generation, and camera view transfer in Figure 5. These visualizations demonstrate the model’s ability to maintain spatiotemporal consistency across diverse modalities on a single unified canvas. Figure 5: Qualitative results across diverse video tasks. We visualize the model’s predictions for depth-flow dual estimation, flow-guided generation, and camera view transfer. For the demonstration videos in the context, we display only the initial frame (t=0t=0) for brevity, while the query and generated target quadrants are shown as temporal rollouts. The results highlight ViGeo’s ability to synthesize high-fidelity, motion-consistent video while adhering to the geometric and semantic constraints provided by the in-context examples. A.2 Camera Control In addition to Table 4, we provide a more detailed comparison between XFactor [Mitchel et al., 2025] and ViGeo in Table 5. Here, we consider Reconstruction Metrics along with the Covisible Metrics, which calculate the metrics on pixels that are co-visible (estimated using VGGT [Wang et al., 2025b]) from both poses of the query input and query target. In both cases, ViGeo obtains superior performance. Table 5: Quantitative comparison of our method on camera view transfer. The best results are bolded. Full Frame Covisible Task Dataset Method PSNR ↑ SSIM ↑ LPIPS ↓ PSNR ↑ SSIM ↑ Camera View Transfer BRIDGE XFactor (ICLR 2026) 13.268 0.502 0.604 14.279 0.524 ViGeo 16.773 0.622 0.299 16.908 0.608 DROID XFactor (ICLR 2026) 12.047 0.480 0.666 12.186 0.498 ViGeo 14.470 0.564 0.377 14.416 0.569 Table 6: Quantitative comparison of camera_view_transfer where the demo and query videos are from different hold-out environments. DROID (N=99N=99) BRIDGE (N=100N=100) Method RotErr ↓ TransDir ↓ RotErr ↓ TransDir ↓ XFactor (ICLR 2026) 68.92 63.64 39.48 49.12 ViGeo 31.58 18.77 27.32 22.79 During training, ViGeo leverages the environment context from both the demo and query, and this yields high reconstruction performance (Table 4). A natural next question is to challenge what would happen if the demo and query videos are from different environments? This means that the model can no longer leverage the information from the demo environment and has to extract exclusively the camera pose from the demo pair and the environment context from the query input. We construct this experiment by cross-pairing the demos and queries from different environments. As shown in Table 6, our performance worsens compared to the more aligned scenario. However, we argue that this failure mode is expected and comes naturally with the current task setup. During training time, we do not provide training scenarios where the environment of the demo videos and query videos is not the same, causing the model to build a shortcut towards using the context from both query and demo rows. A future next step is to construct synthetic demo videos that share the same relative pose difference as the query videos to encourage better pose and context disentanglement. A.3 Supplementary Experiment on Task Internalization: Flow → Depth Video Generation Similar to the depth-to-flow experiment in Section 5.1, we show the result on the held-out flow-to-depth cross-modal video translation task, where a similar task internalization pattern is observed. Figure 6: Query formats that resemble training inputs trigger task internalization. A.4 Failure Mode Analysis of XFactor The severe performance drop of XFactor [Mitchel et al., 2025] on the camera_view_transfer task (Section 4.2) warrants further investigation. To understand why a model trained on extensive camera trajectory data fails in robotic environments, we conduct a qualitative failure mode analysis. We hypothesize that the extreme camera rotations inherent to robotic manipulation represent a critical out-of-distribution (OOD) shift for XFactor. To test this, we evaluate the model across four demo-query pair combinations using two distinct domains: a general outdoor scene (representative of standard trajectory datasets like DL3DV) and an indoor robotic manipulation setup (captured via left and right side cameras observing a Franka robot, a common configuration in the DROID dataset [Khazatsky et al., 2024]). The task requires XFactor to observe the relative camera transformation in the demo pair (demo input to demo target) and apply that exact transformation to the query input. The results, illustrated in Figure 7, are consistent with this hypothesis. When the demo transformation is sourced from the standard outdoor dataset (Figure 7a, b), XFactor successfully applies the motion to both outdoor and indoor query images. However, when the demo transformation is sourced from the indoor robotic cameras (Figure 7c, d), the failure appears to be driven by the camera motion rather than by the visual appearance of robotic scenes. We attribute this failure to a gap between training and test rotation distributions. XFactor is trained on large scene- and object-level video datasets in which human camera operators rarely induce extreme roll or steep pitch. In contrast, the robotic demo used in Figures 7c and d involves a relative rotation of roughly 80° yaw, 75° pitch, and 30° roll, a combination that is likely underrepresented in XFactor’s training data. (a) Setup: Demo: Standard → Query: Standard (b) Setup: Demo: Standard → Query: Robotic (c) Setup: Demo: Robotic → Query: Standard (d) Setup: Demo: Robotic → Query: Robotic Figure 7: Qualitative failure mode analysis of XFactor. We construct a 2x2 grid of cross-domain demo and query pairs. While XFactor easily extracts and applies standard, human-captured camera motions (a, b), it completely collapses when forced to process the extreme roll, pitch, and yaw typical of robotic side-camera setups (c, d). Appendix B A Deep Dive to Video Analogy Figure 8: Robust perception across modality shifts: in-distribution vs. held-out guidance. Per-checkpoint MAE for modal-guided RGB generation under varying prompt styles. ViGeo successfully consumes out-of-distribution renderings as structural guidance. Left: Edge-guided generation works comparably well with seen (Black BG) and unseen (White BG) edge maps. Middle: Depth-guided generation maintains low error when prompted with unseen Inferno and Turbo colormaps, closely tracking the seen grayscale baseline. Right: Optical flow-guided generation adapts seamlessly to unseen black backgrounds. For reference, the ‘Static baseline’ represents the MAE obtained by trivially repeating the first frame of the ground-truth video. B.1 Additional result for task internalization Table 7: Quantitative effect of task internalization under the repeated first-frame hint (F0F_0 repeated), and its mitigation. Both directions are held out from training; the finetuning data contains neither depth nor optical flow. Best results are bolded. n=55n=55 sequences per cell. Depth-to-Flow Flow-to-Depth Method MAE ↓ SSIM ↑ MAE ↓ SSIM ↑ ViGeo (base) 0.375 0.335 0.333 0.257 ViGeo (+ finetuning) 0.142 0.818 0.108 0.793 Under the repeated first-frame hint, the base model produces an RGB copy rather than the demanded modality, giving an MAE between their outputs and targets of 0.3750.375 and SSIM of 0.3350.335 on depth-to-flow and 0.3330.333/0.2570.257 on flow-to-depth. After finetuning, error (against their target) falls by 62%62\% and 68%68\% respectively, to 0.1420.142 and 0.1080.108, while SSIM rises to 0.8180.818 and 0.7930.793. B.2 Seeing through the style: Perceptual Invariance We ask: is the model’s ability to comprehend input prompts brittle to visual styling shifts? Setup. To evaluate this, we test the model on modal-guided video generation (predicting an RGB video from its first frame and a modal guidance video) but deliberately perturb the rendering style of the guidance. Specifically, we prompt the model with modalities in visual conventions it has never seen: depth maps rendered in Inferno and Turbo colormaps (instead of the training-standard grayscale), optical flow on a black background (instead of the training-standard Middlebury white), and edge maps on a white background (instead of the training-standard black). We then measure the prediction error (MAE) of the resulting generated RGB videos with the groundtruth video. Findings. The empirical results demonstrate perceptual robustness (Figure 8). Despite the drastic pixel-level domain shifts in the guidance videos, the model consistently executes the correct structural generation. For depth-guided generation, the prediction error for unseen Inferno and Turbo prompts closely tracks the baseline grayscale performance. Similarly, optical flow guidance remains robust when inverted to a black background, and edge-guided generation succeeds regardless of the background polarity. We characterize this capability as perceptual invariance. Implications. The model’s perceptual robustness provides deeper insight into how video analogies are processed within the canvas. We hypothesize that during the encoding phase, the model decouples high-frequency spatiotemporal structure (e.g., motion boundaries, geometric gradients) from low-frequency semantic styling (the absolute RGB colormap). It can essentially "see through" the color palette to extract the underlying geometric intent. For in-context learning frameworks, this implies that input guidance can afford to be highly diverse and out-of-distribution. Appendix C Task Taxonomy C.1 Camera Pose C.1.1 Training tasks. We organise camera-pose training into two task families. Camera motion transfer (1): In camera_motion_transfer, the model is asked to generate a continuous video sequence along a target camera trajectory given (a) a single static frame to anchor appearance and (b) a demonstration of the same trajectory on a different scene. We curate this from two complementary sources. Layout is 2×22×2. (i)CamCloneMaster [Luo et al., 2025]: each demo and query row pairs a frozen first frame (left cell) with the corresponding full video (right cell); the demo and query rows are drawn from different scenes that share the same Unreal Engine 5-rendered camera trajectory, so the model must transfer the demonstrated motion to the query’s appearance. (i) SpatialVID [Wang et al., 2025a]: we precompute a per-window descriptor of the camera path from the dataset’s COLMAP poses and match windows that follow approximately the same trajectory across different real-world scenes. The matched pair is then assembled into the same layout. Camera reangling (2): The model is asked to synthesise the same scene as observed from a target camera placement B given a video at an initial placement A and a demonstration of the A→BA→B correspondence. We use synchronised multi-camera real-world recordings from BridgeData-v2 [Walke et al., 2023], DROID [Khazatsky et al., 2024], and synthetic synchronised recordings from SynCamMaster [Bai et al., 2024]. (i) camera_view_transfer: the demo row shows the same scene from two camera placements side-by-side as [demo@A ∣ demo@B]; the query row shows another scene from the same pair of placements as [query@A ∣ query@B], where the right cell is the target the model must generate. Layout: 2×22×2. (i) repose_hinted: we expand the pair task to a 2×32×3 grid by inserting a frozen first-frame at placement B as a static “hint” between the two views, giving the layout [A ∣ hint(B[0]B[0]) ∣ B] per row. The hint anchors appearance at the target placement so the model only has to recover the temporal motion, and provides a graded curriculum from the easier hinted variant to the harder unhinted camera_view_transfer. C.1.2 Evaluation tasks. For evaluation, we consider camera_view_transfer and camera_motion_transfer. For camera_view_transfer, we measure generalisation to unseen environments, we hold out a small set of physical environments at the source level and exclude every scene, session, and trajectory belonging to them from training. From Bridge we hold out two environments, toysink2_bww and tabletop_dark_wood; from DROID we hold out five buildings, Bayes -- Kitchen 3, AHG 1st floor kitchen, Bayes -- ICMS, Bayes -- Kitchen Level 1, and Smith Hall 121. The held-out evaluation set therefore measures whether the model can reangle a query scene captured in a physical environment it has never seen during training. For camera_motion_transfer, we use the 55 evaluation subset data and 100 drone subset data from DL3DV [Ling et al., 2024]. C.2 Multimodal Table 8: Per-task schema for Modal-Guided Generation. The demo and query rows share the same cell schema; the rightmost cell is the model’s target (present in the demo, absent and predicted in the query). Convention: (red film icon) = RGB video; ⋅· (gray film icon, prefixed with the modality name) = modality video. Analogously, = RGB image (single frame) and ⋅· = modality image. Task Row Schema Notes Training tasks est_depth [ ∣ depth ] est_normal [ ∣ normal ] est_semantic [ ∣ semantic ] per-pixel semantic class (NYU-40 palette). est_flow [ ∣ flow ] est_point [ ∣ points ∗ ∣ †] ∗static dots on a black image frame, indicating the initial points to be tracked. †RGB video with point tracks overlaid. multimodal_quad [ ∣ quad ∗] ∗nested 2×2 modality grid with four modalities each occupying a quadrant gen_depth_guided [ ∣ depth ∣ ] — gen_normal_guided [ ∣ normal ∣ ] — gen_edge_guided [ ∣ edge ∣ ] — gen_position_guided [ ∣ XYZ ∗ ∣ ] ∗a video where each pixel’s color encodes the 3D location of the surface it shows. complete_seg [ ∗ ∣ ] ∗selected objects are masked (black) throughout the video except in one keyframe per object where each is revealed; the model must reconstruct the full appearance and trajectory of every masked object from that single visible frame. track_seg [ ∣ hint ∗ ∣ masklet †] ∗an image where the target object is hinted with cross or scribbles. †an RGB video overlaid with the target object’s contour or mask on every frame. Held-out evaluation tasks gen_flow_guided [ ∣ flow viz ∣ ] — gen_point_guided [ ∣ points ∗ ∣ ] ∗ moving dots on a black frame. gen_event_guided [ ∗ ∣ event viz ∣ ] ∗the first and last images are given for video interpolation. Table 9: Per-task schema for Video Manipulation. Demo and query rows share the same cell schema; the rightmost cell is the model’s target. Demo and query use independently sampled degradation parameters, so the model must recover the operation from the demo row rather than memorize a fixed setting. Held-out tasks test new operation families or unseen degradations. Task Row schema Notes Training tasks zoom-in_static [full ∣ crop] Static zoom-in to the same region zoom-in_animated [full ∣ dolly-in] Animated zoom-in to the same region zoom-out_static [crop ∣ full] — zoom-out_animated [crop ∣ dolly-out] — change_params [orig ∣ degraded] Samples a tuning curve over time and applies it to the original image; variants include saturation, temperature, tone, and chromatic magnitude grid_warp [warped ∣ orig] shared Gaussian-bump field shuffle_strip_solve [shuffled ∣ orig] 1D strips, per-row permutation shuffle_strips_apply [orig ∣ shuffled] 1D strips, shared permutation shuffle_temporal [shuffled ∣ orig] / [orig ∣ shuffled] restore + apply variants Held-out evaluation tasks 3d_inpaint [masked ∣ orig] 10–20 black space-time cubes as masks 3d_outpaint [border-blk ∣ orig] 15–30 black space-time cubes as masks; the (1−mask)(1-mask) region is blacked out deblur_spatial [blurred ∣ sharp] spatial blur deblur_motion [blurred ∣ sharp] motion-blur kernel deblur_temporal [blurred ∣ sharp] temporal averaging temporal_interp_easy [N frames ∣ full] 16 random frames given for interpolation temporal_interp_hard [N frames ∣ full] 8 random frames given for interpolation jigsaw_solve [shuffled-2D ∣ orig] 2D tile grid (train was 1D strips) jigsaw_apply [orig ∣ shuffled-2D] 2D tile grid, shared perm low_light [dark+noise ∣ orig] γ-darken + Gaussian noise temporal_flip [reversed ∣ orig] global temporal symmetry horizontal_flip [mirrored ∣ orig] global spatial symmetry Table 8 summarizes the multimodal task taxonomy for both the 20 training tasks and 3 testing tasks. C.2.1 Training tasks. Training data is curated from three complementary sources: SpatialVID [Wang et al., 2025a], SA-V [Kirillov et al., 2023], and Hypersim [Roberts et al., 2021]. Per-pixel modalities for SpatialVID are produced by off-the-shelf estimators: Video Depth Anything [Chen et al., 2025b] for depth, WAFT [Wang and Deng, 2025] for optical flow, OneFormer [Jain et al., 2023] for semantic segmentation, DSINE [Bae and Davison, 2024] for surface normals, and AllTracker [Harley et al., 2025] for point trajectories. SpatialVID — modality estimation (3): The model is asked to recover a pixel-level modality from RGB. Layout: 2×22×2 [RGB ∣ modality] × 2 rows, with demo and query drawn from different scenes. We include est_depth, est_normal, and est_flow. SpatialVID — modality-guided RGB generation (3): The model is asked to synthesise an RGB video given a frozen first frame (for appearance) and a per-frame modality video (for geometry/motion). Layout: 2×32×3 [frozen first frame ∣ modality video ∣ RGB video] × 2 rows. We include gen_depth_guided, gen_normal_guided, and gen_edge_guided, where the edge variant computes Sobel/XDoG/Canny edges on the fly with the method drawn per-sample from a fixed mixture. SpatialVID — multimodal quad (1): A 2×22×2 layout where the right cell is itself a nested 2×22×2 grid containing four modalities of the same scene (RGB, depth, normal, flow), each rendered as a 240×240240×240 block. The task forces the model to consume and reproduce multiple modalities of the same scene jointly, and exposes the cross-modality consistency the demo and query rows must share. We refer to it as multimodal_quad. SpatialVID — point estimation (1): In est_point, the model is asked to track a sparse set of points across an RGB video. We sample N∈5,10,20,40N∈\5,10,20,40\ trackable points per video using AllTracker, with semantic-aware filtering (visibility >0.7>0.7, confidence >0.7>0.7, in-frame fraction ≥80%≥80\%, sky/road/earth excluded via OneFormer’s ADE20K labels, person/car preferred when available) so that the chosen points lie on physically meaningful, persistent surfaces rather than on the sky or distant background. The points are rendered as small colored dots (radius 88–1212, jittered colors). Layout: 2×32×3 [RGB video ∣ static initial dots on black ∣ dots overlaid on RGB video] × 2 rows. Hypersim — indoor multimodal (6): Hypersim provides diverse, per-pixel modalities, including RGB, depth, surface normal in camera and world frames, semantic, world-space position, and instance) on synthetic indoor scenes. We sample 21 keyframes per (scene, camera) and repeat each keyframe four times to yield 8282 frames. We define four 2×22×2 tasks: est_depth; est_normal (with camera- and world-frame variants); est_semantic (NYU-40 palette); and complete_seg, in which we pick 44–55 instance-segmented objects that are visible in at least 1818 of the 2121 keyframes and render each object unmasked in only one keyframe and mask the objects in all other frames to black, requiring the model to reconstruct the full video from sparse single-frame appearance observations. We additionally define two 2×32×3-layout tasks: gen_position_guided (frozen first-frame world-space XYZ rendered as RGB serves as guidance for the full position video; and gen_depth_guided. SA-V — video segmentation (1): SA-V provides ∼190 190K masklet annotations over real-world video. We use a 2×32×3 layout for track_seg (left: RGB video; mid: black video whose first frame carries a hint of the masklet, one of coloured semi-transparent mask, contour outline, three cross markers, bounding box, scribble drawn per-sample; right: the masklet rendered as a coloured overlay on every frame, the model’s target). C.2.2 Evaluation tasks. We evaluate the model on two held-out modality-guided RGB-generation tasks built from the SpatialVID source pool, using the same 2×32×3 layout as the corresponding training tasks but with modalities the model has never been trained to consume as guidance: (i) gen_flow_guided: the middle column is a per-frame optical-flow visualisation produced by WAFT, and the model must synthesise the corresponding RGB video; (i) gen_point_guided: the middle column shows N=16N=16 moving points (from AllTracker, with the same semantic-aware filtering used at training time) on black, and the model must produce the matching RGB video where the points implicitly anchor a sparse, deterministic motion field. (i) gen_event_guided: the left video is sparse RGB frame videos; the middle column is a event camera data visualization, and the model must synthesise the corresponding RGB video. For gen_flow_guided, we compare with Go-With-The-Flow [Burgert et al., 2025] and FloVD [Jin et al., 2025]. FloVD is a generative model based on CogVideoX that contains a flow synthesis module and a flow-conditioned video synthesis module (FVSM). We use its FVSM module for evaluation. For gen_event_guided, we compare with CBMNet [Kim et al., 2023] and RE-VDM [Chen et al., 2025a], both taking first and end frame video and raw event camera data (CBMNet gets binned grids) for video frame interpolation. For our model, it takes in two boundary anchor frames, identical to the frames the event baselines are given, with all intermediate frames blanked. Specifically, we prepare the input video by "placing frame 0 at idx 0, and frame 12 at idx 12; while other 79 frames are black". C.3 Video Manipulation Table 9 summarize the video manipulation taxonomy for both the training and testing tasks. C.3.1 Training tasks. We consider three categories of tasks and enumerate in total 99 tasks. Zoom tasks (4): We consider four variants spanning the cross product of zoom-in, zoom-out × static, animated: zoom-in_static, zoom-in_animated, zoom-out_static, zoom-out_animated, all sourced from the 4K (3840×21603840×2160) data from DL3DV [Ling et al., 2024]. For every scene we deterministically sample two nested square regions in the source frame: an outer “wide” square of side S∈[720, 1920]S∈[720,\,1920] px (i.e. up to 4×4× the close-up size, capped by the frame’s shorter dimension) and an inner 480×480480×480 “close-up” bounding box placed at a random offset inside the wide square. Both the outer square and the inner bounding box are shared between the demo and query rows, so the model must localise the close-up region by comparing the two rows. The two views are rendered as 480×480480×480 blocks: the wide view is the outer square downsampled to 480×480480×480, and the close-up view is the inner bounding box at native pixel scale. In the static variants both rows display these two views side by side as still cells, ordered [full ∣ crop] for zoom-in and [crop ∣ full] for zoom-out, with no temporal zoom motion. In the animated variants one of the two cells is replaced by a synthetic dolly-zoom: at every frame we recompute the crop window size, smoothly sweeping it from the outer square to the inner bounding box (zoom-in) or the reverse (zoom-out), and resize back to 480×480480×480. Low-level image tasks (1): We consider a change_params task in which low-level image attributes are altered with a smooth time-varying parameter that is shared between the demo and query rows, so the model must recover the underlying control signal from pixel differences alone. The control trajectory p(t)∈[0,1]p(t)∈[0,1], t=1,…,82t=1,…,82, is generated as follows: we draw 22–33 random anchor values uniformly from 120,220,…,1920\ 120, 220,…, 1920\, prepend the two endpoints 0,1\0,1\ to guarantee that both extremes are visited, randomly shuffle the combined list to determine the visit order, then concatenate equal-length segments interpolating between consecutive anchors using one of four randomly chosen easing modes (linear, ease-in, ease-out, ease-in-out). Each task maps p(t)p(t) to a different image-domain parameter: (i) saturation, a vectorised RGB saturation factor in [0,2][0,2] blending each frame with its luminance map; (i) temperature, a warm/cool shift implemented by per-frame R/B channel offsets up to ±40± 40; (i) tone, a per-frame gamma correction with γ∈[0.4, 2.8]γ∈[0.4,\,2.8] applied via cached LUTs; (iv) focus, a Gaussian blur whose kernel size sweeps from 11 to 5151 pixels along the trajectory; and (v) chromatic aberration, per-frame integer translations of the R and B channels by up to ±20± 20 px along a smooth 2D path, simulating chromatic aberration as a controllable channel-shift effect. For all five, the same trajectory is used for both rows, so the demo row implicitly defines the time-varying parameter that the query row must match. Spatiotemporal correspondence tasks (4): These tasks geometrically or temporally permute the input and require the model to either invert the permutation (restore) or replicate it (apply). We consider: (i) grid_warp, which composes 22–66 Gaussian “bumps” on a 720×720720×720 canvas with sliding, rotational, or pulsing behaviours and produces a smooth shared spatial deformation field; (i) shuffle_strip_solve, which slices each 480×480480×480 frame into 33–66 horizontal or vertical strips and applies an independent non-identity permutation per row in a [shuffled ∣ original] layout, so the model must learn to unshuffle by attending to the demo row; (i) shuffle_strips_apply, which uses the same strip permutation for both rows in a [original ∣ shuffled] layout, so the model must learn to apply the permutation demonstrated in the demo row; and (iv) shuffle_temporal, which permutes frames along the time axis with two hash-selected sub-variants: restore (independent shuffle per row) and apply (shared shuffle across rows). We note that shuffle_strip_solve/shuffle_strips_apply differ from the classical Jigsaw pretext task in that our strips are one-dimensional bands (horizontal or vertical only), whereas Jigsaw permutes a 2D tile grid, and our setup explicitly provides a paired demo–query row rather than a single-image classification target. C.3.2 Evaluation tasks. We use the 5555 scenes from the DL3DV evaluation split and apply 1515 video-processing tasks grouped into five categories. All evaluation tasks share the same 2×22×2 grid layout [degraded ∣ original] × 2 rows: each row pairs a degraded input (left) with the clean target (right), and the demo and query rows use independently sampled degradation parameters so that the model recovers the operation from the demo row rather than memorising a fixed setting. Inpainting (2): Tasks that mask out parts of the video and require the model to fill them in. (i) 3d_inpaint, 1010–2020 black space-time cubes of size 5050–150150 px and 1010–4040 frames placed at random (x,y,t)(x,y,t) offsets, producing time-varying occlusions; (i) 3d_outpaint, the inverse of 3d_inpaint: only 1515–3030 space-time cubes show the original RGB content while everything else is black, so the model must reconstruct the full video from sparse 3D “windows”. Deblurring (3): Tasks that degrade the input with a blur and require the model to recover sharp content. (i) deblur_spatial, a per-scene-randomised spatial blur drawn uniformly from Gaussian, box, median with kernel size 1111–2727 px; (i) deblur_motion, a directional motion blur using a linear streak kernel of size 1515–3333 px at a uniformly sampled angle in [0,180∘)[0,180 ); (i) deblur_temporal, a temporal blur produced by averaging each frame with its ±4± 4-frame neighbours (window size 99, edges truncated), simulating long-exposure or low-frame-rate capture. Temporal interpolation (2): Tasks that delete most frames and require the model to interpolate the missing ones. We sparsify the input by keeping exactly N randomly chosen frames intact and zeroing out the rest, with two difficulty levels controlled by N: (i) temporal_interp_easy (N=16N=16); (i) temporal_interp_hard (N=8N=8); The set of visible frame indices is sampled independently per row. Spatial jigsaw (2): Tasks that permute a 2×22×2 or 3×33×3 tile grid of each frame. (i) jigsaw_solve, where each row is shuffled with an independently sampled non-identity permutation in a [shuffled ∣ original] layout, so the model must invert the demonstrated permutation; (i) jigsaw_apply, where both rows share the same permutation in an [original ∣ shuffled] layout, so the model must replicate the demonstrated permutation. Unlike the strip-shuffle training tasks (which act on 11D bands), these jigsaw tasks act on a true 22D tile grid and serve as the canonical out-of-distribution counterpart. Photometric & symmetry (3): Three additional tasks covering low-level photometric correction and global symmetry operations. (i) low_light, a low-light enhancement task that gamma-darkens each frame with γ∈[0.15, 0.30]γ∈[0.15,\,0.30] (effective exponent 1/γ∈[3.3, 6.7]1/γ∈[3.3,\,6.7]) and adds Gaussian noise with σ∈[10, 30]σ∈[10,\,30]; (i) temporal_flip, which reverses the playback order of the input video, requiring the model to play the scene backwards; (i) horizontal_flip, which mirrors each frame left-to-right, requiring the model to produce the left-right reflected video. C.4 Camera Pose Metrics We follow the formulation in CamI2V [Zheng et al., 2024]. Given a generated video of S frames, we extract camera extrinsics using VGGT [Wang et al., 2025b] and compare against the ground-truth camera trajectory. Both predicted and ground-truth trajectories are first converted to camera-to-world (c2w) poses, made relative to the first frame (setting frame 0 as identity), and independently scale-normalized by dividing all translations by the distance to the furthest camera from the origin: ^i=imaxj‖j‖2,i=1,…,S−1, t_i= t_i _j\|t_j\|_2, i=1,…,S-1, (2) where it_i denotes the translation of the i-th frame relative to frame 0. This normalization is applied independently to both GT and predicted trajectories, making the metrics scale-invariant. Rotation Error (RotErr). The sum of geodesic (angular) distances between predicted and ground-truth rotations: RotErr=∑i=0S−1arccos(tr(iestigt⊤)−12),RotErr= _i=0^S-1 \! ( tr(R_i^estR_i^gt )-12 ), (3) where iest,igt∈SO(3)R_i^est,R_i^gt∈ SO(3) are the predicted and ground-truth rotation matrices at frame i, and the argument of arccos is clipped to [−1,1][-1,1]. Translation Error (TransErr). The sum of ℓ2 _2 distances between predicted and ground-truth translations after scale normalization: TransErr=∑i=0S−1‖^iest−^igt‖2.TransErr= _i=0^S-1 \| t_i^est- t_i^gt \|_2. (4) Camera Motion Error (CamErr). The sum of Frobenius norms of the 3×43× 4 pose matrix differences: CamErr=∑i=0S−1‖[iest∣^iest]−[igt∣^igt]‖F.CamErr= _i=0^S-1 \|[R_i^est t_i^est]-[R_i^gt t_i^gt] \|_F. (5) Camera_view_transfer. For the camera view task, we treat the input image (pose A) and the generated output image (target pose B) as a two-frame sequence (S=2S=2). Ground-truth extrinsics are obtained by running VGGT on the two real images from the demo pair. Predicted extrinsics are obtained by running VGGT on the query input image and the first frame of the generated output. We then apply the same RotErr, TransErr, and CamErr formulation with S=2S=2. Since frame 0 is canonicalized to identity, only the relative pose of the generated view contributes to the error. Camera_motion_transfer. For the camera motion task, the ground-truth is the full S=81S=81 frame trajectory from the reference video. Predicted extrinsics are extracted from the generated video using VGGT. Since all metrics are sums over S frames, values are not directly comparable between the two tasks. Appendix D Limitations As the first work to explore video analogies as a unified framework for diverse tasks, we acknowledge several current limitations. We thoroughly detail these in Section 4.3, which discusses test tasks our model struggles to perform (such as flipping and deblurring) , and Section 5, which characterizes bounds on generalization like task internalization. Beyond these specific failure modes, our framework opens several promising avenues for future work. First, incorporating text conditioning. In its current form, our model relies entirely on visual prompts; future work could investigate how integrating text-based task and video prompts might enhance the performance and generative quality of video analogies. Second, expanding task diversity. Despite our best efforts, the diversity of our training tasks remains bounded. As noted in Section 4 and Section 5, increasing task variety and training the model to generate a wider array of visual appearances are integral steps toward making the system more robust and generalizable. Finally, exploring diverse visual modalities. Future research could extend this approach to other complex data structures, such as spectrograms, sonograms, 3D point clouds, sinograms, or volumetric 3D images (e.g., CT and MRI scans).