Paper deep dive

Gen Through the Lens: Camera-Controllable Video Generation via Cross-Coordinate Conditioning Fusion

Shuai Li, Jinxu Xiang, Xinyuan Li, Hanlei Guo, Xiyang Tan, Alan Zhao, Xiaotong Zhao, Ran Zhang

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 66

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:We present Gen Through the Lens, a camera-controllable image-to-video generation framework that synthesizes videos from a given reference image while enabling realistic and physically consistent camera operations. Unlike previous models that treat camera motion as abstract conditioning or rely on a single coordinate system, our approach explicitly decomposes and fuses control signals across three coordinate spaces: the camera coordinate (modeled by intrinsic parameters), the world coordinate (represented by spatial geometry hints), and the object coordinate (described by 3D bounding boxes of target subjects). Through cross-coordinate conditioning fusion, these heterogeneous cues are unified to guide the generative process, enabling precise control of camera motion, focus, and framing while preserving temporal and spatial coherence. Extensive experiments demonstrate that our method produces controllable and visually consistent results across diverse camera operations, outperforming existing approaches in both realism and editability.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

63,776 characters extracted from source content.

Expand or collapse full text

Gen Through the Lens: Camera-Controllable Video Generation via Cross-Coordinate Conditioning Fusion Shuai Li*, Jinxu Xiang*, Xinyuan Li, Hanlei Guo, Xiyang Tan, Alan Zhao, Xiaotong Zhao, Ran Zhang† Affiliation: Tencent Video Solar System Figure 1. Overview of Gen Through the Lens, the first image-to-video generation framework that jointly enables controllable camera motion, object motion, and depth-of-field effects. Starting from a single input image, our model supports cinematic camera trajectories, zoom and focus transitions, and stable object dynamics in a physically consistent manner. The lower left shows a modeling software environment for editing camera trajectories, object motion, and depth-of-field information. The second column shows the three complementary conditioning signals, and the right side shows the generated video results, with the model’s key controllable capabilities summarized below. Abstract. We present Gen Through the Lens, a camera-controllable image-to-video generation framework that synthesizes videos from a given reference image while enabling realistic and physically consistent camera operations. Unlike previous models that treat camera motion as abstract conditioning or rely on a single coordinate system, our approach explicitly decomposes and fuses control signals across three coordinate spaces: the camera coordinate (modeled by intrinsic parameters), the world coordinate (represented by spatial geometry hints), and the object coordinate (described by 3D bounding boxes of target subjects). Through cross-coordinate conditioning fusion, these heterogeneous cues are unified to guide the generative process, enabling precise control of camera motion, focus, and framing while preserving temporal and spatial coherence. Extensive experiments demonstrate that our method produces controllable and visually consistent results across diverse camera operations, outperforming existing approaches in both realism and editability. 11footnotetext: Equal contribution. † Corresponding author. 1. Introduction Recent advances in video generation have achieved impressive visual realism, yet providing precise and physically meaningful control over camera behavior remains a fundamental challenge. Most existing generative models either treat camera motion as an implicit byproduct of appearance statistics or encode it using low-level signals such as optical flow or relative pose embeddings, making it difficult to specify complex camera operations such as metric-scale motion, depth-of-field variation, or coupled changes in camera intrinsics and extrinsics. A key difficulty lies in the heterogeneous nature of camera-related information, which simultaneously involves intrinsic parameters (e.g., focal length and distortion), extrinsic motion in 3D space, and interactions with scene geometry and object motion. Collapsing these factors into a single coordinate system or abstract latent codes obscures their physical meaning and limits controllability, particularly for advanced cinematic effects such as depth-of-field control or the Hitchcock zoom (Hitchcock, 1958). In this work, we present Gen Through the Lens (GTL), an image-to-video generation framework that enables explicit and physically grounded control over camera motion, object motion, and depth-of-field by decomposing and fusing camera-related signals across multiple coordinate spaces. Our approach introduces a cross-coordinate conditioning formulation that jointly models (i) the camera coordinate, capturing intrinsic parameters and lens effects, (i) the world coordinate, encoding metric-scale camera motion and spatial geometry, and (i) the object coordinate, representing object motion through 6D bounding boxes parameterized by 3D translation and 3D rotation. By preserving the physical meaning of each coordinate space and explicitly fusing them during generation, our framework enables precise and interpretable control over camera motion, focus, and framing. Building upon this formulation, we design three complementary conditioning signals corresponding to the three coordinate spaces: (1) a camera motion condition, which visualizes metric-scale camera extrinsics and intrinsics in the world coordinate space and enables explicit control over camera translation, rotation, and zoom; (2) an object motion condition, which represents foreground dynamics using 3D bounding boxes with 6D pose, thereby enabling disentangled control over object translation and rotation; and (3) a depth-of-field condition, which encodes focus depth and defocus behavior using layered depth cues, allowing explicit and time-varying control of bokeh effects. These conditions are geometrically consistent and mutually complementary, jointly enabling controllable video synthesis with coherent camera motion, stable object dynamics, and physically meaningful depth-of-field behavior. However, effectively leveraging physically grounded conditions requires accurate and metrically consistent camera supervision, which is difficult to obtain from unconstrained videos due to scale ambiguity, temporal instability, and intrinsic–extrinsic inconsistencies. To address this challenge, we introduce a metric-scale dataset labeling pipeline that combines relative camera pose estimation with monocular metric depth and robust scale recovery, constructing a unified metric camera space for reliable camera-controllable supervision. Extensive quantitative evaluations demonstrate that our method significantly improves camera controllability, geometric consistency, and motion stability compared to state-of-the-art approaches. In addition, qualitative results showcase compelling cinematic applications, including controllable depth-of-field effects and stable synthesis of the Hitchcock zoom. Together, these results highlight the importance of explicit, cross-coordinate camera modeling for controllable and physically consistent video generation. 2. Related Work 2.1. Camera-Controllable Video Generation Camera motion control in generative video has evolved from coarse motion cues to explicit trajectory conditioning. Early text-based approaches such as AnimateDiff (Guo et al., 2023) and MotionDirector (Zhao et al., 2024) typically fine-tune LoRA modules for a small set of motion patterns, providing convenient but imprecise control. Recent trajectory-guided methods instead condition video diffusion on camera parameters. MotionCtrl (Wang et al., 2024b) conditions generation on raw camera parameters repeated over the spatial grid, while CameraCtrl (He et al., 2024) and CamCo (Xu et al., 2024) encode camera parameters as pixel-wise Plücker ray embeddings to obtain more discriminative viewpoint control. Several works lift the reference image to 3D and render proxy views for conditioning, such as ViewCrafter (Yu et al., 2024) and I2VControl-Camera (Feng et al., 2024), improving spatial coherence under viewpoint changes. To better handle long-range motion and cross-frame geometry, CamI2V (Zheng et al., 2024) introduces epipolar attention mechanisms, and RealCam-I2V (Li et al., 2025a) further targets real-world usability by aligning trajectories to metric scale. Beyond pure conditioning, GEN3C (Ren et al., 2025) strengthens controllability by maintaining a 3D cache rendered along user-specified trajectories, which also improves temporal 3D consistency. Despite these advances, most trajectory-based controllers still rely on approximate geometry or relative-scale supervision and treat the camera as a simplified projection model, which can lead to scale drift, limited robustness under large rotations or occlusions, and weak support for optical phenomena beyond pose changes. 2.2. Object-Controllable Video Generation Object control in generative video has progressed from coarse region constraints to explicit trajectory-driven motion specification. Early approaches primarily rely on spatial conditions to anchor object placement but provide limited control over object dynamics. Recent works instead adopt drag-style interaction, where sparse point trajectories convey user intent and are injected into diffusion models through explicit motion representations. DragNUWA (Yin et al., 2023) introduces joint text-image-trajectory conditioning for open-domain object manipulation, while ImageConductor (Li et al., 2025b) decouples camera and object motions via separate LoRA branches for more precise interactive control. Many methods use optical flow as an intermediate representation: sparse trajectories are converted to sparse or dense flow to guide temporal attention or feature warping, as in Motion-I2V (Shi et al., 2024) and MOFA-Video (Niu et al., 2024). Beyond flow, Tora (Zhang et al., 2025) encodes trajectories into a trajectory map and integrates them into a scalable diffusion-transformer pipeline via a trajectory extractor and fusion module, and DragAnything (Wu et al., 2024) uses an entity representation with mask-based feature reorganization to improve object-level coherence under dragging. Despite improved controllability, these methods remain sensitive to trajectory placement and depth or occlusion ambiguities, can introduce local distortions under large shape variations, and often suffer from control conflicts when object motion and global camera motion are specified simultaneously. 2.3. Collaborative Motion Control Collaborative motion control targets video generation with simultaneous camera motion and local object dynamics, where naive 2D motion controls often lead to local conflicts under complex camera movements. Early approaches unify camera and object motion in a single 2D representation. AnimateAnything (Lei et al., 2024) converts both motions into a unified optical-flow field, but entangles global camera-induced displacement with object-centric motion, limiting geometric interpretability. Subsequent works adopt point-based motion control derived from 2D perception signals. MotionPrompting (Geng et al., 2024) and I2VControl-Camera (Feng et al., 2024) leverage dense point embeddings to guide motion generation, but collaborative control remains rooted in 2D cues and is typically specified at region or segmentation-mask granularity, restricting fine-grained camera–object interaction modeling. Perception-as-Control (Chen et al., 2025) introduces a 3D-aware motion representation with separate encoders for camera and object control to reduce local conflicts. Despite improved motion harmony, collaborative approaches still depend on upstream perception quality, simplified 3D surrogates may miss motion factors, and overall controllability remains constrained by the underlying generator. 3. Method In this section, we present a metric-scale supervision and conditioning pipeline that first recovers semantic, camera, object, and depth-related supervision from real-world videos, and then constructs geometry-aware conditioning signals for controllable video generation. Together, these components enable disentangled yet coherent control over camera motion, object motion, depth-of-field, and zoom. The overall data processing workflow is illustrated in Fig. 2. Detailed implementation of each component is provided in Appendix Sec. A. Figure 2. Overview of the data processing pipeline. Recovered metric-scale camera parameters and human-centered 3D geometry are converted into geometry-aware conditions for camera motion, object motion, and depth-of-field control. 3.1. Label Prompt We use InternVL2.5 (Chen et al., 2024) to generate dense textual descriptions for each video, which serve as semantic supervision during training. To avoid information leakage and ensure disentanglement between camera control and content description, we explicitly constrain the prompt to exclude camera-related information, such as camera motion, viewpoint, and shooting angles. The prompt is designed to encourage temporally coherent, detailed, and factually accurate descriptions that focus on scene content and object dynamics. 3.2. Camera Modeling Metric-scale camera extrinsics and accurate camera intrinsics are essential for injecting reliable camera motion condition into camera-controllable generative models. Wan2.1 (Wan et al., 2025) is trained on a RealEstate-style dataset (Zhou et al., 2018) that effectively normalizes camera trajectories under a fixed foreground depth assumption; when this assumption is violated at inference, the generated motion exhibits noticeable scale inconsistencies. To address this issue, we build a labeling pipeline that recovers metric-scale camera extrinsics and intrinsics for each video, forming a unified metric camera space. Accurate camera-controllable video generation requires camera extrinsics and intrinsics that are both geometrically reliable and consistent under metric scale. However, existing methods typically specialize in only part of this problem, such as temporally stable camera motion, per-frame metric depth, or accurate intrinsic estimation. We therefore combine three complementary approaches—Pi3 (Wang et al., 2025e), Moge2 (Wang et al., 2025c), and MapAnything (Keetha et al., 2025)—to recover metric-consistent camera parameters. Specifically, Pi3 provides temporally stable camera extrinsics up to an unknown global scale, Moge2 predicts per-frame metric depth, and MapAnything provides reliable camera intrinsics. We first estimate a robust global scale factor from Pi3 and Moge2 using pairwise 3D distance ratios across the video, which resolves the scale ambiguity in camera extrinsics. We then align the camera intrinsics predicted by MapAnything to the same metric space using a consistent scaling strategy, ensuring geometrically accurate projection across frames. The resulting metric-scale camera parameters are used as supervision for training. Quantitative evaluation of the recovered camera parameters is reported in Table 1. Based on the recovered parameters, we construct a camera motion condition using an intermediate world-coordinate representation rather than raw Plücker embeddings (Plücker, 1865) or per-frame point clouds. We normalize camera poses to the first frame and represent camera trajectories using a structured grid in world coordinates, which is rendered with the recovered intrinsics to ensure geometric consistency across frames. This representation encodes camera motion in a structured and controllable form. Figure 3. Overview of the proposed framework. The model jointly conditions video generation on camera motion, object motion, and depth-of-field signals across camera, world, and object coordinate spaces. 3.3. 3D Bounding Box Modeling Controlling object motion solely from 2D image coordinates inevitably discards spatial information, including depth ordering under occlusion and object orientation. This ambiguity motivates the need for an explicit object motion condition. Following Cinemaster (Wang et al., 2025b), we represent object motion using 3D bounding boxes in full 6D, including 3D translation and 3D rotation. We focus on human subjects, as their geometry and motion can be reliably reconstructed from monocular videos using off-the-shelf Human Mesh Recovery (HMR) methods, enabling scalable automatic annotation. We employ TokenHMR (Dwivedi et al., 2024) for identity-consistent tracking, and fit bounding boxes only to the rigid torso vertices of the recovered SMPL mesh (Loper et al., 2023). This design mitigates large shape variations caused by articulated limbs or head motion (e.g., bending or sitting), resulting in more stable bounding boxes and temporally consistent 6D object motion supervision. We then convert the recovered 3D boxes into an object motion condition that provides explicit local spatial cues. Complementing the camera motion condition, which defines the global metric space, the 3D bounding box condition indicates precise screen-space occupancy, relative depth ordering, and orientation. The condition is rendered as a projected 3D bounding box with depth-aware filling and consistent edge coloring, enabling the model to infer object rotation and placement without explicit angle representations. As shown in Fig. 5, although the model is trained only on humans, the learned object motion condition generalizes well to generic non-human objects. Method Cam Trans. Err. ↓ Cam Rot. Err. (°) ↓ VGGT 41.25 20.34 Pi3 40.36 17.21 MapAnything 12.39 17.43 Ours 11.74 17.11 Table 1. Metric-scale camera pose accuracy on the MultiCamVideo Dataset with ground-truth camera annotations. Methods that estimate camera poses only up to scale (VGGT and Pi3) exhibit large errors on sequences with significant camera motion. Our method recovers metrically consistent camera parameters and achieves lower errors, validating the effectiveness of the proposed pipeline. 3.4. Depth-of-Field Modeling To model depth-of-field effects, we construct depth-of-field condition signals using recovered metric depth and a physically motivated bokeh operator following AkiRa (Wang et al., 2025d), where blur is determined by the deviation between pixel depth and the focal plane. We use the same operator for both bokeh-augmented data generation and condition construction, ensuring consistency between training signals and supervision. The recovered metric camera space enables flexible focus modeling. By specifying time-varying focus depths, we can synthesize diverse behaviors such as smooth focus transitions and subject tracking. To improve temporal stability, we apply a boundary-aware smoothing step to mitigate artifacts near large depth discontinuities. For conditioning, we adopt a layered representation instead of directly encoding pixel-wise focus distance. We approximate unknown background depth using a small set of canonical depth planes and combine them with foreground depth derived from 3D bounding boxes. The resulting representation captures the interaction between focus distance and scene depth while avoiding reliance on accurate full-scene depth estimation. 3.5. Zoom Video Augmentation In practical cinematography, zoom is an essential component of camera behavior, yet recovering time-varying focal lengths directly from unconstrained videos is highly unreliable, and such sequences are difficult to curate at scale. We therefore synthesize zoom effects through data augmentation. Specifically, we model focal-length changes by applying a temporal center crop with randomly sampled crop factors and zoom intervals, and resizing the result back to the native resolution to simulate push-in and pull-out shots while keeping the final image resolution fixed. Once the zoom parameters are sampled for a video, the same transformation is consistently applied to all conditioning streams—including the camera motion condition video, the object motion condition video, and the depth-of-field condition video—to preserve pixel-wise alignment. 4. Experiments Method FID ↓ FVD ↓ Qwen VQA Score ↑ Cam Trans. Err. ↓ Cam Rot. Err. (°) ↓ mIoU ↑ BBox Center Err. (%) ↓ BBox Rot. Err. (°) ↓ Traj. Err. (%) ↓ PAC 70.17 172.51 0.56 18.15 8.77 0.52 6.37 41.83 8.68 RC-I2V 55.12 98.76 0.70 9.86 4.63 0.45∗0.45 7.42∗7.42 47.34∗47.34 7.29 GEN3C 64.24 114.33 0.62 11.38 5.02 0.47∗0.47 8.24∗8.24 46.22∗46.22 8.39 GTL 14B 53.71 86.54 0.81 8.59 4.89 0.61 4.40 21.73 6.37 Table 2. Quantitative comparison with state-of-the-art methods. Lower is better (↓ ) for error-based metrics and latent space distance, while higher is better (↑ ) for overlap and semantic metrics. Screen-space errors are reported as percentages of the image diagonal. Metrics marked with ∗ should be interpreted with caution, as RealCam-I2V and GEN3C do not support explicit control of object motion or target object positioning, which may bias object-related metrics toward worse values. 4.1. Training Paradigm We curate a dataset with significant camera and object dynamics from DynPose-100K (Rockwell et al., 2025), MultiCamVideo (Bai et al., 2025), BEDLAM2.0 (Tesch et al., 2025), and internal data, totaling 110k samples with camera trajectories and 3D bounding box annotations. We further construct a zoom dataset (8k) and a bokeh dataset (15k). All training videos are unified to 81 frames at 480p resolution. We build GTL 14B on Wan2.1-14B (Wan et al., 2025), freezing the encoder, decoder, and DiT, and training only the Context Adapter. Training follows a three-stage curriculum from camera to object and finally to bokeh control. We initialize from partial VACE DiT weights (Jiang et al., 2025). Stage 1 trains camera control using trajectory annotations; Stage 2 adds 3D bounding boxes and zoom data for object motion and zoom; Stage 3 introduces bokeh data for joint control. The three stages run for 11k, 13k, and 7k iterations with batch size 8 and learning rate 5×10−55× 10^-5, totaling approximately 288 GPU hours on 8 NVIDIA H800 GPUs. An overview of the model and training pipeline is shown in Fig. 3. 4.2. Evaluation Metrics We present a comprehensive evaluation of our method across multiple complementary aspects, including camera controllability, geometric consistency, motion fidelity, and visual quality. For metrics that rely on extracted signals, we apply the same extraction pipeline to both the real input videos and the generated videos, and compare the recovered features, using explicit ground truth when available and real-video signals as pseudo-ground-truth otherwise. Brief metric descriptions are given below, with full details provided in Appendix B. Camera motion accuracy. Camera controllability is evaluated using two metrics: (i) camera translation error, which quantifies relative deviation of the camera trajectory in 3D space; and (i) camera rotation error, which measures orientation accuracy using geodesic distance on SO⁡(3)SO(3). All camera trajectories are normalized to the first frame to ensure a consistent reference across sequences. Object-level geometric stability. To assess foreground consistency under camera motion, we report the mean Intersection-over-Union (mIoU) measured by SAM2 (Ravi et al., 2024) between predicted and reference foreground masks. In addition, we evaluate 3D bounding box pose accuracy using both screen-space center deviation and 3D rotational error, which measure how accurately the generated videos follow the object controls. Trajectory-based motion accuracy. We assess motion fidelity using a trajectory consistency error computed from point tracks extracted by CoTracker (Karaev et al., 2024). This metric measures the alignment between induced 2D motion and the reference video by comparing point trajectories across frames, and reflects the combined effects of camera and object motion. When evaluated over longer trajectories, it becomes sensitive to accumulated drift, serving as an indicator of long-term motion stability. Visual quality and semantic alignment. We report Fréchet Inception Distance (FID) (Heusel et al., 2017) and Fréchet Video Distance (FVD) (Unterthiner et al., 2019) to evaluate perceptual realism and temporal coherence. To assess semantic alignment with text prompts, we additionally report the Qwen VQA Score (Wang et al., 2024a), which measures the consistency between generated videos and input prompts using a vision–language model. 4.3. Evaluation Datasets We evaluate our method on a held-out test set consisting of 1k video clips drawn from DynPose-100K, the MultiCamVideo Dataset, BEDLAM2.0, and our proprietary internal data. Together, these datasets cover a broad spectrum of foreground occupancy ratios, camera motion complexity, and data sources, including real-world daily-life recordings, cinematic videos, and synthesized scene videos. All evaluation videos are disjoint from the training set. Videos are processed using the same metric-scale annotation pipeline described in Sec. 3, producing camera parameters, foreground masks, and 3D bounding box poses. This unified annotation enables consistent and fair comparison across different methods under identical evaluation conditions. Unless otherwise specified, all quantitative results reported later are computed on this evaluation set. 4.4. Camera Pose Estimation Accuracy Subsequent camera-related evaluations rely on metric-scale camera poses extracted from generated videos. To ensure that reported errors reflect generation quality rather than estimation inaccuracies, we first assess the accuracy of our camera pose extraction pipeline. We compare our method with representative approaches, including VGGT (Wang et al., 2025a), Pi3 (Wang et al., 2025e), and MapAnything (Keetha et al., 2025), using camera translation and rotation errors. As shown in Table 1, our method achieves improved metric-scale accuracy, validating the effectiveness of the proposed pipeline for reliable supervision. We further provide qualitative validation by visualizing the extracted camera motion condition alongside the input videos. As illustrated in Fig. 4, the recovered motion faithfully reflects scene structure and camera dynamics, with consistent changes in position and orientation, making it directly interpretable. An ablation study comparing training with camera parameters from MapAnything and our extraction pipeline is provided in Appendix Table 5. Figure 4. Qualitative comparison of camera controllability with state-of-the-art methods. Gen3C and RealCam-I2V use identical camera motion extracted from the original video, with the first frame as input. CamCloneMaster and Seedance 2.0 use the original video as the camera reference. Our Camera condition encodes recovered extrinsics and intrinsics. For clarity, text prompts are omitted in this figure and provided in the Appendix Sec. E. Figure 5. Qualitative comparison of object motion control. We compare our method with Perception-as-Control under aligned camera conditions and comparable object motion controls, focusing on the controllability and stability of foreground object motion. For clarity, text prompts are omitted in this figure and provided in the Appendix Sec. E. Figure 6. Disentanglement between camera and object motion. Keeping the object fixed in screen space under camera motion requires the object to move together with the camera in 3D. The prompts used here avoid specifying object motion and are provided in Appendix Sec. E. 4.5. Comparison with State-of-the-Art Having established the accuracy of our camera pose extraction pipeline, we evaluate the proposed framework GTL 14B against representative state-of-the-art methods, including Perception-as-Control (PAC) (Chen et al., 2025), RealCam-I2V (Li et al., 2025a), and GEN3C (Ren et al., 2025). Quantitative comparisons across camera motion accuracy, object-level stability, trajectory consistency, and visual quality are reported in Table 2. The inference hyperparameters for GTL 14B are provided in Appendix Sec. C. As summarized in Table 2, GTL 14B achieves lower camera translation and rotation errors, indicating more accurate camera motion recovery under metric scale. With explicit 3D bounding box conditioning, the model better regulates object translation and rotation, reducing object-centric errors. Joint camera-object control further improves motion stability and trajectory consistency, suggesting that explicit geometry-aware conditions provide stronger motion supervision than implicit image-space guidance alone. Perceptual and semantic metrics further show that these geometric improvements do not come at the cost of visual quality. Recent works such as CamCloneMaster (Luo et al., 2025) require a reference video to specify camera motion. In our setting, this would require using the ground-truth video as input, which introduces an unfair advantage. We therefore exclude it from quantitative evaluation and report qualitative results in Fig. 4. We also include results from Seedance 2.0 (Seedance et al., 2026). Due to its less explicit separation between reference and target generation, it often transfers appearance details from the reference video (e.g., the knot on the flower), leading to inconsistencies with the intended control signal. The qualitative results in the following figures corroborate these findings. Fig. 4 shows that, when using only camera motion conditions, our method produces stable and geometrically consistent camera behavior under challenging camera trajectories. Fig. 5 further demonstrates that jointly applying camera and object motion conditions yields consistent camera motion and well-controlled object dynamics. Despite being trained primarily on human-centered motion supervision, the learned object motion condition generalizes effectively to non-human objects and complex scenes. Additional qualitative results are shown in Fig. 9. To further validate disentanglement between camera and object motion, we visualize a controlled scenario in Fig. 6. Keeping the object fixed in screen space under camera motion requires the object to move together with the camera in 3D. This demonstrates that our model disentangles apparent 2D motion from physical 3D object motion, enabling independent control of camera behavior and object dynamics. 4.6. Ablation Study To isolate the effect of individual design choices, we conduct ablation studies using a smaller Wan2.1-1.3B model trained on the same dataset for 30K steps under identical settings. Although the 1.3B model exhibits slightly lower visual fidelity than the 14B model, it provides a reliable and efficient proxy for comparing relative performance across different conditioning designs. The ablation studies focus on relative metric differences rather than absolute generation quality, which are largely unaffected by the backbone scale. Additional ablation studies are provided in Appendix Sec. D. Effect of Camera Condition Representation. We study the impact of different camera condition representations on camera controllability. As an alternative representation, we consider a Plücker ray map (Plücker, 1865), where camera motion is encoded using dense per-pixel rays parameterized by direction and moment. All variants use identical camera extrinsics and intrinsics as input, differing only in the representation of the camera condition. We evaluate controllability using camera translation and rotation errors, as reported in Table 3. The results show that our geometry-aware cubic grid representation provides more stable and accurate camera control than ray-based representations. Effect of 3D Bounding Box Conditioning. We further analyze the role of explicit 3D bounding box conditioning in object motion control, as shown in Table 4. Besides our proposed rotation-aware 3D bounding box representation, we consider a simplified baseline using only binary foreground masks as object-level conditions. The mask-based representation provides only 2D spatial extent and lacks explicit depth ordering and rotational cues. Both variants are trained with identical camera motion and depth-of-field conditions. We evaluate object controllability using bounding box center error, bounding box rotation error, and trajectory consistency measured by CoTracker. Results demonstrate that explicit 3D bounding box conditioning significantly improves rotational stability and long-term motion consistency. Camera Condition Cam Trans. Err. ↓ Cam Rot. Err. (°) ↓ Ray Map (No Grid) 19.26 8.98 Color Cubic Grid (GTL 1.3B) 10.15 5.07 Table 3. Ablation studies on geometry-aware conditioning. Comparison of camera condition representations. Object Condition BBox Center Err. (%) ↓ BBox Rot. Err. (°) ↓ Traj. Err. (%) ↓ 2D Mask 5.94 45.15 8.37 3D BBox (GTL 1.3B) 6.13 27.96 7.03 Table 4. Ablation studies on geometry-aware conditioning. Comparison of object motion conditioning. Figure 7. Visualization of depth-of-field conditions and generated results. Our representation enables controllable bokeh strength, time-varying focus shifts, and stable depth-of-field under camera and object motion. Note that focus cues are encoded through subtle pixel-level variations, which may not be visually salient. The corresponding text prompts are provided in Appendix Sec. E. 5. Applications Our geometry-aware conditioning enables expressive cinematic effects by explicitly disentangling camera extrinsics, intrinsics, and depth-of-field. We present two representative applications—controllable bokeh and the Hitchcock zoom—to illustrate its practical advantages. 5.1. Controllable Depth-of-Field and Bokeh Our framework enables explicit and physically meaningful control over depth-of-field effects in generated videos. By leveraging metric depth information recovered in the metric camera space, we can synthesize a wide range of bokeh behaviors with a time-varying focus depth. In particular, the focal plane can be dynamically adjusted over time, allowing smooth focus transitions as well as focus-follow effects that track a moving subject. Figure 7 shows that bokeh in our model is driven by the explicit depth-of-field condition, where both the relative depth to the focal plane and the bokeh strength are integrated into the conditioning video to drive the resulting blur. Because the focus depth is specified as a continuous, time-varying signal, our method naturally supports zoom-dependent and temporally varying focus behaviors commonly observed in real cinematography, as illustrated by the second example. 5.2. Hitchcock Zoom The Hitchcock zoom (Hitchcock, 1958) is a classic cinematic effect in which the camera translates in space while adjusting focal length in the opposite direction, preserving the apparent size of the foreground subject while dramatically altering background perspective. This coupling of camera translation and zoom poses a significant challenge for generative models, as it requires accurate disentanglement and coordinated control of camera extrinsics and intrinsics. As illustrated in Fig. 8, our framework explicitly encodes camera translation and focal length variation in the camera condition. Consequently, even when forward camera motion and inverse zoom produce visually similar image content, they correspond to distinct patterns in our camera condition, enabling stable and faithful synthesis of the Hitchcock zoom. Figure 8. Hitchcock zoom effect synthesized by our model. In this example, the camera moves backward by 3 meters while the zoom is adjusted accordingly to approximately preserve the projected size of the subject in the image, as indicated by the object motion condition video. The corresponding text prompts are provided in Appendix Sec. E. 6. Conclusion In this paper, we present Gen Through the Lens, a camera-controllable image-to-video generation framework that enables explicit and physically grounded control over camera motion, object motion, and depth-of-field effects from a single reference image. By decomposing camera-related signals into camera, world, and object coordinate spaces and unifying them through cross-coordinate conditioning fusion, our approach preserves the physical meaning of each control signal while enabling coherent, disentangled, and cinematographically meaningful video generation. Extensive experiments demonstrate consistent improvements over prior methods in camera accuracy, motion stability, and editability, without compromising visual quality. Overall, this work highlights the importance of explicit multi-coordinate camera modeling for controllable and physically meaningful video synthesis, and provides a flexible foundation for camera-aware generative tools in cinematic content creation and film production. Figure 9. Additional qualitative results corresponding to the experiments. Text prompts are omitted in this figure and provided in the Appendix Sec. E. References Bai et al. (2025) J. Bai, M. Xia, X. Fu, X. Wang, L. Mu, J. Cao, Z. Liu, H. Hu, X. Bai, P. Wan, et al. ReCamMaster: camera-controlled generative rendering from a single video. arXiv preprint arXiv:2503.11647. Cited by: §4.1. Chen et al. (2025) Y. Chen, Y. Men, Y. Yao, M. Cui, and L. Bo Perception-as-control: fine-grained controllable image animation with 3d-aware motion representation. arXiv preprint arXiv:2501.05020. Cited by: §2.3, §4.5. Chen et al. (2024) Z. Chen, W. Wang, Y. Cao, Y. Liu, Z. Gao, E. Cui, J. Zhu, S. Ye, H. Tian, Z. Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271. Cited by: §3.1. Dwivedi et al. (2024) S. K. Dwivedi, Y. Sun, P. Patel, Y. Feng, and M. J. Black TokenHMR: advancing human mesh recovery with a tokenized pose representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 1323–1333. Cited by: §A.3, §3.3. Feng et al. (2024) W. Feng, J. Liu, P. Tu, T. Qi, M. Sun, T. Ma, S. Zhao, S. Zhou, and Q. He I2VControl-camera: precise video camera control with adjustable motion strength. arXiv preprint arXiv:2411.06525. Cited by: §2.1, §2.3. Geng et al. (2024) D. Geng, C. Herrmann, J. Hur, F. Cole, S. Zhang, T. Pfaff, T. Lopez-Guevara, C. Doersch, Y. Aytar, M. Rubinstein, C. Sun, O. Wang, A. Owens, and D. Sun Motion prompting: controlling video generation with motion trajectories. arXiv preprint arXiv:2412.02700. Cited by: §2.3. Guo et al. (2023) Y. Guo, C. Yang, A. Rao, Z. Liang, Y. Wang, Y. Qiao, M. Agrawala, D. Lin, and B. Dai AnimateDiff: animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725. Cited by: §2.1. He et al. (2024) H. He, Y. Xu, Y. Guo, G. Wetzstein, B. Dai, H. Li, and C. Yang CameraCtrl: enabling camera control for text-to-video generation. arXiv preprint arXiv:2404.02101. Cited by: §2.1. Heusel et al. (2017) M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter GANs trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30. Cited by: §4.2. Hitchcock (1958) A. Hitchcock Vertigo. Note: Paramount Pictures Cited by: §1, §5.2. Jiang et al. (2025) Z. Jiang, Z. Han, C. Mao, J. Zhang, Y. Pan, and Y. Liu VACE: all-in-one video creation and editing. External Links: 2503.07598, Link Cited by: §4.1. Karaev et al. (2024) N. Karaev, I. Rocco, B. Graham, N. Neverova, A. Vedaldi, and C. Rupprecht CoTracker: it is better to track together. In European conference on computer vision, p. 18–35. Cited by: §4.2. Keetha et al. (2025) N. Keetha, N. Müller, J. Schönberger, L. Porzi, Y. Zhang, T. Fischer, A. Knapitsch, D. Zauss, E. Weber, N. Antunes, et al. MapAnything: universal feed-forward metric 3d reconstruction. arXiv preprint arXiv:2509.13414. Cited by: §A.2, §3.2, §4.4. Lei et al. (2024) G. Lei, C. Wang, H. Li, R. Zhang, Y. Wang, and W. Xu AnimateAnything: consistent and controllable animation for video generation. arXiv preprint arXiv:2411.10836. Cited by: §2.3. Li et al. (2025a) T. Li, G. Zheng, R. Jiang, S. Zhan, T. Wu, Y. Lu, Y. Lin, C. Deng, Y. Xiong, M. Chen, et al. RealCam-i2v: real-world image-to-video generation with interactive complex camera control. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 28785–28796. Cited by: §2.1, §4.5. Li et al. (2025b) Y. Li, X. Wang, Z. Zhang, Z. Wang, Z. Yuan, L. Xie, Y. Shan, and Y. Zou Image conductor: precision control for interactive video synthesis. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 5031–5038. Cited by: §2.2. Loper et al. (2023) M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black SMPL: a skinned multi-person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, p. 851–866. Cited by: §A.3, §3.3. Luo et al. (2025) Y. Luo, X. Shi, J. Bai, M. Xia, T. Xue, X. Wang, P. Wan, D. Zhang, and K. Gai CamCloneMaster: enabling reference-based camera control for video generation. In Proceedings of the SIGGRAPH Asia 2025 Conference Papers, p. 1–10. Cited by: §4.5. Müller et al. (2022) T. Müller, A. Evans, C. Schied, and A. Keller Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph. 41 (4), p. 102:1–102:15. External Links: Link, Document Cited by: §A.2. Niu et al. (2024) M. Niu, X. Cun, X. Wang, Y. Zhang, Y. Shan, and Y. Zheng MOFA-video: controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model. In European Conference on Computer Vision, p. 111–128. Cited by: §2.2. Plücker (1865) J. Plücker XVII. on a new geometry of space. Philosophical Transactions of the Royal Society of London (155), p. 725–791. Cited by: §3.2, §4.6. Ravi et al. (2024) N. Ravi, V. Gabeur, Y. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, et al. SAM 2: segment anything in images and videos. arXiv preprint arXiv:2408.00714. Cited by: §4.2. Ren et al. (2025) X. Ren, T. Shen, J. Huang, H. Ling, Y. Lu, M. Nimier-David, T. Müller, A. Keller, S. Fidler, and J. Gao GEN3C: 3d-informed world-consistent video generation with precise camera control. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 6121–6132. Cited by: §2.1, §4.5. Rockwell et al. (2025) C. Rockwell, J. Tung, T. Lin, M. Liu, D. F. Fouhey, and C. Lin Dynamic camera poses and where to find them. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 12444–12455. Cited by: §4.1. Seedance et al. (2026) T. Seedance, D. Chen, L. Chen, X. Chen, Y. Chen, Z. Chen, Z. Chen, F. Cheng, T. Cheng, Y. Cheng, et al. Seedance 2.0: advancing video generation for world complexity. arXiv preprint arXiv:2604.14148. Cited by: §4.5. Shi et al. (2024) X. Shi, Z. Huang, F. Wang, W. Bian, D. Li, Y. Zhang, M. Zhang, K. C. Cheung, S. See, H. Qin, et al. Motion-i2v: consistent and controllable image-to-video generation with explicit motion modeling. In ACM SIGGRAPH 2024 Conference Papers, p. 1–11. Cited by: §2.2. Tesch et al. (2025) J. Tesch, G. Becherini, P. Achar, A. Yiannakidis, M. Kocabas, P. Patel, and M. J. Black BEDLAM2.0: synthetic humans and cameras in motion. arXiv preprint arXiv:2511.14394. Cited by: §4.1. Unterthiner et al. (2019) T. Unterthiner, S. Van Steenkiste, K. Kurach, R. Marinier, M. Michalski, and S. Gelly FVD: a new metric for video generation. In ICLR Workshop on Deep Generative Models for Highly Structured Data, Cited by: §4.2. Wan et al. (2025) T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, et al. Wan: open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. Cited by: §3.2, §4.1. Wang et al. (2025a) J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny VGGT: visual geometry grounded transformer. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 5294–5306. Cited by: §4.4. Wang et al. (2024a) P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, et al. Qwen2-vl: enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191. Cited by: §4.2. Wang et al. (2025b) Q. Wang, Y. Luo, X. Shi, X. Jia, H. Lu, T. Xue, X. Wang, P. Wan, D. Zhang, and K. Gai CineMaster: a 3d-aware and controllable framework for cinematic text-to-video generation. In ACM SIGGRAPH 2025 Conference Papers, p. 1–10. Cited by: §3.3. Wang et al. (2025c) R. Wang, S. Xu, Y. Dong, Y. Deng, J. Xiang, Z. Lv, G. Sun, X. Tong, and J. Yang MoGe-2: accurate monocular geometry with metric scale and sharp details. arXiv preprint arXiv:2507.02546. Cited by: §A.2, §3.2. Wang et al. (2025d) X. Wang, R. Courant, M. Christie, and V. Kalogeiton AKiRa: augmentation kit on rays for optical video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 2609–2619. Cited by: §3.4. Wang et al. (2025e) Y. Wang, J. Zhou, H. Zhu, W. Chang, Y. Zhou, Z. Li, J. Chen, J. Pang, C. Shen, and T. He Pi3: permutation-equivariant visual geometry learning. arXiv preprint arXiv:2507.13347. Cited by: §A.2, §3.2, §4.4. Wang et al. (2024b) Z. Wang, Z. Yuan, X. Wang, Y. Li, T. Chen, M. Xia, P. Luo, and Y. Shan MotionCtrl: a unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, p. 1–11. Cited by: §2.1. Wu et al. (2024) W. Wu, Z. Li, Y. Gu, R. Zhao, Y. He, D. J. Zhang, M. Z. Shou, Y. Li, T. Gao, and D. Zhang DragAnything: motion control for anything using entity representation. External Links: 2403.07420 Cited by: §2.2. Xu et al. (2024) D. Xu, W. Nie, C. Liu, S. Liu, J. Kautz, Z. Wang, and A. Vahdat CamCo: camera-controllable 3d-consistent image-to-video generation. arXiv preprint arXiv:2406.02509. Cited by: §2.1. Yin et al. (2023) S. Yin, C. Wu, J. Liang, J. Shi, H. Li, G. Ming, and N. Duan DragNUWA: fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308.08089. Cited by: §2.2. Yu et al. (2024) W. Yu, J. Xing, L. Yuan, W. Hu, X. Li, Z. Huang, X. Gao, T. Wong, Y. Shan, and Y. Tian ViewCrafter: taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409.02048. Cited by: §2.1. Zhang et al. (2025) Z. Zhang, J. Liao, M. Li, Z. Dai, B. Qiu, S. Zhu, L. Qin, and W. Wang Tora: trajectory-oriented diffusion transformer for video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 2063–2073. Cited by: §2.2. Zhao et al. (2024) R. Zhao, Y. Gu, J. Z. Wu, D. J. Zhang, J. Liu, W. Wu, J. Keppo, and M. Z. Shou MotionDirector: motion customization of text-to-video diffusion models. In European Conference on Computer Vision, p. 273–290. Cited by: §2.1. Zhao et al. (2023) W. Zhao, L. Bai, Y. Rao, J. Zhou, and J. Lu UniPC: a unified predictor-corrector framework for fast sampling of diffusion models. External Links: 2302.04867, Link Cited by: Appendix C. Zheng et al. (2024) G. Zheng, T. Li, R. Jiang, Y. Lu, T. Wu, and X. Li CamI2V: camera-controlled image-to-video diffusion model. arXiv preprint arXiv:2410.15957. Cited by: §2.1. Zhou et al. (2018) T. Zhou, R. Tucker, J. Flynn, G. Fyffe, and N. Snavely Stereo magnification: learning view synthesis using multiplane images. arXiv preprint arXiv:1805.09817. Cited by: §3.2. Appendix A Additional Details on Geometry-Aware Conditioning This appendix provides implementation details for the label prompt, camera motion condition, object motion condition, depth-of-field condition, and zoom augmentation. A.1. Label Prompt Details To ensure disentaglement between camera control and content description, we explicitly constrain the captions to exclude any camera related information. We use the following prompt for dense video caption generation with InternVL2.5: You are a professional video description master, your task is to carefully observe the input video, and give a detailed and dense description of the video, please follow the instructions below: 1. It is required to pay attention to the timing of the video, reflecting the development process of the events in the video over time and the details. 2. The output content should not include camera-related information, such as camera shooting angles and camera movements. 3. Require the output to capture as much video content as possible, including details, considering both completeness and specificity. It should provide comprehensive information that is tied to specific elements of the video. 4. The output description is required to be accurate and coherent, without misinformation and misunderstanding. The output must be factually accurate, coherent in content description, and consistent with the video content. 5. Do not give the info about camera such as camera shooting Angle and camera Movement. The Output should no more than 225 words. A.2. Camera Modeling and Camera Motion Condition Metric-Scale Camera Recovery. We combine Pi3 (Wang et al., 2025e) and Moge2 (Wang et al., 2025c) to recover metric-scale camera motion. Pi3 provides temporally stable camera poses and scene structure up to an unknown global scale, while Moge2 predicts per-frame metric depth with limited temporal consistency. To estimate the global scale, we randomly sample point pairs (a,b)(x_a,x_b) from reliable image regions, back-project them into 3D using Pi3 and Moge2, and compute distance ratios dkMoge/dkPi3d^Moge_k/d^Pi3_k. We discard the top and bottom 20%20\% of ratios both within each frame and globally across the video, and average the remaining values to obtain a single video-level scale factor s. After scaling Pi3 translations and associated 3D structure, we apply a Gaussian temporal filter with a window size of 0.50.5 seconds independently to translation and rotation sequences. For camera intrinsics, we use MapAnything (Keetha et al., 2025) as an initial estimator. Its intrinsic scale is aligned to the recovered metric camera space using the same pairwise-distance scaling strategy, and further adjusted to account for rendering resolution. Camera Motion Condition. We discretize the surrounding 3D space using a regular grid of axis-aligned cubes. Cubes are placed at uniform intervals of 0.80.8 meters along the x, y, and z axes in world coordinates, and each cube has an edge length of 0.150.15 meters. Only cubes within a distance range of [1.5,10][1.5,10] meters from the camera center are considered visible. When more than 300 cubes are visible, we render only the 300 closest cubes. All camera poses are normalized to the first frame. The visible cubes are rendered using the recovered camera intrinsics and extrinsics. To encode spatial position, each cube is assigned a color from a periodic spatial color field with a period of 1010 meters along each axis. Specifically, we randomly sample N=269N=269 anchor points (i,i)i=1N\(p_i,c_i)\_i=1^N in a unit cube [0,1]3[0,1]^3, where i∈ℝ3p_i ^3 satisfy ‖i−j‖2>0.12\|p_i-p_j\|_2>0.12 for all i≠ji≠ j under periodic boundary conditions, and i∈[0,255]3c_i∈[0,255]^3 are RGB colors. The anchor set is tiled periodically in [−1,2][-1,2] along each axis. For a 3D query point x, we find its k=10k=10 nearest anchors with distances dii=1k\d_i\_i=1^k and compute the interpolated color as (1) wi=1di+ε−1dmax+ε,w~i=wi∑j=1kwj,w_i= 1d_i+ - 1d_ + , w_i= w_i _j=1^kw_j, (2) ⁡()=∑i=1kw~ii.c(x)= _i=1^k w_i\,c_i. We approximate this dense color field using a multi-grid hash encoding (Müller et al., 2022) with a lightweight MLP, which is then used to paint all cubes efficiently. A.3. 3D Bounding Box Modeling and Object Motion Condition We use TokenHMR (Dwivedi et al., 2024) to recover identity-consistent human meshes across frames. For each tracked individual, we extract torso vertices from the recovered SMPL mesh (Loper et al., 2023) and fit a tight 3D bounding box. The object motion condition is rendered as a projected 3D bounding box on a black background. Pixels inside the projected box are assigned grayscale values in the range [25,255][25,255] using per-video relative depth normalization, while background pixels remain at 00. During training, if no object motion is present, we use an all-black video as the object motion condition. During inference, we use the same per-video depth normalization rule as in training. The near bound is set to the minimum of 11 meter and the closest distance from the camera to any point on the object bounding box. The far bound is set to the maximum of 1010 meters and the farthest distance from the camera to the bounding box. Depth values inside the box are normalized to this range and mapped to grayscale intensities. To encode orientation, the 12 edges of the 3D bounding box are rendered using distinct colors sampled uniformly from the hue channel of HSV space. The color assignment is fixed across frames and the dataset. A.4. Depth-of-Field Modeling and Depth-of-Field Condition We use the same physically motivated bokeh operator for both bokeh video construction and depth-of-field condition generation. Given a per-frame focus depth, the operator computes blur radius from the depth deviation to the focal plane. To stabilize blur near large depth discontinuities, we identify regions with abrupt depth changes and apply spatially adaptive smoothing to the blur-radius field. The depth-of-field condition adopts a three-channel layered representation based on a synthetic checkerboard image. All channels share the same initial black-and-white checkerboard pattern, where the image width is divided into 30 vertical segments and the height is adjusted to maintain approximately square cells. Each channel corresponds to a canonical background depth in world coordinates, fixed at 5 meters, 15 meters, and 25 meters. For each frame, foreground pixels inside the projected 3D bounding box are assigned the object’s metric depth, while background pixels retain the channel-specific depth. The bokeh operator is applied independently to each depth map, producing three grayscale videos that are stacked into RGB channels to form the final depth-of-field condition video. Because blur depends jointly on depth and bokeh strength, the resulting condition exhibits color variation across channels. During training, if no bokeh effect is present, we use the original unblurred checkerboard as the condition. During inference, the same procedure can be applied with manually specified bounding boxes to enable user-controlled depth-of-field effects. A.5. Zoom Augmentation We simulate zoom using temporal center cropping followed by resizing to the original resolution. Crop factors and zoom intervals are sampled randomly. The same zoom transformation is applied consistently to the camera motion condition video, object motion condition video, and depth-of-field condition video to preserve pixel-wise alignment. Appendix B Evaluation Metrics This appendix provides detailed definitions of all evaluation metrics used in the main paper. B.1. Camera Translation Error To quantify camera path accuracy in a scale-invariant manner, we compute the average relative translation error between predicted and ground-truth camera positions. All camera trajectories are first normalized with respect to the camera pose at frame t=0t=0, such that translations t represent relative displacements from the initial camera position, and rotations R are defined relative to the initial orientation. The translation error is defined as: (3) ℰtrans=1T∑t=1T‖tpred−tgt‖2max⁡(max1≤τ≤T⁡‖τgt‖2, 1)∗100,E_trans= 1T _t=1^T \|t^pred_t-t^gt_t \|_2 ( _1≤τ≤ T \|t^gt_τ \|_2,\ 1)*100, The error metric is summarized as a single scalar value. B.2. Camera Rotation Error Camera orientation accuracy is evaluated using the geodesic distance between predicted and ground-truth rotations: (4) ℰrot=1T∑t=1Tarccos⁡(trace⁡(tpredtgt⊤)−12),E_rot= 1T _t=1^T ( trace (R^pred_tR^gt _t )-12 ), where tpredR^pred_t and tgtR^gt_t denote rotation matrices at frame t. The error is reported in degrees. B.3. Mask Intersection-over-Union (mIoU) Foreground spatial consistency is evaluated using mean Intersection-over-Union (mIoU): (5) mIoU=1T∑t=1T|Mtpred∩Mtgt||Mtpred∪Mtgt|,mIoU= 1T _t=1^T |M^pred_t∩ M^gt_t||M^pred_t∪ M^gt_t|, where MtpredM^pred_t and MtgtM^gt_t denote predicted and reference foreground masks at frame t, obtained using SAM2. B.4. 3D Bounding Box Pose Error We evaluate object-level geometric stability using both translation and rotation errors of the 3D bounding box. We employ a greedy algorithm to match M trajectory pairs (M=2M=2 in our setting), computing the error exclusively on frames t∈commt _comm where M bounding boxes are all detected. The screen-space center deviation is defined as: (6) ℰbbox-trans=1M⋅|comm|∑m=1M∑t∈comm‖t,mpred−t,mgt‖2H2+W2,E_bbox-trans= 1M· |T_comm | _m=1^M _t _comm \|c^pred_t,m-c^gt_t,m \|_2 H^2+W^2, where tc_t is the projected 2D center of the bounding box, and H, W are image height and width. Similarly, bounding box rotation error is measured using geodesic distance: (7) ℰbbox-rot=1M⋅|comm|∑m=1M∑t∈commarccos⁡(trace⁡(bbox,t,mpredbbox,t,mgt⊤)−12). splitE_bbox-rot=\,& 1M· |T_comm |\\ & _m=1^M _t _comm ( trace (R^pred_bbox,t,mR^gt _bbox,t,m )-12 ). split B.5. Trajectory Consistency Error Long-term motion fidelity is evaluated using trajectory consistency error computed from point tracks extracted by CoTracker: (8) ℰtraj=1NT∑i=1N∑t=1T‖i,tpred−i,tgt‖2H2+W2,E_traj= 1NT _i=1^N _t=1^T \|p^pred_i,t-p^gt_i,t \|_2 H^2+W^2, where i,tp_i,t denotes the 2D position of point i at frame t. B.6. Visual Quality and Semantic Alignment We evaluate perceptual quality using Fréchet Inception Distance (FID) and Fréchet Video Distance (FVD). Semantic alignment between generated videos and input prompts is evaluated using the Qwen VQA Score, computed by querying a vision–language model on sampled frames and averaging the confidence scores. Appendix C Inference hyperparameters During inference, we used the unipc (Zhao et al., 2023) sampling method, with the number of steps set to 20. The classifier-free guidance scale is set to 6. Appendix D Additional Ablation Study To evaluate the effectiveness of our camera extraction pipeline, we replace our metric-scale camera parameters with those directly estimated by MapAnything. We then train a GTL-1.3B model using these parameters as-is, while keeping all other components and training settings identical to the main model. We observe that MapAnything produces camera parameters with inconsistent intrinsic across frames, leading to noticeable instability in the resulting camera motion condition. This inconsistency introduces additional difficulty during training and degrades controllability. As shown in Table 5, the model trained with our camera pipeline achieves significantly better performance in camera-related metrics compared to the MapAnything baseline, demonstrating the superiority of our camera extraction pipeline for controllable camera modeling. Camera Extraction Method Cam Trans. Err. ↓ Cam Rot. Err. (°) ↓ MapAnything 13.53 6.84 GTL 10.15 5.07 Table 5. Ablation study on camera extraction pipeline. We compare camera parameters directly estimated by MapAnything with those produced by our pipeline under the same training setting. The results show that our camera extraction pipeline provides more reliable supervision, leading to improved camera translation and rotation accuracy. Appendix E Text Prompts for Experiments We provide the text prompts corresponding to all qualitative results for reproducibility. For fairness, all methods share the same prompts, except for camera-related descriptions, which are ignored by our model but used by baseline methods when applicable. Camera-related descriptions are marked with [Camera]. For Seedance 2.0, we use a different prompt format that incorporates both the input image and reference camera motion. Specifically, the prompt follows the template: “Use the input image as the first frame, follow only the camera motion from the reference video, and generate a new video described as: <PROMPT>.”, where <PROMPT> corresponds to the descriptions listed below. Camera Experiments (Fig. 4). • During the day, sunlight streamed in through the windows. Indoors, a man in a white suit, yellow tie, and glasses, with his hair meticulously styled, held a large bouquet of red roses with a serious expression. [Camera: The camera began with an overhead shot from above the ceiling, then rotated downwards, finally focusing on the pink ribbon decorating the rose bouquet.] Bounding Box Experiments (Fig. 5). • During the daytime, in a European-style residential complex, a red sedan slowly drives away from the complex’s gate, traveling along a gray road. The complex is surrounded by two-story European-style residential buildings. A row of bicycles is parked to the left of the building entrance, and a motorcycle is parked to the right. [Camera: The camera follows the car as it moves to the right, and when the car reaches the complex gate, the camera stops moving and begins to pan to the right until the car drives out of the complex gate.] • The video begins with a medium shot of a bright, modern office lobby. In the foreground is a professional woman with her hair tied back. Standing slightly behind her is a man wearing a black beanie. Initially, the woman has a warm and professional smile, while the man’s expression is more reserved. The woman walks forward, always facing the camera, with the man following closely behind. The woman guides the man through the office space. As they move forward, the camera smoothly moves backward, keeping them centered in the frame. More environmental details gradually become visible, including a marble elevator and staircase in the background. Disentanglement Experiments (Fig. 6). • A man on a city street, surrounded by buildings. Depth-of-Field Experiments (Fig. 7). • This video clip documents an encounter between two men on a European street. The scene opens with a medium shot. In the foreground, on the right, a man is looking at the camera. In the background, another man is walking away down the street. The man in the foreground turns around and begins to walk forward. As he walks forward, the man he was following stops, turns around to face him, and stands there expressionlessly. • A woman is talking to a girl. Hitchcock Experiments (Fig. 8). • A man standing on a green grassy field, with majestic mountains in the background under a dim, overcast sky. Additional Experiments (Fig. 9). • A woman with long brown hair, wearing a black suit and carrying a large black shoulder bag, walks slowly towards the hotel reception desk with her back to the camera. The lobby is predominantly yellow, with crystal chandeliers hanging from the ceiling, and red load-bearing columns with gold accents symmetrically positioned on either side of the reception desk. [Camera: Starting from the rooftop, the camera moves downward into the interior, then smoothly pushes forward through the hall] • Two rows of armored warriors stand motionless on either side. In the center of the main reception area sits a man in a suit, exuding an imposing presence. [Camera: The camera starts slightly facing the ground, then levels to a forward view and continues a steady forward push] • A paper airplane floated in the air, slowly drifting away into the distance. [Camera: The camera remains stationary and rotates to the right]

Instant research discovery