Paper deep dive
Video Generative Models as Geometry Learner
Haosen Yang, Jifei Song, Zhensong Zhang, Xiatian Zhu, Jiankang Deng
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent generative approaches to geometry estimation adapt pretrained image diffusion models and treat the task as image-conditioned generation. Leveraging off-the-shelf image diffusion models, they either (i) train task-specific geometry models (for depth and surface normal estimation) independently, losing the opportunity of exploring the intrinsic correlation of these geometric targets, or (ii) jointly fine-tune modified image diffusion backbones (e.g., altered self-attention), which typically demands substantial labeled data. To overcome these limitations in a principled fashion, we repurpose pretrained video generative models as a unified and data-efficient framework for geometry estimation, formulated innovatively as a next-frames prediction task. Our method, GeoNeXt, inherits naturally structured knowledge and richer priors from the video model, while further adapting them for joint modeling of images and geometry targets (image <-> geometry), enabling more data efficient and effective learning of geometry. Extensive experiments validate our method for zero-shot monocular depth and surface normal estimation across diverse datasets, outperforming both previous task-specific and unified generative competitors while using substantially less training data. Notably, our method rivals discriminative state-of-the-art approaches trained on over 100x more data and even standouts on several benchmarks.
Tags
Links
- Source: https://arxiv.org/abs/2608.28549v1
- Canonical: https://arxiv.org/abs/2608.28549v1
Trouble viewing inline? Open PDF directly ā
Full Text
55,162 characters extracted from source content.
Expand or collapse full text
Video Generative Models as Geometry Learner Haosen Yang Affiliation: University of Surrey, Guildford, UK Jifei Song Affiliation: University of Surrey, Guildford, UK Zhensong Zhang Affiliation: Independent Researcher Xiatian Zhu11 1 Corresponding authors. Affiliation: University of Surrey, Guildford, UK Jiankang Deng11 1 Corresponding authors. Affiliation: Imperial College London, London, UK https://happy-hsy.github.io/projects/GeoNeXt/ Abstract Recent generative approaches to geometry estimation adapt pretrained image diffusion models and treat the task as image-conditioned generation. Leveraging off-the-shelf image diffusion models, they either (i) train task-specific geometry models (for depth and surface normal estimation) independently, losing the opportunity of exploring the intrinsic correlation of these geometric targets, or (i) jointly fine-tune modified image diffusion backbones (e.g., altered self-attention), which typically demands substantial labeled data. To overcome these limitations in a principled fashion, we repurpose pretrained video generative models as a unified and data-efficient framework for geometry estimation, formulated innovatively as a next-frames prediction task. Our method, GeoNeXt, inherits naturally structured knowledge and richer priors from the video model, while further adapting them for joint modeling of images and geometry targets (imageāgeometryimage ) enabling more data efficient and effective learning of geometry. Extensive experiments validate our method for zero-shot monocular depth and surface normal estimation across diverse datasets, outperforming both previous task-specific and unified generative competitors while using substantially less training data. Notably, our method rivals discriminative state-of-the-art approaches trained on over 100Ć more data and even standouts on several benchmarks. Figure 1: We present GeoNeXt, a unified geometry estimation model for both depth and surface normals, repurposed from a off-the-shelf generative video model. Our idea is to leverage the temporal coherence and rich priors of a pretrained video generative model, adapting them for imageāgeometry joint modeling. The predicted geometry provides strong structural cues that enable diverse applications, e.g., (1) controllable image generation, (2) image relighting, and (3) mesh reconstruction. 1 Introduction Monocular 3D geometry estimation (e.g., depth and surface normal) is a fundamental problem in 3D vision, with applications spanning autonomous driving [18, 19], 3D surface reconstruction [63], and inverse rendering [30, 58]. However, recovering underlying 3D geometry from a single image remains challenging, as it requires comprehensive and precise geometric reasoning. Early discriminative approaches typically relied on supervised or semi-supervised learning with large collection of paired RGB images and depth maps. To this end, recent efforts [59, 60, 13, 3] focus on building data engines that leverage ever-growing amounts of unlabeled data to improve generalization. While straightforward, these approaches demand substantial compute and are sensitive to pseudo-label noise, leading to predictions lacking fine-grained, high-frequency detail. More promisingly, several works [29, 16] have leveraged the priors of pretrained text-to-image (T2I) generative models (e.g., Stable Diffusion [43]) for zero-shot 3D geometry (e.g., depth and normal) estimation, attaining competitive performance with limited task-specific supervision. Within this paradigm, 3D geometry is typically treated as a āspecial image,ā and the problem is formulated as image-conditioned generation. Prior efforts [29, 16, 20, 17, 22, 32, 52] can be largely categorized into two tracks. The first family (e.g., [29]) trains task-specific models independently with minimal architectural modifications, thereby best leveraging pretrained capacity under limited supervision. However, this design precludes parameter sharing and fails to exploit inter-task dependencies, necessitating a distinct model for each geometric target. The second family (e.g., [16]) avoids maintaining separate models, opting instead for a joint model that predicts both depth and normals by modifying the architecture of diffusion model (e.g., adding cross-attention or switcher modules). However, these architectural changes substantially increase data requirements, because they widen the gap between pretraining and finetuning, reducing transfer efficiency. Both families reuse image diffusion models for geometry estimation without explicitly modeling imageāgeometry interactions. We conjugate this strategy could lead to cross-modal inconsistencies and degraded geometric fidelity, with reduced clarity and loss of fine detail. To address these limitations, we depart from image-generation models and explore pretrained video generative models to monocular geometry estimation. Our intuition is as follows: (i) Video diffusion models are pretrained on large-scale video datasets, in addition to images, yielding richer generative priors that enable high-quality synthesis across a wide range of domains. (i) Their architectures and pretraining impart temporal-attention priors that capture cross-frame dependencies, which we leverage for joint modeling between images and geometry targets (image ā geometry). (i) Even without explicit geometric supervision during pretraining, such priors can be adapted to predict geometry [25, 48, 61, 27]. Building on this intuition, we propose GeoNeXt, a unified and data-efficient framework that reformulates geometry estimation as next-frame prediction in a video diffusion model, enabling coherent joint modeling of images and geometric modalities such as depth and surface normals. Given an RGB image, we cast geometry prediction as next-frame generation: the target geometry (depth and normals) is modeled as the subsequent frame conditioned on RGB image. Following Stable Video Diffusion [6], we replicate the input image into the geometry slots (depth and normals) to provide a strong, consistent conditioning signal for the geometry objective. Crucially, we preserve the pretrained modelās native generation order and synthesize the image and its geometry in lockstep. Rather than training a geometry-only generator, GeoNeXt co-generates image and geometry along a shared denoising trajectory, which preserves fine-grained detail and enhances imageāgeometry consistency. Our adaptation is lightweight: we fine-tune only the denoising U-Net with minimal architectural changes (e.g., removing the text/CLIP branch), training on modest RGBādepthānormal triplets from datasets such as Hypersim and Virtual KITTI (e.g., Hypersim[42], Virtual KITTI[18]). Leveraging video-diffusion priors, GeoNeXt exhibits strong zero-shot generalization, effectively transferring image-to-video generative capability to imageāgeometry prediction without requiring large additional datasets. Our contributions are: (1) We introduce GeoNeXt, a novel generative approach for jointly estimating depth and surface normals, which repurposes a video generative model as a unified geometry learner, formulated as a next-frames prediction task. (2) We comprehensively study the optimal fine-tuning protocol for adapting the pretrained video generative model to geometry estimation. In particular, we analyze GeoNeXtās generative formulation, architectural design, and the influence of reconstruction order on model robustness and overall performance. (3) Extensive experiments validate the effectiveness of GeoNeXt for zero-shot monocular depth and surface normal estimation across diverse datasets, outperforming both task-specific and unified generative competitors even with substantially less training data, while competitively rivaling discriminative models trained with even larger data and compute. 2 Related Work 2.1 Monocular depth and normal estimation Estimating depth and surface normals from a single image is an ill-posed but fundamental problem, as both capture complementary information about 3D scene geometry. Early approaches relied heavily on comprehensive scene understanding and were typically limited to specific domains [14, 15, 33, 64, 1, 36, 2, 55, 65]. Estimating depth and surface normals in unconstrained environments requires models capable of generalizing beyond the training distribution, which remains a difficult challenge in practice. To improve generalization, recent depth estimation works [59, 60, 35, 41] have focused on collecting large and diverse datasets and developing training strategies that leverage them effectively. Since metric depth depends on camera intrinsics, most approaches, including ours, predict affine-invariant depth, which is consistent across different imaging setups. In contrast, models that attempt to estimate absolute metric depth are often limited to fixed camera configurations [14] or must explicitly condition on known intrinsic parameters [5, 21]. A similar trend can be observed in monocular surface normal estimation, where large-scale pretrained models [13, 2, 3] have become the dominant paradigm, while others [3] further improve accuracy by incorporating task-specific inductive biases designed for normal estimation. To avoid maintaining separate models, several studies have explored depth and surface normal estimation as a multi-task learning problem [24, 34, 56, 65]. These methods typically employ multiple task-specific decoder branches while sharing a common backbone to enable information exchange in-between. Although this design allows for joint estimation, the models remain discriminative. Moreover, existing approaches rely on limited training datasets and exhibit poor generalization, often missing fine geometric details in challenging cases. In contrast, our method tackles depth and surface normal joint estimation without large-scale annotated datasets. Instead, we repurpose the broad priors learned by pretrained video generative models and reformulate the task as next-frame prediction, enabling more efficient and effective geometry learning. 2.2 Diffusion Models for Geometry Estimation Recently, diffusion models [23, 51, 28] have demonstrated remarkable capabilities in both image and video generation, producing high-quality results across a wide range of domains [7, 57, 37, 38, 43, 6, 4]. Several studies have explored extending diffusion models to geometry-related tasks, including depth estimation and surface normal prediction. Recent attempts [26, 46, 45, 66] introduced diffusion-based frameworks for geometric prediction in collaboration with discriminative pipelines, achieving promising results but remaining limited to their specific training domains. More recent efforts [29, 12, 20, 17, 22, 62] have explored leveraging pretrained latent diffusion models (LDMs) for geometry estimation tasks, particularly for depth and surface normal prediction across diverse real-world settings, often requiring only modest amounts of fine-tuning data. Within this paradigm, 3D geometry (e.g., depth and surface normals) is typically represented as a āspecialized image,ā framing the problem as image-conditioned generation. However, existing methods still train independent, task-specific models for each prediction objective, which limits scalability and cross-task consistency. While GeoWizard [16] and Orchid [32] attempt joint depthānormal prediction by modifying the diffusion model architecture or retraining the entire model, including the VAE, these designs substantially increase data requirements and widen the gap between pretraining and fine-tuning. Inspired by monocular image-based geometry estimation, several works [25, 48, 48, 27] extend this paradigm to the video domain, leveraging pretrained video diffusion models to enhance temporal consistency and mitigate flickering artifacts across frame sequences. Our approach moves beyond these efforts by exploring the potential of pretrained video generative models for joint geometry estimation. GeoNeXt adapts the temporal priors learned by video generative models for unified modeling between images and geometry targets (imageāgeometryimage ), enabling more efficient and effective geometry learning. 3 Method Figure 2: Overview of the GeoNeXt fine-tuning protocol. (a) We start from the pretrained Stable Video Diffusion model and encode the RGB image x, depth d, and surface normal n into the latent space using the frozen Stable Diffusion VAE. (b) Noise is sampled and added to the latent z. The image latent is replicated across the geometry slots (depth and normal), and further conditioned by concatenating the geometry latents. (c) We fine-tune only the GeoNeXt U-Net. (d) The model is optimized with the standard diffusion objective over image, depth, and normal latents to ensure fine-grained alignment and consistency between image and geometry. In this work, we adapt a pretrained video diffusion model for monocular geometry estimation jointly, leveraging its strong generative priors learned from large-scale, high-quality data. Given an input image x, our objective is to predict its corresponding geometric informationānamely, a depth map d and a surface-normal map n. Section 3.1 provides a brief overview of the notation for image and video diffusion, followed by a detailed description of GeoNeXt in Section 3.2. Finally, we describe the inference strategy in Section 3.3. An overview of GeoNeXt training is presented in Fig. 2. 3.1 Preliminaries of Diffusion Models Diffusion models [23, 50, 43, 51] model a data distribution via (i) a forward noising process that gradually maps data to a simple reference distribution (e.g., Gaussian), and (i) a learned reverse process that denoises samples back to the data manifold. In latent image diffusion, one first maps an image 0x_0 to a compact latent 0=ā°ā”(0)z_0=E(x_0) to reduce computation. The forward (Markov) process then perturbs 0z_0 according to a noise schedule Ī²tt=1T\ _t\_t=1^T: qā”(tā£tā1)=ā”(t|1āĪ²tātā1,Ī²tā),q(z_t _t-1)\,=\,N (z_t\, |\, 1- _t\,z_t-1,\, _tI ), (1) and the reverse process uses a learned denoiser Īø=(Ī¼Īø,Ī£Īø)Z_Īø=( _Īø, _Īø) to progressively reconstruct: pĪøā(tā1ā£t)=ā”(tā1|Ī¼Īøā(t,t),Ī£Īøā(t,t)).p_Īø(z_t-1 _t)\,=\,N (z_t-1\, |\, _Īø(z_t,t),\, _Īø(z_t,t) ). (2) Prior works [29, 22] formulates monocular geometry estimation as imageāconditioned diffusion: a geometry targetādepth d or normals nāis encoded to a latent, noise is added, and the RGB image provides conditioning during denoising (often by concatenation). In contrast, we build on the video diffusion model Stable Video Diffusion (SVD) [6], which denoises a compact latent sequence and follows the EDM formulation [28]. In EDM, the forward process adds i.i.d. Gaussian noise with variance Ļt2 _t^2 to a clean sample 0ā¼pā”()z_0 p(z): t=0+ĻtāĻµ,Ļµā¼ā”(,),z_t\;=\;z_0+ _t\, Īµ, Īµ (0,I), (3) where tā¼pā”(,Ļt)z_t p(z; _t) denotes the noised sample at noise level Ļt _t. As ĻtāĻmax _tā _ , the sample distribution becomes effectively Gaussian; generation then proceeds by progressively denoising from Ļmax _ to Ļ0=0 _0=0. EDM adopts a preconditioned denoiser to maintain stability across noise levels [28, 44]. Rather than predicting directly from tz_t, the model combines the noisy input with the network output using noiseādependent weights: Īøā(t,Ļt,) _Īø(z_t, _t,c) =cskipā(Ļt)āt =c_skip( _t)\,z_t (4) +coutā(Ļt)āFĪøā(cinā(Ļt)āt,cnoiseā(Ļt),) +c_out( _t)\,F_Īø (c_in( _t)\,z_t,\;c_noise( _t),\;c ) where FĪøF_Īø is a learnable network (e.g., a UNet), and cin,cout,cskip,cnoisec_in,c_out,c_skip,c_noise are scalar functions of Ļt _t that control input scaling, output scaling, the skip path, and the noise embedding. 3.2 Geometry as Next Frames Generative Formulation To jointly estimate depth and surface normals, we formulate monocular geometry prediction as an image-conditioned, image-to-video diffusion problem, where geometry (d for depth and n for surface normals) is modeled as the subsequent frame(s) following the input image. The model learns the conditional distribution pā”(ā£)p(g ), where =,g=\d,n\ and āāHĆWĆ3x ^HĆ WĆ 3. We define the clean imageāgeometry triplet =(,,).G=(x,\,d,\,n). (5) The triplet component is corrupted to tG_t by adding Gaussian noise according to Eq. (3), while x is replicated and kept fixed as the conditioning signal. During the reverse process, the conditional denoiser ĪøZ_Īø iteratively removes noise, mapping tā¦tā1G_t _t-1 conditioned on x, while simultaneously denoising the image to maintain consistency between the image and geometry. Additional conditioning is introduced by concatenating the latent image features with the noisy geometry input before feeding them into the denoiser. Following EDM [28], training minimizes the standard noise-prediction objective using the preconditioned parameterization in Eq. (4). At inference time, (,)(d,n) are reconstructed by first initializing with Gaussian noise Tā¼ā”(,)g_T (0,I), and then progressively denoising it through iterative applications of Īøā(t,,t)Z_Īø(g_t,x,t) from timestep T down to 00. Latent space transformation Instead of operating in pixel space, we follow latent diffusion models (LDMs) [43], which perform the diffusion process in a compact, low-dimensional latent space. This offers substantial computational savings and scales to high-resolution synthesis. The latent space is provided by the bottleneck of a VAE trained independently of the denoiser, yielding representations that are compact yet perceptually aligned with image space. To express our formulation in this latent space, we encode the imageāgeometry triplet with the VAE encoder ā°E and decode with D: ()=ā°ā”(),^=ā”(()),z^(G)=E(G), G=D (z^(G) ), (6) where =(,,) G=(x,\,d,\,n) denotes the imageāgeometry triplet (depth d and surface normals n). Here, ()z^(G) is the latent code of the triplet and G is its reconstruction; ā°E and D are the VAE encoder and decoder, respectively. For depth, which is single-channel, we tile the map to three channels to match the VAEās RGB encoder; at decoding, we average the three output channels to recover a single-channel depth prediction. We adopt a frozen image VAE [43] to encode both the image and its corresponding geometry into the latent space used to train our conditional denoiser. Both depth and normal maps are preprocessed by normalizing the depth values to [0,1][0,1], followed by applying (xā0.5)Ć2(x-0.5)Ć 2 to align them with the VAE input range of [ā1,1][-1,1]. Adapted denoising U-Net Stable Video Diffusion is an image-to-video diffusion model that synthesizes a video sequence conditioned on a single input image. The conditioning image can be injected to the UNet by (i) concatenating its latent with the input noise latent, and (i) providing its CLIP [39] embedding to intermediate features via cross-attention. In our paradigm, geometry (e.g., depth and surface normal) is modeled as the next frames in a image to geomery, so we adapt the conditioning (i) to suit image-to-geomerty generation. Additionally, the temporal attention mechanism enables the image condition to be propagated to the geometry branch, enhancing structural consistency. Furthermore, as illustrated in Fig. 2, given the encoded latents of the image, depth, and normal, we concatenate the geometry latents (depth and normal) with the image latent to form the final conditioning input. Instead of generating geometry alone, the image is also reconstructed to provide a consistent conditioning signal. We disable conditioning (i) because it requires resizing the image, which leads to geometry distortions in the depth and normal maps. 3.3 Inference At inference, we initialize the latent variables t(x)z_t^(x), t(d)z_t^(d), and t(n)z_t^(n) from standard Gaussian noise, and encode the input image into its latent 0(x)z_0^(x). The initialized noises and the image latent 0(x)z_0^(x) are concatenated and fed into the denoiser U-Net, which iteratively predicts the latents of the image, depth, and surface normal across diffusion steps. After denoising, the image latent is discarded, and the final depth map and surface normal are decoded from their corresponding latents via the VAE decoder. By jointly generating the image, depth, and surface normal during inference, GeoNeXt enhances consistency between image appearance and geometric structure. 4 Experiments Implementation We implement GeoNeXt in PyTorch, building upon the image-to-video variant of Stable Video Diffusion [6] as a representative choice. As our method is architecture-agnostic and can be seamlessly integrated with more advanced DiT-based models (e.g., WAN [54]), we report the corresponding results and implementation details in the supplementary material. The original cross-attention conditioning in SVD is disabled. During training, we adopt the standard EDM noise schedule [13], where the noise level Ļt _t is sampled from a distribution pā”(Ļ)=ā”(0.7,1.6)p(Ļ)=N(0.7,1.6), following SVD. At inference, we use the EDM sampler with only 5 denoising steps. Following [29], we ensemble the outputs from 5 inference runs initialized with different noise seeds. We train using the Adam optimizer with a learning rate of 5Ć10ā65Ć 10^-6 and apply random horizontal flipping for data augmentation. For depth estimation, we operate in disparity space, i.e., d=1/dā²d=1/d , where d denotes the predicted disparity and dā²d represents the corresponding true depth. Training datasets We employ two synthetic datasets that together cover both indoor and outdoor environments. (1) Hypersim [42] contains 461 photorealistic indoor scenes. We use the official training split with approximately 54K samples and remove incomplete ones, resulting in about 39K valid samples. All RGB images, depth maps, and normal maps are resized to 576Ć768576Ć 768. (2) Virtual KITTI 2 [9] is a synthetic driving dataset comprising five urban scenes captured under diverse illumination and weather conditions. We utilize four scenes for training, providing roughly 20K samples, which are cropped to 352Ć1216352Ć 1216 with a far-plane depth limit of 80 m. Following Marigold [29], we sample each training batch from the two datasets with a 9:1 ratio, selecting from Hypersim with 90% probability and from Virtual KITTI with 10%. Evaluation datasets We evaluate GeoNeXt on depth and surface normal estimation using real-world datasets that are not seen during training. For depth estimation, we test on five datasets: NYUv2 [49] and ScanNet [11], which include indoor scenes; KITTI [18], which captures diverse outdoor driving scenes; ETH3D [47], a high-resolution dataset covering both indoor and outdoor environments; and DIODE [53], another high-resolution dataset providing dense and accurate depth maps across varied conditions. For surface normal estimation, we evaluate on five datasets as well: NYUv2 [49], ScanNet [11], and iBims-1 [31], which contain real indoor scenes; Sintel [8], which features highly dynamic outdoor scenes; and OASIS [10], which consists of in-the-wild Internet images. Table 1: Quantitative comparison on zero-shot affine-invariant depth estimation. We compare GeoNeXt with representative discriminative and generative methods across five datasets. The upper block lists large-scale discriminative approaches, the middle block reports generative methods trained for depth estimation only, and the lower block presents unified geometry estimation methods, which serve as our main comparison focus. * indicates results reproduced by us. Bold marks the best performance. Method Training Data NYUv2 KITTI ETH3D ScanNet DIODE AbsRelā Ī“1ā _1 AbsRelā Ī“1ā _1 AbsRelā Ī“1ā _1 AbsRelā Ī“1ā _1 AbsRelā Ī“1ā _1 Large-scale discriminative depth models MiDaS [41] 2M 11.1 88.5 23.6 63.0 18.4 75.2 12.1 84.6 33.2 71.5 Omnidata [13] 12.2M 7.4 94.5 14.9 83.5 16.6 77.8 7.5 93.6 33.9 74.2 DPT [40] 1.4M 9.8 90.3 10.0 90.1 7.8 94.6 8.2 93.4 18.2 75.8 DA [59] 62.6M 4.3 98.1 7.6 94.7 12.7 88.2 4.3 98.1 26.0 75.9 DA-V2 [60] 62.6M 4.5 97.9 7.4 94.6 13.1 86.5 4.2 97.8 26.5 73.4 Generative methods for depth estimation only Marigold [29] 74K 5.5 96.4 9.9 91.6 6.5 95.9 6.4 95.2 30.8 77.3 E2E-FT [17] 74K 5.4 96.5 9.6 92.1 6.4 95.9 5.8 96.5 30.3 77.6 DepthFM [20]* 63K 6.9 95.5 9.8 91.5 6.6 95.8 7.8 93.1 25.2 77.9 Lotus-G [22]* 59K 5.4 96.6 8.5 92.2 5.9 97.0 5.9 95.6 23.0 73.0 Generative methods for unified geometry estimation GeoWizard [16]* 208K 5.6 96.3 14.4 82.0 6.8 95.8 6.4 95.2 33.0 73.5 GeoNeXt 59K 5.3 96.8 8.2 92.6 5.6 97.2 5.9 95.8 22.6 74.3 Figure 3: Qualitative comparison of monocular depth estimation methods across diverse datasets. GeoNeXt demonstrates superior reconstruction of fine structures. Figure 4: Qualitative comparison of surface normal estimation methods across diverse datasets. GeoNeXt produces more accurate and detailed normal reconstructions, effectively capturing fine geometric structures and surface orientations. Table 2: Quantitative results on zero-shot surface normal estimation. We evaluate GeoNeXt against representative discriminative and generative approaches across five benchmark datasets. The top section summarizes large-scale discriminative baselines, the middle section reports generative methods trained exclusively for surface normal prediction, and the bottom section highlights unified geometry estimation frameworks for direct comparison. * indicates results reproduced by us. Bold marks the best performance. Method Training Data NYUv2 ScanNet iBims-1 Sintel OASIS meanāā 11.25āā11.25 \!\! meanāā 11.25āā11.25 \!\! meanāā 11.25āā11.25 \!\! meanāā 11.25āā11.25 \!\! meanāā 11.25āā11.25 \!\! Large-scale discriminative normal models Omnidata [13] 12.2M 23.1 45.8 22.9 47.4 19.0 62.1 41.5 11.4 24.9 31.0 EESNU [2] 2.5M 16.2 58.6 - - 20.0 58.5 42.1 11.5 27.7 24.0 DSINE [3] 160K 16.4 59.6 16.2 61.0 17.1 67.4 34.9 21.5 24.4 28.8 Generative methods for surface normal estimation only. Marigold [29] 74K 20.9 50.5 21.3 45.6 18.5 64.7 - - - - StableNormal [62] 250K 18.6 53.5 17.1 57.4 18.2 65.0 36.7 14.1 26.5 23.5 E2E-FT [17] 74K 16.5 60.4 14.7 66.1 16.1 69.7 33.5 22.3 23.2 29.4 Lotus-G [22]* 59K 16.6 59.4 15.1 63.8 17.2 66.3 33.6 21.0 22.9 29.3 Generative methods for unified geometry estimation. GeoWizard [16]* 208K 18.9 50.1 17.2 53.8 19.4 62.8 40.3 12.8 25.0 23.7 GeoNeXt 59K 16.7 60.0 16.0 62.8 16.4 69.2 33.0 21.5 22.8 30.8 4.1 Results Analysis Zero-shot Depth Estimation Comparison We present the quantitative results of zero-shot affine-invariant depth estimation in Table 1. The proposed GeoNeXt is compared with three categories of state-of-the-art methods: (1) large-scale discriminative depth estimation models; (2) generative methods for depth estimation only; and (3) generative methods for unified geometry estimation. Compared with discriminative models such as DepthAnything, which rely on 63.5M training images, our model uses nearly 100Ć less data yet achieves highly competitive results. On several benchmarks, GeoNeXt even surpasses them, achieving, for example, an AbsRel of 13.1 vs. 5.6 on ETH3D. Next, we compare GeoNeXt with recent generative methods designed solely for depth estimation. To demonstrate the advantages of the next-frame prediction formulation within video diffusion models, we compare our approach with image diffusion based generative methods, including Marigold, E2E-FT, DepthFM, Lotus-G, and GeoWizard. In particular, Lotus-G-depth is trained on a similar scale (59K samples) but optimized only for depth prediction, requiring a separate model. In contrast, our method formulates geometry estimation as a next-frame prediction task and models geometric consistency within a unified framework. As shown in Table 1, our model achieves superior or comparable performance across datasets. We further compare with unified geometry estimation methods such as GeoWizard, which also rely on image diffusion models but introduce additional architectural modifications such as cross-attention and switching modules. Although GeoWizard is trained on a substantially larger dataset (208K samples), GeoNeXt still maintains superior performance. For example, GeoNeXt surpasses GeoWizard by 6.2 in AbsRel and 9.4 in Ī“1 _1 on KITTI, and by 10.4 in AbsRel and 1.8 in Ī“1 _1 on DIODE, despite using significantly less training data.In addition, Figure 3 presents qualitative comparisons across multiple datasets. Our method demonstrates clearly superior visual results, particularly in reconstructing fine-grained structures (e.g., chair, poles, and fences), preserving object boundaries. Zero-shot Normal Estimation Comparison To thoroughly evaluate our method, we conduct zero-shot surface normal estimation and compare its performance with three categories of state-of-the-art approaches, following the same experimental setup as used for depth estimation, as shown in Table 2. Despite being trained on only 59K samples, our model delivers competitive or superior accuracy relative to discriminative baselines such as Omnidata and EESNU. Compared with normal-specific generative methods based on image diffusion models and trained on a similar data scale, such as Lotus-G-normal, GeoNeXt achieves the best performance across most datasets. Moreover, compared to unified geometry estimation models like GeoWizard, GeoNeXt attains the overall strongest results, for example, mean angular errors of 16.4 on iBims-1 and 33.0 on Sintel, while jointly estimating both depth and normals using significantly less training data. These results quantitatively demonstrate the effectiveness and data efficiency of our approach. We further present qualitative comparisons in Figure 4, where our method produces more accurate surface normal predictions, particularly on curved and deformable surfaces. The results demonstrate strong cross-domain generalization, with our model achieving consistent and detailed normal reconstructions that faithfully capture fine geometric structures and surface orientations across both synthetic and real-world scenes. 4.2 Ablation Study We conduct ablation studies for GeoNeXt on depth and surface normal estimation. For evaluation, we select two zero-shot sets for each task: NYU and ETH3D for depth estimation, and NYU and iBims-N for surface normal estimation. Table 3: Ablation on the generative formulation, model architecture, and joint estimation. Upper block: Models trained for unified geometry estimation; Lower block: Mfiodels trained for depth or surface normal estimation individually. Method NYUv2 ETH3D NYUv2 iBims-N AbsRelā Ī“1ā _1 AbsRelā Ī“1ā _1 Meanā 11.25āā11.25 Meanā 11.25āā11.25 Unified geometry estimation (a) w/o Image Recon. 6.5 95.2 6.7 96.0 17.9 55.8 18.5 65.2 (b) w/ CLIP Embed. 5.6 96.5 5.8 97.0 16.9 59.6 16.5 69.0 (c) Full model 5.3 96.8 5.6 97.2 16.7 60.2 16.4 69.2 Separated geometry estimation (d) Depth only 5.9 96.5 6.4 96.3 ā ā ā ā (e) Surface Normal only ā ā ā ā 17.2 58.8 17.3 67.1 Generative formulation We hypothesize that a key factor behind the effectiveness of GeoNeXt lies in its formulation and architecture, which jointly synthesize the image and its geometry in lockstep, rather than employing a geometry-only generator. To validate this hypothesis, we first evaluate a variant without the image reconstruction branch. Following the setup of Marigold, which formulates the task as image-conditioned generation, we treat the first and second frames as depth and normal targets, respectively. As shown in Table 3(a), removing image reconstruction leads to a noticeable drop in performance, confirming that reconstructing the input image enhances imageāgeometry consistency and contributes significantly to the overall effectiveness of the generative formulation. Model architecture Another important component of GeoNeXt is the CLIP embedding module in the U-Net, which resizes the input image to extract semantic features for conditioning the generation process. To evaluate its effect, we remove this CLIP-based conditioning. As shown in Table 3(b), excluding the CLIP embedding slightly improves performance, suggesting that resizing the image for CLIP feature extraction may distort intrinsic structural information, thereby hindering stable and consistent geometry generation. Joint Depth and Normal Estimation We further investigate the effectiveness of joint geometry estimation. To this end, we train two separate models to estimate depth and surface normals independently. As shown in Table 3(d) and (e), this setup results in a clear performance drop across all evaluation metrics, indicating that joint training more effectively captures the correlation between the two geometric representations. Depth and Normal Reconstruction Order Table 4: Ablation on reconstruction order. We compare two configurations that assign depth and normal to different frames. The results are nearly identical, demonstrating the robustness of the model to reconstruction order. Method NYUv2 ETH3D NYUv2 iBims-N AbsRelā Ī“1ā _1 AbsRelā Ī“1ā _1 Meanā 11.25āā11.25 Meanā 11.25āā11.25 normal-depth 5.7 96.6 5.5 97.2 16.6 60.0 16.5 69.4 depth-normal 5.3 96.8 5.6 97.2 16.7 60.2 16.4 69.2 We also analyze the impact of reconstruction order between depth and surface normals. As shown in Table 4, both reconstruction orders, where depth and normal frames are swapped, achieve nearly identical performance across all metrics. These marginal variations suggest that GeoNeXt successfully models the intrinsic relationships among image, depth, and normal representations, making the reconstruction order largely flexible without significantly affecting performance. 4.3 Cost-effectiveness analysis Table 5: Cost-effectiveness analysis. Upper: models trained on geometric targets; we report combined depth and normal inference time and parameter count. Lower: unified geometry estimation variants. NFEs: number of function evaluations required to obtain a prediction, defined as diffusion steps Ć ensemble size for diffusion models. Method ETH3D iBims-N NFEs Inference time (s) Param (B) AbsRelā Ī“1ā _1 Meanā 11.25āā11.25 Separated geometry estimation Marigold [29] 7.0 95.5 18.5 65.6 1Ć11Ć 1 0.8 1.9 Marigold [29] 6.5 95.9 18.5 64.7 50Ć1050Ć 10 180.5 1.9 Lotus-G [22] 5.9 97.0 17.2 66.3 1Ć11Ć 1 0.8 1.9 Unified geometry estimation GeoWizard [16] 7.3 96.3 19.9 82.0 1Ć11Ć 1 1.2 0.9 GeoWizard [16] 6.8 96.8 19.4 82.5 50Ć1050Ć 10 272.1 0.9 GeoNeXt 5.8 97.0 16.8 68.9 1Ć11Ć 1 1.0 1.5 GeoNeXt 5.6 97.2 16.4 69.2 5Ć55Ć 5 10.0 1.5 We present a cost-effectiveness analysis of different approaches in Table 5. All inference times are measured on a single NVIDIA A5000 using an input resolution of 768Ć768768Ć 768. Compared with models trained separately for depth and surface normals, our unified approach achieves comparable inference time while delivering the best overall performance, even though Lotus-G leverages more advanced training strategies. Moreover, separated models require storing and maintaining two independent checkpoints, which increases the overall storage footprint. In real-world deployment, these approaches also introduce considerable I/O overhead, as separate models (e.g., for depth and normals) need to be loaded onto and off the GPU individually. The resulting data transfer latency can substantially exceed the actual computation time required for inference. We note that this I/O overhead is not included in the reported runtime measurements. GeoWizard achieves unified geometry estimation by modifying the attention forward pass and introducing a geometry-switch mechanism. However, it requires substantially more training data to converge. When applied with a single prediction, its performance is limited. Although increasing the number of denoising steps or applying large-scale ensembling can improve its accuracy, this comes at the cost of significantly higher inference time, making it less practical compared with our method. In contrast, our single-step prediction already surpasses all baseline methods. By incorporating only a few additional denoising iterations along with a lightweight ensemble strategy, we can further enhance performance while preserving fast and computationally efficient inference. 5 Conclusion We propose GeoNeXt, a novel approach for jointly estimating geometry from a single image. The core idea is to repurpose pretrained video generative models as a unified and data-efficient framework for geometry estimation, formulated as a next-frame prediction task. GeoNeXt naturally inherits temporal coherence and rich generative priors from video models, while further adapting them for joint modeling between images and geometry targets (imageāgeometryimage ). We systematically explore three key aspects: a joint generative formulation within the video generative framework, architectural adaptations for geometry estimation, and robustness to reconstruction order. Our results highlight the importance of preserving fine-grained details in both depth and surface normal estimation, supported by an effective training protocol. Extensive quantitative and qualitative evaluations demonstrate the effectiveness of our approach, with GeoNeXt achieving strong performance even with limited training data. References [1] S. Aich, J. M. U. Vianney, M. A. Islam, and M. K. B. Liu (2021) Bidirectional attention network for monocular depth estimation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), p. 11746ā11752. Cited by: Ā§2.1. [2] G. Bae, I. Budvytis, and R. Cipolla (2021) Estimating and exploiting the aleatoric uncertainty in surface normal estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 13137ā13146. Cited by: Ā§2.1, Ā§2.1, Table 2. [3] G. Bae and A. J. Davison (2024) Rethinking inductive biases for surface normal estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 9535ā9545. Cited by: Ā§1, Ā§2.1, Table 2. [4] J. Betker, G. Goh, L. Jing, T. Brooks, J. Wang, L. Li, L. Ouyang, J. Zhuang, J. Lee, Y. Guo, et al. (2023) Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf 2 (3), p. 8. Cited by: Ā§2.2. [5] S. F. Bhat, I. Alhashim, and P. Wonka (2021) Adabins: depth estimation using adaptive bins. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 4009ā4018. Cited by: Ā§2.1. [6] A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y. Levi, Z. English, V. Voleti, A. Letts, et al. (2023) Stable video diffusion: scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127. Cited by: Ā§1, Ā§2.2, Ā§3.1, Ā§4. [7] T. Brooks, A. Holynski, and A. A. Efros (2023) Instructpix2pix: learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 18392ā18402. Cited by: Ā§2.2. [8] D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black (2012) A naturalistic open source movie for optical flow evaluation. In European conference on computer vision, p. 611ā625. Cited by: Ā§4. [9] Y. Cabon, N. Murray, and M. Humenberger (2020) Virtual kitti 2. arXiv preprint arXiv:2001.10773. Cited by: Ā§4. [10] W. Chen, S. Qian, D. Fan, N. Kojima, M. Hamilton, and J. Deng (2020) Oasis: a large-scale dataset for single image 3d in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 679ā688. Cited by: Ā§4. [11] A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. NieĆner (2017) Scannet: richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 5828ā5839. Cited by: Ā§4. [12] Y. Duan, X. Guo, and Z. Zhu (2024) Diffusiondepth: diffusion denoising approach for monocular depth estimation. In European Conference on Computer Vision, p. 432ā449. Cited by: Ā§2.2. [13] A. Eftekhar, A. Sax, J. Malik, and A. Zamir (2021) Omnidata: a scalable pipeline for making multi-task mid-level vision datasets from 3d scans. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 10786ā10796. Cited by: Ā§1, Ā§2.1, Ā§4, Table 1, Table 2. [14] D. Eigen, C. Puhrsch, and R. Fergus (2014) Depth map prediction from a single image using a multi-scale deep network. Advances in neural information processing systems 27. Cited by: Ā§2.1, Ā§2.1. [15] H. Fu, M. Gong, C. Wang, K. Batmanghelich, and D. Tao (2018) Deep ordinal regression network for monocular depth estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 2002ā2011. Cited by: Ā§2.1. [16] X. Fu, W. Yin, M. Hu, K. Wang, Y. Ma, P. Tan, S. Shen, D. Lin, and X. Long (2024) Geowizard: unleashing the diffusion priors for 3d geometry estimation from a single image. In European Conference on Computer Vision, p. 241ā258. Cited by: Ā§1, Ā§1, Ā§2.2, Table 1, Table 2, Table 5, Table 5. [17] G. M. Garcia, K. Abou Zeid, C. Schmidt, D. De Geus, A. Hermans, and B. Leibe (2025) Fine-tuning image-conditional diffusion models is easier than you think. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), p. 753ā762. Cited by: Ā§1, Ā§2.2, Table 1, Table 2. [18] A. Geiger, P. Lenz, C. Stiller, and R. Urtasun (2013) Vision meets robotics: the kitti dataset. The international journal of robotics research 32 (11), p. 1231ā1237. Cited by: Ā§1, Ā§1, Ā§4. [19] C. Godard, O. Mac Aodha, and G. J. Brostow (2017) Unsupervised monocular depth estimation with left-right consistency. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 270ā279. Cited by: Ā§1. [20] M. Gui, J. Schusterbauer, U. Prestel, P. Ma, D. Kotovenko, O. Grebenkova, S. A. Baumann, V. T. Hu, and B. Ommer (2025) DepthFM: fast generative monocular depth estimation with flow matching. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 3203ā3211. Cited by: Ā§1, Ā§2.2, Table 1. [21] V. Guizilini, I. Vasiljevic, D. Chen, R. AmbruČ, and A. Gaidon (2023) Towards zero-shot scale-aware monocular depth estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 9233ā9243. Cited by: Ā§2.1. [22] J. He, H. Li, W. Yin, Y. Liang, L. Li, K. Zhou, H. Zhang, B. Liu, and Y. Chen (2024) Lotus: diffusion-based visual foundation model for high-quality dense prediction. arXiv preprint arXiv:2409.18124. Cited by: Ā§1, Ā§2.2, Ā§3.1, Table 1, Table 2, Table 5. [23] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. Advances in neural information processing systems 33, p. 6840ā6851. Cited by: Ā§2.2, Ā§3.1. [24] M. Hu, W. Yin, C. Zhang, Z. Cai, X. Long, H. Chen, K. Wang, G. Yu, C. Shen, and S. Shen (2024) Metric3d v2: a versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence. Cited by: Ā§2.1. [25] W. Hu, X. Gao, X. Li, S. Zhao, X. Cun, Y. Zhang, L. Quan, and Y. Shan (2025) Depthcrafter: generating consistent long depth sequences for open-world videos. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 2005ā2015. Cited by: Ā§1, Ā§2.2. [26] Y. Ji, Z. Chen, E. Xie, L. Hong, X. Liu, Z. Liu, T. Lu, Z. Li, and P. Luo (2023) Ddp: diffusion model for dense visual prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 21741ā21752. Cited by: Ā§2.2. [27] Z. Jiang, C. Zheng, I. Laina, D. Larlus, and A. Vedaldi (2025) Geo4d: leveraging video generators for geometric 4d scene reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 20658ā20671. Cited by: Ā§1, Ā§2.2. [28] T. Karras, M. Aittala, T. Aila, and S. Laine (2022) Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems 35, p. 26565ā26577. Cited by: Ā§2.2, Ā§3.1, Ā§3.1, Ā§3.2. [29] B. Ke, A. Obukhov, S. Huang, N. Metzger, R. C. Daudt, and K. Schindler (2024) Repurposing diffusion-based image generators for monocular depth estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 9492ā9502. Cited by: Ā§1, Ā§2.2, Ā§3.1, Ā§4, Ā§4, Table 1, Table 2, Table 5, Table 5. [30] B. Kerbl, G. Kopanas, T. LeimkĆ¼hler, and G. Drettakis (2023) 3D gaussian splatting for real-time radiance field rendering.. ACM Trans. Graph. 42 (4), p. 139ā1. Cited by: Ā§1. [31] T. Koch, L. Liebel, F. Fraundorfer, and M. Korner (2018) Evaluation of cnn-based single-image depth estimation methods. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, p. 0ā0. Cited by: Ā§4. [32] A. Krishnan, X. Yan, V. Casser, and A. Kundu (2025) Orchid: image latent diffusion for joint appearance and geometry generation. arXiv preprint arXiv:2501.13087. Cited by: Ā§1, Ā§2.2. [33] J. H. Lee, M. Han, D. W. Ko, and I. H. Suh (2019) From big to small: multi-scale local planar guidance for monocular depth estimation. arXiv preprint arXiv:1907.10326. Cited by: Ā§2.1. [34] B. Li, C. Shen, Y. Dai, A. Van Den Hengel, and M. He (2015) Depth and surface normal estimation from monocular images using regression on deep features and hierarchical crfs. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 1119ā1127. Cited by: Ā§2.1. [35] Z. Li and N. Snavely (2018) Megadepth: learning single-view depth prediction from internet photos. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 2041ā2050. Cited by: Ā§2.1. [36] Z. Li, Z. Chen, X. Liu, and J. Jiang (2023) Depthformer: exploiting long-range correlation and local information for accurate monocular depth estimation. Machine Intelligence Research 20 (6), p. 837ā854. Cited by: Ā§2.1. [37] H. Liu, H. Yang, E. M. Huijben, M. Schuiveling, R. Su, J. P. Pluim, and M. Veta (2025) Pathopainter: augmenting histopathology segmentation via tumor-aware inpainting. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 408ā417. Cited by: Ā§2.2. [38] B. Poole, A. Jain, J. T. Barron, and B. Mildenhall (2022) Dreamfusion: text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988. Cited by: Ā§2.2. [39] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. (2021) Learning transferable visual models from natural language supervision. In International conference on machine learning, p. 8748ā8763. Cited by: Ā§3.2. [40] R. Ranftl, A. Bochkovskiy, and V. Koltun (2021) Vision transformers for dense prediction. In Proceedings of the IEEE/CVF international conference on computer vision, p. 12179ā12188. Cited by: Table 1. [41] R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun (2020) Towards robust monocular depth estimation: mixing datasets for zero-shot cross-dataset transfer. IEEE transactions on pattern analysis and machine intelligence 44 (3), p. 1623ā1637. Cited by: Ā§2.1, Table 1. [42] M. Roberts, J. Ramapuram, A. Ranjan, A. Kumar, M. A. Bautista, N. Paczan, R. Webb, and J. M. Susskind (2021) Hypersim: a photorealistic synthetic dataset for holistic indoor scene understanding. In Proceedings of the IEEE/CVF international conference on computer vision, p. 10912ā10922. Cited by: Ā§1, Ā§4. [43] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 10684ā10695. Cited by: Ā§1, Ā§2.2, Ā§3.1, Ā§3.2, Ā§3.2. [44] T. Salimans and J. Ho (2022) Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512. Cited by: Ā§3.1. [45] S. Saxena, C. Herrmann, J. Hur, A. Kar, M. Norouzi, D. Sun, and D. J. Fleet (2023) The surprising effectiveness of diffusion models for optical flow and monocular depth estimation. Advances in Neural Information Processing Systems 36, p. 39443ā39469. Cited by: Ā§2.2. [46] S. Saxena, A. Kar, M. Norouzi, and D. J. Fleet (2023) Monocular depth estimation using diffusion models. arXiv preprint arXiv:2302.14816. Cited by: Ā§2.2. [47] T. Schops, J. L. Schonberger, S. Galliani, T. Sattler, K. Schindler, M. Pollefeys, and A. Geiger (2017) A multi-view stereo benchmark with high-resolution images and multi-camera videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 3260ā3269. Cited by: Ā§4. [48] J. Shao, Y. Yang, H. Zhou, Y. Zhang, Y. Shen, V. Guizilini, Y. Wang, M. Poggi, and Y. Liao (2025) Learning temporally consistent video depth from video diffusion priors. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 22841ā22852. Cited by: Ā§1, Ā§2.2. [49] N. Silberman, D. Hoiem, P. Kohli, and R. Fergus (2012) Indoor segmentation and support inference from rgbd images. In European conference on computer vision, p. 746ā760. Cited by: Ā§4. [50] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli (2015) Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, p. 2256ā2265. Cited by: Ā§3.1. [51] J. Song, C. Meng, and S. Ermon (2020) Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. Cited by: Ā§2.2, Ā§3.1. [52] Y. Sun, X. Yu, Z. Huang, Y. Huang, Y. Guo, Z. Yang, Y. Cao, and X. Qi (2025) Unigeo: taming video diffusion for unified consistent geometry estimation. arXiv preprint arXiv:2505.24521. Cited by: Ā§1. [53] I. Vasiljevic, N. Kolkin, S. Zhang, R. Luo, H. Wang, F. Z. Dai, A. F. Daniele, M. Mostajabi, S. Basart, M. R. Walter, et al. (2019) Diode: a dense indoor and outdoor depth dataset. arXiv preprint arXiv:1908.00463. Cited by: Ā§4. [54] T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, et al. (2025) Wan: open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. Cited by: Ā§4. [55] P. Wang, X. Shen, B. Russell, S. Cohen, B. Price, and A. L. Yuille (2016) Surge: surface regularized geometry estimation from a single image. Advances in Neural Information Processing Systems 29. Cited by: Ā§2.1. [56] D. Xu, W. Ouyang, X. Wang, and N. Sebe (2018) Pad-net: multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 675ā684. Cited by: Ā§2.1. [57] H. Yang, A. Bulat, I. Hadji, H. X. Pham, X. Zhu, G. Tzimiropoulos, and B. Martinez (2025) Fam diffusion: frequency and attention modulation for high-resolution image generation with stable diffusion. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 2459ā2468. Cited by: Ā§2.2. [58] H. Yang, C. Zhang, W. Wang, M. Volino, A. Hilton, L. Zhang, and X. Zhu (2025) Improving gaussian splatting with localized points management. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 21696ā21705. Cited by: Ā§1. [59] L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao (2024) Depth anything: unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 10371ā10381. Cited by: Ā§1, Ā§2.1, Table 1. [60] L. Yang, B. Kang, Z. Huang, Z. Zhao, X. Xu, J. Feng, and H. Zhao (2024) Depth anything v2. Advances in Neural Information Processing Systems 37, p. 21875ā21911. Cited by: Ā§1, Ā§2.1, Table 1. [61] Z. Yang, Z. Pan, Y. Yang, X. Zhu, and L. Zhang (2024) Driving scene synthesis on free-form trajectories with generative prior. arXiv preprint arXiv:2412.01717. Cited by: Ā§1. [62] C. Ye, L. Qiu, X. Gu, Q. Zuo, Y. Wu, Z. Dong, L. Bo, Y. Xiu, and X. Han (2024) Stablenormal: reducing diffusion variance for stable and sharp normal. ACM Transactions on Graphics (TOG) 43 (6), p. 1ā18. Cited by: Ā§2.2, Table 2. [63] Z. Yu, S. Peng, M. Niemeyer, T. Sattler, and A. Geiger (2022) Monosdf: exploring monocular geometric cues for neural implicit surface reconstruction. Advances in neural information processing systems 35, p. 25018ā25032. Cited by: Ā§1. [64] W. Yuan, X. Gu, Z. Dai, S. Zhu, and P. Tan (2022) New crfs: neural window fully-connected crfs for monocular depth estimation. arXiv preprint arXiv:2203.01502. Cited by: Ā§2.1. [65] Z. Zhang, Z. Cui, C. Xu, Y. Yan, N. Sebe, and J. Yang (2019) Pattern-affinitive propagation across depth, surface normal and semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 4106ā4115. Cited by: Ā§2.1, Ā§2.1. [66] W. Zhao, Y. Rao, Z. Liu, B. Liu, J. Zhou, and J. Lu (2023) Unleashing text-to-image diffusion models for visual perception. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 5729ā5739. Cited by: Ā§2.2.