Paper deep dive
Learning Implicit Constitutive Laws for Dynamic 3D Gaussian Splatting from Monocular Videos
Xiaoyang Liu, Kai Han
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/25/2026, 8:12:57 AM
Summary
The paper introduces GCA (Gaussian Constitutive Alignment), a framework for learning implicit constitutive laws for dynamic 3D Gaussian Splatting from monocular videos. It addresses geometric and material ambiguities in single-view settings by combining LoRA-based adaptation with two modules: Rank-based Depth-Geometric Anchors (RDGA) for robust geometric constraints via scale-invariant depth alignment, and a Constitutive Prior Regularizer (CPR) that uses classical constitutive models as soft differentiable priors. GCA outperforms existing methods, achieving 48% lower Chamfer Distance on synthetic benchmarks.
Entities (9)
Relation Signals (7)
GCA → learns → implicit constitutive laws
confidence 95% · a framework for learning implicit constitutive laws from monocular dynamic video of deformable objects represented by 3D Gaussians
GCA → uses → RDGA
confidence 95% · our framework unifies LoRA-based adaptation with two key alignment modules. First, we propose Rank-based Depth-Geometric Anchors (RDGA)
GCA → uses → CPR
confidence 95% · Second, a Constitutive Prior Regularizer (CPR) integrates classical constitutive models as soft differentiable priors
GCA → uses → LoRA
confidence 92% · our framework unifies LoRA-based adaptation with two key alignment modules
CPR → addresses → material ambiguity
confidence 90% · CPR integrates classical constitutive models as soft differentiable priors... resolving material ambiguity without sacrificing generalization
RDGA → addresses → Geometric Ambiguity
confidence 90% · RDGA establish robust geometric constraints from monocular dynamic observations via scale-invariant rank-based depth alignment
GCA → outperforms → NeuMA
confidence 90% · GCA outperforms existing methods, achieving 48% lower Chamfer Distance than the strongest baseline [NeuMA] on synthetic benchmarks
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We present GCA (Gaussian Constitutive Alignment), a framework for learning implicit constitutive laws from monocular dynamic video of deformable objects represented by 3D Gaussians. Given a static multi-view scan for geometric initialization, our method learns intrinsic physical dynamics solely from a single fixed-viewpoint video of the moving object. Existing implicit methods often suffer from local minima under noisy supervision and lack physical interpretability, while explicit approaches rely on predefined constitutive equations, limiting generalizability and becoming unstable in monocular settings. To address these challenges, our framework unifies LoRA-based adaptation with two key alignment modules. First, we propose Rank-based Depth-Geometric Anchors (RDGA) to establish robust geometric constraints from monocular dynamic observations via scale-invariant rank-based depth alignment, reducing the reliance on unreliable pixel-level color supervision. Second, a Constitutive Prior Regularizer (CPR) integrates classical constitutive models as soft differentiable priors, regularizing the optimization while preserving the flexibility of implicit modeling---even when the actual material is absent from the hypotheses. Extensive experiments on synthetic, real-to-sim, and real-world datasets demonstrate that GCA outperforms existing methods, achieving 48% lower Chamfer Distance than the strongest baseline on synthetic benchmarks while remaining robust under monocular supervision.
Tags
Links
- Source: https://arxiv.org/abs/2608.22102v1
- Canonical: https://arxiv.org/abs/2608.22102v1
Trouble viewing inline? Open PDF directly →
Full Text
44,330 characters extracted from source content.
Expand or collapse full text
Learning Implicit Constitutive Laws for Dynamic 3D Gaussian Splatting from Monocular Videos Xiaoyang Liu Affiliation: Visual AI Lab, The University of Hong Kong, Hong Kong Kai Han Thanks: Corresponding author. Affiliation: Visual AI Lab, The University of Hong Kong, Hong Kong Affiliation: E-mail xiaoyangliu@connect.hku.hk, kaihanx@hku.hk Abstract We present GCA (Gaussian Constitutive Alignment), a framework for learning implicit constitutive laws from monocular dynamic video of deformable objects represented by 3D Gaussians. Given a static multi-view scan for geometric initialization, our method learns intrinsic physical dynamics solely from a single fixed-viewpoint video of the moving object. Existing implicit methods often suffer from local minima under noisy supervision and lack physical interpretability, while explicit approaches rely on predefined constitutive equations, limiting generalizability and becoming unstable in monocular settings. To address these challenges, our framework unifies LoRA-based adaptation with two key alignment modules. First, we propose Rank-based Depth-Geometric Anchors (RDGA) to establish robust geometric constraints from monocular dynamic observations via scale-invariant rank-based depth alignment, reducing the reliance on unreliable pixel-level color supervision. Second, a Constitutive Prior Regularizer (CPR) integrates classical constitutive models as soft differentiable priors, regularizing the optimization while preserving the flexibility of implicit modeling—even when the actual material is absent from the hypotheses. Extensive experiments on synthetic, real-to-sim, and real-world datasets demonstrate that GCA outperforms existing methods, achieving 48% lower Chamfer Distance than the strongest baseline on synthetic benchmarks while remaining robust under monocular supervision. Keywords: Constitutive laws Differentiable physics Monocular video Visual-physical alignment Figure 1: The core idea of GCA. Learning implicit constitutive laws from dynamic video faces two key challenges: geometric ambiguity from sparse visual observations (left), and material ambiguity from unknown constitutive models and parameters (right). GCA bridges both sides through visual–physical bidirectional alignment: Rank-based Depth-Geometric Anchors (RDGA) establish robust geometric constraints via scale-invariant depth consensus, while a Constitutive Prior Regularizer (CPR) softly guides optimization using classical constitutive hypotheses. Together, they enable reliable learning of implicit constitutive laws (bottom) that generalize across diverse material types. 1 Introduction Understanding the intrinsic dynamics of objects is crucial for spatial intelligence, enabling accurate digital modeling, interaction, and manipulation that follows physical laws [53, 26, 3, 40]. While humans effortlessly infer basic physical properties from videos (e.g., bouncing balls or viscous fluid flow), extracting precise physical models from visual signals remains an open challenge. Prior works have employed models to understand intrinsic dynamics from visual observations [8, 25, 23, 24, 35]. A common approach combines differentiable physics simulators [13, 51] with differentiable renderers [38, 28] for optimization. Regarding constitutive law modeling, existing approaches fall into two paradigms with complementary limitations: Explicit modeling [24, 34, 30, 55] builds upon classical continuum mechanics with predefined constitutive models [12, 39] (e.g., hyperelastic models [46]) and explicit parameters such as Young’s modulus and Poisson’s ratio. While enabling physical interpretability, effectiveness critically depends on correct model specification: (1) manual model selection and fine parameter tuning are required for different materials [30]; (2) complex real-world materials with deeply coupled properties remain intractable [34]; and critically, (3) these methods perform poorly under monocular supervision, as observed in our experiments (Table 5). Implicit parameterization models constitutive relations through neural networks [37, 29]. NeuMA [7] introduces the first method to align implicit constitutive models with visual observations. However, implicit models easily converge to suboptimal solutions when handling noisy and sparse supervision [49]—a problem that becomes severe in monocular settings. These conflicting trade-offs lead to our core question: How to reliably learn intrinsic dynamics from monocular video while preserving the generalization advantages of implicit modeling? Following prior work in dynamic 3D Gaussian splatting [56, 7], we assume access to a static multi-view orbital scan for initial 3D geometry reconstruction. The focus of this work is to learn intrinsic constitutive laws from a single fixed-viewpoint video of the object undergoing dynamic deformation. This monocular dynamic setting introduces severe geometric ambiguities (single viewpoint) and material ambiguities (underconstrained physical parameters) that make the inverse problem significantly ill-posed. To address this challenge, we propose GCA, a framework that achieves visual-physical bidirectional alignment for learning implicit constitutive laws from monocular dynamic video. Our method unifies LoRA-based adaptation with two coordinated alignment modules: (i) Rank-based Depth-Geometric Anchors (RDGA) extract robust geometric constraints from sparse observations using a scale-invariant rank-based depth alignment mechanism. Unlike pixel-level color supervision, which suffers from domain shifts in monocular video, RDGA is designed to mitigate the scale-shift ambiguity of monocular depth estimators. (i) Constitutive Prior Regularizer (CPR) treats classical constitutive models as soft regularization priors rather than hard constraints, resolving material ambiguity without sacrificing generalization, and remains effective even when the true material is absent from the hypotheses. Extensive experiments validate that GCA achieves state-of-the-art performance: 48% lower Chamfer Distance than NeuMA on synthetic benchmarks, robust performance on a challenging real-to-sim dataset, and superior visual quality in real-world monocular experiments. We also show that naively adapting explicit methods to monocular settings leads to optimization divergence, highlighting the importance of robust alignment. 2 Related work Physics-grounded dynamic 3D generation. NeRF-based models [41, 15, 27, 17] are constrained by predefined material assumptions, while recent Gaussian-based methods [28, 16, 48] show substantial progress; SpringGaus [56] reconstructs elastic dynamics but still relies on an explicit spring-mass model. Diffusion-guided approaches [55, 34, 24, 32] inherit imprecise physics priors [10, 42, 31] and typically assume rigid [35] or elastic [56] bodies. NeuMA [7] first optimizes neural constitutive laws from observational images without predefined laws, but single-modality visual optimization suffers from local minima under sparse monocular supervision; naively augmenting explicit methods (e.g., PAC-NeRF [30]) with monocular depth diverges, motivating our hybrid alignment. Material constitutive laws. Conventional approaches [2, 39, 9, 5, 34] enforce explicit laws via predefined nonlinear polynomial bases (e.g., elastic [18]/plastic [12]/fluid [9] models) and tune parameters such as Young’s modulus or Poisson’s ratio, requiring manual specification [33]. Implicit neural modeling [43, 6, 36] bypasses this: NCLaw [37] pioneers hybrid N–PDE training but requires particle-level annotations; NeuMA [7] avoids particle ground truth via LoRA-based [21] alignment with differentiable rendering, yet pure visual supervision lacks physical interpretability [1] and sparse observations introduce optimization ambiguity. Our framework softly regularizes implicit optimization with physical priors, unifying the stability of explicit laws with the generalization of implicit ones under sparse supervision. 3 Method 3.1 Problem statement Given static 3D Gaussians [28] of an object (i)=(i),α(i),(i),(i)G(i)=\p(i),α(i),A(i),c(i)\, where (i),α(i),(i),(i)p(i),α(i),A(i),c(i) are the center, opacity, covariance matrix, and spherical harmonic coefficients of each Gaussian primitive, and its corresponding monocular dynamic video Itt=1T\I_t\_t=1^T, we aim to learn implicit constitutive laws through a dynamical system ℳθM_θ governed by elastoplastic dynamics [19]: ρ0ϕ¨=∇⋅+ρ0,=ℰ(e),e=∇ϕ, _0 φ=∇·P+ _0b, =E(F_e), _e=∇φ, (1) where P is the first Piola–Kirchhoff stress tensor, ρ0 _0 is the object density, and b is the body force. ϕφ denotes the deformation map, ϕ¨ φ is its acceleration, and ℰE is the elastic constitutive law. We discretize Eq. 1 and obtain the dynamical system ℳθM_θ: t+1=ℳθ(t),∀t=0,1,…,T−1,s_t+1=M_θ(s_t), ∀\,t=0,1,…,T-1, (2) where t=t,t,ets_t=\x_t,v_t,F_e^t\ denotes the state at time step t, consisting of particle positions, velocities, and elastic deformation gradients. θ denotes the neural parameters in ℳM. Additional details on preprocessing Gaussians to particles are provided in the supplementary material. To align ℳθM_θ with observation ItI_t, we use a differentiable 3DGS renderer ℛR producing I^t=ℛ(t,,) I_t=R(s_t;K,Q), where ,K,Q denote camera intrinsic and extrinsic matrices. Relying solely on this rendering-based supervision, however, is insufficient to overcome the challenges posed by sparse and noisy monocular video. The inherent geometric ambiguities from the single viewpoint and material ambiguities in the dynamics make the optimization landscape intractable. To establish a robust learning pipeline, we propose GCA (see Fig. 2), which combines LoRA-based adaptation with two coordinated alignment modules—Rank-based Depth-Geometric Anchors (RDGA) and a Constitutive Prior Regularizer (CPR)—detailed below. Figure 2: Overview of GCA. Our framework builds on Low-Rank Adaptation (LoRA) to fine-tune neural material laws while maintaining integration with PDE-based simulation, and introduces two coordinated alignment modules: (i) Rank-based Depth-Geometric Anchors (RDGA) that resolve geometric ambiguities via scale-invariant rank-based depth alignment, and (i) a Constitutive Prior Regularizer (CPR) that provides soft physical priors while preserving generalization. 3.2 Neural material constitutive laws Our work adopts the same dynamical system ℳθM_θ as NCLaw [37] for state transitions. ℳθM_θ is composed of the neural elasticity law ℰθeE_ _e, semi-implicit Euler integration [22, 47], and neural plasticity law θpP_ _p. We use the basic physical prior model ℳ0=ℰ0,0M_0=\E_0,P_0\ provided by NCLaw. To align the model with observations without compromising its fundamental capabilities, we use LoRA [21] for fine-tuning instead of training all parameters. Specifically, ℳθ=ℰθe,θpM_θ=\E_ _e,P_ _p\, where ℰθe=ℰ0+ΔℰθeE_ _e=E_0+ _ _e and θp=0+ΔθpP_ _p=P_0+ _ _p. Remark. While our dynamical system builds upon the NCLaw architecture [37], NCLaw itself requires dense particle-level ground-truth supervision, which is unavailable in visual observation settings. The core challenge addressed by GCA is not the design of neural constitutive architectures, but the visual-physical alignment problem: how to provide reliable supervisory signals to train such architectures from noisy, sparse, single-viewpoint video. As shown in Table 5, naively adapting existing methods to monocular settings leads to optimization instability, highlighting the importance of robust alignment. Moreover, as demonstrated in Table 4, LoRA not only improves parameter efficiency but also acts as a regularizer—full fine-tuning tends to overfit visual noise, yielding worse physical accuracy. Algorithm 1 Time stepping 0: State t=t,t,ets_t=\x_t,v_t,F_e^t\ 0: Next state t+1s_t+1 Stress Evaluation: for each material point i=1i=1 to N do t(i)←ℰθe(et,(i),θe)P_t^(i) _ _e(F_e^t,(i); _e) end for Euler Integration: t+1,t+1,etrial←ℐ(t,t,t)x_t+1,v_t+1,F_e^trial (x_t,v_t,P_t) Plasticity Update: for each material point i=1i=1 to N do et+1,(i)←θp(etrial,(i),θp)F_e^t+1,(i) _ _p(F_e^trial,(i); _p) end for The dynamical system ℳθM_θ advances physical states through three stages as shown in Algorithm 1: (1) Stress evaluation via neural constitutive law ℰθeE_ _e that computes first Piola–Kirchhoff stress tP_t from elastic deformation gradient etF_e^t; (2) Dynamics integration where operator ℐI implements semi-implicit Euler integration to update positions t+1x_t+1 and velocities t+1v_t+1; (3) Plasticity update through network θpP_ _p that modifies etrialF_e^trial to account for plastic deformation. 3.3 Rank-based depth-geometric anchors Pixel-level color matching, adopted by methods like NeuMA [7], struggles under monocular video due to lighting variations, self-occlusions during rapid rotation, and geometric ambiguities. To overcome this, we propose Rank-based Depth-Geometric Anchors (RDGA), which establish robust geometric constraints by enforcing rank-based consensus between rendered depth and estimated depth, focusing on stable interior regions. Design rationale. Monocular depth estimators suffer from inherent scale and shift ambiguity—absolute depth values are unreliable across frames. Standard metric losses (L1/L2) amplify this noise, as validated in Table 3 where replacing our rank-based formulation with L1 loss degrades performance below the baseline. Our Spearman rank correlation is scale-invariant: it only requires that relative depth orderings are preserved, which modern estimators achieve reliably even under large deformations. Given a rendered image Irender∈ℝH×W×3I_render ^H× W× 3, a rendered depth map Drender∈ℝH×WD_render ^H× W, and a ground-truth (GT) image Igt∈ℝH×W×3I_gt ^H× W× 3, we first compute a silhouette consistency loss: ℒmask=‖Mrender−Mgt‖22,L_mask=\|M_render-M_gt\|_2^2, (3) where MrenderM_render and MgtM_gt are object region masks on rendered and GT images. We utilize a pre-trained depth estimation network D (e.g., DAV [52]) to generate relative depth maps DgtD_gt. A feature matching network ℱF (e.g., SuperGlue [44, 11]) extracts N 2D correspondence pairs (pi,qi)i=1N\(p_i,q_i)\_i=1^N, where pi=(xi,yi)p_i=(x_i,y_i) and qi=(xi′,yi′)q_i=(x _i,y _i) denote matched coordinates in IrenderI_render and IgtI_gt. To mitigate depth estimation errors near object boundaries, we define the edge region ℳedgeM_edge through morphological opening: ℳedge=ℳ∘Kr×r,M_edge=M K_r× r, (4) where ∘ denotes morphological opening with an r×r× r rectangular kernel K. The stable interior region is ℳstable=ℳ∖ℳedgeM_stable=M _edge. For each correspondence pair (pi,qi)(p_i,q_i), we validate depth consensus in local neighborhoods. Let NpiN_p_i and NqiN_q_i represent k×k× k regions centered at pip_i in DrenderD_render and qiq_i in DgtD_gt. We compute the Spearman rank correlation coefficient γi _i [45]: γi=1−6∑j=1k2(rj−sj)2k2(k4−1), _i=1- 6 _j=1^k^2(r_j-s_j)^2k^2(k^4-1), (5) where rjr_j and sjs_j are the ranks of the j-th depth value in NpiN_p_i and NqiN_q_i. A consensus indicator function filters pairs: ϕ(pi,qi)=(γi>τ).φ(p_i,q_i)=I( _i>τ). (6) Only pairs in ℳstableM_stable satisfying ϕ(pi,qi)=1φ(p_i,q_i)=1 are retained: =(pi,qi)∣pi∈ℳstable∧ϕ(pi,qi)=1A=\(p_i,q_i) p_i _stable φ(p_i,q_i)=1\. The geometric alignment objective combines global and anchor-level terms: ℒgeo=λ1‖Prender−Pgt‖2⏟global alignment+λ2∑(pi,qi)∈(−γi)⏟anchor-level supervision,L_geo= _1 \|P_render-P_gt \|_2_global alignment+ _2 _(p_i,q_i) (- _i)_anchor-level supervision, (7) where PrenderP_render and PgtP_gt represent matched point sets. The first term maintains global geometric consistency while the second focuses on precise local relationships through verified anchor pairs. 3.4 Constitutive prior regularizer To resolve material ambiguity in sparse-view settings, we design a constitutive prior regularization process that evaluates candidate constitutive laws through parameter stability analysis. Constitutive hypotheses. Following NCLaw [37], our elastic hypothesis set ℋeH_e includes four models: (1) corotated elasticity, (2) St. Venant–Kirchhoff (StVK) elasticity, (3) volume elasticity, and (4) sigma elasticity. The plastic hypothesis set ℋpH_p also includes four models: (1) identity plasticity, (2) sigma plasticity, (3) von Mises plasticity [39], and (4) Drucker–Prager plasticity [12]. These candidates cover common elastic, volumetric, and plastic response families. Full formulations are provided in the supplementary material. Importantly, CPR acts as a soft regularization prior—it remains effective even when the true material behavior does not match any of the hypotheses, as validated in Table 6. The elastic deformation gradient et∈ℝN×3×3F_e^t ^N× 3× 3 serves as input, with physics residuals ℛe,ℛpR_e,R_p quantifying deviation from plausible laws. For each candidate law, we use et[,:,:]F_e^t[S,:,:] to evaluate alignment, where ⊂1,…,NS⊂\1,…,N\ is a fixed subset of indices (||=256|S|=256 by default). The Constitutive Prior Regularizer (Algorithm 2) operates through three phases. First, it performs standard elastic-plastic simulation: elastic stress prediction via ℰθeE_ _e, Euler integration through ℐI, and plasticity correction using θpP_ _p. Next, it solves inverse problems to estimate explicit parameters Θ for each candidate law. The credibility weights ω are computed as inverse variance measures (with smoothing factor ϵε), assigning higher confidence to laws with stable parameter estimates. Finally, physical residuals ℛeR_e and ℛpR_p penalize deviations using weighted combinations. Algorithm 2 Constitutive prior regularizer 0: et∈ℝN×3×3F_e^t ^N× 3× 3, ℋe=ℋekk=1KH_e=\H_e^k\_k=1^K, ℋp=ℋpmm=1MH_p=\H_p^m\_m=1^M, ℰθeE_ _e, θpP_ _p, ϵε, ⊂1,…,NS⊂\1,…,N\ et+1F_e^t+1, ℛetR_e^t, ℛptR_p^t Elastic Stress Prediction: t←ℰθe(et)P^t _ _e(F_e^t) Euler Integration: etrial←ℐ(t)F_e^trial (P^t) Plasticity Correction: et+1←θp(etrial)F_e^t+1 _ _p(F_e^trial) Parameter Solving: et,←et[,:,:]F_e^t,S _e^t[S,:,:]; etrial,←etrial[,:,:]F_e^trial,S _e^trial[S,:,:] for k=1k=1 to K do Θ^ek←argminΘek‖ℰθe(et,)−ℋek(et,,Θek)‖F2 _e^k← _ _e^k\|E_ _e(F_e^t,S)-H^k_e(F_e^t,S; _e^k)\|_F^2 ωek←1VarΘ^ek+ϵ _e^k← 1Var\ _e^k\+ε end for for m=1m=1 to M do Θ^pm←argminΘpm‖θp(etrial,)−ℋpm(etrial,,Θpm)‖F2 _p^m← _ _p^m\|P_ _p(F_e^trial,S)-H^m_p(F_e^trial,S; _p^m)\|_F^2 ωpm←1VarΘ^pm+ϵ _p^m← 1Var\ _p^m\+ε end for ℛet←∑k=1Kωek‖ℰθe(et)−ℋek(et,[Θ^ek])‖F2R_e^t← _k=1^K _e^k\|E_ _e(F_e^t)-H_e^k(F_e^t;E[ _e^k])\|_F^2 ℛpt←∑m=1Mωpm‖θp(etrial)−ℋpm(etrial,[Θ^pm])‖F2R_p^t← _m=1^M _p^m\|P_ _p(F_e^trial)-H_p^m(F_e^trial;E[ _p^m])\|_F^2 Overall optimization objective. The total loss for optimizing the neural elastic model ℰθeE_ _e and neural plasticity model θpP_ _p is ℒ=λmℒmask+λgℒgeo+ℛe+ℛp,L= _mL_mask+ _gL_geo+R_e+R_p, (8) where λm _m and λg _g are balance factors. A theoretical analysis of the convergence properties is provided in the supplementary material. 4 Experiments 4.1 Experimental setup Datasets. We conduct systematic validation across three dimensions. For synthetic experiments, we construct a challenging benchmark of six material types (elastomers, gels, rubber, plasticine, granular materials, and non-Newtonian fluids) across diverse object geometries; the benchmark introduces (1) randomized lighting interference, (2) reduced frame rates matching real-world constraints, and (3) compound material modeling, with additional details provided in the supplementary material. For the real-to-sim bridge, we capture high-quality 3D Gaussian splatting models of real objects (dragon, wolf, pudding) and simulate dynamic sequences with complex material properties, providing ground-truth physics while retaining real-world geometric complexity. For real-world validation, we adopt the SpringGaus [56] dataset; while its original setup employs tri-view supervision, we strictly constrain our approach to single-view video, consistent with our problem setting. All experiments run on a single NVIDIA A800 80GB GPU. Baseline methods. We evaluate against: (1) NCLaw [37] (implicit, requires particle GT); (2) NeuMA [7] (implicit, visual supervision); (3) SpringGaus [56] (explicit, real-world); and (4) PAC-NeRF [30] (explicit, multi-view), which we additionally adapt to monocular settings by augmenting with monocular depth supervision. We exclude PhysDreamer [55] and Physics3D [34] as their reliance on diffusion guidance and predefined explicit models is orthogonal to our implicit learning objective. Evaluation metrics. We use: (1) Chamfer Distance (CD) [4, 14] for geometric consistency, (2) SSIM [50] for structural similarity, (3) PSNR [20] for pixel-level accuracy, and (4) LPIPS [54] for perceptual similarity. 4.2 Evaluation on synthetic dataset We evaluate physical simulation accuracy by computing Chamfer Distance between predicted and ground-truth particle positions. Table 1 shows that under our challenging benchmark with color inconsistency and sparse supervision, our method achieves 48% lower average CD than NeuMA. Table 1: Quantitative comparison on synthetic dataset (Chamfer Distance ↓ ). Our method achieves 48% lower average CD than NeuMA across diverse materials. Material Elastomer Gel Rubber Plasticine Granular Non-Newt. Object Ball Duck Pawn Cat Fish Bottle Average NCLaw [37] 4.085 2.934 2.031 1.909 0.536 1.631 2.188 NeuMA [7] 1.123 1.863 0.517 0.844 0.322 1.056 0.954 Ours 0.922 0.702 0.200 0.318 0.077 0.757 0.496 Figure 3 illustrates temporal CD variations during simulation. Our method maintains alignment with ground truth throughout the simulation, while baselines gradually deviate. Further rendering metrics are provided in Fig. 4. Figure 3: Chamfer Distance during physical simulation. Our method consistently maintains lower CD throughout the simulation. Figure 4: Rendering metrics on synthetic dataset. Our method achieves superior PSNR, SSIM, and LPIPS. 4.3 Evaluation on real-to-sim dataset We assess generalization to complex geometries derived from real objects using our real-to-sim dataset. Table 2 shows GCA consistently outperforms baselines. Figure 5 provides qualitative comparison, demonstrating superior ability to capture plausible dynamics for complex objects. Figure 5: Qualitative comparison on real-to-sim dataset. Our method accurately captures complex dynamics for real-world geometries. Table 2: Chamfer Distance on real-to-sim dataset (↓ ). Our method achieves the lowest error across all objects. Object Plast. Dragon Sand Wolf Gel Pudding NCLaw [37] 25.021 49.420 38.149 NeuMA [7] 3.527 9.803 13.804 Ours 2.081 5.842 0.906 4.4 Evaluation on real-world dataset For real-world validation, we conduct monocular supervision experiments on the SpringGaus dataset [56]. While the original setup uses tri-view supervision, we strictly use single-view video. As shown in Fig. 6, GCA successfully disentangles implicit physical properties and demonstrates strong generalization under monocular constraints. Figure 6: Qualitative comparison on real-world dataset. Our method achieves better visual consistency using only monocular supervision. The PSNR and LPIPS values are computed after filtering the background. 4.5 Ablation and analysis We conduct comprehensive ablation studies to validate each component and design choice. Component-wise ablation. Table 3 provides a granular breakdown of each component’s contribution. Table 3: Detailed ablation study (Chamfer Distance ↓ ). Each component contributes meaningfully. Replacing rank-based depth alignment with L1 loss degrades performance below the baseline, validating our scale-invariant design. Configuration Elast. Gel Rubber Plast. Gran. Non-Newt. Note NeuMA (Baseline) 1.123 1.863 0.517 0.844 0.322 1.056 Prior best w/o Global Align. >100>100 (Diverged) Divergence w/o Anchor Superv. 1.024 0.963 0.637 0.510 0.164 1.166 Loss of detail Replace Rank w/ L1 1.064 1.098 0.846 0.919 0.147 0.989 Scale ambiguity w/o RDGA 3.842 3.045 1.975 1.821 0.559 1.531 No geom. anchor w/o CPR 1.024 0.694 0.243 0.510 0.084 0.769 No physics prior Ours (Full) 0.922 0.702 0.200 0.318 0.077 0.757 Best performance (1) Global alignment is foundational: removing it causes complete optimization divergence, as the model cannot capture overall motion trends. (2) Rank-based >> Metric-based: replacing our rank-based loss with standard L1 depth loss yields CD of 0.844 (avg.), significantly worse than our full model (0.496) and showing degraded performance on 5 of 6 materials compared to our rank-based formulation. This is because monocular depth estimators suffer from scale-shift ambiguity—forcing absolute scale match introduces noise rather than valid supervision. Our rank-based consensus mitigates this issue. (3) RDGA is critical: without it, performance degrades sharply across all materials. (4) CPR provides consistent benefit: the regularizer improves performance on 5 of 6 materials. The single exception (Gel: 0.694 vs. 0.702) is within noise margins and reflects that simple elastic materials may not require additional physics guidance. (5) Color supervision is unreliable under monocular shading. In monocular videos of dynamic objects, self-occlusions and rapid rotation cause severe lighting changes and shading artifacts. Geometric consistency via depth provides a more invariant signal for learning physical dynamics. Computational efficiency. Table 4 shows LoRA reduces GPU memory by 60% and training time by 87% versus full fine-tuning, while achieving superior CD (0.50 vs. 1.35). Full fine-tuning overfits visual noise. Table 4: Computational efficiency analysis. LoRA fine-tuning achieves superior physical accuracy with lower cost. Method Mem. (GB) Train (min) Infer. (s) ↓ CD ↓ NeuMA [7] 28.9 73.3 32.4 1.31 Ours (Full F.T.) 76.9 408.6 32.5 1.35 Ours (LoRA) 30.5 51.2 31.5 0.50 Explicit methods are unstable under monocular supervision. A natural question is whether existing explicit methods can be adapted to monocular settings by adding depth supervision. Table 5 provides empirical evidence: PAC-NeRF augmented with monocular depth becomes unstable and diverges in our monocular setting (CD >100>100). The rigid parameterization of explicit models (e.g., Young’s modulus) makes them overly sensitive to geometric noise inherent in monocular depth estimates. These results suggest that directly adding monocular depth supervision to existing explicit frameworks is insufficient—the specific design of RDGA (rank-based, edge-aware) and CPR (soft regularization) is essential. Table 5: Comparison with explicit methods under monocular setting (CD ↓ ). Explicit methods diverge even with depth supervision; GCA remains robust. Method Supervision Ball Cat Status PAC-NeRF [30] Multi-view 0.85 0.29 Reference NeuMA [7] Multi-view 0.98 0.35 Reference PAC-NeRF + Depth Monocular >>100 >>100 Diverged NeuMA [7] Monocular 1.12 0.84 Suboptimal Ours Monocular 0.92 0.32 Robust Direct law-level validation. Beyond trajectory-level alignment, we further evaluate whether the learned implicit constitutive law matches the underlying physical response. Using an analytic JellyDuck law as ground truth, we query the learned elasticity under held-out deformation gradients that are not used for image supervision. As shown in Fig. 7, GCA follows the ground-truth response more closely than NeuMA and reduces the average constitutive response error from 39.6% to 8.0%. This indicates that the improvement is not merely due to better trajectory fitting, but also to more accurate law-level recovery. Method Uni. Sh. Avg. NeuMA [7] 52.0 27.1 39.6 Ours 12.0 3.9 8.0 Response error (%) ↓ Figure 7: Direct law-level validation on held-out deformation gradients. Left: normalized constitutive response under analytic JellyDuck ground truth. Right: quantitative response error. GCA more accurately recovers the underlying constitutive response than NeuMA, reducing the average error from 39.6% to 8.0%. Robustness to out-of-distribution materials. A critical concern is whether CPR becomes counterproductive when the true material is absent from the hypotheses. Table 6 addresses this with two stress tests: (1) a blind test where the ground-truth constitutive model is deliberately removed from the library, and (2) composite materials combining elasticity, plasticity, and fluid properties—behaviors not covered by any single hypothesis. In both cases, GCA still substantially outperforms the baseline, confirming that CPR provides general physical guidance (e.g., energy consistency, stability constraints) rather than requiring exact template matches. The implicit LoRA layers successfully compensate for residual differences between the prior and actual material behavior. Table 6: Robustness to out-of-distribution materials (CD ↓ ). GCA outperforms baselines even when the correct hypothesis is absent or for composite materials. Configuration Elastomer Plasticine Composite (target excl.) (target excl.) (undefined) NeuMA [7] 1.123 0.844 3.628 Ours (Target Removed) 0.976 0.397 – Ours (Full Library) 0.922 0.318 1.275 Generalization. The learned constitutive laws are not tied to the specific geometry used during training: once acquired, they can be applied to novel object shapes and further support multi-object interaction scenarios where different objects follow different learned material laws. Additional qualitative results are provided in the supplementary material. 5 Conclusion We presented GCA, a framework for learning implicit constitutive laws from monocular dynamic video through visual-physical bidirectional alignment. By unifying LoRA-based adaptation with Rank-based Depth-Geometric Anchors (RDGA) and a Constitutive Prior Regularizer (CPR), our method achieves 48% lower Chamfer Distance than the strongest baseline on synthetic data, strong generalization on real-to-sim datasets, and superior quality in real-world monocular experiments—while remaining robust even when the true material is absent from the hypotheses. Limitations. GCA still requires a static multi-view scan for geometric initialization, and extending the entire pipeline to fully monocular reconstruction is left for future work. Its geometric anchors also depend on the quality of monocular depth estimation, although the rank-based formulation mitigates scale-shift ambiguity. Finally, our current experiments focus mainly on single-object dynamics; extending the framework to dense multi-object scenes with heterogeneous materials is a promising direction. Acknowledgements. This work is supported by Hong Kong Research Grants Council – General Research Fund (Grant No. 17213825), Hong Kong Innovation and Technology Commission – Innovation and Technology Fund (Grant No. ITS/488/24FP), and HKU Seed Fund for PI Research. References [1] L. S. Aira, A. Montanaro, E. Aiello, D. Valsesia, and E. Magli (2024) MotionCraft: physics-based zero-shot video generation. arXiv preprint arXiv:2405.13557. Cited by: §2. [2] E. M. Arruda and M. C. Boyce (1993) A three-dimensional constitutive model for the large stretch behavior of rubber elastic materials. Journal of the Mechanics and Physics of Solids. Cited by: §2. [3] A. Billard and D. Kragic (2019) Trends and challenges in robot manipulation. Science. Cited by: §1. [4] M. A. Butt and P. Maragos (1998) Optimum design of chamfer distance transforms. IEEE TIP. Cited by: §4.1. [5] J. Cai, Y. Yang, W. Yuan, Y. He, Z. Dong, L. Bo, H. Cheng, and Q. Chen (2024) Gaussian-informed continuum for physical property identification and simulation. In NeurIPS, Cited by: §2. [6] S. Cai, Z. Mao, Z. Wang, M. Yin, and G. E. Karniadakis (2021) Physics-informed neural networks (pinns) for fluid mechanics: a review. Acta Mechanica Sinica. Cited by: §2. [7] J. Cao, S. Guan, Y. Ge, W. Li, X. Yang, and C. Ma (2024) NeuMA: neural material adaptor for visual grounding of intrinsic dynamics. In NeurIPS, Cited by: §1, §1, §2, §2, §3.3, Figure 7, §4.1, Table 1, Table 2, Table 4, Table 5, Table 5, Table 6. [8] H. Chen, E. Tretschk, T. Stuyck, P. Kadlecek, L. Kavan, E. Vouga, and C. Lassner (2022) Virtual elastic objects. In CVPR, Cited by: §1. [9] R. P. Chhabra and S. A. Patel (2023) Bubbles, drops, and particles in non-newtonian fluids. CRC press. Cited by: §2. [10] F. Croitoru, V. Hondru, R. T. Ionescu, and M. Shah (2023) Diffusion models in vision: a survey. IEEE TPAMI. Cited by: §2. [11] D. DeTone, T. Malisiewicz, and A. Rabinovich (2018) Superpoint: self-supervised interest point detection and description. In CVPR, Cited by: §3.3. [12] D. C. Drucker and W. Prager (1952) Soil mechanics and plastic analysis or limit design. Quarterly of applied mathematics. Cited by: §1, §2, §3.4. [13] M. Dubied, M. Y. Michelis, A. Spielberg, and R. K. Katzschmann (2022) Sim-to-real for soft robots using differentiable fem: recipes for meshing, damping, and actuation. IEEE Robotics and Automation Letters. Cited by: §1. [14] P. Erler, P. Guerrero, S. Ohrhallinger, N. J. Mitra, and M. Wimmer (2020) Points2surf learning implicit surfaces from point clouds. In ECCV, Cited by: §4.1. [15] J. Fang, T. Yi, X. Wang, L. Xie, X. Zhang, W. Liu, M. Nießner, and Q. Tian (2022) Fast dynamic radiance fields with time-aware neural voxels. In SIGGRAPH Asia, Cited by: §2. [16] Y. Feng, X. Feng, Y. Shang, Y. Jiang, C. Yu, Z. Zong, T. Shao, H. Wu, K. Zhou, C. Jiang, and Y. Yang (2024) Gaussian splashing: unified particles for versatile motion synthesis and rendering. arXiv preprint arXiv:2401.15318. Cited by: §2. [17] Y. Feng, Y. Shang, X. Li, T. Shao, C. Jiang, and Y. Yang (2024) Pie-nerf: physics-based interactive elastodynamics with nerf. In CVPR, Cited by: §2. [18] Y. Fung (1967) Elasticity of soft tissues in simple elongation. American Journal of Physiology-Legacy Content. Cited by: §2. [19] Y. Fung (1977) A first course in continuum mechanics. Englewood Cliffs. Cited by: §3.1. [20] A. Hore and D. Ziou (2010) Image quality metrics: psnr vs. ssim. In Int. Conf. Pattern Recog., Cited by: §4.1. [21] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, et al. (2022) LoRA: low-rank adaptation of large language models. In ICLR, Cited by: §2, §3.2. [22] Y. Hu, Y. Fang, Z. Ge, Z. Qu, Y. Zhu, A. Pradhana, and C. Jiang (2018) A moving least squares material point method with displacement discontinuity and two-way rigid body coupling. ACM TOG. Cited by: §3.2. [23] D. Z. Huang, K. Xu, C. Farhat, and E. Darve (2020) Learning constitutive relations from indirect observations using deep neural networks. Journal of Computational Physics. Cited by: §1. [24] T. Huang, Y. Zeng, H. Li, W. Zuo, and R. W. Lau (2024) DreamPhysics: learning physical properties of dynamic 3D gaussians with video diffusion priors. arXiv preprint arXiv:2406.01476. Cited by: §1, §2. [25] Y. Jiang, C. Yu, T. Xie, X. Li, Y. Feng, H. Wang, M. Li, H. Lau, F. Gao, Y. Yang, et al. (2024) Vr-gs: a physical dynamics-aware interactive gaussian splatting system in virtual reality. In SIGGRAPH, Cited by: §1. [26] M. G. Juarez, V. J. Botti, and A. S. Giret (2021) Digital twins: review and challenges. Journal of Computing and Information Science in Engineering. Cited by: §1. [27] T. Kaneko (2024) Improving physics-augmented continuum neural radiance field-based geometry-agnostic system identification with lagrangian particle optimization. In CVPR, Cited by: §2. [28] B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis (2023) 3D gaussian splatting for real-time radiance field rendering. ACM TOG. Cited by: §1, §2, §3.1. [29] X. Li, Y. Cao, M. Li, Y. Yang, C. Schroeder, and C. Jiang (2022) Plasticitynet: learning to simulate metal, sand, and snow for optimization time integration. In NeurIPS, Cited by: §1. [30] X. Li, Y. Qiao, P. Y. Chen, K. M. Jatavallabhula, M. Lin, C. Jiang, and C. Gan (2023) PAC-NeRF: physics-augmented continuum neural radiance fields for geometry-agnostic system identification. In ICLR, Cited by: §1, §2, §4.1, Table 5. [31] Z. Li, R. Tucker, N. Snavely, and A. Holynski (2024) Generative image dynamics. In CVPR, Cited by: §2. [32] Y. Lin, C. Lin, J. Xu, and Y. MU (2025) OmniPhysGS: 3D constitutive gaussians for general physics-based dynamics generation. In ICLR, Cited by: §2. [33] D. Liu, J. Zhang, A. Dinh, E. Park, S. Zhang, and C. Xu (2025) Generative physical ai in vision: a survey. arXiv preprint arXiv:2501.10928. Cited by: §2. [34] F. Liu, H. Wang, S. Yao, S. Zhang, J. Zhou, and Y. Duan (2024) Physics3D: learning physical properties of 3D Gaussians via video diffusion. arXiv preprint arXiv:2406.04338. Cited by: §1, §2, §2, §4.1. [35] S. Liu, Z. Ren, S. Gupta, and S. Wang (2024) PhysGen: rigid-body physics-grounded image-to-video generation. In ECCV, Cited by: §1, §2. [36] L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis (2021) Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence. Cited by: §2. [37] P. Ma, P. Y. Chen, B. Deng, J. B. Tenenbaum, T. Du, C. Gan, and W. Matusik (2023) Learning neural constitutive laws from motion observations for generalizable PDE dynamics. In ICML, Cited by: §1, §2, §3.2, §3.2, §3.4, §4.1, Table 1, Table 2. [38] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng (2021) NeRF: representing scenes as neural radiance fields for view synthesis. Communications of the ACM. Cited by: §1. [39] R. v. Mises (1913) Mechanik der festen körper im plastisch-deformablen zustand. Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, Mathematisch-Physikalische Klasse. Cited by: §1, §2, §3.4. [40] S. Nair, A. Rajeswaran, V. Kumar, C. Finn, and A. Gupta (2022) R3m: a universal visual representation for robot manipulation. arXiv preprint arXiv:2203.12601. Cited by: §1. [41] K. Park, U. Sinha, J. T. Barron, S. Bouaziz, D. B. Goldman, S. M. Seitz, and R. Martin-Brualla (2021) Nerfies: deformable neural radiance fields. In ICCV, Cited by: §2. [42] B. Poole, A. Jain, J. T. Barron, and B. Mildenhall (2023) DreamFusion: text-to-3D using 2D diffusion. In ICLR, Cited by: §2. [43] M. Raissi, P. Perdikaris, and G. E. Karniadakis (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics. Cited by: §2. [44] P. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich (2020) Superglue: learning feature matching with graph neural networks. In CVPR, Cited by: §3.3. [45] P. Sedgwick (2014) Spearman’s rank correlation coefficient. Bmj. Cited by: §3.3. [46] A. Stomakhin, R. Howes, C. A. Schroeder, and J. M. Teran (2012) Energetically consistent invertible elasticity. In Symposium on Computer Animation, Cited by: §1. [47] D. Sulsky, S. Zhou, and H. L. Schreyer (1995) Application of a particle-in-cell method to solid mechanics. Computer physics communications. Cited by: §3.2. [48] X. Tan, Y. Jiang, X. Li, Z. Zong, T. Xie, Y. Yang, and C. Jiang (2024) PhysMotion: physics-grounded dynamics from a single image. arXiv preprint arXiv:2411.17189. Cited by: §2. [49] T. Wang, P. Ma, A. E. Spielberg, Z. Xian, H. Zhang, J. B. Tenenbaum, D. Rus, and C. Gan (2023) Softzoo: a soft robot co-design benchmark for locomotion in diverse environments. arXiv preprint arXiv:2303.09555. Cited by: §1. [50] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli (2004) Image quality assessment: from error visibility to structural similarity. IEEE TIP. Cited by: §4.1. [51] T. Xue, S. Liao, Z. Gan, C. Park, X. Xie, W. K. Liu, and J. Cao (2023) JAX-FEM: a differentiable GPU-accelerated 3D finite element solver for automatic inverse design and mechanistic data science. Computer Physics Communications. Cited by: §1. [52] H. Yang, D. Huang, W. Yin, C. Shen, H. Liu, X. He, B. Lin, W. Ouyang, and T. He (2024) Depth any video with scalable synthetic data. arXiv preprint arXiv:2410.10815. Cited by: §3.3. [53] H. Yin, A. Varava, and D. Kragic (2021) Modeling, learning, perception, and control methods for deformable object manipulation. Science Robotics. Cited by: §1. [54] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang (2018) The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, Cited by: §4.1. [55] T. Zhang, H. Yu, R. Wu, B. Y. Feng, C. Zheng, N. Snavely, J. Wu, and W. T. Freeman (2024) PhysDreamer: physics-based interaction with 3D objects via video generation. In ECCV, Cited by: §1, §2, §4.1. [56] L. Zhong, H. Yu, J. Wu, and Y. Li (2024) Reconstruction and simulation of elastic objects with spring-mass 3D gaussians. In ECCV, Cited by: §1, §2, §4.1, §4.1, §4.4.