Paper deep dive
ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction
Tim Engelbracht, René Zurbrügg, Mayank Mittal, Marco Hutter, Marc Pollefeys, Hermann Blum, Zuria Bauer
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/21/2026, 4:44:19 AM
Summary
The paper introduces ForceTwin, a system that creates physics-informed digital twins of articulated objects by analyzing instrumented human interaction. Using a handheld force-sensing gripper, the system captures synchronized poses and contact wrenches to estimate kinematics and dynamic parameters (inertia, friction, damping) along with a neural residual for state-dependent mechanism forces. This approach significantly improves robotic manipulation success rates compared to visual-language model priors or kinematics-only models.
Entities (10)
Relation Signals (9)
ForceTwin → estimates → inertia
confidence 95% · ...we estimate the articulation, parametric dynamics including inertia...
ForceTwin → estimates → viscous damping
confidence 95% · ...parametric dynamics including inertia, Coulomb friction, viscous damping...
ForceTwin → estimates → Coulomb friction
confidence 95% · ...parametric dynamics including inertia, Coulomb friction, viscous damping...
ForceTwin → improves → Goal-completion
confidence 95% · ForceTwin achieves 87% goal completion... compared with 60% using VLM-prior and 57% using kinematics-only twins
VLM prior → outperformedby → ForceTwin
confidence 95% · ForceTwin achieves 87% goal completion... compared with 60% using VLM-prior
ForceTwin → uses → Hoi! gripper
confidence 95% · We capture all interactions with the handheld Hoi! gripper... from which we estimate the articulation...
ForceTwin → captures → structured neural residual
confidence 90% · ...and a structured neural residual capturing state-dependent mechanism forces.
ForceTwin → deployson → Franka FR3
confidence 90% · As a feedforward dynamics model for impedance control on a Spot and a Franka FR3...
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Manipulating objects requires understanding not only their motion, but also the physical properties that determine it. For articulated objects, these include inertia, friction, and mechanisms such as springs or door closers, whose effects can vary with configuration and velocity. Such properties are not directly observable from appearance: visually identical doors may require very different effort to manipulate. Existing digital-twin pipelines recover primarily kinematics or assign static physical parameters from visual and language priors, which can yield physically implausible estimates. As a result, state-dependent mechanism dynamics remain unidentified and are not represented in standard asset formats. We present ForceTwin, a system for identifying physics-informed digital twins of articulated objects from instrumented human interaction. A person probes an object using a handheld force-sensing gripper, providing synchronized poses and interaction forces from which we estimate the articulation, parametric dynamics including inertia, Coulomb friction, viscous damping, and a structured neural residual capturing state-dependent mechanism forces. ForceTwin nearly halves the inertial-parameter error of a VLM prior. As a feedforward dynamics model for impedance control on a Spot and a Franka FR3, ForceTwin achieves 87% goal completion across nine object-embodiment pairs, compared with 60% using VLM-prior and 57% using kinematics-only twins, with the largest gains on objects whose strong mechanisms cause both baselines to stall. We further use the identified twins to train whole-body door-traversal policies and deploy them in the real world. Project Page: this https URL
Tags
Links
- Source: https://arxiv.org/abs/2609.21751v1
- Canonical: https://arxiv.org/abs/2609.21751v1
Trouble viewing inline? Open PDF directly →
Full Text
46,358 characters extracted from source content.
Expand or collapse full text
ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human InteractionThanks: †Corresponding author: tengelbracht@ethz.ch Tim Engelbracht René Zurbrüg Mayank Mittal Marco Hutter Marc Pollefeys Hermann Blum Affiliation: ETH Zurich NVIDIA Microsoft University of Bonn Zuria Bauer Abstract Manipulating objects requires understanding not only their motion, but also the physical properties that determine it. For articulated objects, these include inertia, friction, and mechanisms such as springs or door closers, whose effects can vary with configuration and velocity. Such properties are not directly observable from appearance: visually identical doors may require very different effort to manipulate. Existing digital-twin pipelines recover primarily kinematics or assign static physical parameters from visual and language priors, which can yield physically implausible estimates. As a result, state-dependent mechanism dynamics remain unidentified and are not represented in standard asset formats. We present ForceTwin, a system for identifying physics-informed digital twins of articulated objects from instrumented human interaction. A person probes an object using a handheld force-sensing gripper, providing synchronized poses and interaction forces from which we estimate the articulation, parametric dynamics including inertia, Coulomb friction, viscous damping, and a structured neural residual capturing state-dependent mechanism forces. ForceTwin nearly halves the inertial-parameter error of a VLM prior. As a feedforward dynamics model for impedance control on a Spot and a Franka FR3, ForceTwin achieves 87% goal completion across nine object-embodiment pairs, compared with 60% using VLM-prior and 57% using kinematics-only twins, with the largest gains on objects whose strong mechanisms cause both baselines to stall. We further use the identified twins to train whole-body door-traversal policies and deploy them in the real world. Project page: timengelbracht.github.io/forcetwin-website I Introduction Robotic manipulation requires more than knowing how an object can move. To open a drawer, a dishwasher, or a door with a closer, a robot must also account for the forces induced by inertia, friction, and internal mechanisms. These dynamics can vary substantially between visually similar objects and are often impossible to infer from appearance alone: the same drawer may be empty or heavily loaded, while a concealed closer can make an otherwise ordinary door strongly resist motion. A digital twin for manipulation must therefore capture not only an object’s geometry and articulation, but also the instance-specific dynamics that govern its interaction. These dynamics have to be measured. Appearance provides at best a prior, while motion alone identifies inertia, friction, and mechanism torque only up to a common scale that the contact wrench resolves. Feedback cannot simply replace the missing model: strong resistance may indicate that a drawer is stuck and the robot should stop, or be the expected behavior of a spring-loaded door that requires additional effort. Existing twin pipelines neither measure this effort nor model its state-dependent component. Reconstruction methods recover kinematic structure [1, 2, 3, 4], while recent methods assign physical parameters from visual and language priors [5, 6]. Robot-probing approaches [7, 8, 9] measure interaction, but recover at most constant friction models and require robot deployment. None identify the nonlinear, state-dependent response of mechanisms such as door closers or spring-loaded catches, which no set of constant parameters can describe, and standard asset formats provide no representation for this response. We take a different route (Fig. 1): a person measures the object. Handheld force-sensing grippers have made in-the-wild interaction capture practical [10, 11]; we repurpose them for system identification. With ForceTwin, a person probes an articulated object, and from the tool trajectory and contact wrenches we identify the object’s kinematics and dynamics (), decoupling identification from robot deployment. Humans can reach objects without a robot present, safely excite their full range across varied speeds, and do so faster than contact-based robot exploration. We model joint effort semi-parametrically. Inertia, Coulomb friction, and viscous damping are estimated under nonnegativity constraints, yielding physically valid parameters that map directly to simulators and controllers. A structured neural residual captures the remaining state-dependent mechanism response. The resulting twin can be used directly for model-based control or exported as a simulation asset. In summary, our contributions are: • We introduce ForceTwin, which identifies instance-specific articulated-object dynamics from instrumented human interaction, including nonlinear, state-dependent mechanism responses. • We formulate identification as a semi-parametric partially linear model, combining physically constrained inertia and friction parameters with a structured neural residual, yielding simulation-compatible parameters and a predictive model of joint effort. • We show that one identified twin serves both control and simulation. As the feedforward model of an impedance controller, it reaches 87% goal completion across nine object–embodiment pairs, where prior-based and kinematics-only twins reach 60% and 57% and stall entirely on objects with strong mechanisms; incorporated into simulation assets, the identified dynamics support policy learning, where we train whole-body door-traversal policies and deploy them in the real world. I Related Work Articulated Object Perception and Kinematics. Estimating articulation models by acting on the world is a classic instance of interactive perception [12]: a robot pushes an object and infers its kinematic model from the observed motion [7, 13]. Later work removes the interaction, estimating articulation from a single RGB-D frame [14], from two observed object states [15], with learned category-independent models [16], or from in-the-wild video [17], and scene-graph pipelines lift such estimates to scene scale for robotic manipulation [2, 4, 1, 18]. All of these methods estimate kinematic structure, i.e., joint types and axes, but not the dynamic parameters that govern interaction forces; our kinematics stage builds on this line of work, adopting the twist formulation of works such as [16, 19] and interactive BIC-based model selection of [13], but treats the kinematic model as the input to dynamic identification rather than the final result. Instrumented Interaction and Physical Model Identification. Closest to our setting are methods that identify physical parameters from forceful interaction. Door inertia and a velocity-dependent deceleration profile have been identified from robot interaction and released-door trajectories [20], and human opening forces on doors and drawers have been captured with an instrumented hook, yielding force-versus-angle profiles used for haptic recognition and monitoring [21]. We share their premise that dynamics must be measured, but differ in what is recovered and for what: our semiparametric formulation yields interpretable, simulation-ready parameters plus a nonlinear mechanism model, rather than a per-object regression, and our capture requires only a handheld gripper rather than a robot. Robot-mounted estimators also go beyond kinematics: coupled recursive filters over vision, wrist forces, and proprioception additionally estimate a quasi-static Coulomb friction model, i.e., stiction and constant kinetic friction, while deliberately neglecting inertial and viscous effects [8], and factor-graph estimators use force factors to disambiguate the articulation but identify no dynamic parameters [9]. Our identification retains all of these terms, inertia, Coulomb and viscous damping, under physical constraints, and adds the nonlinear mechanism response. Ditto [22] builds “digital twins of articulated objects from interaction,” but its twins carry default physical parameters, while the dynamics parameters are not identified. Finally, handheld instrumented grippers [23, 10, 11] have made in-the-wild interaction capture practical, so far primarily to record demonstrations for policy learning; we repurpose this hardware for system identification. Real-to-Sim and Simulation Asset Creation. A complementary line of work turns real scenes into simulation-ready articulated assets: from images alone [24], from a robot push [25], through human-in-the-loop annotation [26], or by retrieving similar “digital cousins” from an asset database [27]. In all four, the resulting physics is defaulted, annotated, or inherited from the retrieved asset. A parallel thread assigns physical parameters from visual and language priors [5, 6, 3], fits them to visual tracking [28], or tunes simulator distributions against real task rollouts [29]. None of these approaches measure the interaction forces of the specific instance being modeled. ForceTwin complements this line of work with per-instance dynamics, identified from force measurements and exported in a format such pipelines can consume. I Method I-A Problem Formulation We consider a rigid articulated part with a single degree of freedom and identify its kinematics and interaction dynamics from instrumented human probing. The observations consist of two synchronized streams, =TTCP,iW,ℱiWi=1N,D= \^WT_TCP,i,^WF_i \_i=1^N, (1) where TTCP,iW^WT_TCP,i is the pose of the tool center point (TCP) in a world frame W, and ℱiW∈ℝ6^WF_i ^6 is the contact wrench applied by the tool to the object after compensating for the weight and inertial loading of the instrumented gripper. We assume that the tool maintains a rigid, non-slip grasp and that the articulated part follows an ideal revolute or prismatic joint during the retained measurements. Unless marked otherwise, all quantities are expressed in W, and bold denotes a stacked multi-component quantity. From D, we estimate the articulation class κ∈revolute,prismaticκ∈\revolute,prismatic\, the joint twist W^W ξ, the observed range =[qmin,qmax]Q=[q_ ,q_ ], and a generalized-effort model τ^=τ^(q,q˙,q¨). τ= τ(q, q, q). We represent the latter by simulator-compatible effective parametric coefficients =[μ,b,I]⊤ β=[μ,b,I] and a structured mechanism residual ℳmech=gθ,cϕM_mech=\g_θ,c_φ\. Together, these quantities form the physical model ℳphys=(κ,W,,,ℳmech).M_phys= (κ,\,^W ξ,\,Q,\, β,\,M_mech ). (2) The model is estimated independently of a particular geometric reconstruction. A physics-informed digital twin is obtained afterwards by registering and associating ℳphysM_phys with a geometric asset or a robot’s object representation. Fig. 1: The ForceTwin pipeline. Instrumented human interaction provides synchronized tool poses and contact wrenches from which we identify an articulation model and its generalized-effort dynamics. Geometry is reconstructed from the same recording or supplied independently, and the physical model is registered to it to produce a twin for robot control or simulation. I-B Kinematics Estimation Under the rigid-grasp assumption, the tracked TCP positions pii=1N\p_i\_i=1^N follow the one-dimensional motion of the articulated part. We represent this motion using a screw in product-of-exponentials form [16, 13, 30], p¯i=exp([W]∧qi)p¯0, p_i= \! ([^W ξ] q_i ) p_0, (3) where p¯i∈ℝ4 p_i ^4 denotes the homogeneous TCP position, qiq_i is the scalar joint coordinate, and W=[ω⊤,v⊤]⊤∈ℝ6^W ξ=[ω ,v ] ^6 is the joint twist, and [⋅]∧[·] maps a twist to its 4×44× 4 matrix form in (3) se(3). We remove the scale ambiguity between W^W ξ and q by imposing ‖ω‖2=1\|ω\|_2=1 for revolute joints and ‖v‖2=1\|v\|_2=1 for prismatic joints. We set the coordinate origin at the first retained configuration and orient the axis so that the first sustained displacement has positive q. These conventions fix the units and signs of the subsequently identified dynamic parameters. A revolute candidate satisfies ω≠0ω≠ 0 and ω⊤v=0ω v=0, while a prismatic candidate satisfies ω=0ω=0. We fit both candidates and select the articulation class using the Bayesian information criterion [31, 13]. We then refine the selected model in a factor graph, similar to [9], jointly estimating the joint geometry and the per-sample states qi\q_i\. For revolute joints, the refinement enforces zero screw pitch. The observed extrema of the refined states define =[qmin,qmax]Q=[q_ ,q_ ]. I-C Dynamics Identification Wrench Compensation and Generalized Effort. The force-torque sensor measures both the contact wrench and the wrench generated by the instrumented gripper itself. We transform the measured wrench into the world frame and subtract the calibrated gravitational and inertial wrench of the gripper, yielding the estimated contact wrench ℱiW^WF_i applied by the tool to the object. We order spatial twists as angular followed by linear velocity and wrenches as moment followed by force. The kinematic estimate then defines the object Jacobian at the TCP, Jobj(qi)=[ω×(pi−o),ω]T,κ=revolute,[v, 0]T,κ=prismatic.J_obj(q_i)= cases bmatrixω×(p_i-o),\,ω bmatrix^T,&κ=revolute,\\[2.0pt] bmatrixv,\,0 bmatrix^T,&κ=prismatic. cases (4) where o is a point on the revolute axis, and JpJ_p denotes the linear block of JobjJ_obj, the three rows that map q˙ q to the TCP velocity. By virtual work, the generalized effort applied by the tool is τi=Jobj(qi)⊤ℱiW. _i=J_obj(q_i) ^WF_i. Under the rigid-grasp and ideal-joint assumptions, wrench components that perform no virtual work along the permitted articulation lie in the null space of Jobj⊤J_obj and do not contribute to τi _i. The projection does not remove errors caused by wrench calibration, joint-axis estimation, grasp compliance, or slip; these effects remain part of the measurement residual. We obtain q˙i q_i and q¨i q_i by differentiating a smoothed joint-state trajectory. Samples near joint limits, with negligible velocity, or with implausibly large differentiated accelerations are removed using excitation gates. Generalized-Effort Model. We define τi _i as the generalized effort applied to the object and model the effort required to realize the observed motion as τ=Iq¨+μsgn(q˙)+bq˙⏟effective parametric dynamics+gθ(q)+cϕ(q,|q˙|)q˙⏟structured mechanism residual+ε,τ= I q+μ\,sgn( q)+b q_effective parametric dynamics+ g_θ(q)+c_φ(q,| q|) q_structured mechanism residual+ , (5) where I denotes rotational inertia for revolute joints and effective mass for prismatic joints, μ is the Coulomb friction magnitude, b is viscous damping, and ε contains measurement noise and unmodeled effects. We write τmech(q,q˙)=gθ(q)+cϕ(q,|q˙|)q˙ _mech(q, q)=g_θ(q)+c_φ(q,| q|) q for the structured residual as a whole. The sign convention in Eq. 5 describes the effort that must be applied to move the object. Consequently, the compensating damping effort is +cϕq˙+c_φ q. The corresponding damping effort exerted by the object is −cϕq˙-c_φ q, whose mechanical power is Pdamp=−cϕ(q,|q˙|)q˙2≤0P_damp=-c_φ(q,| q|) q^2≤ 0 (6) when cϕ≥0c_φ≥ 0. The function gθ(q)g_θ(q) captures configuration-dependent generalized loads. For joints whose motion is orthogonal to gravity, this term primarily represents mechanisms such as door closers and spring-loaded elements. For a horizontal revolute axis, it may also contain gravitational torque unless gravity is compensated using an independently available mass and center-of-mass estimate. We therefore interpret gθg_θ as a configuration-dependent load rather than claiming that it uniquely isolates a mechanical spring. Residual Fitting. We first estimate the effective parametric coefficients using samples with sufficient motion and force excitation. For sample i, define zi=[sgn(q˙i)q˙iq¨i]⊤,=[μbI]⊤.z_i= bmatrixsgn( q_i)& q_i& q_i bmatrix , β= bmatrixμ&b&I bmatrix . (7) Stacking the retained regressors into Z gives ^=argmin≥0‖−Z‖22. β= β≥ 0argmin \| τ-Z β \|_2^2. (8) The elementwise nonnegativity constraint enforces nonnegative effective inertia, Coulomb friction, and viscous damping. These quantities correspond to properties commonly supported by rigid-body simulators, although their exact interpretation and implementation remain backend-dependent. We next fit the structured nonlinear model to the residual ri=τi−zi⊤^,r_i= _i-z_i β, (9) using r^i=τmech(qi,q˙i),cϕ≥0. r_i= _mech(q_i, q_i), c_φ≥ 0. The configuration-dependent head gθg_θ is sign-unconstrained and takes only q as input. The damping coefficient is produced by softplus outputs, with separate heads for positive and negative motion, allowing directional asymmetry while preserving nonnegative damping. The resulting model is partially linear, but its decomposition is not generally unique. In particular, a constant component of cϕc_φ is indistinguishable from the linear damping coefficient b, and configuration-dependent gravity is indistinguishable from a mechanical load represented by gθg_θ. Correlations between q, q˙ q, and q¨ q can introduce further coupling between the fitted terms. We therefore interpret β as effective simulator-compatible parameters and evaluate the predictive fidelity of the complete effort model. The structured residual and probing protocol regularize the decomposition, but do not make every component uniquely identifiable. We fit the model sequentially: nonnegative least squares followed by residual-network training. Training minimizes residual MSE using full-batch AdamW (learning rate and weight decay 10−310^-3), with a random 75/25 training/validation split, at most 1200 epochs, and early stopping after 120 epochs without validation improvement; we retain the best-validation weights for the mechanism term. I-D Twin Assembly The identified physical model is expressed in the probing frame W and joint coordinate defined above. To attach it to a target geometric representation with frame A, we first register the probing frame using TWA^AT_W. The joint twist transforms according to A=AdTWAW.^A ξ=Ad_^AT_W^W ξ. Rigid frame changes preserve the scalar joint coordinate and generalized effort. If the target asset instead uses a different coordinate convention qA=sq+qoff,s∈−1,1,q_A=sq+q_off, s∈\-1,1\, (10) we transform the valid range and reparameterize the mechanism model. For the complete mechanism effort τmech(q,q˙) _mech(q, q), the target convention uses τmechA(qA,q˙A)=sτmech(s(qA−qoff),sq˙A) _mech^A(q_A, q_A)=s\, _mech (s(q_A-q_off),\,s q_A ) . The nonnegative magnitudes I, μ, and b are unchanged under the sign reversal, while their associated signed efforts follow the transformed joint coordinate. We then associate the registered model with the corresponding articulated instance, as illustrated in Fig. 2. The parametric dynamics are mapped to the corresponding joint or actuator properties provided by the target backend. When the backend supports runtime joint-effort evaluation, the nonlinear model is evaluated directly. For backends that cannot execute the learned network, we approximate it with a backend-supported spring-damper representation during export. This conversion is distinct from physical identification and may introduce an additional approximation. In our experiments, the geometric representation is reconstructed from the same recording, so it already shares the probing frame. We associate the manipulated handle with the tracked TCP and use the reconstructed functional relation to identify the corresponding panel. Other geometric backends or existing assets require only a registration and an instance-association procedure. The resulting twin can be consumed either as a feedforward dynamics model for robot control or as an articulated simulation asset. Fig. 2: Construction of a physics-informed twin. (a) Reconstructed panel instance, associated handle, and registered joint axis. (b) Exported articulated twin. The physical model is transformed into the asset frame, reparameterized to match its joint-coordinate convention, and attached to the corresponding articulated instance. IV Experiments Evaluation proceeds along two axes. First, we ask whether the estimated parameters are accurate and whether that accuracy justifies the model’s complexity: Section IV-B measures fidelity against ground truth inertia measurements and real-world behavior, and Section IV-C investigates the contribution of each model component. Second, we investigate the utility of ForceTwin: Section IV-D deploys the twin as a feedforward dynamics model on two physical robots, and Section IV-E exports it as a simulation asset for policy learning. All experiments follow the capture protocol of Section IV-A. IV-A Capture Protocol and Implementation We capture all interactions with the handheld Hoi! gripper [10], which provides posed RGB-D observations, the tool trajectory from Project Aria MPS [32], and the contact wrench from a wrist-mounted force-torque sensor. Once a part is grasped, we open and close it over a range of speeds and accelerations chosen to excite the individual regressor axes: slow, low-acceleration motion for Coulomb friction, a wide speed spectrum for viscous damping, high accelerations for inertia, and passive following for the mechanism. Fig. 3(b) shows an example probing sequence. In our implementation, FunFact [18] reconstructs panels, handles and functional relations from the same recording and therefore already shares the probing frame; we identify the manipulated handle by proximity to the tracked TCP, use FunFact’s handle-to-panel relation to select the associated door or drawer panel, attach the recovered model to that pair, process the instance meshes, and export the articulated asset to USD. Because ℳphysM_phys is independent of the geometric representation, it can equally be registered onto an existing asset: the real-to-sim free-swing experiments of Section IV-B use the exported asset, while the policy-learning experiments of Section IV-E attach the identified dynamics and mechanism to the door asset of the training pipeline, leaving its geometry unchanged. In both cases the identified β is written directly to the joint’s inertia, friction and damping fields, so the parametric half of the twin is native to the simulator, while the mechanism term is evaluated per step as a state-dependent joint effort. An independently obtained representation has to be registered to the probing frame first. Fig. 3: Dynamics model fit for the wood door. (a) Measured generalized effort (gray) against the parametric model alone (dashed) and with the structured mechanism added (green); (b) the probing trajectory. The parametric model tracks the shape of the effort but is offset from it by a configuration-dependent amount, which the learned mechanism recovers. Shaded bands mark the probing regimes for inertia (yellow), viscous damping (purple), and the mechanism (blue). IV-B Estimation Fidelity We evaluate estimation accuracy at three levels: kinematic parameters, compared against annotated ground-truth axes on the real-world test rig; inertial parameters, whose ground truth can be independently measured (unlike friction parameters, for which no such reference exists); and finally aggregate system fidelity, where kinematic and dynamic terms are evaluated jointly. Kinematics Estimation. We evaluate articulation types and axes on our real-world test bench against ground truth annotated in a Leica RTC360 laser scan [10, 17]. After registering estimates to the scan, we measure axis-angle errors for both joint types and axis-position errors for revolute joints [17, 2]. We compare against MoMa-SG [2], which tracks object points during non-instrumented human interaction and is run on our interactions, and OPDMulti [14], which estimates articulation from visual input without interaction. Interaction-based methods outperform the visual baseline (Table I). OPDMulti estimates prismatic directions reasonably well (4.8∘4.8 ), but yields large revolute-axis errors (34.9∘34.9 ) and only 54.1%54.1\% revolute type accuracy; both interaction-based methods classify all joints correctly. Compared with MoMa-SG, our method slightly improves revolute-axis orientation (1.5∘1.5 vs. 1.7∘1.7 ) and halves position error (2323 vs. 4646 m). We attribute this improvement to directly measured tool poses, avoiding reliance on surface-point tracking susceptible to low texture and motion blur. TABLE I: Articulation Axis Estimation. Evaluated on the real-world test bench against 12 axes annotated in a Leica RTC360 laser scan. Metrics follow ArtiPoint [17]: θerr _err is the angle between axis lines, folded to [0,90]∘[0,90] , and dL2d_L_2 the distance between revolute axis lines along their common perpendicular. Errors accumulate only over predictions with the correct joint type. Values are means with the sample standard deviation in smaller type; †no deviation available. Bold indicates best performance, underline second best; tied entries are all marked best. Prismatic Revolute Type acc. [%] ↑ Method θerr _err [∘] ↓ θerr _err [∘] ↓ dL2d_L_2 [m] ↓ Pris. Rev. ForceTwin (Ours) 1.4 ± 0.2 1.5 ± 0.8 23 ± 26 100.0 100.0 MoMa-SG [2] 1.6 ± 0.8 1.7 ± 0.0 46 ± 20 100.0 100.0 OPDMulti [14] 4.8† 34.9 ± 29.1 143 ± 117 100.0 54.1 Dynamics and Mechanism Estimation. We next assess the fidelity of our estimated dynamics parameters against a prior-based paradigm such as [5]. We focus on the inertial parameters, since ground truth is readily obtainable for them. For prismatic joints this is the mass, measured with a scale. For revolute panels, the hinge inertia follows from mass and geometry as I=13mℓ2I= 13m ^2 under the assumption of constant density, while compact parts such as locks and handles are treated as point masses with I=mℓ2I=m ^2. Here ℓ is the distance from the hinge axis: the panel’s extent from the hinge in the first case, and the radial offset of the part in the second. No ground truth is available for the parametric friction parameters or for the nonlinear MLP. To still evaluate these terms, we assess the aggregate fit at the system level: the real-to-sim free-swing experiment below compares the closing behavior of the real object with that of its simulated twin. Fig. 3 shows a qualitative example of the fit. The parametric model tracks the measured effort qualitatively, but with a non-constant offset that reveals the mechanism it cannot represent. Fig. 4: Inertial-parameter fidelity on the four objects for which ground truth is available: estimate against ground truth on logarithmic axes, so an estimate on the diagonal is exact and vertical distance from it is the error. Marker shape distinguishes revolute (circle) from prismatic (square) joints, whose parameters carry the two units given on the axes. Nested shaded bands mark agreement to within ±25%± 25\% (inner) and to within a factor of two (outer). The macro mean of the absolute relative errors is 29.5%29.5\% for ForceTwin and 53.0%53.0\% for the VLM prior. Aggregated over all objects (Fig. 4), the interaction-based estimate is more accurate than the prior-based one (29.5%29.5\% vs. 53.0%53.0\% macro mean absolute relative error), and it is the better estimate on three of the four objects for which ground truth is available. Our accuracy improves systematically with the magnitude of the inertial parameter: the two heaviest parts are estimated to within 10.4%10.4\% and 13.7%13.7\%, whereas the lightest ones reach 63.8%63.8\% and 30.2%30.2\%. We attribute this to the signal-to-noise ratio of the measurement rather than to the estimator itself, as light parts excite only small interaction forces, so that the force-torque noise band occupies a larger fraction of the measured signal. Consistently, the sole case in which the prior-based baseline is more accurate is the smallest object in the set. The prior-based baseline underestimates the inertial parameter for every object, despite being provided with scale information and prompted to reason about material. The largest error occurs for the heavy drawer (83.5%83.5\%), whose effective mass is dominated by its contents, hidden state that cannot be inferred from appearance. Our estimator, in turn, overestimates on all four objects. The bias is consistent in sign, which suggests that residual unmodeled effort is partially absorbed into the inertial term during the parametric fit. This reflects a structural property of the decomposition rather than insufficient optimization: both the parametric terms and the mechanism response depend on the articulation velocity, so the split between them is only partially identifiable from torque observations, and the fit objective is predictive accuracy rather than parameter recovery. The downstream uses of the twin consume the full torque model, whose aggregate fidelity is assessed at system level by the free-swing experiment below. System-Level Fidelity: Real-to-Sim Free Swing. The preceding experiment evaluates individual parameters, but only the inertial ones admit ground truth. We therefore assess the twin at the system level, where inertia, friction and mechanism act jointly. We release the metal door and the wood door from 45∘45 and 90∘90 and compare the measured closing time against simulations parameterized either by our interaction-derived estimates or by the VLM-prior baseline. TABLE I: Real-Robot Articulation. Spot and Franka FR3 track robot-specific opening goals within 6 s using a fixed controller. ForceTwin uses interaction-identified dynamics; VLM prior uses visual and language estimates [5, 6]; Kinematics only omits dynamics compensation [2, 3]. Complete: maximum goal completion (capped at 100%). Track: goal-normalized tracking RMSE (%). Values are means with sample SD; the overall mean equally weights nine object–robot pairs. Mechanism rates the contribution of built-in mechanisms to articulation force. Objects are ordered by ForceTwin tracking error within each robot panel. Bold/underline: best/second best per object and metric, with ties marked best. ForceTwin (Ours) VLM prior Kinematics only Object Mechanism Complete ↑ Track ↓ Complete ↑ Track ↓ Complete ↑ Track ↓ Spot Drawer Weak 88.0 ± 1.8 9.2 ± 2.9 95.2 ± 4.0 34.1 ± 3.1 90.6 ± 3.7 38.0 ± 2.6 Sliding door None 100.0 ± 0.0 10.5 ± 1.9 100.0 ± 0.0 15.4 ± 1.9 100.0 ± 0.0 18.5 ± 2.6 Wood door Medium 100.0 ± 0.0 14.0 ± 1.5 91.2 ± 0.9 24.3 ± 0.0 88.3 ± 4.7 27.3 ± 1.6 Oven Strong 100.0 ± 0.0 33.3 ± 3.1 4.0 ± 0.7 91.1 ± 0.4 2.2 ± 0.1 91.9 ± 0.2 Metal door Strong N/A (object forces exceed Spot’s actuation limits) Franka Sliding door None 93.7 ± 0.7 10.5 ± 0.4 98.7 ± 1.0 17.2 ± 0.9 99.3 ± 1.0 26.9 ± 12.2 Drawer Weak 85.6 ± 9.6 17.6 ± 3.2 93.1 ± 3.2 37.0 ± 0.4 85.1 ± 11.2 22.4 ± 4.2 Metal door Strong 81.2 ± 3.5 24.4 ± 5.5 1.6 ± 0.0 91.0 ± 0.0 1.5 ± 0.0 91.0 ± 0.0 Wood door Medium 69.8 ± 19.4 30.3 ± 11.0 51.7 ± 4.0 47.7 ± 2.8 44.0 ± 21.3 56.0 ± 16.6 Oven Strong 67.6 ± 0.0 49.7 ± 0.0 1.8 ± 1.0 90.8 ± 0.9 0.9 ± 0.1 91.6 ± 0.1 Overall mean 87.3 22.2 59.7 49.8 56.9 51.5 TABLE I: Real-to-Sim Free-Swing Fidelity. Closing times of the physical door and of twins parameterized by our interaction-derived estimates and by the VLM prior, released from two initial opening angles q0q_0. Times are in seconds and the absolute relative error (ARE) is a percentage. The final row is the equal-weight macro mean. Only two estimates are compared, so bold marks the lower ARE and no second rank is reported. ForceTwin VLM prior Object q0q_0 [∘] Real [s] t [s] ARE ↓ t [s] ARE ↓ Metal door 45 5.6 5.9 5.4 2.1 62.5 Metal door 90 8.4 8.8 4.8 2.5 70.2 Wood door 45 3.7 5.0 35.1 2.0 45.9 Wood door 90 5.0 6.0 20.0 2.4 52.0 Macro mean – – – 16.3 – 57.7 Our estimates yield the lower error in all four conditions (Table I), reducing the macro mean from 57.7%57.7\% to 16.3%16.3\%. The VLM-prior baseline closes every door in roughly 22 s regardless of door or release angle, indicating that it does not match the damped dynamics closely. Note that this measure is aggregate and not injective: a large mismatch proves the dynamics are wrong, but a close match does not uniquely identify the individual parameters, since compensating errors could in principle yield the same closing time. It therefore complements the parameter-wise evaluation above. IV-C Model-Class Ablations We evaluate the benefit of added model complexity. Adding viscous damping to the Coulomb-inertia model reduces in-sample effort RMSE by up to 12%12\%. Adding the structured mechanism reduces it by a further 67−73%67-73\% on mechanism-dominated parts such as the doors and the oven, against 12%12\% on the sliding door, which has no mechanism. We fit the two stages sequentially, the parametric coefficients first and the mechanism on their residual. Alternating between them changes the fit RMSE by less than 5%5\% of the parametric baseline while redistributing effort between overlapping terms, so we do not iterate. IV-D Does Dynamics Fidelity Change Manipulation? Fig. 5: Real-world experiments of zero-shot model-based articulation (Table I) across two embodiments: Spot and Franka FR3. We visualize the initial and final articulation states as blended overlays for both embodiments. The recovered quasi-static mechanism torque for each object is plotted over the opening trajectory, highlighting the variation in mechanism behavior across objects. This experiment evaluates whether the fidelity of a twin’s dynamics influences manipulation performance: We use ForceTwin’s object dynamics as a feedforward term of a Cartesian impedance controller, cmd _cmd =Kx(xd−x)+Dx(x˙d−x˙)+f, =K_x(x_d-x)+D_x( x_d- x)+f_f, (11) f _f =(Jp(q)⊤)†[I^q¨d+μ^sgn(q˙d)+b^q˙d+τ^mech(qd,q˙d)]. = (J_p(q) ) [ I q_d+ μ\,sgn( q_d)+ b q_d+ τ_mech(q_d, q_d) ]. where the bracket sums the generalized effort of the identified inertia, Coulomb and viscous damping, and mechanism response along the reference articulation trajectory qd(t)q_d(t), and the transposed pseudoinverse of the object Jacobian’s linear block JpJ_p maps them to an end-effector force added to the impedance feedback. The twin enters only through ff_f, so we vary its source across the two prevalent twin-construction paradigms and ours, keeping robot, controller, and task fixed: kinematics-only twins [14, 3, 2, 33] provide JpJ_p but no dynamics (f=f_f=0), VLM-prior twins [5, 6] supply prior-based parameter estimates to the same feedforward, and ForceTwin supplies the parameters identified from instrumented interaction. In each trial, the robot starts with the handle grasped and tracks qd(t)q_d(t) from the closed configuration to an embodiment-specific goal within 6 s (Fig. 5); We report goal completion and goal-normalized tracking RMSE. Table I reports goal completion and tracking error on five objects across Spot and Franka FR3 under a common 6 s goal. ForceTwin yields the lowest tracking error on every object and the highest macro mean completion (87.3%87.3\%, against 59.7%59.7\% for the VLM prior and 56.9%56.9\% for kinematics only), and the margin scales with the dynamic load. On the drawers and sliding doors all three models complete the goal, and the identified model improves tracking error by factors of 1.31.3 to 44. On the strong-mechanism objects, the oven on both robots and the metal door on the Franka, both baselines stall below 5%5\% completion because their feedforward supplies only a fraction of the required force, while the identified model reaches 6767–100%100\%. The wood door lies between these regimes on both embodiments, where the baselines lose 99–2626 percentage points and roughly double the tracking error. IV-E The Twin as a Training Asset Beyond the feedforward evaluations above, we show that the identified models can be used directly for reinforcement learning. We register the identified wood- and metal-door dynamics, including the recovered nonlinear closing mechanism, onto the door asset in Isaac Lab [34]. Using the motion-imitation pipeline from [35], we train whole-body door-traversal policies. That pipeline randomizes the door parameters during training; here we replace that randomization with the identified dynamics, leaving the policy architecture and training procedure unchanged. We deploy both policies on an ANYmal quadruped with an arm and evaluate them on the two real doors. The wood-door policy succeeds on the wood door in all five trials. The metal-door policy also succeeds on this door, but exhibits a stop-and-go motion: it advances the door and then pauses while the imitation phase catches up (Fig. 6). On the metal door, the metal-door policy succeeds, whereas the wood-door policy fails its initial opening attempt. These experiments demonstrate that the identified models integrate into an existing policy-learning pipeline and support real-world deployment on contact-rich whole-body tasks. Whether training with identified dynamics consistently improves performance over domain randomization requires a controlled comparison, which we leave to future work. Fig. 6: Policy learning with identified door models. Left: We use the identified wood- and metal-door models (real-to-sim) to train whole-body policies, which we then deploy on ANYmal (sim-to-real). Right: Both policies on the real wood door; shaded regions show the average over five trials. The wood-door policy moves the end-effector smoothly, while the metal-door policy stops and starts, pulling hard and then pausing, which gives it higher jerk (bottom) due to the worse sim-to-real calibration. V Limitations Our decomposition of the effort model is not unique: a constant mechanism damping is absorbed by b, and any configuration-dependent load by gθg_θ, including gravity when the joint axis is not vertical. The model is memoryless in (q,q˙)(q, q), so hysteresis, backlash, stiction at rest, and latch states lie outside its class, and both gθg_θ and the joint range hold only where the object was probed. Further, identification is per instance and needs physical interaction: each part is probed individually, and the probing has to excite the individual regressor axes, so passive demonstration recordings do not suffice. Priors scale across a scene where probing does not, so the two are complementary, and transferring identified dynamics to parts that were not probed is left to future work. Identification is also one-shot: a loaded drawer or a readjusted closer changes the object, and refining a twin online from the robot’s own interaction forces remains open. VI Conclusion ForceTwin identifies physics-informed digital twins of articulated objects from instrumented human interaction. A person probes the object with a handheld force-sensing gripper, and from the tool trajectory and contact wrenches we recover the articulation, effective inertia, Coulomb and viscous terms under nonnegativity constraints, and a residual for the nonlinear mechanism. The identified parameters nearly halve the inertial error of a prior-based estimate and reproduce measured door-closing times. A single twin then serves two uses: as the feedforward model of an impedance controller, it operates objects on which kinematics-only and prior-based twins stall, and exported to simulation it is used to train whole-body door-traversal policies transferring to the real objects. ACKNOWLEDGEMENTS This work was supported by SNSF Advanced Grant 216260, the Lamarr Institute, and Google. The authors used generative AI to assist with drafting and editing the manuscript, cleaning up figures and coding; all content was reviewed by the authors, who take full responsibility for it. References [1] Q. Gu et al., “ArtiSG: Functional 3D Scene Graph Construction via Human-demonstrated Articulated Objects Manipulation,” arXiv:2512.24845, 2026. [2] M. Büchner et al., “Articulated 3D Scene Graphs for Open-World Mobile Manipulation,” arXiv:2602.16356, 2026. [3] Z. Huang et al., “REACT3D: Recovering Articulations for Interactive Physical 3D Scenes,” IEEE RA-L, 2026. [4] A. Yu et al., “Pandora: Articulated 3D Scene Graphs from Egocentric Vision,” arXiv:2603.28732, 2026. [5] D. Iliash et al., “Artiverse: A Diverse and Physically Grounded Dataset for Articulated Objects,” in CVPR, 2026. [6] H. Li et al., “PhysGraph: A Physics-aware 3D Scene Graph for Perception and Reasoning,” arXiv:2606.08655, 2026. [7] D. Katz and O. Brock, “Manipulating Articulated Objects with Interactive Perception,” in ICRA, 2008. [8] R. Martín-Martín and O. Brock, “Coupled Recursive Estimation for Online Interactive Perception of Articulated Objects,” IJRR, 2022. [9] R. Buchanan et al., “Online Estimation and Manipulation of Articulated Objects,” Auton. Robots, 2026. [10] T. Engelbracht et al., “Hoi! - A Multimodal Dataset for Force-Grounded, Cross-View Articulated Manipulation,” in CVPR, 2026. [11] H. Choi et al., “In-the-Wild Compliant Manipulation with UMI-FT,” arXiv:2601.09988, 2026. [12] J. Bohg et al., “Interactive Perception: Leveraging Action in Perception and Perception in Action,” IEEE T-RO, 2017. [13] J. Sturm et al., “A Probabilistic Framework for Learning Kinematic Models of Articulated Objects,” JAIR, 2011. [14] X. Sun et al., “OPDMulti: Openable Part Detection for Multiple Objects,” arXiv:2303.14087, 2023. [15] J. Liu et al., “PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objects,” arXiv:2308.07391, 2023. [16] A. Jain et al., “ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory,” arXiv:2008.10518, 2021. [17] A. Werby et al., “Articulated Object Estimation in the Wild,” arXiv:2509.01708, 2025. [18] Z. Fu et al., “FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning,” arXiv:2604.03696, 2026. [19] R. Buchanan et al., “Online Estimation of Articulated Objects with Factor Graphs using Vision and Proprioceptive Sensing,” arXiv:2309.16343, 2023. [20] F. Endres et al., “Learning the Dynamics of Doors for Robotic Manipulation,” in IROS, 2013. [21] A. Jain et al., “Improving Robot Manipulation with Data-Driven Object-Centric Models of Everyday Forces,” Auton. Robots, 2013. [22] Z. Jiang et al., “Ditto: Building Digital Twins of Articulated Objects from Interaction,” in CVPR, 2022. [23] C. Chi et al., “Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots,” arXiv:2402.10329, 2024. [24] Q. Chen et al., “URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Images,” in RSS, 2024. [25] L. Ma et al., “Sim2Real2: Actively Building Explicit Physics Model for Precise Articulated Object Manipulation,” in ICRA, 2023. [26] M. Villasevil et al., “Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation,” in RSS, 2024. [27] T. Dai et al., “Automated Creation of Digital Cousins for Robust Policy Learning,” arXiv:2410.07408, 2024. [28] H. Jiang et al., “PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos,” in ICCV, 2025. [29] Y. Chebotar et al., “Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience,” in ICRA, 2019. [30] K. M. Lynch and F. C. Park, Modern Robotics: Mechanics, Planning, and Control. Cambridge University Press, 2017. [31] G. Schwarz, “Estimating the Dimension of a Model,” The Annals of Statistics, 1978. [32] J. Engel et al., “Project Aria: A New Tool for Egocentric Multi-Modal AI Research,” arXiv:2308.13561, 2023. [33] A. Delitzas et al., “FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videos,” in CVPR, 2026. [34] M. Mittal et al., “Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning,” arXiv:2511.04831, 2025. [35] J.-P. Sleiman et al., “Guided Reinforcement Learning for Robust Multi-Contact Loco-Manipulation,” arXiv:2410.13817, 2024.