Paper deep dive
Articulated 3D Scene Graphs for Open-World Mobile Manipulation
Martin Büchner, Adrian Röfer, Tim Engelbracht, Tim Welschehold, Zuria Bauer, Hermann Blum, Marc Pollefeys, Abhinav Valada
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 7/21/2026, 1:48:15 AM
Summary
The paper introduces MoMa-SG, a framework for constructing semantic-kinematic 3D scene graphs of articulated scenes from RGB-D sequences. It utilizes occlusion-robust point tracking and a novel unified twist estimation formulation to identify revolute and prismatic joint parameters. The system associates objects with articulations and detects parent-child relationships to enable robust mobile manipulation of articulated objects in open-world environments. The authors also release the Arti4D-Semantic dataset.
Entities (8)
Relation Signals (8)
MoMa-SG → uses → Twist Estimation
confidence 95% · estimate articulation models using a novel unified twist estimation formulation
MoMa-SG → uses → Point Tracking
confidence 95% · infer object motion using occlusion-robust point tracking
MoMa-SG → enables → Mobile Manipulation
confidence 90% · MoMa-SG enables the construction of accurate 3D scene graphs... and serves as a backbone for long-horizon mobile manipulation.
MoMa-SG → evaluatedon → Arti4D-Semantic
confidence 90% · We extensively evaluate the performance of MoMa-SG on two datasets
Point Tracking → uses → CoTracker3
confidence 90% · We employ CoTracker3 [22] to obtain point tracks
Interaction Discovery → uses → YOLOv9
confidence 90% · we extract an interaction mask Ht using a pre-trained YOLOv9 [49] model.
MoMa-SG → testedon → Quadruped
confidence 85% · real-world experiments on both a quadruped and a mobile manipulator demonstrate
MoMa-SG → testedon → Mobile Manipulator
confidence 85% · real-world experiments on both a quadruped and a mobile manipulator demonstrate
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Semantics has enabled 3D scene understanding and affordance-driven object interaction. However, robots operating in real-world environments face a critical limitation: they cannot anticipate how objects move. Long-horizon mobile manipulation requires closing the gap between semantics, geometry, and kinematics. In this work, we present MoMa-SG, a novel framework for building semantic-kinematic 3D scene graphs of articulated scenes containing a myriad of interactable objects. Given RGB-D sequences containing multiple object articulations, we temporally segment object interactions and infer object motion using occlusion-robust point tracking. We then lift point trajectories into 3D and estimate articulation models using a novel unified twist estimation formulation that robustly estimates revolute and prismatic joint parameters in a single optimization pass. Next, we associate objects with estimated articulations and detect contained objects by reasoning over parent-child relations at identified opening states. We also introduce the novel Arti4D-Semantic dataset, which uniquely combines hierarchical object semantics including parent-child relation labels with object axis annotations across 62 in-the-wild RGB-D sequences containing 600 object interactions and three distinct observation paradigms. We extensively evaluate the performance of MoMa-SG on two datasets and ablate key design choices of our approach. In addition, real-world experiments on both a quadruped and a mobile manipulator demonstrate that our semantic-kinematic scene graphs enable robust manipulation of articulated objects in everyday home environments. We provide code and data at: this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2602.16356v1
- Canonical: https://arxiv.org/abs/2602.16356v1
Trouble viewing inline? Open PDF directly →
Full Text
105,348 characters extracted from source content.
Expand or collapse full text
Articulated 3D Scene Graphs for Open-World Mobile Manipulation Martin Büchner1 Adrian Röfer1 Tim Engelbracht2 Tim Welschehold1 Zuria Bauer2 Hermann Blum2,3 Marc Pollefeys2 Abhinav Valada1 Abstract Semantics has enabled 3D scene understanding and affordance-driven object interaction. However, robots operating in real-world environments face a critical limitation: they cannot anticipate how objects move. Long-horizon mobile manipulation requires closing the gap between semantics, geometry, and kinematics. In this work, we present MoMa-SG, a novel framework for building semantic-kinematic 3D scene graphs of articulated scenes containing a myriad of interactable objects. Given RGB-D sequences containing multiple object articulations, we temporally segment object interactions and infer object motion using occlusion-robust point tracking. We then lift point trajectories into 3D and estimate articulation models using a novel unified twist estimation formulation that robustly estimates revolute and prismatic joint parameters in a single optimization pass. Next, we associate objects with estimated articulations and detect contained objects by reasoning over parent-child relations at identified opening states. We also introduce the novel Arti4D-Semantic dataset, which uniquely combines hierarchical object semantics including parent-child relation labels with object axis annotations across 62 in-the-wild RGB-D sequences containing 600 object interactions and three distinct observation paradigms. We extensively evaluate the performance of MoMa-SG on two datasets and ablate key design choices of our approach. In addition, real-world experiments on both a quadruped and a mobile manipulator demonstrate that our semantic-kinematic scene graphs enable robust manipulation of articulated objects in everyday home environments. We provide code and data at: https://momasg.cs.uni-freiburg.de. I Introduction Figure 1: MoMa-SG enables the construction of accurate 3D scene graphs over articulated scenes and serves as a backbone for long-horizon mobile manipulation. Traditional approaches in robotic mapping focus on building metrically accurate representations of quasi-static environments. Recently, 3D scene graphs have proposed sparse abstractions of dense maps that allow structured reasoning over objects, their spatial relationships, and higher-level scene layouts [2, 17, 11]. Beyond purely metric-semantic topological representations, 3D scene graphs have further been equipped with open-vocabulary semantic understanding [12, 51, 26, 30], enabling planning from language-specified goals [14]. Furthermore, these representations increasingly encode functional and affordance-level information, capturing how objects can be interacted with [57, 41, 8]. While the aforementioned methods provide a comprehensive manifold for reasoning, planning, and navigation, mobile manipulation tasks require a certain level of kinematic understanding of the surroundings. Especially, since a significant number of objects in human environments are containers, such as cabinets and drawers, which require an in-depth understanding of their kinematic behavior in order to enable compliant manipulation. As such, anticipating how articulated objects move makes a useful prior, However, only observing articulated object interactions provides us with a model that governs their motion. A large number of previous works [23, 50, 4, 13] have studied the problem of estimating object articulation models, however often limited to synthetic data, fiducial markers, or limited viewpoints [28, 33, 53]. In this work, we present MoMa-SG for modeling semantic-kinematic scenes as articulated 3D scene graphs. The ultimate aim of MoMa-SG is to endow mobile manipulators with the ability to reason, navigate, and manipulate in interactive environments containing numerous articulated objects. We strive to accomplish this from single object interactions, i. e., from in-the-wild RGB-D observations of a human interacting with a series of articulated objects (see Fig. 1). Instead of adopting a learning-based manipulation approach, we address this problem by one-shot distillation of those observations into a semantic-kinematic 3D scene graph hierarchy. The proposed graph holds explicit kinematic information, i. e., the underlying articulation model for each articulated object, and the semantic relationship among containers and contained objects inferred through interaction. We deliberately avoid assumptions about a fixed set of semantic categories as well as camera sensors, viewpoints, and acting embodiments, allowing MoMa-SG to operate across a wide range of robots. The observation modes include human demonstrations from both ego-centric, exo-centric as well as robot-centric camera trajectories. We evaluate our proposed method on two distinct mobile manipulators for the task of identifying articulation states as well as opening and closing various articulated objects based on the estimated models. In summary, we make the following contributions: • A unified framework for constructing articulated 3D scene graphs from either ego-centric, exo-centric, or robot-centric observations that allows for one-shot open-world mobile manipulation. • The novel Arti4D-Semantic benchmark that bridges the fields of semantic scene understanding and object articulation understanding in in-the-wild settings. • A novel twist optimization objective that allows for robust estimation of revolute and prismatic articulations from real-world point-level trajectories suffering from drift and occlusions without loss of generality. • We demonstrate real-world mobile manipulation from our articulated scene graphs in two distinct environments across two robotic embodiments. • We make the dataset, evaluation routines, and code publicly available to foster future research in this domain. I Related Work Mobile Manipulation from Kinematic Environment Models: Having a localized kinematic model of an articulation is a direct enabler of robotic manipulation. Under such a model, an object can be manipulated reliably using a form of compliant control [18, 42, 4], even in mobile settings. Arduengo et al. [1] demonstrate an approach that learns articulations from a human demonstration and then acts on these with a mobile robot. KinScene [16] presents a similar system, though with the robot exploring articulations itself in a similar manner as [42]. Other approaches use kinematic models to drive robotic controllers [45, 34]. While grasping is commonly the focus of these interactions, [39] demonstrate also how an articulation model can be used to generate pushing interactions, including selection of the contact point. Only using a forward-kinematic prediction from an articulation model is also sufficient to realize manipulation through learned controllers [15]. Articulated Object Estimation: Estimating the parameters of articulated objects has been the subject of study for almost two decades. Katz et al. [23] presented the first approach, extracting 2D articulations by tracking and processing the relative motions of point clusters. Sturm et al. [46] build on the core concept and present a full framework for extracting entire kinematic graphs from observations of relative object motions. Their approach views the connections between object edges in a graph and fits pair-wise models of distinct articulations to the connections. As most articulated objects in everyday life are one-dimensional, high-dimensional representations have been reduced to a single vector in 3⊆ℝ6 so3 ^6 [19, 13]. The most recent approaches exploit techniques such as Gaussian Splatting or Neural Radiance Fields (NeRF) to extract an articulation model as well as the objects’ parts’ shapes [20, 29, 28]. All of these methods require objects to be observed in different articulatory states. To overcome this challenge robotically, a long line of work has investigated the use of interaction to produce motions in a suspected articulated object [18, 42, 31, 32]. These approaches require prior knowledge of the type of interaction that might cause motion. Learning this type of prior from visual observations has been studied by [35, 54, 7, 58]. Buchanan et al. [4, 5] couple learned priors with robotic interaction and tactile exploration to estimate the articulation of even ambiguous objects. 3D Scene Graphs: 3D Scene Graphs (SGs) unify geometry, semantics, and relational reasoning for robotic perception. Early work constructed SGs from static 3D reconstructions [2, 48], later extended to incremental RGB-D fusion with online updates and persistent object identity [52]. More recent approaches introduce open-vocabulary and hierarchical SGs that support unconstrained semantic labeling and language grounding, primarily for navigation and long-horizon exploration [12, 51]. Dynamic SG formulations model temporal graph evolution and persistent world memory, but typically assume rigid object models [40]. Scene graphs have also been leveraged as an interface between perception and task planning [21]. A complementary line of work treats functionality and affordances as first-class graph components, introducing interaction-relevant nodes and relations that bridge perception and action [41, 57, 8]. Only very recent efforts integrate articulation into SGs by modeling parts and kinematic relations at the scene level [55]. Closest to our work is Pandora [55], however, it assumes full object visibility and fixed open-close interaction patterns, whereas MoMa-SG supports partial observability, diverse articulation motions, and embodiment-agnostic interaction-driven belief refinement. I Approach In the following, we detail our proposed approach MoMa-SG as demonstrated in Fig. 2 for building accurate and actionable semantic-kinematic 3D scene graphs from articulation demonstrations. First, we split the observation stream into individual interaction segments (Sec. I-A). Next, we apply point tracking on the discovered interaction segments and estimate the manipulated object’s articulation parameters (Sec. I-B). This is followed by a 3D part mapping stage, an object-to-articulation association, and the discovery of objects contained in articulated objects (Sec. I-C). Figure 2: MoMa-SG enables the construction of accurate 3D scene graphs over articulated scenes and serves as a backbone for long-horizon mobile manipulation. We first discover interaction segments (Sec. I-A), then attain object articulation models A by estimating twists from point trajectories (Sec. I-B). Next, we match mapped objects O against articulations and discover objects contained in respective articulated parents (Sec. I-C). I-A Interaction Discovery Given a scene-level RGB-D sequence t,tt=1T \I_t,D_t \_t=1^T of interactions with multiple articulated objects, we first extract a set of N temporal interaction segments =(ts(n),te(n))n=1NS=\(t_s^(n),t_e^(n))\_n=1^N that contain frames capturing dynamic object motion. To identify scene interaction, we propose to combine two independent signals: First, a general interaction prior indicates that an interacting agent is visible, and second, a depth disparity measure that effectively encodes scene dynamics. These complementary indicators allow tracking interactions under occlusions or interactions with low dynamics, e. g., hidden hands when opening large doors. Interaction Prior and Depth Disparity: For each RGB frame tI_t, we extract an interaction mask tH_t using a pre-trained YOLOv9 [49] model. We combine this interaction signal with a depth disparity measure that measures whether the scene stays static between depth map tD_t and a previous depth map t−sHD_t-s_H, where sHs_H denotes a fixed stride. For this, we warp t−sHD_t-s_H into the camera view of tD_t, obtaining ~t←s D_t← s. Given the camera poses at the respective timestamps t−sHT_t-s_H and tT_t and the projection function from point cloud to depth map =proj()D= proj(p), we obtain a depth disparity Δt _t (Fig. 2): Δt _t =|t−~t←sH| = |D_t- D_t← s_H | (1) ~t←sH D_t← s_H =proj(t−1t−sHproj−1(t−sH)) = proj (T_t^-1T_t-s_H proj^-1(D_t-s_H) ) (2) Probabilistic Interaction Segmentation: In the following, we combine the depth disparity Δt _t and the interaction mask tH_t. We threshold Δt>τ _t>τ to obtain a binary mask and compute the pixel-wise sum and normalization of both masks as hth_t and δt _t. We further apply median filtering over a time horizon κ. To fuse the two signals, we define the interaction score p(s)p (s ) as the probability of t being part of an interaction segment. We then interpret the normalized scores ht,δt∈[0,1]h_t, _t∈[0,1] as probabilities: δt=p(d=⊤) _t=p (d= ) represents the probability of having dynamic object motion in the scene, and likewise, ht=p(h=⊤)h_t=p (h= ) measures the probability of interacting with the scene. Assuming that p(d)p (d ) and p(h)p (h ) are independent, we obtain the following conditional probability model: p(s) p (s ) =∑d,h∈⊤,⊥p(s,d,h)=∑d,hp(s|d,h)p(d,h) = _d,h∈\ , \p (s,d,h )= _d,hp (s\; |\;d,h )p (d,h ) (3) =∑d,hp(s|d,h)p(d)p(h) = _d,hp (s\; |\;d,h )p (d )p (h ) The values for p(s|d,h)p (s\; |\;d,h ) are tuned on held-out data and describe the conditional probability for a frame belonging to an interaction segment, given the observation of the interaction mask and the dynamics in the scene. The resulting interaction score is depicted in Fig. 2. We parse the segments by identifying consecutive non-zero intervals of length T and disregard segment lengths below TminT_min and above TmaxT_max. I-B Articulation Estimation For each segment sn∈s_n , we aim to optimize a model for the articulation i=ξ,θs,…,θe,MA_i=\ξ, _s,…, _e,M\, where ξ parametrizes the articulation model, the θj _j describe the articulation states, i.e. the opening states at each time-step tj∈snt_j∈ s_n and M is a semantic token characterizing the observed motion. Point Tracking: As we aim for the approach to be robust to occlusion of hands, we opt for a point tracking-based approach in favor of a meshing approach as chosen by Pandora [55]. While ArtiPoint [50] initializes its tracking process from keypoints sampled around hand masks, we opt for a more general approach by relying on the disparity masks tD_t, as outlined in Sec. I-A, while subtracting image areas covered by the interaction prior such that t∖tD_t _t. Ultimately, this allows tracking points through hand occlusions. Similar to ArtiPoint [50], we create query points using the Shi-Tomasi keypoint detection method [43] as it provides dependable keypoints for object tracking. This yields a set of query points ∈ℝK×F×2Q ^K× F× 2 where K represents the number of keyframes considered within sn∈s_n and F is the number of sampled queries. We employ CoTracker3 [22] to obtain point tracks n∈ℝTn×F×2X_n ^T_n× F× 2 across all TnT_n frames of sns_n, including their visibilities n∈0,1Tn×FV_n∈\0,1\^T_n× F throughout each interaction. We then lift nX_n to 3D using RGB-D depth and compensate for camera motion by transforming into world coordinates. Trajectory Filtering: We first filter nX_n to separate static and dynamic points exhibiting non-consistent behavior due to, e. g., noisy depth estimates. In addition, we employ DBSCAN [9]-based trajectory clustering to identify the time-wise longest set of uniform point tracks. Operating on the longest track cluster, rather than the most uniform one, is particularly important. That is because we want to estimate the full magnitude of the observed motion, not just its axis, to later, e. g., estimate opening states. Finally, we smooth out point tracks to filter for noisy depth and keypoint positions. Regularized Twist Estimation: We estimate object articulation models using screw theory, which allows us to represent arbitrary rigid-body motion. A screw motion is parameterized by a twist ξ=⟨ω,v⟩∈(3)ξ= ω,v ∈ se(3), where ω∈ℝ3ω ^3 and v∈ℝ3v ^3 are the rotational and translational velocity components. In the context of articulations, a key advantage of the twist representation is its capacity to model diverse articulation types, including revolute, prismatic, and screw joints, while also being differentiable, thereby enabling gradient-based optimization [19, 13, 4, 50, 25]. Furthermore, given a configurational parameter θ, we are able to produce the equivalent SE(3)SE(3) rigid-body transform using the exponential map exp:(3)→SE(3)exp: se(3)→ SE(3), while the inverse of the exponential map is referred to as the logarithmic map log:SE(3)→(3)log:SE(3)→ se(3), converting back to tangent space [3]. Despite its advantages, we find that the twist parameterization is prone to observational noise and keypoint drift when applied directly on point trajectories in uncontrolled settings [50, 4], which is why one typically first estimates object poses and then applies pose-based twist estimation. However, fitting representative local frames to objects observed in-the-wild remains challenging. In order to resolve this, we propose a novel regularization scheme for point-based twist estimation that addresses the magnitude of ω and v. We find that under real-world observations, e. g., pure prismatic motion, which ideally would represent a screw motion of infinite pitch, often shows a non-negligible rotational component ‖ω‖\|ω\|. Thus, when employing vanilla twist estimation from point trajectories, one could determine the underlying articulation type (prismatic or revolute) from the ratio ‖ω‖/‖v‖\|ω\|/\|v\| and parse the twist by, e. g., neglecting the rotational component of an identified prismatic motion. In practice, finding such a decision rule is only possible while incurring significant type and reconstructed axis errors. Instead, we propose a scaled dot product prior among vectors cast from the observed point trajectories, which are secants of circles with ideally finite (revolute) or infinite (prismatic) radius. The reasoning behind this is that vectors drawn from prismatic motions generally show little to no angular deviation. On the contrary, revolute motions produce distinct angular disagreements. As such, we sample vectors of the form i=it−i0x_i=p_i^t-p_i^0, where i0∈ℝ3p_i^0 ^3 represents an initial point of a track and it∈ℝ3p_i^t ^3 represents a point along the track. From this, we construct a set of vectors ∈ℝM×3X ^M× 3 across all point tracks for which we compute the scaled dot product of all pairs: Cij=i⊤j‖i‖‖j‖,C_ij= x_i x_j\|x_i\|\,\|x_j\|, (4) where we enforce that ix_i and jx_j do not terminate at the same time step t. This yields a normalized Gram matrix =⊤⊙C=XX where N represents the normalization term. Next, we compute the median of its upper-triangular component η=med(triu(⊙⊤)),η=med(triu(N )), (5) and feed η to a shifted sigmoid function σ(k(η−η∗))σ(k(η-η^*)) with bias η∗η^*, which is a tunable hyperparameter. We impose that for revolute motion, the absolute dot product |ω⊤v|≈0|ω v|≈ 0 as ω⟂vω v. Similarly, for prismatic motion, we expect a diminishing rotational term ‖ω‖\|ω\| as solely v governs the observed translational motion. We define λpris=σ(k(η−η∗)) _pris=σ(k(η-η^*)) and λrev=1−σ(k(η−η∗)) _rev=1-σ(k(η-η^*)) and obtain the following optimization problem: ξ^∗,θ1∗,…,θe∗ ξ^*,θ^*_1,…,θ^*_e =minξ^,θs,…,θe∑m=se∑j=1|Pm|‖pjt+ϵ−exp(ξ^θm)⋅pjt‖ = _ ξ, _s,…, _e _m=s^e _j=1^|P_m| p^t+ε_j- ( ξ _m)· p^t_j (6) +α(λrev|ω⊤v‖ω‖‖v‖|+λpris‖ω‖), +α ( _rev | ω v ω v |+ _pris\|ω\| ), where the first term represents the original twist optimization from point trajectories and the second term constitutes our dot product prior with α as an optional weighting term. We solve for an optimal twist ξ as well as observed configurations θm∗ _m^* using non-linear least-squares optimization. Note that the regularization term is fully differentiable while we do not suffer a loss of generality. Mode Understanding: The obtained articulation estimate in Eq. 6 yields configurational scalars θ. In general, however, we cannot infer from that whether we, e. g., observed an opening or a closing action. Thus, we feed a small set of informative frames contained in sns_n to GPT-5-mini and prompt for the observed articulation mode, which is constrained to be OPENING, CLOSING, or combined motions such as OPENING-CLOSING or CLOSING-OPENING. We use a simple heuristic as a critic to verify whether the VLM output is consistent with the distribution of θ’s. Thus, we obtain a grounded understanding MiM_i of the observed articulation iA_i. I-C Articulated 3D Scene Graph Construction Building on the previous results, we construct a hierarchical 3D scene graph =(,ℰ)G=(V,E) consisting of an object node layer O representing articulated objects and a layer of contained objects C that represent children nodes k to j∈j . This representation is supported by a geometric point cloud layer. A small subset ⊆O^A corresponds to the estimated articulated models A. In this work, we assume that the observed articulations A are either prismatic or revolute. An edge (j,k)(j,k) thus connects an articulated parent object j to a contained child node k∈k . We note that child nodes are either of STATIC or ARTICULATED type, i. e., they either follow the same articulation model as their parent or remain static. Intuitive examples for this are a mug hidden behind the kitchen cabinet door as opposed to a milk carton that sits in the fridge door and moves along with its parent. In our approach, we construct O and A independently and later match objects against articulations. 3D Part Mapping: First, we select keyframes using a distance- and rotation-based heuristic while favoring frames showing small motion blur. We then employ an incremental mapping scheme incorporating 2D segmentation masks from Semantic-SAM [27], as its finest granularity level provides dependable part instances. In order to allow for open-world deployment in unknown environments, we do not assume any knowledge of a fixed set of semantic categories at inference time, different from costly tagging-based prompting pipelines [59, 38]. We equip each part-level mask with a CLIP feature to render it semantically retrievable at deployment time using open-vocabulary scoring. In contrast to ConceptGraphs [12], we do not rely on semantic similarities during mask merging, as it frequently produces erroneous merges for object parts. We incrementally merge new detections into existing objects while evaluating whether the object and detection are isolated masks, i. e., they are not intersecting the image boundaries. If an existing object or detection is not isolated, we apply stronger matching filters based on IoU and inlier score to counter higher segmentation errors for partially observed objects at image boundaries. We detail these rules in the supplementary material. Matching Articulations Against Objects: In the following, we identify the object parts ⊆O^A that are undergoing interactions given the estimated articulations A. Per segment sns_n, we first retrieve the k-closest objects given their 3D centroids, and the mean position of the 3D point tracks belonging to the articulation. As we aim to find the 3D part segment that, under motion, best explains the observed point trajectories, we replay close-by objects in O under the estimated articulated model iA_i and evaluate whether the motion pattern matches with our 2D point tracks nX_n. In particular, we divide nX_n into dynamic and static points and measure the ratio of dynamic keypoints falling into the mask as well as static keypoints that fall into the mask exterior of object j. Summing these two terms yields mijm_ij representing the cost of matching articulation i and object j, which we further detail in the supplement. In general, we face under- and over-segmentation of our part segmentation, leading to conflicting masks. To compensate for this, we strive to find an object-to-articulation matching that is mutually exclusive. This comes with two distinct considerations: a) we aim to avoid assigning a single object j to two different articulations in the case of under-segmentation, and b) we aim to rule out mutual 3D overlaps among assigned objects. We thus formulate a program that ensures that a single object j∈j is only matched against an articulation i∈i once while minimizing the mutual 3D overlap among all assigned object parts. We accomplish this using the following linear binary integer program (BIP): minm,y,z∑i∈∑j∈pijmij+λ∑(j,k)∈qjkzjk, m,y,zmin _i _j p_ijm_ij+λ _(j,k) q_jkz_jk, (7) where pijp_ij is the cost of assigning articulation iA_i to object j as described above, qjk≥0q_jk≥ 0 represents the IoU between object j and k. In addition, mij∈0,1m_ij∈\0,1\ becomes 1 if articulation i is assigned to object j, yj∈0,1y_j∈\0,1\ becomes 11 if object j is assigned to any articulation, and lastly, zjk∈0,1,j,k∈z_jk∈\0,1\,j,k denotes if both yj=1y_j=1 and yk=1y_k=1. In order to satisfy the above requirements, we impose the following constraints: ∑ Σ mij∈=1∀i∈ _j m_ij=1\,\,∀ i (8) ∑ Σ miji∈≤1∀j∈ _i m_ij≤ 1\,\,∀ j zjk z_jk ≤yj,zjk≤yk, and zjk≥yj+yk−1, ≤ y_j,z_jk≤ y_k, and z_jk≥ y_j+y_k-1, with the last enforcing zjk=yj∧ykz_jk=y_j\, y_k. Solving this BIP thus provides us with a set of objects O^A constituting the optimal assignment of articulations A and objects O. Discovery of Contained Objects: In the final stage of our approach, we discover contained objects and arrange them as child nodes C of the found articulated parents O^A. We achieve this by evaluating the estimated articulation mode M and identifying the configuration θmax _max representing the maximum-open frame. We then retrieve the 2D projected articulated parent mask under two configurations: maximum opening and closed states. Subsequently, we query a class-agnostic instance segmentation method [56] and identify whether objects show a significant overlap as well as containment wrt. the articulated part mask at the time of opening or the closed object mask. In case of significant agreement with the articulated part, we initialize an ARTICULATED edge (i,c)(i,c) between parent and child. Conversely, we initialize a STATIC edge when the child object covers the closed object mask. IV Arti4D-Semantic Dataset We present the Arti4D-Semantic dataset, which, to the best of our knowledge, constitutes the first hierarchical interactive 3D scene graph dataset tailored towards scene-level articulated object understanding. Our goal is for Arti4D-Semantic to connect the research areas of articulated object estimation and semantic 3D mapping, which have been studied mostly in isolation to date. Arti4D-Semantic extends Arti4D [50] by providing a rich label set of interaction segments, labeled articulated 3D parts, including semantic categories, as well as labels of objects contained within articulated objects. While ego-centric sequences represented the only observation paradigm in Arti4D, the novel Arti4D-Semantic dataset provides two additional ones: exo-centric demonstrations and robot-centric interaction observations using a teleoperated mobile manipulator. We believe the dataset will be invaluable to study the problem of constructing semantic-kinematic representations of 3D scenes under minimal assumptions about camera positions and demonstration protocol. Across all observation paradigms, Arti4D-Semantic contains 62 RGB-D sequences across 5 distinct scenes depicting humans demonstrating interactions with various articulated objects such as drawers, cabinets, doors, or sliding cabinets. For each interacted object, we thus provide instance and semantic information coupled with its axis of motion. Since contained objects are either static in world coordinates or follow the motion pattern of their articulated parent, our dataset provides contained object labels that are themselves a function of the articulation state. Therefore, misidentifications of maximum-opening states are resolvable at the time of evaluation. We provide additional details in the supplementary material. Figure 3: Labels contained in Arti4D-Semantic: Solid circled labels denote articulated parent parts and dashed-circled labels represent articulated labels. V Experimental Evaluation We evaluate MoMa-SG on several different tasks ranging from temporal interaction segmentation (Sec. V-A) and articulation estimation (Sec. V-B) to articulated 3D part segmentation and contained object discovery (Sec. V-C). The majority of experiments are conducted on Arti4D as well as its labeled counterpart Arti4D-Semantic, as they provide true real-world in-the-wild object articulations. In addition, we quantify the performance of the two best-performing articulation estimation methods on an articulated object split of the large-scale DROID dataset [24] in Tab. I. Except for the evaluation of interaction segmentation itself, we rely on ground-truth interaction segments to allow for appropriate comparisons. To ensure comparability, we only evaluate on the ego-centric sequences of Arti4D-Semantic in the following and provide exo-centric and robot-centric results in the supplementary material. Finally, we test the real-world capability of MoMa-SG in two distinct real-world mobile manipulation settings in Sec. V-D. Implementation Details: We set the warp parameters to sH=6s_H=6, τ=0.05τ=0.05, κ=11κ=11. For point tracking, we use a maximum of F=1500F=1500 queries, a stride of 22, and a cosine threshold of η∗=0.994η^*=0.994. The minimum required point trajectory length is 0.10 0.10. The approach requires 24GB VRAM. We detail additional settings in the supplementary material. TABLE I: Temporal Interaction Segmentation Evaluation on Arti4D-Semantic Method 1D-IoU P R Segment-IoU IonI_on [ss] IoffI_off [ss] Pandora [55] 0.359 0.690 0.400 0.689 0.7442 0.725 HMM 0.471 0.871 0.532 0.683 0.770 0.762 ArtiPoint [50] 0.575 0.678 0.714 0.698 1.05 0.558 MoMa-SG 0.649 0.786 0.800 0.718 0.816 0.742 The 1D-IoU metric denotes the intersection over union of the dense binary timeseries signals, P and R, as well as the Segment-IoU denote the precision and recall, as well as the segment-wise IoU, following a linear sum assignment between predicted and ground truth segments under an IoU >> 0.5. IonI_on and IoffI_off measure the absolute start and end errors of matched segments. Baselines: Depending on the evaluated task, we compare against the following set of methods: We employ the publicly-available code of ArtiPoint [50] that is already tailored to Arti4D and performs interaction segmentation using filtered hand visibilities and articulation estimation relying on factor graph estimation of pose trajectories using either the estimator of Buchanan et al. [4] or Sturm et al. [46]. In addition, we compare against ArtGS [29], which performs Gaussian splatting-based reconstruction of the axis of motion. As it is intended to run on quasi-static scenes, we adapt it by masking hands, and providing a sufficient number of frames from before, during, and after the interaction. Furthermore, we compare against Pandora [55], which constructs articulated scene graphs by initializing flood fills near hand positions towards static, non-articulated objects in order to segment the underlying articulated object. The original main evaluation Pandora conducted was on simulated environments generated with Blender that do not involve sudden hand/camera motion, object occlusions, hand occlusions, and noisy camera poses. We find that several assumptions made by Pandora, such as full object visibility throughout interaction and a strict articulation demonstration protocol, i. e., each object is strictly only going from closed to open, limit its performance when operating on non-curated data as introduced with Arti4D-Semantic. To adapt Pandora, we provide it with ground-truth closed- and open-states. Pandora additionally proposes a heuristic for interaction segmentation relying on casting spheres around hand positions and measuring overlap with a static scene reconstruction. Note that MoMa-SG does not rely on a static reconstruction as we do not assume a neutral scene state with all objects being closed and instead are able to match articulations against various object masks. TABLE I: Articulated Object Estimation on Arti4D Method Prismatic joints Revolute joints Type Quality [%] θerr _err[deg] dL2d_L2[m] θerr _err[deg] dL2d_L2[m] Accuracy Prismatic Recall Revolute Recall Ours w/o twist regularization 28.764 - 33.278 0.135 0.542 1.000 0.305 Ours w/ XFeat [36] keypoint queries 15.768 - 31.538 0.152 0.823 0.824 0.821 ArtGS [29] 52.070 - 62.638 0.301 0.619 1.000 0.000 Pandora [55] 46.814 - 50.620 0.195 0.559 0.772 0.444 ArtiPoint [50] w/ Sturm et al. [46] 17.754 - 28.467 0.313 0.872 0.854 0.905 ArtiPoint w/o prior [50] 22.953 - 27.057 0.269 0.776 0.713 0.899 ArtiPoint w/ prior [50] 23.272 - 26.358 0.248 0.776 0.709 0.906 MoMa-SG (ours) 13.190 - 22.982 0.091 0.884 0.917 0.866 We report the axis-angle errors θerr _err, the positional errors dL2d_L2 of revolute joints, and the accuracy of predicted joint types including their respective recalls. The top two rows denote ablations of our method. V-A Interaction Segmentation We evaluate MoMa-SG on segmenting interactions temporally in Tab. I against Pandora, a simple hidden Markov model (HMM) as well as ArtiPoint. In general, we observe the highest 1D-IoU as well as segment recall and segment-based IoU in comparison, whereas the HMM shows significant precision at low recalls. We deduce that introducing the warp prior (Sec. I-A) leads to a significant performance increase compared to ArtiPoint. V-B Articulation Estimation As shown in Tab. I, MoMa-SG outperforms all prior methods in terms of prismatic and revolute axis errors while also exhibiting increased type prediction accuracy. We find that ArtiPoint, when not using ground truth axis types during twist parsing, yields inferior results compared to MoMa-SG, which instead utilizes the cosine prior to estimate object axes from point trajectories. In addition, we observe that Pandora suffers from hand as well as object occlusions, which result in incomplete meshes and severely complicate articulation estimation. We further evaluate Pandora with scaled monocular depth as input, but this did not improve performance. In addition, we find that noise in depth estimates and camera poses hinders the proposed mesh traversal to identify vertex pairs between the opened object mesh and the static scene reconstruction. Furthermore, we ablate how our method benefits from the proposed twist regularization as we observe significant angular error increases as well as a slight increase in translational errors for revolute joints without regularization. On top of that, we observe significant decreases in type accuracy when relying on a pitch-based rule to identify articulation types. We further incur reduced prediction performance of MoMa-SG when switching to XFeat [36] as keypoint extractor instead of GFTT [43]. We visualize qualitative results of our method in Fig. 5. Subsequently, we compare with ArtiPoint, as best-performing baseline, with MoMa-SG on DROID in Tab. I and observe a significant performance increase with our method. For DROID, we rely on robot masks obtained using SAM3 [6] instead of hand/human masks to generate the interaction prior, showcasing the versatility of MoMa-SG. Figure 4: Contained objects discovered using MoMa-SG across different scenes of Arti4D-Semantic. TABLE I: Articulation Estimation on DROID Dataset Method θerrpris _err^pris [∘] θerrev _err^rev [∘] dL2revd_L2^rev [m] Type Acc. [%] ArtiPoint [50] 35.88 25.43 0.278 0.611 MoMa-SG 7.15 16.91 0.115 0.895 Results across 19 articulated object manipulation demos of DROID [24]. The reported types of metrics follow Tab. I. Figure 5: Qualitative results of MoMa-SG on Arti4D-Semantic: Estimated axis positions and corresponding object masks. As demonstrated, we observe minimal errors for a large number of prismatic objects and small errors on revolute objects. TABLE IV: 3D Articulated Part Segmentation on Arti4D-Semantic Object Method IoU Recall Set @0.25 @0.50 @0.75 O Ours w/o bound.-resp. merging 0.481 0.807 0.534 0.086 OpenMask3D 0.082 0.087 0.019 0.000 ConceptGraphs [12] 0.438 0.332 0.127 0.018 MoMa-SG (ours) w/ O 0.533 0.824 0.646 0.163 O^A Ours w/ greedy matching 0.280 0.432 0.279 0.049 Pandora [55] 0.065 0.012 0.000 0.000 MoMa-SG (ours) w/ O^A 0.292 0.454 0.303 0.061 The upper section contains articulation-independent 3D mapping results over free objects O, the lower section contains only articulation-matched objects O^A. The IoU denotes the mutual overlap under a linear assignment of ground truth and predicted masks. The recall indicates the overlap with the ground truth at various IoUs. TABLE V: Discovery of Contained Objects on Arti4D-Semantic Method IoU Recall Relation Accuracy @IoU=0.25 STATIC / ARTICULATED Pandora [55] 0.002 0.035 0.197 MoMa-SG (ours) 0.091 0.166 0.592 The IoU denotes the highest-achieved overlap between a predicted child node and all ground truth contained objects per interaction. The relation accuracy measures the frequency of correctly identified parent-child edges. V-C Object Understanding In general, precise articulation estimation is paramount to matching objects against articulations as well as when identifying contained objects. Our object understanding experiments are two-fold. First, we investigate how well articulated 3D parts are segmented in Tab. IV, and second, we investigate whether those segments allow us to identify contained objects within various articulated objects in Tab. V. We evaluate the segmentation performance of articulated 3D parts in two distinct settings being free objects O and objects matched against articulations O^A. To reflect this ambiguity, we decided to only evaluate the IoU of an optimal assignment of predicted and ground truth objects as well as the recall. As Pandora assumes knowledge of a neutral scene state, it directly segments articulated parts during interaction. Thus, it does not provide free articulated objects, except for detected contained objects. Overall, we find that MoMa-SG significantly outperforms Pandora as the latter struggles to identify complete meshes already in the articulation estimation stage. In the free mask setting, we observe that 3D object merging solely from geometric overlap outperforms ConceptGraphs [12], which relies on semantic similarities for object merging, which produces a significant number of false positive merges whenever, e. g., two neighboring drawers are undersegmented. In addition, we demonstrate that merging objects based on whether their 2D masks intersect with the image boundaries significantly improves mapping. We also report the performance of MoMa-SG when employing greedy object-articulation matching instead of the proposed binary integer program and note additional performance decreases. This underlines the importance of accurately matching objects to articulations, especially in light of real-world retrieval of those objects. Furthermore, we have evaluated the IoU and recall for all contained children nodes to their respective articulated parents in Tab. V. In general, we face a compounding error problem. Inferior results in articulation estimation and object segmentation are detrimental to children detection as well. In general, MoMa-SG outperforms Pandora in both segmented child nodes and relation understanding. We visualize successful children retrievals in Fig. 4 and provide additional results on open-vocabulary semantics in the supplementary material. Figure 6: We portray our real-world mobile manipulation environments of the HSR (A) as well as the Spot (B). We demonstrate that MoMa-SG allows for robust mobile manipulation even under various articulation states of objects. V-D Real-World Mobile Manipulation To demonstrate the real-world utility of MoMa-SG, we conduct two experiments. The first is performed on the Toyota HSR mobile manipulator and evaluates online articulation state estimation of various articulated parents initialized at diverse states (see Fig. 6, top). We estimate states by articulating object point clouds under the estimated ξ using the exponential map and measuring overlap, which enables the robot to autonomously operate articulated objects independent of their configuration. This further enables retrial of unsuccessful articulations due to, e. g., gripping failures. We report results in Tab. VI and achieve mean errors of 1.7cm for prismatics and approx. 6.5∘6.5 on revolutes, demonstrating sufficient real-world articulation estimation to enable retrials. In a second experiment, we demonstrate how MoMa-SG serves as an embodiment-agnostic map for mobile manipulation of articulated objects. The experiment is conducted in environments A and B (see Fig. 6), where we detect object handles online with SAM3 [6], estimate grasps using their predominant axes, and match those against all objects using a Hungarian assignment. We evaluate two robotic embodiments, the HSR and a Boston Dynamics Spot, tasked with operating the articulated objects represented in the graph, and measure overall task success of opening and closing random objects as reported in Tab. VII, where both robots achieve over 80%\% success across articulation types and environments. We are able to attribute the majority of failed actions to gripping failures. Overall, we argue that MoMa-SG yields an embodiment-agnostic scene representation. We provide results on language-enabled mobile manipulation and additional details in the supplementary material. TABLE VI: Real-World Articulation State Estimation Object Trans. Error Rot. Error Type Δd d [ ] σ [ ] Δθ θ [ ] σ [ ] Prismatic 0.0174 0.006 - - Revolute - - 6.477 3.604 Experiments are conducted across 54 various object articulation states. TABLE VII: Real-World Mobile Object Manipulation Object HSR Spot Type Opening Closing Opening Closing Prismatic 0.879 1.000 0.750 - Revolute 0.810 0.808 0.875 - Overall 0.855 0.902 0.808 - Experiments are conducted on two different robot embodiments, namely the Toyota HSR and the Boston Dynamics Spot across two distinct environments. All displayed metrics constitute success rates. V-E Limitations It would be beneficial to reduce the reliance on accurate camera poses and real-world depth measurements. However, scaled monocular depth still does not produce a level of fidelity allowing consistent articulation estimation in uncurated settings. Furthermore, we assume a limited level of depth noise for our depth warp signal to be meaningful, which may not hold under a large number of specular objects. We further observe that the major limiting factor across all methods is accurate, real-time action recognition. VI Conclusion We presented MoMa-SG as a novel embodiment-agnostic method for distilling in-the-wild object articulations into articulated 3D scene graphs that allow for robust mobile manipulation in dynamic human-centered scenes. In addition, we have presented the novel Arti4D-Semantic dataset as the first benchmark for evaluating articulated 3D scene graphs including object containment. In future work, we aim to address semi-static rearrangements. References [1] M. Arduengo, C. Torras, and L. Sentis (2021) Robust and adaptive door operation with a mobile robot. Intelligent Service Robotics 14 (3), p. 409–425. Cited by: §I. [2] I. Armeni, Z. He, J. Gwak, A. R. Zamir, M. Fischer, J. Malik, and S. Savarese (2019) 3d scene graph: a structure for unified semantics, 3d space, and camera. In Proceedings of the IEEE/CVF international conference on computer vision, p. 5664–5673. Cited by: §I, §I. [3] T. D. Barfoot (2024) State estimation for robotics. Cambridge University Press. Cited by: §I-B. [4] R. Buchanan, A. Röfer, J. Moura, A. Valada, and S. Vijayakumar (2024) Online estimation of articulated objects with factor graphs using vision and proprioceptive sensing. In IEEE International Conference on Robotics and Automation, p. 16111–16117. Cited by: §I, §I, §I, §I-B, §I-B, §V. [5] R. Buchanan, A. Röfer, J. Moura, A. Valada, and S. Vijayakumar (2026) Online estimation and manipulation of articulated objects. arXiv preprint arXiv:2601.01438. Cited by: §I. [6] N. Carion, L. Gustafson, Y. Hu, S. Debnath, R. Hu, D. Suris, C. Ryali, K. V. Alwala, H. Khedr, A. Huang, J. Lei, T. Ma, B. Guo, A. Kalla, M. Marks, J. Greer, M. Wang, P. Sun, R. Rädle, T. Afouras, E. Mavroudi, K. Xu, T. Wu, Y. Zhou, L. Momeni, R. Hazra, S. Ding, S. Vaze, F. Porcher, F. Li, S. Li, A. Kamath, H. K. Cheng, P. Dollár, N. Ravi, K. Saenko, P. Zhang, and C. Feichtenhofer (2025) SAM 3: segment anything with concepts. External Links: 2511.16719, Link Cited by: Figure S.6, Figure S.6, §S.1-D3, §V-B, §V-D. [7] B. Eisner*, H. Zhang*, and Held,David (2022) FlowBot3D: learning 3d articulation flow to manipulate articulated objects. In Robotics: Science and Systems, Cited by: §I. [8] T. Engelbracht, R. Zurbrüg, M. Pollefeys, H. Blum, and Z. Bauer (2025) Spotlight: robotic scene understanding through interaction and affordance detection. In IEEE-RAS Int. Conf. on Humanoid Robots, p. 1–8. Cited by: §I, §I. [9] M. Ester, H. Kriegel, J. Sander, and X. Xu (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD’96, p. 226–231. Cited by: §I-B. [10] D. Fox, W. Burgard, F. Dellaert, and S. Thrun (1999) MCL localization: efficient position estimation for mobile robots. In Proc. of the Conf. on Artificial Intelligence (AAAI), Cited by: §S.1-D1. [11] E. Greve, M. Büchner, N. Vödisch, W. Burgard, and A. Valada (2024) Collaborative dynamic 3d scene graphs for automated driving. In IEEE International Conference on Robotics and Automation, p. 11118–11124. Cited by: §I. [12] Q. Gu, A. Kuwajerwala, S. Morin, K. M. Jatavallabhula, B. Sen, A. Agarwal, C. Rivera, W. Paul, K. Ellis, R. Chellappa, et al. (2024) Conceptgraphs: open-vocabulary 3d scene graphs for perception and planning. In IEEE International Conference on Robotics and Automation, p. 5021–5028. Cited by: §S.1-B, §S.1-B, §I, §I, §I-C, §V-C, TABLE IV. [13] N. Heppert, T. Migimatsu, B. Yi, C. Chen, and J. Bohg (2022) Category-independent articulated object tracking with factor graphs. In IEEE/RSJ International Conference on Intelligent Robots and Systems, p. 3800–3807. Cited by: §I, §I, §I-B. [14] D. Honerkamp, M. Büchner, F. Despinoy, T. Welschehold, and A. Valada (2024) Language-grounded dynamic scene graphs for interactive object search with mobile manipulation. IEEE Robotics and Automation Letters. Cited by: §I. [15] D. Honerkamp, T. Welschehold, and A. Valada (2023) N2M2: learning navigation for arbitrary mobile manipulation motions in unseen and dynamic environments. IEEE Transactions on Robotics 39 (5), p. 3601–3619. Cited by: §S.1-F, §I. [16] C. Hsu, B. Abbatematteo, Z. Jiang, Y. Zhu, R. Martín-Martín, and J. Biswas (2024) KinScene: model-based mobile manipulation of articulated scenes. CoRR. Cited by: §I. [17] N. Hughes, Y. Chang, and L. Carlone (2022) Hydra: a real-time spatial perception system for 3d scene graph construction and optimization. arXiv preprint arXiv:2201.13360. Cited by: §I. [18] A. Jain and C. C. Kemp (2009) Pulling open novel doors and drawers with equilibrium point control. In IEEE-RAS Int. Conf. on Humanoid Robots, p. 498–505. Cited by: §I, §I. [19] A. Jain, R. Lioutikov, C. Chuck, and S. Niekum (2021) Screwnet: category-independent articulation model estimation from depth images using screw theory. In IEEE International Conference on Robotics and Automation, p. 13670–13677. Cited by: §I, §I-B. [20] Z. Jiang, C. Hsu, and Y. Zhu (2022) Ditto: building digital twins of articulated objects from interaction. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition, p. 5616–5626. Cited by: §I. [21] Y. Ju, Y. Liang, Y. Wang, N. Gireesh, Y. Ju, S. Lee, Q. Gu, E. Hsieh, F. Huang, and K. Sreenath (2025) MomaGraph: state-aware unified scene graphs with vision-language model for embodied task planning. External Links: 2512.16909, Link Cited by: §I. [22] N. Karaev, Y. Makarov, J. Wang, N. Neverova, A. Vedaldi, and C. Rupprecht (2025) Cotracker3: simpler and better point tracking by pseudo-labelling real videos. In International Conference on Computer Vision, p. 6013–6022. Cited by: Figure S.10, Figure S.10, Figure S.9, Figure S.9, §I-B. [23] D. Katz and O. Brock (2008) Manipulating articulated objects with interactive perception. In 2008 IEEE International Conference on Robotics and Automation, p. 272–277. Cited by: §I, §I. [24] A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y. Chen, K. Ellis, P. D. Fagan, J. Hejna, M. Itkina, M. Lepert, Y. J. Ma, P. T. Miller, J. Wu, S. Belkhale, S. Dass, H. Ha, A. Jain, A. Lee, Y. Lee, M. Memmel, S. Park, I. Radosavovic, K. Wang, A. Zhan, K. Black, C. Chi, K. B. Hatch, S. Lin, J. Lu, J. Mercat, A. Rehman, P. R. Sanketi, A. Sharma, C. Simpson, Q. Vuong, H. R. Walke, B. Wulfe, T. Xiao, J. H. Yang, A. Yavary, T. Z. Zhao, C. Agia, R. Baijal, M. G. Castro, D. Chen, Q. Chen, T. Chung, J. Drake, E. P. Foster, J. Gao, V. Guizilini, D. A. Herrera, M. Heo, K. Hsu, J. Hu, M. Z. Irshad, D. Jackson, C. Le, Y. Li, K. Lin, R. Lin, Z. Ma, A. Maddukuri, S. Mirchandani, D. Morton, T. Nguyen, A. O’Neill, R. Scalise, D. Seale, V. Son, S. Tian, E. Tran, A. E. Wang, Y. Wu, A. Xie, J. Yang, P. Yin, Y. Zhang, O. Bastani, G. Berseth, J. Bohg, K. Goldberg, A. Gupta, A. Gupta, D. Jayaraman, J. J. Lim, J. Malik, R. Martín-Martín, S. Ramamoorthy, D. Sadigh, S. Song, J. Wu, M. C. Yip, Y. Zhu, T. Kollar, S. Levine, and C. Finn (2024) DROID: a large-scale in-the-wild robot manipulation dataset. Robotics: Science and Systems. Cited by: TABLE I, §V. [25] S. Kim, J. Ha, Y. H. Kim, Y. Lee, and F. C. Park (2025) Screwsplat: an end-to-end method for articulated object recognition. Conference on Robot Learning. Cited by: §I-B. [26] S. Koch, N. Vaskevicius, M. Colosi, P. Hermosilla, and T. Ropinski (2024) Open3dsg: open-vocabulary 3d scene graphs from point clouds with queryable objects and open-set relationships. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 14183–14193. Cited by: §I. [27] F. Li, H. Zhang, P. Sun, X. Zou, S. Liu, J. Yang, C. Li, L. Zhang, and J. Gao (2023) Semantic-sam: segment and recognize anything at any granularity. arXiv preprint arXiv:2307.04767. Cited by: §S.1-B, §I-C. [28] J. Liu, A. Mahdavi-Amiri, and M. Savva (2023) Paris: part-level reconstruction and motion analysis for articulated objects. In International Conference on Computer Vision, p. 352–363. Cited by: §I, §I. [29] Y. Liu, B. Jia, R. Lu, J. Ni, S. Zhu, and S. Huang (2025) Building interactable replicas of complex articulated objects via gaussian splatting. In International Conf. on Learning Representations, Cited by: §I, TABLE I, §V. [30] D. Maggio, Y. Chang, N. Hughes, M. Trang, D. Griffith, C. Dougherty, E. Cristofalo, L. Schmid, and L. Carlone (2024) Clio: real-time task-driven open-set 3d scene graphs. IEEE Robotics and Automation Letters 9 (10), p. 8921–8928. External Links: Document Cited by: §I. [31] R. M. Martin and O. Brock (2014) Online interactive perception of articulated objects with multi-level recursive estimation based on task-specific priors. In IEEE/RSJ International Conference on Intelligent Robots and Systems, p. 2494–2501. Cited by: §I. [32] R. Martin-Martin and O. Brock (2022) Coupled recursive estimation for online interactive perception of articulated objects. The International Journal of Robotics Research 41 (8), p. 741–777. Cited by: §I. [33] R. Martín-Martín, C. Eppner, and O. Brock (2019) The rbo dataset of articulated objects and interactions. International Journal of Robotics Research 38 (9), p. 1013–1019. Cited by: §I. [34] M. Mittal, D. Hoeller, F. Farshidian, M. Hutter, and A. Garg (2022) Articulated object interaction in unknown scenes with whole-body mobile manipulation. In IEEE/RSJ International Conference on Intelligent Robots and Systems, p. 1647–1654. Cited by: §I. [35] K. Mo, L. J. Guibas, M. Mukadam, A. Gupta, and S. Tulsiani (2021) Where2act: from pixels to actions for articulated 3d objects. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition, p. 6813–6823. Cited by: §I. [36] G. Potje, F. Cadar, A. Araujo, R. Martins, and E. R. Nascimento (2024) XFeat: accelerated features for lightweight image matching. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition, p. 2682–2691. External Links: Document Cited by: §V-B, TABLE I. [37] S. K. Ramakrishnan, A. Gokaslan, E. Wijmans, O. Maksymets, A. Clegg, J. M. Turner, E. Undersander, W. Galuba, A. Westbury, A. X. Chang, M. Savva, Y. Zhao, and D. Batra (2021) Habitat-matterport 3d dataset (HM3d): 1000 large-scale 3d environments for embodied AI. In Advances in Neural Information Processing Systems, External Links: Link Cited by: §S.1-B. [38] T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y. Chen, F. Yan, Z. Zeng, H. Zhang, F. Li, J. Yang, H. Li, Q. Jiang, and L. Zhang (2024) Grounded sam: assembling open-world models for diverse visual tasks. External Links: 2401.14159 Cited by: §I-C. [39] A. Röfer, G. Bartels, W. Burgard, A. Valada, and M. Beetz (2022) Kineverse: a symbolic articulation model framework for model-agnostic mobile manipulation. IEEE Robotics and Automation Letters 7 (2), p. 3372–3379. Cited by: §I. [40] A. Rosinol, A. Gupta, M. Abate, J. Shi, and L. Carlone (2020) 3D dynamic scene graphs: actionable spatial perception with places, objects, and humans. In Robotics: Science and Systems, External Links: Document Cited by: §I. [41] D. Rotondi, F. Scaparro, H. Blum, and K. O. Arras (2025) FunGraph: functionality aware 3d scene graphs for language-prompted scene interaction. In IEEE/RSJ International Conference on Intelligent Robots and Systems, Cited by: §I, §I. [42] T. Rühr, J. Sturm, D. Pangercic, M. Beetz, and D. Cremers (2012) A generalized framework for opening doors and drawers in kitchen environments. In 2012 IEEE International Conference on Robotics and Automation, p. 3852–3858. Cited by: §I, §I. [43] J. Shi et al. (1994) Good features to track. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition, p. 593–600. Cited by: §I-B, §V-B. [44] J. Straub, T. Whelan, L. Ma, Y. Chen, E. Wijmans, S. Green, J. J. Engel, R. Mur-Artal, C. Ren, S. Verma, A. Clarkson, M. Yan, B. Budge, Y. Yan, X. Pan, J. Yon, Y. Zou, K. Leon, N. Carter, J. Briales, T. Gillingham, E. Mueggler, L. Pesqueira, M. Savva, D. Batra, H. M. Strasdat, R. D. Nardi, M. Goesele, S. Lovegrove, and R. Newcombe (2019) The Replica dataset: a digital replica of indoor spaces. arXiv preprint arXiv:1906.05797. Cited by: §S.1-B. [45] M. Stuede, K. Nuelle, S. Tappe, and T. Ortmaier (2019) Door opening and traversal with an industrial cartesian impedance controlled mobile robot. In IEEE International Conference on Robotics and Automation, p. 966–972. Cited by: §I. [46] J. Sturm, C. Stachniss, and W. Burgard (2011) A probabilistic framework for learning kinematic models of articulated objects. Journal of Artificial Intelligence Research 41, p. 477–526. Cited by: §I, TABLE I, §V. [47] Z. Teed and J. Deng (2021) Droid-slam: deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems 34, p. 16558–16569. Cited by: §S.3. [48] J. Wald, H. Dhamo, N. Navab, and F. Tombari (2020) Learning 3d semantic scene graphs from 3d indoor reconstructions. IEEE/CVF Conf. on Computer Vision and Pattern Recognition, p. 3960–3969. Cited by: §I. [49] C. Wang, I. Yeh, and H. Mark Liao (2024) Yolov9: learning what you want to learn using programmable gradient information. In European Conference on Computer Vision, p. 1–21. Cited by: §I-A. [50] A. Werby, M. Buechner, A. Roefer, C. Huang, W. Burgard, and A. Valada (2025) Articulated object estimation in the wild. Conference on Robot Learning. Cited by: §I, §S.2-A2, §I-B, §I-B, §I-B, §S.3, §IV, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, §V. [51] A. Werby, C. Huang, M. Büchner, A. Valada, and W. Burgard (2024-07) Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation. In Robotics: Science and Systems, Delft, Netherlands. External Links: Document Cited by: §S.1-B, §S.1-B, §I, §I. [52] S. Wu, J. Wald, K. Tateno, N. Navab, and F. Tombari (2021) Scenegraphfusion: incremental 3d scene graph prediction from rgb-d sequences. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition, p. 7515–7525. Cited by: §I. [53] F. Xiang, Y. Qin, K. Mo, Y. Xia, H. Zhu, F. Liu, M. Liu, H. Jiang, Y. Yuan, H. Wang, et al. (2020) Sapien: a simulated part-based interactive environment. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition, p. 11097–11107. Cited by: §I. [54] Z. Xu, Z. He, and S. Song (2022) Universal manipulation policy network for articulated objects. IEEE Robotics and Automation Letters 7 (2), p. 2447–2454. Cited by: §I. [55] A. Yu, Y. Chang, C. Xie, and L. Carlone (2025) Pandora: articulated 3d scene graphs from egocentric vision. In 36th British Machine Vision Conference 2025, Cited by: §I, §I-B, TABLE I, TABLE I, TABLE IV, TABLE V, §V. [56] C. Zhang, D. Han, Y. Qiao, J. U. Kim, S. Bae, S. Lee, and C. S. Hong (2023) Faster segment anything: towards lightweight sam for mobile applications. arXiv preprint arXiv:2306.14289. Cited by: §I-C. [57] C. Zhang, A. Delitzas, F. Wang, R. Zhang, X. Ji, M. Pollefeys, and F. Engelmann (2025) Open-vocabulary functional 3d scene graphs for real-world indoor spaces. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition, p. 19401–19413. Cited by: §I, §I. [58] H. Zhang, B. Eisner, and D. Held (2023) Flowbot++: learning generalized articulated objects manipulation via articulation projection. Conference on Robot Learning. Cited by: §I. [59] Y. Zhang, X. Huang, J. Ma, Z. Li, Z. Luo, Y. Xie, Y. Qin, T. Luo, Y. Li, S. Liu, et al. (2024) Recognize anything: a strong image tagging model. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition, p. 1724–1732. Cited by: §I-C. Throughout this supplementary material, we detail and discuss the following aspects of our proposed approach MoMa-SG: We provide additional details on the proposed articulation estimation scheme in Sec. S.1-A, provide explanations on keyframe construction and open-vocabulary mapping in Sec. S.1-B as well as the proposed object-to-articulation matching (Sec. S.1-C). This is followed by an in-depth explanation on our real-world mobile manipulation stack that includes localization given an ego-centric map, online grasps generation, end-effector trajectory sampling, and mobile manipulation from natural language in Sec. S.1-D. Finally, we provide additional quantitative and qualitative results in Sec. S.2 and add additional details on the proposed Arti4D-Semantic benchmark dataset in Sec. S.3. S.1 Additional Details on MoMa-SG S.1-A Regularized Twist Estimation: We expand upon the proposed geometric prior for regularized twist estimation introduced in Sec. I-B by depicting two estimation outputs. As depicted in Fig. S.2, we robustly estimate a prismatic twist that is governed solely by its translation component v while ‖ω‖\|ω\| approximates zero, even under partially noisy trajectories. Similarly, we display the estimation result on a revolute joint in Fig. S.3, which is prone to misclassification as a prismatic joint as its observed angular range is rather small (approx. 20∘ 20 ). Nonetheless, due to the geometric prior regularization term, we observe orthogonal twist components, effectively rendering the articulation as the revolute case of twists as described in Sec. I-B. Figure S.2: Estimated twist components given an observed prismatic motion. We depict observed 3D point trajectories (left), sampled point pairs used for articulation estimation (middle), and the estimated articulation model (right). Figure S.3: Estimated twist components given an observed revolute motion of small angular range using our proposed regularized twist estimation scheme. We S.1-B Open-Vocabulary 3D Part Mapping: Keyframe Construction: In order to construct O, we perform 3D part mapping on the frames that are not part of interaction segments S. Different from classical 3D scene graph construction methods [51, 12], we employ a keyframe selection heuristic that does not utilize a fixed stride. Owing to the fact that interactive camera trajectories captured in the real world are highly affected by dynamic camera motion, as opposed to those stemming from simulation or typical indoor datasets [44, 37], we select keyframes based on small motion blur. To this end, we compute the image Laplacian across all considered frames and verify whether it exceeds a certain threshold. In general, we find that especially thin boundaries between, e. g., close-by drawers require small motion blur to be accurately segmented as depicted in Fig. S.6. In addition, we only gather keyframes if a certain distance γ>γ∗γ>γ^* is traveled and a rotational difference β>β∗β>β^* is exceeded in between two camera poses k−1T_k-1 and kT_k: γ=∥k−k-1∥,β=arccos(tr(k−1k−1)2),γ= \|t_k-t_k-1 \| , β=arccos ( tr(R_k-1R_k^-1)2 ), (1) where ∈SO(3)R∈ SO(3) represents the rotational component and ∈ℝ3t ^3 the translational component. The above heuristic yields all frames taken as input to our 3D mapping stage. We visualize two exemplary keyframes in Fig. S.4. Input : Detections D, Objects O, Similarity matrix S Output : Updated free objects O for i←0i← 0 to ||rows−1|S|_rows-1 do j←argmax(Si)j (S_i) sij←(i,j)s_ij (i,j) d←id _i o←jo _j if o.isolatedo.isolated then if sij≥ϵisolateds_ij≥ _isolated or overlap(d,od,o) >ϵoverlap> _overlap then Oj←O_j← Merge(o,do,d) end if else if d.isolatedd.isolated then if overlap(o,do,d) >ϵoverlap> _overlap then o←o← Merge(o,do,d) o.isolated←Trueo.isolated end if else o←o← Merge(o,do,d) end if end if end for return O Algorithm 1 Incremental Merging Figure S.4: We depict dense 2D segmentation masks across two distinct keyframes and denote masks considered as isolated and non-isolated, respectively. Part-Aware Mapping: In this paragraph, we expand upon our 3D part mapping approach by providing additional details on the incremental merging scheme. To formalize this, we provide Alg. 1. During incremental mapping, we obtain a list of detections D per considered keyframe and aim to aggregate those as objects O. At each mapping step, we compute the axis-aligned bounding-box IoU across all elements of D as well as O, which yields the similarity matrix S. We sort and threshold S, and iteratively evaluate object-detection pairs, denoted as (d,o)(d,o) in the following. In general, we aim to find part segmentation masks with a certain level of confidence, which are attained by verifying masks via merging given significant point cloud-based overlap. We found that, in particular, part segmentation is prone to under- and over-segmentation, which frequently leads to erroneous merges when utilizing classical pipelines [12, 51]. While it is generally permissible to deal with wrong segmentation in typical search, navigation, or retrieval tasks, we find that mobile manipulation settings involving articulated objects require a higher level of fidelity. We aim to increase segmentation accuracy by evaluating whether the original 2D segmentation mask stemming from Semantic-SAM [27] belonging to either o or d is isolated, i. e., it does not intersect image boundaries. We depict this differentiation for a number of objects on the HSR split in Fig. S.4. If o stems from an isolated mask and sij>ϵisolateds_ij> _isolated or the point cloud-based overlap of d wrt. o is greater than ϵoverlap _overlap we merge d into o. If o is not isolated, i. e., it was originally observed at the image boundaries, we first check whether d is isolated and if true, check whether o is a strong inlier wrt. to d and merge d into o. In this fashion, we update the status of o to be an isolated object as the recent merge of d verified this. In case both d and o are non-isolated we proceed with merging d into o under the initial thresholding of S. We opt for this approach as we observed numerous cases of inconclusive segmentations hindering object articulation understanding. S.1-C Object-to-Articulation Matching In the following, we expand on the object-to-articulation matching logic introduced in the main manuscript in Sec. I-C. We divide obtained point tracks nX_n into dynamic and static points by evaluating the position variance along each track and filter out tracks with significant jumps due to noisy depth at object boundaries. As outlined in the main manuscript, we measure the ratios of dynamic and static points validating an object j. The proposed sum of those ratios yields our object-to-articulation matching score pijp_ij. In detail, we measure the ratio of visible dynamic 2D keypoints xkx_k that fall into the convex set Ω(oj2D) (o_j^2D) of the projected 2D object mask of node j∈j under a certain articulation state θ and, likewise, penalize the ratio of static keypoints within the interior: pij=∑t=tste∑k=1Fint(Ω)(xk,td)∑k=1Fℝd∖Ω(xk,td)+(1−∑k=1Fℝd∖Ω(xk,ts)∑k=1Fint(Ω)(xk,ts)),p_ij= _t=t_s^t_e _k=1^F1_int( )(x_k,t^d) _k=1^F1_R^d (x_k,t^d)+ (1- _k=1^F1_R^d (x_k,t^s) _k=1^F1_int( )(x_k,t^s) ), (2) where xk,tdx_k,t^d and xk,tsx_k,t^s denote dynamic and static points, respectively at time t. The above yields a mean association cost pijp_ij per articulation i and object j. We visualize the constructed convex sets Ω(oj2D) (o_j^2D) in Fig. S.5. Whenever we can not compute the associated convex set we resort to just the 2D-projected points of j (see Fig. S.5, column 1, row 2). Figure S.5: Qualitative results of articulation-to-object association scheme introduced in Sec. I-C. Across all depicted frames, the shaded mask represents the most-appropriate identified free object when articulated under the estimated articulation model that is projected into the camera frame as a red axis. Solid green image boundaries denote successful pairings, dashed yellow intermediate cases represent either wrong segmentations or misidentified articulation magnitudes under accurate articulation models, and dotted red boundaries denote wrongly associated object masks under permissible estimated axes. S.1-D Real-World Mobile Manipulation In this section, we outline the mobile manipulation stack for performing object opening and closing actions in the real world as well as mobile manipulation from natural language inputs. S.1-D1 Global Localization in Ego-Centric Maps In order to localize a robot within e. g., an ego-centric scene, accurate alignment is required while considering sensor drift throughout map construction. To accomplish this, we first localize the robot at the start of a trial globally. In order to allow for safe object manipulation, we perform an additional registration step before each object manipulation action. In the following, we detail the general approach utilizing 2D LiDAR sensors as this is the most common localization modality in mobile manipulation, as with the Toyota HSR. Ground Plane Estimation and Global Alignment: Given the ego-centric 3D map as constructed in Sec. S.1-B, we first perform robust ground plane estimation to align the recorded point cloud to a single plane normal in order to allow for 2D localization in the bird’s-eye-view. We employ random sample consensus (RANSAC) estimation by iterative sampling of three support points of the raw scene point cloud P in order to construct candidate planes described using a normal n_P. We then score all points i∈p_i by evaluating the number of inliers satisfying |⊤i+d|<ϵ,|n p_i+d|<ε, (3) where d constitutes a scalar offset and ϵε is a tuned distance threshold. This yields the predominant n and a set of inlier points. Given all the inlier points, we employ singular value decomposition (SVD) to refine the RANSAC-estimate n of the ground plane, ensuring an optimal fit to the ground surface. Once the ground plane (,d)(n,d) is estimated, we compute a rigid body transformation GW∈SE(3)^WT_G∈ SE(3) using the Rodrigues’ rotation formula to align the ego-centric map such that the ground plane coincides with the xyxy-plane, where z is the upwards-pointing height axis. GW^WT_G represents the transformation that allows us to transform objects of the constructed articulated 3D scene graph observed during the mapping stage into world coordinates when at robot deployment time. Construction of 2D Occupancy Grid: Given the aligned point cloud expressed in the world frame, we convert its 3D points into a 2D occupancy grid by slicing, projecting, and post-processing free and occupied regions. To do so, we slice the point cloud below 0.05m, which produces floor observations, while points in [ 0.15m, 0.30m][ 0.15m, 0.30m] are treated as occupied as they are observed by a 2D LiDAR. All points are then projected to a 2D grid using a world-to-voxel mapping while discarding points outside the grid bounds. Initially, all cells are denoted UNEXPLORED. We add floor points as FREE cells and then overwrite the produced grid with all occupied points, yielding OCCUPIED cells. We apply morphological closing and erosion to remove holes in the observed ground surface. This procedure yields a 2D occupancy grid that represents navigable space as well as object boundaries. At the beginning of each trial, we localize the mobile manipulator using particle filter-based adaptive Monte Carlo localization (AMCL) [10]. S.1-D2 Local Online Ego-centric Map Registration To maintain spatial consistency between the robot’s real-time observations and the precomputed global scene graph, we implement an online registration pipeline. This process aligns a local point cloud, generated from current RGB-D sensor data, with a localized subset of the global 3D map. Local Online Submaps: Before each manipulation action, we navigate to the object to be manipulated and acquire a depth image tD_t online and its corresponding RGB frame. Given the current estimated camera pose CW∈SE(3)^WT_C∈ SE(3), we project the depth map to a point cloud localP_local and truncate depth values beyond δmax=1.5m _max=1.5m, thus focusing on the object in question: localC=π−1(,t())∣t()<δmax^CP_local=\π^-1(u,D_t(u)) _t(u)< _max\ (4) where π−1π^-1 denotes the inverse projection from image coordinates u to 3D space. This local cloud is then transformed into the world frame: localW=CWlocalC^WP_local=^WT_C^CP_local. Egocentric Reference Submaps: In the next step, we register the live observation against a local crop of our ego-centric map subW⊆W^WP_sub ^WP in world coordinates since global fitting is usually prone to local minima. Given the centroid localc_local of localW^WP_local, we construct a kd-tree over the global 3D map and perform radius search, which yields subW=∈W∣‖−local‖2<rlocal^WP_sub=\q∈^WP \|q-c_local\|_2<r_local\ (5) where rlocal=0.8mr_local=0.8m defines the radius of the desired region. Registering the live observation against subW^WP_sub thus allows us to focus the registration process only on the geometry that is relevant to the object to be interacted with. Constrained Local Refinement: Finally, we align our submap subW^WP_sub against localW^WP_local using point-wise iterative closest point (ICP) registration. This results in a transformation W~ WT_W that allows us to transfer various primitives such as an object pose objG∈SE(3)^GT_obj∈ SE(3) from graph coordinates into a refined world frame W~ W of the robot: objW~=W~⋅GW⋅objG WT_obj= WT_W·^WT_G·^GT_obj (6) If necessary, one can further constrain the registration result W~ WT_W in terms of maximum-allowed rotation and translation. S.1-D3 Online Grasp Prediction Figure S.6: Selection of grasps constructed using our handle detection pipeline employing SAM3 [6] and predominant axis estimation. The blue axis of each grasp constitutes its approach axis. We sample grasp poses online using the robot’s head camera (HSR) / gripper camera (Spot) while facing the object to be manipulated. To do so, we employ SAM3 [6] with the prompt “handle“, which yields handle masks ℳk\M_k\. For each mask ℳkM_k, a partial point cloud is reconstructed given the corresponding depth image transformed into world coordinates given the above online registration. After filtering the obtained point cloud, we generate a candidate grasp pose by first computing the handle’s oriented bounding box. Let ∈ℝ3×3R ^3× 3 denote its rotation matrix and ℓ∈ℝ3 ^3 its half-extents. We identify the predominant axis of the found handle point cloud by identifying the longest axis p=:,argmaxiℓi,a_p=R_:,\, _i _i, (7) which corresponds to the pre-dominant principal dimension of the handle. We compute the min and max points along this axis of the bounding box and compute the tool center point as g=12(min+max),t_g= 12(p_ +p_ ), (8) representing its midpoint. In order to construct the grasp approach axis, we employ the normal θn_θ of the object to be opened/closed at its respective articulation state θ. Finally, we compute the closing axis given the cross product of the two aforementioned vectors. The resulting grasp pose is then defined as grasp=[−θ−p×θp.0001]T_grasp= bmatrix-n_θ&-a_p×n_θ&a_p&t_g.\\ 0&0&0&1 bmatrix (9) Naturally, a single observation of the scene can contain multiple handles. To associate candidate grasps with objects in the scene, we maintain world-frame point clouds and corresponding axis-aligned bounding boxes. For each object–grasp pair, we compute a compatibility score that consists of the IoU between grasp and object, the overlap of the grasp wrt. the object, and the distance of the involved centroids. The resulting score matrix defines a bipartite matching problem, which we solve for using the Hungarian algorithm. The grasp assigned to the target object is selected for manipulation. S.1-E End-Effector Trajectory Sampling To manipulate articulated objects, the robot must generate an end-effector trajectory that is kinematically consistent with the object’s articulation model. We synthesize this trajectory by first computing a view-dependent grasp pose and then propagating it under a range of articulation sates using the exponential map of the estimated twist ξ ξ. Given a grasp pose graspW~ WT_grasp in refined world coordinates W~ W and the object’s articulation model iA_i given as a twist ξ^∈(3) ξ∈ se(3) in graph coordinates, we sample an end-effector trajectory eeW~(θ) WT_e(θ) under the configuration states Θ=θ0,θ1,…,θn =\ _0, _1,…, _n\. For each θ∈Θθ∈ , the global end-effector pose is computed by evaluating the twist under the grasp pose in graph coordinates and transforming back to world coordinates. eeW~(θ)=W~⋅GW⋅(exp(ξ^θ)⋅graspG) WT_e(θ)= WT_W·^WT_G· ( ( ξθ)·^GT_grasp ) (10) where exp(⋅) (·) represents the exponential map from (3) se(3) to SE(3)SE(3), ξ ξ is the twist governing the articulation and graspG^GT_grasp is the initial grasp pose transformed into graph coordinates given Eq. 6. Depending on the fact, whether we perform an opening or a closing action, we reverse/negate the range of Θ . S.1-F Language-enabled Mobile Manipulation On top of the methodology of MoMa-SG, we demonstrate how its capabilities are integrated into an LLM-based planner, which allows to perform various object manipulations such as opening or closing objects. To do so, we define a discrete set of high-level actions that span observation, navigation, and manipulation skills and specify arguments to be parsed by each of those functions. In addition, we provide the LLM with the current state of all considered articulated objects at execution time: [(obj1,Mi),..,(objN,MN)][(obj_1,M_i),..,(obj_N,M_N)], where M constitutes a semantic abstraction mapping from articulation states to semantic states as introduced in Sec. I-C. Next, the LLM determines the most appropriate high-level action to be executed in an online, receding horizon fashion based on the provided scene context. Given a high-level action call, we propagate this instruction to various low-level controllers responsible for navigation, inspection, and manipulation. We consider the following set of actions: • open(o)(o): Navigate to an articulated object, door, or container o and open it. • inspect(o)(o): Navigate to an articulated object, door, or container o, register, and inspect its contents. • close(o)(o): Close an articulated object, door, or container o. • retrieve(o,p)(o,p): Retrieve object o from a parent container p. • done()(): Indicate that the task has been completed. Furthermore, we are also able to run the open(), close(), and retrieve() actions with a boolean retrial flag, which triggers an inspect() call after each performed action in order to check whether the desired state was actually reached. If not, we trigger a retrial. We find that this allows to minimize stochastic gripping failures occurring throughout manipulation. We employ the N2M2 reinforcement learning agent [15] as a whole-body controller by adapting it to accept general end-effector SE(3)SE(3) trajectories, which we sample as outlined in Sec. S.1-E. As the language model, we utilize GPT-5. We depict language-based long-horizon object manipulation in LABEL:fig:retrieve Figure S.7: We depict the ability of MoMa-SG to perform long-horizon mobile manipulation of articulated scenes using the Toyota HSR. Throughout this episode (left to right, top to bottom), the robot successfully opened 7 different articulated objects, 3 revolute objects, and 4 prismatic objects. Throughout our experiments, we found that grasping failures are much more dominant than errors due to mediocre articulation axes. Figure S.8: We depict three consecutive opening actions conducted using the Boston Dynamics spot. S.2 Experimental Evaluation S.2-A Metrics S.2-A1 Interaction Segmentation We evaluate the intersection segmentation using several metrics being a 1D-IoU over the time series, as well as a segment-based IoU that involves matching predicted segments against ground truth segments. The 1D IoU is computed by evaluating the binary time series of predicted and ground truth timestamps. In the following, we denote the time series as P and G and measure the Jaccard index: IoU1D=|P∩G||P∪G|if |P∪G|>00if |P∪G|=0IoU_1D= cases |P∩ G||P∪ G|&if |P∪ G|>0\\ 0&if |P∪ G|=0 cases (11) Furthermore, we evaluate temporal segmentation by matching predicted and ground truth interaction segments using bipartite matching. We compute the segment-wise IoU over pairs of predicted p=(sp,ep)p=(s_p,e_p) and a ground-truth interval g=(sg,eg)g=(s_g,e_g), respectively, and compute a segment-IoU: IoU(p,g)=max(0,min(ep,eg)−max(sp,sg))max(ep,eg)−min(sp,sg).IoU(p,g)= \! (0, (e_p,e_g)- (s_p,s_g) ) (e_p,e_g)- (s_p,s_g). (12) Given all pairs, this produces a cost matrix. We solve for an optimal pairing using the Hungarian algorithm to obtain a one-to-one matching maximizing total overlap. We discard pairs below an IoU of 0.50.5. This yields a precision P, a recall R as well as the mean segment-based IoU over all obtained pairs. For matched segment pairs, we additionally report the mean IoU as well as the mean onset and offset errors, where the errors for each pair are defined as |sp−sg||s_p-s_g| and |ep−eg||e_p-e_g|, respectively. TABLE S.1: Split-Wise Articulated Object Estimation on Arti4D-Semantic Split Paradigm GT Poses Prismatic joints Revolute joints Type Quality [%] θerr _err[deg] dL2d_L2[m] θerr _err[deg] dL2d_L2[m] Accuracy Prismatic Recall Revolute Recall RH078 EGO ✓ 15.83 – 11.70 0.038 0.850 0.760 1.000 R080 EGO ✓ 19.59 – 24.947 0.026 0.808 0.802 0.875 DR080 EGO ✓ 11.29 – 21.545 0.095 0.929 0.922 0.939 RH201 EGO ✓ 04.16 – 27.736 0.117 0.931 0.986 0.877 MHZH EGO × 58.91 – 39.567 0.600 0.784 0.727 0.808 MHZH-EXO EXO × 33.93 – 41.80 0.184 0.700 0.667 0.714 HSR ROBOT × 30.99 – 51.05 0.367 0.605 0.684 0.526 Overall 15.03 – 33.32 0.206 0.850 0.878 0.809 We report the axis-angle errors θerr _err, the positional errors dL2d_L2 of revolute joints, and the accuracy of predicted joint types including their respective recalls as in Tab. I. S.2-A2 Articulation Estimation In our work, we opted for using ground truth interaction segmentation in comparison to ArtiPoint in order to ensure comparability among all considered baselines as well as MoMa-SG. Interactions are labeled based on the first and last timestamps that show each particular articulated object in motion. That obviously means that across non-interaction timestamps, there is still dynamic motion induced by interactors/humans moving throughout the scene or grasping objects. Nonetheless, given ground-truth interaction segments, we hold the means to compare actual prediction-ground truth articulation pairs and follow the evaluation protocol introduced for Arti4D [50]: We compute the positional error did_i between a predicted articulation axis a and the corresponding ground truth axis gta_gt while relying on support points ^,gt p,p_gt along those axes: di=(^i−gt)⊤(^i×gt)‖^i×gt‖if ‖^i×gt‖>ϵ‖(^i−gt)×gt‖else ,d_i= cases ( p_i-p_gt) ( a_i×a_gt) a_i×a_gt &if a_i×a_gt >ε\\ ( p_i-p_gt)×a_gt &else cases, (13) where the first case covers non-parallel axes with ϵ=10−4ε=10^-4. If a method yields a twist, we rely on its own heuristic for reconstructing the associated axis of motion. Furthermore, we follow Arti4D in computing the angular axis errors by computing normalized dot-products: Θerr,i=cos−1(gt⊤^i‖gt‖‖^i‖). _err,i= ^-1 ( a_gt a_i _gt a_i ). (14) Similar to Arti4D, we do only report angular errors for prismatic joints, as the axis location of estimated prismatic objects is not informative. S.2-A3 Articulated 3D Part Segmentation In order to evaluate the articulated parent objects, we consider two evaluations: instance segmentation of articulated parents in their static non-opened state as well as semantic accuracy. Since the number of observed non-articulated objects may exceed the actual number of articulated objects, we decided to evaluate the IoU and recall at various IoU’s when matching all predicted (articulated) objects against the set of ground truth articulated objects that are labeled as part of Arti4D-Semantic. In general, we opt for evaluating all articulated objects in their closed state. Given its objects’ articulation model, we are able to transform its associated mask into the closed state. Instance Segmentation: We define a match between a predicted part PiP_i and a ground truth part GjG_j based on their 3D Intersection over Union (IoU) of their axis-aligned bounding boxes. Across all potential pairs (i,j)(i,j), this constitutes an IoU matrix ∈ℝN×MA ^N× M over N predictions and M ground truth objects. To find the optimal global assignment, we solve the bipartite matching problem using the Hungarian algorithm. This yields an optimal set of pairs between predicted objects and ground truth objects. First, we compute the mean over all axis-aligned IoUs of the found object pairs, which yields our IoU measure. Second, we compute the recall |TP|M |TP|M at the IoU thresholds 0.25,0.50,0.75\0.25,0.50,0.75\ in order to quantify the geometric accuracy. Thus, whenever an object pair exhibits an IoU greater than, e. g., 0.250.25, it is counted as a true positive. TABLE S.2: Split-Wise Articulated 3D Part Segmentation on Arti4D-Semantic Split Paradigm GT Poses IoU Recall @0.25 @0.50 @0.75 RH078 EGO ✓ 0.658 0.976 0.862 0.316 R080 EGO ✓ 0.670 0.993 0.867 0.350 DR080 EGO ✓ 0.639 0.948 0.811 0.269 RH201 EGO ✓ 0.631 0.911 0.832 0.271 MHZH EGO × 0.550 0.793 0.728 0.258 MHZH-EXO EXO × 0.588 0.917 0.750 0.104 HSR ROBOT × 0.600 1.000 0.775 0.200 Overall 0.631 0.943 0.818 0.265 We report the same set of metrics as in Tab. IV. Semantic Accuracy: To evaluate the quality of the articulated parts’ CLIP features, we resort to a retrieval-based evaluation. We evaluate whether the actual ground truth category is among the k highest-similarity responses when scoring the CLIP feature of the object against all considered GT objects. For this evaluation, we follow the optimal assignment obtained above. For each matched pair (Pi,Gj)(P_i,G_j), we extract the predicted CLIP feature vector if_i. We then compute the cosine similarity between if_i and the set of all potential category embeddings semC_sem. S(i,c)=i⋅c‖i‖‖c‖S(i,c)= f_i·w_c\|f_i\|\|w_c\| (15) where cw_c is the text embedding for category c∈semc _sem. Thus, we report the Top-k accuracy (for k∈1,2,5,10k∈\1,2,5,10\), measuring to which degree the correct category cgtc_gt is ranked among the top-k highest similarity scores. TABLE S.3: Articulated 3D Part Semantics Evaluation on Arti4D-Semantic Object Set Split top1top_1 top2top_2 top5top_5 top10top_10 O RH078 0.387 0.506 0.660 0.830 R080 0.069 0.174 0.452 0.532 DR080 0.489 0.663 0.774 0.888 RH201 0.410 0.605 0.781 0.915 MHZH 0.357 0.523 0.725 0.980 MHZH-EXO 0.333 0.562 0.896 1.000 HSR 0.444 0.669 0.850 0.975 Overall 0.355 0.529 0.734 0.875 O^A RH078 0.212 0.356 0.641 0.781 R080 0.026 0.123 0.403 0.549 DR080 0.305 0.424 0.609 0.778 RH201 0.374 0.480 0.758 0.935 MHZH 0.287 0.543 0.667 0.782 MHZH-EXO 0.271 0.396 0.583 0.812 HSR 0.531 0.662 0.769 0.875 Overall 0.287 0.426 0.633 0.787 We report open-vocabulary top-k retrieval scores as outlined in Sec. S.2-A. S.2-A4 Contained Objects Next, we provide additional details on the evaluation of retrieved child objects contained within articulated objects,. This includes two types of objects: ARTICULATED child objects exhibit the same motion pattern as their articulated parents (e. g., milk carton in fridge door), while the pose of STATIC child objects is independent of their articulated parent (e. g., jar behind cabinet door). We reflect this discrimination when evaluating contained objects in the following and follow the above evaluation routine by computing the IoU as well as the recall. The logic behind this is that many potential object parts observed could constitute a child object, which is task dependent. A simple example of this are multiple shelves contained in the fridge, which are natural child objects wrt. the fridge. Nonetheless, for our work, we aim to discover distinct contained objects such as the milk carton, the marmelade jar, or the butter, for instance. First of all, we assign detected child objects c to ground truth interaction segments by assessing whether the timestamp of observation lies within the ground truth interaction segment of the articulated parent. Once assigned, each child c is matched to the nearest ground truth child g∈parentg _parent by maximizing the voxelized point cloud-based interaction over union: IoUmax(c)=maxg∈parent(Pc∩PgPc∪Pg)IoU_max(c)= _g _parent ( P_c∩ P_gP_c∪ P_g ) (16) at a resolution of 1.5cm1.5cm, which yields the IoU as reported in Tab. V. Given this assignment, we also report the recall of found child objects above an IoU threshold of 0.250.25. Finally, we measure whether the containment type (STATIC vs. ARTICULATED) for each associated child is predicted correctly in terms of accuracy. Results are aggregated scene-wise and averaged across the entire dataset. S.2-B Articulation Estimation In the following, we expand on the results provided in the main paper and provide split-wise articulated object estimation results in Tab. S.1. While we observe roughly similar errors on the original splits of Arti4D, we also note considerably higher errors on the newly introduced splits of Arti4D-Semantic, which we interpret in the following. In general, both MHZH and MHZH-EXO do not provide ground truth camera poses. As a consequence, performing depth lifting will lead to considerable errors. Furthermore, we generally expect the exo-centric splits to be more difficult as the observation paradigm induces more dynamic motion due to humans close-by, which we visualize in Fig. S.10. This is further exacerbated by the number of objects that are quite texture-less as depicted in Fig. S.9. As a result, we do not obtain a significant number of keypoint trajectories required for accurate articulation estimation. Regarding the HSR split we note a significantly higher level of noise in the contained depth readings, which similarly increases articulation estimation errors. Figure S.9: Tracked keypoints of CoTracker3 [22] throughout interactions of texture-less objects on the MHZH split (ego-centric) of Arti4D-Semantic. Figure S.10: Tracked keypoints of CoTracker3 [22] on the MHZH-EXO split (exo-centric) of Arti4D-Semantic. S.2-C 3D Part Mapping In addition to the articulation estimation results, we provide split-respective results of MoMa-SG on 3D part segmentation in Tab. S.2. Overall, we find that our method performs similar across all considered splits. Furthermore, we have evaluated the quality of the CLIP features associated with each object in Tab. S.3. We follow the evaluation routine outlined in Sec. S.2-A and generally find a slight decrease in performance when pairing free objects with articulations to obtain O^A. Overall, we observe that the correct label is the top response in 28.7% of the cases, while it is among the top-5 in 63.3% of cases. S.2-D Real-World Long-Horizon Mobile Manipulation We provide additional qualitative insights into the performance of MoMa-SG in long-horizon mobile manipulation. First, we demonstrate that MoMa-SG is capable to perform 7 consecutive object manipulations in a row as visualized in Fig. S.7. We find that the predominant error source when performing long-horizon mobile manipulation are failed grasps, i. e., loss of grip while manipulating. Occasionally, we find that handles are wrongly associated with close-by objects when the object to be manipulated is already opened. In addition, we depict consecutive long-horizon opening trials using the Boston Dynamics Spot quadruped in Fig. S.8. We obtain similar long-horizon results on the task of object closing, which is depicted in the supplementary video. We consider the task of closing to be more complicated, as one needs to first infer the precise articulation state of the object to be manipulated, while the robot needs to be positioned well to accomplish this. Furthermore, detecting handles is more challenging for the closing vs. the opening task. Finally, we also show how MoMa-SG is tightly integrated with natural language prompts as depicted in LABEL:fig:retrieve, where the task is to ”get milk from the fridge”. The executed high-level actions (see Sec. S.1-F) for this trial were: open(fridge), which involves navigating to an inspection position based on the object normal of the object to be manipulated, registration to perform fine-grained localization (see Sec. S.1-D), and the sole opening action from sampled end-effector trajectories (see Sec. S.1-E); inspect(fridge) from the previous inspection position to look inside of the fridge; retrieve(milk, fridge) to estimate a grasp for the prompted category ”milk” and to execute an end-effectory trajectory that takes the milk carton from the fridge along the approach axis of the grasp. In general, the retrieval trajectory is interchangeable with another more acceptable motion. Figure S.11: We visualize the three new dataset splits of Arti4D-Semantic: The ego-centric MHZH / exo-centric MHZH-EXO splits (top) and the robot-centric HSR split (bottom). S.3 Arti4D-Semantic In the following, we provide additional details on our proposed benchmark dataset Arti4D-Semantic that employs and extends Arti4D [50]. As part of our work, we do not only provide three additional dataset splits that contribute robot-centric (HSR) as well as exo-centric (MHZH/MHZH-EXO) demonstrations, but also provide labels for all articulated objects as well as the objects contained in them. The labels include instance segmentation of all objects and also semantic categories. We pair these new labels with the articulation axes of the interacted objects and their associated ground truth interaction segments. We visualize the three new splits of Arti4D-Semantic in Fig. S.11. We construct pseudo ground truth camera trajectories using DROID-SLAM [47]. In the following, we provide details on the involved labeling procedure. Across all articulated objects, we manually select a sufficient number of keyframes from which the articulated object is visible in its closed state. We label each articulated object given those keyframes and later fuse those labels by aggregating them in 3D world coordinates as depicted in Fig. S.12. Furthermore, we provide a semantic category to each parent object that is independently verified by another reviewer. Regarding contained objects, we select frames of maximum-opening for each articulated parent. Given those frames, we label each distinct, contained object and provide it with a semantic category. In addition to that, we also label the visible parts of the articulated parent object by differentiating between areas seen in its closed state (front face) and areas seen only during articulation, which provides us with regions-of-interest that let us infer whether a child object is STATIC or ARTICULATED. Thus, whenever a child label mask overlaps with the articulated part mask during interaction, we consider it ARTICULATED, otherwise as STATIC. Overall, our dataset features 29 articulated parent categories and 54 child object categories. We provide an overview of all scenes contained in Arti4D-Semantic in Tab. S.4. Figure S.12: We visualize articulated 3D parent instance labels for two scenes contained in RH078 (top) and DIN080 (bottom), respectively. Each distinctly colored mask represents a closed articulated object. TABLE S.4: Overview of all demonstration sequences of Arti4D-Semantic. The RH078, R080, DR080, RH201, and MHZH splits are ego-centric. The MHZH-EXO split is exo-centric and the HSR split is robot-centric. We list the distribution of articulation types (PRISM / REV), difficulties, and the number of distinct interactions vs. the number of articulated objects. Split GT Paradigm Sequence Recording # # # # Parent Child Poses ID Objects Interactions PRISM / REV EASY / HARD Labels Labels RH078 ✓ EGO RH078-00 scene_2025-04-04-19-14-38 7 7 3 / 4 7 / 0 ✓ ✓ ✓ EGO RH078-01 scene_2025-04-04-19-18-54 6 7 2 / 4 4 / 2 ✓ ✓ ✓ EGO RH078-02 scene_2025-04-07-11-39-17 7 7 3 / 4 7 / 0 ✓ ✓ ✓ EGO RH078-03 scene_2025-04-07-11-41-52 8 9 3 / 5 6 / 2 ✓ ✓ ✓ EGO RH078-04 scene_2025-04-07-11-48-40 7 7 3 / 4 7 / 0 ✓ ✓ ✓ EGO RH078-05 scene_2025-04-09-10-30-11 8 8 5 / 3 3 / 5 ✓ ✓ ✓ EGO RH078-06 scene_2025-04-09-10-32-52 6 6 5 / 1 3 / 3 ✓ ✓ ✓ EGO RH078-07 scene_2025-04-09-10-35-47 7 7 6 / 1 2 / 5 ✓ ✓ ✓ EGO RH078-08 scene_2025-04-09-10-38-38 7 7 5 / 2 3 / 4 ✓ ✓ ✓ EGO RH078-09 scene_2025-04-09-10-46-48 7 8 7 / 0 4 / 3 ✓ ✓ ✓ EGO RH078-10 scene_2025-04-09-10-49-20 7 7 6 / 1 2 / 5 ✓ ✓ R080 ✓ EGO R080-00 scene_2025-04-10-13-11-16 17 17 14 / 3 9 / 8 ✓ ✓ ✓ EGO R080-01 scene_2025-04-10-16-05-09 14 14 13 / 1 8 / 6 ✓ ✓ ✓ EGO R080-02 scene_2025-04-17-15-25-14 11 12 11 / 0 7 / 4 ✓ ✓ ✓ EGO R080-03 scene_2025-04-17-15-33-44 9 9 9 / 0 7 / 2 ✓ ✓ ✓ EGO R080-04 scene_2025-04-22-09-53-49 8 8 8 / 0 5 / 3 ✓ ✓ ✓ EGO R080-05 scene_2025-04-22-09-56-24 10 10 9 / 1 9 / 1 ✓ ✓ ✓ EGO R080-06 scene_2025-04-22-09-58-49 7 8 7 / 0 4 / 3 ✓ ✓ ✓ EGO R080-07 scene_2025-04-22-11-45-15 9 9 8 / 1 7 / 2 ✓ ✓ ✓ EGO R080-08 scene_2025-04-22-11-48-01 9 9 8 / 1 7 / 2 ✓ ✓ ✓ EGO R080-09 scene_2025-04-22-11-50-40 8 8 7 / 1 6 / 2 ✓ ✓ DR080 ✓ EGO DR080-00 scene_2025-04-11-11-44-32 11 11 7 / 4 5 / 6 ✓ ✓ ✓ EGO DR080-01 scene_2025-04-11-12-58-58 10 10 6 / 4 5 / 5 ✓ ✓ ✓ EGO DR080-02 scene_2025-04-11-13-01-59 9 9 5 / 4 4 / 5 ✓ ✓ ✓ EGO DR080-03 scene_2025-04-11-13-18-00 9 10 4 / 5 3 / 6 ✓ ✓ ✓ EGO DR080-04 scene_2025-04-11-13-43-03 11 11 7 / 4 5 / 6 ✓ ✓ ✓ EGO DR080-05 scene_2025-04-11-14-01-06 11 11 7 / 4 5 / 6 ✓ ✓ ✓ EGO DR080-06 scene_2025-04-11-15-43-24 11 12 7 / 4 5 / 6 ✓ ✓ ✓ EGO DR080-07 scene_2025-04-11-15-46-48 11 11 6 / 5 4 / 7 ✓ ✓ RH201 ✓ EGO RH201-00 scene_2025-04-24-17-52-21 11 11 5 / 6 6 / 5 ✓ ✓ ✓ EGO RH201-01 scene_2025-04-24-17-54-13 9 9 5 / 4 5 / 4 ✓ ✓ ✓ EGO RH201-02 scene_2025-04-24-19-18-42 11 11 5 / 6 7 / 4 ✓ ✓ ✓ EGO RH201-03 scene_2025-04-24-19-21-50 8 8 2 / 6 5 / 3 ✓ ✓ ✓ EGO RH201-04 scene_2025-04-24-19-24-09 8 8 5 / 3 5 / 3 ✓ ✓ ✓ EGO RH201-05 scene_2025-04-25-10-36-37 9 9 5 / 4 7 / 2 ✓ ✓ ✓ EGO RH201-06 scene_2025-04-25-10-53-40 8 8 4 / 4 3 / 5 ✓ ✓ ✓ EGO RH201-07 scene_2025-04-25-10-56-33 8 8 3 / 5 3 / 5 ✓ ✓ ✓ EGO RH201-08 scene_2025-04-25-11-11-47 15 16 6 / 9 6 / 9 ✓ ✓ ✓ EGO RH201-09 scene_2025-04-25-11-15-47 7 7 5 / 2 4 / 3 ✓ ✓ ✓ EGO RH201-10 scene_2025-04-25-14-58-42 9 9 6 / 3 4 / 5 ✓ ✓ ✓ EGO RH201-11 scene_2025-04-25-15-02-14 7 7 3 / 4 4 / 3 ✓ ✓ ✓ EGO RH201-12 scene_2025-04-25-15-04-48 7 7 4 / 3 3 / 4 ✓ ✓ ✓ EGO RH201-13 scene_2025-04-25-15-16-29 9 9 5 / 4 3 / 6 ✓ ✓ ✓ EGO RH201-14 scene_2025-04-25-15-19-22 10 10 5 / 5 5 / 5 ✓ ✓ ✓ EGO RH201-15 scene_2025-04-25-15-22-54 8 8 4 / 4 3 / 5 ✓ ✓ MHZH × EGO MHZH-00 scene_20251205_171539 6 6 2 / 4 - / - ✓ ✓ × EGO MHZH-01 scene_20251206_152615 11 11 2 / 9 - / - ✓ ✓ × EGO MHZH-02 scene_20251206_153729 8 8 4 / 4 - / - ✓ ✓ × EGO MHZH-03 scene_20251206_154127 4 4 2 / 2 - / - ✓ ✓ × EGO MHZH-04 scene_20251206_154926 8 8 1 / 7 - / - ✓ ✓ MHZH EXO × EXO MHZH-EXO-00 scene_20251210_123247 4 4 0 / 4 - / - ✓ ✓ × EXO MHZH-EXO-01 scene_20251210_123710 4 4 1 / 3 - / - ✓ ✓ × EXO MHZH-EXO-02 scene_20251210_123920-0 6 6 3 / 3 - / - ✓ ✓ × EXO MHZH-EXO-03 scene_20251210_123920-1 6 6 2 / 4 - / - ✓ ✓ HSR × ROBOT HSR-00 scene_2026-01-05-15-44-54 5 5 2 / 3 - / - ✓ ✓ × ROBOT HSR-01 scene_2026-01-05-15-49-46 5 5 2 / 3 - / - ✓ ✓ × ROBOT HSR-02 scene_2026-01-05-15-53-13 5 5 2 / 3 - / - ✓ ✓ × ROBOT HSR-03 scene_2026-01-05-16-00-17 5 5 3 / 2 - / - ✓ ✓ × ROBOT HSR-04 scene_2026-01-05-16-02-59 5 5 3 / 2 - / - ✓ ✓ × ROBOT HSR-05 scene_2026-01-05-16-05-49 5 5 1 / 3 - / - ✓ ✓ × ROBOT HSR-06 scene_2026-01-05-16-09-34 4 4 2 / 2 - / - ✓ ✓ × ROBOT HSR-07 scene_2026-01-05-16-13-36 5 5 4 / 1 - / - ✓ ✓