Paper deep dive
Harnessing Coupled Stream Completion For Human-Object Ineraction Modeling
Dawei Guan, Di Yang, Jiangtao Wang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Text-conditioned human-object interaction (HOI) generation requires body motion, object trajectories & rotations, and hand articulation to remain coordinated. These components differ in scale and dynamics, but must agree on contact, relative pose, and timing. A shared representation may limit the distinct structure of each stream, while independent generation prevents each stream from responding to changes in the others. Latent supervision alone also does not directly constrain contact after decoding. We propose TRACE, a continuous latent framework that keeps stream states separate and couples their updates. TRACE encodes body, object, and hand motion into separate latents and predicts each stream velocity from the complete current interaction state. Geometric losses on decoded motion further constrain contact and object-relative motion over time. The same model supports completion of any single absent stream from the other two. Frozen flow features also serve as input to a language model for HOI understanding. Experiments on InterAct, OMOMO, and BEHAVE show that joint completion training improves generation and that frozen flow features improve understanding over raw-motion encoding. On InterAct, TRACE achieves the highest contact precision, recall, and F1 among the compared methods.
Tags
Links
- Source: https://arxiv.org/abs/2609.32551v1
- Canonical: https://arxiv.org/abs/2609.32551v1
Trouble viewing inline? Open PDF directly →
Full Text
98,527 characters extracted from source content.
Expand or collapse full text
Harnessing Coupled Stream Completion for Human–Object Interaction Modeling Dawei Guan Affiliation: School of Artificial Intelligence and Data Science, University of Science and Technology of Chinahttps://visdyn.github.io/TRACE/ Di Yang Affiliation: School of Artificial Intelligence and Data Science, University of Science and Technology of Chinahttps://visdyn.github.io/TRACE/ Jiangtao Wang Affiliation: School of Artificial Intelligence and Data Science, University of Science and Technology of Chinahttps://visdyn.github.io/TRACE/ Abstract Text-conditioned human-object interaction (HOI) generation requires body motion, object trajectories & rotations, and hand articulation to remain coordinated. These components differ in scale and dynamics, but must agree on contact, relative pose, and timing. A shared representation may limit the distinct structure of each stream, while independent generation prevents each stream from responding to changes in the others. Latent supervision alone also does not directly constrain contact after decoding. We propose TRACE, a continuous latent framework that keeps stream states separate and couples their updates. TRACE encodes body, object, and hand motion into separate latents and predicts each stream velocity from the complete current interaction state. Geometric losses on decoded motion further constrain contact and object-relative motion over time. The same model supports completion of any single absent stream from the other two. Frozen flow features also serve as input to a language model for HOI understanding. Experiments on InterAct, OMOMO, and BEHAVE show that joint completion training improves generation and that frozen flow features improve understanding over raw-motion encoding. On InterAct, TRACE achieves the highest contact precision, recall, and F1 among the compared methods. Figure 1: Given a text prompt, our TRACE generates coherent human-object interaction motions via a coupled latent flow, while its structured representation further supports stream completion and HOI understanding. 1 Introduction Text-conditioned Human-Object Interaction (HOI) Generation synthesizes a 3D motion sequence in which a human interacts with a dynamic object according to a language description. Realistic HOI motion supports animation, virtual reality, gaming, and robotics simulation. Recent generative models have extended text-conditioned motion synthesis from isolated human actions to interactions with moving objects (Peng et al., 2025; Diller and Dai, 2024; Li et al., 2024a; Wu et al., 2025a; Song et al., 2024; Wang et al., 2026b). Unlike ordinary text-to-motion, the object is not merely a static condition: its generated trajectory changes the feasible human motion throughout the sequence. HOI comprises body motion, object trajectories, and fine-grained hand articulation, which differ substantially in scale, dimensionality, and dynamics. Generating coherent interaction therefore requires preserving these distinct structures while aligning contact locations, relative poses, and interaction timing. These heterogeneity raise a central question: how can each stream retain its own structure while all streams stay coordinated? Recent generators coordinate motion through shared denoisers, separate branches, or auxiliary contact and relation guidance (Peng et al., 2025; Diller and Dai, 2024; Wu et al., 2025a; Song et al., 2024; Xue et al., 2025; Wang et al., 2026b). Two design choices matter here: how to represent each stream and how to exchange information across streams. A shared representation allows information exchange but may limit the distinct structure of each stream. Separate representations preserve this structure, but each update still needs access to the current states of the other streams. With independent dynamics, each update depends only on its own stream and the shared conditions. Even under the same text and geometry conditions, a body update cannot respond to the current object and hand states, and vice versa. The streams may therefore appear plausible individually while producing missed contact, object drift, or inconsistent interaction timing when combined. Compressed latent spaces reduce temporal redundancy and provide a smoother generation domain (Chen et al., 2023; Geng et al., 2025; Guan et al., 2026), but compression alone does not determine how contact information should pass among streams. Latent regression also leaves contact, relations in the object frame, and interaction timing unconstrained after decoding. More importantly, stream decomposition exposes natural conditional relations: any one component can be inferred from others. Conditional HOI methods show that one motion stream can be inferred from others (Li et al., 2023; Zhang et al., 2025). However, a multi-stream model supporting both complete HOI generation and completion of any missing stream, within a shared temporal model remains underexplored. Separate latent states need not imply independent generative dynamics. The distinction between state separation and dynamic coupling is essential. Body, object, and hand streams can retain specialized states and outputs, while each velocity depends on the complete current HOI state. Therefore, the model is joint in the information available to each update, but separate in what it represents and produces. When trajectories drift, every stream can respond to the current mismatch rather than only to static text and geometry conditions. Coordination is learned throughout generation instead of being imposed after motion is produced. Decomposing HOI into three streams naturally defines three completion tasks, each recovering one missing stream from the other two. These conditional relations define three single-missing-stream completion tasks within the same model. Features capable of predicting any missing stream are also encouraged to encode the action, manipulated object, and temporal progression, providing a natural semantic interface for understanding. We introduce TRACE, short for Tri-stream Representation And Stream Completion for Human-Object Interaction ModEling, a coupled continuous latent framework for text-conditioned Human-Object Interaction Generation, stream completion, and HOI understanding. As illustrated in Figure 1, TRACE generates diverse human-object interactions from text and object geometry while supporting stream completion and HOI understanding. TRACE follows a separation-and-coupling principle. A split temporal VAE (Kingma and Welling, 2013) maps body, hand, and object motion into separate but temporally aligned latents, preserving their distinct structures while reducing sequence length. These streams are not generated independently: a coupled latent flow (Lipman et al., 2023; Liu et al., 2023) predicts every stream-specific velocity from the complete evolving HOI state, allowing their updates to respond to one another throughout generation. To ground latent transport in interaction geometry, clean latent estimates are decoded during training and supervised for contact, object-frame anchoring, and interaction timing. The same representation and flow backbone support Text-to-HOI generation and completion of any missing stream, while frozen coupled-flow states provide continuous memory tokens to a language model for HOI understanding. Our contributions can be summarized into three parts: • We introduce TRACE, a novel coupled continuous latent framework for text-conditioned Human-Object Interaction Generation. The coupled flow preserves separate body, object, and hand states, while each velocity depends on the complete current HOI state. Decoded geometric losses further constrain contact and object-relative motion over time. • We reuse the coupled representation for completion and understanding. Completion of each missing stream is jointly trained with Text-to-HOI, while frozen flow features are bridged to a language model for HOI understanding. • Experiments on three datasets evaluate generation, stream completion, and HOI understanding. Ablations quantify the benefits of stream coupling and decoded geometric supervision for contact consistency, along with the effect of the latent representation. 2 Related Work Text-driven HOI Generation. Text-driven HOI generation has progressed from hand-object synthesis to dynamic whole-body interaction. Text2HOI generates detailed hand-object motion, while InterDiff predicts future human-object interactions from motion history (Cha et al., 2024; Xu et al., 2023). Recent diffusion-based methods extend language control to body motion and dynamic objects through initial-state and waypoint conditioning, physics-informed correction, contact or affordance guidance, and relation-aware generation (Li et al., 2024a; Peng et al., 2025; Diller and Dai, 2024; Wu et al., 2025a; Song et al., 2024). Other methods broaden the task toward zero-shot synthesis, long-horizon instruction following with physics tracking, and generation beyond the training domain (Xu et al., 2024; Wu et al., 2025b; Zhang et al., 2024). More recent work further targets geometric fidelity, long-sequence consistency, and modality-aware generation (Xue et al., 2025; Zeng et al., 2025; Geng et al., 2025; Wang et al., 2026b), reflecting growing attention to both interaction realism and controllability under increasingly complex human-object dynamics. These advances increasingly emphasize accurate coordination among articulated human motion, object trajectories, and contact events rather than treating object motion as a secondary condition. Despite this progress, synthesizing text-aligned body, object, and hand motion that remains spatially and temporally coherent throughout the interaction remains challenging. Motion Representation Learning. Motion generation has explored representations for preserving temporal structure and dependencies among motion components. Full-sequence representations jointly encode synchronized human-object trajectories (Xu et al., 2023; Li et al., 2024a), whereas factorized representations separate body, object, hand, or contact components (Cha et al., 2024; Peng et al., 2025; Song et al., 2024; Wang et al., 2026b). Interaction representations introduce explicit relations through contact variables, geometric fields, or kinematic chains (Diller and Dai, 2024; Wu et al., 2025a; Xue et al., 2025; Zeng et al., 2025). Latent representations compress motion into compact generation spaces. MLD uses continuous VAE latents (Chen et al., 2023), ARDHOI predicts continuous HOI tokens (Geng et al., 2025), and FlowCoMotion combines discrete tokens with continuous latents for text-conditioned flow (Guan et al., 2026). Conditional modeling treats observed motion as input. OMOMO and DiffGrasp condition human motion on object trajectories, showing that partial observations constrain missing interaction dynamics (Li et al., 2023; Zhang et al., 2025). These formulations balance compactness, component specialization, and motion coordination. TriDi (Petrov et al., 2025) models static human-object distributions with one diffusion transformer, while Uni-HOI (Zhang et al., 2026) and HOIGPT (Huang et al., 2025) use discrete representations for joint or bidirectional generation. Unlike LIGHT’s asynchronous denoising, TRACE couples temporal body, object, and hand streams through continuous latent flow, with each velocity depending on the complete current state. Motion-Language Understanding. Motion-language understanding studies mappings between motion and text, including retrieval, understanding, and generation. Token-based methods discretize motion into language-like units for reciprocal translation and instruction following (Guo et al., 2022b; Jiang et al., 2023). Alignment-based methods learn shared motion-text spaces (Li et al., 2024b), while MotionGPT3 (Zhu et al., 2026) connects continuous motion latents to language parameters for bidirectional generation. Interaction-centered models extend this interface to human-scene and human-object interactions. HSI-GPT and HSI-GPT2 jointly model scene, motion, and language (Wang et al., 2025; Wang et al., 2026a), while Uni-HOI models a joint distribution over text, human, and object motion (Zhang et al., 2026). HOIGPT (Huang et al., 2025) represents hand-object sequences with discrete codes for bidirectional generation. These studies suggest that richer motion representations can provide language models with more explicit interaction semantics. In particular, representations learned from generation may encode contact evolution, object-relative motion, and coordinated body-hand dynamics that are directly useful for semantic description. Whether frozen human-object generation features retain reusable interaction semantics therefore remains underexplored. 3 Methodology Overview. We formulate HOI generation as synthesizing a T-frame 3D motion sequence in which a human interacts with an object. As shown in Figure 2, the input consists of a text description d and canonical object geometry p (Prokudin et al., 2019), and the output is a synchronized sequence =(b,h,o) x=( x_b, x_h, x_o) of body, hand, and object streams. Following SMPL-H (Loper et al., 2015; Romero et al., 2017), b∈ℝT×22×3 x_b ^T× 22× 3 contains 22 body joint positions, while h∈ℝT×120 x_h ^T× 120 contains 30 hand joint positions and rotations. The object stream o∈ℝT×9 x_o ^T× 9 contains a 6D rotation (Zhou et al., 2019) and 3D translation. TRACE encodes the three streams into separate continuous latents, jointly evolves them through a coupled flow, and decodes them into motion. It also completes any missing stream ^xm x_m, m∈b,o,hm∈\b,o,h\, from d, p, and the observed streams −m x_-m. For HOI understanding, frozen flow hidden states are used to fine-tune a language model for interaction modeling. Figure 2: Overview of TRACE. Left: HOI Generation. We form different modalities, e.g., body, hand, and object, each compressed into latent vectors by its own encoder. After adding stream and temporal embeddings, a coupled flow transformer predicts their velocities, which are integrated by a flow sampler. Then, different modalities’ decoders reconstruct them into clean motion. Right: HOI Extensions. Leveraging its HOI representation, we enable HOI understanding via language model fine-tuning. Joint training with three completion paths, each targeting one stream, adds completion capability while improving HOI Generation. HOI Representation. Following Cha et al. (2024), we retain body, hand, and object motion as three explicit streams with distinct properties: articulated body motion, rigid object trajectories, and fine-grained hand movements. Separate encoders avoid forcing heterogeneous streams into one feature distribution, while temporal alignment preserves the frame-wise correspondence needed to recover contact and relative motion after decoding. The three streams share the same temporally aligned latent grid. The temporal VAE reduces its length by a factor s. We therefore construct a split temporal variational autoencoder (VAE) (Kingma and Welling, 2013), with a separate encoder and decoder (ℰm,m)(E_m,D_m) for each stream m∈b,o,hm∈\b,o,h\. The encoder compresses the original motion m∈ℝt×fm x_m ^t× f_m into a latent vector m∈ℝl×cm z_m ^l× c_m, where fmf_m and cmc_m denote the input and latent dimensions of the stream, respectively, and l=⌈t/s⌉l= t/s with temporal downsampling factor s. The latent encoding and reconstruction are defined as: qm(m∣m)=(m(m),m(m)),^m=m(m).q_m( z_m x_m)=N\! ( μ_m( x_m), _m( x_m) ), x_m=D_m( z_m). (1) Here, m(m) μ_m( x_m) and m(m) _m( x_m) denote the mean and diagonal covariance predicted by ℰmE_m, and ^m x_m denotes the reconstructed motion produced by mD_m. To preserve the motion characteristics of each stream after temporal compression, we train the split VAE with the following objective: ℒvae=ℒrec+ℒregL_vae=L_rec+L_reg. ℒrecL_rec denotes masked reconstruction over valid frames. The regularization term ℒregL_reg aims to promote reasonable Human-Object motion reconstruction, and it consists of four components. Specifically, ℒregL_reg combines object pose reconstruction, temporal velocity consistency, body kinematic consistency, and weak KL regularization, complementing masked reconstruction without overwhelming the continuous latent space. Full details are provided in Appendix A.1. After training, we freeze the VAE and the latent space produced by ℰmE_m. Each mean is standardized independently for each channel as ~m=(m(m)−¯m)/¯m z_m=( μ_m( x_m)- μ_m)/ σ_m, where ¯m μ_m and ¯m σ_m denote the channel normalization parameters of the latent space. The resulting ~m z_m is then used as the target latent vector for flow matching modeling. Latent Flow Modeling. Given the latent vectors ~m z_m produced by the frozen VAE, we learn a direct flow from independent Gaussian sources to the conditional distribution of HOI latents of the three streams, following flow matching (Lipman et al., 2023; Liu et al., 2023). For each stream m∈b,o,hm∈\b,o,h\, we independently sample m,0∼(,) z_m,0 ( 0, I), while sharing the flow time τ∼(0,1)τ (0,1) across the three streams. The intermediate state and its target velocity along a linear path are defined as: m,τ=(1−τ)m,0+τ~m,m,τ=~m−m,0. z_m,τ=(1-τ) z_m,0+τ z_m, u_m,τ= z_m- z_m,0. (2) Let τ=(b,τ,o,τ,h,τ) z_τ=( z_b,τ, z_o,τ, z_h,τ) denote the joint state; conditioned on (τ,τ,,)( z_τ,τ, d, p), the model predicts one velocity for each stream and trains them with: ℒfm=∑m∈b,o,h‖⊙(θ,m−m,τ)‖22cm‖1.L_fm= _m∈\b,o,h\ \| M ( v_θ,m- u_m,τ ) \|_2^2c_m \| M \|_1. (3) Here, M is the temporal validity mask, broadcast across latent channels, and the denominator normalizes each stream by its valid latent elements. Flow matching supervises latent velocity but not decoded interaction geometry. At time τ, the predicted velocity yields a clean latent estimate ^m,1=m,τ+(1−τ)θ,m z_m,1= z_m,τ+(1-τ) v_θ,m. The frozen VAE decodes the three latents while propagating gradients from decoded motion to the flow model. We further impose two interaction objectives: ℒcL_c supervises human-object contact with BCE and Dice losses (Cha et al., 2024; Dwivedi et al., 2025), while ℒA/EL_A/E preserves object anchoring and temporal consistency. The contact term supervises whether interaction occurs, whereas the anchoring and episode terms constrain where it occurs in the object coordinate frame and how the relation evolves over time. The final objective is: ℒflow=ℒfm+λcℒc+ℒA/E.L_flow=L_fm+ _cL_c+L_A/E. (4) It jointly supervises latent transport and decoded interaction geometry. Full details are provided in Appendix A.1. These constraints improve motion quality and interaction consistency (Figure 3 and Table 1). Coupled Flow. Separating latent states does not require factorized transport dynamics. For m∈b,o,hm∈\b,o,h\, an independent stream flow evolves each latent only from its own current state: dm,τdτ=θ,m(m,τ,τ,,),∂θ,m∂n,τ=,n≠m. d z_m,τdτ= v_θ,m( z_m,τ,τ, d, p), ∂ v_θ,m∂ z_n,τ= 0, n≠ m. (5) Although shared textual and geometric conditions specify the interaction, they do not let one stream’s velocity depend on other current states. Under independent Gaussian initialization, this induces conditionally factorized dynamics without a mechanism to reconcile body, object, and hand trajectories during transport. Thus, an object trajectory drifting from the hands cannot alter the body or hand updates, even though all streams share the same text and geometry conditions. We therefore couple the dynamics while preserving separate latent representations. TRACE evolves the separate stream latents through a single flow over the latent space =b×o×hZ=Z_b×Z_o×Z_h. Let τ=(b,τ,o,τ,h,τ) z_τ=( z_b,τ, z_o,τ, z_h,τ) denote the complete HOI state. The coupled dynamics and responses between streams are: dm,τdτ=θ,m(τ,τ,,),θ,mn(τ)=∂θ,m∂n,τ,m,n∈b,o,h. d z_m,τdτ= v_θ,m( z_τ,τ, d, p), J_θ,mn(τ)= ∂ v_θ,m∂ z_n,τ, m,n∈\b,o,h\. (6) Here, θ,mn J_θ,mn measures how stream m responds to stream n. Off-diagonal blocks need not vanish, enabling dependencies between streams. The vector field is joint in input but partitioned in output: each modality retains separate states and velocities while trajectories evolve jointly during transport. This joint input and separated output design couples transport without discarding the latent coordinates of each stream, allowing every frozen decoder to retain the motion statistics learned during VAE pretraining. Empirically, coupled flow improves motion quality and interaction consistency (Figure 5 and Table 3). Architecture. The left of Figure 2 illustrates the coupled flow transformer parameterizing θ v_θ. (i) Latent vectors. Each modal latent is linearly projected into a shared hidden dimension, then augmented with a learned stream embedding and sinusoidal temporal encoding: m0=ϕm(m,τ)+ms+t h_m^0= _m( z_m,τ)+ e_m^s+ e^t, m∈b,o,hm∈\b,o,h\. (i) Flow conditioning. The shared flow time τ is sinusoidally embedded and combined with the text and geometry condition to modulate attention and feed-forward layers via AdaLN (Peebles and Xie, 2023). (i) Text and geometry conditions. The description d is encoded by frozen DistilBERT (Sanh et al., 2019), while object geometry p is represented by BPS (Prokudin et al., 2019) and projected into a geometry token. They are concatenated and injected into each stream via cross-attention in every transformer block. (iv) Joint attention. Joint attention realizes the responses between streams θ,mn J_θ,mn required by the coupled flow. In each block r, the body, object, and hand streams form queries, keys, and values through separate projections. These features are concatenated for joint attention, split into individual streams, and processed by separate output projections and feed-forward networks: mr+1=ℬmr(br,or,hr,τ,,),m∈b,o,h. h_m^r+1=B_m^r ( h_b^r, h_o^r, h_h^r;τ, d, p ), m∈\b,o,h\. (7) Here, ℬmrB_m^r denotes the update of stream m. Since each state depends on all streams, joint attention establishes a differentiable path from n,τ z_n,τ to θ,m v_θ,m, enabling nonzero off-diagonal responses θ,mn J_θ,mn. (v) Routed velocity head. The velocity head maps coupled states to body, object, and hand velocities using a top-k mixture-of-experts (Shazeer et al., 2017). We use a shared top-k router to produce a common mixture of stream-specific output projections for each interaction. A shared router predicts weights from the joint state and flow condition, while each expert uses stream-specific projections: θ,m=∑e∈()πee,m(m),ℒr=E∑e=1E(π¯e−1E)2. v_θ,m= _e ( π) _e\,O_e,m( h_m), _r=E _e=1^E ( π_e- 1E )^2. (8) Here, ()K( π) denotes the selected experts, πe _e the normalized routing weight, and π¯e π_e its batch average. Shared routing weights mix all streams, while e,mO_e,m remains stream-specific. The routing loss prevents expert concentration, giving ℒ=ℒflow+λrℒrL=L_flow+ _rL_r. Stream Completion. The coupled tri-stream representation allows TRACE to recover any missing stream. Unified conditional generators similarly share parameters across multimodal mappings (Li et al., 2025; Petrov et al., 2025), while partial HOI sequences have been used to condition motion completion (Huang et al., 2025). We jointly train standard HOI generation and three single-stream completion tasks. For a missing stream m∈b,o,hm∈\b,o,h\, (−m,,)↦^m( x_-m, d, p) x_m. All four tasks share the same VAE, coupled flow backbone, and sampling head. Joint training adds completion capability without requiring separate generators. For each sample, observation and task embeddings identify the available streams and target m. Observed latents use τn=1 _n=1 for n≠mn≠ m, while the missing stream follows τm=τ _m=τ. Joint attention infers the missing motion from the observed trajectories. To preserve the conditions, observed hidden states are restored after each block and their latents remain fixed throughout sampling. Joint training on all three completion tasks also improves HOI generation through stronger coupled flow coordination (Figure 4, Table 1, E, A and B). HOI Understanding. For HOI understanding, we connect frozen coupled flow states to LLaDA and adapt it through LoRA (Hu et al., 2022) to describe a complete interaction. Given an interaction x, the frozen VAE first maps its body, object, and hand streams into clean normalized latents. We read the frozen flow at τ=1τ=1 under zero conditioning, which is included during flow training through 0.10.1 condition dropout. The final hidden states before the velocity head form the HOI representation =[b,o,h] h=[ h_b, h_o, h_h]. A learned readout embedding is added to each stream before the valid tokens are concatenated. A temporal slot resampler then compresses them into fixed memory tokens, followed by a projection into LLaDA’s embedding space as a continuous prefix (Nie et al., 2025; You et al., 2026): ^=ℱLM(,(ℛ())). d=F_LM ( q,P\! (R( h) ) ). (9) Here, ℛR and P denote the temporal resampler and prefix projection, while q contains the object category and learning query. The language branch reads frozen flow features without altering generation or completion. Full details on fine-tuning the language model are provided in Section A.2 of the Appendix. Empirically, harnessing the interaction relations learned during generation improves HOI understanding over directly encoding raw motion (Table 2). 4 Experiments Dataset. We mainly experiment on InterAct (Xu et al., 2025), with ablations on its major subsets BEHAVE (Bhatnagar et al., 2022) and OMOMO (Li et al., 2023). InterAct provides fine-grained textual annotations, and we follow its official train-test split. Since the data use both SMPL-H (Romero et al., 2017) and SMPL-X (Pavlakos et al., 2019), we convert all sequences to SMPL-H using the official conversion and consistently use SMPL-H joints. Figure 3: Qualitative comparison with baselines. Our method yields more realistic human-object interactions, fewer contact/penetration artifacts, more accurate finger positioning, and better text-motion alignment. Table 1: Quantitative comparisons on the InterAct dataset between our method and baseline approaches. We report R-Precision with a batch size of 256. Method R-Precision ↑ FID ↓ M Dist. ↓ Diversity → FSR ↓ Pene. ↓ Contact → Interaction ↑ Top 1 Top 2 Top 3 CprecC_prec CrecC_rec CF1C_F1 Ground Truth 0.600±0.0040.600 ± 0.004 0.834±0.0010.834 ± 0.001 0.909±0.0010.909 ± 0.001 0.000±0.0000.000 ± 0.000 1.475±0.0031.475 ± 0.003 7.781±0.1407.781 ± 0.140 0.083±0.2040.083 ± 0.204 0.076±0.0000.076 ± 0.000 0.208±0.0000.208 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 HOI-Diff 0.413±0.0100.413 ± 0.010 0.624±0.0090.624 ± 0.009 0.740±0.0110.740 ± 0.011 0.689±0.0310.689 ± 0.031 3.029±0.0073.029 ± 0.007 7.620±0.0857.620 ± 0.085 0.072±0.1600.072 ± 0.160 0.103±0.0100.103 ± 0.010 0.084±0.0060.084 ± 0.006 0.722±0.0020.722 ± 0.002 0.447±0.0260.447 ± 0.026 0.501±0.0220.501 ± 0.022 CHOIS 0.439¯±0.005 0.439 ± 0.005 0.660¯±0.002 0.660 ± 0.002 0.766¯±0.003 0.766 ± 0.003 0.572±0.0080.572 ± 0.008 2.781±0.0172.781 ± 0.017 7.717¯±0.115 7.717 ± 0.115 0.119±0.2340.119 ± 0.234 0.131±0.0130.131 ± 0.013 0.115±0.0020.115 ± 0.002 0.710±0.0120.710 ± 0.012 0.520±0.0040.520 ± 0.004 0.541±0.0070.541 ± 0.007 InterDiff 0.501±0.0090.501 ± 0.009 0.722±0.0030.722 ± 0.003 0.824±0.0070.824 ± 0.007 0.215±0.0010.215 ± 0.001 2.461±0.0132.461 ± 0.013 7.717¯±0.115 7.717 ± 0.115 0.092±0.1780.092 ± 0.178 0.116±0.0110.116 ± 0.011 0.124±0.0030.124 ± 0.003 0.715±0.0000.715 ± 0.000 0.562±0.0130.562 ± 0.013 0.584±0.0030.584 ± 0.003 Text2HOI 0.428±0.0030.428 ± 0.003 0.637±0.0000.637 ± 0.000 0.745±0.0000.745 ± 0.000 0.331±0.0030.331 ± 0.003 2.665±0.0022.665 ± 0.002 7.631±0.0007.631 ± 0.000 0.055¯±0.123 0.055 ± 0.123 0.105¯±0.003 0.105 ± 0.003 0.102±0.0010.102 ± 0.001 0.711±0.0010.711 ± 0.001 0.488±0.0000.488 ± 0.000 0.532±0.0020.532 ± 0.002 ROG 0.402±0.0070.402 ± 0.007 0.613±0.0080.613 ± 0.008 0.739±0.0090.739 ± 0.009 0.326±0.0210.326 ± 0.021 2.584±0.0182.584 ± 0.018 7.758±0.0767.758 ± 0.076 0.058±0.1470.058 ± 0.147 0.107±0.0050.107 ± 0.005 0.168±0.0030.168 ± 0.003 0.786±0.0050.786 ± 0.005 0.646±0.0090.646 ± 0.009 0.667±0.0070.667 ± 0.007 LIGHT 0.421±0.0140.421 ± 0.014 0.637±0.0160.637 ± 0.016 0.754±0.0160.754 ± 0.016 0.148±0.0140.148 ± 0.014 2.756±0.0182.756 ± 0.018 7.712±0.0507.712 ± 0.050 0.078±0.1970.078 ± 0.197 0.132±0.0010.132 ± 0.001 0.132±0.0010.132 ± 0.001 0.731±0.0020.731 ± 0.002 0.615±0.0160.615 ± 0.016 0.627±0.0100.627 ± 0.010 TRACE 0.404±0.0030.404 ± 0.003 0.615±0.0070.615 ± 0.007 0.744±0.0050.744 ± 0.005 0.199±0.0150.199 ± 0.015 2.256±0.0062.256 ± 0.006 7.927±0.0517.927 ± 0.051 0.047±0.1460.047 ± 0.146 0.117±0.0010.117 ± 0.001 0.196¯±0.001 0.196 ± 0.001 0.830¯±0.001 0.830 ± 0.001 0.810¯±0.002 0.810 ± 0.002 0.799¯±0.004 0.799 ± 0.004 TRACE joint joint 0.407±0.0030.407 ± 0.003 0.619±0.0080.619 ± 0.008 0.755±0.0010.755 ± 0.001 0.188¯±0.007 0.188 ± 0.007 2.262¯±0.008 2.262 ± 0.008 7.911±0.0637.911 ± 0.063 0.047±0.1480.047 ± 0.148 0.111±0.0010.111 ± 0.001 0.198±0.0010.198 ± 0.001 0.835±0.0010.835 ± 0.001 0.811±0.0030.811 ± 0.003 0.806±0.0030.806 ± 0.003 Metrics. Following Guo et al. (2022a), we repeat evaluation 20 times and report mean ± 95% confidence intervals. We evaluate HOI realism, diversity, text alignment, and physical plausibility. FID measures distribution similarity, Diversity measures motion variation, and R-Precision and M Dist assess motion-text alignment. Following LIGHT (Wang et al., 2026b), we report Foot Skating Ratio (FSR), Penetration Ratio (Pene), Contact Ratio (Contact), and frame-wise contact precision (CprecC_prec), recall (CrecC_rec), and F1 (CF1C_F1). These complementary criteria capture distribution-level fidelity and interaction-level correctness, which is essential because visually plausible motions may still exhibit weak semantic alignment or physically implausible contact. We therefore report all metrics under the same evaluator and test split, enabling consistent comparisons across generation, completion, and understanding settings in our experiments. For HOI understanding, following Jiang et al. (2023), we use Bleu@1/4, Rouge, Cider, and BertScore to evaluate generated descriptions, together with R-Precision and M Dist for motion-text alignment. Implementation Details. Object geometry is represented by a 10241024-point BPS (Prokudin et al., 2019). The split VAE uses temporal downsampling s=4s=4 and latent dimensions (cb,co,ch)=(128,64,128)(c_b,c_o,c_h)=(128,64,128). The coupled-flow backbone is a 1212 layer transformer with hidden dimension 10241024 and 1616 heads. The velocity head uses 44 experts with top-22 routing. Models are trained on one NVIDIA A800 GPU with BF16. The language branch uses a two-layer temporal resampler with 3232 memory tokens and rank 88 LoRA. Training uses AdamW with cosine learning-rate decay and gradient clipping for stable optimization of the coupled-flow backbone. We keep the VAE frozen during flow training and optimize the flow model jointly. For understanding, only the resampler, LoRA parameters, and output projection are updated, leaving the generator unchanged. Additional loss weights and schedules are provided in the Appendix A.1. Baselines. We compare against six recent HOI generation methods. HOI-Diff (Peng et al., 2025) uses transformer-based diffusion with affordance-guided sampling. CHOIS (Li et al., 2024a) is adapted to text-only conditioning by removing state and waypoint inputs. InterDiff (Xu et al., 2023) replaces its motion-history encoder with a text encoder. Text2HOI (Cha et al., 2024) is extended to full-body HOI while retaining its static contact-map estimator. ROG (Xue et al., 2025) combines motion diffusion with relation guidance and is retrained on InterAct. LIGHT (Wang et al., 2026b) follows its original setting. All methods use the same preprocessing, split, and evaluator, while adapted baselines retain their original objectives and training schedules. For HOI understanding, we compare with HOIGPT (Huang et al., 2025), TM2T (Guo et al., 2022b), MotionGPT (Jiang et al., 2023), and LaMPM2T (Li et al., 2024b). HOIGPT is extended with a body stream, while other baselines only adapt their motion encoders to our datasets. All models are retrained on InterAct with their original understanding architectures and objectives. TRACE Raw replaces the frozen VAE-flow encoder with a frame projection of normalized motion, followed by the same temporal resampler and prefix projection. Both variants share object prompts, LLaDA/LoRA configurations, language objectives, and training schedules, isolating frozen coupled-flow features from raw-motion encoding. Quantitative Evaluation. As shown in Table 1, TRACE achieves the strongest interaction accuracy: joint training produces the highest contact precision, recall, and F1, while our variants yield the lowest M Dist and FSR. Table 2 shows that TRACE performs best on R@1, M Dist, BLEU@4, and CIDEr for HOI understanding, with coupled-flow features outperforming raw-motion encoding on all metrics. Tables A, B, C, and D report generation and understanding on OMOMO and BEHAVE, confirming the benefits of joint training and flow-based HOI representations. Qualitative Evaluation. As shown in Figure 3, TRACE better follows descriptions while maintaining coordinated human-object interaction and stable contact, whereas baselines often omit actions or exhibit object drift, penetration, and floating artifacts. Figure A in the Appendix further shows the advantage of the complete tri-stream representation for HOI understanding. Figure 4: Qualitative results of stream completion. From left to right, we show the ground-truth interaction and the completion of object, hand, and body streams. Gray denotes observed motion, while pink, magenta, and teal denote the generated object, hands, and body, respectively. Table 2: Comparison of HOI understanding on InterAct. Our model is trained for 100 epochs. Methods R-Precision ↑ M Dist. ↓ Bleu@1 ↑ Bleu@4 ↑ Rouge ↑ Cider ↑ BertScore ↑ Top 1 Top 2 Top 3 Real 0.592 0.757 0.853 1.486 – – – – – TM2T 0.206 0.351 0.466 3.342 35.18 24.12 56.17 2.61 52.16 MotionGPT 0.224 0.391 0.504 3.086 39.86 29.63 60.84 3.42 57.83 LaMPM2T 0.243 0.415 0.532 2.894 44.37 34.42 66.14 4.09 63.18 HOIGPT 0.256 0.452 0.570 2.756 50.31 39.22 71.08 4.89 69.05 TRACE Raw Raw 0.232 0.408 0.512 3.020 41.57 30.82 62.24 3.55 58.81 TRACE 0.260 0.449 0.567 2.742 49.87 39.54 70.18 4.95 67.47 Stream Completion. As shown in Figure 4, TRACE completes a missing body, object, or hand stream from the other two observed streams. For body completion, the generated posture follows the fixed hand-object trajectories and preserves their spatial constraints over time. Object completion recovers a rigid trajectory consistent with the observed human motion, while hand completion aligns fine-grained movements with the body and object. In all three settings, the missing component is inferred from cross-stream dependencies rather than generated independently. The completed component remains coordinated with the observed trajectories, while the observed motion is preserved. Table E in the appendix reports stream-specific reconstruction and physical metrics on OMOMO, BEHAVE, and InterAct. Impact of Stream Modeling. We compare unified, independent, and coupled stream modeling. As shown in Table 3, the coupled model performs best in R-Precision, FID, M Dist, FSR, Contact, and all three interaction metrics. These results show that neither merging the streams nor evolving them independently captures their interaction dependencies as effectively as coupled flow. Figure 5 further shows that the alternative strategies often produce object drift or misaligned hand-object contact, whereas coupled modeling maintains coordinated trajectories throughout the interaction. Tables F and G in the Appendix exhibit the same overall trend on OMOMO and BEHAVE. Figure 5: Qualitative comparison of stream modeling. We compare unified, independent, and coupled stream modeling. Red boxes highlight object drift and hand-object misalignment in the first two variants, while coupled modeling yields coherent interactions. Table 3: Ablation study of stream modeling on InterAct. We report R-Precision with a batch size of 256. Strategy R-Precision ↑ FID ↓ M Dist. ↓ Diversity → FSR ↓ Pene. ↓ Contact → Interaction ↑ Top 1 Top 2 Top 3 CprecC_prec CrecC_rec CF1C_F1 GT 0.600±0.0040.600 ± 0.004 0.834±0.0010.834 ± 0.001 0.909±0.0010.909 ± 0.001 0.000±0.0000.000 ± 0.000 1.475±0.0031.475 ± 0.003 7.781±0.1407.781 ± 0.140 0.083±0.2040.083 ± 0.204 0.076±0.0000.076 ± 0.000 0.208±0.0000.208 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 Independent 0.383±0.0050.383 ± 0.005 0.584±0.0070.584 ± 0.007 0.710±0.0050.710 ± 0.005 0.261±0.0170.261 ± 0.017 2.447±0.0082.447 ± 0.008 8.003±0.0568.003 ± 0.056 0.052¯±0.109 0.052 ± 0.109 0.103±0.0010.103 ± 0.001 0.146±0.0020.146 ± 0.002 0.756±0.0030.756 ± 0.003 0.579±0.0030.579 ± 0.003 0.606±0.0030.606 ± 0.003 Unified 0.400¯±0.004 0.400 ± 0.004 0.609¯±0.006 0.609 ± 0.006 0.737¯±0.004 0.737 ± 0.004 0.213¯±0.016 0.213 ± 0.016 2.306¯±0.007 2.306 ± 0.007 7.896±0.0547.896 ± 0.054 0.059±0.1330.059 ± 0.133 0.130±0.0010.130 ± 0.001 0.186¯±0.002 0.186 ± 0.002 0.818¯±0.002 0.818 ± 0.002 0.764¯±0.002 0.764 ± 0.002 0.760¯±0.002 0.760 ± 0.002 Coupled 0.404±0.0030.404 ± 0.003 0.615±0.0070.615 ± 0.007 0.744±0.0050.744 ± 0.005 0.199±0.0150.199 ± 0.015 2.256±0.0062.256 ± 0.006 7.927±0.0517.927 ± 0.051 0.047±0.1460.047 ± 0.146 0.117¯±0.001 0.117 ± 0.001 0.196±0.0010.196 ± 0.001 0.830±0.0010.830 ± 0.001 0.810±0.0020.810 ± 0.002 0.799±0.0040.799 ± 0.004 Table 4: Ablation study of latent representation on InterAct. We report R-Precision with a batch size of 256. Space R-Precision ↑ FID ↓ M Dist. ↓ Diversity → FSR ↓ Pene. ↓ Contact → Interaction ↑ Top 1 Top 2 Top 3 CprecC_prec CrecC_rec CF1C_F1 GT 0.600±0.0040.600 ± 0.004 0.834±0.0010.834 ± 0.001 0.909±0.0010.909 ± 0.001 0.000±0.0000.000 ± 0.000 1.475±0.0031.475 ± 0.003 7.781±0.1407.781 ± 0.140 0.083±0.2040.083 ± 0.204 0.076±0.0000.076 ± 0.000 0.208±0.0000.208 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 Raw 0.399¯±0.003 0.399 ± 0.003 0.607¯±0.006 0.607 ± 0.006 0.735¯±0.004 0.735 ± 0.004 0.228¯±0.014 0.228 ± 0.014 2.314¯±0.007 2.314 ± 0.007 7.918±0.0527.918 ± 0.052 0.055¯±0.132 0.055 ± 0.132 0.135¯±0.001 0.135 ± 0.001 0.182¯±0.002 0.182 ± 0.002 0.809¯±0.002 0.809 ± 0.002 0.750¯±0.003 0.750 ± 0.003 0.748¯±0.003 0.748 ± 0.003 VQ 0.384±0.0040.384 ± 0.004 0.586±0.0060.586 ± 0.006 0.711±0.0040.711 ± 0.004 0.269±0.0160.269 ± 0.016 2.406±0.0072.406 ± 0.007 7.884±0.0537.884 ± 0.053 0.063±0.0010.063 ± 0.001 0.146±0.1840.146 ± 0.184 0.158±0.0020.158 ± 0.002 0.771±0.0020.771 ± 0.002 0.681±0.0030.681 ± 0.003 0.685±0.0030.685 ± 0.003 VAE 0.404±0.0030.404 ± 0.003 0.615±0.0070.615 ± 0.007 0.744±0.0050.744 ± 0.005 0.199±0.0150.199 ± 0.015 2.256±0.0062.256 ± 0.006 7.927±0.0517.927 ± 0.051 0.047±0.1460.047 ± 0.146 0.117±0.0010.117 ± 0.001 0.196±0.0010.196 ± 0.001 0.830±0.0010.830 ± 0.001 0.810±0.0020.810 ± 0.002 0.799±0.0040.799 ± 0.004 Impact of Latent Representation. We compare raw motion, discrete VQ latents, and continuous VAE latents under the same coupled-flow architecture. Table 4 shows that the VAE achieves the best R-Precision, FID, M Dist, FSR, penetration, contact, and interaction metrics. Raw motion also outperforms VQ on most metrics, suggesting that discrete quantization discards fine-grained interaction cues. By preserving continuous stream-specific information while reducing temporal redundancy, the VAE provides a more effective space for coupled flow modeling. Table 5: Ablation study of geometric constraints on InterAct. We report R-Precision with a batch size of 256. ℒcL_c ℒA/EL_A/E R@3 ↑ FID ↓ FSR ↓ Pene. ↓ Contact → Interaction ↑ CprecC_prec CrecC_rec CF1C_F1 × × 0.736±0.0040.736 ± 0.004 0.220±0.0160.220 ± 0.016 0.053±0.1380.053 ± 0.138 0.137±0.0020.137 ± 0.002 0.173±0.0020.173 ± 0.002 0.796±0.0030.796 ± 0.003 0.708±0.0040.708 ± 0.004 0.715±0.0040.715 ± 0.004 ✓ × 0.740¯±0.003 0.740 ± 0.003 0.208¯±0.015 0.208 ± 0.015 0.050¯±0.104 0.050 ± 0.104 0.126¯±0.001 0.126 ± 0.001 0.187¯±0.002 0.187 ± 0.002 0.814¯±0.002 0.814 ± 0.002 0.770¯±0.003 0.770 ± 0.003 0.765¯±0.003 0.765 ± 0.003 ✓ ✓ 0.744±0.0050.744 ± 0.005 0.199±0.0150.199 ± 0.015 0.047±0.1460.047 ± 0.146 0.117±0.0010.117 ± 0.001 0.196±0.0010.196 ± 0.001 0.830±0.0010.830 ± 0.001 0.810±0.0020.810 ± 0.002 0.799±0.0040.799 ± 0.004 Impact of Geometric Constraints. Table 5 evaluates the geometric constraints applied to the decoded interaction. Adding ℒcL_c to the flow objective consistently improves R@3, FID, FSR, penetration, and contact accuracy. Further introducing ℒA/EL_A/E achieves the best result on every reported metric. This progression shows that contact supervision improves local alignment, while object-local anchoring and temporal consistency further stabilize interaction geometry over time. 5 Conclusion We introduced TRACE, a structured continuous-latent framework for Text-to-HOI generation. TRACE preserves separate body, object, and hand states while conditioning each stream-specific velocity on the complete evolving HOI state; decoded-space geometric constraints further ground latent generation in contact locations, object-local relations, and interaction timing. The same generator handles Text-to-HOI and three stream completion tasks, while frozen interaction features support HOI understanding through a language bridge. Experiments across three datasets demonstrate generation and interaction performance, completion capability, and consistent understanding gains over raw-motion encoding. Together, these results show that structured cross-stream coupling provides a unified representation for coordinated generation, conditional reconstruction, and semantic interpretation across diverse human-object interaction tasks. References Bhatnagar et al. (2022) B. L. Bhatnagar, X. Xie, I. Petrov, C. Sminchisescu, C. Theobalt, and G. Pons-Moll BEHAVE: dataset and method for tracking human object interactions. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §4. Cha et al. (2024) J. Cha, J. Kim, J. S. Yoon, and S. Baek Text2HOI: text-guided 3d motion generation for hand-object interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 1577–1585. Cited by: §2, §2, §3, §3, §4. Chen et al. (2023) X. Chen, B. Jiang, W. Liu, Z. Huang, B. Fu, T. Chen, and G. Yu Executing your commands via motion diffusion in latent space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 18000–18010. Cited by: §1, §2. Diller and Dai (2024) C. Diller and A. Dai CG-HOI: contact-guided 3d human-object interaction generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §1, §1, §2, §2. Dwivedi et al. (2025) S. K. Dwivedi, D. Antić, S. Tripathi, O. Taheri, C. Schmid, M. J. Black, and D. Tzionas InteractVLM: 3d interaction reasoning from 2d foundational models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 22605–22615. Cited by: §3. Geng et al. (2025) Z. Geng, Z. Hayder, W. Liu, and A. S. Mian Auto-regressive diffusion for generating 3d human-object interactions. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence, Cited by: §1, §2, §2. Guan et al. (2026) D. Guan, D. Yang, C. Jin, and J. Wang FlowCoMotion: text-to-motion generation via token-latent flow modeling. arXiv preprint arXiv:2604.11083. Cited by: §1, §2. Guo et al. (2022a) C. Guo, S. Zou, X. Zuo, S. Wang, W. Ji, X. Li, and L. Cheng Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 5152–5161. Cited by: §4. Guo et al. (2022b) C. Guo, X. Zuo, S. Wang, and L. Cheng TM2T: stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts. In ECCV, Cited by: §2, §4. Hu et al. (2022) E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, Cited by: §A.2, §3. Huang et al. (2025) M. Huang, F. Chu, B. Tekin, K. J. Liang, H. Ma, W. Wang, X. Chen, P. Gleize, H. Xue, S. Lyu, K. Kitani, M. Feiszli, and H. Tang HOIGPT: learning long sequence hand-object interaction with language models. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, USA. Cited by: §2, §2, §3, §4. Jiang et al. (2023) B. Jiang, X. Chen, W. Liu, J. Yu, G. Yu, and T. Chen Motiongpt: human motion as a foreign language. Advances in Neural Information Processing Systems. Cited by: §2, §4, §4. Kingma and Welling (2013) D. P. Kingma and M. Welling Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114. Cited by: §1, §3. Li et al. (2024a) J. Li, A. Clegg, R. Mottaghi, J. Wu, X. Puig, and C. K. Liu Controllable human-object interaction synthesis. In ECCV, Cited by: §1, §2, §2, §4. Li et al. (2023) J. Li, J. Wu, and C. K. Liu Object motion guided human motion synthesis. ACM Trans. Graph. 42 (6). Cited by: §1, §2, §4. Li et al. (2025) S. Li, K. Kallidromitis, A. Gokul, Z. Liao, Y. Kato, K. Kozuka, and A. Grover OmniFlow: any-to-any generation with multi-modal rectified flows. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 13178–13188. Cited by: §3. Li et al. (2024b) Z. Li, W. Yuan, Y. He, L. Qiu, S. Zhu, X. Gu, W. Shen, Y. Dong, Z. Dong, and L. T. Yang LaMP: language-motion pretraining for motion generation, retrieval, and captioning. arXiv preprint arXiv:2410.07093. Cited by: §2, §4. Lipman et al. (2023) Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations (ICLR), Cited by: §1, §3. Liu et al. (2023) X. Liu, C. Gong, and qiang liu Flow straight and fast: learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations (ICLR), Cited by: §1, §3. Loper et al. (2015) M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black SMPL: a skinned multi-person linear model. ACM Trans. Graph.. Cited by: §3. Nie et al. (2025) S. Nie, F. Zhu, Z. You, X. Zhang, J. Ou, J. Hu, J. Zhou, Y. Lin, J. Wen, and C. Li Large language diffusion models. arXiv preprint arXiv:2502.09992. Cited by: §3. Pavlakos et al. (2019) G. Pavlakos, V. Choutas, N. Ghorbani, T. Bolkart, A. A. A. Osman, D. Tzionas, and M. J. Black Expressive body capture: 3d hands, face, and body from a single image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Cited by: §4. Peebles and Xie (2023) W. Peebles and S. Xie Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 4195–4205. Cited by: §3. Peng et al. (2025) X. Peng, Y. Xie, Z. Wu, V. Jampani, D. Sun, and H. Jiang HOI-diff: text-driven synthesis of 3d human-object interactions using diffusion models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Cited by: §1, §1, §2, §2, §4. Petrov et al. (2025) I. A. Petrov, R. Marin, J. Chibane, and G. Pons-Moll TriDi: trilateral diffusion of 3d humans, objects, and interactions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Cited by: §2, §3. Prokudin et al. (2019) S. Prokudin, C. Lassner, and J. Romero Efficient learning on point clouds with basis point sets. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Vol. , p. 4331–4340. External Links: Document Cited by: §3, §3, §4. Romero et al. (2017) J. Romero, D. Tzionas, and M. J. Black Embodied hands: modeling and capturing hands and bodies together. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia) 36 (6). Cited by: §3, §4. Sanh et al. (2019) V. Sanh, L. Debut, J. Chaumond, and T. Wolf DistilBERT, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. Cited by: §3. Shazeer et al. (2017) N. Shazeer, *. Mirhoseini, *. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean Outrageously large neural networks: the sparsely-gated mixture-of-experts layer. In International Conference on Learning Representations, Cited by: §3. Song et al. (2024) W. Song, X. Zhang, S. Li, Y. Gao, A. Hao, X. Hou, C. Chen, N. Li, and H. Qin HOIAnimator: generating text-prompt human-object animations using novel perceptive diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §1, §1, §2, §2. Wang et al. (2026a) Y. Wang, X. Li, Y. Li, X. Hou, and S. Wang HSI-gpt2: a dual-granularity large motion reasoning model with diffusion refinement for human-scene interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §2. Wang et al. (2025) Y. Wang, Y. Li, X. Li, and S. Wang HSI-gpt: a general-purpose large scene-motion-language model for human scene interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §2. Wang et al. (2026b) Z. Wang, S. Xu, C. Guo, B. Zhou, J. Gong, J. Wang, Y. Wang, and L. Gui Unleashing guidance without classifiers for human-object interaction animation. In ICLR, Cited by: §1, §1, §2, §2, §4, §4. Wu et al. (2025a) Q. Wu, Y. Shi, X. Huang, L. Xu, J. Yu, and J. Wang THOR: text to human-object interaction diffusion via relation intervention. In 2025 IEEE International Conference on Multimedia and Expo (ICME), Cited by: §1, §1, §2, §2. Wu et al. (2025b) Z. Wu, J. Li, P. Xu, and C. K. Liu Human-object interaction from human-level instructions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: §2. Xu et al. (2025) S. Xu, D. Li, Y. Zhang, X. Xu, Q. Long, Z. Wang, Y. Lu, S. Dong, H. Jiang, A. Gupta, Y. Wang, and L. Gui InterAct: advancing large-scale versatile 3d human-object interaction generation. In CVPR, Cited by: §4. Xu et al. (2023) S. Xu, Z. Li, Y. Wang, and L. Gui InterDiff: generating 3d human-object interactions with physics-informed diffusion. In ICCV, Cited by: §2, §2, §4. Xu et al. (2024) S. Xu, Z. Wang, Y. Wang, and L. Gui InterDreamer: zero-shot text to 3d dynamic human-object interaction. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, Cited by: §2. Xue et al. (2025) M. Xue, Y. Liu, L. Guo, S. Huang, and C. Ding Guiding human-object interactions with rich geometry and relations. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 22714–22723. Cited by: §1, §2, §2, §4. You et al. (2026) Z. You, S. Nie, X. Zhang, J. ZHOU, Z. Lu, J. Wen, and C. Li Llada-v: large language diffusion models with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 10093–10105. Cited by: §3. Zeng et al. (2025) L. Zeng, G. Huang, Y. Wei, S. Gu, Y. Tang, J. Meng, and W. Zheng ChainHOI: joint-based kinematic chain modeling for human-object interaction generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §2, §2. Zhang et al. (2026) M. Zhang, J. Zhang, and Z. Tu Uni-hoi: a unified framework for learning the joint distribution of text and human-object interaction. arXiv preprint arXiv:2604.27491. Cited by: §2, §2. Zhang et al. (2024) Y. Zhang, H. Yang, C. Luo, J. Peng, Y. Wang, and Z. Zhang Ood-hoi: text-driven 3d whole-body human-object interactions generation beyond training domains. arXiv preprint arXiv:2411.18660. Cited by: §2. Zhang et al. (2025) Y. Zhang, Q. He, Y. Wan, Y. Zhang, X. Deng, C. Ma, and H. Wang DiffGrasp: whole-body grasping synthesis guided by object motion using a diffusion model. In Proceedings of the AAAI Conference on Artificial Intelligence, Cited by: §1, §2. Zhou et al. (2019) Y. Zhou, C. Barnes, L. Jingwan, Y. Jimei, and L. Hao On the continuity of rotation representations in neural networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §3. Zhu et al. (2026) B. Zhu, B. Jiang, S. Wang, S. TANG, T. Chen, L. Luo, Y. Zheng, and X. Chen MotionGPT3: human motion as a second modality. In International Conference on Learning Representations, Cited by: §2. Appendix In this Appendix, we provide additional details of the training objectives, completion training, and language adaptation in Sec. A, followed by additional experimental results in Sec. B. LLM Usage. We employ large language models (LLMs), such as ChatGPT, to assist in polishing our paper. Specifically, LLMs are used to correct grammatical errors, refine word choice, and improve overall fluency. We do not use LLMs to formulate our methodology or run experiments. Appendix A Additional Method Details This section first presents the objectives and training procedure of the generation and completion model, followed by details of the language adaptation branch. A.1 Generation and Completion Training Using the frozen VAE, we independently train the generation-only and joint generation-completion TRACE variants for 1000 epochs under the same optimization budget. VAE Objectives. The split VAE is trained using masked reconstruction together with four motion regularizers. All losses are averaged over valid frames. The reconstruction term combines the complete motion and object-stream errors, while the regularization term supervises object pose, temporal velocity, body kinematics, and latent distribution: ℒrec _rec =ℒ1(^,)+λoℒ1(^o,o), =L_1( x, x)+ _oL_1( x_o, x_o), (10) ℒreg _reg =λobjℒobj+λvelℒvel+λkinℒkin+βℒKL. = _objL_obj+ _velL_vel+ _kinL_kin+ _KL. Here, ℒobjL_obj applies Smooth L1 loss to object translation and a rotation loss 1−(tr(^⊤)−1)/21-(tr( R R)-1)/2. ℒvelL_vel matches first-order differences of human joint positions and object translation, while ℒkinL_kin matches the body bone vectors defined by the SMPL-H kinematic tree. ℒKLL_KL is averaged over the valid latent positions of the three streams. We set λo=0.25 _o=0.25, λobj=0.25 _obj=0.25, λvel=0.05 _vel=0.05, λkin=0.1 _kin=0.1, and β=10−4β=10^-4. Decoded Interaction Constraints. Given the clean latent estimate ^m,1 z_m,1, the frozen VAE reconstructs the complete interaction x. Let dt,jd_t,j and d^t,j d_t,j denote the ground-truth and predicted distances from human joint j to the object surface at frame t. We define the contact label and its differentiable prediction as ct,j=[dt,j<γ]c_t,j=1[d_t,j<γ] and c^t,j=sigmoid((γ−d^t,j)/σc) c_t,j=sigmoid((γ- d_t,j)/ _c), respectively. The contact objective is ℒc=ℒBCE(^,)+2ℒDice(^,),L_c=L_BCE( c, c)+2L_Dice( c, c), (11) where positive contacts receive a weight of 33. We use γ=0.05γ=0.05 m and σc=0.01 _c=0.01 m. To supervise relative interaction geometry, let t,j a_t,j be a human anchor and define its coordinates in the object frame as t,j=t⊤(t,j−t) r_t,j= R_t ( a_t,j- t_t). A soft contact weight wt,jw_t,j selects anchors close to the ground-truth object. Using Smooth L1 loss ρ(⋅)ρ(·), the anchoring and temporal episode objectives are ℒA=∑t,jwt,jρ((^t,j−t,j)/sA)∑t,jwt,j,L_A= _t,jw_t,j\,ρ\! (( r_t,j- r_t,j)/s_A ) _t,jw_t,j, (12) ℒE=1||∑s∈∑t,jwt,jwt+s,jρ((Δs^t,j−Δst,j)/sE)∑t,jwt,jwt+s,j,L_E= 1|S| _s _t,jw_t,jw_t+s,j\,ρ\! (( _s r_t,j- _s r_t,j)/s_E ) _t,jw_t,jw_t+s,j, (13) where Δst,j=t+s,j−t,j _s r_t,j= r_t+s,j- r_t,j and =1,4,8S=\1,4,8\. We use ℒA/E=0.01ℒA+0.005ℒEL_A/E=0.01L_A+0.005L_E, with sA=0.05s_A=0.05 m and sE=0.02s_E=0.02 m. These terms constrain both the contact location in the object frame and its evolution over time. Joint Generation and Completion. We represent each task using an observation mask ∈0,13 ω∈\0,1\^3, where ωm=1 _m=1 indicates that stream m is observed. Text-to-HOI uses =(0,0,0) ω=(0,0,0), whereas each completion task observes two streams and generates the remaining one. For an observed stream, we directly use its clean latent with flow time τm=1 _m=1 and set its target velocity to zero. The missing stream follows the same flow used for generation. Let ()=m∣ωm=0G( ω)=\m _m=0\ denote the generated streams. Its flow loss is normalized as ℒfm=3|()|∑m∈()ℒm,L_fm ω= 3|G( ω)| _m ( ω)L_m, (14) which retains the three-stream loss scale for both generation and completion. The four tasks are sampled with a ratio of 2:1:1:12:1:1:1 for Text-to-HOI, body completion, object completion, and hand completion. Observation and task embeddings indicate the available streams and the stream to be generated. A zero-initialized completion adapter is activated only for the missing stream in each transformer block. Because joint attention also updates observed hidden states, we restore them after every block and clamp their latents throughout Euler sampling. The original observed streams are copied into the final motion after decoding, ensuring that only the missing component is modified. Training Configuration. We optimize all models using AdamW with weight decay 10−410^-4 and gradient clipping at 1.01.0. The split VAE is trained for 1,0001,000 epochs with batch size 6464 and learning rate 2×10−42× 10^-4. The Text-to-HOI flow is trained for 10001000 epochs with batch size 128128 and learning rate 10−410^-4. Its learning rate is warmed up over the first 2%2\% of training and then reduced by cosine decay. The joint generation-completion model is also trained for 10001000 epochs. We set the routing-balance weight to 0.010.01 and use a completion-adapter dimension of 128128. Decoded interaction constraints are evaluated on at most 3232 samples using contact-aware windows of 6464 frames and flow times within [0.2,0.8][0.2,0.8]; their weights are linearly warmed up over the first 10%10\% of training. All models are trained with bf16 precision on NVIDIA A800 GPUs. At inference, we integrate the coupled flow using 3232 Euler steps. A.2 HOI Understanding and Language Adaptation We detail how frozen coupled-flow states are converted into continuous memory tokens and used to adapt the language model for HOI understanding. HOI Feature Extraction. Given a complete interaction x, the frozen VAE encodes its body, object, and hand streams into normalized latent vectors ~mm∈b,o,h\ z_m\_m∈\b,o,h\. We pass these latents through the frozen coupled flow transformer at τ=1τ=1. Both the text and geometry conditions are replaced with zeros, ensuring that the extracted representation depends only on the interaction motion. Denote the frozen coupled flow transformer before its velocity head by θT_θ. The extracted states are: (bL,oL,hL)=θ(~b,~o,~h,1,,). ( h_b^L, h_o^L, h_h^L )=T_θ ( z_b, z_o, z_h;1, 0, 0 ). (15) Since each state has exchanged information with the other streams through joint attention, hidden states encode both the motion of stream m and its relation to the complete interaction. We use =[bL,oL,hL] h=[ h_b^L, h_o^L, h_h^L] as the motion representation for language adaptation. HOI Language Bridge. The flow representation contains a number of tokens proportional to the motion duration, whereas the language model requires a compact input with a fixed size. We first add a learned readout stream embedding mr e_m^r to distinguish body, object, and hand tokens: r=[bL+br;oL+or;hL+hr]. h_r= [ h_b^L+ e_b^r; h_o^L+ e_o^r; h_h^L+ e_h^r ]. (16) A temporal resampler uses learned slots to attend to the valid tokens in r h_r. Each resampler block applies cross-attention from the slots to the HOI tokens, followed by slot self-attention and a feed-forward network. We use two blocks, eight attention heads, and 3232 learned slots. The resampled features are mapped into the embedding space of the language model using LayerNorm and a linear projection: =ℛ(r,),=(). S=R ( h_r, V ), E=P ( S ). (17) Here, V masks padded motion tokens, S denotes the resampled slots, and E is the continuous prefix passed to the language model. Language Model Fine-tuning. We use the following prompt for all samples: Object category: [CATEGORY] Question: Describe the human-object interaction. Answer: The textual query contains the object category and the instruction Describe the human-object interaction. We freeze the VAE, coupled flow transformer, and LLaDA backbone, and optimize the HOI language bridge together with LoRA parameters (Hu et al., 2022). Training uses a masked diffusion objective computed only over answer tokens. For each caption, we mask either a random nonempty subset of its tokens or the complete answer with probability 0.80.8, and train the language model to recover the original description. Raw-motion and flow representations use identical category prompts and contrast the correct HOI memory with a same-category mismatch. Let ℓi+ _i^+ denote the caption loss using the correct memory and ℓi− _i^- the loss using another interaction from the same category. The complete language objective is: ℒlang=ℒcap+λg1|ℐ|∑i∈ℐmax(0,δ+ℓi+−ℓi−),L_lang=L_cap+ _g 1|I| _i (0,δ+ _i^+- _i^- ), (18) where ℐI contains samples with a valid paired interaction. We set λg=0.5 _g=0.5 and δ=0.1δ=0.1. The bridge is first trained alone for 55 epochs, after which the LoRA parameters are enabled. We train for 100100 epochs with learning rates of 5×10−55× 10^-5 for the bridge and 2×10−62× 10^-6 for LoRA. LoRA uses rank 88, scale 1616, and dropout 0.050.05. During inference, LLaDA converts the continuous HOI memory into a textual description through 4848 denoising steps. Appendix B Additional Experimental Results This section reports additional generation and understanding results on OMOMO and BEHAVE, evaluates all three stream completion tasks across datasets, and presents qualitative results for HOI understanding. HOI Generation on Additional Datasets. Table A: Quantitative comparisons on the OMOMO dataset between our method and baseline approaches. We report R-Precision with a batch size of 256. Method R-Precision ↑ FID ↓ M Dist. ↓ Diversity → FSR ↓ Pene. ↓ Contact → Interaction ↑ Top 1 Top 2 Top 3 CprecC_prec CrecC_rec CF1C_F1 Ground Truth 0.318±0.0030.318 ± 0.003 0.560±0.0010.560 ± 0.001 0.731±0.0010.731 ± 0.001 0.000±0.0000.000 ± 0.000 1.499±0.0001.499 ± 0.000 7.400±0.0477.400 ± 0.047 0.012±0.0390.012 ± 0.039 0.067±0.0000.067 ± 0.000 0.262±0.0000.262 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 HOI-Diff 0.221±0.0050.221 ± 0.005 0.403±0.0010.403 ± 0.001 0.542±0.0010.542 ± 0.001 0.987±0.0230.987 ± 0.023 2.520±0.0222.520 ± 0.022 7.248±0.0667.248 ± 0.066 0.021±0.0580.021 ± 0.058 0.131±0.0050.131 ± 0.005 0.135±0.0020.135 ± 0.002 0.862±0.0000.862 ± 0.000 0.696±0.0000.696 ± 0.000 0.739±0.0010.739 ± 0.001 CHOIS 0.291±0.0000.291 ± 0.000 0.515±0.0090.515 ± 0.009 0.676±0.0080.676 ± 0.008 0.132±0.0160.132 ± 0.016 1.746±0.0021.746 ± 0.002 7.417±0.1047.417 ± 0.104 0.019±0.0530.019 ± 0.053 0.096±0.0040.096 ± 0.004 0.230¯±0.001 0.230 ± 0.001 0.937¯±0.000 0.937 ± 0.000 0.900¯±0.004 0.900 ± 0.004 0.910±0.0020.910 ± 0.002 InterDiff 0.312±0.0030.312 ± 0.003 0.548±0.0040.548 ± 0.004 0.718±0.0040.718 ± 0.004 0.163±0.0090.163 ± 0.009 1.587±0.0061.587 ± 0.006 7.387±0.0457.387 ± 0.045 0.025±0.0620.025 ± 0.062 0.100±0.0020.100 ± 0.002 0.206±0.0000.206 ± 0.000 0.940±0.0160.940 ± 0.016 0.863±0.0160.863 ± 0.016 0.890±0.0170.890 ± 0.017 Text2HOI 0.286±0.0010.286 ± 0.001 0.513±0.0010.513 ± 0.001 0.655±0.0010.655 ± 0.001 0.168±0.0000.168 ± 0.000 1.832±0.0061.832 ± 0.006 7.359±0.0347.359 ± 0.034 0.020±0.0560.020 ± 0.056 0.127±0.0020.127 ± 0.002 0.163±0.0000.163 ± 0.000 0.919±0.0010.919 ± 0.001 0.757±0.0020.757 ± 0.002 0.807±0.0020.807 ± 0.002 ROG 0.309±0.0060.309 ± 0.006 0.545±0.0070.545 ± 0.007 0.713±0.0060.713 ± 0.006 0.136±0.0180.136 ± 0.018 1.674±0.0121.674 ± 0.012 7.408±0.0807.408 ± 0.080 0.016¯±0.050 0.016 ± 0.050 0.093±0.0030.093 ± 0.003 0.221±0.0020.221 ± 0.002 0.928±0.0040.928 ± 0.004 0.846±0.0080.846 ± 0.008 0.876±0.0060.876 ± 0.006 LIGHT 0.302±0.0040.302 ± 0.004 0.538±0.0150.538 ± 0.015 0.707±0.0110.707 ± 0.011 0.099±0.0180.099 ± 0.018 1.657±0.0101.657 ± 0.010 7.443±0.0437.443 ± 0.043 0.013±0.0450.013 ± 0.045 0.100±0.0010.100 ± 0.001 0.194±0.0000.194 ± 0.000 0.940±0.0000.940 ± 0.000 0.851±0.0080.851 ± 0.008 0.881±0.0050.881 ± 0.005 TRACE 0.342¯±0.003 0.342 ± 0.003 0.601¯±0.006 0.601 ± 0.006 0.764¯±0.002 0.764 ± 0.002 0.087¯±0.009 0.087 ± 0.009 1.557¯±0.017 1.557 ± 0.017 7.422±0.1227.422 ± 0.122 0.018±0.0540.018 ± 0.054 0.089¯±0.005 0.089 ± 0.005 0.227±0.0020.227 ± 0.002 0.917±0.0090.917 ± 0.009 0.889±0.0090.889 ± 0.009 0.894±0.0100.894 ± 0.010 TRACE jointTRACE joint 0.344±0.0040.344 ± 0.004 0.611±0.0070.611 ± 0.007 0.768±0.0040.768 ± 0.004 0.077±0.0080.077 ± 0.008 1.541±0.0121.541 ± 0.012 7.433±0.0997.433 ± 0.099 0.018±0.0420.018 ± 0.042 0.084±0.0020.084 ± 0.002 0.232±0.0010.232 ± 0.001 0.919±0.0050.919 ± 0.005 0.901±0.0030.901 ± 0.003 0.904¯±0.004 0.904 ± 0.004 Table B: Quantitative comparisons on the BEHAVE dataset between our method and baseline approaches. We report R-Precision with a batch size of 256. Method R-Precision ↑ FID ↓ M Dist. ↓ Diversity → FSR ↓ Pene. ↓ Contact → Interaction ↑ Top 1 Top 2 Top 3 CprecC_prec CrecC_rec CF1C_F1 Ground Truth 0.559±0.0050.559 ± 0.005 0.910±0.0000.910 ± 0.000 0.926±0.0000.926 ± 0.000 0.000±0.0000.000 ± 0.000 1.483±0.0181.483 ± 0.018 6.940±0.1536.940 ± 0.153 0.181±0.2240.181 ± 0.224 0.094±0.0010.094 ± 0.001 0.204±0.0010.204 ± 0.001 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 HOI-Diff 0.167±0.0090.167 ± 0.009 0.298±0.0120.298 ± 0.012 0.358±0.0180.358 ± 0.018 1.463±0.0441.463 ± 0.044 4.152±0.0384.152 ± 0.038 6.723±0.0746.723 ± 0.074 0.179±0.2380.179 ± 0.238 0.171±0.0070.171 ± 0.007 0.100±0.0030.100 ± 0.003 0.646±0.0080.646 ± 0.008 0.515±0.0100.515 ± 0.010 0.519±0.0070.519 ± 0.007 CHOIS 0.219±0.0000.219 ± 0.000 0.396±0.0030.396 ± 0.003 0.492±0.0220.492 ± 0.022 0.705±0.0270.705 ± 0.027 3.423±0.0453.423 ± 0.045 6.863±0.0616.863 ± 0.061 0.191±0.2530.191 ± 0.253 0.128±0.0110.128 ± 0.011 0.170¯±0.003 0.170 ± 0.003 0.717±0.0180.717 ± 0.018 0.662±0.0000.662 ± 0.000 0.650±0.0050.650 ± 0.005 InterDiff 0.258±0.0270.258 ± 0.027 0.443±0.0190.443 ± 0.019 0.531±0.0270.531 ± 0.027 0.630±0.0330.630 ± 0.033 3.219±0.0403.219 ± 0.040 6.862±0.0666.862 ± 0.066 0.208±0.2640.208 ± 0.264 0.143±0.0090.143 ± 0.009 0.173±0.0050.173 ± 0.005 0.723±0.0010.723 ± 0.001 0.682¯±0.013 0.682 ± 0.013 0.676±0.0040.676 ± 0.004 Text2HOI 0.135±0.0030.135 ± 0.003 0.242±0.0000.242 ± 0.000 0.322±0.0080.322 ± 0.008 1.291±0.0731.291 ± 0.073 3.815±0.0263.815 ± 0.026 6.544±0.0136.544 ± 0.013 0.177±0.2330.177 ± 0.233 0.179±0.0030.179 ± 0.003 0.128±0.0010.128 ± 0.001 0.703±0.0110.703 ± 0.011 0.579±0.0070.579 ± 0.007 0.587±0.0080.587 ± 0.008 ROG 0.351±0.0100.351 ± 0.010 0.528±0.0130.528 ± 0.013 0.620±0.0160.620 ± 0.016 0.612±0.0460.612 ± 0.046 3.169±0.0313.169 ± 0.031 6.948±0.0846.948 ± 0.084 0.168±0.2310.168 ± 0.231 0.106±0.0060.106 ± 0.006 0.151±0.0030.151 ± 0.003 0.749¯±0.006 0.749 ± 0.006 0.628±0.0100.628 ± 0.010 0.656±0.0080.656 ± 0.008 LIGHT 0.277±0.0110.277 ± 0.011 0.475±0.0140.475 ± 0.014 0.561±0.0410.561 ± 0.041 0.481±0.0870.481 ± 0.087 3.277±0.0763.277 ± 0.076 6.961¯±0.066 6.961 ± 0.066 0.171±0.2450.171 ± 0.245 0.140±0.0040.140 ± 0.004 0.170¯±0.001 0.170 ± 0.001 0.731±0.0060.731 ± 0.006 0.720±0.0110.720 ± 0.011 0.697±0.0070.697 ± 0.007 TRACE 0.438¯±0.009 0.438 ± 0.009 0.584¯±0.004 0.584 ± 0.004 0.681¯±0.009 0.681 ± 0.009 0.688±0.0010.688 ± 0.001 3.145¯±0.011 3.145 ± 0.011 6.978±0.4476.978 ± 0.447 0.169¯±0.001 0.169 ± 0.001 0.163±0.0060.163 ± 0.006 0.141±0.0030.141 ± 0.003 0.737±0.0020.737 ± 0.002 0.650±0.0020.650 ± 0.002 0.653±0.0030.653 ± 0.003 TRACE joint joint 0.466±0.0040.466 ± 0.004 0.613±0.0170.613 ± 0.017 0.688±0.0170.688 ± 0.017 0.578¯±0.013 0.578 ± 0.013 3.044±0.0013.044 ± 0.001 6.906±0.3576.906 ± 0.357 0.175±0.0010.175 ± 0.001 0.122¯±0.006 0.122 ± 0.006 0.161±0.0010.161 ± 0.001 0.758±0.0090.758 ± 0.009 0.682¯±0.002 0.682 ± 0.002 0.682¯±0.006 0.682 ± 0.006 Tables A and B report generation results on OMOMO and BEHAVE, respectively. Following prior methods, all models are trained separately on each dataset using the corresponding training split. TRACE performs favorably on most generation metrics, producing lower feature distribution distances and stronger alignment between generated motion and text. The improvements in contact and physical metrics further indicate that the generated human and object trajectories remain coordinated. Although OMOMO and BEHAVE contain substantially fewer interactions than InterAct, the coupled flow can still learn effective dependencies among body, object, and hand motion. These results show that TRACE remains effective under different dataset scales and interaction distributions. Table C: Comparison of HOI understanding on OMOMO. Our model is trained for 100 epochs. Methods R-Precision ↑ M Dist. ↓ Bleu@1 ↑ Bleu@4 ↑ Rouge ↑ Cider ↑ BertScore ↑ Top 1 Top 2 Top 3 Real 0.345 0.609 0.778 1.499 – – – – – TM2T 0.231 0.421 0.598 2.173 80.62 74.18 90.87 7.46 91.72 MotionGPT 0.254 0.459 0.632 2.046 84.51 79.36 91.92 8.16 92.31 LaMPM2T 0.276 0.490 0.661 1.928 88.74 85.61 92.96 9.37 93.21 HOIGPT 0.291 0.522 0.688 1.824 91.56 85.18 93.58 9.08 93.14 TRACE Raw Raw 0.260 0.472 0.658 1.965 86.29 82.91 92.48 7.31 93.18 TRACE 0.316 0.553 0.715 1.692 93.77 90.12 94.20 8.80 93.30 Table D: Comparison of HOI understanding on BEHAVE. Our model is trained for 100 epochs. Methods R-Precision ↑ M Dist. ↓ Bleu@1 ↑ Bleu@4 ↑ Rouge ↑ Cider ↑ BertScore ↑ Top 1 Top 2 Top 3 Real 0.913 0.944 0.956 1.498 – – – – – TM2T 0.246 0.383 0.501 4.143 23.18 9.62 37.16 4.9 22.18 MotionGPT 0.276 0.421 0.544 3.846 26.43 11.57 40.13 5.9 24.16 LaMPM2T 0.306 0.459 0.586 3.596 29.14 13.92 43.09 6.7 26.12 HOIGPT 0.336 0.497 0.632 3.301 32.64 16.52 46.57 8.32 30.61 TRACE Raw Raw 0.290 0.438 0.555 3.756 27.52 12.27 41.29 6.1 24.35 TRACE 0.350 0.513 0.619 3.484 30.58 13.83 44.34 7.4 25.95 HOI Understanding on Additional Datasets. Tables C and D compare understanding on OMOMO and BEHAVE. Across both datasets, frozen coupled-flow features improve every reported metric over raw-motion encoding. Against external baselines, performance varies by dataset: TRACE is competitive on OMOMO and improves R@1 and R@2 on BEHAVE, but does not lead most BEHAVE language metrics. These results support the internal representation comparison without implying uniform superiority over all baselines. Table E: Stream completion on InterAct, OMOMO, and BEHAVE. For each dataset, one jointly trained checkpoint is evaluated under three conditioning settings. Dataset Setting Target Human reconstruction ↓ Object reconstruction ↓ Interaction ↑ B-MPJPE Root H-JPE Center V2V CprecC_ prec CrecC_ rec CF1C_ F1 InterAct Text/BPS only Object – – – 69.90 74.79 0.420 0.172 0.195 Hands – – 726.46 – – 0.390 0.148 0.164 Body 170.74 574.55 – – – – – – Streams only Object – – – 19.25 34.77 0.503 0.309 0.330 Hands – – 44.91 – – 0.669 0.613 0.610 Body 55.21 72.89 – – – – – – Full Object – – – 15.34 26.51 0.584 0.411 0.434 Hands – – 47.19 – – 0.679 0.629 0.625 Body 60.74 80.01 – – – – – – OMOMO Text/BPS only Object – – – 95.44 100.47 0.407 0.093 0.128 Hands – – 969.38 – – 0.379 0.075 0.107 Body 200.85 781.40 – – – – – – Streams only Object – – – 18.25 36.37 0.530 0.324 0.354 Hands – – 69.75 – – 0.669 0.595 0.594 Body 50.89 83.30 – – – – – – Full Object – – – 14.16 25.30 0.612 0.414 0.445 Hands – – 71.34 – – 0.665 0.592 0.592 Body 51.80 83.81 – – – – – – BEHAVE Text/BPS only Object – – – 52.70 60.37 0.382 0.135 0.155 Hands – – 580.85 – – 0.354 0.113 0.129 Body 183.56 419.61 – – – – – – Streams only Object – – – 21.99 37.55 0.407 0.261 0.275 Hands – – 61.76 – – 0.586 0.540 0.543 Body 68.75 80.87 – – – – – – Full Object – – – 18.58 31.34 0.484 0.322 0.342 Hands – – 76.61 – – 0.589 0.533 0.535 Body 68.87 81.30 – – – – – – Stream Completion on Three Datasets. Table E compares three conditioning settings for object, hand, and body completion. Text/BPS-only uses no observed motion, Streams-only removes the text and BPS embeddings, and Full uses all conditions. For Text/BPS-only, the generated target is evaluated with the synchronized ground-truth non-target streams. In the other settings, observed streams are copied unchanged. Each dataset uses one TRACE checkpoint, with results averaged over two evaluation seeds. B-MPJPE measures root-relative body-joint error, Root measures root-translation error, and H-JPE measures hand-joint error, all in m. Center and V2V measure object-center and surface errors in cm. CprecC_ prec, CrecC_ rec, and CF1C_ F1 are macro-averaged either-hand contact metrics under a 5cm threshold; contact is omitted when both hand and object are observed. Across all datasets, observed streams provide the main completion signal. Full conditioning consistently yields the lowest object reconstruction errors and the highest contact scores for object completion. For hand and body completion, Streams-only remains comparable and sometimes produces lower paired reconstruction errors, showing that the observed trajectories already provide most of the required motion information. Figure A: Qualitative HOI understanding. Adding hand and object evidence produces more complete interaction descriptions. T denotes the reference description. Qualitative Evaluation of HOI Understanding. Figure A analyzes the effects of motion evidence and representation on HOI understanding. In the plastic box and tripod examples, partial body and hand observations yield coarse descriptions, while including all three streams recovers more complete object movement and temporal progression. The floor lamp example further compares two representations under the same complete input. Coupled flow features recognize the orientation adjustment, whereas direct raw motion encoding omits this detail. These results show that both complete HOI evidence and the interaction relations learned during generation benefit HOI understanding. Table F: Ablation study of stream modeling on OMOMO. We report R-Precision with a batch size of 256. Strategy R-Precision ↑ FID ↓ M Dist. ↓ Diversity → FSR ↓ Pene. ↓ Contact → Interaction ↑ Top 1 Top 2 Top 3 CprecC_prec CrecC_rec CF1C_F1 GT 0.318±0.0030.318 ± 0.003 0.560±0.0010.560 ± 0.001 0.731±0.0010.731 ± 0.001 0.000±0.0000.000 ± 0.000 1.499±0.0001.499 ± 0.000 7.400±0.0477.400 ± 0.047 0.012±0.0390.012 ± 0.039 0.067±0.0000.067 ± 0.000 0.262±0.0000.262 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 Independent 0.325±0.0040.325 ± 0.004 0.570±0.0070.570 ± 0.007 0.731±0.0040.731 ± 0.004 0.120±0.0110.120 ± 0.011 1.701±0.0201.701 ± 0.020 7.467±0.1247.467 ± 0.124 0.015±0.0780.015 ± 0.078 0.077±0.0060.077 ± 0.006 0.174±0.0020.174 ± 0.002 0.849±0.0110.849 ± 0.011 0.651±0.0120.651 ± 0.012 0.696±0.0120.696 ± 0.012 Unified 0.339¯±0.003 0.339 ± 0.003 0.597¯±0.006 0.597 ± 0.006 0.759¯±0.003 0.759 ± 0.003 0.093¯±0.010 0.093 ± 0.010 1.590¯±0.018 1.590 ± 0.018 7.411±0.1187.411 ± 0.118 0.022±0.0320.022 ± 0.032 0.097±0.0060.097 ± 0.006 0.218¯±0.002 0.218 ± 0.002 0.906¯±0.010 0.906 ± 0.010 0.842¯±0.011 0.842 ± 0.011 0.855¯±0.011 0.855 ± 0.011 Coupled 0.342±0.0030.342 ± 0.003 0.601±0.0060.601 ± 0.006 0.764±0.0020.764 ± 0.002 0.087±0.0090.087 ± 0.009 1.557±0.0171.557 ± 0.017 7.422¯±0.122 7.422 ± 0.122 0.018¯±0.054 0.018 ± 0.054 0.089¯±0.005 0.089 ± 0.005 0.227±0.0020.227 ± 0.002 0.917±0.0090.917 ± 0.009 0.889±0.0090.889 ± 0.009 0.894±0.0100.894 ± 0.010 Table G: Ablation study of stream modeling on BEHAVE. We report R-Precision with a batch size of 256. Strategy R-Precision ↑ FID ↓ M Dist. ↓ Diversity → FSR ↓ Pene. ↓ Contact → Interaction ↑ Top 1 Top 2 Top 3 CprecC_prec CrecC_rec CF1C_F1 GT 0.559±0.0050.559 ± 0.005 0.910±0.0000.910 ± 0.000 0.926±0.0000.926 ± 0.000 0.000±0.0000.000 ± 0.000 1.483±0.0181.483 ± 0.018 6.940±0.1536.940 ± 0.153 0.181±0.2240.181 ± 0.224 0.094±0.0010.094 ± 0.001 0.204±0.0010.204 ± 0.001 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 1.000±0.0001.000 ± 0.000 Independent 0.414±0.0110.414 ± 0.011 0.550±0.0070.550 ± 0.007 0.643±0.0110.643 ± 0.011 0.842±0.0140.842 ± 0.014 3.392±0.0483.392 ± 0.048 7.056±0.0517.056 ± 0.051 0.194±0.1220.194 ± 0.122 0.142±0.0070.142 ± 0.007 0.104±0.0030.104 ± 0.003 0.676±0.0040.676 ± 0.004 0.451±0.0050.451 ± 0.005 0.486±0.0050.486 ± 0.005 Unified 0.435¯±0.010 0.435 ± 0.010 0.580¯±0.006 0.580 ± 0.006 0.678¯±0.010 0.678 ± 0.010 0.716¯±0.012 0.716 ± 0.012 3.188¯±0.046 3.188 ± 0.046 6.956±0.0306.956 ± 0.030 0.177¯±0.194 0.177 ± 0.194 0.174±0.0070.174 ± 0.007 0.136¯±0.003 0.136 ± 0.003 0.730¯±0.003 0.730 ± 0.003 0.622¯±0.004 0.622 ± 0.004 0.631¯±0.004 0.631 ± 0.004 Coupled 0.438±0.0090.438 ± 0.009 0.584±0.0040.584 ± 0.004 0.681±0.0090.681 ± 0.009 0.688±0.0170.688 ± 0.017 3.145±0.0403.145 ± 0.040 6.978¯±0.047 6.978 ± 0.047 0.169±0.1390.169 ± 0.139 0.163¯±0.006 0.163 ± 0.006 0.141±0.0030.141 ± 0.003 0.737±0.0020.737 ± 0.002 0.650±0.0090.650 ± 0.009 0.653±0.0030.653 ± 0.003 Impact of Stream Modeling on Additional Datasets. Tables F and G extend the comparison of unified, independent, and coupled stream modeling to OMOMO and BEHAVE. The coupled formulation achieves the strongest overall generation and interaction results on both datasets, consistent with the InterAct results in the main paper. This confirms that preserving separate stream states while coupling their evolution remains effective across different dataset scales and interaction distributions. Table H: Inference efficiency on InterAct. All methods generate a batch of 64 interactions, each containing 300 frames, on one NVIDIA A800 GPU using BF16. Method Time / batch (s) ↓ Time / sequence (s) ↓ Relative latency (×) ↓ HOI-Diff 35.20±0.1235.20^± 0.12 0.5500.550 10.210.2 InterDiff 42.88±0.3542.88^± 0.35 0.6700.670 12.512.5 LIGHT 192.76±0.57192.76^± 0.57 3.0123.012 56.056.0 TRACE 3.44±0.013.44^± 0.01 0.0540.054 1.01.0 Inference efficiency. We evaluate all methods on one NVIDIA A800 GPU using BF16, a batch size of 6464, and sequences of 300300 frames. Each method uses its official inference configuration. After five warm-up runs, we measure 1010 runs with CUDA synchronization before and after timing. Runtime includes sampling and motion decoding but excludes data loading, rendering, and file I/O. As shown in Table H, TRACE requires 3.44 seconds per batch and is 10.2×10.2×, 12.5×12.5×, and 56.0×56.0× faster than HOI-Diff, InterDiff, and LIGHT, respectively.