Paper deep dive
ECHO-G: Embodied Co-speech Humanoid mOtion Generation
Yizhao Li, Pusen Gao, Ming Wang, Shaojie Shen, Shuo Yang, Hao Xu
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Generating full-body co-speech motion for humanoid robots requires coordinating speech prosody, linguistic content, and embodiment-specific motion. To this end, we present ECHO-G, a framework jointly conditioned on speech audio and timed transcripts. Its Speech-Grounded Diffusion Transformer (SGDiT) combines frame-aligned acoustic features with token-level linguistic context, preserving their distinct granularities. Trained with rectified flow matching, it models one-to-many utterance-motion relationships directly in robot space. To support training and evaluation, we introduce a BEAT2-derived audio-text-robot dataset and a benchmark covering co-speech characteristics, robot-motion quality, and runtime efficiency. Comparative evaluation supports direct robot-space generation over the evaluated human-motion generation and retargeting pipelines, while modality ablations highlight the benefits of joint audio-text conditioning. We further demonstrate deployment on a physical humanoid robot. A complementary video-rating study also favors joint conditioning over the alternatives. The dataset and training, inference, and evaluation code are available through our project page.
Tags
Links
- Source: https://arxiv.org/abs/2609.39575v1
- Canonical: https://arxiv.org/abs/2609.39575v1
Trouble viewing inline? Open PDF directly →
Full Text
38,832 characters extracted from source content.
Expand or collapse full text
ECHO-G: Embodied Co-speech Humanoid mOtion Generation Yizhao Li Affiliation: Yizhao Li and Ming Wang are with Beihang University. Emails: liyizhao@buaa.edu.cn, wangming@buaa.edu.cn. Affiliation: Yizhao Li, Pusen Gao, and Shuo Yang are with Mondo Robotics. Email: shuo.yang@mondorobotics.com. Pusen Gao Affiliation: Pusen Gao and Shaojie Shen are with The Hong Kong University of Science and Technology. Emails: pgaoak@connect.ust.hk, eeshaojie@ust.hk. Affiliation: Yizhao Li, Pusen Gao, and Shuo Yang are with Mondo Robotics. Email: shuo.yang@mondorobotics.com. Ming Wang Affiliation: Yizhao Li and Ming Wang are with Beihang University. Emails: liyizhao@buaa.edu.cn, wangming@buaa.edu.cn. Shaojie Shen Affiliation: Pusen Gao and Shaojie Shen are with The Hong Kong University of Science and Technology. Emails: pgaoak@connect.ust.hk, eeshaojie@ust.hk. Shuo Yang Affiliation: Yizhao Li, Pusen Gao, and Shuo Yang are with Mondo Robotics. Email: shuo.yang@mondorobotics.com. Hao Xu †thanks: *Yizhao Li and Pusen Gao contributed equally to this work.†thanks: †Corresponding author: Hao Xu. Affiliation: Hao Xu is with Nanjing University. Email: xuhao3e8@nju.edu.cn. Abstract Generating full-body co-speech motion for humanoid robots requires coordinating speech prosody, linguistic content, and embodiment-specific motion. To this end, we present ECHO-G, a framework jointly conditioned on speech audio and timed transcripts. Its Speech-Grounded Diffusion Transformer (SGDiT) combines frame-aligned acoustic features with token-level linguistic context, preserving their distinct granularities. Trained with rectified flow matching, it models one-to-many utterance–motion relationships directly in robot space. To support training and evaluation, we introduce a BEAT2-derived audio–text–robot dataset and a benchmark covering co-speech characteristics, robot-motion quality, and runtime efficiency. Comparative evaluation supports direct robot-space generation over the evaluated human-motion generation and retargeting pipelines, while modality ablations highlight the benefits of joint audio–text conditioning. We further demonstrate deployment on a physical humanoid robot. A complementary video-rating study also favors joint conditioning over the alternatives. The dataset and training, inference, and evaluation code are available through our project page. Index Terms— Human and Humanoid Motion Analysis and Synthesis, Gesture, Posture and Facial Expressions, Humanoid Robot Systems, co-speech gesture generation, flow matching. I Introduction In human communication, gestures complement spoken content, convey emphasis, and organize the temporal structure of speech [1, 2]. Inspired by this coordination, we aim to equip speaking humanoids with body motion that reflects both how an utterance is spoken and what it conveys, while respecting the robot’s embodiment. We study full-body co-speech motion generation from the audio and timed transcript of the robot’s own utterance. Human co-speech research provides a foundation for learning speech–motion relationships. Representative methods explicitly combine acoustic features with transcript-derived linguistic features to model speech rhythm and content [3, 4, 5]. These methods primarily generate human-motion representations. Extending speech-conditioned generation to humanoids requires robot-specific motion references and an interface for their physical execution. Recent robot-oriented methods address the generation of such references from speech. RoboGesture [6] studies audio-driven streaming generation of upper-body and hand gestures, while PhysDrift [7] explores robot-native generation with speech and text encoders. These advances suggest a full-body robot co-speech generator should combine densely sampled acoustic cues with token-level linguistic content while preserving their distinct granularities. The one-to-many relationship between utterances and gestures [8] further motivates a generative formulation, and real-robot deployment favors direct prediction in robot space. In addition, existing public releases do not consistently provide paired audio–text–robot training data together with a benchmark spanning co-speech characteristics, robot-motion quality, and runtime efficiency. Motivated by these considerations, we present ECHO-G, a framework for full-body humanoid co-speech generation from speech audio and timed transcripts (Fig. ). Its Speech-Grounded Diffusion Transformer (SGDiT) adds frame-aligned acoustic features to motion tokens and retrieves token-level linguistic context through global–local cross-attention, preserving the distinct granularities of the two conditions. Trained with rectified flow matching [9], SGDiT models the one-to-many relationship between utterances and full-body robot motion, enabling different motions to be sampled for the same utterance. A fixed pretrained whole-body motion tracker executes the joint-position components of the generated references. To support training and evaluation in humanoid co-speech generation, we construct a BEAT2-derived robot-space dataset through retargeting and embodiment-specific quality filtering [4, 10]. We publicly release the dataset and code for training, inference, and evaluation, together with a benchmark covering co-speech characteristics, robot-motion quality, and runtime efficiency. Pipeline comparisons and modality ablations assess the generation-space and conditioning choices, while video ratings and physical demonstrations provide complementary perceptual and deployment evidence. Our contributions are threefold: • We present ECHO-G, a full-body humanoid co-speech generation framework that jointly uses speech audio and timed transcripts, and demonstrate its deployment on a physical humanoid. • We develop SGDiT, a rectified-flow model combining frame-aligned acoustic conditioning with global–local transcript cross-attention for one-to-many full-body robot-motion generation. • We release a BEAT2-derived dataset pairing speech audio and timed transcripts with full-body robot motion, together with training, inference, and evaluation code. The accompanying benchmark covers co-speech characteristics, robot-motion quality, and runtime efficiency. I Related Work Fig. 2: SGDiT architecture and tracking interface. Frame-aligned acoustic features are combined with noisy motion-frame features to form motion tokens. Contextual transcript embeddings provide a shared key–value memory for global and temporally biased text attention. Their attention distributions are fused within each transformer block before value aggregation and residual injection into the motion stream. Rectified-flow sampling produces robot-motion references, whose joint-position components are passed to a fixed whole-body motion tracker for execution. I-A Humanoid Whole-Body Motion Generation Recent whole-body tracking systems [11, 12] enable humanoids to execute diverse motion references. Such references can be obtained by retargeting human motion, as in GMR [10] and OmniRetarget [13], or generated from language instructions, as in FRoM-W1 [14] and TextOp [15]. OMG [16] further unifies language, audio, and human-motion conditioning within a shared generative framework. Within this broader setting, robot co-speech generation focuses on gestures accompanying spoken utterances. Yoon et al. [17] generate transcript-conditioned upper-body gestures and demonstrate execution on NAO. RoboPerform [18] generates humanoid motion from speech audio using a generic text prompt rather than the utterance transcript. RoboGesture [6] combines hierarchical semantic–acoustic conditioning with streaming generation of upper-body and hand gestures. PhysDrift [7] uses separate speech and text encoders for one-step robot-native motion generation. However, these approaches either omit utterance-specific audio–text conditioning, focus on upper-body motion, or do not fully specify the temporal organization of their multimodal features. ECHO-G combines frame-aligned acoustic features with token-level transcript embeddings for full-body robot-space generation, preserving their distinct granularities. We also provide paired audio–text–robot data and code for training, inference, and evaluation. I-B Holistic Human Co-Speech Motion Generation BEAT [19] provides multimodal speech–gesture data and introduces CaMN for integrating audio, text, and auxiliary conditions. Building on BEAT2, EMAGE [4] combines adaptive content–rhythm fusion with masked gesture modeling and compositional motion priors for holistic generation. DiffSHEG [20] jointly generates expressions and gestures through diffusion, while GestureLSM [5] combines flow matching, latent shortcut learning, and spatiotemporal modeling of body regions for efficient gesture generation. These methods primarily synthesize human-motion representations. Evaluation considers distributional fidelity, motion variation, and speech–motion alignment. Yoon et al. [3] introduced Fréchet Gesture Distance (FGD), and EMAGE [4] adopted skeleton-aware features for distributional evaluation. Audio2Gestures [8] examines motion diversity and multimodality, while beat-alignment measures assess temporal correspondence between motion and audio [21, 4]. Our benchmark adapts these evaluation dimensions to robot motion and complements them with measures of robot-motion quality and runtime efficiency. I Method Fig. 3: Temporal conditioning in the local transcript-attention branch. Orange and purple heatmaps show global and local attention weights, respectively. The blue heatmap represents the clipped Gaussian log-prior b~τn b_τ n derived from motion-frame times uτu_τ and token-center times mnm_n. Multiplying global weights by the exponentiated log-prior and renormalizing yields the local weights. Matrices are transposed for display; colored and dashed arrows indicate stronger and weaker attention, respectively. As shown in Fig. 2, ECHO-G generates full-body robot-motion references from speech audio and timed transcripts. SGDiT integrates acoustic and linguistic conditions within a rectified-flow model, while a fixed whole-body motion tracker executes the generated joint-position references. I-A Problem Formulation and Motion Representation Given speech audio a and its word-timed transcript y, ECHO-G models a conditional distribution over full-body robot-motion sequences: pθ(∣a,y),p_θ(R a,y), (1) where θ denotes the generator parameters and =[1,…,T]⊤∈ℝT×DR=[r_1,…,r_T] ^T× D contains T motion frames. For each frame τ=1,…,Tτ=1,…,T, we use D=39D=39 features: τ=[τ⊤,δψτ,(τloc)⊤,τ⊤]⊤,r_τ= [ ρ_τ ,\,δ _τ,\,(v_τ^loc) ,\,q_τ ] , (2) where τ∈ℝ6 ρ_τ ^6 encodes the base orientation using a 6-D rotation representation, δψτ∈ℝδ _τ is the inter-frame yaw increment, τloc∈ℝ3v_τ^loc ^3 is the base linear velocity in a yaw-aligned local frame, and τ∈ℝ29q_τ ^29 contains the robot’s joint angles in a fixed order. Absolute root translation is omitted to make the learning target invariant to global position offsets. The generator operates on normalized motion features: τ=(τ−)⊘,x_τ=(r_τ- μ) σ, (3) where ,∈ℝD μ, σ ^D are the feature-wise mean and standard deviation computed from the training split, and ⊘ denotes element-wise division. We denote the normalized sequence by =[1,…,T]⊤X=[x_1,…,x_T] . Generated sequences are denormalized before evaluation or execution. I-B Audio–Text Conditioning Acoustic features extracted by a frozen speech encoder [22] are linearly interpolated to the T motion frames. The transcript text in y is tokenized into w1:Nw_1:N and encoded by a frozen language model [23]. Separate layer normalization and learned affine projections map the two feature sequences to ∈ℝT×dA ^T× d and ∈ℝN×dH ^N× d, respectively, where d is the generator’s hidden dimension and N is the number of transcript tokens. The projected conditions retain their frame-level and token-level organization. Using tokenizer character offsets, we derive approximate token intervals B=(sn,en)n=1NB=\(s_n,e_n)\_n=1^N from the word-level timestamps, where sns_n and ene_n are the associated start and end times. Temporal conditioning uses token centers mn=(sn+en)/2m_n=(s_n+e_n)/2 and motion-frame times uτ=(τ−1)/fu_τ=(τ-1)/f, where f is the motion frame rate. Both times are measured from the utterance onset. The combined condition is c=(,,B)c=(A,H,B). I-C Speech-Grounded Diffusion Transformer SGDiT maps a noisy normalized motion sequence t∈ℝT×DX_t ^T× D and conditions c to the flow velocity vθ(t,t,c)v_θ(X_t,t,c). Its transformer blocks combine temporal self-attention, transcript cross-attention, and feed-forward processing. Flow time t∈[0,1]t∈[0,1] modulates the blocks through adaptive layer normalization [24]. Acoustic conditioning. Each input motion token combines a projected motion frame, its aligned acoustic condition, and a positional embedding: τ=rt,τ+τ+τ,z_τ=W_rx_t,τ+A_τ+p_τ, (4) where t,τx_t,τ is frame τ of tX_t, r∈ℝd×DW_r ^d× D is a learned projection, and τ∈ℝdp_τ ^d is a learned frame-position embedding. Temporal self-attention then exchanges information bidirectionally across the acoustically conditioned motion sequence. Transcript conditioning. Cross-attention retrieves linguistic context through two paths over the same transcript features. The global path provides content-based access to the full token sequence, while the local path adds a preference for temporally nearby tokens. Updated motion features, after normalization and flow-time modulation, provide queries τQ_τ; the projected transcript features H provide keys nK_n and values nV_n shared by both paths. For one attention head, the content scores and global attention are Sτn S_τ n =γ^τ⊤^n, =γ\, Q_τ K_n, (5) g ^g =maskedSoftmax(), =maskedSoftmax(S), where hats denote L2-normalized queries and keys, γ is a bounded learned logit scale, and masked softmax normalizes over non-padding transcript tokens. The local path adds a Gaussian temporal prior to the shared content scores, yielding the temporally reweighted attention illustrated in Fig. 3: bτn b_τ n =−(uτ−mn−δ)22σ2, =- (u_τ-m_n-δ)^22σ^2, (6) b~τn b_τ n =max(bτn,−κ), = (b_τ n,-κ), l ^l =maskedSoftmax(+~), =maskedSoftmax(S+ b), where σ and δ control the temporal width and offset, and κ bounds the log penalty. Both paths use the same token padding mask. The two distributions are combined using temporal support: gτ g_τ =αmaxn∈Iexp(bτn), =α _n∈ I (b_τ n), (7) Πτn _τ n =(1−gτ)Πτng+gτΠτnl, =(1-g_τ) ^g_τ n+g_τ ^l_τ n, where I contains the non-padding token indices and α is the learned base mixing coefficient. Support uses the unclipped prior, reducing the local contribution when the frame is distant from all offset-adjusted token centers. The mixed weights aggregate the shared values into a text-conditioned update, which is projected and added to the motion features through a gated residual connection. A linear output head produces the final T×DT× D flow-velocity prediction. I-D Training Objective We train SGDiT with rectified flow matching [9, 25]. For a normalized motion–condition pair (,c)(X,c), we sample standard Gaussian noise ϵ∈ℝT×D ε ^T× D and a sequence-level flow time t∼(0,1)t (0,1). The interpolated motion and target velocity are t=t+(1−t)ϵ,⋆=−ϵ.X_t=tX+(1-t) ε, =X- ε. (8) The predicted velocity ^=vθ(t,t,c) V=v_θ(X_t,t,c) is supervised by matching the target flow and its adjacent-frame differences: ℒflow _flow =[MSE(^,⋆)], =E [MSE( V,V ) ], (9) ℒtemp _temp =[MSE(Δτ^,Δτ⋆)], =E [MSE( _τ V, _τV ) ], ℒgen _gen =ℒflow+λtempℒtemp. =L_flow+ _tempL_temp. Here Δτ _τ denotes first differences along the motion-frame axis, and λtemp _temp weights the temporal term. MSE is averaged over feature dimensions and valid frames; the temporal term uses only adjacent pairs of valid frames. Fig. 4: Conditioning comparison on an utterance outside BEAT2. Rows show joint audio–text conditioning (Ours), text-only, and audio-only outputs at matched frame indices. In the selected frames, joint conditioning exhibits broader arm extensions, whereas the unimodal outputs generally keep the hands closer to the torso. Colored arrows and circles highlight selected arm and hand movements. I-E Inference and Tracking At inference, the output length T is specified by the input clip duration at the motion frame rate. We initialize (0)∈ℝT×DX^(0) ^T× D with standard Gaussian noise and integrate the learned velocity field from flow time 00 to 11, keeping c fixed. Using K uniform Euler steps, we update tk t_k =kK, = kK, (10) (k+1) ^(k+1) =(k)+1Kvθ((k),tk,c), =X^(k)+ 1Kv_θ(X^(k),t_k,c), for k=0,…,K−1k=0,…,K-1. The final state ^=(K) X=X^(K) is converted to robot-motion references by inverting the normalization in (3). Their joint-angle components are supplied to the fixed SONIC motion tracker [11] as joint-position references and executed alongside speech playback. IV Experiments and Results IV-A Experimental Setup IV-A1 Datasets and Preprocessing We construct a robot-space co-speech dataset from BEAT2 [4]. The original long-form recordings are segmented into 22,192 utterance-level clips at speech pauses using word-level forced alignment, retaining the corresponding audio, transcript, SMPL-X motion, and word timestamps. Each motion sequence is retargeted to the 29-DoF Unitree G1 using GMR [10]. The retargeted motions are converted to a unified Z-up coordinate system and foot-ground aligned using a clip-wise vertical root offset, followed by recomputation of forward kinematics and motion derivatives. We further canonicalize each sequence by removing its initial global yaw while preserving subsequent root dynamics. Robot motions remain at the native 30 fps throughout processing. We then filter the resulting speech–robot pairs using both robot-motion quality checks and cross-modal consistency checks. For robot-motion quality, we discard clips that violate criteria on foot contact, self-collision, joint continuity and limits, smoothness, or severe high-frequency motion artifacts. We also retain only pairs whose audio and motion durations differ by at most one motion frame. We adopt a speaker-held-out split, holding out three English speakers for validation and using the remaining speakers for training. After filtering, the final dataset contains 14,987 training clips and 3,242 validation clips. IV-A2 Evaluation Metrics All methods use the same held-out split as the candidate input set, with eligibility determined by their native input and output-length requirements. Prediction–reference comparisons use the common temporal prefix of each eligible pair. Co-Speech Motion Characteristics. Fréchet Gesture Distance (FGD) [3, 4] measures distributional discrepancy using a shared skeleton-aware G1 joint-motion encoder. Div, M, and BA are computed from forward-kinematic body positions with fixed base rotation and translation. Diversity (Div) measures the frame-wise L1 deviation from each sequence’s temporal mean pose. For stochastic generators, Multimodality (M) [8] averages pairwise L1 distances among 20 samples generated for each input condition. Beat Alignment (BA) [21, 4] matches speech onsets to the nearest detected upper-body motion beats. We report absolute gaps to the matched reference statistics for Div and BA. Robot Motion Quality. Fig. 5: Real-robot execution on an utterance outside BEAT2. ECHO-G’s predicted joint angles are supplied as joint-position references to a fixed SONIC motion tracker while the corresponding speech is played. Frames progress from left to right through the book-recommendation utterance shown below, illustrating changes in arm extension, hand height, and torso posture. Colored arrows highlight selected arm movements, while the orange skeletal overlays outline the body configuration. Body jerk [16] is estimated from third-order finite differences of reconstructed world-space body positions, scaled by the cube of the frame rate. We average jerk magnitudes over all valid frame–body pairs and report the absolute gap between generated and reference means (Δ ) [26]. Foot-ground error measures the vertical distance of the lowest sole-proxy surface from the ground plane. Contact sliding speed measures the maximum horizontal sole-point speed per foot, averaged over detected contact intervals. Efficiency. End-to-end inference time covers input audio and aligned-transcript reading, online feature encoding, motion generation, and GMR or VAE-based motion mapping when applicable, ending at the robot-motion reference. We report total processing time divided by the total number of output frames. Peak RAM increase is the maximum request-window system memory usage above the corresponding pre-load idle baseline. IV-A3 Implementation Details We use frozen wav2vec 2.0 large XLSR and Qwen3.5-4B models to obtain 1024-D acoustic features and 2560-D contextual token embeddings, respectively. SGDiT contains 12 transformer blocks with a hidden dimension of 768, 8 attention heads, a feed-forward dimension of 2048, and learned temporal positional embeddings. The attention parameters γ,σ,δ,αγ,σ,δ,α are learned separately for each layer and head. Attention parameters are constrained to γ∈[1,16]γ∈[1,16], σ∈[0.25,2]σ∈[0.25,2] s, δ∈[−0.5,0.5]δ∈[-0.5,0.5] s, and α∈[0,0.5]α∈[0,0.5] using sigmoid/tanh mappings, with log-prior clipping threshold κ=12κ=12. We train for 63,000 optimizer steps on NVIDIA RTX 4090 hardware using FP32 computation. AdamW uses an initial learning rate of 3×10−43× 10^-4 with cosine decay and no warm-up, weight decay of 10−410^-4, an effective batch size of 48, and gradient clipping at 1.0. We set λtemp=0.5 _temp=0.5 and jointly drop the acoustic and text encoder features together with the time-distance inputs with probability 0.1. The exponential moving average (EMA) decay is 0.999. For physical deployment, motion generation runs on a Jetson AGX Orin, which is also used for the efficiency evaluation of all compared methods. Inference uses EMA weights from the checkpoint with the lowest EMA validation loss. Sampling uses eight Euler steps with classifier-free guidance (CFG) scale 1.0. We generate at most 600 motion frames at 30 fps and limit transcripts to 256 tokenizer tokens. Efficiency is measured with batch size one after warm-up. IV-A4 Baselines and Ablations We use the publicly released pretrained checkpoints of EMAGE [4] and GestureLSM [5], and retarget their generated human motions to G1 using GMR [10]. We additionally construct Human-Retargeted using the same audio–text conditioning design, backbone configuration, data split, and optimization settings as Ours, but generate normalized 136-D human motion. After denormalization using human-motion training statistics, a pretrained VAE-based mapping converts the samples to 39-D robot references. This mapping remains frozen during human-motion generator training. Audio-only and Text-only are trained separately in robot space using only the indicated modality. Both variants share the data split, backbone configuration, and optimization settings with Ours. Text-only retains the supplied clip duration and word timings but does not use acoustic features. IV-B Quantitative Results TABLE I: Comparison of direct robot-space generation with human-motion generation followed by retargeting or learned mapping on the BEAT2 speaker-held-out evaluation data. Δ , Δ , and Δ denote absolute deviations from matched ground-truth statistics over each method’s evaluated motion range. Best and second-best results are shown in bold and underlined text, respectively. Method Co-Speech Motion Characteristics Robot Motion Quality Efficiency FGD ↓ Δ ↓ M ↑ Δ ↓ Δ ↓ Foot Err. (m) ↓ C-Slide (m/s) ↓ E2E Time (ms/frame) ↓ Peak RAM Δ (MB) ↓ EMAGE+GMR 4.976 0.749 0 0.172 26.951 0.013 0.163 21.3 2661 GestureLSM+GMR 5.008 0.561 1.016 0.158 24.444 0.010 0.169 20.5 3804 Human-Retargeted 4.725 0.408 1.498 0.161 8.001 0.003 0.050 6.56 18714 Ours 2.278 0.320 1.786 0.063 9.039 0.008 0.052 5.96 18542 TABLE I: Ablation of conditioning modalities for direct robot-space generation on the BEAT2 speaker-held-out evaluation data. Δ , Δ , and Δ denote absolute deviations from the shared ground-truth statistics. Best and second-best results are shown in bold and underlined text, respectively. Ties at the displayed precision receive identical highlighting. Method FGD ↓ Δ ↓ M ↑ Δ ↓ Δ ↓ Foot Err. (m) ↓ C-Slide (m/s) ↓ Audio-only 2.360 0.360 1.702 0.081 4.497 0.008 0.038 Text-only 2.436 0.429 1.681 0.113 5.940 0.009 0.049 Ours 2.278 0.320 1.786 0.063 9.039 0.008 0.052 Table I compares direct robot-space generation with human-motion generation followed by retargeting or learned mapping. ECHO-G achieves the best results on all four co-speech metrics and the lowest processing time per output frame. It also improves all three robot-motion quality metrics over EMAGE+GMR and GestureLSM+GMR. Human-Retargeted yields a smaller Jerk gap, lower foot-ground error, and less contact sliding. These results support direct robot-space generation for co-speech modeling with reduced processing time. Table I evaluates conditioning modalities within the robot-space generator. Joint audio–text conditioning achieves the lowest FGD, Δ , and Δ and the highest M, outperforming both unimodal variants on the reported co-speech metrics. Audio-only yields the smallest Jerk gap and contact sliding speed, and matches joint conditioning in foot-ground error at the displayed precision. These results support combining acoustic and linguistic information to improve the evaluated co-speech characteristics. IV-C Qualitative Results We visualize motions generated from independently prepared audio–text utterances outside BEAT2. These examples provide qualitative evidence of generalization to speech inputs beyond the source dataset. In Fig. 4, audio-only conditioning produces rhythm-responsive motion, but gestures around semantically salient phrases remain small and less clearly related to the spoken content. Text-only conditioning produces content-related gestures, but their timing is less consistently aligned with the audio. Joint conditioning combines speech-responsive timing with more expansive, content-related gestures and fluid transitions in this example. IV-D Real-Robot Deployment We conduct real-robot experiments on a Unitree G1, executing the generated joint-position references through a fixed SONIC motion tracker [11] while playing the corresponding speech audio. Fig. 5 shows a representative trial using an independently prepared utterance outside BEAT2, with the robot accompanying its speech with generated body movements. Videos of additional real-robot trials are provided on the project page. IV-E User Study TABLE I: Mean user-study ratings from 45 participants on a five-point scale. Higher is better. Unimodal scores are pooled as described in the text. Best and second-best means are shown in bold and underlined text, respectively. Method Overall Human- likeness Rhythm Matching Motion Quality EMAGE+GMR 1.80 1.87 1.95 1.59 Human-Retargeted 2.65 2.67 2.38 2.90 Unimodal (pooled) 3.05 3.01 3.13 3.01 Ours 3.49 3.45 3.31 3.70 We conducted a video-rating study with 45 participants using G1 kinematic renderings in MuJoCo. Each participant completed nine trials, with three randomly selected from each of three criterion-specific pools comprising 33 utterances in total. The criteria were human-likeness, rhythm matching, and motion quality. Each trial presented four videos of the same utterance under matched rendering conditions, with hidden method identities and randomized positions. Participants rated each video on a five-point scale. Each trial compared ECHO-G, Human-Retargeted, EMAGE+GMR, and an audio-only or text-only variant. Unimodal ratings were pooled over the observed trial allocation. Overall denotes the equally weighted mean of the three criterion scores. As shown in Table I, joint audio–text conditioning receives the highest overall mean rating and the highest mean ratings across all three criteria, providing perceptual support for our method. V Discussion and Limitations The experimental results support direct robot-space modeling and joint audio–text conditioning for humanoid co-speech generation. Benchmark comparisons show the benefits of these choices for the reported co-speech characteristics, with the direct generation pipeline also requiring less processing time. Qualitative examples, user ratings, and physical demonstrations provide complementary evidence of expressive gestures, perceived quality, and robot execution. Further improvement is needed to better reconcile expressive behavior with robot-motion consistency. Several limitations remain. First, the gains in co-speech characteristics do not consistently translate into smaller Jerk gaps or better foot-contact measures, leaving room to improve expressive behavior and robot-motion consistency together. Second, the model learns broad speech–gesture associations, with limited training examples of explicit deictic or instructional gestures. This may constrain instruction-aware gesture generation when an utterance calls for a specific semantic motion. Third, the current system requires complete speech audio and timed transcripts and does not yet support causal streaming generation. VI Conclusion We presented ECHO-G for full-body humanoid co-speech generation from speech audio and timed transcripts. SGDiT combines frame-aligned acoustic conditioning with global–local transcript cross-attention to generate robot-motion references. Quantitative evaluation, a video-rating study, and physical demonstrations provide complementary evidence for the framework. The released dataset, benchmark, and code support reproducible research on humanoid co-speech generation. Future work will focus on jointly improving gesture expressiveness and robot-motion consistency, enriching training data for instruction-aware semantic gestures, and extending the framework to causal streaming generation. References [1] D. McNeill (1992) Hand and mind: what gestures reveal about thought. University of Chicago Press, Chicago, IL, USA. Cited by: §I. [2] A. Kendon (2004) Gesture: visible action as utterance. Cambridge University Press, Cambridge, UK. External Links: Document Cited by: §I. [3] Y. Yoon, B. Cha, J. Lee, M. Jang, J. Lee, J. Kim, and G. Lee (2020) Speech gesture generation from the trimodal context of text, audio, and speaker identity. ACM Transactions on Graphics 39 (6), p. 1–16. External Links: Document Cited by: §I, §I-B, §IV-A2. [4] H. Liu, Z. Zhu, G. Becherini, Y. Peng, M. Su, Y. Zhou, X. Zhe, N. Iwamoto, B. Zheng, and M. J. Black (2024) EMAGE: towards unified holistic co-speech gesture generation via expressive masked audio gesture modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 1144–1154. Cited by: §I, §I, §I-B, §I-B, §IV-A1, §IV-A2, §IV-A4. [5] P. Liu, L. Song, J. Huang, H. Liu, and C. Xu (2025) GestureLSM: latent shortcut based co-speech gesture generation with spatial-temporal modeling. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 10929–10939. Cited by: §I, §I-B, §IV-A4. [6] Z. Wang, Z. Ren, P. Shi, Z. Wang, C. Lin, T. Wang, Z. Qi, L. Zhao, H. Wang, and L. Yi (2026) RoboGesture: real-time semantic-aligned co-speech gestures generation for humanoid interaction. In Proceedings of the European Conference on Computer Vision (ECCV), Cited by: §I, §I-A. [7] Z. Liang, X. Xing, M. Yang, W. Zhou, and X. Xu (2026) PhysDrift: bridging the embodiment gap in humanoid co-speech motion generation. arXiv preprint arXiv:2606.19935. Cited by: §I, §I-A. [8] J. Li, D. Kang, W. Pei, X. Zhe, Y. Zhang, Z. He, and L. Bao (2021) Audio2Gestures: generating diverse gestures from speech audio with conditional variational autoencoders. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 11293–11302. Cited by: §I, §I-B, §IV-A2. [9] X. Liu, C. Gong, and Q. Liu (2023) Flow straight and fast: learning to generate and transfer data with rectified flow. In International Conference on Learning Representations, Cited by: §I, §I-D. [10] J. P. Araujo, Y. Ze, P. Xu, J. Wu, and C. K. Liu (2025) Retargeting matters: general motion retargeting for humanoid motion tracking. arXiv preprint arXiv:2510.02252. Cited by: §I, §I-A, §IV-A1, §IV-A4. [11] Z. Luo, Y. Yuan, T. Wang, C. Li, F. Castañeda, S. Chen, Z. Cao, J. Li, D. Minor, Q. Ben, et al. (2026) SONIC: supersizing motion tracking for natural humanoid whole-body control. Science Robotics 11 (117), p. eaed4592. External Links: Document Cited by: §I-A, §I-E, §IV-D. [12] M. Chen, K. Wang, B. Zhang, X. Ma, Z. Yang, Y. Ren, Q. Huang, Z. Zhu, Y. Wang, and Z. Su (2026) HoloMotion-1 technical report. arXiv preprint arXiv:2605.15336. Cited by: §I-A. [13] L. Yang, X. Huang, Z. Wu, A. Kanazawa, P. Abbeel, C. Sferrazza, C. K. Liu, R. Duan, and G. Shi (2025) OmniRetarget: interaction-preserving data generation for humanoid whole-body loco-manipulation and scene interaction. arXiv preprint arXiv:2509.26633. Cited by: §I-A. [14] P. Li, Z. Zhuang, Y. Gao, Y. Dong, S. Li, C. Jiang, S. Dou, Z. Xi, E. Zhou, J. Huang, et al. (2026) FRoM-W1: towards general humanoid whole-body control with language instructions. arXiv preprint arXiv:2601.12799. Cited by: §I-A. [15] W. Xie, J. Zheng, J. Han, J. Shi, W. Zhang, C. Bai, and X. Li (2026) TextOp: real-time interactive text-driven humanoid robot motion generation and control. arXiv preprint arXiv:2602.07439. Cited by: §I-A. [16] S. Huang, K. Lee, D. Qiao, G. He, Z. Wang, Y. Li, S. Zhu, and H. Zhao (2026) OMG: omni-modal motion generation for generalist humanoid control. arXiv preprint arXiv:2606.10340. Cited by: §I-A, §IV-A2. [17] Y. Yoon, W. Ko, M. Jang, J. Lee, J. Kim, and G. Lee (2019) Robots learn social skills: end-to-end learning of co-speech gesture generation for humanoid robots. In 2019 International Conference on Robotics and Automation (ICRA), p. 4303–4309. External Links: Document Cited by: §I-A. [18] Z. Li, C. Chi, Y. Wei, B. Zhu, T. Huang, Z. Sun, Y. Peng, P. Wang, Z. Wang, F. Liu, C. Xu, and S. Zhang (2026) Do you have freestyle? expressive humanoid locomotion via audio control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 956–965. Cited by: §I-A. [19] H. Liu, Z. Zhu, N. Iwamoto, Y. Peng, Z. Li, Y. Zhou, E. Bozkurt, and B. Zheng (2022) BEAT: a large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis. In European Conference on Computer Vision, p. 612–630. Cited by: §I-B. [20] J. Chen, Y. Liu, J. Wang, A. Zeng, Y. Li, and Q. Chen (2024) DiffSHEG: a diffusion-based approach for real-time speech-driven holistic 3D expression and gesture generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 7352–7361. Cited by: §I-B. [21] R. Li, S. Yang, D. A. Ross, and A. Kanazawa (2021) AI choreographer: music conditioned 3D dance generation with AIST++. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 13401–13412. Cited by: §I-B, §IV-A2. [22] A. Baevski, Y. Zhou, A. Mohamed, and M. Auli (2020) wav2vec 2.0: a framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems, Vol. 33, p. 12449–12460. Cited by: §I-B. [23] Qwen Team (2026) Qwen3.5: towards native multimodal agents. External Links: Link Cited by: §I-B. [24] W. Peebles and S. Xie (2023) Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 4195–4205. Cited by: §I-C. [25] Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le (2023) Flow Matching for generative modeling. In International Conference on Learning Representations, External Links: Link Cited by: §I-D. [26] F. Fang, S. Yang, and W. Yang (2026) CoordSpeaker: exploiting gesture captioning for coordinated caption-empowered co-speech gesture generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 30761–30771. Cited by: §IV-A2.