Paper deep dive
GestAdapt: Workspace-Conditioned Co-Speech Gesture Generation for Humanoid Robots
Bosong Ding, Xianglin Zhang, Miao Xin, Murat Kirtay, Giacomo Spigler
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 10/1/2026, 8:11:59 AM
Summary
The paper introduces GestAdapt, a framework for generating co-speech gestures for humanoid robots that adapts to workspace constraints. It unifies six co-speech corpora into a shared motion representation and uses a diffusion Transformer conditioned on speech, motion history, and a prescribed wrist workspace box. The system employs analytic guidance during sampling to minimize workspace violations and uses Gaussian Mixture Regression (GMR) for retargeting to specific robot embodiments like Reachy2. Evaluations show improved gesture quality and workspace compliance compared to baselines without workspace conditioning.
Entities (18)
Relation Signals (14)
Giacomo Spigler → affiliatedwith → Tilburg University
confidence 95% · Giacomo Spigler *Corresponding author. Affiliation: Department of Intelligent Systems, Tilburg University
Murat Kirtay → affiliatedwith → Tilburg University
confidence 95% · Murat Kirtay Affiliation: Department of Intelligent Systems, Tilburg University, The Netherlands.
Miao Xin → affiliatedwith → CUMTB
confidence 95% · Miao Xin Affiliation: School of Artificial Intelligence, China University of Mining and Technology-Beijing (CUMTB)
Xianglin Zhang → affiliatedwith → TU Dresden
confidence 95% · Xianglin Zhang Affiliation: TU-Dresden, Dresden, Germany.
Bosong Ding → affiliatedwith → Tilburg University
confidence 95% · Bosong Ding Affiliation: Department of Intelligent Systems, Tilburg University, The Netherlands.
GestAdapt → evaluatedon → Reachy2
confidence 95% · In a real robot evaluation, all compared motions are retargeted to the Reachy2 humanoid robot
GestAdapt → uses → Diffusion Transformer
confidence 95% · A diffusion Transformer generates future motion conditioned on speech, motion history, and a specified workspace box.
GestAdapt → trainedon → ZEGGS
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Co-speech gestures for robots must adapt not only to speech and embodiment, but also to the workspace available for performing the motion. Since the same speech can be accompanied by different gestures, a robot can respond to workspace constraints, e.g., gestures for speech next to a wall. In these scenarios, the robot should gesture in a suitable motion rather than simply correcting an unconstrained one. To achieve this goal, we present GestAdapt, a workspace-conditioned framework that conditions co-speech gesture generation on a prescribed wrist workspace. The GestAdapt framework learns from six complementary co-speech corpora through a shared motion representation and supports retargeting to different robot embodiments. Quantitative evaluation shows that generated motions remain close to the real-motion distribution while respecting the workspace. In a user study, gestures generated under modified workspace constraints receive a mean quality score of 3.24/5, above our no-workspace variant (2.43/5) and below the reference motions (3.68/5). In a real robot evaluation, all compared motions are retargeted to the Reachy2 humanoid robot under identical workspace constraints. Motions generated with our framework rank first in 69.7\% of comparisons, higher than our no-workspace variant baseline and retargeted ground-truth motions constrained afterward. Overall, the results support adapting gestures to the available workspace during generation, rather than modifying unconstrained trajectories afterward to satisfy workspace constraints, potentially compromising gesture naturalness.
Tags
Links
- Source: https://arxiv.org/abs/2609.38400v1
- Canonical: https://arxiv.org/abs/2609.38400v1
Trouble viewing inline? Open PDF directly →
Full Text
45,625 characters extracted from source content.
Expand or collapse full text
GestAdapt: Workspace-Conditioned Co-Speech Gesture Generation for Humanoid Robots Bosong Ding Affiliation: Department of Intelligent Systems, Tilburg University, The Netherlands. Xianglin Zhang Affiliation: TU-Dresden, Dresden, Germany. Miao Xin Affiliation: School of Artificial Intelligence, China University of Mining and Technology-Beijing (CUMTB), Beijing, China. Murat Kirtay Affiliation: Department of Intelligent Systems, Tilburg University, The Netherlands. Giacomo Spigler †thanks: *Corresponding author. Affiliation: Department of Intelligent Systems, Tilburg University, The Netherlands. Abstract Co-speech gestures for robots must adapt not only to speech and embodiment, but also to the workspace available for performing the motion. Since the same speech can be accompanied by different gestures, a robot can respond to workspace constraints, e.g., gestures for speech next to a wall. In these scenarios, the robot should gesture in a suitable motion rather than simply correcting an unconstrained one. To achieve this goal, we present GestAdapt, a workspace-conditioned framework that conditions co-speech gesture generation on a prescribed wrist workspace. The GestAdapt framework learns from six complementary co-speech corpora through a shared motion representation and supports retargeting to different robot embodiments. Quantitative evaluation shows that generated motions remain close to the real-motion distribution while respecting the workspace. In a user study, gestures generated under modified workspace constraints receive a mean quality score of 3.24/5, above our no-workspace variant (2.43/5) and below the reference motions (3.68/5). In a real robot evaluation, all compared motions are retargeted to the Reachy2 humanoid robot under identical workspace constraints. Motions generated with our framework rank first in 69.7% of comparisons, higher than our no-workspace variant baseline and retargeted ground-truth motions constrained afterward. Overall, the results support adapting gestures to the available workspace during generation, rather than modifying unconstrained trajectories afterward to satisfy workspace constraints, potentially compromising gesture naturalness. I INTRODUCTION Co-speech gestures are body movements that naturally accompany speech, and co-speech gesture generation aims to synthesize such movements from speech [1]. However, the space available for gesturing can vary substantially across environments. For example, a person may express the same content while standing in an open room, sitting across a table, or standing close to a wall. Although the speech remains unchanged, the surrounding space constrains the range of feasible gestures. Humanoid robots face similar limitations. A gesture-generation model without spatial conditioning may produce broad arm movements that are inappropriate for the robot’s surroundings or limited joint ranges. Consequently, a speech-conditioned model that does not account for constraints may generate motions that reduce motion quality, violate the robot’s physical limits, or even risk damaging the hardware. In such settings, a gesture that appears natural in the training data may therefore be difficult or unsafe for the robot to execute. Fig. 1: Examples of workspace-adaptive co-speech gestures by a Reachy2 humanoid robot. The chosen gesture space changes across three settings: (a) free space, (b) a nearby wall, and (c) a tabletop workspace. The robot adapts its gesture motion to remain within the specified workspace. Most co-speech gesture models focus on naturalness, synchronization, semantics, style, and diversity [2, 3, 4, 5, 6]. A smaller body of work has brought co-speech gesture generation onto humanoid robots [7, 8, 9, 10]. These works additionally require that generated motion remain executable under the robot’s embodiment and kinematic constraints and emphasize motion transfer, embodiment-aware generation, kinematic feasibility, and safe execution. Despite the growing number of studies, generating gestures that respect the limitations of the surrounding environment has been identified as an important open challenge in co-speech gesture generation [1]. More recent approaches have begun to address this problem for human or virtual-character motion: InteracTalker combines co-speech motion with object-aware interaction [11], while Puppeteer directly conditions gesture generation on posture and surrounding object geometry [12]. This leaves a vital problem at the intersection of the two directions: generating co-speech motion according to the physical space available to a robot while considering its embodiment and execution constraints. Co-speech gesture generation is inherently one-to-many: the same speech can be accompanied by different hand configurations, movement directions, and amplitudes while remaining natural and communicative [13, 5]. Existing methods already exploit this variability by conditioning on speaker, motion style, or partially specified motion [2, 3, 14, 6]. For a robot, physical cues can serve as an additional conditioning signal. Together, these allow the model to generate a different but plausible gesture that naturally fits the available space, without needing to compress or correct the motion, as shown in Fig. 1. To this end, we present GestAdapt, a workspace-conditioned framework for robot co-speech gesture generation. We combine six co-speech corpora in a unified upper-body and hand representation. Spatial supervision comes from the wrist-motion envelopes without requiring scene annotations. A diffusion Transformer generates future motion conditioned on speech, motion history, and a specified workspace box. Learned conditioning allows the gesture itself to change with the available space, while sampling-time analytic guidance reduces residual wrist violations. The generated motion is then transferred to the target robot through embodiment-specific retargeting and executed with constrained workspace limits and hard physical constraints. We evaluate GestAdapt through quantitative motion metrics, human evaluation under changed workspace conditions, and physical robot experiments. The results show that the model can adapt gesture motion to new spatial constraints while preserving plausible and smooth motion. In the robot evaluation, where all compared motions undergo the same retargeting and constrained execution pipeline, participants prefer workspace-conditioned generation over the same generator trained without workspace conditioning and directly retargeted human motion. Together, these results support treating target space as part of gesture generation itself, rather than only as a constraint imposed afterward. I RELATED WORK I-A Co-Speech Gesture Generation Learning-based co-speech gesture generation has progressed from predicting motion from speech acoustics to incorporating linguistic content, speaker identity, and multimodal speech representations [15, 16, 2]. Recent work has further improved holistic body-hand coordination, style control, and motion diversity [17, 3, 13, 5, 6]. At the same time, a number of co-speech motion corpora have been introduced, including BEAT [4], SHOW [17], Streamer [18], TED [19], Trinity [20], and ZEGGS [14]. These datasets were collected independently with different acquisition setups, motion representations, and gesture distributions; as a result, most prior gesture models are developed within a particular corpus and its associated representation. Unlike these studies, our GestAdapt framework unifies multiple corpora in a shared motion space, using their complementary motion and spatial variation to support general workspace-conditioned gesture generation. I-B Gesture Generation for Robots Early learning-based systems demonstrated speech-driven gesture generation on humanoid robots [7, 21]. Transferring human motion to a robot requires accounting for differences in limb proportions, joint configurations, and motion limits, often through retargeting and constrained optimization [22]. More recent work incorporates embodiment requirements more directly. PhysDrift [8], for example, learns robot-native joint trajectories from speech using embodiment-aware motion curation and regularization. RoboGesture [9] combines robot-centric gesture generation with semantic-acoustic alignment and an MPC-based collision-avoidance filter for execution. These methods address important challenges in embodiment transfer, speech alignment, and physical execution. We study a different but complementary question: how should the gesture itself change when the space available to the robot changes? Our model conditions generation on the target workspace before embodiment-specific retargeting and constrained execution. I-C Physical and Spatial Constraints in Motion Generation Scene-conditioned human motion models incorporate environmental geometry for locomotion and human-object interaction [23, 24], while robot motion planning routinely considers kinematic and collision constraints [25, 26]. Spatial context has also begun to enter the co-speech gesture generation literature. InteracTalker integrates speech-driven gestures with prompt-based object interactions [11], and Puppeteer conditions gesture synthesis on posture and surrounding object geometry [12]. Our setting differs regarding its supervision and deployment requirements. Instead of learning from scene-annotated gesture recordings, we derive weak spatial conditions from wrist-motion envelopes across existing speech-motion corpora. We use an explicit workspace box to control gesture extent and evaluate the generated motions after robot retargeting under shared execution constraints. This connects spatially conditioned gesture generation with embodiment-specific robot gesture execution without requiring a full scene representation. I Unified Dataset Fig. 2: Complementary motion and spatial support across the six co-speech corpora. The Principal Component Analysis (PCA) views show the motion distributions, while the front, side, and top views visualize the corresponding wrist-motion envelopes. Heatmap intensity indicates the number of envelopes covering each location, and colored contours show the spatial support of individual corpora. I-A Combining Multiple Co-Speech Corpora Learning spatially adaptive gestures ideally requires the same speech to be performed under different physical constraints. Existing co-speech datasets do not provide such paired recordings or explicit free-space annotations. Therefore, we combine six datasets with aligned speech and motion: BEAT [4], SHOW [17], Streamer [18], TED-Expressive [19], Trinity [20], and ZEGGS [14]. These datasets differ in recording settings and gesture distributions, providing complementary variation in posture, gesture scale, and upper-body workspace. As shown in Fig. 2, their motion distributions overlap, while their kinematic and spatial support remain distinct. Combining them expands both the motion and workspace coverage available for training. Since their native motion representations are incompatible, we first map them into a shared geometric representation. I-B Robot-Centered Unified Motion Representation Fig. 3: Unified motion representation across the six source datasets. Motion is first evaluated on each native kinematic hierarchy, and 38 anatomically corresponding joints are then selected and aligned to a common skeleton. The unified representation contains 33 dimensions for the upper body and palm axes and 45 dimensions for each hand, resulting in 123 dimensions. The six datasets use different skeleton topologies, body proportions, and rotation conventions, making their native representations incompatible. We therefore manually evaluate each motion on its original kinematic hierarchy and extract anatomically corresponding Cartesian landmarks, then align them to a common upper-body and hand skeleton. We perform forward kinematics before simplification, so intermediate joints still contribute to the retained geometry. To remove dataset-specific bone lengths and rotation conventions, we represent motion using 3D bone directions. For generation, seven torso and arm directions describe the upper body, each palm is represented by its forward and normal axes, and finger directions are expressed in the corresponding palm frame. This results in a 123-dimensional motion representation at 15 Hz as shown in Fig. 3. The explicit palm axes resolve the axial orientation that cannot be recovered from a single bone direction. We further verify that this representation preserves the geometry required for robot retargeting. Reconstruction introduces less than 0.6∘0.6 average palm-orientation error on BEAT, Trinity, and ZEGGS, while wrist positions remain effectively unchanged. After retargeting to the G1, GR3, and Reachy2 humanoid robots, the mean differences in shoulder, elbow, and wrist joint trajectories remain below 1∘1 across all evaluated source-robot combinations. I-C Spatial Representation and Dataset Construction The source datasets contain motion but no corresponding scene geometry or free-space annotations. We therefore construct spatial supervision directly from wrist trajectories, which compactly capture where a gesture is expressed and how far it reaches. As task-level endpoints, the wrists provide direct control over gesture extent while leaving some flexibility to the elbow and shoulder joints. This is desirable for a redundant humanoid arm, where similar wrist motion can often be realized by multiple proximal-arm configurations. The feasible regions of these proximal links are generally different from the wrist workspace. For example, a wrist may extend beyond a table edge while the shoulder remains behind it, and the elbow may pass below the tabletop outside its boundary. Applying the same box to the entire upper body can impose unnecessary or unnatural constraints. A whole-upper-body envelope is also less informative about hand placement because its boundaries are often dominated by relatively stationary proximal joints: in our cross-recording analysis, nearest upper-body envelopes differ by only 1.19 cm at the median, compared with 11.59 cm for the corresponding wrist envelopes. We treat the wrist box as a task-level spatial condition, not a complete collision-free region. It only constrains wrist placement so that during applications, the safe region of the movement can be determined by the upstream policy. The generator still predicts upper-body motion; an embodiment-specific quadratic program (QP) retargets it under soft workspace and configured hard physical constraints. For each motion segment, we compute the smallest torso-centered, axis-aligned box containing both wrist trajectories. The six boundaries are normalized by a canonical arm length of 6969 cm. The box specifies gesture extent without determining which hand moves, its trajectory, or the corresponding arm and finger configuration. A practical issue arises when envelopes are computed from fixed temporal windows: nearly identical overlapping windows can receive different labels when an outward wrist movement falls just outside one window. To reduce this effect, each training example contains four observed frames and 30 target frames, with four additional future frames used only to test envelope stability. We discard windows whose envelope changes sharply after this extension, together with segments dominated by stationary wrists or silence. These additional frames are never provided to the generator. Starting from 799,996 candidate windows, this procedure retains 378,438 examples. We split the data by complete recording to prevent overlapping windows from appearing in both training and test sets, resulting in 359,088 training and 19,350 held-out examples. IV Method Fig. 4: Overview of the speech-to-robot framework. Speech, motion history, and a workspace box condition a diffusion model to generate future gesture motion. The workspace is encoded as six spatial tokens and injected through cross-attention, while analytic guidance during denoising further corrects wrist motion toward the requested workspace. The generated motion is then retargeted to the target robot and executed under embodiment-specific constraints. Our framework generates co-speech gestures that adapt to a specified workspace and can be executed on different robot embodiments. As illustrated in Fig. 4, given speech, recent motion history, and a workspace box, GestAdapt generates motion with a workspace-conditioned diffusion model, applies analytic denoising guidance, and retargets the result under embodiment-specific execution constraints. IV-A Problem Formulation Given speech audio A, recent motion history H, and a workspace W, we aim to generate a future gesture sequence X as X∼pθ(X∣A,H,W),W=(ℓx,ux,ℓy,uy,ℓz,uz).X p_θ(X A,H,W), W=( _x,u_x, _y,u_y, _z,u_z). (1) Motion is represented at 15 Hz, with h=4h=4 history frames and T=30T=30 window frames. W denotes the normalized workspace box specifying the desired region for wrist motion. At inference time, it can designate a selected box for gesturing; selecting W from scene or task information is outside our scope. The workspace constrains only the gesture’s spatial extent, without prescribing hand assignment, trajectory, or gesture type, leaving the generator free to produce multiple plausible gestures. IV-B Workspace-Conditioned Diffusion Transformer Model Because co-speech gesture is one-to-many, we use a diffusion Transformer to generate multiple plausible motions conditioned on speech and workspace, as stochastic denoising naturally preserves multiple plausible motion realizations. At each denoising step, temporal motion tokens combine the current noisy future motion, four observed history frames, frame-aligned speech features extracted from 16 kHz audio by a jointly trained four-layer 1D convolutional encoder, and the diffusion timestep. The audio encoder produces a 32-dimensional feature for each of the 34 motion frames for temporal alignment. The denoiser contains eight Transformer blocks with hidden dimension D=512D=512. The six boundaries of the workspace are embedded by a shared MLP and combined with a learned boundary identity embedding. After temporal self-attention, each Transformer block injects these workspace tokens through residual cross-attention: Z(l)=Z~(l)+CrossAttnl(LN(Z~(l)),C,C),Z^(l)= Z^(l)+CrossAttn_l (LN( Z^(l)),C,C ), (2) where Z~(l) Z^(l) contains the temporal motion features and C∈ℝ6×DC ^6× D contains the six workspace tokens. The normalized motion features provide the queries, while C provides the keys and values through separate learned projections. This allows each temporal position to attend to the workspace boundaries throughout denoising. Holistic co-speech motion contains skeleton components with substantially different motion statistics and relationships to speech. Accordingly, recent holistic gesture models commonly use part-wise representations or decoders rather than modeling all motion variables identically [17, 6]. We adopt a similar part-aware design for a task-specific reason. In our unified representation, the body and palm variables form a compact 33-dimensional component that determines the arm configuration and wrist/palm geometry, where workspace adaptation is primarily expressed, whereas finger articulation occupies the remaining 90 dimensions. We therefore use separate output heads for body/palm motion and finger motion. The Transformer backbone remains shared, preserving temporal and cross-part interactions, while the output projections specialize to the different structures of spatially relevant upper-body motion and high-dimensional finger articulation. Both branches are trained jointly using the standard dimension-normalized DDPM noise-prediction objective over 500 diffusion steps, with a linear noise schedule from β1=10−4 _1=10^-4 to β500=0.02 _500=0.02. The model is trained for 780,000 updates, and inference uses the full 500-step ancestral DDPM sampler. Our representation ablation in Sec. V-D shows improved motion quality and workspace compliance over the 126D holistic design. IV-C Analytic Workspace Guidance Workspace conditioning substantially reduces spatial violations, but residual wrist boundary crossings can still occur during sampling. We therefore apply an analytic guidance term at sampling time, following the general idea of test-time guidance in diffusion-based motion generation [27]. The guidance introduces no additional training objective. At reverse step k, we reconstruct an estimate X^0 X_0 of the clean motion from the current noisy state XkX_k and the detached denoiser prediction. Wrist positions are then obtained differentiably through forward kinematics. For each future frame t, wrist s, and spatial axis a∈x,y,za∈\x,y,z\, we define the workspace violation as vt,s,a=max(ℓa−w^t,s,a,w^t,s,a−ua,0),v_t,s,a= ( _a- w_t,s,a, w_t,s,a-u_a,0 ), where ℓa _a and uau_a denote the lower and upper workspace boundaries. We use the guidance cost CW C_W =16T∑t,s,avt,s,a2+(maxt,s,avt,s,a)2, = 16T _t,s,av_t,s,a^2+ ( _t,s,av_t,s,a )^2, (3) ϵ~k ε_k =ϵk+λk∇XkCW, = _k+ _k _X_kC_W, For the 500-step reverse process, we set λk=G1−α¯k _k=G 1- α_k for k≤250k≤ 250 and zero otherwise, with G=96G=96 by default. The denoiser prediction is detached when differentiating CWC_W through the clean-motion reconstruction and forward kinematics. We apply no gradient normalization, clipping, or state projection. Thus, G=0G=0 retains workspace conditioning without analytic guidance, which reduces residual violations but provides no hard feasibility guarantee. TABLE I: Quantitative results over 5 runs. Method FGD↓FGD BC↑BC Jerk ↓ Violation ↓ Trimodal (deterministic) [2] 11.045±0.00011.045± 0.000 0.5283±0.00000.5283± 0.0000 10.202±0.00010.202± 0.000 6.064±0.0006.064± 0.000 StyleGestures[3] 11.579±0.27411.579± 0.274 0.6356±0.00190.6356± 0.0019 6.526±0.0056.526± 0.005 19.667±0.18919.667± 0.189 DiffSHEG[28] 15.579±0.95715.579± 0.957 0.4889±0.00380.4889± 0.0038 5.163±0.1175.163± 0.117 16.481±0.36616.481± 0.366 Ours w/o Workspace 6.933±0.4196.933± 0.419 0.4989±0.00760.4989± 0.0076 4.695±0.1054.695± 0.105 9.815±0.2629.815± 0.262 Ours 5.019±0.2235.019± 0.223 0.4515±0.00360.4515± 0.0036 4.295±0.1044.295± 0.104 1.095±0.0171.095± 0.017 IV-D GMR-Based Robot Retargeting and Constrained Execution The diffusion model generates motion in a unified body representation rather than robot joint space. Here, we use GMR[29] to map each generated sequence to the target embodiment. Generated upper-arm and forearm directions and palm orientations are transformed into the robot frame using embodiment-specific coordinate calibration and link lengths. GMR then solves for robot joint configurations using the robot kinematic model, Jacobians, and joint limits. The requested workspace is registered once in the robot base frame and converted from the normalized generation coordinates to a metric box using the embodiment-specific scale. At each frame, we augment the existing GMR/Mink tracking problem with soft workspace constraints: minΔq,s≥0 _ q,\,s≥ 0 12ΔqHIKΔq+cIKΔq+12sΛs 12 q TH_IK q+c_IK T q+ 12s T s (4) s.t. .t. AWΔq−s≤bW, A_W q-s≤ b_W, AhardΔq≤bhard, A_hard q≤ b_hard, where Δq q is the joint position increment and s≥0s≥ 0 represents the slack variables introduced to soften the workspace constraints. The matrix HIKH_IK and vector cIKc_IK define the quadratic and linear costs of the baseline inverse kinematics (IK) objective. The diagonal weight matrix Λ penalizes workspace slack, trading off tracking accuracy and workspace adherence. The matrices AW,AhardA_W,A_hard and vectors bW,bhardb_W,b_hard denote the Jacobian-based linear mappings and upper bounds for the soft workspace constraints and the hard physical constraints (e.g., joint limits and self-collisions), respectively. Workspace constraints are soft, whereas joint ranges, per-frame motion limits, and configured self-collision constraints remain hard. Generation and execution therefore solve different parts of the problem. The diffusion model can change the gesture itself in response to the target space, while GMR resolves that motion for a particular robot and maintains embodiment-specific feasibility. Accordingly, we evaluate generation-stage wrist containment separately from robot-stage tracking and execution feasibility. V Experiments V-A Metrics Our primary goal is that, when the workspace changes, the generated motion should remain plausible under the distribution of real human gestures, remain responsive to the accompanying speech, and stay smooth, while placing the wrists within the prescribed spatial region. Therefore, we evaluate these complementary properties using four metrics. FGD (Fréchet Gesture Distance) measures the distributional discrepancy between generated and real gestures in the feature space of a pretrained gesture encoder. We compute FGD between the generated motions and the corresponding real motions in the held-out test set; a lower FGD value indicates that the generated samples remain closer to the unseen real gesture distribution. BC (Beat Consistency) measures speech-motion synchronization by comparing the temporal locations of motion beats with acoustic beats extracted from the corresponding audio. Jerk measures the temporal smoothness of the generated wrist trajectories through their third-order temporal derivative. Finally, Violation measures workspace compliance by computing the maximum distance by which either wrist moves outside its prescribed workspace, averaged over the test set. V-B Quantitative Results We compare our full model against three representative external baselines that can be retrained under a common experimental protocol without altering their core formulations. Specifically, Trimodal [2], StyleGestures [3], and DiffSHEG [28] are retrained on the same six datasets and unified motion representation. We restrict the comparison to methods that can be adapted to this setting with minimal structural changes; methods tightly coupled to dataset-specific motion representations or auxiliary inputs would require substantial modification, making a controlled comparison difficult to interpret. We additionally evaluate Ours w/o Workspace, an internal control that uses the same gesture-generation backbone and training setup as the full model but excludes workspace-conditioned design and analytic workspace guidance. This provides a more direct comparison for isolating the effect of workspace-conditioned generation from the underlying motion-generation architecture. As shown in Table I, ours w/o Workspace achieves an FGD of 6.933. The full workspace-aware generation pipeline reduces FGD to 5.019 and mean maximum workspace violation from 9.815 cm to 1.095 cm. Table I further distinguishes learned conditioning from analytic guidance: without guidance, the workspace-conditioned model already achieves an FGD of 5.021 and a violation of 1.272 cm, while the default guidance further reduces violation to 1.095 cm. Although our model has a lower BC, we report BC for comparability with prior work and interpret it only as a diagnostic of rhythmic alignment rather than perceptual quality. Prior studies have shown limited agreement between objective gesture metrics and human judgments, while conventional BC can be sensitive to spurious motion beats; speech–gesture timing itself also exhibits cross-linguistic variation[30]. We therefore complement it with human evaluation of overall gesture quality, including naturalness, expressiveness, and appropriateness to the accompanying speech. The comparison with the three external baselines should not be interpreted as fully information-matched. In Table I, GestAdapt is conditioned on each test sequence’s reference workspace, whereas the external baselines retain their original input modalities. We therefore use these comparisons to assess conditional motion quality and workspace containment rather than to establish superiority under identical input information. In contrast, Ours w/o Workspace shares the same underlying generator, training data, and motion representation as the full model, and serves as the controlled comparison for evaluating the effect of workspace-conditioned generation. Sec. V-C further evaluates this distinction under changed workspace conditions; in the robot study, all motion sources are processed through the same workspace-constrained execution pipeline, enabling comparison under identical downstream constraints. V-C User Study Objective metrics alone do not fully capture perceived gesture quality, as also observed in the GENEA Challenge, where most objective metrics correlated poorly with human judgments [30]. We therefore conduct two user studies with 30 participants on 12 randomly selected 8-second test clips. In both studies, participants evaluate overall gesture quality considering naturalness, expressiveness, and appropriateness to the accompanying speech. The first study evaluates generated human-skeleton motions under workspace constraints against general motion, while the second evaluates the motions after retargeting and constrained execution on a physical robot. All comparisons are anonymized. Table I reports means and 95% confidence intervals across participants. We use Ours w/o Workspace as the primary generated-motion control, which allows the study to isolate whether conditioning gesture generation on the target workspace improves perceived motion quality under changed spatial constraints. To isolate the advantage of workspace guidance, Ours w/o Workspace is denoised with the same analytic workspace guidance, followed by the same robot retargeting, which differs from the setting in Table I. Unseen-workspace generation. Each 8-second clip contains four generation windows. We randomly select one window and use its workspace constraint for all four windows, such that three of the four windows must be generated under a workspace different from their original one. The full sequence is generated recursively using the model’s previous predictions as motion history, without access to future reference motion. Participants independently rate the ground truth, Ours w/o Workspace, and our generated motions from 1 to 5 for overall gesture quality. Only our full model receives the modified workspace through learned conditioning; the no-conditioning variant receives the same workspace only through analytic sampling guidance. Despite this additional constraint, our method achieves a mean score of 3.24/53.24/5, compared with 2.43/52.43/5 for Ours w/o Workspace and 3.68/53.68/5 for the ground truth, showing that it can substantially alter the gesture to satisfy a new workspace while preserving perceived gesture quality. Robot evaluation. We next evaluate whether the generated motions retain their expressiveness under shared workspace-constrained robot retargeting. For each 8-second clip, we randomly sample workspace constraints from unrelated test-set windows. Our method generates motions directly under these constraints, whereas Ours w/o Workspace uses it only through analytic guidance. We then post-process all three using the same workspace constraints, QP post-processing, and robot controller. Participants rank the anonymized robot motions using the same overall-quality criteria. Our method achieves a mean rank of 1.371.37 and is ranked first in 69.7%69.7\% of the trials, compared with 2.112.11 and 19.7%19.7\% for Ours w/o Workspace, and 2.532.53 and 10.6%10.6\% for the ground truth. This indicates that incorporating the target workspace directly into the learned generator improves perceived gesture quality beyond analytic workspace guidance and downstream constrained retargeting alone. TABLE I: User-study results with 95% confidence intervals Study 1 Study 2 Method Mean score ↑ Mean rank ↓ Top-1 (%) ↑ Ground truth 3.68±0.203.68± 0.20 2.53±0.062.53± 0.06 10.6±3.910.6± 3.9 Ours w/o Workspace 2.43±0.182.43± 0.18 2.11±0.062.11± 0.06 19.7±4.819.7± 4.8 Ours 3.24±0.203.24± 0.20 1.37±0.091.37± 0.09 69.7±7.169.7± 7.1 TABLE I: Ablation results over 5 runs. Model or setting FGD ↓ Jerk ↓ Violation ↓ (a) Six-corpus model versus single-corpus experts Ours 5.019±0.2235.019± 0.223 4.295±0.1044.295± 0.104 1.095±0.0171.095± 0.017 BEAT expert 11.211±0.24111.211± 0.241 6.707±0.0446.707± 0.044 2.950±0.0402.950± 0.040 Trinity expert 11.918±0.44611.918± 0.446 5.790±0.0865.790± 0.086 5.142±0.0985.142± 0.098 SHOW expert 12.453±0.50112.453± 0.501 4.935±0.0784.935± 0.078 3.472±0.1063.472± 0.106 ZEGGS expert 12.862±0.40512.862± 0.405 7.671±0.1147.671± 0.114 7.507±0.1207.507± 0.120 Streamer expert 13.979±0.80413.979± 0.804 3.286±0.0353.286± 0.035 2.176±0.0362.176± 0.036 TED expert 14.230±0.48714.230± 0.487 15.073±0.38715.073± 0.387 2.651±0.0822.651± 0.082 (b) Analytic denoising-guidance strength G=0G=0 5.021±0.2215.021± 0.221 4.291±0.1014.291± 0.101 1.272±0.0151.272± 0.015 G=64G=64 5.020±0.2225.020± 0.222 4.294±0.1044.294± 0.104 1.140±0.0161.140± 0.016 =G=96 (Ours) 5.019±0.2235.019± 0.223 4.295±0.1044.295± 0.104 1.095±0.0171.095± 0.017 G=128G=128 Diverged Diverged Diverged (c) Motion representation Holistic representation 5.942±0.3895.942± 0.389 4.602±0.1004.602± 0.100 1.679±0.0381.679± 0.038 Factorized representation (Ours) 5.019±0.2235.019± 0.223 4.295±0.1044.295± 0.104 1.095±0.0171.095± 0.017 V-D Ablation Studies We ablate three design choices: multi-corpus training, analytic denoising guidance, and the motion representation. The results are presented in Table I. Multi-corpus training. A single corpus covers only a limited range of gesture patterns and spatial extents. Since our workspace labels are derived from observed motion, each corpus also provides limited workspace-motion combinations. All corpus-specific experts are trained to convergence and evaluated on the same pooled test set and target workspaces. Although the six-corpus model also achieves lower FGD, the more important difference is workspace compliance: its mean maximum violation is 1.0951.095 cm, compared with 2.1762.176–7.5077.507 cm for the single-corpus experts. This suggests that, under unfamiliar target boxes, models trained on limited spatial support tend to preserve corpus-specific motion patterns at the expense of workspace compliance, motivating broader cross-corpus training. Analytic denoising guidance. With learned workspace conditioning alone (G=0G=0), the mean maximum violation is already 1.272 cm. Analytic guidance further reduces it to 1.095 cm at G=96G=96, with negligible changes in FGD and jerk, indicating that workspace adaptation is primarily learned by the generator while guidance mainly corrects residual violations; G=128G=128 destabilizes sampling. Motion representation. We compare a 126D holistic representation using world-frame directions for all 42 bones with our 123D design, which explicitly represents palm axes and expresses finger directions in the corresponding palm frame, with separate body/palm and finger output heads. The factorized design improves FGD from 5.942 to 5.019 and violation from 1.679 to 1.095 cm. VI CONCLUSION We presented GestAdapt, a workspace-conditioned framework for robot co-speech gesture generation. By unifying six co-speech datasets with wrist-envelope supervision, GestAdapt generates gestures conditioned on speech, recent motion, and target region. Analytic guidance reduces wrist violations, while constrained retargeting handles embodiment-specific gesture execution requirements. Quantitative evaluation shows low workspace violations and favorable motion performance. In the user study, GestAdapt outperforms its no-workspace counterpart and remains close to the reference motions. Participants prefer GestAdapt to both alternatives after identical workspace-constrained robot retargeting and execution. These findings support incorporating the target workspace into gesture generation rather than introducing it only through downstream correction. Current limitations include the generation of wrist boxes and the lack of whole-body collision guarantees; future work will extend the method toward scene-level geometry and whole-body collision constraints. ACKNOWLEDGMENT This work was supported by the Dutch Research Council (NWO) through an NWO XS grant (Grant ID FLRUI47134). References [1] S. Nyatsanga, T. Kucherenko, C. Ahuja, G. E. Henter, and M. Neff (2023) A comprehensive review of data-driven co-speech gesture generation. Computer Graphics Forum 42 (2), p. 569–596. External Links: Document Cited by: §I, §I. [2] Y. Yoon, B. Cha, J. Lee, M. Jang, J. Lee, J. Kim, and G. Lee (2020) Speech gesture generation from the trimodal context of text, audio, and speaker identity. ACM Transactions on Graphics 39 (6), p. 222:1–222:16. External Links: Document Cited by: §I, §I, §I-A, TABLE I, §V-B. [3] S. Alexanderson, G. E. Henter, T. Kucherenko, and J. Beskow (2020) Style-controllable speech-driven gesture synthesis using normalising flows. Computer Graphics Forum 39 (2), p. 487–496. External Links: Document Cited by: §I, §I, §I-A, TABLE I, §V-B. [4] H. Liu, Z. Zhu, N. Iwamoto, Y. Peng, Z. Li, Y. Zhou, E. Bozkurt, and B. Zheng (2022) Beat: a large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis. In European conference on computer vision, p. 612–630. Cited by: §I, §I-A, §I-A. [5] L. Zhu, X. Liu, X. Liu, R. Qian, Z. Liu, and L. Yu (2023) Taming diffusion models for audio-driven co-speech gesture generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 10544–10553. Cited by: §I, §I, §I-A. [6] H. Liu, Z. Zhu, G. Becherini, Y. Peng, M. Su, Y. Zhou, X. Zhe, N. Iwamoto, B. Zheng, and M. J. Black (2024) Emage: towards unified holistic co-speech gesture generation via expressive masked audio gesture modeling. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 1144–1154. Cited by: §I, §I, §I-A, §IV-B. [7] Y. Yoon, W. Ko, M. Jang, J. Lee, J. Kim, and G. Lee (2019) Robots learn social skills: end-to-end learning of co-speech gesture generation for humanoid robots. In 2019 International Conference on Robotics and Automation (ICRA), p. 4303–4309. External Links: Document Cited by: §I, §I-B. [8] Z. Liang, X. Xing, M. Yang, W. Zhou, and X. Xu (2026) PhysDrift: bridging the embodiment gap in humanoid co-speech motion generation. External Links: 2606.19935, Document Cited by: §I, §I-B. [9] Z. Wang, Z. Ren, P. Shi, Z. Wang, C. Lin, T. Wang, Z. Qi, L. Zhao, H. Wang, and L. Yi (2026) RoboGesture: real-time semantic-aligned co-speech gestures generation for humanoid interaction. Note: Accepted at ECCV 2026 External Links: 2608.28693 Cited by: §I, §I-B. [10] B. Ding, S. Ancel, G. Spigler, and M. Kirtay (2026) Co-speech with you: training-free personalization of robot co-speech gestures. External Links: 2609.13876 Cited by: §I. [11] S. Rajan, K. Bhosikar, and C. Sharma (2026) InteracTalker: prompt-based human-object interaction with co-speech gesture generation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), p. 1438–1447. Cited by: §I, §I-C. [12] V. Adeli, S. Mehraban, J. Rommann, H. Sanborn, C. Clifford, and B. Taati (2026) Puppeteer: object-grounded posture-aware co-speech gesture generation. External Links: 2609.00369, Document Cited by: §I, §I-C. [13] J. Li, D. Kang, W. Pei, X. Zhe, Y. Zhang, Z. He, and L. Bao (2021) Audio2Gestures: generating diverse gestures from speech audio with conditional variational autoencoders. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 11293–11302. Cited by: §I, §I-A. [14] S. Ghorbani, Y. Ferstl, D. Holden, N. F. Troje, and M. Carbonneau (2023) ZeroEGGS: zero-shot example-based gesture generation from speech. Computer Graphics Forum 42 (1), p. 206–216. External Links: Document Cited by: §I, §I-A, §I-A. [15] S. Ginosar, A. Bar, G. Kohavi, C. Chan, A. Owens, and J. Malik (2019) Learning individual styles of conversational gesture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 3497–3506. External Links: Document Cited by: §I-A. [16] T. Kucherenko, P. Jonell, S. van Waveren, G. E. Henter, S. Alexanderson, I. Leite, and H. Kjellström (2020) Gesticulator: a framework for semantically-aware speech-driven gesture generation. In Proceedings of the 2020 International Conference on Multimodal Interaction (ICMI), p. 242–250. External Links: Document Cited by: §I-A. [17] H. Yi, H. Liang, Y. Liu, Q. Cao, Y. Wen, T. Bolkart, D. Tao, and M. J. Black (2023) Generating holistic 3d human motion from speech. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 469–480. External Links: Document Cited by: §I-A, §I-A, §I-A, §IV-B. [18] Q. Yang, L. Huang, K. Wang, J. Guan, S. He, F. Li, H. Zhou, L. Yu, Y. Li, H. Feng, and H. Xie (2025) GestureHYDRA: semantic co-speech gesture synthesis via hybrid modality diffusion transformer and cascaded-synchronized retrieval-augmented generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 12615–12625. Cited by: §I-A, §I-A. [19] X. Liu, Q. Wu, H. Zhou, Y. Xu, R. Qian, X. Lin, X. Zhou, W. Wu, B. Dai, and B. Zhou (2022) Learning hierarchical cross-modal association for co-speech gesture generation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 10452–10462. Cited by: §I-A, §I-A. [20] Y. Ferstl and R. McDonnell (2018) Investigating the use of recurrent motion modelling for speech gesture generation. In Proceedings of the 18th international conference on intelligent virtual agents, p. 93–98. Cited by: §I-A, §I-A. [21] C. Yu and A. Tapus (2020) SRG3^3: speech-driven robot gesture generation with GAN. In 2020 16th International Conference on Control, Automation, Robotics and Vision (ICARCV), p. 759–766. External Links: Document Cited by: §I-B. [22] B. Dariush, M. Gienger, A. Arumbakkam, C. Goerick, Y. Zhu, and K. Fujimura (2008) Online and markerless motion retargeting with kinematic constraints. In 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, p. 191–198. External Links: Document Cited by: §I-B. [23] M. Hassan, D. Ceylan, R. Villegas, J. Saito, J. Yang, Y. Zhou, and M. J. Black (2021) Stochastic scene-aware motion prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 11354–11364. External Links: Document Cited by: §I-C. [24] K. Zhao, Y. Zhang, S. Wang, T. Beeler, and S. Tang (2023) DIMOS: synthesizing diverse human motions in 3d indoor scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 14738–14749. Cited by: §I-C. [25] J. J. Kuffner, K. Nishiwaki, S. Kagami, M. Inaba, and H. Inoue (2001) Motion planning for humanoid robots under obstacle and dynamic balance constraints. In Proceedings 2001 IEEE International Conference on Robotics and Automation (ICRA), Vol. 1, p. 692–698. External Links: Document Cited by: §I-C. [26] N. Ratliff, M. Zucker, J. A. Bagnell, and S. Srinivasa (2009) CHOMP: gradient optimization techniques for efficient motion planning. In 2009 IEEE International Conference on Robotics and Automation (ICRA), p. 489–494. External Links: Document Cited by: §I-C. [27] Q. Liao, T. E. Truong, X. Huang, Y. Gao, G. Tevet, K. Sreenath, and C. K. Liu (2026) Beyondmimic: from motion tracking to versatile humanoid control via guided diffusion. Science Robotics 11 (117), p. eadx8924. Cited by: §IV-C. [28] J. Chen, Y. Liu, J. Wang, A. Zeng, Y. Li, and Q. Chen (2024) Diffsheg: a diffusion-based approach for real-time speech-driven holistic 3d expression and gesture generation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 7352–7361. Cited by: TABLE I, §V-B. [29] J. P. Araujo, Y. Ze, P. Xu, J. Wu, and C. K. Liu (2025) Retargeting matters: general motion retargeting for humanoid motion tracking. arXiv preprint arXiv:2510.02252. Cited by: §IV-D. [30] T. Kucherenko*, P. Wolfert*, Y. Yoon*, C. Viegas, T. Nikolov, M. Tsakov, and G. E. Henter (2024) Evaluating gesture generation in a large-scale open challenge: the genea challenge 2022. ACM Transactions on Graphics 43 (3), p. 1–28. Cited by: §V-B, §V-C.