Paper deep dive
Towards Professional Tennis Styles for Humanoid Robots with Adaptive Motion Planning and Tracking
Tao Huang, Ruofei Liu, Xuchen Tang, Xinyin Zhang, Junli Ren, Huayi Wang, Feiyu Jia, Yukai Qi, Kangning Yin, Weishuai Zeng, Lipeng Chen, Xi Li, Ting Wu, Kailin Li, Ruoli Dai, Jingbo Wang, Lei Han, Jiangmiao Pang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/21/2026, 4:10:58 AM
Summary
The paper introduces AdaPT, an Adaptive Motion Planning and Tracking framework that enables humanoid robots to perform professional tennis serving and rallying by learning from broadcast videos. It addresses the sim-to-real gap through speed-adaptive tracking and planning, demonstrating successful real-world deployment on Unitree G1 and Dobot Atom robots without motion capture.
Entities (9)
Relation Signals (7)
ADAPT → runson → Unitree G1
confidence 95% · Real-world experiments on the Unitree G1 demonstrate the effectiveness of our adaptation mechanism
ADAPT → runson → Dobot Atom
confidence 95% · We further deploy AdaPT policies on the full-size Dobot Atom humanoid robot
ADAPT → mitigates → Sim-to-Real Gap
confidence 92% · AdaPT... mitigate compounding errors... bridging the sim-to-real gap
ADAPT → learnsstylefrom → Roger Federer
confidence 90% · AdaPT captures and reproduces the distinctive playing styles of three professional players—Roger Federer
ADAPT → learnsstylefrom → Rafael Nadal
confidence 90% · AdaPT captures and reproduces the distinctive playing styles of three professional players—Rafael Nadal
ADAPT → learnsstylefrom → Novak Djokovic
confidence 90% · AdaPT captures and reproduces the distinctive playing styles of three professional players—Novak Djokovic
GVHMR → usedfor → Motion Reconstruction
confidence 85% · We reconstruct SMPL motions from the videos using GVHMR
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Humanoid robots have recently demonstrated promising capabilities in real-world ball sports. However, achieving professional motion styles while maintaining strong task performance remains challenging. In this work, we propose AdaPT, an Adaptive Motion Planning and Tracking framework that learns professional tennis serving and rally styles directly from broadcast videos. This hierarchical design is motivated by the key insight that the planner generates stylistic kinematic motions, while the tracker executes them with minimal interference with planning. Despite its effectiveness in simulation, a substantial sim-to-real gap emerges: tracking performance inevitably degrades on real robots, and this degradation is partially overlooked by autoregressive planning and further compounded by noisy perception. To address these issues, our adaptation mechanism improves tracking robustness by learning to track randomized execution speeds, while conditioning the planner on a learned motion-speed adapter to mitigate compounding errors. Real-world experiments on the Unitree G1 demonstrate the effectiveness of our adaptation mechanism in bridging the sim-to-real gap. We further deploy AdaPT policies on the full-size Dobot Atom humanoid robot (1.7m) and demonstrate in-the-wild serving without motion capture. Beyond these results, our real-world experiments reveal both algorithmic and engineering insights for future humanoid ball-sports systems. Videos and code are available on our \href{this https URL}{project website}.
Tags
Links
- Source: https://arxiv.org/abs/2608.20087v1
- Canonical: https://arxiv.org/abs/2608.20087v1
Trouble viewing inline? Open PDF directly →
Full Text
84,211 characters extracted from source content.
Expand or collapse full text
Stylized Humanoid Tennis with Adaptive Motion Planning and Tracking : *Equal Contribution † Advising ‡ Lead Towards Professional Tennis Styles for Humanoid Robots with Adaptive Motion Planning and Tracking Tao Huang Affiliation: Noitom Robotics Affiliation: Shanghai AI Laboratory Affiliation: Shanghai Jiao Tong University Ruofei Liu Affiliation: Noitom Robotics Affiliation: Shanghai Jiao Tong University Xuchen Tang Affiliation: Noitom Robotics Xinyin Zhang Affiliation: Noitom Robotics Junli Ren Affiliation: Shanghai AI Laboratory Huayi Wang Affiliation: Shanghai AI Laboratory Feiyu Jia Affiliation: Shanghai AI Laboratory Yukai Qi Affiliation: Noitom Robotics Kangning Yin Affiliation: Shanghai AI Laboratory Weishuai Zeng Affiliation: Shanghai AI Laboratory Lipeng Chen Affiliation: Shanghai Jiao Tong University Xi Li Affiliation: Dobot Robotics Ting Wu Affiliation: Noitom Robotics Kailin Li Affiliation: Shanghai AI Laboratory Ruoli Dai Affiliation: Noitom Robotics Jingbo Wang Affiliation: Shanghai AI Laboratory Lei Han Affiliation: Noitom Robotics Jiangmiao Pang Affiliation: Shanghai AI Laboratory Abstract Humanoid robots have recently demonstrated promising capabilities in real-world ball sports. However, achieving professional motion styles while maintaining strong task performance remains challenging. In this work, we propose AdaPT, an Adaptive motion Planning and Tracking framework that learns professional tennis serving and rally styles directly from broadcast videos. This hierarchical design is motivated by the key insight that the planner generates stylistic kinematic motions, while the tracker executes them with minimal interference with planning. Despite its effectiveness in simulation, a substantial sim-to-real gap emerges: tracking performance inevitably degrades on real robots, and this degradation is partially overlooked by autoregressive planning and further compounded by noisy perception. To address these issues, our adaptation mechanism improves tracking robustness by learning to track randomized execution speeds, while conditioning the planner on a learned motion-speed adapter to mitigate compounding errors. Real-world experiments on the Unitree G1 demonstrate the effectiveness of our adaptation mechanism in bridging the sim-to-real gap. We further deploy AdaPT policies on the full-size Dobot Atom humanoid robot (∼ 1.7m) and demonstrate in-the-wild serving without motion capture. Beyond these results, our real-world experiments reveal both algorithmic and engineering insights for future humanoid ball-sports systems. Videos and code are available on our project website. 1 Introduction Humanoid robots have recently demonstrated promising capabilities in dynamic ball sports, including tennis [49], table tennis [36, 12, 30], and badminton [19, 4]. Existing work mainly focuses on task performance (e.g., hitting and ball return) while paying less attention to professional styles. Such styles have been proven not merely aesthetic, but reflect structured whole-body coordination that is critical for efficient force generation and rapid recovery in human sports biomechanics [6, 16]. For humanoid robots, we envision that these styles can be leveraged to improve motion efficiency and task performance, embracing their temporally limited actuation and energy constraints. In this work, we study tennis, a challenging ball sport involving diverse strokes, high-speed interaction, and full-body coordination. We first construct a lightweight pipeline to extract professional-player motions from broadcast videos and annotate stroke type, spin type, and serving toss timing. To preserve motion style while ensuring task robustness, we adopt a decoupled planning-and-tracking framework [18, 47, 13, 43], separating stylistic high-level motion generation from low-level execution. For rallying, we built on Vid2Player3D [47] and train an MVAE-based motion generator [18, 47] and a planner that infers motion latents and tracking speed from predicted future ball trajectories. For serving, we build on AdaMimic [13] and train a residual tracker to balance task performance and stylistic fidelity. While this decoupled design is effective in simulation, a substantial sim-to-real gap emerges when deploying on real humanoid systems. In particular, tracking performance inevitably degrades due to imperfect actuator modeling in simulation [15, 9], and such errors are further amplified by the autoregressive nature of motion planning and perception noise of hardware, leading to compounding deviations over long-horizon execution [48]. To address these issues, we introduce AdaPT (Adaptive Motion Planning and Tracking), a framework that jointly models motion planning and tracking with explicit speed adaptation. It improves tracking robustness by exposing the policy to randomized motion execution speeds during training. Meanwhile, the high-level planner learns a motion-speed representation, enabling it to adapt planning to varying tracking capabilities and mitigate error accumulation in real-world deployment. We validate AdaPT on both the Unitree G1 and the full-size Dobot Atom humanoid robots. Using broadcast video data, AdaPT captures and reproduces the distinctive playing styles of three professional players—Rafael Nadal, Roger Federer, and Novak Djokovic—for both rallying and serving. We further show that AdaPT generalizes to motion-capture (MoCap) data, demonstrating its applicability across diverse motion sources. Beyond controlled laboratory settings, we deploy AdaPT in the wild, achieving successful serving with a YOLO-based [29] camera perception system for ball localization and HTC VIVE Ultimate Trackers [11] for robot localization. Together, these results demonstrate the robustness, versatility, and practical deployment potential of AdaPT for future in-the-wild humanoid ball sports. We overview the real-world performance in and summarize our core contributions as follows: • A unified adaptive motion planning and tracking framework that mitigates the sim-to-real gap by improving robustness to tracking errors and enabling the planner to adapt to the tracker’s capabilities. • Professional humanoid tennis across motion sources and embodiments is demonstrated through rallying and serving from both video and motion-capture data, on both the Unitree G1 and full-size Dobot Atom robots. • Real-world deployment beyond motion capture is further demonstrated via successful in-the-wild serving. • Algorithmic and engineering insights from real-world experiments that provide practical design guidance for future humanoid ball-sports systems. 2 Related Work Robotic ball sports. Robotic systems have demonstrated impressive capabilities in a variety of ball sports [14, 38, 32, 24, 23, 39, 3, 5, 44]. Recent humanoid systems have achieved promising performance in football [7, 31, 42], basketball [40], badminton [19, 4], table tennis [36, 30, 12], and tennis [49], primarily focusing on task success such as hitting and ball returning. In contrast, we investigate how humanoid robots can acquire professional athletic styles while maintaining robust task execution, a setting that introduces a substantially larger sim-to-real challenge due to the increased requirements on timing, coordination, and force generation. We further study stylized humanoid serving, a whole-body skill that remains largely unexplored in real-world humanoid ball sports. Athletic humanoid motion learning. Recent works have explored learning athletic humanoid skills from motion data. AdaMimic [13] demonstrates adaptable motion tracking from a single reference motion, but does not incorporate perception and has not been applied to dynamic ball sports. We extend its adaptation mechanism to stylized tennis serving, improving robustness to ball toss variations while reducing policy optimization difficulty. For ball sports, PULSE [21] introduces a continuous motion prior that has been successfully adopted in various downstream tasks, including the ball sports benchmarks in SMPLOlympics [22]. LATENT [49] further extends this paradigm to real humanoid tennis. While these methods achieve strong task performance, the motion prior is tightly coupled with environment dynamics, making it difficult to preserve professional motion styles from references. Vid2Player3D [47] addresses this issue through decoupled motion planning [18] and tracking, where planning focuses solely on generating kinematic motions. However, we find that this decoupled formulation suffers from a substantial sim-to-real gap during real-world deployment, as tracking degradation, autoregressive planning errors, and perception noise jointly lead to compounding execution errors. Building on Vid2Player3D, we introduce adaptive planning and tracking mechanisms to mitigate these issues, enabling robust deployment on real humanoid robots. Furthermore, we provide empirical insights into the trade-offs between the above approaches for real-world humanoid tennis. 3 AdaPT: Adaptive Motion Planning and Tracking for Humanoid Tennis Our goal is to learn professional humanoid tennis skills through the decoupled motion planning and tracking framework, where the motion planning is performed independently of environment dynamics and focuses on generating kinematic motions to ensure stylistic imitation (Fig. 2). Intuitively, rally and serve exhibit fundamentally different characteristics, motivating distinct method designs. Figure 1: t-SNE visualization of motions and tracker rollouts. Characteristics of rally and serve. For rally skills, the robot must react to unknown incoming ball trajectories and generate diverse motions that can cover large return regions while supporting different stroke types. In contrast, serving is a self-initiated process from ball toss to racket contact, which is largely independent of external environment dynamics and exhibits much lower motion diversity, as verified by the motion distribution shown in Fig. 1. From a reinforcement learning (RL) perspective, they correspond to self-play with an unknown opponent policy and standard single-agent RL problems, respectively. Methodology choices. Based on these observations, we build the rally method upon Vid2Player3D [47] and learn an MVAE-based motion generator [18] to provide the planner with diverse motion generation capabilities. For serve, we instead build upon AdaMimic [13], where we train a residual tracker to better balance tasks and styles. Overview of AdaPT. With the curated tennis motion dataset (Section 3.1), we train motion trackers using randomized speeds to enable speed-adaptive tracking (Section 3.2). We then train planners with an extra planning dimension on tracking speed for improved task performance while preserving styles (Section 3.3), followed by the details of (Section 3.4) and real-world deployment (Section 3.5). 3.1 Motion Data from Broadcast Videos Collection. To learn professional tennis styles, we collect short tennis clips from publicly available broadcast videos of Rafael Nadal, Roger Federer, and Novak Djokovic, given their distinctive playing styles and the availability of sufficient public videos. Each clip is approximately 2 seconds long and contains a complete rally motion and associated footwork. In addition, we collect high-fidelity motion-capture (MoCap) data from professional human tennis athletes to provide additional styles, demonstrating the compatibility of our framework with diverse motion data sources. We list the statistics in Table 1. Table 1: Statistics of motion data from broadcast videos of three players. Player Total Time # of Clips Stroke Ratio Spin Ratio Forehand Backhand Flat & Top Slice Rafael Nadal 267min 5664 54% 46% 89% 11% Roger Federer 179min 5264 48% 52% 84% 16% Novak Djokovic 189min 5288 49% 51% 93% 7% Mr. Black (MoCap) 219min 2987 57% 43% 76% 24% Processing. We reconstruct SMPL [20] motions from the videos using GVHMR [35] and retarget them to the humanoid robot with GMR [2]. Since GVHMR often produces inaccurate wrist estimates due to viewpoint occlusions, we refine racket-related wrist joints according to player-specific stroke styles following Vid2Player3D. We further apply randomized wrist perturbations during retargeting to diversify racket swing directions and reduce model biases. This is only applied to the rally data. Labeling. Each motion clip is annotated with player identity, stroke type, spin type, and ball contact timing. The stroke labels include forehand, backhand, and serve, while the spin labels distinguish between topspin and slice. These labels provide additional motion semantics for policy learning. For serving motions, we additionally annotate the ball release timing to estimate the toss trajectory. Correction. Since the retargeted robot motions may violate physical constraints and are not directly suitable for tracking, we train a general motion tracker on the retargeted dataset to refine them into physically plausible motion clips, resulting in the final motions rallyD_rally and serveD_serve. Figure 2: AdaPT learns stylized humanoid tennis skills through the pipeline: (a) broadcast videos are reconstructed, retargeted, and physically corrected to obtain tennis motions; (b) speed-adaptive policies learn to track serve and rally motions; (c) high-level planners select or generate style-consistent motions for serve and rally; (d) our robot localization or ball detection modules enable sim-to-real deployment, especially for serving without MoCap. 3.2 Speed-adaptive Motion Tracking Problem formulation. We formulate motion tracking as a Markov decision process [28]. At timestep t, the observation is defined as ttrack=[^t,t,˙t,˙t], o^track_t=[ q_t, q_t, θ_t, ω_t], where ^t q_t and t q_t denote the reference and current robot configurations, respectively, each comprising the root orientation and joint positions, following [17, 41]. ˙t θ_t and ˙t ω_t denote the robot’s joint velocities and base angular velocity, respectively. We find that incorporating root orientation is particularly important for learning rally styles, as it helps capture torso rotation during stroke execution. In contrast, this information is less critical for the relatively structured serving motions and is therefore omitted from the serving tracker. The policy outputs a tracking action ttrack=πtrack(ttrack), a^track_t= _track( o^track_t), conditioned on the current observation. The action ttrack a^track_t is executed as the target command of a PD controller [41] during each control period. The agent receives a reward rtr_t to minimize global and local tracking errors. Objective. Our speed adaptation builds on the idea introduced in AdaMimic [13]. While AdaMimic does not pre-train the tracker to handle varying motion speeds, we find such adaptability to be important for real-robot performance. To this end, we randomly vary the speed of the reference motion by blending the current and previous reference states: ^tα=(1−α)^t−1+α^t, q_t^α=(1-α) q_t-1+α q_t, where the blend coefficient α is uniformly sampled from [αmin,αmax][ _min, _max].The blending interpolates or extrapolates the reference motion, effectively slowing down or speeding up its temporal progression. This exposes the policy to varying execution speeds during training and improves its robustness at deployment. The training objective is defined as πtrack[∑t=0γtrt|^tα,α∼Unif.(αmin,αmax)],E_ _track [ _t=0γ^tr_t\;|\; q_t^α,α .( _ min, _max) ], We train trackers πtrackrally _track^rally and πtrackserve _track^serve for rally and serve. 3.3 Motion Planning with Speed Adaptation 3.3.1 Learning Rally with Motion Generator Motion generator. To model the diverse motions required for rally skills, we follow Vid2Player3D [47] and train an MVAE-based motion generator [18] on the corrected motion dataset rallyD_rally. Please refer to their paper for more implementation details. In addition, we augment the training objective with auxiliary prediction losses on stroke type and spin type, which improves controllability through downstream reward design. After training, we obtain an MVAE decoder DrallyD_rally, which takes as input a latent variable t z_t together with the autoregressive state ^t−1mvae p_t-1^mvae, and predicts the next motion state: ^tmvae=Drally(t,^t−1mvae). p_t^mvae=D_rally( z_t, p_t-1^mvae). High-level planning policy. On top of the low-level speed-adaptive tracker, we train a high-level planning policy πplanrallyπ^rally_plan that predicts both motion generation commands and tracking speed adaptation signals. Specifically, compared with the tracker, the planning policy additionally receives observations of the robot pose and future ball trajectory estimation and outputs a latent motion command for the MVAE decoder together with a speed adaptation variable (t,αt)=πplanrally(trally)( z_t, _t)= _plan^rally( o_t^rally). The speed adaptation signal αt _t allows the planner to dynamically adjust motion execution speed according to the incoming ball trajectory. The reference motion for tracker at timestep t is ^trally=f((1−αt)⋅^t−1mvae+αt⋅^tmvae), q_t^rally=f((1- _t)· p_t-1^mvae+ _t· p_t^mvae), where f converts the MVAE motion representation into the tracker reference state. The weighted MVAE state is then used to condition the next stage of generation. 3.3.2 Learning Serving with Residual Tracker High-level planning policy. Unlike rally, serving motions exhibit significantly lower motion diversity and are largely self-driven. Therefore, instead of learning an additional motion generator, we directly track motions from the serving dataset serveD_serve. On top of the low-level tracker, we train a high-level planning policy πplanserve _plan^serve to adapt motion execution speed during serving. Specifically, the planner receives observations of the robot state, ball state, and its last action, and predicts a speed adaptation variable αt=πplanserve(tserve). _t= _plan^serve( o_t^serve). The resulting tracking reference ^tserve=(1−αt)^t−1+αt^t q_t^serve=(1- _t) q_t-1+ _t q_t is then constructed by interpolating adjacent serving references. Residual motion tracker. Directly tracking serving motions alone is insufficient, since real-world toss trajectories inevitably introduce variations that cannot be fully handled by fixed reference tracking. To address this issue, we build upon AdaMimic [13] and train a residual tracker that predicts an additional correction term on top of the reference-tracking action: tserve=ttrack+Δt, a^serve_t= a^track_t+ a_t, where Δt=πtrackdelta(tserve) a_t=π^delta_track( o^serve_t) is a residual action that enables adaptive racket control under toss variations while preserving the underlying serving style. Keyframing. We observe that, when optimizing primarily for the striking reward, the policy may converge to a local optimum that neglects the backswing, resulting in less expressive serving motions. A similar issue was observed in AdaMimic [13] for jumping tasks, where task rewards alone can favor solutions that achieve the objective while deviating from the desired motion style. To address this issue, we designate the frame with the deepest backswing as a keyframe and introduce an additional sparse local tracking reward around this phase. Specifically, we impose local tracking objectives on the racket and wrist poses, with a reward weight 50× larger than that of the standard tracking reward. This keyframe-guided supervision encourages the policy to preserve the characteristic backswing while retaining flexibility in the remaining motion phases. Tossing guidance. We model the tennis toss trajectory as a parabola [46], where the ball release and striking positions define the start and end points of the trajectory. Assuming gravity-only ballistic motion during the ascending phase, we estimate the expected toss velocities from the release-to-strike duration in the data. The dense tossing reward is computed as the deviation between the actual ball trajectory and the predicted parabolic trajectory over time. Figure 3: Serve hardware. Ball-tossing control. During the tossing, the end-effector undergoes two phases: grasping and release. We set it closed before the tossing time and open afterward, allowing our serving system to accommodate different end-effector hardware (examples in Fig. 3). To overcome real-world end-effector control latency and simulation inaccuracies in toss collisions, we randomize the release timing and randomly perturb ball velocity after release during training. Table 2: Simulation results. Evaluation of all candidate methods across different metrics for rally and serve tasks by learning the motion of three master-level players. ↑ denotes higher is better, while ↓ denotes the opposite. Overall, AdaPT consistently achieves strong motion fidelity while maintaining high task performance, demonstrating an effective balance between stylistic imitation and task performance across both rallying and serving. Stroke Rafael Nadal Roger Federer Novak Djokovic Succ. ↑ Ebo↓E_bo EFID↓E_FID Epow↓E_pow Eacc↓E_acc Succ. ↑ Ebo↓E_bo EFID↓E_FID Epow↓E_pow Eacc↓E_acc Succ. ↑ Ebo↓E_bo EFID↓E_FID Epow↓E_pow Eacc↓E_acc RL-Scratch [34] 97.4 1.6±0.6± 0.6 297.3±16.3± 16.3 92.7±20.0± 20.0 123.1±8.9± 8.9 98.8 2.5±1.8± 1.8 299.0±29.8± 29.8 130.8±27.9± 27.9 145.0±8.3± 8.3 98.8 2.5±1.8± 1.8 307.7±18.4± 18.4 130.8±27.9± 27.9 145.0±8.3± 8.3 AMP [27] 98.1 2.7±0.9± 0.9 172.1±8.0± 8.0 118.1±39.3± 39.3 126.9±12.5± 12.5 65.1 2.1±1.2± 1.2 116.0±32.3± 32.3 112.7±62.4± 62.4 122.5±16.7± 16.7 21.6 2.2±1.4± 1.4 66.9±39.7± 39.7 185.6±92.2± 92.2 155.2±57.7± 57.7 PULSE [21] 86.2 2.5±1.4± 1.4 29.7±3.5± 3.5 92.2±31.1± 31.1 107.1±11.4± 11.4 92.2 2.5±1.1± 1.1 17.9±1.5± 1.5 94.1±26.7± 26.7 101.0±9.5± 9.5 86.1 3.4±1.7± 1.7 32.9±6.3± 6.3 78.6±29.6± 29.6 92.8±7.5± 7.5 NCP [50] 85.6 4.8±2.3± 2.3 9.2±3.5± 3.5 33.0±13.2± 13.2 55.3±5.6± 5.6 89.1 3.6±1.5± 1.5 6.8±4.7± 4.7 24.2±15.3± 15.3 48.6±5.2± 5.2 69.0 4.9±2.3± 2.3 24.0±2.8± 2.8 22.8±6.2± 6.2 56.6±4.6± 4.6 Vid2Player3D [47] 82.6 4.8±2.0± 2.0 5.1±5.2± 5.2 15.1±5.6± 5.6 41.1±5.6± 5.6 81.5 2.9±1.2± 1.2 13.4±3.2± 3.2 18.5±8.3± 8.3 41.9±7.9± 7.9 83.2 3.3±1.9± 1.9 6.3±3.2± 3.2 24.0±9.7± 9.7 40.6±7.3± 7.3 AdaPT (ours) 91.5 2.9±1.4± 1.4 5.4±2.8± 2.8 16.3±4.3± 4.3 43.3±5.0± 5.0 96.3 2.2±1.1± 1.1 6.1±1.3± 1.3 25.2±6.7± 6.7 47.7±4.1± 4.1 92.3 2.8±1.7± 1.7 6.3±4.0± 4.0 22.4±7.7± 7.7 41.4±6.4± 6.4 Serve Rafael Nadal Roger Federer Novak Djokovic Succ. ↑ Ebounce↓E_bounce Empkpeglobal↓E^global_mpkpe Empkpelocal↓E^local_mpkpe Eacc↓E_acc Succ. ↑ Ebounce↓E_bounce Empkpeglobal↓E^global_mpkpe Empkpelocal↓E^local_mpkpe Eacc↓E_acc Succ. ↑ Ebounce↓E_bounce Empkpeglobal↓E^global_mpkpe Empkpelocal↓E^local_mpkpe Eacc↓E_acc RL-Scratch [34] 99.9 −- 192.1±6.5± 6.5 190.5±8.0± 8.0 13.8±3.4± 3.4 1.1 10.9±1.5± 1.5 166.1±34.9± 34.9 165.0±21.7± 21.7 51.8±8.8± 8.8 0.0 −- 193.1±38.6± 38.6 188.0±29.3± 29.3 52.3±9.1± 9.1 AMP [27] 74.2 5.5±2.3± 2.3 119.8±20.9± 20.9 138.9±30.6± 30.6 36.5±7.5± 7.5 94.5 4.3±2.2± 2.2 197.9±11.9± 11.9 222.5±34.8± 34.8 62.1±10.8± 10.8 95.1 3.3±1.8± 1.8 271.7±64.2± 64.2 265.0±48.0± 48.0 58.5±38.2± 38.2 AdaMimic [13] 0.0 −- 54.3±6.1± 6.1 63.2±15.8± 15.8 63.6±73.4± 73.4 0.0 −- 67.4±8.2± 8.2 63.5±18.6± 18.6 55.8±6.9± 6.9 4.7 12.1±3.0± 3.0 80.1±24.7± 24.7 86.9±47.2± 47.2 88.0±90.2± 90.2 DeepMimic [26] 99.7 2.1±1.2± 1.2 73.8±3.9± 3.9 79.3±9.0± 9.0 27.7±3.4± 3.4 100.0 1.9±0.9± 0.9 107.7±2.8± 2.8 268.7±8.1± 8.1 51.2±5.1± 5.1 100.0 2.4±1.4± 1.4 70.2±3.7± 3.7 70.1±5.6± 5.6 35.5±4.3± 4.3 DeepMimic-Distill 99.3 2.2±1.3± 1.3 60.6±4.6± 4.6 64.3±6.8± 6.8 27.5±4.7± 4.7 100.0 2.1±1.3± 1.3 74.0±3.4± 3.4 64.0±4.7± 4.7 44.7±4.7± 4.7 100.0 2.1±1.2± 1.2 60.3±3.9± 3.9 58.9±4.9± 4.9 34.8±4.1± 4.1 AdaPT-w/o-Planner 99.4 2.3±1.5± 1.5 62.0±3.5± 3.5 60.6±5.8± 5.8 29.0±3.5± 3.5 99.9 2.0±1.2± 1.2 66.4±3.7± 3.7 62.2±6.0± 6.0 46.7±5.6± 5.6 100.0 2.6±1.5± 1.5 70.7±3.5± 3.5 78.6±8.8± 8.8 37.5±4.3± 4.3 AdaPT (ours) 99.7 2.2±1.2± 1.2 58.1±5.8± 5.8 59.9±7.3± 7.3 25.2±6.8± 6.8 99.9 2.3±1.4± 1.4 66.2±3.7± 3.7 60.6±5.7± 5.7 42.0±4.6± 4.6 99.9 2.4±1.4± 1.4 53.4±4.4± 4.4 54.3±6.8± 6.8 35.8±4.8± 4.8 3.4 Training Details All policies are trained in Mjlab [45] using PPO [34] with 4096 parallel simulation environments on 4 NVIDIA RTX 4090 GPUs. Both the policy and value networks are parameterized as 3-layer MLPs. We present more details, such as reward and MVAE states in the Appendix. 3.5 Real-world Deployment Ball localization. We consider two ball perception setups for real-world deployment. In the motion-capture setup, we directly obtain the 3D ball position from markers attached to the ball. In the camera-based setup, we use a YOLO26 [29] detector followed by stereo triangulation to estimate the 3D ball position, rather than relying on hand-crafted HSV-based color segmentation and region-of-interest [44]. We find our solution is more robust and accurate. For rallying, we use a 6-camera stereo vision system following ESTHER [44], while a single stereo camera is sufficient for serving due to its more structured setup. Although we adopt a similar multi-view setup to ESTHER, we later analyze its limitations in our system in the experiments. Future trajectory prediction. Given the estimated 3D ball position and velocity, we predict future ball trajectories using a simple physics-based model that ignores spin and considers only gravity and aerodynamic drag. We propagate the trajectory forward under a constant drag model to obtain short-horizon predictions. Robot localization. For robot localization, we use MoCap in controlled laboratory experiments and HTC VIVE Ultimate Trackers [11] for in-the-wild deployment. The motion capture system provides accurate full-body pose estimation, while the VIVE trackers provide the robot’s global pose without requiring a MoCap environment, enabling deployment outside the laboratory. High-level policy switching. We find that a single rally policy can suffer from motion drift when no ball is present: the autoregressive MVAE may gradually deviate from its stationary state, causing undesired oscillations. To address this issue, we employ two high-level policies: a standing policy and a rally policy. The standing policy maintains a stable posture when the robot is not actively responding to the ball, while the rally policy generates task-specific motions for ball interception and hitting. We switch between the two policies according to the current ball state. 4 Experiments 4.1 Experimental Setup Hardware devices. The main experiments are conducted in an approximately 20m×8m20m× 8m motion-capture arena equipped with 35 Noitom MCC-400 cameras [25] for tracking the robot root pose and ball position. We use the Unitree G1 robot for testing. The tennis ball is customized to be reflective by attaching reflective materials to standard tennis balls. Due to the limited capture space, we adopt the ‘Tennis 10s’ system, whose singles court size is approximately 18×6.518× 6.5 m. This setting is also well-matched to the physical capabilities of G1, with a height of 1.35 m. We use a 23-inch junior tennis racket with a wrist mount fixed at a 25-degree offset to approximate a natural racket-gripping posture. Fig. 4 visualizes the experiment setup. Figure 4: Experimental setup for rally and serve task with motion capture. Evaluation. For fair rally evaluation in real robot experiments, we use an Acemate ball feeding machine [1], and symmetrically place it to launch balls into predefined target regions with reproducible trajectories, repeating 25 times for each region. These launcher configurations are unseen during training, enabling reliable evaluation of forehand and backhand generalization. Each trial launches one ball every 4.5s. For serving evaluation, we use the Synria Gloria-D parallel gripper [37] to grasp and toss the tennis ball with 15 trials per model. We also evaluate the Unitree Dex-3 dexterous hand on simpler amateur-level motions; its weight (much heavier than the parallel gripper) prevents G1 from balancing under professional serving motions. Metrics. We employ three shared metrics for both tasks: hit success rate, bounce-position error EboE_bo, and joint acceleration error EaccE_acc, which assess motion smoothness. The target position is fixed at the center of the opponent’s court for rally shots, and randomly sampled inside the diagonal service box as a policy command for serves. For rally, we adopt the Fréchet Inception Distance EFIDE_FID [10] in joint space to measure style similarity to reference motions, alongside the power metric EpowE_pow for motion intensity. For serve, we present global and local Mean Per-Keyframe Position Error (MPKPE), namely EmpkpeglobalE^global_mpkpe and EmpkpelocalE^local_mpkpe. Baselines. We employ several shared baselines across both tasks. RL-Scratch (with PPO [34]) learns the policy without motion data, while AMP [27] incorporates adversarial style rewards from reference motions. For rally, we also compare against large-scale motion-prior approaches, including the continuous-latent-prior PULSE [21] and the discrete-prior NCP [50, 8]. We additionally consider the recent LATENT [49] for humanoid tennis, which is built upon PULSE. Since its full implementation is not publicly available, we use PULSE as a proxy baseline to provide insight into it. Vid2Player3D [47] is a closely related simulation method that adopts a planner-tracker architecture without adaptation mechanisms. For serving, we include DeepMimic [26] and AdaMimic [13], where the latter extends single-reference imitation with speed adaptation. Remark. All subsequent experiments are evaluated using data from broadcast videos, while MoCap data is used only for real-world robot demonstrations. Table 3: Real-robot evaluation of rally skills. The results demonstrate that both the Adaptive Tracker and Adaptive Planner contribute to robust real-world rally performance over all three players. The performance gap between simulation and real-world deployment is further analyzed in Section 4.5. Player Method Adaptive Tracker Adaptive Planner Hit Rate (%) ↑ Net Clearance Rate (%) ↑ EFIDE_FID ↓ EaccE_acc ↓ Sim. Real-FH Real-BH Sim. Real-FH Real-BH Sim. Real-FH Real-BH Sim. Real-FH Real-BH Rafael Nadal Vid2Player3D ∙ 0.6458pt 0.8$ $ ∙ 0.6458pt 0.8$ $ 82.6 28.0 28.0 68.5 28.5 14.8 7.9 15.4 10.4 45.2 18.8 14.7 AdaPT (ours) ✓ ✓ 91.5 44.0 56.0 72.8 27.2 35.7 5.3 19.4 12.9 43.0 19.9 16.3 w/o-AdaPlanner ✓ ∙ 0.6458pt 0.8$ $ 71.7 44.0 28.0 42.1 27.2 28.5 5.5 22.2 14.3 40.8 16.2 14.5 w/o-AdaTracker ∙ 0.6458pt 0.8$ $ ✓ 80.6 40.0 36.0 62.9 50.0 44.4 8.0 18.3 12.9 45.7 19.2 19.1 Roger Federer Vid2Player3D ∙ 0.6458pt 0.8$ $ ∙ 0.6458pt 0.8$ $ 81.5 24.0 8.0 88.3 16.6 0.0 5.1 9.1 9.2 41.1 16.5 15.3 AdaPT (ours) ✓ ✓ 96.3 64.0 48.0 86.9 56.3 50.0 5.4 15.1 13.9 16.3 14.9 21.6 w/o-AdaPlanner ✓ ∙ 0.6458pt 0.8$ $ 80.9 52.0 16.0 67.5 15.3 0.0 10.7 18.8 17.9 41.3 17.0 15.9 w/o-AdaTracker ∙ 0.6458pt 0.8$ $ ✓ 87.5 48.0 20.0 86.5 33.3 20.0 8.6 14.2 12.1 47.7 19.9 23.2 Novak Djokovic Vid2Player3D ∙ 0.6458pt 0.8$ $ ∙ 0.6458pt 0.8$ $ 83.2 8.0 20.0 70.0 0.0 20.0 6.3 17.7 13.7 40.6 16.1 19.4 AdaPT (ours) ✓ ✓ 92.3 48.0 48.0 75.6 33.3 33.3 6.3 12.0 11.8 41.4 19.6 18.2 w/o-AdaPlanner ✓ ∙ 0.6458pt 0.8$ $ 86.4 32.0 20.0 55.5 25.0 20.0 6.2 19.5 14.6 41.5 15.4 16.7 w/o-AdaTracker ∙ 0.6458pt 0.8$ $ ✓ 83.9 24.0 24.0 78.5 16.7 33.3 12.0 12.8 11.1 42.5 11.0 20.3 4.2 Simulation Results and Analyses We present the main simulation results in Table 2 and analyze different methods under rally and serve settings, aiming to extract key insights on the algorithmic side. Figure 5: Analysis of failure of RL-Scratch serving. RL-Scratch verifies the importance of motion data. In rally, it achieves high hit success, indicating strong reactive control capability in continuous ball interactions. However, its generated motions exhibit large deviations from human-like stroke dynamics, leading to high FID. This gap becomes more pronounced in serve, where precise ball placement and stable whole-body coordination are required. In this setting, RL-Scratch succeeds for only one player style. This reflects a strong sensitivity to player-specific serving strategies, where different players exhibit distinct ball-tossing trajectories and pre-serve motion patterns. Specifically, the narrow striking window provides little RL exploration guidance, while unstable postures can prevent basic balance. The Nadal case is an exception: its stable initial posture enables balance and reward hacking by keeping the racket attached to the ball (Appendix C). Table 4: Real-robot evaluation of serve skills. The results demonstrate that both the Adaptive Tracker and Adaptive Planner contribute to robust real-world rally performance over all three players. Performance drops between simulation and real-world deployment are discussed in Section 4.5. Player Method Succ. ↑ EBo3m↓E_Bo^3m EBo6m↓E_Bo^6m EBo9m↓E_Bo^9m Edoftrack↓E_dof^track Eacc↓E_acc Rafael Nadal DeepMimic 80.0 0.7±0.3± 0.3 3.5±0.1± 0.1 6.4±0.6± 0.6 0.18±0.01± 0.01 9.9±2.9± 2.9 AdaPT (ours) 66.7 1.5±0.5± 0.5 1.5±0.4± 0.4 5.3±1.7± 1.7 0.17±0.01± 0.01 12.3±0.5± 0.5 w/o-Planner 53.3 1.8±1.1± 1.1 2.8±0.9± 0.9 4.6±1.7± 1.7 0.18±0.01± 0.01 13.5±0.2± 0.2 w/o-TossRew 60.0 1.3±0.4± 0.4 2.5±1.4± 1.4 – 0.18±0.01± 0.01 11.7±0.7± 0.7 Roger Federer DeepMimic 53.3 1.6±1.0± 1.0 4.2±1.5± 1.5 7.8±0.0± 0.0 0.36±0.03± 0.03 28.4±0.9± 0.9 AdaPT (ours) 73.3 1.4±1.7± 1.7 2.8±1.2± 1.2 6.4±1.4± 1.4 0.17±0.01± 0.01 22.0±1.0± 1.0 w/o-Planner 53.3 1.1±0.2± 0.2 4.0±0.8± 0.8 5.1±1.0± 1.0 0.21±0.02± 0.02 29.8±5.3± 5.3 w/o-TossRew 73.3 1.2±0.7± 0.7 3.4±0.6± 0.6 7.0±0.6± 0.6 0.18±0.02± 0.02 29.2±5.9± 5.9 Novak Djokovic DeepMimic 60.0 2.5±0.00± 0.00 4.2±0.00± 0.00 – 0.23±0.01± 0.01 19.1±2.0± 2.0 AdaPT (ours) 86.7 1.8±0.8± 0.8 2.0±1.3± 1.3 5.6±1.7± 1.7 0.14±0.02± 0.02 22.8±8.0± 8.0 w/o-Planner 53.3 2.5±0.4± 0.4 2.3±0.8± 0.8 7.6±0.1± 0.1 0.17±0.02± 0.02 22.7±6.6± 6.6 w/o-TossRew 80.0 1.6±0.2± 0.2 3.8±0.5± 0.5 6.5±1.1± 1.1 0.18±0.01± 0.01 19.0±6.7± 6.7 AMP struggles at capturing phase and whole-body coordination. In rally, AMP produces more human-like stroke motions, but footwork remains highly sensitive to task objectives and is often distorted by ball recovery demands, leading to significantly reduced whole-body coordination across player styles. In serving, this limitation is amplified: without explicit phase structure, the model fails to capture the full force-generation process from preparation to strike [13]. As a result, AMP can underperform RL-Scratch despite style rewards, suggesting that adversarial imitation alone is insufficient to capture perceptive, phase-consistent, and whole-body coordination in ball sports. Decoupled planning and tracking improves rally styles. Motion-prior methods such as PULSE and NCP, despite their hierarchical structure, still exhibit strong coupling between planning and tracking. In particular, their VAE-based encoders condition on both the motion reference and the current robot proprioception, causing the latent representation to be heavily entangled with robot state. This limits the expressiveness of the motion prior, leading to higher EFIDE_FID, while the coupling enables more responsive control. This coupling is also reflected in physically noticeable artifacts, including distorted swing motions and inconsistent footwork patterns during ball recovery, suggesting a loss of stylistic motion under interaction-driven execution. In contrast, Vid2Player3D explicitly decouples planning and tracking, which improves stylistic consistency and results in significantly better EFIDE_FID, while introducing a degradation in task performance due to reduced responsiveness to interaction dynamics. To address this trade-off, our proposed speed-adaptive mechanism better balances between task responsiveness and styles in the rally task. Introducing phase-aware speed adaptation improves serving style. Directly applying DeepMimic with task conditioning achieves reasonable serve success, but often sacrifices stylistic details such as wrist rotation and follow-through due to coupling between task reward and motion style [36, 30]. While AdaMimic enables joint optimization of motion speed, tracking, and task execution, full-trajectory optimization leads to unstable learning and suboptimal convergence in the serving task. AdaPT addresses these issues via adaptation and a phase mask. 4.3 Real-world Results and Analyses In real-world evaluation, ceiling collisions may occur for high-arcing rally shots, making estimation of landing positions unreliable. Instead, we report the net clearance rate in Table 3. Serves are more controlled in height, enabling reliable evaluation of landing accuracy in Table 4. Significant sim-to-real gap of planning-tracking architecture. Directly applying Vid2Player3D leads to a significant sim-to-real degradation in rally performance (Table 3). While it achieves reasonable results in simulation, its real-robot performance drops substantially across both forehand and backhand settings, especially in net clearance rate and hitting consistency. This highlights the limitation of the decoupled planner and tracker in real-world dynamics. Effect of adaptive tracking and planning in rally task. Both adaptive tracking and planning improve real-robot rally performance by addressing different sim-to-real gaps. The adaptive tracker reduces low-level execution errors, yielding consistent gains over the non-adaptive variant across forehand and backhand settings. In contrast, the adaptive planner improves high-level temporal coordination by adjusting execution speed, leading to higher hit rates compared to Vid2Player3D. However, each component alone is insufficient: tracking alone cannot resolve timing errors, while planning alone remains sensitive to control noise. Combining both yields the best performance, demonstrating their complementarity in mitigating both temporal and execution-level errors. Figure 6: Planner analysis. The planners successfully learn to adjust motion speed to (a, b) minimize tracking error and (c) adapt to varied ball-tossing height, (d) exhibiting better style than DeepMimic. Effect of speed adaptation on stylized serving. Compared with DeepMimic, our two-stage serve training better preserves player-specific motion styles and achieves lower tracking errors (Table 4). However, directly tracking stylized motions is insufficient for robust real-world serving, as fixed execution timing can degrade task accuracy under disturbances. Our adaptive mechanism addresses this by adjusting execution speed during deployment, enabling faster racket acceleration and stronger long-distance serves (low Ebo9mE_bo^9m error), while preserving realistic wrist pronation and player-specific serving styles (Fig. 4). 4.4 Analyses of Key Components We analyze several key components in our system that are critical for learning professional tennis skills, providing insights for humanoid ball sports in the future. Figure 7: Sensitivity of planner-tracker methods to the horizon of the future ball trajectory. Trade-off between rally responsiveness and motion style fidelity. We find that incorporating future ball trajectory observations consistently improves task performance in both simulation and the real world. We attribute this to the decoupled planner-tracker architecture, where the planner benefits from longer-horizon information for improved foresight in trajectory planning. This observation motivates the hypothesis that tightly coupled policies, such as NCP and PULSE, may rely less on long-horizon observations due to their stronger real-time responsiveness. To validate this, we evaluate different future observation horizons for both types of methods in Fig. 7. It confirms a key trade-off: Key trade-off: Decoupled planning-tracking architectures rely on longer and more accurate lookahead ball predictions to preserve temporally coherent motion styles. In contrast, coupled architectures, such as NCP- or PULSE-style policies, react directly to current observations and offer better real-time responsiveness, often at the cost of motion naturalness. This finding provides a possible explanation for why observing only the current ball position and velocity is already sufficient for learning rally in LATENT [49], which is built upon a PULSE-style tightly coupled architecture. Adaptation works for serve. We analyze tracking errors and residual actions across motion phases in Fig. 6. AdaPT achieves the lowest tracking error during the hitting phase with smaller residual actions. Meanwhile, the planner dynamically adjusts the motion speed according to the ball-toss height, enabling adaptation to varying tosses. Figure 8: Tossing guidance as a reward is crucial to imitate tossing style and control landing. Tossing guidance is important for serving. Interestingly, we observe that, without tossing guidance, the model can still achieve reasonable serve success rates, but exhibits significantly worse control over ball landing positions (see Table 4). To better understand this phenomenon, we analyze real-robot serving trajectories and find that, without tossing guidance, the policy tends to produce overly low tosses in order to simplify ball interception and improve stability. However, this leads to insufficient reaction time and limited controllability over the final landing location. In contrast, introducing tossing guidance encourages a higher and more consistent toss trajectory. This not only provides the robot with more preparation time but also enables stronger racket acceleration and more effective “whipping” motions. As a result, the robot can better reproduce fine-grained technical details such as wrist pronation, while achieving more accurate control of the serve landing position. 4.5 Analyses of Sim-to-real Gap Post-bounce trajectory prediction is critical for rally. Future ball trajectory prediction is affected by inevitable velocity estimation noise and irregular ball-ground interactions caused by the ball’s reflective surface, particularly after the bounce (Fig. 9a). More importantly, post-bounce prediction errors exhibit a stronger negative correlation with rally success than pre-bounce errors (Fig. 9b,c), indicating that accurate post-bounce trajectory prediction is particularly important for reliable ball interception. Figure 9: Impact of trajectory prediction errors on rally success. (a) Future ball trajectory prediction errors. (b,c) Correlation between pre- and post-bounce prediction errors and rally success, respectively. Planning and tracking drifts limit rally. Real-robot ablations show that AdaPlanner and AdaTracker respectively mitigate planning and tracking drift (Table 5). Simulation further reveals that both drifts are negatively correlated with hit success (Fig. 10), explaining the improved real-world performance of both single-component variants over Vid2Player3D and their further gains when combined. Method EdoftrackE_dof^track ↓ EMVAEFIDE_MVAE^FID ↓ Succ. ↑ Vid2Player3D 0.29 20.0 16% AdaPT 0.23 4.8 56% w/o AdaPlanner 0.21 15.0 34% w/o AdaTracker 0.27 10.3 34% Table 5: AdaPlanner and AdaTracker mitigate planning and tracking drift in the real robot experiments. Figure 10: The negative correlation between motion fidelity and hit success in the simulation. Initial pose and ball perturbations improve serving robustness. We identify initial pose randomization and ball perturbation as important factors in reducing the sim-to-real gap for serving. Ball perturbation substantially improves success and prevents OOD-induced falls, while pose randomization reduces bounce error and improves robustness to variations in the initial orientation (Table 6 and Fig. 11). Method Succ. ↑ Fall. ↓ EBo9m↓E_Bo^9m Edoftrack↓E_dof^track Eacc↓E_acc w/o-DR 20% 40.0 6.2 0.18 15.1 w/o-BallDR 40% 40.0 4.8 0.17 13.8 w/o-PoseDR 80% 20.0 5.3 0.17 14.4 AdaPT 90% 0.0 2.1 0.14 11.0 Table 6: Effect of domain randomization on real-world serve robustness. Figure 11: Visualization of serve’s domain randomization. Large trajectory estimation noise with camera. We further compare ball perception using the motion capture system and the multi-camera vision setup. Although the cameras are jointly calibrated, we observe larger localization noise and temporals jitter in vision-based detections (Fig. 12), which can lead to inaccurate trajectory prediction. We further evaluate two ball detection approaches and find that YOLO-based detection consistently outperforms HSV-based ball extraction with region-of-interest (ROI) filtering [44] for rallying, leading to a higher hit success rate. For service, YOLO achieves performance comparable to the motion capture system and substantially better than the HSV-based approach, likely because the cameras are positioned closer to the ball during the motion. Since our adaptation mechanism relies heavily on accurate future ball trajectory estimation, perception errors can substantially degrade real-world performance. These results suggest that camera-based perception remains a promising direction for scalable humanoid tennis systems, while highlighting the importance of accurate and temporally stable ball localization in real tennis court environments. Figure 12: Large trajectory estimation noise with camera influence on real performance. But YOLO-based perception is sufficient for serve. 5 Conclusion and Limitations We proposed AdaPT, a solution for stylized humanoid tennis playing. AdaPT learns from professional player motions and enables both rallying and serving. Through decoupled planning and tracking, AdaPT achieves strong task performance while preserving player-specific styles in real-world humanoid robots, G1 and Atom. Limitations. The proposed system still relies primarily on motion capture to obtain the robot root and ball positions. Future work should explore more robust camera-based perception for in-the-wild deployment. Meanwhile, the observed trade-off between responsiveness and style fidelity suggests the need for more effective learning algorithms that can better balance real-time adaptability with motion consistency. Additionally, developing a motion capture-free localization pipeline for robust ball and robot state estimation remains an important direction for real-world deployment. On the hardware side, more capable wrist actuation may further enable more stable dexterous serving motions and subsequently improve strike precision. We leave more discussion in Appendix C. Acknowledgements We sincerely thank our teammates from the Intern Robotics team at Shanghai AI Laboratory, including Xiao Chen, Zirui Wang, Xiaojie Niu, Jiahe Chen, Weixiang Zhong, and Furui Xu, for their valuable suggestions on the early-stage preparation and algorithm design of this work. We are grateful to the Dobot engineering team, including Shiwen Liao, Yiliang Huang, Jiajun Wang, and Shuhai Jiang, for their support in designing the camera-based ball localization and robot spatial localization systems. In particular, we thank Tongbiao Cai for his patience and dedication in continuously refining the camera-based detection system and working alongside us through countless iterations and experiments. We also thank Wude Wang for his invaluable support in deploying our system on the Atom humanoid robot and resolving numerous hardware issues. We also thank the Motion Capture Team of Noitom Robotics and all the athletes who participated in our motion capture sessions for helping us rapidly collect high-quality rally and serve data, which was essential for our iterative development and evaluation. Finally, we dedicate this work to Roger Federer, Rafael Nadal, Novak Djokovic, and all the great tennis players whose perseverance, passion, and refusal to give up have continually inspired us to move forward. References [1] Acemate (2026) Acemate tennis robot. External Links: Link Cited by: §4.1. [2] J. P. Araujo, Y. Ze, P. Xu, J. Wu, and C. K. Liu (2026) Retargeting matters: general motion retargeting for humanoid motion tracking. In International Conference on Robotics and Automation (ICRA), Cited by: §3.1. [3] D. Büchler, S. Guist, R. Calandra, V. Berenz, B. Schölkopf, and J. Peters (2022) Learning to play table tennis from scratch using muscular robots. Transactions on Robotics (TRO). Cited by: §2. [4] Y. Chen, S. Dong, X. Ji, J. Sun, Z. Luo, L. Zhao, J. Zhang, W. Li, J. Ma, B. Xu, et al. (2026) Learning human-like badminton skills for humanoid robots. arXiv preprint arXiv:2602.08370. Cited by: §1, §2. [5] D. B. DAmbrosio, S. Abeyruwan, L. Graesser, A. Iscen, H. B. Amor, A. Bewley, B. J. Reed, K. Reymann, L. Takayama, Y. Tassa, et al. (2025) Achieving human level competitive robot table tennis. In International Conference on Robotics and Automation (ICRA), Cited by: §2. [6] B. Elliott (2006) Biomechanics and tennis. British journal of sports medicine. Cited by: §1. [7] T. Haarnoja, B. Moran, G. Lever, S. H. Huang, D. Tirumala, J. Humplik, M. Wulfmeier, S. Tunyasuvunakool, N. Y. Siegel, R. Hafner, et al. (2024) Learning agile soccer skills for a bipedal robot with deep reinforcement learning. Science Robotics. Cited by: §2. [8] L. Han, Q. Zhu, J. Sheng, C. Zhang, T. Li, Y. Zhang, H. Zhang, Y. Liu, C. Zhou, R. Zhao, et al. (2024) Lifelike agility and play in quadrupedal robots using reinforcement learning and generative pre-trained models. Nature Machine Intelligence 6 (7), p. 787–798. Cited by: §4.1. [9] T. He, J. Gao, W. Xiao, Y. Zhang, Z. Wang, J. Wang, Z. Luo, G. He, N. Sobanbabu, C. Pan, Z. Yi, G. Qu, K. Kitani, J. Hodgins, L. ". Fan, Y. Zhu, C. Liu, and G. Shi (2025) ASAP: aligning simulation and real-world physics for learning agile humanoid whole-body skills. In Robotics Science and Systems (RSS), Cited by: §1. [10] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §4.1. [11] HTC VIVE (2024) VIVE ultimate tracker – full-body tracking for standalone vr. Note: Accessed: 2026-01-18 External Links: Link Cited by: §1, §3.5. [12] M. Hu, W. Chen, W. Li, F. Mandali, Z. He, R. Zhang, P. Krisna, K. Christian, L. Benaharon, D. Ma, et al. (2025) Towards versatile humanoid table tennis: unified reinforcement learning with prediction augmentation. arXiv preprint arXiv:2509.21690. Cited by: §1, §2. [13] T. Huang, H. Wang, J. Ren, K. Yin, Z. Wang, X. Chen, F. Jia, W. Zhang, J. Long, J. Wang, et al. (2026) Towards adaptable humanoid control via adaptive motion tracking. In International Conference on Robotics and Automation (ICRA), Cited by: 3rd item, §1, §2, §3.2, §3.3.2, §3.3.2, Table 2, §3, §4.1, §4.2. [14] X. Huang, Z. Li, Y. Xiang, Y. Ni, Y. Chi, Y. Li, L. Yang, X. B. Peng, and K. Sreenath (2023) Creating a dynamic quadrupedal robotic goalkeeper with reinforcement learning. In International Conference on Intelligent Robots and Systems (IROS), Cited by: §2. [15] J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter (2019) Learning agile and dynamic motor skills for legged robots. Science Robotics. Cited by: §1. [16] W. B. Kibler and A. Sciascia (2004) Kinetic chain contributions to elbow function and dysfunction in sports. Clinics in sports medicine. Cited by: §1. [17] Q. Liao, T. E. Truong, X. Huang, G. Tevet, K. Sreenath, and C. K. Liu (2025) BeyondMimic: from motion tracking to versatile humanoid control via guided diffusion. ArXiv abs/2508.08241. Cited by: 1st item, §3.2. [18] H. Y. Ling, F. Zinno, G. Cheng, and M. Van De Panne (2020) Character controllers using motion vaes. ACM Transactions on Graphics (TOG). Cited by: §1, §2, §3.3.1, §3. [19] C. Liu, L. Jiang, Y. Wang, K. Yao, J. Fu, and X. Ren (2025) Humanoid whole-body badminton via multi-stage reinforcement learning. arXiv preprint arXiv:2511.11218. Cited by: §1, §2. [20] M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black (2015) SMPL: a skinned multi-person linear model. ACM Transactions on Graphics (TOG). Cited by: §3.1. [21] Z. Luo, J. Cao, J. Merel, A. Winkler, J. Huang, K. Kitani, and W. Xu (2024) Universal humanoid motion representations for physics-based control. In International Conference on Learning Representations (ICLR), Cited by: 3rd item, §2, Table 2, §4.1. [22] Z. Luo, J. Wang, K. Liu, H. Zhang, C. Tessler, J. Wang, Y. Yuan, J. Cao, Z. Lin, F. Wang, et al. (2024) Smplolympics: sports environments for physically simulated humanoids. arXiv preprint arXiv:2407.00187. Cited by: §2. [23] Y. Ma, A. Cramariuc, F. Farshidian, and M. Hutter (2025) Learning coordinated badminton skills for legged manipulators. Science Robotics. Cited by: §2. [24] S. Mori, K. Tanaka, S. Nishikawa, R. Niiyama, and Y. Kuniyoshi (2018) High-speed and lightweight humanoid robot arm for a skillful badminton robot. Robotics and Automation Letters. Cited by: §2. [25] Noitom (2026) MCC-400 cameras. External Links: Link Cited by: §4.1. [26] X. B. Peng, P. Abbeel, S. Levine, and M. Van de Panne (2018) Deepmimic: example-guided deep reinforcement learning of physics-based character skills. Transactions on Graphics (TOG). Cited by: 4th item, Table 2, §4.1. [27] X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa (2021) Amp: adversarial motion priors for stylized physics-based character control. Transactions on Graphics (TOG). Cited by: 2nd item, 2nd item, Table 2, Table 2, §4.1. [28] M. L. Puterman (2014) Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons. Cited by: §3.2. [29] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi (2016) You only look once: unified, real-time object detection. In Computer Vision and Pattern Recognition (CVPR), p. 779–788. Cited by: §B.1, §1, §3.5. [30] J. Ren, Y. Li, K. Zhang, P. Fu, H. Jiang, Y. Pan, G. Zeng, T. Huang, W. Guo, P. Lu, et al. (2026) SMASH: mastering scalable whole-body skills for humanoid ping-pong with egocentric vision. arXiv preprint arXiv:2604.01158. Cited by: §1, §2, §4.2. [31] J. Ren, J. Long, T. Huang, H. Wang, Z. Wang, F. Jia, W. Zhang, J. Wang, P. Luo, and J. Pang (2025) Humanoid goalkeeper: learning from position conditioned task-motion constraints. arXiv preprint arXiv:2510.18002. Cited by: §2. [32] M. Riedmiller, T. Gabel, R. Hafner, and S. Lange (2009) Reinforcement learning for robot soccer. Autonomous Robots. Cited by: §2. [33] N. Rudin, D. Hoeller, P. Reist, and M. Hutter (2024) Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning (CoRL), Cited by: §A.5. [34] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. Cited by: 1st item, 1st item, §3.4, Table 2, Table 2, §4.1. [35] Z. Shen, H. Pi, Y. Xia, Z. Cen, S. Peng, Z. Hu, H. Bao, R. Hu, and X. Zhou (2024) World-grounded human motion recovery via gravity-view coordinates. In SIGGRAPH Asia, Cited by: §3.1. [36] Z. Su, B. Zhang, N. Rahmanian, Y. Gao, Q. Liao, C. Regan, K. Sreenath, and S. S. Sastry (2025) HITTER: A humanoid table tennis robot via hierarchical planning and learning. Cited by: §1, §2, §4.2. [37] Synria (2026) Synria gloria-d parallel gripper. External Links: Link Cited by: §4.1. [38] D. Tirumala, M. Wulfmeier, B. Moran, S. Huang, J. Humplik, G. Lever, T. Haarnoja, L. Hasenclever, A. Byravan, N. Batchelor, et al. (2024) Learning robot soccer from egocentric vision with deep reinforcement learning. In Conference on Robot Learning (CoRL), Cited by: §2. [39] H. Wang, Z. Shi, C. Zhu, Y. Qiao, C. Zhang, F. Yang, P. Ren, L. Lu, and D. Xuan (2025) Integrating learning-based manipulation and physics-based locomotion for whole-body badminton robot control. In International Conference on Robotics and Automation (ICRA), Cited by: §2. [40] Y. Wang, Q. Zhao, Y. F. Lau, R. Yu, H. W. Tsui, Q. Chen, J. Wang, J. Pang, and P. Tan (2026) HumanX: toward agile and generalizable humanoid interaction skills from human videos. arXiv preprint arXiv:2602.02473. Cited by: §2. [41] Y. Wang, S. Zhu, P. Zhi, Y. Li, J. Li, Y. Li, Y. Xiao, X. Wang, B. Jia, and S. Huang (2026) Omnixtreme: breaking the generality barrier in high-dynamic humanoid control. In Robotics Science and Systems (RSS), Cited by: §3.2. [42] Y. Wang, C. Luo, P. Chen, J. Liu, W. Sun, T. Guo, K. Yang, B. Hu, Y. Zhang, and M. Zhao (2025) Learning vision-driven reactive soccer skills for humanoid robots. arXiv preprint arXiv:2511.03996. Cited by: §2. [43] M. Xu, Y. Shi, K. Yin, and X. B. Peng (2025) Parc: physics-based augmentation with reinforcement learning for character controllers. In ACM SIGGRAH Asia, Cited by: §1. [44] Z. Zaidi, D. Martin, N. Belles, V. Zakharov, A. Krishna, K. M. Lee, P. Wagstaff, S. Naik, M. Sklar, S. Choi, et al. (2023) Athletic mobile manipulator system for robotic wheelchair tennis. IEEE Robotics and Automation Letters. Cited by: §B.1, §2, §3.5, §4.5. [45] K. Zakka, Q. Liao, B. Yi, L. L. Lay, K. Sreenath, and P. Abbeel (2026) Mjlab: a lightweight framework for gpu-accelerated robot learning. arXiv preprint arXiv:2601.22074. Cited by: §3.4. [46] A. Zeng, S. Song, J. Lee, A. Rodriguez, and T. A. Funkhouser (2020) TossingBot: learning to throw arbitrary objects with residual physics. IEEE Transactions on Robotics (TRO) 36. Cited by: §3.3.2. [47] H. Zhang, Y. Yuan, V. Makoviychuk, Y. Guo, S. Fidler, X. Peng, and K. Fatahalian (2023) Learning physically simulated tennis skills from broadcast videos. ACM Transactions on Graphics (TOG). Cited by: 4th item, §A.1, §A.5, §1, §2, §3.3.1, Table 2, §3, §4.1. [48] Z. Zhang, K. Wen, M. Xu, J. He, C. Li, T. Miki, C. Schwarke, C. Zhang, X. B. Peng, and M. Hutter (2026) Learning whole-body humanoid locomotion via motion generation and motion tracking. arXiv preprint arXiv:2604.17335. Cited by: §1. [49] Z. Zhang, H. Lu, Y. Lian, Z. Chen, Y. Liu, C. Lin, H. Xue, Z. Zeng, Z. Qi, S. Zheng, et al. (2026) Learning athletic humanoid tennis skills from imperfect human motion data. arXiv preprint arXiv:2603.12686. Cited by: §1, §2, §2, §4.1, §4.4. [50] Q. Zhu, H. Zhang, M. Lan, and L. Han (2023) Neural categorical priors for physics-based character control. ACM Transactions on Graphics (TOG). Cited by: 3rd item, Table 2, §4.1. Appendix of AdaPT Appendix A Implementation Details A.1 Motion VAE and Residual Tracker Motion VAE. We follow the MotionVAE implementation in Vid2Player3D [47], which performs autoregressive motion prediction by conditioning on a short history and predicting the next future frame. Our MotionVAE state design largely follows the original formulation, with two modifications. First, we replace joint rotations with joint angles. Second, we remove global position information since reliable estimation of absolute court coordinates is not available in our pipeline. However, we find our current representation is sufficient for learning effective rally skills. To mitigate the loss of global orientation, we apply data augmentation by randomly rotating the initial frame of each motion clip along the yaw axis within [−15∘,15∘][-15 ,15 ], improving coverage over different court-facing directions. In addition, we extend the MotionVAE prediction targets with four auxiliary dimensions to better structure the latent motion space and facilitate reward design. Specifically, we predict a 2D phase variable normalized to [0,2π][0,2π], where π corresponds to the ball-strike frame, as well as two categorical attributes: spin type and stroke type. These auxiliary predictions provide additional supervision signals and enable easy reward design for rally. Residual Tracker. Compared to the Motion Tracker in the first stage, the Residual Tracker additionally incorporates the ball position and the target landing point as observations. It outputs a 27-dimensional residual dof-position within the range of [−1,1][-1,1]. To encourage the residual action to be as small as possible, an L2 loss that penalizes deviations from zero is added as an optimization objective. Additionally, the Residual Tracker also serves as a planner by outputting an extra phase increment αt _t. This phase increment is only active during the interval [ϕrelease,ϕhit+Δϕdelay][ _release, _hit+ _delay], where ϕrelease _release denotes the ball release phase, ϕhit _hit is the striking phase, and Δϕdelay _delay is a constant phase offset. During the remaining stages, we use the fixed phase increment. A.2 Reward Functions. Rally. The rally reward encourages solid racket–ball contact, deep shots after impact, and accurate opponent-court landing. The reward terms are listed below (Table 7). • Racket–ball rewards RracketR_racket (ωracket=5.0 _racket=5.0) give dense guidance before a valid strike and a constant reward afterward. Before contact, the agent is encouraged to bring the racket close to the ball and to align with the MVAE swing phase, with forehand/backhand-specific phase targets selected from the predicted skill label. A segment-persistent hit latch is triggered when geometric and kinematic contact criteria are met (distance, racket-face orientation, and approach speed, or custom racket–ball collision when enabled). Once latched, the term stays active until the next incoming-ball segment. • Landing rewards RlandingR_landing (ωlanding=100.0 _landing=100.0) supervise opponent-court placement using a single ball-dynamics rollout per segment. Shortly after impact, the current ball state is rolled forward under the same aerodynamic and bounce model as in the simulation to estimate bounce location and trajectory apex. The target landing point is defined relative to the robot’s base on the opponent’s half of the court (depth and lateral centering). Landing quality combines position error and apex-height shaping. A net-crossing constraint requires the predicted trajectory to clear the net by at least 1.1m1.1\,m. Otherwise, the landing reward is zeroed. The resulting score is cached and applied densely for the remainder of the segment. • Forward-velocity reward RforwardR_forward (ωforward=50.0 _forward=50.0) activates only after a valid hit and encourages deep shots along the forward court direction. Table 7: Reward functions for rally training. Term Weight Description Racket–ball 5.05.0 Racket–ball contact and MVAE phase alignment; latched after a valid hit. Opponent landing 100.0100.0 Predicted opponent-court bounce; must clear the net (≥1.1m≥ 1.1\,m). Forward velocity 50.050.0 Deep forward ball speed after hit. Spin selection −1.0-1.0 MVAE spin-skill prediction should be consistent with the desired one. Self-collision −10.0-10.0 Robot self-collisions. Facing backward −20.0-20.0 Base facing away from opponent court. Serve. The rewards for serving training consists of four components that jointly encourage accurate motion tracking, stable ball tossing, successful ball striking, and controlled ball landing (Table 8). • Tracking rewards follow BeyondMimic [17] to preserve reference motion. Since the wrist motion plays a key role in professional serving style, we additionally introduce a wrist tracking reward RwristR_wrist (ωwrist=1.0 _wrist=1.0) on the striking wrist DoF positions. • Tossing rewards RtossR_toss (ωtoss=5.0 _toss=5.0) enforce accurate ball release position and velocity tracking. The target toss trajectory is constructed from the dataset by extracting the palm position at release and the racket position at impact. Assuming ballistic motion under gravity, we compute the reference trajectory using a parabolic motion model. To improve robustness under sim-to-real gap in ball handling, we further introduce a palm-upward reward Rpalm-upR_palm -up (ωpalm-up=1.5 _palm -up=1.5), which encourages an upward palm orientation before the toss. • Hitting reward RhitR_hit (ωhit=3.0 _hit=3.0) is a binary reward indicating whether the racket successfully strikes the ball. Once triggered, it remains active until episode termination. A hit is considered valid if: (1) the distance between ball and racket center is below dthreshold=0.15md_threshold=0.15\,m, (2) the timestep lies within a th=0.3st_h=0.3\,s window around the estimated hit time, and (3) the pre-impact ball apex exceeds hthreshold=1.8mh_threshold=1.8\,m. • Bounce reward RbounceR_bounce (ωbounce=6.0 _bounce=6.0) penalizes the error between predicted and target landing positions. To enable efficient supervision, we estimate the landing position using an analytic ball dynamics model immediately after impact. In addition, a net-crossing reward RnetR_net (ωnet=1.0 _net=1.0) is introduced, which encourages the ball to pass over the net with height exceeding 1.1 m. Table 8: Reward functions for serving training. Term Weight Description Wrist tracking 1.01.0 Tracks striking wrist DoF positions. Tossing 5.05.0 Enforces accurate ball release position and velocity. Palm-upward 1.51.5 Encourages an upward palm orientation before the toss. Hitting 3.03.0 Binary reward for racket-ball strike. Bounce 6.06.0 Penalizes error between predicted and target landing positions. Net-crossing 1.01.0 Encourages the ball to pass over the net with height exceeding 1.1m1.1\,m. Action rate −0.2-0.2 Penalizes large action variations between consecutive timesteps. Joint limits −10.0-10.0 Penalizes joints that exceed their joint limits. Collisions −10.0-10.0 Penalizes self-collision of the robot and collision between the racket and the ground. A.3 Baseline Implementation Rally. We evaluate our method against a set of representative baselines for the rally task, covering motion-free reinforcement learning, adversarial motion priors, large-scale motion prior models, and decoupled planner–tracker systems. Overall, these baselines are selected to assess the roles of motion priors, explicit motion decomposition, and adaptive control in long-horizon interactive rally behaviors. For a fair comparison, all methods share the same observations, rewards, and low-level action space. • RL-Scratch [34] learns the policy without any motion data, relying solely on task rewards. • AMP [27] incorporates adversarial motion priors learned from reference motion, using a discriminator over short-horizon 10-step DoF trajectories to shape the policy behavior. • PULSE [21] and NCP [50] are motion-prior methods with continuous and discrete latent representations, respectively. We follow their official implementation. • Vid2Player3D [47] is a simulation-based framework for humanoid tennis that employs a decoupled planner–tracker pipeline without adaptive mechanisms. Serve. We compare our method against representative baselines that learn with or without a single reference motion. These span three categories: (1) task-driven pure RL, (2) motion priors as rewards, and (3) motion-tracking approaches. These baselines are selected to evaluate the contributions of our proposed two-stage training and adaptive mechanism. • RL-Scratch [34] removes all tracking-related rewards and learns the serving skill purely from task rewards. Specifically, only the ball-hitting reward and the landing-point reward are retained. • AMP [27] employs an adversarial motion prior with a discriminator conditioned on 10-step DoF position trajectories. The AMP reward and task reward are weighted by 0.1 and 0.9, respectively. • AdaMimic [13] follows a two-stage training procedure. In stage 1, a pure motion-tracking controller is trained without task rewards or speed adaptation, whereas AdaPT learns speed-adaptive tracking already in this stage. In stage 2, the controller is conditioned on the predicted high-level adaptation variable αt _t to modulate speed throughout the serving motion. • DeepMimic [26] adopts a single-stage training scheme that jointly optimizes tracking and task rewards without a planner. DeepMimic-Distill further distills a student policy from the stage 1 tracking policy by using the teacher’s actions as an imitation reward during training. A.4 Domain Randomization. Table 9: Domain randomization sampling distributions for rally training. Domain Randomization Sampling Distribution Physical parameters Foot tangential friction μ [0.3, 1.2]U[0.3,\,1.2] Encoder bias Δq q [−0.01, 0.01]U[-0.01,\,0.01] Base COM offset (Δx,Δy,Δz)( x,\, y,\, z) [−0.08, 0.08]U[-0.08,\,0.08] PD gain scale (kp,kd)(k_p,\,k_d) [0.95, 1.05]U[0.95,\,1.05] Torque limit scale [0.9, 1.0]U[0.9,\,1.0] Actuator delay ℓ (physics steps) [0, 4⋅decimation]U [0,\,4·decimation ] Push interval Δt t [1, 3]sU[1,\,3]\,s Push root linear velocity (vx,vy,vz)(v_x,\,v_y,\,v_z) [−0.5, 0.5],[−0.5, 0.5],[−0.2, 0.2]U[-0.5,\,0.5],\;U[-0.5,\,0.5],\;U[-0.2,\,0.2] Push root angular velocity (ωroll,ωpitch,ωyaw)( _roll,\, _pitch,\, _yaw) [−0.5, 0.5],[−0.5, 0.5],[−0.8, 0.8]U[-0.5,\,0.5],\;U[-0.5,\,0.5],\;U[-0.8,\,0.8] Joint position observation noise [±0.01]U[± 0.01] Joint velocity observation noise [±0.5]U[± 0.5] Projected gravity observation noise [±0.05]U[± 0.05] Root quaternion observation noise [±0.03]U[± 0.03] Base angular velocity observation noise [±0.2]U[± 0.2] MVAE init root offset (Δx,Δy)( x,\, y) [−0.5, 0.5]mU[-0.5,\,0.5]\,m MVAE init yaw offset Δψ ψ [−30∘, 30∘]U[-30 ,\,30 ] Ball trajectory & dynamics Serve spawn (xrel,yrel,zrel)(x_rel,\,y_rel,\,z_rel) [6, 15],[−2, 2],[0.6, 1.7]mU[6,\,15],\;U[-2,\,2],\;U[0.6,\,1.7]\,m Flight time T [0.7, 1.4]sU[0.7,\,1.4]\,s Landing target (xrel,yrel,ztgt)(x_rel,\,y_rel,\,z_tgt) [−1, 3],[−1.5, 1.5], 0mU[-1,\,3],\;U[-1.5,\,1.5],\;0\,m Re-serve interval [2.0, 2.5]sU[2.0,\,2.5]\,s Drag coefficient CdC_d [0.4, 0.8]U[0.4,\,0.8] Ground restitution e [0.5, 0.75]U[0.5,\,0.75] Horizontal bounce damping dxyd_xy [0.6, 0.8]U[0.6,\,0.8] Post-bounce velocity noise (Δvx,Δvy,Δvz)( v_x,\, v_y,\, v_z) [−0.25, 0.25]m/sU[-0.25,\,0.25]\,m/s Ball position noise Δ [−0.02, 0.02]3U[-0.02,\,0.02]^3 Ball velocity scale [0.9, 1.1]U[0.9,\,1.1] Ball packet drop L∼[3, 5]L [3,\,5] steps ; ptrigger=0.1p_trigger=0.1; Table 10: Domain randomization sampling distributions for serve training. Domain Randomization Sampling Distribution Physical parameters Foot tangential friction μ [0.3, 1.2]U[0.3,\,1.2] Encoder bias Δq q [−0.01, 0.01]U[-0.01,\,0.01] Base COM offset (Δx,Δy,Δz)( x,\, y,\, z) [−0.08, 0.08]U[-0.08,\,0.08] PD gain scale (kp,kd)(k_p,\,k_d) [0.95, 1.05]U[0.95,\,1.05] Torque limit scale [0.9, 1.0]U[0.9,\,1.0] Actuator delay ℓ (physics steps) [0, 2⋅decimation]U [0,\,2·decimation ] Push interval Δt t [1, 3]sU[1,\,3]\,s Push root linear velocity (vx,vy,vz)(v_x,\,v_y,\,v_z) [−0.5, 0.5],[−0.5, 0.5],[−0.2, 0.2]U[-0.5,\,0.5],\;U[-0.5,\,0.5],\;U[-0.2,\,0.2] Push root angular velocity (ωroll,ωpitch,ωyaw)( _roll,\, _pitch,\, _yaw) [−0.52, 0.52],[−0.52, 0.52],[−0.78, 0.78]U[-0.52,\,0.52],\;U[-0.52,\,0.52],\;U[-0.78,\,0.78] Joint position observation noise [±0.01]U[± 0.01] Joint velocity observation noise [±0.5]U[± 0.5] Projected gravity observation noise [±0.05]U[± 0.05] Base angular velocity observation noise [±0.2]U[± 0.2] Motion reference initialization Root position offset (Δx,Δy,Δz)( x,\, y,\, z) [−0.05, 0.05],[−0.05, 0.05],[−0.01, 0.01]U[-0.05,\,0.05],\;U[-0.05,\,0.05],\;U[-0.01,\,0.01] Root orientation offset (Δroll,Δpitch,Δyaw)( ,\, ,\, ) [−0.1, 0.1],[−0.1, 0.1],[−0.2, 0.2]U[-0.1,\,0.1],\;U[-0.1,\,0.1],\;U[-0.2,\,0.2] Root linear velocity offset (Δvx,Δvy,Δvz)( v_x,\, v_y,\, v_z) [−0.5, 0.5],[−0.5, 0.5],[−0.2, 0.2]U[-0.5,\,0.5],\;U[-0.5,\,0.5],\;U[-0.2,\,0.2] Root angular velocity offset (Δωroll,Δωpitch,Δωyaw)( _roll,\, _pitch,\, _yaw) [−0.52, 0.52],[−0.52, 0.52],[−0.78, 0.78]U[-0.52,\,0.52],\;U[-0.52,\,0.52],\;U[-0.78,\,0.78] Joint position offset [−0.1, 0.1]U[-0.1,\,0.1] Ball trajectory & observations Toss time delay Δttoss t_toss [−0.02, 0.05]sU[-0.02,\,0.05]\,s Toss velocity perturbation (Δvx,Δvy,Δvz)( v_x,\, v_y,\, v_z) [−0.1, 0.1]U[-0.1,\,0.1] Landing target (xtgt,ytgt)(x_tgt,\,y_tgt) [6.5, 20.0],[−1.0, 1.0]mU[6.5,\,20.0],\;U[-1.0,\,1.0]\,m Ball position observation noise [±0.1]U[± 0.1] Ball position observation delay ℓ (physics steps) [0, 5]U[0,\,5] Ball observation dropout pdrop=0.1p_drop=0.1 We present the domain randomization terms for rally and serve tasks, including physical parameters, ball trajectories and motion reference initialization, in Table 9 and Table 10, respectively. Importantly, we find that adding noise to the ball trajectory is essential for robust learning. Even in motion-capture settings, the ball observations are subject to noise, packet loss, and latency, which can significantly degrade policy performance if not accounted for during training. In addition, due to the limited vertical field of view in our setup, we mask ball observations when the ball exceeds a height threshold. These engineering efforts are crucial in practice and have a significant impact on the final performance. The performance of the rally will drop from 80% to 55% without it. A.5 PPO Implementation. Our PPO implementation follows the framework outlined in [33]. The actor and critic networks consist of 3-layer MLPs. Each iteration includes 25 steps per environment, with 5 learning epochs and 4 mini-batches per epoch. The discount factor γ is set to 0.99, and the clip ratio is set to 0.2. In the rally task, the entropy coefficient is set as 0 for training stability, following [47], and 0.01 in other trainings by default. In the serving task, the weight of the L2 loss for the residual action is 0.01. Appendix B Implementation Details in the Real B.1 Ball Localization with Cameras Ball detection. We also evaluate the motion-based ROI extraction and HSV-based color thresholding used in ESTHER [44]. However, we find this heuristic-based approach more sensitive to visual noise and background motion. Therefore, our primary deployment pipeline uses a YOLO-based detector [29] followed by stereo triangulation for 3D ball localization. Multi-view 3D reconstruction. Given the 2D detections, we estimate 3D ball positions via calibrated multi-view geometry. Each camera provides a candidate observation, which is projected into the world coordinate frame using the pre-computed camera calibration parameters. To improve robustness across viewpoints, we perform confidence-weighted fusion over the six cameras. Specifically, each observation is weighted inversely proportional to the estimated distance between the detected ball and the corresponding camera, ensuring that closer and more reliable views contribute more strongly to the final 3D estimate. Temporal filtering. The fused 3D positions are further processed using an Extended Kalman Filter (EKF) to obtain a temporally consistent ball trajectory. The EKF operates in the world coordinate frame and estimates both position and velocity under a constant-acceleration motion model, which is suitable for airborne ball dynamics under gravity. B.2 Ball Future Trajectory Estimation. The ball’s current position is expressed in a robot-centric frame, t=wball−wbasep_t=x^ball_w-x^base_w, which provides local spatial awareness for interaction. Future ball trajectories are obtained by rolling out a physics-based prediction model that approximates ball dynamics under gravity, aerodynamic drag, and ground contact. We omit the rotation of the ball because of the difficulty of the measurement in the real world. The continuous-time dynamics can be written as ˙=−kCd‖, v=g-kC_d\|v\|v, where g is gravity, and k and CdC_d models quadratic air resistance. Ground interactions are handled using a simple restitution model with velocity reflection and damping. To provide anticipatory information, we append a K-step rollout of predicted future ball positions ^t+1,…,^t+K\ u_t+1,…, u_t+K\ computed from this dynamics model. To improve robustness to perception and modeling errors, we inject stochastic perturbations into the ball command, including position and velocity noise, short observation delays, and intermittent packet loss, simulating realistic estimator uncertainty. The corresponding noise ranges and domain randomization parameters are summarized in Table 9. On the real robot, we estimate ball velocity using a five-step temporal window over consecutive position observations, providing a smoothed velocity estimate for the command representation. B.3 System Identification We collect real-world motion capture data from 20 ball trajectories on the physical court to identify key parameters of the ball dynamics model. Given the analytic ball model used in simulation, we fit two primary physical parameters: the coefficient of restitution and the horizontal damping coefficient. These parameters govern the ball’s post-impact energy loss and in-plane velocity decay, respectively. Parameter estimation is performed by minimizing the trajectory discrepancy between the observed motion capture data and the model-predicted rollout under identical initial conditions. The resulting calibrated parameters are used consistently in simulation with randomization to reduce the sim-to-real gap in ball dynamics. Appendix C More Limitations Sim. Real Ebo↓E_bo 2.81±1.73± 1.73 3.90±2.74± 2.74 Return quality. The real-world return quality is less consistent than in simulation and is sensitive to the incoming ball landing position and velocity. We quantitatively evaluate EboE_bo (Tab. →) and visualize the landing locations in Figure. →. This raises the difficulty of human-humanoid play, and we will address this problem in the future. Limited locomotion ability. We share another important trade-off: autoregressive generation enables cyclic rally motions but sacrifices locomotion capability. Fig. → shows lower hit rates of AdaPT at distant positions. This is less noticeable in NCP or PULSE. Addressing this problem is valuable in the future. Appendix D More Real-world Experiments Serve with a dexterous hand. For the serving task, our method supports various end-effectors for the tossing arm. In initial tests, we used Unitree’s Dex-3 as the end-effector. However, its substantial weight poses a collision risk during real-robot deployment, as professional players typically swing the tossing arm at high speeds. To mitigate this, we recorded an amateur serving motion with reduced tossing arm swing and tested it on the real robot (Fig. 13), achieving a 90% success rate over 10 trials. In future work, we plan to adopt a lighter dexterous hand to enable more professional serving motions. Figure 13: Amateur serving motion using the Unitree Dex-3 end-effector. In-the-wild serve. We visualize the setup in Fig. 14. Figure 14: The setup of in-the-wild serve.