Paper deep dive
ModPack: An Extensible Teleoperation Interface for Bimanual Mobile Manipulation
Joshua Citron, Renee Zbizika, Zeyi Liu, Shuran Song
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 88%
Last extracted: 7/23/2026, 2:20:47 AM
Summary
The paper introduces ModPack, a modular and extensible teleoperation system for bimanual mobile manipulation. It features a wearable backpack core that integrates computation, power, and communication, supporting plug-and-play modules for joint-level control, mobile base tracking, and active perception. The system enables cross-robot adaptation, haptic feedback, and data collection for policy learning, demonstrated on two distinct robot platforms.
Entities (12)
Relation Signals (11)
ModPack → includes → Backpack Core
confidence 95% · At the core of ModPack is a self-contained wearable “backpack”
ModPack → usedon → RB-Y1m
confidence 95% · Experiments across two distinct robot platforms... RB-Y1m robot
ModPack → supports → Leader Arms
confidence 92% · Built on top of this shared interface, the system supports plug-and-play capability modules including joint-level teleoperation
ModPack → provides → Haptic Feedback
confidence 90% · haptic feedback, enabling safer and more precise teleoperation
ModPack → supports → Active Perception
confidence 90% · Built on top of this shared interface, the system supports plug-and-play capability modules including... active perception.
ModPack → usedon → Customized Mobile Robot
confidence 90% · customized mobile robot with ARX5 arms
ModPack → uses → Vision Pro
confidence 85% · By wirelessly connecting to a VR headset, such as the Vision Pro
ModPack → uses → iPhone
confidence 85% · We track the operator’s base motion through an iPhone mounted on ModPack
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Existing teleoperation systems are often tailored to specific robot hardware and task domains, limiting their scalability and adaptability. We present ModPack, a modular and extensible teleoperation system designed to support diverse robot embodiments and task requirements within a unified framework. At the core of ModPack is a self-contained wearable "backpack" that integrates onboard computation, power, communication, and data storage. Built on top of this shared interface, the system supports plug-and-play capability modules including joint-level teleoperation with haptic feedback, mobile manipulation, and active perception. Experiments across two distinct robot platforms and real-world mobile manipulation tasks demonstrate that ModPack provides a flexible and reusable framework for data collection and policy learning. To support future research, we open-source the complete hardware design and software stack. Project website: this https URL
Tags
Links
- Source: https://arxiv.org/abs/2607.19479v1
- Canonical: https://arxiv.org/abs/2607.19479v1
Trouble viewing inline? Open PDF directly →
Full Text
45,739 characters extracted from source content.
Expand or collapse full text
ModPack: An Extensible Teleoperation Interface for Bimanual Mobile Manipulation Joshua Citron Renee Zbizika Zeyi Liu Shuran Song Stanford University modpack-robotics.github.io Abstract Existing teleoperation systems are often tailored to specific robot hardware and task domains, limiting their scalability and adaptability. We present ModPack, a modular and extensible teleoperation system designed to support diverse robot embodiments and task requirements within a unified framework. At the core of ModPack is a self-contained wearable “backpack” that integrates onboard computation, power, communication, and data storage. Built on top of this shared interface, the system supports plug-and-play capability modules including joint-level teleoperation with haptic feedback, mobile manipulation, and active perception. Experiments across two distinct robot platforms and real-world mobile manipulation tasks demonstrate that ModPack provides a flexible and reusable framework for data collection and policy learning. To support future research, we open-source the complete hardware design and software stack. Keywords: Mobile Manipulation, Teleoperation Systems, Imitation Learning 1 Introduction Figure 1: ModPack is an extensible teleoperation interface for bimanual mobile manipulation, built around a wearable backpack base with plug-and-play capability modules. For example, the active perception module, shown on the left, lets operators control the robot’s head camera to search for cloth placement locations, while the leader arms module with haptic feedback, shown on the right, supports a box transfer task by preventing excessive force and facilitating bimanual coordination. Swappable leader arms further allow adaptation to different robot embodiments. Designing a teleoperation interface involves an inherent trade-off between system simplicity and functional capability. The most capable systems are often more complex and sometimes unnecessary for simpler applications. For instance, while active perception is vital for long-horizon search-and-reach tasks, it may introduce redundant overhead for tabletop manipulation. As a result, prior work has often required researchers to carefully and rigidly tailor teleoperation interfaces to match the requirements of a given robot hardware and task domain. This task- and robot-specific specialization, while effective, leads to a fragmented design space. Systems optimized for one setting are often difficult to extend or adapt to others, limiting scalability and slowing iteration. In particular, adding new capabilities typically requires substantial redesign of both hardware and software pipelines. This lack of modularity presents a key bottleneck for building unified data collection and control systems that generalize across tasks and robot platforms. In this work, we introduce ModPack – a teleoperation system that explicitly addresses this trade-off through an extensible and modular design. At its core, ModPack is a self-contained “backpack” unit that provides integrated power, onboard computation, and data storage. This unit serves as a common substrate, decoupling core system infrastructure from task- or robot- specific functionalities. Built on this foundation, we propose a plug-and-play interface that allows operators to flexibly add or remove capability modules based on task requirements. We demonstrate the versatility of this design through several modular extensions, each enabling a distinct capability: • Cross-robot joint-level control: Through swappable leader-arm modules, as shown in Figure 1, the system can adapt to diverse robot arm kinematics, enabling precise joint-level control across different robot platforms. • Mobility: We track the operator’s base motion through an iPhone mounted on ModPack, enabling mobile manipulation for tasks that require large workspace coverage. • Active perception: By wirelessly connecting to a VR headset, such as the Vision Pro, ModPack streams real-time robot head-camera observations while allowing the operator to control the camera pose. This module enables demonstrations of active perception behaviors, such as object search, occlusion handling, and viewpoint selection. • Haptic feedback: By leveraging force feedback from the robot and active motors on the leader arms, our system provides real-time haptic feedback, enabling safer and more precise teleoperation of contact-rich tasks. While these capabilities have been explored individually, integrating them into a unified, flexible, and robust teleoperation system remains challenging. First, simultaneous control of the arms, mobile base, and head introduces ambiguity in operator motion; ModPack addresses this by disentangling operator movements into component-wise commands for arm manipulation, base motion, and active perception. Second, the system must process high-rate heterogeneous data, including proprioception, visual observations, and force/torque signals for haptic feedback; we collect these streams with low-latency communications through message-queue servers. Third, extended operation requires an unobtrusive and comfortable interface; we reduce operator fatigue through gravity compensation on the exoskeleton leader arms. Finally, all capabilities must be integrated into a compact wearable form factor; we package power, computation, sensing, and communication hardware into a self-contained backpack while balancing constraints on weight, thermal management, and usability. We evaluate ModPack through real-world bimanual mobile manipulation experiments across two distinct robot platforms. Using our teleoperation device, we collect demonstrations, train behavior cloning policies, and evaluate deployment success rates. To support future research, we will open-source the full hardware design, software stack, and policy learning and deployment framework. 2 Related work Teleoperation for Robot Manipulation. Teleoperation serves as a primary paradigm for collecting high-quality demonstration data for robot learning [21, 2, 3, 26, 20, 14]. Early approaches such as kinesthetic teaching [1] enable direct physical guidance but are labor-intensive and difficult to scale. To improve usability, many systems decouple the operator from the robot, using input devices (e.g., spacemouse, joysticks, or VR controllers) to command end-effector poses with inverse kinematics [25, 13]. In contrast, kinematic replicas such as GELLO [29] enable one-to-one joint mapping, improving control fidelity and granularity. As tasks move beyond static tabletop settings, teleoperation for mobile manipulation becomes critical. Mobile ALOHA [10] enables simultaneous base and bimanual arm control via a leader–follower setup, though base and arm control are mostly decoupled. TidyBot++ [28] supports joint control of a mobile base and a single arm, but does not scale naturally to coordinated bimanual tasks. To achieve more natural human-like control, recent works explore exoskeleton-based teleoperation with isomorphic mappings. AirExo and AirExo-2 [8, 9] provide high-fidelity arm control and cross-robot adaptability, but lack integrated mobile base control. Full-body systems such as CHILD [19] and PAPRLE [15] support both arm and leg control, though typically not simultaneously. Most closely related, Supersuit [4] enables whole-body teleoperation with base and torso tracking, but does not provide active perception or haptic feedback. In contrast, ModPack covers key capabilities for mobile teleoperation, and in addition, allows users to select different combinations of components for their target system through its modular design. Please see Appendix A for a detailed comparison with prior works. Teleoperation with Haptic Feedback. Haptic feedback is critical for collecting high quality demonstrations for contact-rich tasks. A key challenge in haptic feedback for teleoperation is balancing feedback fidelity with hardware portability. High-fidelity bilateral systems can provide rich force feedback, but often rely on stationary or off-the-shelf robot arms that are too heavy and cumbersome for wearable mobile interfaces [17, 23]. In contrast, lightweight wearable systems [34, 22, 27] and vibrotactile feedback [7] improve portability, but typically sacrifice one-to-one joint-level transparency and kinesthetic resistance. In this work, we address this trade-off with a wearable teleoperation interface built from motorized GELLO arms integrated into a backpack-mounted frame. By actuating each joint directly, our system provides high-fidelity, joint-level haptic feedback while preserving the portability required for mobile teleoperation. Teleoperation with Active Perception. Active perception is the ability to adaptively change viewpoint to acquire task-relevant visual information. Recent works study active perception with head- or body-mounted sensing [31, 33, 32]. In teleoperation, active perception is often achieved by tracking the operator’s head pose with a VR headset and mapping it to robot camera motion while streaming the robot’s egocentric view [5, 30]. However, existing systems primarily operate in stationary settings, where camera and base motion are decoupled by design. Mobile teleoperation introduces a key challenge: disentangling viewpoint changes caused by head motion from those caused by body or base motion. Our system tracks the operator’s head and body motion with the Vision Pro and a backpack-mounted iPhone, respectively, and compensates for base motion before sending adjusted robot head commands. This decoupling enables operators to naturally walk, look around, and control the arms during data collection. 3 Method Figure 2: ModPack consists of a backpack core that supports plug-and-play modules, such as a Vision Pro for active perception and an iPhone for base control. Separate 6-DoF and 7-DoF leader arms are configured to control a customized mobile robot (top right) and an RB-Y1m robot (bottom right), respectively. ModPack is a self-contained, 3D-printed backpack system that provides onboard power, storage, and compute for mobile teleoperation. Its primary design goal is extensibility: users can easily add, remove, or replace hardware and software modules to support new robot embodiments with minimal setup. We describe the core system, implemented modules—including leader arms, mobile base, and active perception—and integrations with two distinct mobile robot platforms. 3.1 ModPack Core At the core of ModPack is a portable backpack that serves as a shared base platform for modular hardware extensions and provides the software interfaces for integration. This section describes the core hardware and software architecture that enables these extensions. Hardware. The ModPack core is a wearable backpack base unit that moves with the operator while providing a configurable mounting and housing structure for modular hardware components. The backpack is fabricated from 3D-printed parts, improving accessibility and enabling rapid modification by the research community. As shown in Fig. 2, the main backpack assembly consists of 3D-printed front and rear panels secured with toggle latches, allowing convenient access to the internal components. The front panel includes two top-mounted slots for inserting support bars used to mount the leader-arm modules. The internal volume is organized into five removable shelves that house the onboard compute, power, and optional accessories. Following ergonomic observations from prior backpack-based systems [12], which suggest that vertically distributed loads near the thoracic region can reduce carrying strain, the PC and power banks are arranged along the operator’s spine. To further reduce weight and improve thermal ventilation, excess material is removed from both the structural panels and internal shelves. Refer to Appendix B.1 for 3D printing details and Appendix C.1 for the power circuit diagram. Software. The onboard mini PC serves as a standalone compute unit for module activation, data logging, and target-robot startup and shutdown. Target robots communicate with the backpack PC through a lightweight bidirectional “bridge” API. The underlying communication is abstracted from the user and implemented using a lightweight message queue, following [11]. This abstraction decouples the core backpack stack from embodiment-specific implementations, allowing the same system to support different robots and arbitrary subsets of extended modules. A diagram of the software stack can be found in Appendix G. 3.2 Extensible Modules The core system is extensible through modular components, each consisting of a hardware unit and its corresponding software interface. This design enables physical modules to be swapped across target robots while preserving a unified API and minimizing code changes. Below, we describe three modules—leader arms, mobile base, and active perception. 3.2.1 Leader Arms As shown in Fig. 2, we design and 3D-print two leader-arm variants: a pair of 6-DoF ARX5 leader arms for a customized mobile manipulation platform [30], and a pair of 7-DoF leader arms for the RB-Y1m robot. Details of both robot platforms are provided in Appendix F. Given the CAD model of a target robot, each leader arm is constructed to be kinematically equivalent to its corresponding follower arm, following GELLO [29], enabling direct joint-space teleoperation. The leader arms are actuated with off-the-shelf Dynamixel servo motors. Additional details on 3D printing and motor selection are provided in Appendix B.1 and D, respectively. Gravity Compensation. To alleviate operator fatigue during teleoperation, we implement active gravity compensation [24] on the leader arms. The gravity torque vector τg=G(q) _g=G(q) is computed using the Orocos Kinematics and Dynamics Library (KDL) in conjunction with the leader arm’s URDF model. To enhance stability, we optionally append a joint-space damping term to form the augmented torque vector at time t: τt′=τg,t+Kdq˙tτ _t= _g,t+K_d q_t, where KdK_d is a diagonal damping gain matrix. To ensure smooth and stable assistance, the augmented torque for each joint i, denoted as τi,t′τ _i,t, is filtered using two cascaded exponential moving average (EMA) filters: τi,tinter=ατi,t−1inter+(1−α)τi,t′,τi,tfinal=ατi,t−1final+(1−α)τi,tinter, _i,t^inter=α _i,t-1^inter+(1-α)τ _i,t, _i,t^final=α _i,t-1^final+(1-α) _i,t^inter, (1) where α∈[0,1)α∈[0,1) is the smoothing factor. The final torque vector is scaled by a gain factor and commanded to the leader arm motors, subject to the maximum current constraints configured. Figure 3: Haptic feedback. The top plot shows the average joint torque on RB-Y1m left arm (blue), along with the corresponding haptic feedback torque norm commanded to the left leader arm (orange). The bottom panel shows operator’s response to the haptic feedback to adjust the force exerted on the box. Haptic Feedback To enhance operator situational awareness, we implement haptic feedback for follower robots equipped with force/torque (F/T) sensors, in our case, the RB-Y1m robot. When enabled, the system reads the net wrench at the end-effector, FeeF_e, and maps it to joint-space torques via the Jacobian transpose: τc=kc⋅JTFee, _c=k_c· J^TF_e, where kck_c is a tuneable gain. The final control torque τf _f commanded to the leader motors is the superposition of the gravity compensation torque and the haptic feedback component: τf=τgi,tfinal+τc _f= _g_i,t^final+ _c. To maintain hardware longevity and accommodate lower-cost actuators, we implement a current budgeting strategy. This limits peak torque commands and ensures the motors operate within conservative amperage constraints, facilitating safe operation without compromising the transparency of the force feedback. 3.2.2 Mobile Base We track the operator’s SE(2)SE(2) pose using an iPhone mounted on the backpack, which runs a modified WebXR session following [28]. To ensure intuitive base control, we implement an egocentric mapping procedure that couples the robot’s base translation directly to the operator’s displacement within their local frame. For example, when the operator walks forward, the robot moves forward in its own local frame. To recover the operator’s pose in the robot frame, we first transform the raw pose from WebXR (pxr,Rxr)(p_xr,R_xr) to align with the robot’s coordinate convention and obtain (pr,Rr)(p_r,R_r). The key challenge is then that the operator and iPhone rotate about different centers, as the iPhone is mounted to the backpack and not the operator. To account for the physical displacement between the iPhone and the operator, we apply sequential offsets to recover the operator’s center of rotation pop_o: a device-specific offset pdp_d to find the camera mount center pcp_c [28], then an operator-specific offset pup_u to recover the user’s true center of rotation pop_o, where RcR_c is the camera mounting orientation: pc=pr+Rrpd,po=pc+Rcpup_c=p_r+R_rp_d, p_o=p_c+R_cp_u (2) This transformation ensures that the tracked trajectory represents the user’s actual body movement rather than the motion of the backpack-mounted sensor. Details about offsets can be found in Appendix E.1. To synchronize the coordinate systems, we map the operator’s tracking pose (po,Ro)(p_o,R_o) from the WebXR session frame into the robot’s world frame. Given the session’s initial pose (ps,Rs)(p_s,R_s) and the robot’s global starting pose (pw,Rw)(p_w,R_w), the world-relative target position ptp_t is computed via: Rrel=RwRs−1,pt=pw+Rrel(po−ps)R_rel=R_wR_s^-1, p_t=p_w+R_rel(p_o-p_s) (3) The target orientation matrix is similarly updated as Rt=RrelRoR_t=R_relR_o. From the resulting world-relative target state (pt,Rt)(p_t,R_t), we extract the planar x,yx,y coordinates and the yaw heading, which are then issued as absolute target setpoints to the robot base controller. 3.2.3 Active Perception We implement the active perception module with an Apple Vision Pro running a custom visionOS app, which streams the operator’s head pose as robot neck commands similar to [30]. The interface provides a low-latency egocentric view of the robot, rendering either monocular RGB video or 3D point clouds in real time. To map the operator’s head motion, we track the current device pose Tc∈SE(3)T_c∈ SE(3) relative to the initial session pose Ts∈SE(3)T_s∈ SE(3), yielding the relative transformation Trel=Ts−1TcT_rel=T_s^-1T_c. We then convert this motion into the robot head coordinate convention using a fixed basis transformation B∈SO(3)B∈ SO(3), mapping the relative translation and rotation as pr=Bprelp_r=Bp_rel and Rr=BRrelB⊤R_r=BR_relB . Together, these define the robot-frame head command Tr∈SE(3)T_r∈ SE(3). Finally, we transform TrT_r into the robot world frame using the initial tracking reference TinitT_init, yielding the global target head pose Ttarget=TinitTrT_target=T_initT_r. Decoupling Head and Base Motion. We cannot directly command TtargetT_target, as it is expressed in the robot world frame and includes motion already induced by the mobile base. To avoid redundant commands during simultaneous navigation and active perception, we compensate for the current base pose Tbase∈SE(3)T_base∈ SE(3) and express the target in the local robot base frame: Tneck=Tbase−1TtargetT_neck=T_base^-1T_target. This isolates the residual neck motion intended by the operator. 3.3 Policy Learning We train a standard transformer-based Diffusion Policy model [6] for each task using the collected teleoperation demonstrations. For the customized mobile robot, the policy takes as input RGB-D observations from the head camera and RGB observations from the left and right wrist cameras. For the RB-Y1m robot, the policy takes as input RGB observations from two head cameras and the left and right wrist cameras, and optionally left and right arm joint torques. All visual inputs are encoded using a CLIP-pretrained ViT, with depth images repeated across three channels before encoding. Torque observations are encoded with sequential CausalConv layers following [16]. The resulting observation tokens are concatenated and cross-attended to the DiT model as conditioning inputs. Refer to Appendix H for additional details. 4 Experiments To demonstrate ModPack’s adaptibility to different robot embodiments, we evaluate Cloth Placement with Active Perception task on the customized bimanual robot with ARX5 arms from [30] with added holonomic base [28], and the Box Transfer with Haptic Feedback on an RB-Y1m robot. During collection, ModPack was configured to use the appropriate leader arms module, the mobile base module, and the active perception module. For the RB-Y1m robot, we also utilize haptic feedback on the leader arm. We discuss the specifics of each task below, followed by a summary of the results. Figure 4: Summary of Cloth Placement task. a) The robot picks up a cloth, uses active perception to locate a basket placed on either side, navigates toward it, and places the cloth inside. b) Different evaluation scenarios with varying towel colors and basket locations. c) Task success rates across policy variants over input visual modalities. d) Failure cases across different policies. 4.1 Cloth Placement with Active Perception Task Description. As shown in Figure 4 a) and b): a cloth is placed in front of the robot on a counter, where a basket is either to the left or to the right of the robot. The robot must move forward to grasp the cloth and then reverse, at which point it should locate the basket on either side and traverse towards it, finishing the task once it places the cloth in the basket. We collect data with two cloths (blue, pink), two basket locations (left, right), and randomize initial positions for the cloth. The task is considered a success if the cloth is dropped into the basket. We collected 125 demonstrations in total for this task. Results. We evaluate the task across scenarios with varying basket locations (left, right) and cloth colors (blue, pink). We compare three policies with different visual observation modalities: head camera only [Head Cam], wrist cameras only [Wrist Cam], and all cameras (head + both wrist cameras) [All Cam]. For each policy, we perform 25 rollouts with randomized cloth initial positions. See a breakdown of the evaluation scenarios in Appendix I.1. Quantitative results are shown in Figure 4 c). [Head Cam] policy achieves the highest success rate, succeeding in 22/25 rollouts. [All Cam] policy succeeds in 20/25 rollouts, while [Wrist Cam] policy succeeds on only 3/25 rollouts. We hypothesize that the head camera performs best because the cloth admits many viable grasps, reducing the need for close-up wrist observations, while the head camera provides depth and a broader egocentric view for grasping and basket localization. Moreover, using only the head camera avoids potential out of distribution wrist camera observations caused by variations in wrist pose and cloth deformation. In contrast, [Wrist Cam] most often fails due to placement errors, as the policy lacks sufficient context of the basket location. Figure 4 d) shows a common failure mode for the [Wrist Cam] policy, which often dropped the cloth on the edge or out of the basket. 4.2 Box Transfer with Haptic Feedback Figure 5: Summary of Box Transfer task. a) The robot picks up a box, rotates to observe the existing box on the shelf and infer the target placement location, and then places the box accordingly. b) Different task scenarios with varying shelf availability configurations. c) Task success rates across policy variants, comparing the full policy against head-camera-only and wrist-camera-only ablations. d) Failure cases across different policies. Task Description. As shown in Figure 5 a) and b), the robot is required to pick up a box from a cabinet, move to the rack, and place the box onto the unoccupied shelf, which can be either at the top or bottom. A trial is considered successful if the robot places the box in a stable pose on the unoccupied shelf. We collected 102 demonstrations in total, half for each of the two scenarios. Figure 6: Attention Analysis. Results. We evaluate the task across two scenarios where either the top or bottom shelf is occupied and the robot has to place the box on the other unoccupied shelf. We compare three policy variants: one using all RGB camera views (two head cameras and both wrist cameras) [All Cam], one using only the two head cameras [Head Cam], and one that takes in all camera views plus joint torque information for both arms [All Cam + Torque]. For each policy, we conduct 20 rollouts across two evaluation scenarios, varying the cart location where the box is initially placed and the robot’s starting positions. A detailed breakdown of the evaluation scenarios is provided in Appendix J.1. Quantitative results are shown in Fig. 5 c). The policy conditioned on all camera views and joint torques [All Cam + Torque] achieves the highest success rate, succeeding in 12/20 rollouts, while [Head Cam] policy succeeds in 11/20 rollouts, and [All Cam] policy succeeds in 6/20 rollouts. As seen in Figure 5 d), a common failure mode of the [All Cam] policy is bad grasp poses. We hypothesize that incorporating projected joint torques improves performance by providing a coarse alignment signal for grasp timing and depth. We provide a qualitative analysis of the model’s attention patterns. As shown in Figure 6 a), we visualize the attention weights over image patches from the final layer of the ViT encoder for both the [All Cam] and [All Cam + Torque] policies. The torque-conditioned policy appears to attend more strongly to task-relevant regions, such as the edges of the box that are informative for grasping, whereas the vision-only policy attends to less relevant regions. In Figure 6 b), we visualize the aggregated attention weights assigned to the torque tokens in the DiT model for the [All Cam + Torque] policy. The attention weights for both the left- and right-arm torque tokens increase as the robot approaches the box and initiates grasping, suggesting that the policy increasingly relies on torque information during contact-critical phases of the task. 5 Limitations One limitation of the current system is the torque capacity of the Dynamixel motors, particularly when simultaneously supporting gravity compensation and haptic feedback. Higher-torque motors could mitigate this limitation, but would also increase the weight and bulk of the wearable system. Another limitation is the battery requirement for extended untethered operation: powering both the leader-arm motors and onboard compute for several hours requires a relatively large battery setup, increasing backpack weight. Future work could reduce this weight by sharing batteries across the two leader arms and replacing the current mini PC with a lighter-weight compute platform. In the current design, we prioritize longer battery life and larger onboard storage capacity as a practical tradeoff. 6 Conclusion We present ModPack, a modular and extensible teleoperation system for collecting demonstrations across diverse mobile manipulation platforms and task settings. Built around a self-contained wearable backpack, ModPack provides a unified interface for plug-and-play modules including joint-level leader-arm control with haptic feedback, mobile-base control, and active perception. We demonstrate the flexibility of this design by deploying the system on two distinct robot embodiments and collecting demonstrations for real-world bimanual mobile manipulation tasks. Policies trained on data collected with ModPack achieve strong deployment performance, validating its utility as a practical data collection interface for robot learning. Acknowledgments The authors would like to thank all REALab members as well as Chen Chen for feedback and guidance during this project. In particular, we thank Yihuai Gao and Jeff Liu for their work on robologger as well as Xiaomeng Xu, Dian Wang, and Jisang Park for their help with the RB-Y1 robot. We would also like to thank Haochen Shi for his advice on hardware and gravity compensation. This work was supported in part by the NSF Graduate Fellowship, NSF Award #2143601, #2037101, and #2132519, Apple and Amazon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the sponsors. References [1] B. D. Argall, S. Chernova, M. Veloso, and B. Browning (2009-05) A survey of robot learning from demonstration. Robot. Auton. Syst. 57 (5), p. 469–483. External Links: ISSN 0921-8890, Link, Document Cited by: §2. [2] J. Barreiros, A. Beaulieu, A. Bhat, R. Cory, E. Cousineau, H. Dai, C. Fang, K. Hashimoto, M. Z. Irshad, M. Itkina, et al. (2026) A careful examination of large behavior models for multitask dexterous manipulation. Science Robotics 11 (113), p. eaea6201. Cited by: §2. [3] J. Bjorck, F. Castañeda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y. Fang, D. Fox, F. Hu, S. Huang, et al. (2025) Gr00t n1: an open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734. Cited by: §2. [4] T. Chen, H. Wu, J. Wang, X. Li, Z. Jin, and L. Fang (2026) SuperSuit: an isomorphic bimodal interface for scalable mobile manipulation. arXiv preprint arXiv:2603.06280. Cited by: Table 1, §2. [5] X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang (2024) Open-television: teleoperation with immersive active visual feedback. arXiv preprint arXiv:2407.01512. Cited by: Table 1, §2. [6] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song (2023) Diffusion policy: visuomotor policy learning via action diffusion. the intl. Journal of Robotics Research. Cited by: §3.3. [7] R. Ding, Y. Qin, J. Zhu, C. Jia, S. Yang, R. Yang, X. Qi, and X. Wang (2025) Bunny-visionpro: real-time bimanual dexterous teleoperation for imitation learning. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 12248–12255. Cited by: Table 1, §2. [8] H. Fang, H. Fang, Y. Wang, J. Ren, J. Chen, R. Zhang, W. Wang, and C. Lu (2024) Airexo: low-cost exoskeletons for learning whole-arm manipulation in the wild. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 15031–15038. Cited by: §2. [9] H. Fang, C. Wang, Y. Wang, J. Chen, S. Xia, J. Lv, Z. He, X. Yi, Y. Guo, X. Zhan, et al. (2025) Airexo-2: scaling up generalizable robotic imitation learning with low-cost exoskeletons. arXiv preprint arXiv:2503.03081. Cited by: Table 1, §2. [10] Z. Fu, T. Z. Zhao, and C. Finn (2024) Mobile aloha: learning bimanual mobile manipulation with low-cost whole-body teleoperation. arXiv preprint arXiv:2401.02117. Cited by: Table 1, §2. [11] Y. Gao, J. Liu, S. Li, and S. Song (2026) Gated memory policy. External Links: 2604.18933, Link Cited by: Appendix G, §3.1. [12] M. Genitrini, F. Dotti, E. Bianca, and A. Ferri (2022) Impact of backpacks on ergonomics: biomechanical and physiological effects: a narrative review. International Journal of Environmental Research and Public Health 19 (11), p. 6737. External Links: Document Cited by: §3.1. [13] A. Iyer, Z. Peng, Y. Dai, I. Guzey, S. Haldar, S. Chintala, and L. Pinto (2024) Open teach: a versatile teleoperation system for robotic manipulation. arXiv preprint arXiv:2403.07870. Cited by: §2. [14] A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y. Chen, K. Ellis, et al. (2024) Droid: a large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945. Cited by: §2. [15] O. Kwon, S. Yamsani, N. Myers, S. Taylor, J. Hong, K. Park, A. Alspach, and J. Kim (2025) PAPRLE (plug-and-play robotic limb environment): a modular ecosystem for robotic limbs. arXiv preprint arXiv:2507.05555. Cited by: Table 1, §2. [16] M. A. Lee, Y. Zhu, K. Srinivasan, P. Shah, S. Savarese, L. Fei-Fei, A. Garg, and J. Bohg (2019) Making sense of vision and touch: self-supervised learning of multimodal representations for contact-rich tasks. In 2019 IEEE International Conference on Robotics and Automation (ICRA), External Links: Link Cited by: §3.3. [17] C. Lenz and S. Behnke (2023-03) Bimanual telemanipulation with force and haptic feedback through an anthropomorphic avatar system. Robotics and Autonomous Systems 161, p. 104338. External Links: ISSN 0921-8890, Link, Document Cited by: §2. [18] C. Lu, X. Cheng, J. Li, S. Yang, M. Ji, C. Yuan, G. Yang, S. Yi, and X. Wang (2025) Mobile-television: predictive motion priors for humanoid whole-body control. In 2025 IEEE International Conference on Robotics and Automation (ICRA), p. 5364–5371. Cited by: Table 1. [19] N. Myers, O. Kwon, S. Yamsani, and J. Kim (2025) CHILD (controller for humanoid imitation and live demonstration): a whole-body humanoid teleoperation system. In 2025 IEEE-RAS 24th International Conference on Humanoid Robots (Humanoids), p. 1–6. Cited by: Table 1, §2. [20] A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. (2024) Open x-embodiment: robotic learning datasets and rt-x models: open x-embodiment collaboration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 6892–6903. Cited by: §2. [21] R. Punamiya, S. Kareer, Z. Liu, J. Citron, R. Qiu, X. Cai, A. Gavryushin, J. Chen, D. Liconti, L. Y. Zhu, et al. (2026) EgoVerse: an egocentric human dataset for robot learning from around the world. arXiv preprint arXiv:2604.07607. Cited by: §2. [22] A. Purushottam, J. Yan, C. Xu, Y. Sim, and J. Ramos (2024) Wheeled humanoid bilateral teleoperation with position-force control modes for dynamic loco-manipulation. External Links: 2407.12189, Link Cited by: Table 1, §2. [23] M. Schwarz, C. Lenz, A. Rochow, M. Schreiber, and S. Behnke (2021) NimbRo avatar: interactive immersive telepresence with force-feedback telemanipulation. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vol. , p. 5312–5319. External Links: Document Cited by: §2. [24] H. Shi, S. Hu, Y. Hou, W. Wang, K. Liu, and S. Song (2026) Minimalist compliance control. External Links: 2603.00913, Link Cited by: §3.2.1. [25] D. Shivin, A. Wensi, J. Yuqian, S. Samik, H. Jiaheng, Z. Ruohan, S. Peter, A. Ben, and M. Roberto (2024) TeleMoMa: a modular and versatile teleoperation system for mobile manipulation. arXiv (Cornell University). External Links: Link Cited by: §2. [26] G. R. Team, S. Abeyruwan, J. Ainslie, J. Alayrac, M. G. Arenas, T. Armstrong, A. Balakrishna, R. Baruch, M. Bauza, M. Blokzijl, et al. (2025) Gemini robotics: bringing ai into the physical world. arXiv preprint arXiv:2503.20020. Cited by: §2. [27] F. Wu, Z. Jiao, W. Li, Z. Zhang, H. Li, J. Wu, B. Jia, and S. Dong (2025) A vr-based robotic teleoperation system with haptic feedback and adaptive collision avoidance. IEEE Transactions on Consumer Electronics 71 (3), p. 8752–8761. External Links: Document Cited by: Table 1, §2. [28] J. Wu, W. Chong, R. Holmberg, A. Prasad, Y. Gao, O. Khatib, S. Song, S. Rusinkiewicz, and J. Bohg (2024) Tidybot++: an open-source holonomic mobile manipulator for robot learning. arXiv preprint arXiv:2412.10447. Cited by: §F.1, §2, §3.2.2, §3.2.2, §4. [29] P. Wu, Y. Shentu, Z. Yi, X. Lin, and P. Abbeel (2024) Gello: a general, low-cost, and intuitive teleoperation framework for robot manipulators. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 12156–12163. Cited by: Table 1, §2, §3.2.1. [30] H. Xiong, X. Xu, J. Wu, Y. Hou, J. Bohg, and S. Song (2025) Vision in action: learning active perception from human demonstrations. In Conference on Robot Learning, p. 5450–5463. Cited by: Table 1, §F.1, §F.1, §2, §3.2.1, §3.2.3, §4. [31] X. Xu, J. Park, H. Zhang, E. Cousineau, A. Bhat, J. Barreiros, D. Wang, and S. Song (2026) HoMMI: learning whole-body mobile manipulation from human demonstrations. External Links: 2603.03243, Link Cited by: §F.2, §2. [32] J. Yu, Y. Shentu, D. Wu, P. Abbeel, K. Goldberg, and P. Wu (2025) Egomi: learning active vision and whole-body manipulation from egocentric human demonstrations. arXiv preprint arXiv:2511.00153. Cited by: §2. [33] Q. Zeng, C. Li, J. St. John, Z. Zhou, J. Wen, G. Feng, Y. Zhu, and Y. Xu (2025) ActiveUMI: robotic manipulation with active perception from robot-free human demonstrations. External Links: 2510.01607, Link Cited by: §2. [34] Z. Zhang and C. Qian (2023) Wearable teleoperation controller with 2-dof robotic arm and haptic feedback for enhanced interaction in virtual reality. Frontiers in Neurorobotics Volume 17 - 2023. External Links: Link, Document, ISSN 1662-5218 Cited by: Table 1, §2. Appendix Appendix A Comparison with Prior Works Cross-Robot Joint-Level Haptic Mobile Active Adaptability Control Feedback Manipulation Perception Open-Television [5] ✓ ✗ ✗ ✗ ✓ Mobile-Television [18] ✓ ✗ ✗ ✓ ✓ GELLO [29] ✓ ✓ ✗ ✗ ✗ Mobile ALOHA [10] ✗ ✓ ✗ ✓ ✗ AirExo-2 [9] ✓ ✓ ✗ ✓ ✗ ViA [30] ✗ ✓ ✗ ✗ ✓ CHILD [19], PAPRLE [15], SuperSuit [4] ✓ ✓ ✗ ✓ ✗ DLM w/ Feedback [22] ✗ ✗ ✓ ✓ ✗ VR + Haptic [34, 27, 7] ✓ ✗ ✓ ✗ ✗ ModPack ✓ ✓ ✓ ✓ ✓ Table 1: Comparison with existing teleoperation systems across key capabilities. ModPack supports the key capabilities studied in prior work within a unified, modular interface. Appendix B Hardware Overview B.1 3D Printing Any 3D printed components of the backpack were printed using the Bambu Labs H2D for larger components such as leader arms and backpack panels, as well as the smaller Bambu X1 Carbon for smaller pieces. The choice for the backpack to be fabricated in two parts is such that it can be easily 3D printed. The print settings can be found in Table 2. Table 2: 3D Print Settings Parameter Value Material PLA Infill 15 % Infill Pattern Gyroid Layer Height 0.2 m Wall Count 2 Nozzle Diameter 0.4 m Support Strategy Tree Appendix C Backpack Design C.1 Backpack Circuit Figure 7: Circuit diagram for ModPack Empirically, when ModPack is run with one battery per arm and a third for the Geekom A5 mini PC, the system battery life is around 4-6 hours. Figure 7 displays the layout of the power circuit. Appendix D Leader Arms D.1 ARX (6-dof) Leader Arm D.1.1 Motor Selection Motors were selected to provide the minimum amount of torque necessary to hold the arm for mass assumptions made using OnShape materials library. All motors are Dynamixel brand, with the models and control modes enumerated below. For this leader arm, we choose to not actuate joints 5 and 6 to allow for better wrist mobility. Motor Specifications 6-DoF Leader Arm Joint Motor Model Stall Torque (Nm) Control Mode 1 XM540-W270-T 10.6 Torque (Current) 2 XM540-W270-T 10.6 Torque (Current) 3 XM430-W350-T 4.1 Torque (Current) 4 XM430-W350-T 4.1 Torque (Current) 5 XL430-W250-T 1.4 N/A 6 XL430-W250-T 1.4 N/A Gripper XL330-M288-T 0.52 N/A D.1.2 Motor Parameters Each joint motor is clipped to an empirically determined current limit, which was chosen to provide stable gravity compensation without making the joint too stiff. Damping gains were chosen empirically to reduce jitter. Joint Control Parameters Joint Current Limit (mA) Damping Gain kdk_d 1 100 0.1 2 200 0.1 3 200 0.1 4 60 0.1 5 500 0.1 6 100 0.1 Gripper 100 — D.2 RB-Y1m (7-dof) Leader Arm D.2.1 Motor Selection Motors were selected to provide the minimum amount of torque necessary to hold the arm for mass assumptions made using OnShape materials library, as well as have a high enough maximum current limit to support haptic feedback. All motors are Dynamixel brand, with the models and control modes enumerated below. For this leader arm, we choose to not actuate joints 5 and 7 to allow for more stable control and better wrist mobility, respectively. Motor Specifications 7-DoF Leader Arm Joint Motor Model Stall Torque (Nm) Control Mode 1 XM540-W270-T 10.6 Torque (Current) 2 XM540-W270-T 10.6 Torque (Current) 3 XM540-W270-T 10.6 Torque (Current) 4 XM540-W270-T 10.6 Torque (Current) 5 XM540-W270-T 10.6 N/A 6 XM430-W350-T 4.1 Torque (Current) 7 XM430-W350-T 4.1 N/A Gripper XL330-M288-T 0.52 N/A D.2.2 Motor Parameters Each joint motor is clipped to an empirically determined current limit, which was chosen to provide stable gravity compensation without making the joint too stiff. An additional current budget is allocated to actuated motors to allow for haptic feedback torques, which are rate limited to 50 Nm/s and then scaled by 0.8 on the left arm specifically, which empirically prevented jitter. Joint Control Parameters 7-DoF Leader Arm Joint Gravity Current Limit (mA) Contact Current Limit (mA) 1 200 150 2 50 50 3 100 100 4 100 100 5 — — 6 100 80 7 — — Gripper — — Appendix E Base Motion E.1 WebXR Calibration There are two offsets mentioned in the paper: the device offset pdp_d and the offset from mount center to the operator’s center of rotation pup_u. pdp_d is a fixed offset given an iPhone 17, but should be re-examined for other models. This can be done using calipers to get the offset to center of iPhone. pup_u can be measured from roughly the center of operator’s head to the mount center, though in practice does not need to be changed much from operator to operator. Appendix F Robots F.1 Customized mobile robot [30] Our customized mobile robot consists of bimanual 6-DoF ARX5 arms as well as an additional 6-DoF ARX5 for active perception as presented in [30]. The neck is equipped with an iPhone that records depth as well as RBG images, while the other two arms have a Logitech webcam to capture wrist RBG images. Details on capture rate are left for Appendix H. This setup sits on a Tidybot++ base, a four-wheeled base from [28]. F.2 RB-Y1m The RB-Y1m robot we use consists of bimanual setup with 7-DoF arms, as well as a 6-DoF actuated torso on a holonomic mobile base. Using the setup from [31], we make use of a 2-DoF active neck as well as fin-ray fingers as grippers. Continuing with the setup from [31], we use 2 wide-angle cameras, FLIR BFS-PGE-23S3C-CS, and one FLIR BFS-PGE-50S5C-C on each wrist of the two 7-DoF arms. Appendix G Software Diagram Software is split into two distinct sides as seen in Figure 8, with ModPack handling lifecycle management and modules while also setting relevant configurations for downstream robots. The per-robot configurations define the bridge topics as well as what modules are turned on, meaning that all the end user must do is call the relevant bridge API for bi-directional communication with ModPack through robot-message-queue [11]. This means that actions flow from module commands to robot through robot-message-queue servers. Data is saved in a zarr format using robologger, an open source logging pipeline. This pipeline has a main logger than coordinates individual process loggers. Figure 8: Software diagram for ModPack. Appendix H Policy During training, we synchronize all sensory inputs to the downsampled head-camera timestamps (often 10-20 Hz). Proprioceptive states are interpolated at each head-camera timestamp, with positions and gripper states interpolated linearly and orientations interpolated using SLERP. Other modalities are aligned by selecting the nearest preceding measurement to the head-camera timestamp. For the custom mobile robot, the action space consists of a 3-DoF mobile-base command (x,y,θ)(x,y,θ), a 6-DoF head-camera end-effector pose, and bimanual arm commands. We represent the head pose using 3D translation and the 6D rotation representation, yielding a 9-dimensional head action. Each 6-DoF arm is controlled by six joint positions and a one-dimensional gripper position, resulting in 7 dimensions per arm. The full action dimension is therefore 3+9+7+7=263+9+7+7=26. We predict the base command relative to the current base state, while the head, arm, and gripper commands are represented as absolute targets. For the RB-Y1M robot, the action space consists of a 3-DoF mobile-base command (x,y,θ)(x,y,θ), a 2-DoF neck command corresponding to pitch and yaw, and bimanual arm commands. Each 7-DoF arm is controlled by seven joint positions and a one-dimensional gripper position, yielding 8 dimensions per arm. The full action dimension is therefore 3+2+8+8=213+2+8+8=21. As with the custom mobile robot, we predict the base command relative to the current base state, while the neck, arm, and gripper commands are represented as absolute targets. During inference, the model denoises an action chunk of length 16 using DDIM with 16 inference steps. We execute the first 8 actions for the cloth placement task and the full 16-action chunk for the box transfer task. Appendix I Cloth Placement with Active Perception I.1 Demonstration Collection and Rollout Conditions Demos Pink Cloth Blue Cloth Basket Left 33 31 Basket Right 28 33 Rollouts Pink Cloth Blue Cloth Basket Left 6 6 Basket Right 6 7 Table 3: Demonstration and rollout splits across task configurations. Appendix J Box Transfer with Haptic Feedback J.1 Demonstration Collection and Rollout Conditions Demos Target Shelf High 54 Low 48 Rollouts Target Shelf High 10 Low 10 Table 4: Demonstration and evaluation rollout splits across target shelf configurations.