Paper deep dive
Calibrate Once, Fly Any Team: Residual-Grounded Low-Fidelity Training for Cooperative Drone Swarms
Maxim Mednikov, Oren Gal
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/16/2026, 7:35:13 AM
Summary
This paper introduces a mixed-fidelity training scheme for cooperative drone swarms that eliminates high-fidelity (HF) reinforcement learning. A shared, decentralized policy is optimized in a low-fidelity (LF) point-mass simulator, corrected by a per-agent residual ensemble fitted offline using short calibration flights in an HF simulator. This approach scales efficiently with team size (tested 3-18 agents), outperforming uncorrected LF and from-scratch HF baselines while avoiding the high crash rates and computational costs of direct HF training.
Entities (17)
Relation Signals (15)
Calibrate Once, Fly Any Team → uses → pmDrones
confidence 95% · A single shared, decentralized policy is optimized inside a fully-differentiable, JAX-native low-fidelity (LF) point-mass simulator [pmDrones].
Calibrate Once, Fly Any Team → uses → crazyFlow
confidence 95% · The simulator is corrected by a small, per-agent bagged residual ensemble fit once, offline, using short calibration flights in the HF simulator [crazyFlow].
Calibrate Once, Fly Any Team → optimizespolicyvia → Back-Propagation Through Time
confidence 93% · A shared multi-agent policy is optimized entirely inside the LF simulator... via Back-Propagation Through Time (BPTT).
Calibrate Once, Fly Any Team → correctswith → Residual Ensemble
confidence 92% · The simulator is corrected by a small, per-agent bagged residual ensemble fit once, offline.
Oren Gal → affiliatedwith → Swarm & AI Lab
confidence 90% · Oren Gal Affiliation: Swarm & AI Lab (SAIL), University of Haifa, Israel
Maxim Mednikov → affiliatedwith → Swarm & AI Lab
confidence 90% · Maxim Mednikov Affiliation: Swarm & AI Lab (SAIL), University of Haifa, Israel
crazyFlow → implementedin → MuJoCo
confidence 90% · The HF environment is crazyFlow, a MuJoCo/mujoco-mjx-based... simulator.
pmDrones → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Training multi-agent drone-swarm policies directly in high-fidelity (HF) rigid-body physics is accurate but computationally expensive. This cost scales poorly with team size, as each additional agent multiplies contact-resolution complexity and sharply raises the in-simulation crash rate. To address this, we propose a mixed-fidelity training scheme that eliminates HF reinforcement learning entirely. A single shared, decentralized policy is optimized inside a fully-differentiable, JAX-native low-fidelity (LF) point-mass simulator. The simulator is corrected by a small, per-agent bagged residual ensemble fit once, offline, using short calibration flights in the HF simulator. Because calibration requires only one isolated drone, the data collection budget does not compound with team size. Reference trajectories are generated by rolling out an existing LF-only policy and tracked in the HF simulator by a zero-training PD controller. Evaluated across four cooperative drone tasks and team sizes from 3 to 18, the residual-corrected policy outperforms an uncorrected LF baseline in all combinations, and a from-scratch HF policy in 22 of 24 combinations tested. It trails an HF-finetuned policy by a margin that narrows steadily with team size. Ultimately, the proposed method achieves near-equivalent performance at the largest team sizes at a fraction of the computational cost, completely avoiding the high crash rates typical of HF training.
Tags
Links
- Source: https://arxiv.org/abs/2609.17265v1
- Canonical: https://arxiv.org/abs/2609.17265v1
Trouble viewing inline? Open PDF directly →
Full Text
44,286 characters extracted from source content.
Expand or collapse full text
Calibrate Once, Fly Any Team: Residual-Grounded Low-Fidelity Training for Cooperative Drone Swarms Maxim Mednikov Affiliation: Swarm & AI Lab (SAIL), University of Haifa, Israel mmedniko@campus.haifa.ac.il Oren Gal Affiliation: Swarm & AI Lab (SAIL), University of Haifa, Israel orengal@univ.haifa.ac.il Abstract Training multi-agent drone-swarm policies directly in high-fidelity (HF) rigid-body physics is accurate but computationally expensive. This cost scales poorly with team size, as each additional agent multiplies contact-resolution complexity and sharply raises the in-simulation crash rate. To address this, we propose a mixed-fidelity training scheme that eliminates HF reinforcement learning entirely. A single shared, decentralized policy is optimized inside a fully-differentiable, JAX-native low-fidelity (LF) point-mass simulator. The simulator is corrected by a small, per-agent bagged residual ensemble fit once, offline, using short calibration flights in the HF simulator. Because calibration requires only one isolated drone, the data collection budget does not compound with team size. Reference trajectories are generated by rolling out an existing LF-only policy and tracked in the HF simulator by a zero-training PD controller. Evaluated across four cooperative drone tasks and team sizes from 3 to 18, the residual-corrected policy outperforms an uncorrected LF baseline in all combinations, and a from-scratch HF policy in 22 of 24 combinations tested. It trails an HF-finetuned policy by a margin that narrows steadily with team size. Ultimately, the proposed method achieves near-equivalent performance at the largest team sizes at a fraction of the computational cost, completely avoiding the high crash rates typical of HF training. I Introduction Reinforcement learning for robotic control faces a persistent trade-off between simulator speed and simulation fidelity [1]. A low-fidelity (LF) point-mass simulator parallelizes massively and iterates quickly, but its dynamics diverge from a real rigid body: rotational inertia, momentum, and attitude dynamics introduce a delay between a commanded attitude change and the resulting displacement that a point-mass model does not exhibit. A policy trained only on LF physics routinely over-corrects and fails to transfer, an instance of the broader sim-to-real reality gap [2]. Training directly in high-fidelity (HF) rigid-body physics does not scale well to multi-agent settings. Every additional HF agent multiplies the cost of resolving rigid-body contacts, and a joint N-agent rollout must also resolve genuine inter-agent collisions; in our own experiments the in-simulation crash rate climbs sharply as team size grows, purely from running reinforcement learning directly in HF physics, and it previews the cost and risk such training would ultimately incur on physical vehicles. Prior work on multi-robot transfer instead trains inside a deliberately abstract or reduced-fidelity simulator and reintroduces real-world detail only at deployment [3, 4, 5]; we instead keep a single differentiable LF simulator throughout optimization and correct it in-place with a learned per-agent residual, rather than switching representations between training and deployment. This work asks whether the fidelity gap can instead be closed with a small learned correction applied to the cheap simulator, fit once, offline, from a short calibration rollout in the HF simulator. The recipe has three steps. First, a zero-training PD controller flies the HF simulator along a reference trajectory that itself comes from cheaply rolling out an existing LF-only policy; any adequate trajectory follower would serve here, and nothing downstream depends on the choice. Second, a small per-agent residual acceleration model (a bootstrap-bagged ensemble of MLPs) is fit once on that calibration data and then frozen. Third, a shared multi-agent policy is optimized entirely inside the LF simulator, corrected by the frozen residual at every step, via Back-Propagation Through Time (BPTT) [6, 7, 8]; this is the only stage that dominates total training cost, and it never runs HF physics at all. Because the residual’s features are strictly per-agent, with no information about team size or teammates, its calibration data is gathered from one isolated drone at a time. For each team size we regenerate the reference trajectories and re-fit the residual and shared policy from that single-drone data. No stage of the pipeline ever flies two drones together, so calibration carries essentially no crash risk. These collection flights can also run in parallel across separated drones that they never share airspace, cutting wall-clock collection time further. In summary, this paper contributes: (i) a training pipeline that grounds a cheap, differentiable LF simulator in HF rigid-body dynamics with a per-agent residual, fit once offline from a zero-training controller’s calibration rollouts and then frozen, removing HF reinforcement learning from multi-agent policy training entirely; (i) a strictly per-agent residual paired with a fully-shared policy, so grounding a team of any size needs only round-robin single-drone calibration flights, never a joint multi-agent rollout, and policy-training cost stays flat in team size; and (i) an evaluation across four cooperative tasks and every N∈3,5,7,9,12,18N∈\3,5,7,9,12,18\ in which the resulting policy beats an uncorrected LF baseline in all 24 combinations and a from-scratch HF baseline in 22 of 24, with its lead over the latter widening with team size, and closes the gap to an HF fine-tune, the strongest baseline, steadily as teams grow, to under 1%1\% at the largest sizes, all at a small fraction of the HF flight time and training crashes both HF-trained baselines incur. All results are in simulation; physical-drone validation is future work. I Related Work I-A Sim-to-real transfer. Conventional approaches to closing the reality gap train entirely in simulation via domain randomization over dynamics and appearance parameters [9, 10, 11, 12], or by correcting the simulator’s internal parameters to match observed real-world behavior [13, 2]. These methods typically target a single, fixed-fidelity simulator. In contrast, we address the fidelity gap between two simulators of different computational costs using a small learned residual, rather than tuning the LF simulator’s native parameters. I-B Differentiable simulation and BPTT. Policy optimization via Backpropagation-Through-Time (BPTT) in a differentiable simulator avoids the high variance and sample-inefficiency of stochastic policy-gradient methods, provided the environment exposes gradients end-to-end [6, 7]. For a T-step rollout with at=πθ(st)a_t= _θ(s_t), st+1=f(st,at)s_t+1=f(s_t,a_t), s0∼ρ0s_0 _0, and J(θ)=∑t=0T−1r(st,at)J(θ)= _t=0^T-1r(s_t,a_t), a differentiable step f yields the policy gradient in closed form: ∇θJ(θ)=∑t=0T−1(∑i=0t−1∂rt∂st(∏j=i+1t−1∂sj+1∂sj)∂si+1∂θ+∂rt∂at∂at∂θ), _θJ(θ)= _t=0^T-1 ( _i=0^t-1 ∂ r_t∂ s_t ( _j=i+1^t-1 ∂ s_j+1∂ s_j ) ∂ s_i+1∂θ+ ∂ r_t∂ a_t ∂ a_t∂θ ), (1) where ∂si+1/∂θ∂ s_i+1/∂θ and ∂at/∂θ∂ a_t/∂θ are the policy’s direct parameter sensitivities and the product term is the state-transition Jacobian chain carrying them forward to step t. BPTT evaluates this sum by reverse-mode automatic differentiation, without sampled return estimates, before a standard gradient update (e.g., Adam) θk+1=θk+α∇θJ(θk) _k+1= _k+α _θJ( _k). This approach has been applied to single-quadrotor control [14] and to swarm navigation in a differentiable point-mass simulator [8]. We build on this foundation but add a frozen, offline-fit dynamics correction for cooperative multi-agent tasks, rather than deploying the uncorrected policy. For the swarm architecture we use full parameter sharing [15] rather than decentralized communication [16], graph networks [17], or per-agent low-rank adaptation [18]: our formulation is homogeneous (every agent’s residual and policy inputs are structurally identical), so one shared policy suffices and preserves the scalability of single-drone calibration (Section I-C). I-C Residual dynamics correction and online adaptation. Learning a residual correction over an imperfect nominal model, rather than replacing it, is an established approach in robot learning [19], aligned with model-based RL methods that combine learned dynamics with analytical models via uncertainty-aware ensembles for sample-efficient planning [20]. Such learned models often suffer from compounding errors over long rollouts [21], a failure mode analogous to the closed-loop instability addressed in our residual design (Section I-A2). Recent quadrotor work addresses the sim-to-real gap with online, frequently updated residual models [22, 23]. learning_on_the_fly [24] fits an online residual ensemble for a single quadrotor and is the architectural basis for our residual-acceleration formulation. Whereas some swarm work learns residual interaction forces between members such as downwash [25], our residual stays strictly per-agent, using no information about teammates or team size (Section I-C). Unlike continuous online refitting against the active policy, we fit a small bagged ensemble [24] once offline and freeze it, trading an online loop’s tighter distribution matching for a pipeline that runs the HF simulator exactly once per team size. Concurrent work integrates residual models with BPTT for real-world quadrotor adaptation [26] or conditions policy and residual on online-inferred latent dynamics [27]; ours is calibrated entirely in HF simulation, not on hardware, and uses no multi-agent features. Closest is Sym2Real [28], which builds a base model in a cheap LF simulator by symbolic regression and adapts it with a learned residual from roughly ten real trajectories. It targets single-agent LF-to-real transfer; we extend the paradigm to cooperative swarms, grounding teams of any scale with a per-agent residual calibrated one drone at a time, never a joint multi-drone HF flight. I-D PD Velocity Control of Quadrotors For a quadrotor with fixed yaw, any twice-differentiable position trajectory determines the required thrust magnitude and attitude in closed form [29]. We use this property with a standard PD controller to track trajectories and collect calibration data in the HF simulator. Given a current state (,)(p,v) and a target (∗,∗,∗)(p^*,v^*,a^*), the controller computes a desired thrust vector incorporating feedforward acceleration and gravity compensation: =∗+kp(∗−)+kd(∗−)+[0,0,g]⊤.T=a^*+k_p(p^*-p)+k_d(v^*-v)+[0,0,g] . (2) This thrust vector is then mapped directly into the vehicle’s body frame via inverse dynamics to yield the roll, pitch, and thrust commands: ϕcmd=−arcsin(Ty∥),θcmd=atan2(Tx,Tz),Tcmd=m∥. _cmd=- \! ( T_y ), _cmd=atan2(T_x,T_z), T_cmd=m . (3) This deterministic, zero-training controller is used only to fly the calibration data (Section I-B); any sufficiently accurate trajectory follower would serve without altering the downstream pipeline. Fig. 1: Overview of the training pipeline (Section I-B). Stage 1 (left): an existing LF-only policy is rolled out in the LF simulator to generate a reference trajectory, which is tracked in the HF simulator by the PD controller (Section I-D); the residual ensemble is then fit, via an L2 regression loss, against the gap between that HF flight and the LF simulator’s own (simplified) dynamics equations. Stage 2 (right): the fitted residual is frozen and composed with the LF simulator into a corrected dynamics model, inside which a freshly-initialized policy is trained end-to-end via BPTT, with no further HF-simulation contact. I Methodology The proposed pipeline (Figure 1) consists of a shared policy optimized entirely within LF physics, dynamically corrected at every integration step by a per-agent residual. This section details the network architectures and the four-stage training pipeline. I-A Policy and Residual Both networks are multi-layer perceptrons with two hidden layers of width 128128. The policy has a tanh output layer; the residual is a five-member bootstrap-bagged ensemble whose per-member tanh output bounds the correction to ±35m/s2± 35\,m/s^2. I-A1 Policy The shared policy is applied independently to each agent’s observation vector. It is optimized directly in LF physics via backpropagation-through-time (Section I-B): the full trajectory rollout is differentiated end-to-end with respect to the policy weights, requiring neither a critic nor an advantage estimator. I-A2 Residual Each member is independently initialized and trained on its own bootstrap resample of the calibration transitions; at deployment the member outputs are plain-averaged, with no disagreement gating or uncertainty weighting. From a 12-dimensional per-agent feature vector (position p, velocity v, attitude (ϕ,θ,ψ)(φ,θ,ψ), and nominal point-mass acceleration a), the ensemble predicts an additive corrective acceleration Δ=[Δax,Δay,Δaz] =[ a_x, a_y, a_z]. Because this correction depends only on an agent’s own state and action, calibration data can come from a single isolated drone (Section I-C). Two choices keep the residual well-behaved on out-of-distribution states. First, it is fit once, offline, and frozen, removing the moving-target instability of online adaptation. Second, its output is bounded: because the correction enters the LF integration additively, an unbounded residual evaluated off-distribution can act as an unstable linear map that diverges to floating-point overflow. A saturating tanh output structurally prevents this, with no need for artificial episode resets or severed gradient paths. Finally, in tasks with extended station-keeping, near-hover states are under-represented in the dynamic calibration flights (Section IV); a per-sample loss reweighting upweights low-velocity transitions during the fit to compensate. I-B Training Pipeline The residual is fit and the policy trained in four decoupled stages, once per scenario and team size (Figure 1): 1. Generate reference trajectories. An existing, uncorrected LF-only policy (the baseline of Section IV) is rolled out in the LF simulator. Positions and velocities are logged every step, and accelerations reconstructed by exact finite-differencing of velocity to match the LF simulator’s native integration. 2. Fly the reference in the HF simulator. Each trajectory in the pool drives an isolated drone rather than a joint N-agent rollout: trajectory i exercises role imodNi N in a deterministic round-robin schedule, covering all roles without multi-drone flight. The PD controller (Section I-D) tracks it in the HF simulator, the only stage that runs HF physics, logging per-step state, action, and next state. With a fixed number of trajectories per role, the calibration budget scales linearly with N (Section V-B analyzes subsampling it). 3. Fit the residual. For each transition, the regression target is the acceleration offset that, added to the nominal LF step, reproduces the observed HF next state exactly. The ensemble is fit to these targets once, offline, by full-batch supervised regression (Section I-A2), using no reward signal or policy gradient, and frozen thereafter. 4. Train the policy. A freshly initialized shared policy is optimized by BPTT entirely in the LF simulator. The frozen correction is added to each agent’s acceleration every step as a stop-gradient constant, so the BPTT gradient flows only through the nominal point-mass dynamics, never the residual network’s input Jacobian. An ensemble trained on a narrow calibration manifold has arbitrary off-distribution Jacobians, so backpropagating through it would hand the policy spurious gradient directions and destabilize optimization; severing that path retains the forward physical correction without injecting gradient noise. No HF simulation occurs here. The resulting policy is then deployed in the HF simulator for evaluation (Section IV). Stage 1 uses an existing LF-only policy rather than an idealized geometric target: an LF policy commands more aggressive maneuvers than the vehicle can realize, so replaying its actions open-loop would exceed the HF flight envelope. Tracking its achieved path in closed loop keeps the calibration data dynamically feasible, without HF reinforcement learning. I-C Single-Drone Calibration at Any Team Size The pipeline scales by eliminating multi-drone HF flights entirely. Because the residual uses only per-agent features, HF calibration data comes from one isolated drone at a time (Stage 2), and simulating N LF agents reduces to N independent point-mass updates under the residual, so policy-optimization cost is largely invariant to team size. We calibrate the residual and train the policy separately for each N∈3,5,7,9,12,18N∈\3,5,7,9,12,18\. The only cost that scales with N is single-drone calibration collection, driven by the need to maintain uniform role coverage in the round-robin trajectory pool, not by the sample complexity of multi-agent interaction. Every tested swarm size is thus grounded and trained without a single multi-agent HF flight. IV Experiments IV-A Environments Both simulation fidelities share a [−1,1]3[-1,1]^3 action space, a dt=0.02dt=0.02 s RL timestep, and a 0.150.15 m agent collision radius. IV-A1 Low-Fidelity Model (pmDrones) The LF environment is pmDrones, a JAX-native [30], fully JIT/vmap-parallelized point-mass simulator. Commanded roll and pitch track the target angles via a first-order lag at every simulation step. Thrust tracks its command instantaneously, is remapped to a non-negative range, and is converted to acceleration via standard attitude-to-thrust-vector decomposition on the post-lag angles (with yaw locked at ψ=0ψ=0): ax a_x =Tmsinθcosϕ, = Tm θ φ, (4) ay a_y =−Tmsinϕ, =- Tm φ, az a_z =Tmcosθcosϕ−g = Tm θ φ-g The learned residual correction Δ (Section I-A2) is added to a before integration, such that Δ= =0 recovers the nominal pmDrones physics. Unlike high-fidelity multirotors, this nominal point-mass model completely omits body rotational inertia, rotor gyroscopic effects, and velocity-dependent aerodynamic drag. Consequently, the residual ensemble must independently capture these structural discrepancy terms to prevent simulated trajectories from diverging upon transfer to the full rigid-body evaluation platform. IV-A2 High-Fidelity Model (crazyFlow) The HF environment is crazyFlow [31], a MuJoCo/mujoco-mjx-based, GPU-accelerated, differentiable rigid-body simulator driven by the same [ϕcmd,θcmd,Tcmd][ _cmd, _cmd,T_cmd] convention. It models aerodynamic effects such as rotor drag that the point-mass model omits, and is system-identified against physical Crazyflie hardware rather than hand-tuned, reportedly reaching sub-centimeter tracking without domain randomization; this makes it an accurate proxy for real-world deployment. We treat it strictly as ground truth: it is never used inside the policy-optimization loop, only to generate calibration data (Section I-B) and evaluate deployed policies, so all HF results here are in simulation, with hardware deployment left to future work. IV-B Tasks Fig. 2: Top-view flight trajectories for each of the four tasks, one representative episode at N=5N=5 (older positions faded). Circular markers mark each agent’s start; stars mark its end. Fig. 3: Team-size scaling across all four tasks (N∈3,5,7,9,12,18N∈\3,5,7,9,12,18\) for the four configurations of Section IV-C (Hybrid is ours). Top: HF-simulation training time; middle: HF-simulation training crash events; bottom: deployed evaluation reward (mean ± std). We evaluate the pipeline on four cooperative drone tasks (Figure 2). These tasks are implemented identically across both pmDrones and crazyFlow, differing only in physics fidelity, with all agents sharing a single policy. Every task also adds a per-collision penalty to its reward: • Spread: Each agent is assigned a unique landmark on a randomly rotated formation circle per episode. Agents are rewarded for proximity to their assigned landmarks and for achieving full formation coverage as a team. This task uses the specialized near-hover loss reweighting during the residual fit (Section I-A2). • Circle: Agents must match a target tangential velocity while maintaining a fixed-radius orbit at a specified altitude. • Figure8: The entire swarm tracks a single moving target along a lemniscate trajectory. This shared objective encourages collective movement, while the collision penalty prevents spatial overlap. • Pitchandexpand: Agents track a spherical shell centered on a static origin, with a radius that oscillates sinusoidally over time. The reward function encourages both radial proximity to the shell and matching the shell’s purely radial velocity (via a bounded exp(−∥−ideal∥)−1 (- -v_ideal )-1 term) to penalize drifting across the surface. IV-C Comparisons We evaluate four training and deployment configurations on crazyFlow (HF) rollouts: • Low-Fidelity Baseline: Trained entirely in LF physics without a residual or HF involvement, isolating the contribution of the residual correction. • High-Fidelity Scratch: Trained and evaluated entirely in HF physics from random initialization, serving as an accurate but computationally expensive baseline. • High-Fidelity HotStart: The Low-Fidelity Baseline fine-tuned briefly within crazyFlow, providing a stronger HF reference. • Hybrid (ours): The proposed pipeline (Section I-B): trained in LF physics under a frozen, offline-fit residual ensemble, entirely avoiding HF reinforcement learning. All configurations share the Hybrid’s BPTT optimizer and policy architecture. Both HF baselines run in a single crazyFlow world, so their reported HF costs reflect one physical drone team, and both use a 100-epoch early-stopping patience. High-Fidelity Scratch gets a linear-in-N epoch budget large enough to converge (≈ 1.9k epochs at N=18N=18). High-Fidelity HotStart, fine-tuning the converged LF policy at a reduced learning rate, needs far fewer: across all 24 combinations and 5 seeds it early-stops well under its 500-epoch cap (115–335 epochs). To confirm the budget is not the limiting factor: despite 33–10×10× more epochs (median ≈6×≈ 6×), Scratch never exceeds HotStart’s training reward in any of the 24 cells (e.g. spread, N=12N=12: HotStart plateaus by epoch 185 above what a 1400-epoch Scratch run reaches). V Results We evaluate sim-to-sim transfer, deploying each policy from pmDrones (LF) onto crazyFlow (HF). To mirror real deployment, evaluation uses a single crazyFlow world, not a batched ensemble. For every (scenario, team size) we report deployed episode reward and, for the configurations that train in HF, their HF flight-time and crash cost. A crash is registered per agent at the onset of contact, either inter-agent (center-to-center distance below twice the collision radius) or with the floor, so that a sustained contact counts once and k agents colliding at the same step count as k. Floor contact in the first 0.50.5 s after a reset is ignored, since every scenario spawns at ground level and liftoff takes ≈0.3≈ 0.3 s. HF flight time is the simulated flight duration multiplied by the team size N (fleet-seconds), the airtime that would accrue across N physical drones flying together. Each policy is evaluated over 3232 episodes with independent initial conditions, and every training configuration is trained for 55 random seeds. V-A Deployed Reward Across Team Sizes Figure 3 evaluates all four configurations across the 24 conditions formed by four scenarios and six team sizes (N∈3,5,7,9,12,18N∈\3,5,7,9,12,18\). Each setting is calibrated from a pool of 60(N+1)60(N+1) single-drone reference trajectories of 1010 s duration, scaling from 240240 trajectories at N=3N=3 to 11401140 at N=18N=18. These reference flights are distributed evenly across the N team roles via deterministic round-robin assignment (Section I-B); Section V-B analyzes the minimal calibration budget required for convergence. Hybrid outperforms the uncorrected Low-Fidelity Baseline in all 24 conditions, showing that a residual fit purely to single-drone PD tracking data substantially bridges the sim-to-sim gap. It also exceeds High-Fidelity Scratch in 22 of 24 settings: the from-scratch HF policy wins only at N=3N=3 on circle and figure8; at every larger N≥5N≥ 5 Hybrid attains higher rewards, often by wide margins (Fig. 3, bottom row). Against High-Fidelity HotStart, the strongest baseline, and the only one trained directly on the target dynamics, Hybrid trails at small team sizes, but the margin narrows steadily with scale, from ≈22%≈ 22\% at N=3N=3 (mean over tasks) to under 1%1\% at N≥12N≥12, where Hybrid matches HotStart on spread and exceeds it on pitchandexpand. Team size is the governing factor: both HF-trained baselines degrade sharply toward N=18N=18, while Hybrid degrades only mildly. This performance divergence tracks training-time collision dynamics. Running BPTT directly in HF physics forces the policy through inter-agent contacts, and crash frequency grows with swarm size (Fig. 3, middle row): at N=18N=18 Scratch incurs 66–10×10410× 10^4 crash events per run, a 1616–63%63\% crash-step rate that biases optimization toward sub-optimal minima (most visibly on pitchandexpand at N≥9N≥ 9 where agents learn overly conservative collision-avoidance reflexes), and HotStart still shows 22–15%15\%. Because Hybrid optimizes entirely in LF physics, it is structurally immune to these contact artifacts, incurring zero training crashes at every team size. HF exposure shows the same disparity (Fig. 3, top row). Hybrid’s HF footprint is only Stage-2 calibration: 2020 fleet-minutes at N=3N=3, 190190 at N=18N=18, flown by a deterministic controller on isolated vehicles rather than exploratory policies. HotStart needs 7070–11001100 fleet-minutes and Scratch 300300–59005900. While these baseline costs grow rapidly with N, Hybrid’s calibration budget stays bounded and user-defined, and Section V-B reduces it further. V-B Calibration-Data Efficiency Fig. 4: Deployed reward vs. total HF calibration budget. One panel per team size (N∈3,7,12N∈\3,7,12\), one line per scenario; band is ±1± 1 std over training seeds. y: deployed reward relative to that (scenario, N)’s own full-calibration-pool Hybrid (Section IV-C) (%; 0=0= matched, negative == worse). x: the total number of single-drone PD-controller calibration trajectories used to fit the residual ensemble, not per agent, and the same budget regardless of team size (log scale; x=0x=0 is the uncorrected LF policy). The evaluations in Section IV-C applied a fixed per-role trajectory budget. To evaluate calibration data requirements, we vary this budget from 00 (no residual correction) to 240240 trajectories, retraining the residual ensemble and Hybrid policy across five seeds (Figure 4). This analysis requires no additional simulations, as each condition subsamples the trajectory pool collected in Section I-B. Deployed reward increases rapidly with initial calibration data before saturating. Without residual correction (x=0x=0), the uncorrected LF policy trails the full-data Hybrid by 1818–164%164\%. This gap expands with team size, reaching shortfalls of −161%-161\% in figure8 and −164%-164\% in circle at N=12N=12. Calibration sensitivity differs across tasks: • Circle converges fastest, reaching within roughly ten percent of the full-data fit by about 88 calibration trajectories at every team size (a single trajectory already suffices at N=3N=3). • Spread and Pitchandexpand plateau by approximately 3232 trajectories. • Figure8 requires the largest dataset. Below 3232 trajectories at N≥7N≥ 7, cross-seed spread spans 3030–110110 percentage points, narrowing once the budget reaches 3232–6464 trajectories. Across all four tasks and tested team sizes (N∈3,7,12N∈\3,7,12\), approximately 3232 trajectories, about five fleet-minutes of single-drone flight, recover deployed performance to within roughly 1010–15%15\% of the full-pool baseline, with 6464 trajectories further reducing variance. This saturation threshold remains independent of team size. While larger swarms exhibit higher variance under limited data due to covering broader regions of the state space, the total single-drone calibration volume required for convergence does not compound with N. V-C Out-of-Distribution Deployment Fig. 5: Deployment on an off-spec HF drone. Rows: the three disturbances (heavier drone, excess drag, steady wind); columns: scenarios; x: team size. Series: uncorrected LF policy deployed directly, zero-training PD controller tracking the LF-policy reference, and Hybrid after a small warm-started recalibration on the disturbed drone; dotted grey is Hybrid’s clean-conditions reward. Band is ±1± 1 std over 55 training seeds. The pipeline so far assumes calibration and deployment occur on an identical nominal model. To test robustness under mismatch, we examine whether the residual-grounding stage can re-adapt the policy to an off-spec platform. We use circle and figure8: both keep agents at non-zero velocity throughout, so velocity-coupled and persistent-force disturbances are fully engaged (unlike in station-keeping tasks), and they span distinct coordination structures: a shared target vs. distributed formation control. We evaluate three disturbances applied across both calibration and evaluation within crazyFlow at team sizes N∈3,7,12N∈\3,7,12\: • Heavier drone (+20%+20\% mass): Rescales the required thrust profile. • Excess aerodynamic drag (c=2.0c=2.0 s-1): Introduces an unmodeled velocity-coupled drag force, =−cmf=-c\,m\,v. • Steady wind (1.51.5 m/s2 bias): Applies a persistent directional acceleration to each agent. The uncorrected LF baseline and the PD controller are deployed unmodified. For Hybrid we run a brief warm-started recalibration from its clean checkpoints: 20 PD calibration trajectories on the disturbed model, 150 epochs of residual refit, and 400 BPTT epochs of policy fine-tuning under the re-fit residual. All configurations are then evaluated on the disturbed HF system. On figure8, the recalibrated Hybrid shows strong resilience, maintaining returns between −81-81 and −102-102 across all disturbances and swarm sizes, closely matching its clean baseline of approximately −82-82. In contrast, the uncorrected policy drops to between −120-120 and −292-292. On circle, adaptation provides substantial recovery, though performance depends more on the underlying disturbance: • Under the heavier mass profile, the uncorrected baseline drops from −147-147 to −398-398 as team size scales. The recalibrated Hybrid maintains returns between −66-66 and −180-180, tracking its clean baseline range of −63-63 to −167-167. • Under steady wind, the uncorrected policy degrades to −204-204 through −489-489, whereas Hybrid stabilizes between −73-73 and −174-174. • Under excess drag, performance degradation is less severe overall; Hybrid’s margin over the PD tracking baseline narrows, but it reliably outperforms the uncorrected policy. Although the PD controller tracks the agile figure8 reference loosely (trailing even the uncorrected policy), this does not undermine calibration: the residual is fit from observed transition pairs, not tracking precision. Overall, a few minutes of single-drone recalibration lets the frozen residual compensate for substantial platform mismatch, and because that recalibration uses only local tracking transitions, its cost stays decoupled from swarm size. V-D Residual Model Class Task (N) Affine SINDy MLP ens. Spread (N=3N=3) −32.9±6.0-32.9±6.0 −28.1±0.4-28.1±0.4 −27.7±2.7-27.7 2.7 Spread (N=12N=12) −88.3±11.7-88.3±11.7 −77.1±0.5-77.1 0.5 −77.5±3.7-77.5±3.7 Circle (N=3N=3) −289.6±220.1-289.6±220.1 −48.0±1.8-48.0 1.8 −61.5±4.1-61.5±4.1 Circle (N=12N=12) −276.7±165.6-276.7±165.6 −287.3±63.2-287.3±63.2 −168.3±8.1-168.3 8.1 Figure8 (N=3N=3) −91.0±2.8-91.0±2.8 −76.9±2.3-76.9 2.3 −85.2±4.5-85.2±4.5 Figure8 (N=12N=12) −85.6±1.0-85.6±1.0 −117.5±20.3-117.5±20.3 −81.1±0.7-81.1 0.7 Pitchandexpand (N=3N=3) −255.4±247.2-255.4±247.2 −90.8±3.8-90.8 3.8 −106.0±12.1-106.0±12.1 Pitchandexpand (N=12N=12) −173.3±15.5-173.3±15.5 −159.9±9.7-159.9±9.7 −151.9±7.2-151.9 7.2 TABLE I: Deployed reward (mean ± std over five seeds) across residual classes. Bold indicates best mean per row. To assess sensitivity to the residual’s parameterization, we evaluate three frozen model classes under an identical pipeline: (i) an affine correction, (i) a SINDy model [32] fit by sequentially-thresholded least squares over a physically grounded library (thrust, attitude projections, linear velocity), and (i) the bootstrap-bagged MLP ensemble. All share the same bounded output; the affine and MLP variants are five-member bagged ensembles averaged uniformly, while the SINDy model is a single sparse fit. At N=3N=3, performance is comparable across classes, with SINDy achieving the highest mean return. However, stability diverges: the affine model collapses on circle and pitchandexpand at N=3N=3 (std>200std>200), while SINDy degrades at N=12N=12 on circle and figure8. The MLP ensemble is the only class that avoids divergence across all settings, making it our default; the pipeline itself remains model-agnostic. That the simpler classes stay competitive at all partly reflects crazyFlow’s deterministic, near-affine gap; a stochastic or more strongly nonlinear gap would widen the margin in favor of the expressive ensemble. VI Conclusion We bridge the sim-to-sim fidelity gap for cooperative drone swarms using an offline-fit, frozen residual acceleration model. A bootstrap-bagged MLP ensemble fit to short, single-drone PD tracking flights allows the cooperative policy to be optimized entirely within nominal point-mass physics, circumventing multi-agent HF reinforcement learning. Because the dynamics correction operates strictly per-agent, swarms across all tested team sizes are grounded and trained without requiring a single multi-drone flight. Across four cooperative tasks and team sizes N∈3,5,7,9,12,18N∈\3,5,7,9,12,18\, the proposed hybrid pipeline outperforms an uncorrected LF baseline in all 24 configurations and exceeds a from-scratch HF baseline in 22 of 24 (trailing only at N=3N=3). Its performance gap relative to an HF fine-tuning baseline narrows steadily from ≈22%≈ 22\% at N=3N=3 to under 1%1\% at N≥12N≥12, while consuming a fraction of the HF simulation time. Furthermore, while multi-agent contact dynamics cause training crash-step rates in HF baselines to exceed 60%60\% at N=18N=18, our LF optimization avoids physical collision instabilities entirely, maintaining zero training crashes. The total HF requirement is restricted to an initial calibration phase that can be compressed to roughly 3232 single-drone trajectories (about five fleet-minutes) without substantial performance loss. Several directions remain. All evaluations are in simulation; although crazyFlow is system-identified against physical hardware, real-world validation is an essential next step. crazyFlow is also deterministic, without sensor or actuation noise; evaluating the framework under stochastic dynamics and coupled, multi-axis disturbances warrants further study. Still, the methodology is agnostic to aerial robotics, needing only an analytical nominal model, a high-fidelity reference, and a basic tracking controller, and this per-agent residual paradigm is a scalable foundation for other multi-agent systems, including ground rovers and autonomous surface vessels. References [1] J. Collins, S. Chand, A. Vanderkop, and D. Howard (2021) A Review of Physics Simulators for Robotic Applications. IEEE Access 9, p. 51416–51431. External Links: ISSN 2169-3536, Document Cited by: §I. [2] E. Aljalbout, J. Xing, A. Romero, I. Akinola, C. R. Garrett, E. Heiden, A. Gupta, T. Hermans, Y. Narang, D. Fox, D. Scaramuzza, and F. Ramos (2025) The Reality Gap in Robotics: Challenges, Solutions, and Best Practices. arXiv. External Links: 2510.20808, Document Cited by: §I, §I-A. [3] A. Labiosa and J. P. Hanna (2025) Multi-Robot Collaboration through Reinforcement Learning and Abstract Simulation. arXiv. External Links: 2503.05092, Document Cited by: §I. [4] E. Noorani, Z. Serlin, B. Price, and A. Velasquez (2025) From Abstraction to Reality: DARPA’s Vision for Robust Sim-to-Real Autonomy. arXiv. External Links: 2503.11007, Document Cited by: §I. [5] C. V. Samak, T. V. Samak, and V. N. Krovi (2025) Mixed-Reality Digital Twins: Leveraging the Physical and Virtual Worlds for Hybrid Sim2Real Transition of Multi-Agent Reinforcement Learning Policies. IEEE Robotics and Automation Letters 10 (9), p. 9040–9047. External Links: 2403.10996, ISSN 2377-3766, 2377-3774, Document Cited by: §I. [6] M. A. Z. Mora, M. Peychev, S. Ha, M. Vechev, and S. Coros (2021) PODS: Policy Optimization via Differentiable Simulation. In Proceedings of the 38th International Conference on Machine Learning, p. 7805–7817. External Links: ISSN 2640-3498 Cited by: §I, §I-B. [7] J. Xu, V. Makoviychuk, Y. Narang, F. Ramos, W. Matusik, A. Garg, and M. Macklin (2022) Accelerated Policy Learning with Parallel Differentiable Simulation. Cited by: §I, §I-B. [8] Y. Zhang, Y. Hu, Y. Song, D. Zhou, and W. Lin (2024) Back to Newton’s Laws: Learning Vision-based Agile Flight via Differentiable Physics. arXiv. External Links: 2407.10648, Document Cited by: §I, §I-B. [9] OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba (2019) Learning Dexterous In-Hand Manipulation. arXiv. External Links: 1808.00177, Document Cited by: §I-A. [10] J. Tan, T. Zhang, E. Coumans, A. Iscen, Y. Bai, D. Hafner, S. Bohez, and V. Vanhoucke (2018) Sim-to-Real: Learning Agile Locomotion For Quadruped Robots. arXiv. External Links: 1804.10332, Document Cited by: §I-A. [11] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel (2018) Sim-to-Real Transfer of Robotic Control with Dynamics Randomization. In 2018 IEEE International Conference on Robotics and Automation (ICRA), p. 3803–3810. External Links: 1710.06537, Document Cited by: §I-A. [12] J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter (2019) Learning agile and dynamic motor skills for legged robots. Science Robotics 4 (26), p. eaau5872. External Links: 1901.08652, ISSN 2470-9476, Document Cited by: §I-A. [13] P. Huang, X. Zhang, Z. Cao, S. Liu, M. Xu, W. Ding, J. Francis, B. Chen, and D. Zhao (2023) What Went Wrong? Closing the Sim-to-Real Gap via Differentiable Causal Discovery. arXiv. External Links: 2306.15864, Document Cited by: §I-A. [14] J. Heeg, Y. Song, and D. Scaramuzza (2025) Learning Quadrotor Control From Visual Features Using Differentiable Simulation. In 2025 IEEE International Conference on Robotics and Automation (ICRA), p. 4033–4039. External Links: 2410.15979, Document Cited by: §I-B. [15] J. K. Terry, N. Grammel, S. Son, B. Black, and A. Agrawal (2023) Revisiting Parameter Sharing in Multi-Agent Deep Reinforcement Learning. arXiv. External Links: 2005.13625, Document Cited by: §I-B. [16] S. Batra, Z. Huang, A. Petrenko, T. Kumar, A. Molchanov, and G. S. Sukhatme (2021) Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning. arXiv. External Links: 2109.07735, Document Cited by: §I-B. [17] E. Tolstaya, F. Gama, J. Paulos, G. Pappas, V. Kumar, and A. Ribeiro (2017) Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks. Foundations and Trends® in Machine Learning 10 (1-2), p. 1–141. External Links: 1903.10527, ISSN 1935-8237, 1935-8245, Document Cited by: §I-B. [18] B. Zhang, A. Kapoor, and M. Sun (2025) Low-Rank Agent-Specific Adaptation (LoRASA) for Multi-Agent Policy Learning. arXiv. External Links: 2502.05573, Document Cited by: §I-B. [19] T. Silver, K. Allen, J. Tenenbaum, and L. Kaelbling (2019) Residual Policy Learning. arXiv. External Links: 1812.06298, Document Cited by: §I-C. [20] K. Chua, R. Calandra, R. McAllister, and S. Levine (2018) Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models. arXiv. External Links: 1805.12114, Document Cited by: §I-C. [21] N. Lambert, K. Pister, and R. Calandra (2022) Investigating Compounding Prediction Errors in Learned Dynamics Models. arXiv. External Links: 2203.09637, Document Cited by: §I-C. [22] K. Huang, R. Rana, A. Spitzer, G. Shi, and B. Boots (2023) DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control. arXiv. External Links: 2310.09053, Document Cited by: §I-C. [23] M. O’Connell, G. Shi, X. Shi, K. Azizzadenesheli, A. Anandkumar, Y. Yue, and S. Chung (2022) Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds. Science Robotics 7 (66), p. eabm6597. External Links: 2205.06908, ISSN 2470-9476, Document Cited by: §I-C. [24] J. Pan, J. Xing, R. Reiter, Y. Zhai, E. Aljalbout, and D. Scaramuzza (2026) Learning on the Fly: Rapid Policy Adaptation via Differentiable Simulation. arXiv. External Links: 2508.21065, Document Cited by: §I-C, §I-C. [25] G. Shi, W. Hönig, X. Shi, Y. Yue, and S. Chung (2021) Neural-Swarm2: Planning and Control of Heterogeneous Multirotor Swarms using Learned Interactions. arXiv. External Links: 2012.05457, Document Cited by: §I-C. [26] Y. Ren, Z. Zhu, J. Xing, and D. Scaramuzza (2026) Learning Agile Quadrotor Flight in the Real World. arXiv. External Links: 2602.10111, Document Cited by: §I-C. [27] J. Xing, Z. Zhu, Y. Ren, I. Geles, Y. Zhai, R. Reiter, and D. Scaramuzza (2026) Continual Robot Policy Learning via Variational Neural Dynamics. arXiv. External Links: 2606.27353, Document Cited by: §I-C. [28] E. Lee, S. A. Moore, and B. Chen (2026) Sym2Real: Symbolic Dynamics with Residual Learning for Data-Efficient Adaptive Control. arXiv. External Links: 2509.15412, Document Cited by: §I-C. [29] D. Mellinger and V. Kumar (2011) Minimum snap trajectory generation and control for quadrotors. In 2011 IEEE International Conference on Robotics and Automation, p. 2520–2525. External Links: ISSN 1050-4729, Document Cited by: §I-D. [30] (2026) Jax-ml/jax. Note: jax-ml Cited by: §IV-A1. [31] M. Schuck, M. P. Rath, Y. Hua, A. Goudar, S. Zhou, and A. P. Schoellig (2026) Crazyflow: An Accurate, GPU-Accelerated, Differentiable Drone Simulator in JAX. arXiv. External Links: 2606.01478, Document Cited by: §IV-A2. [32] S. L. Brunton, J. L. Proctor, and J. N. Kutz (2016) Discovering governing equations from data: Sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences 113 (15), p. 3932–3937. External Links: 1509.03580, ISSN 0027-8424, 1091-6490, Document Cited by: §V-D.