Paper deep dive
Deep-Unfolded Coordination
Hunter Kuperman, Minchan Jung, Rahul V. Ghosh, Alex Oshin, Evangelos A. Theodorou
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 99%
Last extracted: 6/20/2026, 5:00:50 AM
Summary
The paper introduces 'Deep Coordinator', a deep-unfolding framework designed to accelerate multi-agent robotics coordination by dynamically adjusting the hyperparameters of the ADMM-DDP distributed solver. Unlike traditional methods that use fixed or supervised learning approaches, Deep Coordinator uses an unsupervised learning scheme and the Implicit Function Theorem (IFT) to adapt penalty parameters at solve-time in response to optimizer performance. This approach allows the system to handle non-convex optimization problems more effectively. Experimental results on car and quadrotor fleets demonstrate that Deep Coordinator achieves comparable solution quality while being 6.18-9.44x faster than conventional solvers and maintaining performance when scaled to systems up to 8x larger than the training size.
Entities (6)
Relation Signals (4)
Deep Coordinator → adaptshyperparametersof → ADMM-DDP
confidence 100% · Deep Coordinator, a deep-unfolding framework that learns to dynamically adjust the hyperparameters of ADMM-DDP
Deep Coordinator → uses → Deep-Unfolding
confidence 100% · we propose Deep Coordinator, a deep-unfolding framework
Deep Coordinator → utilizes → Implicit Function Theorem
confidence 100% · we utilize the Implicit Function Theorem (IFT) [9] to derive a novel gradient computation scheme
LSTM → parameterizes → Deep Coordinator
confidence 90% · We last consider a shared feedback policy parametrized by a Long Short-Term Memory (LSTM) network
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Distributed optimization is a highly scalable and structurally transparent technique to solve multi-agent robotics problems; however, such methods often suffer from the need for highly-specialized, problem-specific hyperparameter tunings. In this work, we propose Deep Coordinator, a deep-unfolding framework that learns to dynamically adjust the hyperparameters of ADMM-DDP, a popular distributed solver for robotics tasks, at solve-time in response to optimizer performance. Our architecture consists of unrolling a fixed number of ADMM-DDP iterations into a neural network with learnable functions between layers mapping the optimizer state to the next hyperparameters. To the best of our knowledge, Deep Coordinator is the first deep-unfolding framework to adapt the penalty parameters of a non-convex optimizer at solve-time; we show that the mainstream supervised approach can yield degenerate solutions when training such models, and propose an unsupervised learning scheme. On simulations with fleets of cars and quadrotors, Deep Coordinator produces trajectories of comparable quality 6.18-9.44x faster than conventional solvers. Furthermore, Deep Coordinator retains its performance benefits when deployed to systems up to 8x larger than trained on.
Tags
Links
- Source: https://arxiv.org/abs/2606.19920v1
- Canonical: https://arxiv.org/abs/2606.19920v1
Trouble viewing inline? Open PDF directly →
Full Text
132,662 characters extracted from source content.
Expand or collapse full text
Deep-Unfolded Coordination Hunter Kuperman Minchan Jung Rahul V. Ghosh11footnotemark: 1 Alex Oshin Evangelos A. Theodorou Autonomous Control and Decision Systems Laboratory Georgia Institute of Technology United States These authors contributed equally (equal second authorship) Abstract Distributed optimization is a highly scalable and structurally transparent technique to solve multi-agent robotics problems; however, such methods often suffer from the need for highly-specialized, problem-specific hyperparameter tunings. In this work, we propose Deep Coordinator, a deep-unfolding framework that learns to dynamically adjust the hyperparameters of ADMM-DDP, a popular distributed solver for robotics tasks, at solve-time in response to optimizer performance. Our architecture consists of unrolling a fixed number of ADMM-DDP iterations into a neural network with learnable functions between layers mapping the optimizer state to the next hyperparameters. To the best of our knowledge, Deep Coordinator is the first deep-unfolding framework to adapt the penalty parameters of a non-convex optimizer at solve-time; we show that the mainstream supervised approach can yield degenerate solutions when training such models, and propose an unsupervised learning scheme. On simulations with fleets of cars and quadrotors, Deep Coordinator produces trajectories of comparable quality 6.18-9.44x faster than conventional solvers. Furthermore, Deep Coordinator retains its performance benefits when deployed to systems up to 8x larger than trained on. 1 Introduction Figure 1: Wall-clock time of Deep Coordinator trained on smaller-scale systems but deployed to larger-scale ones, where Deep Coordinator yields significant speedups across task types and scales. Plot titles indicate multi-agent task type. Results are averaged over 20 problems for each task. Multi-agent robotic systems, characterized by their large dimensionality, inter-agent constraints, and the need to coordinate effectively, span a variety of applications. Drone swarms performing search and rescue, for example, must coordinate their coverage patterns while avoiding collision [4]; teams of robots collaborating to lift heavy objects must manage weight distribution while maintaining balance [23]; and fleets of autonomous vehicles must jointly plan maneuvers such as lane changes and merges without compromising safety [6]. Many of these applications benefit from scaling to systems with hundreds or thousands of agents. However, in robotics tasks, controllers typically only have milliseconds to solve the next problem, and existing methods often cannot deliver solutions fast enough to be practical at scale. Two main approaches have arisen to solve such multi-agent problems, each with complementary strengths. Multi-Agent Reinforcement Learning (MARL) [25] leverages data to learn decentralized policies for each agent, resulting in fast inference time and the ability to specialize to complex problem distributions. However, MARL policies are frequently parameterized by model-free neural networks, meaning the resulting models are difficult to interpret and—although recent work [10, 22] has made progress in this area—often generalize poorly to out-of-distribution tasks. Conversely, distributed optimization decomposes large tasks into smaller optimization problems that can be solved quicker and in parallel. A popular method under this paradigm is ADMM-DDP [20], which combines the Alternating Direction Method of Multipliers (ADMM) [2] with Differential Dynamic Programming (DDP) [13]. Their method introduces safe copy-variables of the control and state which always satisfy constraints, enabling constraint satisfaction to be handled separately from solving the underlying control task. Although ADMM-DDP efficiently decomposes the problem structure and is highly interpretable, its convergence speed is heavily dependent on the tuning of coupled hyperparameters, the number of which increases significantly with the size of the problem. Despite this, ADMM-DDP is often hand-tuned, leading to slow convergence. Furthermore, performant tunings can vary significantly between problems, limiting the generalization capabilities of any one tuning. Rather than choosing between the speed and specializability of data-driven learning or the safety and interpretability of structured optimization, we view these characteristics as composable. In recent years, deep-unfolding [14] has emerged as a promising technique to accelerate optimization algorithms by parameterizing optimizer steps with trainable weights that can be learned from data. Deep-unfolding reinterprets these optimizer iterations as differentiable layers in a neural network, which can then be trained end-to-end to learn the optimal weights. The resulting learned optimizers inherit the interpretability and structure of their traditional counterparts, while often converging significantly faster and demonstrating improved generalization [19, 15, 12]. We claim that multi-agent robotics tasks are particularly well-suited for deep-unfolding, for the following reasons: 1. Multi-agent distributed optimizers often have tunable parameters for each agent, yielding hundreds or thousands of total hyperparameters. Deep-unfolding can efficiently learn complex relationships between these hyperparameters that are difficult to identify manually. 2. Control algorithms are often deployed in a model-predictive control (MPC) fashion, where the optimizer re-solves similar problems, differing only in their initial conditions and constraints, at each timestep. Deep-unfolded networks can be trained exclusively on this problem distribution, yielding specialized hyperparameter policies that improve convergence. 3. In robotics, slow convergence can lead to suboptimal or unsafe behavior. Given the time constraints, the performance benefits of deep-unfolded networks are particularly valuable. We propose Deep Coordinator, a deep-unfolding framework for learning to dynamically adapt ADMM-DDP hyperparameters at solve-time in response to optimizer performance. Our architecture consists of unrolling a fixed number of ADMM-DDP iterations into a neural network with learnable functions between layers mapping the optimizer state to the next hyperparameters, and can be deployed to larger or different teams of agents than trained on. For training, we utilize the Implicit Function Theorem (IFT) [9] to derive a novel gradient computation scheme which exploits the distributed structure of ADMM-DDP to efficiently differentiate through Deep Coordinator end-to-end. To the best of our knowledge, Deep Coordinator is the first deep-unfolding framework to adapt the penalty parameters of a non-convex optimizer at solve-time. We show that the mainstream supervised learning approach can yield degenerate solutions when training such non-convex deep-unfolded models, and propose an unsupervised training framework. To showcase the performance of Deep Coordinator, we benchmark our model on three multi-agent tasks involving fleets of cars and quadrotors. Across problems, Deep Coordinator finds solutions of the same cost and comparable constraint satisfaction 6.18-9.44x faster than its traditional optimizer counterparts. Furthermore, Deep Coordinator retains its performance benefits when deployed to systems with up to 8x larger team sizes than trained on. Lastly, we validate the improved performance of the unsupervised framework when compared to the mainstream supervised approach. 2 Related Work Deep-Unfolding as Control. The authors of [19] note that selecting the optimal hyperparameters in a deep-unfolded network can itself be interpreted as a robotic control problem. From this perspective, the optimizer steps at each iteration serve as a dynamics function which propagates the intermediate solutions through time, while the hyperparameters act as control inputs which drive the system towards an optimum. Under this interpretation, the mainstream approach of learning fixed, per-iteration parameters in deep-unfolding can be viewed as a form of open-loop control, since the hyperparameters are selected without feedback. In this sense, schemes that adapt hyperparameters in response to the optimizer state “close the loop.” Learning to Optimize ADMM. Deep-unfolding belongs to a broader class of methods called Learning to Optimize (L2O), which use machine learning to improve the performance of optimization algorithms. Several recent L2O works have targeted ADMM-based distributed optimization. One work [15] unfolds a generic distributed ADMM algorithm and learns an open-loop hyperparameter sequence. Another work [3] reformulates ADMM as a Graph Neural Network (GNN) and learns feedback policies on step size and communication weights. Reinterpreting ADMM as a GNN enables the learned model to explicitly reason over the underlying graph topology induced by the problem splitting; however, their framework does not consider learning penalty parameters and is limited to unconstrained, convex problems. In addition, [19] introduce and unfold a distributed quadratic programming algorithm based on OSQP [21]. They adapt hyperparameters using closed-loop feedback policies, which are parameterized as Multi-Layer Perceptrons (MLPs) taking local and global residuals as input. Each agent uses the same MLP weights, enabling policies to be deployed to problems with more agents than trained on. Although it demonstrates strong performance, their framework is limited to convex quadratic programs, and, while their model adjusts the penalty parameters per-agent, it does not adapt the entire penalty vector. Learning to Optimize in Robotics. Several other recent works have applied L2O methods to robotics tasks. The authors of [12] unfold centralized projected gradient descent, replacing several terms in the update equations with learned matrices, and apply the resulting learned optimizer to a linear MPC task. Another work [24] introduces a framework to meta-learn ADMM-DDP hyperparameters from example problems and derives an IFT-based gradient computation framework to differentiate through the ADMM-DDP optimizer. This approach demonstrates promising results, but is limited to producing a fixed hyperparameter sequence without utilizing the closed-loop form of adaptivity; additionally, the presented gradient computation framework does not handle cases where learnable functions are introduced between iterations. 3 ADMM-DDP 3.1 Problem Formulation Before we present our framework, we first review the mathematical foundation of ADMM-DDP. Consider a team of N agents collaborating on a task. Let i,t∈ℝnx x_i,t ^n_x and i,t∈ℝnu u_i,t ^n_u denote the state and control of agent i at time t, respectively. Each agent i is subject to the dynamics i,t+1=fi(i,t,i,t) x_i,t+1=f_i( x_i,t, u_i,t) and seeks to minimize its cost Ji(i,i)=ℓT(i,T)+∑t=0T−1ℓt(i,t,i,t)J_i( x_i, u_i)= _T( x_i,T)+ _t=0^T-1 _t( x_i,t, u_i,t) where T denotes the planning horizon, ℓt _t denotes the stage cost, and ℓT _T denotes the terminal cost. We consider the following optimization problem (i,ii=1N)∗=argmini,i (\ x_i, u_i\_i=1^N )^*= *argmin_\ x_i, u_i\ ∑i=1NJi(i,i), _i=1^NJ_i( x_i, u_i), subject to i,t+1=fi(i,t,i,t),∀i,t x_i,t+1=f_i( x_i,t, u_i,t), ∀ i,t gi,t(i,t,i,t)≤0,∀i,t g_i,t( x_i,t, u_i,t)≤ 0, ∀ i,t hij,t(i,t,i,t,j,t,j,t)≤0,∀i,j,t h_ij,t( x_i,t, u_i,t, x_j,t, u_j,t)≤ 0, ∀ i,j,t where gi,tg_i,t defines the local constraints for agent i and hij,th_ij,t defines the inter-agent constraints between agents i and j. 3.2 The ADMM-DDP Algorithm Problem (3.1) is a multi-agent nonlinear program (NLP) with a large number of variables and constraints, and cannot be solved scalably in a centralized fashion. To solve this problem efficiently, we consider the ADMM-DDP formulation employed in [24], introducing safe copy-variables ~ x and ~ u and decomposing the original task into the distributed problem min,,~,~ _ x, u, x, u ∑i=1N[Ji(i,i)+ℐfi(i,i)+ℐgi(~i,~i)+∑j≠i(ℐhij(~i,~j,~i,~j))], _i=1^N [J_i( x_i, u_i)+I_f_i( x_i, u_i)+I_g_i( x_i, u_i)+ _j≠ i (I_h_ij( x_i, x_j, u_i, u_j) ) ], (B) subject to =~,=~, 5.0pt x= x, 5.0pt u= u, where ℐfiI_f_i, ℐgiI_g_i, ℐhijI_h_ij are indicator functions for the dynamics, local, and inter-agent constraints, respectively. The augmented Lagrangian for this problem is ℒ= = ∑i=1N[Ji(i,i)+ℐfi(i,i)+ℐgi(~i,~i)+∑j≠i(ℐhij(~i,~j,~i,~j)) _i=1^N [J_i( x_i, u_i)+I_f_i( x_i, u_i)+I_g_i( x_i, u_i)+ _j≠ i (I_h_ij( x_i, x_j, u_i, u_j) ) +iT(i−~i)+iT(i−~i)+i2∥i−~i∥22+i2∥i−~i∥22] + λ_i^T( x_i- x_i)+ ξ_i^T( u_i- u_i)+ ρ_i2\| x_i- x_i\|_2^2+ μ_i2\| u_i- u_i\|_2^2 ] where ,∈ℝN×(T+1)×nx ρ, λ ^N×(T+1)× n_x and ,∈ℝN×T×nu μ, ξ ^N× T× n_u are the penalty parameter and dual variable for the consensus constraints on x and u, respectively. Minimizing the augmented Lagrangian with respect to each variable yields the following subproblems to solve at ADMM iteration a+1a+1, whose mathematical formulations are given in Appendix A: Subproblem 1: First, consider solving for the nominal trajectory updates. This yields the agent-wise unconstrained control problems (1). These subproblems can be solved in parallel using DDP to find dynamically-feasible trajectories that minimize each agent’s individual costs. Subproblem 2: Second, consider solving for the safe copy variable updates. Minimizing the augmented Lagrangian yields optimization problems (2) decoupled across time. The ~,~ x, u variables find a safe trajectory that respects the agent-specific and inter-agent constraints, and can be updated in parallel across the time horizon using an NLP solver. Subproblem 3: Third, consider updating the dual variables via dual ascent. This yields the parallelizable updates (3) that are decoupled for each agent-time pair. 4 Deep Coordinator In this section, we present the Deep Coordinator framework. For notational concision, we stack the ADMM iterates into a single vector a=[a,a,~a,~a,a,a]. v^a=[ x^a, u^a, x^a, u^a, λ^a, ξ^a]. 4.1 The Deep Coordinator Framework Suppose K is a predetermined maximum number of optimizer iterations we intend to unfold. We unroll K ADMM-DDP iterations into a neural network, re-interpreting each ADMM iteration as its own layer. We then construct the Deep Coordinator architecture by introducing penalty parameter adaptation policies between ADMM-DDP layers, of the form a=πwa(a−1,a−1,), split θ^a= _w^a( θ^a-1, v^a-1, χ), split where a=[a,a] θ^a=[ ρ^a, μ^a], πwa _w^a is a learnable feedback policy, waw^a are the network weights, and χ represents problem-specific data. There are several reasonable choices of πwa _w^a, which we discuss in Appendix B. We denote the resulting neural network with Πw _w. Running Deep Coordinator amounts to propagating the initial iterates and problem data through Πw _w, i.e., alternating between computing the next hyperparameters and executing a step of ADMM-DDP with these hyperparameters. We summarize this process in Algorithm 1, and the architecture is visualized in Fig. 2. To learn the optimal policy weights, we train Deep Coordinator end-to-end on a dataset of M sample tasks. Our training objective is the following bi-level optimization problem to find optimal weights w∗=argminw∑m=1MLm(Πw(0,m)), splitw^*= *argmin_w\;& _m=1^M\;L_m( _w( v^0, χ_m)), split where 1,…,M∈ χ_1,..., χ_M are sample tasks and LmL_m is an upper-level loss function which evaluates the trajectories generated by Deep Coordinator. We discuss choices of LmL_m in Subsection 4.2. Figure 2: The Deep Coordinator architecture. Our framework unfolds ADMM-DDP iterations into a deep learning architecture. We train policies for determining the optimal hyperparameters in an end-to-end fashion to maximize task performance. 4.2 Designing the Training Loop The mainstream approach [12, 3, 18, 19] to train deep-unfolded networks has been to minimize a supervised loss function which measures the distance to a ground-truth solution. This strategy has proven effective when unfolding convex solvers, which admit a unique global minimizer to utilize as the ground-truth. However, the multi-agent robotics tasks Deep Coordinator solves are heavily non-convex and often have many local optima. Whether or not a supervised approach is well-suited for this case is an open question. To build intuition for why the supervised framework can be poorly suited for training non-convex deep-unfolded models, we present a 1D example employing a simpler optimizer in Appendix C. Motivated by this, we adopt the following unsupervised loss function L(K)=γcost∑iJi(iK)+γconst(∑i,tmax0,gi,t(i,tK)2+∑i,j,tmax0,hij,t(i,tK,j,tK)2), splitL( v^K)= _ cost _iJ_i( v_i^K)+ _ const ( _i,t \0,g_i,t( v_i,t^K)\^2+ _i,j,t \0,h_ij,t( v_i,t^K, v_j,t^K)\^2 ), split where γcost _ cost and γconst _ const are tunable parameters. This loss penalizes a linear combination of the total cost and constraint violation and proved empirically reliable in our experiments. A natural alternative scheme is to employ a loss function which utilizes the ADMM-DDP residuals (e.g., a linear combination of the primal and dual residuals). However, when the initial trajectory is constraint-satisfying, both ADMM-DDP residuals can be made zero by degenerately scaling the penalty parameters arbitrarily high. In this case, the quadratic coupling term scaled by the penalty parameters acts as a proximal term, halting the optimization process. With arbitrarily high penalty parameters, the dual residual becomes zero since the iterates do not change, while the primal residual remains zero since the initial solution was feasible. As a result, residual minimization can reward stalled optimization rather than progress towards lower cost solutions. 4.3 Computing Gradients Training Deep Coordinator requires backpropagating through K iterations of ADMM-DDP, where each iteration involves solving Subproblems (1) and (2) using iterative optimizers. Unrolling these inner optimization loops using automatic differentiation [1] is computationally intractable, requiring O(K⋅T⋅Ninner)O(K· T· N_inner) time and memory, where NinnerN_inner is the number of inner solver iterations. Moreover, this unrolling approach does not support the use of black box optimizers that do not provide gradients. To address these shortcomings, we utilize the IFT [9] to derive a gradient computation scheme that avoids differentiating through the inner solver iterations and only requires O(K⋅T)O(K· T) time and memory. A detailed discussion of this framework is in Appendices F and G. Input: Problem data χ, weights w, number of iterations K, initialization 0 v^0 for a=1,…,Ka=1,…,K do Predict hyperparameters: a=πwa(a−1,a−1,) θ^a= _w^a( θ^a-1, v^a-1, χ); Solve Subproblem 1 with DDP: a,a← x^a, u^a← Eq. (1); Solve Subproblem 2 with NLP: ~a,~a← x^a, u^a← Eq. (2); Update dual variables: a,a← λ^a, ξ^a← Eq. (3); end for return K v^K Algorithm 1 Deep Coordinator 5 Experimental Results To evaluate the performance of Deep Coordinator, we benchmark our framework on several tasks involving fleets of cars and quadrotors. We compare Deep Coordinator models with three different parameterizations of the policy πwa _w^a, which we discuss below, to analyze the effect of feedback. Learned Scalar. We first consider an open-loop policy parameterized by a single scalar for ρ and a single scalar for μ which are shared across all agents, timesteps, and algorithm iterations. Learned Scalar Per Iteration. We second consider a policy parameterized by K open-loop scalars for ρ and K open-loop scalars for μ which are shared across all agents and timesteps, but can differ across iterations. LSTM. We last consider a shared feedback policy parametrized by a Long Short-Term Memory (LSTM) network [5]. The LSTM encodes an architectural prior that memory of past iterations is important in selecting the next hyperparameters. For each task, we unfold Deep Coordinator for K=30K=30 iterations and train with 80 train and 20 test problems. We compare Deep Coordinator against two un-learned benchmarks: Vanilla ADMM-DDP, which employs a fixed, manually-tuned scalar, and Adaptive ADMM-DDP, which employs the penalty parameter adaptation rule proposed in [20]. For details on the training setup, LSTM architecture, and baseline optimizers, consult Appendix H. Additional experiments ablating the effect of feedback and comparing the supervised and unsupervised loss can be found in Appendix D. 5.1 Main Experiments We study three types of control problems, which we discuss below. For additional details on problem generation and agent dynamics, consult Appendix H.1. Car Obstacle Field. We first consider a multi-agent avoidance scenario with a randomized obstacle field. In this task, teams of 15 Dubins vehicles must reach a target formation across a field of three circular obstacles. The centers and radii of the obstacles are randomized for each task instance. The results and a representative Deep Coordinator trajectory are shown in Fig. 3. Figure 3: Top: Results of the car obstacle field task on 20 unseen test instances. Error bars indicate standard deviation. Bottom: Trajectories of 15 vehicles performing the car obstacle field task generated by Deep Coordinator, with snapshots from different timesteps. Car Intersection. We second study a more complex intersection task where 8 Dubins vehicles must navigate a four-way junction. The initial and target states of the agents are randomly sampled, requiring significantly different trajectories for each problem instance. The results and a representative Deep Coordinator trajectory are shown in Fig. 4. Figure 4: Top: Results of car intersection tasks on 20 unseen test instances. Error bars indicate standard deviation. Bottom: Trajectories of 8 vehicles performing the car intersection task generated by Deep Coordinator, with snapshots from different timesteps. Quadrotor Obstacle Field. We third examine a quadrotor maneuvering problem where teams of 10 quadrotors navigate a field of 7 cylindrical obstacles. To maximize interactions between agents, the two columns of agents must swap their relative positions while crossing the field. Both displacements to the quadrotors’ initial altitudes and the obstacles’ centers and radii are randomized across task instances. The results and a representative Deep Coordinator trajectory are shown in Fig. 5. Figure 5: Top: Results of quadrotor obstacle field tasks on 20 unseen test instances. Error bars indicate standard deviation. Bottom: Trajectories of 10 quadrotors performing the obstacle field task generated by Deep Coordinator, with snapshots from different timesteps. For each task, we evaluate the cost, maximum constraint violation, and wall-clock time of all models on 20 unseen test problems, and average the results. The cost and constraint plots report values at the end of the K=30K=30 iteration budget. To compare wall-clock times, Deep Coordinator is run for 30 iterations. Then, in order to show the relative improvement of our method, Vanilla ADMM-DDP is run until its cost is within 5% of Deep Coordinator’s. We omit timings for Adaptive ADMM-DDP as, though it quickly finds low-cost solutions, its trajectories significantly violate safety constraints. The optimal trajectory is computed by running Vanilla ADMM-DDP until convergence. Our experiments indicate that Deep Coordinator significantly outperforms its traditional optimizer counterparts. We partition our analysis into three parts: within the 30-iteration budget, when allowing Vanilla ADMM-DDP to run until it reaches the same cost as Deep Coordinator, and compared to the optimal solution. Within the 30-iteration budget, Deep Coordinator finds much higher-quality solutions than its traditional counterparts. The LSTM policy, in particular, Pareto dominates Vanilla ADMM-DDP. On the car and quadrotor obstacle field tasks, LSTM Deep Coordinator is able to find solutions with significantly lower cost (1.94-3.59x) and constraint violation (2.61-5.40x) than Vanilla ADMM-DDP. In the intersection task, where the agents must manage more complex right-of-way interactions, the performance benefit of Deep Coordinator is lower, but still yields 2.82x lower cost and 1.22x lower constraint violation trajectories. Adaptive ADMM-DDP quickly converges to slightly lower cost solutions than Deep Coordinator, but does so with much higher constraint violation—up to an order of magnitude greater than the other solvers—and is therefore unsafe to use. The most significant benefit of the Deep Coordinator framework is the reduction in wall-clock time required to reach high-quality solutions. When allowing Vanilla ADMM-DDP to run for longer, Deep Coordinator finds trajectories of comparable cost 6.18-9.44x faster with comparable constraint violation. For additional context, figures depicting the cost and constraint violation over an extended wall-clock time are in Appendix I.1. We emphasize that the cost-thresholded Vanilla solution is not the optimal trajectory; rather, it is an intermediate iterate that has not yet converged and, in general, exhibits higher constraint violation. Compared to the optimal trajectories, Deep Coordinator finds solutions of comparable cost, but higher constraint violation. This is expected, as Deep Coordinator is significantly constrained by its iteration budget, taking 13.8-18.6x less time than it takes to converge to the optimal solution. Figure 6: Deployment results as the number of agents scales. For each task type and scaling factor, we test on 20 problems and average the results. From top to bottom, the rows correspond to the car obstacle field, car intersection, and quadrotor obstacle field task, respectively. 5.2 Scaling Experiments Next, we study how Deep Coordinator’s learned update policies generalize to larger and different teams of agents than those seen during training. We deploy the trained Deep Coordinator models to scaled tasks. For each task type and scaling factor, we test on twenty problems and average the results. Fig. 6 summarizes the results, while a table of numerical values is in Table 3 in Appendix I. For details on scaled problem generation, consult Appendix H.2. In the car and quadrotor obstacle field tasks, scaling largely preserves the local structure observed by each agent. All three Deep Coordinator parameterizations generally preserve their advantage over Vanilla ADMM-DDP as the number of agents increases. The car intersection task presents a more significant challenge, as increasing the number of agents fundamentally alters the underlying coordination problem and the optimal right-of-way behavior. Nevertheless, Deep Coordinator continues to find significantly lower-cost and higher-constraint satisfaction solutions than Vanilla ADMM-DDP, while retaining its 5x speedup in the LSPI and LSTM cases. Standard black-box neural networks trained on such a narrow and small dataset of tasks typically perform poorly in such out-of-distribution regimes due to overfitting to the agent topologies and spatial trajectories found in the training dataset. Deep Coordinator avoids this by delegating the specific inter-agent coordination behavior to the ADMM-DDP optimizer and instead learning to control the optimization process itself. The feedback policies it learns are local, taking ADMM quantities specific to a single (agent, timestep) pair which reflect whether that agent’s trajectory is consistent with its neighbors’, regardless of how many neighbors are present or how they are arranged. These signals remain structurally similar across tasks, meaning the policy stays in-distribution with respect to the optimizer, even when deployed on problems well outside of its training distribution. In this sense, the architectural prior of the ADMM-DDP structure, rather than the expressive capacity of the neural network alone, enables Deep Coordinator’s generalization. We perform additional experiments and analysis ablating the impact of feedback and empirically validating the performance improvements of the unsupervised loss in Appendix D. 6 Limitations In principle, non-convex deep-unfolded optimizers can be trained to find globally-optimal solutions. However, our work is focused on accelerating the convergence of ADMM-DDP to high-quality local optima. Training to find global solutions would require incorporating a mechanism to explore the optimization landscape (e.g., training with the exploration mechanism of RL or unfolding a global optimizer). Future work will explore training with RL and unfolding global optimizers to train models which can quickly converge to the globally-optimal trajectories. Furthermore, while Deep Coordinator performs well within its iteration budget, it has no canonical mechanism to extend the optimization process beyond this, if needed. In principle, the Learned Scalar and LSTM policies can be deployed for more than K ADMM-DDP iterations; however, there is no guarantee that the policy will generalize well beyond this point. Other deep-unfolding work [18] proposes a progressive training scheme, which is a promising mechanism to address this. References [1] A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind (2017-Jan.) Automatic differentiation in machine learning: a survey. Journal of Machine Learning Research 18 (1), p. 5595–5637. External Links: ISSN 1532-4435 Cited by: §4.3. [2] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein (2011-Jan.) Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning 3, p. 1–122. External Links: Document Cited by: §1. [3] H. Doerks, P. Häusner, D. H. Escobar, and J. Sjölund (2025) Learning to accelerate distributed ADMM using graph neural networks. arXiv:2509.05288. Note: The authors demonstrate an equivalence between distributed ADMM and Graph Neural Networks and propose unfolding ADMM into a GNN with closed-loop feedback on algorithm hyperparameters. Their approach is elegent and well-developed, and can inform the architecture in our paper. [56] The authors demonstrate that distributed ADMM can be represented as a GNN and unfold it, introducing additional per-agent neural networks that map previous iterates and residuals to the next hyperparameters. This is an extension of the “closed loop” form of adaptation introduced in THE DUDO4LSQP PAPER; however, their approach has the benefit of naturally scaling to different graph topologies and problem dimensions while still incorporating the structure of the underlying network into policies. [57] It’s worth noting that the authors only consider learning scalar per-agent or global hyperparameters, and do not consider learning the entire penalty parameter vector as in THE RLQP PAPER. In principle, their approach could be adapted to address this by introducing per-agent, per-constrant networks. Their approach could also be adapted to include other forms of hyperparameter adaptivity such as the “meta-learning” and “warm-starting” approaches adopted in THE LEARNING TO COORINDATE PAPER and THE LEARNING TO WARM-START PAPER, respectively, to yield a single, unified framework that outperforms each individually. We should implement this and perform an ablation study, if we adopt a GNN-based architecture in our paper. [58] The authors utilize a supervised loss based on distance to a pre-computed ground-truth solution. This seems reasonable, as they only consider convex problems which admit unique minimizers; however, it’s unclear if their approach would be stable if using a loss function that is applicable to non-convex problems, such as the semi-supervised, residual-based loss adopted in THE LEARNING TO COORDINATE PAPER. [59] The authors benchmark their work on two distributed problem types. Their algorithm achieves superior performance compared to the vanilla optimizer, but doesn’t achieve the level of improvements seen in THE DUDO4LSQP PAPER, despite the more principled architecture. Perhaps this is due to the fact that the latter also unfolds OSQP-specific hyperparameters which could more directly effect the convergence speed of the optimizer. External Links: 2509.05288, Link Cited by: §2, §4.2. [4] A. Ferraro, V. Nardi, E. D’Amato, I. Notaro, and V. Scordamaglia (2023) Multi-drone systems for search and rescue operations: problems, technical solutions and open issues. In 2023 IEEE International Workshop on Technologies for Defense and Security (TechDefense), Vol. , p. 159–164. External Links: Document Cited by: §1. [5] S. Hochreiter and J. Schmidhuber (1997) Long short-term memory. Neural Computation 9 (8), p. 1735–1780. Cited by: §5. [6] M. Hua, X. Qi, D. Chen, K. Jiang, Z. E. Liu, H. Sun, Q. Zhou, and H. Xu (2025) Multi-agent reinforcement learning for connected and automated vehicles control: recent advancements and future prospects. IEEE Transactions on Automation Science and Engineering 22 (), p. 16266–16286. External Links: Document Cited by: §1. [7] W. Jin, S. Mou, and G. J. Pappas (2021) Safe Pontryagin differentiable programming. Advances in Neural Information Processing Systems 34, p. 16034–16050. Cited by: §F.2. [8] W. Jin, Z. Wang, Z. Yang, and S. Mou (2020-Dec.) Pontryagin differentiable programming: an end-to-end learning and control framework. In 34th International Conference on Neural Information Processing Systems, External Links: ISBN 9781713829546 Cited by: Appendix F. [9] S. G. Krantz and H. R. Parks (2003) The implicit function theorem. Birkhäuser Boston. External Links: ISBN 9781461200598, Link, Document Cited by: §1, §4.3. [10] Y. Li, S. Zhang, J. Sun, Y. Du, Y. Wen, X. Wang, and W. Pan (2024) Cooperative open-ended learning framework for zero-shot coordination. External Links: 2302.04831, Link Cited by: §1. [11] I. Loshchilov and F. Hutter (2017) Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. Cited by: §H.5. [12] D. Lupu and I. Necoara (2023-06) Deep unfolding projected first order methods-based architectures: application to linear model predictive control. In 2023 European Control Conference (ECC), p. 1–6. Note: The authors propose a framework for deep-unfolding projected first-order optimizers in an MPC context. Their framework is distinct from other deep-unfolding approaches and appears to be the first application of deep-unfolding to robotics problems. [26] In particular, the authors unfold projected gradient descent along with it’s accelerated counterpart. Interestingly, their framework learns weight matrices which act directly on the optimization variable—in contrast to typical deep-unfolding methods that learn functions to compute optimizer hyperparameters which affect iterates indirectly. This seems reasonable, as it’s unclear what hyperparameters beyond step size should be unfolded in the PGD case. Still, though, we’ve seen that just unfolding step size can provide significant performance improvements in THE LEARNING ALGORITHM HYPERPARAMETERS PAPER. [27] Interestingly, since their approach does not maintain the underlying structure of the PGD optimizer, the learned model is not garaunteed to converge. In some sense, they’re using PGD as a tool to guide architectural design rather than an explicit step in the neural network layers. This structure is closer to a learning-to-optimize approach than a deep-unfolding one; despite this, the authors still label their approach as the latter. [28] Unlike the work of THE LEARNING TO OPTIMIZE PAPER and THE DUDO4LSQP PAPER, they only consider the centralized and first-order cases. Nevertheless, the work is important as it was the first to explore the intersection between deep-unfolding and robotics. [29] They benchmark their work on a linear-dynamics, quadratic-cost system. Their framework achieves similar trajectories to the ground-truth, but computes solutions 5x faster than CVX. It’s difficult to determine the difference in trajectory quality from the ground-truth solution based on their discussion. External Links: Link, Document Cited by: §1, §2, §4.2. [13] D. Mayne (1966-01) A second-order gradient method for determining optimal trajectories of non-linear discrete-time systems. International Journal of Control 3 (1), p. 85–95. Note: Mayne proposes a novel control algorithm, DDP, to compute trajectories over non-linear dynamics and cost functions. This is the original DDP paper and is well-known across robotics. In particular, it is the basis of the ADMM-DDP algorithm that we’re unfolding for our paper. [7] Mayne’s proposed algorithm solves control problems by repeatedly quadratizing the dynamics and cost functions, analytically solving the quadratic problem, and taking a step towards the solution. Each iteration of quadratization consists of a forwards-pass and a backwards-pass. In contrast to previous second-order methods, Mayne’s formulation exploits the time-varying nature of the control problem to yield a much more efficient algorithm than pre-existing approaches. [8] Naturally, this method does not introduce any learnable components into the algorithm; however, it is important in that it is the basis for the vanilla ADMM-DDP algorithm used in both THE LEARNING TO OPTIMIZE PAPER and our proposed paper. Depending on page limits and the complexity of our approach to meta-gradient computation in ADMM-DDP, it may be useful to reproduce Mayne’s derivation of DDP in our paper. Regardless, we should refer to it when discussing ADMM-DDP. External Links: ISSN 1366-5820, Link, Document Cited by: §1. [14] V. Monga, Y. Li, and Y. Eldar (2021-03) Algorithm unrolling: interpretable, efficient deep learning for signal and image processing. IEEE Signal Processing Magazine 38, p. 18–44. External Links: Document Cited by: §1. [15] Y. Noah and N. Shlezinger (2025-04) Distributed Learn-to-Optimize: Limited Communications Optimization Over Networks via Deep Unfolded Distributed ADMM. IEEE Transactions on Mobile Computing 24 (4), p. 3012–3024. Note: The authors propose a framework for deep-unfolding generic distributed ADMM optimizers. Although their architecture only learns fixed hyperparameters, it’s application to any D-ADMM algorithm is generalizing. [31] Their framework learns non-adaptive D-ADMM hyperparameters for each iteration. Unlike the work in THE LEARNING TO WARM-START PAPER, THE LEARNING TO COORDINATE PAPER, and THE DUDO4LSQP PAPER, the authors don’t consider any form of adaptivity in their framework. This seems limiting, and their approach is likely restricted to learning a much smaller class of functions than the other frameworks, although we haven’t seen a formal proof of this. [32] The authors propose both an agent-varying and a shared hyperparameter scheme, enabling them to train and deploy on different graph topologies. The latter is similar to the shared network weights approach taken in THE DUDO4LSQP PAPER. This could probably be improved in both cases by introducing an auxillary neural network with a topology-agnostic architecture, like a Graph Neural Network or a Graph Transformer. [33] The authors motivate their approach by drawing a connection between the fixed iteration budget and inter-agent communication constraints. This is a strength of their paper; it’s well-developed and fits particularly well with deep-unfolding, which requires the user to fix some number of iterations a priori. In the multi-agent robotics contexts that we’re considering in our paper, this connection is particularly appealing, as agents may realistically be unable to communicate with neighbors for long periods of time. [34] Interestingly, the authors benchmark their algorithms against deep neural network architectures rather than vanilla, non-unfolded optimizers. Their framework does not achieve the many-orders-of-magnitude performance improvements deep-unfolding has seen in other contexts, but seems to outperform their baselines. External Links: ISSN 2161-9875, Link, Document Cited by: §1, §2. [16] A. Oshin, H. Almubarak, and E. A. Theodorou (2024-07) Differentiable Robust Model Predictive ControlDifferentiable Robust Model Predictive Control. In Proceedings of Robotics: Science and Systems, Delft, Netherlands. Note: The authors propose a method for differentiating through the DDP optimizer. Their approach is elegant, well-articulated, and could be usefully applied to the deep-unfolding and ADMM-DDP contexts we consider in our paper. [11] The authors approach utilizes the Implicit Function Theorem to differentiate through the DDP optimality conditions. They make the critical observation that DDP is particularly well-suited for an application of IFT, as the gradient formulation it yields can be computed by re-using components of the DDP optimization process, making it more efficient than pre-existing approaches. [12] Their meta-gradient computation through DDP seems particularly well-suited to a deep-unfolding context. Despite this, the authors motivate their work as enabling constant hyperparemeter tuning through a bi-level optimization problem. Perhaps this is due to the fact that this paper was published in 2023, when deep-unfolding was less prominent in the literature. [13] A similar IFT-based approach is used in THE DUDO4LSQP to achieve the gradients of solving each QP subproblem. Notably, however, it seems that this is not the approach taken in Learning to Coordinate THE L2C PAPER. We may be able to utilize this approach to obtain a more elgent and/or efficient form of computing meta-gradients through ADMM-DDP, either by using this as a step in deriving a new auxilliary ADMM-based meta-gradient computation algorithm or by using it to compute centralized DDP gradients and enabling autograd to perform the remaining steps of differentiation. [14] A strength of this paper is how effectively it articulates it’s methodology and application of the IFT. It’s also worth noting that the authors propose a differentiable tube-based MPC algorithm to achieve robustness while maintaining differentiability, but this is beyond the scope of our work. External Links: Document Cited by: §F.3, Appendix F, §G.1, §G.3. [17] F. Sabatino (2015) Quadrotor control: modeling, nonlinear control design, and simulation. Master’s Thesis, KTH Royal Institute of Technology. Cited by: §H.1.3. [18] R. Sambharya and B. Stellato (2024) Learning algorithm hyperparameters for fast parametric convex optimization. arXiv:2411.15717. Note: The authors propose an deep-unfolding framework for fixed-point optimization algorithms, learning non-adaptive scalar hyperparameters for each optimizer iteration. Although their framework is much simpler than SOTA deep-unfolding approaches, it’s well-executed and they make several interesting design decisions. [16] Notably, their approach does not adapt hyperparameters in any way at test time. This is a stark contrast to THE LEARNING TO COORDINATE PAPER (which employs a “meta-learning” form of adaptation) and THE DUDO4LSQP PAPER (which employs a “closed-loop” form of adaptation). As a result, their framework is restricted to learning a simpler class of functions and doesn’t achieve the same level of speedups as the other two approaches. Furthermore, they only apply their approach to centralized, convex optimizers. A benefit of this approach, though, is it’s data-efficiency, sometimes needing only 10 examples to learn something meaningful. [17] For training, they utilize a supervised loss based on distance to the ground-truth solution to the optimization problem. This works as they only apply their method to convex problems; however, it cannot be directly applied to the non-convex control problems we are tackling in this paper, as they often don’t admit unique solutions. Still, some version of this that recomputes the ground-truth solutions every few epochs is worth exploring. [18] They employ an interesting training scheme that partitions the network into contiguous chunks of layers and updates the parameters of one chunk before performing inference on and updating the parameters of the next chunk. This cirriculum-learning-like approach may be useful for training our network, although it’s unclear if it will be stable or converge faster in our more-complex context. [19] They also propose learning a single fixed set of hyperparameters to be used for all iterations after the final one. This is a a convenient and elegant method of attaining convergence garauntees without needing to construct an additional proof. [20] They benchmark their method in numerical experiments across a variety of optimizers and problem types. Based on these experiments, their method works well. Interestingly, we see that the optimizer learns a “jagged” sequence of hyperparameters in the GD and PGD cases, similarly to what we’ve observed in our deep unfolding work with QPs. External Links: 2411.15717, Link Cited by: §4.2, §6. [19] A. D. Saravanos, H. Kuperman, A. Oshin, A. T. Abdul, V. Pacelli, and E. Theodorou (2025) Deep distributed optimization for large-scale quadratic programming. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §1, §2, §2, §4.2. [20] A. D. Saravanos, Y. Aoyama, H. Zhu, and E. A. Theodorou (2023-12) Distributed differential dynamic programming architectures for large-scale multiagent control. IEEE Transactions on RoboticsTrans. Rob. 39 (6), p. 4387–4407. External Links: Link, Document Cited by: Appendix B, §1, §5. [21] B. Stellato, G. Banjac, P. Goulart, A. Bemporad, and S. Boyd (2020) OSQP: An operator splitting solver for quadratic programs. Mathematical Programming Computation 12 (4), p. 637–672. Cited by: §F.2, §2. [22] C. Sun, S. Huang, and D. Pompili (2024) LLM-based multi-agent reinforcement learning: current and future directions. External Links: 2405.11106, Link Cited by: §1. [23] E. Tuci, M. H. M. Alkilabi, and O. Akanyeti (2018-05) Cooperative object transport in multi-robot systems: a review of the state-of-the-art. Frontiers in Robotics and AI 5. External Links: ISSN 2296-9144, Link, Document Cited by: §1. [24] B. Wang, Y. Gao, T. Sun, and L. Zhao (2025) Learning to Coordinate: Distributed Meta-Trajectory Optimization Via Differentiable ADMM-DDP. arXiv:2509.01630. Note: The authors propose a framework for meta-learning ADMM-DDP (from THE ORIGINAL ADMM-DDP PAPER) hyperparameters in a supervised fashion. Their paper is the current SOTA in unfolded distributed MPC (that we’re aware of) and forms the foundation for much of our work. It’l also be an important algorithm to benchmark against. [2] Similar to deep-unfolding approaches, they fix a maximum number of optimizer iterations; however, instead of learning fixed hyperparameters, they learn small agent-wise neural networks that map problem- and agent-specific data to that agent’s hyperparameters. This “meta-learning” form of adaptivity is interesting. It’s distinct from the “closed-loop” form of adaptivity in THE DUDO4LSQP PAPER, but seems isomorphic to the warm-starting architectures we’ve been experimenting with. These forms of adaptivity are not mutually exclusive, and we should use both together and perform an ablation study. [3] For training, the authors also propose a method of differentiating through the ADMM-DDP optimizer which computes meta-gradients by solving an auxilliary ADMM-LQR problem. If applicable to our architecture, this approach may speed up our training and yield better accuracy. Similarly to ALEX’S WORK, they use the IFT; although, Oshin’s approach is better-articulated. It’s unclear if directly applying the approach in ALEX’S WORK would yield the same forumation; this is worth testing. It’s also unclear if their approach is faster than using the IFT to seperately differentiate through each ADMM subblock and using autograd to synthesize these gradients into the meta-gradient. In our paper, we should benchmark both and present some discussion of the results to the community. [4] They use a residual-based and tracking-based loss function, rather than some supervised loss based on optimal solitions. This is a reasonable choice, since ADMM-DDP problems are non-convex and local minima may not be global minimizers. This approach is also worth trying and benchmarking against, in addition to a supervised-based approach. [5] They apply their algorithm to a multi-lift quadrotor scenerio, which seems like a good choice for our paper, as well. Their algorithm seems to work well in quickly generating dynamically-feasible trajectories; although, it’s difficult to tell if they have an advantage in trajectory quality. External Links: 2509.01630, Link Cited by: §2, §3.2. [25] K. Zhang, Z. Yang, and T. Basar (2019) Multi-agent reinforcement learning: A selective overview of theories and algorithms. CoRR abs/1911.10635. External Links: Link, 1911.10635 Cited by: §1. Appendix A Detailed Description of the ADMM-DDP Subproblems In this appendix, we provide the mathematical breakdown of each subproblem of ADMM-DDP. Subproblem 1. For each agent i=1,…,Ni=1,...,N, we solve ia+1,ia+1=argmini,i x_i^a+1, u_i^a+1= *argmin_\ x_i, u_i\ ∑t=0T−1ℓ^i,t(i,t,i,t)+ℓ^i,T(i,T) _t=0^T-1 _i,t( x_i,t, u_i,t)+ _i,T( x_i,T) (1) subject to i,t+1=fi(i,t,i,t) x_i,t+1=f_i( x_i,t, u_i,t) where the augmented stage costs are defined as ℓ^i,t(i,t,i,t) _i,t( x_i,t, u_i,t) =ℓi,t(i,t,i,t)+i,t2‖i,t−~i,ta+i,t−1i,ta‖22+i,t2‖i,t−~i,ta+i,t−1i,ta‖22, = _i,t( x_i,t, u_i,t)+ ρ_i,t2 \| x_i,t- x_i,t^a+ ρ_i,t^-1 λ_i,t^a \|_2^2+ μ_i,t2 \| u_i,t- u_i,t^a+ μ_i,t^-1 ξ_i,t^a \|_2^2, and the terminal cost is defined as ℓ^i,T(i,T) _i,T( x_i,T) =ℓi,T(i,T)+i,T2‖i,T−~i,Ta+i,T−1i,Ta‖22. = _i,T( x_i,T)+ ρ_i,T2 \| x_i,T- x_i,T^a+ ρ_i,T^-1 λ_i,T^a \|_2^2. Subproblem 2. For each time t=0,…,T−1t=0,…,T-1, we solve ~ta+1,~ta+1=argmin~t,~t -10.00002pt x_t^a+1, u_t^a+1= *argmin_ x_t, u_t ∑i=1N[i,t2‖i,ta+1−~i,t+i,t−1i,ta‖22+i,t2‖i,ta+1−~i,t+i,t−1i,ta‖22] _i=1^N [ ρ_i,t2 \| x_i,t^a+1- x_i,t+ ρ_i,t^-1 λ_i,t^a \|_2^2+ μ_i,t2 \| u_i,t^a+1- u_i,t+ μ_i,t^-1 ξ_i,t^a \|_2^2 ] (2) subject to gi,t(~i,t,~i,t)≤0, g_i,t( x_i,t, u_i,t)≤ 0, hij,t(~i,t,~j,t,~i,t,~j,t)≤0. h_ij,t( x_i,t, x_j,t, u_i,t, u_j,t)≤ 0. For time t=Tt=T, the problem is of the form ~Ta+1=argmin~T x_T^a+1= *argmin_ x_T ∑i=1Ni,T2‖i,Ta+1−~i,T+i,T−1i,Ta‖22, _i=1^N ρ_i,T2 \| x_i,T^a+1- x_i,T+ ρ_i,T^-1 λ_i,T^a \|_2^2, subject to gi,T(~i,T)≤0, g_i,T( x_i,T)≤ 0, hij,T(~i,T,~j,T)≤0. h_ij,T( x_i,T, x_j,T)≤ 0. Subproblem 3. Across all agents and timesteps, we update the dual variables with i,ta+1 λ_i,t^a+1 =i,ta+i,t(i,ta+1−~i,ta+1), = λ_i,t^a+ ρ_i,t( x_i,t^a+1- x_i,t^a+1), (3) i,ta+1 ξ_i,t^a+1 =i,ta+i,t(i,ta+1−~i,ta+1). = ξ_i,t^a+ μ_i,t( u_i,t^a+1- u_i,t^a+1). Appendix B Choosing a Feedback Scheme There are many reasonable choices of feedback policies πwa _w^a. For example, a straightforward approach is to learn σ and χincr,decr _ \incr,\, decr\ in the ADMM-DDP penalty parameter adaptation rule proposed in [20]. However, given the distributed nature of ADMM-DDP, we particularly desire feedback schemes that have the capacity to: 1. Independently adapt each element of the penalty vectors a∈ℝN×(T+1)×nx ρ^a ^N×(T+1)× n_x and a∈ℝN×T×nu μ^a ^N× T× n_u, and 2. Deploy to problems with longer time horizons and larger or different graph topologies underlying the agents than trained on The first property allows for policies that capture complex relationships between hyperparameters that would be impossible to represent with simpler schemes (e.g., re-scaling a fixed vector). The second property not only increases the flexibility of the trained optimizer, but also allows models to be trained on smaller problems then deployed to much larger ones. This significantly reduces the computational burden of training. Since the penalty vectors scale with the number of agents and timesteps, any policy that has a fixed output size—such as a single MLP—cannot have property two. To address this, we consider shared per-agent, per-timestep policies which utilize shared weights. Under this scheme, a shared policy φwa _w^a maps agent- and timestep-specific inputs to the associated components of the hyperparameter vector. While the policy weights are shared, the varying inputs enable the network to provide context-aware local updates. To ensure scalability, these inputs are derived from a−1 θ^a-1, a−1 v^a-1 and χ in a manner which is invariant to the problem dimension; examples of such inputs include the relevant components of the ADMM residual vector, dual variables, or trajectories. The overall feedback scheme can be formulated as (πwa)i,t=φwa∘ψi,t, split( _w^a)_i,t= _w^a _i,t, split where ψi,t _i,t extracts the local inputs for agent i and timestep t. Appendix C Illustrative Example For The Supervised vs Unsupervised Loss To build intuition for why the supervised framework can be poorly suited for training non-convex deep-unfolded models, we present the following illustrative 1D example employing a simpler optimizer. Consider the task of unfolding K=10K=10 iterations of a gradient descent algorithm, characterized by the update equation xa+1=xa−α∇xf(xa) splitx^a+1=x^a-α _xf(x^a) split (4) where f is the objective function of the current problem. In this case, we can learn a scalar per iteration for the step-size hyperparameter α. Consider the following training problem min∈ℝ(x−0.3)8−(x−0.25)2+0.55475 split _ x \ (x-0.3 )^8- (x-0.25 )^2+0.55475 split (5) where the constant term is added to set the global minimum to zero. This function exhibits a local minimum at x≈−0.4850x≈-0.4850 and a global minimum at x≈1.1017x≈ 1.1017. Suppose we train the resulting unfolded model using the supervised scheme introduced in Section 4.2 with the global minimizer as the ground-truth solution. Lastly, suppose the initial iterate x0x^0 is zero. We’ve plotted the objective function, along with the resulting loss function and the initial iterate in Fig. 7. Figure 7: A comparison of the loss function (solid blue) and objective function (dashed red) in the illustrative example. The initial iterate is shown as a grey dot. With sufficiently small initial alpha values, the optimizer will converge to the local minimum at x≈−0.4850x≈-0.4850. This is suboptimal with respect to the loss, which penalizes distance away from the global minimizer. To find the loss-minimizing solution, the upper-level training scheme is forced to discover that large step sizes cause the optimizer to jump to the other basin. If it is unable to do so, the locally loss-minimizing choice of alpha—among those which do not cause the optimizer to ascend on the objective function—is to set all alpha values to zero. This prevents the optimizer from moving further away from the initial iterate. In this manner, when the initial iterates of an unfolded optimizer begin in a basin which does not contain the ground-truth solution, the training scheme may push the parameters to degenerate values. Appendix D Additional Analysis and Experiments D.1 Ablating the Impact of Feedback We now compare the performance of the three feedback policy parameterizations, based on the results in Section 5. The Learned Scalar policy represents the optimal hand-tuning of the penalty parameters. We, therefore, expect the more sophisticated policies to outperform LS. Indeed, they do. The LSTM policy, in particular, achieves lower cost and constraint violation than LS in all cases. We also observe that the LSTM exhibits favorable behavior compared to LSPI. In the quadrotor obstacle field task, the LSTM achieves moderately (1.85x) lower constraint violation while incurring slightly higher (1.05x) cost. In the car obstacle field and car intersection tasks, the LSTM achieves similar (0.83x-0.93x lower) constraint violation while incurring lower costs (1.17-1.31x) than LSPI. This improvement is meaningful, but lower than in the quadrotor case. The difference in the prioritization of cost and constraint satisfaction is determined by the choice of scaling parameters in the unsupervised loss. In principle, this difference could be alleviated by performing a finer-grained sweep of these parameters. Figure 8: Comparison of cost and constraint values generated by policies trained under different training schemes. Hatched bars correspond to policies trained using the supervised scheme, while plain bars correspond to policies trained using the unsupervised scheme. The rows correspond to car obstacle field (top), car intersection (middle), and quadrotor obstacle field (bottom). Error bars indicate standard deviation. In addition to its favorable performance over the other policies, the LSTM adds minimal overhead in wall-clock time. This is because the computational cost of solving the inner-loop ADMM-DDP subproblems is significantly higher than the cost of forward-propagating the LSTM. Furthermore, the LSTM achieves lower wall-clock time than LSPI in the quadrotor case. This is due to the penalty parameters predicted by LSPI yielding inner-loop ADMM-DDP subproblems which take longer to solve. A candidate explanation for the superior performance of the LSTM is that the increased number of parameters in the LSTM-based policies allow the model to find a better (potentially degenerate, constant) solution rather than actually incorporating feedback. Analysis of the policy outputs demonstrates that this is not the case. For example, the LSTM trained on the car obstacle field task produces high variance outputs with values below 10110^1 and above 102.10^2. We have included figures showing a representative sample of the outputs of each policy in Appendix E. D.2 Comparing the Unsupervised and Supervised Losses Finally, we compare the impact of our supervised and unsupervised loss functions on Deep Coordinator’s performance by evaluating the cost and maximum constraint violation achieved by models trained on the two loss functions. For a detailed description of the supervised and unsupervised training schemes, consult Appendix H.5. The results are shown in Fig. 8. The supervised loss yields Deep Coordinator models that often find lower cost trajectories than the unsupervised loss. In particular, the cost achieved by supervised Deep Coordinator models on the car intersection task is 1.66-3.49x lower than that achieved by unsupervised Deep Coordinator models. However, the resulting trajectories are often degenerate, and heavily violate constraints. Additionally, 3 of the 9 supervised policies produced numerically unstable inner-loop problems when attempting to scale, failing to complete their iteration budget. This further indicates the degenerate nature of the trajectories produced by the supervised method. Across our three problems, the unsupervised Deep Coordinator models typically yield significantly lower constraint violation than their supervised counterparts. On car problems, LSPI and LSTM Deep Coordinator models achieve between 3.17-4.98x lower constraint violation when trained using the unsupervised loss in lieu of the supervised loss. This effect also occurs in the quadrotor obstacle field case; however the cost values are much closer between frameworks. Overall, the unsupervised Deep Coordinator models tend to sacrifice some trajectory cost for considerably higher trajectory feasibility, indicating that it is better-aligned with the deployment objectives of Deep Coordinator. Appendix E Sample Policy Output In this appendix, we illustrate how the outputs of the feedback policy differ between policy parameterizations. For each feedback scheme, we display the output ρ and μ values. We plot a line for each (agent, timestep, state/control dimension) tuple, representing the policy output averaged over the problem instances for this component of the penalty vector. For each line, the area ± one standard deviation around the mean is drawn as a translucent shaded region around the mean. A colormap over the flattened (agent, timestep, state/control dimension) dimensions is used to distinguish between lines. The plots below are from the unsupervised models for the car obstacle field task. The LS policy employs a single constant value for ρ and μ, applied uniformly to every problem and iteration. Its output is therefore flat across iterations with zero variance. Figure 9: Sample output from unsupervised LS policy at each iteration. The LSPI policy is also problem agnostic but learns a separate scalar at each iteration, so its mean varies along the iterations while the variance remains zero. Figure 10: Sample output from unsupervised LSPI at each iteration. The LSTM policy produces a distinct value for each entry, with variance even within the same agent, timestep, and state or control dimension. Figure 11: Sample output from unsupervised LSTM policy at each iteration. Appendix F Overview of the Implicit Differentiation Framework In this appendix, we provide an overview of our IFT-based differentiation framework. For a detailed derivation and proofs of the theorems, consult Appendix G. Our approach is based on differentiating through Subproblem 2 in its barrier-augmented form, which avoids the gradient discontinuities that arise when the active set changes and that compromise training stability. We derive an analogous approach for computing gradients through Subproblem 1 based on [16]. This approach is based on DDP and is favorable because it is more memory-efficient compared to the Pontryagin-based methods [8] and exploits the temporal sparsity of the control problem to compute gradients efficiently by reusing Hessian factorizations from the last iteration of DDP. F.1 Backpropagation Through Time The forward pass of the architecture is given by Algorithm 1. After K iterations, the final iterate K v^K is used to compute the task loss L(K)L( v^K). To compute the hypergradient ∇wL _wL with respect to the policy weights w, we apply the chain rule at iteration a: ∇wL=∂a∂w⊤∇aL, _wL= θ^aw _ θ^aL, (6) where the term ∇aL _ θ^aL captures how the penalty parameters influence the loss through their effect on the solutions to subproblems 1 and 2. Since computing ∇aL _ θ^aL requires differentiating through both subproblems, we introduce adjoint variables δaδ z^a and δ~aδ z^a that satisfy adjoint equations derived from the subproblem KKT conditions. These adjoint variables capture the sensitivity of the loss with respect to the solutions of their respective subproblems. F.2 Barrier-augmentation Approach Subproblem 2 solves an NLP problem with inequality constraints enforcing safety and inter-agent constraints. At convergence, the solution ~a=[~a,~a,~a] z^a=[ x^a, u^a, y^a] satisfies the KKT conditions ∇~aℒ~(~a;~a)=0 _ z^a L( z^a; s^a)=0, where ~a y^a are the Lagrange multipliers for the safety constraints, ℒ~ L is the Lagrangian, and ~a=[a,a−1,a−1,a] s^a=[ z^a, λ^a-1, ξ^a-1, θ^a] collects the inputs to Subproblem 2. Applying the IFT yields ∂~a∂~a=−∇~~2ℒ~−1∇~~2ℒ~. z^a s^a=- _ z z^2 L^-1 _ z s^2 L. (7) The main challenge is computing the adjoint variable δ~a=−∇~~2ℒ~−1∇~aLδ z^a=- _ z z^2 L^-1 _ z^aL efficiently. We avoid instantiating the full Hessian matrix by exploiting the fact that the optimization is distributed across time, allowing us to solve T linear systems in parallel. To obtain smooth gradients, we differentiate through a barrier-augmented form of the subproblem [7] rather than detecting the active set [21, Section 4]. By writing ℓ~t _t for the per-step objective and gt,jg_t,j for the j-th inequality constraint at time t, the barrier-augmented objective is ℒˇt(~t;~t)=ℓ~t(~t;~t)−τ∑jlog(−gt,j(~t;~t)), L_t( xu_t; s_t)= _t( xu_t; s_t)-τ _j \! (-g_t,j( xu_t; s_t) ), (8) with barrier parameter τ>0τ>0. Because every constraint contributes a smooth term to ℒˇt L_t regardless of its activeness, differentiating through (8) avoids the discontinuities incurred when the active set changes. Proposition 1 (Differentiation through Subproblem 2). Let ~t=(~t,~t) xu_t=( x_t, u_t) denote the primal variables at time t, and suppose the forward solve returns a strictly feasible stationary point of (8), i.e. gt,j(~t;~t)<0g_t,j( xu_t; s_t)<0 for all j and ∇~tℒˇt=0 _ xu_t L_t=0, with ∇~t2ℒˇt _ xu_t^2 L_t nonsingular. Then the adjoint variable δ~tδ xu_t is the solution of the linear system ∇~t2ℒˇtδ~t=−∇~tL, _ xu_t^2 L_t\,δ xu_t=- _ xu_tL, (9) where the Hessian on the left-hand side expands as ∇~t2ℒˇt=∇~t2ℓ~t−τ∑j∇~t2gt,jgt,j+τ∑j∇~tgt,j∇~tgt,j⊤gt,j2. _ xu_t^2 L_t= _ xu_t^2 _t-τ _j _ xu_t^2g_t,jg_t,j+τ _j _ xu_tg_t,j _ xu_tg_t,j g_t,j^2. (10) The linear system (9) is of dimension n, where n is the number of variables, so solving the linear system requires O(n3)O(n^3) time complexity and O(n2)O(n^2) memory, independent of the number of constraints. To fully recover the adjoint δ~tδ z_t we would also need the dual adjoint δ~tδ y_t. In our setting this is unnecessary for two reasons. First, the task loss L is not a function of the safety multipliers ~t y_t, so ∇~tL=0 _ y_tL=0. Second, the multipliers do not feed back as inputs to either subproblem, so they carry no sensitivity downstream. Consequently δ~tδ y_t contributes nothing to the hypergradient, and our implementation does not compute it. For completeness, we give its closed form in Corollary 2. Corollary 2 (Recover adjoint variable). Using the barrier multiplier estimate ~t,j=−τ/gt,j y_t,j=-τ/g_t,j, the dual adjoint corresponding to constraint gt,jg_t,j is δ~t,j=τgt,j2∇~tgt,j⊤δ~t,δ y_t,j= τg_t,j^2 _ xu_tg_t,j δ xu_t, and the full dual adjoint δ~tδ y_t is recovered by concatenating the δ~t,jδ y_t,j over j. The barrier approach provides two advantages over active-set detection based methods. First, the gradient is continuous even when the active set changes, avoiding jumps incurred by active-set detection. Second, the barrier parameter τ acts as a tunable knob over the degree of smoothing, trading off the fidelity of gradient against the conditioning of the linear system (9). F.3 Differentiation Through Subproblem 1 Subproblem 1 solves an optimal control problem that is decoupled across agents. At convergence, the solution a=[a,a,a] z^a=[ x^a, u^a, y^a] satisfies the KKT conditions ∇aℒ¯(a;a)=0 _ z^a L( z^a; s^a)=0, where a y^a are the Lagrange multipliers for the dynamics constraints, ℒ¯ L is the Lagrangian, and a=[~a−1,a−1,a−1,a] s^a=[ z^a-1, λ^a-1, ξ^a-1, θ^a] collects all the inputs to Subproblem 1. Applying the IFT yields ∂a∂a=−∇2ℒ¯−1∇2ℒ¯. z^a s^a=- _ z z^2 L^-1 _ z s^2 L. (11) The adjoint variable δa=−∇2ℒ¯−1∇aLδ z^a=- _ z z^2 L^-1 _ z^aL can be computed efficiently without forming the full Hessian by exploiting two key properties of the optimization: (1) the temporal sparsity of the problem allows an efficient backwards-in-time recursion, and (2) the problem is decoupled across agents, allowing for parallel computation. Furthermore, the Hessian factorization from the final DDP iteration of the forward pass can be reused, avoiding expensive matrix refactorizations [16]. Theorem 3 (Differentiation through Subproblem 1). The gradients of the loss with respect to the Subproblem 1 inputs are given by ∇x~i,tL=δxi,t⊙(−i,t),∇λi,tL=δxi,t,∇i,tL=δxi,t⊙(xi,t−x~i,t),∇u~i,tL=δui,t⊙(−i,t),∇ξi,tL=δui,t,∇i,tL=δui,t⊙(ui,t−u~i,t). aligned _ x_i,tL&=δ x_i,t (- ρ_i,t),\\ _ _i,tL&=δ x_i,t,\\ _ ρ_i,tL&=δ x_i,t (x_i,t- x_i,t), aligned aligned _ u_i,tL&=δ u_i,t (- μ_i,t),\\ _ _i,tL&=δ u_i,t,\\ _ μ_i,tL&=δ u_i,t (u_i,t- u_i,t). aligned These expressions reveal an intuitive relationship: the gradient with respect to the penalty parameters is proportional to the primal residual a−~a−1 x^a- x^a-1 weighted by the loss sensitivity δaδ x^a. This provides direct signal for the policies to learn to adapt the penalty parameters for constraint satisfaction in an end-to-end fashion based on the task performance. Appendix G Detailed Derivation of the Implicit Differentiation Framework In this appendix, we provide a detailed derivation of the gradient computation scheme for training Deep Coordinator. The derivation is organized into three parts: (1) the overall backpropagation through one ADMM-DDP iteration, (2) implicit differentiation through Subproblem 2 with proofs of Proposition 1 and Corollary 2, and (3) implicit differentiation through Subproblem 1 with a proof of Theorem 3. G.1 Backpropagation Through One ADMM-DDP Iteration To simplify the notation, let =(,) ζ=( λ, ξ) be the vector of combined dual variables. At each iteration a∈1,…,Ka∈\1,…,K\, the Deep Coordinator forward pass consists of the following operations: 1. Predict hyperparameters via a=πwa(a−1,a−1,) θ^a= _w^a( θ^a-1, v^a-1, χ) (12) 2. Solve Subproblem 1 for dynamically-feasible trajectories via a=(a,a,a)=DDP(a), z^a=( x^a, u^a, y^a)=DDP( s^a), (13) where a y^a are the Lagrange multipliers for the dynamics constraints and a=(~a−1,a−1,a) s^a=( z^a-1, ζ^a-1, θ^a) collects all inputs to Subproblem 1 at iteration a. 3. Solve Subproblem 2 for safe trajectories via ~a=(~a,~a,~a)=NLP(~a), z^a=( x^a, u^a, y^a)=NLP( s^a), (14) where ~a y^a are the Lagrange multipliers for the safety constraints and ~a=(a,a−1,a) s^a=( z^a, ζ^a-1, θ^a) collects all inputs to Subproblem 2 at iteration a. 4. Update dual variables via a=a−1+a⊙((a,a)−(~a,~a)). ζ^a= ζ^a-1+ θ^a (( x^a, u^a)-( x^a, u^a)). (15) After K iterations, the final iterate K=(K,~K,K) v^K=( z^K, z^K, ζ^K) is used to compute the task loss L(K)L( v^K). Our goal is to compute the hypergradient ∇wL _wL to train the feedback policies via gradient descent. Throughout the derivation, we use dfdx fx to denote the total derivative, which includes both direct and indirect relationships through other variables, while ∂f∂x fx denotes the partial derivative where all other variables are held constant. Consider the final iteration a=Ka=K. We consider reverse-mode automatic differentiation (AD) of L with respect to w. By the chain rule, the hypergradient is (∇wL(K))⊤=dLdw=∂L∂K∂K∂w+∂L∂~K∂~K∂w+∂L∂K∂K∂w.( _wL( v^K)) = Lw= L z^K z^Kw+ L z^K z^Kw+ L ζ^K ζ^Kw. (16) As we are performing reverse-mode AD, the order that we compute the derivatives matters for efficiency. First, we differentiate through the dual variable update (15). Since K ζ^K depends on K z^K, ~K z^K, K−1 ζ^K-1, and K θ^K through (15), we expand, yielding ∂K∂w=∂K∂K∂K∂w+∂K∂~K∂~K∂w+∂K∂K−1∂K−1∂w+∂K∂K∂K∂w. ζ^Kw= ζ^K z^K z^Kw+ ζ^K z^K z^Kw+ ζ^K ζ^K-1 ζ^K-1w+ ζ^K θ^K θ^Kw. Substituting into (16) and combining like terms yields dLdw Lw =(∂L∂K+∂L∂K∂K∂K)∂K∂w = ( L z^K+ L ζ^K ζ^K z^K ) z^Kw (17) +(∂L∂~K+∂L∂K∂K∂~K)∂~K∂w+∂L∂K∂K∂K−1∂K−1∂w+∂L∂K∂K∂K∂K∂w. + ( L z^K+ L ζ^K ζ^K z^K ) z^Kw+ L ζ^K ζ^K ζ^K-1 ζ^K-1w+ L ζ^K ζ^K θ^K θ^Kw. (18) Since ~K z^K depends on K z^K and K−1 ζ^K-1, we differentiate through Subproblem 2 next. Define the total derivative of L with respect to ~K z^K as dLd~K=∂L∂~K+∂L∂K∂K∂~K, L z^K= L z^K+ L ζ^K ζ^K z^K, so the second term of (17) can be written more compactly as dLd~K∂~K∂w=∇~KL⊤∂~K∂w L z^K z^Kw= _ z^KL z^Kw. Computing ∂~K∂w=d~Kd~K∂~K∂w z^Kw= z^K s^K s^Kw involves differentiating through Subproblem 2 (14) in order to compute d~Kd~K z^K s^K. Let ℒ~(~;~) L( z; s) denote the Lagrangian for Subproblem 2. At convergence, the solution satisfies the KKT conditions ∇~Kℒ~(~K;~K)=0 _ z^K L( z^K; s^K)=0. Applying the IFT yields d~Kd~K=−∇~K,~Kℒ~−1∇~K,~Kℒ~. z^K s^K=- _ z^K, z^K L^-1 _ z^K, s^K L. Now, we introduce the Subproblem 2 adjoint variable δ~Kδ z^K, which can be computed by solving the linear system δ~K:=−∇~K,~Kℒ~−1∇~KL⇔∇~K,~Kℒ~δ~K=−∇~KL.δ z^K =- _ z^K, z^K L^-1 _ z^KL _ z^K, z^K L\,δ z^K=- _ z^KL. (19) We utilize barrier-augmentation for formulating and solving this linear system efficiently by exploiting the structure of Subproblem 2. Namely, since Subproblem 2 decomposes across time, we solve T independent linear systems in parallel (one per time step). Moreover, barrier-augmentation considers the effects from all constraints altogether, which provide smooth gradients. The derivation is provided in the following subsection. Substituting the adjoint variable (19) into the hypergradient (17) and expanding ~K s^K using the chain rule yields dLdw=(∂L∂K+∂L∂K∂K∂K+(δ~K)⊤∇~K,Kℒ~)∂K∂w Lw= ( L z^K+ L ζ^K ζ^K z^K+(δ z^K) _ z^K, z^K L ) z^Kw (20) +((δ~K)⊤∇~K,K−1ℒ~+∂L∂K∂K∂K−1)∂K−1∂w + ((δ z^K) _ z^K, ζ^K-1 L+ L ζ^K ζ^K ζ^K-1 ) ζ^K-1w +((δ~K)⊤∇~K,θKℒ~+∂L∂K∂K∂K)∂K∂w. + ((δ z^K) _ z^K,θ^K L+ L ζ^K ζ^K θ^K ) θ^Kw. Define the total derivative of L with respect to K z^K as dLdK:=∂L∂K+∂L∂K∂K∂K+(δ~K)⊤∇~K,Kℒ~, L z^K = L z^K+ L ζ^K ζ^K z^K+(δ z^K) _ z^K, z^K L, so the first term of (20) can be written more compactly as dLdK∂K∂w=∇KL⊤∂K∂w L z^K z^Kw= _ z^KL z^Kw. Computing ∂K∂w=dKdK∂K∂w z^Kw= z^K s^K s^Kw involves differentiating through Subproblem 1 (13) in order to compute dKdK z^K s^K. Let ℒ¯(;) L( z; s) denote the Lagrangian for Subproblem 1. At convergence, the solution satisfies the KKT conditions ∇Kℒ¯(K;K)=0 _ z^K L( z^K; s^K)=0. Applying the IFT yields dKdK=−∇K,Kℒ¯−1∇K,Kℒ¯. z^K s^K=- _ z^K, z^K L^-1 _ z^K, s^K L. Next, we introduce the Subproblem 1 adjoint variable δKδ z^K, which can be computed by solving the linear system. δK:=−∇K,Kℒ¯−1∇KL⇔∇K,Kℒ¯δK=−∇KL.δ z^K =- _ z^K, z^K L^-1 _ z^KL _ z^K, z^K L\,δ z^K=- _ z^KL. (21) Our method exploits the agent-wise decomposition of Subproblem 1 and the temporal sparsity of the control problem to compute δKδ z^K by avoiding forming the entire Hessian ∇K,Kℒ¯ _ z^K, z^K L. Each individual agent’s subproblem can be differentiated through by reusing the Hessian factorizations from the DDP forward pass [16, Theorem 5]. Moreover, Theorem 3 provides closed-form expressions for the gradients with respect to all of the Subproblem 1 inputs, with proof provided in the below subsection. These expressions are equivalent to the matrix multiplication (δK)⊤∇K,Kℒ¯(δ z^K) _ z^K, s^K L by expanding the Hessian with respect to each input. Substituting the adjoint variable (21) into the hypergradient (20) and expanding K s^K using the chain rule yields dLdw Lw =(δK)⊤∇K,~K−1ℒ¯∂~K−1∂w =(δ z^K) _ z^K, z^K-1 L z^K-1w +((δ~K)⊤∇~K,K−1ℒ~+∂L∂K∂K∂K−1+(δK)⊤∇K,K−1ℒ¯)∂K−1∂w + ((δ z^K) _ z^K, ζ^K-1 L+ L ζ^K ζ^K ζ^K-1+(δ z^K) _ z^K, ζ^K-1 L ) ζ^K-1w +((δ~K)⊤∇~K,θKℒ~+∂L∂K∂K∂K+(δK)⊤∇K,Kℒ¯)∂K∂w. + ((δ z^K) _ z^K,θ^K L+ L ζ^K ζ^K θ^K+(δ z^K) _ z^K, θ^K L ) θ^Kw. Finally, let ∂L∂K:=(δ~K)⊤∇~K,θKℒ~+∂L∂K∂K∂K+(δK)⊤∇K,Kℒ¯, L θ^K =(δ z^K) _ z^K,θ^K L+ L ζ^K ζ^K θ^K+(δ z^K) _ z^K, θ^K L, and note that K=πwK(K−1,) θ^K= _w^K( v^K-1, χ), so ∂K∂w=∂πwK∂wK∂wK∂w+∂πwK∂K−1∂K−1∂w, θ^Kw= _w^Kw^K w^Kw+ _w^K v^K-1 v^K-1w, where the first term gives the direct contribution of the weights wKw_K for iteration K and the second term propagates gradients backward to iteration K−1K-1. Expanding K−1 v^K-1 yields the final hypergradient: dLdw=∂L∂K∂πwK∂K−1∂K−1∂w+((δK)⊤∇K,~K−1ℒ¯+∂L∂K∂πwK∂~K−1)∂~K−1∂w Lw= L θ^K _w^K z^K-1 z^K-1w+ ((δ z^K) _ z^K, z^K-1 L+ L θ^K _w^K z^K-1 ) z^K-1w +((δ~K)⊤∇~K,K−1ℒ~+∂L∂K∂K∂K−1+(δK)⊤∇K,K−1ℒ¯+∂L∂K∂πwK∂K−1)∂K−1∂w + ((δ z^K) _ z^K, ζ^K-1 L+ L ζ^K ζ^K ζ^K-1+(δ z^K) _ z^K, ζ^K-1 L+ L θ^K _w^K ζ^K-1 ) ζ^K-1w +∂L∂K∂πwK∂wK∂wK∂w. + L θ^K _w^Kw^K w^Kw. To simplify the notation, define the accumulated partial derivatives through iteration K: ∂L∂K−1 L z^K-1 :=∂L∂K∂πwK∂K−1, = L θ^K _w^K z^K-1, ∂L∂~K−1 L z^K-1 :=(δK)⊤∇K,~K−1ℒ¯+∂L∂K∂πwK∂~K−1, =(δ z^K) _ z^K, z^K-1 L+ L θ^K _w^K z^K-1, ∂L∂K−1 L ζ^K-1 :=(δ~K)⊤∇~K,K−1ℒ~+∂L∂K∂K∂K−1+(δK)⊤∇K,K−1ℒ¯+∂L∂K∂πwK∂K−1, =(δ z^K) _ z^K, ζ^K-1 L+ L ζ^K ζ^K ζ^K-1+(δ z^K) _ z^K, ζ^K-1 L+ L θ^K _w^K ζ^K-1, ∂L∂wK Lw^K :=∂L∂K∂πwK∂wK. = L θ^K _w^Kw^K. These derivatives capture how K−1 z^K-1, ~K−1 z^K-1, K−1 ζ^K-1, and wKw^K affect the loss through the final iteration (iteration K) of Deep Coordinator. Using these definitions, the hypergradient simplifies to dLdw=∂L∂K−1∂K−1∂w+∂L∂~K−1∂~K−1∂w+∂L∂K−1∂K−1∂w+∂L∂wK∂wK∂w. Lw= L z^K-1 z^K-1w+ L z^K-1 z^K-1w+ L ζ^K-1 ζ^K-1w+ Lw^K w^Kw. This is equivalent to the initial hypergradient presented in (16), except the backpropagation is now through iteration a=K−1a=K-1. The process repeats recursively from a=K,K−1,…,1a=K,K-1,…,1, accumulating gradients via backpropagation through time. G.2 Implicit Differentiation Through Subproblem 2 G.2.1 Barrier Augmentation We now prove Proposition 1, which gives the per-timestep linear system for the primal adjoint δ~tδ xu_t. Since the Subproblem 2 is separable across time, the adjoint system decouples into T independent per-step system solved in parallel. The forward solve returns a strictly feasible point gt,j<0g_t,j<0, which we treat as a stationary point of (8) and impose the stationarity condition for every t: ∇~tℒˇt=∇~tℓ~t−τ∑j(∇~tgt,jgt,j)=0. _ xu_t L_t= _ xu_t _t-τ _j ( _ xu_tg_t,jg_t,j )=0. (22) To compute δ~tδ xu_t, we solve the following linear system: ∇~t2ℒˇtδ~t=−∇~tL, _ xu_t^2 L_tδ xu_t=- _ xu_tL, (23) where the Hessian on the left-hand side expands as ∇~t2ℒˇt=∇~t2ℓ~t−τ∑j∇~t2gt,jgt,j+τ∑j∇~tgt,j∇~tgt,j⊤gt,j2. _ xu_t^2 L_t= _ xu_t^2 _t-τ _j _ xu_t^2g_t,jg_t,j+τ _j _ xu_tg_t,j _ xu_tg_t,j g_t,j^2. (24) As a constraint becomes nearly active, gt,jg_t,j is small in magnitude, so the last term scales as τ/gt,j2τ/g_t,j^2 and dominates the Hessian, leaving it poorly conditioned. Moreover, for nonconvex constraints the term ∇~t2gt,j _ xu_t^2g_t,j may render ∇~t2ℒˇt _ xu_t^2 L_t indefinite. To prevent this ill-conditioning and mitigate possible indefiniteness, in practice, we regularize the Hessian and instead solve (∇~t2ℒˇt+δI)δ~t=−∇~tL,( _ xu_t^2 L_t+δ I)δ xu_t=- _ xu_tL, (25) where δ>0δ>0 is set to 10−510^-5 in our implementation. Note that the source of ill-conditioning here differs from the active-set method. Rather than the rank deficiency caused by linearly dependent active constraints, it stems from the large curvature of nearly-active barrier terms and the possible indefiniteness of nonconvex constraints. In addition, τ controls how much the transition between active sets is smoothed; we set it to 10−610^-6 for training. G.2.2 Recover Dual Adjoint We now derive Corollary 2, which recovers the dual adjoint δ~tδ y_t from the primal adjoint δ~tδ xu_t. Under barrier augmentation the inequality multipliers are not independent variables. Actually, at convergence, they are tied to the primal solution through the perturbed complementarity relation ~t,j=−τ/gt,j, y_t,j=-τ/g_t,j, where ~t,j y_t,j is the barrier multiplier estimate for the j-th constraint gt,jg_t,j. Since this expresses ~t,j y_t,j as a function of ~t xu_t through gt,j(~t)g_t,j( xu_t), differentiating both sides with respect to ~t xu_t and contracting with the primal adjoint δ~tδ xu_t gives δ~t,j=∇~t(−τgt,j)⊤δ~t=τgt,j2∇~tgt,j⊤δ~t,δ y_t,j= _ xu_t\! (- τg_t,j )^\! \!δ xu_t= τg_t,j^2\, _ xu_tg_t,j \,δ xu_t, and the full dual adjoint δ~tδ y_t is recovered by concatenating δ~t,jδ y_t,j over j. Because δ~tδ y_t follows in closed form once δ~tδ xu_t is known, no separate dual linear system is required. As previously noted, δ~tδ y_t does not enter the hypergradient and is reported only for completeness. G.3 Implicit Differentiation Through Subproblem 1 Finally, we prove Theorem 3, which gives closed-form expressions for the gradients of L with respect to the Subproblem 1 inputs. Proof. Recall that for Subproblem 1, each agent solves a control problem with the form minx,u _x,u ∑t=0T−1(ℓ(xt,ut)+12xt⊤(ρtI)xt+(λt−ρt⊙x~t)⊤xt _t=0^T-1 ( (x_t,u_t)+ 12x_t ( _tI)x_t+( _t- _t x_t) x_t +12ut⊤(μtI)ut+(ξt−μt⊙u~t)⊤ut) + 12u_t ( _tI)u_t+( _t- _t u_t) u_t ) +ℓT(xT)+12xT⊤(ρTI)xT+(λT−ρT⊙x~T)⊤xT, + _T(x_T)+ 12x_T ( _TI)x_T+( _T- _T x_T) x_T, subject to xt+1=f(xt,ut),x0=x¯0, x_t+1=f(x_t,u_t), x_0= x_0, where we have simplified the quadratic penalty terms and dropped the agent index i for clarity. Let ℒ¯(x,u,y;s) L(x,u,y;s) denote the Lagrangian for this optimization, where recall s=(x~,u~,λ,ξ,ρ,μ)s=( x, u,λ,ξ,ρ,μ). The optimal solution z∗=(x∗,u∗,y∗)z^*=(x^*,u^*,y^*) satisfies the KKT conditions ∇zℒ¯(z∗;s)=0 _z L(z^*;s)=0, namely ∇y0ℒ¯ _y_0 L =x¯0−x0∗=0, = x_0-x_0^*=0, ⋮ 5.0pt ∇xtℒ¯ _x_t L =∇xℓt(xt∗,ut∗)+ρt⊙(xt∗−x~t)+λt+∂ft∂xt⊤yt+1∗−yt∗=0, = _x _t(x_t^*,u_t^*)+ _t (x_t^*- x_t)+ _t+ f_tx_t y_t+1^*-y_t^*=0, ∇utℒ¯ _u_t L =∇uℓt(xt∗,ut∗)+μt⊙(ut∗−u~t)+ξt+∂ft∂ut⊤yt+1∗=0, = _u _t(x_t^*,u_t^*)+ _t (u_t^*- u_t)+ _t+ f_tu_t y_t+1^*=0, ∇yt+1ℒ¯ _y_t+1 L =f(xt∗,ut∗)−xt+1∗=0, =f(x_t^*,u_t^*)-x_t+1^*=0, ⋮ 5.0pt ∇xTℒ¯ _x_T L =∇ℓT(xT∗)+ρT⊙(xT∗−x~T)+λT−yT∗=0. =∇ _T(x_T^*)+ _T (x_T^*- x_T)+ _T-y_T^*=0. Given ∇zL _zL, reverse-mode AD through this optimization involves computing the adjoint vector δz=−∇zzℒ¯−1∇zLδ z=- _z L^-1 _zL, as discussed above. This linear system can be solved efficiently through a single DDP iteration that reuses the Hessian factorizations from the last iteration of DDP from the forward pass [16, Theorem 5]. This vector is then multiplied by the Hessian ∇zsℒ¯ _zs L yielding ∇sL=∇zsℒ¯⊤δz _sL= _zs L δ z. The individual blocks of the Hessian ∇zsℒ¯ _zs L are given by ∇xt,x~tℒ¯=diag(−ρt),∇xt,λtℒ¯=I,∇xt,ρtℒ¯=diag(xt−x~t),∇ut,u~tℒ¯=diag(−μt),∇ut,ξtℒ¯=I,∇ut,μtℒ¯=diag(ut−u~t), aligned _x_t, x_t L&=diag(- _t),\\ _x_t, _t L&=I,\\ _x_t, _t L&=diag(x_t- x_t),\\ aligned aligned _u_t, u_t L&=diag(- _t),\\ _u_t, _t L&=I,\\ _u_t, _t L&=diag(u_t- u_t), aligned (26) which can be derived straightforwardly by noting that each input (x~t x_t, λt _t, ρt _t, etc.) at time t only depends on its respective state/control at time t. Multiplying each of these terms by the respective elements of δzδ z yields (26). ∎ Appendix H Details on Experimental Setup H.1 Details on Problem Types In this appendix, we include details on problem generation and agent dynamics. Note that we warm-start Subproblem 2 with the solution from Subproblem 1. Also, for fast computation of Subproblem 2, we linearize all constraints at current nominal trajectory making the problem a linearly-constrained quadratic programming problem. H.1.1 Car Obstacle Field We first consider a multi-agent avoidance task with a randomized obstacle field. Each agent has Dubins vehicle dynamics, with the state given by (xi,t,yi,t,θi,t)∈ℝ3(x_i,t,y_i,t, _i,t) ^3 and controls given by (vi,t,ωi,t)∈ℝ2(v_i,t, _i,t) ^2, where (xi,t,yi,t)(x_i,t,y_i,t) is the agent’s position, θi,t _i,t is its orientation, vi,tv_i,t is its linear forward velocity, and ωi,t _i,t is its angular velocity. The dynamics are Euler-discretized with timestep dt=0.1dt=0.1. Each agent has quadratic cost ℓt(i,t,i,t) _t( x_i,t, u_i,t) =12i,t⊤ii,t+12i,t⊤ii,t = 12 x_i,t Q_i x_i,t+ 12 u_i,t R_i u_i,t (27) ℓT(i,T) _T( x_i,T) =12i,T⊤ifi,T = 12 x_i,T Q_i^f x_i,T with i=3 Q_i= I_3, i=diag(0.3,0.3) R_i=diag(0.3,0.3) and if=100⋅i Q_i^f=100· Q_i, where 3∈ℝ3×3 I_3 ^3× 3 is the identity matrix. The controls are subject to −vmax≤vi,t≤vmax-v_ ≤ v_i,t≤ v_ and −ωmax≤ωi,t≤ωmax - _ ≤ _i,t≤ _ , where vmax=5v_ =5 m/s and ωmax=5 _ =5 rad/s. Each vehicle must also satisfy the circular obstacle constraints defined by ‖(xi,t,yi,t)−‖2≥rcirc+dsafe, (x_i,t,y_i,t)- c _2≥ r_circ+d_safe, (28) where =(c1,c2) c=(c_1,c_2) is the center of the obstacle, rcirc>0r_circ>0 is its radius, and dsafe>0d_safe>0 is the desired safety distance. The agents must also abide by the inter-agent distance constraints ‖(xi,t,yi,t)−(xj,t,yj,t)‖2≥dsafe∀(i,j),t. (x_i,t,y_i,t)-(x_j,t,y_j,t) _2≥ d_safe\ ∀(i,j),t. (29) We consider N=15N=15 agents initialized in a stationary grid and a target formation across the obstacle field. The agents must navigate to the target formation across a field of 3 circular obstacles with randomized radii and centers. H.1.2 Car Intersection Next, we consider a more complex task where both the initial and target positions of the agents are randomized, requiring significantly different solutions for each problem instance. We consider N=8N=8 agents with Dubins vehicle dynamics discretized in dt=0.05dt=0.05 and random initial conditions chosen from one of 16 fixed locations at a four-way road intersection. The remaining 8 positions are then randomly assigned as targets. The agents must navigate the intersection without running off the road or colliding to reach their target. The quadratic cost (27) is chosen with parameters i=diag(1,1,0) Q_i=diag(1,1,0), i=diag(0.1,0.1) R_i=diag(0.1,0.1) and if=100⋅3 Q_i^f=100· I_3 imposed for each agent. The control box constraints are set to vmax=5v_ =5 m/s and ωmax=1 _ =1 rad/s. We center the intersection at ∈ℝ2 c ^2, where each roadway j∈1,2j∈\1,2\ has a heading j θ_j and width wj>0w_j>0. Road j is the set of points whose lateral distance to the road centerline is at most wj/2w_j/2. Thus, for an agent i, its road membership constraint with respect to road j is defined by si,tj=|(j)⊤(i,t−)|−(wj2−dsafe)≤0,s_i,t^j=| n( θ_j) ( p_i,t- c)|- ( w_j2-d_safe )≤ 0, (30) where (j)=(−sinj,cosj)∈ℝ2 n( θ_j)=(- θ_j, θ_j) ^2. The intersection constraint is then given by softminβ(si,t1,si,t2)≤0,softmin_β(s_i,t^1,s_i,t^2)≤ 0, (31) where β is the reciprocal of the temperature of the softmin function. For our problem setup, we set =(0,0) c=(0,0), 1=0 θ_1=0, 2=π/2 θ_2=π/2, w1=w2=10w_1=w_2=10, and β=100β=100. H.1.3 Quadrotor Obstacle Field Finally, we deploy Deep Coordinator on multi-agent quadrotor maneuvering problems. We adopt the dynamics model from [17] where the state is given by the positions i,t p_i,t, body velocities i,t v_i,t, Euler angles i,t θ_i,t, and body rates i,t _i,t of the quadrotor, and the controls are the collective thrust Fi,tF_i,t and torques τi,tx,τi,ty,τi,tz _i,t^x, _i,t^y, _i,t^z. We discretize the dynamics using the Euler method with timestep dt=0.1dt=0.1. The problem consists of N=10N=10 agents and a time horizon of T=100T=100 timesteps. Each agent has a quadratic cost (27) with i=bdiag(3,0.13,0.73,0.13) Q_i=bdiag( I_3,0.1 I_3,0.7 I_3,0.1 I_3), i=4 R_i= I_4, and if=100⋅i Q_i^f=100· Q_i. The control limits for each quadrotor are Fmin≤Fi,t≤Fmax,−τmax≤τi,tx,τi,ty,τi,tz≤τmax,F_ ≤ F_i,t≤ F_ , - _ ≤ _i,t^x, _i,t^y, _i,t^z≤ _ , where Fmin=5F_ =5, Fmax=15F_ =15 N, and τmax=0.2 _ =0.2 N⋅·m. We use constraints identical to (28) to enforce cylindrical obstacle constraints and impose inter-agent distance constraints by extending (29) to ℝ3R^3. The drones are initialized with random altitude displacements pi,0zp_i,0^z and aim to reach their designated target state while navigating across a field with 7 cylindrical obstacles with randomized radii and centers. To maximize interactions between agents, we set target states in a grid such that the agents initialized farthest from the obstacle are assigned the most distant targets. H.2 Details on Scaled Problem Types In this appendix, we discuss how the problems for scaling experiments are generated. We let M denote the scaling factor relative to the training problems. H.2.1 Scaled Car Obstacle Field For each problem, we randomly sample an obstacle field containing 2M+12M+1 circular obstacles. The obstacle centers coc_o have y-positions uniformly spaced along [−12M−9.5, 12M+2.5][-12M-9.5,\,12M+2.5]; the offset of 2.5 is added to ensure that more agents are initialized at the same y-coordinate as the obstacle centers, as was the case in the original task, which forces agents to break the symmetry when passing the obstacle. The x-positions and radii are sampled independently via co,x∼([−7, 7])c_o,x ([-7,\,7]) and ro∼([2.75, 4.75])r_o ([2.75,\,4.75]), similarly to the original task. Again, a safety radius of 1.0 is enforced. Problems contain 15M15M Dubins vehicle agents, initially arranged in a 5M×35M× 3 grid where the agent in row n and column m has position pinit,mn=(−25+5m,−2.5−10M+5n) p_ init,mn=(-5+5m,-5-0M+5n) for n=0,…,5M−1n=0,...,5M-1 and m=0,…,2m=0,...,2. The agents begin with an initial heading of 0 rad. Each agent’s target state is again xtarget,i=xinit,i+(40,0,0)x_ target,i=x_ init,i+(40,0,0). Obstacle constraints are enforced between all obstacles and agents. To minimize the number of inter-agent constraints, each agent only imposes these constraints on the agents initialized in its own row, the four rows above it, and the four rows below it. Figure 12: Car obstacle field scaling experiments. Deep Coordinator trained on N=15N=15 agents and deployed to N=60N=60 agents. In these scaled car obstacle field problems, scaling is nearly isomorphic. In other words, each agent observes a structurally similar local problem at each scaling factor even as the total number of agents increases. Thus, this setting enables us to analyze whether Deep Coordinator produces transferable coordination rules rather than memorizing the training problem size. H.2.2 Scaled Car Intersection For each problem, we employ the roadway constraint described in Appendix H.1.2 with c=(0,0),θ1=0,θ2=π/2,w1=w2=10c=(0,0),\, _1=0,\, _2=π/2,\,w_1=w_2=10 and β=100β=100. We similarly generate the initial and target state sets as inits _ inits =⋃d∈(−d,±h,0),(±h,−d,−π/2),(d,±h,π),(±h,d,π/2), = _d \(-d,± h,0),(± h,-d,-π/2),(d,± h,π),(± h,d,π/2)\, (32a) targets _ targets =⋃d∈(−d,±h,π),(±h,−d,π/2),(d,±h,0),(±h,d,−π/2), = _d \(-d,± h,π),(± h,-d,π/2),(d,± h,0),(± h,d,-π/2)\, (32b) where =8.0,10.0,12.0D=\8.0,10.0,12.0\ when N=12N=12, and =8.0,9.0,10.0,11.0D=\8.0,9.0,10.0,11.0\ when N=16N=16. We then sample a subset of initial and target states uniformly at random without replacement, with target positions distinct from all sampled initial positions. Then, we perturb the sampled initial and target positions of each agent to get the final states xinit,i x_ init,i =x¯init,i+vi,vi∼(3,σbdiag(2,1)), = x_ init,i+v_i,\ v_i ( 0_3, ( I_2, 0_1)), (33a) xtarget,i x_ target,i =x¯target,i+wi,wi∼(3,σbdiag(2,1)), = x_ target,i+w_i,\ w_i ( 0_3, ( I_2, 0_1)), (33b) where σ=0.1σ=0.1, 2∈ℝ2×2 I_2 ^2× 2 is the identity matrix, and 3∈ℝ3 0_3 ^3, 1∈ℝ 0_1 are zero vectors. Figure 13: Car intersection scaling experiments. Deep Coordinator trained on N=8N=8 agents and deployed to N=16N=16 agents. Unlike the obstacle field problems, the larger intersection problems are not locally isomorphic to smaller problems. We increase the density of the agents, introducing more competing right-of-way decisions and tighter coupling between agents. H.2.3 Scaled Quadrotor Obstacle Field For each problem, we randomly sample an obstacle field containing 7M7M cylindrical obstacles. The obstacle centers coc_o have y-positions uniformly spaced along [−18M, 18M][-18M,\,18M] and x-positions and radii are sampled independently via co,x∼([−7, 7])c_o,x ([-7,\,7]) and ro∼([1.5, 2])r_o ([1.5,\,2]), similarly to the original task. Again, a safety radius of 1.0 is enforced. Problems contain 10M10M quadrotor agents, initially arranged in a 5M×25M× 2 grid where the agent in row n and column m has x,yx,y position (px,py)init,mn=(−20+5m,−2.5−10M+5n) (p^x,p^y)_ init,\,mn=(-0+5m,-5-0M+5n) for n=0,…,5M−1n=0,...,5M-1 and m=0,1m=0,1. The initial z position of each quadrotor is independently sampled via pinit,iz∼([−12, 12])p_ init,\,i^z ([-12,\,12]) and all other state variables are again initialized to the hovering condition. Each agent’s target state is given by ptarget,i=pinit,i⊙(−1,1,…,1)p_ target,\,i=p_ init,\,i (-1,1,...,1). The target z position for all agents is set to zero, and the hovering condition is enforced at the target state. Obstacle constraints are enforced between all obstacles and agents. To minimize the number of inter-agent constraints, each agent only imposes these constraints on the agents initialized in its own row, the four rows above it, and the four rows below it. Similar to the scaled car obstacle field problems, scaling is nearly isomorphic in our scaled quadrotor obstacle field problems. Figure 14: Quadrotor obstacle field scaling experiments. Deep Coordinator trained on N=10N=10 agents and deployed to N=80N=80 agents. H.3 LSTM Architecture In this appendix, we describe the LSTM feedback architecture used in the experiments. Recall that we learn policies πwa _w^a which adapt hyperparameters a=[a,a] θ^a=[ ρ^a, μ^a] at every network layer via a=πwa(a−1,a−1,). split θ^a= _w^a( θ^a-1, v^a-1, χ). split (34) In the version of Deep Coordinator parameterized by an LSTM, we employ the shared per-agent, per-timestep feedback scheme. We predict ρ and μ using two different policies with identical architectures but different weights wρw_ρ and wμw_μ. This structure ensures that Deep Coordinator learns policies that can generalize to larger problems but are also aware of the different roles played by ρ and μ. To minimize the number of weights, wρw_ρ and wμw_μ are shared across all ADMM-DDP iterations. The policies πwρ _w_ρ and πwμ _w_μ are executed for each agent i, timestep t, and state or control dimension d to generate the relevant component of the penalty vector. Each instantiation of the policy is given 14 inputs derived from the previous hyperparameters θa−1θ^a-1, the previous iterate a−1 v^a-1 and the problem context χ. The first three inputs are the relevant components of the dual variable and ADMM residuals. Formally, these inputs may be expressed as i,t,da−1 λ^a-1_i,t,d , (35) prim,x,i,t,da−1 r_ prim,x,i,t,d^a-1 =i,t,da−1−~i,t,da−1, = x^a-1_i,t,d- x^a-1_i,t,d, (36) dual,x,i,t,da−1 r_ dual,x,i,t,d^a-1 =i,t,da−1(~i,t,da−1−~i,t,da−2), = ρ^a-1_i,t,d( x^a-1_i,t,d- x^a-2_i,t,d), (37) for πwρa _w_ρ^a, and i,t,da−1 ξ^a-1_i,t,d , (38) prim,u,i,t,da−1 r_ prim,u,i,t,d^a-1 =i,t,da−1−~i,t,da−1, = u^a-1_i,t,d- u^a-1_i,t,d, (39) dual,u,i,t,da−1 r_ dual,u,i,t,d^a-1 =i,t,da−1(~i,t,da−1−~i,t,da−2), = μ^a-1_i,t,d( u^a-1_i,t,d- u^a-2_i,t,d), (40) for πwμa _w_μ^a. Both policies then receive an input representing the global average state constraint violation G(a−1,a−1)=1M∑gi,t∈max0,gi,t(i,ta−1,i,ta−1)2+1M∑hij,t∈max0,hij,t(i,ta−1,i,ta−1,j,ta−1,j,ta−1)2 splitG( x^a-1, u^a-1)&= 1M _g_i,t∈ χ \0,g_i,t( x_i,t^a-1, u_i,t^a-1)\^2\\ & + 1M _h_ij,t∈ χ \0,h_ij,t( x_i,t^a-1, u_i,t^a-1, x_j,t^a-1, u_j,t^a-1)\^2 split (41) where M is the total number of constraints, gi,tg_i,t are the single-agent constraints for agent i at time step t, and hij,th_ij,t are the inter-agent constraints for agents i and j at timestep t. By providing the constraint violation as an input to πwa _w^a, we enable our policy to adapt the penalty parameters to both indirect (, λ, ξ) and direct (gi,t,hij,tg_i,t,h_ij,t) metrics regarding trajectory safety. Finally, both policies also receive a positional-encoding vector PEa∈ℝ10PE^a ^10, enabling the feedback functions to adapt their behavior at each of the K unfolded ADMM-DDP iterations while keeping shared weights across all layers. We use the transformer-style positional encoding where entry l is given by (PEa)l=sin(a2l/10)if l is even,cos(a2l/10)if l is odd.(PE^a)_l= cases ( a2^l/10 )&if $l$ is even,\\ ( a2^l/10 )&if $l$ is odd. cases (42) Each of these inputs is then concatenated into the vectors [i,t,da−1,prim,x,i,t,da−1,dual,x,i,t,da−1,G(a−1,a−1),PEa], [ λ^a-1_i,t,d,\, r_ prim,x,i,t,d^a-1,\, r_ dual,x,i,t,d^a-1,\,G( x^a-1,\, u^a-1),PE^a], (43a) [i,t,da−1,prim,u,i,t,da−1,dual,u,i,t,da−1,G(a−1,a−1),PEa]. [ ξ^a-1_i,t,d,\, r_ prim,u,i,t,d^a-1,\, r_ dual,u,i,t,d^a-1,\,G( x^a-1, u^a-1),\,PE^a]. (43b) We parameterize πwρ _w_ρ, πwμ _w_μ as recurrent neural networks consisting of a single LSTM cell at each layer. At each unrolled iteration of ADMM-DDP, the LSTM receives the input vector of dimension din=14d_in=14 and maintains a hidden state and cell state of dimension dh=32d_h=32. To compute the final penalty parameter value, the output of the LSTM is transformed by applying a multilayer perceptron (MLP) to the hidden state. The MLP consists of three fully-connected hidden layers of size 32 with sigmoid activation functions, σ(z)=11+e−z,σ(z)= 11+e^-z, (44) applied between layers and an exp(⋅) (·) activation applied to the final layer. This design decouples temporal representation learning, handled by the LSTM, from output regression, handled by the MLP. The final output dimension from the MLP is dout=1d_ out=1, enabling our policy to remain agnostic of problem dimensions by batching over the agent, time, and state or control dimension. Lastly, we scale the output from the LSTM architecture by learned scaling parameters γρa _ρ^a and γμa _μ^a. We initialize these parameters to match the tuning of Vanilla ADMM-DDP to ensure Deep Coordinator does not converge to the degenerate behavior of producing near-zero a ρ^a and a μ^a at every iteration. H.4 Baseline Solvers H.4.1 Vanilla ADMM-DDP Vanilla ADMM-DDP refers to ADMM-DDP using constant hyperparameters across all iterations. To tune Vanilla ADMM-DDP, we select the lowest values of ρ,μ∈…,10,20,30,…,90,100,200,300,…ρ,μ∈\...,10,20,30,...,90,100,200,300,...\ that yield trajectories which are constraint-satisfying and reach the target after D iterations, where D is the number of iterations used to generate the dataset. This yields ρ=300,μ=100ρ=300,μ=100 for car obstacle field, ρ=500,μ=400ρ=500,μ=400 for car intersection, and ρ=200,μ=400ρ=200,μ=400 for quadrotor obstacle field. H.4.2 Adaptive ADMM-DDP Adaptive ADMM-DDP refers to ADMM-DDP which adapts penalty parameters a ρ^a and a μ^a using the ADMM adaptation rule, i,ta=i,ta−1χadaptif ∥prim,x,i,ta−1∥2≥σadapt∥dual,x,i,ta−1∥2,i,ta−1/χadaptif ∥dual,x,i,ta−1∥2≥σadapt∥prim,x,i,ta−1∥2, ρ_i,t^a= cases ρ_i,t^a-1 _ adapt&if r_ prim,x,i,t^a-1 _2≥ _ adapt r_ dual,x,i,t^a-1 _2,\\ ρ_i,t^a-1/ _ adapt&if r_ dual,x,i,t^a-1 _2≥ _ adapt r_ prim,x,i,t^a-1 _2, cases (45) where χadapt>0 _ adapt>0 and σadapt>0 _ adapt>0, and a μ^a is adapted similarly. We initialize the penalty parameters to be identical to those used in Vanilla ADMM-DDP and tune the adaptive scaling terms χadapt _ adapt and σadapt_ _ adapt until the adaptive solver is numerically stable with minimal constraint violation over 30 iterations. This yields χadapt=1.2 _ adapt=1.2 and σadapt=10 _ adapt=10. H.5 Training Setup For both the supervised and unsupervised training schemes, we train policies on 80 problems split into 4 batches, and select the model which shows minimum loss value within 50 epochs. For training, we utilize the AdamW [11] optimizer with the weight decay set to 10−310^-3. The learning rate γrate _rate is tuned separately for each (training scheme, task, policy) tuple. H.5.1 Supervised Training For supervised training, the ground truth solution ∗(m) v^*( χ_m) is obtained by allowing Vanilla ADMM-DDP as described in H.4.1 to run until convergence. For the car intersection and quadrotor obstacle field task, Deep Coordinator is trained with the following supervised loss on all iterates Lj(aa=1K)=∑a=1Kγa‖a−∗(j)‖22,L_j(\ v^a\_a=1^K)= _a=1^Kγ^a|| v^a- v^*( χ_j)||_2^2, (46) where a v^a is the iterate produced by Deep Coordinator at iteration a and γaγ^a is a scaling parameter. The scaling parameter is included to equalize the contributions from all iterates, as later iterates exhibit smaller distances. Under the control-theoretic interpretation of deep-unfolding, these intermediate costs can be interpreted as stage costs that encourage the optimizer to find a smooth path to the optimal solution. In our experiments, we set γa=exp(a+1−K/4)γ^a= ( a+1-KK/4). However, the benefit of considering the intermediate iterates does not hold for all training tasks. In particular, Deep Coordinator models trained on the car obstacle field task exhibited improved performance when trained with the following supervised loss which considers only the final iterate L(K)=‖K−∗(j)‖22,L( v^K)=|| v^K- v^*( χ_j)||_2^2, (47) where K v^K is the iterate at the final iteration. Importantly, for both choices of supervised loss, we evaluate the norm over the dual variables , λ, ξ in addition to the primal variables. The dual variables capture how difficult it is to satisfy the constraints, and thus play an important role in shaping the optimizer’s dynamics. Including , λ, ξ in the loss allows the learned optimizer to take this coupling information into account, improving generalizability and convergence towards the optimal solution ∗ v^*. We tuned learning rate γrate _rate for each policy by sweeping values in …,1×10−3,2×10−3,5×10−3,1×10−2,2×10−2,5×10−2,…\...,1× 10^-3,2× 10^-3,5× 10^-3,1× 10^-2,2× 10^-2,5× 10^-2,...\. Task Parameter LS LSPI LSTM Car Obstacle Field γrate _rate 0.10.1 0.050.05 0.0050.005 Car Intersection γrate _rate 0.10.1 0.040.04 0.0020.002 Quadrotor Obstacle Field γrate _rate 0.10.1 0.10.1 0.0020.002 Table 1: Tuned parameters of supervised scheme for each policy. H.5.2 Unsupervised Training For unsupervised training, we utilize the same 80 environments as in supervised training, though the ground-truth solution ∗(m) v^*( χ_m) is not necessary in this case. By adjusting γcost _cost and γconst _const, we can control the trade-off between constraint satisfaction and cost minimization. However, it is not necessary to tune all three of γcost, _cost, γconst, _const, and γrate _rate, since the latter acts as a scaling factor for the two other terms. Instead, we fix γconst=2×107 _const=2× 10^7 and coarsely sweep γcost _cost over values in …, 0.015, 0.05, 0.15, 0.5,…\...,\,0.015,\,0.05,\,0.15,\\ \,0.5,\,...\ and the learning rate γrate _rate over values in …,1×10−3,2×10−3,5×10−3,1×10−2,2×10−2,5×10−2,1×10−1,2×10−1,5×10−1,…\...,1× 10^-3,2× 10^-3,5× 10^-3,1× 10^-2,2× 10^-2,5× 10^-2,1× 10^-1,2× 10^-1,5× 10^-1,...\. Note that the resulting performance is fairly robust to the precise choice of γcost _cost and γconst _const, as indicated by the coarseness of the sweep. Task Parameter LS LSPI LSTM Car Obstacle Field γrate _rate 0.20.2 0.050.05 0.010.01 γcost _cost 0.050.05 0.0150.015 0.0150.015 γconst _const 2×1072\!×\!10^7 2×1072\!×\!10^7 2×1072\!×\!10^7 Car Intersection γrate _rate 0.10.1 0.050.05 0.0020.002 γcost _cost 0.150.15 0.0150.015 0.050.05 γconst _const 2×1072\!×\!10^7 2×1072\!×\!10^7 2×1072\!×\!10^7 Quadrotor Obstacle Field γrate _rate 0.050.05 0.050.05 0.0020.002 γcost _cost 0.050.05 0.050.05 0.0150.015 γconst _const 2×1072\!×\!10^7 2×1072\!×\!10^7 2×1072\!×\!10^7 Table 2: Tuned parameters of unsupervised scheme for each policy. Appendix I Detailed Experimental Results I.1 Cost and Constraint Violation Over an Extended Wall-Clock Time Fig. 15-17 show the cost and maximum constraint violation of Deep Coordinator and the baseline models on 20 unseen, training-scale problems over an extended wall-clock time. Since the wall-clock time required to complete an iteration of ADMM-DDP may vary between problem instances, the values are sub-sampled at discrete time intervals. The horizontal dashed lines represent the value found by the model at K=30K=30 iterations. Adaptive ADMM-DDP is truncated at 30 iterations as it frequently produces numerically unstable subproblems when run for longer, causing the optimization process to fail. All graphs end when Vanilla ADMM-DDP converges on all problems. Figure 15: Results of the car obstacle field task over time on 20 unseen test instances. Figure 16: Results of the car intersection task over time on 20 unseen test instances. Figure 17: Results of the quadrotor obstacle field task over time on 20 unseen test instances. I.2 Detailed Numerical Results of Scaling Experiments Task N Metric ADMM-DDP Deep Coordinator Vanilla Adaptive LS LSPI LSTM A 15 Final Cost 1235076.81235076.8 305770.9 305770.9 346303.4346303.4 450818.9450818.9 344902.0¯ 344902.0 Max Const. 0.3950.395 2.52.5 0.2420.242 0.061 0.061 0.073¯ 0.073 Speed-up - - 9.35¯ 9.35 4.834.83 9.44 9.44 30 Final Cost 2453478.62453478.6 610535.4 610535.4 691183.5691183.5 892261.2892261.2 687550.9¯ 687550.9 Max Const. 0.4170.417 2.42.4 0.2970.297 0.057 0.057 0.100¯ 0.100 Speed-up - - 9.62¯ 9.62 4.994.99 9.78 9.78 60 Final Cost 4909243.04909243.0 1311819.8 1311819.8 1383651.41383651.4 1786253.31786253.3 1378203.1¯ 1378203.1 Max Const. 0.5150.515 2.82.8 0.3600.360 0.067 0.067 0.125¯ 0.125 Speed-up - - 9.67¯ 9.67 5.095.09 9.84 9.84 B 8 Final Cost 156365.7156365.7 32893.1 32893.1 115555.8115555.8 64493.664493.6 55195.8¯ 55195.8 Max Const. 0.1000.100 1.31.3 0.0950.095 0.076 0.076 0.081¯ 0.081 Speed-up - - 1.871.87 5.07¯ 5.07 6.18 6.18 12 Final Cost 191742.3191742.3 39049.7 39049.7 144439.9144439.9 76969.776969.7 72047.7¯ 72047.7 Max Const. 0.1520.152 1.31.3 0.1620.162 0.099¯ 0.099 0.092 0.092 Speed-up - - 1.931.93 5.47¯ 5.47 5.92 5.92 16 Final Cost 232346.3232346.3 45185.4 45185.4 170993.5170993.5 89625.4¯ 89625.4 93419.093419.0 Max Const. 0.314¯ 0.314 1.41.4 0.3250.325 0.4110.411 0.293 0.293 Speed-up - - 1.861.86 23.77 23.77 23.31¯ 23.31 C 10 Final Cost 590101.0590101.0 211639.1 211639.1 321112.6321112.6 290707.0¯ 290707.0 303822.8303822.8 Max Const. 0.1580.158 4.54.5 0.2190.219 0.112¯ 0.112 0.061 0.061 Speed-up - - 5.035.03 6.34 6.34 6.24¯ 6.24 20 Final Cost 1141216.91141216.9 434224.8 434224.8 641816.3641816.3 580886.1¯ 580886.1 607757.9607757.9 Max Const. 0.1850.185 5.65.6 0.2540.254 0.153¯ 0.153 0.083 0.083 Speed-up - - 5.415.41 7.57 7.57 6.74¯ 6.74 40 Final Cost 2297989.12297989.1 875745.0 875745.0 1287241.81287241.8 1163967.6¯ 1163967.6 1218391.91218391.9 Max Const. 0.2130.213 6.36.3 0.2920.292 0.203¯ 0.203 0.144 0.144 Speed-up - - 6.306.30 19.11 19.11 11.52¯ 11.52 80 Final Cost 4641287.94641287.9 1815901.7 1815901.7 2585792.72585792.7 2339338.8¯ 2339338.8 2448767.62448767.6 Max Const. 0.3890.389 6.06.0 0.4680.468 0.288¯ 0.288 0.279 0.279 Speed-up - - 10.6110.61 48.61 48.61 30.92¯ 30.92 Table 3: Results for scaling experiments. Task A: Car obstacle field; Task B: Car intersection; Task C: Quadrotor obstacle field. The policies are trained using the smallest number of agents within deployment settings. The final cost and maximum constraint violation are obtained after 30 iterations. The speed-up ratio compares Deep Coordinator against the Baseline (Vanilla), where the vanilla optimizer runs until reaching a cost within 5% of Deep Coordinator. Results are averaged over 20 problem instances for all scales. Bold indicates the best-performing model, while underline¯ underline indicates the second-best.