Paper deep dive
Multi-agent rendezvous in fluid flows via reinforcement learning
Bocheng Li, Jingran Qiu, Lihao Zhao
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/9/2026, 4:34:02 AM
Summary
This study investigates multi-agent rendezvous in vortical fluid flows using multi-agent reinforcement learning (MARL). The authors develop a Deep Set Proximal Policy Optimization (DS-PPO) framework to train agents to navigate Taylor-Green Vortices, demonstrating that MARL strategies significantly outperform naive navigation approaches. The learned strategies exhibit strong transferability across varying vortex intensities, scales, and swarm sizes. Theoretical analysis reveals that fluid deformation and large finite-time Lyapunov exponents hinder rendezvous by separating agents, suggesting that target planning should avoid high-deformation regions. The research highlights MARL's capability to extract non-intuitive, physics-informed swarm intelligence for complex fluid environments.
Entities (8)
Relation Signals (7)
Multi-agent reinforcement learning (MARL) → outperforms → Naive strategy
confidence 95% · Compared to a naive strategy, where agents navigate toward their counterparts, MARL strategies significantly improve the rendezvous rate.
Finite-time Lyapunov exponents → identifies → fluid separation regions
confidence 90% · Large finite-time Lyapunov exponents identify where fluid effects separate adjacent agents, suggesting that targets should be planned in weak-deformation regions.
Fluid deformation → impeds → rendezvous process
confidence 90% · Furthermore, a theoretical analysis demonstrates that fluid deformation impedes the rendezvous process.
Deep Set Proximal Policy Optimization (DS-PPO) → implementedas → Multi-agent reinforcement learning (MARL)
confidence 90% · We apply Deep Set Proximal Policy Optimization (DS-PPO) algorithm to explore the rendezvous problem.
Taylor-Green Vortices (TGV) → models → fluid environment
confidence 90% · We consider a typical vortical flow field, i.e., steady Taylor-Green Vortices (TGV) flow, which contain periodic counter-rotating vortices
Multi-agent reinforcement learning (MARL) → explores → Swarm intelligence
confidence 85% · Our findings reveal the important role that agent-fluid interactions play in multi-agent tasks and highlight the MARL capability to explore swarm intelligence in complex flow environments.
Heuristic strategy → extractedfrom →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Rendezvous is a critical task for multi-agent systems, requiring agents to coordinate to meet at an unspecified location. However, achieving this in fluid environments presents a challenge, as it remains unclear how agents can exploit underlying fluid kinematics to facilitate convergence. In this study, we adopt a multi-agent reinforcement learning (MARL) approach to develop physics-informed rendezvous strategies in vortical flows. Compared to a naive strategy, where agents navigate toward their counterparts, MARL strategies significantly improve the rendezvous rate. MARL strategies also show transferability across varying vortex intensities, vortex scales, and swarm sizes. By breaking the symmetry of the state-action map, MARL strategy leverages a non-intuitive mechanism that prevents agents from becoming trapped in separate vortices, thereby enhancing rendezvous success. Additionally, a heuristic strategy is extracted from the learned strategy and also outperforms the naive strategy. Furthermore, a theoretical analysis demonstrates that fluid deformation impedes the rendezvous process. Large finite-time Lyapunov exponents identify where fluid effects separate adjacent agents, suggesting that targets should be planned in weak-deformation regions. Our findings reveal the important role that agent-fluid interactions play in multi-agent tasks and highlight the MARL capability to explore swarm intelligence in complex flow environments.
Tags
Links
- Source: https://arxiv.org/abs/2606.11274v1
- Canonical: https://arxiv.org/abs/2606.11274v1
Trouble viewing inline? Open PDF directly →
Full Text
50,977 characters extracted from source content.
Expand or collapse full text
Multi-agent rendezvous in fluid flows via reinforcement learning Bocheng Li1 Jingran Qiu2,∗ Lihao Zhao1,∗ 1AML, Department of Engineering Mechanics, Tsinghua University, Beijing, 100084 China. 2Department of Physics, Gothenburg University, Gothenburg, 41296 Sweden. Abstract Rendezvous is a critical task for multi-agent systems, requiring agents to coordinate to meet at an unspecified location. However, achieving this in fluid environments presents a challenge, as it remains unclear how agents can exploit underlying fluid kinematics to facilitate convergence. In this study, we adopt a multi-agent reinforcement learning (MARL) approach to develop physics-informed rendezvous strategies in vortical flows. Compared to a naive strategy, where agents navigate toward their counterparts, MARL strategies significantly improve the rendezvous rate. MARL strategies also show transferability across varying vortex intensities, vortex scales, and swarm sizes. By breaking the symmetry of the state-action map, MARL strategy leverages a non-intuitive mechanism that prevents agents from becoming trapped in separate vortices, thereby enhancing rendezvous success. Additionally, a heuristic strategy is extracted from the learned strategy and also outperforms the naive strategy. Furthermore, a theoretical analysis demonstrates that fluid deformation impedes the rendezvous process. Large finite-time Lyapunov exponents identify where fluid effects separate adjacent agents, suggesting that targets should be planned in weak-deformation regions. Our findings reveal the important role that agent-fluid interactions play in multi-agent tasks and highlight the MARL capability to explore swarm intelligence in complex flow environments. I Introduction The rendezvous problem is one of the fundamental problems in multi-agent systems, in which agents in a swarm follow a strategy to eventually rendezvous at an unspecified location [21]. In nature, rendezvous is essential for creature mating and information sharing [2]. For multi-vehicle systems, such as robots and even spacecrafts, rendezvous is essential for payload delivery and logistics [35, 8]. Meanwhile, the rendezvous is a premise task for realizing swarm intelligence [10], such as flocking to efficiently migrate [12], cloak themselves [22], etc. Therefore, investigating the rendezvous problem is of significant value for agents across length scales and has drawn much attention in recent years. Many rendezvous processes take place in fluid environments. The motions of aquatic swimmers [23], oceanic ships [32], and balloons [4], are significantly influenced by the ambient fluid motions. Fluid dynamics is typically non-linear, which makes traditional policies suboptimal. Few studies have considered flow effects. Sang et al. [29] use a particle swarm optimization algorithm to optimize the rendezvous paths of wave gliders, where only large-scale currents are considered, while small-scale structures create a challenge for the rendezvous task. Zaidi et al. [39] achieved time-varying rendezvous for the gust-disturbed drones with a leader-follower method, where the wind gust is treated as disturbance since it is much weaker than the propelling ability of drones. Overall, how ambient fluid motions affect the rendezvous task remains unknown. Three categories of methods have been proposed to study the rendezvous problem, i.e., virtual forces, probabilistic approaches, and artificial evolution methods [3]. Multi-agent reinforcement learning (MARL) is an evolution method, where neural networks link sensory inputs to actuator outputs, and these networks evolve with experience [9]. In earlier studies, reinforcement learning (RL) performs well in controlling agents in various flow fields. For instance, ones study the navigation of microswimmers in vortical and turbulent flows to reveal the response mechanism to fluid signals for different goals [11, 26, 13, 1, 36, 19, 37, 17]. Other agents, including the glider [27], airship [40], and stratospheric balloon [4], are also effectively controlled to take advantage of wind effects. These works show that RL is capable to utilize physical mechanisms to tackle the navigation problems in flow fields, which inspires us to investigate how a MARL approach can discover rendezvous strategies. To the best of the authors’ knowledge, the only study applying MARL to agents in flow fields is that Borra et al. [7] investigated the pursuit-evasion problem of two microswimmers by a decentralized MARL method. Hence, attempts to exploit MARL on different multi-agent problems are meaningful, especially on the current rendezvous problem. How does ambient fluid motion affect the rendezvous task? Can we find an effective rendezvous strategy by utilizing flow signals in MARL implementation? If so, what is the mechanism? These questions motivate us to devise a training approach based on MARL and carry out numerical experiments to explore and interpret the rendezvous strategies. This work reveals important flow effects on the rendezvous task and provides effective and robust strategies. Our approach to applying MARL to problems with agent-fluid interactions shows promise and provides guidance for further exploitation of MARL to specific agents in complex flow environments. I Methods In this study, agents are modeled as point-masses, allowing inter-agent collisions to be neglected, and each agent possesses global observability of all other agents [16]. An example swarm consisting of n=3n=3 agents is shown in Fig. 1(a). The swarm is considered to successfully rendezvous when the distance between any two agents is less than a set value drd_r, as shown in Fig. 1(b). Since the rendezvous process of each swarm is dependent on its initial condition, a statistical indicator is defined to evaluate the effectiveness of a rendezvous strategy as the rendezvous rate Rr=Nr/NR_r=N_r/N, where N is the number of swarms randomly initialized in region Φ (x∈[0,L],y∈[0,L])(x∈[0,L],y∈[0,L]), and NrN_r is the number of swarms that successfully rendezvous within a time horizon TtT_t. RrR_r is obtained through Monte Carlo method based on numerical simulations. In the following sections I.1 to I.4, fluid-agent interaction models and training details are presented. Figure 1: (a) Three agents moving in Taylor-Green Vortices of L/L0=2πL/L_0=2π. The black arrow represents the agent orientation p. The red lines are the streamlines, and the background color indicates the vortex criterion Q value. (b) A rendezvous trajectory of two agents. The radius of the ending circle represents the distance recognized as a successful rendezvous. (c) The Taylor-Green Vortices of different vortex scale L0L_0. The shown region is Φ . The red lines are streamlines, and the background color stands for the flow speed. (d) The symbols of agent motion, where vs,ωsv_s, _s represent the propelling and steering speed, respectively. , v, u are the velocity of the agent and local fluid, respectively. (e) The state symbols for a swarm. up(i),uq(i)u_p^(i),u_q^(i) denote the fluid velocity components on the local coordinates of agent i. Point ’c’ represents the spatial center of this swarm. (i,j) d^(i,j) is the vector from position i to position j, and θ(i,j)θ^(i,j) is the angle between (i,j) d^(i,j) and (i) p^(i). I.1 Flow field We consider a typical vortical flow field, i.e., steady Taylor-Green Vortices (TGV) flow, which contain periodic counter-rotating vortices with intensity u0u_0 and scale L0L_0 [34], as shown in Fig. 1(a). The flow velocity components are ux=u0cos(xL0)sin(yL0),uy=−u0sin(xL0)cos(yL0). gatheredu_x=u_0 ( xL_0) ( yL_0),\\ u_y=-u_0 ( xL_0) ( yL_0). gathered (1) The maximal flow speed is u0=maxux2+uy2u_0= u_x^2+u_y^2. The larger L0L_0 corresponds to larger velocity variation and denser vortices, as shown in Fig. 1(c). Since TGV flow exhibits typical rotation and deformation, we characterize its structural features using the Q-criterion, defined as the rotational kinetic energy minus deformation kinetic energy: Q=12(ΩijΩij−EijEij)=(u0L0)2cos(x+yL0)cos(x−yL0), gatheredQ= 12( _ij _ij-E_ijE_ij)=( u_0L_0)^2 ( x+yL_0) ( x-yL_0), gathered (2) where Ωij,Eij _ij,E_ij represent the components of the rotation matrix and deformation matrix of fluid motion, respectively. The Q-criterion is an important diagnostic associated with the rendezvous behavior, which is discussed in the following analysis in Section I.4. I.2 Agent dynamics In the flow field, agents can propel themselves and are also subjected to fluid velocity u, as shown in Fig. 1(d). An ideal model governs the agent motion [6]: ddt==+vs,ddt=×=ωsz×, gathered d xdt= v= u+v_s p,\\ d pdt= ω× p= _s e_z× p, gathered (3) where x denotes the agent position, and z e_z is the unit vector of the z-axis (normal vector of the plane). The agent propels at a fixed speed vsv_s directing to orientation p, and its steering speed ωs _s is the action controlled by a strategy, whose magnitude is restricted not to exceed ωm _m. The time scale in this system is defined as τ=ωm−1τ= _m^-1. The agent trajectories are obtained by integrating (3) with Adams-Bashforth scheme. I.3 Multi-agent reinforcement learning Figure 2: The diagram of DS-PPO network architecture, with the state of the i-th agent in the swarm as input. For the actor network, the output variable is the mean and standard deviation of the action distribution (μa,σa)( _a, _a). For the critic network, the output is the value estimate V()V( s) of input states. Each MLP block denotes a two-layer fully connected network with ReLU activation in between, where the numbers of neurons are indicated in the brackets. To achieve rendezvous, agents take the action ωs _s to adjust the propelling direction based on perceived signals following a specific strategy, which can be obtained by MARL training. As illustrated in Fig. 1(e), the signals consist of the local states (flow velocity components up(i)u_p^(i), uq(i)u_q^(i)), and the shared states (relative positions of other agents d(i,j)d^(i,j), θ(i,j)(j=1,2,…,i−1,i+1,…,n)θ^(i,j)(j=1,2,...,i-1,i+1,...,n), where d(i,j)=|(i,j)|d^(i,j)=| d^(i,j)|). A naive strategy is defined as one where each agent always steers to orient to the center of this swarm, regardless of the flow effects [28]. For the i-th agent, the naive strategy is ωs(i)=sgn(θ(i,c))ωm,θ(i,c)=cos−1<(i,c),(i)>|(i,c)||(i)|,(i,c)=∑j=1n(i,j)n. gathered _s^(i)=sgn(θ^(i,c)) _m,\\ θ^(i,c)= ^-1 < d^(i,c), p^(i)>| d^(i,c)|| p^(i)|,\\ d^(i,c)= _j=1^n d^(i,j)n. gathered (4) We apply Deep Set Proximal Policy Optimization (DS-PPO) algorithm to explore the rendezvous problem. The network structure is motivated by Deep Sets [38, 20], which can preserve the permutation invariance of the individuals in a swarm. DS-PPO avoids dimensional explosion as a fully decentralized MARL algorithm, and it adapts to different swarm sizes n because the shared information is processed with a pooling layer. The network architecture is shown in Fig. 2. All agents in the swarm share the same network and parameters. For the i-th agent, the input of the network consists of two parts, the local states l(i)=up(i),uq(i) s_l^(i)=\u_p^(i),u_q^(i)\ measured by an agent, and the shared states s(i,j)=d(i,j),θ(i,j)(j=1,2,…,i−1,i+1,…,n) s_s^(i,j)=\d^(i,j),θ^(i,j)\(j=1,2,...,i-1,i+1,...,n) that it receives from all individuals in the swarm. The local state is fed to a fully connected network ϕlocal(l(i)) _local( s_l^(i)). The shared state is fed to another fully connected network ϕshared(s(i,j)) _shared( s_s^(i,j)), followed by an average pooling ϕ¯shared(i)=1n−1∑j=0,j≠inϕshared(s(i,j)) φ_shared^(i)= 1n-1 _j=0,j≠ i^n _shared( s_s^(i,j)). We note that all the shared states, s(i,j),j=1,…,n s_s^(i,j),j=1,...,n, are fed to the same network ϕshared _shared, which keeps the permutation invariance of the agents in the swarm. Finally, ϕlocal(l(i)) _local( s_l^(i)) and ϕ¯shared(i) φ_shared^(i) are concatenated and fed into the output layer (μa(i),σa(i))=ρ(ϕlocal(l(i)),ϕ¯shared(i))( _a^(i), _a^(i))=ρ( _local( s_l^(i)), φ_shared^(i)), where μa(i) _a^(i) and σa(i) _a^(i) are the mean and standard deviation of the action distribution for this agent. We use the same network architecture for both the actor and critic networks. The hyperparameters of the network architecture are given in Fig. 2, where nmid=200n_mid=200. The navigational strategy, modeled by the network, can be optimized by training. The training process is to search for a network that maximizes a reward, whose growth represents the rise of the rendezvous rate. The reward is designed to evaluate the action taken at each state. Having the total distance between agent i and other agents D(i)=∑j=1,j≠ind(i,j)D^(i)= _ subarraycj=1,j≠ i subarray^nd^(i,j), the reward given to agent i at the m-th time step R(i,m)=D(i,m)−D(i,m+1)R^(i,m)=D^(i,m)-D^(i,m+1), which means that if the total distance decreases, a positive reward is given in training. The strategy is trained by updating the networks following the standard way of Proximal Policy Optimization [30], which requires the trajectories of agents with data including states, actions and rewards. Here, all the trajectories of individuals in a swarm are used in the training. The training is performed in an episodic way. In each episode, ten swarms are initialized in Φ(x,y∈[0,L]) (x,y∈[0,L]), and the agents move until all swarms successfully rendezvous, or the maximum duration of an episode Tt=100τT_t=100τ is reached. The value of TtT_t is chosen such that any two-agent swarm using naive strategy has sufficient time to rendezvous in a quiescent flow field. Every 0.025τ0.025τ, the agents decide their actions according to the actor network, and every 5τ5τ, the networks are updated once. During updating, γ=0.99γ=0.99 is used in the calculation of generalized advantage estimation. For better convergence, the learning rate α decays with episode E as α=max[0,α0(1−E0)]α= [0, _0 (1- EE_0 ) ], where hyperparameters are set as α0=10−4,E0=985 _0=10^-4,E_0=985. In each case, we train the networks for no more than 1000 episodes. Once training converges (i.e. the total reward in one episode does not change with further training), we evaluate the strategy by setting σa=0 _a=0, so that the action is ω=μaωmω= _a _m. I.4 Cases and parameters The steering and propelling abilities, i.e., ωm _m and vsv_s, are taken as characteristic variables. Parameters that may influence the rendezvous rate RrR_r are non-dimensionalized and listed in Table 1. To understand the effects of flow intensity and scale on rendezvous task, we consider different TGV intensities and scales, βu∈0,1,2,3,4,5 _u∈\0,1,2,3,4,5\, βl∈5,7,10,14,20,40 _l∈\5,7,10,14,20,40\. Furthermore, we also consider different swarm sizes n∈2,3,4n∈\2,3,4\. In order to test the transferability of MARL strategy, we trained the strategy for a fixed βl=10 _l=10, and tested it for varying βl _l. Hereinafter, we denote each learned MARL strategy as ’NnnUβu _u’, representing the optimal model selected from five independent training trials. Table 1: Dimensionless parameters in the rendezvous task. Parameter Dimensionless Value Initial domain length L βL=Lωm/vs _L=L _m/v_s 20π20π Rendezvous distance drd_r βd=drωm/vs _d=d_r _m/v_s 11 Motion duration TtT_t βT=Ttωm _T=T_t _m 100100 TGV intensity u0u_0 βu=u0/vs _u=u_0/v_s 0,1,2,3,4,5 TGV scale L0L_0 βl=L0ωm/vs _l=L_0 _m/v_s 5,7,10,14,20,40 Swarm size n n 2,3,4 I Results I.1 Performance of two-agent strategies First, we examine how fluid motion affects the success of rendezvous. The performance of two-agent strategies is evaluated and shown in Fig. 3(a,b,c). Fig. 3(a,b) shows that the rendezvous rate of the naive strategy decreases as βu _u increases or βl _l decreases, indicating greater difficulty in achieving rendezvous. When βu=0,1 _u=0,1, all swarms rendezvous, because the agents overcome the flow to approach their partners with a propelling speed vsv_s large enough compared to the maximal flow speed u0u_0. In addition, Fig. 1(c) shows that with the increase of vortex scale βl _l, the flow field becomes less spatially heterogeneous and the velocity gradients are smaller, so that agents at different locations experience more similar fluid velocities u and thus the rendezvous task becomes easier. Therefore, the rendezvous problem is highly dependent on the characteristics of the underlying flow field. Figure 3: (a) The rendezvous rates of the naive strategy used in different vortex intensities for two-agent swarms while fixing βl=10 _l=10. (b) The rendezvous rates of the naive and the learned N2U4 strategy used in different vortex scales for two-agent swarms while fixing βu=4 _u=4. The horizontal axis is on a logarithmic scale. (c) The performance of six MARL strategies N2Uβu _u trained at vortex intensity βu _u used in different vortex intensities on the horizontal axis while fixing βl=10 _l=10. The color indicates the RrR_r ratio of MARL strategy over the naive strategy used at the same βu _u. The parameter values during testing are shown along the horizontal axis. The performance of smart rendezvous strategy obtained by MARL training is demonstrated in Fig. 3(c), which shows the rendezvous rate trained and tested under different βu _u relative to the naive strategy. The diagonal elements in Fig. 3(c) indicates that, when the MARL strategy is trained and tested with the same βu _u, it outperforms the naive baseline, demonstrating the validity of our MARL framework. Notably, N2U4 strategy achieves a 59% improvement in RrR_r. We assess the transferability by testing MARL strategy in all considered vortex intensities. As shown in Fig. 3(c), most MARL strategies outperform the naive one in the considered flow intensities. Besides, the performances of both N2U0 and N2U1 strategies are close to the naive strategy even when they are tested in a strong flow of large βu _u, as shown in the lowest two rows of Fig. 3(c). This means that the agents learn a smart strategy similar to the naive strategy when they are trained in a weak-flow environment. In terms of the transferability to different vortex scale, we test an example, i.e., N2U4 strategy, in all considered vortex scales while fixing the intensity as βu=4 _u=4. Fig. 3(b) illustrates that N2U4 strategy is more effective in TGVs of smaller βl _l, indicating the advantage of N2U4 strategy in tackling small-scale structures in flow field. In general, in flow fields with dense vortices, MARL strategy shows strong transferability to flow variations. I.2 Performance of multi-agent strategies Figure 4: (a) The rendezvous rates of strategies for different swarm sizes and in different flow intensities. In the TGVs of (b)βu=0 _u=0 or (c)βu=2 _u=2, the transferability of strategies to the swarm size. The color indicates the RrR_r ratio of MARL strategy over the naive strategy used in the case of swarm size n. In this section, we investigate the strategy for swarms of different sizes. The results for swarms of three and four agents in βu=0,2 _u=0,2 are shown in Fig. 4(a). The performance of the naive strategy decreases with larger n, which means that the rendezvous task for a larger swarm is more difficult. The reason is that the rendezvous requires the distances between any two of the n agents to be smaller than the threshold, which is more difficult for larger swarms because the number of agent pairs is n(n−1)/2n(n-1)/2. Nevertheless, the advantages of MARL strategies over naive strategy demonstrate that our MARL approach is still effective for large swarm. Our MARL implementation is devised to be independent of swarm size, which means that each strategy can be used in cases where the swarm size is different from that in training. Thus, the strategy transferability to different n is also tested, as shown in Fig. 4(b,c). The results show that the strategies trained at n=3n=3 still outperform the naive strategy when tested for n=2,4n=2,4. The element in the lower triangle in Fig. 4(b,c), where n during training is larger than that during testing, shows better performance than the upper triangle. The lack of full transferability is not discouraging since transferability is usually not guaranteed in RL problems. To further improve generalization, one could train a single strategy over all considered parameters. I.3 Mechanism of MARL strategy To understand the mechanism of MARL strategy, we analyze the dependence of action on states by sampling state-action data. Here, we use N2U4 strategy as an example, because N2U4 strategy shows the best performance in Fig. 3(c). For a two-agent swarm, the states include up,uq,d,θu_p,u_q,d,θ defined in Fig. 1(e). As seen in Fig. 5(a), the points (d,θ,ωs)(d,θ, _s) approximately form a 2D manifold, suggesting that the control action is primarily governed by d and θ. Therefore, we focus on the dependency of action on d,θd,θ. Figure 5: (a) The 3D scatter diagram of state-action samples. (b) The mean action of ωs _s at different (d,θ)(d,θ) state. The state regions are divided into three parts by the sharp changes in action (red dashed lines). (c) The variance of action ωs _s at different (d,θ)(d,θ) caused by the other states up,uqu_p,u_q instead of network stochasticity, because no network stochasticity exists at action decision. (d) The piecewise function fitted from data in (a). (e) The performance of different state-action maps, where the third bar means the agents take actions by N2U4 strategy but input the state of flow speed up,uqu_p,u_q as zeros. The trajectories of an agent pair moving in TGV of βu=8 _u=8 with (f) naive and (g) N2U4 strategies. Opacity increases with time, and two colors distinguish two agents. Some points on the trajectory are drawn, where the line segments are the tail of the agents, representing orientations. According to the yellow-blue color bar in (b), ωs _s is described in (a, b, d), and in (f, g), the action ωs _s is also indicated by the tail color. The TRA distributions for (h) tracer agent whose vs=0v_s=0, (i) agent taking naive strategy, (j) agent taking N2U4 strategy moving in TGV of βu=8 _u=8. TRA¯ TRA is the average over the region of x,y∈[−0.25L,1.25L]x,y∈[-0.25L,1.25L]. Based on the sampled data, we obtain the mean and variance of action over different upu_p and uqu_q by Gaussian-weighted gridded statistics (see Appendix B), as shown in Fig. 5(b,c). The mean value of action is the average steering speed at a certain d,θd,θ. In Fig. 5(c), the large variance indicates where action is sensitive to the changes in flow velocities up,uqu_p,u_q. Because the regions of large variance occupy only a small portion, the information of the relative position d,θd,θ dominate the strategy, whereas the local flow velocity up,uqu_p,u_q have a minor effect. This is verified by masking the flow signals up,uqu_p,u_q by zero, which leads to a marginal 4% decrease in rendezvous rate, as shown in Fig. 5(e). In other words, N2U4 strategy remains effective even in the absence of local flow information. To fit the state-action samples with an explicit function, we divide the state region into three parts as shown in Fig. 5(b). Then we obtain a state-action function S(d/πL0,θ/π)S(d/π L_0,θ/π) through piecewise fitting by third-order polynomials. The function is displayed in Fig. 5(d), and its mathematical expression is in Appendix C. This function provides a heuristic controlling approach for the two-agent rendezvous task, and the corresponding RrR_r in Fig. 5(f) proves its effectiveness. The mechanism of N2U4 strategy is revealed by Fig. 5(a,b). At small d, the ωs _s is an increasing odd function of θ. The same sign of ωs _ s and θ indicates that the two agents steer towards each other when they are close, similar to the mechanism of the naive strategy. When d is large, ωs _s is no longer an odd function of θ. To illustrate the consequence, we show the trajectories of smart agents in Fig. 5(g). When the agent is in the bottom-right vortex, it experiences a state corresponding to region 2 in Fig. 5(b), and it does not steer towards the direction of the other agent. This allows it to escape the bottom-right vortex and later rendezvous with its partner which remains in the lower-left vortex as shown in Fig. 5(g). In contrast, naive agents are trapped in separate vortices as shown in Fig. 5(f). This vortex-induced trapping effect can be quantitatively verified. We employ Lagrangian coherent structure (LCS) theory, which is a tool for describing transport and mixing processes in fluid systems. This theory has been successfully used to analyze the behaviors of inertial [24], self-propelled [5, 33], and controlled agents [18] in flow fields. We focus on elliptic Lagrangian Coherent Structure (eLCS) since it detects regions exhibiting sustained coherent rotation. The trajectory rotation average (TRA) is a major indicator in eLCS, which evaluates the rotation intensity of the agent trajectory [14]. TRA is expressed as: TRA((t0))=1tN−t0∑i=0N−1cos−1<(ti),(ti+1)>|(ti)||(ti+1)|, ( x(t_0))= 1t_N-t_0 _i=0^N-1 ^-1 < v(t_i), v(t_i+1)>| v(t_i)|| v(t_i+1)|, (5) where (t),(t) x(t), v(t) represent the position and velocity vector of the agent at time t. The time window tN−t0t_N-t_0 is divided into N terms, and in each term, the angle variation between the initial and next velocity vectors (ti),(ti+1) v(t_i), v(t_i+1) is calculated. The sum of angle variations over the time window represents the rotation of the trajectory started at (t0) x(t_0). Here, we set N=20N=20 and tN−t0=5τt_N-t_0=5τ. Fig. 5(h) shows large TRA of ambient flow motion, obtained from trajectories of tracer agents which are consistent with streamlines. Moreover, comparing Fig. 5(i, j), we can see that TRA for the naive agent is larger than that for N2U4 agent, especially in vortices with negative vorticity, as indicated in Fig. 5(g). This coincides with the mechanism of avoiding trapping in a negative vortex analyzed in the previous paragraph. From the above analysis, we can conclude that the non-trapping mechanism of MARL strategy is powerful in environments with dense vortices. This mechanism explains why MARL strategy shows less advantage over naive strategy with the increase of βl _l as shown in Fig. 3(d). I.4 Rendezvous distributions To further examine these findings, we analyze the rendezvous behaviors in a statistical way to understand the flow effects. We define P(x,y)P(x,y) as the probability density of a swarm, randomly initialized in region Φ , that successfully reach rendezvous in one episode at position (x,y)(x,y). For the two-agent naive and MARL strategies, P(x,y)P(x,y) is obtained by Gaussian kernel density estimation (see Appendix A), as shown in Fig. 6. The distributions of P of naive strategy in different flow intensities are shown in Fig. 6(a, b). The largest P is found at the center of Φ , where the two agents, initialized at random positions, move statistically equal distances to meet. When flow is present (Fig. 6(b)), agents rendezvous near vortex centers, indicated by the large P in vortical regions. Comparing Fig. 6(b) with (d), it is found that N2U4 strategies increase P, which explains the higher RrR_r achieved by MARL strategies displayed in Fig. 3(a). Figure 6: The distributions of rendezvous probability density P(x,y)P(x,y) for (a-b) naive strategy and (c-d) MARL strategies used in the training TGVs, where color indicates the logarithm of P(x,y)P(x,y). (e) The FTLE σ distribution of TGV of βu=4 _u=4. (f) The statistical correlation between rendezvous probability P(x,y)P(x,y) and FTLE σ. To explain the correlation between rendezvous distribution and the flow structure, we analyze the distance evolution of two agents caused by fluid motion by considering the finite-time Lyapunov exponent (FTLE), which is widely used to describe the hyperbolic LCS [15]. With the flow map t0t()=(t;t0,) F^t_t_0( x_0)= x(t;t_0, x_0) representing position evolution of a tracer agent starting at time t0t_0 and position x_0, the evolution of the relative position of a pair of agents (t)=∇t0t() r(t)=∇ F^t_t_0( x_0) r_0. The magnitude of (t) r(t) is: |(t)|2 | r(t)|^2 =[∇t0t()]T∇t0t() =[∇ F^t_t_0( x_0) r_0]^T∇ F^t_t_0( x_0) r_0 (6) =T[∇t0t()]Tt0t() = r_0^T[∇ F^t_t_0( x_0)]^T F^t_t_0( x_0) r_0 =T, = r_0^T C r_0, where =[∇t0t()]T∇t0t() C=[∇ F^t_t_0( x_0)]^T∇ F^t_t_0( x_0) is the Cauchy-Green tensor. With the eigenvalues λi _i of C, the FTLE value is: σ=logmax(λi)t−t0.σ= ( _i)t-t_0. (7) Since C is a symmetric matrix, the Rayleigh quotient R=TT=|(t)|2TR= r_0^T C r_0 r_0^T r_0= | r(t)|^2 r_0^T r_0 describes the stretching of agent distance, which satisfies Rmax=maxλiR_ = _i. Hence, |max(t)|=|(t0)|eσ(t−t0)| r_ (t)|=| r(t_0)|e^σ(t-t_0). In the two-dimensional system, the fluid is incompressible, thus λmaxλmin=det=1 _ _ = C=1 so that |min(t)|=|(t0)|e−σ(t−t0)| r_ (t)|=| r(t_0)|e^-σ(t-t_0). Physically, a large FTLE measures strong deformation of a fluid particle, which leads to exponentially increasing separation but limited compression of adjacent agents in the fluid particle. Therefore, it is difficult for agents to rendezvous if they experience spaces with large FTLE. We calculate FTLE distribution for TGV of βu=4,βl=10 _u=4, _l=10, with the end time tE−t0=−2.5τt_E-t_0=-2.5τ. By integrating trajectories backward in time, we can find whether agents experienced strong deformation before finally rendezvousing at position x_0. The FTLE distribution is displayed in Fig. 6(e), which presents a negative correlation with rendezvous probability in Fig. 6(b). Mean log(P(x,y)L02) (P(x,y)L_0^2) of different FTLE is shown in Fig. 6(b). The presence of negative correlation verifies the above theoretical analysis. Although FTLE is calculated from a Lagrangian property of the flow, it still reflects Eulerian characteristics of the flow. Comparing Fig. 1(a) and Fig. 6(b), we can see that high FTLE occurs at the vortex edge, where Eulerian deformation is strong, corresponding to a small Q value. The connection between FTLE and the Eulerian deformation can be understood as follows. From an Eulerian perspective, the evolution of the separation between two agents satisfies ˙(t)=∇(,t)(t) r(t)=∇ u( x,t) r(t). Meanwhile, the deformation gradient obeys ∇˙t0t()=∇(,t)∇t0t()∇ F^t_t_0( x_0)=∇ u( x,t)∇ F^t_t_0( x_0), which yields ˙=[∇t0t()]T∇t0t() C=[∇ F^t_t_0( x_0)]^T E∇ F^t_t_0( x_0), where Eulerian deformation tensor of fluid =[∇(,t)]T+∇(,t) E=[∇ u( x,t)]^T+∇ u( x,t). The Cauchy-Green tensor C is initially the identity tensor, corresponding to σ=0σ=0. As the deformation induced by E accumulates along trajectories, the eigenvalues of C separate, leading to an increase in the FTLE. Therefore, large FTLE values are associated with trajectories that experience strong accumulated Eulerian deformation. Since large FTLE is correlated with a small rendezvous probability (Fig. 6(f)), lower rendezvous rates are expected near vortex edges, where the Eulerian deformation is strong. Moreover, the rendezvous task becomes increasingly difficult for larger βu _u or smaller βl _l, as discussed in Sec. I.1, because the magnitude of E scales with βu/βl _u/ _l. IV Conclusions In this study, we have demonstrated that a multi-agent reinforcement learning approach is highly effective in solving the rendezvous problem under the influence of ambient fluid motions. The strategies yielded through the DS-PPO framework significantly outperform naive baselines across various flow intensities and swarm sizes. These results highlight the capability of MARL in exploring swarm intelligence within complex flow environments where traditional control policies often remain suboptimal. The superiority of MARL strategy, specifically N2U4, is attributed to its capacity to mitigate ’vortex trapping’, which inherently limits naive strategies. While naive agents tend to get trapped in separate vortices, the trained RL agents can escape trapping by a symmetry-breaking mechanism in the state-action map. This behavior is quantitatively supported by the TRA analysis, which reveals that RL agents exhibit weaker rotation than naive ones. This suggests that MARL does not merely optimize paths but evolves environmental intelligence to form beneficial coherent structures. Furthermore, by analyzing the statistical distributions of successful rendezvous, we uncover how local fluid kinematics govern agent coordination. We demonstrate that fluid deformation is an inherent obstacle: regions with high finite-time Lyapunov exponents (FTLE) exponentially drive agents apart. These findings provide a crucial practical guideline, strongly suggesting that future multi-agent rendezvous targets should be strategically planned in regions exhibiting weak fluid deformation. Based on these understanding, more detailed fluid effects can be further interpreted by integrating high-fidelity methods such as the immersed boundary method [25], to account for precise agent-fluid and agent-agent interactions. Additionally, exploring the effects of communication limitations, sensory noise, and physical obstacles will be crucial for the application of these strategies to oceanic or atmospheric vehicles. Beyond rendezvous, the framework demonstrated here holds significant potential for other homogeneous collective tasks, such as flocking or collaborative search in turbulent environments. Acknowledgements.B. L. and L. Z. acknowledge the support from National Science Foundation of China (Grants No. 12472224, 27892252104 and 12388101). J. Q. acknowledges the support from Vetenskapsrådet (Grants No. 2018-03974 and 2023-03617). Appendix A Gaussian kernel density estimation on rendezvous distribution We define the rendezvous probability density P(x,y)P(x,y) that satisfies: Rr(A)=∬AP(x,y)dxdy, gatheredR_r(A)= _AP(x,y)dxdy, gathered (8) where Rr(A)R_r(A) denotes the probability that a swarm, randomly initialized in Φ(x,y∈[0,L]) (x,y∈[0,L]), rendezvous at the position inside the area A. With (xi,yi)(x_i,y_i) describing the center position of swarm i when the swarm is judged to rendezvous, the rendezvous probability densities P(x,y)P(x,y) can be obtained with Gaussian kernel density estimation method as: P(x,y)=1Nhxhy∑i=1NrK(x−xihx)K(y−yihy),K(m)=12πexp(−m2/2), gatheredP(x,y)= 1Nh_xh_y _i=1^N_rK ( x-x_ih_x )K ( y-y_ih_y ),\\ K(m)= 1 2π (-m^2/2),\\ gathered (9) where K(m)K(m) is the Gaussian kernel function, and hx,hyh_x,h_y are the bandwidths selected with Silverman [31] rule. Appendix B Gaussian-weighted gridded statistics We again use Gaussian kernel function in (9) to obtain action statistics in the state space. This approach allows for continuous field reconstruction from numerous discrete data points. Given a set of data samples (xi,yi,zi)(x_i,y_i,z_i), we can quantify the mean and variance of action z¯ z, var[z]var[z] at point (x,y)(x,y) as follows: z¯(x,y)=∑ℛK(x−xih)K(y−yih)zi∑ℛK(x−xih)K(y−yih),var[z](x,y)=∑ℛK(x−xih)K(y−yih)(z¯−zi)2∑ℛK(x−xih)K(y−yih), gathered z(x,y)= _RK( x-x_ih)K( y-y_ih)z_i _RK( x-x_ih)K( y-y_ih),\\ var[z](x,y)= _RK( x-x_ih)K( y-y_ih)( z-z_i)^2 _RK( x-x_ih)K( y-y_ih),\\ gathered (10) where ℛR represents the region of |xi−x|<H,|yi−y|<H|x_i-x|<H,|y_i-y|<H. Here, we set H=40h,h=0.01H=40h,h=0.01. In our work, x,y,zx,y,z correspond to the states d/πL0,θ/πd/π L_0,θ/π and action ω/ωmω/ _m, respectively, to analyze the dependencies of the action on these two states, as shown in Fig. 5(b,c). Appendix C Fitting the map from state to action The explicit state-action function S(d^,θ^)S( d, θ) is fitted by third-order polynomials with the least-square method, where d^=d/πL0,θ^=θ/π d=d/π L_0, θ=θ/π. In addition, the function is bounded to [-1,1], and we use the clip function to restrict it. The expression of S(d^,θ^)S( d, θ) is: S(d^,θ^)=clip[f1(d^,θ^)],if θ^>0clip[f2(d^,θ^)],if θ^≤0 and d^+1.1θ^≥0−1,if d^+1.1θ^<0S( d, θ)= casesclip[f_1( d, θ)],&if θ>0\\ clip[f_2( d, θ)],&if θ≤ 0 and d+1.1 θ≥ 0\\ -1,&if d+1.1 θ<0 cases (11) f1(d^,θ^) f_1( d, θ) =−0.16d^3θ^3+0.22d^3θ^2−0.10d^3θ =-16 d^3 θ^3+22 d^3 θ^2-10 d^3 θ (12) +0.02d^3+1.27d^2θ^3−1.77d^2θ^2+0.70d^2θ +02 d^3+27 d^2 θ^3-77 d^2 θ^2+70 d^2 θ −0.07d^2−3.20d^θ^3+4.64d^θ^2−1.75d^θ -07 d^2-20 d θ^3+64 d θ^2-75 d θ +0.07d^+3.07θ^3−5.76θ^2+3.55θ^+0.26, +07 d+07 θ^3-76 θ^2+55 θ+26, f2(d^,θ^) f_2( d, θ) =2.21d^3θ^3+2.25d^3θ^2−0.51d^3θ^+0.14d^3 =21 d^3 θ^3+25 d^3 θ^2-51 d^3 θ+14 d^3 (13) −8.84d^2θ^3−9.71d^2θ^2+3.06d^2θ^−0.50d^2 -84 d^2 θ^3-71 d^2 θ^2+06 d^2 θ-50 d^2 +13.00d^θ^3+17.79d^θ^2−3.95d^θ^+0.55d +300 d θ^3+779 d θ^2-95 d θ+55 d −5.92θ^3−10.88θ^2+1.96θ^+0.11. -92 θ^3-088 θ^2+96 θ+11. The fitted function is shown in Fig. 5(d). References [1] J. K. Alageshan, A. K. Verma, J. Bec, and R. Pandit (2020-04) Machine learning strategies for path-planning microswimmers in turbulent flows. Physical Review E 101 (4), p. 043110. External Links: ISSN 2470-0045, 2470-0053, Document Cited by: §I. [2] A. Amirkhani and A. H. Barshooi (2022-06) Consensus in multi-agent systems: a review. Artificial Intelligence Review 55 (5), p. 3897–3935. External Links: ISSN 0269-2821, 1573-7462, Document Cited by: §I. [3] L. Bayındır (2016-01) A review of swarm robotics tasks. Neurocomputing 172, p. 292–321. External Links: ISSN 09252312, Document Cited by: §I. [4] M. G. Bellemare, S. Candido, P. S. Castro, J. Gong, M. C. Machado, S. Moitra, S. S. Ponda, and Z. Wang (2020-12) Autonomous navigation of stratospheric balloons using reinforcement learning. Nature 588 (7836), p. 77–82. External Links: ISSN 0028-0836, 1476-4687, Document Cited by: §I, §I. [5] S. A. Berman, J. Buggeln, D. A. Brantley, K. A. Mitchell, and T. H. Solomon (2021-01) Transport barriers to self-propelled particles in fluid flows. Physical Review Fluids 6 (1), p. L012501. External Links: ISSN 2469-990X, Document Cited by: §I.3. [6] L. Biferale, F. Bonaccorso, M. Buzzicotti, P. Clark Di Leoni, and K. Gustavsson (2019-10) Zermelo’s problem: Optimal point-to-point navigation in 2D turbulent flows using reinforcement learning. Chaos: An Interdisciplinary Journal of Nonlinear Science 29 (10), p. 103138. External Links: ISSN 1054-1500, 1089-7682, Document Cited by: §I.2. [7] F. Borra, L. Biferale, M. Cencini, and A. Celani (2022-02) Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number. Physical Review Fluids 7 (2), p. 023103. External Links: ISSN 2469-990X, Document Cited by: §I. [8] L. Breger and J. P. How (2008-09) Safe Trajectories for Autonomous Rendezvous of Spacecraft. Journal of Guidance, Control, and Dynamics 31 (5), p. 1478–1489. External Links: ISSN 0731-5090, 1533-3884, Document Cited by: §I. [9] L. Canese, G. C. Cardarilli, L. Di Nunzio, R. Fazzolari, D. Giardino, M. Re, and S. Spanò (2021-05) Multi-Agent Reinforcement Learning: A Review of Challenges and Applications. Applied Sciences 11 (11), p. 4948. External Links: ISSN 2076-3417, Document Cited by: §I. [10] D. L. Chase and O. Peleg (2025) The physics of sensing and decision-making by animal groups. Annual Review of Biophysics 54 (Volume 54, 2025), p. 329–351. External Links: Document, ISSN 1936-1238 Cited by: §I. [11] S. Colabrese, K. Gustavsson, A. Celani, and L. Biferale (2017-04) Flow Navigation by Smart Microswimmers via Reinforcement Learning. Physical Review Letters 118 (15), p. 158004. External Links: ISSN 0031-9007, 1079-7114, Document Cited by: §I. [12] K. V. Ganduri and B. P. Pathri (2024-11) Swarm Intelligence in Action: Particle Swarm Optimization and Rendezvous Algorithms for Swarm Robotics. Journal of Field Robotics, p. rob.22466. External Links: ISSN 1556-4959, 1556-4967, Document Cited by: §I. [13] P. Gunnarson, I. Mandralis, G. Novati, P. Koumoutsakos, and J. O. Dabiri (2021-12) Learning efficient navigation in vortical flow fields. Nature Communications 12 (1), p. 7143. External Links: ISSN 2041-1723, Document Cited by: §I. [14] G. Haller, N. Aksamit, and A. P. Encinas-Bartos (2021-04) Quasi-objective coherent structure diagnostics from single trajectories. Chaos: An Interdisciplinary Journal of Nonlinear Science 31 (4), p. 043131. External Links: ISSN 1054-1500, 1089-7682, Document Cited by: §I.3. [15] G. Haller (2015) Lagrangian coherent structures. Annual Review of Fluid Mechanics 47 (Volume 47, 2015), p. 137–162. External Links: Document, Link, ISSN 1545-4479 Cited by: §I.4. [16] M. Hüttenrauch, S. Adrian, and G. Neumann (2019) Deep reinforcement learning for swarm systems. Journal of machine learning research 20 (54), p. 1–31. External Links: ISSN 1532-4435, 1533-7928 Cited by: §I. [17] Y. Jiao, H. Hang, J. Merel, and E. Kanso (2025-03) Sensing flow gradients is necessary for learning autonomous underwater navigation. Nature Communications 16 (1), p. 3044. External Links: ISSN 2041-1723, Document Cited by: §I. [18] K. Krishna, S. L. Brunton, and Z. Song (2023) Finite Time Lyapunov Exponent Analysis of Model Predictive Control and Reinforcement Learning. IEEE Access 11, p. 118916–118930. External Links: ISSN 2169-3536, Document Cited by: §I.3. [19] B. Li, J. Qiu, and L. Zhao (2025) Escape from the predator-induced flow: smart prey strategies with steering and swimming actions. Soft Matter 21 (13), p. 2413–2421. External Links: ISSN 1744-683X, 1744-6848, Document Cited by: §I. [20] Y. Li, L. Wang, J. Yang, E. Wang, Z. Wang, T. Zhao, and H. Zha (2021) Permutation invariant policy optimization for mean-field multi-agent reinforcement learning: a principled approach. arXiv preprint arXiv:2105.08268. Cited by: §I.3. [21] J. Lin, A. S. Morse, and B. D. O. Anderson (2003) The multi-agent rendezvous problem. In 42nd IEEE International Conference on Decision and Control, Vol. 2, Maui, HI, USA, p. 1508–1513. External Links: Document, ISBN 978-0-7803-7924-4 Cited by: §I. [22] M. Mirzakhanloo, S. Esmaeilzadeh, and M. Alam (2020-11) Active cloaking in Stokes flows via reinforcement learning. Journal of Fluid Mechanics 903, p. A34. External Links: ISSN 0022-1120, 1469-7645, Document Cited by: §I. [23] T. J. Pedley and J. O. Kessler (1992) Hydrodynamic phenomena in suspensions of swimming microorganisms. Annual Review of Fluid Mechanics 24 (Volume 24, 1992), p. 313–358. External Links: Document, ISSN 1545-4479 Cited by: §I. [24] J. Peng and J. O. Dabiri (2009-03) Transport of inertial particles by Lagrangian coherent structures: application to predator–prey interaction in jellyfish feeding. Journal of Fluid Mechanics 623, p. 75–84. External Links: ISSN 0022-1120, 1469-7645, Document Cited by: §I.3. [25] C. S. Peskin (2002-01) The immersed boundary method. Acta Numerica 11, p. 479–517. External Links: ISSN 0962-4929, 1474-0508, Document Cited by: §IV. [26] J. Qiu, N. Mousavi, K. Gustavsson, C. Xu, B. Mehlig, and L. Zhao (2022-02) Navigation of micro-swimmers in steady flow: the importance of symmetries. Journal of Fluid Mechanics 932, p. A10. External Links: ISSN 0022-1120, 1469-7645, Document Cited by: §I. [27] G. Reddy, J. Wong-Ng, A. Celani, T. J. Sejnowski, and M. Vergassola (2018-10) Glider soaring via reinforcement learning in the field. Nature 562 (7726), p. 236–239. External Links: ISSN 0028-0836, 1476-4687, Document Cited by: §I. [28] R. Ribeiro, D. Silvestre, and C. Silvestre (2020-09) A Rendezvous Algorithm for Multi-agent Systems in Disconnected Network Topologies. In 2020 28th Mediterranean Conference on Control and Automation (MED), Saint-Raphaël, France, p. 592–597. External Links: Document, ISBN 978-1-72815-742-9 Cited by: §I.3. [29] H. Sang, J. Ji, X. Sun, J. Yun, S. Zhang, and F. Liu (2024-05) A path planning for formation rendezvous of the wave gliders considering ocean current disturbance. Ocean Engineering 299, p. 117285. External Links: ISSN 00298018, Document Cited by: §I. [30] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017-08) Proximal Policy Optimization Algorithms. arXiv. External Links: 1707.06347, Document Cited by: §I.3. [31] B.W. Silverman (2018-02) Density Estimation for Statistics and Data Analysis. 1 edition, Routledge. External Links: Document, ISBN 978-1-315-14091-9 Cited by: Appendix A. [32] Z. Song, D. Lipinski, and K. Mohseni (2017-09) Multi-vehicle cooperation and nearly fuel-optimal flock guidance in strong background flows. Ocean Engineering 141, p. 388–404. External Links: ISSN 00298018, Document Cited by: §I. [33] L. Storm, J. Qiu, K. Gustavsson, and B. Mehlig (2025) Transport barriers for microswimmers in unsteady flow. arXiv preprint arXiv:2509.16430. Cited by: §I.3. [34] G.I. Taylor (1923-10) LXXV. On the Decay of Vortices in a Viscous Fluid. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 46 (274), p. 671–674. External Links: ISSN 1941-5982, 1941-5990, Document Cited by: §I.1. [35] R. E. Wang, J. C. Kew, D. Lee, T. E. Lee, T. Zhang, B. Ichter, J. Tan, and A. Faust (2020-11) Model-based Reinforcement Learning for Decentralized Multiagent Rendezvous. arXiv. External Links: 2003.06906, Document Cited by: §I. [36] A. Xu, H. Wu, and H. Xi (2023-02) Long-distance migration with minimal energy consumption in a thermal turbulent environment. Physical Review Fluids 8 (2), p. 023502. External Links: ISSN 2469-990X, Document Cited by: §I. [37] L. Yang, J. Jiang, F. Ji, Y. Li, K. Yung, A. Ferreira, and L. Zhang (2024-06) Machine learning for micro- and nanorobots. Nature Machine Intelligence 6 (6), p. 605–618. External Links: ISSN 2522-5839, Document Cited by: §I. [38] M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola (2017) Deep sets. Advances in neural information processing systems 30. Cited by: §I.3. [39] A. Zaidi, M. Kazim, R. Weng, S. Ali, M. T. Raza, G. Abbas, N. Ullah, A. Mohammad, and A. A. Al-Ahmadi (2022) Adaptive Active Disturbance Rejection Control for Rendezvous of a Swarm of Drones. IEEE Access 10, p. 90355–90368. External Links: ISSN 2169-3536, Document Cited by: §I. [40] B. Zheng, M. Zhu, X. Guo, J. Ou, and J. Yuan (2024-07) Path planning of stratospheric airship in dynamic wind field based on deep reinforcement learning. Aerospace Science and Technology 150, p. 109173. External Links: ISSN 12709638, Document Cited by: §I.