Paper deep dive
Implicit Q-learning-bootstrapped ant colony optimization for maritime moving-target observation scheduling with agile satellites
He Wang, Junyu Wu, Yeye Liu, Yifan Zhou, Jie Zhang, Hui Li, Yanjie Song, Liang Li
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Maritime moving-target observation scheduling with agile Earth observation satellites is a dynamic, sequence-dependent combinatorial optimization problem. Sea-surface targets move continuously, causing feasible observation windows to vary with target motion and satellite orbital geometry. The scheduler must jointly determine task selection, satellite assignment, observation-window selection, and observation ordering under time-window, attitude-maneuvering, onboard-resource, and cloud-affected availability constraints. This paper proposes an implicit Q-learning-bootstrapped ant colony optimization method, termed IQACO, for multi-satellite maritime moving-target observation scheduling. Rather than directly learning a task-selection policy, IQACO embeds an offline implicit Q-learning module into constructive ant colony optimization to adaptively adjust the pheromone factor, heuristic factor, and evaporation rate. A compact search-state representation captures pheromone distribution, current and historical-best solution quality, and iteration progress. During online scheduling, ant colony optimization constructs feasible observation sequences, while the learned policy regulates exploration and exploitation according to the current search state. Experiments on 14 scenarios with different scales and satellite configurations show that IQACO obtains the highest mean observation benefit in every scenario, improves the result of conventional ant colony optimization by 3.40\%--9.40\%, accelerates convergence, and remains stable under different objective-weight settings. These results demonstrate that offline value learning provides an effective adaptive search-control mechanism for constrained maritime moving-target observation scheduling.
Tags
Links
- Source: https://arxiv.org/abs/2608.24471v1
- Canonical: https://arxiv.org/abs/2608.24471v1
Trouble viewing inline? Open PDF directly →
Full Text
58,233 characters extracted from source content.
Expand or collapse full text
Implicit Q-learning-bootstrapped ant colony optimization for maritime moving-target observation scheduling with agile satellites He Wang Email: wang_he@hrbeu.edu.cn Address: College of Intelligent Science and Engineering, Harbin Engineering University, Harbin 150001, China Junyu Wu Email: wujunyu@hrbeu.edu.cn Address: College of Intelligent Science and Engineering, Harbin Engineering University, Harbin 150001, China Yeye Liu Email: 20240042@nuc.edu.cn Address: School of Electrical and Control Engineering, North University of China, Taiyuan 030051, China Yifan Zhou Email: ercuniociao@163.com Address: College of Intelligent Science and Engineering, Harbin Engineering University, Harbin 150001, China Jie Zhang Email: jiezhang@hrbeu.edu.cn Address: College of Intelligent Science and Engineering, Harbin Engineering University, Harbin 150001, China Hui Li Email: lihuiheu@hrbeu.edu.cn Address: College of Intelligent Science and Engineering, Harbin Engineering University, Harbin 150001, China Yanjie Song Email: songyj_2017@163.com Address: School of Information Science and Technology, Dalian Maritime University, Dalian 116026, China Liang Li Email: liliang@hrbeu.edu.cn Corresponding author: Corresponding author. Address: College of Intelligent Science and Engineering, Harbin Engineering University, Harbin 150001, China Abstract Maritime moving-target observation scheduling with agile Earth observation satellites is a dynamic, sequence-dependent combinatorial optimization problem. Sea-surface targets move continuously, causing feasible observation windows to vary with target motion and satellite orbital geometry. The scheduler must jointly determine task selection, satellite assignment, observation-window selection, and observation ordering under time-window, attitude-maneuvering, onboard-resource, and cloud-affected availability constraints. This paper proposes an implicit Q-learning-bootstrapped ant colony optimization method, termed IQACO, for multi-satellite maritime moving-target observation scheduling. Rather than directly learning a task-selection policy, IQACO embeds an offline implicit Q-learning module into constructive ant colony optimization to adaptively adjust the pheromone factor, heuristic factor, and evaporation rate. A compact search-state representation captures pheromone distribution, current and historical-best solution quality, and iteration progress. During online scheduling, ant colony optimization constructs feasible observation sequences, while the learned policy regulates exploration and exploitation according to the current search state. Experiments on 14 scenarios with different scales and satellite configurations show that IQACO obtains the highest mean observation benefit in every scenario, improves the result of conventional ant colony optimization by 3.40%–9.40%, accelerates convergence, and remains stable under different objective-weight settings. These results demonstrate that offline value learning provides an effective adaptive search-control mechanism for constrained maritime moving-target observation scheduling. Keywords: Agile satellite scheduling , Maritime moving target , Ant colony optimization , Offline reinforcement learning , Implicit Q-learning 1 Introduction Maritime moving-target observation using agile Earth observation satellites (AEOSs) is important for vessel traffic monitoring, maritime search and rescue, illegal fishing detection, maritime security, and environmental surveillance. Compared with conventional satellites with limited pointing capability, AEOSs can rapidly slew their payloads toward sea-surface targets within limited visibility intervals. When multiple satellites are coordinated to observe numerous moving targets, the scheduling problem is no longer a simple target-selection problem, but a coupled decision-making problem involving satellite-task assignment, observation-window selection, observation sequencing, and feasibility checking under attitude-maneuvering and resource constraints [1, 2, 3]. The overall scheduling scenario is illustrated in Fig. 1. Figure 1: Schematic illustration of multi-satellite maritime moving target scheduling with time-varying observation windows, operational constraints, and IQL-guided adaptive ACO optimization. Maritime moving-target scheduling is more challenging than static-target scheduling. Sea-surface targets continuously change their relative geometry with satellite orbits and sensor footprints, making feasible observation windows strongly time dependent. In addition, schedule feasibility depends on the execution order of selected tasks because attitude-maneuvering time and maneuvering energy are required between consecutive observations. Frequent slews and high-resolution imaging further consume limited onboard energy and storage resources [4, 5, 6, 7]. For optical payloads, ocean cloud cover may reduce the practical value of geometrically visible windows [8, 9, 10]. These factors jointly lead to a large-scale, sequence-dependent, and resource-constrained combinatorial optimization problem. AEOS scheduling has been studied for more than two decades. Existing methods include exact optimization, heuristic and local-search methods, metaheuristics, and learning-based approaches [11]. Exact methods, such as integer programming, branch-and-bound, column generation, and constraint programming, can provide rigorous solutions for small or medium instances, but their computational cost increases rapidly when sequence-dependent transitions, onboard resources, and uncertainty are jointly considered [2, 12, 13]. Heuristic and local-search methods are efficient and easy to implement, but their performance often depends on hand-crafted rules and problem-specific assumptions [14, 15]. Metaheuristics, such as GA, PSO, WOA, ACO, memetic algorithms, and hybrid neighborhood search, have therefore been widely used for large-scale AEOS scheduling [1, 16, 17, 18, 19]. Among metaheuristics, ACO is naturally suitable for sequence construction because pheromone information and heuristic information can guide task-transition decisions. However, conventional ACO usually relies on fixed parameters, including the pheromone factor α, heuristic factor β, and evaporation rate ρ [20]. Fixed parameters may not maintain an appropriate exploration–exploitation balance when target density, satellite number, feasible-window distribution, and resource constraints change across scenarios. Therefore, an adaptive parameter-control mechanism is needed to adjust the search behavior according to the current optimization state. Reinforcement learning (RL) has recently attracted attention in AEOS scheduling because it can learn scheduling policies or value functions from data [21, 22, 23, 24, 25, 26]. However, many existing RL-based schedulers directly learn task-selection or sequence-generation policies. Such methods often face large discrete action spaces, dynamically changing feasible action sets, and limited generalization to scenarios with different target-window distributions. Moreover, stand-alone RL schedulers must explicitly handle hard operational constraints during every decision step, which increases the learning burden and may reduce feasibility reliability. Offline RL provides another possible way to introduce learning into satellite scheduling because it learns from collected decision trajectories without costly online trial-and-error [27, 28]. Implicit Q-learning (IQL) is particularly suitable for offline learning because it avoids explicit evaluation of out-of-distribution actions [29]. Nevertheless, directly using IQL as a stand-alone scheduler remains difficult for discrete, constraint-intensive, and sequence-dependent AEOS scheduling. A more practical strategy is to use IQL as a value-guided parameter controller within a constructive metaheuristic. In this way, the learning module adapts the search behavior, while the deterministic decoder remains responsible for feasible schedule construction. To address these issues, this article develops an implicit Q-learning-bootstrapped ant colony optimization method, termed IQACO, for multi-satellite maritime moving-target observation scheduling. IQACO uses an offline-trained IQL policy to adaptively adjust α, β, and ρ according to a compact five-dimensional search state. ACO remains responsible for constructing feasible schedules under time-window, attitude-maneuvering, energy, storage, and cloud-affected availability constraints. This design preserves the feasibility-oriented search capability of ACO while introducing value-guided exploration–exploitation control. The main contributions of this article are summarized as follows: 1. A multi-satellite maritime moving-target observation scheduling model is formulated by integrating time-varying observation windows, satellite-task-window assignment, sequence-dependent attitude maneuvering, onboard energy and storage resources, and cloud-affected availability. 2. An IQACO method is developed to adaptively regulate ACO parameters using offline IQL. The learned policy adjusts the pheromone factor, heuristic factor, and evaporation rate according to the current search state, improving exploration–exploitation balance during feasible schedule construction. 3. Comprehensive experiments are conducted on 14 scenarios with different problem scales and satellite configurations. The results verify the convergence performance, final observation benefit, objective-weight sensitivity, and training behavior of the proposed method. The remainder of this article is organized as follows. Section 2 formulates the scheduling model. Section 3 presents the proposed IQACO method. Section 4 reports the simulation experiments and result analysis. Section 5 concludes this article. 2 Scheduling Model 2.1 Problem Description We consider multi-satellite maritime moving-target observation scheduling over a finite planning horizon THT_H. The scheduling objects include a set of agile satellites and a set of maritime moving targets. Before optimization, target trajectories and satellite ephemerides are propagated to generate feasible task–satellite observation windows. The scheduler then determines task selection, satellite assignment, observation-window selection, and task ordering on each satellite under time-window, attitude-maneuvering, onboard energy, storage-capacity, and cloud-affected availability constraints. Compared with static-target scheduling, maritime moving-target observation has stronger temporal and spatial variability. The relative geometry between satellites and targets changes with both vessel motion and orbital motion, making feasible observation windows time dependent. A geometrically visible window may also have low practical availability due to ocean cloud cover. For agile satellites, schedule feasibility further depends on the execution order of selected tasks because attitude-maneuvering time and energy are required between consecutive observations. Therefore, the problem is a sequence-dependent and resource-constrained combinatorial optimization problem. 2.2 Sets and Parameters The main notation used in the scheduling model is listed in Table 1. An original moving target may have multiple observation requirements within the planning horizon; these requirements are expanded into independent scheduling tasks. If task i cannot be observed by satellite s, the corresponding feasible-window set i,sW_i,s is empty. Two binary variables are used to describe the scheduling decision. The variable xi,s,kx_i,s,k indicates whether task i is assigned to satellite s and executed in window k. It is set to one if the corresponding task–satellite–window combination is selected, and zero otherwise: xi,s,k=1,selected,0,otherwise.x_i,s,k= cases1,&selected,\\ 0,&otherwise. cases (1) The variable yijsy_ij^s describes the immediate-successor relation in the task sequence of satellite s. It is set to one if satellite s executes task j immediately after task i, and zero otherwise: yijs=1,if j immediately follows i,0,otherwise.y_ij^s= cases1,&if j immediately follows i,\\ 0,&otherwise. cases (2) Thus, xi,s,kx_i,s,k determines task selection, satellite assignment, and observation-window selection, while yijsy_ij^s describes the task order on each satellite. The latter is also used to compute the attitude-maneuvering time and energy between consecutive observations. Table 1: Notation used in the scheduling model Symbol Description T, S Sets of scheduling tasks and agile satellites i,sW_i,s Feasible observation-window set for task i and satellite s i,ji,j Task indices s Satellite index k,lk,l Observation-window indices NTN_T, NSN_S Numbers of tasks and satellites THT_H Planning-horizon length ti,s,kstartt_i,s,k^start Start time of window k ti,s,kendt_i,s,k^end End time of window k ti,s,kobst_i,s,k^obs Observation start time did_i, pip_i Observation duration and nominal benefit of task i ci,s,kc_i,s,k, cminc_ Cloud-affected availability factor and threshold θijs _ij^s Slewing angle from task i to task j Tijs,manT_ij^s,man Maneuvering time from task i to task j ωsmax _s Maximum angular rate of satellite s asmaxa_s Maximum angular acceleration of satellite s ei,s,kobse_i,s,k^obs Imaging energy eijs,mane_ij^s,man Maneuvering energy EsmaxE_s , EsuseE_s^use Energy capacity and energy use of satellite s mi,s,km_i,s,k, MsmaxM_s Generated data volume and storage capacity xi,s,kx_i,s,k Task-window assignment variable yijsy_ij^s Immediate-successor variable 2.3 Constraints A feasible schedule must satisfy operational constraints related to task assignment, observation windows, task sequencing, attitude maneuvering, onboard resources, and cloud-affected availability. These constraints are used as feasibility rules in the constructive decoder. 2.3.1 Task Uniqueness Constraint Each task is executed at most once: ∑s∈∑k∈i,sxi,s,k≤1,∀i∈. _s _k _i,sx_i,s,k≤ 1, ∀ i . (3) 2.3.2 Time-Window Feasibility Constraint The observation interval must be contained within the selected window: ti,s,kstart t_i,s,k^start ≤ti,s,kobs, ≤ t_i,s,k^obs, (4a) ti,s,kobs+di t_i,s,k^obs+d_i ≤ti,s,kend. ≤ t_i,s,k^end. (4b) These inequalities are enforced only for selected task–satellite–window nodes with xi,s,k=1x_i,s,k=1. 2.3.3 Sequence-Linking Constraint The sequence variable yijsy_ij^s is valid only when both tasks i and j are assigned to satellite s: yijs y_ij^s ≤∑k∈i,sxi,s,k, ≤ _k _i,sx_i,s,k, ∀i,j∈,i≠j,s∈, ∀ i,j ,\ i≠ j,\ s , (5) yijs y_ij^s ≤∑l∈j,sxj,s,l, ≤ _l _j,sx_j,s,l, ∀i,j∈,i≠j,s∈. ∀ i,j ,\ i≠ j,\ s . (6) Each scheduled task has at most one immediate successor and predecessor: ∑j∈j≠iyijs _ subarraycj \\ j≠ i subarrayy_ij^s ≤∑k∈i,sxi,s,k, ≤ _k _i,sx_i,s,k, ∀i∈,s∈, ∀ i ,\ s , (7) ∑i∈i≠jyijs _ subarrayci \\ i≠ j subarrayy_ij^s ≤∑l∈j,sxj,s,l, ≤ _l _j,sx_j,s,l, ∀j∈,s∈. ∀ j ,\ s . (8) These constraints define the consistency between task assignment and immediate-successor relations. The actual satellite-specific task sequences are generated explicitly by the constructive ACO procedure, where successor relations, temporal feasibility, and resource feasibility are checked during schedule construction. 2.3.4 Attitude Maneuvering Constraint For agile satellites, an attitude maneuver is required between two consecutive observations assigned to the same satellite. The required slewing angle θijs _ij^s is computed from the angular separation between the payload pointing directions of tasks i and j. A rate- and acceleration-limited maneuvering model is used, as illustrated in Fig. 2. If the required angle is large enough, the satellite reaches the maximum angular rate and follows a trapezoidal profile; otherwise, it follows a triangular profile. tω(t)ω(t)ωsmax _s tsacct_s^accTijs,manT_ij^s,man(a) Trapezoidalacc.const.dec.tω(t)ω(t)ωsmax _s Tijs,manT_ij^s,manωpeak _peak(b) Triangularacc.dec. Figure 2: Attitude-maneuvering angular-rate profiles. Let ωsmax _s and asmaxa_s denote the maximum angular rate and maximum angular acceleration of satellite s, respectively. The acceleration or deceleration time is tsacc=ωsmaxasmax.t_s^acc= _s a_s . (9) When the satellite can reach the maximum angular rate, the maneuvering time is Tijs,man=2tsacc+θijs−(ωsmax)2/asmaxωsmax,T_ij^s,man=2t_s^acc+ _ij^s-( _s )^2/a_s _s , (10) where θijs≥(ωsmax)2/asmax _ij^s≥( _s )^2/a_s . When the maximum angular rate cannot be reached, the maneuvering time is Tijs,man=2θijsasmax,T_ij^s,man=2 _ij^sa_s , (11) where θijs<(ωsmax)2/asmax _ij^s<( _s )^2/a_s . When task j immediately follows task i on satellite s and their selected windows are k and l, respectively, the observation start times must satisfy tj,s,lobs≥ti,s,kobs+di+Tijs,man.t_j,s,l^obs≥ t_i,s,k^obs+d_i+T_ij^s,man. (12) This constraint reserves sufficient transition time between consecutive observations on the same satellite. 2.3.5 Onboard Energy Constraint The total energy use, including imaging and maneuvering energy, must not exceed the onboard capacity: ∑i∈∑k∈i,sei,s,kobsxi,s,k+∑i,j∈i≠jeijs,manyijs≤Esmax,∀s∈. _i _k _i,se_i,s,k^obsx_i,s,k+ _ subarrayci,j \\ i≠ j subarraye_ij^s,many_ij^s≤ E_s , ∀ s . (13) 2.3.6 Onboard Storage Constraint The generated data volume must not exceed the storage capacity: ∑i∈∑k∈i,smi,s,kxi,s,k≤Msmax,∀s∈. _i _k _i,sm_i,s,kx_i,s,k≤ M_s , ∀ s . (14) 2.3.7 Cloud-Availability Constraint For optical observations, a geometrically visible window may still have low practical value due to cloud cover. Let ci,s,kc_i,s,k denote the cloud-affected availability of executing task i by satellite s in window k. Candidate windows with availability lower than the minimum acceptable threshold cminc_ are excluded: xi,s,k=0,ifci,s,k<cmin.x_i,s,k=0, c_i,s,k<c_ . (15) For retained windows, ci,s,kc_i,s,k is further used as a benefit attenuation factor in the objective function. 2.4 Objective Function Under the above feasibility constraints, the objective is to maximize the overall observation performance of the satellite constellation. Three normalized components are considered: observation benefit, energy efficiency, and workload balance. Normalization allows the weights to express mission preferences rather than compensate for different physical scales. The objective function is formulated as maxF=η1Fp+η2Fe+η3Fb, F= _1F_p+ _2F_e+ _3F_b, (16) where FpF_p, FeF_e, and FbF_b denote the normalized observation-benefit, energy-efficiency, and workload-balance terms, respectively. The objective weights satisfy η1+η2+η3=1,η1,η2,η3≥0. _1+ _2+ _3=1, _1, _2, _3≥ 0. (17) The normalized observation-benefit term is defined as Fp=∑i∈∑s∈∑k∈i,spici,s,kxi,s,k∑i∈pi.F_p= _i _s _k _i,sp_ic_i,s,kx_i,s,k _i p_i. (18) This term measures the effective benefit obtained from the selected tasks. The factor ci,s,kc_i,s,k reduces the benefit of a window with low cloud-affected availability. For satellite s, the energy use is composed of imaging energy and attitude-maneuvering energy: Esuse=∑i∈∑k∈i,sei,s,kobsxi,s,k+∑i,j∈i≠jeijs,manyijs.E_s^use= _i _k _i,se_i,s,k^obsx_i,s,k+ _ subarrayci,j \\ i≠ j subarraye_ij^s,many_ij^s. (19) The energy-efficiency term is then defined as Fe=1−∑s∈Esuse∑s∈Esmax.F_e=1- _s E_s^use _s E_s . (20) A larger FeF_e indicates lower relative energy use. The workload of satellite s is defined as the total duration of its selected observations: Ls=∑i∈∑k∈i,sdixi,s,k.L_s= _i _k _i,sd_ix_i,s,k. (21) The average workload of the constellation is L¯=1NS∑s∈Ls. L= 1N_S _s L_s. (22) The workload-balance term is defined as Fb=(1+1NS∑s∈(Ls−L¯)2L¯+ϵ)−1,F_b= (1+ 1N_S _s (L_s- L)^2 L+ε )^-1, (23) where ϵε is a small positive constant used to avoid division by zero. A larger FbF_b indicates a more balanced workload distribution among satellites. Therefore, the objective favors high-benefit and practically available observations while reducing relative energy consumption and avoiding excessive workload concentration on a small number of satellites. 3 Method 3.1 Method Overview IQACO consists of an offline IQL training stage and an online ACO scheduling stage, as shown in Fig. 3. In the offline stage, ACO is executed on training scenarios with exploratory parameter adjustments, and the resulting transitions (t,t,Rt,t+1,dt)(s_t,a_t,R_t,s_t+1,d_t) are collected to train an IQL policy. In the online stage, ACO constructs feasible schedules at each iteration, updates the best solution and pheromone matrix, and then uses the trained policy to adjust α, β, and ρ according to the current search state. Unlike direct RL schedulers that output discrete task-selection actions, IQACO only learns continuous parameter adjustments for ACO. Feasible schedule construction is still handled by the deterministic decoder, which reduces the learning burden and improves constraint satisfaction. Thus, IQL is used as a value-guided search controller rather than a replacement for the scheduling algorithm. Offline IQL training stage Online IQACO scheduling stageTrainingscenariosExploratoryACO runsOffline dataset(t,t,Rt,t+1,dt)(s_t,a_t,R_t,s_t+1,d_t)IQL trainingQ,V,πQ,V,π networksTrained policyπϕ() _φ(s)TestscenarioExtractsearch statets_tPolicy inferencet=πϕ(t)a_t= _φ(s_t)Updateα,β,ρα,β, scheduleconstructionEvaluate schedulesand update pheromoneBest scheduleΠ∗ Figure 3: Workflow of the proposed IQACO method. 3.2 Solution Encoding and ACO-Based Schedule Construction A candidate observation is represented by a task–satellite–window node z=(i,s,k)z=(i,s,k), where i, s, and k denote the task, satellite, and feasible observation window, respectively. Candidate nodes are pre-filtered according to visibility, time-window feasibility, and cloud-affected availability. A complete schedule is an ordered node list Π=z1,z2,…,z|Π| =\z_1,z_2,…,z_| |\, which can be decomposed into satellite-specific sequences Π=Π1,Π2,…,ΠNS =\ _1, _2,…, _N_S\, as shown in Fig. 4. The upper level of the representation stores the complete multi-satellite schedule, whereas the lower level preserves the chronological node sequence assigned to each satellite. Because each node explicitly records the selected task, satellite, and observation window, predecessor–successor relations can be recovered directly for maneuver-time, maneuver-energy, temporal-feasibility, and resource-feasibility checks. This hierarchical encoding therefore captures task selection, satellite assignment, window selection, and observation ordering within a single constructive representation. Π1 _1Π2 _2Π3 _3⋮ ΠNS _N_S(i1,1,k1)(i_1,1,k_1)(i3,1,k2)(i_3,1,k_2)(i7,1,k1)(i_7,1,k_1)k1k_1k2k_2k1k_1(i2,2,k1)(i_2,2,k_1)(i5,2,k3)(i_5,2,k_3)(i8,2,k2)(i_8,2,k_2)k1k_1k3k_3k2k_2(i4,3,k2)(i_4,3,k_2)(i6,3,k1)(i_6,3,k_1)(i9,3,k2)(i_9,3,k_2)k2k_2k1k_1k2k_2⋮ ⋮ ⋮ (ia,NS,kb)(i_a,N_S,k_b)(ic,NS,kd)(i_c,N_S,k_d)(ie,NS,kf)(i_e,N_S,k_f)kbk_bkdk_dkfk_f Π Figure 4: Hierarchical solution encoding. The global schedule is decomposed into chronological satellite-specific sequences of task–satellite–window nodes. Each ant incrementally selects schedulable tasks from a feasible candidate set mC^m that excludes candidates violating operational constraints. Algorithm 1 summarizes this construction. The evaporation rate ρ is applied only during global pheromone updating, not during single-ant construction. For the mmth ant, the next candidate node z=(iz,sz,kz)∈mz=(i_z,s_z,k_z) ^m is selected with probability Pm(z|u)=[τu,iz]α[ηm(u,z)]β∑q∈m[τuq,iq]α[ηm(uq,q)]β,P_m(z|u)= [ _u,i_z]^α[ _m(u,z)]^β _q ^m[ _u_q,i_q]^α[ _m(u_q,q)]^β, (24) where u is the latest scheduled task on satellite szs_z, τu,iz _u,i_z is the task-level pheromone intensity from u to candidate task izi_z, and ηm(u,z) _m(u,z) is the heuristic value of node z under the current partial schedule. For another candidate node q=(iq,sq,kq)q=(i_q,s_q,k_q) in the denominator, uqu_q denotes the latest scheduled task on satellite sqs_q. The parameters α and β control the relative influence of pheromone information and heuristic information. Algorithm 1 Schedule Construction by One Ant 1: Candidate observation nodes, pheromone matrix τ, and ACO parameters α and β 2: A feasible schedule Πm ^m 3: Initialize Πm←∅ ^m← , ←U , and satellite states with virtual initial nodes 4: while true do 5: Build the feasible candidate set mC^m from U 6: Remove candidates violating time-window, attitude-maneuvering, energy, or storage constraints 7: if m=∅C^m= then 8: break 9: end if 10: Compute the selection probability Pm(z|u)P_m(z|u) for each z∈mz ^m 11: Select a candidate node z=(iz,sz,kz)z=(i_z,s_z,k_z) by roulette-wheel selection 12: Insert z into the satellite-specific sequence Πszm ^m_s_z 13: Update the state of satellite szs_z, including its latest task and resource states 14: Update the unscheduled task set ←∖izU \i_z\ 15: end while 16: Merge all satellite-specific sequences into Πm ^m 17: return Πm ^m The heuristic value combines benefit contribution, energy effect, and workload balance: ηm(u,z)=χ1G(u,z)+χ2Em(u,z)+χ3Bm(z), _m(u,z)= _1G(u,z)+ _2E_m(u,z)+ _3B_m(z), (25) where G(u,z)G(u,z) denotes the benefit contribution of selecting candidate node z after task u, Em(u,z)E_m(u,z) denotes the energy-efficiency contribution considering the additional observation and maneuvering energy, and Bm(z)B_m(z) denotes the workload-balance contribution after inserting z. The coefficients χ1 _1, χ2 _2, and χ3 _3 are nonnegative heuristic weights. These terms are normalized before aggregation so that the heuristic value remains comparable across different scenarios and resource scales. After all ants have constructed their schedules, each schedule is evaluated by the objective function in Section 2. The pheromone matrix is then updated by evaporation and solution-quality-based deposition: τij←max(1−ρ)τij,τmin,τij←minτij+Δτij,τmax, _ij← \(1-ρ) _ij, _ \, _ij← \ _ij+ _ij, _ \, (26) with the deposition increment Δτij=∑m=1NAFmFmaxI((i,j)∈Πm), _ij= _m=1^N_A F^mF_ I ((i,j)∈ ^m ), (27) where NAN_A is the number of ants, FmF^m is the objective value of schedule Πm ^m, FmaxF_ is the current best objective, and I(⋅)I(·) is the indicator function. 3.3 Markov Decision Process Formulation The adaptive control of ACO parameters is formulated as an MDP ℳ=(,,,R,γ)M=(X,A,P,R,γ), where each decision step corresponds to one ACO iteration. After the ttth iteration, the search state is extracted from the pheromone matrix and current scheduling results. The IQL policy then outputs a continuous action to adjust the pheromone factor α, heuristic factor β, and evaporation rate ρ for the next iteration. Here, X is the state space, A is the action space, P is the transition process induced by one ACO iteration, R is the reward function, and γ is the discount factor. t=[τ¯t,στ,t2,ftcur,ftgb,rtiter]T,s_t= [ τ_t,\, _τ,t^2,\,f_t^cur,\,f_t^gb,\,r_t^iter ]^T, (28) where τ¯t τ_t and στ,t2 _τ,t^2 are the normalized mean and variance of the pheromone matrix, respectively. The variables ftcurf_t^cur and ftgbf_t^gb denote the best objective value of the current iteration and the global best objective value found so far, respectively. The variable rtiterr_t^iter is the normalized iteration ratio. These variables describe the pheromone distribution, current search quality, historical best performance, and search progress. The action is a continuous parameter-adjustment vector: t=[Δαt,Δβt,Δρt]T.a_t= [ _t,\, _t,\, _t ]^T. (29) After receiving the action, the ACO parameters are updated as αt+1 _t+1 =clip(αt+Δαt,αmin,αmax), =clip( _t+ _t, _ , _ ), (30) βt+1 _t+1 =clip(βt+Δβt,βmin,βmax), =clip( _t+ _t, _ , _ ), (31) ρt+1 _t+1 =clip(ρt+Δρt,ρmin,ρmax), =clip( _t+ _t, _ , _ ), (32) where clip(⋅)clip(·) restricts each parameter to its feasible range. The reward encourages solution improvement while maintaining search diversity. Let ft+1curf_t+1^cur be the best objective after applying the adjusted parameters. The improvement term is It=max(0,ft+1cur−ftgb).I_t= (0,\,f_t+1^cur-f_t^gb ). (33) The final reward is formulated as Rt=clip(cIIt+ξDt+κBt,Rmin,Rmax),R_t=clip (c_II_t+ξ D_t+κ B_t,\,R_ ,R_ ), (34) where DtD_t is a diversity-related term that measures the dispersion of the current search process, and BtB_t is a bonus term activated when a new global-best solution is obtained. The coefficients cIc_I, ξ, and κ balance immediate objective improvement, diversity preservation, and global progress. The clipping operation limits excessively large learning targets and improves the stability of offline training. 3.4 Offline IQL Training The offline dataset is collected by running ACO with exploratory parameter adjustments on training scenarios. During data collection, ACO first performs one iteration to obtain the initial search state. At each subsequent decision step, an exploratory action is sampled to update α, β, and ρ, and the next ACO iteration is executed with the updated parameters. The resulting reward, next state, and terminal indicator are recorded. The dataset is written as =(t,t,Rt,t+1,dt)t=1ND,D=\(s_t,a_t,R_t,s_t+1,d_t)\_t=1^N_D, (35) where dtd_t is the terminal indicator and NDN_D is the number of collected transitions. Algorithm 2 summarizes the transition collection process. Algorithm 2 Offline Transition Collection for IQL 1: Training scenarios, maximum iteration number TmaxT_ , and exploratory action strategy 2: Offline dataset D 3: Initialize ←∅D← 4: for each training scenario do 5: for each training episode do 6: Initialize an ACO scheduler 7: Run one ACO iteration and extract the initial state 0s_0 8: for t=0t=0 to Tmax−2T_ -2 do 9: Extract the current state ts_t 10: Sample an exploratory action ta_t 11: Update α, β, and ρ using ta_t 12: Run the next ACO iteration with the updated parameters 13: Compute the reward RtR_t 14: Extract the next state t+1s_t+1 15: Determine the terminal indicator dtd_t 16: Store (t,t,Rt,t+1,dt)(s_t,a_t,R_t,s_t+1,d_t) in D 17: end for 18: end for 19: end for 20: return D IQL learns a value function Vψ()V_ψ(s), two Q-functions Qϑ1(,)Q_ _1(s,a) and Qϑ2(,)Q_ _2(s,a), and a policy function πϕ() _φ(s) from D. The Q-learning target is yt=Rt+γ(1−dt)Vψ¯(t+1)y_t=R_t+γ(1-d_t)V_ ψ(s_t+1), where Vψ¯V_ ψ is the target value network. The Q loss is ℒQ(ϑ1,ϑ2)=∑j=12[(Qϑj(t,t)−yt)2].L_Q( _1, _2)= _j=1^2E_D [ (Q_ _j(s_t,a_t)-y_t )^2 ]. (36) The value function uses expectile regression with Q^=minjQϑj Q= _jQ_ _j: ℒV(ψ)=[Lτe(Q^(t,t)−Vψ(t))],Lτe(u)=|τe−(u<0)|u2. splitL_V(ψ)&=E_D [L_ _e ( Q(s_t,a_t)-V_ψ(s_t) ) ],\\ L_ _e(u)&=| _e-I(u<0)|u^2. split (37) The policy is trained by advantage-weighted regression with A(t,t)=Q^(t,t)−Vψ(t)A(s_t,a_t)= Q(s_t,a_t)-V_ψ(s_t): ℒπ(ϕ)=[exp(A(t,t)λ)‖πϕ(t)−t‖22].L_π(φ)=E_D [ ( A(s_t,a_t)λ ) \| _φ(s_t)-a_t \|_2^2 ]. (38) 3.5 Overall IQACO Procedure Algorithm 3 summarizes IQACO. In the offline stage, transitions are collected and used to train the IQL networks; the policy πϕ _φ is exported. In the online stage, ants construct schedules using current α and β, the best schedule and pheromone matrix are updated with ρ, and πϕ _φ outputs parameter adjustments [Δα,Δβ,Δρ]T[ α, β, ρ]^T for the next iteration. Algorithm 3 IQL-Bootstrapped Ant Colony Optimization 1: Training scenarios, test scenario, ACO settings, maximum iteration number TmaxT_ , and number of ants NAN_A 2: Best schedule Π∗ 3: Offline training stage 4: Collect transition dataset D using Algorithm 2 5: Train the IQL networks VψV_ψ, Qϑ1Q_ _1, Qϑ2Q_ _2, and πϕ _φ using D 6: Export the trained policy πϕ _φ 7: 8: Online scheduling stage 9: Initialize the ACO scheduler, pheromone matrix τ, parameters α, β, ρ, and best schedule Π∗←∅ ← 10: for t=0t=0 to Tmax−1T_ -1 do 11: for m=1m=1 to NAN_A do 12: Construct a feasible schedule Πm ^m using Algorithm 1 13: end for 14: Evaluate all schedules by the objective function 15: Update the best schedule Π∗ 16: Update the pheromone matrix using ρ 17: if t<Tmax−1t<T_ -1 then 18: Extract the search state ts_t 19: Obtain the action t=πϕ(t)a_t= _φ(s_t) 20: Update α, β, and ρ for the next iteration 21: end if 22: end for 23: return Π∗ Since the policy input is only five-dimensional and policy inference is performed once per ACO iteration, the additional online overhead of IQACO is small compared with schedule construction and feasibility checking. Table 2: Configuration of the experimental scenarios Scenario Original targets Satellites Scene 01 100 3 Scene 02 120 3 Scene 03 100 4 Scene 04 120 4 Scene 05 140 4 Scene 06 160 4 Scene 07 140 5 Scene 08 160 5 Scene 09 180 5 Scene 10 200 5 Scene 11 180 6 Scene 12 200 6 Scene 13 220 6 Scene 14 240 6 4 Simulation Experiments All algorithms are implemented in C++20 (compiled with -std=c++20 -O2 using MinGW-w64 GCC 13.2.0) and executed on a workstation with an Intel Core Ultra 9 285H and 64 GB RAM. The IQL module is implemented in Python with PyTorch; the trained policy is exported in ONNX format and loaded via the ONNX Runtime C++ API for CPU inference. 4.1 Scenario Configuration Fourteen testing scenarios are constructed with 100–240 maritime moving targets and 3–6 satellites, as listed in Table 2. Each original target has 2–4 observation requirements, which are expanded into independent scheduling tasks. The IQL policy is trained on separately generated scenarios with the same parameter ranges but different target distributions and satellite initial conditions, and the 14 scenarios in Table 2 are used only for testing. Targets move within a representative East Asian domain (φ∈[6∘,45∘] ∈[6 ,45 ], λ∈[105∘,145∘]λ∈[105 ,145 ]), with speed vi∼U(5,15)v_i U(5,15) m/s and direction-persistence probability ∼U(0.70,0.95) U(0.70,0.95) over a 24-h horizon (1-s trajectory step). The satellite cone angle is 25∘25 at 400 km altitude. Key parameters: di=60d_i=60 s, pi∈1,2,3p_i∈\1,2,3\, Esmax=500E_s =500 Wh, Psimg=750P_s^img=750 W, Psatt=30P_s^att=30 W, Msmax=2000M_s =2000 GB, Rsdata=4.0R_s^data=4.0 Gbps. Cloud-affected availability ci,s,kc_i,s,k is assigned by latitude: c=0.60c=0.60 for |φ|<10∘| |<10 , c=0.70c=0.70 for 10∘≤|φ|<25∘10 ≤| |<25 , c=0.80c=0.80 for 25∘≤|φ|<45∘25 ≤| |<45 , and used as cloud-affected availability factors in window screening and objective evaluation. 4.2 Compared Algorithms and Parameter Settings IQACO is compared with GA, PSO, WOA, and conventional ACO [30, 31, 32]. For fairness, all algorithms use the same scheduling model, objective function, constraint-checking procedure, and evaluation budget; only the search mechanism differs. Each algorithm is executed 20 times per scenario under the same stopping criterion (NFE=20000N_FE=20000). Baseline parameters follow commonly used empirical settings without problem-specific tuning. For IQACO, the initial ACO parameters are α=1.0α=1.0, β=2.0β=2.0, and ρ=0.1ρ=0.1. The IQL policy outputs [Δα,Δβ,Δρ]T[ α, β, ρ]^T with Δα∈[−0.2,0.2] α∈[-0.2,0.2], Δβ∈[−0.4,0.4] β∈[-0.4,0.4], and Δρ∈[−0.1,0.1] ρ∈[-0.1,0.1], clipped to α∈[1.0,5.0]α∈[1.0,5.0], β∈[1.0,5.0]β∈[1.0,5.0], and ρ∈[0.1,0.5]ρ∈[0.1,0.5]. IQL hyperparameters are listed in Table 3. Table 3: IQL training hyperparameters Parameter Description Value γ Discount factor 0.99 τe _e Expectile parameter 0.6 λ Inverse temperature 5.0 State dimension Input features 5 Action dimension Parameter-adjustment action 3 Epochs Training epochs 200 Learning rate Adam optimizer 3×10−43× 10^-4 4.3 Results and Discussion 4.3.1 Convergence Analysis The convergence behavior of the five algorithms is examined on six representative scenarios spanning moderate and large problem scales. Each algorithm is independently executed 20 times per scenario, and the best-so-far objective value is recorded. Figs. 5 and 6 report the mean convergence curves with one-standard-deviation bands, whereas Figs. 7 and 8 show the corresponding final-benefit distributions. Figure 5: Convergence comparison on Scenes 02, 05, and 08. The solid line denotes the mean best-so-far objective value over 20 independent runs, and the shaded band denotes ±1± 1 standard deviation. Figure 6: Convergence comparison on Scenes 10, 12, and 14. Solid lines show the mean best-so-far objective value over 20 independent runs, and shaded bands show ±1± 1 standard deviation. Figure 7: Distribution of the final observation benefit on Scenes 02, 05, and 08. The box denotes the interquartile range, the central mark denotes the median, the triangle denotes the mean, and the whiskers denote the most extreme nonoutlier values. Figure 8: Distribution of the final observation benefit on Scenes 10, 12, and 14. Each box summarizes 20 independent runs; the box denotes the interquartile range, the central mark denotes the median, the triangle denotes the mean, and the whiskers denote the most extreme nonoutlier values. Table 4: Statistical Summary of Final Observation Benefit Over 20 Independent Runs Scene GA PSO WOA ACO IQACO Gain 01 0.5857±0.01280.5857± 0.0128 0.6003±0.00440.6003± 0.0044 0.5867±0.01540.5867± 0.0154 0.6348±0.01000.6348± 0.0100 0.6564±0.0167‡0.6564± 0.0167 3.40%3.40\% 02 0.5647±0.01090.5647± 0.0109 0.5832±0.00440.5832± 0.0044 0.5737±0.01340.5737± 0.0134 0.6216±0.00840.6216± 0.0084 0.6434±0.0110‡0.6434± 0.0110 3.51%3.51\% 03 0.6155±0.01220.6155± 0.0122 0.6159±0.00330.6159± 0.0033 0.6109±0.01080.6109± 0.0108 0.6481±0.01050.6481± 0.0105 0.6666±0.0149‡0.6666± 0.0149 2.85%2.85\% 04 0.5900±0.01100.5900± 0.0110 0.6028±0.00340.6028± 0.0034 0.5884±0.01450.5884± 0.0145 0.6293±0.01270.6293± 0.0127 0.6518±0.0152‡0.6518± 0.0152 3.58%3.58\% 05 0.5798±0.01200.5798± 0.0120 0.5954±0.00400.5954± 0.0040 0.5866±0.01720.5866± 0.0172 0.6176±0.01190.6176± 0.0119 0.6494±0.0111‡0.6494± 0.0111 5.15%5.15\% 06 0.5588±0.01040.5588± 0.0104 0.5774±0.00610.5774± 0.0061 0.5565±0.01180.5565± 0.0118 0.5985±0.01210.5985± 0.0121 0.6343±0.0130‡0.6343± 0.0130 5.98%5.98\% 07 0.6038±0.00850.6038± 0.0085 0.6052±0.00250.6052± 0.0025 0.6026±0.00650.6026± 0.0065 0.6263±0.00930.6263± 0.0093 0.6719±0.0109‡0.6719± 0.0109 7.28%7.28\% 08 0.5876±0.00910.5876± 0.0091 0.5891±0.00380.5891± 0.0038 0.5854±0.01110.5854± 0.0111 0.6066±0.00960.6066± 0.0096 0.6523±0.0130‡0.6523± 0.0130 7.53%7.53\% 09 0.5616±0.00940.5616± 0.0094 0.5679±0.00310.5679± 0.0031 0.5551±0.00800.5551± 0.0080 0.5793±0.00850.5793± 0.0085 0.6254±0.0147‡0.6254± 0.0147 7.96%7.96\% 10 0.5485±0.01080.5485± 0.0108 0.5626±0.00620.5626± 0.0062 0.5452±0.01070.5452± 0.0107 0.5735±0.00870.5735± 0.0087 0.6186±0.0175‡0.6186± 0.0175 7.86%7.86\% 11 0.5731±0.00840.5731± 0.0084 0.5717±0.00330.5717± 0.0033 0.5698±0.00670.5698± 0.0067 0.5872±0.00980.5872± 0.0098 0.6381±0.0132‡0.6381± 0.0132 8.67%8.67\% 12 0.5678±0.00840.5678± 0.0084 0.5711±0.00300.5711± 0.0030 0.5629±0.00740.5629± 0.0074 0.5762±0.00680.5762± 0.0068 0.6151±0.0129‡0.6151± 0.0129 6.75%6.75\% 13 0.5580±0.00940.5580± 0.0094 0.5654±0.00370.5654± 0.0037 0.5570±0.00800.5570± 0.0080 0.5697±0.00810.5697± 0.0081 0.6158±0.0163‡0.6158± 0.0163 8.09%8.09\% 14 0.5440±0.00650.5440± 0.0065 0.5532±0.00350.5532± 0.0035 0.5434±0.00670.5434± 0.0067 0.5502±0.00820.5502± 0.0082 0.6019±0.0120‡0.6019± 0.0120 9.40%9.40\% ‡ indicates that IQACO is significantly better than all baseline algorithms at p<0.05p<0.05 by the Wilcoxon signed-rank test. Figs. 5 and 7 present the convergence behavior and final-benefit distributions for the moderate-scale cases (Scenes 02, 05, and 08). ACO-based methods generally outperform the other competing methods, confirming that constructive search is suitable for this sequence-dependent scheduling problem. Compared with conventional ACO, IQACO reaches higher best-so-far values and shifts the final-benefit distributions upward, indicating that IQL-guided parameter adjustment improves search quality across repeated runs. Fig. 6 extends the convergence comparison to the larger cases (Scenes 10, 12, and 14). IQACO reaches a higher best-so-far objective level than the four competing algorithms in all three scenes, and the final separation becomes larger as the number of targets and satellites increases. This behavior indicates that fixed ACO parameters become less effective when the feasible-node set and sequence-dependent transitions grow more complex, whereas the IQL controller can adjust the exploration–exploitation balance during the search. The corresponding distributions in Fig. 8 support the same conclusion: IQACO is concentrated at higher observation-benefit levels than the baseline algorithms. Together, these results show that adaptive parameter control is particularly beneficial for large, strongly sequence-dependent scheduling instances. Table 4 reports the complete statistical results on all 14 scenarios. IQACO obtains the highest mean observation benefit in every scenario, and the Wilcoxon signed-rank test confirms significance at p<0.05p<0.05. The gain over conventional ACO increases from 3.40%3.40\% in Scene 01 to 9.40%9.40\% in Scene 14, suggesting that adaptive parameter control becomes more beneficial as the scheduling scale and sequence-dependency complexity increase. 4.3.2 Weight Sensitivity Analysis Weight sensitivity is evaluated on Scene 01, 04, 07, and 11, which cover different constellation sizes from three to six satellites. Five weight configurations W1–W5 are tested: (η1,η2,η3)=(0.90,0.05,0.05)( _1, _2, _3)=(0.90,0.05,0.05), (0.80,0.10,0.10)(0.80,0.10,0.10), (0.70,0.15,0.15)(0.70,0.15,0.15), (0.60,0.20,0.20)(0.60,0.20,0.20), and (0.50,0.25,0.25)(0.50,0.25,0.25). Fig. 9 presents the mean and standard deviation over 10 runs. IQACO achieves the highest or near-highest values under most configurations. Under benefit-dominated settings, the differences among algorithms are relatively small. As the weights shift toward more balanced multi-objective preferences, the advantage of IQACO becomes more pronounced. This result indicates that adaptive parameter control helps maintain search performance when the objective emphasis changes from benefit maximization to joint consideration of benefit, energy efficiency, and workload balance. Figure 9: Weight sensitivity results on four representative scenarios under five weight configurations. 4.3.3 IQL Training Convergence Analysis To further examine the training behavior of the IQL module, the main loss functions and learning statistics are recorded during offline training. Figs. 10–12 show the variations of the total loss, Q loss, value loss, policy loss, action mean-square error, mean Q value, mean V value, mean advantage, and mean action weight with respect to the training epoch. The training curves show that the IQL module remains stable during offline learning. In the early training stage, the total loss and the main component losses decrease rapidly, indicating that the networks learn useful state–action value information from the offline transition samples. As the number of epochs increases, these losses gradually enter a bounded fluctuation range, and no evident divergence is observed. The decreases in policy loss and action mean-square error indicate that the policy network gradually fits parameter-adjustment actions with higher estimated advantages. The mean Q value and mean V value also become more stable in the later training stage, suggesting that the value estimates converge to relatively consistent levels. The mean advantage and mean action weight remain within reasonable ranges, which indicates that the advantage-weighted regression does not produce excessively large sample weights. Overall, the IQL module can learn a stable parameter-adjustment policy from offline ACO search trajectories and provide adaptive parameter control for online IQACO search on unseen test scenarios. Figure 10: Training loss terms of the IQL module. (a) Total loss. (b) Q loss. (c) V loss. Figure 11: Policy and advantage indicators of the IQL module. (a) Policy loss. (b) Action MSE. (c) Mean advantage. Figure 12: Value estimation and weight statistics. (a) Mean Q-value. (b) Mean V-value. (c) Mean weight. 5 Conclusion This article addressed multi-satellite maritime moving-target observation scheduling by developing an IQL-guided adaptive ACO framework. The proposed method improves constructive search by learning how to adjust key ACO parameters rather than directly generating scheduling decisions. In this way, the feasibility advantage of ACO-based schedule construction is preserved, while offline value learning is used to regulate the exploration–exploitation behavior during the search process. Experimental results on 14 scenarios show that IQACO achieves higher objective values and faster convergence than the competing methods. The performance gain becomes more evident in larger-scale scenarios, indicating that adaptive parameter control is particularly useful when the task-window distribution and sequence-dependent constraints become more complex. Weight-sensitivity experiments further show that IQACO maintains competitive performance under different mission-preference settings. However, the current study is still based on simulated target distributions, simplified cloud-availability modeling, and an offline policy trained within a fixed scenario distribution. Future work will focus on incorporating real AIS trajectories, time-varying cloud fields, and more realistic satellite operation constraints. Transfer learning, online adaptation, and hybrid learning-search mechanisms will also be investigated to improve robustness in operational maritime surveillance applications. Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Data availability The data and code supporting the findings of this study are available from the corresponding author upon reasonable request. References [1] X. Wang, G. Wu, L. Xing, and W. Pedrycz Agile Earth observation satellite scheduling over 20 years: Formulations, methods, and future directions IEEE Syst. J., vol. 15, no. 3, p. 3881–3892, Sep. 2021. [2] G. Peng, G. Song, Y. He, B. Deng, and S. Zhao An exact algorithm for agile Earth observation satellite scheduling with time-dependent profits Comput. Oper. Res., vol. 120, Aug. 2020. [3] G. Peng, G. Song, Y. He, J. Yu, S. Xiang, L. Xing, and P. Vansteenwegen Solving the agile Earth observation satellite scheduling problem with time-dependent transition times IEEE Trans. Syst., Man, Cybern., Syst., vol. 52, no. 3, p. 1614–1625, Mar. 2022. [4] V. Antuori, N. Beldiceanu, E. Hebrard, and D. Wojtowicz Solving the agile Earth observation satellite scheduling problem In Proc. 31st Int. Conf. Principles Practice Constraint Program., 2025. [5] A. Chatterjee and R. Tharmarasa Reward factor-based multiple agile satellites scheduling with energy and memory constraints IEEE Trans. Aerosp. Electron. Syst., vol. 58, no. 4, p. 3090–3103, Aug. 2022. [6] L. He, B. Liang, J. Li, and M. Sheng Joint observation and transmission scheduling in agile satellite networks IEEE Trans. Mobile Comput., vol. 21, no. 12, p. 4381–4396, Dec. 2022. [7] A. M. Mercado-Martínez, B. Soret, and A. Jurado-Navas An energy-efficient learning solution for the Agile Earth Observation Satellite Scheduling Problem In Proc. ICMLCN, Barcelona, Spain, 2025, p. 1–7. [8] X. Wang, G. Song, R. Leus, and C. Han Robust Earth observation satellite scheduling with uncertainty of cloud coverage IEEE Trans. Aerosp. Electron. Syst., vol. 56, no. 3, p. 2450–2461, Jun. 2020. [9] X. Wang, Y. Gu, G. Wu, and J. R. Woodward Robust scheduling for multiple agile Earth observation satellites under cloud coverage uncertainty Comput. Ind. Eng., vol. 156, Jun. 2021. [10] Y. Chen, J. Xue, W. Gu, and M. Shao An effective Genetic Programming Hyper-Heuristic for Uncertain Agile Satellite Scheduling In Proc. BigDIA, Nha Trang, Vietnam, 2025, p. 311–318. [11] B. Ferrari, J.-F. Cordeau, M. Delorme, M. Iori, and R. Orosei Satellite Scheduling Problems: A survey of applications in Earth and outer space observation Comput. Oper. Res., vol. 173, 2025. [12] L. He, X. Liu, G. Laporte, Y. Chen, and Y. Chen An improved adaptive large neighborhood search algorithm for multiple agile satellites scheduling Comput. Oper. Res., vol. 100, p. 12–25, Dec. 2018. [13] G. Peng, J. Wang, G. Song, A. Gunawan, L. Xing, and P. Vansteenwegen Branch-and-cut-and-price for agile earth observation satellite scheduling Eur. J. Oper. Res., vol. 326, no. 3, p. 427–438, 2025. [14] X. Liu, G. Laporte, Y. Chen, and R. He An adaptive large neighborhood search metaheuristic for agile satellite scheduling with time-dependent transition time Comput. Oper. Res., vol. 86, p. 41–53, Oct. 2017. [15] Y. Gu, C. Han, Y. Chen, and W. W. Xing Mission Replanning for Multiple Agile Earth Observation Satellites Based on Cloud Coverage Forecasting IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens., vol. 15, p. 594–608, 2022. [16] H. Wang, W. Huang, S. Magnússon, T. Lindgren, R. Wang, and Y. Song A Strategy Fusion-Based Multiobjective Optimization Approach for Agile Earth Observation Satellite Scheduling Problem IEEE Trans. Geosci. Remote Sens., vol. 62, 2024. [17] F. Yao, Y. Chen, L. Wang, Z. Chang, P.-Q. Huang, and Y. Wang A bilevel evolutionary algorithm for large-scale multiobjective task scheduling in multiagile Earth observation satellite systems IEEE Trans. Syst., Man, Cybern., Syst., vol. 54, no. 6, p. 3512–3524, Jun. 2024. [18] B. Wang, Y. Feng, G. Zhang, L. Zhang, and Y. Yang Memetic multiobjective discrete Jaya algorithm for cooperative scheduling of multiple agile Earth observation satellites IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 6, p. 8086–8099, Dec. 2024. [19] Y. Du, T. Wang, B. Xin, L. Wang, Y. Chen, and L. Xing A data-driven parallel scheduling approach for multiple agile Earth observation satellites IEEE Trans. Evol. Comput., vol. 24, no. 4, p. 679–693, Aug. 2020. [20] X. Zhou, H. Ma, J. Gu, H. Chen, and W. Deng Parameter adaptation-based ant colony optimization with dynamic hybrid mechanism Eng. Appl. Artif. Intell., vol. 114, Sep. 2022. [21] A. Herrmann and H. Schaub Reinforcement Learning for the Agile Earth-Observing Satellite Scheduling Problem IEEE Trans. Aerosp. Electron. Syst., vol. 59, no. 5, p. 5235–5247, Oct. 2023. [22] J. Chun, W. Yang, X. Liu, G. Wu, L. He, and L. Xing Deep reinforcement learning for the agile Earth observation satellite scheduling problem Mathematics, vol. 11, no. 19, Sep. 2023. [23] A. Jacquet, G. Infantes, N. Meuleau, E. Benazera, S. Roussel, V. Baudoui, and J. Guerra Earth observation satellite scheduling with graph neural networks arXiv:2408.15041, 2024. [24] Z. Liu, W. Xiong, C. Han, and X. Yu Deep reinforcement learning with local attention for single agile optical satellite scheduling problem Sensors, vol. 24, no. 19, Oct. 2024. [25] X. He, J. Xiang, M. Yan, C. Zhang, Z. Xie, and X. Liang Agile Earth observation satellite constellation mission planning based on multi-agent transformer IEICE Trans. Fundam. Electron., Commun. Comput. Sci., vol. E108-A, no. 9, p. 1316–1319, Sep. 2025. [26] L. Xu, S. Liu, and S. Qiu An autonomous mission planning method for Earth observation satellites based on reinforcement learning In Proc. IAF Earth Observation Symp., 76th Int. Astronautical Congr., Sydney, Australia, 2025, p. 605–611. [27] R. Figueiredo Prudencio, M. R. O. A. Maximo, and E. L. Colombini A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 8, p. 10237–10257, Aug. 2024. [28] A. Riahi Samani, X. Zhao, and F. Chen Distribution shift, generalization and OOD challenge in offline reinforcement learning: A comprehensive survey Neural Comput. Appl., vol. 38, 2026. [29] I. Kostrikov, A. Nair, and S. Levine Offline reinforcement learning with implicit Q-learning In Proc. Int. Conf. Learn. Representations, 2022. [30] A. Nait Chabane and O. Guenounou An enhanced genetic algorithm for optimized task allocation and planning in heterogeneous multi-robot systems Complex Intell. Syst., vol. 11, 2025. [31] L. Abualigah Particle Swarm Optimization: Advances, Applications, and Experimental Insights Comput. Mater. Contin., vol. 82, no. 2, p. 1539–1592, 2025. [32] L. Han, H. Zhou, Y. Zhang, Y. Wu, and M. Xu An enhanced whale optimization algorithm for task scheduling in edge computing environments Front. Big Data, vol. 7, 2024.