Paper deep dive
Reinforcement learning for post-coronagraphic wavefront control
Manuela Castañeda-Medina, Yann Gutierrez, Johan Mazoyer, Baptiste Abeloos, Laurent Mugnier, Olivier Herscovici-Schiller
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/21/2026, 3:40:27 AM
Summary
This paper presents a deep reinforcement learning (RL) method for post-coronagraphic wavefront control in high-contrast imaging of exoplanets. The approach uses an RL agent to control a deformable mirror by observing focal-plane images and physics-informed wavefront sensing data. The method is validated in simulations, demonstrating the ability to create dark holes (regions of suppressed starlight) with contrast levels approaching or exceeding traditional Pair-Wise Probing combined with Electric Field Conjugation (PWP+EFC) methods, while requiring fewer correction iterations.
Entities (7)
Relation Signals (6)
Reinforcement Learning → usedfor → Wavefront Control
confidence 95% · Deep reinforcement learning offers a promising alternative to traditional focal-plane wavefront control techniques
Reinforcement Learning → comparedwith → PWP+EFC
confidence 93% · we compare the resulting controller with PWP+EFC
Reinforcement Learning → controls → Deformable Mirror
confidence 92% · The agent controls a deformable mirror using observations consisting of focal-plane measurements
Reinforcement Learning → achieves → Dark Hole
confidence 90% · the method is validated... where it successfully creates dark holes
Asterix → usedforsimulation → Reinforcement Learning
confidence 88% · All experiments in this work were performed using the Asterix simulator
PWP+EFC → usedin → Roman Space Telescope
confidence 85% · It is part of the high-order wavefront sensing and control architecture developed and tested for the Roman Coronagraph Instrument
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Direct imaging of exoplanets is limited by the extreme contrast between the star and the planets, which is mitigated using a coronagraph. However, optical aberrations cause starlight leakage through the coronagraph, producing speckles that obscure the planetary signal. Achieving the required contrast levels demands wavefront control with subnanometric precision. Deep reinforcement learning offers a promising alternative to traditional focal-plane wavefront control techniques by enabling adaptive correction strategies learned directly from interaction with the system. In this work, we present a fully data-driven method for post-coronagraphic aberration correction in a simulated high-contrast imaging testbed. The agent controls a deformable mirror using observations consisting of focal-plane measurements (images) and physics-informed wavefront sensing information derived from these images. We evaluate different observation representations and control strategies, and the method is validated on simplified simulations of a high-contrast imaging testbed, where it successfully creates dark holes, i.e., regions of the focal plane in which residual starlight is strongly suppressed, while approaching the performance of conventional wavefront control methods.
Tags
Links
- Source: https://arxiv.org/abs/2609.20880v1
- Canonical: https://arxiv.org/abs/2609.20880v1
Trouble viewing inline? Open PDF directly →
Full Text
34,962 characters extracted from source content.
Expand or collapse full text
Reinforcement learning for post-coronagraphic wavefront control Manuela Casta ̃neda-Medina a,b,c , Yann Gutierrez a,b,c , Johan Mazoyer c , Baptiste Abeloos a , Laurent M. Mugnier b , and Olivier Herscovici-Schiller a a DTIS, ONERA, Universit ́e Paris-Saclay, 91120 Palaiseau, France b DOTA, ONERA, Universit ́e Paris Saclay, 92322 Chˆatillon, France c LIRA, Observatoire de Paris, Universit ́e PSL, Sorbonne Universit ́e, Universit ́e Paris Cit ́e, CNRS, 5 place Jules Janssen, 92195 Meudon, France ABSTRACT Direct imaging of exoplanets is limited by the extreme contrast between the star and the planets, which is mitigated using a coronagraph. However, optical aberrations cause starlight leakage through the coronagraph, producing speckles that obscure the planetary signal. Achieving the required contrast levels demands wavefront control with subnanometric precision. Deep reinforcement learning offers a promising alternative to traditional focal-plane wavefront control techniques by enabling adaptive correction strategies learned directly from inter- action with the system. In this work, we present a fully data-driven method for post-coronagraphic aberration correction in a simulated high-contrast imaging testbed. The agent controls a deformable mirror using ob- servations consisting of focal-plane measurements (images) and physics-informed wavefront sensing information derived from these images. We evaluate different observation representations and control strategies, and the method is validated on simplified simulations of a high-contrast imaging testbed, where it successfully creates dark holes, i.e., regions of the focal plane in which residual starlight is strongly suppressed, while approaching the performance of conventional wavefront control methods. Keywords: Exoplanet imaging, Wavefront control, Deep reinforcement learning, Dark hole, High-contrast imag- ing 1. INTRODUCTION To date, more than 6,000 exoplanets have been detected using a variety of observational techniques. Direct imaging aims to capture photons emitted or reflected by the planet itself, 1 enabling spectroscopic analysis of exoplanet atmospheres and the detection of molecular signatures related to their composition and physical con- ditions. However, direct imaging remains extremely challenging because exoplanets can be up to ten orders of magnitude fainter than their host stars and are observed at small angular separations. Coronagraphs suppress the stellar core and diffracted light, but optical aberrations introduced along optical paths not shared by the tele- scope’s sensing and science channels, known as non-common-path aberrations (NCPAs), cause residual starlight to leak through the coronagraph. Slowly evolving aberrations produce quasi-static speckles in the focal plane that can mimic or obscure planetary signals. Active wavefront sensing and control is therefore required to estimate and correct these aberrations using deformable mirrors, creating a dark hole, i.e., a region in the focal plane where residual starlight is strongly suppressed. 2 Although these challenges concern both ground- and space-based observatories, this work focuses on space- based applications. The Coronagraph Instrument aboard the Nancy Grace Roman Space Telescope will provide an in-space demonstration of active coronagraphy and wavefront control, 3 paving the way for the Habitable Worlds Observatory (HWO), which aims to directly image and characterize Earth-like exoplanets around nearby Sun-like stars. 4 Classical approaches, such as Pair-Wise Probing combined with Electric Field Conjugation Send correspondence to lauramanuela.castanedamedina@onera.fr arXiv:2609.20880v1 [astro-ph.IM] 16 Sep 2026 (PWP+EFC), rely on a linearized optical model to estimate and cancel speckles and constitute a state-of-the- art baseline for high-contrast coronagraphy. 5 These methods perform well under accurate calibration but can degrade under model mismatch, temporal drifts, or measurement noise. In this work, dark-hole generation is framed as a sequential decision problem solved with reinforcement learning (RL). 6 The primary objective is to investigate model-free wavefront control, in which the agent learns the correction strategy rather than computing each DM command by explicitly inverting a linearized optical model. In the present implementation, however, the policy observations still include model-dependent information obtained through PWP and EFC. This approach aims to reduce the number of correction iterations and, when possible, improve the final contrast, while exploring robust strategies for future space telescopes dedicated to exoplanet characterization. 2. METHOD 2.1 Reinforcement Learning Reinforcement learning (RL) is a framework where an agent learns to make decisions by interacting with an environment in order to maximize a reward signal. 6 RL problems are commonly modeled as a Markov decision process, defined by (S,A,P,R,γ),(1) where S is the state space, A the action space, P (s ′ |s,a) the transition probability, R(s,a) the reward function, and γ ∈ [0, 1] the discount factor. The agent’s behavior is described by a policy π(a|s), which is optimized to maximize the expected return. Reinforcement learning has previously been investigated for adaptive optics control, particularly to address temporal delays, calibration errors, and nonlinearities in ground-based systems. Nousiainen et al. developed a model-based RL controller for atmospheric turbulence correction and demonstrated it in both numerical simula- tions and laboratory experiments. 7–9 In parallel, Guti ́errez et al. introduced a model-free, image-based approach in which an RL agent learns to estimate and correct non-coronagraphic wavefront aberrations directly from focal-plane phase-diversity images. 10 This approach was subsequently extended to post-coronagraphic wavefront control by Guti ́errez et al., 11 demonstrating the potential of RL agents to control a deformable mirror and create a dark hole from focal- plane measurements. Building on this work, we investigate model-free RL for dark-hole creation in a simplified coronagraphic simulation. We focus on the design of the observations and reward function, compare the resulting controller with PWP+EFC, and evaluate its initial robustness to photon noise. In our approach, wavefront correction is framed as an RL problem in which the environment represents the coronagraphic optical system, the actions correspond to commands applied to the DM actuators, and the observations combine focal-plane images with physics-informed wavefront sensing and control information derived from these images. The reward is computed from the residual intensity, or contrast, within the dark hole. At each interaction step, the agent proposes a DM command, which is propagated through the optical system to produce a new focal-plane image. The resulting observation and reward define a transition that is used to train the policy. 2.2 RL-based wavefront control Wavefront correction is formulated as an RL problem in which the environment represents the coronagraphic optical system. An episode begins with a newly generated static entrance aberration and consists of five correction steps. At each step, the agent receives an observation, proposes an incremental command for the deformable mirror, and observes the resulting coronagraphic focal-plane measurements. The observation contains four complementary components: the current science image, the complex electric field estimated through pair-wise probing, the change in the estimated electric field with respect to the preceding step, and the correction proposed by EFC. The science image provides a direct measurement of the residual intensity, whereas the electric-field estimate contains phase and amplitude information that cannot be recovered from a single intensity image. The electric-field variation provides information about the response of the optical system to the preceding action, while the EFC command supplies a physics-informed correction based on the synthetic optical model. Telescope Deformable mirror Agent Coronagraph Detector Data pre-processing (Observation) Aberrations A t (DM actuator commands) O t R t Environment Figure 1: RL pipeline in wavefront control The action is a continuous vector containing one incremental command for each controlled DM actuator. The policy output is bounded to [−1, 1] and scaled by the environment before being added to the current DM shape. In the configuration considered here, the action space therefore contains 62 controlled degrees of freedom. Finally, the reward encourages both the attainment of deep contrast and the improvement obtained during the current correction step. Denoting by C t the mean contrast inside the dark hole after step t, we studied several contrast-based metrics and found that the following reward gave the best results: R t = [− log 10 (C t )] log 10 C t−1 C t .(2) The first factor favors low absolute contrast, while the second rewards an improvement relative to the preceding step. Consequently, an action that reduces the dark-hole contrast receives a positive reward, whereas an action that degrades it receives a negative reward. 2.3 PWP + EFC Pair-wise probing combined with Electric Field Conjugation (PWP+EFC) is a classical wavefront sensing and control approach widely used for high-contrast coronagraphic imaging. Because non-common-path aberrations must be sensed at the science detector itself, PWP estimates the complex electric field directly in the corona- graphic focal plane rather than relying on a separate wavefront sensor. Its objective is to estimate this field and compute the deformable mirror commands required to suppress residual starlight within the dark hole. The first step, pair-wise probing, estimates the complex electric field associated with the speckle pattern. A synthetic linear model of the optical system is used to predict the focal-plane electric field generated by a known DM probe pattern. Each probe is applied with both positive and negative signs, producing the intensity images I ± k , which can be modeled, within a first-order Taylor expansion, by I ± k =|E± iC[Aφ k ]| 2 ,(3) where E is the unknown focal-plane electric field, A is the unaberrated pupil-plane electric field, φ k is the phase introduced by the k-th DM probe, and C denotes propagation from the pupil plane to the coronagraphic focal plane. Hence, iC[Aφ k ] is the modeled focal-plane field introduced by the probe. Subtracting the two images cancels the terms that are unchanged between the exposures, including the unprobed speckle intensity and the probe intensity, yielding I + k − I − k = 4 (Re (E) Re (iC[Aφ k ]) + Im (E) Im (iC[Aφ k ])).(4) Thus, the intensity difference isolates the interference between the unknown speckle field and the known probe field. Because a detector measures intensity rather than the complex field directly, at least two probe pairs producing linearly independent focal-plane modulations are required to recover the real and imaginary compo- nents of E at each pixel. The resulting linear system is inverted using the synthetic probe model to obtain the electric-field estimate. A detailed description of the PWP formalism and its implementation is provided by Potier et al. 12 Once the electric field has been estimated, Electric Field Conjugation computes the DM command that minimizes the residual intensity within the dark hole. A synthetic linear model of the optical system is used to construct the interaction matrix G, which describes the linearly approximated change in the focal-plane electric field produced by each DM actuator. Under this linear approximation, the corrected electric field is written as E c = b E + Gu,(5) where b E is the estimated electric field, G is the interaction matrix, and u is the vector of DM commands. Equation (5) can be minimized using different inverse-problem strategies. In EFC, singular value decomposition (SVD) is used to compute a regularized pseudoinverse of G, yielding the control matrix G † . Regularization limits the contribution of poorly constrained modes and prevents noise amplification. A detailed derivation of the EFC formalism, as well as a discussion of other inversion approaches, is provided by Groff et al. 13 At each iteration, PWP estimates the current electric field, EFC computes the corresponding correction, and the DM is updated. PWP+EFC has demonstrated strong performance under accurate calibration and stable conditions. It is part of the high-order wavefront sensing and control architecture developed and tested for the Roman Coronagraph Instrument 3 and has been demonstrated on sky with VLT/SPHERE, 14 making it a reference method for model-based focal-plane wavefront control. However, PWP+EFC performance depends on the validity of two linear approximations and on the accuracy of the synthetic optical model. First, PWP relies on a first-order approximation of the field introduced by the probes; at large probe amplitudes, neglected higher-order terms can bias the electric-field estimate. Second, EFC relies on a linearized relationship between DM commands and changes in the focal-plane electric field; at large DM strokes, this approximation can become inaccurate and limit loop convergence. 15 Even within these linear regimes, discrepancies between the model and the real instrument can produce inaccurate electric-field estimates and suboptimal EFC commands. 16 3. EXPERIMENTAL SETUP 3.1 Optical simulation All experiments in this work were performed using the Asterix simulator, a Python-based library for high- contrast imaging simulations. 17 The simulated monochromatic wavelength was λ 0 = 500 nm. The entrance pupil was circular and unobstructed, sampled on an 80-pixel-diameter pupil plane, with the actuator-grid configuration setting the effective pupil diameter to 2.4 m. The science detector contained 48× 48 pixels sampled at 3 pixels per λ/D, corresponding to a 16λ/D field of view. The coronagraph was enabled with an FQPM focal-plane mask, and performance was evaluated in a full square dark hole spanning 0–4λ/D. Each episode used a newly generated static phase-only entrance aberration, represented by a phase screen with RMS phase σ φ = 0.1 rad. The controller acted on the valid pupil actuators of an 8× 8 DM3 actuator grid; after pupil thresholding, this yielded 62 controlled degrees of freedom. To assess the noise-robustness of our method, we performed a set of experiments by introducing photon noise (described in Section 4.2). In these simplified experiments, the goal was to adjust the number of photon so that the SNR of the speckles remains at a constant over the whole correction. The requested SNR was defined as the mean Poisson SNR over the dark hole: SNR DH = D p I i βN tel E i∈DH ,(6) where I i is the noiseless contrast-normalized intensity in dark-hole pixel i, N tel is the number of photons entering the telescope during an exposure and β is the conversion factor between contrast-normalized intensity and the fraction of entering telescope photons expected in a detector pixel. Thus, the expected photon count in pixel i is μ i = I i βN tel .(7) At reset, N tel was calibrated from the initial dark-hole image to obtain the target value of SNR DH . After each correction step, it was updated according to N tel (t) = N tel (0) C DH (0) C DH (t) ,(8) where C DH (t) is the mean dark-hole contrast at step t. Equation (8) means that, in order to simulate an experiment where the mean dark-hole SNR remains approx- imately constant, the effective photon count is linearly increased as the contrast decreases. In a real experiment with a fixed-magnitude source, this would be achieved by linearly increasing the exposure time. Photon noise was then generated by drawing n i ∼ Poisson(I i βN tel )(9) and converting the resulting photon counts back to contrast units by dividing by βN tel . 3.2 Training and Evaluation Setup At each environment reset, an entrance-pupil aberration was independently generated as a zero-mean Gaussian random phase screen with spatial power spectral density PSD(ρ)∝ " 1 + ρ ρ c 2 # −1 , ρ c = 4.3,(10) where ρ = q k 2 x + k 2 y ,(11) k x and k y are the discrete Fourier-frequency indices of the actuator grid. Thus, ρ and the characteristic frequency ρ c are expressed in FFT bins. If the 8×8 actuator grid is assumed to span one pupil diameter, ρ c = 4.3 corresponds approximately to 4.3 cycles per pupil, or to a characteristic spatial scale of D/4.3 ≃ 0.23D. The phase screen was synthesized in the 8× 8 actuator influence-function basis, with piston removed, and rescaled to an RMS phase of 0.1 rad over the entrance pupil. Each realization remained static throughout the corresponding episode. Training episodes contained at most five control steps, with early termination enabled. Two pair-wise probe patterns were used, each applied with positive and negative signs. The amplitude of each signed probe command was 20 nm in optical path difference, corresponding to a 40 nm difference between the positive and negative commands. The policy generated incremental commands for the 62 active DM actuators located within the entrance pupil. Training was allowed to run for a maximum of 5×10 7 environment interactions but could be stopped earlier once convergence was observed. The results reported here were obtained using the best saved policy checkpoint, selected during training, at 6.6× 10 6 environment interactions. The selected policy was evaluated without further parameter updates. Actions were generated deterministi- cally, with no explicit action noise, and generalized state-dependent exploration was disabled. The evaluation comprised 100 independently sampled episodes, each initialized with an aberration drawn from the same distri- bution as during training and held static throughout the episode. Episode length was set at 5 steps, for both training and evaluation. Performance was quantified by the arithmetic mean of the normalized detector intensity over the pixels within the dark hole. The initial uncorrected contrast and the contrast after each correction step were recorded. At every step, the distribution across the 100 episodes was summarized by its median and 25th–75th percentile interval. Once the RL configuration had been selected, its performance was compared with the PWP+EFC reference. For each of the 100 comparison episodes, the RL and PWP+EFC environments were reset using the same seed, producing matched aberration realizations for the two controllers. Both methods used the same optical configuration and were evaluated over five correction steps. Here, one correction step denotes one policy action for the RL controller and one EFC correction for PWP+EFC. In both cases, the electric-field estimate used at each step was obtained from two probe pairs, requiring four probed detector images, in addition to the unprobed science image. 4. RESULTS 4.1 Dark-Hole Creation with RL Figure 2 shows the initial and final optical states during one representative episode of five control steps. Fig- ure 2(a) shows the initial entrance aberration at the start of the episode, while Fig. 2(b) shows the final residual phase after correction. The residual phase exhibits the spatial structure imposed by the deformable-mirror correction applied to suppress the stellar residuals in the controlled region. Initial aberration 0.3 0.2 0.1 0.0 0.1 0.2 0.3 rad (a) Initial science frame 10 9 10 8 10 7 10 6 10 5 10 4 10 3 intensity (c) EEF real 0.0004 0.0003 0.0002 0.0001 0.0000 0.0001 0.0002 0.0003 obs (e) Final residual phase 0.004 0.002 0.000 0.002 0.004 rad (b) Final science frame 10 14 10 13 10 12 10 11 10 10 10 9 intensity (d) EEF imag 0.0002 0.0001 0.0000 0.0001 0.0002 obs (f) Figure 2: Initial and final optical states during dark-hole generation. (a) Initial entrance aberration at the start of the episode. (b) Final residual phase after correction. (c) Initial science frame before correction. (d) Final science frame after the RL control sequence. (e) Real part of the residual estimated electric field ( ˆ E) after five control steps. (f) Imaginary part of the residual estimated electric field ( ˆ E) after five control steps. Figures 2(c) and 2(d) show the corresponding science frames before and after correction. The speckle field is initially distributed inside the dark-hole region, leading to a high residual intensity. After the RL control sequence, the final science frame shows the resulting dark hole, where the intensity has been strongly reduced with respect to the initial image. Figures 2(e) and 2(f) show the real and imaginary parts of the residual estimated electric field after five control steps. The visible pixelation reflects imperfections in the PWP estimation in regions where the probes do not sufficiently modulate the electric field, making the corresponding estimation system poorly conditioned. The residual estimated electric field presents the dark-hole geometry, with a significantly reduced magnitude inside the controlled region. These qualitative results show that the learned policy is able to generate a dark hole within a short correction sequence. Figure 3 shows the evolution of the mean dark-hole contrast during training. The agent required approximately 1.2× 10 6 episodes to reach its best performance. The first stages of training were the most efficient: within the first 3× 10 5 interactions, the contrast decreased from about 10 −4 to 10 −9 . After this initial improvement, further optimization became progressively harder. The last two orders of magnitude required most of the remaining training time, showing that each additional contrast gain becomes increasingly difficult at deeper contrast levels. With an appropriate learning-rate schedule, the agent was able to continue improving and eventually reached a contrast of approximately 4× 10 −11 . 0.00.20.40.60.81.01.2 Episodes ×10 6 10 −10 10 −9 10 −8 10 −7 10 −6 10 −5 10 −4 10 −3 Average contrast Training curves - Average Contrast SAC (5 it) PWP+EFC (100 it) PWP+EFC (5 it) Figure 3: Evolution of the mean dark-hole contrast during SAC training. Figure 4 compares the contrast evolution obtained with the RL controller and the classical PWP+EFC approach over 100 independent episodes. For each correction step, the marker represents the median dark-hole contrast reached by each method, while the vertical bars indicate the interquartile range between the 25th and 75th percentiles. The RL controller reaches deeper contrast levels in fewer iterations. By the fourth correction step, it achieves a contrast more than one order of magnitude lower than PWP+EFC. However, the final correction step appears more difficult for the learned policy: the improvement becomes marginal and, in some cases, the contrast slightly degrades. As a result, the final gap between RL and PWP+EFC is reduced, with PWP+EFC remaining within about one order of magnitude of the RL performance at the end of the episode. The dispersion of the results also differs between the two methods. PWP+EFC shows a small interquartile range at all correction steps, indicating stable and consistent behavior across episodes. In contrast, the RL controller presents a larger spread, especially during the last two steps, suggesting that its performance becomes more sensitive near the deepest-contrast regime. 012345 Decision step within episode 10 −11 10 −10 10 −9 10 −8 10 −7 10 −6 10 −5 10 −4 Average contrast RL vs EFC contrast over 100 episodes RL median EFC-only median Figure 4: Evolution of the dark-hole contrast obtained with the RL controller and PWP+EFC over 100 inde- pendently sampled aberrations. Markers indicate the median, and vertical bars show the 25th–75th percentile interval. 4.2 Photon Noise Figure 5 shows an initial evaluation of the learned policy under photon noise, comparing the noiseless case, denoted as SNR =∞, with noisy observations at SNR = 30 and SNR = 10. Overall, the performance degradation remains limited. For SNR = 30, the contrast evolution is close to the noiseless case, while for SNR = 10 the final contrast remains within less than one order of magnitude of the noiseless performance. The main effect of photon noise is observed in the convergence behavior. Without noise, the agent reaches the deepest contrast levels more rapidly and follows a smoother correction path. In the noisy cases, the controller may require more steps to identify an effective correction direction, leading to a slower decrease of the dark-hole contrast. Nevertheless, these preliminary results suggest that the learned policy retains a significant part of its correction capability under moderate photon noise. 5. CONCLUSION This work presented a proof of concept for reinforcement-learning-based dark-hole generation in a simplified high-contrast imaging simulation. The objective was to investigate whether a model-free controller could learn to control a deformable mirror directly from image-based feedback and generate a dark hole without relying on an explicit linearized optical model. Achieving stable learning required several design choices, including reward optimization, hyperparameter tuning, observation-space design, and learning-rate management. Under the simplified simulation conditions considered here, the trained RL controller was able to create a dark hole within five correction steps and reach deep contrast levels. Compared with the classical PWP+EFC baseline, the RL controller reached lower contrast faster, achieving an improvement of more than one order of magnitude by the fourth correction step. An initial photon-noise study also showed limited degradation at moderate noise levels, suggesting that the learned policy retains part of its correction capability under noisy observations. 0.00.20.40.60.81.01.2 Episodes ×10 6 10 −10 10 −9 10 −8 10 −7 10 −6 10 −5 10 −4 10 −3 Average contrast Training curves - Average Contrast SNR =∞ SNR = 30 SNR = 10 PWP+EFC (100 it) PWP+EFC (5 it) Figure 5: Training curve SAC These results remain preliminary and were obtained under idealized simulation conditions. The next step is to progressively introduce more realistic effects and evaluate the robustness of the learned controller. Since PWP+EFC relies on a calibrated linear model, its performance is expected to become increasingly sensitive to model mismatch, temporal drifts, vibrations, misalignments, and other experimental uncertainties. In contrast, the model-free nature of RL may provide improved robustness if such effects are represented during training, for example through observation or domain randomization. The long-term objective is experimental validation on the THD2 testbed at Observatoire de Paris–PSL. 18 Three main challenges must be addressed to achieve this objective. First, the current framework must be extended from phase-only aberrations to mixed phase-and-amplitude aberrations. In this configuration, correction with a single deformable mirror requires restricting the dark hole to one side of the focal plane. Second, the number of controlled modes must be increased to represent realistic high-order wavefront errors, despite the resulting growth in the dimensionality of the action space and the associated learning difficulty. Finally, transferring the controller to the experimental system will require strategies such as transfer learning or fine-tuning to adapt the simulated policy to the characteristics and uncertainties of the testbed. Addressing these challenges will be essential to determine whether reinforcement learning can become a robust alternative or complement to model-based wavefront-control strategies for future exoplanet imaging missions. ACKNOWLEDGMENTS The Ph.D. work of Manuela Casta ̃neda-Medina is co-funded by CNES and ONERA. This work was carried out as part of a doctoral research project jointly hosted by ONERA and the Observatoire de Paris–PSL. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon Europe research and innovation programme under the grant agreement #101230218. Software - This study made use of the following Python packages: Asterix, 17 Astropy, 19 Matplotlib, 20 NumPy, 21 PyTorch, 22 SciPy, 23 and Stable-Baselines3. 24 OpenAI Codex was used to assist with code development, debugging, and refactoring, as well as with language editing and manuscript revision. All AI-assisted content and code was reviewed, tested, and validated by the authors. REFERENCES [1] Currie, T., Biller, B., Lagrange, A.-M., Marois, C., Guyon, O., Nielsen, E., Bonnefoy, M., and Rosa, R. J. D., “Direct Imaging and Spectroscopy of Extrasolar Planets,” in [Protostars and Planets VII ], Inutsuka, S., Aikawa, Y., Muto, T., Tomida, K., and Tamura, M., eds., Astronomical Society of the Pacific Conference Series 534, 799–850, Astronomical Society of the Pacific (2023). [2] Galicher, R. and Mazoyer, J., “Imaging exoplanets with coronagraphic instruments,” Comptes Rendus. Physique 24(S2), 69–113 (2023). [3] Cady, E., Bowman, N., Greenbaum, A. Z., Ingalls, J. G., Kern, B., Krist, J., Marx, D., Poberezhskiy, I., Riggs, A. J. E., Ruane, G., Seo, B.-J., Shi, F., and Zhou, H., “High-order wavefront sensing and control for the Roman Coronagraph Instrument (CGI): architecture and measured performance,” Journal of Astronomical Telescopes, Instruments, and Systems 11(2), 021408 (2025). [4] Feinberg, L. D., Sitarski, B. N., McElwain, M. W., Arney, G., Baker, C., Bolcar, M. R., Levine, M., Liu, A., Mennesson, B., Roberge, A., Smith, J. S., Zhao, F., and Ziemer, J., “Habitable Worlds Observatory’s Concept and Technology Maturation: Initial Feasibility and Trade Space Exploration,” (2026). [5] Give’on, A., Belikov, R., Shaklan, S., and Kasdin, N. J., “Closed loop, DM diversity-based, wavefront correction algorithm for high contrast imaging systems,” Optics Express 15(19), 12338–12343 (2007). [6] Sutton, R. S. and Barto, A. G., [Reinforcement Learning: An Introduction ], MIT Press, Cambridge, MA, 2 ed. (2018). [7] Nousiainen, J., Rajani, C., Helin, T., Correia, C. M. W., Berdeu, T. B., V ́erinaud, C., Heritier, C. T., and Kasper, M., “Adaptive optics control using model-based reinforcement learning,” Optics Express 29(10), 15327–15344 (2021). [8] Nousiainen, J., Rajani, C., Helin, T., Heritier, C. T., and Kasper, M., “Toward on-sky adaptive optics control using reinforcement learning: Model-based policy optimization for adaptive optics,” Astronomy & Astrophysics 664, A71 (2022). [9] Nousiainen, J., Engler, B., Kasper, M., Rajani, C., Helin, T., Heritier, C. T., Quanz, S. P., and Glauser, A. M., “Laboratory experiments of model-based reinforcement learning for adaptive optics control,” Journal of Astronomical Telescopes, Instruments, and Systems 10(1), 019001 (2024). [10] Gutierrez, Y., Mazoyer, J., Mugnier, L. M., Herscovici-Schiller, O., and Abeloos, B., “Image-based wavefront correction using model-free reinforcement learning,” Optics Express 32(18), 31247–31269 (2024). [11] Gutierrez, Y., Mazoyer, J., Herscovici-Schiller, O., Mugnier, L. M., Abeloos, B., and Laginja, I., “A Deep Reinforcement Learning Approach to Wavefront Control for Exoplanet Imaging,” in [Space Telescopes and Instrumentation 2024: Optical, Infrared, and Millimeter Wave ], Coyle, L. E., Matsuura, S., and Perrin, M. D., eds., Proceedings of SPIE 13092, 130926H, SPIE (2024). [12] Potier, A., Baudoz, P., Galicher, R., Singh, G., and Boccaletti, A., “Comparing focal plane wavefront control techniques: Numerical simulations and laboratory experiments,” Astronomy & Astrophysics 635, A192 (2020). [13] Groff, T. D., Riggs, A. J. E., Kern, B., and Kasdin, N. J., “Methods and limitations of focal plane sensing, estimation, and control in high-contrast imaging,” Journal of Astronomical Telescopes, Instruments, and Systems 2, 011009 (Jan. 2016). [14] Potier, A., Mazoyer, J., Wahhaj, Z., Baudoz, P., Chauvin, G., Galicher, R., and Ruane, G., “Increasing the raw contrast of VLT/SPHERE with the dark-hole technique. I. On-sky wavefront correction and coherent differential imaging,” Astronomy & Astrophysics 665, A136 (2022). [15] Laginja, I., Baudoz, P., Mazoyer, J., Potier, A., Galicher, R., and Boussaha, F., “Extended linearity in the high-order wavefront sensor for the Roman Coronagraph,” Astronomy & Astrophysics 698, A130 (2025). [16] Zhou, H., Krist, J., Seo, B.-J., Kern, B., Cady, E., and Poberezhskiy, I., “Roman CGI testbed HOWFSC modeling and validation,” in [Space Telescopes and Instrumentation 2020: Optical, Infrared, and Millimeter Wave ], Proceedings of SPIE 11443, 114431W (2020). [17] Mazoyer, J., Laginja, I., Potier, A., and Galicher, R., “Asterix: Simulate your high-contrast instruments,” (2025). Software documentation. [18] Laginja, I., Baudoz, P., Potier, A., Mazoyer, J., Galicher, R., Por, E. H., Soummer, R., Sevin, A., Pougheon, E., Paviot, C., Doelman, D., Landman, R., Snik, F., Bettonvil, F., Rietjens, J., van Dijk, C., Peeters, K., Eigenraam, A., Krasteva, M., and Taccola, M., “First high-contrast results on THD2 testbed after infras- tructure upgrade,” Conference presentation 14145-70 at SPIE Astronomical Telescopes + Instrumentation, Space Telescopes and Instrumentation 2026: Optical, Infrared, and Millimeter Wave (2026). [19] Astropy Collaboration et al., “The astropy project: Sustaining and growing a community-oriented open- source project and the latest major release (v5.0) of the core package,” The Astrophysical Journal 935, 167 (08 2022). [20] Hunter, J. D., “Matplotlib: A 2d graphics environment,” Computing in Science & Engineering 9(3), 90–95 (2007). [21] Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk, M. H., et al., “Array programming with NumPy,” Nature 585(7825), 357–362 (2020). [22] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., K ̈opf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S., “PyTorch: An imperative style, high-performance deep learning library,” in [Advances in Neural Information Processing Systems ], 32, 8024–8035 (2019). [23] Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M., Wilson, J., Millman, K. J., et al., “Scipy 1.0: Fundamental algorithms for scientific computing in python,” Nature Methods 17(3), 261–272 (2020). [24] Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., and Dormann, N., “Stable-baselines3: Reliable reinforcement learning implementations,” Journal of Machine Learning Research 22(268), 1–8 (2021).