Paper deep dive
Two-Stage Active Distribution Network Voltage Control via LLM-RL Collaboration: A Hybrid Knowledge-Data-Driven Approach
Xu Yang, Chenhui Lin, Xiang Ma, Dong Liu, Ran Zheng, Haotian Liu, Wenchuan Wu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/20/2026, 12:19:15 PM
Summary
This paper proposes a hybrid knowledge-data-driven approach for two-stage voltage control in active distribution networks (ADNs) by collaborating a Large Language Model (LLM) agent and a Reinforcement Learning (RL) agent. The LLM agent handles day-ahead scheduling of mechanical assets (OLTC and Shunt Capacitors) using coarse forecasts and semantic grid codes, while the RL agent manages intra-day reactive power control of PV inverters using precise node-level measurements. The framework includes a self-evolution mechanism for the LLM and a pretrain-finetune pipeline for the RL agent to enhance coordination and performance.
Entities (8)
Relation Signals (8)
LLM Agent β controls β Shunt Capacitor
confidence 95% Β· generates scheduling strategies for ... shunt capacitors (SCs)
RL Agent β controls β PV Inverter
confidence 95% Β· deriving reactive power generation strategies for PV inverters
LLM Agent β controls β On-Load Tap Changer
confidence 95% Β· generates scheduling strategies for on-load tap changer (OLTC)
LLM Agent β handles β Day-Ahead Scheduling
confidence 95% Β· In the day-ahead stage, the LLM agent receives coarse region-level forecasts and generates scheduling strategies for on-load tap changer (OLTC) and shunt capacitors (SCs)
RL Agent β handles β Intra-Day Reactive Control
confidence 95% Β· in the intra-day stage, based on accurate node-level measurements, the RL agent refines terminal voltages by deriving reactive power generation strategies for PV inverters
Active Distribution Network β contains β Distributed Photovoltaics
confidence 90% Β· The growing integration of distributed photovoltaics (PVs) into active distribution networks (ADNs)
LLM Agent β uses β Reflexion
confidence 90% Β· we propose a Reflexion-based self-evolution mechanism for the LLM policy
RL Agent β uses β Pretrain-Finetune Pipeline
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The growing integration of distributed photovoltaics (PVs) into active distribution networks (ADNs) has exacerbated operational challenges, making it imperative to coordinate diverse equipment to mitigate voltage violations and enhance power quality. Although existing data-driven approaches have demonstrated effectiveness in the voltage control problem, they often require extensive trial-and-error exploration and struggle to incorporate heterogeneous information, such as day-ahead forecasts and semantic-based grid codes. Considering the operational scenarios and requirements in real-world ADNs, in this paper, we propose a hybrid knowledge-data-driven approach that leverages dynamic collaboration between a large language model (LLM) agent and a reinforcement learning (RL) agent to achieve two-stage voltage control. In the day-ahead stage, the LLM agent receives coarse region-level forecasts and generates scheduling strategies for on-load tap changer (OLTC) and shunt capacitors (SCs) to regulate the overall voltage profile. Then in the intra-day stage, based on accurate node-level measurements, the RL agent refines terminal voltages by deriving reactive power generation strategies for PV inverters. On top of the LLM-RL collaboration framework, we further propose a self-evolution mechanism for the LLM agent and a pretrain-finetune pipeline for the RL agent, effectively enhancing and coordinating the policies for both agents. The proposed approach not only aligns more closely with practical operational characteristics but also effectively utilizes the inherent knowledge and reasoning capabilities of the LLM agent, significantly improving training efficiency and voltage control performance. Comprehensive comparisons and ablation studies demonstrate the effectiveness of the proposed method.
Tags
Links
- Source: https://arxiv.org/abs/2602.21715v1
- Canonical: https://arxiv.org/abs/2602.21715v1
Trouble viewing inline? Open PDF directly β
Full Text
58,321 characters extracted from source content.
Expand or collapse full text
1 Two-Stage Active Distribution Network Voltage Control via LLM-RL Collaboration: A Hybrid Knowledge-Data- Driven Approach Xu Yang, Graduate Student Member, IEEE, Chenhui Lin, Senior Member, IEEE Xiang Ma, Dong Liu, Ran Zheng, Haotian Liu, Member, IEEE, and Wenchuan Wu, Fellow, IEEE AbstractβThe growing integration of distributed photovoltaics (PVs) into active distribution networks (ADNs) has exacerbated operational challenges, making it imperative to coordinate diverse equipment to mitigate voltage violations and enhance power quality. Although existing data-driven approaches have demonstrated effectiveness in the voltage control problem, they often require extensive trial-and-error exploration and struggle to incorporate heterogeneous information, such as day-ahead forecasts and semantic-based grid codes. Considering the operational scenarios and requirements in real-world ADNs, in this paper, we propose a hybrid knowledge-data-driven approach that leverages dynamic collaboration between a large language model (LLM) agent and a reinforcement learning (RL) agent to achieve two-stage voltage control. In the day-ahead stage, the LLM agent receives coarse region-level forecasts and generates scheduling strategies for on-load tap changer (OLTC) and shunt capacitors (SCs) to regulate the overall voltage profile. Then in the intra-day stage, based on accurate node-level measurements, the RL agent refines terminal voltages by deriving reactive power generation strategies for PV inverters. On top of the LLM-RL collaboration framework, we further propose a self-evolution mechanism for the LLM agent and a pretrain-finetune pipeline for the RL agent, effectively enhancing and coordinating the policies for both agents. The proposed approach not only aligns more closely with practical operational characteristics but also effectively utilizes the inherent knowledge and reasoning capabilities of the LLM agent, significantly improving training efficiency and voltage control performance. Comprehensive comparisons and ablation studies demonstrate the effectiveness of the proposed method. Index TermsβActive distribution network, two-stage voltage control, large language model, reinforcement learning, knowledge-data-driven. I. INTRODUCTION SPONDING to climate emergencies, pollution from fossil fuels, and carbon neutrality commitments, renewable energy resources, particularly distributed photovoltaics (PVs), are being increasingly integrated into distribution networks, transforming them into active distribution networks (ADNs) [1], [2]. While this integration substantially reduces carbon emissions at the distribution level, the growing penetration of This work was supported in part by the State Grid Zhejiang Electric Power Company Limited Science and Technology Project βResearch on Key Technologies for Cooperative Regulation of Flexible Resources in Active Distribution Networks Based on Highly Safe and Scalable Artificial Intelligenceβ under Grant 5211JH250006 (Corresponding author: Wenchuan Wu). PV has introduced significant operational challenges, notably voltage violations at the network extremities. Also, the inherent intermittency and stochastic nature of PV generation further exacerbate voltage fluctuations, power imbalances, and overall power quality degradation, posing serious threats to ADN stability and security [3], [4]. In this context, traditional passive voltage control methods are no longer sufficient to meet the demands of ADN operation. The ADN operators must adopt proactive voltage control approaches that jointly utilize mechanical assets such as on- load tap changer (OLTC) and shunt capacitors (SCs), as well as flexible resources like PV inverters capable of dynamic reactive support. Due to temporal coupling constraints and the requirements for manual on-site intervention, OLTC and SCs usually necessitate day-ahead scheduling based on the forecasted information or historical records. In contrast, during intra-day operation, the fast reactive power regulation capability of PV inverters can be leveraged to enable rapid control based on real-time measurements. A variety of data-driven approaches, primarily based on deep reinforcement learning (RL), have been developed to enable efficient resolution of the aforementioned voltage control problem [5]-[14]. These methods fundamentally operate by allowing an intelligent RL agent to iteratively interact with the external environment and enhance its policy based on the received feedback. For example, researchers in [5]-[7] proposed RL-based voltage control methods for ADNs, optimizing voltage profiles by utilizing internal distributed energy resources. Researchers in [8]-[11] extended this paradigm into multi-agent settings, where different system regions or resource clusters are assigned to individual RL agents that collaborate to achieve overall voltage optimization. Additionally, researchers in [12]-[14] addressed the coordination of different equipment operating on two timescales, enabling hierarchical voltage regulation across various temporal horizons. Although the above RL-based data-driven methods have demonstrated effective and promising performance, their practical application to real-world ADN voltage control, especially in the day-ahead stage, still faces four key Xu Yang, Chenhui Lin, Haotian Liu, and Wenchuan Wu are with the Sichuan Energy Internet Research Institute, Tsinghua University, Chengdu 610213 , China. Xiang Ma, Dong Liu, and Ran Zheng are with the State Grid Zhejiang Electric Power Co., Ltd., Jinhua Power Supply Company, Jinhua 321017, China. R 2 challenges: 1) Incomplete and heterogeneous information. The effectiveness of data-driven methods typically hinges on the availability of complete and high-quality data, which corresponds to accurate node-level measurements in the context of voltage control problem. However, limited by the resolution of current meteorological forecasts and the accuracy of PV/load predictions, only coarse-grained information, i.e., hourly region-level forecasts are available in the day-ahead stage. Moreover, some unstructured and heterogeneous information such as historical records and voltage reports, may also be provided in the day-ahead stage. Processing and integrating such incomplete and heterogeneous data pose significant challenges for existing RL algorithms. 2) Semantics-based operational constraints. The scheduling of ADN mechanical assets usually involves numerous semantic-based grid codes, such as: βThe number of daily adjustments of an OLTC is subject to an upper limit,β or βSCs can only be committed during a specific time window each day due to the requirement for manual switching.β These grid codes are essentially operational constraints on the agentβs action space. However, incorporating such constraints into an RL policy typically requires elaborate techniques, such as penalty mechanism [14] or action masking [15], which reduces the training efficiency of the RL agent. 3) Instructions adapted to long-tail requirements. Since mechanical assets may encounter unforeseen events such as faults or maintenance, ADN operators may occasionally issue operational instructions described in natural language. Therefore, the agentβs policy must be capable of adapting to such instructions. However, RL agents are inherently unable to process emergent natural-language instructions and exhibit poor generalization to unseen scenarios, which can result in performance degradation or even infeasible control actions. 4) Ambiguous policy improvement direction. Finally, the RL agent relies solely on a scalar reward signal to guide its training process. However, such a scalar reward carries very limited information and fails to provide a clear direction for policy evolution. For example, when the overall voltage is low, the optimal action, such as raising the OLTC tap position, is intuitively straightforward. Yet, the RL agent must undergo extensive exploration to discover this simple rule, resulting in lower training efficiency or worse performance. The rapid advancement of large language models (LLMs) in recent years has initiated widespread applications across various domains [16]-[18] and offered innovative solutions to the above challenges. On the one hand, LLMs possess inherent natural-language processing and strong information integration abilities, enabling them to receive, analyze, and process incomplete and heterogeneous information. On the other hand, LLMs embed a vast amount of prior knowledge and exhibit powerful reasoning capabilities, allowing them to comprehend semantic-based operational constraints and instructions and generate valid responses accordingly. Building on these strengths, some pioneering studies have begun applying LLMs to the power system dispatch domain [19]-[21], while others have investigated how LLMs can assist the training process of RL agents [22]-[24]. In these studies, LLMs are generally limited to auxiliary subtasks such as code programming or function synthesis and are primarily treated as a natural- language interface, rather than being used for policy formulation. Furthermore, research on how to improve LLM policies remains comparatively scarce. Fig. 1. Overall scheme of the proposed approach. Considering practical requirements and operational scenarios in ADN voltage control, we propose a hybrid knowledge-data- driven approach that combines respective advantages of LLM and RL. As shown in Fig. 1, in the day-ahead stage, the ADN operator receives hourly region-level forecasts and utilizes an LLM agent to generate scheduling strategies for the OLTC and SCs. Then in the intra-day stage, the ADN operator receives accurate node-level measurements and utilizes an RL agent to generate real-time reactive power generation strategies for PV inverters. On the one hand, this setup aligns well with the two- stage nature of voltage control problem: in the day-ahead stage, coarse information is used to dispatch mechanical assets for regulating the overall voltage profile, while in the intra-day stage, fine-grained information enables PV inverters to refine terminal voltages. On the other hand, the proposed approach synergistically leverages the complementary advantages of LLM and RL, which not only exploit the LLMβs information processing capabilities and embedded knowledge to enhance policy learning efficiency but also RLβs precise computation and rapid response to optimize control performance. In essence, different from the existing LLM-assisted RL paradigm, the proposed approach addresses a complex multi- stage constrained optimization problem through LLM-RL collaboration using different aspects of information. From the LLMβs perspective, the RL agent serves as a callable specialized tool capable of performing inverter reactive control tasks, thereby extending the LLMβs capability boundaries. Conversely, from the RLβs perspective, the LLM agent constitutes an integral part of the external environment that processes heterogeneous or semantic information, effectively reducing problem complexity and shrinking the action space that needs to be explored, thus accelerating RL policy 3 convergence. Building upon the proposed LLM-RL collaboration framework, the key challenge lies in how to improve and coordinate the policies of the LLM agent and the RL agent. To this end, we propose a Reflexion [25]-based self-evolution mechanism for the LLM policy, which continuously enhances its decision-making through environmental feedback and knowledge base updates. Meanwhile, we also propose a pretrain-finetune pipeline for the RL agent to quickly conform to the LLM policy and thereby strengthen the coordination between two agents. The main contributions of this paper can be summarized as follows: 1) Comprehensive problem formulation. We present a more comprehensive formulation of the two-stage voltage control problem in ADNs. Compared to existing studies, we explicitly account for the differences in available information between the day-ahead and intra-day stages: in the day-ahead stage, only hourly region-level forecasts are accessible, whereas accurate node-level measurements become available only during the intra-day stage. This setup better reflects the operational conditions of real-world ADNs. 2) LLM-RL collaboration framework. To optimize the formulated problem, we propose an LLM-RL collaboration framework that assigns day-ahead OLTC and SCs scheduling to the LLM agent and intra-day PV inverter reactive control to the RL agent. The proposed framework is essentially a hybrid knowledge-data-driven approach that effectively combines the strengths of LLM and RL, not only exploiting the LLMβs information processing capabilities and embedded domain knowledge, but also leveraging RLβs precise computation and rapid response. 3) Policy improvement mechanism for LLM and RL. To enable policy improvement and coordination within the LLM- RL collaboration framework, we also propose dedicated enhancement methods, including a Reflexion-based self- evolution mechanism for the LLM policy and a pretrain- finetune pipeline for the RL policy, enabling efficient adaptation and joint convergence toward optimal performance. Comprehensive comparisons and ablation studies validate the effectiveness and superiority of the proposed method. I. PRELIMINARIES A. Two-Stage Voltage Control Problem Formulation In this paper, we consider an ADN whose set of nodes is denoted by ο. Based on the spatial boundaries defined by weather forecasts or PV/load predictions, this ADN can be partitioned into ν regions. The objective of the two-stage voltage control problem is to minimize voltage deviations across the system and eliminate violations as much as possible: min β β οΏ½ν ν,ν‘ βν ν οΏ½ νβ1 ν‘=0 νβο (1) where ν ν,ν‘ is the voltage magnitude at node ν, time ν‘; ν ν is the voltage reference value; and ν is the length of the voltage control process, which is set to one day in this study. In order to achieve this objective, the ADN operator can dispatch the following controllable equipment: an OLTC installed at the root node; a set of SCs distributed across the network, whose set of nodes is denoted as ο ν ; and several PV inverters installed at terminal nodes of the network, whose set of nodes is denoted as ο ν . We also assume that SCs are not deployed at PV user nodes, i.e., ο ν β©ο ν =β . The OLTC serves to raise or lower the overall system voltage through adjustment of its tap position: ν 0,ν‘ =ν ν +ν½ ν‘ ννν ν ν‘ (2) where ν 0,ν‘ is the root node voltage at time ν‘; ν ν‘ is the voltage difference between two adjacent tap positions; and ν½ ν‘ ννν is an integer indicating the tap position of the OLTC at time ν‘. For example, for an OLTC with 11 tap positions, ν½ ν‘ ννν β β5,β4,...,β1,0,1,...,4,5 . The SCs serve to provide reactive power compensation through commitment during periods of peak load, thereby raising the voltage in corresponding local regions: ν ν,ν‘ =ν½ ν,ν‘ ν ν ν βν ν,ν‘ ννΏ ,βν βο ν (3) where ν ν,ν‘ is the reactive injection at node ν, time ν‘; ν ν is the SC capacity; ν ν,ν‘ ν νΏ is the reactive load at node ν, time ν‘; and ν½ ν,ν‘ ν β 0,1 is the commitment status of the SC at node ν, time ν‘. The PV inverters leverage their spare capacity to generate reactive power, thereby refining terminal voltages: ν ν,ν‘ =ν ν,ν‘ ν βν ν,ν‘ ν νΏ ,βν βο ν (4) οΏ½ν ν,ν‘ ν οΏ½β€ν ν οΏ½ ( ν ν ) 2 βοΏ½ν ν,ν‘ ν οΏ½ 2 ,βν βο ν (5) where ν ν,ν‘ ν is the reactive generation of the PV inverter at node ν, time ν‘; ν ν is the PV installed capacity; ν ν,ν‘ ν is the active generation of the PV inverter at node ν, time ν‘; and ν ν β [ 0,1 ] is the reactive capacity factor of the inverter, which is jointly determined by the inverter configurations and ancillary services provided by the PV user. In addition to the above equipment constraints, the ADN voltage control problem must also satisfy power flow constraints (6)-(9) and voltage limits (10): ν ν,ν‘ =οΏ½ ν ν,ν‘ ν βν ν,ν‘ ν νΏ ,βν βο ν βν ν,ν‘ ννΏ ,βν βο\ο ν (6) ν ν,ν‘ =βν ν,ν‘ ννΏ ,βν βο\οΏ½ο ν βͺο ν οΏ½ (7) ν ν,ν‘ =ν ν,ν‘ β ν ν,ν‘ οΏ½νΊ νν cosν νν,ν‘ +ν΅ νν sinν νν,ν‘ οΏ½ νβο ,βν βο (8) ν ν,ν‘ =ν ν,ν‘ β ν ν,ν‘ οΏ½νΊ νν sinν νν,ν‘ βν΅ νν cosν νν,ν‘ οΏ½ νβο ,βν βο (9) ν ννν β€ν ν,ν‘ β€ν νν‘ ν ,βν βο (10) where ν ν,ν‘ is the active injection at node ν, time ν‘; ν ν,ν‘ ννΏ is the active load at node ν, time ν‘; νΊ νν and ν΅ νν are the real and imaginary parts of the admittance element between nodes ν and ν, respectively; ν νν,ν‘ is the voltage phase difference between nodes ν and ν; and ν ννν and ν νν‘ ν are the lower limit and upper limit of the voltage magnitude, respectively. Eqs. (1)-(10) represent the mathematical modeling of the two-stage voltage control problem adopted by most existing studies. However, in real-world ADN operations, particularly in scheduling mechanical assets, a series of grid codes must be followed, as the control of these devices often involves 4 temporal coupling constraints and requirements for manual on- site intervention. In this paper, we consider two common grid code requirements for OLTC and SCs, i.e., βThe number of daily OLTC adjustments is capped by an upper limit,β and βSCs can only be committed during a specific time window each day.β As can be seen, these grid code requirements can be easily expressed in natural language, yet effectively embedding them into a data-driven RL policy requires complicated design. This is one of the main motivations for introducing LLMs, as LLMs can readily comprehend such operational constraints and generate compliant scheduling strategies. The solution to the above constrained optimization problem is usually carried out in two stages, i.e., the day-ahead stage and the intra-day stage. In the day-ahead stage, the ADN operator obtains hourly region-level forecasts and determines scheduling strategies for OLTC and SCs to proactively regulate the overall voltage profile for the following day, preventing severe voltage violations. Then in the intra-day stage, real-time accurate node- level measurements become accessible, and the ADN operator leverages the fast response capability of PV inverters to further refine terminal voltages. B. LLM as Decision-Making Models In recent years, the rapid development of LLMs has become one of the most transformative and significant developments in artificial intelligence, bringing profound impacts to numerous specialized domains. Built upon the transformer architecture [26] and trained on vast amounts of pretraining data, existing LLMs have acquired powerful information processing capabilities and embedded domain knowledge, making it possible to employ them as decision-making models. Some advanced research in artificial intelligence has begun to apply LLMs as decision-making models in autonomous driving [27], [28] and robotic control [29], [30]. These models take text- based or multimodal environmental descriptions as input and generate corresponding control strategies in an end-to-end manner, offering a novel pathway for solving complex decision-making problems. In power systems, dispatch problems, such as ADN voltage control, are quintessential decision-making problems. However, existing applications of LLMs in the power domain have largely focused on tasks like simulation setup [31], [32], document analysis [33], and scenario generation [34]. The few studies related to dispatching typically assign LLMs subtasks such as code programming and function synthesis, which remain fundamentally natural-language processing problems and do not fully exploit the capabilities of existing LLMs. Therefore, in this paper, we innovatively introduce the LLMs as decision-making models and apply them to the ADN voltage control problem. Through the proposed input and output formulation, LLM-RL collaboration framework, and policy improvement mechanisms, the LLM agent successfully addresses the challenge of coordinating mechanical assets under grid code constraints and enhances voltage control performance in ADNs. C. MDP and RL The intra-day strategies are provided by the RL agent. In RL, the sequential decision-making process is usually modeled as a Markov decision process (MDP) [35], which can be described by a tuple ( ο,ο,ν,ν,νΎ ) , where ο is the state space of the environment; ο is the action space of the RL agent; ν is the state transition function; ν is the reward function; and νΎ β [ 0,1 ) is the discount factor for future rewards. At time ν‘, the RL agent receives the state observation ν ν‘ βο of the current environment and decides its control action ν ν‘ β ο based on its policy ν:οΓοβ [ 0,β ) , i.e., ν ν‘ ~ν ( β |ν ν‘ ) . Then the environment transfers to the next state ν ν‘+1 βο based on ν:οΓοΓοβ [ 0,β ) and gives the RL agent a reward ν ν‘ ( ν ν‘ ,ν ν‘ ) as feedback. Using the feedback information from the environment, the objective of the RL agent is to optimize its policy ν so that the expected cumulative rewards ν½ ( ν ) can be maximized, i.e., max ν ν½ ( ν ) =νΌ ν‘,ν [β νΎ ν‘ ν ν‘ νβ1 ν‘=0 ] (11) Detailed MDP settings of the voltage control problem and RL training mechanism will be described in the next section. I. METHODS A. LLM-RL Collaboration Framework Fig. 2. Illustrative prompts for LLM strategies generation. As shown in Fig. 1, the proposed LLM-RL collaboration framework employs an LLM agent and an RL agent to handle day-ahead and intra-day decision-making, respectively, achieving voltage optimization through their collaborative interaction. In the day-ahead stage, the input to the LLM agent consists of hourly region-level forecasts, specifically the net load forecasts for the ν regions and the overall ADN system. These inputs are characterized by coarse granularity and may 5 exhibit certain deviations due to the limited accuracy of current PV/load predictions. Based on this information, the LLM agent outputs scheduling strategies for the OLTC and SCs that comply with grid code requirements. Then in the intra-day stage, the input to the RL agent consists of accurate node-level measurements, specifically the real-time loads and PV generation. These inputs are characterized by high resolution and relatively high accuracy. Based on this information, the RL agent outputs reactive control strategies for the PV inverters that comply with reactive capacity constraints. The key innovation of this collaboration framework lies in the introduction of the LLM agent, which enables the exploitation of its advantages in information processing and knowledge integration. To adapt an existing general-purpose LLM into the required domain-specific LLM agent, that is, one capable of accurately understanding the voltage control problem and generating day- ahead OLTC and SCs scheduling strategies that satisfy operational constraints, we propose the tailored prompts. As illustrated in Fig. 2, the prompts consist of the following components: 1) Environment and task description: This component provides a detailed description of the environment and task, including the system overview, descriptions of the OLTC and SCs, and a clear specification of the day-ahead scheduling task, which requires the LLM agent to generate day-ahead strategies based on the hourly forecasts for the following day. 2) Output format: A standardized output format can effectively reduce the occurrence of hallucinations or internal errors. In this component, we specify the required output format for the LLM agent, namely, the timing and corresponding tap positions for OLTC adjustments, as well as the commitment statuses and time windows for SCs. Once the LLM agent generates the content, we apply regular expressions to extract structured scheduling strategies from the response and validate their compliance with grid code requirements. If the strategies satisfy these operational constraints, the policy is deemed valid; otherwise, the LLM agent is prompted to regenerate the day- ahead scheduling strategies in another dialogue. 3) Chain-of-Thought (CoT) guidance: CoT reasoning refers to decomposing a complex problem into a sequence of simple, executable inference steps [36]. For domain-specific problems, CoT can effectively assist in generating reliable outputs. Specifically, in the context of voltage control, the LLM agent is first instructed to analyze the trend and magnitude of net load forecasts; based on which it determines the commitment actions for SCs in each region and the tap positions for the OLTC; and finally, it organizes these decisions into a formal final answer which returns to the ADN operator. 4) Few-shot examples as knowledge embedding: The above approach only ensures that the generated strategies are feasible, but it does not enable improvement of the LLM policy. To realize policy improvement, we further construct a knowledge base ο ννΏ that stores information from several historical days. For a specific historical day, we store its day-ahead forecasting data, generated OLTC and SCs actions, and the resulting total reward and voltage profiles. When making decisions, the LLM agent retrieves the historical day most similar to the current scenario from the knowledge base and embeds it as a few-shot example in the prompts for reference during reasoning. This prevents the LLM agentβs decision-making from being arbitrary and enables continuous policy optimization through iterative updates of the knowledge base. Complete prompts for the LLM agent are provided in an online supplementary file [37]. And the details of LLM policy improvement, historical knowledge retrieval, and knowledge base update will be elaborated in the following subsections. On the other hand, the role of the RL agent is to adapt the OLTC and SCs settings prescribed by the LLM agent in the day-ahead stage and further optimize the terminal voltages during the intra-day stage by adjusting the reactive power generation of PV inverters. Although the RL policy can be readily improved using off-the-shelf RL algorithms, rapidly adapting it to the LLM policy within the proposed LLM-RL collaboration framework poses a significant challenge. Because when the statuses of the OLTC and SCs change, the RL agent actually faces a different environment. Therefore, the training mechanism of the RL agent should be carefully designed to enhance its learning capability and adaptation rate. As shown in Fig. 3, to address this issue, we further propose a pretrain-finetune pipeline for the RL agent. Specifically, the RL agent first undergoes extensive exploration in a pretraining environment to learn a general control policy for PV inverters. Subsequently, the pretrained RL agent is transferred to a finetuning environment, where it adapts to the current LLM policy. Notably, in the pretraining environment, the actions of OLTC and SCs are randomly assigned, whereas in the finetuning environment, these actions are provided by the well- improved LLM agent. This design enables the RL agent to acquire versatile control strategies across diverse settings during pretraining, thereby facilitating rapid adaptation to the LLM policy during finetuning. Fig. 3. Pretrain-finetune pipeline for RL agent. B. Intra-Day PV Reactive Control Policy Improvement Within the pretrain-finetune pipeline, in order to leverage RL algorithms to improve the intra-day PV reactive control policy, the intra-day voltage control problem should first be formulated as an MDP. Corresponding definitions are designed as follows: 1) State space: The state ν ν‘ βο of the MDP consists of two 6 components, i.e., the real-time node-level measurements ( ν· ν‘ ννΏ ,νΈ ν‘ ννΏ ,ν½ ν‘ ,ν· ν‘ ν ,νΈ ν‘ ν ) from the ADN monitoring system and OLTC and SCs actions ( ν½ ν‘ ννν ,ν· ν‘ ν ) previously decided by the LLM agent, where ν· ν‘ ννΏ ,νΈ ν‘ ννΏ ,ν½ ν‘ are the vectors of the ADNβs active loads, reactive loads, and voltage magnitudes; ν· ν‘ ν ,νΈ ν‘ ν are the vectors of the PVsβ active and reactive generation; and ν· ν‘ ν is the vector of the SCsβ commitment statuses. Specifically, OLTC and SCs actions ( ν½ ν‘ ννν ,ν· ν‘ ν ) are embedded into the state space using one-hot encoding. Incorporating the LLM agentβs actions into the state space enhances the RL agentβs environmental awareness and improves its adaptation to the LLM policy. 2) Action space: The action ν ν‘ βο of the MDP is constructed with the reactive generation νΈ ν‘ ν of all controllable PV inverters in the ADN. Specifically, we constrain the action of the RL agent in [ βν ν ,ν ν ] so that the reactive capacity constraint eq. (5) is always satisfied. 3) Reward function: The reward function of the MDP aligns with the objective of the voltage control problem: ν ν‘ =β β οΏ½ν ν,ν‘ βν ν οΏ½ νβο | ο | (12) The RL algorithm utilized in this paper is proximal policy optimization (PPO) proposed in [38]. In PPO, the RL agentβs policy ν is approximated by a neural network ν ν with parameters ν. After completing one episode of iteration between the old policy ν ν β² and the external environment, the objective function for ν ν can be expressed as: ν½ ν β² ( ν ) =νΌ ( ν ν‘ ,ν‘ ν‘ ) ~ν ν β² οΏ½ν ν‘ ( ν ) ν΄ ν β² ,ν‘ οΏ½ (13) where ( ν ν‘ ,ν ν‘ ) ~ν ν β² is the batch of iteration data generated from the old policy ν ν β² ; ν ν‘ ( ν ) =ν ν ( ν ν‘ |ν ν‘ ) /ν ν β² ( ν ν‘ |ν ν‘ ) is the importance sampling ratio; and ν΄ ν β² ,ν‘ is an estimator of the advantage function using generalized advantage estimator (GAE) technique. In order to constrain the updated policy within a trust region and ensure more stable training, a common practice in PPO is to use gradient clipping, i.e., ν½ ν β² νννΆν ( ν ) =νΌ ( ν ν‘ ,ν‘ ν‘ ) ~ν ν β² οΏ½minοΏ½ν ν‘ ( ν ) ν΄ ν β² ,ν‘ ,ν νν ( ν ν‘ ( ν ) , 1βν ν ν , 1 +ν ν ν ) ν΄ ν β² ,ν‘ οΏ½ (14) where the clip function confines ν ν‘ ( ν ) to the interval [ 1β ν ν ν , 1 +ν ν ν ] so that the updated policy does not deviate excessively from the old policy. Then based on the calculated ν½ ν β² νννΆν ( ν ) , the policy network ν ν is optimized ν ν ν times by updating its parameters. C. Day-Ahead OLTC and SCs Scheduling Policy Improvement After ensuring that the LLM agent can comprehend the problem and generate day-ahead strategies that comply with operational constraints, the next consideration is how to improve the LLM policy accordingly. In this paper, we propose a Reflexion-based self-evolution mechanism for the LLM agent. Similar to RL algorithms, Reflexion enhances the agentβs policy through interaction with an environment and feedback from it. Specifically, Reflexion aims to improve the modelβs reasoning and decision-making capabilities on complex tasks via a self-reflection process that encompasses execution, evaluation, reflection, memory, and iterative refinement. For the voltage control problem discussed in this paper, a knowledge base ο ννΏ is firstly maintained for the LLM agent, storing data from several historical days. When a decision is required for a new day ν ν , the LLM agent queries the knowledge base and retrieves the historical scenario most similar to current situation (with the highest similarity νΜ), which is then embedded as a few-shot example into the prompts provided to the LLM agent. Specifically, for the new day ν ν and a historical day ν ν , their similarity ν νν is calculated as follows: ν νν =νν νν Γνν νν (15) where νν νν captures the temporal similarity between ν ν and ν ν ; and νν νν captures the magnitude similarity between ν ν and ν ν . νν νν and νν νν are designed as follows: νν νν = β ννΏ οΏ½ ν,ν‘ ΓννΏ οΏ½ ν,ν‘ 23 ν‘=0 οΏ½ β οΏ½ννΏ οΏ½ ν,ν‘ οΏ½ 2 23 ν‘=0 Γ οΏ½ β οΏ½ννΏ οΏ½ ν,ν‘ οΏ½ 2 23 ν‘=0 (16) νν νν = min οΏ½ β ννΏ οΏ½ ν,ν‘ 23 ν‘=0 οΏ½,οΏ½ β ννΏ οΏ½ ν,ν‘ 23 ν‘=0 οΏ½ max οΏ½ β ννΏ οΏ½ ν,ν‘ 23 ν‘=0 οΏ½,οΏ½ β ννΏ οΏ½ ν,ν‘ 23 ν‘=0 οΏ½ (17) where νΏν΄νΏ οΏ½ ν,ν‘ ( ν‘ β [ 0,23 ]) is the hourly net load forecast for the overall ADN system in ν ν . After the OLTC and SCs actions are executed in the training environment, the LLM agent then receives the corresponding results and feedback, including the total reward of the day and hourly voltage profiles, i.e., average voltage for each region and the overall ADN system. Based on the feedback and following the Reflexion paradigm, we instruct the LLM agent to refine its original actions. For example, if the overall voltage is too low during a specific time period, the LLM agent will increase the OLTC tap position for the corresponding period. After performing ν ννΏ rounds of Reflexion, we obtain ν ννΏ +1 candidate rewards for the new day ν ν , and we denote the highest one as ν ν . Fig. 4. Knowledge base updating process for LLM agent. After Reflexion concludes, the next step is to update the knowledge base ο ννΏ , where we set an updating threshold 7 ν ννΏ . Then the update process should be handled in two cases: If νΜ<ν ννΏ , it indicates that the new day ν ν is a novel situation which have not been encountered before, then the new day ν ν , the reward ν ν , and its corresponding OLTC and SCs actions are directly appended into ο ννΏ . If νΜβ₯ν ννΏ , it indicates that ν ν has been encountered before and can be partially represented by a historical day ν ν . We then compare the total rewards ν ν and ν ν to examine their performance. If ν ν >ν ν , it means the LLM agent has discovered a better set of actions for this situation, and we replace ν ν in ο ννΏ with ν ν . Conversely, if ν ν β€ν ν , it implies that the previously stored day ν ν corresponds to a superior strategy, thus ν ν is retained in ο ννΏ and ν ν is discarded. This updating process is also illustrated in Fig. 4. Drawing an intuitive analogy between the proposed Reflexion-based self-evolution method and RL algorithms, the knowledge base ο ννΏ and the retrieved historical day ν ν serves as the policy parameters ν, instructing the agent to generate corresponding actions; the Reflexion process is analogous to the optimization process of ν½ ν β² νννΆν ( ν ) , where the agent identifies directions for policy improvements; ν ννΏ is like ν ν ν , which constrains differences between the old policy and the updated one; and ν ννΏ is like ν ν ν , which denotes the number of times an episode of data can be reused. Through the iterative Reflexion process and updates of the knowledge base, the LLM policy can be progressively refined. In addition, it should be noted that Reflexion is essentially a βtrainingβ process based on iterative experiences, and thus it occurs only in an offline manner. During online execution, the knowledge base is frozen and the LLM directly retrieves the most similar historical scenario from the knowledge base for inference, as it is infeasible and impractical for online Reflexion. IV. NUMERICAL STUDIES A. Cases and Methods Setup In this section, numerical simulations are conducted on IEEE 33-bus [39] and 141-bus [40] distribution systems. OLTC at root node has a total of 11 tap positions, each corresponding to a voltage regulation step of 0.6%. Voltage limitations are set at [0.95, 1.05] p.u. The 33-bus system is partitioned into three regions, and the 141-bus system is partitioned into five regions, in which each region is equipped with one SC and two PV inverters. We use one year of PV and load data to simulate power flow in the distribution systems. In the day-ahead stage, the LLM agent receives hourly region-level forecasts, to which we add 5% white noise to emulate the prediction errors in current PV and load predictions. For the RL agent, intra-day control interval is 15 minutes and the length of the control process is 96 steps. Corresponding configurations are listed in Table I, and more detailed specifications and topologies are provided in the online supplementary file [37]. As for the LLM and RL used in this paper, considering the randomness in LLM responses and the PPO algorithm, we use 3 independent random seeds for each experiment. All experiments are run on a computer with a 2.2GHz Intel Core i9-14900HX CPU and a 32GB RAM. Hyperparameters for the LLM agent and the RL agent are presented in Table I. TABLE I C ONFIGURATIONS OF THE TEST SYSTEMS Parameter Value No. Region 3, 5 No. SC 3, 5 No. PV 6, 10 No. OLTC Position 11 OLTC Adjustment Limits 4 OLTC Step 0.6% SC Capacity 0.15MVAR, 0.4MVAR ν ν 0.3, 0.15 For comparison and ablation studies, in order to demonstrate the effectiveness of the proposed approach, in the offline training phase, we set the proposed method against three baselines: 1) Pure- RL, in which the day-ahead and intra-day policies are optimized by two separate RL agents, is designed to validate the effectiveness of the LLM-RL collaboration framework. To ensure compliance with grid code requirements, we further incorporate a soft constraint and a hard constraint into the day-ahead scheduling strategies of Pure-RL, following established practices in the literature. The soft constraint refers to the penalty mechanism: when the RL agent proposes an action that violates grid code limits, it receives a corresponding negative penalty; while the hard constraint refers to action masking, which enforces that the states of the OLTC and SCs remain unchanged once their maximum allowable switching counts are reached. It should be noted that although Pure-RL receives a penalty term as feedback during training, this penalty term is excluded from the reward metrics presented in the following results to ensure a fair comparison. 2) No-PT, in which the RL agent is not pretrained and is directly finetuned in the environment defined by the LLM policy, is designed to validate the necessity of the proposed pretrain-finetune pipeline. 3) No-Reflexion: in which the Reflexion process is omitted, is designed to demonstrate the effectiveness of the proposed Reflexion-based mechanism in complex tasks. TABLE I H YPERPARAMETERS OF THE LLM AGENT AND THE RL AGENT Agent Parameter Value LLM Model Qwen-Plus Version β2025-12-01β Temperature (Training) 0.7 Top-P (Training) 0.8 Temperature (Test) 0.2 Top-P (Test) 0.6 ν ν νΏ 0.7 ν ν νΏ 3 RL Mini Batch Size 32 Hidden Layers 2 Hidden Units 256 νΎ 0.9 Learning Rate (Pretrain) 1e-4 β 1e-5 (Linear Decay) Learning Rate (Finetune) 1e-5 ν νν 0.3 ν νν 4 Then, in the online execution phase, in addition to the aforementioned methods, we also design another three baselines: 1) Original, in which the OLTC is fixed at the reference voltage position, SC commitment is disabled, and PV inverters are 8 prohibited from participating reactive control, and this baseline is used to demonstrate the necessity of proactive voltage control measures in ADNs. 2) No-LLM, in which the mechanical assets OLTC and SCs are disabled, is designed to validate their capability of regulating the overall voltage profile. 3) No-RL, in which the PV reactive generation is disabled, is designed to validate its capability of refining terminal voltages. B. Training Performance of Proposed and Baseline Methods To compare the performance of Proposed, Pure-RL, No-PT, and No-Reflexion during the offline training phase, we plot their training curves in Figs. 5-6. Solid lines in the figures represent the mean values across multiple random seeds, while the shaded areas indicate the corresponding error bounds. Also, considering that voltage control performance may vary significantly over the course of a year, we apply a moving average with a window length of 25 episodes to smooth these curves. Fig. 5. Training performance in IEEE 33-bus system. Fig. 6. Training performance in IEEE 141-bus system. As can be seen, the training phase begins with the LLM improvement; once the LLM policy stabilizes, we freeze its knowledge base and introduce a pretrained RL agent into the environment for RL finetuning. In the 33-bus system, the LLM agent undergoes improvement over 200 episodes, while the RL agent is pretrained for 1500 episodes and finetuned for 150 episodes. In the 141-bus system, the LLM agentβs improvement spans 250 episodes, and the RL agent is pretrained for 2500 episodes and finetuned for 200 episodes. In addition, for better visualization and comparison, we present the complete training curves of Proposed and Pure-RL in the figures, but only show the first half of the training curve for No-Reflexion and the second half for No-PT. However, it should be noted that all these four methods undergo the complete LLM improvement and RL finetuning procedures, and their converged performance will be compared in the next subsection. First, examining the training curve of Proposed reveals that, during the first half, the LLM policy progressively improves and converges steadily through interaction with the environment and continuous updates of the knowledge base, demonstrating the effectiveness of the proposed Reflexion- based self-evolution mechanism. In the second half of the curve, after the RL agent is introduced, the reward further improves using the PV reactive generation capacity. Moreover, since the RL agent has been well-pretrained beforehand, it can quickly adapt to the day-ahead strategies provided by the LLM agent, leading to stable convergence and enabling efficient coordination between day-ahead and intra-day operations. Second, we compare the performance of Pure-RL. It can be observed that, even without iteration and improvement, the initial LLM policy already outperforms the RL policy on the day-ahead voltage scheduling task. This is because the LLM possesses extensive embedded knowledge and, guided by the proposed prompts, can generate compliant and reasonable strategies. For example, it lowers the OLTC tap position when PV generation is high and dispatches SCs during peak load periods. Such intuitive operational knowledge is largely absent in the RL agent, resulting in an initial policy that is relatively random and ineffective. And even after interacting with the environment, Pure-RL shows no obvious performance improvement. On the one hand, the RL agent must handle operational constraints and a large action space, requiring substantially more exploration than the LLM agent to develop an effective understanding of the environment. On the other hand, the feedback received by the RL agent is a simple scalar reward, whereas the LLM agent can process heterogeneous information, such as voltage profiles and system states, which provides much richer contextual cues. Compared to a scalar reward, this diverse information offers the LLM agent clear directions for policy improvement, while the RL policy remains largely stochastic. In summary, on the day-ahead voltage scheduling task, the LLM demonstrates at least three advantages over RL, i.e., embedded knowledge driven strategy generation, semantic comprehension enhanced constraint handling, and reasoning backed policy improvement. These advantages underscore both the superiority and necessity of applying LLMs to specialized dispatch tasks, as well as the effectiveness of the proposed LLM-RL collaboration framework. Third, we observe the training curve of No-PT. Since the only difference between No-PT and Proposed is whether the introduced RL agent has undergone pretraining, and both 9 methods share the same LLM agent, their curves are identical in the first half and diverge only in the second half. It can be found that, due to the absence of pretraining, the RL agent in No-PT learns and converges more slowly than that in Proposed in the 33-bus system. And in the 141-bus system, it even converges to a suboptimal policy. This demonstrates the effectiveness of the proposed pretrain-finetune pipeline. The pretraining process not only provides the RL agent with a high- quality warm start policy but also enhances its ability to adapt to diverse OLTC and SCs settings. Therefore, pretraining the RL agent is essential within the proposed LLM-RL collaboration framework. Finally, an analysis of No-Reflexion reveals that, although it is also capable of self-evolution through interaction with the environment, its improvement rate is significantly lower than that of Proposed. This is because No-Reflexion does not perform in-depth reasoning and reflection on environmental feedback and instead relies on trial-and-error learning. We still use RL algorithms by analogy, which in fact indicates that the interaction samples of No-Reflexion are not fully utilized, thereby reducing βsample efficiencyβ and degrading training performance. C. Execution Performance of Proposed and Baseline Methods After the LLM agent and the RL agent are fully trained and converged, the system enters the online execution phase. During this phase, the LLM agentβs knowledge base and the RL agentβs policy parameters are no longer updated, and Reflexion is disabled. Instead, day-ahead and intra-day strategies are generated directly. In real-world ADNs, the latest knowledge base and policy parameters are stored and maintained by the ADN operator. In the day-ahead stage, the LLM agent determines the scheduling of OLTC and SCs based on forecast information, whose strategies are generated within minutes. Then during the intra-day operation, the RL agent determines the reactive power generation based on real-time measurements, with strategy generation occurring in milliseconds, which is totally sufficient to meet the requirements of real-time control. To evaluate the execution performance of the proposed and baseline methods, we select 25 additional episodes for each random seed to test their converged policies, resulting in a total of 75 test episodes per method. We present the tested average nodal voltage deviations and the nodal voltage violation rates in Tables I and IV, in which the best performance is marked in bold. TABLE I E XECUTION PERFORMANCE OF IEEE 33-BUS SYSTEM Method Voltage Deviation (p.u.) Violation Rate (%) Mean Std. Mean Std. Original 2.50e-02 4.95e-03 10.7 6.06 No-LLM 1.14e-02 2.48e-03 3.65e-02 1.62e-01 No-RL 1.69e-02 3.18e-03 1.21 2.79 Proposed 8.22e-03 2.08e-03 2.60e-03 1.38e-02 Pure-RL 2.06e-02 7.09e-03 3.24 5.31 No-PT 1.02e-02 2.28e-03 1.26 1.53 No-Reflexion 8.60e-03 1.97e-03 6.08e-03 2.42e-02 * Mean and Std. denote mean values and standard deviations across 75 episodes. TABLE IV E XECUTION PERFORMANCE OF IEEE 141-BUS SYSTEM Method Voltage Deviation (p.u.) Violation Rate (%) Mean Std. Mean Std. Original 2.16e-02 3.35e-03 3.74 4.82 No-LLM 1.21e-02 3.23e-03 1.46 2.87 No-RL 1.21e-02 3.26e-03 1.35e-01 3.64e-01 Proposed 8.37e-03 2.21e-03 1.58e-02 8.48e-02 Pure-RL 2.30e-02 6.30e-03 2.89 3.46 No-PT 1.11e-02 2.89e-03 7.67e-01 1.59 No-Reflexion 1.00e-02 2.33e-03 4.70e-02 3.20e-01 * Mean and Std. denote mean values and standard deviations across 75 episodes. As shown in Tables I and IV, Original exhibits the worst performance on the voltage control problem, resulting in large voltage deviations and frequent voltage violations, which demonstrate the necessity of proactive control in ADNs. Although No-LLM and No-RL show significant improvements over Original, their performance still falls short of that of Proposed. This not only confirms the effectiveness of the proposed method but also underscores the importance of coordinating mechanical assets with flexible resources. Especially in the context of widespread integration of distributed energy resources into ADNs, coordinated control of multiple equipment has become a critical approach for addressing voltage challenges. The test results of Pure-RL show that, even at the final stage of training, it still fails to discover a high-quality control policy and performs worse than Original in certain scenarios. This again validates the effectiveness of incorporating the LLM agent, which can significantly improve both optimization efficiency and solution quality on specific problems. As for No- PT and No-Reflexion, their performance is consistent with their training behavior, i.e., the absence of pretraining and Reflexion process leads to performance degradation compared to Proposed. These comparisons also demonstrate the effectiveness of the proposed pretrain-finetune pipeline and the Reflexion-based self-evolution mechanism. V. CONCLUSION To address voltage challenges in ADNs caused by high PV penetration and the inability of existing data-driven methods to effectively handle heterogeneous information, semantic constraints, and dynamic instructions, we propose a hybrid LLM-RL collaboration framework for two-stage voltage control: the LLM agent schedules OLTCs and SCs using day- ahead forecasts and historical records, while the RL agent adjusts PV inverter reactive power based on real-time measurements. Building upon this framework, we further introduce a Reflexion-based self-evolution mechanism tailored for the LLM agent and a pretrain-finetune pipeline designed for the RL agent, enabling effective policy enhancement and efficient coordination between these two agents. Experimental results show that our approach significantly outperforms baseline methods in both voltage regulation performance and training efficiency, demonstrating the effectiveness and necessity of synergizing knowledge-driven reasoning with data-driven control. 10 Drawing from our research and operational experience, we found that LLMs excel at knowledge-based reasoning and generation but are less capable in precise numerical computation. This is also one of the key motivations for proposing the LLM-RL collaboration framework. In future work, how to fully leverage the strengths of LLMs and achieve deeper integration with various domain-specific models represents an important and promising research direction. R EFERENCES [1] X. Su, L. Fang, J. Yang, F. Shahnia, Y. Fu and Z. Y. Dong, βSpatial- Temporal Coordinated Volt/VAR Control for Active Distribution Systems,β IEEE Trans. Power Systems, vol. 39, no. 6, p. 7077-7088, Nov. 2024. [2] S. S. Al Kaabi, H. H. Zeineldin and V. Khadkikar, βPlanning Active Distribution Networks Considering Multi-DG Configurations,β IEEE Trans. Power Systems, vol. 29, no. 2, p. 785-793, Mar. 2014. [3] X. Xu, Y. Li, Z. Yan, H. Ma and M. Shahidehpour, βHierarchical Central- Local Inverter-Based Voltage Control in Distribution Networks Considering Stochastic PV Power Admissible Range,β IEEE Trans. Smart Grid, vol. 14, no. 3, p. 1868-1879, May 2023. [4] R. Yan, Q. Xing and Y. Xu, βMulti-Agent Safe Graph Reinforcement Learning for PV Inverters-Based Real-Time Decentralized Volt/Var Control in Zoned Distribution Networks,β IEEE Trans. Smart Grid, vol. 15, no. 1, p. 299-311, Jan. 2024. [5] L. Ge, J. Li, L. Hou and J. Lai, βAutonomous Voltage Regulation for Smart Distribution Network With High-Proportion PVs: A Graph Meta- Reinforcement Learning Approach,β IEEE Trans. Sustainable Energy, vol. 16, no. 4, p. 2768-2781, Oct. 2025. [6] H. Liu and W. Wu, βTwo-Stage Deep Reinforcement Learning for Inverter- Based Volt-VAR Control in Active Distribution Networks,β IEEE Trans. Smart Grid, vol. 12, no. 3, p. 2037-2047, May 2021. [7] R. Wang, X. Bi and S. Bu, βReal-Time Coordination of Dynamic Network Reconfiguration and Volt-VAR Control in Active Distribution Network: A Graph-Aware Deep Reinforcement Learning Approach,β IEEE Trans. Smart Grid, vol. 15, no. 3, p. 3288-3302, May 2024. [8] X. Yang, H. Liu and W. Wu, βAttention-Enhanced Multi-Agent Reinforcement Learning Against Observation Perturbations for Distributed Volt-VAR Control,β IEEE Trans. Smart Grid, vol. 15, no. 6, p. 5761-5772, Nov. 2024. [9] H. Liu and W. Wu, βOnline Multi-Agent Reinforcement Learning for Decentralized Inverter-Based Volt-VAR Control,β IEEE Trans. Smart Grid, vol. 12, no. 4, p. 2980-2990, Jul. 2021. [10] X. Zheng, S. Yu, H. Cao, T. Shi, S. Xue and T. Ding, βSensitivity-Based Heterogeneous Ordered Multi-Agent Reinforcement Learning for Distributed Volt-Var Control in Active Distribution Network,β IEEE Trans. Smart Grid, vol. 16, no. 3, p. 2115-2126, May 2025. [11] C. Mu, Z. Liu, J. Yan, H. Jia and X. Zhang, βGraph Multi-Agent Reinforcement Learning for Inverter-Based Active Voltage Control,β IEEE Trans. Smart Grid, vol. 15, no. 2, p. 1399-1409, Mar. 2024. [12] H. Liu, W. Wu and Y. Wang, βBi-Level Off-Policy Reinforcement Learning for Two-Timescale Volt/VAR Control in Active Distribution Networks,β IEEE Trans. Power Systems, vol. 38, no. 1, p. 385-395, Jan. 2023. [13] T. Zhang, L. Yu, D. Yue, C. Dou, X. Xie and G. P. Hancke, βTwo- Timescale Coordinated Voltage Regulation for High Renewable- Penetrated Active Distribution Networks Considering Hybrid Devices,β IEEE Trans. Industrial Informatics, vol. 20, no. 3, p. 3456-3467, Mar. 2024. [14] D. Cao et al., βDeep Reinforcement Learning Enabled Physical-Model- Free Two-Timescale Voltage Control Method for Active Distribution Systems,β IEEE Trans. Smart Grid, vol. 13, no. 1, p. 149-165, Jan. 2022. [15] J. -Y. Oh, S. Oh, G. -S. Lee, Y. T. Yoon and S. Jo, βSequential Control of Individual Switches for Real-Time Distribution Network Reconfiguration Using Deep Reinforcement Learning,β IEEE Trans. Smart Grid, vol. 16, no. 5, p. 3666-3683, Sep. 2025. [16] Y. Nie, Y. Kong, X. Dong et al., βA Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges,β arXiv: 2406.11903, Jun. 2024. [17] J. Qiu, K. Lam, G. Li et al., βLLM-based agentic systems in medicine and healthcare,β Nature Machine Intelligence, vol. 6, no. 12, p. 1418-1420, 2024. [18] S. Yue, W. Chen, S. Wang et al., βDISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services,β arXiv: 2309.11325, Sep. 2023. [19] X. Yang et al., βLarge Language Model Powered Automated Modeling and Optimization of Active Distribution Network Dispatch Problems,β IEEE Trans. Smart Grid, early access. [20] A. Jena, F. Ding, J. Wang, Y. Yao and L. Xie, βLLM-Based Adaptive Distribution Voltage Regulation Under Frequent Topology Changes: An In-Context MPC Framework,β IEEE Trans. Smart Grid, vol. 16, no. 5, p. 4297-4300, Sep. 2025. [21] F. Bernier, J. Cao, M. Cordy and S. Ghamizi, βPowerGraph-LLM: Novel Power Grid Graph Embedding and Optimization With Large Language Models,β IEEE Trans. Power Systems, vol. 40, no. 6, p. 5483-5486, Nov. 2025. [22] X. Yang, C. Lin, H. Liu and W. Wu, βRL2: Reinforce Large Language Model to Assist Safe Reinforcement Learning for Energy Management of Active Distribution Networks,β IEEE Trans. Smart Grid, vol. 16, no. 4, p. 3419-3431, Jul. 2025. [23] Z. Yan and Y. Xu, βReal-Time Optimal Power Flow With Linguistic Stipulations: Integrating GPT-Agent and Deep Reinforcement Learning,β IEEE Trans. Power Systems, vol. 39, no. 2, p. 4747-4750, Mar. 2024. [24] Y. Cao et al., βSurvey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods,β IEEE Trans. Neural Networks and Learning Systems, vol. 36, no. 6, p. 9737-9757, Jun. 2025. [25] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, βReflexion: Language agents with verbal reinforcement learning,β Advances in Neural Information Processing Systems, vol. 36, 2024. [26] A. Vaswani, N. Shazeer, N. Parmar et al., βAttention is all you need,β Advances in neural information processing systems, 2017, no. 30. [27] Z. Xu et al., βDriveGPT4: Interpretable End-to-End Autonomous Driving Via Large Language Model,β IEEE Robotics and Automation Letters, vol. 9, no. 10, p. 8186-8193, Oct. 2024. [28] L. Wen, D. Fu, X. Li et al., βDiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models,β arXiv: 2309.16292, Feb. 2024. [29] A. Brohan, Y. Chebotar, C. Finn et al., βDo as I can, not as I say: Grounding language in robotic affordances,β in Conference on robot learning, 2023, p. 287-318. [30] W. Huang, F. Xia, T. Xiao et al., βInner Monologue: Embodied Reasoning through Planning with Language Models,β in Conference on Robot Learning, 2023, p. 1769-1782. [31] M. Jia, Z. Cui and G. Hug, βEnhancing LLMs for Power System Simulations: A Feedback-Driven Multi-Agent Framework,β IEEE Trans. Smart Grid, vol. 16, no. 6, p. 5556-5572, Nov. 2025. [32] R. S. Bonadia, F. C. L. Trindade, W. Freitas and B. Venkatesh, βOn the Potential of ChatGPT to Generate Distribution Systems for Load Flow Studies Using OpenDSS,β IEEE Trans. Power Systems, vol. 38, no. 6, p. 5965-5968, Nov. 2023. [33] S. Majumder, L. Dong, F. Doudi et al., βExploring the capabilities and limitations of large language models in the electric energy sector,β Joule, vol. 8, no. 6, p. 1544-1549, Jun. 2024. [34] K. Deng, Y. Zhou, H. Zeng, Z. Wang and Q. Guo, βPower Grid Model Generation Based on the Tool-Augmented Large Language Model,β IEEE Trans. Power Systems, vol. 40, no. 6, p. 5487-5490, Nov. 2025. [35] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, vol. 1. Cambridge, MA, USA: MIT Press, 1998. [36] J. Wei, X. Wang, D. Schuurmans et al., βChain-of -Thought Prompting Elicits Reasoning in Large Language Models,β Advances in neural information processing systems, vol. 35, p. 24824-24837, 2022. [37] X. Yang et al., βSupplementary files for Two-Stage Active Distribution Network Voltage Control via LLM-RL Collaboration: A Hybrid Knowledge-Data-Driven Approach,β Feb. 2026, [Online]. Available: https://github.com/YangXuSteve/Knowledge-Data-Driven. [38] J. Schulman, F. Wolski, P. Dhariwal, A. Radford and O. Klimov, βProximal policy optimization algorithms,β arXiv:1707.06347, Aug. 2017. [39] M. E. Baran and F. F. Wu, βNetwork reconfiguration in distribution systems for loss reduction and load balancing,β IEEE Trans. Power Delivery, vol. 4, no. 2, p. 1401β1407, Apr. 1989. [40] H. M. Khodr, F. G. Olsina, P. M. De Oliveira-De Jesus, and J. M. Yusta, βMaximum savings approach for location and sizing of capacitors in distribution systems,β Electric Power Systems Research, vol. 78, no. 7, p. 1192-1203, Jul. 2008.