Paper deep dive
Can Large Language Models Reason and Optimize Under Constraints?
Fabien Bernier, Salah Ghamizi, Pantelis Dogoulis, Maxime Cordy
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/26/2026, 1:42:57 AM
Summary
This paper investigates the capability of Large Language Models (LLMs) to perform constrained reasoning and numerical optimization, specifically focusing on the Optimal Power Flow (OPF) problem. The authors evaluate various models (vanilla, SFT, and GRPO-trained) across different grid topologies and find a consistent performance plateau where models fail to satisfy physical constraints, suggesting reliance on pattern matching rather than genuine reasoning.
Entities (5)
Relation Signals (3)
LLMs → evaluatedon → Optimal Power Flow
confidence 100% · we investigate whether LLMs can reason and optimize under the physical and operational constraints of Optimal Power Flow (OPF) problem.
SFT → improves → Response Formatting
confidence 90% · Supervised fine-tuning improves response formatting but fails to improve physical feasibility
GRPO → improves → Constraint Satisfaction
confidence 80% · Reinforcement learning with constraint-satisfaction rewards, however, yields modest but meaningful improvements on some grid topologies.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large Language Models (LLMs) have demonstrated great capabilities across diverse natural language tasks; yet their ability to solve abstraction and optimization problems with constraints remains scarcely explored. In this paper, we investigate whether LLMs can reason and optimize under the physical and operational constraints of Optimal Power Flow (OPF) problem. We introduce a challenging evaluation setup that requires a set of fundamental skills such as reasoning, structured input handling, arithmetic, and constrained optimization. Our evaluation reveals that SoTA LLMs fail in most of the tasks, and that reasoning LLMs still fail in the most complex settings. Our findings highlight critical gaps in LLMs' ability to handle structured reasoning under constraints, and this work provides a rigorous testing environment for developing more capable LLM assistants that can tackle real-world power grid optimization problems.
Tags
Links
- Source: https://arxiv.org/abs/2603.23004v1
- Canonical: https://arxiv.org/abs/2603.23004v1
Trouble viewing inline? Open PDF directly →
Full Text
45,293 characters extracted from source content.
Expand or collapse full text
Can Large Language Models Reason and Optimize Under Constraints? Fabien Bernier 1 , Salah Ghamizi 1,2 , Pantelis Dogoulis 1 , Maxime Cordy 1 1 SnT – University of Luxembourg, Luxembourg 2 Luxembourg Institute of Health (LIH), Luxembourg Abstract Large Language Models (LLMs) have demon- strated great capabilities across diverse natural language tasks; yet their ability to solve abstrac- tion and optimization problems with constraints remains scarcely explored. In this paper, we investigate whether LLMs can reason and op- timize under the physical and operational con- straints of Optimal Power Flow (OPF) problem. We introduce a challenging evaluation setup that requires a set of fundamental skills such as reasoning, structured input handling, arith- metic, and constrained optimization. Our eval- uation reveals that SoTA LLMs fail in most of the tasks, and that reasoning LLMs still fail in the most complex settings. Our findings high- light critical gaps in LLMs’ ability to handle structured reasoning under constraints, and this work provides a rigorous testing environment for developing more capable LLM assistants that can tackle real-world power grid optimiza- tion problems. 1 Introduction Large Language Models (LLMs) have achieved no- table performance across a wide range of natural language understanding and generation tasks, from open-ended dialogue and code synthesis to math- ematical reasoning and scientific question answer- ing (Guo et al., 2025; Srivastava et al., 2023; Wang et al., 2024). Yet a critical question remains largely unanswered: can LLMs reason and optimize under constraints? Real-world decision-making prob- lems, spanning power grid management, financial operations, and cyber-security, require not only language competence but also the ability to jointly interpret structured inputs, perform multi-step arith- metic, satisfy interacting physical or logical con- straints, and converge to feasible, near-optimal so- lutions. These challenges go far beyond what cur- rent benchmarks assess. The rewards of closing this gap are high. In safety-critical infrastructure such as power grids, for instance, LLMs capable of genuine constrained reasoning could act as world models, avoiding heavy simulations and improving autonomous plan- ning / operational assistance that current systems cannot support. More broadly, advancing LLM competence on structured optimization problems would unlock a new class of AI assistants for high- value engineering and operational domains. Existing benchmarks, however, fall short in eval- uating these capabilities in a rigorous and realis- tic manner. General reasoning benchmarks such as MMLU (Wang et al., 2024) and GPQA (Rein et al., 2024) assess broad knowledge and expert- level question answering, but do not require iter- ative numerical optimization or constraint satis- faction over structured physical systems. Logical reasoning benchmarks such as ARC-AGI (Chol- let, 2019), SATBench (Wei et al., 2025) and Ze- braLogic (Lin et al., 2025) probe constraint sat- isfaction in formal or combinatorial settings, but rely on synthetic puzzles disconnected from real- world engineering complexity. Domain-specific work on LLMs for power systems (Huang et al., 2024; Ren et al., 2025) or finance has largely fo- cused on agents augmented with external solvers or reinforcement learning, missing the core ques- tion of whether the model itself can reason through constraints end to end. To address this gap, we introduce a new chal- lenging task that evaluates LLMs in performing end-to-end constrained reasoning and numerical optimization over realistic power-grid operation scenarios using well known datasets. To further investigate LLMs abilities on such complex tasks, we assess them extensively after applying super- vised fine-tuning (SFT) and group relative policy optimization (GRPO). Our empirical evaluation of state-of-the-art LLMs, including both instruction-tuned models and reasoning-augmented variants, across small and medium sizes, reveals a consistent failure pat- arXiv:2603.23004v1 [cs.AI] 24 Mar 2026 Study 1: Few-Shot In-Context Learning Grid topology & parameters "bus": "x": [ [ 345.0, ... "load": ... Prompt Input Few-Shot Examples Input → Output ... Input → Output Input → Output ... Input → Output Input → Output ... Input → Output Target Query Unseen Input → ? LLM Processing (native reasoning) zero external tool Study 2: Supervised Fine-Tuning (SFT) Large Training Dataset Input loads 1 → Truth 1 Input loads 2 → Truth 2 Input loads 3 → Truth 3 ... Supervised Training Loop Grid A Fine-Tuned “Specialist” Study 3: Group Relative Policy Optimization (GRPO) Grid A Large Training Dataset Input loads 1 → Truth 1 Input loads 2 → Truth 2 Input loads 3 → Truth 3 ... Grid A GRPO Reinforcement Loop ●Formatting (binary) ●Physics Constraints Satisfaction (%) REWARD Reasoning “Specialist” Grid A Evaluation ●Llama 3.1 8B ●Qwen 3 8B ●Qwen 2.5 32B NON-REASONING ●Deepseek R1 Llama 3.1 8B ●Deepseek R1 Qwen 3 8B ●QwQ 32B REASONING Models Evaluation Metrics MSE Generators and buses Constraint Satisfactions (%) Breaches of Kirchhoff’s Laws / Thermal limits Format Validity (%) Parsable JSON/structure Insights Study 1: Variable MSE, constraints plateau, large grid = fail Study 2: MSE ↘ , constraints == Study 3: MSE ↘ , constraints ↗ Figure 1: Overview of our study. LLMs are evaluated with simple in-context learning, and in the evaluation settings after supervised fine-tuning (for non-reasoning models) and group relative policy optimization (for reasoning models); all models outputs are evaluated with the same metrics: MSE, constraint satisfaction, and structure validity. tern. Across virtually all tasks requiring genuine optimization under constraints, models remain at a constraint satisfaction rate of approximately 55– 60%, regardless of architecture, scale, or training regime. Reasoning models, despite their extended chain-of-thought generation, do not systematically outperform their non-reasoning counterparts. Su- pervised fine-tuning improves response formatting but fails to improve physical feasibility, confirm- ing shortcuts in reasoning. Reinforcement learn- ing with constraint-satisfaction rewards, however, yields modest but meaningful improvements on some grid topologies. In summary, our contributions are as follows: (1) a new task for evaluating LLM reasoning under physical constraints via the OPF problem, (2) a rigorous evaluation protocol with constraint-aware metrics, and (3) an extensive empirical study span- ning vanilla, SFT, and GRPO-trained models, re- vealing a consistent failure plateau in constraint satisfaction. 2 Related work 2.1 LLMs for Power Systems The application of LLMs to energy and power sys- tems has seen significant expansion across vari- ous sub-domains. Within the specific area of op- timization, existing literature mainly explores the use of LLMs as high-level agents. For instance, LLM-SUC (Ren et al., 2025) focuses on train- ing LLM-based agents via reinforcement learn- ing to optimize financial costs in energy markets. Specialized frameworks such as LLM4OPF and LLM4EV (Huang et al., 2024) target specific tech- nical challenges in Optimal Power Flow and elec- tric vehicle scheduling. However, these agent- based approaches often rely on reinforcement learn- ing for cost optimization, or external tool calling, rather than direct multi-constraint reasoning. In simulation tasks, DALINE-GPT (Mirshekali et al., 2025) has demonstrated high success rates in generating simulation code (more specifically for MATPOWER), although it still struggles to construct small distribution networks without in- tensive prompt engineering. Finally, to move be- yond traditional regression tasks, the existing work mostly focuses on load forecasting: models like EF-LLM (Qiu et al., 2024) and LFLLM (Liu et al., 2024b) use fine-tuning to handle short-term fore- casting across multiple voltage levels. Newer cross- modal approaches, such as the GPT-Agent (Yan et al., 2024) for PV power forecasting, have also integrated linguistic weather data with numerical vectors to improve interpretability and accuracy, but still lack the structured reasoning required to solve multi-constraint optimization problems with interacting variables. Indeed, even recent work that directly targets optimization problems such as OPF (Bernier et al., 2025) evaluates LLM per- formance solely through MSE, without verifying whether the produced solutions satisfy the under- lying physical and operational constraints, letting feasibility guarantees largely unaddressed. 2.2 LLM Reasoning Capabilities Traditional reasoning benchmarks have smoothly transitioned from testing basic linguistic under- standing to assessing massive multitask knowledge (e.g. MMLU (Wang et al., 2024), BIG-bench (Sri- vastava et al., 2023)) and expert-level reasoning (GPQA (Rein et al., 2024)). Despite this evolu- tion, several limitations persist. Logical reason- ing benchmarks such as SATBench (Wei et al., 2025) and ZebraLogic (Lin et al., 2025) evalu- ate constraint-satisfaction abilities, but they often rely on synthetic puzzles or formal logic state- ments that do not reflect the noise and complex- ity of real-world engineering tasks. Other bench- marks focus on narrow aspects of reasoning: Plan- bench (Valmeekam et al., 2023) assesses plan- ning and reasoning about change, while Causal- bench (Wang, 2024) evaluates causal inference but does not address optimization or structured con- straint satisfaction. Recent surveys indicate that while LLMs achieve high scores on static benchmarks, they often rely on shortcut learning-memorizing solution templates rather than engaging in generalizable reasoning. This leads to a lack of process credibility, where models may reach the correct answer through flawed logical steps. (Guo et al., 2025) highlights a significant divergence between Supervised Fine- Tuning (SFT) and Reinforcement Learning (RL) approaches: while SFT helps models follow spe- cific output formats, it can lead to pattern match- ing that occasionally suppresses pre-trained reason- ing logic or fails on out-of-distribution tasks (Chu et al., 2025). State-of-the-art models are addition- ally fragile when exposed to small perturbations or adversarial inputs, often failing tasks they previ- ously solved if the context is slightly modified (Ni et al., 2025). Furthermore, widespread data contam- ination in training sets often inflates performance metrics, masking the models’ true inability to han- dle novel, out-of-distribution optimization prob- lems (Ni et al., 2025). 2.3 LLMs on Constraint Satisfaction Tasks Prior work on LLMs and constrained reasoning can be organized into three main categories. In the first one, works are focused on the problem formulation from natural language. NL4Opt evaluates whether models can recover decision variables, objectives, and constraints from textual optimization descrip- tions (Ramamonjison et al., 2023). More recent work in the constraint-programming community extends this direction by asking LLMs to gener- ate executable constraint models from combinato- rial problem statements and by introducing broader benchmarks for this capability (Michailidis et al., 2024, 2025). While this line of work is closely re- lated to optimization, it primarily evaluates model- ing and formalization accuracy, rather than whether an LLM can itself carry out constrained search and produce a valid solution end to end. A second direction of research focuses on direct constraint satisfaction and combinatorial reasoning. The authors in (Madusanka et al., 2024), exam- ine satisfiability judgments under different logical fragments expressed in natural language, while in LR 2 Bench (Chen et al., 2025), the authors eval- uate CSP-style tasks that require long reasoning chains, reflection, and backtracking. In a related optimization setting, EHOP (Duchnowski et al., 2025) shows that LLM performance on NP-hard problems is highly sensitive to surface presenta- tion, suggesting dependence on familiar formula- tions rather than robust abstraction over constraints. Taken together, these studies suggest that current models remain fragile when constraints interact strongly, search depth increases, or problem formu- lations deviate from familiar templates. A third direction is based on the work to im- prove reliability through neuro-symbolic methods. Logic-LM and SatLM translate natural-language problems into symbolic or declarative representa- tions and then rely on external solvers for infer- ence (Pan et al., 2023; Ye et al., 2023). In the constraint-programming literature, LLMs have also been used to generate streamlining constraints that accelerate downstream search (Voboril et al., 2025). These methods often improve accuracy, but they also move away from fully end-to-end reasoning. In contrast, our work focuses on a demanding task requiring end-to-end model competence: jointly interpreting context, satisfying the constraints, and producing accurate responses in a hard numerical problem. 3 Method We design a rigorous evaluation framework to as- sess whether LLMs can reason and optimize under the physical and operational constraints of a com- plex real-world problem. We study the Optimal Power Flow (OPF) problem, which is a fundamen- tal to operational planning and grid management. This task requires input interpretation, multi-step arithmetic, and simultaneous satisfaction of inter- acting physical constraints. 3.1 Problem Formulation of Optimization with Reasoning Non-Convex Non-Linear Programming (NLP) problems provide a powerful framework for model- ing and evaluating logical reasoning capabilities of the LLMs. In such problems, solutions must satisfy a set of constraints over variables and their possible value, while iteratively converging towards an op- timal solution that minimizes an expected reward or fitness function. The OPF problem seeks to min- imize a cost function (like generation fuel cost or system losses) subject to the physical laws of the electrical grid. In practice, solving such a problem by an LLM would demonstrate true reasoning capa- bilities given it requires solving multiple complex, iterative sub-problems: Abstraction. Natural language parsing and un- derstanding of a textual power grid description, and a topological representation of its components. The main challenge lies in the capability of the LLM to maintain a "chain of thought" when building its representation. If the logical chain breaks at step two, the final output will fail to represent an OPF solution. Mathematics. Translating that description into algebraic equations and inequalities. The main challenges for an LLM lie in identifying the right formulation (complexe AC OPF, or simplified DC OPF approximation) and its ability to reason about the trade-offs between computational efficiency and physical accuracy, and structuring those equa- tions into a syntax that an optimization algorithm can solve. Multi-Step optimization.Computing the numer- ical solution iteratively using well known algo- rithms such as Newton-Raphson and Interior Point methods. The main challenges are the mathemati- cal bottleneck and the exponential error compound- ing in iterative methods. For instance, to solve a 9-bus system, the very first step is to construct the Admittance Matrix. For an AC OPF, this is a 9×9 matrix of complex numbers. To solve the power flow equations, algorithms must iteratively invert or factorize these matrices (often building a Jacobian matrix of partial derivatives). LLMs may not reli- ably perform matrix inversion or solve systems of linear equations without tools, and there is high risk for the LLM to introduce a small rounding error. In the next iteration, that error is fed back into the non-linear equations, compounding exponentially. Given the complexity of the problem at hand, our study explores to which extent LLMs could approximate the solution, especially when provided examples, and fine-tuned for this task. 3.2 Power Grid Optimization task In this task, LLMs must optimize power generation and distribution across a network while satisfying multiple physical and operational constraints. The input consists of a power grid topology represented as a heterogeneous graph, where nodes represent different components (buses, generators, loads, and transmission lines), each with distinct feature sets. The grid configuration includes active and reac- tive power demands at load nodes and generator capacity limits. Dataset. We build our dataset usingPyTorch Geometric framework and its interface to OPFData(Lovett et al., 2024), a large-scale col- lection of solved OPF instances derived from PGLib-OPFbenchmark grids. Each instance is en- coded as a heterogeneous graph in which the net- work’s physical backbone is represented by bus nodes connected through relations corresponding to AC lines and transformers. The resulting represen- tation supports learning tasks that map grid states and parameters to OPF-relevant quantities (e.g., op- erating points or feasibility/optimality surrogates), andOPFDataadditionally includes variants with topological perturbations to evaluate robustness un- der contingency-like structural changes, referred to asN(i.e., original in-distribution scenarios) and N-1scenarios (i.e., out of distribution scenarios where one line is disconnected). Concretely, we evaluate on three benchmark topologies of increas- ing complexity:case14(14 buses),case30and case118. Mean Squared Error. We measure the numer- ical accuracy of the predicted OPF solutions us- ing the Mean Squared Error (MSE) between the model’s predictions and the ground-truth solver outputs in the dataset. We report the sum of two MSE terms: the generator MSE, computed over the active and reactive power outputs for each genera- tor, and the bus MSE, computed over the voltage magnitude and voltage angle at each bus. Beyond constraint satisfaction, measuring MSE ensures the solution remains numerically close from the opti- mal dispatch. Evaluated Constraints. We assess each pre- dicted OPF solution using three feasibility criteria. For each busi, we compute the active and the reac- tive power balance residuals∆P i = P spec i − P imp i and∆Q i = Q spec i − Q imp i , where the specified injections are obtained from predicted generation minus demand, and the implied injections are those induced by the predicted voltage values under the instance’s topology. A bus satisfies the power-flow constraints when: q ∆P 2 i + ∆Q 2 i ≤ ε pf . Voltage feasibility is defined by the bound:v min i ≤ ˆv i ≤ v max i , and generator feasibility is defined by the active-power constraint:P min g ≤ ˆ P g ≤ P max g . For each sample, we compute the fraction of buses and generators satisfying each of these conditions, and we report the dataset-level average of these per- sample fractions. In addition, we report an aggre- gate constraint satisfaction score obtained by micro- averaging all individual (bus and generator level) constraint checks across the full evaluation set, so that each checked constraint instance contributes equally to the final measure. Mathematically, lets indexes the evaluation samples, and denote byn (s) pf , n (s) v , andn (s) g the number of evaluated buses and generators for the power-flow, voltage, and gen- erator constraints, respectively. Letn (s) pf,sat ,n (s) v,sat , andn (s) g,sat denote the corresponding numbers of satisfied constraints. The final constraint score is defined as: CS = 100× P s n (s) pf,sat + n (s) v,sat + n (s) g,sat P s n (s) pf + n (s) v + n (s) g (1) 4 Experimental Protocol 4.1 Models Model sizes. We evaluate two model sizes to study the impact of scale and training regime on OPF reasoning: (i) small models with 8B pa- rameters and (i) medium models with approxi- mately 32B parameters. The small models com- prise Llama 3.1 8B and Qwen3 8B as baselines, alongside their DeepSeek-distilled reasoning coun- terparts: Deepseek R1 Distill Llama 3.1 8B and DeepSeek R1 Qwen3 8B. The medium sizes com- prise Qwen 2.5 32B as a vanilla baseline and QwQ (32B) as its reasoning-focused counterpart. 4.2 Fine-tuning SFT. To evaluate the impact of standard align- ment in constraint satisfaction, we apply Super- vised Fine-Tuning (SFT) to the vanilla instruct models. We use LoRA (Hu et al., 2022) to train adapters on the training set of OPF problems. This process optimizes the model to predict the ground- truth solutions in zero-shot directly, ensuring the output format and basic physical constraints are learned through direct imitation of high-quality ex- amples. GRPO. Reasoning variants are trained using Group Relative Policy Optimization (GRPO) (Liu et al., 2024a), a reinforcement learning objective that aligns model preferences with structured valid- ity and feasibility rewards. Unlike standard RLHF, GRPO eliminates the need for a separate critic model, by sampling a group of multiple outputs for each prompt and using the relative reward of each output compared to the group average to com- pute the policy gradient. This allows the model to reason through constraints iteratively. The reward Rdecomposes into a structural component and a constraint-satisfaction component: R = r json + r CS The termr json ∈ 0, 0.5rewards adherence to the required JSON schema (0 for an unparsable output; 0.5 for a well-formed output). The term r CS ∈ [0, 1]captures feasibility by measuring the fraction of evaluated constraints satisfied by the proposed solution as defined in equation 1. 4.3 Evaluation Vanilla models are assessed using in-context learn- ing (ICL). We provide the same prompt templates and demonstration format across models, including fine-tuned ones. Across all configurations, models are instructed to return outputs in a standardized JSON schema that encodes the candidate dispatch, key feasibility indicators, and any auxiliary reason- ing traces permitted by the prompt. At inference time, we request the same JSON schema used during training and parse model out- puts with strict validation. For ICL conditions, we reuse the same samples across models and capacity tiers to ensure comparability. For zero-shot condi- tions, no pre-samples are provided and the prompt only specifies the schema, objectives, and any re- quired units or conventions. For every experiment, the model is assessed on 1,000 different conver- sations, the last message of each being the query input. In all cases, performance is computed on the parsed outputs; malformed outputs can’t be parsed and are not evaluated. 5 Results We report the performance of models on the three OPF benchmark cases (14, 30, 118), evaluated across three metrics: satisfied constraints, defined in equation 1 (%, higher is better), valid responses (%, higher is better), and total MSE (lower is bet- ter). The table detailing the satisfied constraints decomposition is available in the data appendix. 5.1 Vanilla models Figure 2 reports that across all three cases, the con- straint satisfaction rate remains globally flat, hover- ing around 55–60% for every model regardless of architecture or training regime. This plateau is ex- plained by the large non satisfaction of power flow constraints for all models, representing∼40% of all the evaluated constraints; suggesting that mod- els are not actively reasoning through the power- flow equations but are instead producing outputs that satisfy a fixed subset of constraints by default, likely through pattern matching rather than genuine optimization. Non-reasoning vs. reasoning models. Reasoning-augmentedmodelsdonotsys- tematically improve constraint satisfaction over their vanilla counterparts.In several settings, reasoning models even degrade performance on the MSE axis: for instance, DeepSeek-R1-Qwen3-8B shows a sharp MSE spike on case14 and case30 (reaching values above10 1 ), indicating that extended chain-of-thought generation may lead to numerically unstable outputs. The only consistent advantage of reasoning models is a higher rate of valid (parsable) responses in some configurations, yet this benefit is not universal. Effect of model size. Scaling from 8B to 32B parameters yields limited gains on the constraint satisfaction score, which remains near 55–60% for both Qwen2.5-32B and QwQ-32B, matching the Model CaseScenario Gen. limits Voltage limits PF Deepseek-R1 Qwen3-8B 14 VanillaN100%98.7%7.5% GRPON100%99.6%6.25% GRPON − 1100%98.6%3.19% 30 VanillaN100%98.9%11.7% GRPON100%99.5%12.5% GRPON − 199.8%97%9.71% Deepseek-R1 Llama3.1-8B 14 VanillaN99.9%98.4%3.94% GRPON100%99.9%1.37% GRPON − 199.9%99.8%0.83% 30 VanillaN100%99.3%10.6% GRPON100%99.6%24.6% GRPON − 1100%100%17.8% Table 1: % of satisfied constraints by model, case, and scenario. 8B models. The most visible size-related effect is on response validity: larger models tend to produce well-formed JSON outputs more reliably, particu- larly on the larger case118 topology where smaller models completely fail in generating any parsable response. Regarding the MSE: although Qwq-32b achieves competitive MSE on cases 14 and 30, the performance doesn’t improve monotically with scale. These results suggest that simply scaling model size is insufficient to solve complex opti- mization like OPF. 5.2 Supervised Fine-Tuning Performance on scenario N.Figure 3 shows that fine-tuning consistently improves response validity and MSE across all models and cases. Constraint satisfaction, however, remains largely unchanged after SFT at around 55–60%, indicating that the models learn to produce well-formatted but not necessarily physically feasible solutions, confirm- ing this reasoning shortcut established in the state of the art. Scenario N vs. N-1. Under theN − 1contin- gency setting, valid response rates unsurprisingly drop noticeably for all grids, and MSE increases across all models, indicating that topological per- turbations expose the limits of SFT-based gener- alization. Constraint satisfaction scores remain at the same plateau as in the vanilla andNset- tings, confirming that simple fine-tuning on nomi- nal cases does not confer robustness to structural grid changes. 5.3 Impact of GRPO Figure 4 reports the performance of reasoning mod- els before and after GRPO fine-tuning on theNand llama3.1-8b deepseek-r1-llama3.1-8b qwen3-8b deepseek-r1-qwen3-8b qwen2.5-32b qwq-32b 0 20 40 60 80 100 % case14 llama3.1-8b deepseek-r1-llama3.1-8b qwen3-8b deepseek-r1-qwen3-8b qwen2.5-32b qwq-32b case30 llama3.1-8b deepseek-r1-llama3.1-8b qwen3-8b deepseek-r1-qwen3-8b qwen2.5-32b qwq-32b case118 10 2 10 1 10 0 10 1 10 3 10 2 10 1 10 0 10 1 10 1 10 0 MSE Satisfied constraints (%) Valid responses (%) MSE Figure 2: Performance of vanilla models (before fine-tuning) on cases 14, 30 and 118. CaseModelin-contextzero-shot Valid (%) CS (%)↑MSE↓ Valid (%) CS (%)↑ MSE↓ case14llama3.1-8b95.057.83.91e-0399.763.85.26e-03 qwen3-8b99.358.43.52e-0332.859.52.94e-02 qwen2.5-32b93.558.22.48e-0378.864.75.40e-03 case30llama3.1-8b47.155.14.22e-0382.561.92.60e-02 qwen3-8b97.556.51.43e-0335.655.12.46e-02 qwen2.5-32b86.255.78.37e-0496.459.81.01e-02 case118llama3.1-8b0.0–0.0– qwen3-8b38.357.29.71e-020.0– qwen2.5-32b35.355.61.68e+000.0– Table 2: Performance of fine-tuned models, in-context learning vs. zero-shot inference. 0 20 40 60 80 100 case14 % llama3.1-8bqwen3-8bqwen2.5-32b 0 20 40 60 80 100 case30 % vanilla scenario N fine-tuned scenario N fine-tuned scenario N 1 0 20 40 60 80 100 case118 % vanilla scenario N fine-tuned scenario N fine-tuned scenario N 1 vanilla scenario N fine-tuned scenario N fine-tuned scenario N 1 10 3 10 2 MSE 10 3 10 2 MSE 10 1 10 0 MSE Satisfied constraints (%) Valid responses (%) MSE Figure 3: Performance of vanilla models (before fine- tuning) and of fine-tuned models on N and N-1 cases. N − 1 test scenarios. GRPO training yields mixed results: while it improves response validity (can be 0 25 50 75 100 case14 % deepseek-r1 llama3.1-8b deepseek-r1 qwen3-8b vanilla scenario N GRPO scenario N GRPO scenario N 1 0 25 50 75 100 case30 % vanilla scenario N GRPO scenario N GRPO scenario N 1 10 2 10 1 10 0 10 1 MSE 10 2 10 1 10 0 10 1 MSE Satisfied constraints (%) Valid responses (%) MSE Figure 4: Performance of reasoning models, before GRPO (vanilla) and after GRPO, on N and N-1 scenar- ios. attributed to the incorporation of the correct format reward), constraint satisfaction still mainly remains Case Modelbasicenhanced Valid (%) CS (%)↑ MSE↓ Valid (%) CS (%)↑ MSE↓ case14llama3.1-8b100.058.04.80e-03100.058.04.92e-03 qwen3-8b99.858.83.61e-0399.659.03.61e-03 case30llama3.1-8b100.054.61.65e-0399.955.41.57e-03 qwen3-8b91.254.71.56e-0398.560.41.66e-03 Table 3: Performance of inference with the basic system prompt defined above, and inference with an LLM-enhanced system prompt. at near 55–60%, except for the case 30, clearly get- ting over the 60% bar for the first time, including for theN − 1contingency setting. Table 1 espe- cially shows that GRPO improves the power flow equations constraint on case 30 with a significant increase (+132%) for Deepseek-R1-Llama3.1-8B, but degrades it on case 14. 5.4 Ablations Table 2 compares fine-tuned models evaluated in their training regime (zero-shot) against the same models prompted with in-context learning exam- ples (ICL). Across all models and grid sizes, ICL consistently yields lower MSE and higher valid response rates than zero-shot inference, suggest- ing that in-context examples provide structural guidance that compensates for the limitations of imitation-based fine-tuning, even when the model has already been exposed to the output format dur- ing training. Impact of prompt To assess whether prompt quality influences model performance, we designed an enhanced version of the original system prompt using ChatGPT, enriching it with more explicit physical context, clearer formatting instructions, emphasizing on constraints and being domain- specific terminology related to power systems and OPF. As shown in Figure 3, this enhancement has virtually no impact: all metrics remain essentially identical. The performance bottleneck might there- fore not lie in the clarity or richness of the prompt formulation, but rather in the models’ fundamental inability to reason over power-flow constraint. Uncertainty and Calibration. To complement constraint satisfaction and MSE, we evaluate the calibration of constraint satisfaction predictions us- ing Expected Calibration Error (ECE), Brier Score, and Negative Log-Likelihood (NLL) on Llama 3.1 8B forcase14. The vanilla model already exhibits poor calibration (ECE = 0.696, Brier = 0.485, NLL = 1.192), and SFT does not improve it; scores slightly worsen on scenarioN(ECE = 0.720, Brier = 0.519, NLL = 1.275). Interestingly, SFT on the N−1out-of-distribution scenario yields marginally better calibration (ECE = 0.677, Brier = 0.458, NLL = 1.131) than the in-distribution counterpart, suggesting that the model’s confidence is not mean- ingfully shaped by fine-tuning. Conclusion We introduced a rigorous evaluation framework for assessing whether LLMs can reason and opti- mize under the physical and operational constraints of the Optimal Power Flow problem: a task de- manding structured input interpretation, multi-step arithmetic, and simultaneous satisfaction of inter- acting physical constraints. Our empirical study yielded three consistent findings. All models, re- gardless of architecture or scale, show a plateau in constraint satisfaction, suggesting pattern match- ing rather than genuine optimization. Supervised fine-tuning reliably improves response formatting and marginally reduces MSE, but leaves constraint satisfaction essentially unchanged, confirming the shortcut-learning behavior reported in prior work. Reinforcement learning with constraint-satisfaction rewards yields modest but meaningful improve- ments on some topologies, leading some models exceed the constraints plateau, including in out-of- distribution examples. Across all settings, no single model dominates on both metrics simultaneously. Qwen-family mod- els tend to produce lower MSE and better-formed outputs, while Llama-based models show more sta- ble constraint satisfaction under GRPO fine-tuning. Scaling to 32B parameters improves response va- lidity on larger grids but does not resolve the con- straint satisfaction plateau. Future work should include more datasets from other domains, as well as richer reward signals that decompose individual constraint violations to guide more targeted reinforcement learning. 6 Limitations 6.1 The "Memorization" vs. "Optimization" trap Since LLMs cannot actually perform 10 iterations of Newton-Raphson in their "latent space," they often resort to Result Guessing. If the model rec- ognizes the input as "similar to a standard power grid, or if they recognize similar financial or cyber- security datasets, it will provide values that look like a valid solution. Our results using the N-1 out of distribution test set demonstrate an increased error compared to the N case (in-distribution) that suggest that even under fine-tuning and GRPO, LLMs mainly rely on memorization mechanisms. 6.2 The lack of tools Using an LLM to evaluate OPF without tools is actually testing its calculating limitations, not only its logical reasoning. One could argue that a better paradigm is to restrict LLMs to abstraction tasks only: Reading the grid data, understanding the physics, deducing the correct constraints, and writ- ing the mathematical formulation into solver code (like Pyomo or JuMP). Then leaving the solver (Gurobi, Ipopt) execute the heavy numerical matrix arithmetic and return the result. However, multi- ple research ave demonstrated the capabilities of deep learning models in reliable OPF predictions (our OPF papers), and a previous study suggested that LLM could also solve the OPF problem with small grids. In addition, our results suggest that not only the predicted solutions by LLMs exhibits high MSE, but even the understanding (valid out- puts) and the basic constraint satisfaction are not supported. Acknowledgements This work was partially funded by FNR CORE project LEAP (17042283) and by Creos Luxem- bourg S.A. Dr.Ghamizi is supported by the Luxem- bourg National Research Fund (FNR) CORE C24/IS/18942843. Ethical Considerations This work studies the ability of LLMs to reason under physical and operational constraints in op- timal power flow settings, with the goal of better understanding their current limitations rather than advocating for their autonomous deployment in real-world power systems. Since power grid opera- tion is extremely safety-critical, one important ethi- cal implication of our findings is that LLM outputs should not be treated as reliable decision support mechanisms yet. However, at the same time our work may have positive societal value by provid- ing a challenging benhcmark that helps identify failure modes and motivate safer hybrid systems combining LLMs with formal solvers and human oversight. More broadly, we hope that this work encourages future research on trustworthy reasoing an robustness for LLM-based systems deployed in critical application domains. References Fabien Bernier, Jun Cao, Maxime Cordy, and Salah Ghamizi. 2025. Powergraph-llm: Novel power grid graph embedding and optimization with large lan- guage models. IEEE Transactions on Power Systems. Jianghao Chen, Zhenlin Wei, Zhenjiang Ren, Ziyong Li, and Jiajun Zhang. 2025. Lr 2 bench: Evaluating long-chain reflective reasoning capabilities of large language models via constraint satisfaction problems. In Findings of the Association for Computational Linguistics: ACL 2025, pages 6006–6032. François Chollet. 2019. On the measure of intelligence. Preprint, arXiv:1911.01547. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Sheng- bang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. 2025. Sft mem- orizes, rl generalizes: A comparative study of foundation model post-training.arXiv preprint arXiv:2501.17161. Alex Duchnowski, Ellie Pavlick, and Alexander Koller. 2025. A knapsack by any other name: Presentation impacts LLM performance on NP-hard problems. In Findings of the Association for Computational Lin- guistics: EMNLP 2025, pages 6628–6651, Suzhou, China. Association for Computational Linguistics. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, and 1 others. 2025. Deepseek- r1 incentivizes reasoning in llms through reinforce- ment learning. Nature, 645(8081):633–638. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3. Chenghao Huang, Siyang Li, Ruohong Liu, Hao Wang, and Yize Chen. 2024. Large foundation models for power systems. In 2024 IEEE Power & Energy Soci- ety General Meeting (PESGM), pages 1–5. IEEE. Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, and Yejin Choi. 2025. Zebralogic: On the scaling limits of llms for logical reasoning. arXiv preprint arXiv:2502.01100. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024a. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. Guolong Liu, Yan Bai, Keen Wen, Xinlei Wang, Yanli Liu, Gaoqi Liang, Junhua Zhao, and Zhao Yang Dong. 2024b. Lfllm: A large language model for load forecasting. Authorea Preprints. Sean Lovett, Miha Zgubic, Sofia Liguori, Sephora Madjiheurem, Hamish Tomlinson, Sophie Elster, Chris Apps, Sims Witherspoon, and Luis Piloto. 2024. Opfdata: Large-scale datasets for ac optimal power flow with topological perturbations. Preprint, arXiv:2406.07234. Tharindu Madusanka,Ian Pratt-Hartmann,and Riza Theresa Batista-Navarro. 2024. Natural lan- guage satisfiability: Exploring the problem distri- bution and evaluating transformer-based language models. In Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15278–15294. Kostis Michailidis, Dimos Tsouros, and Tias Guns. 2025.Cp-bench:Evaluating large language models for constraint modelling. arXiv preprint arXiv:2506.06052. Kostis Michailidis, Dimos Tsouros, Tias Guns, and P. Shaw. 2024. Constraint modelling with llms using in-context learning. In 30th International Confer- ence on Principles and Practice of Constraint Pro- gramming (CP 2024), volume 307, pages 20:1–20:15. Schloss Dagstuhl–Leibniz-Zentrum für Informatik. Hamid Mirshekali, Mohammad Reza Shadi, Fatemehsa- dat Ghanadi Ladani, and Hamid Reza Shaker. 2025. A review of large language models for energy sys- tems: Applications, challenges, and future prospects. IEEE Access. Shiwen Ni, Guhong Chen, Shuaimin Li, Xuanang Chen, Siyi Li, Bingli Wang, Qiyao Wang, Xingjian Wang, Yifan Zhang, Liyang Fan, and 1 others. 2025. A survey on large language model benchmarks. arXiv preprint arXiv:2508.15361. Liangming Pan, Alon Albalak, Xinyi Wang, and William Wang. 2023. Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3806–3824. Zihang Qiu, Chaojie Li, Zhongyang Wang, Renyou Xie, Borui Zhang, Huadong Mo, Guo Chen, and Zhaoyang Dong. 2024. Ef-llm: Energy forecast- ing llm with ai-assisted automation, enhanced sparse prediction, hallucination detection. arXiv preprint arXiv:2411.00852. Rindranirina Ramamonjison, Timothy Yu, Raymond Li, Haley Li, Giuseppe Carenini, Bissan Ghaddar, Shiqi He, Mahdi Mostajabdaveh, Amin Banitalebi- Dehkordi, Zirui Zhou, and 1 others. 2023. Nl4opt competition: Formulating optimization problems based on their natural language descriptions. In NeurIPS 2022 competition track, pages 189–203. PMLR. David Rein, Betty Li Hou, Asa Cooper Stickland, Jack- son Petty, Richard Yuanzhe Pang, Julien Dirani, Ju- lian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First conference on language modeling. Xinxing Ren, Chun Sing Lai, Gareth Taylor, and Zekun Guo. 2025. Can large language model agents balance energy systems? arXiv preprint arXiv:2502.10557. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, and 1 others. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on machine learning research. Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. 2023. Planbench: An extensible benchmark for eval- uating large language models on planning and reason- ing about change. Advances in Neural Information Processing Systems, 36:38975–38987. Florentina Voboril, Vaidyanathan Peruvemba Ra- maswamy, and Stefan Szeider. 2025. Generating streamlining constraints with large language models. Journal of Artificial Intelligence Research, 84. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Ad- vances in Neural Information Processing Systems, 37:95266–95290. Zeyu Wang. 2024. Causalbench: A comprehensive benchmark for evaluating causal reasoning capabil- ities of large language models. In Proceedings of the 10th SIGHAN Workshop on Chinese Language Processing (SIGHAN-10), pages 143–151. Anjiang Wei, Yuheng Wu, Yingjia Wan, Tarun Suresh, Huanmi Tan, Zhanke Zhou, Sanmi Koyejo, Ke Wang, and Alex Aiken. 2025. Satbench: Benchmarking llms’ logical reasoning via automated puzzle genera- tion from sat formulas. In Proceedings of the 2025 Conference on Empirical Methods in Natural Lan- guage Processing, pages 33820–33837. Ziming Yan, Du Zhenyuan, Yan Xu, and Ziyan Zhou. 2024. Probabilistic pv power forecasting by a multi- modal method using gpt-agent to interpret weather conditions. In 2024 IEEE 19th Conference on In- dustrial Electronics and Applications (ICIEA), pages 1–6. IEEE. Xi Ye, Qiaochu Chen, Isil Dillig, and Greg Durrett. 2023. Satlm: Satisfiability-aided language models using declarative prompting. Advances in Neural Information Processing Systems, 36:45548–45580. Model Case Scenario Gen. limits Voltage limits PF QwQ 32B 14VanillaN100%100%0.415% 30VanillaN100%100%0.199% Qwen2.5-32B 14 VanillaN100%100%1.19% SFTN100%100%1.41% SFTN − 1100%100%0.426% 30 VanillaN100%100%0.134% SFTN100%100%2.51% SFTN − 1100%100%0.747% LLaMA 3.1-8B 14 VanillaN100%100%1% SFTN100%99.9%0.609% SFTN − 199.4%96.4%0.208% 30 VanillaN100%100%0.217% SFTN100%98.8%2.35% SFTN − 1100%98.8%0.774% Qwen3-8B 14 VanillaN100%100%2.77% SFTN100%100%1.92% SFTN − 1100%100%0.609% 30 VanillaN100%100%0.241% SFTN100%100%4.22% SFTN − 199.8%100%2.98% Deepseek-R1 Qwen3-8B 14 VanillaN100%98.7%7.5% GRPON100%99.6%6.25% GRPON − 1100%98.6%3.19% 30 VanillaN100%98.9%11.7% GRPON100%99.5%12.5% GRPON − 199.8%97%9.71% Deepseek-R1 Llama3.1-8B 14 VanillaN99.9%98.4%3.94% GRPON100%99.9%1.37% GRPON − 199.9%99.8%0.829% 30 VanillaN100%99.3%10.6% GRPON100%99.6%24.6% GRPON − 1100%100%17.8% Table 4: % of satisfied constraints by model, case, and scenario You are a powergrid operator running Optimal Power Flow simulation. and you need to return a JSON formatted responses based on the provided input JSON. The input is the description of the components of the grid, including the buses, generators, loads, lines, and external grid. The output is the solution to the optimal power flow problem. You will get a few examples of Input and Output json. You need to return the correct Output for the last given Input. "bus": "x": [ [ 345.0, 1.1, 0.9, -93.428, 8.825, 1.0 ], [ 345.0, 1.1, 0.9, -138.405, -25.953, 1.0 ], ... [ 345.0, 1.1, 0.9, 124.925, 48.899, 1.0 ] ] , "load": "x": [...] , "ext_grid": "x": [...] , "gen": "x": [...] , "line": "x": [...] , "('bus', 'to', 'load')": "edge_index": [ [ 4, 6, 8 ], [ 0, 1, 2 ] ] , "('bus', 'to', 'ext_grid')": "edge_index": [ [ 0 ], [ 0 ] ] , "('bus', 'to', 'gen')": "edge_index": [ [ 1, 2 ], [ 0, 1 ] ] , ... system user "res_gen": "0": "p_mw": 138.405, "q_mvar": 25.953 , "1": "p_mw": 97.112, "q_mvar": -2.994 , "res_ext_grid": "0": "p_mw": 93.428, "q_mvar": -8.825 , "res_bus": "0": "vm_pu": 1.0, "va_degree": 0.0 , "1": "vm_pu": 1.066, "va_degree": 0.086 , "2": "vm_pu": 1.048, "va_degree": 0.052 , "3": "vm_pu": 1.007, "va_degree": -0.053 , "4": "vm_pu": 1.006, "va_degree": -0.092 , "5": "vm_pu": 1.051, "va_degree": 0.0 , "6": "vm_pu": 1.041, "va_degree": -0.033 , "7": "vm_pu": 1.054, "va_degree": 0.009 , "8": "vm_pu": 0.995, "va_degree": -0.097 assistant MSE Constraints satisfaction generators buses Power flow equations for every bus Voltage magnitude for every bus Active power for every generator sum % of satisfied constraints Format validity MSE Constraints satisfaction % of satisfied constraints generators bus Figure 5: Prompt used for the OPF task assessment.