Paper deep dive
InsightSR: Refining Symbolic Regression Search Spaces via Parallel Semantic and Structural LLM Guidance
Yating Ling, Wenjing Cun, Zhitang Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/27/2026, 5:06:06 AM
Summary
The paper introduces InsightSR, a framework that integrates Large Language Models (LLMs) with the PySR genetic programming engine to refine symbolic regression search spaces. It employs two parallel pathways: a Semantic Seed Pathway for dimensionally consistent functional skeletons and a Structural Feature Pathway for nonlinear feature transformations. A feedback loop iteratively refines these components, achieving high exact recovery rates on benchmarks like Feynman and LLM-SRBench while maintaining physical consistency and out-of-distribution generalization.
Entities (10)
Relation Signals (9)
InsightSR → achievesperformanceon → Feynman Benchmark
confidence 95% · InsightSR achieves a 95% exact recovery rate on the Feynman benchmark
InsightSR → achievesperformanceon → LLM-SRBench
confidence 95% · 80.18% accuracy on the LLM-SRBench LSR-Transform task
InsightSR → hascomponent → Semantic Seed Pathway
confidence 95% · InsightSR uses LLMs to progressively transform the search space itself through two complementary pathways: a Semantic Seed Pathway...
InsightSR → hascomponent → Structural Feature Pathway
confidence 95% · InsightSR uses LLMs to progressively transform the search space itself through two complementary pathways: ... and a Structural Feature Pathway
InsightSR → uses → Large Language Models
confidence 95% · We present InsightSR, a framework that embeds Large Language Models (LLMs) as a guiding layer
InsightSR → uses → PySR
confidence 95% · We present InsightSR, a framework that embeds Large Language Models (LLMs) as a guiding layer around the PySR genetic programming engine.
InsightSR → outperforms → Genetic Programming
confidence 90% · substantially outperforming state-of-the-art genetic programming and neural-symbolic methods
Semantic Seed Pathway → proposes →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Symbolic regression (SR) seeks to discover parsimonious mathematical laws from observational data, yet conventional approaches often struggle with the vast combinatorial search space of physically meaningful expressions. We present InsightSR, a framework that embeds Large Language Models (LLMs) as a guiding layer around the PySR genetic programming engine. Rather than relying on LLMs to generate expressions directly, InsightSR uses LLMs to progressively transform the search space itself through two complementary pathways: a Semantic Seed Pathway that proposes dimensionally consistent functional skeletons, and a Structural Feature Pathway that recommends nonlinear feature transformations. These transformations accumulate over iterations, broadening the input space and shifting the symbolic search from constructing deep expression trees over raw variables to assembling shallow trees over a rich, semantically informed feature set. A post-generation feedback loop evaluates candidates, categorizes features by their empirical utility, and refines the guidance for the next iteration, transforming the discovery process from open-ended generation into iterative, self-correcting refinement. Across three benchmarks, InsightSR achieves a 95% exact recovery rate on the Feynman benchmark and 80.18% accuracy on the LLM-SRBench LSR-Transform task, substantially outperforming state-of-the-art genetic programming and neural-symbolic methods while maintaining strong out-of-distribution generalization on real-world datasets.
Tags
Links
- Source: https://arxiv.org/abs/2608.25291v1
- Canonical: https://arxiv.org/abs/2608.25291v1
Trouble viewing inline? Open PDF directly →
Full Text
31,455 characters extracted from source content.
Expand or collapse full text
InsightSR: Refining Symbolic Regression Search Spaces via Parallel Semantic and Structural LLM Guidance Yating Ling, Wenjing Cun, Zhitang Chen Abstract Symbolic regression (SR) seeks to discover parsimo- nious mathematical laws from observational data, yet conventional approaches often struggle with the vast combinatorial search space of physically meaningful expressions. We present InsightSR, a framework that embeds Large Language Models (LLMs) as a guiding layer around the PySR genetic programming engine. Rather than relying on LLMs to generate expressions directly, InsightSR uses LLMs to progressively trans- form the search space itself through two complemen- tary pathways: a Semantic Seed Pathway that proposes dimensionally consistent functional skeletons, and a Structural Feature Pathway that recommends nonlin- ear feature transformations. These transformations ac- cumulate over iterations, broadening the input space and shifting the symbolic search from constructing deep expression trees over raw variables to assembling shal- low trees over a rich, semantically informed feature set. A post-generation feedback loop evaluates candi- dates, categorizes features by their empirical utility, and refines the guidance for the next iteration, transform- ing the discovery process from open-ended generation into iterative, self-correcting refinement. Across three benchmarks, InsightSR achieves a 95% exact recovery rate on the Feynman benchmark and 80.18% accuracy on the LLM-SRBench LSR-Transform task, substan- tially outperforming state-of-the-art genetic program- ming and neural-symbolic methods while maintaining strong out-of-distribution generalization on real-world datasets. Introduction Symbolic regression (SR) aims to distill underlying mathe- matical laws directly from observed data, serving as a cor- nerstone for automated scientific discovery (Schmidt and Lipson 2009; Brunton, Proctor, and Kutz 2016). While SR produces interpretable closed-form expressions, its op- timization is inherently NP-hard (Virgolin and Pissis 2022) and the combinatorial search space of candidate expressions grows exponentially with expression complexity. A further challenge is the difficulty of incorporating physical con- straints, such as dimensional consistency and known conser- vation laws, into the search process. As a result, purely data- driven SR methods frequently yield expressions that achieve high numerical accuracy but violate basic physical princi- ples. Existing approaches predominantly rely on stochastic search over the discrete space of mathematical expressions. The most widely used are Genetic Programming (GP) vari- ants (Koza 1994; Cranmer 2023; Virgolin et al. 2021), which evolve populations of expression trees through mutation and crossover, and more recent methods based on Monte Carlo Tree Search (MCTS) (Sun et al. 2022; Shojaee et al. 2023), which frame discovery as a sequential decision-making task. Although these methods can recover approximate symbolic forms, they suffer from low sample efficiency and tend to overfit under observational noise. More critically, without mechanisms to enforce physical constraints, they frequently produce expressions that are mathematically flexible but physically meaningless, lacking the parsimony and domain consistency necessary for scientific interpretation. The integration of neural networks has further expanded the SR toolkit. Deep reinforcement learning methods such as DSR (Petersen et al. 2021) and uDSR (Landajuela et al. 2022) treat equation discovery as a policy optimization prob- lem, using learned gradients to guide the search through ex- pression space. Transformer-based models such as NeSym- ReS (Biggio et al. 2021) and E2E (Kamienny et al. 2022) leverage large-scale pre-training to map data patterns di- rectly to mathematical expressions. Other specialized ap- proaches, such as PhySO (Landajuela et al. 2021), incorpo- rate dimensional constraints into the search to enforce phys- ical consistency. Although these neural methods acceler- ate discovery, they are fundamentally data-driven: their per- formance depends heavily on the training distribution, and they often fail to generalize to out-of-distribution regimes. Moreover, the physical constraints they enforce are typically hard-coded during training rather than dynamically reasoned about, limiting their adaptability to novel physical contexts. Recent work has explored integrating Large Language Models (LLMs) into SR to leverage scientific priors en- coded. LLM-SR (Shojaee et al. 2025a) generates equa- tion skeletons via an LLM and optimizes their parame- ters through BFGS, storing successful patterns in an ex- perience buffer to guide future iterations. SR-LLM (Guo et al. 2025) casts SR as a reinforcement learning prob- lem with an LLM-based policy network, coupled with retrieval-augmented generation for incremental knowledge arXiv:2608.25291v1 [cs.LG] 26 Aug 2026 accumulation. LaSR (Li et al. 2024) focuses on LLM- guided feature engineering, recommending transformations to expand the input space. PiSR via LLM (Taskin, Xie, and Lazebnik 2026) incorporates LLM-derived evaluations into the loss function as a physics-informed regulariza- tion term. While promising, these methods are often con- strained by two limitations. First, they incur substantial in- ference overhead, as they require frequent LLM queries throughout the search process. Second, and more impor- tantly, each method employs LLM guidance along a single dimension—either structural hypothesis generation or input space transformation—without exploiting the potential syn- ergy of integrating both. This separation overlooks a key op- portunity: structural priors can inform which features to en- gineer, and engineered features can simplify the structural forms needed to fit the data. To address these limitations, we propose InsightSR, a framework that embeds LLMs as a guiding layer around the PySR genetic programming engine (Cranmer 2023). Rather than relying on the LLM to generate candidate expressions directly, InsightSR uses the LLM to progressively transform the search space through two complementary pathways. A Semantic Seed Pathway proposes dimensionally consistent functional skeletons that provide a physics-informed warm- start, while a Structural Feature Pathway recommends non- linear feature transformations that accumulate over genera- tions, progressively broadening the input space. This shifts the burden of the search from constructing deep expression trees over raw variables to assembling shallow combinations over a semantically enriched feature set. A learning-from- results feedback loop closes each generation: the LLM eval- uates the Pareto-optimal candidates, categorizes features by their empirical utility, and updates a knowledge base that in- forms both pathways in the subsequent iteration, transform- ing symbolic discovery from open-ended generation into it- erative, self-correcting refinement. Our work makes three key contributions: 1. Semantic Seed Pathway. We introduce a mechanism that leverages domain-aware LLMs to propose dimension- ally consistent functional skeletons that serve as physics- informed seeds for the evolutionary search. By auditing and reconfiguring candidate topologies to align with tar- get physical units, this pathway prunes the search space of physically inconsistent expressions before numerical fitting begins. The LLM also provides per-operator com- plexity biases that steer the genetic programming engine toward physically plausible operator combinations. 2. Structural Feature Pathway. We design a complemen- tary pathway in which the LLM recommends nonlin- ear feature transformations based on the problem con- text and the performance history of prior generations. These transformations accumulate across iterations, pro- gressively broadening the input space. This shifts the evo- lutionary search from constructing deep expression trees over raw variables to assembling shallow combinations over an enriched feature basis, substantially reducing the structural complexity that the symbolic engine must re- solve. Algorithm 1 InsightSR Search Procedure Require: Dataset (X,y), problem contextC, unitsU Ensure: Optimal symbolic expression f ∗ Initialize knowledge baseK←∅ for g = 1 to G do // Dual-Path Heuristic Guidance F seed ← LLM.SEMANTICSEEDS(C,U,K) ▷ Seed Pathway X aug ← LLM.STRUCTURALFEATURES(C,K) ▷ Feature Pathway G ← MergeGuidance(F seed , X aug ) candidates← PYSR.GUIDEDSEARCH(X aug ,y,G) eval, insights← LLM.STRATEGICANALYSIS(candidates) K← KB.ACCUMULATE(insights, eval) if Loss(f ∗ ) < 10 −10 then break end if end for return Best expression f ∗ from candidates 3. Closed-Loop Iterative Refinement. We implement a learning-from-results mechanism that closes the discov- ery loop. After each generation, the LLM evaluates candi- dates along multiple dimensions including numerical ac- curacy, physical interpretability, and feature utility. These assessments are accumulated in a dynamic knowledge base that informs both the semantic and structural path- ways in subsequent iterations, enabling the search to self- correct and progressively converge toward the optimal symbolic form. Method The goal of SR is to discover an analytical expression f : R d → R that accurately describes the relationship in a dataset D = (x i ,y i ) N i=1 . Following the standard GP for- mulation, we treat this as a multi-objective optimization that balances numerical accuracy against structural parsimony: min f∈F L(f,D) = MSE(y,f (X)) + λ·C(f ),(1) whereC(f ) denotes the structural complexity of f (mea- sured by the number of nodes in its expression tree) and λ controls the parsimony penalty. InsightSR organizes the discovery process into a four- phase iterative loop, summarized in Figure 1 and formal- ized in Algorithm 1. Each generation embeds LLM-derived domain knowledge at three intervention points—before, during, and after the evolutionary search—shifting PySR from undirected stochastic exploration toward purposefully guided optimization. We detail each phase below. Contextual Initialization and Unit Synthesis The framework first resolves the physical dimensions of each variable. It extracts available dimensional information from the dataset metadata and problem description. For vari- ables whose units remain ambiguous or unspecified, an LLM infers probable dimensions based on the semantic context of the problem and the physical role of the target variable. The resulting unit assignments persist as dimensional constraints Figure 1: The system architecture of InsightSR. The framework follows a four-phase iterative loop per generation: (1) Contex- tual Initialization, where the LLM performs automated unit synthesis and domain analysis; (2) Parallel Guidance Strategy, which concurrently generates seed expressions (Strategy A) and recommends feature enhancements (Strategy B); (3) Search with PySR, where the integrated heuristics steer the PySR engine; and (4) Strategic Analysis, where the LLM evaluates the top-K candidates and formulates a plan for the next iteration. throughout the discovery loop. By enforcing dimensional homogeneity, the framework prunes physically inconsistent candidates from the search space, ensuring that the symbolic search remains grounded in the underlying physics. Parallel Guidance Strategy The core of InsightSR is a parallel guidance strategy in which the LLM simultaneously contributes along two com- plementary dimensions: top-down semantic seeding and bottom-up feature engineering. Semantic Seed Pathway. This pathway leverages domain-aware LLMs to propose dimensionally consistent functional skeletons that serve as physics-informed seeds for the evolutionary search. Given the problem metadata and the unit constraints resolved in Section 3.1, the LLM generates candidate expression topologies whose dimensional consis- tency is audited before they enter the population. Each skele- ton contains symbolic constants that are subsequently opti- mized numerically. By seeding the population with physi- cally plausible structures, this pathway prunes a vast region of dimensionally inconsistent expressions from the search space before the evolutionary optimization begins. Structural Feature Pathway. In parallel, this pathway enriches the input space through LLM-guided feature engi- neering. At each generation, the LLM analyzes the perfor- mance history and identifies high-utility nonlinear transfor- mations φ(x) based on patterns observed in surviving candi- dates. These transformations are appended to the input ma- trix: X aug = [x 1 ,..., x d ,φ 1 (x),...,φ k (x)],(2) where φ j (x) includes operations such as power-law terms x n i and transcendental mappings sin(x i ). Crucially, these features accumulate across generations, progressively broadening the input space. This shifts the search from con- structing deep expression trees to discover complex nonlin- ear relationships to assembling shallow combinations over an already-enriched feature set, substantially reducing the structural depth that the evolutionary engine must resolve. The two pathways converge within the PySR engine. Rather than searching the unconstrained space F over raw inputs x, the optimizer operates over an informed manifold: f ∗ = arg min f∈F seed L(f (X aug ),D).(3) Semantic seeds provide macro-level structural constraints grounded in physics, while engineered features supply micro-level building blocks derived from empirical patterns. Together, they confine the search to a physically plausible region of the expression space without sacrificing the nu- merical flexibility needed for high-precision fitting. Search with PySR The synthesized guidance consists of three components: symbolic seeds, operator preferences, and the augmented feature matrix X aug . These are passed to a modified PySR engine for evolutionary search. To steer the search to- ward LLM-informed functional structures, we introduce a complexity-biasing mechanism that adjusts the structural penalty of each expression tree. The adjusted complexity C ′ (f ) is computed as the sum of baseline operator costs c o modified by an LLM-provided bias ω o : C ′ (f ) = X o∈f (c o + ω o ).(4) Operators recommended by the LLM receive a negative bias ω o , granting them a complexity discount that promotes their survival on the Pareto front. Conversely, operators deemed physically implausible are penalized with a positive bias, in- creasing their effective cost and suppressing their propaga- tion across generations. In addition, we employ a warm-start mechanism that retains the best candidate from the previous generation as an elite member of the new population. This ensures that prior refinements are preserved and built upon, enabling progressive convergence toward the optimal sym- bolic form. Multi-Dimensional Evaluation and Strategic Feedback After each evolutionary search completes, the LLM serves as a post-generation critic to evaluate the Pareto-optimal candidates and guide the next iteration. The evaluation syn- thesizes multiple criteria: numerical accuracy, physical in- terpretability, and the completeness with which the expres- sion incorporates the relevant variables. In parallel, the sys- tem performs a diagnostic analysis of the feature space, cat- egorizing each engineered transformation as effective or in- effective based on its contribution to reducing the residual error. These assessments are stored in a dynamic knowledge base, enabling the framework to learn from both successful patterns and failure modes across generations. The knowl- edge base then informs the next iteration in three ways: it refines the seed expressions proposed by the Semantic Seed Pathway, adjusts the operator complexity biases used by the PySR engine, and prioritizes feature transformations for the Structural Feature Pathway. By closing the loop between empirical search and semantic reasoning, the framework progressively narrows the search toward physically mean- ingful expressions until convergence. Experimental Setup Benchmarks We evaluate on three benchmarks summarized in Table 1. The Feynman 100 benchmark (Udrescu and Tegmark 2020) consists of 100 physics equations derived from the Feynman Lectures on Physics, covering classical mechan- ics, electromagnetism, quantum mechanics, and thermody- namics. Each problem has a known ground-truth expression, enabling exact recovery evaluation. The LLM-SRBench (Shojaee et al. 2025b) comprises five categories: LSR-Transform and LSR-Synth across Physics, Chemistry, Biology, and Material Science. Each category features different data-generating mechanisms and varying complexity, assessing cross-domain generalization. The Real-World benchmark (Shojaee et al. 2025a) in- cludes four real-world datasets (Oscillator 1, Oscillator 2, E. coli, Stress-Strain). Each dataset provides both in- distribution (ID) and out-of-distribution (OOD) test splits, enabling evaluation of out-of-distribution generalization. Implementation Details Table 2 lists the key hyperparameters used in our experi- ments. Evaluation Metrics The Coefficient of Determination (R 2 ) measures the propor- tion of variance captured by the model, defined as: R 2 = 1− P n i=1 (y i − ˆy i ) 2 P n i=1 (y i − ̄y) 2 (5) where y i represents the ground truth, ˆy i is the predicted value, and ̄y is the empirical mean. Consistent with our op- timization objective, the Normalized Mean Squared Error (NMSE) is defined as NMSE = 1− R 2 . The Accuracy under Tolerance (acc τ ) is defined as a bi- nary success metric for each problem. A discovery is consid- ered successful (1) only if the maximum relative error across all n test samples is within the threshold τ ; otherwise, it is marked as failure (0): acc τ = I max 1≤i≤n y i − ˆy i y i ≤ τ (6) where I(·) is the indicator function. The Exact Recovery Rate is defined as a special case where the symbolic form is mathematically equivalent to the ground truth, typically resulting in acc τ → 1 for τ → 0. Results Feynman Benchmark Figure 2 summarizes the performance on the Feynman benchmark. InsightSR achieves a 95% exact recovery rate and an average R 2 of 0.9999, outperforming the previous state of the art. Figure 3 shows the Pareto front of loss ver- sus expression complexity. The median loss declines consis- tently as complexity increases, and the narrowing interquar- tile range at higher complexity levels indicates enhanced numerical stability. We further evaluated robustness to data perturbations. As shown in Table 3, accuracy remains high under moderate noise, with Acc 0.1 reaching 79% at a noise level of 0.001. LLM-SRBench We further evaluate our method on the LLM-SRBench across five scientific domains. As summarized in Ta- ble 4, our method achieves a substantial gain in the LSR- Transform category, reaching an Acc 0.1 of 80.18% com- pared to the previous best of 50.45% (LaSR). Table 1: Benchmark datasets overview BenchmarkProblemsExample Feynman (Udrescu and Tegmark 2020)100 m 0 √ 1−v 2 /c 2 LLM-SRBench (Shojaee et al. 2025b)240−kA(t) 2 + k z A(t) 2 βA(t) 4 +1 Real-World (Shojaee et al. 2025a)40.8 sin(x)− 0.5v 3 − 0.2x 3 − 0.5xv− x cos(x) Table 2: Hyperparameters ParameterValue LLMQwen-3.5 27B LLM Temperature0.3 PySR Iterations100–500 LLM Generations3–30 Populations30 Population Size50 Parsimony0.003 Max Expression Size20–25 Early Stop Threshold10 −10 Loss FunctionMSE NGGP GP-GOMEA gplearn PySR QLattice PSRN E2E PhySO SR-LLM Ours 0 20 40 60 80 100 Exact Recovery Rate (%) Exact Recovery Rate Avg R 2 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 Avg R 2 Figure 2: Feynman benchmark: exact recovery rate (bars) with average R 2 overlay (diamonds). Performance is consistent across the four domain-specific synthetic datasets. Specifically, our method achieves an ac- curacy of 66.67% in Chemistry, 50.00% in Biology, 40.91% in Physics, and 92.00% in Material Science. In terms of nu- merical precision, our method yields NMSE values that are several orders of magnitude lower than other LLM-based solvers. These results are further illustrated in Figure 4, which shows the relative performance across all methods. Real-World Datasets Table 5 compares the NMSE results of our method against several baselines on four real-world datasets, covering both in-distribution (ID) and out-of-distribution (OOD) scenar- ios. Overall, our method obtains the lowest NMSE in most test cases. On the Oscillator 1 and Oscillator 2 datasets, our method achieves ID NMSE values of 9.55× 10 −11 and 2.45× 10 −9 , respectively, representing an improvement of several orders of magnitude over the previous best LLM-based method (LLM-SR). For the E. coli dataset, our method yields an ID NMSE of 1.76× 10 −3 and an OOD NMSE of 1.41× 10 −2 . In the Stress-Strain task, our results are 2.00×10 −2 (ID) and Figure 3: Loss vs. expression complexity Pareto front. The solid line shows the median; shaded region shows the 25th– 75th percentile. Table 3: Noise vs Accuracy Performance noiseAcc 0.1 (%) 0.0537 0.0168 0.00572 0.00179 097 5.32× 10 −2 (OOD), which are comparable to or better than LLM-SR. These results are further illustrated in Figure 5. Qualitative Case Study To provide an intuitive understanding of our method’s per- formance, we compare the ground-truth equations against the discovered expressions across all three benchmarks. Ta- ble 6 highlights how our method recovers the core functional forms, with matching terms bolded in the discovered col- umn. Discussion The experimental results across three benchmarks demon- strate that InsightSR addresses two central challenges in symbolic regression: the combinatorial explosion of the search space and the difficulty of maintaining physical con- sistency under purely data-driven optimization. The dual- pathway design contributes along both dimensions. The Se- mantic Seed Pathway provides a physics-informed warm- start, which is reflected in the 95% exact recovery rate on Table 4: LLM-SRBench results. Method LSR-TransformChemistryBiologyPhysicsMaterial Sci. Acc 0.1 ↑ NMSE↓ Acc 0.1 ↑ NMSE↓ Acc 0.1 ↑ NMSE↓ Acc 0.1 ↑ NMSE↓ Acc 0.1 ↑ NMSE↓ Direct Prompting6.310.26313.882.2e-24.160.4659.090.0650.000.048 SGA8.110.23216.665.5e-412.510.0139.090.05136.116.0e-4 LaSR50.450.00138.929.1e-520.831.5e-431.819.9e-472.049.2e-6 LLM-SR39.640.00952.774.1e-629.163.1e-636.367.6e-588.283.2e-9 Ours80.181.3e-1466.678.0e-750.007.3e-740.911.6e-592.004.7e-8 Table 5: Real-world datasets results. ID: in-distribution, OOD: out-of-distribution. Method Oscillator 1Oscillator 2E. coliStress-Strain IDOODIDOODIDOODIDOOD GPlearn1.55e-025.57e-017.55e-013.19e+001.08e+001.04e+001.06e-014.09e-01 NeSymReS4.70e-035.38e-012.49e-016.47e-01—1.45e+007.93e-016.38e-01 E2E8.20e-033.72e-011.40e-011.91e-016.32e-011.45e+002.26e-015.87e-01 DSR8.70e-032.45e-015.80e-021.95e-019.45e-012.43e+003.33e-011.11e+00 uDSR3.00e-047.00e-043.20e-031.50e-033.32e-015.46e+005.02e-021.76e-01 PySR9.00e-043.11e-012.00e-049.80e-033.76e-021.01e+003.31e-021.30e-01 LLM-SR4.65e-075.00e-042.12e-073.81e-052.14e-022.64e-022.10e-025.16e-02 Ours9.55e-111.15e-042.45e-091.03e-051.76e-031.41e-022.00e-025.32e-02 020406080100 acc_0.1 (%) LSR-Transform Chemistry Biology Physics Material Sci. 6.3 13.9 4.2 9.1 8.1 16.7 12.5 9.1 36.1 50.5 38.9 20.8 31.8 72.0 39.6 52.8 29.2 36.4 88.3 80.2 66.7 50.0 40.9 92.0 Direct Prompting SGA LaSR LLM-SR Ours Figure 4: LLM-SRBench Acc 0.1 comparison across five cat- egories. the Feynman benchmark: by seeding the population with di- mensionally consistent skeletons, the search bypasses vast regions of physically meaningless expressions that would otherwise dominate the early generations of a standard GP run. The Structural Feature Pathway addresses the depth bottleneck inherent in GP-based SR. By accumulating LLM-recommended transformations across generations, In- sightSR progressively broadens the input space, shifting the search from constructing deep expression trees to assem- bling shallow combinations over an enriched feature ba- sis. This reduction in required structural depth is directly reflected in the 80.18% accuracy on the LSR-Transform task, where target expressions exhibit complex functional forms that would otherwise require prohibitively deep trees to discover. The two pathways operate synergistically: se- mantic constraints narrow the space of candidate structures, while engineered features simplify the structural complexity GPlearn NeSymReS E2E DSR uDSR PySR LLM-SR Ours 10 9 10 7 10 5 10 3 10 1 NMSE (log scale) ID OOD GPlearn NeSymReS E2E DSR uDSR PySR LLM-SR Ours 10 8 10 6 10 4 10 2 10 0 NMSE (log scale) ID OOD GPlearn NeSymReS E2E DSR uDSR PySR LLM-SR Ours 10 3 10 2 10 1 10 0 10 1 10 2 10 3 NMSE (log scale) ID OOD GPlearn NeSymReS E2E DSR uDSR PySR LLM-SR Ours 10 1 10 0 NMSE (log scale) ID OOD Figure 5: LLM-SR benchmark NMSE comparison. needed to fit the data, together enabling the PySR engine to operate far more effectively than in its standalone configura- tion. The performance on real-world datasets, such as the Os- cillator and E. coli benchmarks, further highlights the frame- work’s capacity for out-of-distribution generalization. Un- like purely neural symbolic methods that may overfit to spe- cific training distributions, the learning-from-results feed- back mechanism in InsightSR helps the evolved expres- sions remain parsimonious and physically interpretable. The complexity-biasing mechanism, informed by the LLM’s strategic analysis, prioritizes operators that align with es- tablished domain knowledge. This adaptive regularization is crucial for scientific discovery, where the ultimate objective is to identify governing principles that remain valid beyond the observed data range. Table 6: Ground truth vs. discovered equations. Matching terms are bolded. BenchmarkProblemGround TruthDiscovered Feynman I.6.2a e −θ 2 /2 √ 2π 0.3989· e −θ 2 /2 I.8.14 p (x 2 − x 1 ) 2 + (y 2 − y 1 ) 2 p (x 1 − x 2 ) 2 + (y 1 − y 2 ) 2 LLM-SRBench I.6.15b 8πE f εr 3 3 sin(2θ) 4π 3 · E f εr 3 sinθ cosθ I.48.2 −c p 1− c 4 m 2 /E 2 n −c p 1− m 2 c 4 /E 2 n Real-World Oscillator 10.8 sinx− 0.5v 3 −0.2x 3 − 0.5xv− x cosx −0.5(v 3 + xv) +sin x(−0.275cos x + 0.075) Oscillator 20.3 sint− 0.5v 3 −xv− 5xe 0.5x 0.3sin t− 0.5v 3 −xv + e x (x 6 − 5) + const While the integration of LLM guidance enhances search efficiency, the framework’s reliance on initial metadata sug- gests a potential sensitivity to problem descriptions. In sce- narios where variable semantics are ambiguous, the accu- racy of the unit synthesis module becomes a critical factor. The current “learning-from-results” loop remains computa- tionally viable by strategically querying the LLM between generations rather than at every search step. Future research may investigate the use of multi-modal priors to further re- fine the initialization phase and enhance the framework’s ro- bustness in the absence of explicit textual metadata. Conclusion This paper presented InsightSR, a framework that embeds Large Language Models as a guiding layer around the PySR genetic programming engine. Through two complementary pathways, semantic seeding and structural feature engi- neering, the framework progressively transforms the search space, shifting symbolic regression from deep tree construc- tion over raw variables to shallow assembly over an en- riched feature basis. A closed-loop feedback mechanism enables the system to learn from search outcomes and re- fine its guidance across generations. Extensive evaluations demonstrate that InsightSR achieves state-of-the-art accu- racy across three benchmarks while maintaining strong out- of-distribution generalization on real-world datasets. These results suggest that strategic integration of LLM reasoning with established evolutionary search engines offers a practi- cal and effective path toward automated scientific discovery. References Biggio, L.; Bendinelli, T.; Neitz, A.; Lucchi, A.; and Paras- candolo, G. 2021. Neural symbolic regression that scales. In International Conference on Machine Learning, 936–945. Brunton, S. L.; Proctor, J. L.; and Kutz, J. N. 2016. Discov- ering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences 113(15):3932–3937. Cranmer, M. 2023. Pysr: Fast & parallel symbolic re- gression in python/julia. Journal of Open Source Software 8(86):5319. Guo, Z.; Wang, S.; Tian, Y.; Yang, J.; Yu, H.; Na, X.; Kov ́ acs, L.; Li, L.; Ioannou, P. A.; and Wang, F.-Y. 2025. Sr-llm: An incremental symbolic regression framework driven by llm- based retrieval-augmented generation. Proceedings of the National Academy of Sciences 122(52):e2516995122. Kamienny, P.-A.; d’Ascoli, S.; Lample, G.; and Charton, F. 2022. End-to-end symbolic regression with transformers. In Advances in Neural Information Processing Systems, vol- ume 35, 10228–10240. Koza, J. R. 1994. Genetic Programming: On the Program- ming of Computers by Means of Natural Selection. MIT Press. Landajuela, M.; Petersen, B. K.; Kim, S.; Santiago, C. P.; Glatt, R.; Mundhenk, T. N.; Pettit, J. F.; and Faissol, D. M. 2021. Discovering symbolic models from deep learning with inductive biases. In Advances in Neural Information Pro- cessing Systems, volume 34, 13521–13533. Landajuela, M.; Lee, C. S.; Yang, J.; Glatt, R.; Santiago, C. P.; Aravena, I.; Mundhenk, T.; Mulcahy, G.; and Petersen, B. K. 2022. A unified framework for deep symbolic regres- sion. Advances in Neural Information Processing Systems 35:33985–33998. Li, K.; Zhang, C.; Ling, Z.; and Zhang, Y. 2024. Lasr: Llm- guided feature engineering for symbolic regression. arXiv preprint arXiv:2410.07262. Petersen, B. K.; Landajuela, M.; Mundhenk, T. N.; Santi- ago, C. P.; Kim, S.; and Kim, J. T. 2021. Deep symbolic regression. In Advances in Neural Information Processing Systems, volume 34, 13895–13907. Schmidt, M., and Lipson, H. 2009. Distilling free-form natural laws from experimental data. science 324(5923):81– 85. Shojaee, P.; Meidani, K.; Barati Farimani, A.; and Reddy, C. 2023. Transformer-based planning for symbolic regres- sion. Advances in Neural Information Processing Systems 36:45907–45919. Shojaee, P.; Meidani, K.; Gupta, S.; Farimani, A. B.; and Reddy, C. K. 2025a. Llm-sr: Scientific equation discovery via programming with large language models. In Interna- tional Conference on Learning Representations. Shojaee, P.; Nguyen, N.-H.; Meidani, K.; Farimani, A. B.; Doan, K. D.; and Reddy, C. K. 2025b. Llm-srbench: A new benchmark for scientific equation discovery with large language models. arXiv preprint arXiv:2504.10415. Sun, F.; Liu, Y.; Wang, J.-X.; and Sun, H. 2022. Symbolic physics learner: Discovering governing equations via monte carlo tree search. arXiv preprint arXiv:2205.13134. Taskin, B.; Xie, W.; and Lazebnik, T. 2026. Knowledge inte- gration for physics-informed symbolic regression using pre- trained large language models. Scientific Reports 16:1614. Udrescu, S.-M., and Tegmark, M. 2020. Ai feynman: A physics-inspired method for symbolic regression. Science Advances 6(16):eaay2631. Virgolin, M., and Pissis, S. P. 2022. Symbolic regression is np-hard. arXiv preprint arXiv:2207.01018. Virgolin, M.; Alderliesten, T.; Witteveen, C.; and Bosman, P. A. 2021. Improving model-based genetic programming for symbolic regression of small expressions. Evolutionary computation 29(2):211–237.