Paper deep dive

LIGO-PINN: Learned Initialization via Gated Optimization to Alleviate Convergence Failures in Physics Informed Neural Networks

Nilay Anurag, Shital Adhikari, Taniya Kapoor, Nikhil Muralidhar

Year: 2026Venue: arXiv preprintArea: cs.LGType: PreprintEmbeddings: 93

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%

Last extracted: 7/17/2026, 4:33:32 AM

Summary

The paper introduces LIGO-PINN, a framework that uses learned weight initialization via gated layerwise optimization to alleviate catastrophic convergence failures in Physics-Informed Neural Networks (PINNs) when solving challenging partial differential equations (PDEs). By combining invariance encoding from easy PDE tasks with a progressive gating mechanism, LIGO-PINN stabilizes training dynamics and significantly outperforms state-of-the-art baselines across 1D, 2D, and 3D PDE domains.

Entities (9)

Partial Differential Equations · mathematical-concept · 99%Physics-Informed Neural Networks · model · 99%LIGO-PINN · method · 98%Gated Layer-wise Optimization · technique · 95%Invariance Encoding · technique · 95%1D Convection · pde-domain · 92%2D Helmholtz · pde-domain · 92%2D Incompressible Navier-Stokes · pde-domain · 92%Meta-learning · algorithm · 90%

Relation Signals (8)

Physics-Informed Neural Networks → solves → Partial Differential Equations

confidence 98% · Physics-informed neural networks (PINNs) have had a broad research impact in modeling domains governed by partial differential equations (PDE).

LIGO-PINN → alleviates → PINN Convergence Failures

confidence 95% · LIGO-PINN is proposed to overcome PINN convergence failures and alleviate catastrophic training dynamics.

LIGO-PINN → uses → Gated Layer-wise Optimization

confidence 95% · we augment the IE procedure with the novel GLO mechanism by introducing a gated update of layers during each meta-update

LIGO-PINN → uses → Invariance Encoding

confidence 95% · Our proposed LIGO-PINN method, has two facets (i) Invariance Encoding (ii) Gated Layer-wise Optimization.

LIGO-PINN → evaluatedon → 1D Convection

confidence 92% · We evaluate our proposed LIGO-PINN model on three diverse PDE domains... 1D Convection.

LIGO-PINN → evaluatedon → 2D Helmholtz

confidence 92% · We evaluate our proposed LIGO-PINN model on three diverse PDE domains... 2D Helmholtz.

LIGO-PINN → evaluatedon → 2D Incompressible Navier-Stokes

confidence 92% · We finally consider modeling the challenging 2D viscous flow over a cylinder, using the non-dimensional 2D incompressible Navier–Stokes equations

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Physics-informed neural networks (PINNs) have had a broad research impact in modeling domains governed by partial differential equations (PDE). However, PINNs have been shown to perform poorly, sometimes even converging to trivial solutions, in challenging PDE domains, or when generalizing to unseen but related PDE domains. Previously proposed solutions detail hyperparameter tuning to reduce loss imbalance between data-driven and physics guided losses, curriculum learning based training strategies, or dynamic re-sampling of hard collocation points. These methods face certain pitfalls: hyperparameter tuning is expensive, designing a training curriculum is ambiguous in multi-parameter PDE settings, and dynamic resampling still fails in complex PDE settings. Complementary to this line of thinking, we believe the initial PINN network weights also play a crucial role in the emergence of catastrophic failures during training, yet the effect of PINN weight initialization has been surprisingly under-investigated. To this end, we propose a framework for Learned Initialization via Gated Layerwise Optimization (LIGO-PINN) to overcome PINN convergence failures. Through rigorous evaluation on 1D and 2D PDE domains, including a challenging 2D fluid dynamics setting, we demonstrate that our methodology outperforms state-of-the-art methods designed to alleviate PINN failures, achieving a 91.5% average performance improvement across six baselines and 81% over the strongest baseline. We also verify that LIGO-PINN generalizes to 3D unstructured domains. Finally, we analyze training dynamics across all three PDE domains to explain both LIGO-PINN's improvement and the convergence failure of traditional PINNs. Code: this https URL Keywords: Machine Learning, Physics-Informed Neural Networks, Deep Learning, PDE Modeling

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

91,506 characters extracted from source content.

Expand or collapse full text

LIGO-PINN: Learned Initialization via Gated Optimization to Alleviate Convergence Failures in Physics Informed Neural Networks Nilay Anurag 1 Shital Adhikari 1,* Taniya Kapoor 2 Nikhil Muralidhar 1,* 1 Department of Computer Science, Stevens Institute of Technology 2 Artificial Intelligence Group, Wageningen University & Research, nanurag, sadhikar1, nmurali1@stevens.edu, taniya.kapoor@wur.nl Abstract Physics-informed neural networks (PINNs) have had a broad research impact in modeling domains governed by partial differential equations (PDE). However, PINNs have been shown to perform poorly, sometimes even converging to trivial solutions, in challenging PDE domains, or when generalizing to unseen but related PDE domains. Previously proposed solutions to alleviate this well known PINN failure mode, detail hyperparameter tuning to reduce loss imbalance between data- driven and physics guided losses, curriculum learning based training strategies or ones involving dynamic re-sampling of hard collocation points. We observe that these methods face certain pitfalls e.g., hyperparameter tuning is expensive, designing a training curriculum is ambiguous in multi-parameter PDE settings and dynamic resampling of collocation points still fails in complex PDE settings. Com- plementary to this line of thinking, we believe the initial PINN network weights also play a crucial role in the emergence of catastrophic failures during training. However, the effect of PINN weight initialization has been surprisingly under- investigated. To this end, we propose a novel PINN training methodology based on Learned Initialization, to address catastrophic PINN failures. Specifically, we propose a framework for Learned Initialization via Gated Layerwise Optimization (LIGO-PINN) to overcome PINN convergence failures. Through rigorous eval- uation on 1D and 2D PDE domains, including a challenging 2D fluid dynamics setting, we demonstrate that our proposed methodology outperforms state-of-the- art methods, including those specifically designed to alleviate PINN failures. We show that LIGO-PINN achieves a 91.5% average performance improvement across six state-of-the-art PINN baselines and 81% performance improvement over the strongest baseline, when evaluated across three diverse PDE domains. We also verify that LIGO-PINN generalizes to 3D unstructured domains. Finally, we con- duct a rigorous analysis to identify and explain the performance improvement of LIGO-PINN and the convergence failure of traditional PINNs, by analyzing training dynamics across all three PDE domains. Our code is publicly available at https://github.com/scailab/ligo-pinn. 1 Introduction Partial differential equations (PDEs) govern dynamics in heat transfer, fluid flow, electromagnetics, and other physical systems. Efficient PDE solvers are central to scientific progress. Physics- Informed Neural Networks (PINNs) [Raissi et al., 2017b, 2018] embed PDE constraints into neural network loss functions, enabling both forward and inverse problem solving. PINNs have been *Corresponding Authors: sadhikar1@stevens.edu, nmurali1@stevens.edu Preprint. arXiv:2607.14233v1 [cs.LG] 15 Jul 2026 applied widely [Cuomo et al., 2022b], but often fail in hard PDE settings, converging to trivial or inaccurate solutions. A key challenge is the imbalance between data-driven and physics-informed loss terms [Wang et al., 2021a], which causes optimization pathologies. Remedies include curriculum learning [Krishnapriyan et al., 2021], adaptive sampling [Daw et al., 2023], and spectral/Neural Tangent Kernel (NTK) analyses [Wang et al., 2021b, 2022b, Lau et al., 2024]. Yet these approaches typically succeed only in narrow PDE regimes and lack generality. Meanwhile, the machine learning community has shown that initial weights strongly influence trainability in deep networks [Arpit et al., 2019, Schoenholz et al., 2017, Taki, 2017, Yang and Schoenholz, 2017]. Standard weight initialization schemes such as Xavier or Kaiming can place networks in chaotic or overly ordered regimes, leading to poor convergence. Despite this, nearly all PINN studies rely on off-the-shelf initializers without task-specific adaptation. To address this gap, we introduce Learned-Initialization via Gated Layerwise Optimization for PINNs (LIGO-PINN), a framework to systematically initialize PINN network weights that alleviates catastrophic training dynamics. Our contributions are as follows: • We propose a method to systematically learn the initial PINN weights, to mitigate catas- trophic training failures, commonly observed in PINNs, particularly in challenging PDE regimes. •As part of the learned initialization method, we introduce a novel gated layer-wise optimiza- tion (GLO) procedure and show that it produces initial PINN weights that mitigate spectral bias during PINN training. •Through extensive experiments on three diverse PDE domains, we demonstrate consistent improvements over state-of-the-art baselines, especially in extrapolation settings. 2 Related Work We now review major research directions investigating PINN training failures [Cuomo et al., 2022a]. Temporal Dynamics, Sampling, and Curriculum-Based Methods. A seminal work [Wang et al., 2021a] uncovered the imbalance in gradients between data-driven and physics-based loss terms as one of the reasons for catastrophic PINN failures, causing converge to trivial solutions. Early work also highlighted the importance of respecting causal order of PDE dynamics during training, in time-dependent problems [Wang et al., 2022a]. Building on this, residual-based sampling techniques dynamically reweigh collocation points in regions with high PDE residuals [Daw et al., 2023, Toloubidokhti et al., 2023]. In parallel, curriculum learning approaches schedule training tasks to be ordered by increased difficulty, guiding PINNs from simpler to complex PDE regimes [Krishnapriyan et al., 2021]. While these strategies reduce domain errors, they require careful tuning when PDE parameters vary widely and do not generalize well across different PDE systems. Gradient-Level Interventions Efforts to address the issue of loss imbalance in PINNs have also explored gradient-level solutions. Kim et al. [2021] introduced a Dynamic Pulling Method (DPM) which employs a pseudo-inverse operation to align losses in the same direction, to improve stability, effectiveness of training. Wang et al. [2025] investigated effect of higher-order optimizers. Meta-Learning for PINNs. Several recent works have explored meta learning concepts in PDE systems. Psaros et al. [Psaros et al., 2022] uses meta learning to discover optimal loss function for different PINN system. In contrast LIGO-PINN uses meta learning to inform PINN weight initialization. Hyper-LR-PINNs [Cho et al., 2023] employ a hyper-network architecture to generate low-rank, task-specific weights for each PDE domain. Although they have trained on similar PDE domains to ours, their experiments have not explored extrapolation regimes as extreme as ours in this paper. Qin et al. [2022] is the closest paper to ours and also employs a model-agnostic meta- learning (MAML) approach for PINN training. However, their evaluation is restricted to simple PDE configurations and unlike our setup, they do not investigate gated optimization strategies or generalization to unseen PDE configurations. In contrast to other approaches to alleviate catastrophic training failures in PINN, LIGO-PINN is the only approach to systematically explore the effect of learned weight initialization on PINN training dynamics. Further, our gated layer-wise optimization is the first approach of its kind applied in the context of addressing convergence failures in PINNs across diverse PDE settings. 2 3 Problem Formulation We follow the standard PINN setting as in [Raissi et al., 2017a], also described briefly below; See Appendix A for detailed formulation. Letz∈ Ω⊂R d denote the spatial coordinate within domain Ω (with boundary ∂Ω), t∈ [0,T ] time, and u(z,t) the physical state solving the PDE in Eq. 1. F u(z,t);γ = f (z,t),z∈ Ω, t∈ [0,T ], B u(z,t) = u b (z,t),z∈ ∂Ω,(1) Here,Fis the (possibly nonlinear) differential operator representing the underlying physical law,γ are the PDE parameters (e.g., viscosity, diffusivity), andf (z,t)is a known forcing or source term. The boundary operatorBenforces boundary or initial conditions, andu b (z,t)specifies the prescribed values along ∂Ω and the subscript ‘b’ is added to indicate solution along the boundary ∂Ω. A neural networkˆu Θ (z,t)parameterized by weightsΘis trained to satisfy the PDE at interior (collocation) points and to match boundary/initial data on∂Ω. WithN Ω interior samples(z i ,t i )and N ∂Ω boundary/initial samples (z j ,t j ), the objective is listed in Eq. 2. L(Θ) = λ r 1 N Ω N Ω X i=1 F (ˆu Θ (z i ,t i );γ)− f (z i ,t i ) 2 + 1 N ∂Ω N ∂Ω X j=1 ˆu Θ (z j ,t j )− u b (z j ,t j ) 2 .(2) Here, λ r balances the contributions of the physics residual and boundary losses. Previous work has tried to alleviate PINN training failures by hyper-parameter tuning ofλ r Wang et al. [2021a], learning improved training curricula Krishnapriyan et al. [2021] or by intelligently re-sampling collocation points to focus on hard regions of the domain Daw et al. [2023], Wu et al. [2023] but has all seen limited success as our investigations of these methods in Sec. 5 reveal. Complementary to these works, we hypothesize that PINN convergence failures in challenging (a.k.a. hard) PDE domains, can be alleviated, if PINN weightsΘ, instead of being randomly initialized, are systematically initialized prior to training on the hard domain. To this end, we propose a Learned- Initialization mechanism to systematically learn better initial weights, such that PINNs trained with these learned initial weights overcome catastrophic training failures. Our proposed LIGO-PINN method, has two facets (i) Invariance Encoding (i) Gated Layer-wise Optimization. Algorithm 1: Physics Informed Neural Network with Learned Initialization via Gated Optimization Input: Tasks p(Γ easy ), initial weights Θ 0 , inner LR η inner , outer LR η outer , total meta-iterationsR, Num. Sampled Tasks k Output: Learned Initial Weights Θ R for r ← 0 do // meta-iterations r = 1,...,R Sample batchT i k i=1 ∼ p(Γ easy ) Copy Global WeightsΘ ′ r,i k i=1 := Θ r for i = 1 to k do Θ ′ r,i ← Θ ′ r,i − η inner ∇ Θ ′ r,i L T i (Θ ′ r,i ) Compute Meta Gradients [∇ θ 0 r L meta ,...,∇ θ L−1 r L meta ] for l = 0 to L− 1 do g l (r) = 1l≤⌊ rL R ⌋ θ l r+1 ← θ l r − η outer g l (r)∇ θ l r L meta Invariance Encoding (IE). If the goal of the PINN is to learn a challeng- ing PDE domain governed by PDE pa- rameterγ hard , IE seeks to first train a randomly initialized PINN model on a set of relatively easier configurations of the PDE with parametersΓ easy = γ 1 ,...,γ K . This enables a distil- lation of the invariant physics across Γ easy into the PINN weightsΘ. Our approach for IE is grounded in foun- dational works of meta-learning [Finn et al., 2017]. IE begins with randomly initialized global PINN weightsΘ 0 and a pool ofKtasks inΓ easy . The IE learn- ing process is carried out forRitera- tions1, 2,...,Rand at each train- ing iterationr, tasksT 1 ,...,T k are randomly sampled from the task distributionp(Γ easy )of easy tasks andkidentical copies Θ ′ r,1 ,..., Θ ′ r,k of the current global PINN weightsΘ r are created. EachΘ ′ r,i is then optimized via. gradient descent, employing data from taskT i at iterationrfor ‘J’ inner iterations (for simplicity we assume ‘J’ = 1). LetΘ ′ r,i represent the updated version ofΘ ′ r,i after ‘J’ inner gradient-descent update iterations. The common representations across allktask modelsΘ ′ r,1 ,..., Θ ′ r,k is distilled into Θ r via. a ‘meta-update’ as indicated in Eq. 3 whereL T i (Θ ′ r,i )represents the loss in Eq. 2 calculated w.r.t parameters Θ ′ r,i with data from taskT i and η outer represents the learning rate. Θ r+1 = Θ r − η outer ∇ Θ r L meta ;∇ Θ r L meta =∇ Θ r k X i=1 L T i (Θ ′ r,i )(3) 3 Gated Layer-wise Optimization (GLO). Traditional meta-learning approaches [Finn et al., 2017, Rajeswaran et al., 2019] are not physics-informed and treat all tasks inΓ easy as resulting in similar training dynamics across all layers. Thus, the meta-update in Eq. 3 is a simple summation of all task gradients. However, based on previous work [Wang et al., 2021a], we know that non-trivial dynamics between the physics-based loss and data-driven loss affects different layers in the neural network differently. To address these gaps that manifest uniquely in the physics-informed PDE modeling context, we augment the IE procedure with the novel GLO mechanism by introducing a gated update of layers during each meta-update, such that a gating mechanismg(r), unlocks deeper layers, exposing their parameters to gradient-based optimization, gradually, over the training iterations from 1 toR. Let us consider that the global PINN is an ‘L’ layer neural network whose parameters at iterationr are represented asΘ r =θ 0 r ,...,θ L−1 r withθ i r representing the weights corresponding to layeri at iterationr. GLO allows us to prioritize learning shallow layers prior to deeper layers providing fine-grained (layer level) control over training dynamics in contrast to traditional IE approaches. Specifically, at each iterationr ∈ 1, 2,...,R, the (binary) gating value for layerlis calculated with a gating mechanism detailed in Eq. 4. g l (r) = 1, l ≤ rL R , 0, otherwise, l = 0,...,L− 1.(4) The gating schedule begins with the first layer unmasked, followed by linear unmasking of deeper layers as training progresses, ensuring localized (and more stable) effects of non-trivial loss dynamics. Although the proposedg(r)∈0, 1 L , results in a linear gating schedule, GLO can easily admit any general gating mechanism. θ l r+1 = θ l r − η outer g l (r)∇ θ l r L meta , l = 0,...,L− 1,(5) Ifg(r) = [g 0 (r), ..., g L−1 (r)]and each meta-update gradient vector is represented as∇ Θ r L meta = [∇ θ 0 r L meta , ..., ∇ θ L−1 r L meta ], then the meta-update at iterationr + 1for layerlis detailed in Eq. 5. Here, eachg l (r)controls the meta-update strength of layerl. Alg. 1 details the entire procedure of our proposed LIGO-PINN framework. The learned weightsΘ R resulting from Alg. 1, can be employed as initial weights for PINN training on γ hard (i.e., the hard PDE context). 4 Dataset Description and Experimental Setup We evaluate our proposed LIGO-PINN model on three diverse PDE domains, described below. 1D Convection. The 1D convection PDE (Eq. 6) models transport of a scalar fieldu(z,t)with constant velocity. In Eq. 6,z ∈ [0, 2π]denotes the spatial coordinate andt ∈ [0, 1]denotes the temporal coordinate.βis the advection coefficient,u t andu z denote partial derivatives in time and space.u(z, 0) = sin(z)defines the initial condition and periodic boundaries impose u(0,t) = u(2π,t). N [u](z,t) := u t (z,t) + β u z (z,t) = 0,(6) This problem represents convection-dominated motion, where high-speed phase propagation and increasingly oscillatory space-time structure make PINN training challenging. In line with prior work Krishnapriyan et al. [2021], we considerΓ easy asβ ∈5, 10, 15, 20, 25and hard regime as β ∈40, 50, 60, 70, 80. Selection methodology for easy, hard tasks is detailed in Appendix B.2. 2D Helmholtz. The steady-state Helmholtz PDE (Eq. 7) describes oscillatory wave fields on a 2D domain. Letz = (x 1 ,x 2 )and let∆ = ∂ x 1 x 1 + ∂ x 2 x 2 denote the Laplace operator. Eq. 7 is defined on the domain[−1, 1] 2 with Dirichlet boundary conditionu = 0on∂Ω. Using the analytic solution u(z) = sin(a 1 πx 1 ) sin(a 2 πx 2 ), the source term becomes q(z) = k 2 − (a 1 π) 2 − (a 2 π) 2 u(z). N [u](z) := ∆u(z) + k 2 u(z)− q(z) = 0,(7) The coefficients(a 1 ,a 2 )control the oscillation frequency along x and y direction respectively, and larger values induce more frequent oscillations posing greater challenge for PINNs.Γ easy is considered(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)based on the case considered in Wang et al. [2021a] and hard regime is selected to have significantly more challenging propagation dynam- ics and both isotropic and anisotropic propagation dynamics are tested in the hard regimes 4 (4, 4), (4, 5), (5, 5), (4, 6), (5, 6), (6, 6). Our work has significantly expanded the investigation regime, compared to previous PINN-based investigations of 2D Helmholtz. Selection methodology for easy, hard tasks is detailed in Appendix B.2.. 2D Incompressible Navier–Stokes. We finally consider modeling the challenging 2D viscous flow over a cylinder, using the non-dimensional 2D incompressible Navier–Stokes equations, represented in Eq. 8 in compact momentum-continuity form. Hereu(x,y,t)is the velocity field,p(x,y,t)is the scalar pressure field,νis the kinematic viscosity (parameterized through the Reynolds number). Domain (x,y,t)∈ Ω× [0,T ], where Ω contains a unit-diameter cylinder. ∂u ∂t + (u·∇)u +∇p− ν∇ 2 u = 0, ∇·u = 0.(8) This system exhibits vortex shedding and transitional flow behavior, and the coupling between momentum and incompressibility creates multiple interacting residual terms, making it one of the most challenging PDE families for PINNs. The complexity of the dynamics increases with increase in Reynolds number (Re). We considerΓ easy to beRe∈100, 200, 300, 400, 500where relatively more steady, laminar dynamics are exhibited and in line with [Lee et al., 2025], we consider Re∈600, 800, 1000 to be hard regime where less steady dynamics are exhibited. To ensure a fair evaluation, all PINN baseline methods are trained for 56K epochs on the target domainΓ hard , while LIGO-PINN is trained only for 50K epochs on the target domain. This difference accounts for the 6K iterations employed for the learned initialization procedure which is absent in baseline methods which are randomly initialized. All other training parameters (detailed in Appendix B for full reproducibility) are identical including model architecture, training data, optimization settings. 5 Results & Discussion In this section, we demonstrate with rigorous quantitative and qualitative experiments, the effects of catastrophic failures on PINN performance and how these failures can be alleviated via Learned Initialization. To validate our approach, we compare LIGO-PINN against a suite of state-of-the- art models, that have been designed specifically to overcome catastrophic failures of PINNs in challenging PDE domains; the full descriptions of these baselines can be found in Appendix C.2. Specifically, we undertake this investigation by answering three research questions (RQ): RQ1. (Qualitative) How does learned initialization alleviate catastrophic PINN training dynamics? RQ2. (Quantitative) How does LIGO-PINN compare with other state-of-the-art (SoTA) approaches proposed to alleviate PINN catastrophic failures? RQ3. (Ablation) What is the effect of invariance encoding and gated layer-wise optimization in LIGO-PINN? 5.1 (RQ1) How does learned initialization alleviate catastrophic PINN training dynamics? We answer this question in three stages.First, we demonstrate qualitatively the catastrophic failures PINNs undergo in challenging PDE contexts.Second, we investigate how the training losses of the PINNs evolve in challenging domains.Third, we characterize the effect of the catastrophic training dynamics by analyzing the loss landscape neighborhood of the converged models. In each case, we contrast the analysis of a randomly initialized PINN, with LIGO-PINN behavior in the same context. Qualitative Failure Modes. In Fig. 1a, we demonstrate the solution predicted by a PINN (with random weight initialization) when trained on a challenging 1D convection domain withβ = 70. It can be clearly seen that the PINN model (center plot in Fig. 1a) has failed to estimate a good solution of the domain. The perfect solution of the domain can be seen in the left plot (labeled ‘Analytical Solution’) of Fig. 1a. We conduct a similar qualitative investigation of a separate PDE domain 2D-Helmholtz comprising two PDE parametersa 1 ,a 2 , with(a 1 = 6,a 2 = 6)in Fig. 1b. Once again, we notice that the PINN model (center plot of Fig. 1b) is unable to provide a good estimate of the challenging PDE domain solution (i.e., ‘Analytical Solution’ represented on the left plot in Fig. 1b). Turning our attention to the performance of LIGO-PINN in each of these two contexts (i.e., right plot in each of Fig. 1a and Fig. 1b), we notice that the estimation of LIGO-PINN closely aligns with the analytical solution of the corresponding PDE domain. We now highlight that the only difference 5 0.00.51.0 t 0 2 4 6 x Analytic Solution 0.00.51.0 t 0 2 4 6 x PINN Prediction 0.00.51.0 t 0 2 4 6 x LeIn-PINN Prediction −1.0 −0.5 0.0 0.5 LIGO-PINN Prediction (a) 1D Convection (β=70) −101 x 1 −1.0 −0.5 0.0 0.5 1.0 x 2 Analytic Solution −101 x 1 −1.0 −0.5 0.0 0.5 1.0 x 2 PINN Prediction −101 x 1 −1.0 −0.5 0.0 0.5 1.0 x 2 LeIn-PINN Prediction −1.0 −0.5 0.0 0.5 1.0 LIGO-PINN Prediction (b) 2D Helmholtz (a 1 =6,a 2 =6) Figure 1: Qualitative performance comparison of a randomly initialized PINN model and LIGO- PINN on two different and challenging PDE domains. We notice that both in the 1D convection (Fig. 1a) and 2D Helmholtz (Fig. 1b), LIGO-PINN faithfully recreates the analytical solution while the randomly initialized PINN variant experiences catastrophic failure. between the PINN and LIGO-PINN procedures is the weight initialization. Specifically, PINN weights have been initialized randomly (i.e., Xavier initialization) while the weights of LIGO-PINN have been initialized by our proposed learned initialization method. Thus, we infer that the learned initialization strategy in LIGO-PINN successfully alleviates catastrophic failures to which traditional PINNs are susceptible, when modeling complex PDE dynamics. ExtensiontoMulti-Physics. We also investigate LIGO-PINN performance on 2D incompressible, Navier-Stokes cylinder wake at high Reynolds numbers. A challenging multi-physics setting. We investigate both forward (pressure prediction) and inverse settings (recovering viscous and advection coefficients of Navier-Stokes equations). x y 0.1 0.0 0.1 Vorticity (a) Flow past a cylinder with vor- tex shedding (FEM solver) −1.5 0.0 1.5 x 0 4 8 t −0.8 0.0 0.8 y −0.25 0.00 0.25 0.50 0.75 1.00 1.25 1.50 Velocity Component (b) Streamwise velocity u x at t. −1.5 0.0 1.5 x 0 4 8 t −0.8 0.0 0.8 y −1.00 −0.75 −0.50 −0.25 0.00 0.25 0.50 0.75 Velocity Component (c) Transverse velocity u y at t. −2−1012 x −1.0 −0.5 0.0 0.5 1.0 y −0.10 −0.08 −0.06 −0.04 −0.02 0.00 (d) Ground-truth pressure p. −2−1012 x −1.0 −0.5 0.0 0.5 1.0 y 0.30 0.35 0.40 0.45 0.50 0.55 (e) PINN pressure (failure). −2−1012 x −1.0 −0.5 0.0 0.5 1.0 y −0.2 0.0 0.2 0.4 0.6 (f) LIGO-PINN pressure. Figure 2: Navier-Stokes cylinder wake atRe = 1000. (a) Simulation setup with vortex shedding (FEM solver). The black box indicates the region used for training and evaluation. (b–c) Velocity componentsu x ,u y at a representative time slice. Also showcased are collocation points at which velocity training data is sampled. (d) Ground-truth pressure. (e) Standard PINN prediction (failure). (f) LIGO-PINN prediction, showing faithful reconstruction of ground-truth pressure. Consistent with our findings on 1D-Convection and 2D-Helmholtz, we see from Fig. 2, that LIGO- PINN yields substantially lower errors and qualitatively faithful pressure fields (Fig. 2f) compared to randomly initialized PINN (Fig. 2e). Ground truth pressure field is depicted in Fig. 2d. This highlights the advantage of learned initialization to model complex fluid dynamics. Training Dynamics Analysis. We complement the qualitative analysis in Fig. 1 with an investigation (Fig.3) of the training dynamics in the same PDE contexts. Specifically, we highlight in Fig.3, the evolution of the total training loss (Eq. 2) of PINNs and LIGO-PINN on the same 1D convection, 2D Helmholtz and 2D Navier-Stokes domains. We note that the training loss plots in Fig.3 has been averaged over five independent training runs to capture the expected training dynamics. For the 1D convection problem (β = 70) the result depicted in Fig. 3a, shows that random initialization of the PINN leads to an early plateau in the loss curve, whereas our learned initialization method 6 01000020000300004000050000 Epoch 10 −4 10 −3 10 −2 10 −1 Training Loss PINN LeIn-PINN LIGO-PINN (a) 1D Convection (β = 70) 01000020000300004000050000 Epoch 10 −2 10 −1 10 0 10 1 10 2 10 3 10 4 10 5 10 6 Training Loss PINN LeIn-PINN LIGO-PINN (b) 2D Helmholtz (a 1 = 6, a 2 = 6) 0800016000240003200040000 Epoch 10 3 10 2 10 1 10 0 Training Loss PINN LeIn-PINN LIGO-PINN (c) 2D Navier-Stokes (Re= 1000) Figure 3: Loss curves highlighting training dynamics of randomly initialized PINNs and LIGO-PINN, across three PDE domains. In all three cases, LIGO-PINN converges to a solution with (at least) an order of magnitude lower loss, compared to randomly initialized PINN. (LIGO-PINN) achieves a steady, monotonic decrease, signaling more effective exploration of the solution manifold and ultimately, convergence to a loss value an order of magnitude lower than the corresponding randomly initialized PINN solution. A similar pattern emerges on the 2D Helmholtz and 2D Navier-Stokes problem(a 1 = 6, a 2 = 6)as shown in Fig. 3b,c. We see that the randomly initialized PINN exhibits premature convergence to a sub-optimal solution while, once again LIGO- PINN achieves a loss several orders of magnitude lower. This demonstrates the ability of our learned initialization based solution (LIGO-PINN) to achieve superior convergence dynamics by overcoming loss surface barriers Fort et al. [2020] and hence avoiding catastrophic failures. −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 0 100 200 300 (i) Total Loss 50 100 150 −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 0 1 2 3 (i) Data Loss 1 2 3 −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 0 100 200 300 (i) Residual Loss 50 100 150 (a) 1D Convection PINN Loss Landscape −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 0 200 400 600 (i) Total Loss 200 400 −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 0 2 4 (i) Data Loss 1 2 3 4 −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 0 200 400 600 (i) Residual Loss 200 400 (b) 1D Convection LIGO-PINN Loss Landscape Figure 4: Loss Landscape visualization generated by perturbing converged model weights along its top-2 eigenvectors (ν 1 ,ν 2 ). (a) Randomly Initialized PINN 1D Convection, with (i), (i), (i) representing loss landscape w.r.t total loss, data loss and residual loss respectively. (b) LIGO-PINN 1D Convection with (i), (i), (i) once again representing loss landscape w.r.t total, data, residual loss. Loss Landscape Analysis. Although the loss curves in Fig. 3 show a clear contrast between random and learned initialization, they do not fully reveal how training dynamics affect parameter optimization. To clarify this, we examine the loss landscape neighborhood of each converged model using a Hessian-informed perturbation analysis [Krishnapriyan et al., 2021, Böttcher and Wheeler, 2024]. Fig. 4 presents results for the 1D convection problem (2D Helmholtz in Appendix C.2). Under random initialization (Fig. 4a), the converged PINN exhibits a highly rugged surface with deep valleys and sharp ridges, driven mainly by the residual term (Fig. 4a.i). The presence of such ravines is known to hinder convergence to strong local minima Wu et al. [2017], Hochreiter and Schmidhuber [1997]. In contrast, the data loss landscape is smoother but still challenging to optimize. With LIGO-PINN, the converged landscape is markedly smoother without the sharp peaks of Fig. 4a. Fig. 4b.i shows the total loss surface, while Figs. 4b.i–b.i show data and residual components, all with smoother geometry compared to their counterparts resulting from a randomly initialized PINN. Viewed together with Fig. 3a, these results illustrate that random initialization produces residual-dominated peaks that trap the optimizer in narrow basins, whereas LIGO-PINN yields a smoother landscape that supports improved convergence to better local minima. 7 5.2 (RQ2) How do LIGO-PINNs compare with other state-of-the-art (SoTA) approaches proposed to alleviate PINN catastrophic failures? Recent efforts have begun to investigate catastrophic failures in PINNs, and in this section we compare our learned-initialization-based LIGO-PINN model against six comparative PINN baselines, including vanilla PINNs and recent methods proposed to improve PINN training stability [Krishnapriyan et al., 2021, Wang et al., 2021a, Daw et al., 2023, Qin et al., 2022, Cho et al., 2023]. We provide quantitative comparisons using the mean-absolute error (MAE) metric, across three benchmark PDE systems: the 1D Convection equation, the 2D Helmholtz equation, and the 2D incompressible time-dependent Navier-Stokes system [Raissi et al., 2017b]. In all cases, we evaluate models under challenging PDE settings, and extend our analysis regime far beyond what existing papers have investigated. For Navier-Stokes, we report both forward pressure reconstruction and inverse coefficient estimation in the main text. Table 1: Mean Absolute Error (MAE) on 1D convection system for extrapolation tasks (β ∈ 40, 50, 60, 70, 80). Method4050607080Avg PINN (fixed)0.74330.73370.87200.76710.82530.7883 PINN (dynamic) [Wang et al., 2021a]0.00950.01480.08810.15970.16990.0884 Curr-Reg. [Krishnapriyan et al., 2021]0.08010.25930.32150.37260.40720.2881 R3 [Daw et al., 2023]0.01620.14770.35570.43270.41510.2735 Meta-PDE [Qin et al., 2022]0.15880.21120.37570.35700.51670.3239 HyperLR [Cho et al., 2023]0.40080.45840.49120.53590.55740.4887 LIGO-PINN (Ours)0.00720.01190.01370.01790.08710.0275 1D Convection System Table 1 shows that LIGO-PINN achieves the lowest average MAE across the extrapolation settings and obtains the best result in four out of five tested regimes. LIGO-PINN achieves the strongest average performance, reducing average MAE by 96.5% relative to vanilla PINNs and by 68.9% relative to the strongest non-ours baseline, PINN-Dynamic. Across all six comparative baselines, the MAE reduction ranges from 68.9% to 96.5%. Table 2: Mean Absolute Error (MAE) on 2D Helmholtz PDE across varying domain parametersa xy . Methoda 44 a 45 a 55 a 46 a 56 a 66 Avg PINN (fixed)0.11350.22760.59310.34470.73961.22830.5411 PINN (dynamic) [Wang et al., 2021a]0.16450.39560.31850.31860.84440.47840.4200 Curr-Reg. [Krishnapriyan et al., 2021]0.30800.16300.82230.13310.41770.65200.4160 R3 [Daw et al., 2023]0.49100.41920.51430.47860.45500.39680.4590 Meta-PDE [Qin et al., 2022]0.24730.47360.59820.49540.40770.43290.4425 HyperLR [Cho et al., 2023]0.06130.04350.15990.15700.75852.94370.6874 LIGO-PINN (Ours)0.00080.00170.00220.00320.00740.16980.0307 2D Helmholtz System. The 2D Helmholtz system presents a more challenging oscillatory and multi-parameter setting. As shown in Table 2, the baseline methods remain substantially less accurate than LIGO-PINN across all tested domain parameters. On average, LIGO-PINN reduces MAE by 94.3% relative to vanilla PINNs and by 92.6% relative to the strongest non-ours baseline, PINN- Dynamic. Across all six comparative baselines, the MAE reduction ranges from 92.6% to 95.5%. These results indicate that learned initialization is particularly effective for high-frequency PDE regimes where standard PINN training and existing mitigation strategies remain unstable. Table 3: Pressure (p) prediction errors across Reynolds numbers. Best values per column in bold. Method6008001000Avg. PINN (fixed)0.01830.19340.02570.0791 PINN (dynamic) [Wang et al., 2021a]0.02390.09580.03530.0517 Curr-Reg. [Krishnapriyan et al., 2021]0.30110.25970.23090.2639 R3 [Daw et al., 2023]0.35190.49660.26200.3702 Meta-PDE [Qin et al., 2022]0.31720.29970.2812 0.2994 HyperLR [Cho et al., 2023]0.21550.12530.19100.1773 LIGO-PINN (Ours)0.01640.00650.00580.0096 2D Incompressible Navier–Stokes. Table 3 reports the pressure prediction results across Reynolds numbers. In this fluid dynamics setting, LIGO-PINN achieves an average MAE of0.0096, compared 8 Table 4: Navier–Stokes:λ adv andλ visc estimation errors across Reynolds numbers. Lowest values in each column are in bold. MethodRe_600Re_800Re_1000Avg λ adv λ visc λ adv λ visc λ adv λ visc λ adv λ visc PINN (fixed)0.1212 0.0282 0.1210 0.0441 0.0191 0.00740.0871 0.0266 PINN (dynamic) [Wang et al., 2021a] 0.0472 0.0085 0.1539 0.0370 0.0478 0.00740.0830 0.0176 Curr-Reg. [Krishnapriyan et al., 2021] 0.6251 0.1645 0.7021 0.1479 0.6380 0.1787 0.6551 0.1637 R3 [Daw et al., 2023]0.3841 0.0692 0.5106 0.0722 0.2222 0.11890.3723 0.0868 Meta-PDE [Qin et al., 2022]0.4320 0.1312 0.6910 0.2979 0.6910 0.23700.6046 0.2220 HyperLR0.0509 0.0108 0.0633 0.0133 0.1162 0.01630.0768 0.0135 LIGO-PINN w/o (GLO)0.2679 0.0510 0.2904 0.0374 0.0519 0.00870.2034 0.0324 LIGO-PINN (Ours)0.0291 0.0083 0.0082 0.0068 0.0082 0.0067 0.0152 0.0073 6008001000 Task 10 3 10 2 10 1 10 0 Absolute Error LIGO-PINN w/o (IE,GLO) LIGO-PINN w/o (GLO) LIGO-PINN (a) Empirical Evaluation (MAE). 020406080 Frequency 10 3 10 1 10 1 10 3 Log Amplitude LIGO-PINN w/o (IE,GLO) LIGO-PINN w/o (GLO) LIGO-PINN (b) Spectral Evaluation of Error Fields. Figure 5: The figure depicts results of LIGO-PINN ablation analysis on the most sophisticated PDE domain we evaluated (i.e., 2D Navier-Stokes). Fig. 5a depicts empirical (MAE) based ablation characterization for each ablation variant. Fig. 5b depicts a spectral distribution of the error fields for each ablation variant (lower is better). Overall, LIGO-PINN achieves lower MAE, reduced variance, and mitigates spectral bias relative to ablation variants. with0.0517of PINN (dynamic), (the strongest baseline), and0.0791for vanilla PINNs (i.e., PINN (fixed)). This corresponds to an error reduction of 81.4% relative to the strongest baseline and 87.9% relative to vanilla PINNs. Across all six comparative baselines, the MAE reduction ranges from 81.4% to 97.4%. These results show that learned initialization remains effective even in complex time-dependent fluid dynamics problems. In addition to the forward pressure reconstruction setting, we further evaluate all methods on the inverse formulation of the 2D incompressible Navier–Stokes system. In this setting, the advection and viscosity coefficients(λ adv ,λ visc )are treated as unknown parameters and jointly inferred dur- ing training from sparse observations. This inverse problem is particularly challenging due to the strong coupling between nonlinear momentum dynamics and incompressibility constraints, espe- cially at higher Reynolds numbers. Table 4 reports the coefficient estimation errors across Reynolds numbersRe∈600, 800, 1000. We observe that standard PINN baselines exhibit unstable param- eter recovery and large estimation errors as the flow dynamics become increasingly complex. In contrast, LIGO-PINN consistently achieves lower coefficient estimation error across all evaluated regimes, demonstrating that learned initialization improves not only forward field reconstruction but also inverse parameter identification in challenging multi-physics PDE systems. For the inverse problem involving estimation of the advection (λ adv ) and viscosity (λ visc ) coefficients, LIGO-PINN yields an 80.2%, and 45.9% performance improvement over the strongest baseline (HyperLR in the inverse modeling case). 5.3 (RQ3) What is the effect of invariance encoding and gated layer-wise optimization in LIGO-PINN? (Ablation Analysis) We analyze the effect of Invariance Encoding (IE) and Gated Layer-wise Optimization (GLO) through ablation variants: ‘LIGO-PINN w/o (IE,GLO)’ (random initialization) and ‘LIGO-PINN w/o GLO’ (MAML-style initialization). We focus on the challenging 2D Navier-Stokes setting, a multi-physics, highly coupled domain where PINN failures are most pronounced. The ablation analysis is conducted in two stages, (i) Empirical Evaluation (MAE) (i) Spectral Evaluation of Absolute Error Fields. Empirical Evaluation. Fig. 5a shows that neither simple random initialization (‘w/o (IE,GLO)’) nor just incorporating IE (‘w/o GLO’) improves performance on its own. Effective performance is only 9 demonstrated by LIGO-PINN where both IE and GLO simultaneously influence learning initialization. This highlights the critical role of gated, layer-wise parameter optimization and invariance encoding in learning initial weights that enable stable and accurate convergence in challenging PDE domains. Spectral Evaluation. Fig. 5b shows spectral density of the absolute error fields of each ablation variant (lower is better). LIGO-PINN consistently achieves lower error across both low and high frequencies, while ablation variants exhibit pronounced high-frequency error (highlighting spectral bias). Specifically, this indicates improved capture of fine-scale structures in fluid dynamics by LIGO-PINN compared to variants without IE or GLO. Results on 1D Convection and 2D Helmholtz exhibit similar trends and detailed in Appendix C.6. Additional Results. We also successfully verify that LIGO-PINN performance translates to complex unstructured 3D domains (results in Appendix C.1). For completeness, we also report computational time and memory footprint details in Appendix B.5. Our results depict that LIGO-PINN scales comparably to standard PINNs aside from a one-time meta-training overhead. Appendix C.7 presents sensitivity analyses, to the number and range of sampled tasks during learned initialization phase. 6 Conclusion This work shows that learned initialization can serve as an effective mechanism for reducing catas- trophic failure in PINNs. Rather than relying only on task-specific training heuristics such as curriculum design, adaptive sampling, or loss reweighting, LIGO-PINN learns a weight initialization that places the PINN in a more favorable optimization regime before training begins, leading to more stable training and substantially lower errors on challenging extrapolation problems. Across the evaluated PDE systems, LIGO-PINN reduces average error by 91.5% over six comparative PINN baselines and improves over the strongest baseline in each domain by an average of 81%. The ablation results further indicate that both invariance encoding and gated layer-wise optimization are important contributors to this behavior, reducing volatility and improving spectral stability. These findings suggest that learned initialization is a promising direction for more robust PINN training, es- pecially in regimes where standard training often breaks down. Finally, the results on 3D unstructured domains indicate that the benefits of LIGO-PINN are not ONLY limited to structured 2D settings and generalize to complex domains. Limitations and Future Work. While LIGO-PINN demonstrates strong performance across several challenging PDE systems, the proposed invariance encoding stage requires access to related PDE configurations during initialization. In addition, although the empirical analyses strongly suggest improved optimization behavior and reduced spectral bias, a deeper theoretical understanding of the underlying mechanisms will strengthen the framework and will be explored in the future. Finally, extending LIGO-PINN to real-world large-scale multi-physics and industrial simulation settings is a promising avenue for future investigation. 10 References Devansh Arpit, Víctor Campos, and Yoshua Bengio. How to initialize your network? robust initialization for weightnorm & resnets. Advances in Neural Information Processing Systems, 32, 2019. Lucas Böttcher and Gregory Wheeler. Visualizing high-dimensional loss landscapes with hessian directions. Journal of Statistical Mechanics: Theory and Experiment, 2024(2):023401, February 2024. ISSN 1742-5468. doi: 10.1088/1742-5468/ad13fc. URL http://dx.doi.org/10.1088/1742-5468/ad13fc. Woojin Cho, Kookjin Lee, Donsub Rim, and Noseong Park. Hypernetwork-based meta-learning for low-rank physics-informed neural networks. Advances in Neural Information Processing Systems, 36:11219–11231, 2023. Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. Journal of Scientific Computing, 92(3):88, 2022a. Salvatore Cuomo, Vincenzo Schiano di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. Scientific machine learning through physics-informed neural networks: Where we are and what’s next, 2022b. URL https://arxiv.org/abs/2201.05624. Arka Daw, Jie Bu, Sifan Wang, Paris Perdikaris, and Anuj Karpatne. Mitigating propagation failures in physics- informed neural networks using retain-resample-release (r3) sampling, 2023. URLhttps://arxiv.org/ abs/2207.02338. Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks, July 2017. URL http://arxiv.org/abs/1703.03400. arXiv:1703.03400 [cs]. Stanislav Fort, Gintare Karolina Dziugaite, Mansheej Paul, Sepideh Kharaghani, Daniel M Roy, and Surya Ganguli. Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the neural tangent kernel. Advances in Neural Information Processing Systems, 33:5850–5861, 2020. Sepp Hochreiter and Jürgen Schmidhuber. Flat minima. Neural computation, 9(1):1–42, 1997. Jungeun Kim, Kookjin Lee, Dongeun Lee, Sheo Yon Jhin, and Noseong Park. Dpm: A novel training method for physics-informed neural networks in extrapolation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8146–8154, 2021. Aditi S. Krishnapriyan, Amir Gholami, Shandian Zhe, Robert M. Kirby, and Michael W. Mahoney. Characterizing possible failure modes in physics-informed neural networks, November 2021. URLhttp://arxiv.org/ abs/2109.01050. arXiv:2109.01050 [physics]. Gregory Kang Ruey Lau, Apivich Hemachandra, See-Kiong Ng, and Bryan Kian Hsiang Low. Pinnacle: Pinn adaptive collocation and experimental points selection. arXiv preprint arXiv:2404.07662, 2024. Jongmok Lee, Seungmin Shin, Taewan Kim, Bumsoo Park, Ho Choi, Anna Lee, Minseok Choi, and Seungchul Lee. Physics informed neural networks for fluid flow analysis with repetitive parameter initialization. Scientific Reports, 15(1):16740, 2025. Apostolos F Psaros, Kenji Kawaguchi, and George Em Karniadakis. Meta-learning pinn loss functions. Journal of Computational Physics, 458:111121, June 2022. ISSN 0021-9991. doi: 10.1016/j.jcp.2022.111121. URL http://dx.doi.org/10.1016/j.jcp.2022.111121. Tian Qin, Alex Beatson, Deniz Oktay, Nick McGreivy, and Ryan P. Adams. Meta-pde: Learning to solve pdes quickly without a mesh, 2022. URL https://arxiv.org/abs/2211.01604. Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics informed deep learning (part i): Data- driven discovery of nonlinear partial differential equations, 2017a. URLhttps://arxiv.org/abs/1711. 10566. Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561, 2017b. Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Numerical gaussian processes for time-dependent and nonlinear partial differential equations. SIAM Journal on Scientific Computing, 40(1):A172–A198, 2018. Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. Advances in neural information processing systems, 32, 2019. 11 Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, and Jascha Sohl-Dickstein. Deep information propagation, 2017. URL https://arxiv.org/abs/1611.01232. Masato Taki. Deep residual networks and weight initialization. arXiv preprint arXiv:1709.02956, 2017. Maryam Toloubidokhti, Yubo Ye, Ryan Missel, Xiajun Jiang, Nilesh Kumar, Ruby Shrestha, and Linwei Wang. Dats: Difficulty-aware task sampler for meta-learning physics-informed neural networks. In The Twelfth International Conference on Learning Representations, 2023. Greg Turk and Marc Levoy. Zippered polygon meshes from range images. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’94, pages 311–318, New York, NY, USA, 1994. ACM. doi: 10.1145/192161.192241. Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021a. Sifan Wang, Hanwen Wang, and Paris Perdikaris. On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 384:113938, 2021b. Sifan Wang, Shyam Sankaran, and Paris Perdikaris. Respecting causality is all you need for training physics- informed neural networks. arXiv preprint arXiv:2203.07404, 2022a. Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:110768, 2022b. Sifan Wang, Ananyae Kumar Bhartari, Bowen Li, and Paris Perdikaris. Gradient alignment in physics-informed neural networks: A second-order optimization perspective, 2025. URLhttps://arxiv.org/abs/2502. 00604. Chenxi Wu, Min Zhu, Qinyang Tan, Yadhu Kartha, and Lu Lu. A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 403:115671, 2023. Lei Wu, Zhanxing Zhu, et al. Towards understanding generalization of deep learning: Perspective of loss landscapes. arXiv preprint arXiv:1706.10239, 2017. Ge Yang and Samuel Schoenholz. Mean field residual networks: On the edge of chaos. Advances in neural information processing systems, 30, 2017. 12 Supplementary LIGO-PINN Experiments A Detailed formulation of PINN In this section, we first present the general formulation of Physics-Informed Neural Networks (PINNs), and then demonstrate its instantiation on two canonical PDEs: the 1D Convection Equation and the 2D Helmholtz Equation. A.1 Physics Informed Neural Network A general form of a PDE can be expressed as: F (u(z);γ) = f (z),z∈ Ω,(9) B(u(z)) = g(z),z∈ ∂Ω.(10) Here: • Ω⊂R d represents the spatial domain, while ∂Ω denotes its boundary. •z = [x 1 ,x 2 ,...,x d ,t] ⊤ ∈R d+1 encapsulates space-time coordinates. • u is the unknown solution we aim to approximate. • γ denotes physical parameters of the system. • f (z) encodes system-specific data or forcing terms. • F is a (potentially nonlinear) differential operator characterizing the physical laws of the system. • Bis a boundary operator that enforces initial and boundary conditions, which can be of Dirichlet, Neumann, or periodic types. The PINN framework leverages automatic differentiation to embed the residuals ofFandBinto a composite loss function. This loss function penalizes deviations from the governing equations and boundary conditions, ensuring that the solution adheres to the physical laws while simultaneously fitting observed data. Neural networks are powerful tools due to their universal approximation property, which allows them to approximate any continuous function given sufficient capacity. In the context of PINNs, we aim to approximate the solutionu(z)of the governing equations using a neural networkˆu θ (z), whereθ denotes the learnable parameters of the neural network. The approximation can be written as: ˆu θ (z)≈ u(z),(11) wherez∈R d+1 represents the space-time coordinates. To achieve this, the PINN framework constructs a composite loss function consisting of: 1. PDE Residual Loss: This enforces the governing physical laws as specified by the PDE. 2. Boundary/Data Loss: This ensures compliance with boundary or initial conditions, as well as data consistency when available. PDE Residual LossUsing automatic differentiation tools, the neural network computes derivatives efficiently with respect to its inputs, enabling evaluation of the PDE residual. The residual is given by: R(z) =F (ˆu θ (z);γ)− f (z),(12) whereFrepresents the differential operator,f (z)is the system-specific forcing term, andγare the physical parameters. The residual loss over the domain Ω is formulated as: L r = 1 N Ω N Ω X i=1 |R(z i )| 2 ,(13) whereN Ω represents the number of collocation points in the domain, andz i ∈ Ωare the sampled points. 13 Boundary/Data Loss For enforcing boundary or initial conditions, we define a loss term that penalizes deviations from the specified boundary valuesg(z). For Dirichlet conditions, this can be written as: L b = 1 N ∂Ω N ∂Ω X j=1 |B(ˆu θ (z j ))− g(z j )| 2 ,(14) whereN ∂Ω represents the number of points on the boundary∂Ω, andz j ∈ ∂Ωare the boundary points. Total Loss The total loss for training the PINN combines the residual loss and the boundary/data loss, weighted by respective coefficients λ r and λ d : L total = λ r L r + λ d L b .(15) We calculate the total loss by adding the residual and data losses. Here,λ r andλ d are hyperparameters that control the relative contribution of the residual and data losses during training. The residual loss acts as a regularization term, ensuring that the learned solution adheres to physically consistent laws. To train the neural network, we solve the optimization problem: θ ∗ = arg min θ L total .(16) We aim to formulate a Physics-Informed Neural Network (PINN) by studying two fundamental classification of systems in physics that are crucial for understanding and modeling the physical world. These systems are categorized based on whether their properties depend on time: Time-Independent Systems These systems are used to study the steady-state behavior of a system, where properties such as energy, momentum, or other conserved quantities remain constant over time. Time-independent systems help us analyze static or equilibrium conditions, providing insights into the inherent properties of the system without considering temporal evolution. Time-Dependent Systems These systems are used to study how the state of a system evolves over time. By understanding the temporal dynamics, we can gain insights into transient phenomena, system responses, and changes in state variables due to external forces or intrinsic behaviors. To explore these concepts, we explore the following two systems. A.2 1D Convection Equation This system represents a classic example of a time-dependent system. We consider a one-dimensional convection system to demonstrate the application of Physics-Informed Neural Networks (PINNs). The system models the transport of heat or mass in a medium, where the primary variableu(x,t) evolves over space and time. The spatial domain isx∈ [0, 2π], and the time domain ist∈ [0, 1]. This setup allows for studying convection behavior under periodic boundary conditions, often encountered in cyclic systems. This formulation is adapted from Krishnapriyan et al. [2021] The governing partial differential equation (PDE) is given by: ∂u ∂t + β ∂u ∂x = 0, x∈ Ω, t∈ [0,T ],(17) with the initial condition: u(x, 0) = h(x), x∈ Ω.(18) Here: • β is the convection coefficient (physical parameter of system γ in 9), • h(x) specifies the initial condition for u(x,t), • Ω denotes the spatial domain. To evaluate the performance of PINNs, we generate ground truth data using an analytical solution. For constant β and periodic boundary conditions, the analytical solution is: u analytical (x,t) =F −1 F (h(x))e −iβkt ,(19) where: 14 • F andF −1 are the Fourier transform and its inverse, respectively, • i = √ −1 is the imaginary unit, • k represents the frequency in the Fourier domain. For this problem, we assume: h(x) = sin(x), u(0,t) = u(2π,t),(20) implying periodic boundary conditions. Residual Loss The residual loss for this PDE is given by: L r = 1 N f N f X i=1 ∂ ˆu ∂t + β ∂ ˆu ∂x 2 ,(21) where: • N f : Number of collocation points sampled from the interior of the domain Ω× [0,T ] • ˆu is the predicted solution, • ∂ ˆu ∂t : Temporal derivative of the predicted solution, • ∂ ˆu ∂x : Spatial derivative of the predicted solution, • β:Convection coefficient which is the parameter describing the physical system Data Loss The data loss consists of two components: the loss enforcing the initial condition and the loss enforcing the boundary conditions. It is defined as: L d =L ic +L bc ,(22) where: L ic = 1 N u N u X i=1 (ˆu(x i , 0)− g(x i , 0)) 2 ,(23) L bc = 1 N b N b X j=1 (ˆu(0,t j )− ˆu(2π,t j )) 2 .(24) Here: • L ic : Initial condition loss, ensuring the predicted solutionˆu(x, 0)matches the datag(x, 0) = h(x), • L bc : Boundary condition loss, ensuring periodic boundary conditions ˆu(0,t) = ˆu(2π,t), • N u : Number of points sampled from the initial condition, • N b : Number of points sampled from the boundary condition, • g(x i , 0) = h(x i ): Known solution at initial condition points, • ˆu(0,t j ) and ˆu(2π,t j ): Predicted solutions at boundary points. Total Loss The total loss combines the above components: L(θ) = λ r L r + λ d L d ,(25) where: • λ r ,λ d are weighting coefficients balancing the contributions of the respective terms. 15 A.3 2D Helmholtz Equation We study the vibration states of a two-dimensional membrane using the 2D Helmholtz equation. This equation is widely used to describe stationary wave fields, such as vibrations in membranes, sound waves, and electromagnetic waves. Specifically, we consider a square membrane with spatial coordinatesx 1 ,x 2 ∈ [−1, 1]and analyze its steady-state behavior under fixed boundary conditions. The Helmholtz equation helps identify resonance patterns and steady-state vibration modes in such systems. The formulation is adapted from Wang et al. [2021a]. The governing PDE for the 2D Helmholtz equation is: ∂ 2 u ∂x 2 1 + ∂ 2 u ∂x 2 2 + k 2 u = q(x 1 ,x 2 ),(x 1 ,x 2 )∈ Ω,(26) where Ω = [−1, 1]× [−1, 1] is the spatial domain. The boundary of the domain,∂Ω, represents the fixed edges of the membrane. Consequently, the displacement u(x 1 ,x 2 ) at the boundaries is zero, giving rise to the Dirichlet boundary condition: u(x 1 ,x 2 ) = 0,(x 1 ,x 2 )∈ ∂Ω.(27) We specifically choosek 2 ̸= a 2 1 +a 2 2 to ensure that the selectedkdoes not correspond to an eigenmode of the domain, resulting in a non-zero source termq(x 1 ,x 2 ). Thus, the PDE system is inherently inhomogeneous. Here: • k is the wavenumber and is kept fixed at k = 1. • a 1 ,a 2 ∈Z + are integer mode numbers representing the spatial oscillation patterns. These serve as the primary PDE parameters affecting solution complexity and difficulty. • q(x 1 ,x 2 )is the resulting source term that characterizes external excitation or forcing within the membrane, driving the system response. For this setup, we use an analytical solution of the form: u(x 1 ,x 2 ) = sin(a 1 πx 1 ) sin(a 2 πx 2 ),(28) which satisfies the zero-displacement boundary condition. The corresponding source term is derived as: q(x 1 ,x 2 ) = k 2 − (a 1 π) 2 − (a 2 π) 2 sin(a 1 πx 1 ) sin(a 2 πx 2 ).(29) Residual Loss The residual loss quantifies deviations from the Helmholtz equation at collocation points in Ω: L residual = 1 N f N f X i=1 ∂ 2 ˆu ∂x 2 1 + ∂ 2 ˆu ∂x 2 2 + k 2 ˆu− q(x 1,i ,x 2,i ) 2 ,(30) where: • N f : Number of collocation points sampled from the interior of the domain Ω, • ˆu: Predicted solution obtained from the neural network, • q(x 1,i ,x 2,i ): Source term evaluated at the collocation point (x 1,i ,x 2,i ). Boundary Condition Loss The boundary loss enforces zero displacement at the edges of the membrane, consistent with the fixed boundary condition: L boundary = 1 N b N b X i=1 ˆu(x 1,i ,x 2,i ) 2 ,(31) where N b is the number of points sampled from the boundary ∂Ω. 16 Total Loss The total loss combines the residual and boundary condition losses: L(θ) = λ r L residual + λ b L boundary ,(32) whereλ r andλ b are weighting coefficients balancing the contributions of the residual and boundary losses. A.4 2D Incompressible Navier-Stokes (with learnable coefficients) We consider the two-dimensional incompressible Navier-Stokes equations in non-dimensional form, augmented with learnable coefficients for the advection and viscosity terms: u t + λ adv (u x + vu y ) =−p x + λ vis (u x + u y ),(33) v t + λ adv (uv x + v y ) =−p y + λ vis (v x + v y ),(34) u x + v y = 0.(35) Here,u(t,x,y)andv(t,x,y)are the velocity components,p(t,x,y)is the pressure, andλ adv ,λ vis are trainable scalars. In a forward setting,pis predicted by the network but not supervised; in an inverse setting, λ adv and λ vis are learned from data. Network Outputs Let a neural networkN θ (t,x,y) predict (ˆu, ˆv, ˆp) =N θ (t,x,y), Residual Loss Define the PDE residuals at collocation points(t i ,x i ,y i ) N f i=1 by f u = ˆu t + λ adv (ˆu ˆu x + ˆv ˆu y ) + ˆp x − λ vis (ˆu x + ˆu y ),(36) f v = ˆv t + λ adv (ˆu ˆv x + ˆv ˆv y ) + ˆp y − λ vis (ˆv x + ˆv y ),(37) f c = ˆu x + ˆv y ,(38) and the residual loss L r = 1 N f N f X i=1 |f u (t i ,x i ,y i )| 2 + |f v (t i ,x i ,y i )| 2 + α|f c (t i ,x i ,y i )| 2 ,(39) where α > 0 weights the continuity residual. Data Loss (Vortex Shedding Velocities) Given velocity measurements from the vortex-shedding flow,(t i ,x i ,y i ,u i ,v i ) N d i=1 , we use only velocity supervision: L d = 1 N d N d X i=1 (ˆu(t i ,x i ,y i )− u i ) 2 + (ˆv(t i ,x i ,y i )− v i ) 2 .(40) Pressure is not supervised and is recovered implicitly via the momentum equations. Total Loss The training objective combines residual and data terms: L(θ,λ adv ,λ vis ) = λ r L r + λ d L d ,(41) with weightsλ r ,λ d > 0. For inverse problems,λ adv andλ vis are optimized jointly withθ; for forward problems they are fixed (e.g., λ adv = 1, λ vis = 1/Re). B Experimental Setup B.1 Dataset and Task Construction We evaluate our method across three PDE systems: 1D Convection, 2D Helmholtz, and 2D Navier- Stokes. For each PDE system, tasks are constructed by varying the governing PDE parameters following the experimental protocols established in prior PINN benchmark works. 17 1D Convection. For the 1D Convection system, we use the analytical solution formulation intro- duced in Curr-Reg. Krishnapriyan et al. [2021]. The dataset generation and task construction closely follow the publicly available benchmark implementation provided in their codebase. Task difficulty is controlled through the convection coefficientβ, where each task corresponds to a different value of β. 2D Helmholtz. For the 2D Helmholtz system, we construct our own dataset using the analytical solution formulation described in Wang et al. [2021a]. The source term is analytically derived from the selected closed-form solution, allowing exact supervision of PDE behavior. Task construction is performed by varying the spatial frequency parameters(a 1 ,a 2 ), which control the oscillatory complexity of the PDE solution. 2D Navier-Stokes. For the 2D Navier-Stokes system, we generate fluid simulation data using PhiFlow, a differentiable fluid simulation framework based on finite element style numerical solvers. The simulation setup follows the cylinder wake configuration commonly adopted in PINN literature and is designed to closely match the benchmark setting introduced by Raissi et al. [2017a],. Task difficulty is controlled by varying the Reynolds number (Re), where higher Reynolds regimes correspond to increasingly complex flow dynamics. Across all PDE systems, task families are constructed by systematically varying the corresponding PDE parameters, following the difficulty progression protocols established in the aforementioned baseline works. B.2 Task Difficulty Characterization via PINN Loss Dynamics In many PDE settings, easy vs. hard regimes are known and hence task difficulty is defined with the aid of domain expertise, where practitioners identify regimes that are known to be easy or challenging based on the underlying physics. However, in the absence of such prior knowledge, purely empirical criteria can be adopted, based on PINN training dynamics. Specifically, we observe that the distribution of PINN training losses even after early-stage training (i.e., 10K epochs) already provides a clear and consistent signal of task difficulty. Across PDE systems, this behavior exhibits a distinct transition in error dynamics, allowing us to separate regimes where standard PINN training is effective from those where it begins to fail. 5 101520253040506070 Task parameter () 10 5 10 4 10 3 10 2 Total Loss Easy tasks Hard tasks (a) 1D Convection (1,1)(1,2)(1,3)(1,4)(1,5)(4,4)(4,5)(4,6)(5,5)(5,6)(6,6) Task parameters ((a 1 , a 2 )) 10 5 10 3 10 1 10 1 10 3 Total Loss Easy tasks Hard tasks (b) 2D Helmholtz Figure 6: Distribution of residual losses @ 10K across five random seeds. 1D Convection exhibits a gradual transition in difficulty asβincreases, whereas Helmholtz shows a significantly sharper transition due to increasing oscillatory complexity. 1D Convection. For the convection equation, task complexity is governed by the coefficientβ. As shown in Fig. 6(a), lower values ofβyield consistently low training loss across random seeds, indicating stable optimization. Asβincreases, the magnitude of the loss grows, with a sharp increase in magnitude at the transition from easy to difficult task boundary. Note, despiteβ = 25exhibiting ‘hard task’ dynamics, it is included in the easy task set to maintain consistency with the experimental setup employed in Curr-Reg.( Krishnapriyan et al. [2021]) a state-of-the-art baseline we compare 18 with in our work. Based on this transition, we considerβ ≥ 30as the onset of the hard regime, and treat all tasks with β ≥ 30 as hard tasks in our experiments. 2D Helmholtz. In the 2D Helmholtz equation, task complexity is controlled by the parameters (a 1 ,a 2 )which govern the domain spatial frequency (given by p a 2 1 + a 2 2 ). As shown in Fig. 6(b), low-frequency configurations produce low and stable PINN training losses, indicating that PINNs can effectively learn task dynamics. In contrast, hard-task settings (i.e., high frequency configurations) exhibit a sharp increase in loss magnitude. B.3 Baseline Model Description We now provide brief descriptions of baseline models employed for comparative evaluation. 1. PINN (fixed) Raissi Original Formulation 2.PINN (dynamic) Wang et al. [2021a], Wu et al. [2023]: The current optimized and stable PINN training paradigm. In each iteration, collocation points are randomly resampled using quasi-random low-discrepancy sequences, combined with adaptive learning rate scheduling to improve convergence. 3.Curr-Reg. Krishnapriyan et al. [2021]: Curriculum regularization of PINN which pro- gressively trains models from easier to harder PDE tasks by incrementally introducing complexity during training. If we want to Learn a 1D convection system withβ = 30. We incremently train on β =5, 10, 15, 20, 25, 30 4.R3 Daw et al. [2023]: Employs the adaptive sampling strategy Retain-Resample-Release (R3), focusing training effort by retaining collocation points with high residual errors while periodically resampling the remaining points. 5.Meta-PDE Qin et al. [2022]: A meta-learning framework for PDEs based on a MAML-style formulation. It also includes a learnable per-parameter step-size. 6.HyperLR Cho et al. [2023]: A hypernetwork-based meta-learning method that generates low-rank, task-specific PINN weights for parameterized PDEs. 7.LIGO-PINN w/o (IE, GLO): Effectively standard PINN training from randomly initialized weights. 8.LIGO-PINN w/o GLO: An adaptation of the Model-Agnostic Meta-Learning–based frame- work. B.4 Training Configuration per PDE System We summarize the paper-facing experimental configurations used across all PDE systems. Unless otherwise specified, all models use Xavier weight initialization. Standard PINN-style baselines are optimized with Adam and cosine annealing learning-rate scheduling. For fair comparison, baseline methods and LIGO-PINN share the same network architectures, task definitions, collocation budgets, and evaluation protocols within each PDE system. The training horizon is matched to the method family. One-phase baselines are trained for 56,000 steps. LIGO/MAML-based downstream fine-tuning is run for 50,000 steps. For LIGO-PINN, the LIGO invariance encoding stage is meta-trained exclusively on the easy-task distributionΓ Easy , while evaluation is conducted on the unseen hard-task distributionΓ Hard . During invariance encoding, the model learns shared initialization priors across related PDE tasks. It is then adapted to harder PDE regimes. Subsequently, gated layer-wise optimization (GLO) progressively adapts deeper network layers during task-specific fine-tuning. The corresponding outer and inner optimization steps and learning rates are reported explicitly in the tables below. Unless explicitly stated otherwise, all reported MAE results are averaged across five random seeds. Within each PDE system, the same collocation point budgets, boundary-condition definitions, and evaluation protocols are used across competing methods. Table 5 reports the settings for the 1D Convection system, Table 6 reports the settings for the 2D Helmholtz system, and Table 7 reports the settings for the 2D Navier-Stokes system. 19 Table 5: 1D Convection Experimental Settings Common Settings Equation∂ t u + β ∂ x u = 0 Input/Domain(x,t)∈ [0, 2π]× [0, 1] Dataset SourceAnalytical solution adapted from Krishnapriyan et al. [2021] Boundary ConditionPeriodic Baseline SamplingStatic and dynamic baselines use matched collocation budgets. Adaptive SamplingR3 uses adaptive residual-based resampling. Network Architecture5 hidden layers, 50 neurons each, Tanh, Xavier init. MLP Parameters10,401 Collocation PointsN f = 1000 Boundary PointsN b = 1000 Supervised Data UsageMatched across compared methods. Training Tasks (Γ Easy )β ∈5, 10, 15, 20, 25 Evaluation Tasks (Γ Hard )β ∈, 40, 50, 60, 70, 80 Fine-Tuning Settings OptimizerAdam Base Learning Rate0.01 SchedulerCosine scheduling Residual/Data Weight Ratioλ res : λ data = 0.01 : 1 Layer Gating During EvaluationFalse Baseline Training Steps56,000 LIGO/MAML Fine-Tuning Steps50,000 LIGO Settings LIGO Outer Steps200 LIGO Inner Steps10 Layer GatingTrue LIGO Outer OptimizerAdam LIGO Inner OptimizerSGD LIGO Outer LR0.01 LIGO Inner LR0.005 Reproducibility Notes. We provide the supplementary material needed to reproduce the reported results. Above, we specify the architecture, optimizer, learning-rate, task split, sampling, and training-horizon details used for each PDE system. The supplementary code package includes README-based instructions for environment setup, experiment execution, and result reproduction. We also provide the selected model runs and random-seed settings used for the reported evaluations so that the main benchmark tables and figures can be reproduced consistently. Unless otherwise noted, all final reported MAE values are averaged over five random seeds. Analytical-task PDE systems are reproduced through the provided task-generation code, while the Navier–Stokes experiments use the supplied simulation/data-generation pipeline derived from thePhiFlowframework. Experiment metadata, metrics, and artifacts are tracked with MLflow for auditability and result verification. B.5 Computational Resources All experiments were conducted on an internal Linux GPU server using a four NVIDIA RTX A6000 GPU with 48 GB vRAM per GPU. To improve reproducibility, we report the hardware and software environment, the experiment execution setup, and representative runtime and memory costs for the main benchmark pipeline. The reported specifications correspond to the environment used for the paper-facing experiments. Computation Cost. To contextualize the computational overhead of invariance encoding, we report representative wall-clock time (minutes) and peak GPU memory usage (MB) in Table 9. All measurements were taken on the same NVIDIA RTX A6000 GPU to ensure consistency across stages. Wall-clock time was recorded per run, and peak GPU memory was measured usingnvidia-smi. The 20 Table 6: 2D Helmholtz Experimental Settings Common Settings Equation∆u + k 2 u = q(x 1 ,x 2 ), k = 1 Input/Domain(x 1 ,x 2 )∈ [−1, 1] 2 Dataset SourceAnalytical solution adapted from Wang et al. [2021a] Boundary ConditionDirichlet Baseline SamplingStatic and dynamic baselines use Halton-based sampling. Adaptive SamplingAdaptive baselines use residual-guided updates under the same colloca- tion budget. Spatial Frequency Parameters(a 1 ,a 2 ) Network Architecture5 hidden layers, 50 neurons each, Tanh, Xavier init. MLP Parameters10,401 Collocation PointsN f = 1000 Boundary PointsMatched across compared methods Supervised Data UsageMatched across compared methods. Training Tasks (Γ Easy )(a 1 ,a 2 )∈(1, 1), (1, 2), (1, 3), (1, 4), (1, 5) Evaluation Tasks (Γ Hard )(a 1 ,a 2 )∈(4, 4), (4, 5), (5, 5), (4, 6), (5, 6), (6, 6) Fine-Tuning Settings OptimizerAdam Base Learning Rate0.01 SchedulerCosine scheduling Residual/Data Weight Ratioλ res : λ data = 0.01 : 1 Layer Gating During EvaluationFalse Baseline Training Steps56,000 LIGO/MAML Fine-Tuning Steps50,000 LIGO Settings LIGO Outer Steps200 LIGO Inner Steps10 Layer GatingTrue LIGO Outer OptimizerAdam LIGO Inner OptimizerSGD LIGO Outer LR0.005 LIGO Inner LR0.001 table reports representative benchmark costs for the 1D Convection system and should be interpreted as stage-level execution cost rather than full cumulative project compute. The overhead of LIGO-PINN is defined as the additional runtime introduced by the learned initial- ization meta-learning stage relative to PINN-D training. Across the three PDE systems, PINN-D requires 51.82 GPU-hours, while LIGO-PINN requires 62.57 GPU-hours when both learned initial- ization via meta-learning and downstream fine-tuning are included. This corresponds to an additional 10.75 GPU-hours, or a 20.75% cumulative runtime overhead. This overhead is incurred during the initialization stage and can be amortized when the learned initialization is reused across multiple downstream tasks within the same PDE family. The higher peak memory observed for the Navier–Stokes system is primarily due to the larger dataset and the additional automatic-differentiation graph required by the coupled, higher-order PDE residual terms. Peak memory can be reduced by using a first-order MAML-style update, where higher-order gradients through the inner-loop adaptation are not tracked. This would improve runtime and memory efficiency, but it trades off meta-gradient fidelity and may reduce final accuracy. In this work, we keep the higher-order meta-gradient computation and use the same inner-loop setting across PDE systems to maintain experimental parity. The invariance-encoding stage is a one-time cost. Once learned, the initialization can be reused across multiple tasks within the same PDE family, thereby amortizing the additional meta-training overhead. During the reported comparisons, the collocation budgets and evaluation settings were kept fixed between invariance encoding, downstream fine-tuning, and standard PINN training. This shows that LIGO-PINN retains comparable per-task fine-tuning cost to standard PINNs, with the main additional expense arising from the one-time initialization stage. 21 Table 7: 2D Navier-Stokes Experimental Settings (cylinder wake) Common Settings EquationIncompressible NS with learnable λ adv , λ vis Input/Domain(x,y,t)∈ Ω× [0,T ] Spatial GeometryΩ is a channel with a unit-diameter cylinder Dataset Source PhiFlow fluid simulation framework Reference BenchmarkCylinder wake setup adapted from Raissi et al. [2017b] Baseline SamplingFixed/static and dynamic settings use matched collocation budgets. Adaptive Sampling Adaptive settings use matched budgets with residual-guided resampling. Network Architecture7 hidden layers, 50 neurons each, Tanh, Xavier init. MLP Parameters15,602 Collocation PointsN f = 5000 Training Tasks (Γ Easy )Re∈100, 200, 300, 400, 500 Evaluation Tasks (Γ Hard )Re∈600, 800, 1000 Fine-Tuning Settings OptimizerAdam Base Learning Rate0.01 SchedulerCosine scheduling Residual/Data Weight Ratioλ res : λ data = 1 : 1 Layer Gating During EvaluationFalse Baseline Training Steps56,000 LIGO/MAML Fine-Tuning Steps50,000 LIGO Settings LIGO Outer Steps200 LIGO Inner Steps10 Layer GatingTrue LIGO Outer OptimizerAdam LIGO Inner OptimizerSGD LIGO Outer LR0.001 LIGO Inner LR0.005 Table 8: Hardware and software specifications used for the reported experiments. ComponentSpecification Compute PlatformInternal Linux GPU server GPUNVIDIA RTX A6000 (48 GB vRAM) CPUIntel(R) Xeon(R) Platinum 8358 CPU @ 2.60 GHz System RAM2 TB DDR4 Operating SystemUbuntu 20.04 LTS Python3.10 PyTorch2.4 CUDA12 Experiment TrackingMLflow 2.14 Memory Monitoringnvidia-smi C Additional Results In this section, we present supplementary training results to further evaluate the behavior of our methods. C.1 Do LIGO-PINN generalize effectively to PDEs defined on complex geometries? To evaluate whether our method generalizes to PDEs on complex geometries, we compare LIGO- PINN with PINN (dynamic) for the screened Poisson equation defined on the Stanford Bunny surface Turk and Levoy [1994]. We further evaluate both PINN (dynamic) and LIGO-PINN under multiple random seed initializations to ensure robustness. Both models are trained for 50k epochs under identical training settings. LIGO-PINN achieves a 15% improvement over PINN (dynamic). 22 Table 9: Cumulative computation cost across PDE systems. PINN-D reports target-domain training cost across all evaluation tasks and five random seeds. LIGO-PINN reports the total cost, including cost of learned initialization via meta-learning and downstream fine-tuning. Overhead is computed relative to PINN-D. Peak memory is reported as the maximum observed GPU memory and is not summed across runs. PDERunsPINN-DLIGO-PINN TotalOverhead Time (h)GBTime (h)GBTime (h)% 1D Convection259.732.1010.822.301.0911.23 2D Helmholtz3021.560.7022.881.121.326.10 2D Navier–Stokes1520.531.7028.8719.128.3440.64 Total7051.822.1062.5719.1210.7520.75 Figure 7: Poisson equation on a bunny surface. Overall, we find that the LIGO-PINN achieves a 15% improvement in MAE over PINN (dynamic) even on complex domains. (a) Ground truth field. (b) LIGO-PINN absolute error. (c) PINN (dynamic) absolute error. C.2 Hessian-Based Loss-Landscape Visualization To better understand the local geometry of the trained PINN, we analyze the curvature of each loss component—total loss, data loss, and residual loss—around the converged weights θ ∗ . Concretely: 1. Calculate losses. Let L tot (θ), L data (θ), L res (θ) denote the total, data, and residual losses, respectively, as in Eq. (2). 2. Compute Hessians. For each loss L∈L tot ,L data ,L res , form the Hessian H L =∇ 2 θ L(θ) θ=θ ∗ . 3. Eigen-decomposition. Solve H L v i = λ i v i , λ 1 ≥ λ 2 ≥ ..., and retain the top two eigenpairs(λ 1 ,v 1 ),(λ 2 ,v 2 ), which capture the directions of greatest curvature. 4. Parameter perturbation. For offsets (α,β)∈ [−δ,δ] 2 , define θ(α,β) = θ ∗ + αv 1 + β v 2 . 5. Landscape slice. The two-dimensional landscape f L (α,β) = L θ(α,β) is evaluated on a uniform grid of (α,β) and visualized as a heatmap. 2D Helmholtz Loss-Landscape Visualization. In 8a we see that in case that model has converge to a trivial solution that has trapped in steep 1 dimension wells and stops which does nto faciliates learning. on the other hand 8b we that a converged model has smoother optimization bowls that facilates good convergence 23 −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 600 605 610 (i) Total Loss 600 605 610 −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 0 5 10 (i) Data Loss 5 10 −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 597.8 598.0 (i) Residual Loss 597.8 598.0 (a) 2D Helmholtz PINN Loss Landscape −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 0 1000 2000 (i) Total Loss 1000 2000 −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 0 10 20 (i) Data Loss 10 20 −0.5 0.0 0.5 ν 1 −0.5 0.0 0.5 ν 2 0 1000 2000 (i) Residual Loss 1000 2000 (b) 2D Helmholtz LIGO-PINN Loss Landscape Figure 8: Loss Landscape visualization by pertubating model along its top-2 eigenvectors (ν 1 ,ν 2 ) plots for PINN Training (Total, Data, Residuals) in the two case of PINN in 2D Helmholtz System(a 55 ): (a) Randomly Initialized PINN, (b) LIGO-PINN. C.3 Exploring Effect of different initialization Table 10 compares the extrapolation MAE for PINN-Dynamic under Xavier initialization, PINN with Kaiming initialization, and PINN-Dynamic with LIGO (i.e., nothing but our proposed LIGO- PINN model). The learned initialization (i.e., PINN-Dynamic LIGO) achieves consistently lower error than both Xavier and Kaiming baselines as task get progressively harder, indicating that the learned invariance weights provide a stronger starting point for optimization than standard random initializations. Table 10: Error on 1D convection extrapolation tasks measured by MAE (β ∈ 40, 50, 60, 70). Values show mean with standard deviation in parentheses. Task LIGO-PINN Xavier LIGO-PINN Kaiming LIGO-PINN LIGO 400.0130 (0.0065) 0.0158 (0.0029) 0.0072 (0.0012) 500.0120 (0.0044) 0.0176 (0.0023) 0.0119 (0.0032) 600.0162 (0.0019) 0.0210 (0.0057) 0.0137 (0.0024) 700.0256 (0.0070) 0.0708 (0.0644) 0.0179 (0.0050) Avg0.01670.03130.0127 Layer-wise Parameter Distribution Analysis. To characterize how the learned initialization modifies the PINN parameter landscape, we compare the layer-wise weight distributions of LIGO-PINN against layer-wise weight distributions of a randomly initialized PINN. We report (Table. 11) two statistics for the first, middle, and final layers across all PDEs namely: (i) the absolute difference between the variance of weight distributions per layer∆ =|Var(LIGO-PINN)-Var(Random PINN)|, and (i) the 1-Wasserstein distance, which measures distributional similarity. In Table 11, we find an interesting behavior where the results show a consistent pattern across systems. Specifically, we found a stark distributional dissimilarity (between randomly initialized PINNs and PINNs initialized with learned weights) in shallower layers with increased distributional similarity in deeper layers. This difference in distributional similarity can be attributed to a higher variance of weights in shallower layers of LIGO-PINN, compared to variance of weights in the corresponding 24 Table 11: Layer-wise statistics across PDEs. Variance∆captures the absolute scale difference and Wasserstein distance quantifies distributional similarity. Layer 1D Convection2D Helmholtz2D Navier–Stokes Variance ∆WassersteinVariance ∆WassersteinVariance ∆Wasserstein First Layer0.0581030.1082910.0137710.0387300.0048540.039123 Middle Layer 0.0091420.0245600.0021090.0061870.0010540.004970 Last Layer0.0002640.0026630.0001230.0029500.0004590.004678 layers of randomly initialized PINNs. Previous work has shown that neural networks with higher variance of weights in shallower layers have increased expressivity.Yang and Schoenholz [2017], thereby allowing us to infer that LIGO-PINN with learned weights offer greater expressive power (and thereby larger learning capacity) compared to randomly initialized PINNs, also contributing to their superior performance in challenging PDE domains. C.4 Compatibility with Existing PINN Stabilization Methods We further investigate whether the proposed learned initialization strategy is compatible with existing PINN stabilization approaches. To this end, we integrate our learned initialization into the R3 sampling framework (R3) Daw et al. [2023], a state-of-the-art adaptive collocation sampling method designed to improve PINN convergence stability. Specifically, we compare two variants of the R3 framework: • R3 (Xavier Init.): standard R3 training initialized using Xavier initialization, •R3 (LIGO Init.): R3 initialized using learned weights obtained from our invariance encoding setup. To provide a stronger stress test, both variants were trained for 100K epochs, significantly extending beyond the primary training budget used throughout this work. Evaluation was performed on challenging extrapolation regimes of the 1D Convection system withβ ∈40, 50, 60. For fairness, both models used identical architectures, optimization schedules, and collocation settings. Results are averaged across 5 independent random seeds, and we report both mean MAE and standard deviation. As shown in Table 12, the proposed learned initialization remains compatible with the R3 framework even under substantially extended training schedules. In particular, the learned initialization achieves lower MAE than Xavier initialization in theβ = 40andβ = 60extrapolation settings, including a substantial improvement in the more challengingβ = 60regime. These results suggest that the proposed initialization mechanism is complementary to existing PINN stabilization strategies and can be integrated into adaptive sampling frameworks in a model-agnostic manner. Table 12: Comparison of R3 performance on the 1D Convection system under extended 100K epoch training. Results are reported as mean MAE across training runs 1D Convection (β)R3 (Xavier Init.)R3 (LIGO Init.) 400.00840.0066 500.01460.0105 600.20860.0228 Average MAE0.07720.0133 C.5 Evaluation across baseline 1D Convection.For the 1D convection system, we evaluate all methods under progressively harder extrapolation regimes with convection coefficientsβ ∈ 40, 50, 60, 70, 80. In addition to the mean prediction error, Table 13 also reports the variance across independent runs to characterize optimization stability under increasing transport difficulty. As the convection coefficient increases, 25 standard PINN variants exhibit larger performance degradation and higher variance, indicating unstable convergence dynamics. In contrast, LIGO-PINN maintains consistently low error and reduced variance across most extrapolation settings, demonstrating improved robustness in convection- dominated regimes. 2D Helmholtz. For the 2D Helmholtz system, we investigate extrapolation to high-frequency oscillatory regimes parameterized by(a 1 ,a 2 ). These regimes introduce increasingly complex spatial oscillations that are known to challenge standard PINN optimization. Table 14 reports both the mean and variance of the MAE across independent runs to evaluate reconstruction quality and convergence stability. The results show that baseline PINN methods experience substantial instability and variance in higher-frequency settings, whereas LIGO-PINN consistently achieves lower error with significantly more stable convergence behavior across both isotropic and anisotropic wave propagation regimes. 2D Navier-Stokes (Forward Problem).For the 2D incompressible Navier-Stokes system, we eval- uate the forward pressure reconstruction problem across Reynolds numbers Re∈600, 800, 1000. In this setting, the pressure field is recovered without direct supervision, making optimization particu- larly challenging in high Reynolds number regimes with complex vortex dynamics. Table 15 reports both the mean and variance of the pressure MAE across independent runs to analyze convergence stability in this multi-physics setting. The results show that LIGO-PINN achieves substantially lower prediction error and reduced variability compared to existing baselines, indicating more stable and reliable convergence behavior in complex fluid dynamics regimes. Table 13: Mean Absolute Error (MAE) on 1D convection for extrapolation tasks (β ∈ 40, 50, 60, 70, 80). Values report the mean MAE across 5 random seeds, with corresponding standard deviation in parentheses. Method4050607080 PINN (fixed)0.7433 (0.1522) 0.7337 (0.2567) 0.8720 (0.1020) 0.7671 (0.1478) 0.8253 (0.1761) PINN (dynamic)0.0095(0.0050)0.0148(0.0050)0.0881(0.1534)0.1597(0.1988)0.1699(0.2072) Curr-Reg.0.0801 (0.0074) 0.2593 (0.2183) 0.3215 (0.0845) 0.3726 (0.0272) 0.4072 (0.0248) R30.0162 (0.0125) 0.1477 (0.1823) 0.3557 (0.0138) 0.4327 (0.0334) 0.4151 (0.0166) Meta-PDE0.1588 (0.1405) 0.2112 (0.1270) 0.3757 (0.1548) 0.3570 (0.0214) 0.5167 (0.1058) HyperLR0.4008 (0.0250) 0.4584 (0.0309) 0.4912 (0.0261) 0.5359 (0.0248) 0.5574 (0.0253) LIGO-PINN w/o (GLO) 0.0126 (0.0030) 0.0181 (0.0069) 0.0149 (0.0035) 0.0377 (0.0365) 0.0524 (0.0330) LIGO-PINN (Ours)0.0072(0.0012)0.0119(0.0032)0.0137(0.0024)0.0179(0.0050)0.0871(0.1466) Table 14: Mean Absolute Error (MAE) on the 2D Helmholtz PDE across varyinga xy in the extrap- olation setting. Values report the mean MAE across random seeds, with corresponding standard deviation reported in parentheses. Methoda 44 a 45 a 55 a 46 a 56 a 66 PINN (fixed)0.1135 (0.0757) 0.2276 (0.0993) 0.5931 (0.3411) 0.3447 (0.2861) 0.7396 (0.3906) 1.2283 (0.5588) PINN (dynamic)0.1615(0.2152)0.3956(0.4806)0.3186(0.1753)0.3185(0.1759)0.8444(1.0390)0.4784(0.2969) Curr-Reg.0.3080 (0.4643) 0.1630 (0.1448) 0.8223 (1.3994) 0.1331 (0.1515) 0.4177 (0.1481) 0.6520 (0.5460) R30.4910 (0.1544) 0.4192 (0.0491) 0.5143 (0.1766) 0.4786 (0.1512) 0.4550 (0.0669) 0.3968 (0.0003) Meta-PDE0.2473 (0.1541) 0.4736 (0.3218) 0.5982 (0.3420) 0.4954 (0.3495) 0.4077 (0.0092) 0.4329 (0.0578) HyperLR0.0613 (0.0317) 0.0435 (0.0140) 0.1599 (0.1437) 0.1570 (0.1007) 0.7585 (0.8403) 2.9437 (0.8651) LIGO-PINN w/o (GLO) 0.0014 (0.0004) 0.1604 (0.2161) 0.1657 (0.2195) 0.1617 (0.2148) 0.1625 (0.2141) 0.2134 (0.2951) LIGO-PINN (Ours)0.0007 (0.0002) 0.0017 (0.0004) 0.0031 (0.0003) 0.0032 (0.0007) 0.0048 (0.0010) 0.0800 (0.1600) C.6 Ablation Analysis on 1D Convection and 2D Helmholtz We extend the ablation analysis of LIGO-PINN to additional PDE domains, namely 1D Convection and 2D Helmholtz. As in the main text, we evaluate the effect of Invariance Encoding (IE) and Gated Layer-wise Optimization (GLO) using the ablation variants: ‘LIGO-PINN w/o (IE,GLO)’ (random initialization) and ‘LIGO-PINN w/o GLO’ (MAML-style initialization). (i) Performance Trends. Across both PDE domains, random initialization (‘w/o (IE,GLO)’) results in higher error and larger variance, indicating unstable convergence behavior. Incorporating IE (‘w/o GLO’) improves performance and reduces variance relative to random initialization, demonstrating the benefit of task-aware initialization. However, the absence of GLO still leads to suboptimal 26 Table 15: Pressure error (MAE) for 2D Navier–Stokes across Reynolds numbers. Values are mean with standard deviation in parentheses. Method6008001000Avg PINN (fixed)0.0183 (0.0135)0.1934 (0.3643)0.0257 (0.0188)0.0791 PINN (dynamic)0.0239 (0.0314)0.0958 (0.2027)0.0353 (0.0663)0.0517 Curr-Reg.0.3011 (0.3055)0.2597 (0.2683)0.2309 (0.2555)0.2639 R30.3519 (0.2572)0.4966 (0.3249)0.2620 (0.2021)0.3702 Meta-PDE0.3172 (0.0400)0.2997 (0.0262)0.2812 (0.0342)0.2994 HyperLR0.2155 (0.0594)0.1253 (0.0022)0.1910 (0.0273)0.1773 LIGO-PINN w/o (GLO)0.1510 (0.1859)0.1941 (0.1773)0.0288 (0.0302)0.1246 LIGO-PINN (Ours)0.0164 (0.0187)0.0065 (0.0011)0.0058 (0.0001)0.0096 506070 Task 10 2 Absolute Error LIGO-PINN w/o (IE,GLO) LIGO-PINN w/o (GLO) LIGO-PINN (a) 1D Convection (MAE). a44a55a66 Task 10 3 10 2 10 1 10 0 Absolute Error LIGO-PINN w/o (IE,GLO) LIGO-PINN w/o (GLO) LIGO-PINN (b) 2D Helmholtz (MAE). Figure 9: Empirical (MAE) comparison across ablation variants on 1D Convection and 2D Helmholtz. LIGO-PINN achieves lower error and reduced performance variability across both domains. convergence, particularly as task complexity increases. Full LIGO-PINN, combining IE and GLO, consistently achieves lower error and tighter error distributions, indicating improved training stability and robustness. 050100150200250 Frequency 10 2 10 1 10 0 10 1 10 2 Log Amplitude LIGO-PINN w/o (IE,GLO) LIGO-PINN w/o (GLO) LIGO-PINN (a) 1D Convection (β = 70). 010203040506070 Frequency 10 1 10 0 10 1 10 2 Log Amplitude LIGO-PINN w/o (IE,GLO) LIGO-PINN w/o (GLO) LIGO-PINN (b) 2D Helmholtz (a 1 = 6,a 2 = 6). Figure 10: Spectral distribution of absolute error for LIGO-PINN and ablation variants. Lower spectral density indicates better reconstruction of both low- and high-frequency components. (i) Spectral Bias Analysis. We analyze spectral bias by computing the Fourier transform of the absolute error fields. Across both 1D Convection and 2D Helmholtz, ablation variants exhibit higher spectral density, particularly in the high-frequency regime, indicating poor capture of fine-scale features. In contrast, LIGO-PINN consistently achieves lower spectral density across both low and high frequencies, demonstrating improved representation of complex solution structures. This confirms that the combination of invariance encoding and gated layer-wise optimization effectively mitigates spectral bias beyond what is achieved by either component alone. C.7 Sensitivity Analysis of LIGO-PINN to Number and Range of IE Tasks We further analyze the sensitivity of LIGO-PINN to two factors during the Invariance Encoding (IE) phase: (i) the number of sampled tasksk, and (i) the range of tasks used. For fairness, the baseline task ranges in the main paper were selected to align with prior work Krishnapriyan et al. [2021], which we also compared against through the Curr-Reg. baseline. Sensitivity to Number of Tasksk.We variedk ∈2, 3, 4during meta-training, withk = 3used in the main paper. Evaluation focused on the challenging extrapolation regimes of the 1D convection PDE,β ∈50, 60, 70. Table 16 reports the median MAE across random seeds. The results show 27 that performance remains relatively stable across different values ofk, withk = 3achieving the best median MAE on β = 50 and β = 70, while all settings remain in a similar range. Table 16: Sensitivity of LIGO-PINN to the number of sampled taskskduring the IE phase. We report the median mean absolute error (MAE) across random seeds for 1D Convection extrapolation tasks. 1D Convection (β) k = 2 k = 3 k = 4 500.015350.012850.01525 600.014900.016450.01600 700.027250.016900.01730 Sensitivity to Task Range. We further test the robustness of LIGO-PINN by varying the range of easy tasks selected for IE. In addition to the main setup LIGO-PINN(5-25), we evaluate LIGO- PINN(10-30) and LIGO-PINN(15-35). The evaluation tasks remain the challenging extrapolation regimesβ ∈50, 60, 70. Table 17 reports the median MAE across random seeds. The results show that all LIGO-PINN variants achieve comparable performance across task ranges, suggesting that the method is not overly sensitive to the exact IE task range. Table 17: Sensitivity of LIGO-PINN to the task range used during the IE phase. We report the median mean absolute error (MAE) across random seeds for 1D Convection extrapolation tasks. 1D Convection (β)LIGO (5-25)LIGO (10-30)LIGO (15-35) 500.005350.014650.01520 600.017100.015050.02055 700.015800.025000.01350 Summary.These results suggest that LIGO-PINN is robust to both (i) the number of sampled tasks kand (i) the range of tasks selected during the IE phase. Across the tested settings, median MAE remains consistently low on challenging extrapolation tasks, indicating that the IE phase does not require highly precise tuning of either task count or task range. This supports the practical use of domain knowledge to select reasonable IE task ranges without sacrificing downstream extrapolation performance. 28

Instant research discovery