Paper deep dive
Kirchhoff-Inspired Neural Networks for Evolving High-Order Perception
Tongfei Chen, Jingying Yang, Linlin Yang, Jinhu Lü, David Doermann, Chunyu Xie, Long He, Tian Wang, Juan Zhang, Guodong Guo, Baochang Zhang
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/26/2026, 2:14:20 AM
Summary
The paper introduces the Kirchhoff-Inspired Neural Network (KINN), a novel architecture that models neural representation learning as the evolution of an intrinsic latent potential based on Kirchhoff's circuit laws. By utilizing Kirchhoff Neural Cells (KNC) and Cascaded Kirchhoff Blocks (CKB), the model explicitly decouples and encodes higher-order evolutionary components, improving performance in PDE solving and image classification tasks.
Entities (5)
Relation Signals (3)
Cascaded Kirchhoff Block → composedof → Kirchhoff Neural Cell
confidence 100% · the Cascaded Kirchhoff Block (CKB), which realizes layer-wise cascaded RC nodes
Kirchhoff-Inspired Neural Network → utilizes → Kirchhoff Neural Cell
confidence 100% · we instantiate KINN with the Kirchhoff Neural Cell (KNC)
Kirchhoff-Inspired Neural Network → evaluatedon → Darcy Flow
confidence 95% · We systematically evaluate KINN on neural operator learning... including Darcy Flow
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Deep learning architectures are fundamentally inspired by neuroscience, particularly the structure of the brain's sensory pathways, and have achieved remarkable success in learning informative data representations. Although these architectures mimic the communication mechanisms of biological neurons, their strategies for information encoding and transmission are fundamentally distinct. Biological systems depend on dynamic fluctuations in membrane potential; by contrast, conventional deep networks optimize weights and biases by adjusting the strengths of inter-neural connections, lacking a systematic mechanism to jointly characterize the interplay among signal intensity, coupling structure, and state evolution. To tackle this limitation, we propose the Kirchhoff-Inspired Neural Network (KINN), a state-variable-based network architecture constructed based on Kirchhoff's current law. KINN derives numerically stable state updates from fundamental ordinary differential equations, enabling the explicit decoupling and encoding of higher-order evolutionary components within a single layer while preserving physical consistency, interpretability, and end-to-end trainability. Extensive experiments on partial differential equation (PDE) solving and ImageNet image classification validate that KINN outperforms state-of-the-art existing methods.
Tags
Links
- Source: https://arxiv.org/abs/2603.23977v1
- Canonical: https://arxiv.org/abs/2603.23977v1
Trouble viewing inline? Open PDF directly →
Full Text
55,160 characters extracted from source content.
Expand or collapse full text
Kirchhoff-Inspired Neural Networks for Evolving High-Order Perception Tongfei Chen 1 , Jingying Yang 1 , Linlin Yang 2 , Jinhu L ̈u 6 , David Doermann 4 , Chunyu Xie 5 , Long He 1 , Tian Wang 1 , Juan Zhang 1 , Guodong Guo 3 , Baochang Zhang 1 1* School of Artificial Intelligence (Institute of Artificial Intelligence), Beihang University, Beijing, China. 2 Communication University of China, Beijing, China. 3 Eastern Institute of Technology, Ningbo, Ningbo, China. 4 Department of Computer Science and Engineering, University at Buffalo, New York, Buffalo, USA. 5 360 AI Research, Qihoo 360, Beijing, China. 6 School of Automation Science and Electrical Engineering, Beihang University, Beijing, China. Abstract Deep learning architectures are fundamentally inspired by neuroscience, particu- larly the structure of the brain’s sensory pathways, and have achieved remarkable success in learning informative data representations. Although these architec- tures mimic the communication mechanisms of biological neurons, their strategies for information encoding and transmission are fundamentally distinct. Biological systems depend on dynamic fluctuations in membrane potential; by contrast, con- ventional deep networks optimize weights and biases by adjusting the strengths of inter-neural connections, lacking a systematic mechanism to jointly characterize the interplay among signal intensity, coupling structure, and state evolution. To tackle this limitation, we propose the Kirchhoff-Inspired Neural Network (KINN), a state-variable-based network architecture constructed based on Kirchhoff’s current law. KINN derives numerically stable state updates from fundamental ordinary differential equations, enabling the explicit decoupling and encoding of higher-order evolutionary components within a single layer while preserv- ing physical consistency, interpretability, and end-to-end trainability. Extensive experiments on partial differential equation (PDE) solving and ImageNet image classification validate that KINN outperforms state-of-the-art existing methods. 1 arXiv:2603.23977v1 [cs.LG] 25 Mar 2026 Keywords: Kirchhoff-Inspired Neural Network, Deep Learning, Higher-order neural network, Partial differential equations Deep learning architectures have been inspired by neuroscience, particularly by hier- archical sensory processing [1, 2]. In biological systems, sensory pathways rely on continuous membrane-potential dynamics and long-range transmission via action potentials without decrement along axons [3, 4]. Importantly, temporal development is embedded in the evolving states of neural populations rather than being treated as an external index. At the population level, we therefore view neural information through three complementary aspects: intensity (firing strength), connection (interaction structure among units), and evolution (intrinsic temporal development) [5–7]. Existing deep networks, parameterized by learned weights and architectural con- nectivity, operate as largely instantaneous mappings over discrete inputs, and can effectively encode signal intensity and inter-unit coupling through feedforward trans- formations [8, 9]. Yet, as recognized in neuroscience, information in sensory pathways is encoded and transmitted through continuous membrane-potential dynamics across connected neurons, rather than as largely instantaneous mappings over discrete inputs [10, 11]. Consequently, in many contemporary feedforward and attention-based architectures, the evolutionary aspect of a signal—i.e., how its latent representa- tion changes across positions or steps—is commonly introduced through architectural devices such as positional encodings, attention masks, or gating mechanisms, rather than formulated as an intrinsic dynamical state variable [9, 12]. As a result, a sin- gle update is typically not explicitly designed to jointly capture intensity, interaction structure, and higher-order evolution, which can be particularly important for data governed by continuous physical dynamics such as PDEs. In this paper, we introduce the Kirchhoff-Inspired Neural Network (KINN), which leverages Kirchhoff circuit dynamics to define a neural architecture for modeling higher-order state evolution. Our KINN derives closed-form, numerically stable dis- crete updates from the underlying ordinary differential equations [13, 14], enabling the model to explicitly encode higher-order evolutionary structure induced by cascaded state evolution within a unified representation layer. The resulting framework remains interpretable and fully end-to-end trainable. Inspired by the view that neural computation is carried not only by instantaneous inputs but also by the continuous evolution of membrane potentials, we model rep- resentation learning as the evolution of an intrinsic latent potential rather than as a sequence organized by external positional heuristics. An RC node provides a min- imal physical template for this purpose: capacitance accumulates past input, while conductance regulates state relaxation and the update timescale [15]. Based on this mechanism, we formulate a Kirchhoff architecture in which the hidden potential serves as an internal carrier of evolution and external inputs enter as driving currents. Under zero-order-hold discretization, the resulting dynamics yield a closed-form recurrent update [13, 14]: one term retains the previous state, while the other injects the cur- rent input into the evolving state. In this way, evolution is represented as an intrinsic state-evolution variable. 2 This formulation naturally extends to higher-order state evolution through cas- caded RC nodes. A single RC node realizes a first-order state update, whereas cascading multiple cells yields a higher-order system with additional latent states. From a systems perspective, each added stage enriches the family of admissible tem- poral responses, allowing the model to represent more expressive patterns of state evolution. As a result, sensitivity to complex temporal variation is not introduced by externally appended positional mechanisms, but emerges directly from the model’s internal dynamics. This provides a compact and interpretable route to jointly rep- resent input injection, interaction structure, and higher-order temporal development within a unified framework. To validate the effectiveness of modeling temporal evolution as an intrinsic state- evolution variable, we instantiate KINN with the Kirchhoff Neural Cell (KNC), which represents an RC node, and the Cascaded Kirchhoff Block (CKB), which realizes layer-wise cascaded RC nodes for higher-order state evolution. The resulting architec- ture models the continuous evolution of latent representations under Kirchhoff circuit dynamics [15], enabling multi-order feature integration that captures zeroth-, first-, and higher-order evolutionary components within a single representational layer. We systematically evaluate KINN on neural operator learning, spatiotemporal dynamics prediction, and visual recognition tasks, including Darcy Flow, Shallow Water, Navier– Stokes, and ImageNet-1K [16]. Across these benchmarks, KINN delivers consistent improvements, achieving errors of 1.775× 10 −2 , 2.587× 10 −3 , and 9.875× 10 −3 on Darcy Flow, Shallow Water, and Navier–Stokes, respectively, as well as Top-1 accura- cies of 83.3% (Tiny) and 83.9% (Small) on ImageNet-1K. Taken together, these results indicate that modeling evolution as an intrinsic state variable and elevating represen- tational order through cascade composition provide a unified, stable, and interpretable route to higher-order evolution modeling. Results Cascaded Kirchhoff Neural Cells realize higher-order state evolution This subsection examines the central premise of KINN that temporal evolution is rep- resented as an intrinsic latent-state variable and that higher-order temporal structure emerges through cascade composition. We show that a single Kirchhoff Neural Cell realizes a first-order state-evolution process governed by Kirchhoff-inspired RC dynam- ics and admits a stable discrete implementation under zero-order hold. We further show that cascading such cells systematically increases the effective order of the under- lying dynamical system, thereby providing an explicit and interpretable mechanism for modeling higher-order temporal structure. We first examine how temporal evolution can be represented as an intrinsic state variable, rather than supplied through externally injected positional informa- tion [17, 18]. To this end, we instantiated the proposed Kirchhoff Neural Cell (KNC) and examined its behavior from three complementary perspectives: continuous-time circuit dynamics, discrete-time neural realization, and cascade-induced system com- position. At the single-cell level, KNC models the hidden representation as a latent 3 Kirchhoff Neural Cell realizes first- order state-evolution g. f. e. d. c.b.a. h. ﹘82.8% E r r o r ( D a r c y F l o w ) [ R M S E 1 0 - 2 ] E r r o r ( P o i s s i o n ) [ R M S E % ] E r r o r ( N a v i e r - S t o k e s ) [ R M S E 1 0 - 2 ] E r r o r ( S h a l l o w W a t e r ) [ R M S E 1 0 - 3 ] A c c u r a c y ( I m a g e N e t - 1 K ) [ T o p - 1 A c c % ] ﹘21.7% ﹘27.0% ﹘72.7% + 1.2% KNC Baseline KNC Baseline KNC Baseline KNC Baseline KNC Baseline f.g. Fig. 1 KINN models temporal evolution as an intrinsic state variable and elevates representational order through cascade composition. a, Biological motivation at the neural- population level. A received neural signal can be decomposed into three complementary aspects: connection, indicating which upstream units contribute to the signal; intensity, indicating its instan- taneous or short-time magnitude; and evolution, indicating its continuous temporal development. b, In many contemporary architectures, order information is commonly introduced through posi- tional encoding, in which positional cues are externally injected into discrete feature cells rather than emerging from an internal evolving state. c, Recurrent models introduce a latent state, but a single hidden-state transition typically realizes only shallow first-order state evolution. d, KINN addresses this limitation through cascaded Kirchhoff circuit processing, where multi-stage internal state transi- tions progressively enrich temporal dynamics and enable higher-order evolution. e, A single Kirchhoff Neural Cell (KNC) realizes a first-order state-evolution process under Kirchhoff-inspired RC dynam- ics, in which latent voltage, input injection, leakage dissipation, and coupling modulation jointly determine the state update and readout. f, The full architecture instantiates KINN from key modules, combining KNC and the Cascaded Kirchhoff Block (CKB) to integrate zeroth-, first-, and higher- order evolutionary components within a unified representational hierarchy. g, We evaluate KINN across three task families spanning neural operator learning (Darcy Flow), spatiotemporal dynam- ics prediction (Shallow Water and Navier–Stokes), and visual recognition (ImageNet-1K). h, KINN yields consistent gains over strong baselines across these domains, supporting the view that intrinsic state evolution and cascade-induced order elevation provide a unified, stable, and interpretable route to higher-order evolution modelling. voltage state governed by Kirchhoff-inspired RC dynamics [15, 19]: C dv(t) dt =−(G leak + G p )v(t) + B p u(t),(1) where C denotes the effective capacitance, G leak and G p denote the leakage and coupling conductances, and B p u(t) is the input-driven injection term. Defining α≜ G leak + G p C , β≜ B p C , 4 the dynamics is equivalently written as a first-order state-evolution process [20, 21], ̇ v(t) =−αv(t) + β u(t), showing that the latent evolution is jointly controlled by state relaxation and input injection. To deploy this mechanism in neural computation, we discretize the continuous dynamics under zero-order hold, which yields a closed-form recurrent update [13]: v t+1 = e −α∆t t v t + Z ∆t t 0 e −α(∆t t −τ ) βu t dτ = e −α∆t t v t + β 1− e −α∆t t α u t .(2) This update provides a stable, interpretable state-evolution rule in discrete form: the exponential factor controls retention of past states, whereas the second term injects the current stimulus into the evolving latent representation. Each KNC then emits an output via an explicit readout of the updated state and the current input, and this output, rather than the internal state itself, is propagated to the next stage. We next asked whether composing such state-update/readout units increases the effective order of the end-to-end input-output dynamics. Eliminating the internal states from the cascade yields an end-to-end operator form for the final-stage output: n Y ℓ=1 C ℓ d dt + a ℓ y n (t) = n Y ℓ=1 d o,ℓ C ℓ d dt + d o,ℓ a ℓ + c o,ℓ b ℓ u(t),(3) where u(t) is the external input, y n (t) is the output of the final stage, and c o,ℓ ,d o,ℓ are fixed readout coefficients of the ℓ-th KNC. Equation (3) makes the higher-order nature of the cascade explicit: the left-hand side is the product of n first-order differential operators acting on the final output, showing that recursive composition progressively elevates the effective order of end-to-end evolution; in the generic non-degenerate case, the resulting mapping is n-th order [20, 21]. Thus, within the cascade itself, higher-order evolutionary sensitivity is not manually appended as an external posi- tional heuristic, but emerges from the internal composition of state-evolution units. Consistent with this view, increasing cascade depth led to progressively stronger evo- lution modeling capacity and improved downstream performance (see Sec. ??. The full derivation is provided in Methods. The cascaded Kirchhoff block implements multi-order evolution in neural networks We next asked how the Kirchhoff-inspired state-evolution principle can be realized in a practical neural module. Figure 2 provides a direct visual correspondence from the theoretical formulation to the implemented architecture: the single-cell schematic illustrates a Kirchhoff Neural Cell (KNC) as the basic state-evolution unit, the 5 Linear Linear Norm KNC SiLU SiLUDSConv N-order CKO KNC KNC Cascaded Kirchhoff Block Element-wise Multiplication Element-wise Addition LiftDescen d Act Spectral Act Conv Act SpectralConv Act SpectralConv SpectralCKB FNO with KINN Descen d Down Conv E n c o d e r Down Conv E n c o d e r Conv D e c o d e r Up Conv D e c o d e r Up CKB Conv D e c o d e r Up CKB Conv D e c o d e r Up Lift Down CKB Conv E n c o d e r Down CKB Conv E n c o d e r CKB UNet with KINN )(tu CB/ p )(tv R C G 1 )( pleak )(ty Kirchhoff Neural Cell Kirchhoff Neural Cell e.c.a. b. d. Fig. 2 Architectural instantiation of KINN and its integration into neural operators and encoder–decoder backbones. a, Structure of a single Kirchhoff Neural Cell (KNC). The hidden representation is modeled as a latent voltage state v(t) governed by Kirchhoff-inspired RC dynamics, where input injection, leakage dissipation, and state retention jointly determine the cell update and readout. b, Cascaded Kirchhoff Block (CKB). By stacking multiple KNCs in sequence, the output of each stage is propagated to the next stage, progressively increasing the effective order of the resulting dynamics and enabling higher-order state evolution. c, N -order Cascaded Kirch- hoff Operator (CKO). KNCs are combined with lightweight projection, normalization and nonlinear transformation modules to form a trainable high-order evolution operator for deep architectures. d, FNO with KINN. The proposed Kirchhoff modules are incorporated into Fourier neural operators by inserting CKB-enhanced evolution pathways alongside spectral and convolutional transformations, enabling multi-order feature interaction in operator learning. e, U-Net with KINN. CKB is further embedded into the encoder–decoder hierarchy, where Kirchhoff-inspired evolution modules are placed along the downsampling, bottleneck and upsampling paths to enrich latent dynamics across scales. Together, these instantiations show that KINN is not restricted to a single backbone, but provides a unified and modular mechanism for introducing intrinsic and higher-order state evolution into diverse architectures. cascaded-cell schematic shows how serial composition gives rise to higher-order evo- lution, and the CKB diagram shows how this principle is instantiated as a trainable neural block. At the single-cell level, the continuous-time KNC is implemented as a selective discrete state-update/readout unit. In this implementation, the relaxation, input- injection, and readout roles derived in Eqs. (9)–(11) are preserved, while part of the effective coefficients are made adaptive to the current feature. In this way, the implemented KNC remains faithful to the theoretical relaxation–injection–readout interpretation while gaining the flexibility required for neural representation learning. At the block level, the figure further shows how higher-order evolution is realized by cascading multiple KNCs. Each KNC contributes one state-evolution step, and their serial composition forms the neural counterpart of the higher-order cascade derived in Eqs. (15)–(16). To preserve information from different evolutionary depths, the 6 outputs of all stages are aggregated into a unified multi-order representation, ̄ y = N X k=1 y (k) ,(4) where y (k) denotes the output of the k-th KNC. This aggregation retains shallow, intermediate, and deep evolutionary responses simultaneously, rather than exposing only the deepest cascaded state. In parallel, the block also preserves the current input through a direct input- conditioned modulation path, which serves as an explicit zero-order feature pathway. The final block output is obtained by combining this zero-order path with the aggregated higher-order response through gated residual fusion: Y = X + ̄ y⊙ g,(5) where g is generated directly from the current input. Therefore, the resulting CKB jointly realizes two complementary aspects of representation: zero-order feature preser- vation from the current input, and higher-order feature aggregation from cascaded state evolution. Consistent with this design, replacing conventional single-step tran- sition blocks with CKB led to stronger temporal modeling capacity and improved downstream performance. Full implementation details are provided in the Methods section. Empirical validation across physical field solving benchmarks Darcy Flow Prediction with KINN Methods. To evaluate the model’s ability to learn complex mappings from high- dimensional parameter fields to solution fields, we first focus on the Darcy flow, a classic steady-state elliptic PDE. Unlike temporal forecasting, this task requires the model to infer the global pressure distribution [22] u(x) from a spatially heterogeneous permeability field a(x), governed by −∇· (a(x)∇u(x)) = f (x). Mathematically, the solution at any spatial point is coupled with the global distribu- tion of the permeability field, necessitating a model with high-order spatial reasoning and extensive receptive fields. Our proposed architecture, termed CKB, is built upon the Unet, which has demon- strated superior performance in learning solution operators of PDEs by maintaining continuous-discrete equivalence (CDE) [23]. The CKB model retains the hierarchical Operator U-Net structure but introduces two key structural enhancements to optimize computational efficiency and physical modeling accuracy. • Depthwise Separable Convolutions (dsconv): To optimize parameter utiliza- tion and computational efficiency without sacrificing model capacity, we replace all standard 3× 3 convolutions in the baseline CNO (including those in the encoder, 7 decoder, and residual blocks) with depthwise separable convolutions [24]. This mod- ification significantly reduces the FLOPs and parameter count while maintaining the local receptive fields necessary for capturing multi-scale physical features. • Second-Order Kirchhoff Neural Components (KNC): We seamlessly inte- grate the proposed KNC modules into the deepest two layers of both the encoder and the decoder, as well as the bottleneck layer. These layers correspond to the lowest spatial resolutions and the largest receptive fields, where the network cap- tures macro-scale global dynamics. By embedding KNC at these strategic locations, the model explicitly incorporates second-order physical priors and conservation constraints directly into the latent feature space, enhancing the approximation of complex PDE evolution laws. For the Darcy flow dataset, we adopt a U-Net–based neural operator architecture, shown in Fig. 2(e), and integrate the proposed CKB modules to enhance spatial mod- eling. The network follows an encoder–decoder structure. The input field is first lifted to a latent representation and then processed through a multi-stage encoder that pro- gressively reduces spatial resolution while increasing channel capacity. CKB modules are introduced in the deepest encoder and decoder stages, enabling structured spatial scanning and directional state evolution on high-level features. After multi-scale fea- ture fusion through U-Net skip connections, the reconstructed latent representation is projected back to the physical field to produce the predicted solution. Our proposed CKB architecture naturally aligns with this requirement through two distinct mechanisms. First, the cascaded high-order dynamics derived in our theory (Eq. 16-18) allow the network to approximate the inverse of the Laplacian-like oper- ator more effectively than standard first-order models. Second, the quad-directional selective scan ensures omnidirectional information propagation across the 2D grid, mimicking the physical dissipation of pressure in a porous medium. Results. Quantitative results demonstrate that our Kirchhoff-inspired approach significantly outperforms established baselines. As shown in Table 1, our model achieves a normalized relative L 2 error (nRMSE) of 1.775× 10 −2 , which represents a 4.5× reduction in error compared to U-Net (7.993× 10 −2 ) and a 6.4× reduction compared to the Fourier Neural Operator (FNO, 1.142 × 10 −1 ). Furthermore, the maximum error is reduced to 6.593× 10 −2 , nearly a 3× improvement over baseline methods, indicating superior local fidelity. Visual analysis confirms that while base- lines often exhibit structural blurring or artifacts in high-gradient regions, our method maintains sharp boundaries and recovers the principal dynamical modes with high precision, validating the efficacy of physics-consistent state evolution in steady-state operator learning. Shallow-Water Forecasting with KINN Methods. To evaluate the model’s capability in capturing non-stationary spatiotem- poral evolution, we apply our method to the Shallow Water equations (SWE) dataset. Unlike steady-state problems, SWE emphasizes long-horizon temporal evolution cou- pled with spatial transport, typically exhibiting wave propagation and nonlinear advection [25]. Traditional discrete deep learning models often struggle with such 8 Table 1 Quantitative evaluation on steady-state Darcy Flow. Predictive performance comparison on the 2D Darcy Flow dataset. nRMSE denotes the normalized relative L 2 error. The best results are highlighted in bold, demonstrating significant improvements across all metrics. MethodRMSEnRMSEMax Error U-Net1.545× 10 −2 7.993× 10 −2 1.930× 10 −1 FNO2.129× 10 −2 1.142× 10 −1 1.986× 10 −1 Ours (CKB)3.657× 10 −3 1.775× 10 −2 6.593× 10 −2 tasks, as truncation errors from discrete temporal updates rapidly accumulate, leading to structural divergence and numerical instability over long rollouts [26]. Our architecture addresses this bottleneck through the inherent physical con- sistency of the Kirchhoff Neural Cell (KNC). By adopting an exact exponential integration scheme [13] with zero-order hold (ZOH), the discrete state updates in our network are rigorously A-stable. This mathematical property ensures that the continuous-time physical dissipation and propagation are faithfully preserved in the discrete computational graph, effectively suppressing the spurious amplification of high-frequency errors during temporal iteration. We evaluate the proposed CKB module on the shallow-water equations (SWE) using a modified Fourier Neural Operator architecture (FKNO), illustrated in Fig. 2(d). The model follows a four-layer FNO backbone. The input consists of the pre- vious ten temporal states concatenated with spatial coordinates (x,y) and is first lifted to a latent representation. Each Fourier block combines a spectral convolution captur- ing global interactions with a local operator modeling spatial correlations. In the first block, the local operator is replaced by the proposed CKB module, which performs multi-directional spatial scanning and state-evolution transformations before feature aggregation. The resulting representation is finally projected back to the physical space to predict the next SWE state. Reslults. Quantitative comparisons against data-driven (U-Net, FNO [27]) and physics-informed (PINN [28]) baselines validate this theoretical advantage. As detailed in Table 2, while conventional convolution-based mapping (U-Net) struggles to maintain coherent wave structures (yielding an nRMSE of 9.120×10 −2 ), our Kirchhoff- inspired model achieves state-of-the-art predictive accuracy with an nRMSE of 2.587× 10 −3 , notably outperforming both the spectral interpolation of FNO (3.301×10 −3 ) and the soft-constrained PINN (1.336× 10 −2 ). Moreover, the maximum error is restricted to 4.958× 10 −2 . The bounded error accumulation confirms that the physics-compliant exponential updates act as a stabilizing inductive bias, enabling the network to learn robust dynamical transitions rather than merely overfitting to short-term spatial correlations. CKB Order Ablation on Poisson Problems Methods. To explicitly validate our theoretical claim that cascading Kirchhoff Neural Cells induces higher-order dynamical constraints, we perform an ablation study on the cascade depth (n-pass) using the Poisson equation. The Poisson equation, governed 9 Table 2 Quantitative evaluation on 2D Shallow Water equations. Predictive performance comparison for temporal forecasting. nRMSE denotes the normalized relative L 2 error. Our method exhibits superior stability and accuracy compared to both data-driven and physics-informed baselines. MethodRMSEnRMSEMax Error U-Net9.500× 10 −2 9.120× 10 −2 6.503× 10 −1 PINN1.389× 10 −2 1.336× 10 −2 1.682× 10 −1 FNO3.436× 10 −3 3.301× 10 −3 5.464× 10 −2 Ours (CKB)2.692× 10 −3 2.587× 10 −3 4.958× 10 −2 by the Laplace operator (∆u = f ), intrinsically requires the modeling of second-order spatial derivatives. According to our derivations (Eq. 16-18), a single KNC behaves as a first-order operator, whereas cascading them analytically produces higher-order polynomial transfer functions that are better suited for higher-order PDEs. For the Poisson dataset, we investigate the influence of the CKB order within the UKNO architecture. The backbone follows the U-Net–based neural operator described above, where CKB modules are integrated into the deepest encoder–decoder stages. The standard configuration employs an CKB module composed of KNCs. To evaluate the impact of model capacity and hierarchical dynamics, we vary the number of KNCs within the CKB module. Specifically, we consider configurations with one, two, three, and four KNCs. For comparison, we also include the original U-Net architecture with- out CKB modules as a baseline. All other architectural settings, training procedures, and hyperparameters remain unchanged across the experiments. Results. We evaluate variants of our model from 1-pass (1m) to 4-pass (4m) architectures. As summarized in Table 3, increasing the cascade stages provides a systematic enhancement in predictive accuracy. The 1-pass model, constrained by its first-order nature, yields a baseline OUT relative L1 error of 0.307%. By stepping to a 2-pass cascade, which mathematically aligns with the second-order Laplacian operator, the nRMSE drops from 3.854× 10 −3 to 3.692× 10 −3 . When further extending the cascade to 4 passes, the relative L1 error achieves a global minimum of 0.268%, and the OUT nRMSE significantly decreases to 3.424× 10 −3 . Although we observe a slight optimization fluctuation in the mean error at 3 passes, the maximum error (Max Error) steadily decreases at higher orders, indi- cating that deeper cascades are particularly effective at resolving sharp, high-gradient local structures. These empirical results firmly corroborate our mathematical formu- lation: the performance gain of the Cascaded Kirchhoff Block (CKB) is not merely a consequence of increased parameter count, but a direct result of aligning the neural network’s intrinsic differential order with the underlying physical governing equations. Rollout Horizon Evaluation on Navier–Stokes Dynamics Methods. To further evaluate the temporal stability of the proposed model, we con- duct long-horizon forecasting experiments on the 2D incompressible Navier–Stokes equations in the vorticity formulation with viscosity ν = 10 −3 . This benchmark is widely used to assess the long-term predictive robustness of neural operator models. 10 Table 3 Ablation on cascade stages (n-pass) for the Poisson equation. We compare the relative L1 error, nRMSE, and maximum error on the OUT field. Parameters (Params) and inference times indicate that the multi-pass mechanism maintains high efficiency while scaling the operator’s mathematical order. VariantParams (M)Rel. L1 Error (%)OUT nRMSEOUT Max Error 1-pass (1m)2.600.3073.854× 10 −3 1.074× 10 −2 2-pass (2m)2.830.2923.692× 10 −3 1.033× 10 −2 3-pass (3m)3.070.3904.791× 10 −3 8.764× 10 −3 4-pass (4m)3.310.2683.424× 10 −3 9.403× 10 −3 The Navier–Stokes dynamics exhibit strongly nonlinear convection–diffusion inter- actions and multi-scale vortex structures, making long-term forecasting particularly challenging. Small prediction errors may accumulate and amplify over recursive time steps, eventually leading to structural divergence. To systematically analyze this sta- bilization effect, we evaluate the models over a rigorous 40-step rollout under varying training horizons: 3-step, 5-step, and 10-step temporal unrolling. Results. As detailed in Table 4, our method consistently outperforms the standard Fourier Neural Operator (FNO) across all training strategies under matched parameter budgets ( 470K parameters). For instance, under the 10 → 1 training strategy, our model achieves a rollout relative L 2 error of 1.054×10 −2 , establishing a clear advantage over FNO (1.349× 10 −2 ). The superiority of the CKB architecture becomes even more pronounced in the temporal error trajectories. As observed in the rollout curves, while all models expe- rience an initial error descent followed by gradual accumulation , baseline models exhibit an accelerated error growth phase at later stages (t > 20). In contrast, our model maintains a significantly flatter error trajectory throughout the entire 40-step horizon. Pixel-level error analysis further reveals that baselines suffer from severe error spikes during highly nonlinear events, such as vortex merging. Our approach effectively suppresses these fluctuations, preserving sharper structural boundaries in high-shear regions without suffering from high-frequency spectral distortion. These findings cor- roborate that the cascaded Kirchhoff dynamics provide a robust, physics-aligned inductive bias for long-term spatiotemporal forecasting. Table 4 Long-term rollout performance on the 2D Navier-Stokes equations. We report the average relative L 2 error over a 40-step rollout. Models are evaluated across varying training sequence lengths (from 3-step to 10-step). Our CKB-based model consistently exhibits lower error accumulation and superior stability across all settings under comparable parameter budgets. Method Rollout Relative L 2 Error across Training Horizons 3-step (3→ 1)5-step (5→ 1)10-step (10→ 1) FNO2.226× 10 −2 1.814× 10 −2 1.349× 10 −2 Ours (CKB)1.533× 10 −2 1.449× 10 −2 9.875× 10 −3 11 Directional Scan Ablation on Navier–Stokes Dynamics Methods. For the Navier–Stokes dataset, we examine the impact of directional spatial scanning within the CKB module of the FKNO architecture. The backbone follows the same FNO-based neural operator described in the SWE experiments, where the first local operator is replaced by the proposed CKB module. The CKB module performs structured spatial traversal followed by state-evolution transformations and directional feature aggregation. To analyze the role of directional scanning, we vary the number of traversal directions used in the spatial scanning stage. Specifically, three configurations are evaluated: a single-direction scan using a raster horizontal traversal, a two-direction scheme combining horizontal and vertical scans, and a four-direction configuration employing both forward and reverse horizontal and vertical traversals. In addition, the original FNO model without the CKB module is included as a baseline. All other architectural components, training settings, and hyperparameters remain unchanged across the experiments. Results. As summarized in Table 6, the empirical results on the 64× 64 resolution NS benchmark reveal a clear correlation between directional diversity and predictive accuracy. The baseline FNO yields a relative L 2 error of 1.349× 10 −2 . By introducing even a single-directional Kirchhoff scan (FKNO-1), the error is reduced to 1.043× 10 −2 , demonstrating the superior inductive bias of the Kirchhoff-inspired exponential integration over standard spectral convolutions. More importantly, increasing the scanning complexity further refines the results. The FKNO-4 model achieves the optimal performance with a test L 2 error of 9.875× 10 −3 , outperforming the FKNO-2 (1.026× 10 −2 ) and FKNO-1 variants. This progression confirms that the highly non-linear advection and vortex stretching inher- ent in Navier-Stokes flows require the model to capture spatial dependencies from multiple orientations. While the computational time increases slightly with more direc- tions (from 54.1s for FNO to 83.3s for FKNO-4), the significant gain in precision justifies the multi-directional design. This ablation empirically validates that the four- way scanning mechanism provides a more isotropic and robust representation of the underlying fluid velocity fields. Table 5 Ablation of scanning directions on Navier-Stokes equations. We evaluate the impact of the number of Kirchhoff scanning directions (FKNO-n) on the 64× 64 resolution dataset (10→ 1 time-step prediction). L 2 denotes the relative full-field error. Model VariantDirectionsParametersTraining Time (s)Test L 2 Error Vanilla FNO (Baseline)-465,71754.121.349× 10 −2 FKNO-11476,69871.481.043× 10 −2 FKNO-22481,01879.371.026× 10 −2 FKNO-4 (Ours)4489,65883.269.875× 10 −3 Generalization to large-scale natural image recognition To demonstrate that the physical inductive biases introduced by the Kirchhoff Neural Cell (KNC) and Cascaded Kirchhoff Block (CKB) are not limited to solving explicit 12 physical partial differential equations (PDEs), we evaluate our architecture on the ImageNet-1K image classification benchmark. In the context of computer vision, map- ping raw pixels to high-level semantic labels can be viewed as the continuous evolution of a 2D spatial feature field [29]. Consequently, the high-order receptive fields and sta- ble gradient propagation derived from our physical constraints theoretically provide a superior backbone for visual representation learning. We integrate the CKB as a drop-in replacement into a standard hierarchical vision backbone. Table 6 compares our models (Ours-T and Ours-S) against leading Con- volutional Neural Networks (ConvNeXt [30]), Vision Transformers (Swin [31]), and recent State-Space Models (VMamba [32], MambaVision [33]). At the Tiny scale ( 32M parameters, 4.9G FLOPs), our model achieves a Top-1 accuracy of 83.3%, outper- forming strong SSM baselines like VMamba-T (82.6%) and MambaVision-T (82.3%). Scaling up to the Small size ( 50M parameters, 8.2G FLOPs), Ours-S reaches 83.9% Top-1 accuracy, establishing a new state-of-the-art Pareto frontier for accuracy and computational complexity. Table 6 ImageNet-1K classification performance. Comparison of our Kirchhoff-inspired backbone with state-of-the-art CNNs, Vision Transformers, and visual State Space Models (SSMs). Our models achieve superior accuracy under strictly matched parameter and FLOP constraints. Model#Params (M)FLOPs (G)Top-1 Acc (%) ConvNeXt-T28.64.582.0 Swin-T28.34.481.3 EfficientVMamba-S11.01.378.7 VMamba-T30.04.982.6 MambaVision-T31.84.482.3 Ours-T32.34.983.3 ConvNeXt-S50.28.783.1 VMamba-S50.08.783.6 MambaVision-S50.17.583.2 Ours-S50.38.283.9 To trace the origins of these empirical gains back to our mathematical formu- lation, we conduct rigorous ablations on the module topology and the injection of physical constraints (Table 7). Starting from a baseline configuration (Tiny: 82.6%, Base: 83.3%), we evaluate two expansion strategies: a Parallel routing and a Series (Cascaded) routing. Crucially, the Series cascade yields a substantially higher performance jump (Tiny: +0.6%, Base: +0.9%) compared to the Parallel approach (Tiny: +0.3%, Base: +0.3%). This precisely validates the core tenet of our derivation (Eq. 16-18): a sequential cascade of first-order Kirchhoff cells analytically constructs a higher-order differential operator, which exponentially increases the spatiotemporal expressive capacity of the network, whereas a parallel combination merely acts as a wide first-order ensemble. Furthermore, incorporating the strict physical constraints (e.g., KCL-compliant decay 13 and exponential integration) provides an additional accuracy boost, confirming the utility of physical conservation laws in regularizing deep visual features. Finally, we observe that the physical constraints act as a powerful stabilizing prior during optimization. When evaluating the short-schedule training efficiency (Top-1 accuracy at 30 epochs), our models systematically outpace the VMamba baselines. Specifically, Ours-S achieves 72.5% at epoch 30, a full 1.0% ahead of VMamba-S (71.5%). This accelerated convergence indicates that the intrinsic A-stability and dis- sipative properties of the Kirchhoff dynamics facilitate smoother and faster gradient propagation across the loss landscape. Table 7 Ablation studies on ImageNet-1K. (Top) Impact of routing topologies (Parallel vs. Series cascade) and physical constraints on final accuracy. (Bottom) Training efficiency measured by Top-1 accuracy at 30 epochs, demonstrating faster convergence. Topological & Physical Constraint Ablation ScaleBaselineParallelSeries (Cascade)+ Physical Constraints Tiny82.682.983.283.3 Base83.383.684.284.3 Training Efficiency (Top-1 Acc at 30 epochs) Model-T (Tiny)-S (Small)-B (Base) VMamba70.271.569.4 Ours71.072.571.0 Discussion Recent advances in neural operators, state-space models, and modern vision architec- tures have shown that strong performance can emerge from increasingly expressive mechanisms for long-range interaction and hierarchical feature transformation. Yet, in many such formulations, evolution is introduced primarily as a computational device, rather than as an explicitly grounded state process. The present study departs from this trend by treating representation dynamics as the evolution of an endogenous latent potential governed by conservation and dissipation. In this sense, the proposed frame- work is less an incremental extension of existing sequence or operator models than an alternative modelling viewpoint, one in which stimulus, coupling, and temporal development are embedded within a unified physical state equation. From this perspective, the contribution of our work lies not only in introducing a new architectural block, but also in demonstrating that higher-order representation dynamics can be constructed through structured, cascaded evolution. Starting from Kirchhoff-consistent RC dynamics, the Kirchhoff Neural Cell provides an interpretable first-order primitive. At the same time, the cascaded design extends this primitive into a higher-order mechanism without abandoning its dissipative character. This yields a model family that is both analytically motivated and practically effective. The empir- ical results suggest that such a formulation is sufficiently general to support diverse 14 learning settings, including steady-state operator approximation, long-horizon spa- tiotemporal prediction, and large-scale visual recognition. Rather than relying solely on empirical architectural heuristics, the model benefits from an inductive bias that is physically constrained yet flexible in implementation. At a broader level, these findings indicate that physically motivated state evolu- tion may offer a useful route towards neural architectures that balance expressivity, stability, and interpretability. At the same time, the present results should be inter- preted with appropriate caution. Neural and circuit principles inspire the framework, but it is not intended as a biologically faithful account of neuronal computation. In addition, the current study explores only a limited set of cascade configurations and benchmark families. It therefore remains to be seen how far the same design princi- ple extends to more heterogeneous modalities, irregular dynamical regimes, or larger foundation-model settings. Future work may clarify whether adaptive order selection, richer circuit motifs, or tighter links between continuous-time analysis and discrete learning dynamics can further strengthen this line of inquiry. Methods Mathematical formulation of the Kirchhoff Neural Cell. We view representation learning as the evolution of an intrinsic latent potential v(t), analogous to a membrane-potential-like state. For a single Kirchhoff cell driven by an external input u(t), Kirchhoff’s current law gives i C (t) + i leak (t) + i couple (t) = i in (t),(6) where the branch currents are defined as i C (t) = C dv(t) dt , i leak (t) = G leak v(t), i couple (t) =−G p v(t), i in (t) = B p u(t). (7) Substituting Eq. (7) into Eq. (6) yields C dv(t) dt =−(G leak + G p )v(t) + B p u(t).(8) Defining α≜ G leak + G p C , β≜ B p C , the dynamics can be written in standard state form as ̇ v(t) =−αv(t) + β u(t),(9) which makes explicit that the latent evolution is jointly governed by state relaxation and input injection. To deploy this continuous-time mechanism in neural computation, we discretize Eq. (9) under zero-order hold (ZOH), assuming the input is piecewise constant within 15 each interval [t,t + ∆t t ]. This yields the exact update [13] v t+1 = e −α∆t t v t + Z ∆t t 0 e −α(∆t t −τ ) βu t dτ = e −α∆t t v t + β 1− e −α∆t t α u t .(10) This update provides a stable, interpretable discrete-state evolution rule: the exponen- tial factor controls retention of past states. At the same time, the second term injects the current stimulus into the evolving latent state. A KNC does not expose its internal state directly. Instead, its output is read out from both the updated state and the current input: y t = c o v t+1 + d o u t ,(11) where y t is the output voltage of the cell, and c o and d o are fixed readout coefficients. This separates internal evolution from external emission: the internal state stores and transforms latent dynamics, whereas the output is the quantity propagated to the next cell. For continuous-time analysis, we use the corresponding readout form y(t) = c o v(t) + d o u(t).(12) Higher-order evolution via the Cascaded Kirchhoff Block. Let y 0 (t) ≡ u(t) denote the external input. For the ℓ-th cell in a cascade, we define the state-update and output-readout equations as C ℓ dv ℓ (t) dt =−a ℓ v ℓ (t) + b ℓ y ℓ−1 (t),(13) y ℓ (t) = c o,ℓ v ℓ (t) + d o,ℓ y ℓ−1 (t), ℓ = 1,...,n.(14) Here, v ℓ (t) is the internal state of the ℓ-th KNC, y ℓ (t) is its output, and the next stage receives y ℓ (t) rather than v ℓ (t). This formalizes the intended KNC–CKB hierarchy: each cell first updates its latent voltage state and then emits an output voltage that drives the next cell. Eliminating the internal state from Eqs. (13) and (14) gives v ℓ (t) = y ℓ (t)− d o,ℓ y ℓ−1 (t) c o,ℓ . Substituting this into Eq. (13) yields the output-level operator recursion C ℓ d dt + a ℓ y ℓ (t) = d o,ℓ C ℓ d dt + d o,ℓ a ℓ + c o,ℓ b ℓ y ℓ−1 (t).(15) 16 Equation (15) shows that each stage induces a first-order operator relation between successive outputs. Applying Eq. (15) recursively across n stages yields the end-to-end operator form n Y ℓ=1 C ℓ d dt + a ℓ y n (t) = n Y ℓ=1 d o,ℓ C ℓ d dt + d o,ℓ a ℓ + c o,ℓ b ℓ u(t).(16) Eq. (16) makes the higher-order nature of the cascade explicit: the left-hand side is the product of n first-order differential operators acting on the final-stage output y n (t). Therefore, cascading first-order KNCs yields a progressively higher-order end-to-end evolution process; in the generic non-degenerate case, the resulting mapping is n-th order [20, 21]. This provides a principled mechanism for enriching temporal expres- siveness through recursive composition, rather than through externally appended positional heuristics. The formulation above clarifies three levels of structure in KINN. First, a sin- gle KNC implements a first-order state-evolution unit grounded in RC dynamics. Second, ZOH discretization turns these continuous dynamics into a closed-form recurrent update suitable for neural computation. Third, the CKB composes state- update/readout pairs across stages, so that higher-order temporal sensitivity emerges from the model’s internal recursive structure. Neural architecture of Kirchhoff Neural Cell and Cascaded Kirchhoff Block. The theoretical formulation above defines a Kirchhoff Neural Cell (KNC) as a state- evolution unit with three essential components: a relaxation term, an input-injection term, and a readout term. In continuous time, these roles are played by the coefficients α, β, c o , and d o in Eqs. (9)–(12). In neural implementation, we preserve the same state-update/readout structure, but instantiate part of these coefficients in a selective discrete form so that the effective dynamics can adapt to the current feature. Let the input feature be X ∈R B×L×d ,(17) where B is the batch size, L is the sequence length (or the flattened spatial length for 2D features), and d is the channel dimension. The block output is Y ∈R B×L×d ,(18) with the same shape as the input, so that the block can be seamlessly inserted into deep hierarchical backbones. Given the normalized input e X = Norm(X), the block first splits into an evolution branch and a gate branch: u = φ DSConv(W u e X) , g = φ(W g e X),(19) where W u and W g are learnable linear projections, DSConv(·) denotes depthwise separable convolution, and φ(·) denotes the SiLU activation. The evolution branch 17 provides the driving signal for state evolution, while the gate branch provides an input- dependent modulation path. The DSConv layer introduces lightweight local interaction before the dynamical update. For the discrete neural realization of a single KNC, we use v t to denote the latent state at step t, in direct correspondence with the continuous-time state v(t). The implemented state-update/readout rule is written as v t+1 = ̄ α t ⊙ v t + ̄ β t ⊙ u t , y t = c t ⊙ v t+1 + du t ,(20) where y t is the emitted output of the cell, ̄ α t is the effective retention coefficient, ̄ β t is the effective input-injection coefficient, c t is the readout coefficient, and d is a learned skip/readout parameter. This is the discrete implementation counterpart of Eqs. (9) and (11): ̄ α t corresponds to relaxation/retention, ̄ β t corresponds to input injection, and (c t ,d) correspond to state/input readout. To enhance expressive capacity, part of these effective coefficients are made input- dependent through learnable projections: ∆ t = softplus(W ∆ u t + b ∆ ), b t = W b u t , c t = W c u t ,(21) and the discretized coefficients are formed as ̄ α t = exp(−∆ t ⊙ λ), ̄ β t = 1− exp(−∆ t ⊙ λ) λ ⊙ b t ,(22) where λ is a learned decay parameter. In this parameterization, λ plays the role of the continuous-time decay rate, ∆ t acts as an input-dependent effective discretiza- tion timescale, and the combination exp(−∆ t ⊙ λ) is the selective implementation counterpart of the theoretical retention factor e −α∆t t in Eq. (10). Thus, the net- work implementation preserves the Kirchhoff-inspired relaxation–injection–readout interpretation while allowing the dynamics to adapt to the current feature. The evolution branch is then processed by a cascaded Kirchhoff operator (CKO) composed of serial KNCs. Denoting the k-th KNC byK θ k (·), the cascade is written as y (1) =K θ 1 (u), y (k) =K θ k y (k−1) , k = 2,...,N,(23) where N is the cascade depth. This realizes the discrete counterpart of the higher-order cascade derived in Eqs. (15)–(16): each KNC contributes a first-order state-evolution step, and serial composition progressively enriches the effective order of the end-to-end dynamics. To preserve shallow, intermediate, and deep evolutionary responses, we aggregate the outputs of all KNC stages into the final cascaded representation: ̄ y = N X k=1 y (k) ,(24) 18 where y (k) denotes the output of the k-th KNC in the cascade. Equivalently, this can be viewed as progressively adding all preceding stage responses into the final- stage pathway. Such dense aggregation preserves low-order and intermediate-order evolutionary cues while integrating the deepest response produced by the cascaded operator. The fused feature is then modulated by the gate branch through element-wise multiplication: by = ̄ y⊙ g,(25) and the final block output is obtained by residual addition: Y = X + by.(26) Therefore, the proposed CKB implements the theoretical KNC–CKB hierarchy in a practical neural form: a single KNC realizes Kirchhoff-inspired selective state evolu- tion, while the full block combines local preprocessing, cascaded multi-order evolution, dense cross-stage aggregation, input-aware gating, and residual fusion within a unified architecture. References [1] Fukushima, K. Neocognitron: A self-organizing neural network model for a mech- anism of pattern recognition unaffected by shift in position. Biological Cybernetics 36, 193–202 (1980). [2] Yamins, D. L. K. & DiCarlo, J. J. Using goal-driven deep learning models to understand sensory cortex. Nature Neuroscience 19, 356–365 (2016). [3] Purves, D. et al. in Long-distance signaling by means of action potentials 2 edn, Neuroscience (Sinauer Associates, Sunderland, MA, 2001). [4] Marzvanyan, A. & Alhawaj, A. F. Physiology, sensory receptors. StatPearls (2019). [5] Averbeck, B. B., Latham, P. E. & Pouget, A. Neural correlations, population coding and computation. Nature Reviews Neuroscience 7, 358–366 (2006). [6] Bassett, D. S. & Sporns, O. Network neuroscience. Nature Neuroscience 20, 353–364 (2017). [7] Vyas, S., Golub, M. D., Sussillo, D. & Shenoy, K. V. Computation through neural population dynamics. Annual Review of Neuroscience 43, 249–275 (2020). [8] Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, Vol. 25 (2012). 19 [9] Vaswani, A. et al. Attention is all you need. Advances in Neural Information Processing Systems, Vol. 30 (2017). [10] Debanne, D., Bialowas, A. & Rama, S. What are the mechanisms for analogue and digital signalling in the brain? Nature Reviews Neuroscience 14, 63–69 (2013). [11] Dayan, P. & Abbott, L. F.Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (MIT Press, Cambridge, MA, 2001). [12] Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Computation 9, 1735–1780 (1997). [13] Higham, N. J.Functions of Matrices: Theory and Computation (SIAM, Philadelphia, PA, 2008). [14] Gu, A., Goel, K. & R ́e, C. Efficiently modeling long sequences with structured state spaces. International Conference on Learning Representations (2022). [15] Chua, L. O., Desoer, C. A. & Kuh, E. S. Linear and Nonlinear Circuits (McGraw- Hill, New York, 1987). [16] Deng, J. et al. ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248–255 (IEEE, 2009). [17] Morita, T. Positional encoding helps recurrent neural networks handle a large vocabulary. arXiv preprint arXiv:2402.00236 (2024). [18] Zhao, L. et al. Length extrapolation of transformers: A survey from the perspective of positional encoding. Findings of the Association for Computational Linguistics: EMNLP 2024, 9959–9977 (2024). [19] Nilsson, J. W. & Riedel, S. A. Electric Circuits 12 edn (Pearson, Hoboken, NJ, 2022). [20] Ogata, K. Modern Control Engineering (Prentice Hall, Upper Saddle River, NJ, 2010). [21] Kailath, T. Linear Systems (Prentice-Hall, Englewood Cliffs, NJ, 1980). [22] Stuart, A. M. Inverse problems: A Bayesian perspective. Acta Numerica 19, 451–559 (2010). [23] Guibas, J. et al. Adaptive Fourier neural operators: Efficient token mixers for transformers. arXiv preprint arXiv:2111.13587 (2021). [24] Chollet, F. Xception: Deep learning with depthwise separable convolutions. Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017). 20 [25] Lusch, B., Kutz, J. N. & Brunton, S. L. Deep learning for universal linear embeddings of nonlinear dynamics. Nature Communications 9, 4950 (2018). [26] Butcher, J. C.The Numerical Analysis of Ordinary Differential Equations: Runge–Kutta and General Linear Methods (Wiley, Chichester, 1987). [27] Li, Z. et al. Fourier neural operator for parametric partial differential equations. International Conference on Learning Representations (2021). [28] Raissi, M., Perdikaris, P. & Karniadakis, G. E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics 378, 686–707 (2019). [29] Sermanet, P. et al. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013). [30] Liu, Z. et al. A ConvNet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022). [31] Liu, Z. et al. Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision (2021). [32] Liu, Y. et al. VMamba: Visual state space model. Advances in Neural Information Processing Systems (2024). [33] Hatamizadeh, A. & Kautz, J. MambaVision: A hybrid Mamba-Transformer vision backbone. Proceedings of the Computer Vision and Pattern Recognition Conference, 25261–25270 (2025). 21