Paper deep dive
A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems
Christoph Jürgen Hemmer, Florian Plaswig, Daniel Durstewitz
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/18/2026, 4:22:25 AM
Summary
The paper introduces DynaBase, a minimal two-parameter interpretable architecture derived from the complex DynaMix foundation model for zero-shot reconstruction of dynamical systems. DynaBase forecasts trajectories via a linear blend of the current latent state and the nearest in-context neighbor plus its temporal successor. Despite its extreme simplicity, it achieves competitive zero-shot performance across chaotic and cyclic systems with negligible parameters. Theoretical and empirical analysis reveals that DynaBase spans a spectrum from context parroting to chaotic dynamics, demonstrating that training strategy (short-term MSE vs. long-term DSR loss) fundamentally dictates whether the model learns trivial replication or true underlying dynamics.
Entities (10)
Relation Signals (10)
DynaBase → reducedfrom → DynaMix
confidence 97% · Toward this goal, here we iteratively reduce a recent powerful SOTA model for DS reconstruction, DynaMix ... to a minimal interpretable two-parameter form, which we call DynaBase.
DynaBase → uses → λ
confidence 96% · The model’s two coefficients (α,β) parametrize a continuous spectrum of dynamical behaviors... α,β,γ∈ℝ and α+β+γ=1
DynaBase → enables → Dynamical Systems Reconstruction
confidence 95% · DynaBase produces highly competitive zero-shot DS reconstructions across chaotic and cyclic systems, with a negligible parameter load
Context Parroting → recoveredby → DynaBase
confidence 94% · Theoretical and empirical analysis of DynaBase further leads to a 1-parameter family of maps, with the context-parroting algorithm of (Zhang & Gilpin, 2026) recovered at one end
DynaMix → isa → Foundation Model
confidence 93% · Recent foundation models (FMs) for zero-shot reconstruction of dynamical systems (DS) achieve strong out-of-domain generalization... SOTA model for DS reconstruction, DynaMix
DynaBase → produces → Chaotic Dynamics
confidence 92% · We show that at one end of this subspace DynaBase produces strictly chaotic dynamics, mimicking chaotic DS provided in-context
In-context Learning → underlies → Foundation Models
confidence 91% · inspired by the success of in-context learning in large language and time-series foundation models (FMs)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent foundation models (FMs) for zero-shot reconstruction of dynamical systems (DS) achieve strong out-of-domain generalization but provide little insight into the mechanisms that underlie their forecasts. Such an understanding could help to strip down overladen FM architectures to their bare essence and expose the minimal requirements for in-context learning in the DS domain. Toward this goal, here we iteratively reduce a recent powerful SOTA model for DS reconstruction, DynaMix (Hemmer & Durstewitz, 2025), to a minimal interpretable two-parameter form, which we call DynaBase. DynaBase produces forecasts through a linear blend of the current latent state and the nearest in-context neighbor and its temporal successor. Surprisingly, despite its extreme simplicity, DynaBase produces highly competitive zero-shot DS reconstructions across chaotic and cyclic systems, with a negligible parameter load, many orders of magnitude below that of other FMs. Even more, this extreme simplicity permits direct model optimization on DS reconstruction measures, as well as closed-form one-step analytical solutions on prediction MSE. Theoretical and empirical analysis of DynaBase further leads to a 1-parameter family of maps, with the context-parroting algorithm of (Zhang & Gilpin, 2026) recovered at one end, and chaotic (divergent but bounded) behavior at the other. We further show how different training strategies lead to models either optimal for short-term prediction or for DS reconstruction. Thus, DynaBase not only exposes the minimal mechanisms required for producing zero-shot DS reconstruction, but also reconciles within an accessible mathematical frame divergent observations in the literature.
Tags
Links
- Source: https://arxiv.org/abs/2607.14937v1
- Canonical: https://arxiv.org/abs/2607.14937v1
Trouble viewing inline? Open PDF directly →
Full Text
90,728 characters extracted from source content.
Expand or collapse full text
A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems Christoph Jürgen Hemmer2,1,3111These authors contributed equally to this work. Corresponding authors: christoph.hemmer, daniel.durstewitz@zi-mannheim.de , Florian Plaswig1,311footnotemark: 1 , Daniel Durstewitz1,2,3 1Dept. of Theoretical Neuroscience, Central Institute of Mental Health, Mannheim, Germany 2Interdisciplinary Center for Scientific Computing (IWR), Heidelberg, Germany 3Faculty of Physics and Astronomy, Heidelberg University, Heidelberg, Germany Abstract Recent foundation models (FMs) for zero-shot reconstruction of dynamical systems (DS) achieve strong out-of-domain generalization but provide little insight into the mechanisms that underlie their forecasts. Such an understanding could help to strip down overladen FM architectures to their bare essence and expose the minimal requirements for in-context learning in the DS domain. Toward this goal, here we iteratively reduce a recent powerful SOTA model for DS reconstruction, DynaMix [44], to a minimal interpretable two-parameter form, which we call DynaBase. DynaBase produces forecasts through a linear blend of the current latent state and the nearest in-context neighbor and its temporal successor. Surprisingly, despite its extreme simplicity, DynaBase produces highly competitive zero-shot DS reconstructions across chaotic and cyclic systems, with a negligible parameter load, many orders of magnitude below that of other FMs. Even more, this extreme simplicity permits direct model optimization on DS reconstruction measures, as well as closed-form one-step analytical solutions on prediction MSE. Theoretical and empirical analysis of DynaBase further leads to a 1-parameter family of maps, with the context-parroting algorithm of [92] recovered at one end, and chaotic (divergent but bounded) behavior at the other. We further show how different training strategies lead to models either optimal for short-term prediction or for DS reconstruction. Thus, DynaBase not only exposes the minimal mechanisms required for producing zero-shot DS reconstruction, but also reconciles within an accessible mathematical frame divergent observations in the literature. 1 Introduction Many, if not most, natural and engineered systems, from chemical and molecular processes to climate, ecosystems, brain activity, or stock markets, are naturally described as dynamical systems (DS) [30, 37, 62, 81, 10, 21, 59]. Reconstructing these DS (DSR) in a data-driven way from just time series observations is of fundamental interest to all sciences, which seek tractable models that enable mechanistic insight into the underlying dynamical processes. DSR, hence, aims for generative surrogate models of the underlying dynamics which not only forecast a few steps ahead, but—importantly—reproduce long-term statistical properties of the DS in question, such as its attractor geometry (ergodic distribution) and power spectrum [13, 30, 38, 39, 71, 18, 17, 29]. While traditionally DSR involved custom-training ML/AI models on each particular dataset [14, 45, 39, 70, 71], inspired by the success of in-context learning in large language and time-series foundation models (FMs) [5, 4, 11], a new line of research explored whether zero-shot generalization to previously unseen DS is also feasible [44, 53]. However, as for most if not all FMs, it is currently utterly unclear how SOTA DSR FMs like DynaMix [44] achieve their zero-shot out-of-domain generalization, and what the essential architectural requirements for this are. Insight into how AI models solve their problems is of general interest, but is particularly important in scientific and medical domains where the focus is on mechanistic understanding [73, 30]. Here we tackle this question in the context of DSR by reverse-engineering the pretrained DynaMix model. We produce a series of systematic ablations and simplifications that largely retain, or even improve on, the model’s zero-shot DSR performance. We ultimately arrive at an algebraically minimal form consisting just of a linear combination of the model’s previous state, the nearest neighbor from the context signal, and its immediate temporal successor. A natural symmetry requirement on the context points further condenses the model into a two-parameter form. We show that this recursive map, DynaBase, can generate point, cyclic, and chaotic attractors, and allows for closed-form one-step least-squares fitting to time series data. Theoretical and empirical analysis of DynaBase trained directly for DSR reveals that optimal zero-shot solutions essentially lie along a 1-parameter curve with truly chaotic dynamics at one end and ‘context parroting’ of a time series snippet at the other, a strategy recently observed in transformer-based time series FMs like Chronos [92, 91]. Our observations are in line with a recent literature that demonstrates that tiny ML/AI models, many orders of magnitude below the parameter load of modern FMs, often achieve comparable, or even widely superior performance, on a range of tasks from cognitive and behavioral modeling [54] to challenging reasoning problems [85, 48]. In a nutshell, our main results are: • Minimal zero-shot DSR model: Through iterative simplification of DynaMix [44] we create a minimal 2-parameter map, DynaBase, which exhibits competitive zero-shot DSR performance at essentially zero training costs. • Mechanistic insight: The model’s two coefficients (α,β)(α,β) parametrize a continuous spectrum of dynamical behaviors, with optimal DSR solutions essentially driven to a 1-parameter subspace. We show that at one end of this subspace DynaBase produces strictly chaotic dynamics, mimicking chaotic DS provided in-context, while at the other it yields the exact context-parroting algorithm of [92]. • Impact of training strategy: We further demonstrate that training using either a standard ahead-prediction MSE loss vs. a specific DSR loss yields fundamentally different solutions: Training for ahead-prediction often collapses to simple context parroting and hence does not recover the true dynamics, especially for chaotic systems, which is only achieved by DSR-specific training. 2 Related work Dynamical systems reconstruction (DSR) DSR aims to learn generative models from time series data that capture the underlying system’s dynamical long-term behavior, i.e. its long-term or “climate” statistics and state space geometry [71, 30, 45, 60, 38, 18]. Custom-trained models for this purpose build on library-based methods like SINDy [20, 56], reservoir computing [68, 70, 71, 22, 42], neural ODEs [23, 3, 51], Koopman operators [58, 65, 6, 63, 19, 87], or on various types of RNNs [80, 31, 83, 14, 74, 45, 43, 13, 12]. More important than the particular architecture appears to be the training strategy: sparse or generalized teacher forcing [60, 45] are control-theoretic training techniques that enable long trajectory roll-outs even for chaotic DS, while specific regularization methods based on a system’s invariant measures have been designed to enforce agreement in long-term statistics [35, 71, 47]. All of these methods, however, require purpose-training on each new system, and out-of-domain generalization remains a key challenge [39]. Zero-shot DSR and time-series foundation models (FMs) Multi-environment [90, 50], hyper-network-guided [82], or hierarchical [15] approaches have been used to train DSR models on many DS simultaneously, but all of these still require fine-tuning on new systems or parameter configurations not seen in training. More recently, inspired by the success of in-context inference in LLMs [16, 36, 1], a line of general-purpose time series (TS) FMs pretrained on large corpora of real and synthetic data has emerged, which forecast TS from a short observed context snippet zero-shot. Most of these, like Chronos [5, 4], TimesFM [27], or Panda [53], are based on transformers, with a few exceptions that engage RNNs like the xLSTM [8] or Mamba [40] as their core. All of these, however, lack the ability to reproduce a system’s long-term behavior as required in DSR [39, 44, 29]. They mostly either converge to just fixed points in the limit, or cyclically repeat the context [91, 92], and thus fail to capture DS with inherently chaotic dynamics [44]. DynaMix [44] closes this gap by pretraining, based on DSR-specific training techniques [60], a mixture of AL-RNN [13] experts, weighted by a context-dependent gating network. It delivers competitive short-term forecasts and SOTA long-term performance, while being relatively lightweight with ∼104 10^4 parameters. Mechanisms of in-context learning What is at the root of in-context learning has been intensely studied in the past few years [16, 36, 1], and hypotheses range from transformers implementing gradient-descent in-context [7, 84, 55, 89] to mechanisms that more or less boil down to a simple remixing of their memory contents [77, 86, 88]. For DSR FMs, the analogous problem remains largely open but—at the same time—appears more tractable due to the more constrained and mathematically better defined [39] nature of the question. Recent studies using Chronos suggested that TS FMs may long-term forecast DS by the simple mechanism of context parroting [91], a well known pattern in LLMs [64, 32, 24]. Based on this phenomenon, Zhang and Gilpin [92] explicitly design a ‘context parroting algorithm’ which takes the last D values of the context as a delay-embedded query, retrieves the best-matching motif elsewhere in the context, and copies the subsequent fragment as forecast. They show that this strategy works surprisingly well and actually outperforms most TS FMs. But context parroting, as exhibited by Chronos or conceptualized in this form, can only reproduce discrete-cyclic activity and thus inherently fails to capture defining signatures of chaotic systems, like positive Lyapunov exponents and rather broad power spectra [44, 29]. A superficially similar principle is employed in classical nonlinear forecasting algorithms that retrieve from a library (history) locally neighboring trajectories whose next time step predictions are either just averaged or based on which a locally linear model is fit [33, 49]. Crucially, in contrast to mere context parroting, this neighborhood may change, however, with each prediction step taken. Through iterative reduction of DynaMix we arrive at a mathematical form that contains these classes of algorithms as special cases. 3 Methods 3.1 Background: DynaMix DynaMix [44] is a DSR foundation model based on a mixture-of-experts architecture, using j=1…Jj=1… J almost-linear RNNs (AL-RNNs) [13] as experts, t+1j=Fjexp(t)=jt+jΦ∗(t)+j, z^j_t+1=F^exp_j( z_t)= A_j z_t+ W_j ^*( z_t)+ h_j\;, (1) where tj∈ℝM z^j_t ^M is an M-dimensional latent state, Φ∗(t):=[1:M−P,t,ReLU(M−P+1:M,t)]⊤ ^*( z_t):= [ z_1:M-P,t,\ ReLU( z_M-P+1:M,t) ] is the activation function with M−PM-P linear units and P ReLUs, and j∈diag(ℝM) A_j (R^M), j∈ℝM×M W_j ^M× M, and j∈ℝM h_j ^M are learnable parameters. The different experts are combined into an overall next state prediction by t+1=∑j=1Jwj,texp⋅t+1j z_t+1= _j=1^Jw^exp_j,t· z_t+1^j, where the weights wj,texp∈(0,1)w^exp_j,t∈(0,1) are determined by a gating network through texp(t,~tatt)=σ(MLP(~tatt,t)τexp)∈ℝJ, w^exp_t( z_t, C w^att_t)=σ ( MLP( C w^att_t, z_t) _exp ) ^J\;, (2) where τexp _exp is a temperature parameter and ~=CNN() C=CNN( C) are temporal features obtained from the provided context signal =t∈ℝN×TC C=\ c_t\ ^N× T_C through a CNN. Attention weights tatt w^att_t at each time step t are in turn computed based on a distance between current latent state t z_t and the context C as tatt(t,)=σ(|−(t+ϵ)TC⊤|⊤Nτatt)∈ℝTC, w^att_t( z_t, C)=σ ( | C- ( Dz_t+ ε ) 1_T_C | 1_N _att ) ^T_C\;, (3) where τatt _att is another temperature, ∈ℝN×M D ^N× M a learnable matrix, ϵ∼(0,) ε (0, ) exploration noise, and TC,N 1_\T_C,N\ are column vectors of ones of length TC,NT_C,N, respectively. In [44], DynaMix was trained on ≈6×105≈ 6× 10^5 trajectories from 3434 3d3d DS in the cyclic or chaotic regime using control-theoretic training techniques that ensure DSR criteria are met [60, 45]. After pretraining, given any context time series snippet C, DynaMix is charged with forecasting the long-term trajectory evolution of the underlying DS without any retraining or fine-tuning. 3.2 Reducing DynaMix At a high level, DynaMix can be compactly expressed as a single RNN of the following form t+1=FDynaMix(t,):=∑jFjexp(t)⋅texp(t,~tatt), z_t+1=F_DynaMix( z_t, C):= _jF_j^exp( z_t)· w^exp_t( z_t, C w^att_t)\;, (4) with (see [44]) ~tatt=∑i=1TC−1CNN()i⋅iatt(t,). C w^att_t= _i=1^T_C-1CNN( C)_i· w^att_i( z_t, C)\;. (5) Note that eq. 4 is a weighted sum of piecewise-linear systems FjexpF_j^exp, where, however, the weights texp(t,) w^exp_t( z_t, C) themselves are nonlinear functions of the current state and the context. As already shown in [44], the MLP in eq. 2 may be replaced by a linear layer w/o significant loss in performance, while in Appx. A.3 we show the output nonlinearity σ (softmax) can as well be linearized over the relevant range. Further, the almost-linear term Φ∗(t) ^*( z_t) in eq. 1 can be made fully linear (Appx. A.3), leaving ~=CNN() C=CNN( C) and tatt(t,) w^att_t( z_t, C) as the only nonlinearities: t+1=FLinear(t,)=t+(CNN()tatt(t,))+ z_t+1=F_Linear( z_t, C)= A z_t+ B(CNN( C) w^att_t( z_t, C))+ h (6) We next observe that for τatt→0 _att→ 0, tatt(t,) w^att_t( z_t, C) essentially just boils down to a nearest neighbor selector, τ(t)=argmins‖s−t‖2τ( z_t)= *arg\,min_s c_s- z_t_2. Further assuming a CNN kernel size of 22 and noticing that for a single linear CNN layer (as used in DynaMix) the CNN operations can be combined with B in eq. 6, we obtain fLinear(t,) f_Linear( z_t, C) =zt+(1)τ(t)+(2)τ(t)+1+~ = Az_t+ B^(1) c_τ( z_t)+ B^(2) c_τ( z_t)+1+ h (7) τ(t) τ( z_t) =argmins‖s−t‖2 = *arg\,min_s c_s- z_t_2 (8) where ,(1),(2)∈ℝM×M A, B^(1), B^(2) ^M× M and ~∈ℝM h ^M. We call this form a recursive affine nearest-neighbor (N) map. 3.3 DynaBase In DSR, two vector fields that differ only by a rigid coordinate transformation describe the same underlying flow and therefore yield dynamically equivalent reconstructions [69]. Recognizing this invariance, we can state the following result: Theorem 1 (Uniqueness of recursive affine N maps). Any nearest-neighbor recursive map of the form eq. 7 that is affine in t z_t, 1 c_1, and 2 c_2, equivariant under O(N)O(N) rotations and reflections, and translation equivariant, belongs to the two-parameter linear family fα,βf_α,β, t+1=fα,β(t)=αt+β1+(1−α−β)2. z_t+1=f_α,β( z_t)=α\, z_t+β\, c_1+(1-α-β)\, c_2. (9) Proof. See Appx. B.∎ Exploiting Thm. 1, we obtain a minimal model, which we call DynaBase, as t+1=fα,β(t)=αt+βτ(t)+γτ(t)+1,τ(t)=argmins‖s−t‖2, z_t+1=f_α,β( z_t)=α\, z_t+β\, c_τ( z_t)+γ\, c_τ( z_t)+1, τ( z_t)= _s c_s- z_t_2, (10) with α,β,γ∈ℝα,β,γ and α+β+γ=1α+β+γ=1. Thus, γ=1−α−βγ=1-α-β leaves (α,β)∈ℝ2(α,β) ^2 as the only free parameters of the minimal model (formalized in Thm. 1). If the data were noise-free, a natural self-consistency condition would be given by i+1=fα,β(i)∀i∈[1,TC−1], c_i+1=f_α,β( c_i)\;∀ i∈[1,T_C-1]\ , (11) which guarantees that whenever t z_t coincides with a context point i c_i, update eq. 10 maps t↦i+1 z_t c_i+1 exactly. Using this reasonable assumption, the affine N map could be simplified further by imposing α=−β⟹γ=1,α=-β γ=1, such that the whole construction depends only on a single parameter: t+1=fα(t)=α(t−τ(t))+τ(t)+1, z_t+1=f_α( z_t)=α\,( z_t- c_τ( z_t))+ c_τ( z_t)+1\ , (12) where the local rate of con- or divergence is determined solely by α. 3.4 Training For training DynaBase, we assume we are given N-dimensional training data ∈ℝN×T X ^N× T, of which we define the first TCT_C time steps as the context signal ∈ℝN×TC C ^N× T_C. Given the model’s simplicity, it allows for straightforward training, where we assume t=t z_t= x_t (i.e., the dynamics is directly defined in the observation space). Specifically, we explore the two following approaches: Linear regression As DynaBase is linear in parameters, its one-step-ahead prediction error loss is convex and we can optimize it by least-squares regression in one step using the reparameterized form t+1−τ(t)+1=α(t−τ(t)+1)+β(τ(t)−τ(t)+1). z_t+1- c_τ( z_t)+1=α\,( z_t- c_τ( z_t)+1)+β\,( c_τ( z_t)- c_τ( z_t)+1). (13) Given a training sequence, nearest neighbors (τ(t),τ(t)+1)∀i∈( c_τ( x_t), c_τ( x_t)+1)\;∀ x_i∈ X are simply precomputed based on eq. 8, and ordinary-least-squares is used to solve for (α^,β^)( α, β) in closed form. No backpropagation or teacher forcing is required; fitting is embarrassingly cheap compared to conventional FM pretraining. Grid search However, linear regression only optimizes for short term forecasts, and hence might not be optimal for DSR problems which require capturing long-term behavior. We therefore investigate a second strategy, where the loss is given through a long-term reconstruction measure [52, 60] Loss(,^)=Dstsp(ptrue()∥ppred(^)),Loss( X, X)=D_stsp(p_true( X)\|p_pred( X))\ ,\; (14) i.e. the Kullback-Leibler divergence between the true and model-generated trajectory distributions based on long term rollouts X of DynaBase. DstspD_stsp is a common measure in the DSR literature to assess the agreement in true and reconstructed attractor geometries [52, 60, 38, 45, 66, 91, 53]. Since the model has only 2 trainable parameters, searching for an optimal solution by grid search is fairly straightforward and inexpensive. 4 Results 4.1 Theoretical results We now establish a theoretical characterization of the forecasting properties of DynaBase. Structural characterization By Thm. 1, DynaBase is a unique nearest-neighbor affine family compatible with the symmetry assumptions in Sec. 3.3. We use this structural result as the starting point for the two main mechanism-level consequences below. Theorem 2 (Reduction to context parroting). Let 1:TC∈ℝ1×TC C_1:T_C ^1× T_C be a context sequence and ¯1:TC−D+1∈ℝD×(TC−D+1) C_1:T_C-D+1 ^D×(T_C-D+1) its delay embedding with dimension D, defined as ¯s=(cs,cs+1,…,cs+D−1) c_s=(c_s,c_s+1,…,c_s+D-1). Let f0f_0 denote DynaBase with α=0α=0 (hence β=0,γ=1β=0,γ=1) on the embedded context ¯ C, and consider the discrete-time system t+1=f0(t) z_t+1=f_0( z_t) with initial condition 0=¯TC−D+1 z_0= c_T_C-D+1 and τ(0)∈[1,TC−2D]τ( z_0)∈[1,T_C-2D]. Then the projection xt=zD,tx_t=z_D,t exactly reproduces the context-parroting algorithm of [92]. Proof. See Appx. B.∎ Theorem 2 shows that the context-parroting baseline of [92] is recovered for free from DynaBase by simply setting (α,β)=(0,0)(α,β)=(0,0) and delay-embedding the context. Any DynaBase model with α≠0α≠ 0 is therefore strictly more expressive than parroting in the sense that a larger class of DS can be embedded. Given that the map fαf_α is piecewise-affine, we can also derive an exact continuous-time analogue [61] (t) ζ(t), such that (kh)=k∀k∈ℤ,h∈ℝ ζ(kh)= z_k∀ k ,h , by integrating the equations ddt=logαh−1hlog(α)1−α(τ(ζ)+1−ατ(ζ)) ζt= αh ζ- 1h α1-α( c_τ(ζ)+1-α c_τ(ζ)) (15) where we denote the discrete timestep size of fαf_α as h and logα1−α|α=1 α1-α _α=1 is defined by its continuous extension. Notably, this limit does not exist for α=0α=0 (context parroting), and is only defined in the complex number space for α<0α<0. For α=0α=0, the map collapses onto C at every step, i.e. hops between a discrete set of points, which is not reproducible by a continuous flow unless C contains exactly one (fixed) point. Importantly, context parroting is therefore an inherently discrete special case of DynaBase, defined on a measure-zero set, which cannot reproduce true limit cycles or chaotic behavior. A detailed derivation is given in Appx. A.6. Dynamical characterization We now characterize the dynamical properties of the self-consistent DynaBase – specifically how the parameter α governs the transition from trivial to genuinely chaotic behavior. Theorem 3 (Long-term dynamics). For α=0α=0, every orbit of DynaBase converges after finitely many steps onto a terminal cycle contained in the context sequence, and thus has max. Lyapunov exponent λmax<0 _max<0. Proof. See Appx. B.∎ The case α=0α=0 is therefore dynamically trivial: all orbits eventually collapse to C; context parroting is thus structurally incapable of producing chaos. Theorem 4 (Boundedness). Let fαf_α be the self-consistent DynaBase map with parameter α≥0α≥ 0 and context sequence 1:TC⊂ℝN C_1:T_C ^N. Then α∈[0,1]α∈[0,1] is a sufficient but not a necessary condition for the orbits of fαf_α to remain bounded. More precisely, α∈[0,1]α∈[0,1] guarantees d(t+1)≤d(t)d_ C( z_t+1)≤ d_ C( z_t) for all t. For α>1α>1, there exist Voronoi parcellations by C which allow for strictly chaotic dynamics (i.e., such that λmax>0 _max>0 and the dynamics is bounded). Proof. See Appx. B.∎ The theorem thus establishes that α∈[0,1]α∈[0,1] is sufficient for the orbit to remain confined to – or track – the context sequence, as distances to C are non-increasing. For α>1α>1, the map will diverge almost everywhere. If the arrangement of Voronoi cells through the context C is, however, such that its dynamics still remains bounded, it will almost surely follow a chaotic attractor [2, 41]. An example is discussed in the proof to Thm. 4 and illustrated in Fig. 2b. 4.2 Empirical forecasting comparison Figure 1: DSR performance across all 5454 test DS used in [44]. Minimal models (red: DynaBase training variants and pure context parroting), custom-trained DSR models (gray), and pretrained zero-shot foundation models including a zero-shot variant of DynaBase (blue) are compared for the same context length TC=2000T_C=2000 (values for custom-trained and foundation models taken from [44]). Median± of DstspD_stsp (left, geometrical disagreement), DHD_H (center, temporal disagreement) and MASE (right, 1010-step prediction error). We evaluate DynaBase on the same DSR test set as in [44], reporting the same three complementary measures, two of them specific to DSR (see Appx. A.2 for definitions): the disagreement DstspD_stsp in state space geometry, the long-term temporal disagreement DHD_H, and the short-term mean absolute scaled error (MASE). To evaluate the long-term statistics DstspD_stsp and DHD_H, each system is forecast for T=10,000T=10,000 time steps, following [44] and in agreement more generally with the DSR literature [45, 71, 29] (note that point forecasts become meaningless for chaotic systems beyond a few Lyapunov times). As introduced in Sec. 3.4, we fit DynaBase by either (i) closed-form one-step linear regression, optimizing the one-step MSE in eq. 13, or (i) directly for DSR over the grid of parameter values. As baselines we include context parroting as reported in [91, 92], DynaMix [44] and Chronos [5, 4] as FMs, and three custom-trained DSR baselines (AL-RNN [13], reservoir computers [67], and Neural ODEs [23]) trained directly on the context signal provided to the FMs. As shown in Fig. 1, DynaBase generally performs best, with the variant optimized for DSR (i) better than the one trained for ahead-prediction (i) on the long-term measure DstspD_stsp (while the differences in MASE and DHD_H are not statistically significant, Wilcoxon signed-rank test, p>0.1p>0.1). However, for longer (but not too long) prediction horizons, optimizing for ahead-prediction achieves lower MASE than DSR-optimization (Table 2). This difference illustrates that, not unexpectedly, the choice of training objective determines the model’s forecasting strategy: a one-step MSE penalizes any local deviations from the ground truth, and hence tends to produce smaller α values enabling the map to track the immediate next step tightly, yielding better short-term predictions. In contrast, the grid optimization strategy (i) directly targets the invariant attractor geometry, outperforming (i) on DSR. In both cases, however, training DynaBase with its few parameters is extremely cheap and rapid. Figure 2: a) Comparison of distributions of estimated α α values across all 5454 chaotic test DS [44] for the two training objectives (sect. 3.4). Closed-form minimization of the 1-step MSE yields α<1α<1, outside of the chaotic regime. In contrast, optimization for DSR places most α values firmly (and correctly) into the chaotic regime, with a median α^≈1.01 α≈ 1.01. b) DSR loss landscape (optimization for DstspD_stsp) across (α,β)(α,β) shows a valley at α=−βα=-β in agreement with the self-consistency condition, as well as a minimum at α>1α>1. c) Bifurcation diagram of the self-consistent DynaBase map fαf_α for the 1d1d context sequence =(−1,1,−1) C=(-1,1,-1) (the boundedness of this map was shown in the proof of Thm. 4). The graph shows the map’s long-term behavior (attractor structure) as a function of α, obtained by drawing 1,0001,000 random initial conditions from the interval [−2,2][-2,2] and discarding transients. The red line (α=1α=1) marks the transition from a stable 22-cycle (−1,+1)(-1,+1) to the chaotic regime. The distribution of estimated α α values across the test systems in Fig. 2a makes this point explicit. The two training objectives lead to qualitatively different parameter regimes: linear regression on the 1-step MSE places the map mostly into the parroting/ semi-cyclic regime with α<1α<1, while optimizing directly on DstspD_stsp, in contrast, produces mostly chaotic behavior (in accordance with the ground-truth DS) with α>1α>1. This implies that only DSR-based training has correctly inferred the chaotic ground truth systems, as further confirmed by a high correlation (r≈0.90r≈ 0.90) between max. Lyapunov exponents of the GT DS and the DynaBase map when fitted by DSR, but not when fitted by least-squares (r≈0.18r≈ 0.18), Fig. 11. Moreover, the loss landscape reveals an elongated valley along the α=−βα=-β axis (Fig. 2b), providing empirical support for the self-consistency condition in eq. 12, which reduces DynaBase to a one-parameter map. The bifurcation graph in Fig. 2c further illustrates on a synthetic toy example the cyclic behavior for α<1α<1 and emergence of chaos for α>1α>1. Another interesting observation is that for DSR-specific training, α α tightly concentrates on a single value near α≈1.01α≈ 1.01, across the diverse set of DS used. This suggests that, at least on this benchmark, the α-parameter may be fixed a priori. We test this by estimating α on a separate training set (similar to the one used for training DynaMix in [44]), obtaining α=1.006α=1.006 close to the median in Fig. 2a, which leads to a parameter-free zero-shot forecaster that requires no DS-specific fitting. Applying this to the context signals from the test set, we still achieve performance comparable to the custom-trained models (see Fig. 1, and Figs. 6-9 for zero-shot reconstructions). 4.3 Loss landscape tracks dynamics Figure 3: DynaBase’ loss landscape reflects the dynamics of the target system when trained for DSR. Top: Lorenz-63 in chaotic regime (gray). The geometrical misalignment DstspD_stsp across α exhibits a global minimum (red dot) at α≈1.01α≈ 1.01, which produces a valid reconstruction of the attractor (red). Center: Lorenz-63 in cyclic regime. The DstspD_stsp minimum occurs at α=1α=1. Bottom: A fixed point or cyclic point (discrete cycle) (gray). The DstspD_stsp minimum occurs at α=0α=0, corresponding to context parroting. To examine how DynaBase’ parameters map onto the ground truth system’s dynamical regime, we study the DSR loss landscape on a representative chaotic, cyclic and fixed point system (Fig. 3). For these different dynamical regimes, the loss landscape exhibits distinct global minima which correspond to different dynamical mechanisms: • Context parroting α=−β=0α=-β=0: the map reduces to t+1=τ(t)+1 z_t+1= c_τ( z_t)+1, i.e. the state snaps onto the successor of its nearest context neighbor. For a delay-embedded context signal, this exactly reproduces the context-parroting algorithm of Zhang and Gilpin [92] (see Thm. 2). • Flow mimicry α=−β=1α=-β=1: the map reduces to t+1−t=τ(t)+1−τ(t) z_t+1- z_t= c_τ( z_t)+1- c_τ( z_t), a forward-Euler step driven by the local finite difference ˙≈(τ(t)+1−τ(t))/h c≈( c_τ( z_t)+1- c_τ( z_t))/h of the underlying vector field as sampled from the context. This yields a ‘zero-friction’ direction in state space (neither con- nor divergence), as along the limit cycle (λmax=0 _max=0) of a continuous-time DS. • Chaos α>1α>1: One obtains local divergence almost everywhere and, if bounded, chaotic trajectories are produced (cf. Fig. 12 for Lorenz-63 reconstructions along a bifurcation diagram). Figure 4: a) Voronoi tessellation for the periodic Selkov DS induced by the nearest-neighbor lookup τ(⋅)τ(·) in DynaBase. The context 1:TC C_1:T_C partitions the state space into cells Vs=:τ()=sV_s=\ z:τ( z)=s\, within which DynaBase is described by a strictly affine map t+1=αt+~s z_t+1=α\, z_t+ c_s. b) Out-of-context forecasting on the 2d Selkov system. When the initial condition lies outside of the context-covered region, neither parroting (α=0α=0, left) nor pure flow mimicry (α=1α=1, center) recovers the true vector field. Refitting α α from a single additional short trajectory via eq. 13 produces a local flow estimate that aligns better with the true vector field (right). The nearest-neighbor lookup τ(⋅)τ(·) induces a Voronoi tessellation of the state space (Fig. 4a). Inside each cell Vs=:τ()=sV_s=\ z:τ( z)=s\, DynaBase is a strictly affine map t+1=αt+~s z_t+1=α\, z_t+ c_s, where ~s=βs+γs+1 c_s=β c_s+γ c_s+1 is fixed by the cell, hence belongs to the class of well-studied switching piecewise-linear (PL) systems [26, 78]. Within each cell VsV_s, the model’s Jacobian is simply given by αα I, i.e. a single scalar α controls the local con- or divergence almost everywhere in state space. With α^≈1.01 α≈ 1.01 just slightly above 11 (Fig. 2), DynaBase diverges almost everywhere but still stays sufficiently close to the cell centers to create a chaotic attractor, despite not having a nonlinear activation function (in line with Thm. 3 & Thm. 4). For context parroting with α=0α=0 (Thm. 2), on the other hand, DynaBase snaps onto a cyclic point that perfectly traces out the empirically observed context trajectory. However, a cyclic point would result in this case even if the observed trajectory snippet comes from a chaotic system (Fig. 13). Pure context parroting also prevents generalization to new initial conditions of an observed DS, a feature of DynaMix demonstrated in [44]. This is illustrated for the Selkov DS in Fig. 4b: For α=0α=0, DynaBase strictly follows the context points and thus fails to map the vector field anywhere outside the context. On the other hand, pure context mimicry, f1f_1, while reproducing the λmax=0 _max=0 direction on a limit cycle, fails to capture the convergence when started off the stable cycle, since the finite difference τ(t)+1−τ(t) c_τ( z_t)+1- c_τ( z_t) is always evaluated at the nearest context point. Fitting α α via eq. 14 on a trajectory bit that contains a transient, however, yields a better approximation of the surrounding vector field outside the observed context while still reproducing the cycle (Fig. 4b). 5 Conclusion We reduce DynaMix, a mixture-of-experts DSR foundation model, into DynaBase, a two-parameter nearest-neighbor affine recurrence that can be further reduced to a 1d1d family of maps by imposing a consistency condition. Despite this minimal form, variants of DynaBase match or exceed substantially larger architectures on both long-horizon DSR measures and short-term predictions, while remaining computationally extremely cheap with either training via 1-step closed-form least squares or grid search on a DSR objective. Analysis of this minimal map and its training provides a number of interesting insights into the minimal requirements TS and DSR FMs may need to achieve zero-shot DS forecasts. First, α=0α=0 exactly reproduces context parroting as observed in [91, 92], but this mechanism is inherently unable to reproduce chaotic or even true limit cycle behavior. Second, for α>0α>0 this affine nearest-neighbor form admits the whole spectrum from convergence to fixed points or cycles to strictly chaotic activity for α>1α>1, thus being sufficiently expressive to approximate a large range of DS. Third, for α≈1.01α≈ 1.01 the map can even operate in zero-shot mode, at least on a common DSR benchmark set, exposing the profound role the context alone plays in structuring the forecast. Fourth, optimizing for ahead-predictions biases the map toward convergence to fixed or cyclic points, even when the underlying DS is chaotic, explaining why this training objective is suboptimal if DSR is the goal [60, 45, 71, 47]. More generally, with DynaBase we have established a formal framework that enables to study mechanisms of DSR in FMs and beyond in detail, creating an instrument for mathematical analysis of context-based forecasting and DSR methods. Limitations DynaBase is a minimal 2-parameter model. Although it highlights the fact that complex architectures with tens of thousands to many millions of parameters may not be needed for DSR and time series forecasting, the assumptions on which its reduction is based come at a cost in terms of the diversity of DS and vector field topologies that can be learned, as well as the accuracy with which geometrical properties of the vector field can be captured (as apparent from the remaining mismatch in Fig. 4b). Also, we so far did not explicitly consider the impact of noise (but see Fig. 14, where noise degrades but does not preclude DSR). Currently, DynaBase may be viewed more as a research tool that allows for detailed mathematical analysis of context-based DSR FMs. A more in-depth study of DynaBase, e.g. its DS approximation capabilities or the computation of the full Lyapunov spectrum, which requires the concept of a saltation operator [28, 25], is still pending. Acknowledgements This work was supported by individual grants Du 354/15-1 (# 502196519) and Du 354/18-1 (# 567025973) from the German Research Foundation (DFG), and by the German Ministry for Research, Astronautics, and Technology (BMFTR) through NAILIt (“Neuro-Inspired AI for Learning & Inference in Non-Stationary Environments”, # 01GQ2509A). References [1] E. Akyürek, D. Schuurmans, J. Andreas, T. Ma, and D. Zhou (2023) What learning algorithm is in-context learning? Investigations with linear models. In Proceedings of the 11th International Conference on Learning Representations (ICLR), External Links: Link Cited by: §2, §2. [2] K. T. Alligood, T. D. Sauer, and J. A. Yorke (1996) Chaos: an introduction to dynamical systems. Textbooks in Mathematical Sciences, Springer. External Links: ISBN 978-0-387-94677-1 978-0-387-22492-3, Document Cited by: §4.1. [3] V. M. M. Alvarez, R. Roşca, and C. G. Fălcuţescu (2020) Dynode: neural ordinary differential equations for dynamics modeling in continuous control. arXiv preprint arXiv:2009.04278. Cited by: §2. [4] A. F. Ansari, O. Shchur, J. Küken, A. Auer, B. Han, P. Mercado, S. S. Rangapuram, H. Shen, L. Stella, X. Zhang, M. Goswami, S. Kapoor, D. C. Maddix, P. Guerron, T. Hu, J. Yin, N. Erickson, P. M. Desai, H. Wang, H. Rangwala, G. Karypis, Y. Wang, and M. Bohlke-Schneider (2025) Chronos-2: from univariate to universal forecasting. External Links: 2510.15821, Link Cited by: §A.1, §1, §2, §4.2. [5] A. F. Ansari, L. Stella, A. C. Turkmen, X. Zhang, P. Mercado, H. Shen, O. Shchur, S. S. Rangapuram, S. P. Arango, S. Kapoor, J. Zschiegner, D. C. Maddix, H. Wang, M. W. Mahoney, K. Torkkola, A. G. Wilson, M. Bohlke-Schneider, and B. Wang (2024) Chronos: learning the language of time series. Transactions on Machine Learning Research. External Links: ISSN 2835-8856, Link Cited by: §A.1, §1, §2, §4.2. [6] O. Azencot, N. B. Erichson, V. Lin, and M. W. Mahoney (2020) Forecasting Sequential Data using Consistent Koopman Autoencoders. In Proceedings of the 37th International Conference on Machine Learning, External Links: Link Cited by: §2. [7] Y. Bai, F. Chen, H. Wang, C. Xiong, and S. Mei (2023) Transformers as statisticians: provable in-context learning with in-context algorithm selection. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36, p. 57125–57211. External Links: Link Cited by: §2. [8] M. Beck, K. Pöppel, M. Spanring, A. Auer, O. Prudnikova, M. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter (2024) Xlstm: extended long short-term memory. Advances in Neural Information Processing Systems 37, p. 107547–107603. Cited by: §2. [9] G. Benettin, L. Galgani, A. Giorgilli, and J. Strelcyn (1980) Lyapunov characteristic exponents for smooth dynamical systems and for hamiltonian systems; a method for computing all of them. part 1: theory. Meccanica 15 (1), p. 9–20. Cited by: §A.2. [10] U. S. Bhalla and R. Iyengar (1999) Emergent properties of networks of biological signaling pathways. Science 283 (5400), p. 381–387. External Links: Document Cited by: §1. [11] S. K. Bhethanabhotla, O. Swelam, J. Siems, D. Salinas, and F. Hutter (2024) Mamba4Cast: efficient zero-shot time series forecasting with state space models. arXiv preprint arXiv:2410.09385. Cited by: §1. [12] A. Brändle, L. Eisenmann, F. Götz, and D. Durstewitz (2026) Continuous-time piecewise-linear recurrent neural networks. arXiv preprint arXiv:2602.15649. Cited by: §2. [13] M. Brenner, C. J. Hemmer, Z. Monfared, and D. Durstewitz (2024) Almost-linear rnns yield highly interpretable symbolic codes in dynamical systems reconstruction. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37, p. 36829–36868. Cited by: §A.1, §1, §2, §2, §3.1, §4.2. [14] M. Brenner, F. Hess, J. M. Mikhaeil, L. F. Bereska, Z. Monfared, P. Kuo, and D. Durstewitz (2022-06) Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems. In Proceedings of the 39th International Conference on Machine Learning, p. 2292–2320. Note: ISSN: 2640-3498 External Links: Link Cited by: §1, §2. [15] M. Brenner, E. Weber, G. Koppe, and D. Durstewitz (2025) Learning interpretable hierarchical dynamical systems models from time series data. In The Thirteenth International Conference on Learning Representations (ICLR), External Links: Link Cited by: §2. [16] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. (2020) Language models are few-shot learners. Advances in neural information processing systems 33, p. 1877–1901. Cited by: §2, §2. [17] S. L. Brunton and J. N. Kutz (2019) Data-driven science and engineering: Machine learning, dynamical systems, and control. Cambridge University Press. Cited by: §1. [18] S. L. Brunton, B. W. Brunton, J. L. Proctor, E. Kaiser, and J. N. Kutz (2017-05) Chaos as an intermittently forced linear system. Nature Communications 8 (1), p. 19 (en). Note: Publisher: Nature Publishing Group External Links: ISSN 2041-1723, Link, Document Cited by: §1, §2. [19] S. L. Brunton, M. Budišić, E. Kaiser, and J. N. Kutz (2022) Modern koopman theory for dynamical systems. SIAM Review 64 (2), p. 229–340. External Links: Document, Link, https://doi.org/10.1137/21M1401243 Cited by: §2. [20] S. L. Brunton, J. L. Proctor, and J. N. Kutz (2016) Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences USA 113 (15), p. 3932–3937. External Links: ISSN 0027-8424, Document Cited by: §2. [21] G. Buzsaki (2006-08) Rhythms of the Brain. Oxford University Press (en). Note: Google-Books-ID: ldz58irprjYC External Links: ISBN 978-0-19-804125-2 Cited by: §1. [22] T. L. Carroll and L. M. Pecora (2019-08) Network Structure Effects in Reservoir Computers. Chaos: An Interdisciplinary Journal of Nonlinear Science 29 (8), p. 083130. Note: Comment: accepted for Chaos External Links: ISSN 1054-1500, 1089-7682, Document Cited by: §2. [23] R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud (2018) Neural Ordinary Differential Equations. In Advances in Neural Information Processing Systems 31, External Links: Link Cited by: §A.1, §2, §4.2. [24] S. Chen, H. Sheen, T. Wang, and Z. Yang (2024) Unveiling induction heads: provable training dynamics and feature learning in transformers. Advances in Neural Information Processing Systems 37, p. 66479–66567. Cited by: §2. [25] S. Coombes, M. Şayli, R. Thul, R. Nicks, M. A. Porter, and Y. M. Lai (2024) Oscillatory networks: insights from piecewise-linear modeling. SIAM Review 66 (4), p. 619–679. External Links: Document Cited by: §5. [26] J. Daafouz, P. Riedinger, and C. Iung (2002-11) Stability analysis and control synthesis for switched systems: a switched Lyapunov function approach. IEEE Transactions on Automatic Control 47 (11), p. 1883–1887. Note: Conference Name: IEEE Transactions on Automatic Control External Links: ISSN 1558-2523, Link, Document Cited by: §4.3. [27] A. Das, W. Kong, R. Sen, and Y. Zhou (2024) A decoder-only foundation model for time-series forecasting. In Forty-first International Conference on Machine Learning, Cited by: §2. [28] M. di Bernardo, C. J. Budd, A. R. Champneys, and P. Kowalczyk (2008) Piecewise-smooth dynamical systems: theory and applications. Applied Mathematical Sciences, Vol. 163, Springer, London. External Links: ISBN 978-1-84628-039-9, Document Cited by: §5. [29] D. Durstewitz, C. J. Hemmer, F. Hess, C. R. Doll, and L. Eisenmann (2026) Position: why a dynamical systems perspective is needed to advance time series modeling. arXiv preprint arXiv:2602.16864. Cited by: §A.2, §1, §2, §2, §4.2. [30] D. Durstewitz, G. Koppe, and M. I. Thurm (2023-11) Reconstructing computational system dynamics from neural data with recurrent neural networks. Nature Reviews. Neuroscience 24 (11), p. 693–710 (eng). External Links: ISSN 1471-0048, Document Cited by: §1, §1, §2. [31] D. Durstewitz (2017) A state space approach for piecewise-linear recurrent neural networks for identifying computational dynamics from neural measurements. PLoS Comput. Biol. 13 (6), p. e1005542 (eng). External Links: ISSN 1553-7358, Document Cited by: §2. [32] E. Edelman, N. Tsilivis, B. L. Edelman, E. Malach, and S. Goel (2024) The evolution of statistical induction heads: in-context learning markov chains. Advances in neural information processing systems 37, p. 64273–64311. Cited by: §2. [33] J. D. Farmer and J. J. Sidorowich (1987-08) Predicting chaotic time series. Physical Review Letters 59 (8), p. 845–848. Note: Publisher: American Physical Society External Links: Link, Document Cited by: §2. [34] J. H. Friedman, J. L. Bentley, and R. A. Finkel (1977) An algorithm for finding best matches in logarithmic expected time. ACM Transactions on Mathematical Software (TOMS) 3 (3), p. 209–226. Cited by: §A.6. [35] L. Fumagalli, K. Lüdge, J. de Wiljes, H. Haario, and L. Jaurigue (2025-11) Data-driven performance measures using global properties of attractors for testing black-box surrogate models of chaotic systems. Chaos: An Interdisciplinary Journal of Nonlinear Science 35 (11), p. 113121. External Links: ISSN 1054-1500, Document, Link, https://pubs.aip.org/aip/cha/article-pdf/doi/10.1063/5.0283424/20806395/113121_1_5.0283424.pdf Cited by: §2. [36] S. Garg, D. Tsipras, P. S. Liang, and G. Valiant (2022) What can transformers learn in-context? a case study of simple function classes. Advances in Neural Information Processing Systems 35, p. 30583–30598. Cited by: §2, §2. [37] W. Gilpin (2022) Chaos as an interpretable benchmark for forecasting and data-driven modelling. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), External Links: Link Cited by: Figure 13, Figure 13, Figure 5, Figure 5, §A.4, §1. [38] W. Gilpin (2023) Model scale versus domain knowledge in statistical forecasting of chaotic systems. Physical Review Research 5 (4), p. 043252. Cited by: §A.2, §1, §2, §3.4. [39] N. A. Göring, F. Hess, M. Brenner, Z. Monfared, and D. Durstewitz (2024-07) Out-of-Domain Generalization in Dynamical Systems Reconstruction. In Proceedings of the 41st International Conference on Machine Learning, p. 16071–16114 (en). Note: ISSN: 2640-3498 External Links: Link Cited by: §1, §2, §2, §2. [40] A. Gu and T. Dao (2024) Mamba: linear-time sequence modeling with selective state spaces. In First conference on language modeling, Cited by: §2. [41] J. Guckenheimer and P. Holmes (1983) Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields. Applied Mathematical Sciences, Vol. 42, Springer, New York, NY. External Links: ISBN 978-1-4612-7020-1 978-1-4612-1140-2, Link, Document Cited by: §4.1. [42] X. Han, Y. Zhao, and M. Small (2022-04) A tighter generalization bound for reservoir computing. Chaos: An Interdisciplinary Journal of Nonlinear Science 32 (4), p. 043115. External Links: ISSN 1054-1500, Document Cited by: §2. [43] C. J. Hemmer, M. Brenner, F. Hess, and D. Durstewitz (2024-07) Optimal Recurrent Network Topologies for Dynamical Systems Reconstruction. In Proceedings of the 41st International Conference on Machine Learning, p. 18174–18204 (en). Note: ISSN: 2640-3498 External Links: Link Cited by: §A.2, §2. [44] C. J. Hemmer and D. Durstewitz (2025) True zero-shot inference of dynamical systems preserving long-term statistics. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §A.1, §A.1, §A.1, §A.2, §A.2, §A.3, §A.4, Table 1, Table 1, 1st item, §1, §1, §2, §2, §3.1, §3.1, §3.2, §3.2, Figure 1, Figure 1, Figure 2, Figure 2, §4.2, §4.2, §4.3. [45] F. Hess, Z. Monfared, M. Brenner, and D. Durstewitz (2023-07) Generalized Teacher Forcing for Learning Chaotic Dynamics. In Proceedings of the 40th International Conference on Machine Learning, p. 13017–13049. Note: ISSN: 2640-3498 External Links: Link Cited by: §A.2, §1, §2, §3.1, §3.4, §4.2, §5. [46] H. Hewamalage, K. Ackermann, and C. Bergmeir (2023-01-01) Forecast evaluation for data scientists: common pitfalls and best practices. Data Mining and Knowledge Discovery 37 (2), p. 788–832. Note: (IF 4.8, Q2 in “Computer Science, Artificial Intelligence”, JCR 2022) External Links: Link Cited by: §A.2. [47] R. Jiang, P. Y. Lu, E. Orlova, and R. Willett (2023) Training neural operators to preserve invariant measures of chaotic attractors. Advances in Neural Information Processing Systems 36, p. 27645–27669. Cited by: §2, §5. [48] A. Jolicoeur-Martineau (2025) Less is more: recursive reasoning with tiny networks. External Links: 2510.04871, Link Cited by: §1. [49] H. Kantz and T. Schreiber (2004) Nonlinear time series analysis. Vol. 7, Cambridge university press. Cited by: §A.2, §2. [50] M. Kirchmeyer, Y. Yin, J. Donà, N. Baskiotis, A. Rakotomamonjy, and P. Gallinari (2022) Generalizing to new physical systems via context-informed dynamics model. In International Conference on Machine Learning, p. 11283–11301. Cited by: §2. [51] J. Ko, H. Koh, N. Park, and W. Jhe (2023) Homotopy-based training of neuralodes for accurate dynamics discovery. Advances in Neural Information Processing Systems 36, p. 64725–64752. Cited by: §2. [52] G. Koppe, H. Toutounji, P. Kirsch, S. Lis, and D. Durstewitz (2019) Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRI. PLOS Computational Biology 15 (8), p. e1007263 (en). External Links: ISSN 1553-7358, Document Cited by: §A.2, §3.4, §3.4. [53] J. Lai, A. Bao, and W. Gilpin (2025) Panda: a pretrained forecast model for universal representation of chaotic dynamics. arXiv preprint arXiv:2505.13755. Cited by: §A.2, §1, §2, §3.4. [54] J. Li, M. K. Benna, and M. G. Mattar (2025) Discovering cognitive strategies with tiny recurrent neural networks. Nature 644, p. 993–1001. External Links: Document Cited by: §1. [55] Z. Lin and K. Lee (2024) Dual operating modes of in-context learning. In Forty-first International Conference on Machine Learning, Cited by: §2. [56] J. Loiseau and S. L. Brunton (2018-03) Constrained sparse Galerkin regression. Journal of Fluid Mechanics 838, p. 42–67 (en). Note: Publisher: Cambridge University Press External Links: ISSN 0022-1120, 1469-7645, Document Cited by: §2. [57] E. N. Lorenz (1963) Deterministic nonperiodic flow. Journal of atmospheric sciences 20 (2), p. 130–141. Cited by: §A.4. [58] B. Lusch, J. N. Kutz, and S. L. Brunton (2018-12) Deep learning for universal linear embeddings of nonlinear dynamics. Nat Commun 9 (1), p. 4950. Note: arXiv: 1712.09707 External Links: ISSN 2041-1723, Link, Document Cited by: §2. [59] B. Mandelbrot and R. L. Hudson (2007) The misbehavior of markets: a fractal view of financial turbulence. Basic books. Cited by: §1. [60] J. Mikhaeil, Z. Monfared, and D. Durstewitz (2022-12) On the difficulty of learning chaotic dynamics with RNNs. Advances in Neural Information Processing Systems 35, p. 11297–11312 (en). Cited by: §A.2, §A.2, §2, §2, §3.1, §3.4, §3.4, §5. [61] Z. Monfared and D. Durstewitz (2020) Transformation of ReLU-based recurrent neural networks from discrete-time to continuous-time. In Proceedings of the 37th International Conference on Machine Learning, (en). External Links: Link Cited by: §A.6, §4.1. [62] P. J. Mumby, A. Hastings, and H. J. Edwards (2007) Thresholds and the resilience of caribbean coral reefs. Nature 450 (7166), p. 98–101. External Links: Document, ISBN 1476-4687, Link Cited by: §1. [63] I. Naiman and O. Azencot (2021-10) A Koopman Approach to Understanding Sequence Neural Models. arXiv:2102.07824 [cs, math] (en). Note: arXiv: 2102.07824 External Links: Link Cited by: §2. [64] C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. DasSarma, T. Henighan, B. Mann, A. Askell, Y. Bai, A. Chen, et al. (2022) In-context learning and induction heads. arXiv preprint arXiv:2209.11895. Cited by: §2. [65] S. E. Otto and C. W. Rowley (2019) Linearly recurrent autoencoder networks for learning dynamics. SIAM Journal on Applied Dynamical Systems 18 (1), p. 558–593. External Links: Document, Link, https://doi.org/10.1137/18M1177846 Cited by: §2. [66] M. Pals, A. E. Sağtekin, F. Pei, M. Gloeckler, and J. H. Macke (2024) Inferring stochastic low-rank recurrent neural networks from neural data. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37, p. 18225–18264. External Links: Document Cited by: §A.2, §A.2, §3.4. [67] D. Patel and E. Ott (2023-02) Using machine learning to anticipate tipping points and extrapolate to post-tipping dynamics of non-stationary dynamical systems. Chaos (Woodbury, N.Y.) 33 (2), p. 023143 (eng). External Links: ISSN 1089-7682, Document Cited by: §A.1, §4.2. [68] J. Pathak, Z. Lu, B. R. Hunt, M. Girvan, and E. Ott (2017-12) Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data. Chaos: An Interdisciplinary Journal of Nonlinear Science 27 (12), p. 121102. Note: arXiv: 1710.07313 External Links: ISSN 1054-1500, 1089-7682, Link, Document Cited by: §2. [69] L. Perko (2001) Differential equations and dynamical systems. 3rd ed edition, Texts in applied mathematics, Springer, New York (en). External Links: ISBN 978-0-387-95116-4 Cited by: §3.3. [70] J. A. Platt, S. G. Penny, T. A. Smith, T. Chen, and H. D. Abarbanel (2022) A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics. Neural Networks 153, p. 530–552. Cited by: §1, §2. [71] J. A. Platt, S. G. Penny, T. A. Smith, T. Chen, and H. D. Abarbanel (2023) Constraining chaos: enforcing dynamical invariants in the training of reservoir computers. Chaos: An Interdisciplinary Journal of Nonlinear Science 33 (10). Cited by: §1, §2, §4.2, §5. [72] M. T. Rosenstein, J. J. Collins, and C. J. De Luca (1993) A practical method for calculating largest lyapunov exponents from small data sets. Physica D: Nonlinear Phenomena 65 (1-2), p. 117–134. Cited by: §A.2. [73] C. Rudin (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1 (5), p. 206–215. External Links: Document Cited by: §1. [74] T. K. Rusch, S. Mishra, N. B. Erichson, and M. W. Mahoney (2022) Long expressive memory for sequence modeling. In International Conference on Learning Representations, Cited by: §2. [75] T. Sauer, J. A. Yorke, and M. Casdagli (1991) Embedology. Journal of statistical Physics 65 (3), p. 579–616. Cited by: §A.3. [76] E. E. SEL’KOV (1968) Self-oscillations in glycolysis 1. a simple kinetic model. European Journal of Biochemistry 4 (1), p. 79–86. Cited by: §A.4. [77] L. Shen, A. Mishra, and D. Khashabi (2024-21–27 Jul) Position: do pretrained transformers learn in-context by gradient descent?. In Proceedings of the 41st International Conference on Machine Learning, R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp (Eds.), Proceedings of Machine Learning Research, Vol. 235, p. 44712–44740. External Links: Link Cited by: §2. [78] Z. Sun (2006-03) Switched Linear Systems: Control and Design. Springer Science & Business Media (en). Note: Google-Books-ID: u4GArZN1bmsC External Links: ISBN 978-1-84628-131-0 Cited by: §4.3. [79] F. Takens (1981) Detecting strange attractors in turbulence. In Dynamical Systems and Turbulence, Warwick 1980, Vol. 898, p. 366–381 (en). External Links: ISBN 978-3-540-11171-9 978-3-540-38945-3, Link Cited by: §A.3. [80] A. P. Trischler and G. M.T. D’Eleuterio (2016) Synthesis of recurrent neural networks for dynamical system simulation. Neural Networks 80, p. 67–78 (en). External Links: ISSN 08936080, Link, Document Cited by: §2. [81] E. Tziperman, H. Scher, S. E. Zebiak, and M. A. Cane (1997-08) Controlling spatiotemporal chaos in a realistic el niño prediction model. Phys. Rev. Lett. 79, p. 1034–1037. External Links: Document, Link Cited by: §1. [82] A. Vermani, J. Nassar, H. Jeon, M. Dowling, and I. M. Park (2025) Meta-dynamical state space models for integrative neural data analysis. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §2. [83] P. R. Vlachas, W. Byeon, Z. Y. Wan, T. P. Sapsis, and P. Koumoutsakos (2018) Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks. Proc. R. Soc. A. 474 (2213), p. 20170844 (en). External Links: ISSN 1364-5021, 1471-2946, Link, Document Cited by: §2. [84] J. von Oswald, E. Niklasson, E. Randazzo, J. Sacramento, A. Mordvintsev, A. Zhmoginov, and M. Vladymyrov (2023) Transformers learn in-context by gradient descent. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 202, p. 35151–35174. External Links: Link Cited by: §2. [85] G. Wang, J. Li, Y. Sun, X. Chen, C. Liu, Y. Wu, M. Lu, S. Song, and Y. A. Yadkori (2025) Hierarchical reasoning model. External Links: 2506.21734, Link Cited by: §1. [86] Q. Wang, Y. Wang, X. Ying, and Y. Wang (2025) Can in-context learning really generalize to out-of-distribution tasks?. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §2. [87] R. Wang, Y. Dong, S. Ö. Arik, and R. Yu (2022-10) Koopman Neural Forecaster for Time Series with Temporal Distribution Shifts. arXiv (en). Note: arXiv:2210.03675 [cs, stat] External Links: Link Cited by: §2. [88] W. Wu, T. Hsiao, J. Y. Hu, W. Zhang, and H. Liu (2025-13–19 Jul) In-context learning as conditioned associative memory retrieval. In Proceedings of the 42nd International Conference on Machine Learning, A. Singh, M. Fazel, D. Hsu, S. Lacoste-Julien, F. Berkenkamp, T. Maharaj, K. Wagstaff, and J. Zhu (Eds.), Proceedings of Machine Learning Research, Vol. 267, p. 67300–67325. External Links: Link Cited by: §2. [89] S. M. Xie, A. Raghunathan, P. Liang, and T. Ma (2021) An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXiv:2111.02080. Cited by: §2. [90] Y. Yin, I. Ayed, E. de Bézenac, N. Baskiotis, and P. Gallinari (2021) Leads: learning dynamical systems that generalize across environments. Advances in Neural Information Processing Systems 34, p. 7561–7573. Cited by: §2. [91] Y. Zhang and W. Gilpin (2025) Zero-shot forecasting of chaotic systems. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §A.2, §1, §2, §2, §3.4, §4.2, §5. [92] Y. Zhang and W. Gilpin (2026) Context parroting: a simple but tough-to-beat baseline for foundation models in scientific machine learning. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §A.1, §B.2, §B.2, §B.2, 2nd item, §1, §2, §2, 1st item, §4.1, §4.2, §5, Theorem 2. Appendix A Appendix A.1 Methodological details Training details Training of the DynaBase parameters (α,β)(α,β) uses a sampled training sequence train∈ℝN×T X_train ^N× T drawn from a DS, where the first TCT_C steps serve as context signal C. For the experiments in Fig. 1, we set T=2000T=2000 and TC=1000T_C=1000. At evaluation time, we supplied the training sequence as context alongside the estimated parameter values (α,β)(α,β), and assessed performance on a sequence unseen during training and context construction (using the self-consistent DynaBase fαf_α as defined in eq. 12). This ensures that DynaBase receives exactly the same context and training sequence information as all other comparison methods. To fit DynaBase via linear regression, we compute (α^,β^)( α, β) in closed form via ordinary least-squares. Nearest neighbors τ(⋅)τ(·) are precomputed once via ℓ2 ^2-distances over C and the training sequence X. To cast this as a standard linear regression problem, we define the response vector b and design matrix A based on eq. 10, for each training sample i i b_i =i+1−τ(i)+1, = z_i+1- c_τ( z_i)+1, (16) i A_i =[i−τ(i)+1,τ(i)−τ(i)+1]. = [\; z_i- c_τ( z_i)+1\;,\; c_τ( z_i)- c_τ( z_i)+1\; ]. (17) The regression then takes the form =[α,β]⊤ b= A[α,β] , and the coefficients are obtained by the standard least-squares solution [α^,β^]⊤=(⊤)−1⊤.[ α, β] =( A A)^-1 A b. (18) In the self-consistent setting of eq. 12, the constraint β=−αβ=-α reduces the two-parameter fit to a single unknown scalar, simplifying i=i−τ(i) A_i= z_i- c_τ( z_i). To train parameters via grid search, we sweep across α∈[0,1.2]α∈[0,1.2] with step size Δα=0.002 α=0.002 along the self-consistency line β=−αβ=-α. For each candidate α, DynaBase is rolled out for T=10,000T=10,000 steps using the context C and scored by the KL-divergence loss given in eq. 14 between predicted trajectories X and the training data. The global minimizer α α is returned. Training and evaluation of DynaBase was performed on a single CPU (AMD EPYC 9655 96-Core). Comparison methods We compare DynaBase against three groups of baselines, all evaluated on the same 5454 test DS and the same input data of length T=2000T=2000 as in [44]. Zero-shot foundation models: DynaMix [44], the DSR FM that DynaBase is reduced from (mixture-of-experts of AL-RNNs with context-dependent gating; pretrained on ∼6×105 \!6× 10^5 trajectories from 3434 DS); Chronos-T5-base [5], a transformer FM that quantizes real-valued series into discrete tokens and forecasts autoregressively on a corpus of synthetic and real-world time series; and Chronos-2 [4], its multivariate extension augmenting the T5 backbone with a group-attention mechanism. Evaluations were taken from [44]. Custom-trained DSR models: an AL-RNN [13] (eq. 1), which is used for the experts in DynaMix, a Neural-ODE [23], and a reservoir computer [67]; each is trained from scratch on the same T=2000T=2000 context segment provided to the foundation models. Evaluations were taken from [44]. Context parroting: the algorithm of Zhang and Gilpin [92], which retrieves the best-matching motif in C and copies its successor as forecast. As shown in Thm. 2, this baseline corresponds to the α=0α=0 case of DynaBase on a delay-embedded context. A.2 Performance measures Long-term metrics To evaluate the geometric similarity between true and model-generated reconstructions, we employ a measure, DstspD_stsp, based on the Kullback-Leibler (KL) divergence evaluated in state space [52, 60, 38, 66, 91, 53, 44]. This measure quantifies the (mis)match between the ground-truth spatial distribution of trajectory points, ptrue()p_true( x), and the distribution pgen(|)p_gen( x| z) of points from trajectories freely generated by the model: Dstsp(ptrue()∥pgen(|))=∫ptrue()logptrue()pgen(|)d.D_stsp (p_true( x)\,\|\,p_gen( x| z) )= p_true( x) p_true( x)p_gen( x| z)\,d x. (19) In practice, we approximate this via a discrete binning of the state space into K=mNK=m^N bins, where m is the number of bins per dimension and N is the system dimensionality, estimating occupation probabilities through relative frequency of visits p^i p_i: Dstsp≈∑i=1Kp^true;ilogp^true;ip^gen;i.D_stsp≈ _i=1^K p_true;i p_true;i p_gen;i. (20) To ensure that the system has reached a steady-state distribution, long trajectories (T=10000T=10000) are sampled from the trained model. For the 3d attractors we set m=30m=30 bins per dimension, following [43]. To evaluate long-term temporal reconstruction quality, we calculate the Hellinger distance DHD_H between the power spectra of the ground-truth and model-generated time series [60, 45, 66, 29]. We first apply dimension-wise Fast Fourier Transforms (FFT) to the time series and then smooth the resulting power spectra with a Gaussian kernel (with σ=20σ=20), followed by normalization to enable comparison across dimensions. High-frequency tails dominated by noise are removed. The Hellinger distance between the smoothed true spectrum F(ω)F(ω) and the generated spectrum G(ω)G(ω) is given by DH(F(ω),G(ω))=1−∫−∞F(ω)G(ω)ω∈[0,1],D_H (F(ω),G(ω) )= 1- _-∞^∞ F(ω)\,G(ω)\,dω\;∈[0,1], (21) where values close to 0 indicate high spectral agreement and values close to 11 indicate complete disagreement. DHD_H is reported as the average across all dimensions. Note that both DstspD_stsp and DHD_H are designed to capture long-term reconstruction quality in the limit T→∞T→∞, after transients have settled and the attractor geometry has been traced out reasonably well (which was the case here for T=10000T=10000 steps). Short-term forecast quality For evaluating short-term predictive performance, we use the Mean Absolute Scaled Error (MASE) [46, 44], which is a normalized n-step-ahead prediction error aggregated across all N output dimensions: MASE=1N∑i=1N1n∑t=1n|xi,t−x^i,t|1T∑t=1T|xi,t−xi,t−1|,MASE= 1N _i=1^N 1n _t=1^n |x_i,t- x_i,t | 1T _t=1^T |x_i,t-x_i,t-1 |, (22) where T denotes the total length of the ground truth time series, n is the forecast horizon, xi,tx_i,t the true value, and x^i,t x_i,t the model prediction for dimension i at time t. We evaluate at n=10n=10 steps. Lyapunov exponent To numerically estimate the maximum Lyapunov exponent λmax _max from a time series X, we use the Rosenstein algorithm [72]. For each state i x_i, its nearest neighbor j x_j is identified subject to |i−j|>lt|i-j|>l_t, which excludes direct temporal neighbors on the same trajectory segment. We set lt=100l_t=100 (based on the mean periodicity [72]). The divergence between neighboring trajectories is then tracked over time steps k=0,…,kmaxk=0,…,k_ , di(k)=‖i+k−j(i)+k‖2.d_i(k)=\| x_i+k- x_j(i)+k\|_2\;. (23) Since nearby trajectories diverge as di(k)≈di(0)eλmaxkΔt,d_i(k)≈ d_i(0)\,e _maxk t\;, (24) where Δt t is the sampling interval, taking the logarithm and averaging over all pairs gives ⟨lnd(k)⟩≈λmaxkΔt+C. d(k) ≈ _ k t+C\;. (25) For chaotic systems, this quantity grows approximately linearly with k for a suitable kmaxk_ (here: 100100) [49], with slope λmax>0 _ >0, whereas periodic systems yield λmax=0 _ =0. When the equations of motion ˙=f() x=f(x) are known, the full Lyapunov spectrum λi\ _i\ can be computed via a QR decomposition algorithm [9], which integrates the system along its variational equations Q˙=J()Q Q=J(x)Q, where J()=∂f/∂J( x)=∂ f/∂ x is the Jacobian of f and Q is an n×n× n matrix of perturbation vectors initialized to the identity InI_n. At regular time intervals, a QR decomposition Q=Q~RQ= QR is performed and the columns of Q~ Q, which form an orthonormal basis, replace Q. The diagonal entries RiiR_i record the local expansion rates, and the k-th Lyapunov exponent is obtained as the time-averaged logarithmic growth rate, λk=limT→∞1T∑jlog|Rkk(j)|, _k= _T→∞ 1T _j |R_k^(j) |, (26) where the sum runs over all QR factorizations performed up to time T. A.3 Details on DynaMix reduction Table 1: Zero-shot performance of DynaMix and model simplifications across all 54 test DS used in [44]. DynaMix and simplified models are compared for the same context length TC=2000T_C=2000. Reported are median± for geometrical disagreement DstspD_stsp, temporal disagreement DHD_H, and 10-step-ahead prediction error MASE Model Dstsp↓D_stsp DH↓D_H MASE↓MASE Original DynaMix 3.80±1.403.80± 1.40 0.16±0.060.16± 0.06 0.35±0.170.35± 0.17 Linear experts & gating (M>NM>N) 3.10±1.193.10± 1.19 0.13±0.070.13± 0.07 0.26±0.110.26± 0.11 Linear experts & gating (M=NM=N) 2.85±1.302.85± 1.30 0.13±0.060.13± 0.06 0.23±0.100.23± 0.10 Here we provide the supporting evidence and derivation details for the successive reduction of DynaMix to the recursive affine N map eq. 7 described in Sec. 3.2. Linear approximation As noted in [44], replacing the MLP in eq. 5 with a linear layer does not reduce performance. Here we further show that dropping the softmax nonlinearity σ in eq. 3 and the piecewise-linear activation Φ∗ ^* in eq. 1, reducing the mixture of AL-RNN experts to a single affine map FLinearF_Linear as in eq. 6, does not cause performance to degrade either. Table 1 confirms this: Linear experts & gating matches, or even improves upon, the original DynaMix checkpoint on all three measures across the DS test set. Latent dimension The AL-RNN experts in DynaMix operate in an M-dimensional latent space distinct from the N-dimensional observation space, requiring the projection D in eq. 3. Replacing the expert block with FLinearF_Linear eliminates this distinction, yielding M=NM=N and = D= I, such that the model operates entirely in observation space. Table 1 (Linear experts & gating, M=NM=N) confirms this does not reduce performance on the present test set. Working in a single space is also a prerequisite for the O(N)O(N) equivariance exploited in Thm. 1, since rotations and the nearest-neighbor distance must act on the same space in which the state t z_t and the context C live. Note we can always perform a classical delay-embedding [79, 75] should this dimensionality not be sufficient to properly represent an empirically observed DS. Nearest-neighbor selector In the publicly available DynaMix checkpoint, the learned attention temperature τatt _att converged to |τatt,learned|≈0.06| _att,learned|≈ 0.06, indicating that the soft attention eq. 3 is already operating close to its τatt→0 _att→ 0 limit, i.e. effectively selecting a single nearest context point at each step. Two deliberate departures from DynaMix are made in passing to the hard limit. First, DynaMix injects Gaussian exploration noise ε∼(0,) (0, ) into the attention eq. 3; in favor of a minimal and deterministic model, we set =0 =0. Second, DynaMix computes context distances with the ℓ1 ^1-norm, whereas we adopt the Euclidean ℓ2 ^2-norm. The invariance of ℓ2 ^2 distances under orthogonal transformations R∈O(N)R∈ O(N), is used in Thm. 1. Together these choices yield the deterministic selector τ(t)=argmins‖s−t‖2τ( z_t)= *arg\,min_s\| c_s- z_t\|_2. CNN expansion To make the recursive map fully explicit in terms of individual context points, we expand the CNN operation. Since DynaMix uses a single linear CNN layer with kernel size 22, for s∈1…TC−1s∈ 1… T_C-1 we have ~s=CNN()s=(1)s+(2)s+1+, c_s=CNN( C)_s= K^(1) c_s+ K^(2) c_s+1+ b, (27) where (1),(2)∈ℝN×N K^(1), K^(2) ^N× N and ∈ℝN b ^N are learned weights. Substituting into eq. 6 with the nearest-neighbor selector and absorbing (1) B K^(1), (2) B K^(2) and the bias into (1) B^(1), (2) B^(2), and ~ h respectively, yields the recursive affine N map eq. 7 as stated in the main text. A.4 Datasets Our DS dataset consists of simulated time series of length 10510^5, produced from 3d3d DS collected in [37]. For the test set, we used the same 5454 distinct systems as in [44] to enable a fair comparison among all the baseline models in relation to the originally published results. To evaluate DynaBase in a zero-shot setting, we estimated α using a training set of 3030 different chaotic DS, analogous to the setup in [44]. Figure 5: DSR test set of 54 different DS gathered from [37]. Ground truth (gray) and DynaBase reconstructions using a context window of TC=2000T_C=2000 (red). In addition, we conduct experiments on the 2d2d Selkov system [76], which models the kinetics of an open monosubstrate enzyme reaction and is defined by the equations dxdt dxdt =−x+ay+x2y, =-x+ay+x^2y, (28) dydt dydt =b−ay−x2y, =b-ay-x^2y, where we chose a=0.1a=0.1 and b=0.5b=0.5. The system was solved numerically with integration time step Δt=0.3 t=0.3 using scipy.integrate with the RK45 solver. For Fig. 12 we used the Lorenz-63 attractor [57], originally formulated by Edward Lorenz in 1963 [57] to model atmospheric convection. It is defined by dxdt dxdt =σ(y−x) =σ(y-x) (29) dydt dydt =x(ρ−z)−y =x(ρ-z)-y dzdt dzdt =xy−βz, =xy-β z, where σ,ρ,βσ,ρ,β, are control parameters of the system (we fix σ=10σ=10 and ρ=28ρ=28, and vary β∈[0,1]β∈[0,1], producing regimes with point attractors, cyclic, and chaotic dynamics). The system was solved numerically with integration time step Δt=0.02 t=0.02 using scipy.integrate with the RK45 solver. A.5 Further empirical results Figure 6: Zero-shot reconstruction of chaotic Lorenz-63 system using DynaBase with α=1.006α=1.006. Left: State space, center: time graphs, right: power spectrum. Figure 7: Zero-shot reconstruction of chaotic Sprott-F system using DynaBase with α=1.006α=1.006. Left: State space, center: time graphs, right: power spectrum. Figure 8: Zero-shot reconstruction of chaotic Sprott-D system using DynaBase with α=1.006α=1.006. Left: State space, center: time graphs, right: power spectrum. Figure 9: Zero-shot reconstruction using α=1α=1 of cyclic Lorenz-63 system using DynaBase. Left: State space, center: time graphs, right: power spectrum. Figure 10: Illustration of context parroting and flow mimicry. Left: context parroting — at (α,β)=(0,0)(α,β)=(0,0) the state snaps onto the successor τ(t)+1 c_τ( z_t)+1 of its nearest context neighbor, producing a piecewise-constant map that replays the context. Right: flow mimicry — at (α,β)=(1,−1)(α,β)=(1,-1) the state is updated by adding the finite-difference increment τ(t)+1−τ(t) c_τ( z_t)+1- c_τ( z_t) evaluated at the nearest context neighbor, a local Euler step on the context vector field. Figure 11: Lyapunov exponents estimated via the Rosenstein algorithm (see Appx. A.2) across test DS for both linear regression (blue) and DSR-based training (red) vs. ground truth exponents. Only DSR training produces long-term dynamics that closely follows that of the ground truth, including chaotic attractors with λmax>0 _max>0. Figure 12: Different reconstructions by DynaBase along a bifurcation diagram of the Lorenz-63 system with system parameters σ=10σ=10, ρ=28ρ=28 and β∈[0,1]β∈[0,1]. For each reconstruction, the estimated alpha value together with the GT system’s true Lyapunov exponent (estimated via QR decomposition, see Sec. A.2) is indicated. Figure 13: DynaBase forecasting of a chaotic laser system [37], using the same context once with α>1α>1 and once with α=0α=0. Note that despite the system’s non-periodic context trajectory, parroting (α=0α=0) incorrectly produces a discrete cycle. Shown are time graphs (left), state space (center), and power spectrum (right). Figure 14: DynaBase reconstruction of Lorenz-63 adding 15%15\% Gaussian noise to the context signal. Left: State space, center: time graphs, right: power spectrum. Table 2: Short-term forecasting accuracy (MASE, median ± MAD) on the DS test set for three DynaBase training strategies (linear regression (LR), DSR objective, and zero-shot) across various prediction horizons n. Best median values in bold. Note that LR training outperforms the other approaches on this measure for intermediate forecasting horizons, while for larger horizons (n=300n=300) performance differences narrow down again due to the general limitations caused by exponential trajectory divergence in chaotic systems. Model MASE(n=10)MASE\;(n=10) MASE(n=50)MASE\;(n=50) MASE(n=100)MASE\;(n=100) MASE(n=200)MASE\;(n=200) MASE(n=300)MASE\;(n=300) DynaBase LR 0.13±0.08 0.13± 0.08 0.51±0.40 0.51± 0.40 0.71±0.62 0.71± 0.62 1.06±0.93 1.06± 0.93 2.64±2.18 2.64± 2.18 DynaBase DSR & Gating 0.13±0.08 0.13± 0.08 0.62±0.480.62± 0.48 1.06±0.891.06± 0.89 2.22±1.642.22± 1.64 2.80±1.962.80± 1.96 DynaBase zero-shot 0.13±0.07 0.13± 0.07 0.61±0.480.61± 0.48 0.94±0.800.94± 0.80 2.29±1.692.29± 1.69 3.07±2.093.07± 2.09 A.6 Further theoretical results Details on continuous-time limit. We begin by writing eq. 12 explicitly as a piecewise-affine discrete-time system k+1=fα(k)=αk+(τ(k)+1−ατ(k)). z_k+1=f_α( z_k)=α I z_k+( c_τ( z_k)+1-α c_τ( z_k))\ . (30) For values of α∈ℝ/0,1α /\0,1\, Thm. 3 in [61] guarantees the existence of an equivalent affine continuous-time system, in the sense that (t0)=k0,(t0+h)=k0+1, ζ(t_0)= z_k_0,\; ζ(t_0+h)= z_k_0+1\ , (31) where h denotes the discrete timestep at which observations k z_k are sampled. The equivalent continuous-time system is then given by ddt=logαh−1hlogα1−α(τ()+1−ατ()). ζt= αh ζ- 1h α1-α( c_τ( ζ)+1-α c_τ( ζ))\ . (32) For the case α=1α=1, we define log(α)1−α|α=1:=−1 α1-α _α=1:=-1 by its continuous extension. Such an extension does, however, not exist for α=0α=0. For α<0α<0, the logarithm is complex-valued. Hence, a real-valued continuous-time limit for fαf_α exists only for α>0α>0. Computational complexity The minimal architecture of DynaBase makes it computationally highly efficient for zero-shot inference. By exploiting discrete nearest-neighbor lookup and efficient heuristic search algorithms such as k-d trees [34], the time complexity of running DynaBase autoregressively over TFT_F forecast steps is =(TFlogTC),T=O(T_F T_C), (33) with a one-time overhead of (TClogTC)O(T_C T_C) for constructing the k-d tree. In higher dimensions, query time degrades toward (TC)O(T_C) in the worst case. Provided the generated trajectory is not retained, the space complexity reduces to =(TC).S=O(T_C). (34) These bounds allow DynaBase to generate long forecasts over extended contexts at low computational cost — in contrast to standard transformer architectures, which incur quadratic time and space complexity in context length. Appendix B Proofs of Theorems In the following we use the same notation as introduced in Sec. 3.3, with =1,…,TC∈ℝN×TC C=\ c_1,…, c_T_C\ ^N× T_C a context sequence, τ()=argmins‖s−‖2τ( z)= *arg\,min_s c_s- z_2 the nearest-neighbor lookup, and fα,β()=α+βτ()+γτ()+1,γ=1−α−β,f_α,β( z)=α z+β c_τ( z)+γ c_τ( z)+1, γ=1-α-β, (35) the two-parameter DynaBase map; fαf_α denotes its self-consistent reduction with β=−αβ=-α, γ=1γ=1. Ties in τ are broken by the smallest index; the tie set has measure zero and does not affect any of the statements below. B.1 Proof of Theorem 1 (Uniqueness of form) Proof. We start from the affine ansatz eq. 7, fLinear(,)=+(1)τ()+(2)τ()+1+~,f_Linear( z, C)= A z+ B^(1) c_τ( z)+ B^(2) c_τ( z)+1+ h, (36) with ,(1),(2)∈ℝN×N A, B^(1), B^(2) ^N× N and ∈ℝN h ^N, and impose the two structural assumptions in turn. Orthogonal equivariance. O(N)O(N)-equivariance requires QfLinear(,)=fLinear(Q,Q)Q\,f_Linear( z, C)=f_Linear(Q z,Q C) for all Q∈O(N)Q∈ O(N), where Q:=QssQ C:=\Q c_s\_s. Since τ is itself O(N)O(N)-equivariant, substituting eq. 36 and matching coefficients of the independent vectors ,τ(),τ()+1 z, c_τ( z), c_τ( z)+1 gives Q=Q,Q(1)=(1)Q,Q(2)=(2)Q,Q=,∀Q∈O(N).Q A= AQ, Q B^(1)= B^(1)Q, Q B^(2)= B^(2)Q, Q h= h, ∀ Q∈ O(N). (37) The standard representation of O(N)O(N) on ℝNR^N is irreducible for N≥2N≥ 2, so by Schur’s lemma every commuting matrix is a scalar multiple of the identity, and the only O(N)O(N)-fixed vector is 0: =α,(1)=β,(2)=γ,=,α,β,γ∈ℝ. A=α I, B^(1)=β I, B^(2)=γ I, h= 0, α,β,γ . (38) Hence eq. 36 reduces to f()=α+βτ()+γτ()+1f( z)=α z+β c_τ( z)+γ c_τ( z)+1. Translation equivariance. For any ∈ℝN v ^N, applying the joint shift ↦+ z z+ v, ↦+ C C+ v leaves τ invariant. Substituting into the previous form, f(+;+)=α(+)+β(τ()+)+γ(τ()+1+)=f(;)+(α+β+γ).f( z+ v; C+ v)=α( z+ v)+β( c_τ( z)+ v)+γ( c_τ( z)+1+ v)=f( z; C)+(α+β+γ) v. (39) Translation equivariance demands f(+;+)=f(;)+f( z+ v; C+ v)=f( z; C)+ v for all v, hence α+β+γ=1α+β+γ=1. Setting γ=1−α−βγ=1-α-β yields the two-parameter family fα,β()=α+βτ()+(1−α−β)τ()+1.f_α,β( z)=α z+β c_τ( z)+(1-α-β)\, c_τ( z)+1. (40) ∎ B.2 Proof of Theorem 2 (Reduction to context parroting) Proof. For α=β=0α=β=0, the DynaBase map on the delay-embedded context ¯1:TC−D+1∈ℝD×(TC−D+1) C_1:T_C-D+1 ^D×(T_C-D+1) reduces to the deterministic lookup t+1=f0(t)=¯τ(t)+1,τ()=argmin1≤s≤TC−D‖¯s−‖2. z_t+1=f_0( z_t)= c_τ( z_t)+1, τ( z)= *arg\,min_1≤ s≤ T_C-D c_s- z_2. (41) Initial step. Starting from 0=¯TC−D+1 z_0= c_T_C-D+1 (the most recent D context entries), the first application of eq. 41, assuming the final D steps are omitted for τ(0)τ( z_0) (see [92]), yields s⋆:=τ(0)=argmin1≤s≤TC−2D‖¯s−¯TC−D+1‖2.s :=τ( z_0)= *arg\,min_1≤ s≤ T_C-2D c_s- c_T_C-D+1_2. (42) Since ¯s=(cs,…,cs+D−1) c_s=(c_s,…,c_s+D-1), this distance is the Euclidean distance between two D-length motifs of the original context—identical to the motif-matching step of [92, Algorithm 1]. Hence s⋆s coincides with the best-matching motif index produced by context parroting. Induction step. Suppose k=¯s⋆+k z_k= c_s +k with s⋆+k≤TC−Ds +k≤ T_C-D. Then τ(k)=s⋆+kτ( z_k)=s +k at zero distance, and k+1=¯s⋆+k+1=(cs⋆+k+1,…,cs⋆+k+D). z_k+1= c_s +k+1=(c_s +k+1,…,c_s +k+D). (43) By induction, t=¯s⋆+t z_t= c_s +t for all 0≤t≤TC−D−s⋆0≤ t≤ T_C-D-s . Extracting observations from the delay embedded vectors results in xt=zD,t=cs⋆+t+D−1,x_t=z_D,t=c_s +t+D-1, (44) which is exactly the forecast emitted by [92, Algorithm 1] after locating the best-matching motif s⋆s . Wrap-around. Once s⋆+ts +t exceeds TC−DT_C-D, both algorithms treat the current tail as a new query and repeat the motif search. Both procedures execute the same operation, so the equivalence extends to arbitrary forecast horizons. ∎ B.3 Proof of Theorem 3 (Long-term dynamics) Proof. For α=0α=0, fαf_α reduces to f0()=τ()+1f_0( z)= c_τ( z)+1, with finite range 2,…,TC\ c_2,…, c_T_C\. After one iteration every orbit lies in this finite set, on which f0f_0 acts as the deterministic graph map s↦τ(s)+1=s+1 c_s c_τ( c_s)+1= c_s+1 for s<TCs<T_C. Each node has out-degree exactly one, so iteration on a finite directed functional graph must enter a cycle in at most TC−1T_C-1 steps and remain periodic thereafter. The differential of f0f_0 vanishes on the interior of every Voronoi cell (the map is piecewise constant), so the Lyapunov exponent within each cell is λmax=−∞<0 _max=-∞<0 everywhere. Since the saltation operator s,s+1=S_s,s+1=I between two Voronoi cells VsV_s and Vs+1V_s+1 is always finite and nonzero for α=0α=0, we obtain λmax=−∞ _max=-∞ globally. ∎ B.4 Proof of Theorem 4 (Boundedness) Proof. For ∈ℝN z ^N define the distance to the context set d():=min∈‖−‖2=‖−τ()‖2.d_ C( z):= _ c∈ C c- z_2= z- c_τ( z)_2. (45) Sufficiency (α∈[0,1]α∈[0,1]). Fix any t z_t and let t+1=fα(t)=α(t−τ(t))+τ(t)+1 z_t+1=f_α( z_t)=α( z_t- c_τ( z_t))+ c_τ( z_t)+1 as in eq. 12. Since τ(t)+1∈ c_τ( z_t)+1∈ C, d(t+1)≤‖t+1−τ(t)+1‖2=|α|‖t−τ(t)‖2=|α|d(t).d_ C( z_t+1)≤ z_t+1- c_τ( z_t)+1_2=|α|\, z_t- c_τ( z_t)_2=|α|\,d_ C( z_t). (46) For α∈[0,1]α∈[0,1], this gives d(t+1)≤d(t)d_ C( z_t+1)≤ d_ C( z_t). By induction, d(t)≤d(0)d_ C( z_t)≤ d_ C( z_0) for all t≥0t≥ 0, so ‖t‖2≤max1≤s≤TC−1‖s‖2+d(0)<∞, z_t_2≤ _1≤ s≤ T_C-1 c_s_2+d_ C( z_0)<∞, (47) and the orbit is bounded. Non-necessity (α>1α>1 counterexample). Let N=1N=1, α=2α=2, and =(c1,c2,c3)=(−1,1,−1) C=(c_1,c_2,c_3)=(-1,1,-1). The induced Voronoi cells are V1=z≤0V_1=\z≤ 0\ and V2=z>0V_2=\z>0\, with successors c2=1c_2=1 and c3=−1c_3=-1, respectively. Substituting into eq. 12 gives the piecewise-affine map f2(z)=2z+3,z≤0,2z−3,z>0.f_2(z)= cases2z+3,&z≤ 0,\\ 2z-3,&z>0. cases (48) A direct check shows f2([−3,3])⊆[−3,3]f_2([-3,3]) [-3,3]: for z∈[−3,0]z∈[-3,0] we have 2z+3∈[−3,3]2z+3∈[-3,3], and for z∈(0,3]z∈(0,3] we have 2z−3∈(−3,3]2z-3∈(-3,3]. Hence any orbit with z0∈[−3,3]z_0∈[-3,3] remains in [−3,3][-3,3] for all t≥0t≥ 0, proving bounded dynamics for α=2>1α=2>1. This map is conjugate to the doubling/sawtooth map xt+1=2xtmod1x_t+1=2x_t 1, whose Lyapunov exponent equals log2 2 on its invariant set. ∎