Paper deep dive
Temporal Geometry of Deep Networks: Hyperbolic Representations of Training Dynamics for Intrinsic Explainability
Ambarish Moharil
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Intrinsic explainability remains a challenging problem, particularly in contexts where multilayer perceptrons (MLPs) require dynamic re-training within an optimization environment. This paper investigates how MLPs and their training dynamics can be represented and studied in non-Euclidean spaces; our representation features the Poincaré model of hyperbolic geometry. We aim to capture the geometric evolution of their weighted topology and self-organization over time. Instead of restricting the analysis to single checkpoints---as per established measure-based explainability methods---we construct temporal \textit{parameter graphs}, i.e., snapshots over time $T$ steps of the optimization/training process for MLPs. This reflects the view that neural networks encode information not only in their weights but also in the trajectory traced during training. Drawing on the idea that many complex networks admit embeddings in hidden metric spaces where distances correspond to connection likelihood, we present a geometric and temporal graph-based metalearning framework for obtaining dynamic hyperbolic representations of the underlying neural parameter graphs. Our model embeds temporal parameter graphs in the Poincaré model ball, and learns from them while maintaining equivariance to within-snapshot neuron permutations and invariance to permutations of past snapshots. In doing so, the approach preserves functional equivalence over time and recovers the latent evolving geometry of the network. Experiments on regression and classification tasks with trained MLPs show strong meta-network performance, accompanied by hyperbolic temporal representations. This reveals how the network structure emerges over time under specific training environments, thus providing insights into the network's self-organization.
Tags
Links
- Source: https://arxiv.org/abs/2610.03000v1
- Canonical: https://arxiv.org/abs/2610.03000v1
Trouble viewing inline? Open PDF directly →
Full Text
146,792 characters extracted from source content.
Expand or collapse full text
Temporal Geometry of Deep Networks: Hyperbolic Representations of Training Dynamics for Intrinsic Explainability Ambarish Moharil Affiliation: Data Governance Department Affiliation: Jheronimus Academy of Data Science Affiliation: Eindhoven University of Technology Email: a.moharil@tue.nl Abstract Intrinsic explainability remains a challenging problem, particularly in contexts where multilayer perceptrons (MLPs) require dynamic re-training within an optimization environment. This paper investigates how MLPs and their training dynamics can be represented and studied in non-Euclidean spaces; our representation features the Poincaré model of hyperbolic geometry. We aim to capture the geometric evolution of their weighted topology and self-organization over time. Instead of restricting the analysis to single checkpoints—as per established measure-based explainability methods—we construct temporal parameter graphs, i.e., snapshots over time T steps of the optimization/training process for MLPs. This reflects the view that neural networks encode information not only in their weights but also in the trajectory traced during training. Drawing on the idea that many complex networks admit embeddings in hidden metric spaces where distances correspond to connection likelihood, we present a geometric and temporal graph-based metalearning framework for obtaining dynamic hyperbolic representations of the underlying neural parameter graphs. Our model embeds temporal parameter graphs in the Poincaré model ball, and learns from them while maintaining equivariance to within-snapshot neuron permutations and invariance to permutations of past snapshots. In doing so, the approach preserves functional equivalence over time and recovers the latent evolving geometry of the network. Experiments on regression and classification tasks with trained MLPs show strong meta-network performance, accompanied by hyperbolic temporal representations. This reveals how the network structure emerges over time under specific training environments, thus providing insights into the network’s self-organization. 1 Introduction The central problem that we address in this work is the geometric evolution of neural networks, specifically how the weighted topology of a deep neural network (DNN) model reorganizes during training and how this self-organization shapes the (neural network) function. We argue that studying the weighted topology is key to figure out the role of neural weights, that is the parameters used for any decision, are not just incidental outputs of some optimization process, but also the structures through which generalization, compression, and transfer learning arise (Han et al., 2016; Li et al., 2020). Our focus in this work is not only to investigate the performance of hyperbolic and temporal meta-networks, but also to expose how a neural network’s structure forms and shifts during training. A temporal geometric view makes it possible to track the emergence of hierarchy, the consolidation of functional modules, and the redistribution of influence across layers by studying the evolution of its parameter graph. This perspective offers intrinsic explanations grounded in the model’s own dynamics and self-organization, rather than relying on post-hoc probes for explanation. A recent approach has been to treat neural networks as data objects themselves and to train meta-networks that take neural parameters as inputs to predict the generalization performance of the underlying model (accuracy, R2R^2, etc.), classifying the implicit neural representations (INRs) of the underlying neural model or to generate weights for transfer learning across domains (Unterthiner et al., 2021; Schürholt et al., 2021; Dupont et al., 2022; Navon et al., 2024; Zhou et al., 2023b; You et al., 2020; Lim et al., 2023; Kofinas et al., 2024). Early works on the topic flattened weights and biases into a single tensor, but this discarded structural relations and ignored the underlying geometric priors of the weight space, such as functional invariance and equivariance to permutations (Hecht-Nielsen, 1990; Kofinas et al., 2024). Later work enforced data symmetries, such as permutation equivariance, ensuring that reordering the neurons did not affect the network functionality (Zhou et al., 2023a; Zhou et al., 2023b; Navon et al., 2024; Hecht-Nielsen, 1990). These methods, although robust, often require architecture-specific tuning and Euclidean representations, making it challenging to interpret the network’s internal self-organization. A complementary direction has leveraged graph-theoretic and network-science perspectives. Here, neurons and biases become network nodes, whereas the model weights are tied to the existence of an edge link between two nodes, with meta-learning being performed on these parameter graphs (You et al., 2020; Lim et al., 2023; Kofinas et al., 2024). These methods have performed well for INR classification and generalization prediction tasks, but usually operate on single checkpoints (along the training trajectory) and make zero-shot predictions, leaving the temporal traces of optimization underused (Lim et al., 2023; Kofinas et al., 2024). On the other hand, few studies on complex network graphs in physical domains (Simas and Rocha, 2014; Allard et al., 2017) have considered how geometric priors could inform edge weight dynamics; a link that can be crucial in making deep learning interpretable. Network science points to two priors of particular relevance: small-world effects and modularity (Song et al., 2005; Kim et al., 2007a; Wei et al., 2013; Fronczak et al., 2024). They also often admit embeddings in metric spaces where the distances reflect the connection probabilities (Allard et al., 2017; Krioukov et al., 2010). If parameter graphs display similar tendencies, embedding them in metric spaces with distance-biased learning can yield compact, interpretable representations. Most existing methods use Euclidean embeddings (Lim et al., 2023; Kofinas et al., 2024), but these require high dimensions and distort structure when projected (Nickel et al., 2014; Nickel and Kiela, 2017). On the other hand, hyperbolic spaces preserve hierarchical relations with low distortion (Gromov, 1987), yet no prior work has embedded neural parameter graphs within hyperbolic spaces while preserving permutation symmetry. The closest example addresses static graphs of multilayer brain networks (Guillemaud et al., 2025). We close this gap by developing a geometric temporal hyperbolic framework for parameter graphs that treats training itself as an object of study. Conceptually, we view the optimization of a neural network as a trajectory across weighted parameter graphs evolving in a negatively curved space, and we argue that intrinsic explanations should be grounded in this evolution rather than in isolated checkpoints. Methodologically, we (a) construct a temporal hyperbolic meta network (GTH–GMN) that (b) models MLP parameter traces as permutation equivariant parameter graphs, (c) embedding them in the Poincaré ball, and (d) coupling distance-biased hyperbolic attention with meta evolution of the attention kernels over time (Pareja et al., 2019; Li et al., 2024). The architecture is designed to respect the symmetries of weight space: it is equivariant to the neuron permutations within each snapshot and invariant to permutations of past snapshots, so that functionally equivalent networks share the same representation, while their latent geometry can still be recovered. A signed weight regressor links geodesic distance to edge magnitude through a power law prior (Allard et al., 2017), decoupling magnitude from polarity and importing scale free inductive biases from network science (Song et al., 2005; Kim et al., 2007a; Wei et al., 2013; Fronczak et al., 2024). Empirically, we show on INR classification, CIFAR–10 generalization prediction, and sinusoid regression, that this temporal hyperbolic encoder matches or heavily approaches strong Euclidean and tensor-based baselines, while producing compact embeddings whose radial and angular structure tracks the self-organization of the underlying networks. In this way, the method offers a concrete route toward intrinsic explainability through the geometry of parameter graph trajectories (Unterthiner et al., 2021; Schürholt et al., 2021; Dupont et al., 2022; Navon et al., 2024; Zhou et al., 2023b; You et al., 2020; Lim et al., 2023; Kofinas et al., 2024; Allard et al., 2017; Krioukov et al., 2010; Nickel et al., 2014; Nickel and Kiela, 2017; Gromov, 1987; Guillemaud et al., 2025; Pareja et al., 2019; Li et al., 2024). 2 Related Work Neural Meta Networks. Neural networks can themselves be treated as data. Early meta-network approaches flattened parameters or relied on simple statistics, which ignored neuron permutation symmetry and had limited cross-architecture generalization (Unterthiner et al., 2021; Eilertsen et al., 2020; Dupont et al., 2022; De Luigi et al., 2023). Subsequent works developed permutation-equivariant meta-networks that respect weight-space symmetries (Schürholt et al., 2021; Navon et al., 2024; Zhou et al., 2023a; Zhou et al., 2023b; Maron et al., 2019). Recently, Graph-based approaches have extended this view by modeling parameters as graphs (Lim et al., 2023; Kofinas et al., 2024). Graph Meta Networks (GMN) construct parameter graphs to learn graph neural networks (GNNs) that remain equivariant to neural DAG automorphisms, covering CNNs, Transformers, and multilayer perceptrons (MLPs) (Lim et al., 2023). These graph-based approaches model individual neurons and biases as parameter graph nodes, where edge existence depends on the weight between adjacent nodes, enabling GNNs or relational transformers to process architectures with varying depth, width, and skip connections (Lim et al., 2023; Kofinas et al., 2024). Other work has used higher-level graphs for NAS, federated learning, and parameter sharing (Liu et al., 2018; You et al., 2020; Litany et al., 2022), though often at a coarser level of abstraction. GNN provides a foundation of permutation-equivariant operators for nodes and edges (Hamilton, 2020; Maron et al., 2019; Kim et al., 2021; Lim et al., 2023). Our contribution builds on this trajectory but addresses two gaps. First, most approaches rely on single checkpoints and do not exploit the temporal traces generated during optimization. Second, embeddings are almost always Euclidean, which limits their capacity to represent hierarchical or heavy-tailed structures. We therefore focus on temporal parameter graphs of MLPs, embedding them in hyperbolic space to capture both the evolving geometry and functional symmetries over the course of training. Hyperbolic GNNs. Poincaré and Lorentz embeddings first established that hierarchical relations admit compact, faithful representations in negatively curved spaces (Nickel and Kiela, 2017; Nickel and Kiela, 2018) ( For background on hyperbolic geometry, refer to Appendix B). Building on this insight, hyperbolic neural models extended message passing, attention, and diffusion to non-Euclidean manifolds for static graphs (Ganea et al., 2018; Chami et al., 2019; Zhang et al., 2019; Wen et al., 2024). The key challenge is that hyperbolic spaces are not vector spaces, so Euclidean modules do not apply verbatim (Yang et al., 2023). Two strategies have emerged: design fully manifold counterparts for core layers (Chen et al., 2021), or lift computations to tangent spaces and map back while enforcing model-specific constraints (Ganea et al., 2018; Chami et al., 2019; Zhang et al., 2021). Manifold optimization further supports these pipelines through Riemannian adaptive methods (Sakai and Iiduka, 2020). In our framework, node embeddings are also updated intrinsically: alongside the Euclidean parameter pass for kernels and regressors, we run a dedicated Riemannian optimizer that updates positions z(t)z^(t) on the Poincaré ball in a geometry-consistent second phase. Temporal Learning in Euclidean and Hyperbolic Spaces. Early approaches to temporal graph learning largely relied on Euclidean neural architectures. Sequence-aware graph models combined recurrent updates with graph convolutions. For example, the spatial operators in convolution-based long-short term memory (LSTM) networks were replaced with Chebyshev filters so that temporal dynamics and topology could be processed jointly (Seo et al., 2016; Shi et al., 2015; Defferrard et al., 2017). Later work introduced uncertainty modeling over time by placing latent distributions at each step (Hajiramezanali et al., 2019), and eventually allowed the graph encoder itself to evolve through recurrent updates, so the parameters of the convolutional layers changed along with the temporal signal (Pareja et al., 2019). However, Euclidean geometry can flatten or obscure the heavy-tailed and hierarchical patterns that naturally arise in many temporal networks (Krioukov et al., 2010; Nickel and Kiela, 2017). This motivated the development of temporal models built directly in hyperbolic space, where message passing and recurrence are designed to respect the curvature. These methods combine hyperbolic graph layers, hyperbolic recurrent units, and temporal attention mechanisms, along with geometric consistency losses that stabilize learning (Yang et al., 2023). Other approaches use hyperbolic diffusion operators, dilated causal convolutions, and curved decoders adapted to long-horizon forecasting (Bai et al., 2023). Hyperbolic attention variants compute similarity using geodesic distances and perform aggregation using Einstein gyromidpoints. Extensions that incorporate temporal context improve multi-step forecasting and link prediction (Li et al., 2024; Li et al., 2023). Continuous time formulations push this further by combining hyperbolic variational encoders with temporal encodings for event stream modeling (Sun et al., 2021a). Across these developments, the unifying ideas are operators that preserve the manifold, mappings that respect curvature, and explicit mechanisms that maintain temporal coherence. Despite this progress, difficulties remain in scaling hyperbolic temporal models and in adapting curvature over long sequences (Zhang et al., 2021; Yang et al., 2023). To address these limitations, we follow the kernel evolution strategy introduced in recurrent graph encoders (Pareja et al., 2019), which updates the parameters of the attention kernels directly over time. This avoids storing the entire sequence of hyperbolic embeddings, significantly reduces memory usage, and enables efficient learning on long optimization traces while maintaining temporal smoothness (Pareja et al., 2019) (related work in Appendix C). 3 Research Method Figure 1 provides an overview of our approach. Given a training trace of an MLP, we encode each checkpoint as a parameter graph where nodes represent neurons or biases and edges represent weights. The sequence of graphs is embedded in the Poincaré ball and processed with a hyperbolic graph attention network, while temporal evolution is handled through kernel meta–evolution that updates attention parameters recurrently across snapshots. The model outputs link probabilities and signed weight predictions, and node embeddings are refined by intrinsic Riemannian optimization. This yields compact temporal representations that capture both structural symmetries and geometric self-organization of the network over training. Figure 1: Architecture of the Geometric Temporal Hyperbolic Graph Meta-Network (GTH-GMN). 3.1 Temporal Graph Construction from MLP Traces We represent the evolving weight matrices of an MLP as a sequence of graph snapshots of parameters, ensuring that the permutation symmetries are respected. At each epoch t, individual neurons and biases are represented as nodes (u∈Vtu∈ V_t), while the parameters define the weighted edges (u,v∈Et\u,v\∈ E_t with signed label wuv,t⋆w _uv,t). Nodes with zero weights at epoch t do not register adjacency (Lim et al., 2023; Kofinas et al., 2024). For an MLP with L layers and widths (n0,…,nL)(n_0,…,n_L), this yields a snapshot Gt=(Xt,Et,Wt)G_t=(X_t,E_t,W_t). Input and output layers are fixed so that reordering neurons within hidden layers results only in an equivariant graph structure (Lim et al., 2023). The set of nodes includes inputs, hidden neurons, and explicit bias nodes (Lim et al., 2023; Kofinas et al., 2024). Each node feature (xu,tx_u,t) contains a layer label ℓ(u) (u) and a type τ(u)τ(u). The neuron edges connect successive layers, while the bias edges connect a bias node to its layer. Directed pairs are symmetrized by averaging across antiparallel directions, treating missing partners as zero, so that the undirected edge set EtE_t preserves signed magnitudes without spurious artifacts. Each node (xu,tx_u,t) is further endowed with structural signals that remain invariant under permutations. From the edges incident to u we compute absolute and signed strength scores and then standardize them across the graph (Unterthiner et al., 2021). The resulting z–scores (s~abs(u,t),s~sgn(u,t) s^abs(u,t), s^sgn(u,t)) quantify how strongly a node is embedded in its neighborhood. Together with ℓ(u) (u) and τ(u)τ(u), they yield the feature vector xu,t=[ℓ(u),τ(u),s~abs(u,t),s~sgn(u,t)].x_u,t= [ (u),τ(u), s^abs(u,t), s^sgn(u,t) ]. Edge attributes (euv,te_uv,t) are deliberately minimal: they encode only whether an edge is a bias connection and whether it spans layers, never leaking functions of wuv,t⋆w _uv,t into the model. Repeating this construction produces a temporal sequence Gtt=1T\G_t\_t=1^T that captures the self-organization of the MLP while remaining invariant to hidden layer permutations. All descriptors (xu,t,euv,tx_u,t,e_uv,t) are embedded in the Euclidean tangent space at the origin T0cd≃ℝnT_0B^d_c ^n, which provides a stable learning backbone. Discrete labels (ℓ(u),τ(u) (u),τ(u)) are mapped through learned embeddings, while standardized strength features are passed through a small MLP. A residual block refines the result, producing a n -dimensional representation Xu,t∈T0X_u,t∈ T_0. For edges, the binary context euv,t=(is_bias,Δℓ)e_uv,t=(is\_bias, ) is encoded in a bias term buv,tb_uv,t and a gate of near identity guv,tg_uv,t using an MLP, which modulates attention scores without altering the geometry of the manifold. Thus, normalization, dropout, and mixing remain stable in Euclidean space, while curvature is introduced only in the subsequent hyperbolic message passing layers (Li et al., 2024; Zhang et al., 2019; Ganea et al., 2018). The final output is a temporal sequence of parameter graphs with node characteristics (Xu,tX_u,t) and edge characteristics (euv,t,buv,t,guv,te_uv,t,b_uv,t,g_uv,t) that respect the permutation symmetry and provide a clean basis for the prediction and regression of semi-supervised link probability. 3.2 Hyperbolic Graph Attention Layer: Intuition and Formulation The goal of this layer is to compute the updated positions of the nodes on the ball by applying manifold-aware structural self-attention, which respects the curvature of the geometry as well as the temporal evolution of parameter graphs. Each parameter graph snapshot GtG_t lives on the Poincaré ball, which naturally encodes hierarchical and heavy–tailed structure. A hyperbolic attention layer, therefore, cannot simply apply Euclidean linear maps: every projection, comparison, and aggregation must respect the curvature of the underlying manifold. We follow a simple principle throughout: compute in the tangent space, transport to the ball only when geometry matters, and aggregate in a model where barycenters (a manifold-consistent version of weighted average) are well defined. This yields a stable attention mechanism whose scores and updates have direct geometric meaning (Li et al., 2024; Li et al., 2023). Hyperbolic Affine Maps (Q/K/V on the Ball). Each attention head first computes hyperbolic query, key, and value vectors, which act as the basic attention primitives (equation 2): the queries encode how a node seeks information, the keys encode how it is compared to its neighbors in hyperbolic space, and the values carry the features that will be aggregated along curved geodesics. Since linear maps are not directly compatible with curved geometry, we realize them as tangent space projections, followed by exponential lifting, which transports the updated tangent vector back onto the manifold along the geodesic defined at that point (Ganea et al., 2018; Li et al., 2024). Given the Euclidean parameters W and b, the hyperbolic affine map (Φ ) is defined as ΦW,b(t)(x)=expΦW(t)(x)(PT0→ΦW(t)(x)(b)),ΦW(t)(x)=exp0(Wlog0(x)), ^(t)_W,b(x)= _ ^(t)_W(x)\! (PT_0→ ^(t)_W(x)(b) ), ^(t)_W(x)= _0\! (W\, _0(x) ), (1) where log0 _0 and exp0 _0 move between the tangent space and the ball at the origin, and PTPT denotes parallel transport. This construction should be viewed as the hyperbolic analogue of a linear map plus bias: the matrix W acts in the flat tangent space, while b is transported to the tangent space at ΦW(t)(x) _W^(t)(x) where it is applied before lifting back onto the ball. Applying Φ to each node yields the corresponding hyperbolic query, key, and value vectors as follows: qi(r,t)=ΦWq(r,t),bq(r,t)(t)(zi(t)),ki(r,t)=ΦWk(r,t),bk(r,t)(t)(zi(t)),vi(r,t)=ΦWv(r,t),bv(r,t)(t)(zi(t)).q^(r,t)_i= ^(t)_W^(r,t)_q,b^(r,t)_q(z^(t)_i), k^(r,t)_i= ^(t)_W^(r,t)_k,b^(r,t)_k(z^(t)_i), v^(r,t)_i= ^(t)_W^(r,t)_v,b^(r,t)_v(z^(t)_i). (2) Attention Logits and Structural Gating. Attention on the Poincaré ball must measure similarity by geodesic distance, in line with prior hyperbolic graph attention formulations (Zhang et al., 2019; Li et al., 2023; Li et al., 2024). For each directed edge (u→v)(u\!→\!v), we therefore score proximity as the negative hyperbolic distance, since smaller geodesic distances correspond to stronger similarity, and using the negative ensures that nearer nodes receive exponentially larger attention weights under the softmax (equation 3). θu→v(r,t)=−dc(qu(r,t),kv(r,t))+buv(t),θ^(r,t)_u→ v=-\,d_c(q^(r,t)_u,\,k^(r,t)_v)+b^(t)_uv, (3) where buv(t)b^(t)_uv injects edge context such as bias indicator or layer separation. Softmax normalization produces the structural attention coefficients in the following manner αu→v(r,t) α^(r,t)_u→ v =exp(θu→v(r,t))∑w:(w→v)∈Etexp(θ(r,t)w→v),α~(r,t)u→v=αu→v(r,t)guv(t)∑w:(w→v)∈Etα(r,t)w→vg(t)wv. = \! (θ^(r,t)_u→ v ) _w:(w→ v)∈ E_t \! (θ^(r,t)_w→ v ) , α^(r,t)_u→ v= α^(r,t)_u→ v\,g^(t)_uv _w:(w→ v)∈ E_tα^(r,t)_w→ v\,g^(t)_wv. (4) Because some edges carry more structural influence (e.g., cross–layer or bias connections), we modulate these coefficients using a gate guv(t)g^(t)_uv (right part of equation 4). Renormalizing yields α~u→v(r,t) α^(r,t)_u→ v, which behaves like standard attention but incorporates curvature–aware structural reweighting (Li et al., 2023; Li et al., 2024), which is achieved by multiplying each coefficient by its gate and normalizing across the gated neighbors so that their relative influence is correctly rescaled. Geometric Aggregation via Einstein Gyromidpoints. Averaging vectors in curved space cannot be done linearly because Euclidean addition does not respect the manifold’s geometry: the sum of two points typically lies outside the manifold and does not correspond to any meaningful geodesic midpoint. We therefore use Einstein gyromidpoints, computed in the Klein model where closed–form barycenters exist (Zhang et al., 2019; Li et al., 2024). The Poincaré to Klein map (κ) and its inverse are κ(z) κ(z) =2cz1+c‖z‖2,κ−1(y)=1cy1+ 1−‖y‖2. = 2 c\,z1+c\|z\|^2, κ^-1(y)= 1 c\, y1+ \,1-\|y\|^2\,. (5) with Lorentz factor γ(y)=1/1−‖y‖2γ(y)=1/ 1-\|y\|^2. For node v and head r, the Klein barycenter is κ(mv(r,t))=∑u:(u→v)∈Etα~(r,t)u→vγ(κ(v(r,t)u))κ(v(r,t)u)∑u:(u→v)∈Etα~(r,t)u→vγ(κ(v(r,t)u)).κ\! (m^(r,t)_v )= _u:(u→ v)∈ E_t α^(r,t)_u→ v\,γ(κ(v^(r,t)_u))\,κ(v^(r,t)_u) _u:(u→ v)∈ E_t α^(r,t)_u→ v\,γ(κ(v^(r,t)_u)). (6) Mapping back and returning to the tangent space gives the multi–head aggregation (right-part equation 7) for computing the node update, which is simply the average of these tangent updates: mv(r,t)=κ−1(κ(mv(r,t))),m^v(r,t)=log0(mv(r,t)),Mv(t)=1H∑r=1Hm^v(r,t).m^(r,t)_v=κ^-1\! (κ(m^(r,t)_v) ), m^(r,t)_v= _0(m^(r,t)_v), M^(t)_v= 1H _r=1^H m^(r,t)_v. (7) This design has a critical interpretive consequence: attention uses geodesics to decide “who influences whom,” while aggregation respects the global curvature, ensuring that hierarchical structure, hub nodes, and boundary effects are preserved. Each node then receives a gated residual update (addition in the tangent space followed by exponential lifting). A projection step enforces a safety margin inside the ball to avoid numerical instability near the boundary (Li et al., 2024). Temporal Evolution of Attention Kernels. Now, to model temporal smoothness across snapshots, we meta–evolve the attention parameters using a GRU, inspired by (Pareja et al., 2019). For each layer ℓ and head r, we maintain a recurrent state uℓ,r(t)u^(t)_ ,r as follows: pℓ(t−1) p^(t-1)_ =1N∑i=1NX(t−1)ℓ,i,u(t)ℓ,r=GRU(p(t−1)ℓ,u(t−1)ℓ,r). = 1N _i=1^NX^(t-1)_ ,i, u^(t)_ ,r=GRU\! (p^(t-1)_ ,\,u^(t-1)_ ,r ). (8) The GRU emits fresh Q/K/V parameters via an MLP: Wℓ,r,k/q/v(t)=ϕMLP(Woutuℓ,r(t)+bout)∈ℝ3d2+3d,W^(t)_ ,r,k/q/v= _MLP\! (W_outu^(t)_ ,r+b_out ) ^3d^2+3d, (9) which are then used in the hyperbolic affine maps (1)–(2). This mechanism allows the attention kernels to evolve smoothly over time without storing past embeddings, providing temporal consistency while preserving permutation invariance and limiting memory to compact recurrent states. 3.3 Signed Weight Regression Before introducing the mathematical details of this module, we outline its role in the architecture. A standard link predictor estimates whether an edge should exist, but in weighted networks, we also need to recover the actual strength and polarity of each connection. Magnitude reflects how strongly two nodes influence each other, while the sign encodes the direction of that influence. In many real networks, these magnitudes follow heavy-tailed distributions (Song et al., 2005; Krioukov et al., 2010; Wei et al., 2013; Allard et al., 2017), and hyperbolic distance offers a natural geometric proxy for such patterns. The signed regression head therefore serves two purposes: it provides additional supervision for the temporal encoder, and it imposes a geometric prior that ties magnitudes to hyperbolic distances while modeling polarity in the tangent space. In contrast, Euclidean regressors or generic MLP decoders (Goyal et al., 2018) ignore curvature and risk losing this interpretive structure. Magnitude as a Power Law of Hyperbolic Distance. We begin by predicting node-dependent scale and decay factors from the tangent features Xi(t)∈T0cd≃ℝdX^(t)_i∈ T_0B^d_c ^d ,si=fσ(Xi(t)),ki=fκ(Xi(t)),,s_i=f_σ(X^(t)_i), k_i=f_κ(X^(t)_i), where fσf_σ and fκf_κ are MLPs. sis_i & κi _i adjust the local steepness and scale of the magnitude–distance relation. For each edge (u,v)(u,v) these corrections are used symmetrically as (su,sv)(s_u,s_v) and (ku,kv)(k_u,k_v). The slope of the power law is then allowed to vary slightly with local context: αuv=α0+δα⋅tanh(ϕMLP([Xu(t),Xv(t),euv])), _uv= _0+δα· \! ( _MLP([X^(t)_u,X^(t)_v,e_uv]) ), (10) where α0 _0 is a global baseline, δαδα a small deviation, and ϕMLP _MLP an MLP (Allard et al., 2017). This design captures mild structural asymmetries between edge types while keeping the decay law interpretable and stable. Given these components, the predicted magnitude follows a heavy-tailed power law (Allard et al., 2017): |wuv(t)|^=exp(logν+su+sv)exp(−(1−αuvd)(ku+kv))(duv(t))−αuv, |w^(t)_uv|= ( ν+s_u+s_v)\, \! (-\,(1- _uvd)(k_u+k_v) )\, (d^(t)_uv )^- _uv, (11) where ν>0ν>0 is a global scale and duv(t)=dc(zu(t),zv(t))d^(t)_uv=d_c(z^(t)_u,z^(t)_v) is the Poincaré distance. Large magnitudes, therefore, correspond to smaller hyperbolic distances, while small magnitudes produce larger metric separation (Simas and Rocha, 2014). This establishes a direct geometric interpretation of weight strength. Polarity in the Tangent Space. Distances on the manifold are positive definite (Simas and Rocha, 2014) . Polarity is therefore modeled as a directional effect in the tangent space, where inner products are well defined. We map zv(t)z^(t)_v into the Tangent Space at zu(t)z^(t)_u using the logarithmic map (equation 12), δu→v(t)=logzu(t)(zv(t)),ηu=WηXu(t),ξu=ηuλzu(t)‖ηu‖,λzu(t)=21−c‖zu(t)‖2,δ^(t)_u→ v= _z^(t)_u(z^(t)_v), _u=W_ηX^(t)_u, _u= _u _z^(t)_u\| _u\|, _z^(t)_u= 21-c\|z^(t)_u\|^2, (12) and normalize the source feature ηu _u by the conformal factor so that directionality is consistent with the Riemannian metric. The sign logit is then given by a hyperbolically consistent inner product, suv(t)=β⟨δu→v(t),ξu⟩,w^uv(t)=|wuv(t)|^tanh(suv(t)).s^(t)_uv=β\, δ^(t)_u→ v, _u , w^(t)_uv= |w^(t)_uv|\, (s^(t)_uv). (13) This construction separates magnitude and polarity in a geometrically faithful manner (equation 13). A distance-based heavy-tailed law controls magnitude (equation 11), while polarity arises from the angle between transported displacements and conformally normalized source features (equation 12). As a result, short hyperbolic distances naturally represent strong connections, whereas longer distances encode weaker or negligible ones, and the directional alignment in the tangent space determines the sign. The module therefore provides an interpretable decomposition of each weight into strength and direction, consistent with the geometry of the Poincaré ball. 3.4 Link Decoder, Supervision, and Objective To translate the hyperbolic embeddings into edge predictions, we use the Fermi–Dirac decoder (ψ(t)ψ^(t)) (equation 14) (Nickel and Kiela, 2017; Nickel and Kiela, 2018; Wen et al., 2024), which links connectivity directly to the geodesics on the Poincaré ball, for a given time step t: ψ(t)(u,v)=(1+exp(dc(zu(t),zv(t))−RT))−1.ψ^(t)(u,v)= (1+ \! ( d_c(z^(t)_u,z^(t)_v)-RT ) )^-1. (14) Here, the radius R controls the effective neighborhood boundary and the temperature T controls the sharpness of the transition, both learned during training. Under this decoder, nodes that lie close in hyperbolic distance are classified as likely neighbors, and distant nodes as unlikely. Training uses a binary cross-entropy loss together with dynamic negative sampling from both same-layer and uniform distributions. Moreover, connectivity alone does not describe the strength or direction of influence between two parameters. For this reason, we include a signed weight regressor (Section 3.3) that predicts the full value of each positive edge. The regression loss encourages the predicted weight to match the true magnitude, while a separate classification loss predicts its polarity. The magnitude loss measures the difference between predicted and true weights, while the polarity loss compares the predicted sign logit with the binary label indicating whether the weight is positive or negative. Formal expressions for these losses and their components are provided in Appendix G. This design ties geometric separation to functional strength: large-magnitude edges correspond to smaller hyperbolic distances, and the sign is encoded through directions in the tangent space. As a result, the embeddings acquire semantic meaning: hubs drift outward, stronger edges cluster radially, and excitatory versus inhibitory effects appear as angular variation. To maintain coherent geometry across time, we add a small set of regularizers. A slope prior keeps the decay exponents near a global baseline, a temporal smoothness term limits abrupt movement between snapshots, and a ranking loss enforces that stronger interactions correspond to shorter hyperbolic distances. The overall objective is a weighted sum of the Fermi–Dirac cross-entropy, the magnitude and sign supervision terms, and these geometric regularizers. Training follows a gentle curriculum that anneals the ranking margin and progressively hardens negative sampling to stabilize learning before introducing difficult examples. Ablations about specific terms can be found in Appendix J. Optimization proceeds in two distinct phases. This separation is necessary because the attention stack and regressors operate in Euclidean space, whereas the node embeddings evolve on a curved manifold; using a single update rule would either ignore curvature or force the Euclidean modules to inherit manifold constraints they were not designed for. In the first phase, a Euclidean pass updates all kernel weights, regressors, and the link decoder using standard backpropagation. In the second phase, a Riemannian pass refines the node positions z(t)z^(t) directly on the Poincaré ball. This intrinsic step begins by rescaling Euclidean gradients with the conformal factor λz2=2/(1−c‖z‖2) _z^2=2/(1-c\|z\|^2) so that updates follow the manifold geometry rather than the ambient space (Bridson and Haefliger, 1999; Bonnabel, 2013; Nickel and Kiela, 2017). To stabilize learning, first and second moments of these gradients are accumulated in a RAMSGrad–style procedure (Bécigneul and Ganea, 2019; Sakai and Iiduka, 2020), which extends adaptive optimization to curved spaces. Each accepted update is applied via the exponential map, projecting the tangent step back onto the manifold. Finally, moment vectors are parallel transported to the new tangent space, keeping the optimizer’s memory coherent across iterations. This two–phase design decouples the stability of the Euclidean attention stack from the dynamics of hyperbolic embeddings, ensuring that geometry is respected while long training traces remain stable. We describe the algorithm in Appendix G. Overall, this scheme does more than minimize loss: it shapes an embedding space where magnitudes follow radial distance, polarity is encoded in tangent orientation, and temporal smoothness emerges from controlled geodesic motion. Geometry thus becomes a built-in inductive bias for compact and interpretable representations. 4 Experimental Setup and Results Classification of Images via INR Traces. This experiment evaluates whether the temporal evolution of the parameter graph of the intrinsic neural representation (INR) of an image contains enough structure for a meta-network to infer the class of the underlying image. Crucially, the meta-network never sees the image itself; it only receives the evolution of the INR weights over optimization. We fit each image from MNIST and Fashion-MNIST using a shallow INR, recording its parameter values over optimization, yielding a temporal trace of parameter graphs. Each trace contains T∈[80−100]T∈[80-100] checkpoints, i.e., a sequence of signed parameter-graphs. Dataset splits are made at the image level to avoid leakage; we use 45,00045,000 images for training, 5,0005,000 for validation, and 10,00010,000 for testing. A spatio-temporal meta-network is pretrained with intrinsic (hyperbolic) optimization on a joint link-prediction and signed weight-regression objective over the full kept trace per image, along with other losses in our curriculum; within each checkpoint, edge embeddings are pooled to a snapshot vector, and snapshot vectors are then pooled over time to produce a fixed n-dimensional representation. A small feed-forward classifier is trained on these representations to predict the digit or fashion class. The same encoding procedure is applied at validation and test time, and we report classification accuracy on the held-out test images. Results are shown in Table 1. Table 1: INR classification in static vs. temporal settings. Average test accuracies (± standard error) on MNIST and Fashion-MNIST INRs. Baselines are reproduced from DWSNets (Navon et al., 2024) and NFNs (Zhou et al., 2023a). Our spatio-temporal variant uses pooled representations across K traces. Model MNIST INR Fashion-MNIST INR MLP 17.55 ± 0.01 19.91 ± 0.47 MLP + Perm. aug 29.26 ± 0.18 22.76 ± 0.13 MLP + Alignment 58.98 ± 0.52 47.79 ± 1.03 INR2Vec (Arch.) 23.69 ± 0.10 22.33 ± 0.41 Transformer 26.57 ± 0.18 26.97 ± 0.33 DWSNets 85.71 ± 0.57 67.06 ± 0.29 NFNHNP_HNP 92.5 ± 0.07 72.7 ± 1.53 NFNNP_NP 92.9 ± 0.22 75.6 ± 1.07 Ours (Temporal, K-trace pooling) 95.6 ± 0.18 80.72 ± 0.29 Predicting DNN Generalization from Weights. This experiment tests whether the temporal evolution of the parameters of a model contains a predictive signal about its test accuracy. More specifically, can we recover accuracy-related information solely from a deep neural network’s weight structure? Here, each trial i, corresponds to an MLP trained on a subset of the CIFAR-10 dataset. For each trial i we gather checkpoints θi,tt=1Ti\ _i,t\_t=1^T_i and the corresponding test accuracy (per-epoch) yi,ty_i,t, and convert every checkpoint to a parameter graph Gi,tG_i,t with corresponding node/edge features. Traces are encoded over fixed-length (stride-rolled) windows or complete sequences, with last-step supervision, i.e., the intermediate steps evolve while the final step is supervised. We classify the divisions by trial, and the fidelity of the classification is measured using Kendall’s τ (Zhou et al., 2023a; Navon et al., 2024; Lim et al., 2023). See Results in (Table 3). Table 2: Kendall’s τ correlation on the CIFAR-10 benchmark. This table compares the rank correlation between predicted and ground-truth accuracies of neural architectures (INRs) as captured by different meta-network representations. Higher values indicate stronger consistency between the learned representation space and the empirical performance ranking of architectures (Unterthiner et al., 2021; Zhou et al., 2023a). Metanetwork CIFAR-10 NFNHNP_HNP 0.934±0.0010.934± 0.001 NFNNP_NP 0.922±0.0010.922± 0.001 StatNN 0.915±0.0020.915± 0.002 GTH-GMN (ours) 0.846±0.0040.846± 0.004 Table 3: Results of meta-net representations on sinusoid-MLPs. Entries report test MSE of a downstream MLP regressor trained on the learned representations. We compare with other baselines by Lim et al. (2023); Navon et al. (2024); Zhou et al. (2023a). Metanetwork Test MSE MLP 7.39 ± 0.19 MLP + Perm. aug 5.65 ± 0.01 MLP + Alignment 4.47 ± 0.15 INR2Vec (Arch.) 3.86 ± 0.32 Transformer 5.11 ± 0.12 DWSNets 1.39 ± 0.06 GMN 1.13 ± 0.08 GTH-GMN (ours) 1.06 ± 0.24 Predicting Sine Wave Frequency. Additionally, we also evaluate our temporal model on the benchmark introduced by (Navon et al., 2024), in which each input network is an MLP trained to fit a one-dimensional sinusoidal function of the form x↦asin(bx)x a (bx), with coefficients a,b∈ℝa,b varying across samples. The goal is to learn a meta-network encoder that maps these trained MLP temporal parameter graphs to meaningful representations, while remaining invariant to neuron permutations, where representation quality is assessed in a contrastive-learning framework inspired by SimCLR (Lim et al., 2023; Navon et al., 2024; Zhou et al., 2023a; Chen et al., 2020). We introduce a temporal variant: link K successive checkpoints per INR, build an edge–centric hyperbolic representation, mean–pool, and predict (a,b)(a,b) with a small MLP. See results in Table 3. Figure 2: Hyperbolic embeddings of parameter graphs from a 13-layer MLP trained as an INR on CIFAR-10, shown at checkpoints t=11t=11 (left) and t=97t=97 (right). Embeddings are learned in d=10d=10 dimensions with 4 HGAT layers and 4 EvolveGCN–H kernels (tangent dim. 128) and projected to the Poincaré disk via PCA. Nodes are colored by layer index, with edges in gray, illustrating the progressive reorganization and separation of layers over training. 5 Discussion Interpreting the Results. Across the three experiments in Section 4, we aimed to test whether a single temporal hyperbolic meta network can generalize across very different prediction tasks. The results suggest that the evolution of the parameter graph carries a stable and predictive structure. In the INR classification, the encoder achieves average test accuracies of 95.6±0.18%95.6± 0.18\% on MNIST and 80.72±0.29%80.72± 0.29\% on Fashion MNIST, revealing that the way INRs self-organize during training encodes class-specific geometric structures that the model is able to consistently separate. In the CIFAR 10 generalization prediction task, we achieve a relatively modest Kendall’s τ=0.846±0.004τ=0.846± 0.004 compared to the neural functional baselines. Our factorization emphasizes global geometric structure and temporal coherence over preserving all fine-grained tensor details. It has been shown that representation learning of neural networks using Autoencoders often exhibits a smoothening bias as MSE reconstruction loss tends to coarse-averaged representations (Meynent et al., 2025). In contrast, permutation-equivariant neural functionals operate directly on raw weight tensors (Zhou et al., 2023a) and retain more accuracy-correlated micro-structure at the cost of reduced geometric interpretability. This likely contributes to the relatively lower Kendall’s τ achieved over the CIFAR-10 dataset (Fu et al., 2021). We see this as a trade-off between raw performance over the generalization prediction task and generating geometric and interpretable embeddings of the underlying DNN. Moreover, over the sinusoid benchmark, our temporal hyperbolic model reaches the 1.06±0.241.06± 0.24 test MSE, comparable in mean to GMN’s 1.13±0.081.13± 0.08, though with higher variance. This arises from the interaction of curvature (Fu et al., 2021) and temporal coupling: in the Poincaré ball, the effective step size increases with radius, so small early differences can produce divergent trajectories, and the recurrent kernel further amplifies such variation (Nickel and Kiela, 2017; Bonnabel, 2013; Ganea et al., 2018). Variability can be reduced with more conservative Riemannian step control, stronger temporal regularization, or light ensembling, which are standard techniques for stabilizing hyperbolic and recurrent models (Ganea et al., 2018; Chen et al., 2024; Sun et al., 2021b). Implications for Explainability. The embeddings reveal coherent patterns of self-organization: points drift toward the boundary, layers separate in both radial and angular directions, and trajectories evolve smoothly across training (Figure 2). These behaviors suggest several potential signals for intrinsic explainability. Radial position and drift may reflect how influence strengthens or weakens over time; angular separation may indicate the emergence of functional specialization; geometric proximity and repeated strong connections may reveal central processing hubs; and consistent magnitude–distance alignment may highlight connections that are both geometrically and functionally salient. Conversely, nodes whose trajectories collapse together or drift outward uniformly may signal redundancy and pruning opportunities (Nickel and Kiela, 2017; Serrano et al., 2008a). Additionally, Appendix Section K explains how our approach can be extended beyond multilayered perceptrons (MLPs). Acknowledgements I thank Eindhoven University of Technology (TU/e) for providing access to the Umbrella cluster, which supported part of the experimentation. Moreover, I would like to extend my thanks to Dr. Indika Kumara and Dr. Damian Tamburri, from Tilburg University and Eindhoven University respectively for their supervisory contribution during the course of this project. I also acknowledge SURF (w.surf.nl) for enabling use of the National Supercomputer Snellius. The use of large language models (LLMs) was restricted exclusively to polishing the text for brevity under space constraints. The author(s) take full responsibility for the research ideation and the experimentation conducted and reported in this work. References Allard et al. (2017) A. Allard, M. Á. Serrano, G. García-Pérez, and M. Boguñá The geometric nature of weights in real complex networks. Nature Communications 8 (1), p. 14103. External Links: Document, Link, ISSN 2041-1723 Cited by: Appendix C, Appendix C, §1, §1, §1, §3.3, §3.3. Amaral et al. (2000) L. A. N. Amaral, A. Scala, M. Barthélémy, and H. E. Stanley Classes of small-world networks. Proceedings of the National Academy of Sciences 97 (21), p. 11149–11152. External Links: Document, Link, https://w.pnas.org/doi/pdf/10.1073/pnas.200327197 Cited by: Appendix C. Bai et al. (2023) Q. Bai, C. Nie, H. Zhang, D. Zhao, and X. Yuan HGWaveNet: a hyperbolic graph neural network for temporal link prediction. In Proceedings of the ACM Web Conference 2023, W ’23, p. 523–532. External Links: Link, Document Cited by: §2. Bassett and Bullmore (2006) D. S. Bassett and E. Bullmore Small-world brain networks. Neuroscientist 12 (6), p. 512–523. External Links: Document, ISSN 1073-8584 Cited by: Appendix C. Bécigneul and Ganea (2019) G. Bécigneul and O. Ganea Riemannian adaptive optimization methods. External Links: 1810.00760, Link Cited by: Appendix G, §3.4, 6. Bonnabel (2013) S. Bonnabel Stochastic gradient descent on riemannian manifolds. IEEE Transactions on Automatic Control 58 (9), p. 2217–2229. External Links: ISSN 1558-2523, Link, Document Cited by: Appendix G, §3.4, §5. Bridson and Haefliger (1999) M. R. Bridson and A. Haefliger Metric spaces of non-positive curvature. Springer, Berlin, Heidelberg, New York. External Links: Document Cited by: Appendix G, §3.4. Bronstein et al. (2021) M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković Geometric deep learning: grids, groups, graphs, geodesics, and gauges. External Links: 2104.13478, Link Cited by: Appendix D, Definition 2. Bullmore and Sporns (2009) E. Bullmore and O. Sporns Complex brain networks: graph theoretical analysis of structural and functional systems. Nature Reviews Neuroscience 10 (3), p. 186–198. External Links: Document, Link, ISSN 1471-0048 Cited by: Appendix C. Campbell et al. (2023) A. Campbell, A. G. Zippo, L. Passamonti, N. Toschi, and P. Lio DynDepNet: learning time-varying dependency structures from fmri data via dynamic graph structure learning. External Links: 2209.13513, Link Cited by: Appendix C. Chami et al. (2019) I. Chami, R. Ying, C. Ré, and J. Leskovec Hyperbolic graph convolutional neural networks. External Links: 1910.12933, Link Cited by: §2. Chen et al. (2024) H. Chen, H. D. Liu, A. Jacobson, D. I. W. Levin, and C. Zheng Trust-region eigenvalue filtering for projected newton. External Links: 2410.10102, Link Cited by: §5. Chen et al. (2020) T. Chen, S. Kornblith, M. Norouzi, and G. Hinton A simple framework for contrastive learning of visual representations. External Links: 2002.05709, Link Cited by: §E.4, §E.4, §4. Chen et al. (2021) W. Chen, X. Han, Y. Lin, H. Zhao, Z. Liu, P. Li, M. Sun, and J. Zhou Fully hyperbolic neural networks. arXiv preprint arXiv:2105.14686. Cited by: §2. Cohen and Havlin (2010) R. Cohen and S. Havlin Complex networks: structure, robustness and function. Cambridge University Press, Cambridge, UK. External Links: ISBN 9780521841565 Cited by: Appendix C. Cui et al. (2021) H. Cui, W. Dai, Y. Zhu, X. Li, L. He, and C. Yang BrainNNExplainer: an interpretable graph neural network framework for brain network based disease analysis. External Links: 2107.05097, Link Cited by: Appendix C. Cui et al. (2022) H. Cui, W. Dai, Y. Zhu, X. Li, L. He, and C. Yang Interpretable graph neural networks for connectome-based brain disorder analysis. External Links: 2207.00813, Link Cited by: Appendix C. De Luigi et al. (2023) L. De Luigi, A. Cardace, R. Spezialetti, P. Z. Ramirez, S. Salti, and L. Di Stefano Deep learning on implicit neural representations of shapes. arXiv preprint arXiv:2302.05438. Cited by: §2. Defferrard et al. (2017) M. Defferrard, X. Bresson, and P. Vandergheynst Convolutional neural networks on graphs with fast localized spectral filtering. External Links: 1606.09375, Link Cited by: §2. Dupont et al. (2022) E. Dupont, H. Kim, S. Eslami, D. Rezende, and D. Rosenbaum From data to functa: your data point is a function and you can treat it like one. arXiv preprint arXiv:2201.12204. Cited by: §1, §1, §2. Eilertsen et al. (2020) G. Eilertsen, D. Jönsson, T. Ropinski, J. Unger, and A. Ynnerman Classifying the classifier: dissecting the weight space of neural networks. arXiv preprint arXiv:2002.05688. Cited by: §2. Fronczak et al. (2024) A. Fronczak, P. Fronczak, M. J. Samsel, K. Makulski, M. Łepek, and M. J. Mrowiński Scaling theory of fractal complex networks. Scientific Reports 14, p. 9079. External Links: Document Cited by: Appendix C, §1, §1. Fu et al. (2021) X. Fu, J. Li, J. Wu, Q. Sun, C. Ji, S. Wang, J. Tan, H. Peng, and P. S. Yu ACE-hgnn: adaptive curvature exploration hyperbolic graph neural network. In 2021 IEEE International Conference on Data Mining (ICDM), p. 111–120. External Links: Link, Document Cited by: §5. Ganea et al. (2018) O. Ganea, G. Becigneul, and T. Hofmann Hyperbolic neural networks. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31, p. . External Links: Link Cited by: §B.2, §2, §3.1, §3.2, §5. Goyal et al. (2018) P. Goyal, N. Kamra, X. He, and Y. Liu DynGEM: deep embedding method for dynamic graphs. External Links: 1805.11273, Link Cited by: Appendix G, Appendix G, §3.3. Gromov (1987) M. Gromov Hyperbolic groups. In Essays in group theory, p. 75–263. Cited by: §1, §1. Guillemaud et al. (2025) M. Guillemaud, V. Dinkelacker, and M. Chavez Hyperbolic embedding of multilayer networks. External Links: 2505.20378, Link Cited by: Appendix C, §1, §1. Hajiramezanali et al. (2019) E. Hajiramezanali, A. Hasanzadeh, N. Duffield, K. R. Narayanan, M. Zhou, and X. Qian Variational graph recurrent neural networks. In Advances in Neural Information Processing Systems 32 (NeurIPS 2019), External Links: Link Cited by: §2. Hamilton (2020) W. L. Hamilton Graph representation learning. Morgan & Claypool Publishers. Cited by: §2. Han et al. (2025) K. Han, Y. Su, L. He, L. Zhan, S. Plis, V. Calhoun, and C. Yang Rethinking functional brain connectome analysis: do graph deep learning models help?. External Links: 2501.17207, Link Cited by: Appendix C. Han et al. (2016) S. Han, H. Mao, and W. J. Dally Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. External Links: 1510.00149, Link Cited by: §1. Hecht-Nielsen (1990) R. Hecht-Nielsen On the algebraic structure of feedforward network weight spaces. In Advanced Neural Computers, p. 129–135. Cited by: §1. Kan et al. (2023) X. Kan, Z. Li, H. Cui, Y. Yu, R. Xu, S. Yu, Z. Zhang, Y. Guo, and C. Yang R-mixup: riemannian mixup for biological networks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’23, p. 1073–1085. External Links: Link, Document Cited by: Appendix C. Kim et al. (2021) B. Kim, J. C. Ye, and J. Kim Learning dynamic graph representation of brain connectome with spatio-temporal attention. External Links: 2105.13495, Link Cited by: Appendix C, §2. Kim et al. (2007a) J. S. Kim, K.-I. Goh, G. Salvi, E. Oh, B. Kahng, and D. Kim Fractality in complex networks: critical and supercritical skeletons. Physical Review E 75, p. 016110. External Links: Document Cited by: Appendix C, §1, §1. Kim et al. (2007b) J. S. Kim, K.-I. Goh, G. Salvi, E. Oh, B. Kahng, and D. Kim Fractality in complex networks: critical and supercritical skeletons. Physical Review E 75 (1). External Links: ISSN 1550-2376, Link, Document Cited by: Appendix C. Kofinas et al. (2024) M. Kofinas, B. Knyazev, Y. Zhang, Y. Chen, G. J. Burghouts, E. Gavves, C. G. M. Snoek, and D. W. Zhang Graph neural networks for learning equivariant representations of neural networks. External Links: 2403.12143, Link Cited by: §1, §1, §1, §1, §2, §3.1. Krioukov et al. (2010) D. Krioukov, F. Papadopoulos, M. Kitsak, A. Vahdat, and M. Boguñá Hyperbolic geometry of complex networks. Physical Review E 82 (3), p. 036106. External Links: Document Cited by: Appendix C, §1, §1, §2, §3.3. Latora and Marchiori (2001) V. Latora and M. Marchiori Efficient behavior of small-world networks. Phys. Rev. Lett. 87, p. 198701. External Links: Document, Link Cited by: Appendix C. Li et al. (2024) H. Li, H. Jiang, D. Ye, Q. Wang, L. Du, Y. Zeng, L. yuan, Y. Wang, and C. Chen DHGAT: hyperbolic representation learning on dynamic graphs via attention networks. Neurocomputing 568, p. 127038. External Links: ISSN 0925-2312, Document, Link Cited by: §1, §2, §3.1, §3.2, §3.2, §3.2, §3.2, §3.2, §3.2. Li et al. (2020) J. Li, Y. Sun, J. Su, T. Suzuki, and F. Huang Understanding generalization in deep learning via tensor methods. External Links: 2001.05070, Link Cited by: §1. Li et al. (2023) X. Li, Y. Liu, Z. Liu, and P. S. Yu Time-aware hyperbolic graph attention network for session-based recommendation. External Links: 2301.03780, Link Cited by: §2, §3.2, §3.2, §3.2. Li et al. (2021) X. Li, Y. Zhou, N. Dvornek, M. Zhang, S. Gao, J. Zhuang, D. Scheinost, L. H. Staib, P. Ventola, and J. S. Duncan BrainGNN: interpretable brain graph neural network for fmri analysis. Medical Image Analysis 74, p. 102233. External Links: Document, ISSN 1361-8415, Link Cited by: Appendix C. Lim et al. (2023) D. Lim, H. Maron, M. T. Law, J. Lorraine, and J. Lucas Graph metanetworks for processing diverse neural architectures. External Links: 2312.04501, Link Cited by: Appendix K, item 2, §E.3, §1, §1, §1, §1, §2, §3.1, §4, §4, Table 3. Litany et al. (2022) O. Litany, H. Maron, D. Acuna, J. Kautz, G. Chechik, and S. Fidler Federated learning with heterogeneous architectures using graph hypernetworks. arXiv preprint arXiv:2201.08459. Cited by: §2. Liu et al. (2018) H. Liu, K. Simonyan, and Y. Yang Darts: differentiable architecture search. arXiv preprint arXiv:1806.09055. Cited by: §2. Maron et al. (2019) H. Maron, H. Ben-Hamu, N. Shamir, and Y. Lipman Invariant and equivariant graph networks. In International Conference on Learning Representations, External Links: Link Cited by: §2. Meynent et al. (2025) L. Meynent, I. Melev, K. Schürholt, G. Kauermann, and D. Borth Structure is not enough: leveraging behavior for neural network weight reconstruction. External Links: 2503.17138, Link Cited by: §5. Navon et al. (2024) A. Navon, A. Shamsian, E. Fetaya, G. Chechik, N. Dym, and H. Maron Equivariant deep weight space alignment. External Links: 2310.13397, Link Cited by: §E.3, §1, §1, §2, §4, §4, Table 1, Table 3. Newman (2010) M. Newman Networks: an introduction. Oxford University Press, Inc., USA. External Links: ISBN 0199206651 Cited by: Appendix C. Nickel et al. (2014) M. Nickel, X. Jiang, and V. Tresp Reducing the rank in relational factorization models by including observable patterns. Advances in Neural Information Processing Systems 27. Cited by: §1, §1. Nickel and Kiela (2017) M. Nickel and D. Kiela Poincaré embeddings for learning hierarchical representations. External Links: 1705.08039, Link Cited by: Appendix G, Appendix G, Appendix G, §1, §1, §2, §2, §3.4, §3.4, §5, §5. Nickel and Kiela (2018) M. Nickel and D. Kiela Learning continuous hierarchies in the lorentz model of hyperbolic geometry. External Links: 1806.03417, Link Cited by: Appendix G, Appendix G, §2, §3.4. Papadopoulos et al. (2012) F. Papadopoulos, M. Kitsak, M. Á. Serrano, M. Boguñá, and D. Krioukov Popularity versus similarity in growing networks. Nature 489 (7417), p. 537–540. External Links: ISSN 1476-4687, Link, Document Cited by: Appendix C. Papadopoulos et al. (2008) F. Papadopoulos, D. Krioukov, M. Boguñá, and A. Vahdat Efficient navigation in scale-free networks embedded in hyperbolic metric spaces. arXiv preprint. External Links: 0805.1266 Cited by: Appendix C. Pareja et al. (2019) A. Pareja, G. Domeniconi, J. Chen, T. Ma, T. Suzumura, H. Kanezashi, T. Kaler, T. B. Schardl, and C. E. Leiserson EvolveGCN: evolving graph convolutional networks for dynamic graphs. External Links: 1902.10191, Link Cited by: Appendix G, Appendix I, §1, §2, §3.2. Peng et al. (2025) C. Peng, Y. Huang, Q. Dong, S. Yu, F. Xia, C. Zhang, and Y. Jin Biologically plausible brain graph transformer. External Links: 2502.08958, Link Cited by: Appendix C. Redman et al. (2024) W. T. Redman, J. Bello-Rivas, M. Fonoberova, R. Mohr, Y. G. Kevrekidis, and I. Mezić Identifying equivalent training dynamics. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37, p. 23603–23629. External Links: Document, Link Cited by: Appendix K. Rubinov and Sporns (2010) M. Rubinov and O. Sporns Complex network measures of brain connectivity: uses and interpretations. NeuroImage 52 (3), p. 1059–1069. Note: Computational Models of the Brain External Links: ISSN 1053-8119, Document, Link Cited by: Appendix C. Sakai and Iiduka (2020) H. Sakai and H. Iiduka Riemannian adaptive optimization algorithm and its application to natural language processing. External Links: 2004.00897, Link Cited by: Appendix G, §2, §3.4, 6. Scannell et al. (1995) J. W. Scannell, C. Blakemore, and M. P. Young Analysis of connectivity in the cat cerebral cortex. The Journal of Neuroscience 15 (2), p. 1463–1483. External Links: Document, ISSN 0270-6474, Link Cited by: Appendix C. Schürholt et al. (2021) K. Schürholt, D. Kostadinov, and D. Borth Self-supervised representation learning on neural network weights for model characteristic prediction. Advances in Neural Information Processing Systems 34, p. 16481–16493. Cited by: §1, §1, §2. Seo et al. (2016) Y. Seo, M. Defferrard, P. Vandergheynst, and X. Bresson Structured sequence modeling with graph convolutional recurrent networks. External Links: 1612.07659, Link Cited by: §2. Serrano et al. (2008a) M. Á. Serrano, D. Krioukov, and M. Boguñá Self-similarity of complex networks and hidden metric spaces. Physical Review Letters 100, p. 078701. External Links: Document Cited by: §5. Serrano et al. (2008b) M. Á. Serrano, D. Krioukov, and M. Boguñá Self-similarity of complex networks and hidden metric spaces. Physical Review Letters 100 (7). External Links: ISSN 1079-7114, Link, Document Cited by: Appendix C. Shi et al. (2015) X. Shi, Z. Chen, H. Wang, D. Yeung, W. Wong, and W. Woo Convolutional lstm network: a machine learning approach for precipitation nowcasting. External Links: 1506.04214, Link Cited by: §2. Simas and Rocha (2014) T. Simas and L. M. Rocha Distance closures on complex networks. External Links: 1312.2459, Link Cited by: §1, §3.3, §3.3. Song et al. (2005) C. Song, S. Havlin, and H. A. Makse Self-similarity of complex networks. Nature 433, p. 392–395. External Links: Document Cited by: Appendix C, §1, §1, §3.3. Sun et al. (2021a) L. Sun, Z. Zhang, J. Zhang, F. Wang, H. Peng, S. Su, and P. S. Yu Hyperbolic variational graph neural network for modeling dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, p. 4375–4383. Cited by: §2. Sun et al. (2021b) L. Sun, Z. Zhang, J. Zhang, F. Wang, H. Peng, S. Su, and P. S. Yu Hyperbolic variational graph neural network for modeling dynamic graphs. External Links: 2104.02228, Link Cited by: §5. Thapaliya et al. (2025) B. Thapaliya, E. Akbas, J. Chen, R. Sapkota, B. Ray, P. Suresh, V. D. Calhoun, and J. Liu Brain networks and intelligence: a graph neural network based approach to resting state fmri data. Medical Image Analysis 101, p. 103433. External Links: ISSN 1361-8415, Link, Document Cited by: Appendix C. Ungar (2007) A. A. Ungar The hyperbolic square and mobius transformations. Banach Journal of Mathematical Analysis 1 (1), p. 101–116. Cited by: §B.2. Ungar (2022) A. Ungar A gyrovector space approach to hyperbolic geometry. Springer Nature. Cited by: §B.2. Unterthiner et al. (2021) T. Unterthiner, D. Keysers, S. Gelly, O. Bousquet, and I. Tolstikhin Predicting neural network accuracy from weights. External Links: 2002.11448, Link Cited by: §1, §1, §2, §3.1, Table 3. Wei et al. (2013) D. Wei, B. Wei, Y. Hu, H. Zhang, and Y. Deng A new information dimension of complex networks. arXiv preprint arXiv:1311.3527. External Links: Link Cited by: Appendix C, §1, §1, §3.3. Wen et al. (2024) L. Wen, X. Tang, M. Ouyang, X. Shen, J. Yang, D. Zhu, M. Chen, and X. Wei Hyperbolic graph diffusion model. External Links: 2306.07618, Link Cited by: Appendix G, §2, §3.4. Wu et al. (2015) Z. Wu, G. Menichetti, C. Rahmede, and G. Bianconi Emergent complex network geometry. Scientific Reports 5 (1). External Links: ISSN 2045-2322, Link, Document Cited by: Appendix C. Yang et al. (2023) M. Yang, M. Zhou, H. Xiong, and I. King Hyperbolic temporal network embedding. IEEE Transactions on Knowledge and Data Engineering 35 (11), p. 11489–11502. External Links: Document Cited by: Appendix G, §2, §2. You et al. (2020) J. You, J. Leskovec, K. He, and S. Xie Graph structure of neural networks. External Links: 2007.06559, Link Cited by: §1, §1, §1, §2. Yu et al. (2022) Y. Yu, X. Kan, H. Cui, R. Xu, Y. Zheng, X. Song, Y. Zhu, K. Zhang, R. Nabi, Y. Guo, C. Zhang, and C. Yang Learning task-aware effective brain connectivity for fmri analysis with graph neural networks. External Links: 2211.00261, Link Cited by: Appendix C. Zhang et al. (2021) Y. Zhang, X. Wang, C. Shi, N. Liu, and G. Song Lorentzian graph convolutional networks. In Proceedings of the Web Conference 2021 (W ’21), p. 1249–1261. Cited by: §2, §2. Zhang et al. (2019) Y. Zhang, X. Wang, X. Jiang, C. Shi, and Y. Ye Hyperbolic graph attention network. External Links: 1912.03046, Link Cited by: §2, §3.1, §3.2, §3.2. Zhou et al. (2023a) A. Zhou, K. Yang, K. Burns, A. Cardace, Y. Jiang, S. Sokota, J. Z. Kolter, and C. Finn Permutation equivariant neural functionals. External Links: 2302.14040, Link Cited by: §E.3, §1, §2, §4, §4, Table 1, Table 3, Table 3, §5. Zhou et al. (2023b) A. Zhou, K. Yang, Y. Jiang, K. Burns, W. Xu, S. Sokota, J. Z. Kolter, and C. Finn Neural functional transformers. External Links: 2305.13546, Link Cited by: §1, §1, §2. Appendix Appendix A Notations Parameter Graphs & Features GtG_t Parameter-graph snapshot at epoch t V,EtV,\ E_t Node set; edge set at epoch t wuv⋆w _uv Signed edge label on u,v\u,v\ euve_uv Edge context (bias/kernel, interlayer flag) L,d,cL,\ d,\ c #layers; embedding dim; curvature parameter ℓ(u) (u) Layer label of node u τ(u)τ(u) Type label (input / neuron / bias) sabs(u)s_abs(u) Incident absolute-weight sum at u ssgn(u)s_sgn(u) Incident signed-weight sum at u s~abs,s~sgn s_abs,\ s_sgn Standardized node-weight signals x_u Final node feature vector Vℓ,NℓV_ ,\ N_ Nodes in layer ℓ ; their count Temporal States and Recurrence t Time/epoch index Xi(t)X^(t)_i Tangent-backbone feature of node i at t pℓ(t−1)p^(t-1)_ Mean-pooled tangent for layer ℓ uℓ,r(t)u^(t)_ ,r Recurrent state (layer ℓ , head r) θℓ,r(t)θ^(t)_ ,r Packed attention parameters (head r) Hyperbolic Geometry (Poincaré Ball) cdB^d_c d-dimensional Poincaré ball d(zi,zj)d_B(z_i,z_j) Geodesic distance on cdB^d_c expx(⋅),logx(⋅) _x(·),\ _x(·) Exponential / logarithmic maps at x λz _z Conformal factor at point z PTx→y(⋅)PT_x→ y(·) Parallel transport from x to y K(⋅),K−1(⋅)K(·),\ K^-1(·) Poincaré↔ coordinate maps Hyperbolic Attention Block r Attention head index qu(r,t),kv(r,t),vu(r,t)q^(r,t)_u,\ k^(r,t)_v,\ v^(r,t)_u Query, key, value points on cdB^d_c τ(r,t)τ^(r,t) Softmax temperature (head r) buvb_uv Edge bias from context guvg_uv Edge gate from context αu→v(r,t)α^(r,t)_u→ v Attention coefficient (head r) mv(r,t)m^(r,t)_v Head message in tangent space Mv(t)M^(t)_v Multi-head combined message zv(t)z^(t)_v Node position update on cdB^d_c (v)N(v) Neighborhood of node v y Klein coordinate (∥y∥<1 y <1) γ(y)γ(y) Lorentz factor in Klein model Decoders and Regressors ψ(t)(u,v)ψ^(t)(u,v) Fermi–Dirac link probability R,TR,\ T Link radius; temperature (FD decoder) w^uv(t),|w^uv(t)| w^(t)_uv,\ | w^(t)_uv| Predicted signed weight; magnitude αuv _uv Distance–magnitude exponent (edge-wise) ν,βν,\ β Magnitude scale; sign-logit scale δu→v(t)δ^(t)_u→ v Hyperbolic displacement (source u to v) ξu _u Conformal direction for sign prediction suv(t)s^(t)_uv Sign logit ε Small positive constant (stability) Losses and Optimization ℒFDL_FD BCE loss for Fermi–Dirac link decoder ℒwrecL_wrec Magnitude regression loss ℒsignL_sign Sign prediction loss (BCE) ℒαL_α Prior/regularizer on αuv _uv ℒsmoothL_smooth Temporal smoothness penalty ℒrankL_rank Pairwise ranking loss E+,E−E^+,\ E^- Positive edges; sampled negatives γrank _rank Ranking margin parameter ∇zEucℒ,∇zRiemℒ∇^Euc_zL,\ ∇^Riem_zL Euclidean / Riemannian gradients Δi _i Intrinsic update step at node i zi(t+1)z^(t+1)_i Updated node position η Learning rate Appendix B Theoretical Preliminaries B.1 Riemannian Manifold A Riemannian manifold (ℳ,gℳ)(M,g^M) is a smooth manifold ℳM endowed with a Riemannian metric gℳg^M, which assigns to each tangent space TxℳT_xM at a point x∈ℳx a positive definite inner product. The tangent space TxℳT_xM consists of all tangent vectors at x and may be regarded as the best linear, first-order approximation to the manifold in a neighborhood of that point. The metric equips this linear space with a bilinear form ⟨⋅,⋅⟩:Txℳ×Txℳ→ℝ ·,· :T_xM× T_xM , from which a norm is induced by ‖v‖g=g(v,v)\|v\|_g= g(v,v) for any v∈Txℳv∈ T_xM. With this structure, the manifold acquires the ability to measure lengths, angles, and volumes, thereby extending the familiar tools of Euclidean geometry to curved spaces. If γ:[a,b]→ℳγ:[a,b] is a smooth curve, its Riemannian length is defined as L(γ)=∫ab‖γ′(t)‖gtL(γ)= _a^b\|γ (t)\|_g\,dt, a quantity that depends on the metric through the induced norm on tangent vectors. Among all curves connecting two points x,y∈ℳx,y , geodesics play a central role: they are the locally length-minimizing paths, serving as the natural analogue of straight lines in Euclidean space. The Riemannian distance is accordingly given by dℳ(x,y)=infγL(γ)d_M(x,y)= _γL(γ), where the infimum is taken over all smooth curves γ with γ(a)=xγ(a)=x and γ(b)=yγ(b)=y. At this point, fundamental questions of differential geometry naturally arise. Given a point x∈ℳx and a tangent vector v∈Txℳv∈ T_xM, how can one move from the linearized tangent space back to the curved manifold? Conversely, given two points x,y∈ℳx,y , how can their displacement be expressed in the linear coordinates of TxℳT_xM? These questions motivate the introduction of the exponential and logarithmic maps. The exponential map at x, denoted expx:Txℳ→ℳ _x:T_xM , takes a tangent vector v∈Txℳv∈ T_xM and returns the point on the manifold obtained by following the geodesic that begins at x in the direction of v, parameterized so that unit time corresponds to unit length. In this way, the exponential map provides a principled mechanism for “lifting” linear displacements in TxℳT_xM into the curved geometry of ℳM. The logarithmic map, denoted logx:ℳ→Txℳ _x:M→ T_xM, is defined locally as the inverse of the exponential map. For a point y∈ℳy sufficiently close to x, it yields the unique vector v∈Txℳv∈ T_xM such that expx(v)=y _x(v)=y. Thus, the log map addresses the complementary problem: it represents the displacement between manifold points in the linear framework of the tangent space. Taken together, these constructions endow the Riemannian manifold with a powerful interpretative scheme. Tangent vectors become infinitesimal displacements that may be integrated into finite motions along the manifold via expx _x, while finite displacements can be linearized into tangent vectors through logx _x. This duality shows why exponential and logarithmic maps are often viewed as rigorous analogues of addition and subtraction on curved spaces, providing the formal apparatus that underpins the heuristic extension of Euclidean intuition to Riemannian geometry. B.2 Poincaré Ball Model A hyperbolic manifold can be described as a Riemannian manifold of constant negative curvature c with c<0c<0. Among the isomorphic realizations of hyperbolic space, three models are commonly employed: the Poincaré ball model, the Lorentz model, and the Klein model. In what follows, we focus on the Poincaré ball model, which offers a particularly convenient formulation for learning problems in hyperbolic geometry. The Poincaré ball model is denoted as (cn,gxB)(B^n_c,g^B_x), where the domain cn=x∈ℝn:∥x∥2<−1/cB^n_c=\x ^n:\|x\|^2<-1/c\ is the open n-dimensional ball of radius 1/|c|1/ |c|. Its Riemannian metric tensor is defined as gxB=(λxc)2gxEg^B_x=(λ^c_x)^2g^E_x, where gxEg^E_x is the Euclidean metric, i.e. the identity matrix InI_n, and λxc=21+c‖x‖2λ^c_x= 21+c\|x\|^2 is the conformal factor that encodes the curvature dependence. Given two points x,y∈cnx,y ^n_c, the induced geodesic distance is given by dBc(x,y)=1|c|cosh−1(1−2c‖x−y‖2(1+c‖x‖2)(1+c‖y‖2)).d^c_B(x,y)= 1 |c| ^-1 (1- 2c\|x-y\|^2(1+c\|x\|^2)(1+c\|y\|^2) ). (15) Closed-form expressions for the exponential and logarithmic maps in the Poincaré ball model were derived in Ganea et al. (2018). The exponential map at x∈cnx ^n_c applied to a tangent vector v∈Txcnv∈ T_xB^n_c takes the form expxc(v)=x⊕c(tanh(|c|λxc‖v‖2)v|c|‖v‖), ^c_x(v)=x _c ( ( |c| λ^c_x\|v\|2 ) v |c|\|v\| ), (16) where ⊕c _c denotes Möbius addition. This expression formalizes the process of mapping linear displacements in the tangent space to points on the curved manifold along the geodesic through x. The logarithmic map, which serves as the inverse of the exponential map, is given by logxc(y)=2|c|λxctanh−1(|c|∥−x⊕cy∥)−x⊕cy∥−x⊕cy∥. ^c_x(y)= 2 |c|λ^c_x ^-1 ( |c|\|-x _cy\| ) -x _cy\|-x _cy\|. (17) Here, logxc(y) ^c_x(y) produces the unique tangent vector at x that corresponds to the geodesic displacement from x to y. Another key operation in hyperbolic geometry is parallel transport, which allows one to move a tangent vector from TxcnT_xB^n_c to TycnT_yB^n_c along the geodesic connecting x and y, without altering its length or intrinsic orientation. This operation is formally defined as PTx→yc(v)=λxcλycgyr[y,−x]v,PT^c_x→ y(v)= λ^c_xλ^c_y\,gyr[y,-x]\,v, (18) where gyr[y,−x]gyr[y,-x] is the gyration operator associated with Möbius addition, capturing the non-associative structure of hyperbolic translations Ungar (2007); Ungar (2022). Taken together, the distance function, exponential and logarithmic maps, and parallel transport furnish the Poincaré ball with the full differential-geometric apparatus needed for optimization and representation learning in spaces of constant negative curvature. Appendix C Additional Related Work Network science and geometric models The study of complex networks has long emphasized the interplay between efficiency, clustering, and degree heterogeneity. Small-world properties, for instance, can be captured through measures of global and local efficiency (Latora and Marchiori, 2001). Geometry itself may emerge from simple generative rules: simplicial-complex models yield clustered and modular graphs whose spectral dimension is finite, with curvature distributions obeying Gauss–Bonnet relations (Wu et al., 2015). Empirical evidence further suggests that hidden metric spaces shape both connectivity and weights, as shown by the strong coupling between triangle formation and edge strengths (Allard et al., 2017). A recurrent observation is the coexistence of heavy-tailed degree distributions and high clustering (Kim et al., 2007b; Serrano et al., 2008b; Amaral et al., 2000), a structural combination that classical random-graph or lattice-based models cannot reproduce (Newman, 2010; Cohen and Havlin, 2010). Renormalization studies reveal that such networks are often self-similar: under box covering, they remain scale-free, exhibiting fractal organization (Song et al., 2005). This fractality can be traced to critical branching “skeletons,” whereas supercritical skeletons drive small-world behavior (Kim et al., 2007a). Beyond standard box dimension, information-dimension measures have been developed to capture self-similar structure in real and weighted networks (Wei et al., 2013). A recent synthesis unifies these microscopic and macroscopic exponents, demonstrating robust scaling laws across web, brain, and citation networks (Fronczak et al., 2024). Such findings motivate latent-space models in which link probabilities decay with hidden distance (Papadopoulos et al., 2008; Papadopoulos et al., 2012). The hyperbolic framework of Krioukov et al. (2010) provides a particularly powerful resolution, reconciling hierarchical expansion with geometric growth. In this view, the popularity–similarity model represents degree through radial coordinates and similarity through angular ones, thereby unifying hub formation with community structure across diverse real-world systems (Krioukov et al., 2010; Allard et al., 2017). Neuroscience and Graph Learning Neurobiological work has used graph theory to study how anatomical and functional connectomes are organized, translating empirical association matrices into graphs and then asking what principles explain their structure. Early cortical mapping of the cat showed that laminar hierarchy and nonmetric multidimensional scaling reveal coherent system-level topologies that are only partly explained by simple neighborhood wiring, suggesting additional long-range organizing factors (Scannell et al., 1995). This line of inquiry converged on the view that brain networks combine high clustering with short paths, a small world organization that can be quantified by clustering, path length, small worldness, and efficiency, and related to plausible cost-benefit trade-offs (Bassett and Bullmore, 2006). Reviews then systematized how to construct graphs from imaging data and how measures of segregation, integration, centrality, motifs, assortativity, and robustness jointly characterize architecture, while emphasizing that node definitions, edge weighting, thresholding, and null models strongly shape inference (Bullmore and Sporns, 2009; Rubinov and Sporns, 2010). Building on this foundation, learning based approaches seek predictive models that preserve neurobiological interpretability. BrainGNN encodes region identity with region-of-interest-aware convolutions and performs region selection pooling to yield subject-level and cohort-level biomarkers that align with prior findings (Li et al., 2021). BrainNNExplainer complements this with a group-level explainer that learns a sparse global edge mask, improving diagnosis while revealing disorder-specific subgraphs (Cui et al., 2021). Dynamic representations extend beyond static functional connectivity: STAGIN learns spatio-temporal graph embeddings with attention across windows of connectivity (Kim et al., 2021); IBGNN proposes an interpretable backbone that couples edge weight aware message passing with a shared mask for disorder specific biomarkers (Cui et al., 2022); TBDS learns sparse directed acyclic graphs from BOLD time series and couples them to signed edge graph neural networks (Yu et al., 2022); and DynDepNet replaces fixed connectivity with time varying dependencies that are optimized jointly with a recurrent classifier (Campbell et al., 2023). Parallel developments adjust data geometry and architectural bias: R Mixup interpolates correlation and adjacency matrices along log Euclidean geodesics on the symmetric positive definite manifold to stabilize training and improve generalization (Kan et al., 2023); BrainRGIN adapts graph isomorphism style aggregation, region aware pooling, and attention based readouts to predict individual differences with interpretable subnetworks (Thapaliya et al., 2025); and BioBGT encodes hub roles and modular structure within a transformer by combining diffusion based node importance with module aware self attention (Peng et al., 2025). Recent benchmarking advises caution when applying message aggregation to functional connectomes. Broad comparisons indicate that aggregation can underperform strong non-graph baselines as graph density increases, while hybrids that separate global vectorized patterns from localized graph structure provide competitive accuracy and clearer interpretation (Han et al., 2025). In parallel, work on hyperbolic embeddings of multilayer networks, including formulations in the Lorentz model, points toward latent geometric explanations, although current treatments remain focused on static graphs and often rely on stochastic block models to encode topology (Guillemaud et al., 2025). Appendix D Theoretical Propositions & Proofs Setup. Fix an integer L≥2L≥ 2. Let the common vertex set be V=I∪˙H1∪˙⋯∪˙HL−1∪˙O∪˙B,V\;=\;I\; ∪\;H_1\; ∪\;·s\; ∪\;H_L-1\; ∪\;O\; ∪\;B, a disjoint union of inputs I, hidden layers HℓH_ (ℓ=1,…,L−1 =1,…,L-1), outputs O, and bias nodes B=b(1),…,b(L)B=\b^(1),…,b^(L)\. Let T be a nonempty index set of time steps (epochs). For each t∈Tt∈ T, the parameter snapshot is a directed (or undirected as well), edge-labeled graph Gt=(V,Et,θt),G_t\;=\;(V,E_t, _t), where Et⊆V×VE_t V× V is the edge set and θt:Et→ℝ _t:E_t assigns a real weight to each edge present at time t. The family (Gt)t∈T(G_t)_t∈ T is called a trace. Definition 1. A family (φt)t∈T( _t)_t∈ T with φt:V→V _t:V→ V bijective for each t is a snapshot automorphism family for the trace (Gt)t∈T(G_t)_t∈ T if, for every t∈Tt∈ T, 1. Edge preservation at time t: (u,v)∈Et⟺(φt(u),φt(v))∈Etfor all u,v∈V.(u,v)∈ E_t ( _t(u), _t(v))∈ E_t all u,v∈ V. 2. Node label preservation at time t: input nodes, output nodes, and bias nodes are fixed points; that is, φt(i)=i∀i∈I,φt(o)=o∀o∈O,φt(b)=b∀b∈B. _t(i)=i\ \ ∀\,i∈ I, _t(o)=o\ \ ∀\,o∈ O, _t(b)=b\ \ ∀\,b∈ B. (Lim et al., 2023) Definition 2. Let T denote the set of all traces on the fixed vertex set V. An encoder is a map F:→ZF:T→ Z into a representation space Z equipped with a (left) group action ρ of the product group ∏t∈TAut(Gt) _t∈ TAut(G_t) on Z. The encoder F is called snapshot-wise permutation equivariant if, for every trace (Gt)t∈T(G_t)_t∈ T and every snapshot automorphism family (φt)t∈T( _t)_t∈ T with φt∈Aut(Gt) _t (G_t), F((φt⋅Gt)t∈T)=ρ((φt)t∈T)F((Gt)t∈T).F\! (( _t\!·\!G_t)_t∈ T )\;=\;ρ\! (( _t)_t∈ T )\,F\! ((G_t)_t∈ T ). When Z carries an index structure inherited from V at each t (e.g., per-node or per-edge features over time), ρ((φt)t)ρ (( _t)_t ) is the natural permutation of those indices, acting componentwise in t. If, in addition, a readout P:Z→YP:Z→ Y is permutation-invariant with respect to ρ, then P∘FP F is invariant under all snapshot automorphism families (Bronstein et al., 2021). Proposition 1. For any epoch t∈Tt∈ T, a per-snapshot permutation φt:V→V _t:V→ V belongs to the snapshot automorphism family at time t if and only if it fixes all input, output, and bias nodes and permutes only hidden neurons within their own layer; equivalently, By membership, this is exactly the class of permutations that preserve the time-t edges: (u,v)∈Et⟺(φt(u),φt(v))∈Etfor all u,v∈V.(u,v)∈ E_t\; \;( _t(u), _t(v))∈ E_t\ \ for all u,v∈ V. Proof. Fix t∈Tt∈ T. (⇒ ) If φt _t belongs to the snapshot automorphism family at time t, then by node–label preservation, it fixes I, O, and B. Moreover, as a graph isomorphism of GtG_t it preserves lengths of directed paths. In an MLP DAG with edges only between adjacent layers, the layer index ℓ(h) (h) of a hidden node h equals the maximum length of a directed path from an input to h. Since permutations render this maximal path length unchanged under φt _t i.e., ℓ(φt(h))=ℓ(h) ( _t(h))= (h) which implies that φt _t maps each hidden layer to itself. (⇐ ) Conversely, if φt _t fixes I, O, and B, maps each hidden layer to itself, and preserves EtE_t, then it satisfies the snapshot automorphism conditions at time t by definition. ∎ Proposition 2. For any trace (Gt)t∈T(G_t)_t∈ T and any family of per-snapshot permutations (ψt)t∈T( _t)_t∈ T with ψt∈Aut(Gt) _t (G_t) at each t, the GTH-GMN encoder satisfies: 1. Snapshot equivariance: at every time t, relabeling the snapshot by ψt _t relabels the time-t output in the same way 2. Previous-step invariance: the temporal transition from step t−1t-1 to t is invariant to any per-snapshot permutation applied at t−1t-1; Consequently, the full encoder is equivariant to ϕt _t at the current step t, while the update from t−1t-1 to t is unaffected by any admissible relabeling at t−1t-1. Proof. 1. At snapshot t, the parameter graph is Gt=(V,Et,θt)G_t=(V,E_t, _t). Each node u∈Vu∈ V carries raw node features xt(u)=[ℓ(u),τ(u),s~abs(u),s~sgn(u)],x_t(u)\;=\; [\; (u),\;τ(u),\; s_abs(u),\; s_sgn(u)\; ], where ℓ(u) (u) is the layer index, τ(u)∈in,hid,out,biasτ(u)∈\in,hid,out,bias\ the node type, and s~abs,s~sgn s_abs, s_sgn are standardized incident-weight aggregations (as defined in Sec. 3.2) computed at node u from the coalesced signed weights at time t. Each edge (u,v)∈Et(u,v)∈ E_t carries raw edge features et(u,v)=[is_bias(u,v),Δℓ(u,v)],e_t(u,v)\;=\; [\; is\_bias(u,v),\; (u,v)\; ], encoding whether the edge originates at a bias node and the layer difference Δℓ(u,v)=ℓ(v)−ℓ(u) (u,v)= (v)- (u). By construction, these descriptors are structural: they do not depend on arbitrary node indices and are invariant under within-layer permutations that fix inputs/outputs/biases. Group action on features. Let ψt∈Aut(Gt) _t (G_t) be any per-snapshot permutation (automorphism) at time t. Its action on the node and edge features is the natural reindexing: [ψt(xt)]ψt(u)=xt(u),[ψt(et)]ψt(u),ψt(v)=et(u,v). [ _t(x_t) ]_ _t(u)\;=\;x_t(u), [ _t(e_t) ]_ _t(u),\, _t(v)\;=\;e_t(u,v). Equivalently, if Pψt∈0,1|V|×|V|P_ _t∈\0,1\^|V|×|V| is the permutation matrix induced by ψt _t on nodes and ψt:=Pψt⊗PψtP_ _t:=P_ _t P_ _t the induced action on ordered node pairs, then in matrix/tensor form ψt(xt)=Pψtxt,ψt(et)=ψt⋅et, _t(x_t)\;=\;P_ _t\,x_t, _t(e_t)\;=\;P_ _t\,·\,e_t, (where the second equality is understood as reindexing the first two modes of the edge-feature tensor). The values of ℓ,τ,s~abs,s~sgn ,τ, s_abs, s_sgn and [is_bias,Δℓ][ is\_bias, ] are preserved because ψt _t fixes node labels for I∪O∪BI∪ O∪ B and only permutes hidden nodes within layers; hence ℓ and τ are unchanged, and the incident-weight aggregations at u move to ψt(u) _t(u) without alteration of their numeric value. Neighborhoods and reindexing. Write t(u):=v∈V:(u,v)∈EtN_t(u):=\\,v∈ V:(u,v)∈ E_t\,\ for the (directed or undirected) out-neighborhood at time t. Automorphism implies t(ψt(u))=ψt(t(u))N_t( _t(u))= _t(N_t(u)), so any neighbor-wise aggregation satisfies ∑v∈t(ψt(u))f(ψt(v))=∑v∈ψt(t(u))f(v)=∑v∈t(u)f(ψt(v)), _v _t( _t(u))f ( _t(v) )\;=\; _v∈ _t(N_t(u))f (v )\;=\; _v _t(u)f ( _t(v) ), and likewise for means and other symmetric reductions. Equivariance of the hyperbolic graph attention block. Let HGATtHGAT_t denote the per-snapshot hyperbolic attention block at time t, with shared parameters WtW_t supplied by the temporal kernel. Its forward map can be abstracted as (Xt,zt)=HGATt(xt,et,Wt),(X_t,z_t)\;=\;HGAT_t (x_t,e_t;\,W_t ), where Xt∈ℝ|V|×dhX_t ^|V|× d_h are tangent embeddings and zt∈(cdz)|V|z_t∈(B_c^d_z)^|V| are hyperbolic embeddings. Internally, the block computes nodewise (tangent) projections, hyperbolic queries/keys/values via shared Möbius-linear maps, attention logits that depend only on index-symmetric quantities (e.g., dc(zt(u),zt(v))d_c(z_t(u),z_t(v)) and et(u,v)e_t(u,v)), neighbor-wise softmax normalizations, and Möbius-weighted sums over t(u)N_t(u). All these operations commute with the reindexing induced by ψt _t. Concretely, letting ℓuv=g(dc(ϕq(zt(u)),ϕk(zt(v))),et(u,v),Xt(u),Xt(v)),αuv=softmaxv∈t(u)ℓuv, _uv\;=\;g (d_c( _q(z_t(u)), _k(z_t(v))),\,e_t(u,v),\,X_t(u),\,X_t(v) ), _uv\;=\;softmax_v _t(u) _uv, one has ℓψt(u)ψt(v)=ℓuv _ _t(u) _t(v)= _uv and hence αψt(u)ψt(v)=αuv _ _t(u) _t(v)= _uv; messages and updates therefore reindex accordingly. It follows that there exists the induced permutation PψtP_ _t such that HGATt(ψt(xt),ψt(et),Wt)=PψtHGATt(xt,et,Wt),HGAT_t (\, _t(x_t),\, _t(e_t)\,;\,W_t )\;=\;P_ _t\,HGAT_t (\,x_t,\,e_t\,;\,W_t ), i.e., the per-snapshot block is permutation equivariant in our setting on both the tangent and hyperbolic branches. 2. Fix t∈Tt∈ T and let ψ∈Aut(Gt−1)ψ (G_t-1) act on node indices with permutation matrix PψP_ψ. Let Xt−1∈ℝ|V|×dxX_t-1 ^|V|× d_x and Zt−1∈(cdz)|V|Z_t-1∈(B_c^d_z)^|V| denote the tangent and hyperbolic node embeddings produced at time t−1t-1. The action of ψ on embeddings is (PψXt−1)(u)=Xt−1(ψ−1(u)),(Pψzt−1)(u)=zt−1(ψ−1(u)).(P_ψX_t-1)(u)\;=\;X_t-1\! (ψ^-1(u) ), (P_ψz_t-1)(u)\;=\;z_t-1\! (ψ^-1(u) ). Euclidean (tangent) pooling is permutation-invariant. Define the pooled tangent vector(s) p(t−1)=1|V|∑u∈VXt−1(u)or layerwisepℓ(t−1)=1|Hℓ|∑u∈HℓXt−1(u).p^(t-1)\;=\; 1|V| _u∈ VX_t-1(u) layerwise p^(t-1)_ \;=\; 1|H_ | _u∈ H_ X_t-1(u). Because ψ fixes I∪O∪BI∪ O∪ B and only permutes hidden nodes within layers, each HℓH_ is preserved. Summation is order-agnostic; hence 1|V|∑u∈V(PψXt−1)(u)=p(t−1),1|Hℓ|∑u∈Hℓ(PψXt−1)(u)=pℓ(t−1)∀ℓ. 1|V| _u∈ V(P_ψX_t-1)(u)\;=\;p^(t-1), 1|H_ | _u∈ H_ (P_ψX_t-1)(u)\;=\;p^(t-1)_ ∀\, . Einstein gyromidpoint aggregation (Klein computation) and permutation invariance. Since linear averages are not geometrically compatible on the Poincaré ball, we aggregate values using the Einstein (gyro) midpoint aggregation, which we restate here. The computation is performed in the Klein model, where weighted barycenters admit a closed form. Let κ:cd→dκ:B^d_c\!→\!K^d be the Poincaré-to-Klein map, κ(z(t))=2cz(t)1+c‖z(t)‖2,z(t)∈cd,κ\! (z^(t) )\;=\; 2 c\,z^(t)1+c\|z^(t)\|^2, z^(t) ^d_c, (19) and κ−1:d→cdκ^-1:K^d\!→\!B^d_c its inverse, κ−1(y(t))=1cy(t)1+1−‖y(t)‖2,y(t)∈d.κ^-1\! (y^(t) )\;=\; 1 c\, y^(t)1+ 1-\|y^(t)\|^2, y^(t) ^d. (20) We also denote by γ(y(t))=1/1−‖y(t)‖2γ(y^(t))=1/ 1-\|y^(t)\|^2 the Lorentz factor. For node v and head r at time t, the Klein-space barycenter of the incoming neighborhood is κ(mv(r,t))=∑u:(u→v)∈Etα~(r,t)u→vγ(κ(v(r,t)u))κ(v(r,t)u)∑u:(u→v)∈Etα~(r,t)u→vγ(κ(v(r,t)u)).κ\! (m^(r,t)_v )\;=\; Σ _u:\,(u\!→\!v)∈ E_t α^(r,t)_u→ v\,γ\! (κ(v^(r,t)_u) )\,κ\! (v^(r,t)_u )Σ _u:\,(u\!→\!v)∈ E_t α^(r,t)_u→ v\,γ\! (κ(v^(r,t)_u) ). (21) We then map back to the Poincaré ball and linearize at the origin for the residual path: mv(r,t)=κ−1(κ(mv(r,t)))∈cd,m^v(r,t)=log0(mv(r,t))∈T0cd,m^(r,t)_v\;=\;κ^-1\! (κ\! (m^(r,t)_v ) ) ^d_c, m^(r,t)_v\;=\; _0\! (m^(r,t)_v )∈ T_0B^d_c, (22) and average across heads, Mv(t)=1H∑r=1Hm^v(r,t).M^(t)_v\;=\; 1H _r=1^H m^(r,t)_v. (23) Per-snapshot permutation equivariance of the Einstein aggregator. Let ψt∈Aut(Gt) _t (G_t) be any per-snapshot permutation. Automorphism implies u:(u→v)∈Et\u:(u\!→\!v)∈ E_t\ is sent to u′:(u′→ψt(v))∈Et\u :(u \!→\! _t(v))∈ E_t\ with u′=ψt(u)u = _t(u), and snapshot equivariance of attention yields α~ψt(u)→ψt(v)(r,t)=α~u→v(r,t) α^(r,t)_ _t(u)→ _t(v)= α^(r,t)_u→ v. Applying equation 21 at node ψt(v) _t(v) and reindexing the sums by u′=ψt(u)u = _t(u) gives κ(mψt(v)(r,t))=∑uα~u→v(r,t)γ(κ(vψt(u)(r,t)))κ(vψt(u)(r,t))∑uα~u→v(r,t)γ(κ(vψt(u)(r,t))).κ\! (m^(r,t)_ _t(v) )= _u α^(r,t)_u→ v\,γ\! (κ(v^(r,t)_ _t(u)) )\,κ\! (v^(r,t)_ _t(u) ) _u α^(r,t)_u→ v\,γ\! (κ(v^(r,t)_ _t(u)) ). Thus the Einstein weighted midpoint is symmetric in the multiset of weighted Klein vectors and therefore commutes with the reindexing induced by ψt _t; mapping back by κ−1κ^-1 and log0 _0 preserves this equivariance. Consequently, Mψt(v)(t)M^(t)_ _t(v) is exactly the PψtP_ _t-reindexing of Mv(t)M^(t)_v. At time t−1t\!-\!1, let the hyperbolic node embeddings be Zt−1=zu(t−1)u∈V⊂cdZ_t-1=\z^(t-1)_u\_u∈ V ^d_c and the tangent embeddings be Xt−1=Xt−1(u)u∈V⊂ℝdxX_t-1=\X_t-1(u)\_u∈ V ^d_x. We summarize these for the temporal kernel using (i) Euclidean means in the tangent space and (i) Einstein gyromidpoints in Klein space, consistent with Eqs. equation 19–equation 23. Define the global (or layerwise) tangent summaries p(t−1)=1|V|∑u∈VXt−1(u),pℓ(t−1)=1|Hℓ|∑u∈HℓXt−1(u).p^(t-1)\;=\; 1|V| _u∈ VX_t-1(u), p^(t-1)_ \;=\; 1|H_ | _u∈ H_ X_t-1(u). These are permutation-invariant because they are simple averages over sets preserved by automorphisms. Let κ and κ−1κ^-1 be the Poincaré–Klein maps in equation 19–equation 20, and γ(y)=1/ 1−‖y‖2γ(y)=1/ \,1-\|y\|^2\,. With uniform weights, the global Klein barycenter is κ(z¯t−1)=∑u∈Vγ(κ(zu(t−1)))κ(zu(t−1))∑u∈Vγ(κ(zu(t−1))),z¯t−1=κ−1(κ(z¯t−1)).κ\! ( z_t-1 )\;=\; Σ _u∈ Vγ\! (κ(z^(t-1)_u) )\,κ\! (z^(t-1)_u )Σ _u∈ Vγ\! (κ(z^(t-1)_u) ), z_t-1\;=\;κ^-1\! (κ( z_t-1) ). For layerwise pooling, restrict the sums to u∈Hℓu∈ H_ to obtain z¯t−1(ℓ) z^( )_t-1. For compatibility with the temporal GRU/MLP operating in the tangent space, we finally use z¯^t−1=log0(z¯t−1),z¯^t−1(ℓ)=log0(z¯t−1(ℓ)). z_t-1\;=\; _0\! ( z_t-1 ), z^( )_t-1\;=\; _0\! ( z^( )_t-1 ). Let ψ∈Aut(Gt−1)ψ (G_t-1) with induced reindexing (Pψz(t−1))u=zψ−1(u)(t−1)(P_ψz^(t-1))_u=z^(t-1)_ψ^-1(u). In the global Einstein midpoint, the numerator and denominator are sums over the multiset γ(κ(zu(t−1)))κ(zu(t−1))u∈V \γ(κ(z^(t-1)_u))\,κ(z^(t-1)_u) \_u∈ V, which are merely reordered by ψ. Hence κ(z¯t−1(Pψz(t−1)))=κ(z¯t−1(z(t−1))),⟹z¯t−1(Pψz(t−1))=z¯t−1(z(t−1)),κ\! ( z_t-1(P_ψz^(t-1)) )\;=\;κ\! ( z_t-1(z^(t-1)) ), z_t-1(P_ψz^(t-1))\;=\; z_t-1(z^(t-1)), and applying log0 _0 preserves equality: z¯^t−1(Pψz(t−1))=z¯^t−1(z(t−1)) z_t-1(P_ψz^(t-1))= z_t-1(z^(t-1)). The same argument holds layerwise because ψ maps each HℓH_ to itself. Therefore, the hyperbolic pooled summaries z¯^t−1 z_t-1 and z¯^t−1(ℓ)ℓ\ z^( )_t-1\_ are invariant to any admissible permutation at time t−1t-1. Let Xt−1X_t-1 be the tangent embeddings at t−1t-1 (log0(zt−1) _0(z^t-1)), and let p(t−1)p^(t-1) (or pℓ(t−1)\p^(t-1)_ \) denote their mean pool(s). Since p(t−1)p^(t-1) is an ordinary Euclidean average and layer sets are preserved by automorphisms, p(t−1)p^(t-1) is permutation invariant. The Evolve-GCN-H style kernel then consumes only node-agnostic summaries, (Wt,ut)=ϕMLP(GRU(p(t−1),ut−1))(or with layerwise pools),(W_t,u_t)\;=\; _MLP\! (GRU\! (p^(t-1),\ u_t-1 ) )\ \ \ (or with layerwise pools), where ut−1u_t-1 is learned hidden temporal vector, which proves invariance of (Wt,ut)(W_t,u_t) to any per-snapshot permutation at time t−1t-1. This completes the previous-step invariance argument, i.e, the temporal kernel renders the module invariant to permutations in the previous time step (Bronstein et al., 2021). ∎ Appendix E Experiment Details E.1 Problem Formulation We study dynamic parameter graphs extracted from the training of a base multilayer perceptron (MLP). A dynamic parameter graph is a sequence =Gt=(V,Et,Wt)t=1TG=\G_t=(V,E_t,W_t)\_t=1^T of T snapshots, where V is the fixed node set (inputs, hidden units, and biases), EtE_t is the set of undirected edges at time t, and Wt=wuv⋆,(t):(u,v)∈EtW_t=\w ,(t)_uv:(u,v)∈ E_t\ the associated signed weights. The goal is to learn node embeddings zi(t)∈cd:i∈V,t=1,…,T\z^(t)_i ^d_c:i∈ V,\;t=1,…,T\ in the d–dimensional Poincaré ball such that: (i) a Fermi–Dirac decoder ψ(t)(u,v)ψ^(t)(u,v) predicts adjacency labels, (i) a signed regressor w^uv(t) w^(t)_uv estimates wuv⋆,(t)w ,(t)_uv, and (i) trajectories zi(t)t\z^(t)_i\_t evolve smoothly across time. Supervision is provided on positive edges, with negative samples generated adaptively, placing the task in the semi–supervised, temporal graph embedding setting. E.2 Additional Details: Classification of Images via INR Traces Base INR architecture and training. For each image x∈MNIST, Fashion-MNISTx∈\MNIST, Fashion-MNIST\ we train a shallow implicit neural representation (INR) to fit pixel intensities as a continuous function of coordinates. The INR has three fully connected layers of width 32 with tanh activations, mapping u∈[0,1]2u∈[0,1]^2 (pixel coordinates) to grayscale value x(u)x(u). Weights are initialized with Xavier initialization and optimized using Adam (learning rate 5×10−45× 10^-4) for up to 100 epochs, with early stopping if the PSNR of the reconstruction exceeds 40. This produces a sequence of checkpoints θi,tt=1Ti\ _i,t\_t=1^T_i for image i, where Ti∈[80,100]T_i∈[80,100] depending on early stopping. Parameter-graph construction. At each checkpoint t, the INR parameters θi,t _i,t are transformed into a signed parameter graph Gi,tG_i,t: • nodes correspond to input, hidden, and bias units; • edges connect units across successive layers and from biases to their layer, labeled by the signed weight; • node features include layer index, node type, and z–scored sums of incident weights (absolute and signed); • edge features indicate whether the edge originates from a bias node and the layer jump Δℓ . This yields a temporal trace Gi,tt=1Ti\G_i,t\_t=1^T_i per image. Trace dataset and splits. Each image contributes one temporal trace. Dataset splits are made strictly at the image level to prevent leakage across time. We use 45,000 images for training, 5,000 for validation, and 10,000 for testing. All checkpoints from an image remain in the same split. Each trace contains between 80 and 100 checkpoints, ensuring sufficient temporal depth. Meta-network pretraining. We pretrain our Geometric Temporal Hyperbolic Graph Meta-Network (GTH-GMN) on the training traces. The backbone consists of L=4L=4 Hyperbolic Graph Attention (HGAT) layers, each paired with one EvolveGCN–H temporal kernel, so that the number of HGAT layers and temporal kernels is always equal. Embeddings live in cdB^d_c with dimension d=125d=125 and curvature parameter c=−1c=-1. The optimization alternates between: 1. Euclidean updates of the attention kernels, regressors, and decoder; 2. Riemannian updates of node embeddings with trust-region control. We use RAMSGrad with learning rate 10−310^-3 and trust radius τ=0.1τ=0.1. Losses and curriculum. The training objective combines: • Fermi–Dirac link prediction loss (binary cross-entropy); • signed weight regression (L2 loss on log–magnitude + sign classification); • quadratic prior on the learned distance exponent; • temporal smoothness penalty on successive embeddings; • pairwise logistic ranking loss on semi–hard negative pairs. A gentle curriculum is applied: the ranking margin increases over epochs, and mining hardness is annealed to prevent early saturation. Representation extraction. During pretraining, edge embeddings within each checkpoint are pooled into a snapshot vector by mean aggregation. For downstream classification, snapshot vectors are pooled temporally (mean across checkpoints in the trace) to yield a fixed n–dimensional representation per image. Classifier and evaluation. An MLP classifier (n→128→10n→ 128→ 10, with ReLU) is trained on these representations using cross-entropy loss. The classifier is trained on the training set, tuned on the validation set, and evaluated on the test set. Accuracy on the test set is reported in Table 1. E.3 Additional Details: Predicting DNN Generalization from Weights We evaluate on checkpoint traces from implicit neural representation (INR) trials trained for CIFAR-10 classification. For each trial i we collect the ordered checkpoints θi,tt=1Ti\ _i,t\_t=1^T_i together with the corresponding per-epoch test accuracy yi,ty_i,t. Using the architecture metadata stored with each trial, we reconstruct a shape-compatible network skeleton and convert every checkpoint to a parameter-graph Gi,tG_i,t, where nodes and edges encode their respective features. No image content is used; supervision is solely the scalar accuracy aligned to checkpoints. Figure 3: Predicted versus actual generalization performance on CIFAR-10 checkpoint traces. Each point corresponds to a model trial at a given epoch, where predictions are obtained from the learned GTH-GMN embeddings. The strong monotonic trend (Kendall’s τ=0.846±0.004τ=0.846± 0.004) indicates that hyperbolic embeddings capture meaningful structure in the parameter dynamics, even if absolute correlation lags slightly behind NFN baselines. Each trace is processed by the encoder introduced in the previous section over fixed-length, stride-rolled windows or full traces (depending on computational constraints). We adopt the last-step supervision regime: intermediate steps are used only to evolve the state, and supervision is applied to the final step target. Dataset splits are made strictly by trial (not by checkpoint) to prevent leakage across traces. For all experiments, we embed parameter-graphs into a hyperbolic ball of dimension d∈[80,125]d∈[80,125], with four HGAT layers and four EvolveGCN–H temporal kernels of hidden size 128. Hyperbolic embeddings are optimized with the two-phase scheme described in Section 3.4, combining Euclidean updates for kernel parameters with Riemannian gradient steps for node positions. Training employs binary cross-entropy on Fermi–Dirac link logits, signed weight regression, a prior on the learned exponent, temporal smoothness penalties, and a ranking curriculum. We report Kendall’s τ for ranking fidelity, following Zhou et al. (2023a); Navon et al. (2024); Lim et al. (2023). Results are shown in Table 3 and illustrated qualitatively in Figure 3. Our method GTH-GMN achieves τ=0.846±0.004τ=0.846± 0.004, lower than NFN baselines that learn end–to–end from raw weight tensors. This gap is expected, since NFNs exploit stronger task-aligned inductive biases while our approach compresses parameters into permutation-invariant graphs and trains with geometric proxy losses under last-step supervision, inevitably discarding certain accuracy-correlated cues. Nonetheless, GTH-GMN offers a complementary benefit by jointly modeling link structure, signed weights, and temporal evolution in hyperbolic space, producing embeddings that are robust, interpretable, and faithful to training dynamics. E.4 Additional Details: Predicting Sine Wave Frequency In addition to supervised objectives, we employ a SimCLR-style (Chen et al., 2020) contrastive regularizer adapted to temporal checkpoint traces. This setup treats the optimization trajectory of a neural network analogously to an image in a classification task, operating in “parameter-time space” rather than pixel space. For each trial, two correlated views of the same checkpoint subsequence are generated by combining: (i) Gaussian perturbations of node and edge features, and (i) Random temporal masking, where a subset of checkpoints in the subsequence is dropped with probability p=0.5p=0.5. The encoder processes both views to produce pooled representations, which are learned using a contrastive loss (Chen et al., 2020). Positive pairs correspond to augmented views of the same checkpoint subsequence (representing the same underlying function dynamics), while negative pairs are drawn from other trials. This auxiliary objective ensures that the learned geometric representation captures the intrinsic signature of the optimization trajectory. E.5 Additional Details: Weights vs Distance Relationships Figure 4: Relationship between hyperbolic distance d and edge weights w on CIFAR-10 INR traces, shown as log-count hexbins. (Top Left, Bottom Left) Distance versus absolute weight |w||w| (at t∈Tt∈ T), highlighting the concentration of large weights at small distances and the rapid decay at larger distances. (Top Right, Bottom Right) Distance versus log-magnitude log|w| |w| (at t′∈Tt ∈ T), which reveals an approximately linear inverse trend, consistent with a power–law dependence of weight magnitudes on hyperbolic distance. Together, these plots illustrate that the hyperbolic geometry learned by GTH-GMN aligns distances with functional weight strengths, capturing both edge existence and graded magnitude in a geometrically coherent manner. To probe the strength of the geometric relationship captured by GTH-GMN, we visualize the alignment between learned hyperbolic distances and underlying edge weights on the CIFAR-10 INR traces. Figure 4 reports hexbinned joint distributions of hyperbolic distance d against both |w||w| and log|w| |w|. The results reveal a clear inverse correlation: large weights tend to concentrate at smaller hyperbolic distances, while edges at larger distances exhibit systematically lower magnitudes. On the logarithmic scale, the relationship is approximately linear, consistent with the power–law parameterization employed in our signed edge–weight regressor. These findings support the claim that temporal hyperbolic embeddings faithfully encode functional structure: the geometry reflects not only the presence of connections but also their graded strengths across training, thereby grounding the link between parameter values and hyperbolic distance. Appendix F Hyperparameter Investigation Table 5: Architecture and data settings for our Experiments. Component Hyperparameter Range / Value Manifold Curvature |c||c| fixed to 1.01.0 Embedding dimension d 80− 12580\;-\;125 HGAT (per layer) # layers L 44 # heads H 4− 84\;-\;8 (tuned) Head temperature learned, init 0.5− 1.00.5\;-\;1.0 Residual gate learned scalar ∈(0,1)∈(0,1) Geodesic trust radius τℓ _ 0.05− 0.150.05\;-\;0.15 EvolveGCN–H # temporal kernels 44 (equal to HGAT layers) GRU hidden size 64− 12864\;-\;128 Input to GRU mean pool in T0T_0 (prev. snapshot) Graph construction Node features (ℓ,t,sabs,ssgn)( ,t,s^abs,s^sgn) Edge features (is_bias,Δℓ)(is\_bias, ) Temporal window Sequence length K 64−T64\;-\;T (full trace, T∈[80,100]T\!∈[80,100]) Stride 4− 84\;-\;8 Contrastive views Feature noise (Gaussian) σ=0.1− 0.3σ=0.1\;-\;0.3 (SimCLR-style) Temporal masking drop prob. p=0.3− 0.5p=0.3\;-\;0.5 Temperature (NT-Xent) 0.1− 0.50.1\;-\;0.5 Table 6: Optimization and loss settings for INR-trace image classification. Ranges correspond to grid or random search over hyperparameters. Component Hyperparameter Range / Value Euclidean params Optimizer / LR Adam, lr=×10−3−×10−3lr=1\!×\!10^-3\;-\;3\!×\!10^-3 (kernels, heads) Weight decay 10−6− 10−410^-6\;-\;10^-4 LR schedule StepLR (step=10− 20step=10\;-\;20, γ=0.6− 0.8γ=0.6\;-\;0.8) Epochs 30− 5030\;-\;50 Manifold params Optimizer RAMSGrad (Riemannian) (node positions z) LR / trust radius lr=×10−3−×10−3lr=1\!×\!10^-3\;-\;3\!×\!10^-3, τ=0.05− 0.15τ=0.05\;-\;0.15 (β1,β2)( _1, _2) (0.9, 0.99− 0.999)(0.9,\,0.99\;-\;0.999) Conformal precond. divide by λx2 _x^2 Link decoder Fermi–Dirac radius R init 1.5− 2.51.5\;-\;2.5 (learned) Temperature T init 0.3− 0.70.3\;-\;0.7 (learned) Negatives Neg. multiplier 3− 10×3\;-\;10× positives Same-layer ratio 0.3− 0.70.3\;-\;0.7 Losses (Euclid pass) FD BCE (links) weight ramp to 1.01.0 Signed weight recon λwrec=0.2− 0.6 _wrec=0.2\;-\;0.6 Sign BCE λsign=0.2− 0.4 _sign=0.2\;-\;0.4 Exponent prior λα=10−3− 10−2 _α=10^-3\;-\;10^-2 Temporal smoothness βsmooth=0.2− 0.6 _smooth=0.2\;-\;0.6 Ranking (pairs) λrank=0.5− 1.0 _rank=0.5\;-\;1.0, margin γ=0.2− 0.5γ=0.2\;-\;0.5 Losses (z-only pass) Same terms as Euclid pass; X(t)X^(t) detached Practical caps Pos. edges per step 10,000− 20,00010,000\;-\;20,000 FD / distance chunks 20,000− 40,00020,000\;-\;40,000 edges / chunk Hyperparameter Investigation. Tables 5–6 summarize the settings used in our studies. Architecture choices fix the manifold curvature at |c|=1.0|c|=1.0 and sweep embedding dimension d∈[80,125]d∈[80,125]; HGAT uses L=4L=4 layers with H∈[4,8]H∈[4,8], learned head temperatures, a learned residual gate, and per–layer geodesic trust radii τℓ∈[0.05,0.15] _ ∈[0.05,0.15]. EvolveGCN–H mirrors the HGAT depth with GRU size 6464–128128, while the parameter graph encodes (ℓ,t,sabs,ssgn)( ,t,s^abs,s^sgn) at nodes and (is_bias,Δℓ)(is\_bias, ) at edges. Temporal windows use K∈[64,T]K∈[64,T] with stride 44–88, and SimCLR-style augmentations vary feature noise, temporal masking, and NT-Xent temperature. Optimization (Table 6) employs Adam for Euclidean parameters with lr∈[1×10−3,3×10−3]lr∈[1×10^-3,3×10^-3], weight decay 10−610^-6–10−410^-4, StepLR, and 3030–5050 epochs; node positions use Riemannian AMSGrad (RAMSGrad) with matched learning rates, trust radius τ∈[0.05,0.15]τ∈[0.05,0.15], (β1,β2)=(0.9,0.99CLOSE( _1, _2)=(0.9,0.99–OPEN0.999)0.999), and conformal preconditioning. The FD decoder learns radius R and temperature T from initialized ranges, negatives mix same-layer and uniform samples, and losses combine FD BCE, signed weight reconstruction, sign BCE, exponent prior, temporal smoothness, and pairwise ranking with reported weights; the same losses are reused in a z–only pass with X(t)X^(t) detached. For throughput, we cap positive edges and chunk FD/distance computations as listed. Appendix G Algorithms Overview of Algorithms. Algorithm 2 converts a single MLP checkpoint into a permutation–invariant parameter graph by instantiating input, neuron, and bias nodes; coalescing directed parameter incidences into undirected edges with context and signed labels; and standardizing nodewise strength features to form Gt=(Xt,E,Wt)G_t=(X_t,E,W_t). Algorithm 1 then updates node positions intrinsically on the Poincaré ball: Euclidean gradients are rescaled to Riemannian ones, Adam–style moments are accumulated, a geodesic trust region clips the tangent step, the exponential map applies the update with a safety projection, and first moments are parallel transported. The two procedures are independent and can be composed sequentially within each training step. Optimization Curriculum We map hyperbolic geometry to link probabilities with a Fermi–Dirac (FD) decoder acting on Poincaré geodesics (Nickel and Kiela, 2017; Nickel and Kiela, 2018; Wen et al., 2024): ψ(t)(u,v)=(1+exp(dc(zu(t),zv(t))−RT))−1,ψ^(t)(u,v)= (1+ \! ( d_c(z^(t)_u,z^(t)_v)-RT ) )^-1, (24) with learned radius R and temperature T. Training uses positives EtE_t and dynamic negatives from a mixture of same–layer and uniform samplers (mixture capped at 50%). The FD loss is binary cross–entropy, ℒFD=−(u,v)∈E+[logψ(t)(u,v)]−(u,v)∈E−[log(1−ψ(t)(u,v))].L_FD=-\,E_(u,v)∈ E^+\! [ ψ^(t)(u,v) ]-\,E_(u,v)∈ E^-\! [ (1-ψ^(t)(u,v)) ]. (25) To capture edge magnitudes and signs, we add the signed regressor of Section 3.3. For (u,v)∈E+(u,v)∈ E^+ with weight wuv⋆w _uv and prediction w^uv(t) w^(t)_uv from equation 13 (Goyal et al., 2018; Pareja et al., 2019), ℒwrec _wrec =(u,v)∈E+[∥w^uv(t)−wuv⋆∥p], =E_(u,v)∈ E^+\! [ w^(t)_uv-w _uv _p ], (26a) ℒsign _sign =−(u,v)∈E+[yuvlogσ(suv(t))+(1−yuv)log(1−σ(suv(t)))]. =-\,E_(u,v)∈ E^+\! [y_uv\, σ\! (s^(t)_uv )+(1-y_uv)\, \! (1-σ(s^(t)_uv) ) ]. (26b) with p∈1,2p∈\1,2\ and yuv=[wuv⋆>0]y_uv=1[w _uv>0]. We regularize the exponent via ℒα=(u,v)∈E+[(αuv−α0)2],L_α=E_(u,v)∈ E^+\! [( _uv- _0)^2 ], (27) encourage smooth trajectories (Goyal et al., 2018; Yang et al., 2023), ℒsmooth=1|V|∑i∈Vdc(zi(t),zi(t−1))2,L_smooth= 1|V| _i∈ Vd_c\! (z^(t)_i,z^(t-1)_i )^2, (28) and enforce distance–magnitude monotonicity with a logistic pairwise rank loss (Nickel and Kiela, 2017), ℒrank=u,(v+,v−)log(1+exp(dc(zu(t),zv+(t))−dc(zu(t),zv−(t))+γ)).L_rank=E_u,(v^+,v^-) \! (1+ (d_c(z^(t)_u,z^(t)_v^+)-d_c(z^(t)_u,z^(t)_v^-)+γ) ). (29) The total objective is ℒ(t)=λFDℒFD+λwrecℒwrec+λsignℒsign+λαℒα+λsmoothℒsmooth+λrankℒrank,L^(t)= _FDL_FD+ _wrecL_wrec+ _signL_sign+ _αL_α+ _smoothL_smooth+ _rankL_rank, (30) with a gentle curriculum: anneal γ and harden mining over epochs. Training runs in two phases: a Euclidean pass updates kernels, regressors, and decoder; a Riemannian pass updates node positions z(t)z^(t) on the Poincaré ball. With conformal factor λz2=21−c‖z‖2 _z^2= 21-c\|z\|^2, the Riemannian gradient rescales the Euclidean one (Bridson and Haefliger, 1999; Bonnabel, 2013; Nickel and Kiela, 2017; Nickel and Kiela, 2018): ∇zRiemℒ=1λz2∇zEucℒ.∇^Riem_zL= 1 _z^2\,∇^Euc_zL. (31) We maintain first and second moments and take an Adam–style tangent step (Bécigneul and Ganea, 2019; Sakai and Iiduka, 2020): Δi(t)=−ηmi(t)vi(t)+ε. _i^(t)=-\,η\, m^(t)_i v^(t)_i+ . (32) A geodesic trust region clips the move, not its Euclidean norm. With zitry,(t)=expzi(t)(Δi(t)),di(t)=dc(zi(t),zitry,(t)),z^try,(t)_i= _z^(t)_i( _i^(t)), d^(t)_i=d_c\! (z^(t)_i,z^try,(t)_i ), (33) we scale Δi(t) _i^(t) by min1,τ/di(t) \1,τ/d^(t)_i\ and accept the intrinsic update zi(t+1)=expzi(t)(Δi(t)),z^(t+1)_i= _z^(t)_i( _i^(t)), (34) followed by projection into the ball. First–moment vectors are then parallel transported: mi(t+1)=PTzi(t)→zi(t+1)(mi(t)).m^(t+1)_i=PT_z^(t)_i→ z^(t+1)_i (m^(t)_i ). (35) The manifold step optimizes a z–only objective mirroring equation 25–equation 29, with X(t)X^(t) detached. This decouples the hyperbolic graph attention head’s stability from embedding motion and enforces controlled geodesic updates near the boundary. Algorithm 1 Intrinsic Riemannian Optimization of Node Positions on the Poincaré Ball 1: Snapshots Gtt=1T\G_t\_t=1^T; initial positions zi(t)\z^(t)_i\; moments mi(t)=0,vi(t)=0m^(t)_i=0,v^(t)_i=0; learning rate η; trust radius τ; tolerance ε ; curvature c 2: Updated positions zi(t)t=1T\z^(t)_i\_t=1^T 3: for t=1t=1 to T do ⊳ after Euclidean parameter update 4: Detach X(t)X^(t); compute Euclidean gradient ∇zi(t)Eucℒ∇^Euc_z^(t)_iL for losses equation 25–equation 29 5: Convert to Riemannian gradient with conformal factor λz2=21−c‖z‖2 _z^2= 21-c\|z\|^2: ∇zi(t)Riemℒ=λzi(t)−2∇zi(t)Eucℒ∇^Riem_z^(t)_iL= _z^(t)_i^-2\,∇^Euc_z^(t)_iL 6: Update moments (Adam/AMSGrad) (Bécigneul and Ganea, 2019; Sakai and Iiduka, 2020): mi(t)←β1mi(t)+(1−β1)∇zi(t)Riemℒm^(t)_i← _1m^(t)_i+(1- _1)∇^Riem_z^(t)_iL vi(t)←β2vi(t)+(1−β2)(∇zi(t)Riemℒ)⊙2v^(t)_i← _2v^(t)_i+(1- _2) (∇^Riem_z^(t)_iL ) 2 7: Compute tangent step: Δi=−ηmi(t)vi(t)+ε _i=-\,η\, m^(t)_i v^(t)_i+ 8: Trial update and geodesic distance: zitry=expzi(t)(Δi),di=dc(zi(t),zitry)z^try_i= _z^(t)_i( _i), d_i=d_c(z^(t)_i,z^try_i) 9: Trust scaling: si=min1,τ/(di+ε)s_i= \1,τ/(d_i+ )\; Δi←siΔi _i← s_i _i 10: Accept update and project inside ball: zi(t+1)=expzi(t)(Δi),zi(t+1)←Πcd(zi(t+1),ε)z^(t+1)_i= _z^(t)_i( _i), z^(t+1)_i← _B^d_c(z^(t+1)_i; ) 11: Parallel transport first moment: mi(t+1)=PTzi(t)→zi(t+1)(mi(t))m^(t+1)_i=PT_z^(t)_i→ z^(t+1)_i(m^(t)_i) 12: end for Algorithm 2 MLP → Parameter Graph (single snapshot at epoch t) 1: Layer widths (n0,…,nL)(n_0,…,n_L); weights Wℓ=1L\W_ \_ =1^L; biases bℓ=1L\b_ \_ =1^L 2: Snapshot Gt=(Xt,E,Wt)G_t=(X_t,E,W_t) 3: V←i1(0),…,in0(0)V←\\,i^(0)_1,…,i^(0)_n_0\,\ ⊳ input nodes 4: for ℓ←1 ← 1 to L do 5: add bias node b(ℓ)b^( ) to V 6: add neuron nodes i1(ℓ),…,inℓ(ℓ)i^( )_1,…,i^( )_n_ to V 7: set (ℓ(u),τ(u))( (u),τ(u)) for all newly added u ⊳ layer index, type 8: end for 9: →←∅A_→← ⊳ directed edge bag with context + weight 10: for ℓ←1 ← 1 to L do 11: for p←1p← 1 to nℓ−1n_ -1 do 12: for q←1q← 1 to nℓn_ do 13: append (ip(ℓ−1)→iq(ℓ),(is_bias=0,Δℓ=1),w←Wℓ[q,p]) (i^( -1)_p→ i^( )_q,\;(is\_bias=0, =1),\;w← W_ [q,p] ) to →A_→ 14: end for 15: end for 16: for q←1q← 1 to nℓn_ do 17: append (b(ℓ)→iq(ℓ),(is_bias=1,Δℓ=0),w←bℓ[q]) (b^( )→ i^( )_q,\;(is\_bias=1, =0),\;w← b_ [q] ) to →A_→ 18: end for 19: end for 20: M←M← empty map from unordered pairs u,v\u,v\ to running sums 21: for all (u→v,e,w)∈→(u\!→\!v,\;e,\;w) _→ do 22: k←u,vk←\u,v\ 23: update M[k].sum_ctx+=eM[k].sum\_ctx +=e; M[k].sum_w+=wM[k].sum\_w +=w; M[k].cnt+=1M[k].cnt +=1 24: end for 25: E←∅E← , Wt←∅W_t← 26: for all k=u,vk=\u,v\ in M do 27: euv←M[k].sum_ctx/M[k].cnte_uv← M[k].sum\_ctx/M[k].cnt ⊳ coalesced context (mean) 28: wuv⋆←M[k].sum_w/M[k].cntw _uv← M[k].sum\_w/M[k].cnt ⊳ coalesced label (mean) 29: insert u,v\u,v\ into E with context euve_uv; set Wt[u,v]←wuv⋆W_t[\u,v\]← w _uv 30: end for 31: for all u∈Vu∈ V: sabs(u)←∑v:u,v∈E|Wt[u,v]|s^abs(u)← _v:\u,v\∈ E|W_t[\u,v\]|; ssgn(u)←∑v:u,v∈EWt[u,v]s^sgn(u)← _v:\u,v\∈ EW_t[\u,v\] 32: μabs,σabs← _abs, _abs← mean/std of sabs(u):u∈V\s^abs(u):u∈ V\ 33: μsgn,σsgn← _sgn, _sgn← mean/std of ssgn(u):u∈V\s^sgn(u):u∈ V\ 34: for all u∈Vu∈ V do 35: s~abs(u)←sabs(u)−μabsσabs,σabs>00,otherwise s^abs(u)← cases s^abs(u)- _abs _abs,& _abs>0\\ 0,&otherwise cases 36: s~sgn(u)←ssgn(u)−μsgnσsgn,σsgn>00,otherwise s^sgn(u)← cases s^sgn(u)- _sgn _sgn,& _sgn>0\\ 0,&otherwise cases 37: end for 38: Xt←xu=[ℓ(u),τ(u),s~abs(u),s~sgn(u)]:u∈VX_t← \x_u=[\, (u),τ(u), s^abs(u), s^sgn(u)\,]\;:\;u∈ V \ 39: return Gt=(Xt,E,Wt)G_t=(X_t,E,W_t) Appendix H Hyperbolic Embeddings of INRs Figure 5: Hyperbolic embeddings of parameter graphs from a 25-layer MLP trained as an INR on CIFAR-10, shown at checkpoints. Embeddings are learned in d=10,25d=10,25 dimensions with 4 HGAT layers and 4 EvolveGCN–H kernels (tangent dim. 128) and projected to the Poincaré disk via PCA. Nodes are colored by layer index, with edges in gray, illustrating the progressive reorganization and separation of layers over training. We visualize hyperbolic embeddings of parameter graphs from a 25-layer INR–MLP at several checkpoints 5. Embeddings are learned with 4 HGAT layers and 4 EvolveGCN–H kernels (tangent dim. 128) in d=10d\!=\!10 and d=25d\!=\!25, then projected to the Poincaré disk via PCA for display. Points are colored by layer, and edges are gray. Early snapshots show mixed layers and long inter-layer edges. As training proceeds, layers separate along angular directions, intra-layer bands tighten, and high-strength nodes drift radially inward, indicating hub formation and growing specialization. The smooth drift of clusters across checkpoints reflects temporal kernel meta-evolution rather than abrupt reconfiguration. Because these plots are PCA projections from a higher-dimensional ball, exact angles are not preserved; nevertheless, the progressive stratification and the shortening of within-layer geodesics are consistent across panels and align with the learned distance–magnitude coupling. Appendix I Computational cost analysis We analyze the time and space complexity of GTH–GMN in terms of the size of the parameter graphs and the temporal window length. Let a parameter graph snapshot GtG_t have N nodes and E edges. Embeddings live in a d dimensional Poincaré ball, the hyperbolic encoder uses H attention heads and L HGAT layers, the temporal kernel has window length K and GRU hidden size h, and each training batch contains Ebatch+E^+_batch positive edges. Time Complexity of the Hyperbolic Encoder. For a single HGAT layer on GtG_t, hyperbolic queries, keys and values are obtained by applying the affine maps ΦW,b _W,b at each node and head. Each such map performs one logarithmic map log0 _0, one matrix–vector product Wlog0(x)W _0(x), one exponential map exp0 _0, and a bias transport plus exponential at ΦW(x) _W(x), all on d dimensional vectors. The cost per head and node is therefore O(d2+d)O(d^2+d), and aggregated over all nodes and heads this yields CostQKV=O(HNd2).Cost_QKV=O (HNd^2 ). (36) Attention logits and softmax normalization are computed per edge and head. For each directed edge (u,v)(u,v), the Poincaré distance dc(qu,kv)d_c(q_u,k_v) requires a constant number of inner products and norms in ℝdR^d, so O(d)O(d) operations, followed by scalar exponentiation and normalization. Einstein gyromidpoint aggregation in the Klein model combines value vectors from the incoming edges of each node and maps the result back via κ−1κ^-1 and log0 _0, which again contributes O(d)O(d) per edge and head plus O(d)O(d) per node and head. Summed over all edges, nodes and heads, the attention and aggregation stage costs Costatt=O(HEd+HNd).Cost_att=O (HEd+HNd ). (37) The total cost of a single HGAT layer is therefore CostHGAT, layer=O(HNd2+HEd),Cost_HGAT, layer=O (HNd^2+HEd ), (38) and a stack of L layers has CostHGAT=O(LHNd2+LHEd).Cost_HGAT=O (LHNd^2+LHEd ). (39) In the regimes we study, E is much larger than N and d is moderate, so the edge term O(LHEd)O(LHEd) dominates and the cost is effectively linear in the number of edges. The hyperbolic operations (exp, log, distance and parallel transport) are all O(d)O(d) vector calculations and do not change the asymptotic order relative to a Euclidean graph attention network with the same (L,H,d)(L,H,d). Time Complexity of Temporal Kernel Evolution. Temporal evolution follows the EvolveGCN–H scheme (Pareja et al., 2019), where each HGAT layer maintains a GRU with hidden size h over a window of length K. A single GRU step processes a pooled d dimensional summary and costs O(h2+hd)O(h^2+hd). For K steps and L layers, the total temporal cost is Costtemp=O(LK(h2+hd)).Cost_temp=O (LK(h^2+hd) ). (40) This term is linear in the window length and independent of E. For the configurations used in our experiments, HEdHEd dominates K(h2+hd)K(h^2+hd), so temporal coupling is lower order than the HGAT edge processing. Time Complexity of the Link Decoder and Signed Regressor. The Fermi–Dirac decoder and signed weight regressor operate on positive edges and their sampled negatives. For each edge we compute a Poincaré distance dc(zu,zv)d_c(z_u,z_v), a Fermi–Dirac probability, a power law magnitude, and a sign logit from a tangent inner product. Each of these involves a constant number of d dimensional vector operations, so the cost per edge is O(d)O(d). If Ebatch+E^+_batch positive edges are present in the batch and κ negatives are sampled per positive, then Costdec=O(d(1+κ)Ebatch+).Cost_dec=O (d(1+κ)E^+_batch ). (41) In practice Ebatch+E^+_batch is capped (for example at 10410^4 to 2×1042× 10^4), and distance computations are performed in chunks of at most 40 00040\,000 edges to keep buffers bounded. This term is therefore linear in a controlled edge count and does not affect the overall scaling in E and K. Two Phase Optimization Overhead. Each training step consists of a Euclidean pass and a Riemannian refinement pass. In the Euclidean pass we backpropagate through the HGAT stack, temporal kernels and decoders, updating all Euclidean parameters. In the Riemannian pass we update only the node embeddings ziz_i on the Poincaré ball. This involves rescaling gradients by the conformal factor λz2 _z^2, applying RAMSGrad updates in the tangent space, mapping back via the exponential map, and parallel–transporting optimizer moments, all on d dimensional vectors for each node. The cost is therefore CostRiem=O(Nd).Cost_Riem=O(Nd). (42) We reuse the same sampled edges and losses and do not recompute the attention stack or temporal kernels in this phase, so the two phase scheme introduces a constant factor overhead over a single Euclidean training pass with the same architecture while preserving the linear dependence on E and K. Space Complexity. Memory consumption is dominated by current node embeddings, attention buffers and temporal states. Storing node embeddings requires O(Nd)O(Nd) memory. Attention coefficients and intermediate messages are kept for the current batch of edges, which costs O(EH+Ed)O(EH+Ed). The temporal kernels maintain GRU hidden states and pooled summaries of size O(Lh+Kd)O(Lh+Kd). In addition, the decoder holds temporary buffers for at most 40 00040\,000 edges at a time for distance and loss computation, which is bounded by the edge caps discussed above. Crucially, we never store embeddings for all checkpoints in a trace: only the current temporal window and the recurrent state are retained. The overall space complexity is therefore Space=O(Nd+EH+Kh),Space=O (Nd+EH+Kh ), (43) rather than O(TNd)O(TNd) in the length T of the full optimization trace. Scaling Behaviour. Collecting all components, the full time complexity of one training step is O(LHNd2+LHEd)⏟HGAT encoder+O(LK(h2+hd))⏟temporal kernels+O(d(1+κ)Ebatch+)⏟decoder+O(Nd)⏟Riemannian refinement. O\! (LHNd^2+LHEd )_HGAT encoder\;+\; O\! (LK(h^2+hd) )_temporal kernels\;+\; O\! (d(1+κ)E^+_batch )_decoder\;+\; O(Nd)_Riemannian refinement. (44) In our setting, the parameter graphs satisfy E≫NE N and the embedding dimension d is moderate. Under these conditions, the edge term LHEdLHEd dominates the node–dependent terms LHNd2LHNd^2 and NdNd. Moreover, Ebatch+E^+_batch is capped in practice, so the decoder contributes only a controlled linear factor in E. Under these assumptions, the effective leading dependence simplifies to O(LHEd)+O(LK(h2+hd))+O(d(1+κ)Ebatch+),O\! (LHEd )\;+\;O\! (LK(h^2+hd) )\;+\;O\! (d(1+κ)E^+_batch ), (45) which is linear in the number of edges E and in the temporal window length K. Hyperbolic operations and the Riemannian refinement introduce only O(d)O(d) costs per node or edge, adding a modest constant factor relative to a Euclidean graph meta network with the same (L,H,d)(L,H,d) but not altering the asymptotic scaling in model size. Appendix J Ablations Design of the ablations. The full objective can be viewed as a core prediction loss equipped with a set of geometric and temporal regularizers. The core part consists of the Fermi–Dirac link loss ℒFDL_FD, which teaches the model which edges should exist, and the signed regression head ℒwrec+ℒsignL_wrec+L_sign, which aligns the predicted weights with the ground truth values. On top of this, the slope prior ℒαL_α and the ranking loss ℒrankL_rank encourage a monotone relation between hyperbolic distance and weight magnitude, while the temporal smoothness term ℒsmoothL_smooth prevents node embeddings from moving abruptly between consecutive checkpoints (equation 30). In the ablation study we retain ℒFDL_FD so that the model continues to learn meaningful link structure, and we selectively remove geometric terms to examine their individual contribution. Removing ℒrankL_rank tests whether enforcing distance–magnitude ordering matters primarily for interpretability or also for INR accuracy. Removing both ℒrankL_rank and ℒsmoothL_smooth probes the importance of temporal regularity for maintaining a coherent geometry across the optimization trace. Finally, removing the signed regression head (ℒwrecL_wrec, ℒsignL_sign, and ℒαL_α) asks whether explicit supervision on magnitudes and signs is required to bind weights to distance, or whether link prediction alone can sustain the geometric coupling. The following ablations on MNIST and Fashion–MNIST evaluate these questions in detail. We conduct ablations on the INR–trace image classification task because it is the setting in which all components of the composite objective are jointly active: link prediction, signed weight regres- sion, distance–magnitude priors, ranking, and temporal smoothness all contribute to shaping the learned hyperbolic geometry. This task provides long, dense temporal traces (80–100 checkpoints per image), rich variation in weight magnitudes and signs, and a strong supervisory signal, making it the most sensitive and diagnostic environment for assessing how each loss term influences the geometry. Moreover, this is the setting in which our temporal and geometric meta–network attains high classification accuracy, ensuring that ablations probe a regime where the model operates at full capacity rather than in a degraded or low-signal setting. Ablation questions. We design our ablations around three fundamental questions (Table 7): 1. Does the model still produce a meaningful geometry if it is never explicitly told that stronger weights should be closer? (No Ranking Loss) 2. If neither local geometric ordering nor temporal continuity is enforced, does the temporal hyperbolic geometry collapse? (No Ranking and Temporal Smoothness) 3. Can hyperbolic distances encode magnitude and polarity purely from link prediction? (No Sign Magnitude) Each ablation scenario disables a specific subset of loss terms in order to answer exactly one of these questions while keeping the remaining components of the objective intact. Ablations on geometric objectives (MNIST / Fashion-MNIST). We examine four variants of GTH–GMN on the INR classification task in order to isolate the role of each geometric loss. The variants are: • The full objective. • A model without the ranking loss (λrank=0 _rank=0). • A model without both ranking and temporal smoothness (λrank=λsmooth=0 _rank= _smooth=0). • A model without the signed regression head (dropping ℒwrecL_wrec, ℒsignL_sign, and ℒαL_α while keeping the Fermi–Dirac decoder). Table 7: Ablation scenarios and their relation to the three guiding questions. Each row indicates which loss terms are active in the corresponding variant. Variant Question LFDL_FD LwrecL_wrec LsignL_sign LαL_α LrankL_rank LsmoothL_smooth Full objective – ✓ ✓ ✓ ✓ ✓ ✓ No ranking loss (1) ✓ ✓ ✓ ✓ ✗ ✓ No ranking & no smoothness (2) ✓ ✓ ✓ ✓ ✗ ✗ No signed regression head (3) ✓ ✗ ✗ ✗ ✓ ✓ For each condition we report Kendall’s τ between predicted and actual accuracies, and the fitted slope of the distance–log|w| |w| relation (Table 8). Table 8: Ablation Study Table Condition Metric MNIST Value Fashion-MNIST Value Full Objective τ 0.9460.946 0.9350.935 Slope (b) −0.617-0.617 −0.322-0.322 No Ranking Loss τ 0.8110.811 0.7970.797 Slope (b) −0.473-0.473 −0.189-0.189 No Rank/Smooth τ 0.7650.765 0.7640.764 Slope (b) −0.365-0.365 −0.061-0.061 No Signed Head τ 0.7380.738 0.7260.726 Slope (b) −0.211-0.211 +0.125+0.125 Full objective. With all losses active, the model exhibits the strongest alignment between its hyperbolic representations and the underlying predictive signal: Kendall’s τ reaches 0.9460.946 on MNIST and 0.9350.935 on Fashion–MNIST. Geometrically, the distance–weight relation is sharply expressed. The inverse slope is steepest on MNIST (−0.617-0.617) and moderately steep on Fashion–MNIST (−0.322-0.322), reflecting the dataset-specific geometric requirements. The data shows a narrow concentration of points around this trend. Large weights consistently lie close to the origin, weaker weights move outward, and the trajectory across checkpoints remains stable. This is the regime in which the geometry, temporal evolution, and predictive signal remain coherent, forming the reference structure against which the ablations can be interpreted (Figures 6 ,7 ). Figure 6: Kendall τ over the full objective for the INR classification task on MNIST and Fashion-MNIST datasets. GTH-GMN with full objective achieves a Kendall’s τ of 0.9460.946 on MNIST and 0.9350.935 on Fashion–MNIST datasets. Figure 7: Analysis of the relationship between hyperbolic distance (d) and edge weights for the ”full objective” condition. The top row displays results for MNIST, while the bottom row shows Fashion-MNIST. For each dataset, the left image shows the hexbin plot of distance vs. log|w| |w| with a fitted linear regression line (dashed orange). The right image shows the hexbin plot of distance vs. the raw weight magnitude |w||w|. The fitted slope for MNIST in this condition ( −0.617d-0.617d) indicates a strong decay of edge weights, significantly steeper than for Fashion-MNIST (−0.322d-0.322d), reflecting distinct geometric influences on model training for each dataset. Figure 8: Kendall τ over the ”No ranking loss” condition for the INR classification task on MNIST and Fashion-MNIST datasets. GTH-GMN without ℒrankL_rank achieves a Kendall’s τ of 0.8110.811 on MNIST and 0.7970.797 on Fashion–MNIST datasets, indicating a noticeable degradation in the monotonicity of the predictive signal compared to the full objective. Figure 9: Analysis of the relationship between hyperbolic distance (d) and edge weights for the ”No ranking loss” condition. The top row displays results for MNIST, while the bottom row shows Fashion-MNIST. For each dataset, the left image shows the hexbin plot of distance vs. log|w| |w| with a fitted linear regression line (dashed orange). The right image shows the hexbin plot of distance vs. the raw weight magnitude |w||w|. The fitted slope for MNIST in this condition ( −0.473d-0.473d) indicates a moderate decay of edge weights, which is substantially shallower than the full objective model, while the decay for Fashion-MNIST ( −0.189d-0.189d) is notably weaker. This indicates that removing the ranking loss leads to a loss of the crisp, monotone geometric ordering. No ranking loss. Removing ℒrankL_rank has a negative effect on both accuracy and the learned geometry. Kendall’s τ drops to 0.8110.811 in MNIST and 0.7970.797 in Fashion–MNIST, reflecting that the model retains a broad performance gradient, but loses the finer monotonic structure present in the full objective (Figure 8). The fitted slope of the distance–log|w| |w| relation (Figure 9) flattens significantly to −0.473-0.473 on MNIST and −0.189-0.189 on Fashion–MNIST, and the hexbins widen, indicating more frequent local inversions where strong and weak edges intermingle. The geometry remains meaningful, but it no longer enforces a crisp ranking between weight magnitudes. Figure 10: Kendall τ over the ”No ranking and no temporal smoothness” condition for the INR classification task on MNIST and Fashion-MNIST datasets. GTH-GMN without both ℒrankL_rank and ℒsmoothL_smooth achieves a Kendall’s τ of 0.7650.765 on MNIST and 0.7640.764 on Fashion–MNIST datasets, indicating a further decline in the ordering coherence of the predictive signal. Figure 11: Kendall τ over the ”No signed regression head” condition for the INR classification task on MNIST and Fashion-MNIST datasets. GTH-GMN without the signed head achieves a Kendall’s τ of 0.7380.738 on MNIST and 0.7260.726 on Fashion–MNIST datasets, representing the strongest degradation in the geometric ordering of the predictive signal. Figure 12: Analysis of the relationship between hyperbolic distance (d) and edge weights for the ”No ranking and no temporal smoothness loss” condition. The top row displays results for MNIST, while the bottom row shows Fashion-MNIST. For each dataset, the left image shows the hexbin plot of distance vs. log|w| |w| with a fitted linear regression line (dashed orange). The right image shows the hexbin plot of distance vs. the raw weight magnitude |w||w|. The fitted slope for MNIST in this condition (−0.365d-0.365d) indicates a substantially shallower decay of edge weights compared to the full model, while the decay for Fashion-MNIST is minimal ( −0.061d-0.061d), reflecting a loss of coherent geometric structure when both regularization terms are removed. No ranking and no temporal smoothness. When both ℒrankL_rank and ℒsmoothL_smooth are removed, the geometry loses coherence across time as well as within each snapshot. Kendall’s τ declines further to 0.7650.765 on MNIST and 0.7640.764 on Fashion–MNIST (Figure 10), and the distance–log|w| |w| slope becomes much shallower, falling to −0.365-0.365 on MNIST and −0.061-0.061 on Fashion–MNIST. The scatter rises sharply, and the expected ordering “stronger weights lie closer” breaks down in many regions. Without the temporal smoothness term, node embeddings can jump between checkpoints, making the temporal signal harder to follow and reducing the stability of the mapping between weights and their geometric placement. This ablation shows that ranking and smoothness act jointly: one enforces local ordering, the other maintains temporal continuity, and removing both produces a visibly unstable geometric structure (Figure 12). No signed regression head. Removing the signed regression head produces the strongest degradation in the geometry. INR accuracy remains comparatively close to the full model, yet Kendall’s τ drops to 0.7380.738 on MNIST and 0.7260.726 on Fashion–MNIST (Figure 11). The distance–magnitude slopes collapse significantly on MNIST (−0.211-0.211) and even reverse on Fashion–MNIST (+0.125+0.125) (Figure 13), and the hexbins lose almost all discernible structure. Without supervision on magnitudes and signs, the model only learns which edges exist; it no longer learns how strong or influential they are, nor how polarity should be represented in tangent directions. Hyperbolic distances therefore reflect connectivity but not graded strength, and the embedding becomes insensitive to functional variation. This confirms that the signed head is the main mechanism that binds the geometry to weight values rather than merely assisting link prediction. Figure 13: Analysis of the relationship between hyperbolic distance (d) and edge weights for the ”No signed regression head” condition. The top row displays results for MNIST, while the bottom row shows Fashion-MNIST. For each dataset, the left image shows the hexbin plot of distance vs. log|w| |w| with a fitted linear regression line (dashed orange). The right image shows the hexbin plot of distance vs. the raw weight magnitude |w||w|. The fitted slope for MNIST in this condition ( −0.211d-0.211d) is near-zero, and the slope for Fashion-MNIST (+0.125d+0.125d) is positive, indicating that the expected decay of weights with distance has collapsed or reversed. This confirms that the signed head is the primary mechanism binding the hyperbolic geometry to the functional magnitude of the weights. Taken together, these ablations show that the ranking and smoothness losses preserve the temporal and geometric coherence of the embeddings, while the signed head is essential for encoding magnitude–distance coupling. The INR classification task remains robust in several ablations, yet the auxiliary metrics and geometric plots reveal a steady erosion of the hyperbolic structure that supports our explanatory interpretations when each constraint is removed. Appendix K Future Works Our approach of geometrically representing the temporal dynamics of deep neural networks naturally extends theoretically beyond MLPs because any neural architecture can be represented as a parameter graph (Lim et al., 2023). For CNNs, convolutional kernels give structured edge sets between input and output channel nodes forming a multi-graph, with kernel position and related structure stored as edge features. For Transformers, attention layers induce subgraphs that connect query, key, value, and output channels, with head and layer indices and normalization parameters encoded as node and edge features (Lim et al., 2023). The deeper layers form the MLP parameter graphs and can be modeled as stated in this work for both CNNs and Transformers. Since the hyperbolic encoder operates only on node and edge structure, these architectures require no modification to the core method: their parameters simply instantiate richer graph topologies that the same temporal geometric machinery can process. Apart from that, recent work has shown that optimization trajectories admit coarse equivalence classes through topological conjugacy and Koopman spectral analysis (Redman et al., 2024). These dynamical characterizations complement our hyperbolic parameter graph approach. A promising future direction is to incorporate such global dynamical information into the temporal update module or the final task-specific readout, allowing these signals to enrich the geometric structure captured by GTH-GMN through a broader global context.