Paper deep dive
Geometric Mixture-of-Experts with Curvature-Guided Adaptive Routing for Graph Representation Learning
Haifang Cao, Yu Wang, Timing Li, Xinjie Yao, Pengfei Zhu
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/26/2026, 1:29:26 AM
Summary
The paper introduces GeoMoE, a Geometric Mixture-of-Experts framework for graph representation learning that uses Ollivier-Ricci Curvature (ORC) as an explicit geometric prior. GeoMoE adaptively routes node representations across Euclidean, hyperbolic, and spherical experts using a graph-aware gating network, regularized by curvature-guided alignment and contrastive objectives to ensure geometry-consistent and interpretable node-wise adaptation.
Entities (4)
Relation Signals (3)
GeoMoE → utilizes → Ollivier-Ricci Curvature
confidence 100% · GeoMoE leverages Ollivier-Ricci Curvature (ORC) as an intrinsic geometric prior
Ollivier-Ricci Curvature → guides → Expert Routing
confidence 95% · we transform expert routing into a node-adaptive, geometry-consistent, and interpretable mechanism.
GeoMoE → outperforms → GraphMoRE
confidence 90% · Extensive experiments on six benchmark datasets demonstrate that GeoMoE outperforms state-of-the-art baselines
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Graph-structured data typically exhibits complex topological heterogeneity, making it difficult to model accurately within a single Riemannian manifold. While emerging mixed-curvature methods attempt to capture such diversity, they often rely on implicit, task-driven routing that lacks fundamental geometric grounding. To address this challenge, we propose a Geometric Mixture-of-Experts framework (GeoMoE) that adaptively fuses node representations across diverse Riemannian spaces to better accommodate multi-scale topological structures. At its core, GeoMoE leverages Ollivier-Ricci Curvature (ORC) as an intrinsic geometric prior to orchestrate the collaboration of specialized experts. Specifically, we design a graph-aware gating network that assigns node-specific fusion weights, regularized by a curvature-guided alignment loss to ensure interpretable and geometry-consistent routing. Additionally, we introduce a curvature-aware contrastive objective that promotes geometric discriminability by constructing positive and negative pairs according to curvature consistency. Extensive experiments on six benchmark datasets demonstrate that GeoMoE outperforms state-of-the-art baselines across diverse graph types.
Tags
Links
- Source: https://arxiv.org/abs/2603.22317v1
- Canonical: https://arxiv.org/abs/2603.22317v1
Trouble viewing inline? Open PDF directly →
Full Text
91,082 characters extracted from source content.
Expand or collapse full text
Geometric Mixture-of-Experts with Curvature-Guided Adaptive Routing for Graph Representation Learning Haifang Cao Yu Wang Timing Li Xinjie Yao Pengfei Zhu Abstract Graph-structured data typically exhibits complex topological heterogeneity, making it difficult to model accurately within a single Riemannian manifold. While emerging mixed-curvature methods attempt to capture such diversity, they often rely on implicit, task-driven routing that lacks fundamental geometric grounding. To address this challenge, we propose a Geometric Mixture-of-Experts framework (GeoMoE) that adaptively fuses node representations across diverse Riemannian spaces to better accommodate multi-scale topological structures. At its core, GeoMoE leverages Ollivier-Ricci Curvature (ORC) as an intrinsic geometric prior to orchestrate the collaboration of specialized experts. Specifically, we design a graph-aware gating network that assigns node-specific fusion weights, regularized by a curvature-guided alignment loss to ensure interpretable and geometry-consistent routing. Additionally, we introduce a curvature-aware contrastive objective that promotes geometric discriminability by constructing positive and negative pairs according to curvature consistency. Extensive experiments on six benchmark datasets demonstrate that GeoMoE outperforms state-of-the-art baselines across diverse graph types. Machine Learning, ICML Figure 1: (Left) Topological visualization of real-world graphs (IV), combining grid (I), hierarchy (I), and cycle (I) structures. (Right) Performance of single-geometry models vs. our GeoMoE across these structures. 1 Introduction Graph representation learning has become a fundamental tool for analyzing relational data in domains ranging from bioinformatics to social networks (Atz et al., 2021; Khoshraftar and An, 2024; Chen et al., 2020). A critical challenge lies in the profound geometric heterogeneity of real-world graphs. As visualized in Figure 1, local structures are rarely uniform, exhibiting a complex mixture of hierarchical expansions (hyperbolic), tight cycles (spherical), and grids (Euclidean) (Bronstein et al., 2017; Defferrard et al., 2020). This mismatch between the homogeneous geometric prior assumed by most models and the heterogeneous reality of data inevitably distorts vital structural information and creates a significant bottleneck for representation capacity. However, most existing graph representation learning methods are built on a single geometric prior, predominantly Euclidean (Kipf, 2016; Veličković et al., 2017). Even recent extensions to non-Euclidean settings typically assume a globally shared geometry, such as a fixed hyperbolic or spherical manifold (Chami et al., 2019; Liu et al., 2019; Zhang et al., 2021a). Although these approaches effectively capture certain structural patterns, they implicitly impose a uniform geometric bias over the entire graph. As a result, they struggle to faithfully represent graphs whose local geometry diverges from the chosen global space, leading to suboptimal expressiveness and limited adaptability, as visually demonstrated in Figure 1 where single-geometry models only excel on their native substructures. To capture such geometric heterogeneity, a promising direction is to enable graph representations to jointly exploit multiple geometric spaces. Recent works have explored this via mixed-curvature representation learning (Cao et al., 2025; Sun et al., 2022; Yang et al., 2022; Sun et al., 2024). However, these methods still blend geometries at a global level, applying the same geometric configuration to all nodes, thus failing to achieve fine-grained node-wise adaptation. GraphMoRE (Guo et al., 2025) introduces a Mixture of Riemannian Experts framework with topology-aware gating for node-specific expert routing. However, its gating mechanism relies on implicit distortion minimization, lacking supervision from a fundamental geometric prior. This absence of inductive bias results in a routing strategy that is geometrically opaque and prone to expert collapse—where data-driven optimization may gravitate towards trivial solutions rather than adhering to the graph’s intrinsic topology. To bridge these gaps, we propose GeoMoE, a novel geometric mixture-of-experts framework that effectively injects an explicit geometric prior into node-wise expert fusion. The key idea is to utilize Ollivier-Ricci curvature (ORC) (Ollivier, 2009) as a principled descriptor of local graph geometry, accurately capturing neighborhood expansion or contraction and thus indicating whether a region behaves more Euclidean, hyperbolic, or spherical. By treating ORC as an explicit geometric supervision signal, GeoMoE moves beyond purely task-driven routing and enforces geometry-consistent adaptation, aiming to improve both interpretability and robustness under locally varying geometric regimes. Specifically, GeoMoE orchestrates three specialized experts (Euclidean, hyperbolic, spherical) via a graph-aware gating network. Crucially, we introduce a gating alignment loss that regularizes the gating distribution toward an ORC-derived target, making expert preferences explicitly consistent with intrinsic discrete geometry. To further promote geometric discriminability, we design a curvature-aware contrastive objective that aligns embeddings with their curvature-matched experts, while contrasting them against mismatched experts and inter-node hard negatives, thereby disentangling geometrically distinct structures. Our main contributions are summarized as follows: • We propose GeoMoE, a novel geometric mixture-of-experts framework. By integrating Ollivier-Ricci Curvature as an explicit and informative geometric prior, we transform expert routing into a node-adaptive, geometry-consistent, and interpretable mechanism. • We design a tailored optimization scheme comprising a curvature-guided gating alignment and a curvature-aware contrastive objective. This strategy regularizes the routing logic, ensuring precise topological alignment and enhanced geometric discriminability. • We establish formal theoretical guarantees—including a synergy bound for expert fusion, gating-curvature monotonic consistency, and a mutual information lower bound for contrastive learning—with extensive experiments confirming GeoMoE’s state-of-the-art performance and corroborating our theoretical insights. 2 Related work 2.1 Ricci Curvature-Driven Graph Learning Discrete Ricci curvature—predominantly Ollivier–Ricci curvature(ORC) and Forman–Ricci curvature(FRC)—serves as a principled descriptor of local graph geometry (Lin et al., 2011; Ollivier, 2009; Forman, 2003). A major line of research leverages these signals to mitigate structural bottlenecks, such as over-squashing and over-smoothing, via curvature-guided rewiring, sparsification, or Ricci flow-based topology refinement (Sun et al., 2023a; Nguyen et al., 2023; Zhang et al., 2023; Feng and Weber, 2024). Beyond topology refinement, recent advances increasingly integrate curvature as a geometric prior within representation learning. This includes guiding adaptive embedding spaces (Sun et al., 2023b; Zhang et al., 2023) and enhancing domain-specific modeling in biomolecular interaction prediction (Shen et al., 2024) and knowledge graphs (Luo et al., 2025). Most recently, the paradigm expands into information-theoretic frameworks (Fu et al., 2025) and Spectro-Riemannian architectures (Grover et al., 2025), bridging the gap between discrete geometry and continuous representation theory. Despite substantial progress, most curvature-driven methods treat Ricci curvature as an auxiliary heuristic rather than a fundamental guiding principle for representation formation. Moreover, they lack mechanisms for fine-grained node-wise adaptation and explicit geometric discriminability. 2.2 Mixture-of-Experts in Graph Learning The Mixture-of-Experts (MoE) (Shazeer et al., 2017) paradigm has recently been adapted to graph learning to address structural and semantic heterogeneity by partitioning computation across specialized subnetworks (Hu et al., 2021). Early explorations introduced MoE for graph-level classification and node representation, aiming to capture diverse structural motifs through expert diversity (Errica et al., 2021; Wang et al., 2023). This direction has evolved into domain-specific expert designs, such as topology-aware modules for molecular property prediction (Kim et al., 2023; Shirasuna et al., 2024), link predictors with heterogeneous capacity (Ma et al., 2024), and generalizable high-level synthesis modeling (Li et al., 2025). A more recent and impactful trend focuses on fine-grained node-wise routing, where individual nodes are dynamically assigned to the most suitable experts based on local context—e.g., via adaptive filtering (Han et al., 2024), memory-augmented routers (Huang et al., 2025), or multi-view alignment (Wu et al., 2025). Notably, GraphMoRE (Guo et al., 2025) extends this paradigm to non-Euclidean settings, introducing a Mixture of Riemannian Experts to mitigate complex geometric heterogeneity through topology-aware manifold routing. In spite of these advancements, existing routing mechanisms remain largely implicit, relying on task-driven heuristics that lack explicit alignment with the graph’s intrinsic geometry. GeoMoE addresses these limitations by directly internalizing Ollivier-Ricci curvature as a principled supervisory signal. By explicitly guiding expert assignment, GeoMoE ensures that each node is routed to the geometric expert that faithfully reflects its local geometric signature. 3 Preliminaries This section provides the theoretical foundation and mathematical framework necessary for our proposed GeoMoE model. We first establish a consistent notation system and define the Riemannian model spaces utilized by our geometric experts. We then present a rigorous formulation of ORC, which serves as the foundational guiding signal for our gating mechanism and representation formation. 3.1 Notations A graph is formally defined as =(,ℰ)G=(V,E), where =v1,v2,…,vNV=\v_1,v_2,…,v_N\ is the set of N nodes and ℰ⊆×E ×V denotes the set of m edges. The connectivity of G is characterized by the adjacency matrix ∈0,1N×NA∈\0,1\^N× N, where Aij=1A_ij=1 if (vi,vj)∈ℰ(v_i,v_j) and 0 otherwise. Each node viv_i is associated with a d-dimensional feature vector i∈ℝdx_i ^d, forming the node feature matrix ∈ℝN×dX ^N× d. The neighborhood of a node viv_i is denoted by (vi)=vj∣(vi,vj)∈ℰN(v_i)=\v_j (v_i,v_j) \. In geometric representation learning, node embeddings are modeled in Riemannian manifolds to reflect the heterogeneous geometric nature of graph structures. We consider three canonical Riemannian manifolds of constant sectional curvature κ∈ℝκ : the Euclidean space ℝdR^d (κ=0κ=0); the hyperbolic space ℍκdH_κ^d (κ<0κ<0), modeled via the Poincaré ball ∈ℝd:‖<1/−κ\x ^d:\|x\|<1/ -κ\; the spherical space κdS_κ^d (κ>0κ>0), embedded in ℝd+1R^d+1 as ∈ℝd+1:‖=1/κ\x ^d+1:\|x\|=1/ κ\. For a Riemannian manifold ℳκM_κ, we denote its exponential map at point o (typically the origin) as exp:Tℳκ→ℳκ _o:T_oM_κ _κ, and its inverse logarithmic map as log:ℳκ→Tℳκ _o:M_κ→ T_oM_κ, where TℳκT_oM_κ is the associated tangent space at o. The geodesic distance and Riemannian inner product on ℳκM_κ are denoted by dκ(⋅,⋅)d_κ(·,·) and ⟨⋅,⋅⟩κ ·,· _κ, respectively. 3.2 Ollivier–Ricci Curvature Ollivier-Ricci curvature (ORC)(Ollivier, 2009) is a discrete curvature measure grounded in optimal transport theory, quantifying local graph geometry via Wasserstein distance between neighborhood probability measures. We leverage ORC as a geometric prior to guide node-wise expert routing. Definition 3.1 (Edge-level Ollivier-Ricci Curvature). (Ache and Warren, 2019; Yang et al., 2023) Let =(,ℰ)G=(V,E) be a locally finite, connected, and simple graph. For any two distinct nodes vi,vj∈v_i,v_j connected by an edge, their edge-level Ollivier-Ricci curvature κ(vi,vj)κ(v_i,v_j) is defined as: κ(vi,vj)=1−W1(μvi,μvj)d(vi,vj),κ(v_i,v_j)=1- W_1( _v_i, _v_j)d(v_i,v_j), (1) where d(vi,vj)d(v_i,v_j) denotes the shortest-path distance between viv_i and vjv_j on G, and W1(μvi,μvj)W_1( _v_i, _v_j) is the first-order Wasserstein distance (Definition A.1) measuring the discrepancy between two probability measures μvi _v_i and μvj _v_j. Definition 3.2 (Node-level Ollivier-Ricci Curvature). (Ache and Warren, 2019; Yang et al., 2023) The node-level Ollivier-Ricci curvature κ(vi)κ(v_i) for a node vi∈v_i is defined as the average of edge curvatures over its incident edges: κ(vi)=1|(vi)|∑vj∈(vi)κ(vi,vj),κ(v_i)= 1|N(v_i)| _v_j (v_i)κ(v_i,v_j), (2) where κ(vi,vj)κ(v_i,v_j) is given by Equation 1. This scalar summary captures the local geometric signature of viv_i, providing a quantitative basis for the subsequent expert routing. 4 Methodology Our GeoMoE consists of three synergistic components: (i) a tripartite bank of geometric experts for Euclidean, hyperbolic, and spherical spaces; (i) a curvature-guided gating alignment mechanism to synchronize routing decisions with intrinsic geometry via a graph-aware router and a gate alignment loss; and (i) a curvature-aware contrastive learning objective to refine the discriminative power of the fused representations. By integrating ORC as a principled prior, we ensure that the gating mechanism is structurally grounded and the resulting embeddings maintain high geometric fidelity. The overall architecture is illustrated in Figure 2. Figure 2: An overview of GeoMoE. We first encode the input graph via Euclidean, hyperbolic, and spherical geometric experts, then dynamically fuse their outputs through a curvature-guided gating alignment mechanism. The curvature-aware contrastive learning objective further refines the fused representations to enhance geometric discriminability. 4.1 Geometric Mixture-of-Experts Local structures in real-world graphs exhibit profound geometric heterogeneity, with flat, hierarchical, and compact topologies coexisting. To address this, we propose a tri-expert framework that encodes the graph across Euclidean, hyperbolic, and spherical spaces, followed by a dynamic fusion mechanism to effectively reconcile these disparate geometric signatures into a comprehensive representation. Geometric experts. Each expert is tailored to the intrinsic properties of its manifold. Specifically, we instantiate three parallel GNN experts to process signals within their respective optimal geometries: a Euclidean expert for flat patterns, a hyperbolic expert (Poincaré ball) for hierarchies, and a spherical expert for cyclical motifs. By partitioning the representational task across these specialized encoders, GeoMoE ensures that each structural signature is processed within a space of optimal inductive bias. Formally, the encoding process for each expert m∈E,H,Sm∈\E,H,S\ is: m=GNNm(,),h^m=GNN^m(X,A), (3) where GNNmGNN^m denotes the manifold-specific graph neural networks and mh^m represents the resulting node embeddings in the corresponding underlying geometric space. Dynamic feature fusion. To achieve effective integration, we employ a gating-weighted summation strategy. For each node viv_i, embeddings from non-Euclidean manifolds are projected into a unified Euclidean tangent space at the origin o via logarithmic maps for feature alignment, and fused as: fused(vi) ^fused(v_i) =ωiE⋅E(vi) = _i^E·h^E(v_i) (4) +ωiH⋅log(H(vi))+ωiS⋅log(S(vi)), + _i^H· _o(h^H(v_i))+ _i^S· _o(h^S(v_i)), where ωiE,ωiH,ωiS _i^E, _i^H, _i^S are normalized gating weights such that ∑m∈E,H,Sωim=1 _m∈\E,H,S\ _i^m=1. Notably, these weights are not learned in a black-box fashion. Instead, their distribution is explicitly regularized by the curvature-guided alignment and contrastive learning objectives detailed in Section 4.2 and Section 4.3. This ensures that the expert assignment is physically grounded in the local geometry of the graph. Beyond empirical intuition, this synergistic integration aims to overcome the fundamental representation bottlenecks of single-manifold models. By adaptively routing nodes to their curvature-optimal geometric spaces, GeoMoE theoretically expands the expressive capacity of the latent representation compared to any fixed-curvature baseline. To formally justify this, we establish Theorem 4.1 on the synergy bound of our geometric mixture-of-experts architecture. Theorem 4.1 (Synergy Bound of Geometric Mixture-of-Experts). Let ℓm(vi)=ℓ(m(vi),y(vi)) ^m(v_i)= (h^m(v_i),y(v_i)) be the loss of expert m∈E,H,Sm∈\E,H,S\ for node viv_i, and let fused(vi)h^fused(v_i) be the fused representation defined in Equation 4 with gating weights i ω_i. Assume the loss function ℓ(⋅,y) (·,y) is L-Lipschitz continuous. Suppose each expert has an advantage region Ωm⊂ _m with positive probability measure μ(Ωm)>0μ( _m)>0, such that: ∀vi∈Ωm,ℓm(vi)≤ℓn(vi)−ϵ,∀n≠m.∀ v_i∈ _m,\ ^m(v_i)≤ ^n(v_i)-ε,\ ∀ n≠ m. Let eme^m be the corresponding one-hot vector for expert m. If the gating error satisfies [‖i−em‖1∣vi∈Ωm]≤δE[\| ω_i-e^m\|_1 v_i∈ _m]≤δ for a sufficiently small positive constant δ>0δ>0, then: [ℓ(fused,y)]<minm∈E,H,S[ℓm].E[ (h^fused,y)]< _m∈\E,H,S\E[ ^m]. Proof. See Appendix B.1 for the complete proof. ∎ 4.2 Curvature-Guided Gating Alignment Leveraging the property that ORC κ(v)κ(v) characterizes local topology—where positive, negative, and near-zero values indicate spherical, hyperbolic, and Euclidean topologies, respectively—we design an ORC-guided gating alignment mechanism to ensure that expert assignments are physically grounded in the local structural attributes of nodes. Graph-aware gating network. Traditional gating mechanisms often rely solely on node features, ignoring the underlying topological structure. This can lead to suboptimal routing where nodes with similar features but distinct neighborhood structures receive identical weights. To address this, we adopt a graph-aware gating network that integrates local topological information through neighborhood aggregation. First, a graph convolutional layer is employed to fuse the raw node features with the surrounding structural context: agg=GCNgate(,),x_agg=GCN_gate(X,A), (5) where X and A are the input feature and adjacency matrices. Then, the expert weights ω are generated via a multi-layer perceptron with a temperature-controlled Softmax: =Softmax(MLP(agg)τg), ω=Softmax ( MLP(x_agg) _g ), (6) where =[ωE,ωH,ωS]⊤ ω=[ω^E,ω^H,ω^S] represents the weights for the Euclidean, hyperbolic, and spherical experts, and τg _g is the temperature controlling the distribution sharpness. Gating alignment loss. To enforce consistency between learned weights and the geometric prior κ(v)κ(v), we introduce an alignment loss based on Kullback-Leibler (KL) divergence. We first construct the geometric target weights ∗=[ω∗E,ω∗H,ω∗S]⊤ ω^*=[ω^*E,ω^*H,ω^*S] as soft labels derived from κ(v)κ(v): ω∗E=σ(θ−|κ(v)|η),ω∗H=σ(−κ(v)−θη),ω∗S=σ(κ(v)−θη), casesω^*E=σ ( θ-|κ(v)|η ),\\ ω^*H=σ ( -κ(v)-θη ),\\ ω^*S=σ ( κ(v)-θη ), cases (7) where θ>0θ>0 is the curvature threshold for region partitioning, η>0η>0 is a smoothing factor, and σ(⋅)σ(·) is the Sigmoid function. These components quantify target expertise intensities for the Euclidean (near-zero curvature), hyperbolic (negative curvature), and spherical (positive curvature) domains, respectively. After normalizing the targets such that ∗=∗/∑mω∗m ω^*= ω^*/ _mω^*m, the alignment loss is defined as: ℒalign=KL(∗∥).L_align=KL( ω^* ω). (8) Theoretically, minimizing the alignment loss ℒalignL_align enforces a strict monotonic consistency between the gating weights and the node-level ORC values, which guarantees the structural rationality of the expert allocation. We formally establish this pivotal property through the following theorem. Theorem 4.2 (Monotonic Consistency Between Gating Weights and ORC). Let κ(v)κ(v) be the node-level ORC of node v. Under the alignment loss ℒalignL_align with target weights defined in Equation 7, the learned gating weights at convergence (v)=[ωE(v),ωH(v),ωS(v)]⊤ ω(v)=[ω^E(v),ω^H(v),ω^S(v)] satisfy the following regional monotonicity properties: (i) Spherical Domain:On Ω+=v∈∣κ(v)≥θ _+=\v κ(v)≥θ\, the spherical weight ωSω^S is strictly increasing with respect to κ(v)κ(v), i.e., ∂ωS(v)∂κ(v)>0 ∂ω^S(v)∂κ(v)>0; (i) Hyperbolic Domain:On Ω−=v∈∣κ(v)≤−θ _-=\v κ(v)≤-θ\, the hyperbolic weight ωHω^H is strictly decreasing with respect to κ(v)κ(v), i.e., ∂ωH(v)∂κ(v)<0 ∂ω^H(v)∂κ(v)<0; (i) Euclidean Domain:On Ω0=v∈∣|κ(v)|≤θ _0=\v |κ(v)|≤θ\, the Euclidean weight ωEω^E is strictly decreasing with respect to the curvature magnitude |κ(v)||κ(v)|, i.e., ∂ωE(v)∂|κ(v)|<0 ∂ω^E(v)∂|κ(v)|<0. Proof. See Appendix B.2 for the complete proof. ∎ 4.3 Curvature-Aware Contrastive Learning While curvature-guided alignment ensures geometric consistency, it may not guarantee the discriminative power. To address this, we propose a curvature-aware contrastive objective, employing ORC values as geometric priors to assign positive samples to the most compatible geometric experts and construct informative negative sets. This enhances the discriminability and robustness of the fused representations. Positive sample selection. Based on the geometric semantics of ORC values κ(v)κ(v), we define an ORC-driven hard selection mechanism to identify the optimal expert representation as the positive sample pos(v)h^pos(v). This systematic selection logic remains strictly consistent with the ideal weighting scheme previously defined in Section 4.2: pos(v)=E(v),if |κ(v)|≤θ;H(v),if κ(v)<−θ;S(v),if κ(v)>θ.h^pos(v)= casesh^E(v),&if |κ(v)|≤θ;\\ h^H(v),&if κ(v)<-θ;\\ h^S(v),&if κ(v)>θ. cases (9) Here, θ>0θ>0 denotes the identical curvature threshold used for region partitioning as in Section 4.2. This formulation explicitly ensures that the positive samples are highly congruent with the primary gating alignment objectives. Dual-strategy negative construction. To further sharpen the decision boundaries, we design a dual-strategy for constructing negative sets, comprising intra-node and inter-node negative samples. For intra-node negatives, we select the non-optimal expert representations of the current node v, which encourages the model to differentiate between distinct geometric manifolds. Simultaneously, we identify inter-node hard negatives by mining other nodes that exhibit high similarity to pos(v)h^pos(v) but possess distinct geometric properties. Specifically, we compute the cosine similarity between the positive embedding pos(v)h^pos(v) and the fused representations fused(u)h^fused(u) of all nodes u∈∖vu \v\, selecting the top-(K−2)(K-2) most similar embeddings as hard negatives. Contrastive loss. To formalize the training objective, we define sv,pos=sim(fused(v),pos(v))s_v,pos=sim(h^fused(v),h^pos(v)) and sv,j=sim(fused(v),fused(vj))s_v,j=sim(h^fused(v),h^fused(v_j)) as the cosine similarity scores for the positive pair and the j-th negative pair, respectively. Based on the standard InfoNCE framework (Oord et al., 2018), the contrastive loss is mathematically formulated as: ℒcontr=−1N∑v∈logesv,pos/τcesv,pos/τc+∑j∈−(v)esv,j/τc,L_contr=- 1N _v e^s_v,pos/ _ce^s_v,pos/ _c+ _j ^-(v)e^s_v,j/ _c, (10) where −(v)N^-(v) denotes the set of negative samples of node v, and τc _c is the contrastive temperature parameter. Curvature-aware contrastive learning maximizes the mutual information between node representations and their underlying geometric signatures. To characterize this information-theoretic guarantee, we establish the following theorem. Theorem 4.3 (Geometry-Consistent Mutual Information Maximization). Following the InfoNCE principle (Oord et al., 2018; Poole et al., 2019), let fusedh^fused and ∗h^* denote the random variables for the fused representation and the geometric prior embedding (derived from the ORC-guided positive selection), respectively. For a batch containing one positive sample and K negative samples drawn independently from the marginal distribution p(∗)p(h^*), the contrastive loss ℒcontrL_contr serves as a lower bound estimator: I(fused;∗)≥log(K+1)−ℒcontr.I(h^fused;h^*)≥ (K+1)-L_contr. (11) Consequently, minimizing ℒcontrL_contr maximizes the variational lower bound of the mutual information between the node representations and their curvature-driven geometric priors, thereby effectively promoting geometric consistency and mitigating representational collapse in the latent space. Proof. See Appendix B.3 for the detailed derivation. ∎ 4.4 Training Objective The total loss of the model is formulated as a weighted linear combination of the supervised loss, the gating alignment loss, and the contrastive loss. These terms directly correspond to the three core optimization objectives: downstream task fitting, geometric consistency constraints, and enhancement of geometric discriminability, respectively: ℒtotal=αℒtask+βℒalign+γℒcontr,L_total= _task+ _align+ _contr, (12) where ℒtaskL_task denotes the task-specific supervised loss (e.g., cross-entropy), and the non-negative hyperparameters α,β,γα,β,γ govern the trade-off between the predictive accuracy and the geometric regularization terms. We provide the detailed training algorithm in Algorithm 1 of the Appendix. 4.5 Complexity and Scalability Analysis A primary bottleneck in curvature-driven learning is the computational cost. To address this, GeoMoE adopts a strategy that decouples geometric characterization from model optimization. Specifically, ORC computation is treated as an offline pre-processing step. While exact solvers are employed for benchmark datasets, our framework can seamlessly integrate with near linear-time approximations (Ni et al., 2019) for scalability. By utilizing neighborhood overlap measures, the pre-computation complexity reduces from (|ℰ|Δ3logΔ)O(|E| ^3 )—where |ℰ||E| is the edge count and Δ is the maximum degree—to a manageable (|ℰ|Δ)O(|E| ), which is computationally efficient for sparse graphs. Crucially, during the online phase, the curvature-guided gating incurs only (1)O(1) overhead per node, maintaining a training complexity of (|ℰ|)O(|E|) asymptotically equivalent to standard GCNs. This ensures GeoMoE is theoretically scalable to large-scale graphs. Further derivations are detailed in Appendix D. Table 1: Accuracy (%±\%± standard deviation) for Cora, CiteSeer, PubMed, Photo and WikiCS and F1 score (%±\%± standard deviation) for Airport for node classification. Scores are averaged over ten runs. The best results are boldfaced while the runner-ups are underlined. Method Cora CiteSeer PubMed Photo WikiCS Airport GCN (Kipf, 2016) 82.02±0.4582.02_± 0.45 70.24±0.8570.24_± 0.85 78.45±0.3678.45_± 0.36 92.84±0.2592.84_± 0.25 74.29±1.0674.29_± 1.06 83.19±1.7683.19_± 1.76 GAT (Veličković et al., 2017) 82.72±0.6982.72_± 0.69 71.45±1.1571.45_± 1.15 77.23±0.5177.23_± 0.51 92.70±0.5592.70_± 0.55 76.55±0.8876.55_± 0.88 86.10±1.3986.10_± 1.39 SGC (Wu et al., 2019) 81.91±0.6381.91_± 0.63 71.95±0.5571.95_± 0.55 78.60±0.2478.60_± 0.24 91.96±1.2391.96_± 1.23 75.54±1.2575.54_± 1.25 84.07±1.9184.07_± 1.91 GraphCON (Rusch et al., 2022) 82.54±1.2582.54_± 1.25 71.29±0.9671.29_± 0.96 78.66±1.2378.66_± 1.23 91.94±1.1291.94_± 1.12 76.64±0.5976.64_± 0.59 79.25±2.5179.25_± 2.51 NodeFormer (Wu et al., 2022) 82.34±0.8582.34_± 0.85 72.47±1.2772.47_± 1.27 79.13±1.1079.13_± 1.10 93.46±0.6293.46_± 0.62 73.27±0.8773.27_± 0.87 82.66±0.6982.66_± 0.69 ACMP (Wang et al., 2022) 82.65±0.7882.65_± 0.78 73.01±1.1473.01_± 1.14 79.12±0.5479.12_± 0.54 92.24±1.7192.24_± 1.71 77.12±0.5877.12_± 0.58 88.96±1.7488.96_± 1.74 SGFormer (Wu et al., 2023) 83.01±0.7583.01_± 0.75 72.35±0.5472.35_± 0.54 79.25¯±0.65 79.25_± 0.65 90.74±1.5190.74_± 1.51 74.72±0.7674.72_± 0.76 91.67±1.2191.67_± 1.21 RNCGLN (Zhu et al., 2024) 82.58±0.6682.58_± 0.66 73.60¯±0.68 73.60_± 0.68 78.64±0.8778.64_± 0.87 91.71±0.2891.71_± 0.28 77.65±0.8577.65_± 0.85 90.94±0.7790.94_± 0.77 DGMAE (Zheng et al., 2025) 84.24¯±0.58 84.24_± 0.58 72.91±0.3872.91_± 0.38 79.21±0.2879.21_± 0.28 93.06±0.1393.06_± 0.13 78.81¯±0.34 78.81_± 0.34 90.58±0.9590.58_± 0.95 HGNN (Liu et al., 2019) 78.99±0.6178.99_± 0.61 70.20±0.6170.20_± 0.61 76.94±1.1276.94_± 1.12 91.82±0.3591.82_± 0.35 74.89±0.4274.89_± 0.42 84.87±2.0984.87_± 2.09 HGCN (Chami et al., 2019) 78.41±0.7778.41_± 0.77 68.29±0.9568.29_± 0.95 77.03±0.4977.03_± 0.49 93.19±0.2993.19_± 0.29 76.67±0.4476.67_± 0.44 88.82±1.6688.82_± 1.66 GIL (Zhu et al., 2020) 82.49±0.7782.49_± 0.77 71.21±0.9371.21_± 0.93 77.35±0.5677.35_± 0.56 93.92±0.1893.92_± 0.18 77.78±0.8577.78_± 0.85 91.21±1.7291.21_± 1.72 HGAT (Zhang et al., 2021a) 79.71±0.9579.71_± 0.95 69.41±0.7869.41_± 0.78 75.56±0.6775.56_± 0.67 93.38±0.4593.38_± 0.45 77.62±0.4977.62_± 0.49 89.25±0.9689.25_± 0.96 LGCN (Zhang et al., 2021b) 78.93±0.7978.93_± 0.79 68.59±0.6468.59_± 0.64 78.08±0.6578.08_± 0.65 94.21±0.4394.21_± 0.43 76.58±0.6076.58_± 0.60 88.53±1.2688.53_± 1.26 F-HNN (Chen et al., 2022) 81.04±0.7881.04_± 0.78 71.12±0.6671.12_± 0.66 77.66±1.0477.66_± 1.04 94.55±0.5294.55_± 0.52 78.07±0.5178.07_± 0.51 91.85±0.8691.85_± 0.86 MotifRGC (Sun et al., 2024) 81.65±1.0381.65_± 1.03 72.56±1.2172.56_± 1.21 78.33±0.8578.33_± 0.85 93.78±0.9393.78_± 0.93 78.12±0.7278.12_± 0.72 92.31±1.8592.31_± 1.85 LRN (He et al., 2025) 78.34±1.1678.34_± 1.16 67.10±1.1967.10_± 1.19 77.24±0.6777.24_± 0.67 94.58¯±0.39 94.58_± 0.39 77.98±0.4277.98_± 0.42 92.39±0.7292.39_± 0.72 GraphMoRE (Guo et al., 2025) 82.85±0.9782.85_± 0.97 71.64±0.8571.64_± 0.85 78.65±0.6678.65_± 0.66 94.07±1.0594.07_± 1.05 78.27±0.6878.27_± 0.68 92.76¯±0.94 92.76_± 0.94 GeoMoE(ours) 84.75±0.7884.75_± 0.78 74.72±0.6274.72_± 0.62 80.13±0.5880.13_± 0.58 95.18±1.0495.18_± 1.04 79.65±0.5379.65_± 0.53 93.56±1.2193.56_± 1.21 5 Experiments 5.1 Experimental Settings Datasets. We evaluate GeoMoE on six benchmark datasets with diverse structural topologies: three citation networks (Cora, CiteSeer, PubMed)(Sen et al., 2008; Namata et al., 2012), a co-purchase graph (Photo)(Shchur et al., 2018), a reference graph (WikiCS)(Mernyei and Cangea, 2020), and a flight network (Airport)(Chami et al., 2019). These datasets collectively span a wide range of graph geometries, with detailed statistics provided in Table 3 of the Appendix E.1. Baselines. We compare our method against a wide range of state-of-the-art baselines, which can be broadly categorized into two groups based on the underlying geometric spaces: (1) Euclidean-based methods, which model graph representations exclusively in flat Euclidean space; (2) Riemannian methods, which operate in curved Riemannian manifolds (hyperbolic, spherical, or mixed-curvature spaces) to capture latent hierarchical, cyclic, or heterogeneous structures. Implementation Details. We adopt standard splits for three citation networks, official splits for WikiCS, and random splits for Photo (60/20/20%) and Airport (70/15/15%). For a fair comparison, all models are optimized via grid search and evaluated over 10 independent runs with a fixed hidden dimension of 16. Our GeoMoE framework incorporates GCN, HGAT, and Spherical-GCN as the Euclidean, hyperbolic, and spherical experts, respectively. All experiments are conducted on an NVIDIA RTX 3090 GPU, with comprehensive implementation details provided in Appendix E.1. Table 2: Ablation study. In the table, “✓” denotes GeoMoE with the component. The best results are boldfaced. Experts Losses Datasets Variant Euc. Hyp. Sph. ℒalignL_align ℒcontrL_contr Cora CiteSeer PubMed Photo WikiCS Airport (a) ✓ 82.02 70.24 78.45 92.84 74.29 83.19 (b) ✓ 79.71 69.41 75.56 93.38 77.62 89.25 (c) ✓ 76.68 70.57 76.85 91.61 75.85 85.63 (d) ✓ ✓ ✓ ✓ 82.97 71.12 78.73 93.88 78.13 91.97 (e) ✓ ✓ ✓ ✓ 82.75 71.37 78.84 93.42 75.76 87.28 (f) ✓ ✓ ✓ ✓ 83.47 72.89 77.19 93.76 77.85 90.31 (g) ✓ ✓ ✓ 82.23 70.91 78.45 93.52 77.99 91.40 (h) ✓ ✓ ✓ ✓ 83.02 72.90 78.59 93.79 78.23 91.77 (i) ✓ ✓ ✓ ✓ 82.77 73.84 78.81 94.25 78.45 92.19 GeoMoE ✓ ✓ ✓ ✓ ✓ 84.75 74.72 80.13 95.18 79.65 93.56 5.2 Main Results Bridging the Euclidean-Riemannian gap. GeoMoE establishes new state-of-the-art performance by consistently surpassing both Euclidean and Riemannian baselines across all datasets. On citation networks where Euclidean GNNs typically dominate, GeoMoE outperforms the strongest baseline methods by 0.51%–1.12%. More significantly, on datasets favoring non-Euclidean geometry, GeoMoE achieves superior results, surpassing the leading mixture model, GraphMoRE, by an impressive average margin of 1.28%. This performance highlights the efficacy of our design: distinct from GraphMoRE’s implicit routing or fixed-manifold approaches, GeoMoE leverages explicit ORC supervision to ensure precise topological alignment, effectively bridging the performance gap between these competing paradigms. Adaptive superiority across geometric regimes. GeoMoE demonstrates the strongest gains on graphs with diverse geometric characteristics. On CiteSeer—a citation network exhibiting mixed structural motifs—GeoMoE achieves the largest improvement, outperforming the best baseline (RNCGLN) by 1.12%1.12\%. On WikiCS, which combines hierarchical article links with dense community structures, GeoMoE surpasses the top baseline model (DGMAE) by 0.84%0.84\%. Even on relatively homogeneous graphs such as the hierarchical Airport and the community-dense Photo, it consistently improves over specialized baselines by 0.80%0.80\% and 0.60%0.60\%, respectively, by leveraging complementary geometric representations. This consistent advantage confirms that adaptive multi-geometry routing optimally represents the varied curvature regimes found in real-world graphs. 5.3 Ablation Study To dissect the contributions of the core components in GeoMoE, we conduct a series of comprehensive ablation experiments by constructing systematic variants that omit specific experts or regularization losses, as summarized in Table 2. Synergy of multi-geometry experts. The full three-expert GeoMoE consistently outperforms all single- and dual-expert variants. Single-geometry baselines (variants a–c) achieve competitive but limited performance, with each excelling only on graphs matching its geometric bias (e.g., hyperbolic expert on Airport: 89.25%, Euclidean on Cora: 82.02%). Dual-expert combinations (variants d–f) show partial synergy but remain suboptimal, particularly on datasets with mixed structural motifs like CiteSeer and WikiCS. In contrast, GeoMoE achieves the best results everywhere, confirming that Euclidean, hyperbolic, and spherical geometries provide complementary representational capacities that collectively capture a wider range of graph structures. Efficacy of geometric regularization components. The geometric gating alignment loss (ℒalignL_align) and contrastive loss (ℒcontrL_contr) are critical for optimizing performance. Without either loss (variant g), performance is unstable—strong on Photo but underperforming on Cora (82.23%82.23\%) and CiteSeer (70.91%70.91\%). Adding ℒalignL_align alone (variant h) boosts CiteSeer accuracy by 2.0%2.0\% to 72.90%72.90\% via improved routing alignment. Incorporating ℒcontrL_contr independently (variant i) enhances discriminative power, increasing PubMed accuracy to 78.81%78.81\% and Airport F1 to 92.19%92.19\%. Mechanistically, ℒalignL_align anchors routing to topological structures while ℒcontrL_contr ensures the global distinctiveness of multi-manifold representations. GeoMoE, integrating both objectives, achieves optimal performance across all datasets. This synergy confirms that explicit geometric alignment and contrastive refinement are indispensable for fully activating the representational potential of a multi-geometry mixture-of-experts. Figure 3: Consistency between gating weights and ORC on Photo. 5.4 Further Analysis To further understand the underlying mechanism of GeoMoE, we first evaluate the routing consistency with geometric priors across diverse curvature regimes, followed by a sensitivity study on the critical curvature threshold θ. Consistency between gating weights and ORC. We assess whether GeoMoE’s routing behavior aligns with geometric priors. As shown in Figure 3, expert utilization on Photo exhibits a clear curvature-aligned pattern: the hyperbolic expert dominates negatively curved regions, whereas the spherical expert prevails as ORC becomes positive. Crucially, the Euclidean weight peaks at κ≈0κ≈ 0 and decays with curvature magnitude, confirming its specialization in flat topologies. The sharp expert transitions observed around ±θ±θ demonstrate the gate network’s discriminative sensitivity, confirming a curvature-aware manifold partitioning rather than a diffuse mixing strategy. These results empirically support that our alignment objective effectively internalizes ORC as a routing signal, consistent with Theorem 4.2. Figure 4: Sensitivity to curvature threshold θ. Sensitivity to curvature threshold θ. To validate the robustness of our ORC-guided routing mechanism, we analyze the model’s sensitivity to the curvature threshold θ. As shown in Figure 4, GeoMoE peaks at θ=10−4θ=10^-4 on both PubMed and Photo, indicating that subtle curvature signals are crucial for routing. Performance remains stable on PubMed as θ increases, consistent with its Euclidean bias, while accuracy on Photo exhibits a sharper decline beyond θ=10−3θ=10^-3, confirming that forcing Riemannian-inclined nodes into Euclidean space incurs a geometric distortion penalty. Essentially, θ acts as a structural filter that modulates the transition between non-Euclidean and Euclidean representational regimes. The consistent performance across several orders of magnitude ([10−5,10−2][10^-5,10^-2]) demonstrates that our mechanism is highly robust to threshold selection, with a small default value being effective for diverse graph topologies. 6 Conclusion We present GeoMoE, a novel geometric mixture-of-experts framework that leverages Ollivier–Ricci curvature to enable node-wise adaptive routing across Euclidean, hyperbolic, and spherical spaces. By explicitly aligning gating decisions and contrastive learning with intrinsic graph geometry, GeoMoE ensures both interpretability and discriminability. Theoretical analysis validates the geometric consistency of our design, while extensive experiments demonstrate consistent superiority over strong baselines. This work bridges discrete curvature theory with multi-geometry representation learning, offering a principled path toward more faithful graph embeddings. Future work will explore the extension of curvature-aware routing to complex dynamic networks. Impact Statement This paper presents GeoMoE, a framework designed to improve the adaptivity and interpretability of graph representation learning through intrinsic geometric priors. As this work focuses on fundamental algorithmic advancements in machine learning, it has broad potential applications in fields such as social network analysis and bioinformatics. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here. References A. G. Ache and M. W. Warren (2019) Ricci curvature and the manifold learning problem. Advances in Mathematics 342, p. 14–66. Cited by: Definition 3.1, Definition 3.2. K. Atz, F. Grisoni, and G. Schneider (2021) Geometric deep learning on molecular representations. Nature Machine Intelligence 3 (12), p. 1023–1032. Cited by: §1. D. P. Bertsekas (2014) Constrained optimization and lagrange multiplier methods. Academic press. Cited by: §B.2. M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst (2017) Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine 34 (4), p. 18–42. Cited by: §1. S. Bubeck (2015) Convex optimization: algorithms and complexity. Foundations and trends in Machine Learning 8 (3-4), p. 231–357. Cited by: §B.1. H. Cao, Y. Wang, J. Li, P. Zhu, and Q. Hu (2025) Hyperbolic-euclidean deep mutual learning. In Proceedings of the ACM on Web Conference 2025, p. 3073–3083. Cited by: §1. I. Chami, Z. Ying, C. Ré, and J. Leskovec (2019) Hyperbolic graph convolutional neural networks. Advances in neural information processing systems 32. Cited by: §E.1, §1, Table 1, §5.1. F. Chen, Y. Wang, B. Wang, and C. J. Kuo (2020) Graph representation learning: a survey. APSIPA Transactions on Signal and Information Processing 9, p. e15. Cited by: §1. W. Chen, X. Han, Y. Lin, H. Zhao, Z. Liu, P. Li, M. Sun, and J. Zhou (2022) Fully hyperbolic neural networks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 5672–5686. Cited by: Table 1. T. M. Cover (1999) Elements of information theory. John Wiley & Sons. Cited by: §B.3. M. Cuturi (2013) Sinkhorn distances: lightspeed computation of optimal transport. Advances in neural information processing systems 26. Cited by: 1st item. M. Defferrard, M. Milani, F. Gusset, and N. Perraudin (2020) DeepSphere: a graph-based spherical cnn. arXiv preprint arXiv:2012.15000. Cited by: §1. F. Errica, D. Bacciu, and A. Micheli (2021) Graph mixture density networks. In International conference on machine learning, p. 3025–3035. Cited by: §2.2. A. Feng and M. Weber (2024) Graph pooling via ricci flow. arXiv preprint arXiv:2407.04236. Cited by: §D.3, §2.1. Forman (2003) Bochner’s method for cell complexes and combinatorial ricci curvature. Discrete & Computational Geometry 29 (3), p. 323–374. Cited by: §2.1. X. Fu, J. Wang, Y. Gao, Q. Sun, H. Yuan, J. Li, and X. Li (2025) Discrete curvature graph information bottleneck. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 16666–16673. Cited by: §2.1. K. Grover, H. Yu, X. Song, Q. Zhu, H. Xie, V. N. Ioannidis, and C. Faloutsos (2025) Spectro-riemannian graph neural networks. arXiv preprint arXiv:2502.00401. Cited by: §2.1. Z. Guo, Q. Sun, H. Yuan, X. Fu, M. Zhou, Y. Gao, and J. Li (2025) GraphMoRE: mitigating topological heterogeneity via mixture of riemannian experts. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 11754–11762. Cited by: §1, §2.2, Table 1. H. Han, J. Li, W. Huang, X. Tang, H. Lu, C. Luo, H. Liu, and J. Tang (2024) Node-wise filtering in graph neural networks: a mixture of experts approach. arXiv preprint arXiv:2406.03464. Cited by: §2.2. N. He, M. Yang, and R. Ying (2025) Lorentzian residual neural networks. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1, p. 436–447. Cited by: Table 1. F. Hu, L. Wang, S. Wu, L. Wang, and T. Tan (2021) Graph classification by mixture of diverse experts. arXiv preprint arXiv:2103.15622. Cited by: §2.2. X. Huang, W. Chen, B. Hu, and Z. Mao (2025) Graph mixture of experts and memory-augmented routers for multivariate time series anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 17476–17484. Cited by: §2.2. M. I. Jordan and R. A. Jacobs (1994) Hierarchical mixtures of experts and the em algorithm. Neural computation 6 (2), p. 181–214. Cited by: §B.1. J. Jost and S. Liu (2014) Ollivier’s ricci curvature, local clustering and curvature-dimension inequalities on graphs. Discrete & Computational Geometry 51 (2), p. 300–322. Cited by: 2nd item. S. Khoshraftar and A. An (2024) A survey on graph representation learning methods. ACM Transactions on Intelligent Systems and Technology 15 (1), p. 1–55. Cited by: §1. S. Kim, D. Lee, S. Kang, S. Lee, and H. Yu (2023) Learning topology-specific experts for molecular property prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37, p. 8291–8299. Cited by: §2.2. T. Kipf (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907. Cited by: §1, Table 1. P. Langley (2000) Crafting papers on machine learning. In Proceedings of the 17th International Conference on Machine Learning (ICML 2000), P. Langley (Ed.), Stanford, CA, p. 1207–1216. Cited by: §E.2. W. Li, D. Wang, Z. Ding, A. Sohrabizadeh, Z. Qin, J. Cong, and Y. Sun (2025) Hierarchical mixture of experts: generalizable learning for high-level synthesis. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 18476–18484. Cited by: §2.2. Y. Lin, L. Lu, and S. Yau (2011) Ricci curvature of graphs. Tohoku Mathematical Journal, Second Series 63 (4), p. 605–627. Cited by: §2.1. Q. Liu, M. Nickel, and D. Kiela (2019) Hyperbolic graph neural networks. Advances in neural information processing systems 32. Cited by: §1, Table 1. Z. Luo, G. Tadmor, O. Amar, D. Zeevi, and Z. Xu (2025) Local-curvature-aware knowledge graph embedding: an extended ricci flow approach. arXiv preprint arXiv:2512.07332. Cited by: §2.1. L. Ma, H. Han, J. Li, H. Shomer, H. Liu, X. Gao, and J. Tang (2024) Mixture of link predictors on graphs. Advances in Neural Information Processing Systems 37, p. 16043–16070. Cited by: §2.2. P. Mernyei and C. Cangea (2020) Wiki-cs: a wikipedia-based benchmark for graph neural networks. arXiv preprint arXiv:2007.02901. Cited by: §E.1, §5.1. G. Namata, B. London, L. Getoor, B. Huang, and U. Edu (2012) Query-driven active surveying for collective classification. In 10th international workshop on mining and learning with graphs, Vol. 8, p. 1. Cited by: §E.1, §5.1. K. Nguyen, N. M. Hieu, V. D. Nguyen, N. Ho, S. Osher, and T. M. Nguyen (2023) Revisiting over-smoothing and over-squashing using ollivier-ricci curvature. In International Conference on Machine Learning, p. 25956–25979. Cited by: §2.1. C. Ni, Y. Lin, F. Luo, and J. Gao (2019) Community detection on networks with ricci flow. Scientific reports 9 (1), p. 9984. Cited by: 3rd item, §4.5. Y. Ollivier (2009) Ricci curvature of markov chains on metric spaces. Journal of Functional Analysis 256 (3), p. 810–864. Cited by: §1, §2.1, §3.2. A. v. d. Oord, Y. Li, and O. Vinyals (2018) Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748. Cited by: §B.3, §4.3, Theorem 4.3. B. Poole, S. Ozair, A. Van Den Oord, A. Alemi, and G. Tucker (2019) On variational bounds of mutual information. In International conference on machine learning, p. 5171–5180. Cited by: §B.3, Theorem 4.3. T. K. Rusch, B. Chamberlain, J. Rowbottom, S. Mishra, and M. Bronstein (2022) Graph-coupled oscillator networks. In International Conference on Machine Learning, p. 18888–18909. Cited by: Table 1. P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad (2008) Collective classification in network data. AI magazine 29 (3), p. 93–93. Cited by: §E.1, §5.1. N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean (2017) Outrageously large neural networks: the sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538. Cited by: §2.2. O. Shchur, M. Mumme, A. Bojchevski, and S. Günnemann (2018) Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868. Cited by: §E.1, §5.1. C. Shen, P. Ding, J. Wee, J. Bi, J. Luo, and K. Xia (2024) Curvature-enhanced graph convolutional network for biomolecular interaction prediction. Computational and Structural Biotechnology Journal 23, p. 1016–1025. Cited by: §2.1. V. Y. Shirasuna, E. Soares, E. V. Brazil, K. F. A. Gutierrez, R. Cerqueira, S. Takeda, and A. Kishimoto (2024) A multi-view mixture-of-experts based on language and graphs for molecular properties prediction. In ICML 2024 AI for Science Workshop, Cited by: §2.2. L. Sun, Z. Huang, Z. Wang, F. Wang, H. Peng, and P. S. Yu (2024) Motif-aware riemannian graph neural network with generative-contrastive learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, p. 9044–9052. Cited by: §1, Table 1. L. Sun, Z. Huang, H. Wu, J. Ye, H. Peng, Z. Yu, and P. S. Yu (2023a) Deepricci: self-supervised graph structure-feature co-refinement for alleviating over-squashing. In 2023 IEEE International Conference on Data Mining (ICDM), p. 558–567. Cited by: §D.3, §2.1. L. Sun, J. Ye, H. Peng, F. Wang, and P. S. Yu (2023b) Self-supervised continual graph learning in adaptive riemannian spaces. In Proceedings of the AAAI conference on artificial intelligence, Vol. 37, p. 4633–4642. Cited by: §2.1. L. Sun, Z. Zhang, J. Ye, H. Peng, J. Zhang, S. Su, and P. S. Yu (2022) A self-supervised mixed-curvature graph neural network. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36, p. 4146–4155. Cited by: §1. P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio (2017) Graph attention networks. arXiv preprint arXiv:1710.10903. Cited by: §1, Table 1. H. Wang, Z. Jiang, Y. You, Y. Han, G. Liu, J. Srinivasa, R. Kompella, Z. Wang, et al. (2023) Graph mixture of experts: learning on large-scale graphs with explicit diversity modeling. Advances in Neural Information Processing Systems 36, p. 50825–50837. Cited by: §2.2. T. Wang and P. Isola (2020) Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International conference on machine learning, p. 9929–9939. Cited by: item (i). Y. Wang, K. Yi, X. Liu, Y. G. Wang, and S. Jin (2022) Acmp: allen-cahn message passing for graph neural networks with particle phase transition. arXiv preprint arXiv:2206.05437. Cited by: Table 1. F. Wu, A. Souza, T. Zhang, C. Fifty, T. Yu, and K. Weinberger (2019) Simplifying graph convolutional networks. In International conference on machine learning, p. 6861–6871. Cited by: Table 1. Q. Wu, W. Zhao, Z. Li, D. P. Wipf, and J. Yan (2022) Nodeformer: a scalable graph structure learning transformer for node classification. Advances in Neural Information Processing Systems 35, p. 27387–27401. Cited by: Table 1. Q. Wu, W. Zhao, C. Yang, H. Zhang, F. Nie, H. Jiang, Y. Bian, and J. Yan (2023) Sgformer: simplifying and empowering transformers for large-graph representations. Advances in Neural Information Processing Systems 36, p. 64753–64773. Cited by: Table 1. Z. Wu, J. Cai, Y. Zhang, J. Lu, Z. Chen, S. Zhuang, and H. Wang (2025) Where graph meets heterogeneity: multi-view collaborative graph experts. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, Cited by: §2.2. H. Yang, H. Chen, S. Pan, L. Li, P. S. Yu, and G. Xu (2022) Dual space graph contrastive learning. In Proceedings of the ACM web conference 2022, p. 1238–1247. Cited by: §1. M. Yang, M. Zhou, L. Pan, and I. King (2023) κ : Tree-likeness modeling via continuous and discrete curvature learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, p. 2965–2977. Cited by: Definition 3.1, Definition 3.2. X. Zhang, D. Song, and D. Tao (2023) Ricci curvature-based graph sparsification for continual graph representation learning. IEEE Transactions on Neural Networks and Learning Systems. Cited by: §2.1. Y. Zhang, X. Wang, C. Shi, X. Jiang, and Y. Ye (2021a) Hyperbolic graph attention network. IEEE Transactions on Big Data 8 (6), p. 1690–1701. Cited by: §1, Table 1. Y. Zhang, X. Wang, C. Shi, N. Liu, and G. Song (2021b) Lorentzian graph convolutional networks. In Proceedings of the web conference 2021, p. 1249–1261. Cited by: Table 1. Z. Zheng, Y. Yang, Z. Guan, W. Zhao, and W. Lu (2025) Discrepancy-aware graph mask auto-encoder. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, p. 4038–4049. Cited by: Table 1. S. Zhu, S. Pan, C. Zhou, J. Wu, Y. Cao, and B. Wang (2020) Graph geometry interaction learning. Advances in Neural Information Processing Systems 33, p. 7548–7558. Cited by: Table 1. Y. Zhu, L. Feng, Z. Deng, Y. Chen, R. Amor, and M. Witbrock (2024) Robust node classification on graph data with graph and label noise. In Proceedings of the AAAI conference on artificial intelligence, Vol. 38, p. 17220–17227. Cited by: Table 1. Appendix A Supplementary Preliminary Definition A.1 (First-Order Wasserstein Distance). Let μi _i and μj _j be two probability measures defined over the node set V. The first-order Wasserstein distance W1(μi,μj)W_1( _i, _j) is defined as: W1(μi,μj)=infπ∈Π(μi,μj)∑vk∈,vl∈π(vk,vl)d(vk,vl),W_1( _i, _j)= _π∈ ( _i, _j) _v_k ,v_l π(v_k,v_l)\,d(v_k,v_l), (13) where Π(μi,μj) ( _i, _j) is the set of all transport plans from μi _i to μj _j. A transport plan π:×→[0,1]π:V×V→[0,1] satisfies the marginal constraints: ∑vl∈π(vk,vl)=μi(vk)and∑vk∈π(vk,vl)=μj(vl). _v_l π(v_k,v_l)= _i(v_k) _v_k π(v_k,v_l)= _j(v_l). (14) Definition A.2 (Local Probability Measure). Given a graph =(,ℰ)G=(V,E), for a node vi∈v_i , let dvid_v_i denote its degree and (vi)N(v_i) its neighborhood set. A parameter p∈[0,1]p∈[0,1] controls the weight assigned to the center node itself. The local probability measure μvi _v_i over V is defined as: μvi(v)=p,if v=vi,1−pdvi,if v∈(vi),0,otherwise. _v_i(v)= casesp,&if v=v_i,\\ 1-pd_v_i,&if v (v_i),\\ 0,&otherwise. cases (15) Appendix B Theoretical Analysis B.1 Proof of Theorem 4.1 Proof. Let I∗(vi)=argminm∈E,H,Sℓm(vi)I^*(v_i)= _m∈\E,H,S\ ^m(v_i) be the oracle expert selector (Jordan and Jacobs, 1994), and define: ∗(vi)=I∗(vi)(vi),h^*(v_i)=h^I^*(v_i)(v_i), (16) ℓ∗(vi)=ℓ(∗(vi),y(vi))=minmℓm(vi). ^*(v_i)= (h^*(v_i),y(v_i))= _m ^m(v_i). (17) By definition, for any expert m, we have: [ℓ∗]≤[ℓm].E[ ^*] [ ^m]. (18) Furthermore, by the advantage region assumption, there exists η0>0 _0>0 such that: [ℓ∗]<minm[ℓm]−η0.E[ ^*]< _mE[ ^m]- _0. (19) Assume the loss function ℓ(⋅,y) (·,y) is L-Lipschitz in its first argument (Bubeck, 2015): |ℓ(fused(vi),y(vi))−ℓ(∗(vi),y(vi))|≤L⋅‖fused(vi)−∗(vi)‖, split | (h^fused(v_i),y(v_i))- (h^*(v_i),y(v_i)) |\\ ≤ L·\|h^fused(v_i)-h^*(v_i)\|, split (20) where L is Lipschitz constant of the loss function ℓ . Then: ℓ(fused(vi),y(vi))≤ℓ∗(vi)+L⋅‖fused(vi)−∗(vi)‖. (h^fused(v_i),y(v_i))≤ ^*(v_i)+L·\|h^fused(v_i)-h^*(v_i)\|. (21) Taking expectations yields: [ℓ(fused,y)]≤[ℓ∗]+L⋅∥fused−∗∥].E[ (h^fused,y)] [ ^*]+L·E\|h^fused-h^*\|]. (22) Recall: fused(vi)=∑mωimm(vi).h^fused(v_i)= _m _i^mh^m(v_i). (23) Thus: ‖fused(vi)−∗(vi)‖=‖∑mωim(m(vi)−∗(vi))‖≤∑mωim‖m(vi)−∗(vi)‖. split&\|h^fused(v_i)-h^*(v_i)\|\\ &= \| _m _i^m(h^m(v_i)-h^*(v_i)) \|\\ &≤ _m _i^m\|h^m(v_i)-h^*(v_i)\|. split (24) Since all embeddings are projected into a common Euclidean tangent space via logarithmic maps and the hyperbolic and spherical spaces are compact, there exists a finite constant M<∞M<∞ such that: ‖m(vi)−n(vi)‖≤Mfor all m,n∈E,H,S.\|h^m(v_i)-h^n(v_i)\|≤ M all m,n∈\E,H,S\. When m=I∗(vi)m=I^*(v_i), we have m(vi)=∗(vi)h^m(v_i)=h^*(v_i), so that term vanishes. Therefore: ∑mωim‖m(vi)−∗(vi)‖=∑m≠I∗(vi)ωim‖m(vi)−∗(vi)‖≤M∑m≠I∗(vi)ωim. split& _m _i^m\|h^m(v_i)-h^*(v_i)\|\\ &= _m≠ I^*(v_i) _i^m\|h^m(v_i)-h^*(v_i)\|\\ &≤ M _m≠ I^*(v_i) _i^m. split (25) Note that: ‖i−eI∗(vi)‖1=2(1−ωiI∗(vi))=2∑m≠I∗(vi)ωim. split\| ω_i-e^I^*(v_i)\|_1=2 (1- _i^I^*(v_i) )=2 _m≠ I^*(v_i) _i^m. split (26) so ∑m≠I∗(vi)ωim=12‖i−eI∗(vi)‖1. _m≠ I^*(v_i) _i^m= 12\| ω_i-e^I^*(v_i)\|_1. (27) Therefore, ∑mωim‖m(vi)−∗(vi)‖≤M2‖i−eI∗(vi)‖1, _m _i^m\|h^m(v_i)-h^*(v_i)\|≤ M2\| ω_i-e^I^*(v_i)\|_1, (28) and thus ‖fused(vi)−∗(vi)‖≤M2‖i−eI∗(vi)‖1.\|h^fused(v_i)-h^*(v_i)\|≤ M2\| ω_i-e^I^*(v_i)\|_1. (29) Taking expectations: [‖fused−∗‖]≤M2[‖−eI∗‖1].E[\|h^fused-h^*\|]≤ M2E[\| ω-e^I^*\|_1]. (30) By assumption, on each advantage region Ωm _m, [‖−em‖1∣vi∈Ωm]≤δ.E[\| ω-e^m\|_1 v_i∈ _m]≤δ. (31) We now bound [‖−eI∗‖1]E[\| ω-e^I^*\|_1] using the law of total expectation. Partition the node set V into the advantage regions Ωmm∈E,H,S\ _m\_m∈\E,H,S\ and their complement: [‖−eI∗‖1] [\| ω-e^I^*\|_1] (32) =∑m∈E,H,Sμ(Ωm)[‖−eI∗‖1∣vi∈Ωm] = _m∈\E,H,S\μ( _m)\,E[\| ω-e^I^*\|_1 v_i∈ _m] +μ(∖⋃m∈E,H,SΩm)[‖−eI∗‖1∣vi∉⋃mΩm], +μ (V _m∈\E,H,S\ _m )\,E[\| ω-e^I^*\|_1 v_i∉ _m _m], where μ denotes the uniform probability measure over V. On each advantage region Ωm _m, we have I∗=mI^*=m by definition, so the first term becomes: ∑m∈E,H,Sμ(Ωm)[‖−em‖1∣v∈Ωm] _m∈\E,H,S\μ( _m)\,E[\| ω-e^m\|_1 v∈ _m] (33) ≤∑m∈E,H,Sμ(Ωm)⋅δ=δ⋅∑m∈E,H,Sμ(Ωm)≤δ. ≤ _m∈\E,H,S\μ( _m)·δ=δ· _m∈\E,H,S\μ( _m)≤δ. For the complement region, note that the L1L^1 distance between two probability vectors is bounded by 2. Therefore: [‖−eI∗‖1∣v∉⋃mΩm]≤2.E[\| ω-e^I^*\|_1 v∉ _m _m]≤ 2. (34) Combining these bounds, we obtain: [‖−eI∗‖1]≤δ+2⋅μ(∖⋃m∈E,H,SΩm).E[\| ω-e^I^*\|_1]≤δ+2·μ (V _m∈\E,H,S\ _m ). (35) Combining Equation 22, Equation 30 and Equation 35, we have [ℓ(fused,y)]≤[ℓ∗]+LM2(δ+2⋅μ(∖⋃mΩm)). [ (h^fused,y)] [ ^*]+ LM2 (δ+2·μ (V _m _m ) ). (36) If the gating error δ is sufficiently small and the uncovered region V∖∪mVmV _mV_m has sufficiently small measure so that LM2(δ+2⋅μ(∖⋃mΩm))<η0, LM2 (δ+2·μ (V _m _m ) )< _0, then from Equation 19 we obtain the strict inequality [ℓ(fused,y)]<minm∈E,H,S[ℓm].E[ (h^fused,y)]< _m∈\E,H,S\E[ ^m]. ∎ B.2 Proof of Theorem 4.2 Proof. The KL divergence is defined as: KL(ω∗∥ω)=∑m∈E,H,Sω∗mlogω∗mωm.KL(ω^*\|ω)= _m∈\E,H,S\ω^*m ω^*mω^m. (37) During minimization of this loss, the simplex constraint ω∈Δ2=(ωE,ωH,ωS)∣∑m∈E,H,Sωm=1,ωm≥0ω∈ ^2=\(ω^E,ω^H,ω^S) _m∈\E,H,S\ω^m=1,ω^m≥ 0\ must be strictly satisfied. Computing the gradient of the KL divergence with respect to wmw_m without accounting for the simplex constraint can lead to updates that violate ∑m∈E,H,Sωm=1 _m∈\E,H,S\ω^m=1. To enforce this constraint, the Lagrange multiplier method (Bertsekas, 2014) is employed. The Lagrangian is constructed as: ℒ(ω,λ)=KL(ω∗∥ω)+λ(∑m∈E,H,Sωm−1).L(ω,λ)=KL(ω^*\|ω)+λ ( _m∈\E,H,S\ω^m-1 ). (38) Taking the partial derivative with respect to ωmω^m and setting it to zero yields the optimality condition: ∂ℒ∂ωm=−ω∗mωm+λ=0⇒ωm=ω∗mλ. ∂ω^m=- ω^*mω^m+λ=0 ω^m= ω^*mλ. (39) Applying the simplex constraint ∑m∈E,H,Sωm=1 _m∈\E,H,S\ω^m=1 gives λ=1λ=1, and thus the unique global minimizer is ω=ω∗ω=ω^*. Without loss of generality, we focus our analysis on the positive curvature region Ω+=v∈∣κ(v)≥θ _+=\v κ(v)≥θ\. Define: a(v)=σ(−|κ(v)|+θη), a(v)=σ ( -|κ(v)|+θη ), (40) b(v)=σ(−κ(v)−θη), b(v)=σ ( -κ(v)-θη ), c(v)=σ(κ(v)−θη). c(v)=σ ( κ(v)-θη ). Then ω∗S=c(v)/(a(v)+b(v)+c(v))ω^*S=c(v)/(a(v)+b(v)+c(v)). By the quotient rule: ∂ω∗S∂κ(v)=∂c(v)∂κ(v)(a(v)+b(v)+c(v))−c(v)(∂a(v)∂κ(v)+∂b(v)∂κ(v)+∂c(v)∂κ(v))(a(v)+b(v)+c(v))2. ∂ω^*S∂κ(v)= aligned & ∂ c(v)∂κ(v)(a(v)+b(v)+c(v))\\ &-c(v) ( ∂ a(v)∂κ(v)+ ∂ b(v)∂κ(v)+ ∂ c(v)∂κ(v) ) aligned(a(v)+b(v)+c(v))^2. (41) For any v∈Ω+v∈ _+ (i.e., κ(v)≥θ>0κ(v)≥θ>0), the signs of the partial derivatives are: ∂a(v)∂κ(v)=−1ησ′(−κ(v)+θη)<0,∂b(v)∂κ(v)=−1ησ′(−κ(v)−θη)<0,∂c(v)∂κ(v)=1ησ′(κ(v)−θη)>0. cases ∂ a(v)∂κ(v)=- 1ησ ( -κ(v)+θη )<0,\\ ∂ b(v)∂κ(v)=- 1ησ ( -κ(v)-θη )<0,\\ ∂ c(v)∂κ(v)= 1ησ ( κ(v)-θη )>0. cases (42) The numerator of the derivative can be rewritten as: N=∂c(v)∂κ(v)(a(v)+b(v))−c(v)(∂a(v)∂κ(v)+∂b(v)∂κ(v))⏟>0.N= ∂ c(v)∂κ(v)(a(v)+b(v))-c(v) ( ∂ a(v)∂κ(v)+ ∂ b(v)∂κ(v) )_>0. (43) Since ∂a(v)∂κ(v)+∂b(v)∂κ(v)<0 ∂ a(v)∂κ(v)+ ∂ b(v)∂κ(v)<0, the second term is positive. The first term is obviously also positive. It follows immediately that N>0N>0. The denominator is strictly positive, so ∂ω∗S∂κ(v)>0 ∂ω^*S∂κ(v)>0 for all v∈Ω+v∈ _+. Let f(κ)=∂ω∗S∂κf(κ)= ∂ω^*S∂κ be the derivative function defined on the domain of curvature. Given that the graph is finite, the curvature values κ(v)κ(v) are naturally bounded. We thus consider f(κ)f(κ) on the compact set [θ,M][θ,M], where M=maxv∈κ(v)M= _v κ(v) represents the maximum observed curvature. Since f(κ)f(κ) is continuous and strictly positive (f(κ)>0f(κ)>0) for all κ∈[θ,M]κ∈[θ,M], the Extreme Value Theorem guarantees that it attains a positive minimum: C+=minκ∈[θ,M]f(κ)>0.C_+= _κ∈[θ,M]f(κ)>0. (44) Consequently, we have ∂ω∗S∂κ(v)≥C+>0 ∂ω^*S∂κ(v)≥ C_+>0 for all v∈Ω+v∈ _+. At the convergence point, ω=ω∗ω=ω^*. Since the gating network is a differentiable function, it follows that: ∂ωS∂κ(v)=∂ω∗S∂κ(v)≥C+>0. ∂ω^S∂κ(v)= ∂ω^*S∂κ(v)≥ C_+>0. The proofs for the negative curvature region Ω−=v∈∣κ(v)≤−θ _-=\v κ(v)≤-θ\ and the near-zero curvature region Ω0=v∈∣|κ(v)|≤θ _0=\v |κ(v)|≤θ\ are analogous, yielding constants C−>0C_->0 and C0>0C_0>0, respectively. ∎ B.3 Proof of Theorem 4.3 Proof. We establish the mutual information lower bound for the ORC-guided contrastive learning objective. The proof follows the standard InfoNCE variational framework (Oord et al., 2018; Poole et al., 2019) while explicitly incorporating our curvature-driven construction. Let v be a node uniformly sampled from V. Define: • κ=κ(v)κ=κ(v): the node-level ORC, • fused=fused(v)h^fused=h^fused(v): the fused representation, • ∗=∗(κ)h^*=h^*(κ): the ORC-guided geometric prior embedding (positive sample). Let 1−,…,K−∼i.i.d.p(∗)h^-_1,…,h^-_K .i.d. p(h^*) be K negative samples independently drawn from the marginal distribution of positive embeddings. Form the candidate set =∗,1−,…,K−C=\h^*,h^-_1,…,h^-_K\ of size K+1K+1. Note that the subset of negatives 1−,…,K−\h^-_1,…,h^-_K\ corresponds to the set −(v)N^-(v) defined in the main text. Introduce a discrete index random variable J∈0,1,…,KJ∈\0,1,…,K\ indicating which candidate is the true positive; J=0J=0 corresponds to ∗h^*. Under the data generation process, we assume a uniform prior: p(J=j∣)=1K+1,j=0,…,K.p(J=j )= 1K+1, j=0,…,K. (45) Define the model’s predictive distribution via a softmax over similarity scores: qθ(j∣fused,)=exp(s(fused,cj)/τc)∑ℓ=0Kexp(s(fused,cℓ)/τc),q_θ(j ^fused,C)= (s(h^fused,c_j)/ _c ) _ =0^K (s(h^fused,c_ )/ _c ), (46) where c0=∗c_0=h^*, cj=j−c_j=h^-_j for j≥1j≥ 1, s(u,v)=u⊤v/(‖u‖‖v‖)s(u,v)=u v/(\|u\|\|v\|) is cosine similarity, and τc _c is the temperature parameter. In training, we construct the candidate set such that the true positive always occupies the position j=0j=0, i.e., J=0J=0 in the data. The InfoNCE loss can then be written as the expected negative log-likelihood of the true index: ℒcontr=−(fused,)[logqθ(J=0∣fused,)].L_contr=-E_(h^fused,C) [ q_θ(J=0 ^fused,C) ]. (47) Consider the conditional mutual information between fusedh^fused and J given C: I(fused;J∣)=p(fused,J,)[logp(J∣fused,)p(J∣)].I(h^fused;J )=E_p(h^fused,J,C) [ p(J ^fused,C)p(J ) ]. (48) We now introduce the variational distribution qθ(J∣fused,)q_θ(J ^fused,C), then add and subtract logqθ(J∣fused,) q_θ(J ^fused,C) inside the expectation: I(fused;J∣) I(h^fused;J ) (49) =[logp(J∣fused,)p(J∣)] =E [ p(J ^fused,C)p(J ) ] =[logp(J∣fused,)qθ(J∣fused,)+logqθ(J∣fused,)p(J∣)] =E [ p(J ^fused,C)q_θ(J ^fused,C)+ q_θ(J ^fused,C)p(J ) ] =[logqθ(J∣fused,)p(J∣)]⏟variational term+[logp(J∣fused,)qθ(J∣fused,)]⏟KL term. = E [ q_θ(J ^fused,C)p(J ) ]_variational term+ E [ p(J ^fused,C)q_θ(J ^fused,C) ]_KL term. According to the definition of conditional expectation and the law of total expectation, the second term can be rewritten as a two-layer expectation over (fused,)(h^fused,C) and J: [logp(J∣fused,)qθ(J∣fused,)] [ p(J ^fused,C)q_θ(J ^fused,C) ] (50) =fused,[J∼p(⋅∣fused,)[logp(J∣fused,)qθ(J∣fused,)]]. =E_h^fused,C [E_J p(· ^fused,C) [ p(J ^fused,C)q_θ(J ^fused,C) ] ]. The inner expectation is the KL divergence: J∼p(⋅∣fused,)[logp(J∣fused,)qθ(J∣fused,)] _J p(· ^fused,C)[ p(J ^fused,C)q_θ(J ^fused,C)] (51) =KL(p(⋅∣fused,)∥qθ(⋅∣fused,)). =KL (p(· ^fused,C)\,\|\,q_θ(· ^fused,C) ). Hence: [logp(J∣fused,)qθ(J∣fused,)] [ p(J ^fused,C)q_θ(J ^fused,C) ] (52) =fused,[KL(p(⋅∣fused,)∥qθ(⋅∣fused,))]≥ 0, =E_h^fused,C [KL (p(· ^fused,C)\,\|\,q_θ(· ^fused,C) ) ]\;≥ 0, which yields the inequality I(fused;J∣)≥[logqθ(J∣fused,)p(J∣)].I(h^fused;J )\;≥\;E [ q_θ(J ^fused,C)p(J ) ]. (53) Next, we expand the right-hand side: [logqθ(J∣fused,)p(J∣)] [ q_θ(J ^fused,C)p(J ) ] (54) =[logqθ(J∣fused,)−logp(J∣)] =E [ q_θ(J ^fused,C)- p(J ) ] =[logqθ(J∣fused,)]−[logp(J∣)]. =E [ q_θ(J ^fused,C) ]-E [ p(J ) ]. Using the uniform prior assumption in Equation 45, we have p(J∣)=1K+1⇒logp(J∣)=−log(K+1),p(J )= 1K+1 p(J )=- (K+1), so that −[logp(J∣)]=log(K+1).-E [ p(J ) ]= (K+1). (55) Substituting Equation 55 into Equation 54 and then into Equation 53 yields I(fused;J∣) I(h^fused;J ) ≥[logqθ(J∣fused,)]+log(K+1). [ q_θ(J ^fused,C) ]+ (K+1). (56) On the other hand, from the definition of ℒcontrL_contr in Equation 47, we have [logqθ(J∣fused,)]=−ℒcontr. [ q_θ(J ^fused,C) ]=-L_contr. (57) Combining Equation 56 and Equation 57, we obtain I(fused;J∣)≥log(K+1)−ℒcontr.I(h^fused;J )\;≥\; (K+1)-L_contr. (58) Finally, we connect the index J to the curvature-driven geometric prior ∗(κ)h^*(κ). Recall that the positive sample is defined as ∗=∗(κ)h^*=h^*(κ), and the candidate set is =∗,1−,…,K−C=\h^*,h^-_1,…,h^-_K\, where the negatives k−h^-_k are drawn i.i.d. from p(∗)p(h^*). By assumption, these negatives are sampled independently of fusedh^fused, so C introduces no additional dependence between fusedh^fused and J. Furthermore, given C, the index J is a deterministic function of ∗(κ)h^*(κ), as it uniquely identifies the position of the positive sample within the set. This yields a Markov structure fused⟶∗(κ)⟶(,J),h^fused\; \;h^*(κ)\; \;(C,J), and by the data processing inequality (Cover, 1999), we have I(fused;∗(κ))≥I(fused;J∣).I(h^fused;h^*(κ))\;≥\;I(h^fused;J ). (59) Combining Equation 58 and Equation 59, I(fused;∗)≥log(K+1)−ℒcontr,I (h^fused;h^* )\;≥\; (K+1)-L_contr, (60) which is exactly the claimed mutual information lower bound. ∎ Remark on Negative Sampling Assumptions: The derivation of Theorem 4.3 follows the standard assumption that negative samples are drawn i.i.d. from the marginal distribution p(∗)p(h^*), ensuring that the candidate set C is conditionally independent of the anchor fusedh^fused. In our practical implementation, however, we employ a “hard negative” selection strategy (corresponding to −(v)N^-(v)) based on curvature similarity to fusedh^fused. While this introduces a weak dependency between C and fusedh^fused and deviates from the strict i.i.d. assumption, the theoretical bound remains a robust approximation for two reasons: (i) The InfoNCE objective continues to provide a valid lower bound on mutual information when the dependency is weak or bounded, as discussed in recent analyses of contrastive learning (Wang and Isola, 2020). (i) Empirically, curvature-aware hard negatives enhance the discriminative power of the representations, thereby improving the empirical tightness of the bound under the same geometric consistency constraint. Thus, while our practical sampling strategy modifies the idealized assumption, it preserves the underlying principle of mutual information maximization while strengthening the practical efficacy of representation learning. Appendix C Algorithm To provide a clear roadmap for our implementation, we delineate the training procedure of GeoMoE in Algorithm 1. The process begins with the offline precomputation of Ollivier-Ricci curvature (ORC), which serves as a static topological prior for geometric routing. During each training iteration, the model concurrently generates node representations among multiple manifold experts and dynamically determines the gating weights. The optimization follows a multi-objective strategy, where geometric consistency is enforced via ℒalignL_align and ℒcontrL_contr alongside the loss of primary tasks ℒtaskL_task. This structured approach ensures that the model parameters Θ are refined to align with the underlying graph topology effectively. Algorithm 1 Training Process of GeoMoE 0: Graph =(,ℰ)G=(V,E), node features X, adjacency matrix A, labels Y, ORC threshold θ, smoothing factor η, loss weights α,β,γα,β,γ, max epochs T 0: Model parameters Θ and fused node embeddings vifusedvi∈\h_v_i^fused\_v_i 1: Precompute Ollivier-Ricci curvature k(vi)k(v_i) for all vi∈v_i and initialize Θ randomly 2: for t=1t=1 to T do 3: Compute expert embeddings E,H,Sh^E,h^H,h^S via Equation 3; 4: Calculate gating weights w via Equation 6; 5: Obtain fused representations fusedh^fused via Equation 4; 6: Compute gating alignment loss ℒalignL_align via Equation 8; 7: Construct ORC-guided contrastive pairs and compute ℒcontrL_contr via Equation 10; 8: Evaluate supervised loss ℒtaskL_task and formulate total loss ℒtotalL_total via Equation 12; 9: Update Θ via gradient descent. 10: end for Appendix D Complexity and Scalability Analysis In this section, we provide a detailed analysis of the computational complexity of GeoMoE. We distinguish between the offline geometric pre-computation and the online model training, demonstrating that our framework is efficient and scalable to large graph datasets. D.1 Offline Geometric Pre-computation The core geometric signal in our framework is the Ollivier-Ricci Curvature (ORC). The bottleneck lies in computing the 1-Wasserstein distance W1(μu,μv)W_1( _u, _v) for each edge (u,v)∈ℰ(u,v) . Exact Computation. Exact computation of W1W_1 involves solving a Linear Programming (LP) problem. For a graph with maximum degree Δ , the transport plan has size roughly Δ×Δ × . Using standard interior-point methods, the worst-case complexity per edge is (Δ3logΔ)O( ^3 ). For the entire graph, the complexity is: exact=(|ℰ|⋅Δ3logΔ).T_exact=O(|E|· ^3 ). (61) While feasible for the datasets used in our main experiments (e.g., PubMed, roughly 20k nodes), this becomes prohibitive for million-scale graphs. Scalable Approximations. To ensure scalability, GeoMoE supports multiple approximation strategies: • Sinkhorn Algorithm (Cuturi, 2013): By regularizing the optimal transport problem, the Sinkhorn algorithm converts the LP into iterative matrix scaling operations. The complexity per edge is (L⋅Δ2)O(L· ^2), where L is the number of iterations (typically small, e.g., 10−2010-20). This offers a significant speedup over exact LP. • Local Clustering Coefficient Bound (Jost and Liu, 2014): Jost and Liu derived a theoretical lower bound for ORC based on local clustering coefficients. For an edge (u,v)(u,v), the curvature is bounded by geometric properties related to the number of triangles (shared neighbors). Computing this bound relies on triangle enumeration, which typically costs (Δ2)O( ^2) per node but is significantly faster than LP solvers in practice due to the sparsity of real-world graphs. • Neighborhood Overlap Approximation (Ni et al., 2019): For maximum scalability, we approximate the transport cost using the Jaccard index of neighborhoods. The intersection of two sorted neighbor lists of size dud_u and dvd_v takes (du+dv)O(d_u+d_v). Summing over all edges, the total complexity is proportional to the sum of squared degrees, bounded by: approx=(|ℰ|⋅Δ)T_approx=O(|E|· ) (62) In real-world sparse graphs where Δ≪|| |V|, this approximation scales linearly with the number of edges, making it applicable to massive networks. D.2 Online Training Complexity During the training phase, GeoMoE decouples from the curvature calculation logic. The curvature values κ(u,v)κ(u,v) are stored as static edge attributes. • Gating Network: The curvature-guided gating alignment (Section 4.2) performs a simple lookup operation and an MLP projection. This incurs an (1)O(1) cost per node/edge, negligible compared to feature aggregation. • Expert Computation: The three experts (Euclidean, Hyperbolic, Spherical) operate in parallel. Assuming a standard message-passing scheme with hidden dimension d, the complexity is (|ℰ|d+||d2)O(|E|d+|V|d^2). While Riemannian manifold operations (e.g., logarithmic maps) introduce a larger constant factor than Euclidean operations, they do not change the asymptotic complexity and are efficiently parallelized on GPUs. • Overall Complexity: The total training complexity per epoch is: train=(|ℰ|⋅d+||⋅d2).T_train=O(|E|· d+|V|· d^2). (63) Since d is a small constant (e.g., 16 or 64), the training complexity is linear in the number of edges, matching standard GNN baselines such as GCN. D.3 Discussion on Approximation Validity One might question whether approximate curvature compromises the model’s performance. Recent studies (Sun et al., 2023a; Feng and Weber, 2024) suggest that for graph learning tasks, the relative ranking and sign of the curvature (indicating tree-likeness vs. clique-likeness) are more critical than the exact numerical value usually. Since our gating mechanism (Equation 7) relies on thresholds and soft alignments, it is robust to the minor numerical perturbations introduced by approximation methods. Therefore, GeoMoE maintains its effectiveness even under scalable approximation regimes. Appendix E Experiments E.1 Experimental Settings Datasets. We evaluate our method on six benchmark datasets spanning diverse domains. Cora, CiteSeer, and PubMed (Sen et al., 2008; Namata et al., 2012) are canonical citation networks, in which nodes correspond to scientific publications and citations are treated as undirected edges. Photo (Shchur et al., 2018) is derived from the Amazon co-purchase graph, comprising photography-related products as nodes, with edges reflecting frequent co-purchase patterns. WikiCS (Mernyei and Cangea, 2020) constructs a hyperlink graph from Wikipedia, with nodes representing computer science articles and edges denoting internal hyperlinks; each article is annotated with one of ten subfield categories. Airport (Chami et al., 2019) models the global air transportation system, where nodes are airports and edges reflect the existence of commercial airline routes between them. Detailed statistics for all datasets are provided in Table 3 Implementation Details. In our experiments, we adopt the standard data splits for the three citation networks (Cora, CiteSeer, PubMed). For Photo and Airport, we use random splits of 60%/20%/20% and 70%/15%/15% for training, validation, and testing, respectively. For WikiCS, we use the official fixed split provided by the dataset, specifically the first of its 20 predefined training/validation partitions along with the standard test set. The grid search is performed over the following search space: Learning rate: [0.001, 0.005, 0.01, 0.02]; Dropout rate: [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6]; weight decay: [0, 1e-4, 5e-4, 1e-3]; Number of hidden layers: [1, 2, 3]. To ensure statistical reliability, all results are reported as the average of 10 runs with random parameter initializations. We set the dimension of latent representation of all methods as 16. In our GeoMoE framework, we designate GCN and HGAT as the Euclidean and hyperbolic base experts, respectively. All experiments are implemented in Python using the PyTorch framework and conducted on an NVIDIA GeForce RTX 3090 GPU. Table 3: Statistics of the experimental datasets. Dataset #Nodes #Edges #Classes #Features Cora 2,708 5,429 7 1,433 CiteSeer 3,327 4,732 6 3,703 PubMed 19,717 44,338 3 500 Photo 7,650 119,081 8 745 WikiCS 11,701 216,123 10 300 Airport 3,188 18,631 4 4 Figure 5: Impact of negative sample count K. E.2 Further Analysis Impact of negative sample count. We analyze the impact of negative sample count K on model performance across PubMed and Photo datasets. As shown in Figure 5, performance initially improves with increasing K, peaking at K=3K=3 for PubMed and K=4K=4 for Photo. This demonstrates that our inter-node negative sampling strategy effectively enforces geometry-aware discrimination, with optimal regularization achieved using only a modest number of negatives. However, we observe a marginal performance decay as the cardinality exceeds 1010, which can be attributed to the ”false negative” phenomenon where topologically similar nodes are erroneously pushed apart in the embedding space. Overall, GeoMoE maintains robust performance within the range of [3,20][3,20], making it easy to tune in practice.