Paper deep dive
Sheaf-Based Federated Representation Learning
Gabriele D'Acunto, Enrico Grimaldi, Valeria Avino, Mario Edoardo Pandolfo, Leonardo Di Nino, Sergio Barbarossa, Paolo Di Lorenzo
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/13/2026, 3:58:57 AM
Summary
The paper introduces Sheaf-based Federated Representation Learning (SFRL), a framework for heterogeneous federated systems that aligns local latent representations without assuming a shared global latent space. It utilizes a network sheaf structure with learnable orthogonal and Stiefel restriction maps to enforce geometric alignment via a sheaf Laplacian regularizer. The proposed decentralized algorithm, Sheaf-FRL, alternates between local gradient updates and closed-form Procrustes updates for alignment maps, demonstrating superior performance in cooperative classification tasks under model and data heterogeneity.
Entities (6)
Relation Signals (6)
Sheaf-FRL → implements → SFRL
confidence 96% · We develop a decentralized algorithm for solving SFRL, termed Sheaf-FRL
SFRL → uses → Network Sheaf
confidence 95% · we formulate federated representation learning over a learnable network sheaf
SFRL → enforces → Geometric Alignment
confidence 92% · global consistency emerges from the alignment of neighboring latent representations
Sheaf-FRL → uses → Procrustes Updates
confidence 90% · alternates between gradient updates of the local models and closed-form Procrustes updates of the edge-wise restriction maps
Sheaf Laplacian → induces → Quadratic Gluing Regularizer
confidence 88% · This alignment is enforced by a quadratic gluing regularizer induced by the sheaf Laplacian
SFRL → appliedto → Semantic Communication
confidence 85% · As an application, we consider a cooperative classification task in the context of semantic communication
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Heterogeneous federated systems require agents to learn and exchange informative representations despite differences in data distributions, sensing modalities, model architectures, latent dimensionalities, and local learning objectives. To address this challenge, we propose Sheaf-based Federated Representation Learning (SFRL), a general framework that jointly optimizes local objectives with a manifold-constrained geometric alignment regularizer based on learnable sheaf restriction maps. Unlike most existing approaches, SFRL does not assume a shared global latent space. Instead, global consistency emerges from the alignment of neighboring latent representations through orthogonal transformations and isometric embeddings. This alignment is enforced by a quadratic gluing regularizer induced by the sheaf Laplacian, whose learnable restriction maps adapt the geometry to the observed data. The penalty is evaluated on a small set of shared pilot samples, ensuring scalability and communication efficiency. We develop a decentralized algorithm for solving SFRL, termed Sheaf-FRL, which alternates between gradient updates of the local models and closed-form Procrustes updates of the edge-wise restriction maps. We further establish convergence of Sheaf-FRL to first-order stationary points in both deterministic and stochastic settings. As an application, we consider a cooperative classification task in the context of semantic communication, under model and data heterogeneity. Our results show that Sheaf-FRL outperforms baseline approaches in terms of local and post-communication classification accuracy across different levels of local distribution shift and exhibits greater robustness to latent-space dimensionality compression.
Tags
Links
- Source: https://arxiv.org/abs/2608.10016v1
- Canonical: https://arxiv.org/abs/2608.10016v1
Trouble viewing inline? Open PDF directly →
Full Text
140,034 characters extracted from source content.
Expand or collapse full text
Sheaf-Based Federated Representation Learning Gabriele D’Acunto∗ &Enrico Grimaldi∗ &Valeria Avino Mario Edoardo Pandolfo &Leonardo Di Nino &Sergio Barbarossa & Paolo Di Lorenzo Abstract Heterogeneous federated systems require agents to learn and exchange informative representations despite differences in data distributions, sensing modalities, model architectures, latent dimensionalities, and local learning objectives. To address this challenge, we propose Sheaf-based Federated Representation Learning (SFRL), a general framework that jointly optimizes local objectives with a manifold-constrained geometric alignment regularizer based on learnable sheaf restriction maps. Unlike most existing approaches, SFRL does not assume a shared global latent space. Instead, global consistency emerges from the alignment of neighboring latent representations through orthogonal transformations and isometric embeddings. This alignment is enforced by a quadratic gluing regularizer induced by the sheaf Laplacian, whose learnable restriction maps adapt the geometry to the observed data. The penalty is evaluated on a small set of shared pilot samples, ensuring scalability and communication efficiency. We develop a decentralized algorithm for solving SFRL, termed Sheaf-FRL, which alternates between gradient updates of the local models and closed-form Procrustes updates of the edge-wise restriction maps. We further establish convergence of Sheaf-FRL to first-order stationary points in both deterministic and stochastic settings. As an application, we consider a cooperative classification task in the context of semantic communication, under model and data heterogeneity. Our results show that Sheaf-FRL outperforms baseline approaches in terms of local and post-communication classification accuracy across different levels of local distribution shift and exhibits greater robustness to latent-space dimensionality compression. **footnotetext: These authors contributed equally. All authors are with Sapienza University of Rome, Italy. Correspondence to: gabriele.dacunto@uniroma1.it and enrico.grimaldi@uniroma1.it. 1 Introduction Modern machine learning systems increasingly operate in decentralized and heterogeneous environments, where data, models, sensing modalities, and computational resources are distributed across multiple agents, devices, or institutions. In this context, we adopt the term agent to denote any physical or artificial entity performing a task in an environment. Privacy constraints, regulatory requirements, and communication limitations often prevent centralized data aggregation, motivating federated learning (FL), where agents optimize local models on private data while exchanging limited information rather than raw samples (Kairouz et al., 2021). A central challenge in such systems is that agents may learn representations that are useful locally but incompatible across the network. Existing FL methods commonly rely on parameter averaging, global regularization, or shared encoders, and therefore implicitly assume that features learned by different agents are directly comparable (McMahan et al., 2017; Scardapane & Di Lorenzo, 2017; Li et al., 2020b; Karimireddy et al., 2020; Collins et al., 2021; Li et al., 2021; Dong & Voiculescu, 2021). This assumption can fail when agents differ in their data distributions, sensing modalities, architectures, latent dimensions, or local objectives. Independently trained encoders may produce representations that encode equivalent task-relevant information while differing by rotations, reflections, or more general isometric embeddings. Such ambiguity reflects the inherent gauge freedom of representation learning objectives, whereby equivalent solutions may differ by transformations that preserve geometric structure (Wang & Isola, 2020). Consequently, forcing all agents into a single shared latent space can be unnecessarily restrictive and may degrade local representation quality or hinder knowledge transfer. These observations motivate semantic alignment: rather than requiring agents to adopt identical latent coordinates, one should learn how their local representation spaces are related. This perspective is supported by recent work on relative representations, which shows that independently learned latent spaces can support zero-shot communication when their relational structure is preserved (Moschella et al., 2023). Semantic alignment can support heterogeneous federated representation learning across supervised, semi-supervised, and self-supervised settings, as well as cooperative tasks in which agents exploit representations received from their neighbors. For example, it is relevant to multimodal sensing, collaborative robotics, distributed channel charting, transfer across heterogeneous models, and task-aware semantic communication, where latent information generated by one agent must remain useful to another despite differences in their internal representations. A natural mathematical framework for these settings is a cellular sheaf on graph, hereafter referred to as a network sheaf (Curry, 2014). A network sheaf associates a local vector space with each node and linear restriction maps with graph edges, thereby encoding how information in neighboring spaces should be compared or transported. Recent work in graph signal processing and graph neural networks has shown that sheaf-based models can represent vector-valued data coupled through local consistency constraints (Hansen & Ghrist, 2019; Di Nino et al., 2024; 2026; Bodnar et al., 2022; Barbero et al., 2022). This provides a natural abstraction for federated representation learning: each agent maintains its own latent space, while learnable edge-wise maps encode the geometric relations required to align representations across neighboring agents. Global consistency then emerges from local compatibility relations, rather than from a shared latent space. Contributions. We propose a sheaf-based framework for federated representation learning that enables semantic alignment across heterogeneous agents without requiring a shared latent space. Specifically, (i) we formulate federated representation learning over a learnable network sheaf, where agent-specific latent spaces are related through edge-wise geometric transport maps; (i) we introduce a quadratic gluing penalty induced by a learnable sheaf Laplacian, evaluated on a small set of reference representations, hereinafter pilots, to promote alignment while limiting communication; (i) we employ orthogonal and Stiefel restriction maps to capture rotations, reflections, and isometric embeddings across latent spaces of possibly different dimensions; (iv) we develop a fully decentralized alternating algorithm, termed Sheaf-FRL, that combines local gradient updates with closed-form Procrustes updates of the transport maps, and establish convergence to first-order stationary points in deterministic and stochastic settings; and (v) we demonstrate that Sheaf-FRL improves over baseline approaches in learning and representation transfer for collaborative supervised classification in semantic communication, under model and data heterogeneity. The code implementing the proposed method and reproducing all the experiments is publicly available.111https://github.com/SPAICOM/sheaf-based-federated-representation-learning.git 2 Related Work In this section, we review the most relevant literature on federated learning, representation learning, and geometric alignment, and position our work with respect to these directions. Federated learning under heterogeneity. Federated learning (FL) trains a shared model through repeated local stochastic gradient updates followed by either server-side aggregation (McMahan et al., 2017) or neighbor-based combination over a communication graph (Scardapane & Di Lorenzo, 2017). A large body of work studies the impact of statistical and system heterogeneity, which can lead to client drift and unstable convergence, and proposes stabilizing mechanisms such as proximal regularization (FedProx) (Li et al., 2020b) and control variates (SCAFFOLD) (Karimireddy et al., 2020). More broadly, these approaches aim to improve optimization stability while maintaining a single global model. In contrast, we focus on settings where enforcing a shared latent representation is inherently restrictive. Federated multi-task learning. Federated multi-task learning (FMTL) extends FL by training client-specific models while exploiting relationships across tasks or clients (Smith et al., 2017). Early formulations build upon regularized multi-task learning objectives, where task relationships are captured through shared covariance or relational structures (Evgeniou & Pontil, 2004; Evgeniou et al., 2005; Jacob et al., 2008; Zhang & Yeung, 2010; Zhang & Yang, 2017). In the federated setting, this leads to approaches that jointly optimize local models and their coupling structure via alternating or primal-dual procedures (Smith et al., 2017; Jaggi et al., 2014; Ma et al., 2015). A prominent class of methods leverages graph-based regularization, where a predefined or learned graph encodes similarities among clients and enforces smoothness of model parameters across the network. In particular, Laplacian-based approaches penalize discrepancies between neighboring models through total variation terms (Dinh et al., 2022; Issaid et al., 2025). While these methods capture task relatedness, they operate in the parameter space and typically assume homogeneous representations. In contrast, our approach operates in the representation space and models relationships between clients up to learnable transformations, enabling alignment across heterogeneous latent spaces and accommodating different architectures or dimensions. Multimodal Federated Learning. Multimodal Federated Learning (MFL) (Che et al., 2023) extends classical FL to settings where clients possess data from multiple modalities (e.g., text, images, signals), enabling richer representations while preserving privacy. Existing MFL approaches are typically categorized based on data heterogeneity into congruent (shared modalities and feature spaces) and incongruent settings, the latter including vertical, transfer, and hybrid scenarios with partially overlapping or distinct modalities. In general, MFL optimizes a global objective by aggregating client-specific multimodal losses, often defined as weighted combinations of modality-wise contributions. However, most methods assume a shared latent space and rely on centralized or weakly decentralized protocols. Recent graph-based approaches such as CoMFed (Badi et al., 2026) address heterogeneity by aligning compressed latent prototypes across clients, but still require a predefined common embedding space and enforce similarity via projections. In contrast, our sheaf-based framework does not assume a global latent space; instead, it enables decentralized learning in which each client maintains its own representation, and cross-client consistency is enforced through edge-dependent geometric alignments without information loss, naturally accommodating multimodal and heterogeneous settings. Federated Representation Learning. Our problem formulation is closely related Federated Representation Learning (FRL), which aims to learn transferable representations across clients. In the supervised setting, existing approaches either enforce consistency via auxiliary losses on latent embeddings (Li et al., 2021), typically assuming a shared global model, or adopt a split architecture with a common encoder and client-specific heads (Collins et al., 2021; McLaughlin & Su, 2024), still requiring a shared latent dimensionality. More recent methods based on prototype alignment (Tan et al., 2022; Tran et al., 2024) partially address heterogeneity but continue to rely on a common embedding space and often centralized training. In contrast, our framework allows each client to learn its own latent space and enforces consistency in a decentralized manner through structured transport maps, so that classical FRL methods can be interpreted as a special case in which all latent spaces share the same dimensionality and are constrained to coincide via identity alignments. Beyond supervised settings, unsupervised and self-supervised approaches operating on unlabeled data aim to improve representation transferability across clients, often leveraging contrastive learning frameworks (Zhuang et al., 2021; Han et al., 2022; Zhang et al., 2023; Miao & Koyuncu, 2024; Ghalkha et al., 2026). Our proposed framework is not limited to any specific local loss or model architecture; thus, it is also amenable to the latter settings. Representation alignment, synchronization, and manifold methods. Alignment up to orthogonal transformations arises naturally when representations are identifiable only up to rotations or reflections, with the orthogonal Procrustes problem providing a classical closed-form solution (Schönemann, 1966). We leverage this solution in a federated and decentralized setting, both under homogeneous latent spaces (cf. App. D) and under stringent privacy constraints (cf. App. F). Related ideas arise in group synchronization and connection-Laplacian methods (Singer, 2011; Singer & Wu, 2012; Bandeira et al., 2013; Thunberg et al., 2017), which enforce consistency across pairwise transformations; unlike these approaches, where node signals and edge relations are given or observed, possibly with noise, we jointly learn latent representations at the nodes and alignment maps on the edges. To accommodate heterogeneous latent dimensions, we further consider semi-orthogonal maps on the Stiefel manifold, enabling isometric embeddings across representation spaces. This connects our formulation to optimization on matrix manifolds (Absil et al., 2008; Edelman et al., 1998) and federated optimization under manifold constraints (Li & Ma, 2022); however, these works focus on optimizing model parameters subject to geometric constraints, rather than learning alignment maps as a component of representation learning. Positioning of this paper. Overall, our framework generalizes across all paradigms discussed above, which differ mainly in what is assumed to be shared among agents—a single global model, a coupled set of parameters, a common multimodal embedding, or a shared latent space—rather than in the underlying goal of exploiting relatedness across agents. Indeed, methods introduced within one paradigm, such as prototype alignment in FRL or graph-based regularization in FMTL, are often applicable to the others as well. Rather than enforcing any of these forms of sharing, we learn edge-wise geometric transport maps that relate neighboring latent spaces and promote their semantic compatibility. In the homogeneous setting, our formulation recovers the single global model of classical FL as a limiting case, together with orthogonal alignment, synchronization, and connection-Laplacian models. More generally, it extends graph-based FMTL and Laplacian regularization through a learnable sheaf Laplacian acting on latent representations rather than model parameters. Additionally, it relaxes the shared, predefined embedding space assumed by MFL approaches such as CoMFed, and recovers FRL methods as the special case in which all latent spaces coincide via identity alignments. Finally, pilot-restricted gluing enables communication-efficient alignment, while manifold-constrained transport maps admit decentralized optimization through local model updates and closed-form edge-wise Procrustes steps. 3 Our Sheaf-based Representation Learning Framework Figure 1: An illustration of our sheaf-based representation learning framework. In this example, we consider a supervised setting and equip local agents with task-specific heads. Fig. 1 illustrates the proposed sheaf-based representation learning framework. Let =(,ℰ)G=(N,E) denote a finite undirected graph, where ||=N|N|=N is the number of nodes. Each node corresponds to an agent, while edges encode structural relationships among agents. For example, an edge (i,j)∈ℰ(i,j) may capture similarities in tasks, data distributions, modalities, or model architectures, thereby enabling beneficial cooperation between agents i and j through a physical communication channel. Local agents. Each agent i∈i observes a local dataset iT_i comprising MiM_i samples in ℝpiR^p_i, drawn from an underlying distribution PiP_i, as illustrated in Fig. 1. We denote the n-th sample by inx_i^n, with n∈[Mi]n∈[M_i]. Depending on the local learning paradigm, iT_i may additionally include labels, side information, paired observations, or locally generated views of the samples. Each agent learns a local representation model comprising a neural encoder fi:ℝpi→ℱ(i),f_ _i:R^p_i (i)\,, (1) where i _i is the set of learnable parameters, and ℱ(i)≅ℝdiF(i) ^d_i is the latent representation space of agent i. When required by the local task, the encoder is followed by a personalized head parametrized by i ψ_i, i.e., hi:ℱ(i)→i,h_ ψ_i:F(i) _i\,, (2) where iY_i is an agent-dependent output space. We let i θ_i denote the collection of local trainable parameters, including i _i and, when present, i ψ_i. Each agent i∈i aims at minimizing an arbitrary local learning objective ℒi(i)L_i( θ_i) based on the dataset iT_i. For example, in the supervised setting, ℒi(i)=1Mi∑n=1Miℓi(hi∘fi(in)).L_i( θ_i)= 1M_i _n=1^M_i _i\! (h_ ψ_i f_ _i(x_i^n) )\,. (3) Other choices naturally accommodate semi-supervised, unsupervised, or self-supervised learning. Communication over the learnable latent network sheaf. To formalize the relationships, and thus the communication, among the latent representation spaces ℱ(i)F(i) introduced above, we adopt the framework of network sheaves (Bredon, 1997; Curry, 2014). Intuitively, those ℱ(i)F(i) spaces will be structured so as to encode consistency relations between the latent representations of neighboring agents. Specifically, a network sheaf ℱF on G, valued in the category of finite-dimensional real vector spaces and linear maps, consists of the following assignments. To each node i∈i , it associates a did_i-dimensional real vector space ℱ(i)F(i), being the learnable latent representation space of agent i and referred to as node stalk. In this work we consider the general case where node stalks ℱ(i)F(i), i∈i , have different dimensionality did_i. For instance, in Fig. 1, the vector space ℱ(i)F(i) is three-dimensional, while the vector space ℱ(j)F(j) is two-dimensional. For each node stalk, the valuation corresponds to a latent embedding, e.g., those vectors i=fi(i)z_i=f_ _i(x_i) and j=fj(j)z_j=f_ _j(x_j) in Fig. 1. The collection of latent representations across the network, viz. =ii∈z=\z_i\_i , forms a 0-cochain. Accordingly, the 0-cochain space is C0(,ℱ)=⨁i∈ℱ(i)C^0(G,F)= _i F(i). Similarly, to each edge eij=(i,j)∈ℰe_ij=(i,j) , the network sheaf ℱF assigns a dijd_ij-dimensional real vector space ℱ(eij)F(e_ij), representing a shared space encoding pairwise compatibility between agents i and j. The latter space is called edge stalk. The 11-cochain is a collection =eijeij∈ℰy=\y_e_ij\_e_ij with eij∈ℱ(eij)y_e_ij (e_ij), and the corresponding 11-cochain space is C1(,ℱ)=⨁eij∈ℰℱ(eij)C^1(G,F)= _e_ij F(e_ij). Additionally, for each incidence relation i→eiji→ e_ij, where i is a vertex of eije_ij, ℱF specifies a linear restriction map ℱi→eij:ℱ(i)→ℱ(eij),F_i→ e_ij:F(i) (e_ij)\,, which relates the local latent representation at node i to the corresponding edge space and enables the communication among latent spaces. In this work, we focus on learnable restriction maps suitable for dealing with the well-known geometric-invariance of latent representations (Moschella et al., 2023), and that preserve distances and angles. These maps are orthogonal transformations and Stiefel matrices, where the orthogonal and Stiefel manifolds are O(d)≔∈ℝd×d∣⊤=−1,andSt(d,k)≔∈ℝd×k∣⊤=k,k<d.O(d) \O\!∈\!R^d× d\, \,O \!=\!O^-1\\,, (d,k) \V\!∈\!R^d× k\, \,V V\!=\!I_k,\,k<d\\,. (4) As we consider structured restriction maps that preserve the information but do not increase latent perturbations, for each edge eij=(i,j)∈ℰe_ij=(i,j) with di>djd_i>d_j, we set ℱ(eij)=ℱ(i)≅ℝdiF(e_ij)=F(i) ^d_i, as illustrated in Fig. 1. This implies ℱi→eij=ji∈O(di)F_i→ e_ij\!=\!O_ji (d_i) and ℱj→eij=ij∈St(di,dj)F_j→ e_ij\!=\!V_ij\!∈\!St(d_i,d_j). Further discussion on the benefits of choosing the edge stalk dimensionality dij=max(di,dj)d_ij= (d_i,d_j), as opposed to compressing, is provided in App. A. Heterogeneous latent space geometry. In this setting, the encoders fif_ _i are trained independently, on different data, and possibly with different architectures. Then, it is reasonable to expect the induced latent spaces ℱ(i)F(i) to exhibit heterogeneous geometry, in the sense of different scales and anisotropies (Pandolfo et al., 2026; Svendsen et al., 2026). This may occur even when dimensionalities did_i coincide. However, the above geometric restriction maps are effective for measuring inconsistency only when the latent representations iz_i and jz_j adhere to a common geometric reference on the edge stalk. Thus, to make representations from distinct node stalks comparable once transported, each encoder is assumed to include a final whitening layer (Zhang et al., 2021) that approximately normalizes the latent representations to zero mean and identity covariance. This normalization reduces the node-specific geometry to a common Euclidean reference. However, the geometry of ℱ(i)F(i) is key to minimizing the local learning objective ℒi(i)L_i( θ_i). Hence, to let the local head exploit the local geometry, we equip the latter with an initial coloring layer that is assumed to perform the left inverse map of the normalization applied by the whitening layer of the encoder. Centralized formulation under global alignment. The sheaf induces a coboundary operator :C0(;ℱ)→C1(;ℱ) :C^0(G;F)→ C^1(G;F) defined edgewise as ()eij=jii−ijj,eij=(i,j)∈ℰ.( )_e_ij=O_jiz_i-V_ijz_j, e_ij=(i,j) . (5) Equipping C0C^0 and C1C^1 with canonical Euclidean inner products ⟨,′⟩=∑i∈i⊤i′ ,z = _i z_i z_i and ⟨,′⟩=∑eij∈ℰeij⊤eij′ ,y = _e_ij y_e_ij y_e_ij , the adjoint ⊤ is well defined. The associated network sheaf Laplacian is ℱ=⊤,L_F= \,, (6) which is symmetric positive semidefinite. A valuation ⋆∈C0(;ℱ)z ∈ C^0(G;F) belonging to ker(ℱ) (L_F) is said global section and it can be thought of as an assignment of latent representations to the node stalks that does not break local rules. Indeed, from Eq. 6 it is clear that ⋆= z =0 which implies jii⋆=ijj⋆for each eij∈ℰ;O_jiz_i =V_ijz_j \, each e_ij \,; (7) where i⋆∈ℝdiz_i ^d_i and j⋆∈ℝdjz_j ^d_j are the components of ⋆z at node i and j, respectively. Thus, a plausible strategy would be to constrain the learned latent representations across the network to be globally consistent with respect to the learned network sheaf ℱF. For simplicity, assume the existence of a set of shared sample indices across agents, and let M≤mini∈MiM≤ _i M_i be the number of such comparable samples. Then, according to the previous rationale we can pose the following centralized problem formulation mini=(i,i)i∈ji∈O(di),ij∈St(di,dj)eij∈ℰ _ subarrayc\ θ_i=( _i, ψ_i)\_i \\ \O_ji (d_i),\,V_ij (d_i,d_j)\_e_ij subarray ∑i∈ℒi(i) _i L_i( θ_i) (8) s.t. n=fi(in)i∈ker(ℱ),∀n∈[M]; ^n=\f_ _i(x_i^n)\_i ∈ (L_F), ∀ n∈[M]\,; where the network sheaf ℱF is learned through its restriction maps under a known communication topology. However, the constraints in Prob. (8) might be too restrictive in practice. Enforcing exact compatibility across heterogeneous agents is generally unrealistic, and, moreover, satisfying the constraint for all samples would in principle require a centralized solution, as it couples all node representations simultaneously. This limitation motivates the need for a more flexible and decentralized formulation that can accommodate inconsistencies while still promoting coherence across agents. 4 Towards a Scalable and Decentralized Formulation The discussed limitations of Prob. (8) in realistic settings motivate the introduction of a relaxed, distributed formulation. To obtain a tractable formulation suitable for distributed implementation, we replace the hard constraints in Prob. (8) with a soft penalty based on the sheaf total variation, which promotes geometric consistency across agents without enforcing exact agreement: ()≔‖‖22=⊤ℱ=∑eij∈ℰ‖jii−ijj‖22.TV(z) z_2^2=z L_Fz= _e_ij O_jiz_i-V_ijz_j_2^2\,. (9) From Eq. 7, it is easy to see that constraining the learned z to be a global section is equivalent to zeroing Eq. 9. Additional discussion on the geometric consistency promoted by the action of the restriction maps, as well as the use of the total variation penalty, is provided in App. A. Semantic embedding and orientation. We can recast the local terms in Eq. 9 as ‖jii−ijj‖22=(a)‖i−ijj‖22; O_jiz_i-V_ijz_j_2^2 (a)= z_i-V_ijz_j_2^2\,; (10) where in (a)(a) we exploit the transitive action of the orthogonal group on the Stiefel manifold, thus re-parameterizing accordingly ij=ji⊤ijV_ij=O_ji V_ij. Eq. 10 highlights that perfect alignment, and thus null total variation, corresponds to the case in which the higher-dimensional latent space is an isometric embedding of a lower-dimensional latent space. While in general this condition does not always hold (see the discussion in App. A on geometric consistency), the restriction maps in our work adheres to the semantic embedding principle (D’Acunto et al., 2025; 2026): informally, from a probabilistic perspective, the semantics of the lower-dimensional node stalk ℱ(j)F(j) is preserved when embedded into the higher-dimensional ℱ(i)F(i). Another distinguishing feature of our framework is that the restriction map reparameterization in Eq. 10 induces a natural orientation for G which we exploit in the sequel to reduce the communication cost. Specifically, as exemplified in Fig. 1, we endow G with the embedding orientation, that is, from nodes with lower-dimensional latent spaces to those with higher-dimensional ones, accordingly to the direction in which semantics is preserved. Further details on the special case of latent representation spaces with homogeneous dimensionality are provided in App. A. Incoming and outgoing embedding maps. By using Eq. 10, the total variation in Eq. 9 can be rewritten as ()=∑eij∈ℰ‖i−ijj‖22;TV(z)= _e_ij z_i-V_ijz_j_2^2\,; (11) where for the edge eij=(i,j)e_ij=(i,j), node i is the head and j the tail, in accordance with the relation di>djd_i>d_j. Ties occurring when di=dj=d_i=d_j=d are instead broken arbitrarily: since in this case ij=ji⊤∈O(d)V_ij=V_ji (d), the edge contribution in Eq. 11 is invariant to the chosen orientation. Throughout, we therefore adopt the convention that for eije_ij, the first index i is always the head. Now, exploiting the orientation fixed above, for each i∈i , we can distinguish between two sets of neighbors. Specifically, (i)−N(i)^- collects the neighbors j for which i is the head, and (i)+N(i)^+ those for which i is the tail, so that dj≤did_j≤ d_i on (i)−N(i)^- and dj≥did_j≥ d_i on (i)+N(i)^+, with equality holding only on tied edges. The overall neighborhood is thus (i)=(i)−∪(i)+N(i)=N(i)^- (i)^+. Taking advantage of this distinction, starting from Eq. 11, we have () (z) =∑i∈∑j∈(i)−‖i−ijj‖22 = _i _j (i)^- z_i-V_ijz_j_2^2 (12) =12∑i∈[∑j∈(i)−‖i−ijj‖22+∑j∈(i)+‖j−jii‖22]; = 12 _i [ _j (i)^- z_i-V_ijz_j_2^2+ _j (i)^+ z_j-V_jiz_i_2^2 ]\,; which highlights that the contribution of node i to the total variation is made of two terms with different geometric meanings: ()|i=12∑j∈(i)−‖i−ijj‖22⏟Incoming Embedding+12∑j∈(i)+‖j−jii‖22⏟Outgoing Embedding.TV(z)|_i= 12 _j (i)^- z_i-V_ijz_j_2^2_Incoming Embedding+ 12 _j (i)^+ z_j-V_jiz_i_2^2_Outgoing Embedding\,. (13) Leveraging Eq. 13, for each node i, the matrices ijV_ij in the first term are referred to as the incoming embedding maps, while the matrices jiV_ji in the second term are referred to as the outgoing embedding maps. As detailed in the sequel, the expression in Eq. 13 is useful for computing local updates and reducing the communication cost in our proposed decentralized algorithm. Scalable gluing penalty and decentralized problem. For scalability and computational aspects, to enforce geometric consistency via the sheaf total variation, we only use a subset of reference samples chosen among those inz_i^n (pilots), for each i∈i . Let ⊂1,…,Mmin,||=K≪Mmin,A⊂\1,…,M_min\, |A|=K M_min, denote fixed pilot indices, where Mmin=miniMiM_min= _iM_i. For the sake of exposition, we assume the datasets i\T_i\ are aligned. For each agent i, define the pilot feature matrix i(i)=[fi(ik)]k∈=[ik]k∈ℝdi×K.A_i( _i)= [f_ _i (x_i^k ) ]_k = [z_i^k ]_k ^d_i× K. (14) Different strategies can be used to select the pilot set A, trading off semantic reliability, geometric coverage, and communication efficiency; we discuss several of such strategies in App. B. The usage of pilots also highlights that we enforce a weaker notion of alignment among latent spaces. In other words, while a global agreement may occur on these pilots, the local latent representations for the remaining samples may not, in principle, be perfectly alignable via the learned restriction maps. This is consistent with the discussion above, particularly with the fact that we neither assume nor aim for a single global latent representation. Now, exploiting Eq. 12 and introducing a penalty hyperparamer λ>0λ>0, our considered gluing penalty decouples as ℛ(i,ij,ji)=∑i=1Nℛ|i(i,ij,ji);R_A(\ _i\,\V_ij\,\V_ji\)= _i=1^NR_A|_i( _i,\V_ij\,\V_ji\)\,; (15) where ℛ|i(i,ij,ji) _A|_i( _i,\V_ij\,\V_ji\) =λi2K[∑j∈(i)−‖i(i)−ijj(j)‖F2 = _i2K [ _j (i)^- A_i( _i)-V_ijA_j( _j)_F^2 (16) +∑j∈(i)+‖j(j)−jii(i)‖F2]; + _j (i)^+ A_j( _j)-V_jiA_i( _i)_F^2 ]\,; with λi=λ/di _i=λ/d_i and λ>0λ>0. At this point, we are ready to pose the sheaf-based federated representation learning problem. Problem 1 (Sheaf-based Federated Representation Learning). Given (i) an undirected graph =(,ℰ)G=(N,E) endowed with an orientation induced by semantic embeddings, (i) a collection of N local training datasets ii∈\T_i\_i drawn from unknown distributions PiP_i, and (i) a set of reference pilot indices A, the goal of sheaf-based federated representation learning (SFRL) is to learn, for each node i, a local model i=(i,i) θ_i=( _i, ψ_i) together with incoming and outgoing embeddings ij∈St(di,dj)(i,j)∈ℰ\V_ij (d_i,d_j)\_(i,j) and ji∈St(dj,di)(j,i)∈ℰ\V_ji (d_j,d_i)\_(j,i) , respectively. These embedding matrices align the local latent space ℱ(i)F(i), using reference pilots indexed by A, with those of the neighbors in (i)N(i). This is achieved by solving in a decentralized manner mini=(i,i)ij∈St(di,dj)ji∈St(dj,di)∑i∈ℒi(i)+ℛ(i,ij,ji); _ subarrayc\ θ_i=( _i, ψ_i)\\\ \V_ij (d_i,d_j)\\\ \V_ji (d_j,d_i)\ subarray _i L_i( θ_i)+R_A( _i,\V_ij\,\V_ji\)\,; (SFRL) while preserving agents’ privacy and reducing communication costs over the network. Prob. (SFRL) is a regularized nonconvex optimization problem, which couples smooth local neural parameters optimization with manifold-constrained edge transports. It can be interpreted as a scalable decentralized relaxation of the centralized Prob. (8). 5 The Sheaf-FRL Algorithm We solve Prob. (SFRL) in a decentralized manner, by adopting an alternating minimization strategy. The latter consists of the following two updates. 5.1 Step 1: Isometric Embedding Update Recall that (i)⊆∖iN(i) \i\ denotes the set of neighbors of node i, which splits into (i)−N(i)^- and (i)+N(i)^+, collecting the neighbors j for which i is the head and the tail, respectively. Given the neural parameters it=(it,it)\ θ_i^t=( _i^t, ψ_i^t)\ updated at communication round t, each node i∈i updates its alignment map with its neighbors j∈(i)j (i). Let it=i(it)=[fit(ik)]k∈A_i^t=A_i ( _i^t )= [f_ ^t_i (x^k_i ) ]_k denote the pilot feature matrix at communication round t. Since the local gluing penalty in Eq. 16 separates over the oriented edges, each node i can update its alignment maps by solving local problems involving only its neighbors j∈(i)j (i). In particular, for each eij∈ℰe_ij , where i is the head and j the tail, node i optimizes the corresponding incoming and outgoing embedding maps associated with its neighborhood. This leads to two symmetric alignment problems corresponding to the incoming and outgoing embedding terms. Specifically, we consider ijt _ij^t =argminij∈St(di,dj)‖it−ijjt‖F2,∀j∈(i)−, = *arg\,min_V_ij (d_i,d_j)\;\; A_i^t-V_ijA_j^t_F^2\,, ∀\,j (i)^-\,, (P2a) jit _ji^t =argminji∈St(dj,di)‖jt−jiit‖F2,∀j∈(i)+. = *arg\,min_V_ji (d_j,d_i)\;\; A_j^t-V_jiA_i^t_F^2\,, ∀\,j (i)^+\,. (P2b) The objective of Prob. (P2a) (and analogously of (P2b)) can be recast as: ‖it−ijjt‖F2=tr(itit⊤)+tr(jtjt⊤)−2tr(itjt⊤ij⊤). A_i^t-V_ijA_j^t_F^2= A_i^tA_i^t + A_j^tA_j^t -2 A_i^tA_j^t V_ij \,. (17) Minimizing Eq. 17 thus corresponds to maximizing the third term. Denoting by ⊤U W the thin SVD of the cross-covariance term itjt⊤A_i^tA_j^t , the solution to (P2a) is given by ijt=⊤,where∈St(di,dj) and ∈O(dj).V_ij^t=UW \,, (d_i,d_j) and W (d_j)\,. (18) Similarly, the solution to (P2b) is obtained from the thin SVD of jtit⊤A_j^tA_i^t . For each node i∈i , the updates in Eq. 18 are computed locally once its neighbors j∈(i)j (i) share their feature evaluations jt\A_j^t\. Importantly, the orientation is exploited by i to distinguish the edges for which it acts as the head, in which case the incoming embedding map ijtV_ij^t must be computed via (P2a), from those for which it acts as the tail, where the outgoing embedding map jitV_ji^t is updated via (P2b). Remark 1. In the homogeneous case, as well as on tied edges of the heterogeneous setting, St(di,dj)St(d_i,d_j) coincides with O(d)O(d) and Probs. (P2a)–(P2b) reduce to canonical orthogonal Procrustes problems; accordingly, the thin SVD reduces to the full SVD, so that tied edges require no separate treatment. The solutions are attained in closed form by setting ijt=⊤O_ij^t=UW and jit=ijt⊤O_ji^t=O_ij^t , where the factors are obtained from the classical SVD of the cross-covariance matrix itjt⊤A_i^tA_j^t . Conditions ensuring identifiability of the transport maps, both in the homogeneous and heterogeneous settings, are discussed in App. C. 5.2 Step 2: Gradient Update of Neural Parameters In Prob. (SFRL), each agent i updates its local parameters i θ_i through a gradient step that minimizes the global objective with respect to i θ_i, while keeping the neighboring parameters jtj∈(i)\ θ_j^t\_j (i) fixed at the current iteration. The regularization term, instead, depends exclusively on the encoder parameters i _i. Therefore, starting from Eq. 16, for each agent i∈i we obtain ∇iℛ|i= _ _iR_A|_i= λiK[∑j∈(i)−∑k∈(∇ifi(ik))⊤(fi(ik)−ijfj(jk)) _iK [ _j (i)^- _k ( _ _if_ _i(x_i^k) ) (f_ _i (x_i^k )-V_ijf_ _j (x_j^k ) ) (19) −∑j∈(i)+∑k∈(∇ifi(ik))⊤ji⊤(fj(jk)−jifi(ik))] - _j (i)^+ _k ( _ _if_ _i(x_i^k) ) V_ji (f_ _j (x_j^k )-V_jif_ _i (x_i^k ) ) ] =(a) (a)= λiK∑j∈(i)∑k∈(∇ifi(ik))⊤(fi(ik)−ijfj(jk)); _iK _j (i) _k ( _ _if_ _i(x_i^k) ) (f_ _i (x_i^k )-V_ijf_ _j (x_j^k ) )\,; where ∇ifi(ik)∈ℝ|i|×di _ _if_ _i(x_i^k) ^| _i|× d_i denotes the Jacobian of the feature map fif_ _i with respect to the parameter vector i _i, evaluated at ikz_i^k; and in (a)(a), for j∈(i)+j (i)^+, we used that ij=ji⊤V_ij=V_ji and ijji=diV_ijV_ji=I_d_i, which follow from the orthonormality of the columns of ji∈St(dj,di)V_ji (d_j,d_i). This shows that both incoming and outgoing contributions admit a unified expression. Furthermore, let i=[∇iℛ|i|i|]r_i= bmatrix _ _iR_A|_i\\ 0_| ψ_i| bmatrix (20) be the contribution of the sheaf-based regularization to the gradient of the local neural parameters i θ_i. Then, for each agent i∈i , the neural parameters are updated locally via it+1=it−ηθ(∇iℒi(i)+it), θ_i^t+1= θ_i^t- _θ ( _ θ_iL_i( θ_i)+r^t_i ), (21) where ηθ>0 _θ>0 is the stepsize, and itr^t_i is the gradient term in (20) evaluated at time t. Gradient computations in (21) are fully decentralized. Specifically, each agent i can evaluate the gradient with respect to its local parameters i θ_i using only locally available information together with the pilots’ latent representations jtj∈(i)\A_j^t\_j (i) received from its neighboring agents. Since these pilot representations are already exchanged during Step 1 to update the restriction maps, no additional communication is required to evaluate the regularization gradient in (19). The resulting decentralized algorithm, termed Sheaf-FRL, is summarized in Appendix D for both the heterogeneous and homogeneous settings. A detailed convergence analysis, covering both deterministic and stochastic settings, is presented in Appendix E. Finally, the proposed framework naturally accommodates accelerated first-order methods: momentum-based and adaptive schemes, such as Nesterov acceleration or Adam, can be directly incorporated into the local parameter updates (21) without affecting the decentralized structure of the algorithm. Remark 2 (Communication cost). At each iteration, every node i broadcasts its pilot feature matrix it∈ℝdi×KA_i^t ^d_i× K to its neighbors in (i)N(i), transmitting (diK)O(d_iK) scalar values. Therefore, the overall communication volume per iteration scales as (∑i∈diK)O\! ( _i d_iK ). Importantly, only pilot representations are exchanged, while model parameters remain local. Remark 3 (Privacy). In some settings, sharing latent representations may raise privacy concerns, as they can be vulnerable to reconstruction and inference attacks, including model inversion (Fredrikson et al., 2015), membership inference (Shokri et al., 2017), and attribute inference (Melis et al., 2019). This can be mitigated in our framework by exchanging only representations transformed via the restriction maps, thus avoiding direct exposure of local latent vectors. Nevertheless, the cost to pay to enforce strict privacy requirements is additional local computation and higher communication cost. The trade-off between privacy, communication, and local computational cost is discussed in App. F. Remark 4 (Local computational cost of restriction maps update). In principle, the number of alignment updates could be halved, since for each edge eije_ij the isometric embedding is only required at the lower-dimensional node. However, this would require each node i to transmit its transformed embedded representations jiitV_jiA_i^t, whose dimension is at least did_i, in order to evaluate Eq. 19. This would increase the communication cost. To preserve communication efficiency (cf. Prob. 1), we instead exchange the raw pilot features itA_i^t and let both endpoints compute the corresponding restriction maps locally. This design shifts the burden from communication to computation, which is well aligned with the increasing computational capabilities of modern edge devices. In the homogeneous case, where di=d_i=d for all i∈i , this trade-off can be further improved. In this setting, sharing transported representations jiitO_jiA_i^t does not increase communication, and the number of SVD computations can be halved, as detailed in App. D and Alg. 2. 6 Collaborative supervised classification with semantic communication As an application, we test our proposed Sheaf-FRL algorithm in a collaborative and decentralized classification task in the setting of semantic communication, where agents exchange compressed latent representations of data (Gündüz et al., 2022; Barbarossa et al., 2023; Strinati et al., 2024; Pandolfo et al., 2025; Grimaldi et al., 2025). Each agent minimizes a local cross-entropy loss ℒi(i)≔CE(i)L_i( θ_i) ( θ_i), while the representation alignment across agents is promoted via the sheaf regularization term. At a high level, the goal for the agents is to improve their classification accuracy on both (i) their private local latent representations, and (i) the latent representations received by their corresponding neighboring agents j∈(i)j (i). Accordingly, as customary in this application setting, we monitor average private and average communication accuracies. Denote by itestT_i^test the test set for agent i consisting of MitestM_i^test pairs (itest,n,yitest,n)n∈Mitest\(x_i^test,n,y_i^test,n)\_n∈ M_i^test. Then, Aipriv A_i^priv =1Mitest∑n=1Mitest[argmaxc∈[hi∘fi(itest,n)]c=yitest,n] = 1M_i^test _n=1^M_i^test 1[ _c [h_ ψ_i [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0f_ _i(x_i^test,n)]_c=y_i^test,n]\, (22) Aicomm A_i^comm =1|(i)|(∑j∈(i)−1Mjtest∑n=1Mjtest[argmaxc∈[hi∘ij∘fj(jtest,n)]c=yjtest,n]+ = 1|N(i)| ( _j (i)^- 1M_j^test _n=1^M_j^test 1[ _c [h_ ψ_i _ij [rgb]0,.5,.5 [named]pgfstrokecolorrgb0,.5,.5f_ _j(x_j^test,n)]_c=y_j^test,n]+ ∑j∈(i)+1Mjtest∑n=1Mjtest[argmaxc∈[hi∘ji⊤∘fj(jtest,n)]c=yjtest,n]). _j (i)^+ 1M_j^test _n=1^M_j^test 1[ _c [h_ ψ_i _ji [rgb]0,.5,.5 [named]pgfstrokecolorrgb0,.5,.5f_ _j(x_j^test,n)]_c=y_j^test,n] )\,. Model heterogeneity. We consider N=15N=15 agents, each consisting of (i) a CNN local encoder and (i) an MLP local decoder. This choice of qualitative architectural similarity is driven by the task homogeneity of the application setting. Heterogeneity across models is enforced by varying architectural characteristics (encoder and decoder widths) and the dropout hyper-parameter (cf. Tab. 2). Data heterogeneity. We use MNIST as a global dataset T with label space =0,…,9Y=\0,…,9\. We first set aside a global pilot set A, a uniformly sampled 10%10\% subset of T, held out before any agent-level partitioning; A is shared, unmodified, across all agents and used exclusively to align/communicate their representations. The remaining 90%90\% is partitioned disjointly across the N agents to create heterogeneity: each agent i has a target class set i⊂C_i , and its local label distribution is a convex combination of two uniform distributions, the first over its target classes and the second over all classes, viz. Pi(y)≔sUnifi+(1−s)Unif,∀i∈[N];P_i(y) s\,Unif\C_i\+(1-s)\,Unif\Y\\,, ∀ i∈[N]\,; (23) where s∈[0,1]s∈[0,1] is the distribution-shift strength. Samples of each class are split across agents in proportion to Pi(y)i=1N\P_i(y)\_i=1^N. Each agent’s local pool is then split 80%/10%/10%80\%/10\%/10\% into train/val/test, preserving its label skew across all three splits. Network topology. Agents communicate over an undirected, unweighted graph ≔(,ℰ)G (N,E), with density level 40%40\%, built to be connected and to reflect the degree of class overlap across agents, as described below. Each agent i∈[N]i∈[N] is assigned to a distinct node, so that ||=N|N|=N. The edge set ℰE is constructed from the target-class sets ii∈[N]\C_i\_i∈[N] as follows. For every pair (i,j)(i,j), let wij≔|i∩j|w_ij |C_i _j| denote the number of target classes shared by agents i and j, collected into the weight vector w. These weights define the fully-connected, weighted graph ~≔(,ℰ~,) G (N, E,w), with ℰ~≔(i,j)∈[N]×[N]∣i≠j E \(i,j)∈[N]×[N]\, \,i≠ j\. We then extract the maximum-weight spanning tree ^≔(,ℰ^,^) G (N, E, w) from ~ G, and initialize ℰE with the (unweighted) edges of ℰ E, which guarantees connectivity. We next add edges from ℰ~∖ℰ E E, in decreasing order of weight, until the target 40%40\% density is reached. Finally, each edge is assigned an orientation, consistent with the orientation rule in Sec. 4. Baselines. This heterogeneous setup precludes direct comparison with classical federated methods based on model or submodel averaging (Scardapane & Di Lorenzo, 2017; Diao et al., 2021; Li et al., 2020a), or shared representations (Tan et al., 2022; Setayesh et al., 2026; Collins et al., 2021; Liang et al., 2020). Indeed, these methods assume homogeneous architectures or shared latent spaces. We therefore compare against methods explicitly designed for heterogeneous settings, namely ComFed (Badi et al., 2026) and Sheaf-FMTL (Issaid et al., 2025). We also consider a non-cooperative baseline in which models are trained independently and their latent spaces are aligned only after training, providing a natural lower bound. All baselines use the same communication network topology as Sheaf-FRL. Except for ComFed, which learns alignment maps during training, all baselines require post-hoc alignment to enable semantic communication. Specifically, for each edge (i,j)(i,j) we solve (P2a) and (P2b) in closed-form via Eq. 18, fitting isometric maps between whitened latent spaces using the same pilot sets as in Sheaf-FRL. In our experiments, the whitening for these baselines is pursued through classical zero-phase component analysis (Kessy et al., 2018) fitted on the latent representations from the private training set, independently for each agent. Figure 2: Average private accuracy (left) and average communication accuracy (right) against the distribution-shift strength s, for the collaborative supervised classification task with 1515 agents. Markers denote the simple average across agents for the private accuracy, and the degree-weighted average for the communication accuracy, to account for the imbalance in the agents’ degree distribution. Results. Fig. 2 shows the metrics in Eq. 22 against the distribution-shift parameter s, which controls the level of data heterogeneity according to Eq. 23. Overall, Sheaf-FRL consistently outperforms all baselines on both metrics. Looking at the average communication accuracy, the performance gap widens as s increases, suggesting that our framework is more effective at learning latent spaces that are (i) well aligned across agents, thus enhancing transferability, and (i) more relevant to the downstream task, thus being semantically meaningful. Point (i) is further supported by the average private accuracy, where Sheaf-FRL still compares favorably with the baselines, although the margin is less pronounced. Among the baselines, ComFed performs poorly in terms of average communication accuracy, while being comparable to the others in private accuracy. This suggests that the joint learning of general, unstructured alignment maps and compressed representations of a single shared latent space is not well suited for transferability. Sheaf-FMTL, in turn, shows no consistent improvement in communication accuracy over the non-cooperative baseline. This stems from a limitation in how it parameterizes the restriction maps: since Sheaf-FMTL connects parameter spaces rather than latent representation spaces, it inherits their much higher dimensionality. The resulting memory overhead requires applying a high compression factor to the edge stalks, which negatively affects the diffusion of information across the network (Appendix G for further details). 6.1 Robustness to semantic compression An important feature in semantic communication is robustness to latent-space compression (Gündüz et al., 2022; Barbarossa et al., 2023). This can be understood as the ability of methods to extract semantically meaningful information to be communicated. To investigate this aspect, we restrict our focus to two agents, say i and j, connected by an edge (i,j)(i,j), and set the level of distribution shift to s=0.7s=0.7. Then, we monitor the two metrics in Eq. 22 across different bottleneck dimensions. Specifically, although the overall architectures of i and j are different (Appendix G for further details), they share the same latent space dimensionality, d=di=djd=d_i=d_j. Consequently, in this case, we can include among the baselines those methods suitable for heterogeneous architectures sharing the same latent space dimensionality. Specifically, we consider the FedProto (Tan et al., 2022) and FedMuscle (Setayesh et al., 2026) methods, although their communication protocols are centralized and therefore not directly applicable to a decentralized setting. Figure 3: Average private accuracy (left) and average communication accuracy (right) against the bottleneck dimension at distribution-shift strength s=0.7s=0.7, for the robustness-to-compression case study. Markers denote the simple average across the two agents. Results. Fig. 3 shows the results for all the considered methods. Overall, Sheaf-FRL achieves a better trade-off between average classification accuracy (private and communication) and bottleneck dimensionality than the baselines, with the performance gap increasing as the bottleneck dimension decreases, i.e., in the high-compression regime. As expected, all methods suffer as the compression increases; however, the performance of Sheaf-FRL degrades much more gracefully. Notably, it preserves high accuracy even at a bottleneck dimension of 1616, where we approach the minimum number of dimensions required by the network to discriminate the 1010 MNIST classes. With respect to the baselines, their performance is comparable to that of the non-cooperative method, occasionally showing even lower communication accuracy despite employing the same post-training alignment strategy. The results confirm the poor transferability of the representations learned under these collaboration schemes: an agent’s latent space is not readily reusable by another, regardless of whether a post-training alignment pipeline is applied. Moreover, in terms of average communication accuracy, ComFed achieves the lowest performance among the considered methods despite its learnable alignment maps, consistent with the results in Fig. 2. 7 Conclusions and Future Work We introduced Sheaf-based Federated Representation Learning, a framework that enables semantic alignment among heterogeneous agents without imposing a shared global latent space. By modeling agent-specific representations as sections of a learnable network sheaf and enforcing consistency through a quadratic gluing penalty induced by the sheaf Laplacian, SFRL accommodates differences in data distributions, sensing modalities, architectures, and latent dimensionalities, while remaining compatible with supervised, semi-supervised, and self-supervised learning paradigms. We developed Sheaf-FRL, a fully decentralized alternating algorithm that couples local gradient updates with closed-form Procrustes updates of the edge-wise restriction maps, and established its convergence to first-order stationary points in both deterministic and stochastic settings. Applied to a cooperative classification task in semantic communication under model and data heterogeneity, Sheaf-FRL outperformed existing baselines in terms of local and post-communication classification accuracy across different levels of distribution shift. Additionally, it proved to be more robust to latent-space compression. Notably, this is achieved while evaluating the gluing penalty on only a small set of shared pilots, keeping communication overhead low. These results support the sheaf-theoretic perspective as a principled and flexible foundation for representation alignment in heterogeneous federated systems. Future work includes extending the application of SFRL to multi-modal, semi-supervised, and self-supervised settings, as well as improving the efficiency of Sheaf-FRL from both an algorithmic and a communication-strategy perspective—for instance by replacing full pilot representations with prototype pilots—to further reduce communication cost. Acknowledgments The work was supported by the SNS JU project 6G-GOALS (Strinati et al., 2024) under the EU’s Horizon program Grant Agreement No 101139232, and by Huawei Technology France SASU under Grant N. Tg20250616041. References Absil et al. (2008) P.-A. Absil, Robert Mahony, and Rodolphe Sepulchre. Optimization Algorithms on Matrix Manifolds. Princeton University Press, 2008. Badi et al. (2026) Mohamed Badi, Chaouki Ben Issaid, and Mehdi Bennis. Communication-efficient and robust multi-modal federated learning via latent-space consensus. IEEE Wireless Communications Letters, p. 1–1, 2026. doi: 10.1109/LWC.2026.3676885. Bandeira et al. (2013) Afonso S. Bandeira, Amit Singer, and Daniel A. Spielman. A Cheeger inequality for the graph connection Laplacian. SIAM Journal on Matrix Analysis and Applications, 34(4):1611–1630, 2013. doi: 10.1137/120875338. Barbarossa et al. (2023) Sergio Barbarossa, Danilo Comminiello, Eleonora Grassucci, Francesco Pezone, Stefania Sardellitti, and Paolo Di Lorenzo. Semantic communications based on adaptive generative models and information bottleneck. IEEE Communications Magazine, 61(11):36–41, 2023. Barbero et al. (2022) Federico Barbero, Cristian Bodnar, Haitz Sáez de Ocáriz Borde, Michael Bronstein, Petar Veličković, and Pietro Liò. Sheaf neural networks with connection Laplacians. In Topological, Algebraic and Geometric Learning Workshops 2022, p. 28–36. PMLR, 2022. Beck (2017) Amir Beck. First-order methods in optimization. SIAM, 2017. Bodnar et al. (2022) Cristian Bodnar, Francesco Di Giovanni, Benjamin Paul Chamberlain, Pietro Liò, and Michael M. Bronstein. Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in GNNs. In Advances in Neural Information Processing Systems (NeurIPS), 2022. URL https://dblp.org/rec/conf/nips/BodnarGCLB22. Bredon (1997) Glen E Bredon. Sheaf theory, volume 170. Springer Science & Business Media, 1997. Che et al. (2023) Liwei Che, Jiaqi Wang, Yao Zhou, and Fenglong Ma. Multimodal federated learning: A survey. Sensors, 23(15):6986, 2023. Chung et al. (2014) Fan Chung, Wenbo Zhao, and Mark Kempton. Ranking and sparsifying a connection graph. Internet Mathematics, 10(1-2):87–115, 2014. Collins et al. (2021) Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting shared representations for personalized federated learning. In Proceedings of the 38th International Conference on Machine Learning (ICML), volume 139 of PMLR, 2021. Curry (2014) Justin Michael Curry. Sheaves, cosheaves and applications. PhD thesis, University of Pennsylvania, 2014. D’Acunto et al. (2025) Gabriele D’Acunto, Fabio Massimo Zennaro, Yorgos Felekis, and Paolo Di Lorenzo. Causal abstraction learning based on the semantic embedding principle. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=J16AIOkjjY. D’Acunto et al. (2026) Gabriele D’Acunto, Paolo Di Lorenzo, and Sergio Barbarossa. Networks of causal abstractions: A sheaf-theoretic framework. arXiv preprint arXiv:2509.25236, 2026. Di Nino et al. (2024) Leonardo Di Nino, Sergio Barbarossa, and Paolo Di Lorenzo. Learning sheaf Laplacian optimizing restriction maps. In 2024 58th Asilomar Conference on Signals, Systems, and Computers, p. 59–63. IEEE, 2024. Di Nino et al. (2026) Leonardo Di Nino, Gabriele D’Acunto, Sergio Barbarossa, and Paolo Di Lorenzo. Learning the structure of connection graphs. In ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 76–80. IEEE, 2026. Diao et al. (2021) Enmao Diao, Jie Ding, and Vahid Tarokh. HeteroFL: Computation and communication efficient federated learning for heterogeneous clients. In International Conference on Learning Representations (ICLR), 2021. Dinh et al. (2022) Canh T Dinh, Tung T Vu, Nguyen H Tran, Minh N Dao, and Hongyu Zhang. A new look and convergence rate of federated multitask learning with Laplacian regularization. IEEE Transactions on Neural Networks and Learning Systems, 35(6):8075–8085, 2022. Dong & Voiculescu (2021) Nanqing Dong and Irina Voiculescu. Federated contrastive learning for decentralized unlabeled medical images. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 378–387. Springer, 2021. Edelman et al. (1998) Alan Edelman, Tomás A. Arias, and Steven T. Smith. The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications, 20(2):303–353, 1998. doi: 10.1137/S0895479895290954. Evgeniou & Pontil (2004) Theodoros Evgeniou and Massimiliano Pontil. Regularized multi–task learning. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, p. 109–117, 2004. Evgeniou et al. (2005) Theodoros Evgeniou, Charles A Micchelli, Massimiliano Pontil, and John Shawe-Taylor. Learning multiple tasks with kernel methods. Journal of Machine Learning Research, 6(4), 2005. Fiorellino et al. (2024) Simone Fiorellino, Claudio Battiloro, Emilio Calvanese Strinati, and Paolo Di Lorenzo. Dynamic relative representations for goal-oriented semantic communications. In Proc. IEEE EUSIPCO, p. 2107–2111, 2024. Fiorellino et al. (2026) Simone Fiorellino, Claudio Battiloro, Emilio Calvanese Strinati, and Paolo Di Lorenzo. Frame-based zero-shot semantic channel equalization for AI-native communications. IEEE Transactions on Cognitive Communications and Networking, 2026. Fredrikson et al. (2015) Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS), p. 1322–1333, 2015. Ghalkha et al. (2026) Abdulmomen Ghalkha, Zhuojun Tian, Chaouki Ben Issaid, and Mehdi Bennis. SheafAlign: A sheaf-theoretic framework for decentralized multimodal alignment. IEEE Communications Letters, 2026. Grimaldi et al. (2025) Enrico Grimaldi, Mario Edoardo Pandolfo, Gabriele D’Acunto, Sergio Barbarossa, and Paolo Di Lorenzo. Learning network sheaves for AI-native semantic communication. In 2025 59th Asilomar Conference on Signals, Systems, and Computers, p. 1692–1696. IEEE, 2025. Gündüz et al. (2022) Deniz Gündüz, Zhijin Qin, Inaki Estella Aguerri, Harpreet S Dhillon, Zhaohui Yang, Aylin Yener, Kai Kit Wong, and Chan-Byoung Chae. Beyond transmitting bits: Context, semantics, and task-oriented communications. IEEE Journal on Selected Areas in Communications, 41(1):5–41, 2022. Han et al. (2022) Sungwon Han, Sungwon Park, Fangzhao Wu, Sundong Kim, Chuhan Wu, Xing Xie, and Meeyoung Cha. Fedx: Unsupervised federated learning with cross knowledge distillation. In European Conference on Computer Vision, p. 691–707. Springer, 2022. Hansen & Ghrist (2019) Jakob Hansen and Robert Ghrist. Toward a spectral theory of cellular sheaves. Journal of Applied and Computational Topology, 3(4):315–358, 2019. Hüttebräucker et al. (2024) Tomás Hüttebräucker, Simone Fiorellino, Mohamed Sana, Paolo Di Lorenzo, and Emilio Calvanese Strinati. Relative representations of latent spaces enable efficient semantic channel equalization. In Proc. IEEE GLOBECOM, 2024. Issaid et al. (2025) Chaouki Ben Issaid, Praneeth Vepakomma, and Mehdi Bennis. Tackling feature and sample heterogeneity in decentralized multi-task learning: A sheaf-theoretic approach. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https://openreview.net/forum?id=JlPq0LmApB. Jacob et al. (2008) Laurent Jacob, Jean-Philippe Vert, and Francis Bach. Clustered multi-task learning: A convex formulation. Advances in Neural Information Processing Systems, 21, 2008. Jaggi et al. (2014) Martin Jaggi, Virginia Smith, Martin Takáč, Jonathan Terhorst, Sanjay Krishnan, Thomas Hofmann, and Michael I Jordan. Communication-efficient distributed dual coordinate ascent. Advances in Neural Information Processing Systems, 27, 2014. Kairouz et al. (2021) Peter Kairouz, Brendan H. Mcmahan, et al. Advances and Open Problems in Federated Learning, volume 14 of Foundations and Trends® in Machine Learning. 2021. doi: 10.1561/2200000083. URL https://inria.hal.science/hal-02406503. Karimireddy et al. (2020) Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. SCAFFOLD: Stochastic controlled averaging for federated learning. In Proceedings of the 37th International Conference on Machine Learning, PMLR, p. 5132–5143, 2020. Kessy et al. (2018) Agnan Kessy, Alex Lewin, and Korbinian Strimmer. Optimal whitening and decorrelation. The American Statistician, 72(4):309–314, 2018. Li et al. (2020a) Ang Li, Jingwei Sun, Binghui Wang, Lin Duan, Sicheng Li, Yiran Chen, and Hai Li. LotteryFL: Personalized and communication-efficient federated learning with lottery ticket hypothesis on non-iid datasets. arXiv preprint arXiv:2008.03371, 2020a. Li & Ma (2022) Jiaxiang Li and Shiqian Ma. Federated learning on Riemannian manifolds. arXiv preprint arXiv:2206.05668, 2022. Li et al. (2021) Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. Li et al. (2020b) Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. In Proceedings of the Third Conference on Machine Learning and Systems (MLSys), 2020b. Liang et al. (2020) Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B Allen, Randy P Auerbach, David Brent, Ruslan Salakhutdinov, and Louis-Philippe Morency. Think locally, act globally: Federated learning with local and global representations. arXiv preprint arXiv:2001.01523, 2020. Ma et al. (2015) Chenxin Ma, Virginia Smith, Martin Jaggi, Michael Jordan, Peter Richtárik, and Martin Takác. Adding vs. averaging in distributed primal-dual optimization. In International Conference on Machine Learning, p. 1973–1982. PMLR, 2015. McLaughlin & Su (2024) Connor J McLaughlin and Lili Su. Personalized federated learning via feature distribution adaptation. Advances in Neural Information Processing Systems, 37:77038–77059, 2024. McMahan et al. (2017) Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), volume 54, p. 1273–1282. PMLR, 2017. Melis et al. (2019) Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting unintended feature leakage in collaborative learning. In IEEE Symposium on Security and Privacy (S&P), 2019. Miao & Koyuncu (2024) Runxuan Miao and Erdem Koyuncu. Contrastive and non-contrastive strategies for federated self-supervised representation learning and deep clustering. IEEE Journal of Selected Topics in Signal Processing, 18(6):1070–1084, 2024. Moschella et al. (2023) Luca Moschella, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. Relative representations enable zero-shot latent space communication. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=SrC-nwieGJ. Nesterov (2004) Yurii Nesterov. Introductory Lectures on Convex Optimization: A Basic Course, volume 87 of Applied Optimization. Springer, 2004. Pandolfo et al. (2025) Mario Edoardo Pandolfo, Simone Fiorellino, Emilio Calvanese Strinati, and Paolo Di Lorenzo. Latent space alignment for AI-native mimo semantic communications. In 2025 International Joint Conference on Neural Networks (IJCNN), p. 1–8. IEEE, 2025. Pandolfo et al. (2026) Mario Edoardo Pandolfo, Enrico Grimaldi, Lorenzo Marinucci, Leonardo Di Nino, Simone Fiorellino, Sergio Barbarossa, and Paolo Di Lorenzo. SEMASIA: A large-scale dataset of semantically structured latent representations. arXiv preprint arXiv:2605.09485, 2026. Robbins & Siegmund (1971) Herbert Robbins and David Siegmund. A convergence theorem for non negative almost supermartingales and some applications. p. 233–257. Academic Press, 1971. Scardapane & Di Lorenzo (2017) Simone Scardapane and Paolo Di Lorenzo. A framework for parallel and distributed training of neural networks. Neural Networks, 91:42–54, 2017. Schönemann (1966) Peter H. Schönemann. A generalized solution of the orthogonal Procrustes problem. Psychometrika, 31(1):1–10, 1966. Setayesh et al. (2026) Mehdi Setayesh, Mahdi Beitollahi, Yasser H. Khalil, and Hongliang Li. Toward enhancing representation learning in federated multi-task settings. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=nIOIfHHYzk. Shokri et al. (2017) Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In IEEE Symposium on Security and Privacy (S&P), p. 3–18. IEEE, 2017. Singer (2011) Amit Singer. Angular synchronization by eigenvectors and semidefinite programming. Applied and Computational Harmonic Analysis, 30(1):20–36, 2011. Singer & Wu (2012) Amit Singer and H-T Wu. Vector diffusion maps and the connection Laplacian. Communications on Pure and Applied Mathematics, 65(8):1067–1144, 2012. Smith et al. (2017) Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. Federated multi-task learning. Advances in Neural Information Processing Systems, 30, 2017. Strinati et al. (2024) Emilio Calvanese Strinati, Paolo Di Lorenzo, et al. Goal-oriented and semantic communication in 6G AI-native networks: The 6G-GOALS approach. In Proceedings of EuCNC/6G Summit, p. 1–6, 2024. doi: 10.1109/EuCNC/6GSummit60053.2024.10597087. Svendsen et al. (2026) Oscar Thorsted Svendsen, Nikolaj Holst Jakobsen, Fabian Mager, and Hiba Nassar. Improving relative representations with learned anchors and whitened inner products. arXiv preprint arXiv:2605.30596, 2026. Tan et al. (2022) Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. Fedproto: Federated prototype learning across heterogeneous clients. In Proceedings of the AAAI Conference on Artificial Intelligence, 2022. Thunberg et al. (2017) Johan Thunberg, Florian Bernard, and Jorge Gonçalves. Distributed methods for synchronization of orthogonal matrices over graphs. Automatica, 80:243–252, 2017. Tran et al. (2024) Trung-Kien Tran, Ha-Phuong Tran, Thi-Lan Le, and Thanh-Hai Tran. Fedntproto: A prototype-based approach for personalized federated learning. In 2024 International Conference on Multimedia Analysis and Pattern Recognition (MAPR), p. 1–6. IEEE, 2024. Wang & Isola (2020) Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning, p. 9929–9939. PMLR, 2020. Zhang et al. (2023) Fengda Zhang, Kun Kuang, Long Chen, Zhaoyang You, Tao Shen, Jun Xiao, Yin Zhang, Chao Wu, Fei Wu, Yueting Zhuang, and Xiaolin Li. Federated unsupervised representation learning. Frontiers of Information Technology & Electronic Engineering, 24(8):1181–1193, 2023. Zhang et al. (2021) Shengdong Zhang, Ehsan Nezhadarya, Homa Fashandi, Jiayi Liu, Darin Graham, and Mohak Shah. Stochastic whitening batch normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 10978–10987, 2021. Zhang & Yang (2017) Yu Zhang and Qiang Yang. Learning sparse task relations in multi-task learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017. Zhang & Yeung (2010) Yu Zhang and Dit-Yan Yeung. A convex formulation for learning task relationships in multi-task learning. In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence, p. 733–742, 2010. Zhuang et al. (2021) Weiming Zhuang, Xin Gan, Yonggang Wen, Shuai Zhang, and Shuai Yi. Collaborative unsupervised visual representation learning from decentralized data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 4912–4921, 2021. Appendix A Additional discussion of our framework This section elaborates on two salient features of our proposed framework, namely the edge stalk dimensionality and the enforced geometric consistency. We also briefly discuss the homogeneous setting case where all node stalks have the same dimensionality. Embedding vs. compression. Choosing the edge stalk dimensionality dij=max(di,dj)d_ij= (d_i,d_j) avoids introducing an information bottleneck when aligning heterogeneous latent representations. In contrast, as done for instance in (Badi et al., 2026), enforcing dij≤min(di,dj)d_ij≤ (d_i,d_j) would require projecting both representations onto a shared lower-dimensional subspace, implicitly assuming the existence of a common latent structure and potentially discarding node-specific information. Moreover, such a projection only enforces alignment on the retained subspace, leaving the remaining components unconstrained. As a result, the learned restriction maps capture consistency only up to an information loss, potentially leading to inconsistencies when transferring latent representations across models. Our choice instead pursues a geometric notion of consistency across representations, and preserves the semantics of local representations. Importantly, this also makes our framework well-suited for downstream transfer of latent representations among pre-trained models, where alignment based on compression may result in performance degradation on downstream local tasks. On the geometric consistency. To better understand the action of the restriction maps as well as the role of the total variation penalty, let us consider ℱ(i)→ℱ(eij)←ℱ(j)F(i) (e_ij) (j) in Fig. 1. When transported to the edge stalk ℱ(eij)⊆ℝ3F(e_ij) ^3, the latent representation i∈ℱ(i)z_i (i) is rotated and reflected by the orthogonal map jiO_ji. Instead, the latent representation jz_j is embedded via the Stiefel map ijV_ij into the higher-dimensional edge stalk. Interestingly, the image Im(ij)Im (V_ij ) of ijV_ij defines a two-dimensional subspace embedded in ℱ(eij)F(e_ij). Thus, a perfect alignment between the latent representations iz_i and jz_j implies that iz_i is a rotated and reflected version of a vector j→eij=ijjy_j→ e_ij=V_ijz_j belonging to Im(ij)Im (V_ij ). When this occurs for all the latent representations in ℱ(i)F(i), it means that the latter representations lie on a linear two-dimensional subspace of ℱ(i)F(i), as depicted in Fig. 1. In Fig. 1, we notice that there is only a partial alignment between iz_i and jz_j when they are transported onto ℱ(eij)F(e_ij). Indeed, even though the projection Proj(ji(i))Proj(O_ji(z_i)) of ji(i)O_ji(z_i) onto Im(ij)Im (V_ij ) exactly matches ijjV_ijz_j, there is still an orthogonal component that makes non-null the Euclidean distance between the two transported vectors. The latter component can be better visualized in ℱ(i)F(i), where we see that iz_i does not lie on the subspace individuated by the transport of j→eij∈Im(ij)y_j→ e_ij (V_ij ). Thus, the total variation term acts as a regularizer that, for all eij∈ℰe_ij , enforces the zeroing of components orthogonal to Im(ij)Im (V_ij ). Notably, the latter discussion highlights that within our proposed framework we do not assume or leverage the existence a single global latent representation, as instead pursued in recent works (Setayesh et al., 2026; Badi et al., 2026). Indeed, in our framework a global representation may eventually emerge from the alignment of local representations over the network sheaf. Homogeneous setting. When ℱ(i)=ℱ(e)≅ℝdF(i)=F(e) ^d for all i∈i and e∈ℰe , the local total variation term in Eq. 10 reduces to ‖i−ijj‖22, z_i-O_ijz_j_2^2\,, (24) where ij∈O(d)O_ij (d). In this case, alignment between neighboring representations is achieved up to an orthogonal transformation, i.e., rotations and reflections of the latent space. This setting connects Eq. 9 to the Dirichlet energy of the connection Laplacian (Chung et al., 2014), and more generally to synchronization problems over the orthogonal group (Singer & Wu, 2012). In particular, the transport maps satisfy ji=ij⊤O_ji=O_ij , and the two variation terms in Eq. 12 coincide. In contrast, in the heterogeneous setting we allow semi-orthogonal maps on the Stiefel manifold, which act as isometric embeddings and are only left-invertible. Finally, since all node stalks share the same dimension, the orientation of G does not affect the model and can be chosen arbitrarily. Appendix B Pilot Selection Strategies The alignment of latent spaces relies on a set of semantic pilots: inputs whose correspondence across agents is known a priori, so that fi(ik)f_ _i(x_i^k) and fj(jk)f_ _j(x_j^k) can be treated as two latent representations of the same underlying sample. In practice, pilots arise from public datasets, calibration signals, standardized inputs, or an overlapping portion of the local datasets, and we assume that a common index set 1,…,M\1,…,M\ has been agreed upon. The choice of ⊂1,…,MA⊂\1,…,M\, with ||=K≪M|A|=K M, governs the trade-off between communication cost, computational efficiency, geometric coverage, and alignment quality. In the sequel, we outline the most widely used practical selection criteria. Additionally, a normalization step, applied to the resulting pilot matrices for balancing directions in their corresponding latent spaces, is discussed at the end of the section. Original pilots. Each agent transmits the embeddings fi(ik)f_ _i(x_i^k) of the shared inputs themselves, where k∈k . Taking =1,…,MA=\1,…,M\ evaluates the penalty exactly and is the most reliable option, but scales linearly in M in both communication and Procrustes cost; subsampling to a budget K≪MK M instead yields an unbiased Monte Carlo estimator of the full alignment energy. Unsupervised selection: Pilots are drawn uniformly, which requires no side information but may under-represent low-density regions of the representation space and lead to ill-conditioned iA_i. Clustering the representation space and drawing a fixed number of pilots per cluster recovers the same stratification that labels provide in the supervised case, at the cost of one clustering pass, and guarantees that sparsely populated regions are represented. Supervised selection: Labels are exploited since they induce a natural stratification, and a fixed number of pilots is drawn per class, guaranteeing balanced coverage even for small K. Prototype pilots. The shared set is partitioned into K groups and each pilot is defined as the group average (centroid), ¯i,c=1|c|∑k∈cfi(ik),c=1,…,K; f_i,c= 1|S_c| _k _cf_ _i(x_i^k), c=1,…,K\,; which preserves semantic structure while averaging out per-sample noise. When groups are large, the centroid need not be computed over all their members: averaging over a subset c′⊂cS_c _c gives an unbiased estimate at a fraction of the local forward passes, trading estimation variance for computational cost. Unsupervised partitioning: The partition is obtained by clustering the representation space (e.g., via k-means), with the resulting index assignment broadcast once and reused thereafter (Hüttebräucker et al., 2024; Fiorellino et al., 2026). Supervised partitioning: Class labels provide the partition directly, recovering prototype-based federated schemes such as FedProto (Tan et al., 2022). Ensuring correspondence and adaptive selection. To ensure that the transmitted quantities are in correspondence, both original and prototype pilots require the selection to be replicated across agents. Specifically: (i) the sampled indices in the first case, and (i) the partition and the subsets used to estimate the centroids in the second. Furthermore, the selection can be made adaptive by prioritizing the samples that currently exhibit the largest cross-client mismatch, biasing which inputs are transmitted or which groups are refined so as to concentrate the alignment effort on geometrically inconsistent regions. Dynamic selection. Any of the above criteria can be applied per iteration, letting tA_t vary over time and turning the penalty into a stochastic approximation of the full alignment energy. A practical recipe combines a small persistent subset (for stability of the transport maps) with a periodically refreshed remainder (for coverage), which empirically improves generalization of the learned maps beyond the sampled pilots (Fiorellino et al., 2024). Parseval normalization. Independently of how A is selected, the conditioning of the Procrustes step depends on the geometry of the pilot matrix i=[fi(ik)]k∈ℝdi×KA_i= [f_ _i(x_i^k) ]_k ^d_i× K, with K<diK<d_i in the regime of interest. To mitigate degeneracy, we can normalize pilots via ~i=i(i⊤i)−1/2, A_i=A_i(A_i A_i)^-1/2\,, the orthogonal polar factor of iA_i: this enforces ~i⊤~i=K A_i A_i=I_K, i.e., a Parseval frame of the pilot subspace (Fiorellino et al., 2026). This transformation balances directions in latent space and stabilizes the Procrustes updates. Appendix C Identifiability of the Transport Maps The edge-wise transports are computed via (semi-)orthogonal Procrustes problems ij∈argmin∈ij‖i−j‖F2,V_ij∈ _V _ij \|A_i-VA_j \|_F^2, (25) where ij=O(d)C_ij=O(d) in the homogeneous case and ij=St(di,dj)C_ij=St(d_i,d_j) for di>djd_i>d_j. We summarize conditions under which the minimizer is unique. Homogeneous case (di=dj=d_i=d_j=d). Let ij≔ij⊤∈ℝd×dM_ij _iA_j ^d× d. An optimal solution is ij⋆=⊤V_ij =UW , where ij=⊤M_ij=U is an SVD (Schönemann, 1966). The solution is unique if ijM_ij is full rank and has simple singular values. Non-uniqueness arises only when ijM_ij is rank-deficient or has repeated singular values, which induce rotational ambiguity within singular subspaces. In particular, if rank(i)=rank(j)=drank(A_i)=rank(A_j)=d (e.g., K≥dK≥ d with non-degenerate pilots), then ijM_ij is generically full rank and ij⋆V_ij is uniquely defined up to measure-zero degeneracies. Stiefel case (di>djd_i>d_j). Let ij=ij⊤∈ℝdi×djM_ij=A_iA_j ^d_i× d_j and ij=⊤M_ij=U a thin SVD, with ∈ℝdi×djU ^d_i× d_j, ∈ℝdj×djW ^d_j× d_j. A minimizer is ij⋆=⊤∈St(di,dj).V_ij =UW (d_i,d_j)\,. (26) Uniqueness holds if rank(ij)=djrank(M_ij)=d_j and its singular values are simple. Crucially, ijV_ij is identifiable only on the subspace spanned by jA_j: if rank(j)=r<djrank(A_j)=r<d_j, the solution is underdetermined on the orthogonal complement. Hence, identifiability requires rank(j)=djrank(A_j)=d_j (typically ensured when K≥djK≥ d_j and pilots provide sufficient geometric coverage), in which case ij⋆V_ij is generically unique up to measure-zero degeneracies. Appendix D Pseudocode of the Sheaf-FRL Algorithm In this section, we provide additional details on the proposed decentralized algorithm, reported in Alg. 1. The method follows an alternating minimization strategy, where each communication round consists of local feature extraction, decentralized alignment updates, and gradient-based optimization of the neural parameters. Algorithm 1 Sheaf-FRL 0: Oriented =(,ℰ)G=(N,E), pilots A, λ>0λ>0, ηθ>0 _θ>0, iterations T 1: Initialize i0=(i0,i0)i∈\ θ_i^0=( _i^0, ψ_i^0)\_i 2: for t=0,…,T−1t=0,…,T-1 do 3: for all i∈i in parallel do 4: it←[fit(ik)]k∈A_i^t←[f_ _i^t(x_i^k)]_k 5: end for 6: Each node i broadcasts itA_i^t to those j∈(i)j (i) 7: for all i∈i in parallel do 8: for j∈(i)j (i) do 9: if j∈(i)−j (i)^- then 10: [,,⊤]←thinSVD(itjt⊤)[U, ,W ] (A_i^tA_j^t ) 11: ijt←⊤V_ij^t 12: else 13: [,,⊤]←thinSVD(jtit⊤)[U, ,W ] (A_j^tA_i^t ) 14: jit←⊤V_ji^t 15: end if 16: end for 17: end for 18: for all i∈i in parallel do 19: it+1←it−ηθ(∇iℒi(it)+it) θ_i^t+1← θ_i^t- _θ ( _ θ_iL_i( θ_i^t)+r^t_i ) 20: end for 21: end for Heterogeneous case. Alg. 1 implements a fully decentralized procedure in which agents iteratively update both their local representations and the alignment maps across the network. At each iteration t, every agent i first computes its pilot feature matrix itA_i^t using the current encoder (lines 3–5). These pilot representations provide a compact summary of the local latent space and constitute the only information exchanged across agents. The pilot features are then broadcast to neighboring nodes (line 6). Upon receiving jtj∈(i)\A_j^t\_j (i), each agent locally updates the alignment maps associated with its incident edges (lines 7–17). In particular, for each neighbor j, agent i solves a small Procrustes problem based on the cross-covariance of the corresponding pilot features. The orientation of the graph determines whether the update corresponds to an incoming map ijtV_ij^t or an outgoing map jitV_ji^t, ensuring consistency with the underlying sheaf structure. Finally, each agent performs a gradient-based update of its local parameters i θ_i (lines 18–20), combining the gradient of the local loss with the contribution of the sheaf-based regularization. This step enforces alignment of neighboring representations while preserving local task objectives. Overall, the algorithm alternates between closed-form alignment updates and gradient-based parameter updates. All computations are carried out locally, and communication is limited to low-dimensional pilot features, resulting in an efficient and scalable decentralized learning procedure. Algorithm 2 Sheaf-FRL (homogeneous case) 0: Oriented =(,ℰ)G=(N,E), pilots A, λ>0λ>0, ηθ>0 _θ>0, iterations T 1: Initialize i0=(i0,i0)i∈\ θ_i^0=( _i^0, ψ_i^0)\_i and ji0=dj∈(i),i∈\O_ji^0=I_d\_j (i),i 2: for t=0,…,T−1t=0,…,T-1 do 3: for all i∈i in parallel do 4: it←[fit(ik)]k∈A_i^t←[f_ _i^t(x_i^k)]_k 5: end for 6: Each node i exchanges jititO_ji^tA_i^t with all j∈(i)j (i) 7: for all i∈i in parallel do 8: for all j∈(i)+j (i)^+ do 9: [,,⊤]←SVD(ijtjtit⊤)[U, ,W ] (O_ij^tA_j^tA_i^t ) 10: jit←⊤O_ji^t 11: end for 12: end for 13: for all i∈i in parallel do 14: it+1←it−ηθ(∇iℒi(it)+it) θ_i^t+1← θ_i^t- _θ ( _ θ_iL_i( θ_i^t)+r_i^t ) 15: end for 16: end for Homogeneous case. When the latent dimensions are homogeneous, i.e., di=d_i=d for all i∈i , the alignment maps reduce to orthogonal transformations ij∈O(d)O_ij (d), for each (i,j)∈ℰ(i,j) . In this setting, as discussed in Remark 4, the local computational cost of the alignment step can be reduced without increasing communication. Alg. 2 presents a simplified Sheaf-FRL protocol tailored to this scenario. For each agent i, we initialize to the identity the maps used to transport its pilot representations toward its neighbors, i.e., ji0=dO_ji^0=I_d for all j∈(i)j (i). At each iteration t, each agent first computes its pilot representations itA_i^t and sends to each neighbor j∈(i)j (i) the transported features jititO_ji^tA_i^t. Notably, since we exploit the reparameterization in Eq. 24, when j∈−(i)j ^-(i), we have jit=ji0=dO_ji^t=O_ji^0=I_d for all t=0,…,T−1t=0,…,T-1, so that the transmitted features coincide with the original ones and no additional communication cost is incurred. From a local computation perspective, since transported pilot representations are communicated instead of raw ones, agent i does not need to solve the orthogonal Procrustes problem arising in (P2a) for evaluating the alignment and the gradient required for the local parameter update. As a consequence, only the SVDs for j∈(i)+j (i)^+ are required, effectively halving the computational cost of the alignment step, as specified in lines 8-10 of Alg. 2. Overall, this variant preserves the communication efficiency of the general algorithm while reducing the computational burden (cf. Tab. 1). Appendix E Convergence Analysis In this section we establish convergence guarantees for the proposed Sheaf-FRL algorithm in both deterministic and stochastic settings. We first recall the classical descent result for smooth functions. Lemma E.1 (Descent Lemma, Nesterov, 2004, Prop. 1.2.3,Beck, 2017, Sec. 2.1). Let f be differentiable with L-Lipschitz gradient. Then for any x and any η>0η>0, f(−η∇f())≤f()−η(1−Lη2)‖∇f()‖2.f(x-η∇ f(x))≤ f(x)-η (1- Lη2 )\|∇ f(x)\|^2. E.1 Deterministic Setting Let J(,)=∑i∈ℒi(i)+ℛ(ii∈,ij(i,j)∈ℰ,ji(i,j)∈ℰ)J( θ,V)= _i L_i( θ_i)+R_A\! (\ _i\_i ,\V_ij\_(i,j) ,\V_ji\_(i,j) ) (27) in (SFRL) where, with slight abuse of notation, =ii∈=(i,i)i∈ θ=\ θ_i\_i =\( _i, ψ_i)\_i , and =ij(i,j)∈ℰ∪ji(i,j)∈ℰV=\V_ij\_(i,j) ∪\V_ji\_(i,j) . For each edge (i,j)∈ℰ(i,j) , the matrices ij∈St(di,dj)V_ij (d_i,d_j) and ji∈St(dj,di)V_ji (d_j,d_i), where St(di,dj)St(d_i,d_j) denotes a Stiefel manifold as defined in Eq. 4. The feasible set is therefore :=(∏(i,j)∈ℰSt(di,dj))×(∏(i,j)∈ℰSt(dj,di))V:= ( _(i,j) St(d_i,d_j) )× ( _(i,j) St(d_j,d_i) ). The proposed method alternates between an exact minimization with respect to V and a gradient step with respect to θ, and can be compactly cast as: t ^t ∈argmin∈J(t,), ∈ _V J( θ^t,V), (28) t+1 θ^t+1 =t−η∇J(t,t) = θ^t-η _ θJ( θ^t,V^t) (29) We introduce the following assumptions. Assumption 1. The objective J(,)J( θ,V) is bounded from below, with lower bound Jinf≔inf,J(,).J_ _ θ,VJ( θ,V)\,. Assumption 2. For every fixed ∈V , J(,)J( θ,V) is differentiable and its gradient is L-Lipschitz: ‖∇J(1,)−∇J(2,)‖≤L‖1−2‖.\| _ θJ( θ_1,V)- _ θJ( θ_2,V)\|≤ L\| θ_1- θ_2\|. Assumption 3. At each iteration, t∈argmin∈J(t,),V^t∈ _V J( θ^t,V), i.e., tV^t is an exact minimizer of J given t θ^t. Assumption 4. The feasible set V is compact. Under 1-4, the following convergence result holds. Theorem E.2. Suppose Assumptions 1–4 hold and choose 0<η≤1/L0<η≤ 1/L. Then: 1. Descent, Convergence, and Stationarity. The sequence J(t,t)\J( θ^t,V^t)\ is monotonically decreasing and therefore convergent. Moreover, limt→∞‖∇J(t,t)‖=0 _t→∞\| _ θJ( θ^t,V^t)\|=0, and every accumulation point (⋆,⋆)( θ ,V ) is a block-wise first-order stationary point. 2. Sublinear stationarity rate. For all T≥0T≥ 0, min0≤t≤T‖∇J(t,t)‖2≤2(J(0,0)−Jinf)η(T+1). _0≤ t≤ T\| _ θJ( θ^t,V^t)\|^2≤ 2 (J( θ^0,V^0)-J_ )η(T+1)\,. (30) Proof. (i) Descent, Convergence, and Stationarity. Fix tV^t and consider J(,t).J( θ,V^t). By Assumption 2, J(,t)J( θ,V^t) has L-Lipschitz gradient. Applying Lemma E.1 with η≤1/Lη≤ 1/L yields J(t+1,t)≤J(t,t)−η2‖∇J(t,t)‖2.J( θ^t+1,V^t)≤ J( θ^t,V^t)- η2\| _ θJ( θ^t,V^t)\|^2. (31) By exact minimization of V at iteration t+1t+1 (Assumption 3), J(t+1,t+1)≤J(t+1,t)J( θ^t+1,V^t+1)≤ J( θ^t+1,V^t). Combining this inequality with the previous bound yields J(t+1,t+1)≤J(t,t)J( θ^t+1,V^t+1)≤ J( θ^t,V^t), which shows that the sequence J(t,t)\J( θ^t,V^t)\ is monotonically decreasing. Since J is bounded below (Assumption 1), the sequence therefore converges. Summing (31) over t yields ∑t=0∞‖∇J(t,t)‖2<∞, _t=0^∞\| _ θJ( θ^t,V^t)\|^2<∞, which implies limt→∞‖∇J(t,t)‖=0. _t→∞\| _ θJ( θ^t,V^t)\|=0. Let (tk,tk)( θ^t_k,V^t_k) be a convergent subsequence. By continuity of ∇J _ θJ, it holds ∇J(⋆,⋆)=0. _ θJ( θ ,V )=0. Moreover, since tV^t minimizes J(t,⋅)J( θ^t,·) over the compact set V, passing to the limit gives ⋆∈argmin∈J(⋆,).V ∈ _V J( θ ,V). Thus every accumulation point is block-wise first-order stationary. (i) Sublinear Stationarity Rate. From the descent inequality (31), summing from t=0t=0 to T and telescoping yields η2∑t=0T‖∇J(t,t)‖2≤J(0,0)−J(T+1,T). η2 _t=0^T\| _ θJ( θ^t,V^t)\|^2≤ J( θ^0,V^0)-J( θ^T+1,V^T). Since J(T+1,T)≥JinfJ( θ^T+1,V^T)≥ J_ (Assumption 1), ∑t=0T‖∇J(t,t)‖2≤2(J(0,0)−Jinf)η. _t=0^T\| _ θJ( θ^t,V^t)\|^2≤ 2 (J( θ^0,V^0)-J_ )η. Finally, using min0≤t≤Tat≤1T+1∑t=0Tat _0≤ t≤ Ta_t≤ 1T+1 _t=0^Ta_t with at=‖∇J(t,t)‖2a_t=\| _ θJ( θ^t,V^t)\|^2, we obtain min0≤t≤T‖∇J(t,t)‖2≤2(J(0,0)−Jinf)η(T+1), _0≤ t≤ T\| _ θJ( θ^t,V^t)\|^2≤ 2 (J( θ^0,V^0)-J_ )η(T+1), which proves (30). ∎ E.2 Stochastic Setting We consider the same deterministic objective J(,)J( θ,V) in (27), and extend the θ-update to the stochastic (mini-batch) setting while keeping the V-block exactly minimized with respect to the full objective. Let Φtt≥0\ _t\_t≥ 0 be the natural filtration generated by the iterates and the sampling randomness up to time t. At iteration t, we first update the alignment variables by exact minimization, t∈argmin∈J(t,),V^t∈ _V J( θ^t,V), (32) and then form a mini-batch stochastic gradient estimator t≡(t,t;ξt),g^t ( θ^t,V^t; _t), where ξt _t denotes the mini-batch sampling randomness. The stochastic update of the neural parameters is t+1=t−ηtt. θ^t+1= θ^t- _t\,g^t. (33) To analyze the convergence behavior of the stochastic scheme, we impose the following assumptions on the stochastic gradient estimator. Assumption 5 (Unbiased stochastic gradient). For all t≥0t≥ 0, [t∣Φt]=∇J(t,t).E[g^t _t]= _ θJ( θ^t,V^t). Assumption 6 (Bounded conditional variance). There exists σ2<∞σ^2<∞ such that for all t≥0t≥ 0, [∥t−∇J(t,t)∥2|Φt]≤σ2.E\! [ \|g^t- _ θJ( θ^t,V^t) \|^2\ |\ _t ]≤σ^2. Under these assumptions, we establish convergence guarantees for the stochastic variant of the proposed method in the following sections. E.2.1 Convergence Guarantee with Optimized Step Size Theorem E.3 (Stochastic convergence rate). Assume Assumptions 1, 2, 3 and 4, and the stochastic gradient conditions in Assumptions 5 and 6. Let (t,t)t≥0\( θ^t,V^t)\_t≥ 0 be generated by (32)–(33). Define Δ0:=J(0,0)−Jinf _0:=J( θ^0,V^0)-J_ , where Jinf:=inf,J(,)J_ := _ θ,VJ( θ,V). Then, for any horizon T≥0T≥ 0, choosing the constant step size ηt≡η:=min1L,2Δ0Lσ2(T+1) _t≡η:= \! \ 1L, 2 _0Lσ^2(T+1) \ yields the bound min0≤t≤T[‖∇J(t,t)‖2]≤22Lσ2Δ0T+1+2LΔ0T+1. _0≤ t≤ TE\! [ \| _ θJ( θ^t,V^t) \|^2 ]≤ 2 2Lσ^2\, _0T+1\;+\; 2L _0T+1. In particular, the method attains the stochastic convergence rate min0≤t≤T[‖∇J(t,t)‖2]=(1T). _0≤ t≤ TE\! [ \| _ θJ( θ^t,V^t) \|^2 ]=O\! ( 1 T ). Proof. For brevity, define Jt:=J(t,t),∇t:=∇J(t,t),t:=(t,t;ξt).J_t:=J( θ^t,V^t), _t:= _ θJ( θ^t,V^t), ^t:=g( θ^t,V^t; _t). Fix t≥0t≥ 0. By Assumption 2, for the L-smooth map ↦J(,t) θ J( θ,V^t), we have for any vector u, J(t+,t)≤J(t,t)+∇t⊤+L2‖2.J( θ^t+u,V^t)≤ J( θ^t,V^t)+ _t u+ L2\|u\|^2. Applying this with =−ηtu=-η\,g^t yields J(t+1,t)≤J(t,t)−η∇t⊤t+Lη22‖t‖2.J( θ^t+1,V^t)≤ J( θ^t,V^t)-η\, _t g^t+ Lη^22\|g^t\|^2. (34) By exact minimization in V (Assumption 3), J(t+1,t+1)≤J(t+1,t).J( θ^t+1,V^t+1)≤ J( θ^t+1,V^t). (35) Combining (34)–(35) gives Jt+1≤Jt−η∇t⊤t+Lη22‖t‖2.J_t+1≤ J_t-η\, _t g^t+ Lη^22\|g^t\|^2. (36) Taking conditional expectation w.r.t. Φt _t and using unbiasedness (Assumption 5) gives [∇t⊤t∣Φt]=∇t⊤[t∣Φt]=‖∇t‖2.E\! [ _t g^t _t ]= _t E[g^t _t]=\| _t\|^2. Moreover, expanding the second moment and using the bounded conditional variance (Assumption 6) yields [‖t‖2∣Φt] \! [\|g^t\|^2 _t ] =[‖t−∇t+∇t‖2∣Φt] =E\! [\|g^t- _t+ _t\|^2 _t ] =‖∇t‖2+[‖t−∇t‖2∣Φt]+2∇t⊤[t−∇t∣Φt] =\| _t\|^2+E\! [\|g^t- _t\|^2 _t ]+2\, _t E[g^t- _t _t] ≤‖∇t‖2+σ2, ≤\| _t\|^2+σ^2, since [t−∇t∣Φt]=E[g^t- _t _t]=0 by Assumption 5. Substituting these two identities into the conditional expectation of (36) yields [Jt+1∣Φt]≤Jt−η(1−Lη2)‖∇t‖2+Lη22σ2.E[J_t+1 _t]≤ J_t-η (1- Lη2 )\| _t\|^2+ Lη^22σ^2. (37) Assuming 0<η≤1/L0<η≤ 1/L implies 1−Lη2≥121- Lη2≥ 12, hence [Jt+1∣Φt]≤Jt−η2‖∇t‖2+Lη22σ2.E[J_t+1 _t]≤ J_t- η2\| _t\|^2+ Lη^22σ^2. (38) Taking total expectation in (38) and summing from t=0t=0 to T gives [JT+1]≤J0−η2∑t=0T‖∇t‖2+Lη22σ2(T+1).E[J_T+1]≤ J_0- η2 _t=0^TE\| _t\|^2+ Lη^22σ^2(T+1). Rearranging and using the lower bound [JT+1]≥JinfE[J_T+1]≥ J_ (Assumption 1) yields η2∑t=0T‖∇t‖2≤Δ0+Lη22σ2(T+1),Δ0:=J0−Jinf. η2 _t=0^TE\| _t\|^2≤ _0+ Lη^22σ^2(T+1), _0:=J_0-J_ . (39) Dividing by η(T+1)η(T+1) and using min0≤t≤Tat≤1T+1∑t=0Tat _0≤ t≤ Ta_t≤ 1T+1 _t=0^Ta_t gives min0≤t≤T‖∇t‖2≤2Δ0η(T+1)+Lησ2. _0≤ t≤ TE\| _t\|^2≤ 2 _0η(T+1)+Lη\,σ^2. (40) We now optimize the right-hand side in η under the constraint η≤1/Lη≤ 1/L. Let η⋆:=2Δ0Lσ2(T+1).η := 2 _0Lσ^2(T+1). If η⋆≤1/Lη ≤ 1/L, choosing η=η⋆η=η in (40) yields min0≤t≤T‖∇t‖2≤22Lσ2Δ0T+1. _0≤ t≤ TE\| _t\|^2≤ 2 2Lσ^2 _0T+1. If instead η⋆>1/Lη >1/L, we choose η=1/Lη=1/L in (40) to obtain min0≤t≤T‖∇t‖2≤2LΔ0T+1+σ2. _0≤ t≤ TE\| _t\|^2≤ 2L _0T+1+σ^2. Both cases are covered by the choice η=min1L,2Δ0Lσ2(T+1).η= \ 1L,\ 2 _0Lσ^2(T+1) \. Moreover, with this choice we have the unified bound min0≤t≤T‖∇t‖2≤22Lσ2Δ0T+1+2LΔ0T+1, _0≤ t≤ TE\| _t\|^2≤ 2 2Lσ^2 _0T+1+ 2L _0T+1, where the second term is redundant when η=η⋆η=η but remains valid. This proves Theorem E.3. ∎ E.2.2 Almost-sure Convergence with Diminishing Step Sizes We establish an almost-sure convergence guarantee under a Robbins–Monro step-size schedule. Throughout, let Φtt≥0\ _t\_t≥ 0 denote the natural filtration and let the iterates be generated by (32)–(33). Assumption 7 (Robbins–Monro step sizes). The step sizes satisfy 0<ηt≤1/L0< _t≤ 1/L for all t≥0t≥ 0, and ∑t=0∞ηt=∞,∑t=0∞ηt2<∞. _t=0^∞ _t=∞, _t=0^∞ _t^2<∞. Theorem E.4 (Almost-sure convergence). Assume Assumptions 1, 2, 3 and 4, and Assumptions 5 and 6 and Assumption 7. Let (t,t)t≥0\( θ^t,V^t)\_t≥ 0 be generated by (32)–(33). Then: 1. The sequence J(t,t)\J( θ^t,V^t)\ converges almost surely to a finite random variable. 2. Moreover, ∑t=0∞ηt‖∇J(t,t)‖2<∞almost surely. _t=0^∞ _t \| _ θJ( θ^t,V^t) \|^2<∞ surely. 3. Consequently, lim inft→∞‖∇J(t,t)‖=0almost surely. _t→∞ \| _ θJ( θ^t,V^t) \|=0 surely. Proof. We begin by recalling the Robbins–Siegmund lemma, which provides a standard convergence result for quasi-supermartingale sequences. Lemma E.5 (Robbins–Siegmund, Robbins & Siegmund, 1971). Let Xtt≥0\X_t\_t≥ 0 be a nonnegative adapted sequence with respect to a filtration Φtt≥0\ _t\_t≥ 0. Assume there exist nonnegative Φt _t-measurable sequences att≥0\a_t\_t≥ 0 and btt≥0\b_t\_t≥ 0 such that [Xt+1∣Φt]≤Xt−at+bta.s. for all t,E[X_t+1 _t]≤ X_t-a_t+b_t .s. for all t, and ∑t=0∞bt<∞ _t=0^∞b_t<∞ almost surely. Then XtX_t converges almost surely to a finite random variable and ∑t=0∞at<∞ _t=0^∞a_t<∞ almost surely. We now show that the sequence generated by the algorithm satisfies the conditions of this lemma. Define Jt:=J(t,t),∇t:=∇J(t,t),Xt:=Jt−Jinf≥0.J_t:=J( θ^t,V^t), _t:= _ θJ( θ^t,V^t), X_t:=J_t-J_ ≥ 0. From the one-step bound (cf. (38)), for all t≥0t≥ 0, [Jt+1∣Φt]≤Jt−ηt2‖∇t‖2+Lηt22σ2.E[J_t+1 _t]≤ J_t- _t2\| _t\|^2+ L _t^22σ^2. Subtracting JinfJ_ from both sides yields [Xt+1∣Φt]≤Xt−at+bt,at:=ηt2‖∇t‖2,bt:=Lηt22σ2.E[X_t+1 _t]≤ X_t-a_t+b_t, a_t:= _t2\| _t\|^2, b_t:= L _t^22σ^2. By Assumption 7 we have ∑t=0∞ηt2<∞ _t=0^∞ _t^2<∞, hence ∑t=0∞bt<∞ _t=0^∞b_t<∞. Therefore all the conditions of Lemma E.5 are satisfied. Applying the lemma implies that XtX_t converges almost surely to a finite random variable, and ∑t=0∞ηt‖∇t‖2<∞a.s. _t=0^∞ _t\| _t\|^2<∞ .s. Finally, since ∑t=0∞ηt=∞ _t=0^∞ _t=∞ (Assumption 7), the above summability implies lim inft→∞‖∇t‖=0a.s. _t→∞\| _t\|=0 .s. This proves Theorem E.4. ∎ Appendix F Trade-off between Privacy, Communication, and Local Computation Our proposed algorithmic solution in Sec. 5 leverages sharing of raw latent representations to lower the communication cost, according to the objective in Prob. 1. This is compatible with standard regulatory constraints on privacy, for instance when latent representations do not allow direct identification of individuals or when appropriate safeguards such as aggregation, anonymization, or secure communication protocols are in place, as commonly assumed in federated learning systems (Kairouz et al., 2021; McMahan et al., 2017). However, as highlighted in Remark 3, in some circumstances more stringent regulatory constraints on privacy impair the sharing of raw latent representations. For example, this occurs when latent representations may still encode sensitive or identifiable information and are therefore susceptible to reconstruction or inference attacks, such as model inversion (Fredrikson et al., 2015), membership inference (Shokri et al., 2017), or attribute inference (Melis et al., 2019), making their direct exchange incompatible with strict privacy requirements. Our proposed approach naturally extends to this more stringent setting. Instead of sharing raw latent representations, agents exchange their transported counterparts, i.e., latent features mapped through the restriction maps associated with the edges. This limits the direct exposure of local features. However, this enhanced privacy comes at a cost: it requires additional local computation and increases the communication overhead, as detailed below. Concerning the local computational cost, each agent i∈i needs to encode its latent representations toward both sets of neighbors (i)−N(i)^- and (i)+N(i)^+. To this end, we adopt the formulation in Eq. 10 before reparameterization, where both orthogonal and Stiefel maps explicitly appear, allowing consistent encoding across heterogeneous neighbors. Specifically, i must communicate jitit∈ℝdj×KV_ji^tA_i^t ^d_j× K to neighbors j∈(i)+j (i)^+, where jit∈St(dj,di)V_ji^t (d_j,d_i) is the outgoing embedding matrix, and jitit∈ℝdi×KO_ji^tA_i^t ^d_i× K to neighbors j∈(i)−j (i)^-, where jit∈O(di)O_ji^t (d_i) is an orthogonal transformation. Hence, in this setting we cannot exploit the reparameterized form in Eq. 10, which avoids the explicit use of orthogonal maps. In detail, after receiving the encoded latent representations from its neighbors, namely ijtjt from j∈(i)−andijtjt from j∈(i)+;V_ij^tA_j^t from j (i)^- _ij^tA_j^t from j (i)^+\,; (41) under stringent privacy constraints, agent i must solve the following problems to update its restriction maps: jit _ji^t =argminji∈O(di)‖jiit−ijtjt‖F2,,∀,j∈(i)−, = *arg\,min_O_ji (d_i)\; O_jiA_i^t-V_ij^tA_j^t_F^2,, ∀,j (i)^-\,, (P2c) jit _ji^t =argminji∈St(dj,di)‖ijtjt−jiit‖F2,,∀,j∈(i)+. = *arg\,min_V_ji (d_j,d_i)\; O_ij^tA_j^t-V_jiA_i^t_F^2,, ∀,j (i)^+\,. (P2d) Unlike the case of standard privacy constraints, where for j∈(i)−j (i)^- we can only update the incoming restriction map ijV_ij and solve Prob. (P2a) via a thin SVD at a computational cost of (didj2)O(d_id_j^2) with dj≤did_j≤ d_i, Prob. (P2c) is a canonical orthogonal Procrustes problem. Its solution is obtained via a classical SVD of ijtjtit⊤V_ij^tA_j^tA_i^t at a cost of (di3)O(d_i^3), thus leading to an increase in the local computational burden. Conversely, similarly to Problem (P2b), for j∈(i)+j (i)^+ the update of the outgoing restriction map jiV_ji can be computed via a thin SVD of ijtjtit⊤O_ij^tA_j^tA_i^t at a cost of (djdi2)O(d_jd_i^2), where in this case dj≥did_j≥ d_i. Please notice that, more stringent privacy constraints affect only the update of the restriction maps. Indeed, the local gradient computation in Eq. 19 remains unchanged: ∇iℛ|i= _ _iR_A|_i= λiK[∑j∈(i)−∑k∈(∇ifi(ik))⊤ji⊤(jifi(ik)−ijfj(jk)) _iK [ _j (i)^- _k ( _ _if_ _i(x_i^k) ) O_ji (O_jif_ _i (x_i^k )-V_ijf_ _j (x_j^k ) ) (42) −∑j∈(i)+∑k∈(∇ifi(ik))⊤ji⊤(ijfj(jk)−jifi(ik))] - _j (i)^+ _k ( _ _if_ _i(x_i^k) ) V_ji (O_ijf_ _j (x_j^k )-V_jif_ _i (x_i^k ) ) ] = = λiK[∑j∈(i)−∑k∈(∇ifi(ik))⊤(fi(ik)−ji⊤ijfj(jk)) _iK [ _j (i)^- _k ( _ _if_ _i(x_i^k) ) (f_ _i (x_i^k )-O_ji V_ijf_ _j (x_j^k ) ) −∑j∈(i)+∑k∈(∇ifi(ik))⊤(ji⊤ijfj(jk)−fi(ik))] - _j (i)^+ _k ( _ _if_ _i(x_i^k) ) (V_ji O_ijf_ _j (x_j^k )-f_ _i (x_i^k ) ) ] =(a) (a)= λiK∑j∈(i)∑k∈(∇ifi(ik))⊤(fi(ik)−ji⊤ijfj(jk)); _iK _j (i) _k ( _ _if_ _i(x_i^k) ) (f_ _i (x_i^k )-O_ji V_ijf_ _j (x_j^k ) )\,; where in (a)(a), for those j∈(i)+j (i)^+ we used that for a network sheaf ji⊤ij=ji⊤ijV_ji O_ij=O_ji V_ij. Importantly, ji⊤ijO_ji V_ij is exactly the reparameterization exploited in Eq. 10, thus showing the equivalence between Eq. 19 and Eq. 42. Regarding the communication cost, sharing transported latent representations increases the communication overhead toward neighbors j∈(i)+j (i)^+. Under standard privacy constraints, agent i transmits it∈ℝdi×KA_i^t ^d_i× K, whereas in the more stringent setting it must send jiit∈ℝdj×KV_jiA_i^t ^d_j× K, with dj≥did_j≥ d_i. Thus, the communicated payload scales with djd_j, and strictly increases whenever dj>did_j>d_i. Our modified Sheaf-FRL protocols under stringent privacy regulatory constraints for both the heterogeneous and homogeneous cases are given in Algs. 3 and 4, respectively. Additionally, Tab. 1 details the privacy vs. communication and computational costs trade-off. Table 1: Trade-off between privacy and communication and local computation cost per agent. The local computation refers to the update of the restriction maps, i.e., the part influenced by more stringent privacy constraints. To aid comparison, for the heterogeneous case we highlight in blue the larger of did_i and djd_j (either one when di=djd_i=d_j) when considering either j∈(i)−j (i)^- or j∈(i)+j (i)^+. In red, we instead highlight we instead highlight the (potential) increase in cost implied by more stringent privacy constraints. Standard Privacy Strict Privacy Heterogeneous Communication (|(i)|diK)O(|N(i)|d_iK) (|(i)−|diK+K∑j∈(i)+dj)O (|N(i)^-|d_iK+K\! _j (i)^+\! [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0d_j ) Local computation (di∑j∈(i)−dj2+di2∑j∈(i)+dj)O ( [rgb]0,0,1 [named]pgfstrokecolorrgb0,0,1d_i\! _j (i)^-\!d_j^2+d_i^2\! _j (i)^+\! [rgb]0,0,1 [named]pgfstrokecolorrgb0,0,1d_j ) (|(i)−|di3+di2∑j∈(i)+dj)O (|N(i)^-| [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0d_i^3+d_i^2\! _j (i)^+\! [rgb]0,0,1 [named]pgfstrokecolorrgb0,0,1d_j ) Homogeneous Communication (|(i)|dK)O(|N(i)|dK) (|(i)|dK)O(|N(i)|dK) Local computation (|(i)+|d3)O(|N(i)^+|d^3) (|(i)−∪(i)+|d3)O(| [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0N(i)^- (i)^+|d^3) Non-identifiability and privacy guarantees. Tab. 1 highlights a clear trade-off. Under stricter privacy requirements, the proposed protocol incurs higher communication and local computational costs, since agents exchange transported latent representations and must explicitly update the corresponding restriction maps. The benefit, however, is a stronger form of privacy: the communicated messages no longer reveal the latent pilots in their original coordinates. More precisely, when only transported pilots are shared, an observer that does not know the underlying restriction maps cannot uniquely reconstruct the original latent representations, but can only recover geometric information that is invariant under isometries, as formalized in the following proposition. Proposition F.1 (Non-identifiability of latent pilots under unknown maps). Let ∈ℝd×KA ^d× K be a matrix of latent pilot representations, and let =Y=TA, where T is an unknown restriction map satisfying ⊤=dT T=I_d. This includes both the homogeneous case ∈O(d)T (d) and the heterogeneous case ∈St(D,d)T (D,d) with D≥dD≥ d. Then the latent pilots A are not uniquely identifiable from Y without knowledge of T. In particular, A is determined only up to an orthogonal transformation. Proof. Let ∈O(d)Q (d) and ′=A =QA. Then ==⊤′Y=TA=TQ A , and (⊤)⊤(⊤)=d(TQ ) (TQ )=I_d. Thus ⊤TQ is a valid map, implying that A is identifiable only up to orthogonal transformations. ∎ Algorithm 3 Sheaf-FRL under stringent privacy regulatory constraints 0: Oriented =(,ℰ)G=(N,E), pilots A, λ>0λ>0, ηθ>0 _θ>0, iterations T 1: Initialize i0=(i0,i0)i∈\ θ_i^0=( _i^0, ψ_i^0)\_i , ji0∈O(di)j∈(i)−,i∈\O_ji^0 (d_i)\_j (i)^-,\,i , ji0∈St(dj,di)j∈(i)+,i∈\V_ji^0 (d_j,d_i)\_j (i)^+,\,i 2: for t=0,…,T−1t=0,…,T-1 do 3: for all i∈i in parallel do 4: it←[fit(ik)]k∈A_i^t←[f_ _i^t(x_i^k)]_k 5: end for 6: Each node i broadcasts jititO_ji^tA_i^t to those j∈(i)−j (i)^- and jititV_ji^tA_i^t to j∈(i)+j (i)^+ 7: for all i∈i in parallel do 8: for j∈(i)j (i) do 9: if j∈(i)−j (i)^- then 10: [,,⊤]←SVD(ijtjtit⊤)[U, ,W ] (V_ij^tA_j^tA_i^t ) 11: jit←⊤O_ji^t 12: else 13: [,,⊤]←thinSVD(ijtjtit⊤)[U, ,W ] (O_ij^tA_j^tA_i^t ) 14: jit←⊤V_ji^t 15: end if 16: end for 17: end for 18: for all i∈i in parallel do 19: it+1←it−ηθ(∇iℒi(it)+it) θ_i^t+1← θ_i^t- _θ ( _ θ_iL_i( θ_i^t)+r_i^t ) 20: end for 21: end for Algorithm 4 Sheaf-FRL under stringent privacy regulatory constraints (homogeneous case) 0: Oriented =(,ℰ)G=(N,E), pilots A, λ>0λ>0, ηθ>0 _θ>0, iterations T 1: Initialize i0=(i0,i0)i∈\ θ_i^0=( _i^0, ψ_i^0)\_i , ji0∈O(d)j∈(i),i∈\O_ji^0 (d)\_j (i),i 2: for t=0,…,T−1t=0,…,T-1 do 3: for all i∈i in parallel do 4: it←[fit(ik)]k∈A_i^t←[f_ _i^t(x_i^k)]_k 5: end for 6: Each node i exchanges jititO_ji^tA_i^t with all j∈(i)j (i) 7: for all i∈i in parallel do 8: for all j∈(i)j (i) do 9: [,,⊤]←SVD(ijtjtit⊤)[U, ,W ] (O_ij^tA_j^tA_i^t ) 10: jit←⊤O_ji^t 11: end for 12: end for 13: for all i∈i in parallel do 14: it+1←it−ηθ(∇iℒi(it)+it) θ_i^t+1← θ_i^t- _θ ( _ θ_iL_i( θ_i^t)+r_i^t ) 15: end for 16: end for Appendix G Supervised classification experimental details All experiments run on MNIST with the label-shift-induced data heterogeneity of Eq. 23. Every agent trains with stochastic gradient descent (learning rate 10−210^-2, momentum 0.90.9, weight decay 5⋅10−45· 10^-4) and batch size 6464; the fifteen-agent runs additionally clip gradient norms at 1.01.0. A held-out shared pilot set (10%10\% of the training pool, restricted to the classes observed by every agent so that no encoder is asked to embed out-of-distribution samples) serves two purposes: it provides the pilots exchanged during sheaf-based training, and it is the data on which whitening layers of Sheaf-FRL are trained, and (post-training) alignment maps are fitted. Reported accuracies are computed on the private test sets, different from both training and pilot data. Two-agent heterogeneous pair (Fig. 3). The two agents are drawn from a parametric CNN family indexed by the bottleneck dimension d∈16,32,64,128,256,512d∈\16,32,64,128,256,512\, matched so that the last block of both encoders outputs the same d (Tab. 3). Each convolutional block consists of a convolution, batch normalization, ReLU, and max-pooling; the final feature map is globally pooled, so the latent dimension equals the width of the last block. Agent 0 uses a dropout of 0.30.3 while agent 11 uses a dropout of 0.10.1. Agent 0’s target classes are 4,…,9\4,…,9\ and agent 11’s are 0,…,5\0,…,5\, at a fixed shift strength s=0.7s=0.7; the graph has two nodes corresponding to the agents and a single edge connecting them. Training lasts 2020 epochs with the common optimizer above. Fifteen-agent network (Fig. 2). Tab. 2 lists the fifteen architectures and the target classes assigned to each agent. All encoders use batch normalization; the latent (stalk) dimension did_i equals the width of the last convolutional block. The proposed methods run for a total of 100100 epochs. Table 2: The fifteen heterogeneous CNN agents of the multi-agent benchmark (Section 6). Encoder: output channels of each convolutional block; head: hidden widths of the MLP classifier; did_i: latent (stalk) dimension. Total: 3.413.41M parameters. Agent Encoder widths Head widths Dropout did_i Target classes iC_i 0 (32,64,128)(32,64,128) (256,128,64)(256,128,64) 0.300.30 128128 4,5,6,7,8\4,5,6,7,8\ 1 (32,64,128,224)(32,64,128,224) (120)(120) 0.100.10 224224 0,1,2,3,4\0,1,2,3,4\ 2 (16,32,64)(16,32,64) (128)(128) 0.200.20 6464 0,1,2,3,4,5\0,1,2,3,4,5\ 3 (64,128,128)(64,128,128) (256,128)(256,128) 0.250.25 128128 4,5,6,7,8,9\4,5,6,7,8,9\ 4 (32,48,96,192)(32,48,96,192) (192,96)(192,96) 0.300.30 192192 0,1,2,7,8,9\0,1,2,7,8,9\ 5 (24,48,96)(24,48,96) (96)(96) 0.150.15 9696 2,3,4,5,6\2,3,4,5,6\ 6 (32,64,128,256)(32,64,128,256) (256)(256) 0.400.40 256256 1,2,3,4,5,6\1,2,3,4,5,6\ 7 (24,48,96,144)(24,48,96,144) (64,32)(64,32) 0.100.10 144144 3,4,5,6,7\3,4,5,6,7\ 8 (48,96,160)(48,96,160) (160,80)(160,80) 0.200.20 160160 0,2,4,6,8\0,2,4,6,8\ 9 (32,80)(32,80) (100)(100) 0.050.05 8080 5,6,7,8,9\5,6,7,8,9\ 10 (40,80,120)(40,80,120) (120,60)(120,60) 0.200.20 120120 1,2,3,4,5\1,2,3,4,5\ 11 (48,96,192,240)(48,96,192,240) (256,128)(256,128) 0.200.20 240240 0,1,2,8,9\0,1,2,8,9\ 12 (20,40,60,80)(20,40,60,80) (80)(80) 0.200.20 8080 3,4,5,6,7,8\3,4,5,6,7,8\ 13 (32,64,96)(32,64,96) (192,96)(192,96) 0.250.25 9696 1,4,6,8,9\1,4,6,8,9\ 14 (56,112,160)(56,112,160) (128)(128) 0.100.10 160160 0,1,2,3,7\0,1,2,3,7\ Table 3: The shared two-agent architecture family used at every bottleneck dimension d in Fig. 3; both agents’ encoders are matched to end at width d. Encoder/head: output channels of each convolutional block / hidden widths of the MLP classifier. Dropout is 0.30.3 for agent 0 and 0.10.1 for agent 1. d Agent Encoder widths Head widths 16 0 (4,8,16)(4,8,16) (8,4)(8,4) 1 (8,16)(8,16) (8)(8) 32 0 (8,16,32)(8,16,32) (16,8)(16,8) 1 (16,32)(16,32) (16)(16) 64 0 (16,32,64)(16,32,64) (32,16)(32,16) 1 (32,64)(32,64) (32)(32) 128 0 (32,64,128)(32,64,128) (64,32)(64,32) 1 (64,128)(64,128) (64)(64) 256 0 (64,128,256)(64,128,256) (128,64)(128,64) 1 (128,256)(128,256) (128)(128) 512 0 (64,128,256,512)(64,128,256,512) (128,64)(128,64) 1 (64,256,512)(64,256,512) (128,64)(128,64) Selection of the regularization coefficient. For Sheaf-FRL, as well as Sheaf-FMTL, ComFed, FedProto, and FedMuscle baselines, we need to find the best regularization coefficient. The applied strategy in each architecture configuration under consideration—either the bottleneck dimension in the two-agent sweep, or the distribution-shift strength in the fifteen-agent benchmark—is a grid search for the best λ value in the [10−5,1][10^-5,1] interval. Method-specific hyperparameters. Beyond the regularization coefficient, every baseline also carries its own additional, fixed hyperparameters that are untouched by the cross-validation above. Sheaf-FMTL introduces two further constants, the local-regularization weight γ=10−3γ=10^-3 and the personalization rate η=10−2η=10^-2, both held fixed across every shift strength and every bottleneck dimension. For further information to support the selection process of the Sheaf-FMTL hyperparameter γ refer to the next paragraph. ComFed instead learns, for every agent i, an unconstrained linear projection matrix i∈ℝr×diP_i ^r× d_i mapping that agent’s local did_i-dimensional latent space—the right, input-side dimension of iP_i—into a shared space of dimension r, the left, output-side dimension common to every agent, in which class prototypes from all agents are aligned. We fix r=64r=64 in the fifteen-agent benchmark. In the two-agent bottleneck sweep, where every configuration already forces both agents to share a common latent dimension d, we instead set proj_dim=dproj\_dim=d at each bottleneck value, so that ComFed’s shared space coincides with the bottleneck dimension already under study, rather than further compressing or expanding it. FedProto maintains a single global prototype per class, updated as an exponential moving average of the agents’ local per-class embeddings with momentum 0.90.9. FedMuscle organizes training into repeating communication rounds, each consisting of four local epochs of ordinary task-loss training followed by one additional epoch in which a contrastive alignment loss, with temperature 0.10.1, pulls every agent’s representations toward the network average. Memory footprint and choice of γ for Sheaf-FMTL. Let the number of parameters of agents i and j be respectively did_i and djd_j, then the restriction maps matrix of Sheaf-FMTL ijP_ij acts directly in parameter space, and its size is dij×did_ij× d_i with dij=max(1,⌊γ⋅min(di,dj)⌋)d_ij= (1, γ· (d_i,d_j) ). The aggregate storage across all edges grows as O(γ∑(i,j)∈ℰmin(di,dj)di)O(γ _(i,j) (d_i,d_j)\,d_i), with the hyperparameter γ controlling how fast the memory usage scales with the architecture sizes. With the 1515-agent heterogeneous ensemble of models of Tab. 2 and the class-overlap graph of Sec. 6, this places a hard ceiling on the values of γ that fit in the machine memory (GPU RAM), well before any accuracy trade-off becomes relevant. Fig. 4 reports a blue curve providing the theoretical storage occupied by the restriction maps ijP_ij and model parameters in the real 1515-agent setup, and the green markers then represent the empirical peak resident-memory measurements from isolated single-epoch instantiations of Sheaf-FMTL at escalating γ. We use γ=10−3γ=10^-3 throughout the 1515-agent benchmark (Tab. 2)—a margin below the maximum available memory on the considered machine that accounts for the full evaluation protocol rather than a single run. Figure 4: Aggregate memory footprint of the Sheaf-FMTL restriction maps ij\P_ij\ as a function of γ, for the 1515-agent architecture and communication graph of Sec. 6 (log-log axes). Solid curve: exact storage in the half-precision format used at rest. Dashed line: available system memory. Markers: measured peak resident memory of an isolated, single-epoch Sheaf-FMTL run at each γ (crosses mark values at which the run was terminated for exceeding available memory). The operating point used throughout the paper (γ=10−3γ=10^-3) sits below the single-run ceiling, leaving headroom for the repeated model instantiations of the full evaluation protocol (hyperparameter search preceding each reported run).