Paper deep dive
Federated Latent Space Alignment for Multi-user Semantic Communications
Giuseppe Di Poce, Mario Edoardo Pandolfo, Emilio Calvanese Strinati, Paolo Di Lorenzo
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 7/21/2026, 12:08:50 AM
Summary
This paper proposes a federated latent space alignment method for multi-user semantic communications in a downlink scenario. It addresses semantic mismatches caused by differing latent representations in AI-native devices by employing a shared semantic pre-equalizer at the Access Point (AP) and local semantic equalizers at user devices. The optimization is performed using a federated Alternating Direction Method of Multipliers (ADMM) framework to minimize the distance between transmitted and target latent spaces while respecting power and complexity constraints. Numerical results on the CIFAR-10 dataset demonstrate the approach's effectiveness in goal-oriented semantic communication.
Entities (11)
Relation Signals (8)
Giuseppe Di Poce → authored → Federated Latent Space Alignment
confidence 99% · Authors list includes Giuseppe Di Poce
Mario Edoardo Pandolfo → authored → Federated Latent Space Alignment
confidence 99% · Authors list includes Mario Edoardo Pandolfo
Federated Latent Space Alignment → addresses → Latent Space Misalignment
confidence 95% · mitigating latent space misalignment in multi-agent AI-native semantic communications
Federated Latent Space Alignment → employs → Semantic Pre-equalizer
confidence 95% · implements a protocol that shares a semantic pre-equalizer at the AP
Federated Latent Space Alignment → employs → Semantic Equalizer
confidence 95% · shares a semantic pre-equalizer at the AP and local semantic equalizers at user devices
Federated Latent Space Alignment → uses → Federated ADMM
confidence 95% · numerically solved using a federated alternating direction method of multipliers (ADMM) framework
Federated Latent Space Alignment → validateson → CIFAR-10
confidence 95% · Numerical results confirm the robustness of the proposed method in a multi-user goal-oriented semantic communication scenario for image classification... We consider the CIFAR-10 dataset
6G-GOALS → funded →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Semantic communication aims to convey meaning for effective task execution, but differing latent representations in AI-native devices can cause semantic mismatches that hinder mutual understanding. This paper introduces a novel approach to mitigating latent space misalignment in multi-agent AI- native semantic communications. In a downlink scenario, we consider an access point (AP) communicating with multiple users to accomplish a specific AI-driven task. Our method implements a protocol that shares a semantic pre-equalizer at the AP and local semantic equalizers at user devices, fostering mutual understanding and task-oriented communication while considering power and complexity constraints. To achieve this, we employ a federated optimization for the decentralized training of the semantic equalizers at the AP and user sides. Numerical results validate the proposed approach in goal-oriented semantic communication, revealing key trade-offs among accuracy, com- munication overhead, complexity, and the semantic proximity of AI-native communication devices.
Tags
Links
- Source: https://arxiv.org/abs/2602.17271v1
- Canonical: https://arxiv.org/abs/2602.17271v1
Trouble viewing inline? Open PDF directly →
Full Text
32,422 characters extracted from source content.
Expand or collapse full text
Federated Latent Space Alignment for Multi-user Semantic Communications Giuseppe Di Poce3,∗, Mario Edoardo Pandolfo1,2,∗, Emilio Calvanese Strinati3, and Paolo Di Lorenzo2,4 1 DIAG Department, Sapienza University of Rome, via Ariosto 25, Rome, Italy. 2 Consorzio Nazionale Interuniversitario per le Telecomunicazioni (CNIT), Parma, Italy. 3 CEA Leti, University Grenoble Alpes, 38000, Grenoble, France. 4 DIET Department, Sapienza University of Rome, Via Eudossiana 18, Rome, Italy. e-mail: marioedoardo.pandolfo, paolo.dilorenzo@uniroma1.it, giuseppe.dipoce, emilio.calvanese-strinati@cea.fr This work was funded by the 6G-GOALS project (6G SNS-JU Horizon, n.101139232), the EU under NextGenerationEU RESTART (PE00000001), the SNS JU 6GARROW project (Horizon, n. 101192194), and the French government (France 2030 ANR, ref. 22-PEFT-0010). ∗Equal contribution. Abstract Semantic communication aims to convey meaning for effective task execution, but differing latent representations in AI-native devices can cause semantic mismatches that hinder mutual understanding. This paper introduces a novel approach to mitigating latent space misalignment in multi-agent AI-native semantic communications. In a downlink scenario, we consider an access point (AP) communicating with multiple users to accomplish a specific AI-driven task. Our method implements a protocol that shares a semantic pre-equalizer at the AP and local semantic equalizers at user devices, fostering mutual understanding and task-oriented communication while considering power and complexity constraints. To achieve this, we employ a federated optimization for the decentralized training of the semantic equalizers at the AP and user sides. Numerical results validate the proposed approach in goal-oriented semantic communication, revealing key trade-offs among accuracy, communication overhead, complexity, and the semantic proximity of AI-native communication devices. I Introduction and Motivation Traditional communication systems emphasize accurate transmission of bits or symbols. However, the increasing number of connected devices and data-intensive applications is rapidly straining network capacity [1]. Time-critical tasks like autonomous driving and smart surveillance demand reliable, low-latency AI services while managing energy, bandwidth, and computation efficiently. Despite advances, current wireless systems increasingly consume bandwidth and energy to meet data demands, underscoring the need for a new, more efficient communication paradigm. Semantic communication is emerging as a key enabler for 6G networks by representing raw symbols in a compressed, task-relevant form, thereby reducing bandwidth use and latency [2, 3]. Recent research builds on this foundation, exploring joint source-channel coding [3], semantic extraction [4], goal-oriented system design [5], semantic reasoning [6], and generative AI [7]. Artificial intelligence (AI) is a key enabler of semantic communications, where deep neural networks (DNNs) embed raw data into low-dimensional semantic features for transmission [8, 3]. Effective communication requires a shared latent space between transmitter and receiver. However, devices may encode the same information into different latent representations due to independent training procedures or architectural discrepancies. This misalignment introduces semantic noise, which can significantly impair mutual understanding between communicating agents. Such discrepancies are not exceptions but rather common occurrences—particularly in multi-vendor environments where parties are unwilling or unable to share models, training datasets, or other proprietary assets. In these cases, joint end-to-end training of encoding networks or the exchange of DNN models is often infeasible due to privacy concerns and intellectual property constraints. As a result, an effective semantic alignment mechanism becomes essential to ensure consistency and interoperability across heterogeneous systems. This challenge, known as semantic channel equalization, has prompted various alignment solutions, including relative representations (RRs) [9], linear mappings [10], and optimal transport (OT)-based on global invariance [11, 12]. Recent works have further addressed semantic noise and channel impairments via OT-codebooks [13] and dynamic R-based schemes [14, 15]. Despite growing interest in semantic alignment, most studies target single-link scenarios, leaving the multi-user case largely unexplored. Contributions. The goal of this work is to address the latent space mismatch in a downlink semantic communication scenario, where an AP communicates with a set of AI-native users, each equipped with its own DNN model. The proposed method implements a protocol featuring a shared semantic pre-equalizer at the AP and local semantic equalizers at user devices. This architecture facilitates mutual understanding, implements semantic compression, and enables task-oriented communication under power and complexity constraints. Assuming linear semantic equalization modules at the transmitter and receiver, the optimization problem is cast as a block-convex program and is numerically solved using a federated alternating direction method of multipliers (ADMM) framework [16], enabling decentralized training of both the semantic pre-equalizer and equalizers. Finally, a message exchange protocol is introduced to reduce communication overhead and latency, while preserving the privacy of users’ personal latent spaces. As far as we know, this is the first aligner designed specifically for broadcast multi-user semantic communication scenarios available in the literature. Numerical results confirm the robustness of the proposed method in a multi-user goal-oriented semantic communication scenario for image classification. I System Model We consider a downlink system where an AP and L users, all endowed with pre-trained DNNs, communicate semantically via transmitted and interpreted latent representations. A pictorial sketch is illustrated in Fig. 1. Let AP∈ℝds_ AP ^d be the vector of semantic features extracted at the TX side from a data point ∈ℝqz ^q. The set of all semantic vectors APs_ AP represents the TX semantic latent space at the AP. Every user l is trained to interpret a different encoding scheme than the one used by the AP, i.e., it requires the reception of a different semantic feature vector, say l∈ℝmls_l ^m_l (corresponding to z), to correctly interpret the transmitted message or effectively perform a given task (e.g., classification). The set of all semantic vectors ls_l represents the RX semantic latent space at each user l∈1,…,Ll∈\1,…,L\. Latent space mismatches introduce semantic noise, requiring equalization to ensure mutual understanding between the AP and users. Our approach to semantic equalization exploits the presence of MIMO wireless channels between the AP and the users. Assuming, without loss of generality, that d is even, we proceed by pairing the first half of the semantic features in T∈ℝds_T ^d with the second half to form complex symbols, yielding an input vector ∈ℂd2x d2. Then, assuming the AP is endowed with NTN_T antennas, we exploit a semantic pre-equalizer that maps the complex vector ∈ℂd2x d2 into the vector ¯∈ℂKNT x ^KN_T to be transmitted over K wireless channel usages. The semantic pre-equalizer at the AP implements a learnable transformation represented by the function f:ℂd2→ℂKNTf:C d2 ^KN_T. This transformation enables the semantic compression of the TX latent space (since typically d2≫K d2 K), with a compression factor given by ζ=Kd/2. ζ= Kd/2. For each user l∈1,…,Ll∈\1,…,L\, we consider the presence of a MIMO flat Rayleigh fading channel described by the matrix ¯l∈ℂNT×NR H_l ^N_T× N_R, where NRN_R denotes the number of antennas at the RX side, assumed to be the same for all users. For the sake of simplicity, we assume that all channels remain constant over the duration of K consecutive transmissions. Additionally, in our simulations, we consider the case where perfect channel state information (CSI) is available at both TX and RX, while noting that TX may also operate without CSI. Then, at every user l∈1,…,Ll∈\1,…,L\, we have a semantic equalizer that maps the received symbols into a complex vector ^l∈ℂml2 y_l m_l2, via the learnable transformation gl:ℂKNR→ℂml2g_l:C^KN_R m_l2. Overall, for every user l, the considered semantic MIMO communication channel can be compactly written as: ^l=gl(lf()+l),l=1,…,N, y_l=g_l(H_lf(x)+n_l), l=1,…,N, (1) where, l=K⊗¯l∈ℂKNR×KNTH_l=I_K H_l ^KN_R× KN_T, and l∈ℂKNRn_l ^KN_R denotes the noise vector at the l-th user, which follows a complex Gaussian distribution (,n)CN(0, _n) for all l∈1,…,Nl∈\1,…,N\. Finally, at every user l, the complex latent vector ^l∈ℂml2 y_l m_l2 in (1) is converted into an mlm_l-dimensional real latent vector, say ^l s_l, by inverting the halving operation done at the TX side. Our aim is to act on the learnable transformations f (at the BS side, shared among all users) and gll=1L\g_l\_l=1^L (at the user side) in (1) to perform the best possible semantic alignment between the AP and the users’ latent spaces. This can be obtained by minimizing the (semantic) distance between the spaces composed by the vectors ls_l and ^l s_l, for all l=1,…,Ll=1,…,L, over a limited set of available latent vectors, acting as semantic pilots for logic-channel estimation. In the sequel, we will illustrate the proposed federated strategy for latent space alignment. Figure 1: Overview of the proposed system model. I Federated Latent Space Alignment We consider a semantic pre-equalizer function f modeled as a linear transformation, represented by matrix ∈ℂKNT×d2F ^KN_T× d2; similarly, users’ equalization functions gll=1L\g_l\_l=1^L are modeled by matrices l∈ℂml2×KNRG_l m_l2× KN_R, with l∈1,…,Ll∈\1,…,L\. Also, w.l.o.g., we consider the application of pre-whitening to standardize the covariance of the transmitted symbols (x). Under these assumptions, the channel model (1) boils down to: ^l=ll+ll,l=1,…,L. y_l=G_lH_lFx+G_ln_l, l=1,…,L. (2) Our optimization criterion aims at minimizing the (semantic) distance between the signals given by model (2) and the target latent spaces of all users l∈1,…,Ll∈\1,…,L\, with respect to the (shared) linear semantic pre-equalizer F and the (local) equalizers ll=1L\G_l\_l=1^L. To this aim, let us assume that to have a training set of semantic pilots composed of n examples (i,i,l)(x_i,y_i,l), for i∈ri _r and l=1,…,Ll=1,…,L. Although various distance metrics can be applied within this framework, we opt for mean-squared error in what follows, as it offers a straightforward yet effective measure of latent space discrepancies. Mathematically, the problem is cast as an empirical risk minimization: min,ll=1L _F,\G_l\_l=1^L 1Ln∑l=1L∑i∈r‖i,l−l(li+l)‖F2 1Ln _l=1^L _i _rE \|y_i,l-G_l (H_lFx_i+n_l ) \|_F^2 s.t. tr(H)≤PT (FF^H)≤ P_T (3) where the constraint in (3) imposes a maximum budget PTP_T on the power transmitted by the AP. Let us now define ∈ℂd2×nX d2× n be the matrix that collects all training samples, where each column ix_i represents an individual input, and l∈ℂm2×nY_l m2× n be the matrix containing the corresponding latent column vectors i,ly_i,l at the l-th user, for all l=1,…,Ll=1,…,L. Then, leveraging the zero-mean property of the noise components ll=1L\n_l\_l=1^L, the objective function of (3) can be recast as: min,ll=1L _F,\G_l\_l=1^L 1Ln∑l=1L‖l−ll‖F2+∑l=1Ltr(lnlH) \; 1Ln _l=1^L \|Y_l-G_lH_lFX \|_F^2+ _l=1^Ltr (G_l _nG_l^H ) s.t. tr(H)≤PT (FF^H)≤ P_T (4) Problem (4) is non-convex, but enjoys a block-convex structure in the single variables, once having fixed the others. Thus, in the sequel, we will leverage a federated ADMM optimization framework to iteratively optimize the shared semantic pre-equalizer and the local users’ equalizers. I-A Federated ADMM derivation As a first step, we recast (4) in an equivalent manner, introducing the auxiliary variable Z and the set Ω=|tr(H)≤PT. =\Z\,|\, tr(Z^H)≤ P_T\. Then, we equivalently get: min,ll=1L _F,\G_l\_l=1^L 1Ln∑l=1L‖l−ll‖F2+IΩ() \; 1Ln _l=1^L \|Y_l-G_lH_lFX \|_F^2+I_ (Z) +∑l=1Ltr(lnlH) + _l=1^Ltr (G_l _nG_l^H ) s.t. −= F-Z=0 (5) ADMM iteratively minimizes the (scaled) augmented Lagrangian of problem (5) given by: Lρ(,,,) L_ρ(F,G,Z,U) =1Ln∑l=1L[∥l−l∥F2+IΩ() = 1Ln _l=1^L [ \|Y_l-G_lH_lFX \|_F^2+I_ (Z) +tr(∑l=1LlnlH)]+ρ∥−+∥F2 -28.45274pt+tr ( _l=1^LG_l _nG_l^H ) ]+ρ \|F-Z+U \|_F^2 (6) with respect to primal variables, while minimizing it with respect to dual variables [16]; here, U is the (scaled) Lagrange multiplier enforcing the constraint in (5), and ρ is a positive parameter. Now, we derive the single steps of ADMM. The lG_l step. The Lagrangian (6) is separable over the ll=1L\G_l\_l=1^L. Setting the gradient of (6) w.r.t. lHG_l^H to zero, and fixing F at time t, the update of lG_l at time t+1t+1 reads as: l(t+1) _l^(t+1) =l(l(t))H((l(t))(l(t))H+nn)−1 =Y_l(H_lF^(t)X)^H ((H_lF^(t)X)(H_lF^(t)X)^H+n _n )^-1 (7) The F step. With G, Z and U fixed, the update for F at iteration t+1t+1 can be obtained setting the gradient of (6) w.r.t. F to zero. The solution depends on federated information that comes from all users l=1,…,Ll=1,…,L, and can be expressed as: (t+1)=1L∑l=1L^lF^(t+1)= 1L _l=1^L F_l (8) where the single variables ^ll=1L\ F_l\_l=1^L can be found as the solutions of the subproblems: ^l= F_l= argminl1n‖l−l(t+1)ll‖F2 \, _F_l\; 1n||Y_l-G_l^(t+1)H_lF_lX||_F^2 +ρ‖l−(t)+(t)‖F2 \;\;+ρ||F_l-Z^(t)+U^(t)||^2_F (9) for l=1,…,Ll=1,…,L. Setting the gradient of (9) to zero, we get: (l(t+1)l)H(l(t+1)l)l(H)+nρl (G^(t+1)_lH_l)^H(G^(t+1)_lH_l)F_l(XX^H)+nρF_l −nρ((t)−(t))−(l(t+1)l)HlH=. \;\;-nρ(Z^(t)-U^(t))-(G_l^(t+1)H_l)^HY_lX^H=0. (10) Now, letting l=(l(t+1)l)H(l(t+1)l), A_l=(G_l^(t+1)H_l)^H(G_l^(t+1)H_l), (11) =H, B=XX^H, (12) l=nρ((t)−(t))+(l(t+1)l)HlH, C_l=nρ(Z^(t)-U^(t))+(G_l^(t+1)H_l)^HY_lX^H, (13) we can compactly recast (10) as: ll+nρl=l,A_lF_lB+n _l=C_l, (14) which is a Sylvester equation that can be efficiently solved by Bartels-Stewart algorithm. Otherwise, to find the closed form solution of (14), exploiting vec(ll)=(H⊗)vec(l)vec(A_lF_lB)=(B^H A)vec(F_l) and solving for vec(l)vec(F_l), after easy algebra we obtain: ^l=vec−1((H⊗l+nρ)−1vec(l)) F_l=vec^-1 ((B^H A_l+n )^-1vec(C_l) ) (15) The Z step. Minimizing (6) w.r.t. to Z leads to: (t+1) Z^(t+1) =argmin‖(t+1)+(t)−‖F2+IΩ() = _Z\,||F^(t+1)+U^(t)-Z||^2_F+I_ (Z) =ProjΩ((t+1)+(t)), =Proj_ (F^(t+1)+U^(t)), (16) where ProjΩ(⋅)Proj_ (·) denotes the projection operator onto the set Ω . Easy algebra shows that the optimal solution of (I-A) reads as: (t+1)=11+λ^((t+1)+(t)),Z^(t+1)= 11+ λ(F^(t+1)+U^(t)), (17) where λ^=max(0,tr(^^H)PT−1) λ= (0, tr( Z Z^H)P_T-1 ) with ^=(t+1)+(t) Z=F^(t+1)+U^(t). The U step. The Lagrange multiplier update reads as: (t+1)=(t)+(t+1)−(t+1) U^(t+1)=U^(t)+F^(t+1)-Z^(t+1) (18) I-B Protocol for Federated Implementation In this section, we summarize the protocol outlining the main steps of the proposed federated ADMM algorithm, along with the data exchange between the AP and the devices. This protocol is specifically designed to exchange the variables with a reduced payload, while maintaining private users’ latent space representations. The main steps are given by: i) An initial handshaking step, in which users submit requests to the AP, which determines the number of devices involved in the alignment process. i) To update locally their semantic decoder l(t+1)G_l^(t+1) by (7), every user l receives from the AP the variable l(t)=l(t)S_l^(t)=H_lF^(t)X if the AP has CSI and l(t)=(t)S_l^(t)=F^(t)X otherwise. i) To update (t+1)F^(t+1) at transmitter side using (15), the l-th user forwards lA_l in (11) and l=(l(t+1)l)HlP_l=(G_l^(t+1)H_l)^HY_l to the AP. To enforce privacy constraint, typical of a federated paradigm, pre-whitening is applied to lY_l. This step ensures privatization of local latent representation ls_l. iv) Updates of (t+1)Z^(t+1) and (t+1)U^(t+1) are computed by the AP. v) Return to step (i), and repeat until iteration T. All the main steps of the proposed Federated ADMM for semantic alignment are summarized in Algorithm 11. Algorithm 1 Algorithm 1 : Federated ADMM for Semantic Alignment 1: Input: (0)∼(0,1)F^(0) (0,1), (0),(0)=\Z^(0),U^(0)\=0, ρ>0ρ>0. 2: Output: Final values (T),(T),(T),(T)F^(T),G^(T),Z^(T),U^(T). 3: Perform handshaking; 4: for each iteration t=1,…,Tt=\1,…,T\ do 5: The AP transmits l(t)S_l^(t) to all users l=1,…,Ll=1,…,L;The users update locally l(t+1)G^(t+1)_l as in (7); 6: Users send lA_l and lP_l to the AP;The AP updates (t+1)F^(t+1) as in (8); 7: The AP updates (t)Z^(t) as in (17); 8: The AP updates (t)U^(t) as in (18). 9: end for IV Numerical Results This section provides numerical results to assess the performance of the proposed algorithm in a multi-user semantic communication system, transmitting latent representations to perform an image classification task. We consider the CIFAR-10 dataset, comprising 60000 32×3232× 32 color images distributed across 10 classes (6000 images per class). Among them, 42500 images were used for training, 7500 for validation, and 10000 for testing, with classification across 10 labels as the downstream task τ. Latent representations are produced by the backbone of pre-trained models chosen from the timm Python library and are summarized in Table I. We consider square MIMO Rayleigh fading channels with unitary variance. All displayed results are averaged across seeds:27,42,100,123,144,200, considering ρ=1ρ=1, transmitter power constraint PTP_T equal to 11 and iterations T=30T=30.111https://github.com/SPAICOM/multi-agent-semantic-alignment.git Baselines. We compare our methodology with baselines that directly transmit vectors x of the TX latent space while performing semantic alignment and MIMO channel equalization in a disjoint fashion. Specifically, leveraging SVD channel decomposition as ¯l=lllH H_l=U_l _lV_l^H, we set l=KT⊗(lHl+1SNRlNR)−1(ll)HG_l=1_K^T ( _l^H _l+ 1SNR_lI_N_R )^-1 (U_l _l )^H for all l=1,…,Ll=1,…,L, where SNRl represents the Signal-to-Noise Ratio in dB (SNR) at the l-th user; the semantic pre-coder is obtained by solving: =argmin = _F 1Ln∑l=1L‖−l(l+l)‖F2 1Ln _l=1^LE \|X-G_l (H_lFX+n_l ) \|_F^2 s.t. tr(H)≤PT. (FF^H)≤ P_T. (19) We consider two different transmitting strategies: (i) ”First-K” baseline, which transmits the first 2⋅K⋅NT2· K· N_T features of ∈ℝdx ^d, and (i) ”Top-K” baseline, that transmits the largest K features based on their l1l_1 norm. In the latter, along with their features, corresponding indices are also sent, used by the receivers to reconstruct the original signal, assuming perfect index reconstruction. After signal transmission, semantic alignment is performed by users by solving a least square problem: min∈ℂml×d‖l−l‖F2. _Q ^m_l× d||Q_lX-Y_l||_F^2. (20) for l=1,…,Ll=1,…,L, where lQ_l represents the alignment matrix. Furthermore, we benchmarked our approach against a multiplexing scheme in which each l-communication link is endowed with an independently optimized semantic pre-coder, under the assumption of negligible multi-user interference. TABLE I: Models Configuration Group Models AP vit_tiny_patch16 Users efficientvit_m5, levit_128, rexnet_100, vit_small_patch16,32, vit_base_patch16,32, mobilenet_v3_small_075,100, mobilenet_v3_large_100 Heterogeneous vit_small,base_patch16_224, mobilenetv3_small_075, efficientvit_m5 Homogeneous mobilenetv3_small_075,100, rexnet_100, mobilenetv3_large_100 Results. As a first example, Figure 2 presents the average downstream task accuracy versus compression factor ζ for various MIMO antenna configurations, wherein our Federated approach is benchmarked against the aforementioned baselines. The simulation considers a scenario with 10 users and SNR equal to 20 dB. As we can notice from Fig. 2, the proposed method significantly outperforms the First-K and Top-K baseline strategies. Interestingly, the proposed approach exhibits enhanced performance also with respect to the multi-link baseline configuration, which employs different semantic pre-equalizers over L independent parallel channels. In fact, mitigating semantic discrepancies in the latent spaces generated by heterogeneous models and the AP, the shared semantic pre-equalizer ensures robust performance even at high compression rates, underscoring the strong semantic alignment capabilities of our approach, without the need for having separate semantic equalizers and channels for each user. Figure 2: Accuracy vs. ζ with SNR=20 dB. Semantic alignment performance is affected by the degree of semantic similarity among the latent spaces produced by different models. Intuitively, the AP is better able to align users whose latent representations are semantically similar. To quantitatively evaluate this effect, Fig. 3 reports the network mean-squared error (MSE) between the latent spaces of the AP and users, and the task accuracy, as a function of the compression factor ζ, using our proposed method under two distinct settings, displayed in Table I: (i) a homogeneous setup, where four users employ different but semantically similar encoders; and (i) a heterogeneous setup, where the four users are split into two groups with significantly different latent space structures. We also consider three different percentages of semantic pilots available for training, i.e., 10%10\%, 50%50\%, and 100%100\%. As shown in Fig. 3, in both setups the network mean squared error (MSE) decreases as ζ increases, whereas, conversely, the accuracy improves. As expected, the homogeneous configuration results in significantly lower network MSE, along with more stable and higher task accuracy compared to the heterogeneous setting. This is due to more favorable semantic alignment conditions in the homogeneous case. Moreover, reducing the percentage of semantic pilots can facilitate model alignment and enhance overall task accuracy. However, this benefit holds only down to a certain threshold, below which alignment deteriorates and performance drops. This reveals an interesting trade-off between the number and selection of semantic pilots and overall system performance. We plan to explore this aspect in future work. V Conclusions This paper proposes a federated latent space alignment framework for multi-user semantic communications. By combining a shared semantic pre-equalizer at the AP with personalized semantic equalizers on user devices, the framework effectively addresses the challenge of aligning diverse neural representations in the presence of semantic noise. The proposed methodology hinges on a federated ADMM method enabling decentralized training of both the semantic pre-equalizer and the user-side equalizers. Extensive numerical evaluations with heterogeneous user models validate the approach, demonstrating its ability to balance key performance indicators such as accuracy, communication overhead, and computational complexity. Future research includes optimal semantic pilot selection, data-driven semantic clustering, and managing user coexistence in interference-limited scenarios. Key challenges involve interference-aware alignment and dynamic user grouping based on latent space compatibility. Addressing these aspects will be essential for realizing efficient multi-user semantic communication. Figure 3: Alignment struggle as Network MSE and Accuracy vs. ζ for a 4×4 MIMO channel with SNR=20 dB. References [1] C. De Alwis and at al., “Survey on 6g frontiers: Trends, applications, requirements, technologies and future research,” IEEE Open Journal of the Communications Society, vol. 2, p. 836–886, 2021. [2] E. C. Strinati, P. Di Lorenzo et al., “Goal-oriented and semantic communication in 6g ai-native networks: The 6G-goals approach,” in Proc. of EuCNC, 2024, p. 1–6. [3] D. Gündüz and et al., “Beyond transmitting bits: Context, semantics, and task-oriented communications,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 1, p. 5–41, 2022. [4] M. Kountouris and N. Pappas, “Semantics-empowered communication for networked intelligent systems,” IEEE Communications Magazine, vol. 59, no. 6, p. 96–102, 2021. [5] P. Di Lorenzo and at al., “Goal-oriented communications for the IoT: System design and adaptive resource optimization,” IEEE Internet of Things Magazine, vol. 6, no. 4, p. 26–32, 2023. [6] C. K. Thomas, E. C. Strinati, and W. Saad, “Reasoning with the theory of mind for pragmatic semantic communication,” arXiv:2311.18224, 2023. [7] S. Barbarossa, D. Comminiello, E. Grassucci, F. Pezone, S. Sardellitti, and P. Di Lorenzo, “Semantic communications based on adaptive generative models and information bottleneck,” IEEE Communications Magazine, vol. 61, no. 11, p. 36–41, 2023. [8] H. Xie, Z. Qin, G. Y. Li, and B.-H. Juang, “Deep learning enabled semantic communication systems,” IEEE Transactions on Signal Processing, vol. 69, p. 2663–2675, 2021. [9] L. Moschella, V. Maiorca, M. Fumero, A. Norelli, F. Locatello, and E. Rodolà, “Relative representations enable zero-shot latent space communication,” arXiv preprint arXiv:2209.15430, 2022. [10] Z. Lähner and M. Moeller, “On the direct alignment of latent spaces,” in Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models. PMLR, 2024, p. 158–169. [11] D. Alvarez-Melis, S. Jegelka, and T. S. Jaakkola, “Towards optimal transport with global invariances,” in The 22nd International Conference on Artificial Intelligence and Statistics. PMLR, 2019, p. 1870–1879. [12] L. Kühnel, T. Fletcher, S. Joshi, and S. Sommer, “Latent space geometric statistics,” in Proc. of ICPR. Springer, 2021, p. 163–178. [13] M. Sana and E. C. Strinati, “Semantic channel equalizer: Modelling language mismatch in multi-user semantic communications,” in Proc. of IEEE GLOBECOM, 2023, p. 2221–2226. [14] S. Fiorellino, C. Battiloro, E. C. Strinati, and P. Di Lorenzo, “Dynamic relative representations for goal-oriented semantic communications,” in Prof. of EUSIPCO. IEEE, 2024, p. 2107–2111. [15] T. Hüttebräucker, S. Fiorellino, M. Sana, P. Di Lorenzo, and E. C. Strinati, “Relative representations of latent spaces enable efficient semantic channel equalization,” arXiv preprint arXiv:2411.19719, 2024. [16] S. Boyd et al., “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Foundations and Trends® in Machine learning, vol. 3, no. 1, p. 1–122, 2011.