Paper deep dive
Manifold-Matching Autoencoders
Laurent Cheret, Vincent Létourneau, Isar Nejadgholi, Chris Drummond, Hussein Al Osman, Maia Fraser
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 93%
Last extracted: 3/22/2026, 5:43:49 AM
Summary
The paper introduces Manifold-Matching Autoencoders (MMAE), an unsupervised regularization framework for autoencoders that aligns pairwise distances in the latent space with those of a reference space (e.g., input data or PCA-reduced data). By minimizing the mean squared error between these distance matrices, MMAE effectively preserves global geometric and topological structures, outperforming existing topological and geometric autoencoder variants on synthetic and real-world benchmarks while offering a scalable alternative to Multi-Dimensional Scaling (MDS).
Entities (7)
Relation Signals (3)
MMAE → regularizes → Autoencoder
confidence 95% · The Manifold-Matching regularization MM-reg is added to the objective function of the standard AE, forming MMAEs.
MMAE → approximates → MDS
confidence 90% · We also observe that MMAE provides a scalable approximation of Multi-Dimensional Scaling (MDS).
MMAE → outperforms → TopoAE
confidence 85% · We find that this regularization outperforms similar methods on metrics based on preservation of nearest-neighbor distances
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We study a simple unsupervised regularization scheme for autoencoders called Manifold-Matching (MMAE): we align the pairwise distances in the latent space to those of the input data space by minimizing mean squared error. Because alignment occurs on pairwise distances rather than coordinates, it can also be extended to a lower-dimensional representation of the data, adding flexibility to the method. We find that this regularization outperforms similar methods on metrics based on preservation of nearest-neighbor distances and persistent homology-based measures. We also observe that MMAE provides a scalable approximation of Multi-Dimensional Scaling (MDS).
Tags
Links
- Source: https://arxiv.org/abs/2603.16568v1
- Canonical: https://arxiv.org/abs/2603.16568v1
Trouble viewing inline? Open PDF directly →
Full Text
44,421 characters extracted from source content.
Expand or collapse full text
Manifold-Matching Autoencoders Laurent Cheret Vincent Létourneau Isar Nejadgholi Chris Drummond Hussein Al Osman Maia Fraser Abstract We study a simple unsupervised regularization scheme for autoencoders called Manifold-Matching (MMAE): we align the pairwise distances in the latent space to those of the input data space by minimizing mean squared error. Because alignment occurs on pairwise distances rather than coordinates, it can also be extended to a lower dimensional representation of the data, adding flexibility to the method. We found that this regularization outperforms similar methods on metrics based on preservation of closest neighbors distances and persistence homology based measures. We also observe MMAE provides a scalable approximation of ‘Multi-Dimensional Scaling” (MDS). Machine Learning, ICML 1 Introduction Dimensionality reduction is fundamental to modern data analysis, enabling visualization and interpretation of high-dimensional datasets. Autoencoders (Hinton and Salakhutdinov, 2006; Chen and Guo, 2023) learn compressed representations by minimizing reconstruction error, but this objective alone does not guarantee preservation of any particular geometric or topological structure. When the encoder ignores these structures, similar objects in the input space may be mapped to distinct regions of the latent space, creating discontinuities that negatively affect the decoder’s ability to reconstruct (Batson et al., 2021). This problem can also affect other downstream tasks. For example, in anomaly detection or when visualizing developmental trajectories in single-cell data (Chari and Pachter, 2023) or exploring latent spaces in generative models (Chadebec and Allassonnière, 2022; Xu et al., 2024)—additional regularization becomes necessary. 1.1 Topology and Geometry in Autoencoders Following the success of statistical methods using topological data analysis tools like persistence diagrams (Su et al., 2025), there has been a recent effort to improve the preservation of topological features by autoencoders, which we classify as topological or geometrical methods. Topological methods (Moor et al., 2020b; Trofimov et al., 2023) use persistent homology to identify and preserve multi-scale structural features such as connected components, loops, and voids. Geometric methods (Singh and Nag, 2021; Nazari et al., 2023; Lim et al., 2024) focus on preserving local angles and distances. Take, for example, the nested spheres dataset (Moor et al., 2020b), a simple synthetic yet highly nonlinear case: ten 100-dimensional small spheres are nested inside a larger 100D enclosing sphere. A topologically accurate 2D representation should preserve this nesting relationship, with the outer sphere cluster surrounding the inner sphere clusters. To date, only topological autoencoder variants consistently recover this structure in 2D/3D. Other autoencoder variants and nonparametric methods such as UMAP (McInnes et al., 2018), t-SNE (van der Maaten and Hinton, 2008), and PHATE (Moon et al., 2017) fail in this case. Interestingly, we found that the classical method Multidimensional Scaling (MDS) (Torgerson, 1952) also successfully recovers the nesting relationship—a result that, to the best of our knowledge, has not been reported in previous work. Figure 1: Left: Overview of the current approach. The Manifold-Matching regularization M-reg is added to the objective function of the standard AE, forming MMAEs. Top Right: 2D latent spaces of the Nested Spheres dataset (Moor et al., 2020b). Standard AE (Vanilla) using no M-reg and 9 MMAE models using different number of PCA components in their regularization (1→1001→ 100). Bottom Right: MMAE 2D latent spaces “copying” 2D embeddings from UMAP, t-SNE, and PCA across MNIST, F-MNIST, and CIFAR10 datasets. 1.2 Our Approach: Alignment of Pairwise Distances Classical MDS (Torgerson, 1952) utilizes the pairwise distance matrix of the entire dataset to preserve global geometry. In contrast, topological autoencoders like TopoAE (Moor et al., 2020b) and RTD-AE (Trofimov et al., 2023) employ persistent homology signatures on pairwise distance matrices at the mini-batch level. While the former focuses on geometric distances—which may implicitly capture topology—the latter prioritize multi-scale structural connectivity, a focus that has been shown to result in superior preservation of the manifold’s global geometry. One issue arises with these methods, MDS scales poorly with data size due to memory requirements to compute the n×n× n pairwise distance matrix, and topological variants of autoencoders scale poorly with the batch size b due to the persistent homology computations batch-wise. A central question emerges: What happens to latent spaces when global geometry preservation is imposed in autoencoders? We address this by introducing a regularization term called Manifold-Matching (M-reg), defined as the MSE between the pairwise distance matrix DZD_Z of the latent space and a reference distance matrix DED_E computed from either the input data X itself or its respective embedding. Crucially, since both DZD_Z and DED_E are b×b× b matrices (where b is the batch size), the dimensionality of the reference space is decoupled from the bottleneck dimensionality. This means, for example, that a 2D latent space can be regularized using distances from a 50D or 100D reference representation. Note there is strong theoretical justification in the choice to replace the data by its distance matrix. In short, distance preservation implies topology preservation. This is the content of section 2.2. Figure 1 illustrates this effect on the nested spheres dataset. Without regularization, the standard AE projects the inner spheres outside the cluster representing the outer sphere, consistent with prior literature. However, with M-reg, as the number of PCA components in the reference increases, the nesting structure present in the original data emerges, with the inner sphere clusters progressively pulled inside the outer sphere cluster. In the specific case in which the reference embeddings are 2D, a “copying” effect can be seen where the 2D latent space approximates the reference, allowing the autoencoder to extend known representation to new data points. Specifically, our contributions are: (1) We introduce the Manifold-Matching Autoencoder (MMAE), an unsupervised framework for global structure-aware dimensionality reduction. (2) We study its visualization effects on synthetic datasets where the topology is intuitively understood. (3) We extend experiments to real-world benchmarks, showing competitive performance against topological and geometrical autoencoder variants. (4) We provide discussions on global geometry preservation as a proxy for topology preservation. 2 Background We review the key concepts underlying our approach: persistent homology as the gold standard for topological comparison, and the relation between distance preservation and topology. 2.1 Persistent Homology The importance of understanding data topology has been recognized since the 1960s (Rosenblatt, 1962). This concern is tied to the manifold hypothesis: high-dimensional data =ii=1kX=\x_i\_i=1^k with i∈ℝnx_i ^n typically lies on or near a lower-dimensional manifold ℳ⊂ℝnM ^n. Persistent homology provides a principled way to detect the topological features of this manifold across scales (Edelsbrunner et al., 2002; Carlsson, 2009). 2.2 From Distance Preservation to Topology Preservation Multidimensional Scaling (MDS) (Torgerson, 1952) provides a classical approach that finds a low-dimensional configuration of points whose pairwise distances best preserve those of the input distance matrix. The key insight is that while points i,j∈ℝnx_i,x_j ^n may have many coordinates, their Euclidean distance dij=‖i−j‖2d_ij=\|x_i-x_j\|_2 reduces their relationship to a single scalar value. Remarkably, collecting all such pairwise distances into a matrix D contains sufficient information to recover the original geometric configuration. Classical MDS formalizes this by converting distance relationships into geometric configurations through eigendecomposition of the associated Gram matrix (Borg and Groenen, 2005; Schoenberg, 1935). This distance-centric view connects naturally to topology preservation through the stability theorem. For finite metric spaces with Vietoris-Rips persistence diagrams: Theorem 2.1 (Stability (Cohen-Steiner et al., 2007; Chazal et al., 2016)). dB(Dgmp(X),Dgmp(Y))≤2⋅dGH(X,Y)d_B(Dgm_p(X),Dgm_p(Y))≤ 2· d_GH(X,Y) (1) for all homology dimensions p≥0p≥ 0, where dBd_B is the bottleneck distance and dGHd_GH the Gromov-Hausdorff distance. Since uniform distance preservation bounds dGHd_GH, we obtain: Corollary 2.2 (Distance Preservation Implies Topology Preservation). If an encoder fθf_θ satisfies |dX(xi,xj)−dZ(fθ(xi),fθ(xj))|≤ϵ|d_X(x_i,x_j)-d_Z(f_θ(x_i),f_θ(x_j))|≤ε for all pairs, then dB(Dgmp(X),Dgmp(fθ(X)))≤2ϵd_B(Dgm_p(X),Dgm_p(f_θ(X)))≤ 2ε for all p≥0p≥ 0. This result reveals our path forward: rather than computing persistent homology during training, we preserve topology by preserving distances. Manifold-Matching Autoencoders operationalize this principle by aligning the latent space to a reference geometry through pairwise distances. In practice, MMAE applies this principle at the minibatch level, leveraging the theoretical justification that batch-wise topology approximates global topology as batch size increases. A practical consideration is that training operates on mini-batches rather than the full dataset. TopoAE (Moor et al., 2020b) provides theoretical justification for this approach through two key results. First, they establish that the probability of batch topology deviating from full set topology is bounded by geometric sampling: P(dB(Dgm(X),Dgm(X(b)))>ϵ) P(d_B(Dgm(X),Dgm(X^(b)))>ε) (2) ≤P(dH(X,X(b))>ϵ2) ≤ P(d_H(X,X^(b))> ε2) where X(b)X^(b) is a mini-batch of size b and dHd_H is the Hausdorff distance. Second, they show that as batch size approaches dataset size, the expected Hausdorff distance converges to zero, meaning batch-level topology increasingly mirrors global topology. This justifies using batch-wise distance preservation as a proxy for global structure preservation. 3 Related Work The challenge of learning topologically correct representations has motivated several autoencoder variants. We review these methods through our core question: how can we capture global structure efficiently? 3.1 Topological Autoencoders TopoAE (Moor et al., 2020b) pioneered using persistent homology as a training signal. Given distance matrices DXD_X and DZD_Z in input and latent space, it penalizes discrepancies between topologically significant point pairs: ℒtopo=12∑(i,j)∈X(DXij−DZij)2+12∑(k,l)∈Z(DZkl−DXkl)2L_topo= 12 _(i,j) _X(D_X^ij-D_Z^ij)^2+ 12 _(k,l) _Z(D_Z^kl-D_X^kl)^2 (3) where X,ZP_X,P_Z denote topologically significant pairs (births/deaths in persistence diagrams). While theoretically generalizable, the implementation focuses on H0H_0 (connected components) via minimum spanning trees for efficiency. Two limitations arise: (1) the loss is discontinuous under point perturbations, as small changes can abruptly alter the spanning tree (Trofimov et al., 2023); (2) ℒtopo=0L_topo=0 is necessary but not sufficient for topological equivalence, failing to capture higher-order features like loops. 3.2 RTD-AE: Representation Topology Divergence Autoencoders RTD-AE (Trofimov et al., 2023) addresses these limitations with stronger guarantees: nullity of RTD ensures persistence barcodes coincide across all homology degrees. Crucially, the loss is continuous and accounts for feature localization. The method constructs a joint distance matrix over 2n2n points: Djoint=(n×nDXTDXmin(DX,DZ))D_joint= pmatrix0_n× n&D_X^T\\ D_X& (D_X,D_Z) pmatrix (4) with loss ℒRTD=∑(b,d)∈Dgm(Djoint)|d−b|pL_RTD= _(b,d) (D_joint)|d-b|^p summing persistence lifetimes. Notably, their experiments show the gap between PCA and topological methods narrows at higher latent dimensions (64–128D). However, RTD incurs high computational cost with batch size, often requiring two-stage training (reconstruction, then topology). 3.3 Structure Preserving Autoencoders The Structure-Preserving Autoencoder (SPAE) (Singh and Nag, 2021) learns low-dimensional representations where pairwise distances are a linearly scaled version of the input space distances. The method defines a distance ratio rij=dZ(zi,zj)/dX(xi,xj)r_ij=d_Z(z_i,z_j)/d_X(x_i,x_j) and regularizes by minimizing the variance of log-ratios: ℒSPAE=ℒrecon+λ⋅Var[logrij]i<jL_SPAE=L_recon+λ·Var [ r_ij ]_i<j (5) By operating on logarithms, the loss is scale-invariant: it permits any uniform scaling dZ≈c⋅dXd_Z≈ c· d_X while penalizing non-uniform distortions. However, it relies on raw input distances, which become unreliable in high dimensions due to the curse of dimensionality. And the ratio formulation amplifies noise for small distances: when dX≈ϵd_X≈ε, the ratio dZ/dXd_Z/d_X becomes unstable, and log(dZ/dX) (d_Z/d_X) exhibits high variance. The variant of the SPAE-graph replaces Euclidean distances with geodesic distances computed by shortest paths on a similarity graph, but introduces sensitivity to graph sparsity: holes in the data cause path detours that exaggerate distances and produce spurious geometric distortions (Lim et al., 2024). 3.4 Geometric Autoencoders GeomAE (Nazari et al., 2023) shifts the focus from global geometry to the local geometry of the decoder g:ℝn→ℝdg:R^n ^d to prevent spurious stretching in visualizations. For a given latent point z, the pullback metric G(z)=Jg(z)⊤Jg(z)G(z)=J_g(z) J_g(z) captures local volume distortion and anisotropy. This distortion is visualized through indicatrices—unit spheres in the pullback metric that reveal which directions are squeezed or expanded by the decoder. GeomAE minimizes the variance of the generalized Jacobian determinant: ℒgeom=Varz[logdet(G(z))]L_geom=Var_z [ (G(z)) ] (6) which encourages area-preservation and uniform scaling across the embedding. 3.5 Graph Geometry-Preserving Autoencoders GGAE (Lim et al., 2024) replaces Euclidean distances with similarity graphs. By defining a graph with weights Kij=exp(−‖xi−xj‖2/h)K_ij= (-\|x_i-x_j\|^2/h), it can be shown (Shi and Sun, 2016) that if the distribution is supported on a smooth manifold then, as the number of sampled points goes to infinity, the graph Laplacian of this weighted graph converges to the Laplace-Beltrami operator of the distribution manifold. The Riemannian distortion loss: ℒggae=[Tr(H2)−2Tr(H)+n],H=JL−1J⊤L_ggae=E [Tr(H^2)-2Tr(H)+n ], H=JL^-1J (7) penalizes deviation from isometry. Their Batch-Kernel method selects submatrices from a precomputed global kernel, giving mini-batches access to global structure. However, the optimal choice of bandwidth h remains highly dataset-dependent—the authors acknowledge extending to new datasets is burdensome. In cases for large datasets, the GPU makes the computation of the bandwidth efficient but at a high memory cost. 4 Manifold Matching Autoencoder 4.1 Overview MMAE consists of an encoder fθ:ℝD→ℝdf_θ:R^D ^d and decoder gϕ:ℝd→ℝDg_φ:R^d ^D, trained with a combined reconstruction loss and distance-preservation regularization. 4.2 Reference Space Given training data X∈ℝn×DX ^n× D, the reference space E defines the target distance structure that the latent space should preserve. In its simplest form, E=XE=X uses the original input space directly, similarly to Singh and Nag (2021). More generally, E can be any embedding of the data: E=u(X)∈ℝn×kE=u(X) ^n× k (8) where u:ℝn×D→ℝn×ku:R^n× D ^n× k is an embedding. This formulation provides flexibility: the embedding u can be chosen to emphasize different geometric properties—linear projections preserve global structure, while neighborhood-based methods like UMAP preserve local topology. For high-dimensional data where distances concentrate due to the curse of dimensionality (Aggarwal et al., 2001), using a reduced embedding E=u(X)E=u(X) with k<Dk<D can provide more informative distance comparisons than the original space. The key insight is that u acts as a preprocessing step that filters noise and extracts the relevant geometric structure, which the autoencoder then learns to match. We focus on PCA as our main reference space besides the original input. 4.3 Manifold-Matching Regularization (M-reg) For a batch of n points with latent representations Z∈ℝn×dZ ^n× d and reference representations E∈ℝn×kE ^n× k (where k=Dk=D if using original input), we compute pairwise Euclidean distance matrices: DZij D_Z^ij =‖zi−zj‖2 =\|z_i-z_j\|_2 (9) DEij D_E^ij =‖ei−ej‖2 =\|e_i-e_j\|_2 (10) The manifold-matching regularization is the mean squared error between these distance matrices: ℛM=1n2∑i,j(DZij−DEij)2R_M= 1n^2 _i,j(D_Z^ij-D_E^ij)^2 (11) This directly penalizes discrepancies between latent and reference pairwise distances, encouraging the encoder to preserve the metric structure of the reference space. 4.4 Total Objective The full MMAE objective combines reconstruction and manifold-matching: ℒMMAE=ℒrecon+λ⋅ℛML_MMAE=L_recon+λ·R_M (12) where ℒrecon=1n∑i‖xi−gϕ(fθ(xi))‖2L_recon= 1n _i\|x_i-g_φ(f_θ(x_i))\|^2 and λ controls the trade-off between reconstruction fidelity and structure preservation. 5 Experiments Dataset Method DC↑ TA↑ KL0.1↓ Nested Spheres (101D) Vanilla AE -0.40 0.42 0.85 MMAE (Ours) 0.67 0.70 0.23 TopoAE 0.63 0.69 0.29 RTD-AE 0.61 0.69 0.29 GeomAE -0.18 0.48 0.66 GGAE 0.11 0.54 0.46 SPAE 0.55 0.66 0.32 Linked Tori (100D) Vanilla AE 0.71 0.77 0.03 MMAE (Ours) 0.91 0.87 0.003 TopoAE 0.84 0.82 0.008 RTD-AE 0.90 0.86 0.005 GeomAE 0.77 0.78 0.01 GGAE 0.80 0.81 0.02 SPAE 0.89 0.85 0.003 Concentric Spheres (1000D) Vanilla AE 0.51 0.66 0.37 MMAE (Ours) 0.61 0.68 0.30 TopoAE 0.59 0.67 0.31 RTD-AE 0.61 0.68 0.31 GeomAE 0.41 0.61 0.38 GGAE 0.39 0.56 0.39 SPAE 0.59 0.67 0.32 Mammoth (3D) Vanilla AE 0.94 0.91 0.007 MMAE (Ours) 0.99 0.96 0.001 TopoAE 0.96 0.91 0.003 RTD-AE 0.97 0.92 0.002 GeomAE 0.95 0.91 0.007 GGAE 0.95 0.91 0.004 SPAE 0.98 0.93 0.002 Earth (3D) Vanilla AE 0.94 0.92 0.01 MMAE (Ours) 0.98 0.95 0.002 TopoAE 0.95 0.95 0.01 RTD-AE 0.87 0.90 0.02 GeomAE 0.94 0.92 0.009 GGAE 0.87 0.87 0.03 SPAE 0.98 0.93 0.002 Figure 2: 2D latent spaces of synthetic shapes Left: Quantitative metrics. Average over 5 runs (optimized for metric KL0.1KL_0.1) Right: 2D Latent representations: a) Standard AE (Vanilla); b) MMAE; c) TopoAE; d) RTD-AE; e) GeomAE; f) GGAE; g) SPAE. 5.1 Datasets Synthetic Datasets: Spheres (Moor et al., 2020b): 10 small spheres embedded within a larger sphere in 101 dimensions, yielding 11 connected components—a standard benchmark for topology preservation. Linked Tori: Two interlocking tori as a Hopf Link embedded in 100 dimensions via random orthogonal projection. It does not matter the angle-of-view in a 2D representation, there is always going to be one torus overlapping the other. Concentric Spheres: 1000D Nested spherical shells in high dimensions with a coinciding center. Mammoth: 3D point cloud forming a mammoth skeleton structure (MNoichl, 2025). Earth: 3D point cloud of the planet Earth globe (Nazari et al., 2023) colored by six continents: North/Central America, South America, Europe, Asia, and Oceania. Real-World Datasets. MNIST and Fashion-MNIST (Lecun et al., 1998; Xiao et al., 2017): 28×2828× 28 grayscale images (784 dimensions). CIFAR-10 (Krizhevsky, 2009): 32×32×332× 32× 3 color images (3072 dimensions). PBMC3k: Single-cell RNA-seq data with 2,700 cells across 1,838 genes (10x Genomics, 2016), and Paul15: 2,730 mouse bone marrow cells across 3,451 genes (Paul et al., 2015), both representing biological manifold structure and available through the Scanpy library (Wolf et al., 2018). 5.2 Baselines We compare against both topological and geometric autoencoder methods: Vanilla AE: Standard autoencoder with MSE reconstruction loss only. TopoAE (Moor et al., 2020b): Topological regularization via persistent homology on minimum spanning trees. RTD-AE (Trofimov et al., 2023): Representation Topology Divergence with joint filtration construction. GeomAE (Nazari et al., 2023): Geometric regularization minimizing decoder Jacobian variance. GGAE (Lim et al., 2024): Graph geometry preservation via Laplacian-based distortion. SPAE (Singh and Nag, 2021): Structure-preserving regularization via variance of log distance ratios between latent and input spaces. 5.3 Evaluation Metrics We evaluate methods using: Distance Correlation (DC) and Triplet Accuracy (TA) which measure the preservation of the global geometry; KL Density (Moor et al., 2020b) at scale σ=0.1σ=0.1 measuring the preservation of the density (following Moor et al. (2020b) and Trofimov et al. (2023) this is our metric to optimize hyperparameter, as this scale offers good balance of local/global density); Wasserstein Distance (W0W_0) on persistence diagrams for topological preservation of connected components; Trustworthiness (Trust) and Continuity (Cont) (Venna and Kaski, 2001) which are complementary metrics for the preservation of the local neighborhood at scales k∈5,10,50,100k∈\5,10,50,100\; and Reconstruction Error (Rec)as MSE. 5.4 Implementation Details All methods use MLP architectures following Moor et al. (2020b): encoder with symmetric decoder, ReLU activations, weight decay, Adam optimizer (Kingma and Ba, 2017), and batch normalization. A CNN is used for CIFAR10. Hyperparameters such as learning rate, batch size as well as specifics of each model are chosen following a similar optimization procedure of Moor et al. (2020b); Trofimov et al. (2023). Batch sizes were chosen within a range [16;256][16;256] (with the exception of RTD which has a batch size limited to at most 80 due to its significant increase in training time for larger batches), learning rate [10−4;10−2][10^-4;10^-2]. In the case of MMAEs, for real-world datasets, we allow the hyperparameter choice to use a lower dimensional PCA projection down to 80% of the original data dimensionality using Scikit (Pedregosa et al., 2011) PCA implementation, to fight the noise in the distances and help with the curse of dimensionality (Aggarwal et al., 2001). The number of trials in the optimization step is 20 for each model, and the metric to optimize is the KL Density (σ=0.1σ=0.1), same as (Moor et al., 2020b; Trofimov et al., 2023) which measures the preservation of density across the whole representation, a metric tightly bound to the topological variants objectives. The models with the best configurations are retrained 5 times, and the average on the test sets is reported. 5.5 Results Synthetic Datasets: In the nested spheres (Figure 2), models inverting outer and inner spheres achieved negative DC. Only MMAE and topological variants recover the nesting relationship; GGAE shows inconsistent nesting (DC of 0.11). MMAE uniquely preserves the gap between spheres, while TopoAE and RTD-AE produce continuous embeddings. In linked tori, other methods produce a ”bowtie” effect by compressing the overlap region, whereas MMAE maintains constant circular shapes. Despite both tori overlapping in a large region, MMAE achieves the highest DC, TA and lowest KL0.1KL_0.1, finding a nonlinear point arrangement that optimizes density preservation. The concentric spheres by MMAE shows five distinct disks with visible gaps; TopoAE, RTD-AE, and SPAE produce continuous embeddings with no gaps. On the mammoth, RTD-AE, TopoAE, GeomAE, and SPAE spread the rib cage and hips to preserve local structure. Interestingly, our projection seems to improve on the MDS projection by choosing a side view of the mammoth similarly to other models but preserving the global proportions making a more realistic projection of the animal. Even though MMAE collapses one side of the rib cage onto the other instead of spreading, it still achieves the highest DC, TA and lowest KL0.1KL_0.1, balancing local and global structure. The Earth dataset complements nested spheres results: MMAE creates distortions in South America to better preserve relative distance to Africa, and on Oceania to approximate it to Asia achieving best performance. Other methods stretch different continents in different directions significantly increasing oceanic distances (for example from Madagascar to New Zealand); RTD-AE projects South America’s tip nearly into Europe. SPAE’s regularization produces even stretching also performing well. GGAE showed high sensitivity to bandwidth choice on both 3D dataset, often producing circular representations ignoring data topology. Table 1: Representation quality on real-world datasets. Best per dataset in bold. Latent space dimensionality for each dataset: Paul15 (2D), MNIST (16D), FMNIST (64D), PBMC3K (64D), CIFAR10 (128D). Dataset Method Rec↓ DC↑ TA↑ KL0.1↓ Trust↑ Cont↑ W0↓ Paul15 Vanilla AE 0.94 0.42 0.58 0.14 0.67 0.70 272.1 MMAE (Ours) 0.94 0.85 0.77 0.07 0.69 0.82 269.5 TopoAE 0.95 0.50 0.67 0.07 0.67 0.80 269.5 RTD-AE 0.96 0.58 0.69 0.07 0.66 0.79 269.9 GeomAE 0.94 0.37 0.59 0.12 0.68 0.71 273.2 GGAE 0.94 0.23 0.57 0.13 0.63 0.65 277.2 SPAE 0.96 0.59 0.68 0.08 0.65 0.76 269.5 MNIST Vanilla AE 0.15 0.95 0.82 0.002 0.93 0.95 85.65 MMAE (Ours) 0.15 0.99 0.89 0.001 0.96 0.98 71.01 TopoAE 0.17 0.90 0.85 0.005 0.96 0.97 68.19 RTD-AE 0.14 0.97 0.87 0.001 0.97 0.98 56.69 GeomAE 0.15 0.79 0.78 0.01 0.93 0.93 90.10 GGAE 0.56 0.80 0.70 0.01 0.87 0.92 43.02 SPAE 0.39 0.97 0.87 0.001 0.96 0.97 53.87 FMNIST Vanilla AE 0.12 0.92 0.86 0.005 0.98 0.98 26.46 MMAE (Ours) 0.11 0.99 0.95 0.001 1.00 1.00 26.47 TopoAE 0.12 0.94 0.86 0.004 0.98 0.98 31.81 RTD-AE 0.09 0.99 0.95 0.001 1.00 1.00 8.08 GeomAE 0.12 0.91 0.86 0.007 0.98 0.98 31.40 GGAE 0.20 0.71 0.77 0.02 0.94 0.94 70.23 SPAE 0.14 0.98 0.94 0.004 0.99 0.99 21.58 PBMC3k Vanilla AE 0.87 0.67 0.68 0.03 0.70 0.75 238.8 MMAE (Ours) 0.82 0.80 0.73 0.02 0.82 0.86 63.42 TopoAE 0.82 0.72 0.74 0.02 0.82 0.87 114.5 RTD-AE 0.88 0.70 0.70 0.03 0.77 0.82 108.7 GeomAE 0.82 0.63 0.69 0.03 0.71 0.78 213.5 GGAE 0.87 0.15 0.54 0.17 0.62 0.53 175.2 SPAE 0.86 0.68 0.69 0.03 0.75 0.79 75.38 CIFAR10 Vanilla AE 0.27 0.93 0.89 0.01 0.98 0.98 21.73 MMAE (Ours) 0.10 1.00 0.98 0.000 1.00 1.00 1.87 TopoAE 0.14 1.00 0.97 0.001 1.00 1.00 2.95 RTD-AE 0.16 0.99 0.95 0.005 1.00 1.00 5.80 GeomAE 0.95 0.43 0.68 0.10 0.84 0.81 171.2 GGAE 0.20 0.71 0.75 0.05 0.90 0.89 79.41 SPAE 0.09 1.00 0.98 0.000 1.00 1.00 1.57 Real-world Datasets: Both MMAE and SPAE achieve W0W_0 competitive with RTD-AE and TopoAE, improving with bottleneck dimensionality (visible in CIFAR10 at 128D). These distance-based methods also show superior Trust. and Cont. against GeomAE and GGAE. On Paul15 and PBMC3K, MMAE achieves the lowest W0W_0 with highest DC and TA. These high-dimensional, relatively small datasets benefit from PCA-based M-reg which reduces reference dimensionality to ignore noise; SPAE using raw distances performs significantly lower on Paul15 (2D bottleneck), though this weakness vanishes at larger bottlenecks. SPAE’s more relaxed distance preservation provides flexibility for better connectivity (W0W_0) at the cost of global geometry (Table 1). Both regularizations enforcing global geometry produce topology preservation effects. 6 Discussion and Conclusion 6.1 Scalability Figure 3 shows training time scaling with batch size. RTD-AE is limited to batch sizes of 80 before becoming prohibitive; TopoAE also scales poorly. Since batch topology approximates global topology only as batch size increases (Moor et al., 2020b, a), these methods face a fundamental tension. MMAE scales similarly to vanilla autoencoders and geometric variants. MMAE also achieves similar geometric fidelity to classical MDS while requiring significantly less memory through batch-wise optimization. Figure 3: Training time versus batch size. MMAE scales similarly to the standard AE (Vanilla). RTD-AE limited to batch size of 80. 6.2 Two Philosophies for Impossible Embeddings Topological methods like TopoAE and RTD-AE optimize for topological signatures—birth and death of features across scales—ensuring the latent space mirrors input connectivity. They allow distortions in order to maintain connections in the data. The linked tori is an example of this effect, in which the “bowtie” shape appears because the areas overlapping are minimized (here all models show this effect except for MMAE). Although this may be useful for downstream tasks (Hofer et al., 2017), it loses visual interpretation. Geometric methods like GeomAE and GGAE control the stretching of the representation. In the mammoth case this means that GeomAE flattens the skeleton producing an equally stretched representation which looses proportions. In the Earth dataset, this means all continents will be stretched equally which can give different visual results for each continent. This method then looses the global structure of the data in order to equally stretch. GGAE, however seems to preserve some global structure but is more difficult to finetune for each dataset, recognized by the authors Lim et al. (2024), producing a nested relationship in the nested spheres case but loosing all structural information for the two 3D datasets (mammoth and Earth), which limits our interpretation of the results. MMAE and SPAE, instead, preserve global geometric relationships, which, given our results, has shown topology-preserving properties. In comparison to SPAE (Singh and Nag, 2021), which measures variance ratios in distances, MMAE better preserves global geometry proportions. On noisy high-dimensional datasets like PBMC3K and Paul15, SPAE shows lower DC, TA, Trust., and Cont. We hypothesize MMAE succeeds here because it can use reduced PCA dimensionality (down to 80%) to mitigate noise effects. In summary, our quantitative as well as qualitative results seem to confirm the theory that topology preserving properties can be achieved by alignment of the pairwise distance matrices at the batch level. We hypothesize that one way to optimize the topology preservation would be to vary the regularization strength, by starting with a strong M-reg regularization (high λ and then a relaxation of the parameter allowing the model to better reconstruct once it reaches a favorable representation. For example in the nested spheres case, once the model achieves the nesting relationship, even if λ→0λ→ 0, the nesting structure is maintained. 6.3 Parameterizing External Representations Since M-reg only requires pairwise distances, it can encode arbitrary reference representations. Duque et al. (2023) forced autoencoders to match reference coordinates directly, but Figure 1 shows M-reg achieves similar effects: 2D representations from UMAP, t-SNE, and PCA can be copied into the latent space. This is more generic—Duque et al. (2023) could not reproduce results in Figure 2 without an external embedding mechanism. 6.4 Limitations and Future Directions MMAE preserves global geometry rather than explicitly preserving topology. It does not unfold manifolds but minimizes pairwise distance discrepancies, so results vary by dataset. A middle ground may exist: applying M-reg to random point subsets could relax geometric constraints, balancing topology and geometry without persistent homology. Another interesting approach would be to combine the methods shown above to optimize the training time while getting the best of both worlds. For example, starting with M-reg which doesn’t offer significant computational overhead to capture global geometry and then concluding with the regularization of TopoAE or RTD-AE during the final epochs to enhance connectivity. Given continued use of MDS in clustering (Chen et al., 2025) and PCA-like autoencoders (Casella et al., 2022; Ladjal et al., 2019; Pham et al., 2022), plus evidence that latent geometry (Chadebec and Allassonnière, 2022) and density (Xu et al., 2024) improve generative quality, extending MMAE to generative scenarios is promising—particularly beyond 2D/3D bottlenecks where interpolation and sampling tasks may benefit from topology preservation. 6.5 Conclusion We presented Manifold-Matching Autoencoders, a framework for global geometry preservation through pairwise distance alignment leading to topology-preserving effect without significant computational overhead. Unlike MDS, it enables out-of-sample extension and scales to large datasets. Using PCA references allows approximating true global geometry while ignoring noise inherent to high-dimensional data. The approach also enables copying representations from other dimensionality reduction algorithms, providing out-of-sample extension for nonparametric methods. In scenarios beyond visualization, this naive distance alignment proves to be increasingly more competitive in terms of topology preservation, paving the way for the study of topology awareness in scenarios requiring larger bottlenecks. Impact Statement This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here. References 10x Genomics (2016) 3k PBMCs from a Healthy Donor. Note: Single Cell Gene Expression Dataset, Version 1.1.0Accessed via Scanpy datasets External Links: Link Cited by: §5.1. C. C. Aggarwal, A. Hinneburg, and D. A. Keim (2001) On the surprising behavior of distance metrics in high dimensional spaces. In Proceedings of the 8th International Conference on Database Theory, ICDT ’01, Berlin, Heidelberg, p. 420–434. External Links: ISBN 3540414568 Cited by: §4.2, §5.4. J. Batson, C. G. Haaf, Y. Kahn, and D. A. Roberts (2021) Topological obstructions to autoencoding. Journal of High Energy Physics 2021 (4). External Links: ISSN 1029-8479, Link, Document Cited by: §1. I. Borg and P. J. F. Groenen (2005) Modern multidimensional scaling theory and applications. Springer, New York. External Links: Document, ISBN 038728981X Cited by: §2.2. G. Carlsson (2009) Topology and data. Bulletin of the American Mathematical Society 46 (2), p. 255–308. Cited by: §2.1. M. Casella, P. Dolce, M. Ponticorvo, and D. Marocco (2022) From principal component analysis to autoencoders: a comparison on simulated data from psychometric models. In 2022 IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE), Vol. , p. 377–381. External Links: Document Cited by: §6.4. C. Chadebec and S. Allassonnière (2022) A geometric perspective on variational autoencoders. Advances in Neural Information Processing Systems 35, p. 19618–19630. Cited by: §1, §6.4. T. Chari and L. Pachter (2023) The specious art of single-cell genomics. PLOS Computational Biology 19 (8), p. 1–20. External Links: Document, Link Cited by: §1. F. Chazal, V. De Silva, M. Glisse, and S. Oudot (2016) The structure and stability of persistence modules. Springer. Cited by: Theorem 2.1. G. Chen, X. Wang, Q. Sun, and Z. Tang (2025) Multidimensional scaling improves distance-based clustering for microbiome data. Bioinformatics 41 (2), p. btaf042. External Links: ISSN 1367-4811, Document, Link, https://academic.oup.com/bioinformatics/article-pdf/41/2/btaf042/61664392/btaf042.pdf Cited by: §6.4. S. Chen and W. Guo (2023) Auto-encoders in deep learning—a review with new perspectives. Mathematics 11 (8), p. 1777. External Links: Document, Link Cited by: §1. D. Cohen-Steiner, H. Edelsbrunner, and J. Harer (2007) Stability of persistence diagrams. Discrete & Computational Geometry 37 (1), p. 103–120. Cited by: Theorem 2.1. A. F. Duque, S. Morin, G. Wolf, and K. R. Moon (2023) Geometry regularized autoencoders. IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (6), p. 7381–7394. External Links: Document Cited by: §6.3. Edelsbrunner, Letscher, and Zomorodian (2002) Topological persistence and simplification. Discrete & computational geometry 28 (4), p. 511–533. Cited by: §2.1. G. E. Hinton and R. R. Salakhutdinov (2006) Reducing the dimensionality of data with neural networks. Science 313 (5786), p. 504–507. External Links: Document, Link, https://w.science.org/doi/pdf/10.1126/science.1127647 Cited by: §1. C. Hofer, R. Kwitt, M. Niethammer, and A. Uhl (2017) Deep learning with topological signatures. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, Red Hook, NY, USA, p. 1633–1643. External Links: ISBN 9781510860964 Cited by: §6.2. D. P. Kingma and J. Ba (2017) Adam: a method for stochastic optimization. External Links: 1412.6980, Link Cited by: §5.4. A. Krizhevsky (2009) Learning multiple layers of features from tiny images. Technical report University of Toronto. Cited by: §5.1. S. Ladjal, A. Newson, and C. Pham (2019) A pca-like autoencoder. External Links: 1904.01277, Link Cited by: §6.4. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner (1998) Gradient-based learning applied to document recognition. Proceedings of the IEEE 86 (11), p. 2278–2324. External Links: Document Cited by: §5.1. J. Lim, J. Kim, Y. Lee, C. Jang, and F. C. Park (2024) Graph geometry-preserving autoencoders. In Proceedings of the 41st International Conference on Machine Learning, R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp (Eds.), Proceedings of Machine Learning Research, Vol. 235, p. 29795–29815. External Links: Link Cited by: §1.1, §3.3, §3.5, §5.2, §6.2. L. McInnes, J. Healy, N. Saul, and L. Großberger (2018) UMAP: uniform manifold approximation and projection. Journal of Open Source Software 3 (29), p. 861. External Links: Document, Link Cited by: §1.1. MNoichl (2025) MNoichl/umap-examples-mammoth: 0.0.1 External Links: Document, Link Cited by: §5.1. K. R. Moon, D. van Dijk, Z. Wang, W. Chen, M. J. Hirn, R. R. Coifman, N. B. Ivanova, G. Wolf, and S. Krishnaswamy (2017) PHATE: a dimensionality reduction method for visualizing trajectory structures in high-dimensional biological data. bioRxiv. External Links: Document, Link, https://w.biorxiv.org/content/early/2017/03/24/120378.full.pdf Cited by: §1.1. M. Moor, M. Horn, K. Borgwardt, and B. Rieck (2020a) Challenging euclidean topological autoencoders. In NeurIPS 2020 Workshop on Topological Data Analysis and Beyond, External Links: Link Cited by: §6.1. M. Moor, M. Horn, B. Rieck, and K. Borgwardt (2020b) Topological autoencoders. In Proceedings of the 37th International Conference on Machine Learning (ICML), Proceedings of Machine Learning Research. External Links: 1906.00722 Cited by: Figure 1, Figure 1, §1.1, §1.1, §1.2, §2.2, §3.1, §5.1, §5.2, §5.3, §5.4, §6.1. P. Nazari, S. Damrich, and F. A. Hamprecht (2023) Geometric autoencoders - what you see is what you decode. In Proceedings of the 40th International Conference on Machine Learning, A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett (Eds.), Proceedings of Machine Learning Research, Vol. 202, p. 25834–25857. External Links: Link Cited by: §1.1, §3.4, §5.1, §5.2. F. Paul, Y. Arkin, A. Giladi, D. A. Jaitin, E. Kenigsberg, H. Keren-Shaul, D. Winter, D. Lara-Astiaso, M. Gury, A. Weiner, E. David, N. Cohen, F. K. B. Lauridsen, S. Haas, A. Schlitzer, A. Mildner, F. Ginhoux, S. Jung, A. Trumpp, B. T. Porse, A. Tanay, and I. Amit (2015) Transcriptional heterogeneity and lineage commitment in myeloid progenitors. Cell 163 (7), p. 1663–1677. Note: PMID: 26627738 External Links: Document Cited by: §5.1. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and É. Duchesnay (2011) Scikit-learn: machine learning in Python. Journal of Machine Learning Research 12, p. 2825–2830. Cited by: §5.4. C. Pham, S. Ladjal, and A. Newson (2022) PCA-AE: Principal Component Analysis Autoencoder for Organising the Latent Space of Generative Networks. Journal of Mathematical Imaging and Vision 64 (5), p. 569–585. External Links: Link, Document Cited by: §6.4. F. Rosenblatt (1962) Principles of neurodynamics: perceptrons and the theory of brain mechanisms. Cornell Aeronautical Laboratory Report No. VG-1196-G-8, Spartan Books, Cornell Aeronautical Laboratory. Note: Original from the University of Michigan, digitized on 27 Nov. 2007 Cited by: §2.1. I. J. Schoenberg (1935) Remarks to maurice frechet’s article“sur la definition axiomatique d’une classe d’espace distances vectoriellement applicable sur l’espace de hilbert. Annals of Mathematics 36 (3), p. 724–732. Cited by: §2.2. Z. Shi and J. Sun (2016) Convergence of the laplace-beltrami operator from point cloud. External Links: 1403.2141, Link Cited by: §3.5. A. Singh and K. Nag (2021) Structure-preserving deep autoencoder-based dimensionality reduction for data visualization. In 2021 IEEE/ACIS 22nd International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), Vol. , p. 43–48. External Links: Document Cited by: §1.1, §3.3, §4.2, §5.2, §6.2. Z. Su, X. Liu, L. B. Hamdan, V. Maroulas, J. Wu, G. Carlsson, and G. Wei (2025) Topological data analysis and topological deep learning beyond persistent homology: a review. 59 (2), p. 58. External Links: ISSN 1573-7462, Document, Link Cited by: §1.1. W. S. Torgerson (1952) Multidimensional scaling: I. Theory and method. Psychometrika 17 (4), p. 401–419. Cited by: §1.1, §1.2, §2.2. I. Trofimov, D. Cherniavskii, E. Tulchinskii, N. Balabin, S. Barannikov, and E. Burnaev (2023) Learning topology-preserving data representations. In International Conference on Learning Representations, External Links: Link Cited by: §1.1, §1.2, §3.1, §3.2, §5.2, §5.3, §5.4. L. van der Maaten and G. Hinton (2008) Visualizing data using t-sne. Journal of Machine Learning Research 9 (86), p. 2579–2605. External Links: Link Cited by: §1.1. J. Venna and S. Kaski (2001) Neighborhood preservation in nonlinear projection methods: an experimental study. In International conference on artificial neural networks, p. 485–491. Cited by: §5.3. F. A. Wolf, P. Angerer, and F. J. Theis (2018) SCANPY: large-scale single-cell gene expression data analysis. Genome Biology 19 (1), p. 15. Note: PMID: 29409532 External Links: Document, Link Cited by: §5.1. H. Xiao, K. Rasul, and R. Vollgraf (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. External Links: 1708.07747, Link Cited by: §5.1. J. Xu, H. Le, and D. Samaras (2024) Assessing sample quality via the latent space of generative models. In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part LIX, Berlin, Heidelberg, p. 449–464. External Links: ISBN 978-3-031-73201-0, Link, Document Cited by: §1, §6.4.