Paper deep dive
Rare-Aware Autoencoding: Reconstructing Spatially Imbalanced Data
Alejandro Castañeda Garcia, Jan van Gemert, Daan Brinks, Nergis Tömen
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/3/2026, 12:17:17 AM
Summary
The paper introduces 'Rare-Aware Autoencoding', a framework designed to address spatial data imbalance in unsupervised image reconstruction. The authors identify that autoencoders often bias reconstructions toward dominant patterns, leading to blurred details in rare spatial locations. They propose two complementary mechanisms: a self-entropy-based loss function that upweights statistically uncommon spatial locations, and 'Sample Propagation' (spp), a replay mechanism that re-exposes the model to hard-to-reconstruct samples across training batches. The method is validated on simulated and real-world datasets, demonstrating improved reconstruction consistency and detail retention compared to existing baselines.
Entities (5)
Relation Signals (3)
Rare-Aware Autoencoding → addresses → Spatial Data Imbalance
confidence 95% · We address spatial imbalance by two complementary components
Self-entropy-based loss → componentof → Rare-Aware Autoencoding
confidence 95% · We address spatial imbalance by two complementary components: (i) self-entropy-based loss
Sample Propagation → componentof → Rare-Aware Autoencoding
confidence 95% · We address spatial imbalance by two complementary components: ... (ii) Sample Propagation
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Autoencoders can be challenged by spatially non-uniform sampling of image content. This is common in medical imaging, biology, and physics, where informative patterns occur rarely at specific image coordinates, as background dominates these locations in most samples, biasing reconstructions toward the majority appearance. In practice, autoencoders are biased toward dominant patterns resulting in the loss of fine-grained detail and causing blurred reconstructions for rare spatial inputs especially under spatial data imbalance. We address spatial imbalance by two complementary components: (i) self-entropy-based loss that upweights statistically uncommon spatial locations and (ii) Sample Propagation, a replay mechanism that selectively re-exposes the model to hard to reconstruct samples across batches during training. We benchmark existing data balancing strategies, originally developed for supervised classification, in the unsupervised reconstruction setting. Drawing on the limitations of these approaches, our method specifically targets spatial imbalance by encouraging models to focus on statistically rare locations, improving reconstruction consistency compared to existing baselines. We validate in a simulated dataset with controlled spatial imbalance conditions, and in three, uncontrolled, diverse real-world datasets spanning physical, biological, and astronomical domains. Our approach outperforms baselines on various reconstruction metrics, particularly under spatial imbalance distributions. These results highlight the importance of data representation in a batch and emphasize rare samples in unsupervised image reconstruction. We will make all code and related data available.
Tags
Links
- Source: https://arxiv.org/abs/2604.02031v1
- Canonical: https://arxiv.org/abs/2604.02031v1
Trouble viewing inline? Open PDF directly →
Full Text
89,717 characters extracted from source content.
Expand or collapse full text
11institutetext: Delft University of Technology (TU Delft), Delft, The Netherlands 11email: m.a.castanedagarcia@tudelft.nl Rare-Aware Autoencoding: Reconstructing Spatially Imbalanced Data Alejandro Castañeda Garcia Jan van Gemert Daan Brinks Nergis Tömen Abstract Autoencoders can be challenged by spatially non-uniform sampling of image content. This is common in medical imaging, biology, and physics, where informative patterns occur rarely at specific image coordinates, as background dominates these locations in most samples, biasing reconstructions toward the majority appearance. In practice, autoencoders are biased toward dominant patterns resulting in the loss of fine-grained detail and causing blurred reconstructions for rare spatial inputs especially under spatial data imbalance. We address spatial imbalance by two complementary components: (i) self-entropy-based loss that upweights statistically uncommon spatial locations and (i) Sample Propagation, a replay mechanism that selectively re-exposes the model to hard to reconstruct samples across batches during training. We benchmark existing data balancing strategies, originally developed for supervised classification, in the unsupervised reconstruction setting. Drawing on the limitations of these approaches, our method specifically targets spatial imbalance by encouraging models to focus on statistically rare locations, improving reconstruction consistency compared to existing baselines. We validate in a simulated dataset with controlled spatial imbalance conditions, and in three, uncontrolled, diverse real-world datasets spanning physical, biological, and astronomical domains. Our approach outperforms baselines on various reconstruction metrics, particularly under spatial imbalance distributions. These results highlight the importance of data representation in a batch and emphasize rare samples in unsupervised image reconstruction. We will make all code and related data available. 1 Introduction Figure 1: Spatial data imbalance. Top: example frames from a pendulum dataset at different angles. Due to damping, the pendulum spends more time near equilibrium, making some spatial configurations much more frequent than others. Top-right: per-pixel variance over the full dataset, highlighting regions with high motion-induced variability. Bottom-left: temporal intensity traces for four representative pixels: a static background pixel (blue), an extreme-angle pixel (red), a mid-position pixel (green), and an equilibrium pixel (orange). Variance increases as pixels approach the equilibrium region, since the pendulum passes there most often. Bottom-right: histogram of pixel intensities (for the selected pixels), showing the highly non-uniform occupancy of the state space and the resulting spatial imbalance. We investigate small-data imaging problems in the natural sciences, where obtaining and labeling data is expensive, and given this cost, self-supervised and unsupervised learning are particularly relevant. Consequently, autoencoders are a good alternative since they do not need additional labels, as their objective is to reconstruct the input image. Additionaly, autoencoders are essential deep learning models due to their versatility across a wide range of tasks [vincent2008extracting, bengio2013representation] including denoising [vincent2008extracting], anomaly detection [an2015variational], latent dynamics modeling [watter2015embed], generative modeling [diederik_p__kingma_baac31fa], and clustering [makhzani2015adversarial] which makes them relevant for unlabeled small-data imaging problems. Here, we focus on the autoencoder family of models for problems where the spatial locations of objects in an image varies strongly in the dataset. This is especially relevant in the natural sciences, where, e.g., stars in different galaxies have no canonical orientation and the spatial image location of cells in one petri dish, differs from the spatial image locations of cells in a different petri dish. Reasons for variance in the spatial location can be traced back to acquisition difficulties [chen_sun_f5900b3a, linus_ericsson_30c98a67, peng_zheng_d3e44641], either for a single source [finn2017model], or a continuous stream such as video [srivastava2015unsupervised, xiaofeng_zhang_a4031cb]. We make the observation that high variance, in recorded object placements over different spatial image locations in the dataset, leads to an imbalance in the unsupervised learning problem. This imbalance is particularly present when the number of training samples is small, and objects are not equally sampled over spatial image locations in the training dataset; see Fig. 1 for an illustration. Imbalance is a known problem [yifan_zhang_caa37b4d, charika_de_alvis_6e7a21a0, lu_yang_c3b53d32, pintea2023step] for supervised learning when there is an imbalance in the class labels. However, in the unsupervised setting, where labels are not available, the effect of imbalance is less explored. Here, we investigate how imbalance in the spatial image locations affects autoencoders. In this paper we view unsupervised image reconstruction problems, for the first time, through the lens of data imbalance at spatial image locations. Inspired by methods for data imbalance in supervised settings, we design two new independent methods to handle imbalance in the unsupervised setting. One method is inspired by changing the loss and giving a higher weight to rare classes [cao2019learning, lin2017focal]. Since we don’t have class labels, we introduce a self-entropy based loss to prioritize rare data. Similarly, another typical approach in supervised learning is emphasizing data samples [Schaul2016PER, Wu2016OnlineBootstrapping, Gu2020HardPixelMining]. Likewise, we studied sample weighting in an unsupervised setting and propose sample propagation to increase the exposure of rare inputs during training. We list here our main contributions: • We investigate autoencoder reconstruction problems through the lens of spatial location data imbalance. • We propose a self-entropy based reconstruction loss that prioritizes rare spatial pixels to avoid convergence to mean values and decrease blurring. • We introduce Sample Propagation (spp), a hard-example replay mechanism, increasing the exposure of images with spatial imbalance. • We, for the first time, benchmark various data balancing strategies and loss functions for autoencoders. 2 Related work Image reconstruction in representation learning. Autoencoders depend on the latent space to capture relevant image structures [srivastava2015unsupervised, hsieh2018learning, jaques2019physics, adiban2025s]. But when data distributions are biased or imbalanced, traditional loss functions often compress too aggressively or average out rare patterns. This weakens the learned representation and leads to reconstructions that are blurry or uninformative [srivastava2015unsupervised, buda2018systematic, geirhos2020shortcut]. Nevertheless, the spatial imbalance is never discussed; in contrast, we explore the effect of spatial imbalance to improve sample reconstruction to avoid blurring or averaged reconstructions. Reconstruction challenges due to data statistics. In domains like natural sciences, data from a single video often exhibits a spatial capture bias and repetitive patterns [jaques2019physics, oakden2020exploring, puyol2021fairness, garcia2024learning]. This can lead models to prioritize frequent patterns while overlooking transient or spatially rare events, resulting in blur or averaging in regions with high pixel variance [guo2023event, ercan2024hypere2vid, zhang2025reconstruction]. This imbalance leads to two main challenges. First is the averaged reconstructions, where the AE averages out or blurs details in locations with rare pixel values. Second is unbalanced samples, where the images with underrepresented rare locations are not often seen during training. Image blurring. On image reconstruction, focused losses propose plug-in alternatives which do not require architectural changes. These include pixel-level (L1, L2), perceptual (VGG [zhang2018unreasonable], Watson [czolbe2020watson]), frequency-aware (FFL [jiang2021focal]) and class-balanced (LDAM [cao2019learning], Focal [lin2017focal]) losses. Similar, Bredell et al. [bredell2023explicitly] introduced a blur-aware loss (BEL) for variational autoencoders (VAEs) targeting high-frequency errors while preserving ELBO maximization. While these methods offer diverse reconstruction strategies, they have not been compared and evaluated under sample imbalance. Here, we provide such an evaluation in comparison to our proposed method, to properly formalize the spatial imbalance problem in AE reconstruction and show whether the methods dealing with blurriness aid the underlying problem. Imbalanced sample distribution in supervised settings. Data imbalance is extensively studied in classification [yifan_zhang_caa37b4d, pintea2023step], where the goal is to ensure minority classes are properly represented. However, data imbalance has received less attention in reconstruction. Some methods like Online Hard Example Mining [Shrivastava2016OHEM], Prioritized Experience Replay [Schaul2016PER], and hard-pixel mining [Wu2016OnlineBootstrapping, Gu2020HardPixelMining] are ill-suited in a unsupervised setup since their notion of “hardness” depends on previous supervised annotations. Other methods can be transferred to unsupervised settings such as, loss function modifications, like reweighting sample contributions [cao2019learning, lin2017focal], have proven effectiveness. Likewise, Loss-based solutions such as focal loss [lin2017focal]), or margin-based adjustments such as ldam [cao2019learning]. Beyond loss adjustments, strategies such as online batch selection [loshchilov2015online], curriculum learning [bengio2009curriculum] and selection via proxy [Coleman2020SVP] dynamically prioritize harder samples. Mentioned approaches suggest that imbalance can be addressed through both loss design and data sampling. However, their application to autoencoders remains largely unexplored–particularly within the batch learning paradigm [goodfellow2016deep], which is central to modern deep learning pipelines. Ignoring batch-level dynamics can bias gradient estimates, underrepresented rare patterns, and degrade generalization in reconstruction. While the existing methods work on their particular task, we studied them together, for the first time, in a unsupervised setup for image reconstruction with spatial imbalanced data. 3 Methods Figure 2: Addressing reconstruction under spatial data imbalance. Consider a dataset of simulated pendulum dynamics. Black squares are input or reconstructed frames. Left-top: Input distributions exhibit a strong bias, with common, overrepresented samples (blue) and underrepresented ones (red). Left-bottom: Standard autoencoder (AE) backbone (encoder–latent–decoder). The SOTA (cyan) approach tends to converge toward smooth reconstructions dominated by frequent samples. Right: Our approach (orange) introduces two complementary mechanisms: (i) a self-entropy loss that upweights high-surprisal pixels across the batch within each image, and (i) sample propagation that replays high-loss (hard) samples across epochs, ensuring rare events are repeatedly observed. The combined effect yields sharper and more balanced reconstructions. Self-entropy loss Our loss aims to decrease the blurriness caused by converging to the average pixel values. The intuition is to assign greater importance to pixel values that are statistically rare across a batch, in theory we are interested in the full distribution, but in practice we look at the batch level since we do not want to weight spatial locations that the model won’t see in a particular update. The model is encouraged to prioritize spatially imbalanced locations over statistically redundant locations. Specifically, we first measure the self-entropy, associated with each pixel across the batch, those pixel forms the rare location to prioritize denominated as "entropy mask". For image batch X and its reconstruction X X, with X,X^∈[0,1]B×C×H×WX, X∈[0,1]^B× C× H× W, batch size B, number of channels C, width W and height H, we flatten spatial and channel dimensions so xb,ℓx_b, indices sample b∈1,…,Bb∈\1,…,B\ and location ℓ∈1,…,L ∈\1,…,L\ with L=CHWL=CHW. We estimate the distribution using a histogram with J∈ℕJ bins and bin width δ=1/Jδ=1/J. We discretize each xb,ℓx_b, to a bin kb,ℓ=⌊xb,ℓ⋅(J−1)⌋∈0,…,J−1k_b, = x_b, ·(J-1) ∈\0,…,J-1\ and count occurrences cℓ,kc_ ,k of pixel values in the batch, cℓ,k=∑b=1B[kb,ℓ==k]c_ ,k= _b=1^B1[\,k_b, ==k\,] to estimate the pixel value probability (more details in Appendix 0.A.1) p^ℓ(xb,ℓ)≈cℓ,k/Bδ=JBcℓ,k. p_ (x_b, )≈ c_ ,k/Bδ= JB\,c_ ,k. -6.0pt (1) We calculate the self-entropy of each pixel sb,ℓs_b, sb,ℓ=−log(p^ℓ(xb,ℓ)+ε),s_b, =- \! ( p_ (x_b, )+ ), -3.60004pt (2) using ε>0 >0 for numerical stability. We reshape back to (B,C,H,W)(B,C,H,W), and apply per-(image,channel) min–max normalization, and define an "entropy mask" as s~ s: mb,c=minh,wsb,c,h,w,Mb,c=maxh,wsb,c,h,w⇒ m_b,c\!=\! _h,ws_b,c,h,w, M_b,c\!=\! _h,ws_b,c,h,w\; \; (3a) s~b,c,h,w=sb,c,h,w−mb,cMb,c−mb,c+ε∈[0,1]. s_b,c,h,w= s_b,c,h,w-m_b,cM_b,c-m_b,c+ ∈[0,1]. -6.0pt (3b) where b,c,h,wb,c,h,w are indices for each pixel in [1,…,B][1,…,B], [1,…,C][1,…,C], [1,…,H][1,…,H], [1,…,W][1,…,W] respectively. We define our self-entropy loss as a spatially weighted L1 distance. Let the pixel-wise error be: L1b,c,h,w=|X^b,c,h,w−Xb,c,h,w|L1_b,c,h,w= X_b,c,h,w-X_b,c,h,w The corresponding self-entropy weight is defined as: entb,c,h,w=s~b,c,h,w+0.05ent_b,c,h,w= s_b,c,h,w+0.05. where L1∈[0,1]L1∈[0,1] and s~∈[0,1] s∈[0,1]. Intuitively, the L1 term encourages the recovery of sharp details, while the self-entropy weight ent∈[0.05,1.05]ent∈[0.05,1.05] identifies which pixels to prioritize. The constant 0.05 acts as a smoothing factor to ensure that common pixels are not completely ignored during training. To ensure the model reconstructs both rare features and the global image structure, we balance the standard Mean Squared Error (MSE) with our proposed self-entropy loss: Entb _b =1CHW∑c,h,wL1b,c,h,wentb,c,h,w, = 1CHW\! _c,h,wL1_b,c,h,w\,ent_b,c,h,w, (4a) MSEb -2.39996ptMSE_b =1CHW∑c,h,w(X^b,c,h,w−Xb,c,h,w)2, = 1CHW\! _c,h,w ( X_b,c,h,w-X_b,c,h,w )^2, -6.0pt (4b) and combine with a trade-off coefficient λ>0λ>0: ℓb=MSEb+λEntb,ℒ=1B∑b=1Bℓb. _b=MSE_b+λ\,Ent_b, = 1B _b=1^B _b. (5) Sample propagation (Spp) Entropy alone is not enough to solve the problem, while self-entropy prioritizes a particular location, the model may forget it if the samples with this data imbalance are not seen again. To counter data imbalance, we propagate difficult samples across subsequent batches. We find this more favorable to reweighting individual samples per batch as traditional focal methods. Our algorithm identifies samples with rare, difficult-to-reconstruct spatial locations and propagates them into subsequent batches to prevent the model from "forgetting" these features. To avoid overfitting to these specific patterns, we restrict the model update to the top B hardest samples. This ensures that while the effective batch size processed by the model increases, the number of samples used for gradients remains constant, maintaining training efficiency. For a particular batch bib_i with size B during training: 1) Reconstruct: Compute the reconstruction for all samples in batch bib_i. 2) Evaluate: Sort samples based on their individual reconstruction error. 3) Identify: Select the M most difficult samples as "hard" cases. 4) Propagate: Concatenate these M samples into the next training batch bi+1b_i+1. 5) Update: Perform a gradient update using only the B hardest samples from the resulting pool. Sample propagation is presented in Fig. 3. Figure 3: Sample propagation algorithm. First, for each input sample, the model produces a reconstruction error. The color scale encodes error magnitude: red denotes higher error, while decreasing progressively to cooler hues. Second, we compute a error per sample and sort/rank the batch accordingly, yielding an ordered list from highest to lowest reconstruction error. Finally, a propagation step selects the M top-ranked (hardest) samples for targeted optimization and concatenates them to the next batch; gradients are then applied (loss backward) to update the network with an emphasis on hard examples. We use the following technique inspired by the focal-loss [lin2017focal]. With ℓmin=minbℓb _ = _b _b, ℓmax=maxbℓb _ = _b _b, define fb=1+ℓb−ℓminmax(ℓmax−ℓmin,ε)(B−1)∈[1,B],f_b=1+ _b- _ ( _ - _ ,\, )(B-1)∈[1,B],\\ (6) where the final loss is ℒ=1B∑b=1Bfbℓb.L= 1B _b=1^Bf_b\, _b. (7) 4 Experiments To ensure a fair comparison, we set the same training hyperparameters across baselines and proposed methods. The models are trained on an NVIDIA A40 GPU using the Adam optimizer with a learning rate lr=1.0e−3lr=1.0e^-3 and weight decay wd=1.0e−5wd=1.0e^-5. For full architectural details, refer to the supplementary material. We evaluate using Mean Squared Error (MSE↓), Peak Signal-to-Noise Ratio (PSNR↑), and Structural Similarity Index (SSIM↑), following prior work [mustafa2022training, bredell2023explicitly]. For notation, we denote our sample propagation method as sppkk related to memory size M as follows: M=BkM= Bk . For sppksppk algorithm, our internal experiments showed that a fixed amount of memory was more stable than a dynamic memory size; therefore, we compare different memory sizes. For simplicity and readability we refer to "self-entropy" just as "entropy" and experiments denoted with "spp+entropy" we used k=4k=4. 4.1 Experiments on controlled data We use a simulated dataset [garcia2024learning] of a damped pendulum, with a size of (64×64)(64× 64) pixels, where the black background and the damping leads to spatial imbalance and blurry reconstructions. We use MNIST as an example of a homogeneous data distribution with uniform spatial coverage. Results for a fully connected (MLP) AE encoder and decoder are shown in Fig. 4. For the imbalanced pendulum, the autoencoder overfits to dominant spatial locations, leading to incomplete or distorted reconstructions. In contrast, our spp+entropy method maintains reconstruction fidelity in both cases, demonstrating robustness to spatial imbalance distributions that otherwise degrade in conventional reconstruction approaches. Figure 4: Comparison between balanced and imbalanced reconstruction scenarios. The MNIST dataset (left) represents a balanced case where samples are more uniformly distributed across the spatial domain. In contrast, the damped pendulum dataset (right) introduces imbalance, as certain image spatial locations are severely underrepresented across the dataset and within training batches. Our proposed spp+entropy framework reconstructs both balanced and imbalanced data faithfully, while the standard MLP AE pipeline performs well only on homogeneous datasets such as MNIST but fails to preserve structure under imbalance. Figure 5: (Left) Qualitative comparison. The first row shows the ground truth (GT). Methods are arranged by rows: (Left) comparison of baselines and our entropy-based methods (entropy*, entropy+spp*); (right) baselines and our underrepresented-sample-focusing spp variants (spp2*, spp4*, spp8*). Columns include both an Underrepresented sample (Us) and an Average sample (As) to visualize behavior on rare versus frequent cases. The layout highlights how different objectives affect sharp structures and textures: the left column makes visible the tendency of some losses to approach a dataset-average appearance, whereas the spp block illustrates how prioritizing underrepresented samples influences detail retention and stability across memory sizes k. Table 1: (Right) Reconstruction performance on the controlled dataset. The table reports reconstruction metrics (MSE↓, PSNR↑, SSIM↑; mean± over samples) for two complementary proposals: (i) Entropy (top block), which evaluates losses under a setting prone to convergence toward a dataset’s statistical mean, and (i) spp (bottom block), which benchmarks losses against our sample-propagation with different memory sizes (spp2, spp4, spp8) which prioritizes underrepresented samples. ∗* denotes our methods; entries in bold indicate the best mean value within each experiment block for a given metric. Our methods demonstrate better qualitative performance at reconstruction while keeping a high statistical score across the whole dateset. In Fig. 5 and Table 1 we evaluate various baselines. Most baselines target fidelity and perceptual agreement: pixel losses (L1/L2), SSIM for structural similarity [Wang2004SSIM], perceptual (VGG-feature) losses [zhang2018unreasonable], the Watson DCT-based perceptual error model [czolbe2020watson], frequency-aware FFL [jiang2021focal], and margin/imbalance-aware ldam [cao2019learning]. Finally the recent BEL (Blur Error Loss) [bredell2023explicitly] introduced by Bredell et al. Baseline methods for our entropy loss explicitly penalize error in pixel/structure/frequency spaces or encourage larger margins on minority patterns. For the sample-propagation baselines adapt the sampling or loss to class/instance rarity: Focal loss emphasizes hard/rare examples [lin2017focal]; curriculum learning schedules data from easy to hard [bengio2009curriculum]; online batch selection (onbs) prefers high-loss items within each step [loshchilov2015online]; and selection via proxy (svp) uses a lightweight proxy model to guide which samples to revisit [Coleman2020SVP]. As seen in Fig. 5, across datasets and architectures, our two methods provide complementary benefits: the entropy-based loss counteracts collapse toward mean appearances, recovering sharper details and higher PSNR / lower MSE, while the underrepresented-sample focusing (sppkk) extends the exposition of rare samples. We note that the BEL baseline attains strong scores on the toy setting aided by its sensitivity to the synthetic, high-contrast edges—yet this advantage diminishes on real data where textures and structures are more varied (see Sec. 4.2). The qualitative comparisons in Fig. 5 highlights the differences: our methods preserve fine structure and rare patterns that baselines tend to smooth, translating the quantitative gains into more faithful reconstructions. Furthermore, Fig. 6(a) indicates that our objective reduces average error across all pixels and maintains low error on pixels with high surprise—precisely where models typically fail. In contrast, standard losses distribute error more evenly over the surprise axis, leaving a sizable fraction of high-surprise pixels with larger residuals. (a) Per-pixel error vs. surprise with marginal distributions. (Left) Scatter–density plots show absolute reconstruction error against per-pixel surprise (higher = rarer/less expected) for baselines and our methods (entropy*, spp+entropy*). The joint distributions for our methods concentrate near the origin—low error even at moderate–high surprise—while baselines spread more uniformly across surprise values. (Right) The accompanying marginal histograms highlight that our curves accumulate most mass at small absolute differences, with markedly reduced heavy tails compared to baselines. (b) Error vs. sample frequency (grouped by occurrence count). Mean absolute difference against binned sample frequency in the dataset for baselines (in blue) and our sample propagation variants (in red: spp2*, spp4*, spp8*). Baselines are overlapped in a single column as well as our spp variants to improve readability. Our curves remain both lower and flatter across the frequency axis—indicating uniformly low reconstruction error—even in the rarest bins. In contrast, baseline errors rise as frequency decreases, revealing sensitivity to underrepresented cases. Figure 6: Hypothesis testing on synthetic data. On figure (a) we show how our entropy loss reduce the error of individual pixel equally, while on (b) sample propagation reduces the error on underrepresented samples having a similar performance in all samples. Fig. 6(b) shows that propagating underrepresented samples yields improvements that are global (lower average error across all counts) and targeted (especially strong in low-frequency bins). Whereas conventional objectives exhibit a pronounced error ramp as samples become rarer, our sppkk variants maintain a nearly flat error profile—evidence that the model learns features that transfer to tail examples rather than overfitting to frequent modes. Among our methods, larger memories (spp2*, spp4*) further stabilize the tail without sacrificing performance on common cases, supporting the claim that controlled replay of rare patterns mitigates frequency-induced degradation and improves robustness across the full data distribution. Finally, in Fig. 7 we visualize a restricted latent space, using only one-dimensional (1D) space. Our method has lower MSE/higher PSNR with comparable or reduced variance, which is visually apparent in the trajectories: the L2 loss leads to latent space collapse, while BEL partly captures the oscillatory dynamics, but does not yield a faithful trajectory or detailed reconstructions. In contrast, our method remains expressive of the system’s salient, time-varying structure. In short, under restricted capacity, our objective resists premature convergence and captures dynamics more aligned with the real dynamics. Figure 7: Latent–capacity comparison under restricted bottlenecks. Expressivity in the AE latent space when its dimensionality is intentionally constrained to 1D. We report the time–series plots that trace the decoded signal across samples. Under tight bottlenecks, baseline trajectories either collapse (L2) or do not follow the oscillations faithfully (BEL), whereas our method (spp+entropy) has rich latent dynamics that remain coupled to the input sequence. Example reconstructions from each method (upper left) illustrate it. 4.2 Real data experiments We used three different datasets which have spatial distribution imbalance or imbalance due to rare events. Pendulum: A real-world pendulum video [garcia2024learning]. A static background and recurring ball positions dominate the frame distribution, while some ball positions appear infrequently due to the pendulum’s damping. Galaxy zoo: A dataset of galaxy images spanning diverse morphological types [lintott2011galaxy]. The dataset features a uniformly black background and predominantly centered galaxies. Off-center galaxy images and examples that showcase fine structural details (like stars) occur infrequently. Mitosis: Time-lapse microscopy sequence for cell tracking and mitosis detection from the MOTChallenge CTMC-v1 dataset [anjum2020ctmc].The dataset shows uniform imaging conditions and interphase cell instances, while mitotic events appear infrequently due to their brief duration. We evaluate the following autoencoder variants: An MLP Autoencoder (AE) [chen2023auto] consisting of fully connected architecture composed of a multilayer perceptron (MLP) encoder and decoder, a Variational Autoencoder (VAE) [diederik_p__kingma_baac31fa, chen2023auto], a Sparse Autoencoder (SAE) [chen2023auto], and a Masked Autoencoder (MAE) [he2022masked]. Entropy: We analyzed the performance of the proposed entropy loss function (Eq. 5) qualitatively compared to baseline loss functions (Fig. 9). Quantitatively, tables 8, 9, 10 (in the appendix) are summarized in Fig. 8(a) which shows the performance of our method. For the quantitative evaluation, we compare standard pixel-wise loss functions (L1, L2, CE), and structural (SSIM), frequency/feature (FFL), margin-aware (ldam) and perceptual (VGG, Watson, BEL) losses against our proposed entropy* loss and its variant spp+entropy* on four backbones tailored to the experiment (AE, VAE, SAE, MAE). (a) Reconstruction results across datasets and architectures with different losses. Across all datasets and architectures, our losses consistently achieve the best distortion metrics (lowest MSE, highest PSNR), with frequent SSIM ties or improvements. On repetitive dynamics (Pendulum), entropy–based training prevents convergence to oversmoothed solutions, yielding MSE reductions over strong perceptual/pixel baselines. On structured textures (Galaxy), it improves PSNR and MSE while remaining SSIM–competitive. On fine cellular detail (Mitosis), it preserves morphology, improving PSNR and reducing MSE in AE/SAE. Overall, maximizing information content during training generalizes across regimes, sharpening the reconstruction and avoiding collapse. (b) Sample focus reconstruction performance across datasets and architectures. Overall, sppksppk improves average reconstruction quality relative to pixel/perceptual baselines while keeping the standard deviation comparable or lower—indicating better robustness to rare/atypical samples. Taken together, the figure shows that focusing on underrepresented samples improves the general mean–variance reconstruction in favor of our model. Figure 8: Summary figure of our extensive results detailed in appendix tables 8 to 12. Fig. 8(a) shows the performance of our entropy* loss function and 8(b) the results for our spp algorithm. Both sub-figures show the performance on three different datasets Pendulum, Galaxy, and Mitosis over four autoencoder families (AE, VAE, SAE, MAE). For better visualization, we re-normalized the metrics from 0.2to 1.00.2\; to\;1.0 where high is always better. In the presented radar plots more area implies a better overall reconstruction. Baseline methods are shown in shades of blue while our methods are in a red scale. Figure 9: Qualitative comparison under convergence: convergence toward the statistical mean vs. detail preservation. For representative samples from Pendulum, Galaxy, and Mitosis datasets, we show GT and reconstructions with baselines and our methods. The shown samples were randomly drawn from the set; for clarity, we display the architecture per dataset where the effect is most legible, while other samples show similar tendencies. Pixel/perceptual baselines frequently converge to a dataset “mean” appearance—yielding blurred trajectories, deleting stars, and washed-out cellular boundaries, our losses retain more of the high-frequency structure and rare patterns, avoiding averaging and preserving relatively sharper edges and textures. Across three dataset containing repetetive dynamics (Pendulum), structured textures (Galaxy), and fine cellular morphology (Mitosis), our entropy-based objectives consistently deliver the best metrics (lowest MSE, highest PSNR) across all four architectures, while remaining competitive or superior in SSIM. On multiple experiments we see a consistent gain in PSNR while decreasing the MSE an order of magnitude (e.g. Pendulum-SAE, Galaxy-AE), indicating that maximizing information content prevents collapse to oversmoothed reconstructions. Some configurations present a better SSIM than ours, however our method is still comparable (e.g. <0.025<0.025 improvement on Pendulum-VAE, Galaxy-SAE, Mitosis-VAE). Still our method retains a better performance in PSNR and MSE, suggesting sharper and perceptually faithful textures. On Mitosis-VAE, entropy-based training preserves subtle cellular structure while substantially lowering distortion, outperforming baselines for all metrics. Our approach attains near-state-of-the-art SSIM with a clear better MSE/PSNR, highlighting a favorable position with respect to the competitors. Collectively, results imply that entropy maximization acts as a regularizer against convergence in repetitive or low-diversity regimes, improving generalization and detail preservation across architectures and data domains without task-specific tuning. As illustrated in Fig. 9, most baselines frequently converge toward a statistical mean appearance—producing blurred pendulum traces, smoothed galaxy arms, and softened cellular boundaries—whereas our entropy-based and sppkk objectives preserve high-frequency structure and rare patterns, yielding sharper trajectories, crisper spiral features, and better-defined mitotic contours that align with the quantitative gains reported above. Sample propagation In this section, we evaluate the reconstruction performance of the AE models trained with our spp method. Spp is designed to improve learning from underrepresented samples, and we compare its results with baseline approaches. We show a qualitative analysis in Fig. 10, while the statistical results are summarized in Fig. 8(b) the associated exhaustive tables can be found in the appendix 0.A.4. Across all three datasets, sppkk improves average reconstruction. On repetitive dynamics (Pendulum), larger memory leads to a better overall reconstruction, while on structured textures and fine details (Galaxy, Mitosis) smaller memories (spp4, spp8) dominate, suggesting that maintaining a buffer of rare patterns stabilizes learning and helps the model preserve high-frequency structure. Methodically, sppkk mitigates the bias towards frequent samples by increasing the effective sampling of underrepresented samples; this reduces blurring (better PSNR/MSE) and improves structural fidelity (SSIM). Overall, results indicate that targeting underrepresented samples is a principled way to more reliable outputs across data regimes and model families. As shown in Fig. 10, focusing on underrepresented samples also produces visibly sharper reconstructions and better retention of rare patterns. Figure 10: Qualitative comparison with underrepresented-sample focusing (sppkk). Representative cases from the Pendulum, Galaxy, and Mitosis datasets. We compare GT against L2/Focal/onbs/dcl/svp and our spp2*, spp4*, spp8* methods. The shown samples were randomly drawn from the test sets; for clarity, we display the architecture per dataset where the effect is most legible, while other examples show similar tendencies. Pixel/perceptual baselines often smooth toward frequent patterns—fading spiral arms, indistinct mitotic contours, and blurred pendulum traces—whereas sppkk preserves rare/high-frequency structure, yielding relatively crisper edges and textures. 4.3 Downstream tasks Finally, in Table 2, we examine the benefits of applying our method across different tasks. First, we evaluate the image generation task used in BEL [bredell2023explicitly]. To ensure a fair comparison, we reimplemented their model and then built our method on top of it, we also report FID and LPIPS as standard generation metrics, and our approach outperforms the best referenced method. Second, we evaluate anomaly detection using the same dataset, setup, and metrics as the baseline in [collin2021improved], achieving higher accuracy. Finally, we assess the impact of integrating our method into conventional AE training for physical parameter estimation. Using the same synthetic dataset as the Delfys baseline [garcia2024learning], we addressed the inherent autoencoder limitations discussed in [garcia2024learning], matching the performance of their best-reported models. Overall, results demonstrate that our framework consistently enhances performance across diverse domains in particular in tasks which suffer from spatial data imbalance. Table 2: Downstream task performance (bold is better). In this table we compare three different task showing how our method can aid task by increasing the performance. Task Model Metrics Generation FID LPIPS BEL [bredell2023explicitly] 280.5 0.3467 ± 0.0405 Ours 221.8 0.2826 ± 0.0512 Anomaly detection Image AUROC pixel AUROC AESc [collin2021improved] 0.6017 0.7853 Ours 0.6247 0.7875 Pendulum parameter estimation Abs. Err. (frequency) Abs. Err. (damping) Delfys [garcia2024learning] 0.0586 0.064 AE 1.0016 3.32 Ours 0.0516 0.002 5 Discussion Autoencoders are a core tool in unsupervised learning, enabling efficient representation learning across diverse tasks. However, the effect of spatial imbalance remains underexplored, despite its critical impact on downstream performance. Imbalanced spatial distributions can lead to suboptimal representations, reducing generalization and obscuring important variations. This study takes a first step toward addressing an often-overlooked challenge of data imbalance. By tackling this issue systematically, we show how unsupervised autoencoders can be improved without altering their underlying architectures. In summary, our methods address the issue of spatial imbalance by improving the learning of underrepresented spatial locations, promoting a more balanced reconstruction. The self-entropy-based objective mitigates blurring and over-smoothing by maximizing information content, producing sharper, higher-fidelity reconstructions that preserve fine details across architectures. The sample propagation algorithm (spp) improves average accuracy and enhances retention of rare events while reducing focus on already learned features, leading to better performance in general across the whole dataset. Future Work: While our approach demonstrates significant improvements, a deeper understanding of data distribution behaviors is essential to determine when and how these techniques can be effectively applied on task-specific constraints. Limitations: Our study does not focus on a specific application domain, further validation in more complex and domain-specific setups is necessary to assess performance across diverse datasets and tasks. References Appendix 0.A Technical Appendices and Supplementary Material 0.A.1 Mathematical details Probability calculation. Pixel intensities are continuous on [0,1][0,1]. Thus, we define a histogram over a continuous variable, which estimates a density (units: inverse intensity), not a discrete probability mass. Normalizing counts by the batch size, c/Bc/B, yields a frequency for a bin of width δ; converting to a density requires dividing by δ : p^≈c/Bδ=JBc. p\ ≈\ c/Bδ\ =\ JB\,c. -5.69046pt Dividing by B makes the estimate invariant to batch size; multiplying by J=1/δJ=1/δ removes the dependence on the chosen bin width. Using a PMF-like c/Bc/B alone would shrink as bins get narrower (larger J), making −log(⋅)- (·) incomparable across choices of J. The PDF-based surprise s=−logp^s=- p is therefore bin-width–invariant and consistently highlights statistically rare (high-surprise) pixels. 0.A.2 Model Architectures Across all experiments, we relied on the architectures detailed in Table 3, 4, 5, and 6; which define the AE, VAE, SAE, and MAE networks respectively. While the overall structure remained consistent, the layer dimensionality and size were adapted to the specific dataset or method. Table 3: MLP Autoencoder architecture. The input image of size (c,h,w)(c,h,w) is flattened to dimension c⋅h⋅wc·h·w, where c is the number of channels and h,wh,w are the spatial dimensions. The encoder compresses the input into a latent vector of dimension l. The reduction factors did_i denote successive proportions used to shrink the layer dimensionality relative to the input size. The decoder mirrors this process to reconstruct. Stage Layer In Out Output Shape Encoder 1 Linear chwc\,h\,w chw/d1c\,h\,w/d_1 (chwd1)( c\,h\,wd_1) ReLU (chwd1)( c\,h\,wd_1) 2 Linear chw/d1c\,h\,w/d_1 chw/d2c\,h\,w/d_2 (chwd2)( c\,h\,wd_2) ReLU (chwd2)( c\,h\,wd_2) 3 Linear chw/d2c\,h\,w/d_2 l (l)(l) Decoder 1 Linear l chw/d2c\,h\,w/d_2 (chwd2)( c\,h\,wd_2) ReLU (chwd2)( c\,h\,wd_2) 2 Linear chw/d2c\,h\,w/d_2 chw/d1c\,h\,w/d_1 (chwd1)( c\,h\,wd_1) ReLU (chwd1)( c\,h\,wd_1) 3 Linear chw/d1c\,h\,w/d_1 chwc\,h\,w (c,h,w)(c,h,w) Sigmoid (c,h,w)(c,h,w) Table 4: Variational Autoencoder (VAE) architecture. The input (c,h,w)(c,h,w) is encoded through three convolutional stages, where each stage corresponds to a resolution-changing convolution (stride 2) followed by non-linearities. After flattening, fully connected layers reduce the representation to a latent distribution of dimension l, parameterized by mean (μ) and variance (σ). Channel sizes n1,n2,n3n_1,n_2,n_3 are the number of feature maps at each stage, and (hi,wi)(h_i,w_i) are the spatial dimensions after stage i. The decoder mirrors this process with three transposed convolution stages, reconstructing the image back to (c,h,w)(c,h,w). Intermediate fully connected layers are scaled by proportional factors d1d_1 and d2d_2 relative to n3n_3. Stage Layer In Out Kernel / Stride / Pad Output Shape Encoder 1 Conv2d c n1n_1 3×3,s=2,p=13× 3,\;s=2,\;p=1 (n1,h1,w1)(n_1,h_1,w_1) ReLU (n1,h1,w1)(n_1,h_1,w_1) 2 Conv2d n1n_1 n2n_2 3×3,s=2,p=13× 3,\;s=2,\;p=1 (n2,h2,w2)(n_2,h_2,w_2) ReLU (n2,h2,w2)(n_2,h_2,w_2) 3 Conv2d n2n_2 n3n_3 3×3,s=2,p=13× 3,\;s=2,\;p=1 (n3,h3,w3)(n_3,h_3,w_3) ReLU (n3,h3,w3)(n_3,h_3,w_3) Flatten (n3h3w3)(n_3h_3w_3) MLP Encoder Linear n3h3w3n_3h_3w_3 d1ld_1l – (d1l)(d_1l) ReLU (d1l)(d_1l) Linear d1ld_1l d2ld_2l – (d2l)(d_2l) ReLU (d2l)(d_2l) Linear(μ) d2ld_2l l – (l)(l) Linear(σ) d2ld_2l l – (l)(l) MLP Decoder Linear l d2ld_2l – (d2l)(d_2l) ReLU (d2l)(d_2l) Linear d2ld_2l n3h3w3n_3h_3w_3 – (n3h3w3)(n_3h_3w_3) ReLU (n3h3w3)(n_3h_3w_3) Reshape – (n3,h3,w3)(n_3,h_3,w_3) Decoder 1 ConvTrans n3n_3 n2n_2 3×3,s=2,p=1,op=13× 3,\;s=2,\;p=1,\;op=1 (n2,h2,w2)(n_2,h_2,w_2) ReLU (n2,h2,w2)(n_2,h_2,w_2) 2 ConvTrans n2n_2 n1n_1 3×3,s=2,p=1,op=13× 3,\;s=2,\;p=1,\;op=1 (n1,h1,w1)(n_1,h_1,w_1) ReLU (n1,h1,w1)(n_1,h_1,w_1) 3 ConvTrans n1n_1 c 3×3,s=2,p=1,op=13× 3,\;s=2,\;p=1,\;op=1 (c,h,w)(c,h,w) ReLU (c,h,w)(c,h,w) Conv2d c c 3×3,s=1,p=13× 3,\;s=1,\;p=1 (c,h,w)(c,h,w) ReLU (c,h,w)(c,h,w) Conv2d c c 3×3,s=1,p=13× 3,\;s=1,\;p=1 (c,h,w)(c,h,w) Sigmoid (c,h,w)(c,h,w) Table 5: Sparse Autoencoder (SAE) architecture. The encoder reduces the input image (c,h,w)(c,h,w) through three convolutional layers, producing feature maps (n3,h0,w0)(n_3,h_0,w_0) with h0=h/8h_0=h/8 and w0=w/8w_0=w/8. This representation is flattened and mapped by an MLP into a latent vector of dimension l, with proportional factors d1d_1 and d2d_2 controlling the intermediate hidden sizes. A sparsity regularization is applied on l to encourage compact codes. The decoder mirrors this process, expanding the latent code through an MLP, reshaping into convolutional feature maps, and progressively upsampling with transposed convolutions to reconstruct (c,h,w)(c,h,w). Stage Layer (Parameters) Output Shape Encoder Conv2d (c→n1), 3×3,s=2,p=1(c→ n_1),\;3× 3,\;s=2,\;p=1 (n1,h/2,w/2)(n_1,\,h/2,\,w/2) ReLU – – Conv2d (n1→n2), 3×3,s=2,p=1(n_1→ n_2),\;3× 3,\;s=2,\;p=1 (n2,h/4,w/4)(n_2,\,h/4,\,w/4) ReLU – – Conv2d (n2→n3), 3×3,s=2,p=1(n_2→ n_3),\;3× 3,\;s=2,\;p=1 (n3,h/8,w/8)(n_3,\,h/8,\,w/8) ReLU – – Flatten – (n3h0w0)(n_3\,h_0\,w_0) MLP Encoder Linear (n3h0w0→d1l)(n_3h_0w_0→ d_1l) (d1l)(d_1l) ReLU – – Linear (d1l→d2l)(d_1l→ d_2l) (d2l)(d_2l) ReLU – – Linear (d2l→l)(d_2l→ l) (l)(l) MLP Decoder Linear (l→d2l)(l→ d_2l) (d2l)(d_2l) ReLU – – Linear (d2l→d1l)(d_2l→ d_1l) (d1l)(d_1l) ReLU – – Linear (d1l→n3h0w0)(d_1l→ n_3h_0w_0) (n3h0w0)(n_3h_0w_0) Decoder ConvTranspose2d (n3→n2), 3×3,s=2,p=1,op=1(n_3→ n_2),\;3× 3,\;s=2,\;p=1,\;o_p=1 (n2, 2h0, 2w0)(n_2,\,2h_0,\,2w_0) ReLU – – ConvTranspose2d (n2→n1), 3×3,s=2,p=1,op=1(n_2→ n_1),\;3× 3,\;s=2,\;p=1,\;o_p=1 (n1, 4h0, 4w0)(n_1,\,4h_0,\,4w_0) ReLU – – ConvTranspose2d (n1→c), 3×3,s=2,p=1,op=1(n_1→ c),\;3× 3,\;s=2,\;p=1,\;o_p=1 (c, 8h0, 8w0)(c,\,8h_0,\,8w_0) ReLU – – Conv2d (c→c), 3×3,s=1,p=1(c→ c),\;3× 3,\;s=1,\;p=1 (c, 8h0, 8w0)(c,\,8h_0,\,8w_0) ReLU – – Conv2d (c→c), 3×3,s=1,p=1(c→ c),\;3× 3,\;s=1,\;p=1 (c, 8h0, 8w0)(c,\,8h_0,\,8w_0) Sigmoid – – Table 6: Masked Autoencoder (MAE) architecture. The model first embeds the input image (N,C,H,W)(N,C,H,W) into non‐overlapping patches of size P×P× P, producing L=(HW/P2)L=(HW/P^2) tokens of embedding dimension E, where C is the number of input channels. A learnable [CLS] token and fixed 2D sine–cosine positional embeddings are added. The encoder applies d Transformer blocks with h attention heads and MLP expansion ratio r, yielding latent representations of shape (N,L+1,E)(N,L+1,E). The decoder maps embeddings to dimension DdecD_dec (e.g., 16), adds mask tokens and positional encodings, and processes them with ddecd_dec Transformer blocks with hdech_dec heads. Finally, a projection stage normalizes and linearly predicts patch pixels of size P2CP^2C, which are reassembled into the reconstructed image (N,C,H,W)(N,C,H,W). Stage Module Configuration Output Shape Embed PatchEmbed P=16P\!=\!16, D=ED\!=\!E (e.g. 768/1024/1280), C=3C\!=\!3 (N,L,E)(N,L,E) Add Pos. Embed (+CLS) fixed 2D sin‐cos (N,L+1,E)(N,L\!+\!1,E) Encoder Transformer Blocks (×d× d) heads h, MLP‐ratio r (N,L+1,E)(N,L\!+\!1,E) LayerNorm LayerNorm(E)LayerNorm(E) (N,L+1,E)(N,L\!+\!1,E) Decode Linear Embed E→DdecE→ D_dec (512) (N,L+1,Ddec)(N,L\!+\!1,D_dec) Mask Token & Pos. Embed fixed 2D sin‐cos (N,L+1,Ddec)(N,L\!+\!1,D_dec) Transformer Blocks (×ddec× d_dec) heads hdech_dec, MLP‐ratio r (N,L+1,Ddec)(N,L\!+\!1,D_dec) Proj LayerNorm LayerNorm(Ddec)LayerNorm(D_dec) (N,L+1,Ddec)(N,L\!+\!1,D_dec) Linear Predict Ddec→P2CD_dec→ P^2\!C (N,L+1,P2C)(N,L\!+\!1,P^2C) Remove CLS and Unpatchify — (N,C,H,W)(N,C,H,W) Table 7: Experimental configurations across conditions, datasets, and models. Condition Dataset Model Parameters Convergence Pendulum AE d1=200,d2=2000,l=8d_1=200,\;d_2=2000,\;l=8, λ=1.0λ=1.0 VAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2, λ=1.0λ=1.0 SAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2, λ=10.0λ=10.0 MAE patch size =16=16, depth=16=16, heads =16=16, λ=10.0λ=10.0 Mitosis AE d1=200,d2=2000,l=8d_1=200,\;d_2=2000,\;l=8, λ=1.0λ=1.0 VAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2, λ=1.0λ=1.0 SAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2, λ=20.0λ=20.0 MAE patch size =16=16, depth=16=16, heads=16=16, λ=10.0λ=10.0 Galaxy AE d1=200,d2=2000,l=8d_1=200,\;d_2=2000,\;l=8 VAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2 SAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2 MAE patch size =16=16, depth=16=16, heads =16=16, λ=10.0λ=10.0 Sample Unbalance Pendulum AE d1=200,d2=2000,l=8d_1=200,\;d_2=2000,\;l=8, λ=10.0λ=10.0 VAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2, λ=10.0λ=10.0 SAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2, λ=20.0λ=20.0 MAE patch size =16=16, depth =16=16, heads =16=16, λ=10.0λ=10.0 Mitosis AE d1=200,d2=2000,l=8d_1=200,\;d_2=2000,\;l=8, λ=1.0λ=1.0 VAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2, λ=1.0λ=1.0 SAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2, λ=1.0λ=1.0 MAE patch size =16=16, depth =16=16, heads =16=16, λ=10.0λ=10.0 Galaxy AE d1=200,d2=2000,l=8d_1=200,\;d_2=2000,\;l=8, λ=1.0λ=1.0 VAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2, λ=1.0λ=1.0 SAE n1=8,n2=16,n3=32,d1=16,d2=4,l=2n_1=8,\;n_2=16,\;n_3=32,\;d_1=16,\;d_2=4,\;l=2, λ=1.0λ=1.0 MAE patch size =16=16, depth =16=16, heads =16=16, λ=10.0λ=10.0 0.A.2.1 Specification of Individual Experiments As discussed in the main paper, certain architectures have sufficient capacity to overfit the training set, thereby obscuring meaningful differences between reconstruction loss functions. To mitigate this effect and to more clearly expose the advantages of our approach, we systematically adjust dataset parameters in each experiment. The detailed configurations are reported here to ensure reproducibility and to provide a transparent account of how the evaluation setup emphasizes the strengths and limitations of different methods. The parameter selection can be seen in table 7. 0.A.3 Training details For the training we used a A40 GPU. The optimizer and learning rate was fixed for all of the experiment runs using Adam optimizer, learning rate lr=1.0e−3lr=1.0e^-3 and weight decay wd=1.0e−5wd=1.0e^-5. 0.A.4 Experimental results tables for real data On Tables 8 to 10 we show the quantitative results for the real-world data previously summarized on Fig. 8(a) for the entropy experiments. Similarly, for the sample proagation experiments, the tables 11 to 13 show the statistical results corresponding to Fig. 8(b). Table 8: Pendulum reconstruction: mitigating convergence on repetitive dynamics. We compare loss functions across four autoencoder families (AE, VAE, SAE, MAE) on Pendulum. Rows marked with ∗* are our entropy–based objectives. Our losses consistently deliver the best distortion metrics (lowest MSE, highest PSNR) in every architecture: (i) AE: entropy attains 1.66×10−41.66× 10^-4 MSE / 38.4538.45,dB PSNR / 0.9290.929 SSIM, outperforming the strongest baseline (BEL) by ≈ 2× lower MSE, +3.3+3.3,dB PSNR, and +0.040+0.040 SSIM; (i) VAE: entropy* achieves the best MSE/PSNR (1.02×10−41.02× 10^-4, 40.0040.00,dB) with near–state–of–the–art SSIM (0.942), while the SSIM loss peaks in SSIM (0.964) at the cost of notably worse MSE/PSNR; (i) SAE: spp+entropy* yields the strongest MSE/PSNR (1.12×10−41.12× 10^-4, 39.6939.69,dB), and entropy* attains the best SSIM (0.9480.948); (iv) MAE: entropy* is best on all three metrics (5.93×10−45.93× 10^-4, 32.9032.90,dB, 0.8860.886). The consistent gains—often multi–dB PSNR improvements and order–of–magnitude MSE reductions—indicate that maximizing information content during training prevents collapse on repetitive trajectories and preserves fine, non–repetitive details that pixel/perceptual objectives tend to smooth. Model Loss MSE PSNR SSIM AE L1 1.126e−3±6.37e−41.126e^-3± 6.37e^-4 3.078e1±4.06e03.078e^1± 4.06e^0 8.777e−1±4.85e−28.777e^-1± 4.85e^-2 L2 8.224e−4±3.35e−48.224e^-4± 3.35e^-4 3.127e1±2.01e03.127e^1± 2.01e^0 8.548e−1±3.29e−28.548e^-1± 3.29e^-2 CE 5.145e−3±4.95e−45.145e^-3± 4.95e^-4 2.291e1±4.22e−12.291e^1± 4.22e^-1 7.467e−1±1.33e−27.467e^-1± 1.33e^-2 SSIM 1.588e−3±1.29e−41.588e^-3± 1.29e^-4 2.801e1±3.42e−12.801e^1± 3.42e^-1 8.507e−1±1.04e−28.507e^-1± 1.04e^-2 FFL 8.188e−4±3.45e−48.188e^-4± 3.45e^-4 3.128e1±1.98e03.128e^1± 1.98e^0 8.583e−1±3.38e−28.583e^-1± 3.38e^-2 ldam 6.611e−4±3.57e−46.611e^-4± 3.57e^-4 3.229e1±2.00e03.229e^1± 2.00e^0 8.693e−1±2.17e−28.693e^-1± 2.17e^-2 VGG 1.596e−3±1.14e−41.596e^-3± 1.14e^-4 2.798e1±3.03e−12.798e^1± 3.03e^-1 8.442e−1±1.04e−28.442e^-1± 1.04e^-2 Watson 2.296e−3±1.91e−42.296e^-3± 1.91e^-4 2.640e1±3.59e−12.640e^1± 3.59e^-1 6.934e−1±7.87e−36.934e^-1± 7.87e^-3 BEL 3.359e−4±1.67e−43.359e^-4± 1.67e^-4 3.511e1±1.69e03.511e^1± 1.69e^0 8.890e−1±2.21e−28.890e^-1± 2.21e^-2 entropy* 1.661−±1.27e−4 1.661e^-4± 1.27e^-4 3.845±2.13e0 3.845e^1± 2.13e^0 9.294−±1.62e−2 9.294e^-1± 1.62e^-2 spp+entropy* 9.833e−4±1.16e−49.833e^-4± 1.16e^-4 3.010e1±5.08e−13.010e^1± 5.08e^-1 8.380e−1±1.87e−28.380e^-1± 1.87e^-2 VAE L1 8.197e−4±9.51e−58.197e^-4± 9.51e^-5 3.089e1±4.77e−13.089e^1± 4.77e^-1 8.846e−1±1.18e−28.846e^-1± 1.18e^-2 L2 1.499e−3±1.44e−41.499e^-3± 1.44e^-4 2.826e1±4.13e−12.826e^1± 4.13e^-1 7.985e−1±1.12e−27.985e^-1± 1.12e^-2 CE 6.391e−3±5.84e−46.391e^-3± 5.84e^-4 2.196e1±4.02e−12.196e^1± 4.02e^-1 7.675e−1±1.19e−27.675e^-1± 1.19e^-2 SSIM 2.283e−4±4.36e−52.283e^-4± 4.36e^-5 3.648e1±7.01e−13.648e^1± 7.01e^-1 9.644−±4.44e−3 9.644e^-1± 4.44e^-3 FFL 1.663e−3±9.29e−51.663e^-3± 9.29e^-5 2.780e1±2.40e−12.780e^1± 2.40e^-1 6.879e−1±5.93e−36.879e^-1± 5.93e^-3 ldam 2.725e−3±1.81e−42.725e^-3± 1.81e^-4 2.566e1±2.83e−12.566e^1± 2.83e^-1 8.145e−1±1.01e−28.145e^-1± 1.01e^-2 VGG 4.871e−4±1.72e−44.871e^-4± 1.72e^-4 3.329e1±1.07e03.329e^1± 1.07e^0 9.318e−1±1.57e−29.318e^-1± 1.57e^-2 Watson 2.685e−3±3.11e−42.685e^-3± 3.11e^-4 2.574e1±4.95e−12.574e^1± 4.95e^-1 8.183e−1±1.26e−28.183e^-1± 1.26e^-2 BEL 1.483e−3±1.32e−41.483e^-3± 1.32e^-4 2.831e1±3.80e−12.831e^1± 3.80e^-1 8.102e−1±1.08e−28.102e^-1± 1.08e^-2 entropy* 1.021−±2.47e−5 1.021e^-4± 2.47e^-5 4.000±7.94e−1 4.000e^1± 7.94e^-1 9.419e−1±5.57e−39.419e^-1± 5.57e^-3 spp+entropy* 1.064e−4±2.17e−51.064e^-4± 2.17e^-5 3.979e1±6.72e−13.979e^1± 6.72e^-1 9.466e−1±5.72e−39.466e^-1± 5.72e^-3 SAE L1 1.625e−3±1.08e−41.625e^-3± 1.08e^-4 2.790e1±2.97e−12.790e^1± 2.97e^-1 8.470e−1±8.77e−38.470e^-1± 8.77e^-3 L2 1.499e−3±1.36e−41.499e^-3± 1.36e^-4 2.826e1±3.92e−12.826e^1± 3.92e^-1 8.029e−1±1.11e−28.029e^-1± 1.11e^-2 CE 4.927e−3±4.10e−44.927e^-3± 4.10e^-4 2.309e1±3.63e−12.309e^1± 3.63e^-1 7.851e−1±1.06e−27.851e^-1± 1.06e^-2 SSIM 1.675e−3±1.19e−41.675e^-3± 1.19e^-4 2.777e1±3.03e−12.777e^1± 3.03e^-1 8.439e−1±8.40e−38.439e^-1± 8.40e^-3 FFL 1.659e−3±1.12e−41.659e^-3± 1.12e^-4 2.781e1±2.92e−12.781e^1± 2.92e^-1 6.948e−1±6.48e−36.948e^-1± 6.48e^-3 ldam 1.507e−3±1.46e−41.507e^-3± 1.46e^-4 2.824e1±4.16e−12.824e^1± 4.16e^-1 7.932e−1±1.13e−27.932e^-1± 1.13e^-2 VGG 1.639e−3±1.34e−41.639e^-3± 1.34e^-4 2.787e1±3.40e−12.787e^1± 3.40e^-1 8.445e−1±9.75e−38.445e^-1± 9.75e^-3 Watson 2.831e−3±1.72e−42.831e^-3± 1.72e^-4 2.549e1±2.61e−12.549e^1± 2.61e^-1 8.347e−1±1.01e−28.347e^-1± 1.01e^-2 BEL 1.649e−3±1.26e−41.649e^-3± 1.26e^-4 2.784e1±3.27e−12.784e^1± 3.27e^-1 7.747e−1±1.04e−27.747e^-1± 1.04e^-2 entropy* 1.620e−4±2.74e−51.620e^-4± 2.74e^-5 3.795e1±5.93e−13.795e^1± 5.93e^-1 9.479−±4.83e−3 9.479e^-1± 4.83e^-3 spp+entropy* 1.124−±5.10e−5 1.124e^-4± 5.10e^-5 3.969±1.11e0 3.969e^1± 1.11e^0 9.460e−1±6.58e−39.460e^-1± 6.58e^-3 MAE L1 1.423e−3±2.63e−41.423e^-3± 2.63e^-4 2.856e1±9.13e−12.856e^1± 9.13e^-1 8.423e−1±2.22e−28.423e^-1± 2.22e^-2 L2 1.280e−3±2.41e−41.280e^-3± 2.41e^-4 2.902e1±9.39e−12.902e^1± 9.39e^-1 7.989e−1±3.65e−27.989e^-1± 3.65e^-2 CE 2.896e−1±4.87e−32.896e^-1± 4.87e^-3 5.383e0±7.35e−25.383e^0± 7.35e^-2 1.673e−1±4.33e−21.673e^-1± 4.33e^-2 SSIM 2.807e−3±1.88e−42.807e^-3± 1.88e^-4 2.553e1±2.86e−12.553e^1± 2.86e^-1 8.231e−1±7.16e−38.231e^-1± 7.16e^-3 FFL 1.392e−3±2.07e−41.392e^-3± 2.07e^-4 2.862e1±7.19e−12.862e^1± 7.19e^-1 7.121e−1±6.71e−27.121e^-1± 6.71e^-2 ldam 1.263e−3±2.52e−41.263e^-3± 2.52e^-4 2.909e1±1.00e02.909e^1± 1.00e^0 8.166e−1±3.30e−28.166e^-1± 3.30e^-2 VGG 4.748e−3±3.79e−44.748e^-3± 3.79e^-4 2.325e1±3.46e−12.325e^1± 3.46e^-1 7.622e−1±6.39e−37.622e^-1± 6.39e^-3 Watson 1.730e−1±3.66e−31.730e^-1± 3.66e^-3 7.620e0±9.24e−27.620e^0± 9.24e^-2 5.719e−1±4.70e−35.719e^-1± 4.70e^-3 BEL 7.944e−4±3.79e−47.944e^-4± 3.79e^-4 3.142e1±1.86e03.142e^1± 1.86e^0 8.304e−1±3.44e−28.304e^-1± 3.44e^-2 entropy* 5.928−±3.86− 5.928e^-4± 3.86e^-4 3.290±2.17 3.290e^1± 2.17e^0 8.856−±2.27− 8.856e^-1± 2.27e^-2 spp+entropy* 6.137e−4±3.89e−46.137e^-4± 3.89e^-4 3.271e1±2.09e03.271e^1± 2.09e^0 8.847e−1±1.79e−28.847e^-1± 1.79e^-2 Table 9: Reconstruction performance on the Galaxy dataset. We compare loss functions across four autoencoder families (AE, VAE, SAE, MAE). Rows marked with ∗* are our entropy–based objectives. Our method consistently achieves the best distortion metrics (lowest MSE, highest PSNR) in every architecture, while remaining competitive in SSIM (often within ∼0.01 0.01–0.020.02 of the SSIM-trained baseline). Concretely: (i) AE: entropy attains the best MSE/PSNR (9.57×10−49.57×10^-4, 31.5931.59,dB), improving upon the strongest non-ours baseline by ∼38% 38\% lower MSE and +0.69+0.69,dB PSNR; (i) VAE: spp+entropy* achieves the best MSE/PSNR (8.78×10−48.78×10^-4, 31.7931.79,dB), a ∼9% 9\% MSE reduction and +0.73+0.73,dB over the best baseline; (i) SAE: spp+entropy* yields the best MSE/PSNR (1.30×10−31.30×10^-3, 30.3830.38,dB), a ∼32% 32\% MSE drop and +0.58+0.58,dB; (iv) MAE: entropy* provides the top PSNR (30.0130.01,dB) with lower MSE (1.41×10−31.41×10^-3), improving ∼24% 24\% in MSE and +0.90+0.90,dB. These gains indicate that maximizing information content mitigates convergence to oversmoothed solutions on structured astronomical textures, producing sharper, more faithful reconstructions across architectures. Model Loss MSE PSNR SSIM AE L1 1.301e−3±1.61e−31.301e^-3± 1.61e^-3 3.085e1±3.94e03.085e^1± 3.94e^0 8.359e−1±4.64e−28.359e^-1± 4.64e^-2 L2 1.537e−3±1.45e−31.537e^-3± 1.45e^-3 2.954e1±3.42e02.954e^1± 3.42e^0 8.224e−1±4.92e−28.224e^-1± 4.92e^-2 CE 1.707e−1±1.07e−21.707e^-1± 1.07e^-2 7.686e0±2.70e−17.686e^0± 2.70e^-1 5.258e−1±3.48e−25.258e^-1± 3.48e^-2 SSIM 1.268e−3±1.50e−31.268e^-3± 1.50e^-3 3.090e1±3.91e03.090e^1± 3.91e^0 8.527−±4.53e−2 8.527e^-1± 4.53e^-2 FFL 1.874e−3±2.21e−31.874e^-3± 2.21e^-3 2.900e1±3.62e02.900e^1± 3.62e^0 7.923e−1±6.07e−27.923e^-1± 6.07e^-2 ldam 1.670e−3±1.82e−31.670e^-3± 1.82e^-3 2.936e1±3.53e02.936e^1± 3.53e^0 8.174e−1±5.03e−28.174e^-1± 5.03e^-2 VGG 2.990e−3±5.33e−32.990e^-3± 5.33e^-3 2.717e1±3.57e02.717e^1± 3.57e^0 8.027e−1±4.76e−28.027e^-1± 4.76e^-2 Watson 5.510e−3±9.70e−35.510e^-3± 9.70e^-3 2.388e1±2.75e02.388e^1± 2.75e^0 5.897e−1±8.58e−25.897e^-1± 8.58e^-2 BEL 1.686e−3±1.89e−31.686e^-3± 1.89e^-3 2.935e1±3.55e02.935e^1± 3.55e^0 8.198e−1±5.00e−28.198e^-1± 5.00e^-2 entropy* 9.567−±9.13e−4 9.567e^-4± 9.13e^-4 3.159±3.38e0 3.159e^1± 3.38e^0 8.377e−1±4.34e−28.377e^-1± 4.34e^-2 spp+entropy* 1.048e−3±6.15e−41.048e^-3± 6.15e^-4 3.026e1±1.96e03.026e^1± 1.96e^0 8.137e−1±4.75e−28.137e^-1± 4.75e^-2 VAE L1 1.268e−3±2.92e−31.268e^-3± 2.92e^-3 3.045e1±3.12e03.045e^1± 3.12e^0 8.123e−1±5.24e−28.123e^-1± 5.24e^-2 L2 9.686e−4±2.24e−39.686e^-4± 2.24e^-3 3.106e1±2.15e03.106e^1± 2.15e^0 7.954e−1±5.21e−27.954e^-1± 5.21e^-2 CE 1.684e−1±1.31e−21.684e^-1± 1.31e^-2 7.748e0±3.35e−17.748e^0± 3.35e^-1 6.202e−1±5.77e−26.202e^-1± 5.77e^-2 SSIM 9.784e−4±2.85e−39.784e^-4± 2.85e^-3 3.150e1±2.82e03.150e^1± 2.82e^0 8.294−±4.88e−2 8.294e^-1± 4.88e^-2 FFL 1.283e−3±2.63e−31.283e^-3± 2.63e^-3 2.990e1±2.42e02.990e^1± 2.42e^0 7.656e−1±6.00e−27.656e^-1± 6.00e^-2 ldam 1.004e−3±2.25e−31.004e^-3± 2.25e^-3 3.090e1±2.18e03.090e^1± 2.18e^0 7.891e−1±5.34e−27.891e^-1± 5.34e^-2 VGG 2.412e−3±4.94e−32.412e^-3± 4.94e^-3 2.796e1±3.24e02.796e^1± 3.24e^0 8.022e−1±5.49e−28.022e^-1± 5.49e^-2 Watson 1.886e−3±3.41e−31.886e^-3± 3.41e^-3 2.876e1±3.20e02.876e^1± 3.20e^0 6.720e−1±7.09e−26.720e^-1± 7.09e^-2 BEL 3.127e−2±2.64e−33.127e^-2± 2.64e^-3 1.506e1±3.43e−11.506e^1± 3.43e^-1 7.572e−1±6.00e−27.572e^-1± 6.00e^-2 entropy* 8.951e−4±2.69e−38.951e^-4± 2.69e^-3 3.163e1±2.47e03.163e^1± 2.47e^0 8.196e−1±5.30e−28.196e^-1± 5.30e^-2 spp+entropy* 8.780−±2.71e−3 8.780e^-4± 2.71e^-3 3.179±2.42e0 3.179e^1± 2.42e^0 8.171e−1±5.21e−28.171e^-1± 5.21e^-2 SAE L1 1.913e−3±4.06e−31.913e^-3± 4.06e^-3 2.952e1±3.88e02.952e^1± 3.88e^0 8.069e−1±5.59e−28.069e^-1± 5.59e^-2 L2 2.224e−3±3.66e−32.224e^-3± 3.66e^-3 2.834e1±3.48e02.834e^1± 3.48e^0 7.757e−1±5.85e−27.757e^-1± 5.85e^-2 CE 3.027e−1±1.55e−23.027e^-1± 1.55e^-2 5.198e0±3.24e−15.198e^0± 3.24e^-1 6.933e−1±5.27e−26.933e^-1± 5.27e^-2 SSIM 1.909e−3±4.45e−31.909e^-3± 4.45e^-3 2.980e1±4.04e02.980e^1± 4.04e^0 8.135−±5.56e−2 8.135e^-1± 5.56e^-2 FFL 2.320e−3±3.76e−32.320e^-3± 3.76e^-3 2.807e1±3.36e02.807e^1± 3.36e^0 7.583e−1±6.28e−27.583e^-1± 6.28e^-2 ldam 2.228e−3±3.66e−32.228e^-3± 3.66e^-3 2.833e1±3.48e02.833e^1± 3.48e^0 7.795e−1±5.95e−27.795e^-1± 5.95e^-2 VGG 3.534e−3±6.40e−33.534e^-3± 6.40e^-3 2.620e1±3.34e02.620e^1± 3.34e^0 7.730e−1±5.47e−27.730e^-1± 5.47e^-2 Watson 1.476e−1±9.72e−31.476e^-1± 9.72e^-3 8.324e0±4.09e−18.324e^0± 4.09e^-1 2.644e−1±1.04e−22.644e^-1± 1.04e^-2 BEL 2.848e−3±6.37e−32.848e^-3± 6.37e^-3 2.737e1±3.36e02.737e^1± 3.36e^0 7.799e−1±5.22e−27.799e^-1± 5.22e^-2 entropy* 1.374e−3±2.86e−31.374e^-3± 2.86e^-3 3.025e1±3.34e03.025e^1± 3.34e^0 8.103e−1±5.48e−28.103e^-1± 5.48e^-2 spp+entropy* 1.302−±2.89e−3 1.302e^-3± 2.89e^-3 3.038±3.20e0 3.038e^1± 3.20e^0 8.097e−1±5.40e−28.097e^-1± 5.40e^-2 MAE L1 1.885e−3±3.40e−31.885e^-3± 3.40e^-3 2.911e1±3.54e02.911e^1± 3.54e^0 8.019e−1±5.63e−28.019e^-1± 5.63e^-2 L2 1.907e−3±3.09e−31.907e^-3± 3.09e^-3 2.895e1±3.46e02.895e^1± 3.46e^0 7.969e−1±5.55e−27.969e^-1± 5.55e^-2 CE 3.720e−2±8.35e−33.720e^-2± 8.35e^-3 1.438e1±8.25e−11.438e^1± 8.25e^-1 1.883e−1±2.15e−21.883e^-1± 2.15e^-2 SSIM 1.924e−3±3.51e−31.924e^-3± 3.51e^-3 2.903e1±3.54e02.903e^1± 3.54e^0 8.053−±5.47− 8.053e^-1± 5.47e^-2 FFL 2.070e−3±3.25e−32.070e^-3± 3.25e^-3 2.855e1±3.42e02.855e^1± 3.42e^0 7.869e−1±5.63e−27.869e^-1± 5.63e^-2 ldam 1.850e−3±2.55e−31.850e^-3± 2.55e^-3 2.894e1±3.38e02.894e^1± 3.38e^0 7.950e−1±5.60e−27.950e^-1± 5.60e^-2 VGG 6.195e−3±7.06e−36.195e^-3± 7.06e^-3 2.302e1±2.54e02.302e^1± 2.54e^0 1.120e−1±4.62e−31.120e^-1± 4.62e^-3 Watson 7.818e−3±7.99e−37.818e^-3± 7.99e^-3 2.185e1±2.30e02.185e^1± 2.30e^0 5.033e−2±5.19e−35.033e^-2± 5.19e^-3 BEL 1.851e−3±2.61e−31.851e^-3± 2.61e^-3 2.896e1±3.41e02.896e^1± 3.41e^0 7.976e−1±5.54e−27.976e^-1± 5.54e^-2 entropy* 1.409e−3±1.52e−31.409e^-3± 1.52e^-3 3.001±3.41 3.001e^1± 3.41e^0 8.049e−1±5.38e−28.049e^-1± 5.38e^-2 spp+entropy* 1.402−±1.51− 1.402e^-3± 1.51e^-3 2.998e1±3.35e02.998e^1± 3.35e^0 8.053e−1±5.34e−28.053e^-1± 5.34e^-2 Table 10: Reconstruction performance on the Mitosis dataset. We compare loss functions across four autoencoder families (AE, VAE, SAE, MAE); rows marked with ∗* are our entropy–based objectives. AE: entropy attains the best scores on all metrics (6.65×10−46.65×10^-4 MSE, 32.1032.10,dB PSNR, 0.9180.918 SSIM), improving over the strongest non-ours baseline (ldam) by ∼38% 38\% lower MSE, +2.17+2.17,dB PSNR, and +0.047+0.047 SSIM. VAE: our spp+entropy* achieves the best distortion (1.05×10−31.05×10^-3 MSE, 29.9529.95,dB), edging the best baseline by ∼4.6% 4.6\% MSE and +0.16+0.16,dB, while the SSIM-trained loss unsurprisingly peaks in SSIM (0.9090.909). SAE: entropy* is best on all three metrics (3.01×10−33.01×10^-3, 25.4925.49,dB, 0.7400.740), yielding ∼36% 36\% lower MSE, +1.85+1.85,dB PSNR, and +0.044+0.044 SSIM vs. the top baseline. MAE: spp+entropy* leads across metrics (3.88×10−33.88×10^-3, 24.3024.30,dB, 0.6300.630), a ∼29% 29\% MSE drop and +1.56+1.56,dB PSNR over the best baseline with a +0.020+0.020 SSIM gain. Overall, maximizing information content mitigates oversmoothing and better preserves fine cellular structures, delivering consistently lower distortion and competitive–to–superior perceptual quality across architectures. Model Loss MSE PSNR SSIM AE L1 1.546e−3±7.46e−41.546e^-3± 7.46e^-4 2.848e1±1.72e02.848e^1± 1.72e^0 8.588e−1±4.87e−28.588e^-1± 4.87e^-2 L2 1.366e−3±4.27e−41.366e^-3± 4.27e^-4 2.883e1±1.25e02.883e^1± 1.25e^0 8.449e−1±2.80e−28.449e^-1± 2.80e^-2 CE 6.967e−3±1.25e−36.967e^-3± 1.25e^-3 2.163e1±7.03e−12.163e^1± 7.03e^-1 8.927e−2±1.25e−28.927e^-2± 1.25e^-2 SSIM 2.744e−3±1.43e−32.744e^-3± 1.43e^-3 2.626e1±2.48e02.626e^1± 2.48e^0 7.933e−1±8.16e−27.933e^-1± 8.16e^-2 FFL 1.572e−3±5.27e−41.572e^-3± 5.27e^-4 2.823e1±1.25e02.823e^1± 1.25e^0 8.128e−1±3.71e−28.128e^-1± 3.71e^-2 ldam 1.073e−3±4.05e−41.073e^-3± 4.05e^-4 2.993e1±1.38e02.993e^1± 1.38e^0 8.711e−1±2.81e−28.711e^-1± 2.81e^-2 VGG 2.410e−3±1.19e−32.410e^-3± 1.19e^-3 2.673e1±2.23e02.673e^1± 2.23e^0 8.502e−1±5.56e−28.502e^-1± 5.56e^-2 Watson 6.681e−3±1.39e−36.681e^-3± 1.39e^-3 2.183e1±8.25e−12.183e^1± 8.25e^-1 4.361e−1±6.97e−24.361e^-1± 6.97e^-2 BEL 2.422e−3±7.46e−42.422e^-3± 7.46e^-4 2.636e1±1.33e02.636e^1± 1.33e^0 7.628e−1±4.73e−27.628e^-1± 4.73e^-2 entropy* 6.654−±3.09e−4 6.654e^-4± 3.09e^-4 3.210±1.60e0 3.210e^1± 1.60e^0 9.178−±1.99e−2 9.178e^-1± 1.99e^-2 spp+entropy* 1.478e−3±1.09e−41.478e^-3± 1.09e^-4 2.831e1±2.84e−12.831e^1± 2.84e^-1 8.420e−1±2.65e−28.420e^-1± 2.65e^-2 VAE L1 6.592e−3±1.63e−36.592e^-3± 1.63e^-3 2.192e1±9.59e−12.192e^1± 9.59e^-1 5.223e−1±6.81e−25.223e^-1± 6.81e^-2 L2 6.319e−3±1.67e−36.319e^-3± 1.67e^-3 2.212e1±1.03e02.212e^1± 1.03e^0 4.992e−1±4.92e−24.992e^-1± 4.92e^-2 CE 7.555e−3±1.43e−37.555e^-3± 1.43e^-3 2.128e1±7.32e−12.128e^1± 7.32e^-1 5.300e−1±6.07e−25.300e^-1± 6.07e^-2 SSIM 1.096e−3±3.37e−41.096e^-3± 3.37e^-4 2.979e1±1.26e02.979e^1± 1.26e^0 9.094−±1.59e−2 9.094e^-1± 1.59e^-2 FFL 6.396e−3±1.59e−36.396e^-3± 1.59e^-3 2.206e1±9.67e−12.206e^1± 9.67e^-1 4.542e−1±4.10e−24.542e^-1± 4.10e^-2 ldam 6.320e−3±1.61e−36.320e^-3± 1.61e^-3 2.211e1±9.91e−12.211e^1± 9.91e^-1 4.898e−1±4.42e−24.898e^-1± 4.42e^-2 VGG 3.774e−3±1.46e−33.774e^-3± 1.46e^-3 2.456e1±1.69e02.456e^1± 1.69e^0 7.807e−1±4.72e−27.807e^-1± 4.72e^-2 Watson 8.002e−3±1.72e−38.002e^-3± 1.72e^-3 2.105e1±8.30e−12.105e^1± 8.30e^-1 4.519e−1±7.21e−24.519e^-1± 7.21e^-2 BEL 3.611e−3±1.15e−33.611e^-3± 1.15e^-3 2.462e1±1.28e02.462e^1± 1.28e^0 6.562e−1±5.03e−26.562e^-1± 5.03e^-2 entropy* 1.054e−3±2.92e−41.054e^-3± 2.92e^-4 2.992e1±1.11e02.992e^1± 1.11e^0 8.798e−1±1.74e−28.798e^-1± 1.74e^-2 spp+entropy* 1.046−±2.92e−4 1.046e^-3± 2.92e^-4 2.995±1.12e0 2.995e^1± 1.12e^0 8.736e−1±1.95e−28.736e^-1± 1.95e^-2 SAE L1 7.381e−3±1.43e−37.381e^-3± 1.43e^-3 2.139e1±7.52e−12.139e^1± 7.52e^-1 5.305e−1±6.05e−25.305e^-1± 6.05e^-2 L2 6.312e−3±1.62e−36.312e^-3± 1.62e^-3 2.212e1±9.97e−12.212e^1± 9.97e^-1 5.021e−1±4.73e−25.021e^-1± 4.73e^-2 CE 7.345e−3±1.43e−37.345e^-3± 1.43e^-3 2.141e1±7.53e−12.141e^1± 7.53e^-1 5.305e−1±6.05e−25.305e^-1± 6.05e^-2 SSIM 4.709e−3±1.91e−34.709e^-3± 1.91e^-3 2.364e1±1.82e02.364e^1± 1.82e^0 6.965e−1±7.94e−26.965e^-1± 7.94e^-2 FFL 6.385e−3±1.58e−36.385e^-3± 1.58e^-3 2.206e1±9.63e−12.206e^1± 9.63e^-1 4.604e−1±4.16e−24.604e^-1± 4.16e^-2 ldam 6.341e−3±1.58e−36.341e^-3± 1.58e^-3 2.209e1±9.64e−12.209e^1± 9.64e^-1 4.682e−1±3.98e−24.682e^-1± 3.98e^-2 VGG 7.207e−3±1.48e−37.207e^-3± 1.48e^-3 2.150e1±7.92e−12.150e^1± 7.92e^-1 4.167e−1±4.68e−24.167e^-1± 4.68e^-2 Watson 7.676e−3±1.42e−37.676e^-3± 1.42e^-3 2.121e1±7.16e−12.121e^1± 7.16e^-1 3.781e−1±3.80e−23.781e^-1± 3.80e^-2 BEL 6.295e−3±1.62e−36.295e^-3± 1.62e^-3 2.213e1±1.00e02.213e^1± 1.00e^0 5.029e−1±4.63e−25.029e^-1± 4.63e^-2 entropy* 3.014−±1.12− 3.014e^-3± 1.12e^-3 2.549±1.54 2.549e^1± 1.54e^0 7.403−±4.58− 7.403e^-1± 4.58e^-2 spp+entropy* 3.925e−3±1.43e−33.925e^-3± 1.43e^-3 2.435e1±1.59e02.435e^1± 1.59e^0 6.840e−1±6.81e−26.840e^-1± 6.81e^-2 MAE L1 5.527e−3±1.21e−35.527e^-3± 1.21e^-3 2.268e1±9.35e−12.268e^1± 9.35e^-1 5.553e−1±6.15e−25.553e^-1± 6.15e^-2 L2 5.546e−3±1.25e−35.546e^-3± 1.25e^-3 2.266e1±9.22e−12.266e^1± 9.22e^-1 5.165e−1±5.30e−25.165e^-1± 5.30e^-2 CE 2.433e−1±2.11e−32.433e^-1± 2.11e^-3 6.139e0±3.74e−26.139e^0± 3.74e^-2 7.364e−2±5.77e−37.364e^-2± 5.77e^-3 SSIM 5.492e−3±1.40e−35.492e^-3± 1.40e^-3 2.274e1±1.08e02.274e^1± 1.08e^0 6.095e−1±6.05e−26.095e^-1± 6.05e^-2 FFL 5.721e−3±1.31e−35.721e^-3± 1.31e^-3 2.253e1±9.30e−12.253e^1± 9.30e^-1 5.097e−1±5.43e−25.097e^-1± 5.43e^-2 ldam 5.492e−3±1.25e−35.492e^-3± 1.25e^-3 2.271e1±9.29e−12.271e^1± 9.29e^-1 5.319e−1±5.13e−25.319e^-1± 5.13e^-2 VGG 7.683e−3±1.44e−37.683e^-3± 1.44e^-3 2.121e1±7.30e−12.121e^1± 7.30e^-1 6.523e−2±4.06e−36.523e^-2± 4.06e^-3 Watson 7.942e−3±1.53e−37.942e^-3± 1.53e^-3 2.107e1±7.47e−12.107e^1± 7.47e^-1 3.009e−1±5.31e−23.009e^-1± 5.31e^-2 BEL 5.272e−3±1.24e−35.272e^-3± 1.24e^-3 2.290e1±1.02e02.290e^1± 1.02e^0 5.222e−1±6.36e−25.222e^-1± 6.36e^-2 entropy* 3.897e−3±1.07e−33.897e^-3± 1.07e^-3 2.425e1±1.18e02.425e^1± 1.18e^0 6.254e−1±5.43e−26.254e^-1± 5.43e^-2 spp+entropy* 3.883−±1.15− 3.883e^-3± 1.15e^-3 2.430±1.27 2.430e^1± 1.27e^0 6.298−±6.06− 6.298e^-1± 6.06e^-2 Table 11: Pendulum: effect of underrepresented-sample focusing on accuracy and robustness. We compare losses across four architectures (AE, VAE, SAE, MAE). Rows with ∗* denote our sppkk objective that prioritizes underrepresented samples via a memory of size k. Considering both mean and variability (mean± ), spp markedly improves average reconstruction on three architectures while keeping variance acceptable: VAE—spp2 attains the best distortion and perceptual quality (MSE↓ , PSNR↑ , SSIM↑ ) with low–moderate spread; SAE—spp2* achieves the strongest means across all metrics, with spp8* slightly trading peak PSNR for reduced dispersion; MAE—spp2* leads on all metrics and maintains tight standard deviations. For AE, focal loss offers the best means and small variance, while spp4*/spp8* remain competitive in SSIM but do not surpass focal on averages. Across models, increasing memory (spp2→ 4→ 8) generally smooths variance at a modest cost in mean performance, indicating a controllable mean–robustness trade-off. Overall, underrepresented-sample focusing improves average reconstruction quality where data imbalance is most impactful (VAE/SAE/MAE) and provides a tunable handle on stability via the standard deviation. Model Baseline MSE PSNR SSIM AE L2 1.292e−3±3.23e−41.292e^-3± 3.23e^-4 2.905e1±1.27e02.905e^1± 1.27e^0 8.129e−1±3.71e−28.129e^-1± 3.71e^-2 Focal 3.477e−4±1.56e−43.477e^-4± 1.56e^-4 3.489e1±1.54e03.489e^1± 1.54e^0 9.131e−1±1.61e−29.131e^-1± 1.61e^-2 onbs 2.533e−3±7.90e−42.533e^-3± 7.90e^-4 2.636e1±2.22e02.636e^1± 2.22e^0 7.679e−1±3.89e−27.679e^-1± 3.89e^-2 dcl 1.305e−3±3.71e−41.305e^-3± 3.71e^-4 2.904e1±1.34e02.904e^1± 1.34e^0 8.083e−1±3.63e−28.083e^-1± 3.63e^-2 svp 1.009e−3±3.70e−41.009e^-3± 3.70e^-4 3.036e1±2.06e03.036e^1± 2.06e^0 8.315e−1±4.12e−28.315e^-1± 4.12e^-2 spp2* 1.563−±1.49− 1.563e^-3± 1.49e^-4 2.808e1±4.26e−12.808e^1± 4.26e^-1 7.932e−1±1.77e−27.932e^-1± 1.77e^-2 spp4* 1.041e−3±5.05e−41.041e^-3± 5.05e^-4 3.053e1±2.77e03.053e^1± 2.77e^0 8.368−±4.85− 8.368e^-1± 4.85e^-2 spp8* 1.147e−3±4.96e−41.147e^-3± 4.96e^-4 3.012±2.94 3.012e^1± 2.94e^0 8.429e−1±4.67e−28.429e^-1± 4.67e^-2 VAE L2 1.581e−3±9.09e−51.581e^-3± 9.09e^-5 2.802e1±2.48e−12.802e^1± 2.48e^-1 7.832e−1±9.23e−37.832e^-1± 9.23e^-3 Focal 1.524e−3±9.91e−51.524e^-3± 9.91e^-5 2.818e1±2.83e−12.818e^1± 2.83e^-1 7.953e−1±1.08e−27.953e^-1± 1.08e^-2 onbs 2.321e−3±3.76e−42.321e^-3± 3.76e^-4 2.642e1±8.52e−12.642e^1± 8.52e^-1 6.651−±3.38− 6.651e^-1± 3.38e^-2 dcl 1.685e−3±2.03e−41.685e^-3± 2.03e^-4 2.776e1±5.08e−12.776e^1± 5.08e^-1 7.191e−1±2.20e−27.191e^-1± 2.20e^-2 svp 2.744e−3±1.83e−42.744e^-3± 1.83e^-4 2.562e1±2.84e−12.562e^1± 2.84e^-1 8.143e−1±1.00e−28.143e^-1± 1.00e^-2 spp2* 1.806−±8.21− 1.806e^-4± 8.21e^-5 3.772±1.44 3.772e^1± 1.44e^0 9.310e−1±9.63e−39.310e^-1± 9.63e^-3 spp4* 2.754e−3±1.71e−42.754e^-3± 1.71e^-4 2.561e1±2.67e−12.561e^1± 2.67e^-1 8.122e−1±1.02e−28.122e^-1± 1.02e^-2 spp8* 6.442e−4±1.15e−46.442e^-4± 1.15e^-4 3.196e1±6.54e−13.196e^1± 6.54e^-1 8.583e−1±1.13e−28.583e^-1± 1.13e^-2 SAE L2 1.594e−3±9.58e−51.594e^-3± 9.58e^-5 2.798e1±2.58e−12.798e^1± 2.58e^-1 7.757e−1±1.02e−27.757e^-1± 1.02e^-2 Focal 1.587−±8.24− 1.587e^-3± 8.24e^-5 2.800e1±2.24e−12.800e^1± 2.24e^-1 7.902e−1±8.33e−37.902e^-1± 8.33e^-3 onbs 2.540e−3±4.02e−42.540e^-3± 4.02e^-4 2.602e1±8.46e−12.602e^1± 8.46e^-1 5.996−±2.71− 5.996e^-1± 2.71e^-2 dcl 1.691e−3±1.68e−41.691e^-3± 1.68e^-4 2.774e1±4.20e−12.774e^1± 4.20e^-1 7.413e−1±1.94e−27.413e^-1± 1.94e^-2 svp 1.529e−3±1.13e−41.529e^-3± 1.13e^-4 2.817e1±3.18e−12.817e^1± 3.18e^-1 7.939e−1±1.17e−27.939e^-1± 1.17e^-2 spp2* 1.537e−4±1.79e−41.537e^-4± 1.79e^-4 3.906±2.39 3.906e^1± 2.39e^0 9.420e−1±1.44e−29.420e^-1± 1.44e^-2 spp4* 7.563e−4±2.40e−47.563e^-4± 2.40e^-4 3.138e1±1.11e03.138e^1± 1.11e^0 8.656e−1±2.44e−28.656e^-1± 2.44e^-2 spp8* 3.080e−4±1.49e−43.080e^-4± 1.49e^-4 3.545e1±1.57e03.545e^1± 1.57e^0 9.253e−1±1.49e−29.253e^-1± 1.49e^-2 MAE L2 1.271e−3±2.46e−41.271e^-3± 2.46e^-4 2.906e1±9.62e−12.906e^1± 9.62e^-1 8.181e−1±3.08e−28.181e^-1± 3.08e^-2 Focal 1.283e−3±2.35e−41.283e^-3± 2.35e^-4 2.900e1±9.07e−12.900e^1± 9.07e^-1 7.830e−1±3.85e−27.830e^-1± 3.85e^-2 onbs 2.102e−3±3.21e−42.102e^-3± 3.21e^-4 2.683e1±7.05e−12.683e^1± 7.05e^-1 4.907e−1±1.38e−24.907e^-1± 1.38e^-2 dcl 1.331e−3±3.09e−41.331e^-3± 3.09e^-4 2.889e1±1.12e02.889e^1± 1.12e^0 7.887e−1±3.80e−27.887e^-1± 3.80e^-2 svp 1.294e−3±2.32e−41.294e^-3± 2.32e^-4 2.896e1±8.92e−12.896e^1± 8.92e^-1 7.892e−1±3.82e−27.892e^-1± 3.82e^-2 spp2* 5.246−±3.28− 5.246e^-4± 3.28e^-4 3.335±2.00 3.335e^1± 2.00e^0 8.896−±1.53− 8.896e^-1± 1.53e^-2 spp4* 6.862e−4±4.47e−46.862e^-4± 4.47e^-4 3.227e1±2.18e03.227e^1± 2.18e^0 8.837e−1±1.85e−28.837e^-1± 1.85e^-2 spp8* 5.593e−4±3.35e−45.593e^-4± 3.35e^-4 3.299e1±1.81e03.299e^1± 1.81e^0 8.823e−1±1.77e−28.823e^-1± 1.77e^-2 Table 12: Mitosis: underrepresented-sample focusing improves averages with controllable variance. We compare losses across four architectures (AE, VAE, SAE, MAE); rows with ∗* denote our sppkk objective (memory size k). Considering mean± , AE benefits most from larger memory: spp8 attains the best MSE/PSNR/SSIM means, with a small increase in dispersion—indicating sharper reconstructions at slightly higher variability. VAE shows large gains with spp8*/spp4*, which substantially improve all three metrics while reducing MSE variability versus pixel/perceptual baselines, suggesting more stable training on rare cell patterns. For SAE, spp4* offers the strongest averages across metrics but with higher SSIM spread; spp8* trades a bit of mean performance for tighter variance, exposing a tunable accuracy–robustness frontier. MAE mirrors this behavior: spp4* yields the best means with standard deviations comparable to L2/Focal. Overall, prioritizing underrepresented samples consistently improves average reconstruction (MSE↓ , PSNR/SSIM↑ ) and provides a controllable handle on robustness via the standard deviation, especially effective for VAE/MAE and with memory size governing the mean–variance trade-off. Model Baseline MSE PSNR SSIM AE L2 6.270e−3±1.54e−36.270e^-3± 1.54e^-3 2.214e1±9.58e−12.214e^1± 9.58e^-1 4.983e−1±3.89e−24.983e^-1± 3.89e^-2 Focal 6.297e−3±1.52e−36.297e^-3± 1.52e^-3 2.212e1±9.51e−12.212e^1± 9.51e^-1 4.966e−1±4.04e−24.966e^-1± 4.04e^-2 onbs 7.454e−3±1.58e−37.454e^-3± 1.58e^-3 2.136e1±8.30e−12.136e^1± 8.30e^-1 9.822e−2±3.45e−29.822e^-2± 3.45e^-2 dcl 6.309e−3±1.46e−36.309e^-3± 1.46e^-3 2.210e1±8.94e−12.210e^1± 8.94e^-1 4.901e−1±3.15e−24.901e^-1± 3.15e^-2 svp 6.303e−3±1.45e−36.303e^-3± 1.45e^-3 2.210e1±8.91e−12.210e^1± 8.91e^-1 4.912e−1±3.28e−24.912e^-1± 3.28e^-2 spp2* 6.414e−3±1.56e−36.414e^-3± 1.56e^-3 2.204e1±9.48e−12.204e^1± 9.48e^-1 5.136e−1±4.92e−25.136e^-1± 4.92e^-2 spp4* 6.420e−3±1.51e−36.420e^-3± 1.51e^-3 2.203e1±9.19e−12.203e^1± 9.19e^-1 5.119e−1±4.49e−25.119e^-1± 4.49e^-2 spp8* 6.054−±1.59− 6.054e^-3± 1.59e^-3 2.232±1.11 2.232e^1± 1.11e^0 5.291−±5.71− 5.291e^-1± 5.71e^-2 ConvVAE L2 6.328e−3±1.50e−36.328e^-3± 1.50e^-3 2.209e1±9.34e−12.209e^1± 9.34e^-1 4.783e−1±3.57e−24.783e^-1± 3.57e^-2 Focal 6.363e−3±1.48e−36.363e^-3± 1.48e^-3 2.207e1±9.15e−12.207e^1± 9.15e^-1 4.647e−1±3.43e−24.647e^-1± 3.43e^-2 onbs 7.342e−3±1.47e−37.342e^-3± 1.47e^-3 2.142e1±7.75e−12.142e^1± 7.75e^-1 3.648e−1±5.29e−23.648e^-1± 5.29e^-2 dcl 6.350e−3±1.53e−36.350e^-3± 1.53e^-3 2.208e1±9.46e−12.208e^1± 9.46e^-1 4.784e−1±3.76e−24.784e^-1± 3.76e^-2 svp 6.413e−3±1.45e−36.413e^-3± 1.45e^-3 2.203e1±8.89e−12.203e^1± 8.89e^-1 4.458e−1±2.99e−24.458e^-1± 2.99e^-2 spp2* 2.085e−3±5.17e−42.085e^-3± 5.17e^-4 2.693e1±9.98e−12.693e^1± 9.98e^-1 7.855e−1±2.38e−27.855e^-1± 2.38e^-2 spp4* 1.762−±5.11− 1.762e^-3± 5.11e^-4 2.771±1.20 2.771e^1± 1.20e^0 8.045e−1±2.79e−28.045e^-1± 2.79e^-2 spp8* 1.800e−3±4.83e−41.800e^-3± 4.83e^-4 2.759e1±1.07e02.759e^1± 1.07e^0 8.048−±2.28− 8.048e^-1± 2.28e^-2 SAE L2 6.397e−3±1.35e−36.397e^-3± 1.35e^-3 2.203e1±8.47e−12.203e^1± 8.47e^-1 4.535e−1±2.89e−24.535e^-1± 2.89e^-2 Focal 6.454e−3±1.30e−36.454e^-3± 1.30e^-3 2.198e1±8.19e−12.198e^1± 8.19e^-1 4.302e−1±3.18e−24.302e^-1± 3.18e^-2 onbs 7.276e−3±1.44e−37.276e^-3± 1.44e^-3 2.145e1±7.63e−12.145e^1± 7.63e^-1 3.984e−1±4.46e−23.984e^-1± 4.46e^-2 dcl 6.466e−3±1.40e−36.466e^-3± 1.40e^-3 2.199e1±8.72e−12.199e^1± 8.72e^-1 4.381e−1±3.05e−24.381e^-1± 3.05e^-2 svp 6.432e−3±1.31e−36.432e^-3± 1.31e^-3 2.200e1±8.18e−12.200e^1± 8.18e^-1 4.268e−1±2.62e−24.268e^-1± 2.62e^-2 spp2* 6.474e−3±1.35e−36.474e^-3± 1.35e^-3 2.197e1±8.28e−12.197e^1± 8.28e^-1 4.901e−1±3.07e−24.901e^-1± 3.07e^-2 spp4* 4.715−±1.37− 4.715e^-3± 1.37e^-3 2.348±1.42 2.348e^1± 1.42e^0 6.343−±7.53− 6.343e^-1± 7.53e^-2 spp8* 6.007e−3±1.05e−36.007e^-3± 1.05e^-3 2.228e1±7.26e−12.228e^1± 7.26e^-1 5.137e−1±5.08e−25.137e^-1± 5.08e^-2 MAE L2 5.508e−3±1.25e−35.508e^-3± 1.25e^-3 2.269e1±9.20e−12.269e^1± 9.20e^-1 5.370e−1±5.20e−25.370e^-1± 5.20e^-2 Focal 5.528e−3±1.23e−35.528e^-3± 1.23e^-3 2.267e1±9.21e−12.267e^1± 9.21e^-1 5.325e−1±5.34e−25.325e^-1± 5.34e^-2 onbs 7.342e−3±1.63e−37.342e^-3± 1.63e^-3 2.144e1±8.87e−12.144e^1± 8.87e^-1 3.954e−1±5.36e−23.954e^-1± 5.36e^-2 dcl 5.661e−3±1.37e−35.661e^-3± 1.37e^-3 2.259e1±9.83e−12.259e^1± 9.83e^-1 5.087e−1±5.72e−25.087e^-1± 5.72e^-2 svp 5.616e−3±1.28e−35.616e^-3± 1.28e^-3 2.261e1±9.26e−12.261e^1± 9.26e^-1 5.145e−1±5.64e−25.145e^-1± 5.64e^-2 spp2* 3.850e−3±1.05e−33.850e^-3± 1.05e^-3 2.430e1±1.16e02.430e^1± 1.16e^0 6.284e−1±5.22e−26.284e^-1± 5.22e^-2 spp4* 3.765−±1.09− 3.765e^-3± 1.09e^-3 2.442±1.23 2.442e^1± 1.23e^0 6.355−±5.62− 6.355e^-1± 5.62e^-2 spp8* 3.799e−3±1.04e−33.799e^-3± 1.04e^-3 2.436e1±1.16e02.436e^1± 1.16e^0 6.327e−1±5.38e−26.327e^-1± 5.38e^-2 Table 13: Galaxy: underrepresented-sample focusing improves accuracy and stabilizes training. We compare losses across four architectures (AE, VAE, SAE, MAE); rows with ∗* denote our sppkk objective (memory size k). Considering mean± , AE sees consistent gains with markedly lower variability: spp2 attains the best MSE/PSNR/SSIM among AEs (9.19×10−49.19×10^-4, 31.7131.71,dB, 0.8290.829) with substantially smaller standard deviations than pixel/perceptual baselines. VAE benefits from larger memory: spp8* yields the best MSE/PSNR (8.13×10−48.13×10^-4, 31.9131.91,dB) and competitive SSIM (0.816), with variance comparable to or lower than L2/Focal. SAE: spp8* provides the strongest triplet (1.22×10−31.22×10^-3, 30.7030.70,dB, 0.8180.818) while reducing spread relative to baselines. MAE: spp8* achieves the top PSNR/SSIM (30.0230.02,dB, 0.8050.805), whereas spp4* attains the lowest MSE ( 1.40×10−31.40×10^-3 ) with nearly identical variance—illustrating a controllable trade-off between mean accuracy and dispersion via memory size. Overall, prioritizing underrepresented samples improves average reconstruction (MSE↓ , PSNR/SSIM↑ ) while keeping or lowering standard deviations, indicating better robustness across galaxy textures. Model Baseline MSE PSNR SSIM AE L2 1.465e−3±1.23e−41.465e^-3± 1.23e^-4 2.836e1±3.62e−12.836e^1± 3.62e^-1 8.033e−1±1.38e−28.033e^-1± 1.38e^-2 Focal 1.449e−3±1.03e−41.449e^-3± 1.03e^-4 2.840e1±3.06e−12.840e^1± 3.06e^-1 8.087e−1±1.11e−28.087e^-1± 1.11e^-2 onbs 2.476e−3±7.77e−42.476e^-3± 7.77e^-4 2.645e1±2.16e02.645e^1± 2.16e^0 7.642e−1±3.97e−27.642e^-1± 3.97e^-2 dcl 1.470e−3±1.50e−41.470e^-3± 1.50e^-4 2.835e1±4.46e−12.835e^1± 4.46e^-1 7.841e−1±1.52e−27.841e^-1± 1.52e^-2 svp 1.460e−3±1.30e−41.460e^-3± 1.30e^-4 2.837e1±3.85e−12.837e^1± 3.85e^-1 8.055e−1±1.42e−28.055e^-1± 1.42e^-2 spp2* 1.464e−3±3.01e−41.464e^-3± 3.01e^-4 2.851e1±1.38e02.851e^1± 1.38e^0 8.267e−1±2.24e−28.267e^-1± 2.24e^-2 spp4* 2.620−±2.81− 2.620e^-4± 2.81e^-4 3.706±2.90 3.706e^1± 2.90e^0 9.240−±3.00− 9.240e^-1± 3.00e^-2 spp8* 6.298e−4±7.51e−46.298e^-4± 7.51e^-4 3.455e1±4.52e03.455e^1± 4.52e^0 9.110e−1±4.51e−29.110e^-1± 4.51e^-2 VAE L2 1.224e−3±2.12e−31.224e^-3± 2.12e^-3 3.019e1±2.70e03.019e^1± 2.70e^0 7.931e−1±5.30e−27.931e^-1± 5.30e^-2 Focal 1.468e−3±2.13e−31.468e^-3± 2.13e^-3 2.928e1±2.58e02.928e^1± 2.58e^0 7.664e−1±5.47e−27.664e^-1± 5.47e^-2 onbs 3.638e−3±7.04e−33.638e^-3± 7.04e^-3 2.598e1±3.01e02.598e^1± 3.01e^0 7.068e−1±4.80e−27.068e^-1± 4.80e^-2 dcl 2.238e−3±4.07e−32.238e^-3± 4.07e^-3 2.851e1±3.61e02.851e^1± 3.61e^0 7.886e−1±5.62e−27.886e^-1± 5.62e^-2 svp 1.165e−3±2.06e−31.165e^-3± 2.06e^-3 3.025e1±2.41e03.025e^1± 2.41e^0 7.840e−1±5.44e−27.840e^-1± 5.44e^-2 spp2* 9.785e−4±2.34e−39.785e^-4± 2.34e^-3 3.116e1±2.57e03.116e^1± 2.57e^0 8.114e−1±5.14e−28.114e^-1± 5.14e^-2 spp4* 8.682e−4±2.35e−38.682e^-4± 2.35e^-3 3.166e1±2.44e03.166e^1± 2.44e^0 8.163−±4.89− 8.163e^-1± 4.89e^-2 spp8* 8.126−±1.86− 8.126e^-4± 1.86e^-3 3.191±2.45 3.191e^1± 2.45e^0 8.156e−1±4.93e−28.156e^-1± 4.93e^-2 SAE L2 2.179e−3±3.65e−32.179e^-3± 3.65e^-3 2.853e1±3.58e02.853e^1± 3.58e^0 7.945e−1±5.60e−27.945e^-1± 5.60e^-2 Focal 3.497e−3±5.38e−33.497e^-3± 5.38e^-3 2.557e1±2.31e02.557e^1± 2.31e^0 7.822e−1±5.13e−27.822e^-1± 5.13e^-2 onbs 3.353e−3±6.55e−33.353e^-3± 6.55e^-3 2.637e1±3.03e02.637e^1± 3.03e^0 7.151e−1±4.49e−27.151e^-1± 4.49e^-2 dcl 2.884e−3±6.40e−32.884e^-3± 6.40e^-3 2.751e1±3.57e02.751e^1± 3.57e^0 7.923e−1±5.26e−27.923e^-1± 5.26e^-2 svp 2.199e−3±3.72e−32.199e^-3± 3.72e^-3 2.851e1±3.58e02.851e^1± 3.58e^0 7.935e−1±5.54e−27.935e^-1± 5.54e^-2 spp2* 6.160e−3±6.95e−36.160e^-3± 6.95e^-3 2.302e1±2.50e02.302e^1± 2.50e^0 7.397e−1±5.73e−27.397e^-1± 5.73e^-2 spp4* 1.416e−3±2.71e−31.416e^-3± 2.71e^-3 3.022e1±3.50e03.022e^1± 3.50e^0 8.151e−1±5.22e−28.151e^-1± 5.22e^-2 spp8* 1.220−±2.65− 1.220e^-3± 2.65e^-3 3.070±3.27 3.070e^1± 3.27e^0 8.185−±5.11− 8.185e^-1± 5.11e^-2 MAE L2 1.868e−3±2.62e−31.868e^-3± 2.62e^-3 2.892e1±3.38e02.892e^1± 3.38e^0 7.954e−1±5.64e−27.954e^-1± 5.64e^-2 Focal 2.401e−3±2.52e−32.401e^-3± 2.52e^-3 2.727e1±2.72e02.727e^1± 2.72e^0 7.503e−1±5.03e−27.503e^-1± 5.03e^-2 onbs 3.060e−3±6.32e−33.060e^-3± 6.32e^-3 2.692e1±3.20e02.692e^1± 3.20e^0 7.618e−1±5.05e−27.618e^-1± 5.05e^-2 dcl 2.123e−3±3.81e−32.123e^-3± 3.81e^-3 2.864e1±3.55e02.864e^1± 3.55e^0 7.887e−1±6.39e−27.887e^-1± 6.39e^-2 svp 1.903e−3±3.03e−31.903e^-3± 3.03e^-3 2.894e1±3.43e02.894e^1± 3.43e^0 7.907e−1±5.46e−27.907e^-1± 5.46e^-2 spp2* 1.408e−3±1.42e−31.408e^-3± 1.42e^-3 2.994e1±3.35e02.994e^1± 3.35e^0 8.050e−1±5.40e−28.050e^-1± 5.40e^-2 spp4* 1.399−±1.39− 1.399e^-3± 1.39e^-3 2.993e1±3.30e02.993e^1± 3.30e^0 8.048e−1±5.33e−28.048e^-1± 5.33e^-2 spp8* 1.411e−3±1.55e−31.411e^-3± 1.55e^-3 3.002±3.42 3.002e^1± 3.42e^0 8.052−±5.41− 8.052e^-1± 5.41e^-2