Paper deep dive
FederatedFactory: Generative One-Shot Learning for Extremely Non-IID Distributed Scenarios
Andrea Moleri, Christian Internò, Ali Raza, Markus Olhofer, David Klindt, Fabio Stella, Barbara Hammer
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%
Last extracted: 3/22/2026, 5:40:56 AM
Summary
FederatedFactory is a zero-dependency, one-shot federated learning framework that replaces discriminative parameter aggregation with the exchange of generative priors (Factories). By synthesizing class-balanced datasets ex nihilo from localized generative models, it overcomes gradient conflicts in pathologically non-IID (single-class silo) scenarios, achieving performance comparable to centralized training while enabling modular machine unlearning.
Entities (5)
Relation Signals (4)
FederatedFactory → evaluatedon → MedMNIST
confidence 95% · Evaluations across diverse medical imagery benchmarks, including MedMNIST and ISIC2019
FederatedFactory → evaluatedon → ISIC2019
confidence 95% · Evaluations across diverse medical imagery benchmarks, including MedMNIST and ISIC2019
FederatedFactory → supports → Modular Machine Unlearning
confidence 95% · Additionally, this framework facilitates exact modular unlearning through the deterministic deletion of specific generative modules.
FederatedFactory → utilizes → EDM2
confidence 95% · Specifically to our work, we use the computationally efficient EDM2 diffusion model.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Federated Learning (FL) enables distributed optimization without compromising data sovereignty. Yet, where local label distributions are mutually exclusive, standard weight aggregation fails due to conflicting optimization trajectories. Often, FL methods rely on pretrained foundation models, introducing unrealistic assumptions. We introduce FederatedFactory, a zero-dependency framework that inverts the unit of federation from discriminative parameters to generative priors. By exchanging generative modules in a single communication round, our architecture supports ex nihilo synthesis of universally class balanced datasets, eliminating gradient conflict and external prior bias entirely. Evaluations across diverse medical imagery benchmarks, including MedMNIST and ISIC2019, demonstrate that our approach recovers centralized upper-bound performance. Under pathological heterogeneity, it lifts baseline accuracy from a collapsed 11.36% to 90.57% on CIFAR-10 and restores ISIC2019 AUROC to 90.57%. Additionally, this framework facilitates exact modular unlearning through the deterministic deletion of specific generative modules.
Tags
Links
- Source: https://arxiv.org/abs/2603.16370v1
- Canonical: https://arxiv.org/abs/2603.16370v1
Trouble viewing inline? Open PDF directly →
Full Text
69,421 characters extracted from source content.
Expand or collapse full text
FederatedFactory: Generative One-Shot Learning for Extremely Non-IID Distributed Scenarios Andrea Moleri 1,2* , Christian Internò 3* , Ali Raza 1 , Markus Olhofer 1 , David Klindt 4 , Fabio Stella 2† , and Barbara Hammer 3† 1 Honda Research Institute Europe, Germany 2 University of Milan-Bicocca, Italy 3 Bielefeld University, Germany 4 Cold Spring Harbor Laboratory, U.S. * Equal contribution. † Co-advised. Abstract. Federated Learning (FL) enables distributed optimization without compromis- ing data sovereignty. Yet, where local label distributions are mutually exclusive, standard weight aggregation fails due to conflicting optimization trajectories. Often, FL methods rely on pretrained foundation models, introducing unrealistic assumptions. We introduce Fed- eratedFactory, a zero-dependency framework that inverts the unit of federation from discriminative parameters to generative priors. By exchanging generative modules in a sin- gle communication round, our architecture supports ex nihilo synthesis of universally class- balanced datasets, eliminating gradient conflict and external prior bias entirely. Evaluations across diverse medical imagery benchmarks, including MedMNIST and ISIC2019, demon- strate that our approach recovers centralized upper-bound performance. Under pathological heterogeneity, it lifts baseline accuracy from a collapsed 11.36% to 90.57% on CIFAR-10 and restores ISIC2019 AUROC to 90.57%. Additionally, this framework facilitates exact modular unlearning through the deterministic deletion of specific generative modules. Keywords: Federated Learning· Generative Synthesis· Non-IID Data 1 Introduction FL provides a decentralized framework to optimize statistical models across K distinct clients while strictly preserving local data sovereignty [36]. However, the theoretical convergence guarantees of traditional FL rely on independently and identically distributed (IID) data, an assumption violated by extreme statistical heterogeneity in real-world applications like multi-institutional medical imaging [40, 41]. The structural fragility of standard parameter aggregation is exposed most aggressively under pathological label skew [21,25], where each client holds samples from only a few, or even just one, class of data. Specifically, we define and analyze the extreme single-class silo regime, where each client k ∈1,...,K possesses a local datasetD k that contains exclusively one class, yielding mutually disjoint label supports Y k . Optimizing shared discriminative parameters w fails unconditionally in this setting. Lacking the counterfactual data necessary to form an inter- class decision boundary, the local empirical risks L k (w) produces optimization trajectories that interfere with one another. To avoid communication bottlenecks and conflicts in gradient trajectories of iterative FL, One- Shot Federated Learning (OSFL) attempts to aggregate knowledge in a single round of communi- cation [14,32]. Recent generative OSFL methods synthesize datasets using pretrained Foundation Models (FMs) as universal priors [2,49]. While effective for general domains, this dependency is unreliable for specialized applications such as medical diagnosis [23, 35,50]. By projecting a rare target distribution onto an external representation space, these methods effectively discard the off-manifold feature components x ⊥ that constitute the true diagnostic signal [16]. We introduce FederatedFactory, a framework that aims to recover the centralized upper- bound performance under the extreme single-class silo assumption via a novel, strictly zero- dependency Generative OSFL architecture. We invert the unit of federation from discriminative Correspondence: a.moleri@campus.unimib.it; christian.interno@uni-bielefeld.de Code: https://github.com/andreamoleri/FederatedFactory arXiv:2603.16370v1 [cs.LG] 17 Mar 2026 FederatedFactory2 parameter matrices W to localized generative prior parametersθ k . Each client independently trains and transmits a generative Factory G θ k exactly once, supporting both a centralized ar- chitecture (in consortiums where a central aggregator can be trusted) and a fully decentralized Peer-to-Peer (P2P) network mesh (in consortiums in which a central aggregator cannot be trusted). Specifically to our work, we use the computationally efficient EDM2 diffusion model [28,29]. The server concatenates these decoupled generative structures into a universal prior, synthe- sizing class-balanced datasets ex nihilo from a standard latent space Z. By relying exclusively on models trained directly on the true localized data distributions, FederatedFactory explicitly avoids projection errors and eliminates the external prior bias inherent to FM-reliant methods. [ Hypothesis: We hypothesize that shifting the unit of federation from discriminative parameters to generative priors enables OSFL to achieve performance comparable to cen- tralized baselines in pathologically non-IID scenarios, strictly without raw data exchange or reliance on external FMs. We summarize our contributions as follows: – Robustness to Extreme Heterogeneity: FederatedFactory recovers centralized per- formance under pathological single-class silos where standard methods collapse (e.g, CI- FAR10 [31] Accuracy 11.36%→ 90.57%, and ISIC2019 [43] AUROC 47.31%→ 90.57%). – Zero-Dependency Federation: By decoupling data synthesis from external pre-trained FMs, our protocol relies exclusively on localized priors. As formalized in Theorem 1, this bounds the global risk strictly by the local generative error ( ̄ε). – One-Shot Communication Efficiency: FederatedFactory relies on a single communi- cation round (C rounds = 1), avoiding expensive iterations. – Modular Machine Unlearning: The framework guarantees exact modular unlearning [10]. Removing a client requires only the structural deletion of their corresponding parameter co- ordinates (Γ :,k ←∅). 2 Related Work Optimization Under Distribution Shift. Federated methods (e.g., FedDyn [1], FedProx [33], SCAFFOLD [27]) handle statistical heterogeneity by bounding local updates. However, they fail when local label sets are entirely disjoint (Y i ∩Y j = ∅). Despite strong collaboration incentives in this extreme regime (e.g., isolated hospitals needing generalized models), disjoint labels cause actively divergent gradients [51]. Without overlapping classes to anchor a shared feature space, proximal constraints cannot align these conflicting trajectories to form coherent inter-class decision boundaries [33]. One-Shot FL and External Priors. OSFL bypasses iterative gradient conflicts via a single upstream transmission. Recent diffusion frameworks (e.g., FedLMG [46], FedSDE [38], Fed- DEO [47]) synthesize datasets centrally but rely heavily on pretrained FMs (e.g., CLIP [39], Stable Diffusion [4]). Relying on an external manifoldM FM inherently projects away rare features x ⊥ ; in medical imaging, this risks texture bias and semantic hallucination [11]. We eliminate this FM dependency by transmitting locally trained generative priorsθ k , achieving ex nihilo one-shot synthesis exclusively from true local distributions. Other approaches collaboratively train genera- tive models to construct synthetic datasets. Notably, the Diffusion Federated Dataset [15] models generation as cooperative sampling from diffusion energy-based models. While analytically robust, its reliance on iterative communication rounds increases communication overhead. 3 Background and Mathematical Preliminaries Federated Learning under Single-Class Silo Regime. In standard FL model aggregation, a decentralized network of K clients aims to aggregate local empirical risks over shared discrim- inative parameters w ∈ R m in order to effectively approximate the global optimum. The de FederatedFactory3 123 0 0.2 0.4 0.6 0.8 1 Density (a) IID / Uniform (α→∞) 123 0 0.2 0.4 0.6 0.8 1 (b) Dirichlet Skew (α = 0.5) 123 0 0.2 0.4 0.6 0.8 1 (c) Single-Class Silo (α→ 0) Fig. 1: The Spectrum of Heterogeneity. (a) Ideal IID data (uniform overlap). (b) Dirichlet- distributed skew (imbalanced overlap). (c) Single-Class Silo (pathologically disjoint supports), represent- ing the extreme theoretical limit. Takeaway I: Under the single-class silo regime, standard FL collapses. This extreme label skew induces gradient conflict across clients, rendering standard parameter aggregation incapable of forming a coherent global decision boundary. facto standard FL approach [36] formally attempts to minimize the global objective function min w L(w) := 1 K P K k=1 L k (w), where L k (w) = E (x,y)∼p k [ℓ(w;x,y)] is the local objective repre- senting the expected loss over the true data distribution p k at client k, and ℓ denotes the per-sample loss function. While aggregators such as FedAvg [36] succeed when the local datasets D k are IID, this paradigm collapses under severe non-IID settings. We parameterize label skewness via a Dirichlet distribution Dir C (α). As α → 0 in a cross-silo setting with K clients and C global classes, we reach the pathological Single-Class Silo regime (Figure 1c). Here, each client dataset D k contains exactly one unique class, yielding strictly disjoint label sets (Y i ∩Y j =∅ for i̸= j). Consequently, local optimization trajectories diverge, causing severe gradient conflict [48,51]. FederatedFactory Formal Problem Statement. Our objectives challenge the standard set- ting defined in Sec. 3. We operate strictly within the cross-silo regime, which stands in contrast to the more common cross-device setting. While cross-device learning involves massive populations of transient, resource-constrained mobile or IoT units, the cross-silo regime is characterized by a small, fixed number of stakeholders (such as hospitals or financial institutions) possessing localized, high-capacity computational resources and persistent connectivity [26]. We aim to design a FL optimization protocol A that recovers the optimal centralized decision boundary while operating under severe decentralized restrictions. Let D union = S K k=1 D k denote the inaccessible theoretical centralized dataset, and let w ∗ = arg min w E (x,y)∼D union [ℓ(w,x,y)] represent the optimal param- eters obtained via centralized training. We model the federated protocol A as a set of local client functions f k generating upstream messages M k = f k (D k ), and a server aggregation function g such that the final synthesized model has parameters w A = g(M 1 ,...,M K ). We formalize this objective as a constrained optimization problem seeking to minimize the excess global risk: min f 1 ,...,f K ,g E (x,y)∼D union [ℓ(w A ,x,y)]− E (x,y)∼D union [ℓ(w ∗ ,x,y)] s.t. C1 (Pathological Skew): supp(p i (y))∩ supp(p j (y)) =∅, ∀i̸= j C2 (Zero-Dependency): w A without external priorθ FM C3 (Strict Sovereignty): x /∈ M k , ∀x∈D k C4 (One-Shot Comm.): C rounds = 1 (1) Here, ℓ is the per-sample task-specific loss function. C1 mandates convergence in the extreme limit of the Dirichlet distribution (α→ 0), where p i (y) denotes the marginal label distribution at client i. In this pathological regime, discriminative gradient trajectories are actively opposed due to completely disjoint label supports. C2 prohibits reliance on external foundation models (M FM ) or public proxy datasets to synthesize missing counterfactuals, isolating the framework from external prior bias. C3 guarantees no raw samples are transmitted in the uplink communication messages M k . Finally, C4 restricts the system to exactly one asynchronous upstream communication per FederatedFactory4 client (where C rounds is the total number of communications), permanently eliminating iterative communication overhead and bidirectional synchronization requirements. 3.1 Theoretical Guarantee of Zero-Dependency Convergence Recent generative OSFL frameworks ensure global convergence using server-side Foundation Mod- els (FMs) [38,46,47], assuming the FM’s distribution p FM (x) sufficiently overlaps with local client data p k (x). This overlap is bounded by λ, which quantifies the maximum OOD penalty between the pre-trained manifold and private data. While effective for natural images, this assumption fails in specialized modalities (e.g, clinical) where severe domain shifts cause λ→∞. To motivate FederatedFactory, we establish a convergence guarantee that completely by- passes this FM overlap assumption. Let p union (x,y) be the inaccessible true global joint distribu- tion. Under the Single-Class Silo constraint (C1) with disjoint label supports, p union reduces to a strict mixture of local marginals p k (x) weighted by empirical proportions π k =|D k |/|D union |: p union (x,y) = K X k=1 π k p k (x)I(y = y k ) (2) where I(·) is the indicator function isolating the localized class. By transmitting only the localized generative priorθ k (C3, C4), the server constructs a fully synthetic global distribution ˆp syn (x,y) ex nihilo, complying with the zero-dependency constraint (C2): ˆp syn (x,y) = K X k=1 π k p θ k (x)I(y = y k )(3) To prove convergence without external priors, we establish two standard assumptions: Assumption 1 (Local Generative Convergence). The local diffusion training objective (ELBO) directly minimizes the Kullback-Leibler (KL) divergence. We assume this local optimiza- tion error is bounded by ε k for all clients k ∈1,...,K: KL(p k (x)∥ p θ k (x))≤ ε k (4) Assumption 2 (Bounded Risk Function). The task-specific per-sample loss function ℓ(w,x,y) is bounded by a constant M > 0, such that sup w,x,y ℓ(w,x,y)≤ M. Lemma 1 (Global Manifold Recovery). Under constraint C1 and Assumption 1, the KL divergence between the true global distribution and the zero-dependency synthetic distribution is strictly bounded by the weighted sum of the local diffusion errors: KL(p union ∥ ˆp syn )≤ P K k=1 π k ε k . Proof. Based on the fundamental definition of the joint KL divergence, we expand the integral: KL(p union ∥ ˆp syn ) = X y Z p union (x,y) log p union (x,y) ˆp syn (x,y) dx(5) Because the label supports are completely disjoint (C1), for any given class y k , the cross-terms of the mixture evaluate exactly to zero. The summation over y thus collapses perfectly to the individual client indices k. Substituting the marginal definitions: = K X k=1 Z π k p k (x) log π k p k (x) π k p θ k (x) dx = K X k=1 π k KL(p k (x)∥ p θ k (x))(6) Substituting the local convergence bound from Assumption 1 yields the final inequality. ■ We define L true (w) = E (x,y)∼p union [ℓ(w,x,y)] as the true centralized risk, and L syn (w) = E (x,y)∼ ˆp syn [ℓ(w,x,y)] as the surrogate risk evaluated on the synthesized dataset. Let ̄ε = q 1 2 P K k=1 π k ε k denote the aggregate generative error mapped to the Total Variation (TV) space. FederatedFactory5 Theorem 1 (Zero-Dependency Aggregation). Under Assumptions 1 and 2, the excess global risk of the classifier w A = arg min w L syn (w) trained exclusively on the generated synthetic distribution, compared to the optimal centralized classifier w ∗ = arg min w L true (w), is strictly bounded by: L true (w A ) | z Federated Classifier − L true (w ∗ ) |z Centralized Classifier ≤2M ̄ε | z Max Penalty (7) w Risk L L true L syn w ∗ w A ≤ 2M ̄ε Proof. By the integral definition of the Total Variation (TV) distance and Pinsker’s inequality, the expected risk deviation for any arbitrary classifier w is strictly bounded by: |L true (w)−L syn (w)|≤ M · TV(p union ∥ ˆp syn )≤ M r 1 2 KL(p union ∥ ˆp syn )≤ M ̄ε(8) Because the federated classifier w A is optimized to minimize the synthetic risk, it holds that L syn (w A )≤L syn (w ∗ ). Applying the bound M ̄ε symmetrically transitions between true and syn- thetic risks: L true (w A )≤L syn (w A ) + M ̄ε≤L syn (w ∗ ) + M ̄ε≤L true (w ∗ ) + 2M ̄ε(9) which concludes the proof.■ Remark. Theorem 1 formally demonstrates that global convergence is achievable under con- straints C1–C4. By exchanging generative priors, the global distribution approximation error is entirely bounded by ε k . Unlike FM-dependent methods where the bound is dominated by a rigid, often infinite projection error (λ) caused by out-of-distribution local data, FederatedFactory’s error approaches zero simply by training the local models to standard convergence. 4 Methodology To satisfy the optimization objective outlined in Sec. 3, FederatedFactory abandons discrim- inative parameter aggregation entirely. Under the pathological single-class silo regime (C1), gra- dient trajectories are in conflict [48]. We resolve this by replacing the unit of communication from discriminative gradients to localized generative prior parameters, establishing a zero-dependency (C2), no-data sharing (C3), one-shot framework (C4) for distributed synthesis. 4.1 FederatedFactory Instead of transmitting parameter updates, which actively conflict across disjoint label spaces [48], FederatedFactory relies on the transmission of generative model parameters. We define the Factory as a localized, self-contained generative module. Each client k ∈1,...,K independently optimizes a Factory on its private dataset D k . While our framework is architecture-agnostic, we specifically instantiate the Factory using the Score-based Diffusion model EDM2 [29]. In this context, the “generative blueprint” consists of the denoising function G θ k . Unlike tradi- tional Autoencoders [8], diffusion models do not utilize a deterministic encoder to compress data into a latent bottleneck. Instead, the reverse denoising process acts as the fundamental mapping from a standard normal latent space Z ∼N (0,I) to the learned local manifold ˆ M k [20]. While we empirically focus on Diffusion for its high-fidelity clinical synthesis, Federated- Factory natively supports architectures with explicit encoder-decoder splits (e.g., VAEs [30]) or adversarial mappings (GANs [13]), where the shared parametersθ k would correspond to the phys- ical decoder or generator. Reducing the communication payload to independent Factories enables flexible global synthesis. Specifically, we instantiate this framework through two distinct opera- tional modes: (A) a centralized architecture designed for consortiums equipped with a trusted aggregator (Sec. 4.1), and (B) a fully decentralized peer-to-peer (P2P) mesh optimized for envi- ronments in which a centralized entity cannot be trusted (Sec. 4.1). FederatedFactory6 Takeaway I: By transmitting generative Factories, FederatedFactory enables a blueprint-based ex nihilo generation of class-balanced datasets without relying on overlapping data supports or external FMs. Protocol A: Centralized Synthesis. Under this configuration (Figure 2), the framework des- ignates the central aggregator as the point of data generation, restricting network overhead to exactly one upstream transmission per client (C rounds = 1). The server aggregates the generative mappings into a unified libraryΘ =G θ 1 ,...,G θ K . Exploiting the universality of the standard normal latent space Z, the server samples noise vectors z ∼ N (0,I) and projects them through the respective Factory. This materializes a fully synthetic, class-balanced global dataset ˆ D syn ex nihilo, with an arbitrarily large number of samples whose diversity is bounded by the entropy of the localized generative priors: ˆ D syn = K [ k=1 ( ˆ x,y k )| ˆ x = G θ k (z),z∼N (0,I) (10) We then optimize a global classifier w (ResNet-50 [17]) exclusively on ˆ D syn . We adopt this model to isolate the improvements of our generative framework without introducing architectural bottle- necks. We reformulate the standard FL parameter aggregation into an optimization over a global surrogate objective: min w 1 K K X k=1 E z∼N (0,I) [ℓ (w;G θ k (z),y k )](11) Because this synthesis relies exclusively on localized generative priors optimized directly on the true data manifolds, FederatedFactory ensures that the generated distribution includes the rare diagnostic support (as required in Sec. 3.1). Client A (Class 1) Client B (Class 2) Client C (Class 3) Server (Aggregator) Real D 1 Factory A x ∼ p 1 (x) Real D 2 Factory B x ∼ p 2 (x) Real D 3 Factory C x ∼ p 3 (x) (i) Local Training (e.g. EDM2 [29]) G θ 1 G θ 2 G θ 3 z∼N z∼N z∼N Syn ˆ D 1 Syn ˆ D 2 Syn ˆ D 3 ˆx = G θ 1 (z) ˆx = G θ 2 (z) ˆx = G θ 3 (z) (i) Upload (i) Generative Sampling: z → ˆx Params θ 1 Params θ 2 Params θ 3 Global Classifier Inference (iv) Global Training (e.g. ResNet-50 [17]) Fig. 2: Centralized FederatedFactory Protocol. Aggregated Factories produce a fully synthetic dataset ˆ D, enabling a global classifier training without raw data access. Protocol B: Decentralized Synthesis. Under this configuration (Figure 3), each client k broad- casts its generative prior G θ k to all participating peers. Upon receiving the complement Factories FederatedFactory7 for all disjoint classes j ̸= k, every client k locally synthesizes the missing distributions, construct- ing a hybrid dataset D mix k in which local real data D k is augmented with synthetic samples ˆ x: D mix k =D k ∪ [ j̸=k ( ˆ x,y j )| ˆ x = G θ j (z),z∼N (0,I) (12) Client k then optimizes a local discriminative expert classifier f w k exclusively onD mix k . At inference time, the distributed mesh aggregates global decisions using a Product of Experts (PoE) formu- lation [19]. Unlike standard mixture-based ensembling, which acts as a logical disjunction and dilutes predictive certainty, PoE functions as a strict intersection of constraints. For a target sam- ple x, the global inference probability is the renormalized product: p PoE (y | x) = 1 Z Q K k=1 p k (y | x) where Z is the partition function. This enforces a strict consensus veto. To prevent a single over- confident but incorrect expert from indiscriminately zeroing out the consensus, the aggregation operates in log-space with a strict minimum probability floor. Thus, if a local expert assigns a near-zero probability to a spurious feature, the aggregate probability strictly collapses, preserving high-confidence decision boundaries across the distributed network. Client A (Class 1) Client B (Class 2) Client C (Class 3) Real D 1 Factory A x ∼ p 1 (x) G θ 2 G θ 3 Union ˆ D A Real∪ Syn Model w A Self D 1 Real D 2 Factory B x ∼ p 2 (x) G θ 1 G θ 3 Union ˆ D B Real∪ Syn Model w B Self D 2 Real D 3 Factory C x ∼ p 3 (x) G θ 1 G θ 2 Union ˆ D C Real∪ Syn Model w C Self D 3 Broadcast θ 1 Broadcast θ 2 Broadcast θ 3 Product of Experts Inference (i) Local Training (e.g. EDM2 [29]) (i) Broadcast (i) Classifier Training (e.g. ResNet-50 [17]) (iv) Inference: ˆ P (y|x)∝ Q P k (y|x) Fig. 3: Decentralized FederatedFactory Protocol. This architecture depicts the P2P topology where local data flows (x ∼ p k (x)) are augmented with synthetic samples from broadcasted Factories to train local experts, aggregated via PoE. 4.2 Multi-Class Generalization FederatedFactory natively extends to multi-class configurations. When clients possess data spanning multiple classes, each client k ∈1,...,K independently trains a class-specific Factory G θ k (z,c) for every class c ∈ Y k present in its local dataset. To preserve the true global prior probability of the classes during centralized aggregation, the server requires the local sample counts n c,k = |D c,k |. During global data synthesis, the server draws m c,k latent vectors for each specific Factory, strictly ensuring that the synthesized contribution is proportional to the local empirical density (m c,k ∝ n c,k ). To bypass the destructive non-convexity of parametric averaging, FederatedFactory8 (1) Vertical k 1 k 2 k 3 c 1 c 2 c 3 G θ c,k G θ c,k G θ c,k ∅ ∅ ∅ G θ c,k G θ c,k G θ c,k (2) Horizontal k 1 k 2 k 3 c 1 c 2 c 3 G θ c,k ∅ G θ c,k G θ c,k ∅ G θ c,k G θ c,k ∅ G θ c,k (3) Targeted k 1 k 2 k 3 c 1 c 2 c 3 G θ c,k G θ c,k G θ c,k G θ c,k ∅ G θ c,k G θ c,k G θ c,k G θ c,k Client Removal Γ :,k ←∅ Concept Erasure Γ c,: ←∅ Specific Intersection Γ c,k ←∅ Fig. 4: Modular Unlearning Modes in the Generative MatrixΓ. By structuring the global model as a disjoint union of class-client generators G θ c,k , FederatedFactory enables exact erasure without retraining the entire ensemble. which frequently shifts mature weights into high-loss regions, the server instead performs data- space weighted aggregation. For a target sample size N target of class c, and clients S c ⊆1,...,K possessing this class, the server assigns a generation quota Q c,k to each local Factory G θ c,k : Q c,k = N target n c,k P j∈S c n c,j . This allocation ensures the synthetic global dataset ˆ D syn mirrors its empirical sources. Large institutions establish the manifold’s backbone, while smaller clinics inject stochastic diversity. This preserves rare morphological subtypes and scales the global distribution seamlessly, bypassing the interference of weight aggregation across imbalanced models. The multi-class expansion organizes the aggregated models into a “Visual Memory” [10], defining a Generative MatrixΓ ∈F C×K , with C global classes, K clients, and mapping space F. Each entryΓ c,k holds the localized Factory G θ c,k (or ∅ if locally absent). Under a strict single-class silo regime (α → 0),Γ collapses into a severely sparse diagonal, with each client contributing exactly one valid prior. Under general multi-class heterogeneity, it populates organically based on local label supports. 4.3 Modular Machine Unlearning and Exact Erasure Decoupling the generative process intoΓ structurally allows for modular unlearning [6]. Standard FL densely entangles representations across global weights, making localized data deletion in- tractable and typically requiring complete retraining to satisfy right-to-be-forgotten mandates [12]. In FederatedFactory, the global representation is a discrete union of parameter-independent modules. In the single-class silo regime, unlearning trivially requires excising the target client’s generative prior:Θ new =Θ\G θ k . The server then discards the associated synthetic samples and retrains the centralized classifier. In decentralized P2P setups, this is mirrored locally: peers delete the revoked prior G θ k , flush related synthetic data, and retrain experts. Extending this logic throughΓ (Figure 4) enables three granular modes of exact data erasure. Vertical Unlearning (Client Removal): The server nullifies a column (Γ :,k ← ∅) to compre- hensively forget client k. Horizontal Unlearning (Concept Erasure): The server executes a row-wise deletion (Γ c,: ← ∅) to remove an obsolete or restricted class c consortium-wide. Tar- geted Unlearning (Specific Intersection): The server zeroes an exact coordinate (Γ c,k ←∅) to remove a specific class subset from a specific client. Following any deletion, the server clears the invalidated synthetic buffer ˆ D syn and retrains the centralized classifier. Because the target manifold’s underlying generative prior is eradicated, we guarantee exact data removal ( ˆ D syn ∩ ˆ M c,k = ∅). This achieves true exact unlearning without approximations [6,12,44]. FederatedFactory9 Table 1: Robustness to Extreme Statistical Heterogeneity. Mean ± SD of Accuracy and AUROC (%) under moderate (α = 0.1) and pathological (α→ 0) label skew. Best overall performance (Red). Bold indicates the best FL method. Method CIFARBloodMNISTPathMNIST DirichletSilosDirichletSilosDirichletSilos AccAUCAccAUCAccAUCAccAUCAccAUCAccAUC FedAvg [36]89.76 ± 1.1299.22 ± 0.1911.36 ± 1.2850.91 ± 1.7983.46 ± 5.1897.57 ± 1.1121.88 ± 5.3955.23 ± 7.1773.79 ± 16.8690.11 ± 16.0518.05 ± 0.8050.12 ± 1.76 FedDyn [1]61.82 ± 11.4794.87 ± 2.6510.12 ± 0.2451.48 ± 0.8364.93 ± 8.6392.02 ± 2.2119.47 ± 0.0051.44 ± 2.8760.99 ± 8.6692.71 ± 4.4917.76 ± 0.8047.57 ± 3.22 FedProx [33]89.67 ± 1.1599.21 ± 0.1918.08 ± 1.1867.06 ± 1.0284.11 ± 5.3897.61 ± 1.0020.18 ± 1.5753.69 ± 3.3277.09 ± 11.8996.01 ± 4.4418.93 ± 0.5752.97 ± 2.67 Scaffold [27]79.26 ± 5.1097.73 ± 0.8612.99 ± 0.7554.83 ± 1.0882.31 ± 2.9797.51 ± 0.7522.60 ± 1.9667.84 ± 2.9871.88 ± 9.7095.61 ± 1.8831.15 ± 2.1064.87 ± 2.79 FedFact (Cent.)–84.30 ± 0.6698.24 ± 0.13–91.17 ± 0.2699.04 ± 0.06–67.94 ± 3.8493.60 ± 1.07 FedFact (P2P)–90.57 ± 0.0999.14 ± 0.02–86.38 ± 0.3198.36 ± 0.07–67.03 ± 2.2891.48 ± 0.64 Centralized Upper Bound94.69 ± 0.3399.75 ± 0.0394.69 ± 0.3399.75 ± 0.0391.23 ± 0.8799.18 ± 0.1391.23 ± 0.8799.18 ± 0.1384.82 ± 0.8196.96 ± 0.4084.82 ± 0.8196.96 ± 0.40 Method RetinaMNISTISIC2019 DirichletSilosDirichletSilos AccAUCAccAUCAccAUCAccAUC FedAvg [36]45.20 ± 3.8055.02 ± 7.9643.50 ± 0.0048.70 ± 1.9060.35 ± 6.9476.73 ± 12.3348.22 ± 0.0047.31 ± 1.78 FedDyn [1]45.25 ± 3.9154.02 ± 8.8743.50 ± 0.0048.70 ± 1.9253.63 ± 5.4868.48 ± 7.8148.22 ± 0.0043.28 ± 0.99 FedProx [33]45.45 ± 4.3654.52 ± 8.7043.50 ± 0.0048.74 ± 1.9162.83 ± 2.4183.21 ± 1.6048.22 ± 0.0045.49 ± 0.78 Scaffold [27]43.50 ± 0.0052.52 ± 6.1543.50 ± 0.0048.70 ± 1.9050.03 ± 1.7564.37 ± 6.1948.22 ± 0.0044.33 ± 1.02 FedFact (Cent.)–46.75 ± 1.2970.29 ± 1.06–62.08 ± 0.5984.98 ± 0.57 FedFact (P2P)–49.30 ± 1.2571.79 ± 0.64–69.94 ± 0.4690.57 ± 0.08 Centralized Upper Bound47.20 ± 0.7869.69 ± 1.0047.20 ± 0.7869.69 ± 1.0070.38 ± 0.6990.37 ± 0.7470.38 ± 0.6990.37 ± 0.74 DirichletSilos 0 20 40 60 80 100 Accuracy (%) CIFAR DirichletSilos 0 20 40 60 80 100 BloodMNIST DirichletSilos 35 40 45 50 55 60 RetinaMNIST DirichletSilos 0 20 40 60 80 100 PathMNIST DirichletSilos 40 50 60 70 80 90 ISIC2019 FedAvgFedDynFedProxScaffoldFedFact (Centralized)FedFact (Decentralized)Centralized Upper Bound Fig. 5: Results in Pathological Heterogeneity. While standard baselines (FedAvg ◦ ,FedDyn □ , FedProx △ ,Scaffold ⋄ ) collapse as we move from moderate skew (α = 0.1) to extreme silos (α→ 0), FederatedFactory matches theUpper Bound (—) in bothCentralized (+) ,Decentralized (⋆) configurations. 5 Experiments and Results Experimental Setup. To validate FederatedFactory, we evaluate across three dataset regimes: (1) CIFAR-10 [31] as a standard high-variance baseline for foundational stability; (2) three MedMNIST [45] subsets (BloodMNIST, RetinaMNIST, PathMNIST) to test diverse mor- phological heterogeneity; and (3) ISIC2019 [43] as a high-resolution stress test for rare dermato- scopic classes. Across all configurations, we employ an adaptive ResNet-50 [17] backbone trained via an identical SGD optimizer (batch size 128, weight decay 1×10 −4 , initial learning rate 0.1 with cosine annealing [34]) and domain-specific augmentations over equivalent computational budgets (300 centralized/synthetic epochs vs. 200 rounds of 5 local epochs for federated baselines). Follow- ing preliminary grid-search tuning, we report the mean and standard deviation of Test Accuracy and macro-averaged One-vs-Rest (OvR) AUROC across five independent random seeds for all configurations. The centralized upper bound is explicitly trained on the theoretically inaccessible global dataset D union . Results. Table 1 and Figure 5 quantify the transition from moderate heterogeneity (Dirichlet α = 0.1) to the Single-Class Silo extreme. Under moderate skew, FedProx [33] and SCAFFOLD [27] maintain convergence. However, in strictly disjoint silos, parameter aggregation degenerates into a random walk, causing unconditional baseline failure. FedAvg on CIFAR plunges from 89.76% → 11.36% accuracy, while on imbalanced sets like ISIC2019, global models collapse to majority- class prediction (e.g., FedProx drops from 62.83%→ 48.22%). Conversely, FederatedFactory avoids this failure, as demonstrated on RetinaMNIST (49.30% vs. 47.20% Centralized Accuracy) and ISIC2019 (90.57% vs. 90.37% Centralized AUROC). FederatedFactory10 Table 2: Resource Trade-off Analysis. Total computational overhead (FLOPs) vs. communication volume (MBs) across the five benchmarks. Method CIFARBloodMNISTPathMNISTRetinaMNISTISIC2019 FLOPsMBsFLOPsMBsFLOPsMBsFLOPsMBsFLOPsMBs FedAvg [36]1.95× 10 17 358,899.63.57× 10 16 287,069.62.69× 10 17 322,981.53.22× 10 15 179,371.63.11× 10 17 287,163.4 FedDyn [1]1.95× 10 17 358,899.63.57× 10 16 287,069.62.69× 10 17 322,981.53.22× 10 15 179,371.63.11× 10 17 287,163.4 FedProx [33]1.95× 10 17 358,899.63.57× 10 16 287,069.62.69× 10 17 322,981.53.22× 10 15 179,371.63.11× 10 17 287,163.4 Scaffold [27]1.95× 10 17 717,799.13.57× 10 16 574,139.32.69× 10 17 645,962.93.22× 10 15 358,743.23.11× 10 17 574,326.8 FedFact (Server)7.30× 10 18 1,934.14.47× 10 18 1,547.35.03× 10 18 1,740.72.79× 10 18 967.02.52× 10 20 1,435.3 FedFact (Local)8.35× 10 18 19,340.64.97× 10 18 12,378.05.67× 10 18 15,665.92.97× 10 18 4,835.22.54× 10 20 11,482.0 DINOV2 (Block 8) t-SNE CIFAR10 BloodMNISTRetinaMNISTPathMNISTISIC2019 Real Sample Synthetic Sample (FederatedFactory) Synthetic Real Fig. 6: Real vs. Synthetic Data in Representation Space. Top: DINOv2 (Block 8) feature space [37]. FederatedFactory (▲) represents the true target manifolds data (•), preserving good per- formance. Bottom: Visual comparisons across datasets. Columns pair real and generated samples of the same class, demonstrating accurate morphological preservation without memorization. Takeaway I: FederatedFactory fixes the Single-Class Silo collapse, and it also manages to match the centralized data-pooled upper-bound baseline, under more difficult constraints. Communication-Computation Anaysis. Table 2 highlights FederatedFactory’s shift from a bandwidth-bound to a compute-bound regime. Traditional FL baselines minimize edge compute but incur communication penalties from iterative convergence. Bidirectional parameter transmis- sions across 200 rounds create severe bottlenecks, demanding hundreds of gigabytes (e.g., 358,899.6 MB on CIFAR-10), a vulnerability SCAFFOLD strictly doubles for control variates. Conversely, FederatedFactory employs a OSFL protocol, collapsing network exposure and achieving a 99.4% communication reduction (1,934.1 MB on CIFAR-10). This zero-dependency design incurs a “generative tax” in local FLOPs, scaling computational load from ∼ 10 17 in baselines to ∼ 10 18 (up to ∼ 10 20 for ISIC2019). Ultimately, trading computational overhead for reduced communi- cation costs is justified in cross-silo clinical settings, where model trustworthiness takes precedence over computational expense. Qualitative Analysis and Manifold Alignment. To verify that Factories correctly approx- imate the localized manifolds without memorization [5], we project the generated distribution ˆ D syn and the true target distribution D union into a joint feature space, presented alongside qual- itative samples in Figure 6. Visual comparisons (Fig. 6, Top) demonstrate FederatedFactory successfully synthesizes complex morphologies (e.g., ISIC2019 textures, BloodMNIST cells) ex nihilo. To validate this structural fidelity, we project intermediate DINOv2 representations (Block 8) [37] via t-SNE (Fig. 6, Bottom), a feature space proven effective for isolating generative ar- tifacts [24]. The synthetic priors natively span the true empirical distribution without discrete FederatedFactory11 coordinate overlaps. While not a formal privacy guarantee, this empirically confirms continuous diversity and functional manifold mapping over trivial memorization. 6 Conclusion As federated learning (FL) scales [26], resolving extreme statistical heterogeneity is critical [21,51]. To bypass the collapse of standard parameter aggregation under single-class silos [36], Federat- edFactory transfers localized generative priors rather than discriminative gradients. By using independent generative modules to synthesize class-balanced global datasets ex nihilo [20, 28], our zero-dependency framework eliminates gradient conflict and external prior bias [16,23]. This consistently recovers centralized upper-bound performance where standard methods fail. For in- stance, under strictly disjoint silos, it lifts CIFAR-10 accuracy from a collapsed 11.36% (FedAvg) to 90.57% (matching the centralized bound), and improves ISIC2019 AUROC from 47.31% to 90.57% [43]. Remarkably, it achieves this while slashing communication overhead by 99.4% (from 358, 899.6 MB to just 1, 934.1 MB on CIFAR-10) [36]. While traditional FL averages local trajectories, our results prove that transferring the un- derlying data manifold approximation is fundamentally more robust for disjoint label supports. This paradigm shift encourages transmitting localized generative models over fragile discrimina- tive boundaries. Furthermore, structuring the global model as a discrete union of these generative modules inherently facilitates exact modular unlearning [6,44]. Excising a client’s specific gener- ative parameters guarantees exact data erasure, effortlessly complying with right-to-be-forgotten mandates without retraining the global ensemble [12]. Limitations. Trading communication for compute bottlenecks in cross-silo settings introduces a substantial “generative tax.” Hardware profiling shows local computational loads scale by roughly an order of magnitude (e.g., 1.95 × 10 17 to 8.35 × 10 18 FLOPs on CIFAR-10). Furthermore, theoretical convergence assumes local diffusion models converge without mere memorization [5]. While feature space projections empirically confirm continuous diversity, the framework lacks formal privacy guarantees (e.g., Differential Privacy) [9] against advanced data extraction [7] or membership inference attacks on transmitted priors [22]. Broader Impacts. Standard FL fragility endangers multi-institutional collaborations, espe- cially in clinical imaging where data sovereignty is fundamental [40, 41]. FederatedFactory provides a robust, zero-dependency alternative. Moreover, organizing the global model as a dis- crete Generative Matrix inherently supports exact modular unlearning [6,44], ensuring compliance with stringent data privacy regulations like the Right to be Forgotten and strictly protecting lo- calized data rights [12]. References 1. Acar, D.A.E., Zhao, Y., Matas, R., Mattina, M., Whatmough, P., Saligrama, V.: Federated learning based on dynamic regularization. In: International Conference on Learning Representations (ICLR) (2021), https://openreview.net/forum?id=B7v4QMR6Z9w 2. Beitollahi, M., Bie, A., Hemati, S., Brunswic, L.M., Li, X., Chen, X., Zhang, G.: Parametric feature transfer: One-shot federated learning with foundation models. arXiv preprint arXiv:2402.01862 (2024) 3. Bińkowski, M., Sutherland, D.J., Arbel, M., Gretton, A.: Demystifying MMD GANs. In: International Conference on Learning Representations (ICLR) (2018) 4. Blattmann, A., Rombach, R., Oktay, K., Ommer, B.: Semi-parametric neural image synthesis (2022). https://doi.org/10.48550/arXiv.2204.11824, https://arxiv.org/abs/2204.11824v3 5. Bonnaire, T., Urfin, R., Biroli, G., Mézard, M.: Why diffusion models don’t memorize: The role of implicit dynamical regularization in training (2025), https://arxiv.org/abs/2505.17638 6. Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C.A., Jia, H., Travers, A., Zhang, B., Lie, D., Papernot, N.: Machine unlearning. In: IEEE Symposium on Security and Privacy (SP). p. 141–159 (2021) 7. Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramèr, F., Balle, B., Ippolito, D., Wallace, E.: Extracting training data from diffusion models. In: Proceedings of the 32nd USENIX Conference on Security Symposium. SEC ’23, USENIX Association, USA (2023) 8. Davidson, T.R., Falorsi, L., Cao, N.D., Kipf, T., Tomczak, J.M.: Hyperspherical variational auto- encoders. In: Uncertainty in Artificial Intelligence (UAI) (2018), https://arxiv.org/abs/1804.00891 FederatedFactory12 9. Dwork, C., Roth, A.: The algorithmic foundations of differential privacy. Found. Trends Theor. Com- put. Sci. 9(3–4), 211–407 (Aug 2014). https://doi.org/10.1561/0400000042, https://doi.org/ 10.1561/0400000042 10. Geirhos, R., Jaini, P., Stone, A., Medapati, S., Yi, X., Toderici, G., Ogale, A., Shlens, J.: Towards flex- ible perception with visual memory. In: Forty-second International Conference on Machine Learning (2025), https://openreview.net/forum?id=dMYL47aQwb 11. Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F.A., Brendel, W.: Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In: In- ternational Conference on Learning Representations (2019), https://openreview.net/forum?id= Bygh9j09KX 12. Golatkar, A., Achille, A., Soatto, S.: Eternal sunshine of the spotless net: Selective forgetting in deep networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 9304–9312 (2020) 13. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (NeurIPS). vol. 27 (2014) 14. Guha, N., Talwalkar, A., Smith, V.: One-shot federated learning. In: arXiv preprint arXiv:1902.11175 (2019), presented at NeurIPS 2019 Workshop on Federated Learning 15. Hahn, S.J., Lee, J.: Diffusion federated dataset. In: OpenReview (2024), https://openreview.net/ forum?id=1GCWcrZTX8 16. He, H., Xiang, S., Zhang, Y., Zhu, Y., Zhang, J., Lu, Y., Deng, H., Alsentzer, E., Chen, Q., Yu, K.H., et al.: Ai-generated data contamination erodes pathological variability and diagnostic reliability. medRxiv (2026) 17. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). p. 770–778 (2016) 18. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time- scale update rule converge to a local nash equilibrium. In: Advances in Neural Information Processing Systems (NeurIPS). vol. 30 (2017) 19. Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Computation 14(8), 1771–1800 (2002), https://w.cs.toronto.edu/~hinton/absps/training-products-of- experts-by-minimizing-contrastive-divergence.pdf 20. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: Advances in Neural Infor- mation Processing Systems (NeurIPS). vol. 33, p. 6840–6851 (2020) 21. Hsu, T.M.H., Qi, H., Brown, M.: Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335 (2019), https://arxiv.org/abs/1909.06335 22. Hu, H., Pang, J.: Membership inference of diffusion models (2023), https://arxiv.org/abs/2301. 09956 23. Huix, J.P., Ganeshan, A.R., Haslum, J.F., Söderberg, M., Matsoukas, C., Smith, K.: Are natural domain foundation models useful for medical image classification? In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. p. 7977–7987 (2024) 24. Internò, C., Geirhos, R., Olhofer, M., Liu, S., Hammer, B., Klindt, D.: AI-generated video detection via perceptual straightening. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025), https://openreview.net/forum?id=LsmUgStXby 25. Jamali-Rad, H., Abdizadeh, M., Singh, A.: Federated learning with taskonomy for non-iid data. arXiv (2021). https://doi.org/10.48550/arxiv.2103.15947 26. Kairouz, P., McMahan, H.B.: Advances and open problems in federated learning. Foundations and Trends in Machine Learning 14(1-2), 1–210 (06 2021). https://doi.org/10.1561/2200000083, https://doi.org/10.1561/2200000083 27. Karimireddy, S.P., Kale, S., Mohri, M., Reddi, S., Stich, S., Suresh, A.T.: Scaffold: Stochastic con- trolled averaging for federated learning. In: International Conference on Machine Learning (ICML). p. 5132–5143. PMLR (2020), https://proceedings.mlr.press/v119/karimireddy20a.html 28. Karras, T., Aittala, M., Aila, T., Laine, S.: Elucidating the design space of diffusion-based generative models. In: Advances in Neural Information Processing Systems (NeurIPS). vol. 35, p. 26565–26577 (2022), https://arxiv.org/abs/2206.00364 29. Karras, T., Aittala, M., Lehtinen, J., Hellsten, J., Aila, T., Laine, S.: Analyzing and improving the training dynamics of diffusion models. In: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR). p. 24168–24178 (2024), https://openaccess. thecvf.com/content/CVPR2024/html/Karras_Analyzing_and_Improving_the_Training_Dynamics_ of_Diffusion_Models_CVPR_2024_paper.html 30. Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: International Conference on Learning Representations (ICLR) (2013), arXiv preprint arXiv:1312.6114 FederatedFactory13 31. Krizhevsky, A.: Learning multiple layers of features from tiny images. Tech. rep., University of Toronto (2009), https://w.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf 32. Li, Q., He, B., Song, D.: Practical one-shot federated learning for cross-silo setting. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI). p. 1484–1490 (2021) 33. Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in het- erogeneous networks. In: Proceedings of Machine Learning and Systems (MLSys). vol. 2, p. 429–450 (2020), https://proceedings.mlsys.org/paper/2020/hash/1f5fe83998a09396ebe6477d9475ba0c- Abstract.html 34. Loshchilov, I., Hutter, F.: Sgdr: Stochastic gradient descent with warm restarts. In: International Conference on Learning Representations (ICLR) (2016), https://arxiv.org/abs/1608.03983 35. Ma, J., He, Y., Li, F., Han, L., You, C., Wang, B.: Segment anything in medical images. Nature Communications 15(1), 654 (2024) 36. McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR (2017) 37. Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023) 38. Qiu, L., Annunziata, D., Giampaolo, F., Piccialli, F.: FedSDE: Self-distillation with diffu- sion enhanced for one-shot federated learning. In: 2025 IEEE International Conference on Big Data (BigData) (2025), https://github.com/Lynn0925/FedSDE, iEEE Presentation Video: https://w.computer.org/csdl/video-library/video/2dOxHtFy8UM 39. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language su- pervision. In: International Conference on Machine Learning (2021), https://api.semanticscholar. org/CorpusID:231591445 40. Rieke, N., Hancox, J., Li, W., Milletari, F., Roth, H., Albarqouni, S., Bakas, S., Galtier, M.N., Land- man, B., Maier-Hein, K., Ourselin, S., Sheller, M., Summers, R.M., Trask, A., Xu, D., Baust, M., Cardoso, M.J.: The future of digital health with federated learning. npj Digital Medicine volume 3, Article number: 119 (2020) (2020). https://doi.org/10.1038/s41746-020-00323-1 41. Sheller, M.J., Edwards, B., Reina, G.A., Martin, J., Pati, S., Kotrotsou, A., Milchenko, M., Xu, W., Marcus, D., Colen, R.R., Bakas, S.: Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Scientific Reports 10 (2020). https://doi.org/10. 1038/s41598-020-69250-1 42. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). p. 2818–2826 (2016) 43. du Terrail, J.O., Ayed, S.S., Cyffers, E., Grimberg, F., He, C., Loeb, R., Mangold, P., Marchand, T., Marfoq, O., Mushtaq, E., Muzellec, B., Philippenko, C., Silva, S., Teleńczuk, M., Alatur, S., Berry, A., Dieudonné, A., Michele, M., Gouin, A., Yu, D., Bellet, A., Bach, F., Quellec, G., Lorenzi, M., Dieuleveut, A., Jaggi, M., Karimireddy, S.P., Hartley, M.A., Andreux, M.: Flamby: Datasets and benchmarks for cross-silo federated learning in realistic healthcare settings. In: Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS 2022) Datasets and Benchmarks Track (2022), https://arxiv.org/abs/2210.04620 44. Yan, H., Li, X., Guo, Z., Li, H., Li, F., Lin, X.: ARCANE: An efficient architecture for exact machine unlearning. In: Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI). p. 4006–4013 (2022) 45. Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H., Ni, B.: Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data 10(1), 41 (2023) 46. Yang, M., Su, S., Li, B., Xue, X.: One-shot heterogeneous federated learning with local model-guided diffusion models. In: International Conference on Machine Learning (ICML) (2023), https://arxiv. org/abs/2311.08870 47. Yang, M., Su, S., Li, B., Xue, X.: Feddeo: Description-enhanced one-shot federated learning with diffusion models. In: Proceedings of the 32nd ACM International Conference on Multimedia. p. 6666–6675. M ’24, Association for Computing Machinery, New York, NY, USA (2024). https: //doi.org/10.1145/3664647.3681490, https://doi.org/10.1145/3664647.3681490 48. Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., Finn, C.: Gradient surgery for multi-task learning. In: Advances in Neural Information Processing Systems (NeurIPS). vol. 33 (2020) 49. Zaland, O., Jin, S., Pokorny, F.T., Bhuyan, M.: One-shot federated learning with classifier-free dif- fusion models. In: 2025 IEEE International Conference on Multimedia and Expo (ICME). p. 1–6. IEEE (2025) FederatedFactory14 50. Zhang, S., Metaxas, D.N.: On the challenges and perspectives of foundation models for medical image analysis. Medical Image Analysis 91, 102996 (2024) 51. Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., Chandra, V.: Federated learning with non-iid data. arXiv preprint arXiv:1806.00582 (2018), https://arxiv.org/abs/1806.00582 FederatedFactory15 Appendix Contents A FederatedFactory Pseudocode15 B Qualitative Examples of Diffusion Generated Images15 C Empirical Cumulative Distribution Function and t-SNE Analysis15 D Additional FederatedFactory Performance Analysis17 E Additional FederatedFactory Cost Analysis18 F Image Generation Metrics (FID, KID)19 G Computational Environments20 H Datasets and Sources20 A FederatedFactory Pseudocode For reproducibility, we formalize the complete FederatedFactory operational protocol in Al- gorithm 1. The formulation unites both the Centralized and Decentralized architectures into a single execution graph, controlled by the structural indicator T . B Qualitative Examples of Diffusion Generated Images To visually validate the fidelity and diversity of the FederatedFactory synthesis, we perform a nearest-neighbor analysis across all evaluated domains. For each class, we randomly sample latent vectors z∼N (0,I) to generate synthetic images via the localized EDM2 [29] Factory. We then compute the pixel-wise Euclidean (L2) distance across the local training manifold to find each generated sample’s closest real counterpart. Figures 12, 13, 14, 15, and 16 present these comparisons. The top rows display the synthetic samples, while the bottom rows show their nearest real neighbors with the embedded L2 distances (d). Across both low-resolution standard benchmarks (CIFAR-10) and high-resolution medical modalities (ISIC2019), the generative priors successfully capture the underlying semantic mor- phologies and textures. Importantly, structural differences in poses, backgrounds, and boundaries between the synthetic images and their real matches confirm that the model synthesizes diverse data rather than memorizing the training set. C Empirical Cumulative Distribution Function and t-SNE Analysis We evaluate FederatedFactory’s manifold mapping by projecting generated ( ˆ D syn ) and local training samples (D union ) into 2D via t-SNE. As Fig. 7 shows, synthetic data consistently populates the same macroscopic regions as real data across all datasets. Crucially, despite this global semantic alignment, there is no point-to-point overlap. We further formalize the evaluation of inter-class fidelity and intra-class diversity through Empirical Cumulative Distribution Function (ECDF) curves and nearest-neighbor distance histograms (Fig. 8). – Fidelity (Distance to Real): Quantified by the minimum Euclidean (L2) distance in the feature space from each synthetic sample to its absolute nearest real neighbor (ˆx→ x). – Diversity (Distance to Generated): Measured by the L2 distance strictly among the generated samples themselves (ˆx i → ˆx j ), useful to identify center collapse. FederatedFactory16 Algorithm 1 FederatedFactory: Unified Global Optimization Protocol Require: Network of K clients, private local datasets D k , corresponding local label supports Y k . Require: Architecture T ∈Centralized,Decentralized. Require: Generative epochs E gen , Discriminative epochs E disc , Generation quotas Q k . Require: Untrained generative architectures G θ k K k=1 . Phase I: Local Generative Prior Optimization (Asynchronous) 1: for each client k ∈1,...,K in parallel do 2: for e = 1 to E gen do 3:Sample local empirical batch x∼D k 4:Compute diffusion objective: L ELBO (θ k ;x) ▷Minimizes KL(p k ∥ p θ k ) bounded by ε k 5:Update local generative prior:θ k ←θ k − η∇ θ k L ELBO 6: end for 7: end for Phase I: Conditional Architectural Execution & Discriminative Training 8: if T == Centralized then ▷Trusted Aggregator Available 9: Clients transmit optimized parametersθ k to the Server ▷Satisfies C rounds = 1 10: Server initializes empty global synthetic dataset: ˆ D syn ←∅ 11: for each received model G θ k ∈Θ do 12:for i = 1 to Q k do 13:Sample latent noise vector z∼N(0,I) 14:Synthesize counterfactual mapping ˆ x← G θ k (z) 15: ˆ D syn ← ˆ D syn ∪( ˆ x,y k )▷Ex nihilo synthesis without external FM 16:end for 17: end for 18: Server initializes centralized classifier w 19: Optimize w on ˆ D syn for E disc epochs using standard empirical risk minimization 20: else if T == Decentralized then▷Trustless Decentralized Mesh 21: for each client k ∈1,...,K in parallel do 22:Broadcastθ k to all valid peers j ∈1,...,K\k 23:Receive complement priorsΘ =θ j j̸=k 24:Initialize local hybrid dataset: D mix k ←D k 25:for each received complement model G θ j ∈Θ do 26:Generate Q j samples from G θ j (z) and append mappings to D mix k 27:end for 28:Initialize local expert classifier w k 29:Optimize w k exclusively on D mix k for E disc epochs 30: end for 31: end if Phase I: Distributed Global Inference 32: procedure Inference(Target sample x target ) 33: if T == Centralized then 34:return p(y | x target ;w) 35: else if T == Decentralized then 36:Gather local unnormalized probabilities: p k = f w k (x target ), ∀k ∈1,...,K 37:Compute joint consensus: p joint (y) = Q K k=1 p k (y | x target ) 38:Compute partition function: Z = P y ′ p joint (y ′ ) 39:return Renormalized Product of Experts: p PoE (y | x target ) = 1 Z p joint (y) 40: end if 41: end procedure FederatedFactory17 2D t-SNE CIFARBloodMNISTRetinaMNISTPathMNISTISIC2019 Class 0Class 1Class 2Class 3Class 4Class 5Class 6Class 7Class 8Class 9Real SampleSynthetic Sample Fig. 7: Global Manifold Alignment in 2D t-SNE Subspace. Feature space projections comparing true localized training (D union ,◦) and synthesized global ( ˆ D syn ,△) distributions across all five benchmarks. 7.510.012.515.017.5 Dist to Real 0.0 0.2 0.4 0.6 0.8 1.0 Fidelity (ECDF) CIFAR 6810121416 Dist to Real 0.0 0.2 0.4 0.6 0.8 1.0 BloodMNIST 681012 Dist to Real 0.0 0.2 0.4 0.6 0.8 1.0 RetinaMNIST 51015 Dist to Real 0.0 0.2 0.4 0.6 0.8 1.0 PathMNIST 5101520 Dist to Real 0.0 0.2 0.4 0.6 0.8 1.0 ISIC2019 7.510.012.515.017.5 Dist to Real Fidelity (Dist) 6810121416 Dist to Real 681012 Dist to Real 51015 Dist to Real 5101520 Dist to Real 7.510.012.515.017.520.0 Dist to Gen 0.0 0.2 0.4 0.6 0.8 1.0 Diversity (ECDF) 51015 Dist to Gen 0.0 0.2 0.4 0.6 0.8 1.0 0246810 Dist to Gen 0.0 0.2 0.4 0.6 0.8 1.0 051015 Dist to Gen 0.0 0.2 0.4 0.6 0.8 1.0 5.07.510.012.515.0 Dist to Gen 0.0 0.2 0.4 0.6 0.8 1.0 7.510.012.515.017.520.0 Dist to Gen Diversity (Dist) 51015 Dist to Gen 0246810 Dist to Gen 051015 Dist to Gen 5.07.510.012.515.0 Dist to Gen Class 0Class 1Class 2Class 3Class 4Class 5Class 6Class 7Class 8Class 9 Fig. 8: Quantitative Manifold Alignment via ECDF and Density Histograms. Evaluation of inter-class Fidelity (top rows: L2 distance from synthetic samples to their nearest real neighbors) and intra-class Diversity (bottom rows: L2 distance strictly among synthetic samples) across five benchmarks. Steep Fidelity ECDF curves confirm that synthetic distributions tightly bind to the real data manifold with minimal Out-of-Distribution (OOD) deviation. Simultaneously, diversity ECDFs and density histograms shows how the generative factories maintain broad continuous support without mode collapse. These curves also reflect intrinsic domain variances: tightly grouped dis- tributions in RetinaMNIST [45] mirror the low structural variance of retinal crops, while broader ECDFs in ISIC2019 [43] and CIFAR [31] capture their high visual heterogeneity. Ultimately, these metrics mathematically prove FederatedFactory captures the true underlying data support de- spite extreme single-class label skew. D Additional FederatedFactory Performance Analysis To illustrate the catastrophic failure of standard parameter aggregation [36] and our framework’s subsequent recovery, Fig. 9 compares performance magnitudes across all five benchmarks. Under the pathological Single-Class Silo regime, severe gradient interference causes baselines (FedAvg FederatedFactory18 0 20 40 60 80 100 Accuracy (%) CIFAR +72% BloodMNIST +69% RetinaMNIST +6% PathMNIST +37% ISIC2019 +22% 0 20 40 60 80 100 AUROC (%) +32% +31% +23% +29% +43% FedAvgFedDynFedProxScaffoldFederatedFactory (Centralized)FederatedFactory (Decentralized)Data-Pooled Upper Bound Fig. 9: Absolute Performance Recovery under Single-Class Silos. Grouped bar chart detailing the accuracy and AUROC across the evaluated datasets. The gray shaded region represents the col- lapsed performance ceiling of standard iterative federated learning methods. FederatedFactory (in both Centralized andDecentralized modes) successfully escapes this collapsed regime, yielding im- provements (e.g., matching theCentralized Upper Bound ). [36], FedDyn [1], FedProx [33], SCAFFOLD [27]) to collapse into near-random or majority-class predictions [48]. The barplot visualizes the ∆ improvement (arrows) from the best iterative baseline to FederatedFactory. Ultimately, our zero-dependency approach completely bridges this gap, restoring predictive metrics to the theoretical data-pooled upper bound. E Additional FederatedFactory Cost Analysis FederatedFactory transitions from a bandwidth-bound optimization regime to a compute- bound one in order to achieve higher model trustworthiness. We visualize this “Compute-for- Bandwidth Swap” via log-log scatter plot in Fig. 10. Standard FL [1,27,33,36] minimizes local compute (∼ 10 16 –10 17 FLOPs) but requires massive iterative communication (> 10 5 MB). Conversely, FederatedFactory employs a One-Shot FL (OSFL) protocol, trading a “generative tax” in local compute (∼ 10 18 –10 20 FLOPs) for a > 99.4% reduction in network payload (∼ 10 3 MB). This compute-for-bandwidth trade-off is ideal for data-siloed clinical consortiums possessing abundant local compute. 10 3 10 4 10 5 10 6 Communication (MB) 10 15 10 16 10 17 10 18 10 19 10 20 10 21 Total Computations (FLOPs) CIFAR 10 3 10 4 10 5 10 6 Communication (MB) BloodMNIST 10 3 10 4 10 5 10 6 Communication (MB) RetinaMNIST 10 3 10 4 10 5 10 6 Communication (MB) PathMNIST 10 3 10 4 10 5 10 6 Communication (MB) ISIC2019 FedAvgFedDynFedProxScaffoldFederatedFactory (Centralized)FederatedFactory (Decentralized) Fig. 10: The Compute-for-Bandwidth Swap. Log-log scatter comparing computational overhead versus network exposure. Iterative baselines (FedAvg ◦ ,FedDyn □ ,FedProx △ ,Scaffold ⋄ ) cluster bottom-right (low compute, massive bandwidth). Conversely, FederatedFactory (Centralized + , Decentralized ⋆ ) shifts to the top-left, accepting higher local FLOPs to achieve higher trustworthiness. FederatedFactory19 F Image Generation Metrics To quantify synthesized data quality, fidelity, and diversity, we evaluate generated ( ˆ D syn ) against target distributions (D union ) using Fréchet Inception Distance (FID) [18] and Kernel Inception Distance (KID) [3]. These project images into the Inception-V3 activation space [42] to measure statistical divergence between real and synthetic manifolds, improving upon naive pixel-space distances. As Tab. 3 and Fig. 11 illustrates, per-class FID and KID distributions confirm Fed- eratedFactory captures coherent semantic structures. On natural domains (CIFAR-10 [31]), per-class FID averages ∼28.8, with KID reliably < 0.02 (×100). Similarly, high-resolution der- matoscopic lesions (ISIC2019 [43]) maintain strong manifold alignment with median FIDs ∼45 and tightly grouped KIDs. Benchmarks with complex, sparse cellular morphologies (PathMNIST, BloodMNIST [45]) exhibit slightly higher variances. Table 3: Per-Class Generative Alignment Metrics. Quantitative evaluation of the synthesized images via FID and KID (scaled by 10 2 ). Lower values indicate better manifold alignment with the real target distribution. Values are rounded to two decimal places for readability. For CIFAR, classes 0–9 correspond to airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck, respectively. DatasetMetric Class 0123456789 CIFAR FID (↓) 29.07 22.50 27.28 46.67 22.10 35.40 41.02 15.01 19.32 25.75 KID (↓) 1.71 1.38 1.59 3.48 1.50 2.16 3.09 0.58 1.00 1.85 BloodMNIST FID (↓) 84.01 61.54 45.08 50.37 60.58 66.42 50.78 27.19 – KID (↓) 10.76 8.83 4.63 6.33 7.54 7.69 6.76 2.97– RetinaMNIST FID (↓) 38.49 58.40 38.77 44.09 58.88 – KID (↓) 3.52 3.03 2.85 2.90 3.91– PathMNIST FID (↓) 38.20 33.85 77.25 103.85 57.24 64.21 64.68 79.69 70.86 – KID (↓) 3.83 2.92 8.95 14.33 6.75 7.01 7.63 10.43 8.94– ISIC2019 FID (↓) 38.80 41.18 30.01 40.91 45.54 76.67 82.39 55.55 – KID (↓) 1.59 2.18 1.46 1.23 1.90 1.85 1.76 1.92– CIFARBloodMNISTRetinaMNISTPathMNISTISIC2019 20 40 60 80 100 FID Score Per-Class Distribution of Image Fidelity CIFARBloodMNISTRetinaMNISTPathMNISTISIC2019 0 2 4 6 8 10 12 14 KID (×100) Per-Class Distribution of Kernel Inception Distance Density (Violin)Individual Class (Strip)Median (Box) Fig. 11: Quantitative Manifold Alignment via FID and KID. Per-class distributions of FID (left) and KID (right). The raincloud plots visualize the density, raw data points, and median scores. FederatedFactory20 These quantitative metrics represent a strict lower bound on the framework’s synthesis capa- bility. To maintain computational tractability on distributed nodes, we deployed a heavily down- scaled EDM2 [29] architecture (128 embedding dimension, 32 sampling steps) and faced severe long-tail class imbalances (e.g., ISIC2019). Despite these sub-optimal conditions, Federated- Factory synthesizes counterfactuals matching centralized upper-bound classification accuracy (Sec. 5). Because global discriminative performance is mathematically bounded by local gener- ative error (Theorem 1), achieving robust downstream classification with non-ideal generators guarantees significant upside. As local compute scales, enabling larger foundation diffusion models and extended training, ex nihilo synthesis fidelity will proportionally improve, yielding an even stronger global decision boundary. G Computational Environments The core system is equipped with 2× NVIDIA H100 and 4× NVIDIA L40 GPUs. – NVIDIA H100 (80GB VRAM): The two H100 GPUs were exclusively dedicated to train- ing the localized generative priors, specifically the EDM2 diffusion models [28,29]. – NVIDIA L40 (48GB VRAM): The four L40 GPUs were parallelized to handle server-side image synthesis and downstream evaluations. Specifically, they were utilized to synthesize the globally class-balanced datasets ex nihilo, and for training the global discriminative classifiers, i.e. the adaptive ResNet-50 [17] backbone. H Datasets and Sources We gratefully acknowledge the creators, curators, and institutions behind the public datasets used to evaluate FederatedFactory. The empirical validation of our method relies entirely on the public availability of these benchmarks (CIFAR-10 [31], MedMNIST [45], ISIC2019 [43]) FederatedFactory21 Class 0 Synthetic Real d = 8.7d = 6.8d = 7.4d = 7.9d = 13.1d = 5.2d = 6.7d = 7.8d = 8.3d = 10.8 Class 1 Synthetic Real d = 15.7d = 10.1d = 10.3d = 13.2d = 11.5d = 12.9d = 13.3d = 10.9d = 10.3d = 10.6 Class 2 Synthetic Real d = 5.3d = 7.5d = 18.0d = 12.1d = 7.4d = 8.7d = 8.9d = 5.5d = 8.4d = 6.0 Class 3 Synthetic Real d = 7.0d = 8.2d = 12.4d = 10.3d = 11.4d = 6.1d = 7.9d = 10.6d = 14.0d = 7.0 Class 4 Synthetic Real d = 6.7d = 5.7d = 7.0d = 7.7d = 8.0d = 7.6d = 7.5d = 6.3d = 11.9d = 7.2 Class 5 Synthetic Real d = 6.5d = 8.5d = 7.4d = 7.0d = 8.3d = 7.8d = 6.5d = 7.1d = 6.9d = 8.0 Class 6 Synthetic Real d = 7.4d = 6.9d = 9.0d = 12.2d = 8.6d = 12.6d = 9.8d = 4.6d = 8.3d = 7.9 Class 7 Synthetic Real d = 10.6d = 9.4d = 7.5d = 12.2d = 7.6d = 9.3d = 9.3d = 8.5d = 10.3d = 9.1 Class 8 Synthetic Real d = 7.4d = 6.5d = 5.6d = 9.5d = 6.1d = 9.5d = 4.2d = 9.6d = 8.7d = 10.4 Class 9 Synthetic Real d = 10.8d = 8.9d = 10.6d = 9.0d = 10.7d = 9.4d = 9.5d = 14.8d = 9.9d = 12.3 Fig. 12: Nearest Neighbor Analysis on CIFAR-10. Synthetic samples (top rows) paired with their closest real counterparts (bottom rows). Class 9 has been omitted to accommodate layout constraints. FederatedFactory22 Class 0 Synthetic Real d = 6.8d = 5.8d = 6.8d = 7.1d = 6.8d = 6.2d = 6.6d = 6.3d = 5.4d = 5.5 Class 1 Synthetic Real d = 6.2d = 6.8d = 4.9d = 6.1d = 8.6d = 5.6d = 5.1d = 8.4d = 7.1d = 4.8 Class 2 Synthetic Real d = 5.6d = 5.6d = 5.0d = 8.7d = 5.1d = 7.4d = 5.2d = 5.4d = 7.4d = 4.9 Class 3 Synthetic Real d = 5.9d = 5.1d = 6.1d = 5.6d = 7.9d = 5.9d = 6.9d = 6.1d = 7.7d = 6.4 Class 4 Synthetic Real d = 5.6d = 5.7d = 6.0d = 5.5d = 6.7d = 5.2d = 7.0d = 7.5d = 5.4d = 5.9 Class 5 Synthetic Real d = 7.3d = 7.8d = 6.5d = 6.1d = 5.8d = 6.1d = 5.2d = 7.1d = 6.1d = 6.7 Class 6 Synthetic Real d = 6.1d = 4.5d = 6.4d = 5.2d = 6.5d = 5.8d = 6.2d = 6.4d = 6.6d = 5.5 Class 7 Synthetic Real d = 7.0d = 4.9d = 5.6d = 3.6d = 3.6d = 4.9d = 4.1d = 5.1d = 5.7d = 3.9 Fig. 13: Nearest Neighbor Analysis on BloodMNIST. The localized Factories successfully capture the distinct morphological features, shapes, and textures of different blood cell types. FederatedFactory23 Class 0 Synthetic Real d = 2.4d = 6.3d = 0.9d = 2.9d = 0.3d = 3.9d = 3.4d = 3.9d = 4.8d = 4.4 Class 1 Synthetic Real d = 0.8d = 3.0d = 0.8d = 4.4d = 0.8d = 0.7d = 0.6d = 1.2d = 0.8d = 2.1 Class 2 Synthetic Real d = 1.2d = 4.6d = 3.1d = 3.8d = 4.1d = 6.3d = 7.8d = 5.8d = 3.4d = 6.1 Class 3 Synthetic Real d = 5.2d = 4.5d = 3.2d = 3.9d = 3.7d = 5.3d = 4.7d = 6.3d = 5.2d = 4.1 Class 4 Synthetic Real d = 5.5d = 7.1d = 6.4d = 4.8d = 5.2d = 4.1d = 5.3d = 5.8d = 6.1d = 5.5 Class 5 Synthetic Real d = 6.1d = 4.6d = 4.3d = 5.3d = 4.2d = 3.4d = 3.2d = 3.4d = 5.0d = 3.5 Class 6 Synthetic Real d = 5.5d = 6.8d = 6.3d = 5.9d = 7.6d = 6.7d = 7.3d = 7.2d = 5.5d = 7.0 Class 7 Synthetic Real d = 3.5d = 3.9d = 3.6d = 3.0d = 3.0d = 3.6d = 3.1d = 3.2d = 2.3d = 4.9 Class 8 Synthetic Real d = 7.4d = 8.4d = 4.2d = 6.1d = 6.3d = 4.3d = 8.1d = 4.2d = 6.5d = 4.1 Fig. 14: Nearest Neighbor Analysis on PathMNIST. Synthetic histological patches alongside their closest real training samples. FederatedFactory24 Class 0 Synthetic Real d = 7.3d = 4.4d = 6.7d = 6.4d = 7.6d = 4.6d = 6.2d = 4.6d = 9.0d = 5.9 Class 1 Synthetic Real d = 4.8d = 6.3d = 5.4d = 3.6d = 10.9d = 3.4d = 4.3d = 10.9d = 5.3d = 4.0 Class 2 Synthetic Real d = 3.7d = 5.2d = 6.5d = 4.7d = 3.5d = 5.5d = 7.7d = 3.7d = 6.8d = 5.5 Class 3 Synthetic Real d = 6.0d = 5.6d = 3.0d = 4.9d = 3.9d = 4.5d = 8.5d = 4.0d = 6.4d = 5.0 Class 4 Synthetic Real d = 6.7d = 5.4d = 6.4d = 4.3d = 7.9d = 5.5d = 6.0d = 5.6d = 3.1d = 3.5 Fig. 15: Nearest Neighbor Analysis on RetinaMNIST. Despite the intrinsically low structural variance of retinal fundus crops, the generative models maintain sufficient diversity and do not trivially memorize the limited training support. FederatedFactory25 Class 0 Synthetic Real d = 28.8d = 54.3d = 10.1d = 23.0d = 58.2d = 31.8d = 27.9d = 40.4d = 15.0d = 20.4 Class 1 Synthetic Real d = 10.8d = 13.7d = 18.6d = 14.9d = 20.3d = 23.5d = 10.9d = 20.2d = 56.8d = 13.5 Class 2 Synthetic Real d = 12.4d = 25.3d = 17.3d = 12.8d = 12.7d = 14.5d = 12.1d = 13.3d = 14.6d = 14.3 Class 3 Synthetic Real d = 28.5d = 20.4d = 14.0d = 7.4d = 19.5d = 28.3d = 16.0d = 16.1d = 13.4d = 16.3 Class 4 Synthetic Real d = 11.4d = 17.2d = 10.8d = 13.9d = 15.6d = 22.3d = 50.2d = 16.0d = 18.4d = 13.9 Class 5 Synthetic Real d = 28.0d = 18.2d = 10.3d = 17.2d = 63.4d = 28.9d = 18.5d = 18.3d = 16.4d = 19.0 Class 6 Synthetic Real d = 57.0d = 23.9d = 19.9d = 19.5d = 32.7d = 28.1d = 17.5d = 65.8d = 33.2d = 21.9 Class 7 Synthetic Real d = 20.4d = 13.9d = 47.6d = 34.7d = 15.1d = 26.0d = 22.3d = 25.8d = 27.5d = 65.9 Fig. 16: Nearest Neighbor Analysis on ISIC2019. High-resolution dermatoscopic lesions are syn- thesized with high fidelity.