Paper deep dive
Compositional Cross-Modality Translation via Whole-Volume Multitask Latent Flow Matching
Daniele Molino, Alessio Zoboli, Camillo Maria Caruso, Valerio Guarrasi, Paolo Soda
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/11/2026, 5:20:16 AM
Summary
The paper proposes a unified multitask framework for cross-modality medical image translation using whole-volume latent flow matching. By leveraging a pretrained 3D variational autoencoder (VAE) to compress volumetric data into a latent space, the method decouples appearance learning from translation mapping. A conditional flow-matching model is trained jointly across multiple modalities (MRI, CT, CBCT) and anatomical regions, enabling zero-shot generalization and compositional translation (e.g., MRI to CT via intermediate steps) that outperforms patch-based and task-specific baselines.
Entities (14)
Relation Signals (9)
FM-MT Whole-Vol. → enables → Compositional Translation
confidence 95% · Crucially, joint training unlocks capabilities inaccessible to task-specific approaches: ... compositional cross-dataset translation along paths never directly supervised.
FM-MT Whole-Vol. → enables → Zero-Shot Generalization
confidence 95% · Crucially, joint training unlocks capabilities inaccessible to task-specific approaches: zero-shot generalization to anatomical regions unseen during training.
FM-MT Whole-Vol. → trainedon → SynthRAD2025
confidence 95% · We train a single model jointly across inter-modality ... tasks over three multi-center datasets.
FM-MT Whole-Vol. → trainedon → SynthRAD2023
confidence 95% · We train a single model jointly across inter-modality ... tasks over three multi-center datasets.
FM-MT Whole-Vol. → trainedon → BraTS2023
confidence 95% · We train a single model jointly across inter-modality ... tasks over three multi-center datasets.
FM-MT Whole-Vol. → uses → Flow Matching
confidence 95% · Translation is then learned as a conditional flow-matching problem in this latent space.
FM-MT Whole-Vol. → outperforms → FM-ST Patch
confidence 90% · FM-ST Whole-Vol. outperforms FM-ST Patch, with gains in SSIM ranging from +0.12 to +0.37.
→ →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Cross-modality medical image translation can reduce the burden of multi-modal acquisitions, yet the field remains constrained by two coupled limitations: methods operate on 2D slices or 3D patches rather than whole volumes, and train a separate model for each translation task. Both stem from a single cause, the absence of a sufficiently strong volumetric prior, which forces generative models to learn anatomical appearance and cross-modality mapping simultaneously, an ill-posed problem at the scale of available paired datasets. We propose to decouple these objectives. A large-scale pretrained 3D variational autoencoder provides a compact latent representation of volumetric appearance, reducing translation to a conditional flow-matching problem. This compression makes whole-volume processing tractable, while a resolution-aware sampling strategy preserves native anatomical scale. We train a single model jointly across inter-modality (MRI$\to$CT, CBCT$\to$CT) and intra-modality (MRI$\to$MRI) tasks over three multi-center datasets. Across all tasks, whole-volume processing outperforms its patch-based counterpart, and the multi-task model matches task-specific baselines while replacing $N$ networks with one. Crucially, joint training unlocks capabilities inaccessible to task-specific approaches: zero-shot generalization to anatomical regions unseen during training, within 0.15 SSIM of the fully supervised model, and compositional cross-dataset translation along paths never directly supervised. These results suggest that combining a strong volumetric prior with multitask training is a scalable route toward synthesis systems that generalize beyond their training distribution. Code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.08135v1
- Canonical: https://arxiv.org/abs/2608.08135v1
Trouble viewing inline? Open PDF directly →
Full Text
31,191 characters extracted from source content.
Expand or collapse full text
11institutetext: Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma, Rome, Italy 11email: daniele.molino@unicampus.it, alessio.zoboli@alcampus.it, camillomaria.caruso@unicampus.it, p.soda@unicampus.it 22institutetext: Department of Diagnostics and Intervention, Biomedical Engineering and Radiation Physics, Umeå University, Umeå, Sweden 22email: paolo.soda@umu.se 33institutetext: UniCamillus – Saint Camillus International University of Health Sciences, Rome, Italy 33email: valerio.guarrasi@unicamillus.org Compositional Cross-Modality Translation via Whole-Volume Multitask Latent Flow Matching Daniele Molino Corresponding author. Alessio Zoboli Camillo Maria Caruso Valerio Guarrasi Paolo Soda Abstract Cross-modality medical image translation can reduce the burden of multi-modal acquisitions, yet the field remains constrained by two coupled limitations: methods operate on 2D slices or 3D patches rather than whole volumes, and train a separate model for each translation task. Both stem from a single cause, the absence of a sufficiently strong volumetric prior, which forces generative models to learn anatomical appearance and cross-modality mapping simultaneously, an ill-posed problem at the scale of available paired datasets. We propose to decouple these objectives. A large-scale pretrained 3D variational autoencoder provides a compact latent representation of volumetric appearance, reducing translation to a conditional flow-matching problem. This compression makes whole-volume processing tractable, while a resolution-aware sampling strategy preserves native anatomical scale. We train a single model jointly across inter-modality (MRI→ , CBCT→ ) and intra-modality (MRI→ ) tasks over three multi-center datasets. Across all tasks, whole-volume processing outperforms its patch-based counterpart, and the multi-task model matches task-specific baselines while replacing N networks with one. Crucially, joint training unlocks capabilities inaccessible to task-specific approaches: zero-shot generalization to anatomical regions unseen during training, within 0.15 SSIM of the fully supervised model, and compositional cross-dataset translation along paths never directly supervised. These results suggest that combining a strong volumetric prior with multitask training is a scalable route toward synthesis systems that generalize beyond their training distribution. Code is available at https://github.com/arco-group/Whole-Volume-Latent-FM. 1 Introduction Cross-modality medical image translation has emerged as a clinically relevant paradigm for reducing the burden of multi-modal acquisitions. By synthesizing a target modality directly from an available source, it enables applications ranging from MR-only radiotherapy planning [23] to scanner harmonization and data augmentation under strict privacy constraints [3, 2, 4]. The growing clinical interest in this problem has catalyzed dedicated challenges and benchmarks. SynthRad2023 and SynthRad2025 [23, 24] provide multi-center paired datasets spanning five anatomical regions, covering both magnetic resonance imaging (MRI)–computed tomography (CT) and cone-beam CT (CBCT)–CT translation tasks. BraTS2023 [1] complements these with multi-parametric MRI acquisitions for brain tumour cases, enabling intra-modality translation across four MRI sequences. Yet paired datasets remain scarce relative to the scale of modern deep learning: matched multi-modal acquisitions require coordinated imaging protocols and expose patients to additional procedures, while data-sharing regulations strictly limit cross-institutional pooling [12, 13]. Despite progress from GAN-based approaches [6, 11, 25] to diffusion and flow matching models [9, 14, 20], most medical image translation methods still operate on 2D slices or 3D patches [21]. This is largely driven by the cubic memory and compute cost of full-volume generation [19]. However, patch-based models lack global anatomical context, may introduce stitching artifacts [15, 16], and often require resampling to a common voxel grid, thereby altering native scale relationships. Additionally, most approaches train one model per translation task, limiting representation sharing across modalities, anatomical regions, and datasets. Since each model is trained on a single narrow distribution, it cannot exploit the regularities shared across modalities, anatomical regions, and acquisition centers [10, 5]. Furthermore, the task-specific formulation structurally precludes generalization to modality compositions absent from any individual training set, a capability that only emerges from joint training over shared representations [17, 18]. We argue that these limitations stem not from the translation task itself but from the absence of a strong volumetric prior: when a model must simultaneously learn how medical volumes look and how to translate between them, the problem is ill-posed at the scale of available paired datasets. We address these limitations by decoupling volumetric representation learning from cross-modality mapping. A pretrained 3D VAE [7], trained on over 55,000 CT and MRI volumes, maps whole scans to a compact latent space. Translation is then learned as a conditional flow-matching problem in this latent space, making whole-volume processing tractable. On top of this representation, we train a conditional flow-matching model [14] jointly across all available modalities, anatomical regions, and datasets, using a resolution-aware bucketed sampling strategy that preserves each volume’s native resolution without resampling. Our contributions are threefold. (i) We show that whole-volume latent flow matching outperforms its patch-based counterpart, empirically validating that global context improves translation quality. (i) We train a single multitask model across MRI→ , CBCT→ , and MRI→ translation, matching task-specific baselines without significant degradation. (i) We demonstrate that joint training across datasets unlocks capabilities structurally inaccessible to task-specific approaches, such as zero-shot generalization to anatomical regions unseen during training, and compositional translation across modalities and datasets never co-observed during training. Figure 1: Overview of the proposed multitask whole-volume framework. Native-resolution volumes are processed through bucketed sampling and encoded by a frozen pretrained 3D VAE into a compact latent space. A target-modality token conditions the latent flow-matching model, enabling both direct cross-modality translation and compositional inference through chained translation steps. 2 Method The proposed framework comprises two stages, latent compression and latent translation, reflecting a deliberate decoupling of volumetric representation learning from cross-modality mapping. An overview is provided in Fig. 1. Volumetric latent compression. We adopt the frozen encoder ℰ(⋅)E(·) of MAISI [7], a 3D VAE pretrained on over 55,000 heterogeneous CT and MRI volumes. Given an input volume ∈ℝH×W×Dx ^H× W× D, the encoder produces a compact latent representation via the reparameterization trick: =ϕ()+ϕ()⊙ϵ,ϵ∼(,),∈ℝh×w×d×c,z= μ_φ(x)+ σ_φ(x) ε, ε (0,I), ^h× w× d× c, (1) with h≪Hh H, w≪Ww W, d≪Dd D. The decoder (⋅)D(·) defines the inverse mapping ~=() x=D(z). Both are kept frozen, confining translation to a latent space that already encodes the complexity of medical volumes. Conditional latent translation via flow matching. Given source and target latents src=ℰ(src),tar=ℰ(tar)z^src=E(x^src),z^tar=E(x^tar), we define a probability path via linear interpolation: t=(1−t)tar+tϵ,ϵ∼(,),t∈[0,1],z_t=(1-t)\,z^tar+t\, ε, ε (0,I), t∈[0,1], (2) with target velocity field ut=ϵ−taru_t= ε-z^tar. A 3D U-Net [22] θ(⋅)U_θ(·) is trained to approximate this field conditionally on srcz^src and a target modality token m∈ℳm , where ℳM is the set of modalities spanned by the training data; the source latent is injected via channel-wise concatenation, while m is projected through a learnable embedding and combined with the timestep encoding. No source-modality token is used, as the source modality is already specified by srcz^src, while the target token disambiguates the modality to be synthesized. The model is optimized by minimizing the flow matching objective: ℒFM=(zsrc,ztar),ϵ,t[‖vθ(t,t,src,m)−(ϵ−tar)‖1].L_FM=E_(z^src,z^tar),ε,t [ \|v_θ(z_t,t,z^src,m)-( ε-z^tar) \|_1 ]. (3) At inference, the source volume is encoded to srcz^src, the target latent is initialized as 1∼(,)z_1 (0,I), and the learned ODE d/dt=vθ(t,t,src,m)dz/dt=v_θ(z_t,t,z^src,m) is integrated from t=1t=1 to t=0t=0, yielding ^tar z^tar. Because the model is conditioned on an explicit target modality token, it supports compositional inference: given a path A→B→CA→ B→ C with no direct A→CA→ C supervision, we first synthesize ^B z^B from Az^A conditioned on token B, then synthesize ^C z^C from ^B z^B conditioned on token C, decoding only the final output. Native-resolution training via bucketed sampling. Whole-volume training requires handling heterogeneous spatial dimensions without resampling all scans to a single canonical grid. To address this, we use a bucketed sampling strategy. Each volume is cropped or zero-padded so that its spatial dimensions are multiples of 128. Volumes with the same resulting spatial dimensions are assigned to the same bucket, so that each bucket contains only samples with identical tensor shape. During training, each mini-batch is drawn from a single bucket. This guarantees shape consistency within the batch and avoids any additional on-the-fly padding or interpolation. Since volumes are not resampled, the native voxel spacing of each acquisition is preserved; the bucketed strategy only standardizes tensor extents through cropping and zero-padding. As a result, the model can process whole volumes at their native physical resolution while still supporting heterogeneous anatomical regions, modalities, and source datasets across training batches. 3 Experiments Datasets. We evaluate on three public paired datasets spanning inter- and intra-modality translation across heterogeneous anatomical regions and acquisition conditions. SynthRad2023 [23] provides brain and pelvis T1w–CT and CBCT–CT pairs, with 360 pairs per task. SynthRad2025 [24] extends the same tasks to head-and-neck, thorax, and abdomen, with 578 T1w–CT and 579 CBCT–CT pairs. BraTS2023 [1] provides 1,251 co-registered brain MRI cases with T1w, T1c, T2w, and T2f sequences, enabling all pairwise MRI translations. All datasets are split 75/25 into training and test sets. The shared modality vocabulary is ℳ=CT, T1w, T1c, T2w, T2fM=\CT, T1w, T1c, T2w, T2f\, with SynthRad T1 and BraTS native T1 assigned to the same T1w token. Preprocessing. All volumes are processed as whole 3D scans. Before encoding, each volume is cropped to the patient foreground, thereby removing external background, and subsequently zero-padded so that its spatial dimensions are multiples of 128, up to a maximum size of 512×512×256512× 512× 256 voxels. CT and CBCT volumes are intensity-clipped to [−1024,3000][-1024,3000] HU and linearly mapped to [0,1][0,1]. MRI volumes are normalised by the 99.5th percentile of each volume. Experimental configuration. We compare task-specific FM-ST Whole-Vol. models, trained independently for each translation task, with a unified FM-MT Whole-Vol. model trained jointly on SynthRad and BraTS. For external comparison, we include SRGAN [8], the strongest baseline reported in the benchmark of Romoli et al. [21] and patch-based flow matching (FM-ST Patch), which processes 96396^3 patches and reconstructs full volumes by sliding-window inference with 0.625 overlap and Gaussian blending (σ scale =0.125=0.125), thus isolating the contribution of whole-volume latent processing. Results are taken from [21] when available; otherwise, patch-based models are retrained under the same protocol. Evaluation protocol. Translation quality is assessed via Structural Similarity Index (SSIM, ↑ ) and Peak Signal-to-Noise Ratio (PSNR, ↑ ), computed over the foreground region defined by the patient-outline mask. Statistical significance is assessed via Wilcoxon signed-rank tests with Holm correction. Implementation details. The generative U-Net comprises four resolution levels with channel widths 64, 128, 256, and 512, two residual blocks per level, and self-attention at the two deepest scales. Training runs for 200 epochs with the Adam optimizer (lr=10−4=10^-4, polynomial decay, power 2), batch size 4, and mixed precision. The rectified flow scheduler uses 1,000 training timesteps and 30 inference steps. As an indicative upper bound on achievable fidelity, encoding and decoding each volume through the frozen MAISI autoencoder yields PSNR 39.93 dB and SSIM 0.972. All experiments were conducted on a single NVIDIA A40 GPU. 4 Results Table 1: PSNR (dB) and SSIM results across datasets, tasks, and models (mean ± std). Bold: best; underlined: second best. ∗ indicates a significant improvement of FM-MT over FM-ST Whole-Vol. (Wilcoxon signed-rank test with Holm correction, p<0.05p<0.05); all other FM-MT results are not significantly worse than FM-ST Whole-Vol. † indicates model retrained under the protocol of [21]. Metric Dataset Task region SRGAN FM-ST Patch FM-ST Whole-Vol. FM-MT Whole-Vol. PSNR SynthRad23 T1w→ Brain 27.48±1.4727.48± 1.47 24.82±0.8324.82± 0.83 29.35±1.05¯ 29.35± 1.05 29.56±1.23∗29.56± 1.23^* SynthRad23 T1w→ Pelvis 29.15±1.6429.15± 1.64 25.94±1.3725.94± 1.37 28.74±1.97¯ 28.74± 1.97 28.69±1.4528.69± 1.45 SynthRad23 CBCT→ Brain 28.88±2.0528.88± 2.05 25.68±1.8425.68± 1.84 30.82±2.28¯ 30.82± 2.28 30.85±2.3630.85± 2.36 SynthRad23 CBCT→ Pelvis 29.83±2.2929.83± 2.29 24.78±1.2124.78± 1.21 30.59±2.17¯ 30.59± 2.17 30.80±2.2730.80± 2.27 SynthRad25 T1w→ Head-Neck 26.32±3.0126.32± 3.01 21.42±1.4521.42± 1.45 28.22±1.4528.22± 1.45 28.09±0.75¯ 28.09± 0.75 SynthRad25 T1w→ Thorax 25.74±1.5125.74± 1.51 21.83±1.8221.83± 1.82 26.79±1.8926.79± 1.89 26.76±1.63¯ 26.76± 1.63 SynthRad25 T1w→ Abdomen 25.91±2.18†25.91± 2.18 21.67±1.73†21.67± 1.73 27.26±2.59¯ 27.26± 2.59 27.43±2.31∗27.43± 2.31^* SynthRad25 CBCT→ Head-Neck 29.50±1.7029.50± 1.70 26.55±1.9226.55± 1.92 30.91±1.21¯ 30.91± 1.21 31.22±1.35∗31.22± 1.35^* SynthRad25 CBCT→ Thorax 28.07±1.98†28.07± 1.98 24.90±2.12†24.90± 2.12 30.57±1.6930.57± 1.69 30.56±1.79¯ 30.56± 1.79 SynthRad25 CBCT→ Abdomen 28.35±2.10†28.35± 2.10 25.20±2.15†25.20± 2.15 30.82±1.85¯ 30.82± 1.85 31.05±1.92∗31.05± 1.92^* BraTS23 T1w→ 1c Brain 23.84±2.71†23.84± 2.71 22.46±2.38†22.46± 2.38 24.96±2.33¯ 24.96± 2.33 26.51±3.42∗26.51± 3.42^* BraTS23 T1c→ 1w Brain 28.74±4.36†28.74± 4.36 26.91±3.89†26.91± 3.89 30.02±5.2330.02± 5.23 29.86±4.89¯ 29.86± 4.89 BraTS23 T1w→ 2w Brain 25.96±3.01†25.96± 3.01 24.31±2.76†24.31± 2.76 27.45±3.2227.45± 3.22 27.31±3.25¯ 27.31± 3.25 BraTS23 T2w→ 1w Brain 27.42±4.12†27.42± 4.12 25.85±3.67†25.85± 3.67 29.07±4.8729.07± 4.87 28.79±4.83¯ 28.79± 4.83 BraTS23 T2w→ 2f Brain 25.28±2.9025.28± 2.90 23.55±2.5623.55± 2.56 28.92±3.17¯ 28.92± 3.17 29.12±2.9429.12± 2.94 BraTS23 T2f→ 2w Brain 25.41±3.08†25.41± 3.08 23.97±2.74†23.97± 2.74 28.29±3.3428.29± 3.34 27.96±3.41¯ 27.96± 3.41 SSIM SynthRad23 T1w→ Brain 0.84±0.040.84± 0.04 0.77±0.040.77± 0.04 0.86±0.02¯ 0.86± 0.02 0.89±0.02∗0.89± 0.02^* SynthRad23 T1w→ Pelvis 0.88±0.020.88± 0.02 0.73±0.050.73± 0.05 0.86±0.04¯ 0.86± 0.04 0.86±0.03¯ 0.86± 0.03 SynthRad23 CBCT→ Brain 0.89±0.040.89± 0.04 0.78±0.090.78± 0.09 0.91±0.040.91± 0.04 0.91±0.040.91± 0.04 SynthRad23 CBCT→ Pelvis 0.88±0.030.88± 0.03 0.76±0.050.76± 0.05 0.90±0.040.90± 0.04 0.90±0.040.90± 0.04 SynthRad25 T1w→ Head-Neck 0.76±0.080.76± 0.08 0.53±0.050.53± 0.05 0.86±0.030.86± 0.03 0.85±0.02¯ 0.85± 0.02 SynthRad25 T1w→ Thorax 0.73±0.050.73± 0.05 0.56±0.060.56± 0.06 0.77±0.08¯ 0.77± 0.08 0.78±0.070.78± 0.07 SynthRad25 T1w→ Abdomen 0.74±0.07†0.74± 0.07 0.55±0.06†0.55± 0.06 0.78±0.10¯ 0.78± 0.10 0.80±0.09∗0.80± 0.09^* SynthRad25 CBCT→ Head-Neck 0.87±0.040.87± 0.04 0.71±0.090.71± 0.09 0.91±0.02¯ 0.91± 0.02 0.92±0.020.92± 0.02 SynthRad25 CBCT→ Thorax 0.80±0.05†0.80± 0.05 0.65±0.11†0.65± 0.11 0.88±0.030.88± 0.03 0.88±0.030.88± 0.03 SynthRad25 CBCT→ Abdomen 0.81±0.06†0.81± 0.06 0.66±0.10†0.66± 0.10 0.87±0.04¯ 0.87± 0.04 0.89±0.03∗0.89± 0.03^* BraTS23 T1w→ 1c Brain 0.86±0.04†0.86± 0.04 0.78±0.05†0.78± 0.05 0.91±0.01¯ 0.91± 0.01 0.92±0.010.92± 0.01 BraTS23 T1c→ 1w Brain 0.90±0.03†0.90± 0.03 0.81±0.05†0.81± 0.05 0.93±0.020.93± 0.02 0.93±0.010.93± 0.01 BraTS23 T1w→ 2w Brain 0.88±0.04†0.88± 0.04 0.78±0.05†0.78± 0.05 0.92±0.010.92± 0.01 0.92±0.010.92± 0.01 BraTS23 T2w→ 1w Brain 0.90±0.03†0.90± 0.03 0.82±0.05†0.82± 0.05 0.93±0.020.93± 0.02 0.93±0.020.93± 0.02 BraTS23 T2w→ 2f Brain 0.83±0.050.83± 0.05 0.59±0.060.59± 0.06 0.88±0.01¯ 0.88± 0.01 0.89±0.010.89± 0.01 BraTS23 T2f→ 2w Brain 0.87±0.04†0.87± 0.04 0.74±0.05†0.74± 0.05 0.90±0.010.90± 0.01 0.90±0.010.90± 0.01 Whole-volume vs. patch-based processing. Table 1 compares all evaluated models across tasks, datasets, and anatomical regions. FM-ST Patch yields the lowest performance across all evaluated tasks, confirming that patch-based flow matching without a strong volumetric prior fails to exploit the generative architecture’s full capacity. SRGAN improves substantially over FM-ST Patch, yet is outperformed by both whole-volume models on all tasks except Pelvis T1w→ , where it retains a narrow advantage in PSNR and SSIM. Across all tasks and datasets, FM-ST Whole-Vol. outperforms FM-ST Patch, with gains in SSIM ranging from +0.12+0.12 to +0.37+0.37 and in PSNR from +2.8+2.8 to +8.4+8.4 dB. These results empirically validate the central claim of this work: when a strong volumetric prior makes whole-volume processing tractable, operating on full volumes is preferable to patch-based inference at parity of generative architecture. Single-task vs. multitask training. We compare the task-specific FM-ST Whole-Vol. models with the unified FM-MT Whole-Vol. model across all evaluated datasets, translation tasks, and anatomical regions. FM-MT is never significantly worse than the corresponding single-task baseline on any evaluated metric. Significant improvements over FM-ST Whole-Vol. are marked with ∗ in Table 1 and indicate cases in which joint training provides a measurable benefit. These results show that a single multitask model can replace N independently trained task-specific networks without a detectable loss in task-level accuracy. Moreover, the significant gains observed in several settings suggest that joint training can act as a beneficial regularizer by sharing information across anatomically and modality-overlapping translation tasks. The benefits of unification extend beyond pixel-level accuracy, as shown by the generalization and compositional experiments reported next. Representative results are shown in Fig. 2. Figure 2: Representative qualitative results across inter-modality and intra-modality translation tasks. For each example, input, ground truth, and prediction are shown. Leave-One-Region-Out (LORO). To assess zero-shot anatomical generalization, we train five FM-MT models, each excluding one anatomical region, and evaluate on the held-out region. Table 2 reports SSIM averaged over T1w→ and CBCT→ for each excluded region. The model reaches an average SSIM of 0.72 against 0.87 for the full multitask model. The limited degradation across all regions confirms that joint training over the full dataset collection induces a shared latent geometry rich enough to support partial anatomical generalization. Table 2: Leave-One-Region-Out (LORO) evaluation. SSIM values are averaged over T1w→ and CBCT→ tasks. “Full” denotes the multitask model trained on all regions; “LORO” denotes zero-shot performance on the excluded region. region Full LORO Δ Brain 0.90 0.74 −-0.16 Pelvis 0.88 0.71 −-0.17 Abdomen 0.84 0.72 −-0.12 Head-Neck 0.89 0.72 −-0.17 Thorax 0.83 0.70 −-0.13 Average 0.87 0.72 −-0.15 4.1 Compositional Translation via Two-Stage Inference We evaluate the compositional capabilities of the Multitask model through two-stage inference. We consider two scenarios of increasing complexity. Intra-dataset chaining. As a first validation, we evaluate T1w→ 2w→ 2f on BraTS, comparing FM-MT Whole-Vol. with a chain of two FM-ST Whole-Vol. models. This supervised setting serves to characterize error propagation across sequential translations. At stage 1, both approaches achieve comparable performance. At stage 2, however, FM-MT improves over the single-task chain (SSIM 0.95 vs. 0.92, p<0.05p<0.05, Table 4). This suggests that the intermediate T2w generated by the multitask model remains better aligned with the subsequent T2w→ 2f mapping. In contrast, the single-task chain provides the second model with a synthetic T2w produced by an independently trained model, creating a mismatch with the real T2w inputs used during its training. Cross-dataset chaining. We then evaluate a more challenging cross-dataset path, T1w→ 2w→ , where the first step exploits BraTS knowledge and the second step targets SynthRad CT synthesis. This path is never directly supervised, since no T2w→ pairs are available. We report results on brain and head-and-neck, the regions with the closest anatomical overlap with BraTS. As shown in Table 4, FM-MT achieves SSIM 0.84 and 0.80 on brain and head-and-neck, respectively, while the chained FM-ST baseline drops to 0.15 and 0.31. This collapse reflects the distribution mismatch between the synthetic T2w generated by the BraTS-only model and the inputs expected by the SynthRad-only model, which was never trained on T2w representations. Representative examples are shown in Fig. 3. Table 3: Two-stage intra-dataset chaining on BraTS (T1w→ 2w→ 2f). Statistically significant improvement at stage 2 (p<0.05p<0.05, Wilcoxon with Holm correction). Stage 1: T1w→ 2w Stage 2: T2w→pred_pred 2f Model SSIM PSNR SSIM PSNR FM-ST (chained) 0.96 28.45 0.92 28.87 FM-MT (Multitask) 0.96 28.32 0.95 30.80 Table 4: Two-stage cross-dataset chaining (T1w→ 2w→BraTS_BraTS on SynthRad). Brain Head-Neck Model SSIM PSNR SSIM PSNR FM-ST (chained) 0.15 12.88 0.31 13.91 FM-MT (Multitask) 0.84 25.51 0.80 25.82 Figure 3: Cross-dataset compositional translation (T1w→ 2w→ ). Brain (top) and head-and-neck (bottom). 5 Discussion and Conclusion This work shows that whole-volume latent translation improves over patch-based inference, and multitask training can replace task-specific models without significant degradation. Beyond accuracy, the unified model enables zero-shot anatomical generalization and compositional translation paths that are not supported by independently trained models. Limitations remain. Evaluation is based on pixel-level metrics and does not yet establish clinical utility. Future work should assess downstream relevance and expert reader studies. The method also inherits the modality and anatomical coverage of the pretrained VAE prior, and zero-shot region generalization remains below fully supervised performance. Future work will therefore focus on broader priors, few-shot adaptation, and uncertainty estimation for out-of-distribution regions. References [1] U. Baid et al. (2021) The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107.02314. External Links: 2107.02314 Cited by: §1, §3. [2] J. Chen et al. (2025) Medical image translation with deep learning: Advances, datasets and perspectives. Medical Image Analysis 103, p. 103605. Cited by: §1. [3] S. Dayarathna et al. (2024) Deep learning based synthesis of MRI, CT and PET: Review and analysis. Medical Image Analysis 92, p. 103046. Cited by: §1. [4] F. Di Feola, L. Pompilio, C. Assolito, V. Guarrasi, and P. Soda (2025) Texture-Aware StarGAN for CT data harmonization. In 2025 International Joint Conference on Neural Networks (IJCNN), p. 1–8. Cited by: §1. [5] B. Glocker, R. Robinson, D. C. Castro, Q. Dou, and E. Konukoglu (2019) Machine learning with multi-site imaging data: An empirical study on the impact of scanner effects. arXiv preprint arXiv:1910.04597. External Links: 1910.04597 Cited by: §1. [6] I. Goodfellow et al. (2020) Generative adversarial networks. Communications of the ACM 63 (11), p. 139–144. Cited by: §1. [7] P. Guo et al. (2025) MAISI: Medical AI for synthetic imaging. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, p. 4430–4441. Cited by: §1, §2. [8] J. Ha, J. S. Park, D. Crandall, E. Garyfallidis, and X. Zhang (2025) Multi-Resolution Guided 3D GANs for Medical Image Translation. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), p. 4342–4351. Cited by: §3. [9] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, Vol. 33, p. 6840–6851. Cited by: §1. [10] I. Iele, F. Di Feola, V. Guarrasi, and P. Soda (2025) Sample-Aware Test-Time Adaptation for Medical Image-to-Image Translation. arXiv preprint arXiv:2508.00766. Cited by: §1. [11] P. Isola, J. Zhu, T. Zhou, and A. A. Efros (2017) Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 1125–1134. Cited by: §1. [12] G. A. Kaissis, M. R. Makowski, D. Rückert, and R. F. Braren (2020) Secure, privacy-preserving and federated machine learning in medical imaging. Nature Machine Intelligence 2 (6), p. 305–311. Cited by: §1. [13] A. Kazerouni et al. (2022) Diffusion models for medical image analysis: A comprehensive survey. arXiv preprint arXiv:2211.07804. External Links: 2211.07804 Cited by: §1. [14] Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le (2022) Flow matching for generative modeling. arXiv preprint arXiv:2210.02747. External Links: 2210.02747 Cited by: §1. [15] D. Molino, C. M. Caruso, F. Ruffini, P. Soda, and V. Guarrasi (2025) Text-to-CT Generation via 3D Latent Diffusion Model with Contrastive Vision-Language Pretraining. arXiv preprint arXiv:2506.00633. Cited by: §1. [16] D. Molino, C. M. Caruso, P. Soda, and V. Guarrasi (2026) Retrieval-augmented anatomical guidance for text-to-ct generation. arXiv preprint arXiv:2603.08305. Cited by: §1. [17] D. Molino, F. Di Feola, E. Faiella, D. Fazzini, D. Santucci, L. Shen, V. Guarrasi, and P. Soda (2026) XGeM: A multi-prompt foundation model for multimodal medical data generation. Computerized Medical Imaging and Graphics, p. 102718. Cited by: §1. [18] D. Molino, F. Di Feola, L. Shen, P. Soda, and V. Guarrasi (2025) Any-to-any vision-language model for multimodal x-ray imaging and radiological report generation. In 2025 International Joint Conference on Neural Networks (IJCNN), p. 1–8. Cited by: §1. [19] S. Niyas, S. Pawan, M. A. Kumar, and J. Rajan (2022) Medical image segmentation with 3D convolutional neural networks: A survey. Neurocomputing 493, p. 397–413. Cited by: §1. [20] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 10684–10695. Cited by: §1. [21] G. Romoli et al. (2026) Cross modality image translation in medical imaging using generative frameworks. arXiv preprint arXiv:2605.13686. External Links: 2605.13686 Cited by: §1, §3, Table 1. [22] O. Ronneberger, P. Fischer, and T. Brox (2015) U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Lecture Notes in Computer Science, Vol. 9351, p. 234–241. Cited by: §2. [23] A. Thummerer et al. (2023) SynthRad2023 grand challenge dataset: Generating synthetic CT for radiotherapy. Medical Physics 50 (7), p. 4664–4674. Cited by: §1, §3. [24] A. Thummerer et al. (2025) SynthRad2025 grand challenge dataset: Generating synthetic CTs for radiotherapy from head to abdomen. Medical Physics 52 (7), p. e17981. Cited by: §1, §3. [25] J. Zhu, T. Park, P. Isola, and A. A. Efros (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision, p. 2223–2232. Cited by: §1.