Paper deep dive
DoGMA: A Central-Dogma-Guided Foundation Model for Multi-Omics Alignment and Multi-Task Learning in Oncology
Junfei Ling, Bangzheng Pu, Bingsen Xue, Tianle Li, Ruying Hu, Cheng Jin
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/11/2026, 5:22:55 AM
Summary
The paper introduces DoGMA, a foundation model for pan-cancer multi-omics analysis that incorporates the central dogma of molecular biology as an inductive bias. By using directed attention mechanisms in a Transformer-MoE architecture and masked hierarchical omics reconstruction pretraining, DoGMA improves cross-omics information flow, robustness to missing data, and performance on downstream tasks such as cancer subtyping, metastasis prediction, and survival analysis compared to existing bidirectional models.
Entities (15)
Relation Signals (12)
DoGMA → guides → Central Dogma
confidence 95% · DoGMA, a central-dogma-guided foundation model
DoGMA → uses → Transformer-MoE
confidence 95% · Concretely, we build it on a Transformer-MoE architecture
DoGMA → applies → Masked Hierarchical Omics Reconstruction
confidence 92% · We further pretrain our model with masked hierarchical omics reconstruction
Directed Attention → enforces → Central Dogma
confidence 90% · directed attention biases inter-omics communication toward central-dogma information flow
DoGMA → outperforms → Pathformer
confidence 90% · DoGMA achieves the highest average METABRIC test accuracy... outperforming... Pathformer
DoGMA → outperforms → DeePathNet
confidence 90% · DoGMA achieves the highest average METABRIC test accuracy... outperforming... DeePathNet
DoGMA → outperforms → TMO-Net
confidence 90% · Compared with TMO-Net, DoGMA produces clearer cancer-type separation
DoGMA → outperforms → MOGONET
confidence 90% · DoGMA achieves the highest average METABRIC test accuracy... outperforming the strongest non-DoGMA baseline
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Attention mechanisms have been widely utilized in modern deep learning, and many existing multi-omics models inherit their conventional use to allow unrestricted bidirectional interactions. However, the fundamental logic of life is directional. Existing designs often overlook the directionality suggested by the central dogma, potentially limiting transfer across heterogeneous cancers, downstream tasks, and incomplete modality this http URL this work, we present DoGMA, a central-dogma-guided foundation model for pan-cancer multi-omics analysis, arguing that robust transfer requires representations with domain-specific inductive bias. Concretely, we build it on a Transformer-MoE architecture where directed attention biases inter-omics communication toward central-dogma information flow. We further pretrain our model with masked hierarchical omics reconstruction to guide it toward learning central-dogma-consistent interactions. Across diverse downstream tasks, including cancer representation learning, survival prediction, and metastasis prediction, DoGMA consistently demonstrates strong predictive performance. Ablations and analyses further suggest that the performance gains arise from the synergy between central-dogma-guided directed attention and reconstruction-based pretraining, which together promote more biologically consistent cross-omics information exchange. Overall, DoGMA demonstrates that domain-specific inductive biases can improve the robustness and transferability of multi-omics foundation models, offering new insights into the design of attention mechanisms for multi-omics representation learning.
Tags
Links
- Source: https://arxiv.org/abs/2608.08148v1
- Canonical: https://arxiv.org/abs/2608.08148v1
Trouble viewing inline? Open PDF directly →
Full Text
79,502 characters extracted from source content.
Expand or collapse full text
DoGMA: A Central-Dogma-Guided Foundation Model for Multi-Omics Alignment and Multi-Task Learning in Oncology Junfei Ling1 , Bangzheng Pu1 , Bingsen Xue1 , Tianle Li2, Ruying Hu3, Cheng Jin1 Abstract Attention mechanisms have been widely utilized in modern deep learning, and many existing multi-omics models inherit their conventional use to allow unrestricted bidirectional interactions. However, the fundamental logic of life is directional. Existing designs often overlook the directionality suggested by the central dogma, potentially limiting transfer across heterogeneous cancers, downstream tasks, and incomplete modality settings. In this work, we present DoGMA, a central-dogma-guided foundation model for pan-cancer multi-omics analysis, arguing that robust transfer requires representations with domain-specific inductive bias. Concretely, we build it on a Transformer-MoE architecture where directed attention biases inter-omics communication toward central-dogma information flow. We further pretrain our model with masked hierarchical omics reconstruction to guide it toward learning central-dogma-consistent interactions. Across diverse downstream tasks, including cancer representation learning, survival prediction, and metastasis prediction, DoGMA consistently demonstrates strong predictive performance. Ablations and analyses further suggest that the performance gains arise from the synergy between central-dogma-guided directed attention and reconstruction-based pretraining, which together promote more biologically consistent cross-omics information exchange. Overall, DoGMA demonstrates that domain-specific inductive biases can improve the robustness and transferability of multi-omics foundation models, offering new insights into the design of attention mechanisms for multi-omics representation learning. Introduction Figure 1: Overview of DoGMA. Stage 1 uses modality-specific VQ-VAEs to map heterogeneous omics profiles into a unified token space. In Stage 2, learnable tokens represent missing omics, PubMedBERT encodes clinical text, and stacked Central-Dogma-MoE blocks combine directed cross-omics attention favoring DNA-to-RNA-to-protein information flow with modality-specific routing for heterogeneous and incomplete inputs. Masked self-reconstruction and cross-omics reconstruction pretrain the backbone to recover within-modality structure and infer missing molecular states. In Stage 3, the learned representation is transferred to cancer subtyping, metastasis prediction, survival analysis, and causal-chain analysis. Attention mechanisms are well-established in multimodal neural networks (Bahdanau et al. 2014; Vaswani et al. 2017; Lu et al. 2019; Chen et al. 2020; Alayrac et al. 2022) and their success has encouraged multi-omics methods to adopt standard attention for integrating molecular modalities (Moon and Lee 2022; Cai et al. 2024; Liu et al. 2024; Lan et al. 2024). However, this direct transfer overlooks a key distinction that standard attention is typically formulated as an unconstrained and bidirectional interaction process (Schölkopf et al. 2021; Cui et al. 2024; Baltrušaitis et al. 2018; Zeng et al. 2024), whereas the fundamental logic of life is directional (Crick 1970; Li et al. 2023). Under the central dogma, information primarily propagates from genomic and epigenomic regulation to transcriptional and proteomic states, with weak feedback and context-dependent regulation. Thus, multi-omics dependencies are not merely symmetric correlations among modalities, they reflect an ordered biological process that should inform model design. This mismatch is particularly problematic in multi-omics integrative representation learning and downstream transfer. First, multi-omics data are highly heterogeneous due to domain and measurement platform effects. Without the guidance of rational priors, unconstrained cross-modal attention may fit cohort-specific associations or modality-specific artifacts (Ganin and Lempitsky 2015) rather than transferable disease-relevant molecular patterns (Arjovsky et al. 2019). Second, clinical multi-omics data often exhibit severe and non-random missingness: patients frequently lack expensive proteomic or methylation profiles. In this setting, a missing modality is not simply an empty input, but can be viewed as an unobserved molecular layer that may be partially inferred from related omics under plausible biological constraints (Ashuach et al. 2023; Tang et al. 2024). These challenges call for models that encode directional biological inductive bias while remaining flexible under heterogeneity and missingness. In this work, we introduce DoGMA, a central-dogma-guided foundation model for pan-cancer multi-omics analysis. Specifically, we build a Transformer-MoE architecture equipped with central-dogma-guided directed attention, which biases inter-omics communication (Rudin 2019) toward central-dogma information flow. To further encourage the model to internalize these directional dependencies, we introduce a masked hierarchical omics reconstruction objective. This pretraining task encourages DoGMA to learn cross-omics information flow patterns that are more consistent with the extended central-dogma prior. Extensive experiments demonstrate the effectiveness of DoGMA. DoGMA achieves state-of-the-art performance across a wide range of benchmark tasks, including cancer subtyping, distant metastasis prediction, and prognostic analysis, while also maintaining robust performance even when faced with missing omics inputs. We further examine why the proposed design improves downstream transfer. Ablation studies and attention-flow analyses reveal a clear synergy between directed attention and reconstruction-based pretraining. Removing the directed prior weakens alignment with biologically plausible information flow, indicating that unconstrained attention alone is insufficient to recover central-dogma-consistent dependencies. Removing the reconstruction objective reduces the training pressure to use cross-omics information flow for inferring masked molecular states. Together, the two components make the prior operational: the architecture specifies a biologically motivated direction, and pretraining teaches the model to use it for reconstruction and prediction. Intervention analyses further show that the resulting inter-omics interactions are more consistent with known molecular priors. These findings suggest that DoGMA gains from central-dogma-consistent information flow rather than unconstrained modality aggregation. In summary, our work demonstrates that the central dogma can serve as an effective domain-specific inductive bias for multi-omics foundation models. By coupling central-dogma attention with reconstruction-based pretraining, DoGMA promotes more plausible inter-omics information flow patterns and improves transfer across tasks and incomplete modality settings. These findings suggest that central-dogma priors offer a principled route toward more robust and transferable attention mechanisms for biomedical representation learning. Method Unified Latent Tokenization via Single-Omics VQ-VAE To ground our Central Dogma backbone on a stable, noise-invariant vocabulary (Bao et al. 2021; Theodoris et al. 2023), for each modality m∈ℳm , we train a modality-specific Vector-Quantized Variational Autoencoder (VQ-VAE) (Van Den Oord et al. 2017) that maps the high-dimensional profile xmx_m to a continuous latent representation hm=Encm(xm)h_m=Enc_m(x_m), followed by nearest-neighbor quantization in a learnable codebook m=ekk=1KC_m=\e_k\_k=1^K: zm=ek,k=argminj‖hm−ej‖2.z_m=e_k, k= _j\|h_m-e_j\|_2. (1) The VQ-VAE is optimized with the standard reconstruction, codebook, and commitment losses: ℒVQ(m) _VQ^(m) =‖xm−Dm(zm)‖22+‖sg[hm]−zm‖22 =\|x_m-D_m(z_m)\|_2^2+\|sg[h_m]-z_m\|_2^2 (2) +β‖hm−sg[zm]‖22. +β\|h_m-sg[z_m]\|_2^2. Where DmD_m is the decoder and sg[⋅]sg[·] denotes the stop-gradient operator. This stage converts heterogeneous and noisy multi-omics measurements into a sequence of invariant, modality-specific tokens, serving as the atomic units for the subsequent Central Dogma-guided interaction. Central Dogma-Guided Interaction and Fusion To align neural connectivity with biological reality, we propose a fusion architecture governed by the Central Dogma, consisting of L stacked Transformer-MoE blocks. Each block applies Central-Dogma Attention to structure information flow, followed by a modality-specific Mixture-of-Experts (MoE) (Riquelme et al. 2021) for heterogeneous modality fusion. Omics-Specific Residual Central-Dogma Attention Standard self-attention imposes no structural asymmetry on cross-omics interactions (Vaswani et al. 2017). This is overly permissive for multi-omics data, where molecular layers are ordered and their dependencies are not symmetric. We therefore introduce an omics-specific residual Central-Dogma attention mechanism that incorporates a directional prior while retaining pair-specific flexibility. Let =methylation,mutation,copy-number alteration (CNA),gene expression,protein,clinicalA=\methylation, \ mutation, \ copy-number alteration (CNA), \ gene expression, \ protein, \ clinical\ denote the set of omics and clinical-context modalities, ordered by molecular level (Battaglia et al. 2018). Methylation, mutation, and CNA are treated as DNA-level modalities, gene expression as RNA-level, and protein abundance as protein-level. For a sample with observed modalities ⊆S , the input to layer l is H(l−1)=[ha(l−1)]a∈ℝ||×dH^(l-1)=[h^(l-1)_a]_a ^|S|× d. For attention head r, we add an omics-specific structural bias to the scaled dot-product logits: Z(l,r)=Q(l,r)K(l,r)⊤dr+Bdogma(l,r),Z^(l,r)= Q^(l,r)K^(l,r) d_r+B_dogma^(l,r), (3) Attn(l,r)(H(l−1))=Softmax(Z(l,r))V(l,r).Attn^(l,r)\! (H^(l-1) )=Softmax\! (Z^(l,r) )V^(l,r). (4) Rows index target modalities and columns index source modalities. We define a binary topology matrix T∈0,1||×||T∈\0,1\^|A|×|A|, where Tij=1T_ij=1 marks a weak-feedback edge from source modality j to target modality i. Its structural bias combines a fixed global penalty with a learnable residual: Bdogma,ij(l,r)=A+Rij(l,r),if Tij=1,0,if Tij=0,B_dogma,ij^(l,r)= casesA+R_ij^(l,r),&if T_ij=1,\\ 0,&if T_ij=0, cases (5) where A<0A<0 is shared across weak-feedback edges, and Rij(l,r)R_ij^(l,r) is a zero-initialized, omics-pair-specific residual. The fixed term A initializes the model with a coarse Central-Dogma prior, while the residual term R distinguishes specific mechanistic relationships and regulatory patterns between omics pairs. MoE Fusion: Conditional Computation for Heterogeneity After Central-Dogma attention, the updated tokens are fused through a Mixture-of-Experts (MoE) (Riquelme et al. 2021). Let h~a(l) h^(l)_a denote the attention-updated representation of omics a at layer l. For each observed omics a∈a , an expert Experta(⋅)Expert_a(·) processes h~a(l) h^(l)_a, and a gate Ga(⋅)G_a(·) produces its routing logit. The normalized routing weight and layer-wise MoE output are αa(l) _a^(l) =exp(Ga(h~a(l)))∑b∈exp(Gb(h~b(l))), = \! (G_a( h_a^(l)) ) _b \! (G_b( h_b^(l)) ), (6) u(l) u^(l) =∑a∈αa(l)Experta(h~a(l)),u(l)∈ℝd. = _a _a^(l)Expert_a\! ( h_a^(l) ), u^(l) ^d. The final fused representation is obtained by aggregating the MoE outputs across the stacked Transformer-MoE blocks: hfused=Agg(u(1),…,u(L)),h_fused=Agg (u^(1),…,u^(L) ), where Agg(⋅)Agg(·) is implemented as residual accumulation across blocks. Central Dogma-Guided Pretraining The Central-Dogma attention prior defines a structured pattern of omics communication, while pretraining encourages the backbone to exploit it. We optimize the model with masked self-reconstruction, cross-omics reconstruction, and invariant semantic alignment: ℒtotal=ℒmask+λ1ℒcross+ℒinv.L_total=L_mask+ _1L_cross+L_inv. (7) Masked self-reconstruction (ℒmaskL_mask). For each modality m∈ℳm , we replace a subset of latent dimensions Ωm _m in the VQ-derived representation zmz_m with a learnable modality-specific mask embedding. The backbone maps the corrupted latent z~m z_m to h~m h_m, and a modality-specific head rmr_m reconstructs the masked values: ℒmask=∑m∈ℳ1|Ωm|∑i∈Ωm‖zm,i−[rm(h~m)]i‖22.L_mask= _m 1| _m| _i∈ _m \|z_m,i-[r_m( h_m)]_i \|_2^2. (8) This latent-space reconstruction objective (He et al. 2022) promotes within-modality robustness to partial corruption. Cross-omics reconstruction (ℒcrossL_cross). To encourage cross-omics information transfer, each modality-specific decoder ϕm _m reconstructs the original latent zmz_m from the fused representation obtained from corrupted inputs: ℒcross=∑m∈ℳ‖zm−ϕm(hfused)‖22.L_cross= _m \|z_m- _m(h_fused) \|_2^2. (9) This objective requires hfusedh_fused to aggregate complementary information across omics, thereby operationalizing the Central-Dogma communication prior during reconstruction. Domain-invariant and semantic alignment. To suppress cohort- and platform-specific artifacts while preserving phenotype-relevant structure, we combine adversarial domain alignment through a gradient reversal layer (Ganin and Lempitsky 2015) with supervised contrastive learning (Khosla et al. 2020): ℒadv _adv =ℒGRL(Dadv(GRL(hfused)),Ydomain), =L_GRL (D_adv(GRL(h_fused)),Y_domain ), (10) ℒinv _inv =λadvℒadv+λconℒsupcon. = _advL_adv+ _conL_supcon. Together, reconstruction encourages structured cross-omics communication, while invariant alignment promotes transfer across heterogeneous cohorts. Downstream Transfer and Adaptation For downstream tasks, we preserve the pretrained biological prior through hierarchical unfreezing: lower blocks remain frozen, while higher layers are updated for task-specific adaptation. Survival Prediction. For right-censored time-to-event analysis, we map the fused representation hfusedh_fused to a risk score θi _i and minimize the Cox Proportional Hazards loss (Cox 1972): ℒcox=−∑i:δi=1(θi−log∑j∈ℛ(ti)exp(θj)),L_cox=- _i: _i=1 ( _i- _j (t_i) ( _j) ), (11) where ℛ(ti)R(t_i) denotes the risk set at time tit_i. Clinical Classification. For discrete phenotypes (e.g., metastasis), we optimize standard Cross-Entropy loss. Crucially, we address the severe class imbalance inherent in metastatic cohorts via stratified epoch-balanced sampling. Each method was independently trained with five random seeds, and the results are reported as mean ± standard deviation. Results Pretrained representations encode pan-cancer structure Figure 2: Pretrained representation geometry. t-SNE of fused embeddings before downstream fine-tuning. DoGMA forms a more structured pan-cancer manifold than TMO-Net. We first examine whether the pretrained backbone already organizes multi-omics samples into a meaningful latent geometry. Figure 2 shows t-SNE projections of fused embeddings before downstream fine-tuning. Compared with TMO-Net, DoGMA produces clearer cancer-type separation and improves the silhouette score from 0.368 to 0.628. This suggests that central-dogma-guided pretraining improves representation structure before task-specific adaptation, providing a basis for the downstream transfer results. Cross-omics Reconstruction and Reverse Molecular Inference Figure 3: Cross-omics reconstruction and reverse inference. DoGMA improves gene-expression reconstruction from non-expression omics and mutation-status inference from downstream molecular profiles, indicating stronger cross-omics predictive dependencies. Forward reconstruction of transcriptomic states. We first test whether the pretrained representation supports prediction across molecular layers by reconstructing gene-expression profiles from non-expression omics. DoGMA achieves a mean Pearson r of 0.78, compared with 0.64 for TMO-Net (Figure 3a); the corresponding heatmaps are provided in Appendix Figure 6. This result suggests that pretraining preserves cross-omics signals informative for transcriptomic reconstruction. Reverse inference of driver-mutation status. We then infer binary driver-mutation status from downstream molecular profiles, a challenging setting because driver mutations are sparse and their downstream effects are indirect and heterogeneous (Roohani et al. 2024; Bailey et al. 2018; Li et al. 2023). DoGMA achieves an AUC of 0.75 versus 0.57 for TMO-Net (Figure 3b), with consistent gains across representative driver genes (Figure 3c). Together, these results indicate that DoGMA better captures mutation-associated downstream signals and support the hypothesis that directional biological priors help organize predictive information flow across omics (Linder et al. 2025). Cross-Cohort Transfer to METABRIC for Breast Cancer Subtyping Method ER HER2 PAM50 Basal Avg. Val Test Val Test Val Test Val Test Val Test MOGONET 0.966 ± 0.008 0.946 ± 0.009 0.951 ± 0.010 0.851 ± 0.014 0.915 ± 0.012 0.927 ± 0.010 0.972 ± 0.007 0.975 ± 0.006 0.951 ± 0.005 0.925 ± 0.005 Pathformer 0.966 ± 0.008 0.930 ± 0.010 0.963 ± 0.008 0.865 ± 0.014 0.929 ± 0.011 0.932 ± 0.010 0.976 ± 0.007 0.984 ± 0.005 0.959 ± 0.004 0.928 ± 0.005 DeePathNet 0.966 ± 0.008 0.947 ± 0.009 0.964 ± 0.008 0.875 ± 0.013 0.923 ± 0.012 0.950 ± 0.009 0.982 ± 0.006 0.979 ± 0.006 0.960 ± 0.004 0.943 ± 0.005 TMO-Net (scratch) 0.957 ± 0.009 0.928 ± 0.010 0.937 ± 0.011 0.883 ± 0.013 0.884 ± 0.014 0.876 ± 0.013 0.982 ± 0.006 0.981 ± 0.005 0.940 ± 0.005 0.917 ± 0.005 TMO-Net (pretrained) 0.966 ± 0.008 0.925 ± 0.010 0.968 ± 0.008 0.873 ± 0.013 0.892 ± 0.014 0.884 ± 0.013 0.990 ± 0.004 0.988 ± 0.004 0.954 ± 0.005 0.918 ± 0.005 DoGMA (scratch) 0.944 ± 0.010 0.892 ± 0.012 0.953 ± 0.009 0.804 ± 0.016 0.923 ± 0.012 0.888 ± 0.013 0.981 ± 0.006 0.974 ± 0.006 0.950 ± 0.005 0.890 ± 0.006 DoGMA (pretrained) 0.968 ± 0.008 0.947 ± 0.009 0.971 ± 0.007 0.939 ± 0.010 0.921 ± 0.012 0.954 ± 0.008 0.983 ± 0.006 0.981 ± 0.005 0.961 ± 0.004 0.955 ± 0.004 Table 1: Cross-cohort breast cancer subtype prediction from TCGA to METABRIC. Validation (Val) and test accuracies are reported as mean ± standard deviation, and Avg. denotes the mean across the four tasks. The best result in each column is bolded. Pretrained DoGMA achieves the highest average test accuracy and the best or tied-best performance on three tasks. We evaluate cross-cohort transfer by training models on TCGA (Cancer Genome Atlas Research Network and others 2013) breast cancer samples and testing on the independent METABRIC cohort (Curtis et al. 2012; Pereira et al. 2016) across four subtype-related classification tasks. This setting tests whether the learned representation generalizes under shifts in cohort composition and measurement protocols. We compare against representative multi-omics baselines, including MOGONET (Wang et al. 2021), Pathformer (Liu et al. 2024), DeePathNet (Cai et al. 2024), and TMO-Net (Wang et al. 2024), with scratch and pretrained variants where applicable. DoGMA improves external-cohort transfer. As shown in Table 1, pretrained DoGMA achieves the highest average METABRIC test accuracy (0.955), outperforming the strongest non-DoGMA baseline, DeePathNet (0.943), and pretrained TMO-Net (0.918). It achieves the best or tied-best results on ER, HER2, and PAM50, while remaining competitive on Basal. These results suggest that DoGMA learns representations that transfer more effectively across breast cancer cohorts. Pretraining is critical for this transfer. Using the same architecture without pretraining lowers the average METABRIC test accuracy from 0.955 to 0.890. This gap shows that the gain is not attributable to the downstream architecture alone, but depends on pretraining. A more fine-grained attribution of the gains is analyzed in ablation studies. Figure 4: Metastasis prediction on TCGA. Pretrained DoGMA achieves the best performance across accuracy, F1-score, precision, and recall among all compared methods. Error bars show the standard deviation across five runs. Metastasis Prediction on TCGA Metastasis prediction requires integrating molecular signals that may be distributed across multiple omics layers and vary across cancer types (Xie et al. 2024). We evaluated DoGMA on the TCGA cohort for binary classification of metastatic versus primary tumor samples, comparing it with four baseline models. The results are summarized in Figure 4. The pretrained DoGMA model achieves the highest performance across all evaluated metrics, reaching 0.9747 accuracy, 0.9750 precision, 0.9750 recall, and 0.9750 F1-score. The baselines perform competitively but remain consistently below the pretrained model, particularly in recall. Since false negatives are undesirable in metastasis detection, recall provides an important complementary view beyond overall accuracy. We interpret these results as evidence that DoGMA learns a representation that is more predictive of metastatic status under this TCGA setting. This aligns with recent observations that symmetric fusion models often hit a saturation point due to their inability to resolve conflicting signals between modalities (Zhang et al. 2025; Wang et al. 2025). They are also consistent with our broader hypothesis that directional cross-omics pretraining can help the model retain weak, distributed molecular signals associated with tumor progression. Survival Risk Stratification in COAD and READ Figure 5: Survival-risk stratification in COAD and READ. We test whether pretrained multi-omics representations transfer to time-to-event prognosis: (a) C-index distributions compare prognostic discrimination, and (b) Kaplan–Meier curves evaluate risk separation from predicted scores. Pretrained DoGMA shows stronger discrimination and clearer survival stratification. Beyond classification, we ask whether the learned multi-omics representations transfer to time-to-event prognosis in an independent colorectal cancer cohort. We evaluate COAD and READ separately to test prognostic generalization across related but clinically distinct subcohorts, with READ representing the more heterogeneous and sample-limited setting. Prognostic discrimination. Figure 5a compares C-index distributions across baselines and DoGMA variants. Pretrained DoGMA achieves the best overall survival performance and the highest COAD C-index, while DoGMA variants remain among the strongest methods on READ. These results suggest that reconstruction-based pretraining yields representations that transfer beyond classification to survival modeling. Risk-group stratification. We further test whether predicted risk scores separate patients with distinct survival outcomes (Kaplan and Meier 1958). As shown in Figure 5b, pretrained DoGMA produces the clearest high- versus low-risk separation in COAD and the most significant separation in READ. Together, these results indicate that DoGMA learns multi-omics representations that are informative for prognosis in an independent colorectal cancer cohort. Ablation Studies and Mechanistic Analysis Variant Surv. C-ind. Metastasis Breast subtype Acc. Prec. Rec. F1 Val. Test Scratch 0.6708 0.9114 0.8837 0.9500 0.9157 0.9500 0.8900 Standard Attn. 0.6710 0.9114 0.9231 0.9000 0.9114 0.9370 0.8950 w/o Cross- Recon 0.6761 0.9241 0.9048 0.9500 0.9268 0.9596 0.8811 w/o Contra 0.6835 0.8861 0.8605 0.9250 0.8916 0.9660 0.9394 w/o Adv-GRL 0.6788 0.9494 0.9737 0.9250 0.9487 0.9620 0.9140 w/o Self- Recon 0.6807 0.9241 0.9250 0.9250 0.9250 0.9615 0.9271 Single Expert 0.6865 0.8734 0.8571 0.9000 0.8780 0.9586 0.9061 w/o MoE Router 0.6769 0.9367 0.9268 0.9500 0.9383 0.9625 0.9255 Full 0.7207 0.9747 0.9750 0.9750 0.9750 0.9610 0.9550 Table 2: Component ablations of pretraining objectives, attention priors, and MoE variants. Except for Scratch, all variants use the same pretraining framework; each “w/o” variant removes one Full DoGMA component. Best values in each column are bold. The central hypothesis of DoGMA is that its gains do not come simply from a larger fusion backbone, but from the interaction between reconstruction-based pretraining and central-dogma-guided directed attention. We therefore analyze the model from two complementary perspectives: component ablations quantify which objectives and routing modules drive downstream performance, while mechanistic interventions test whether the trained model functionally relies on DoGMA-prior-consistent omics interactions. Component Ablations Table 2 compares parameter-matched ablations of the pretraining objectives, attention prior, and MoE fusion. Scratch omits second-stage pretraining; all other variants remove or replace one component under the same downstream protocol. Full DoGMA performs best on survival, all metastasis metrics, and METABRIC test accuracy, although several variants remain competitive on validation. Among objective ablations, removing cross-omics reconstruction causes the largest external-transfer drop, underscoring the value of recovering each modality from complementary omics context. Self-reconstruction yields a smaller but consistent gain, consistent with latent denoising. Contrastive and adversarial alignment mainly benefit survival and external-cohort transfer, even when validation accuracy changes little. Architectural ablations show a similar trend. Standard attention remains close to Scratch and below Full DoGMA, suggesting that pretraining alone does not recover directed cross-omics structure. Using a single expert or removing the router also reduces performance, with the single-expert variant dropping more. Together, these results attribute DoGMA’s gains to the combination of central-dogma-guided communication, reconstruction-based pretraining, and conditional modality-specific fusion. Attention-flow Alignment A fundamental question is whether DoGMA’s gains arise from internalizing an information-flow pattern that is more consistent with the central dogma. To quantify the qualitative attention-flow pattern (Abnar and Zuidema 2020; Serrano and Smith 2019) shown in Appendix Figure 8, we compute an omics-level directed flow matrix F, where FijF_ij denotes the information flow from source omics j to target omics i, and summarize it with a Dogma Directionality Index (DDI). Let ℰ+E_+ denote DoGMA-prior-consistent directions and ℰrevE_rev denote reverse directions. We define DDI=∑(j→i)∈ℰ+Fij−∑(j→i)∈ℰrevFij∑(j→i)∈ℰ+Fij+∑(j→i)∈ℰrevFij+ϵ.DDI= _(j→ i) _+F_ij- _(j→ i) _revF_ij _(j→ i) _+F_ij+ _(j→ i) _revF_ij+ε. A higher DDI indicates stronger allocation of information flow to DoGMA-prior-consistent directions over reverse directions. Table 3 shows that DDI is positively associated with downstream performance across variants that directly alter information flow. Standard attention and scratch training yield low DDI values (0.2650.265 and 0.3510.351) and weaker task performance. Removing cross-omics or self-reconstruction gives intermediate DDI (0.4990.499–0.5270.527) and partially recovers performance. The full model achieves the highest DDI (0.7390.739) and consistently delivers the best downstream performance. These results suggest that DoGMA’s benefit is not merely architectural: the reconstruction objectives help convert the directional prior into task-useful representations. Metric Scr. Std. −-Cross −-Self Full Mechanistic analysis DDI 0.351 0.265 0.499 0.527 0.739 Survival analysis C-index 0.6708 0.6710 0.6761 0.6807 0.7207 Metastasis prediction Accuracy 0.9114 0.9114 0.9241 0.9241 0.9747 Precision 0.8837 0.9231 0.9048 0.9250 0.9750 Recall 0.9500 0.9000 0.9500 0.9250 0.9750 F1-score 0.9157 0.9114 0.9268 0.9250 0.9750 Breast cancer subtyping Avg. val. acc. 0.9500 0.9370 0.9596 0.9615 0.9610 Avg. test acc. 0.8900 0.8950 0.8811 0.9271 0.9550 Table 3: Mechanistic DDI analysis and downstream performance of selected model variants. Scr. denotes training from scratch; Std. denotes standard attention; −-Cross and −-Self denote pretrained variants without cross-omics and self-reconstruction, respectively. The best result in each row is shown in bold. Metric Fwd. DNA Rev. Self Rand. A. Edge knockout Δ −0.4963-0.4963 −0.3307-0.3307 +0.4374+0.4374 −0.0197-0.0197 −0.1430-0.1430 Δ -index −0.1242-0.1242 −0.0033-0.0033 +0.0010+0.0010 −0.0064-0.0064 −0.0055-0.0055 Δ . Acc. −0.0059-0.0059 −0.0020-0.0020 −0.0059-0.0059 −0.0118-0.0118 0.00000.0000 Δ Acc. −0.0206-0.0206 −0.0048-0.0048 −0.0016-0.0016 −0.0190-0.0190 −0.0079-0.0079 Δ . −0.0909-0.0909 0.00000.0000 −0.0606-0.0606 −0.0909-0.0909 −0.0606-0.0606 Δ . −0.1412-0.1412 0.00000.0000 −0.0991-0.0991 −0.1412-0.1412 −0.0991-0.0991 Δ . 0.00000.0000 0.00000.0000 0.00000.0000 0.00000.0000 0.00000.0000 Δ 1 −0.0808-0.0808 0.00000.0000 −0.0554-0.0554 −0.0808-0.0808 −0.0554-0.0554 Metric Broad KO Prior Reverse Random B. Edge rescue Δ −0.4605-0.4605 +0.5395+0.5395 −1.4605-1.4605 −0.3744-0.3744 Δ -index −0.0979-0.0979 −0.0828-0.0828 −0.1188-0.1188 −0.0916-0.0916 Δ . Acc. −0.0168-0.0168 −0.0035-0.0035 −0.0237-0.0237 −0.0049-0.0049 Δ Acc. −0.0460-0.0460 −0.0012-0.0012 −0.0349-0.0349 −0.0055-0.0055 Δ . −0.0633-0.0633 −0.0380-0.0380 −0.1266-0.1266 −0.1266-0.1266 Δ . −0.0756-0.0756 −0.0499-0.0499 −0.1974-0.1974 −0.1974-0.1974 Δ . 0.00000.0000 0.00000.0000 0.00000.0000 0.00000.0000 Δ 1 −0.0623-0.0623 −0.0370-0.0370 −0.1049-0.1049 −0.1049-0.1049 Table 4: Full DoGMA edge-level interventions. Entries are changes from the intact model under inference-time attention masking with fixed parameters. Panel A removes one edge family; more negative values indicate stronger reliance. Panel B blocks all cross-omics edges and restores one family; larger values indicate stronger rescue. DoGMA-prior is the union of forward and intra-DNA routes. Fwd., DNA, Rev., Self, and Rand. denote forward, intra-DNA, reverse, self, and random-matched edges. Bold marks the strongest knockout or rescue per metric. Counterfactual Edge Intervention We test whether DoGMA functionally uses its learned attention-flow pattern at inference. With parameters fixed, additive masks suppress selected source-to-target omics edges in every attention block. Knockout removes one edge family from the intact model to test necessity; rescue first blocks all cross-omics edges and then restores one family to test recoverability. Table 4 shows a clear asymmetry. Removing DoGMA-forward edges produces the largest survival drop (Δ -index=−0.1242=-0.1242) and a larger metastasis drop than random-matched deletion, whereas intra-DNA edges mainly shift DDI with limited downstream effect. After blocking all cross-omics routes, restoring DoGMA-prior edges gives the strongest rescue, increasing DDI (Δ =+0.5395=+0.5395) and nearly recovering METABRIC test accuracy (Δ Acc.=−0.0012=-0.0012); reverse and random-matched edges leave larger deficits. Thus, DoGMA not only learns central-dogma-like attention patterns but also selectively relies on prior-consistent routes for prediction. The appendix A provides complementary biological plausibility analyses: in silico perturbations of upstream drivers recover literature-supported downstream cancer programs and identify candidate causal-chain intermediates. References S. Abnar and W. Zuidema (2020) Quantifying attention flow in transformers. In Proceedings of the 58th annual meeting of the association for computational linguistics, p. 4190–4197. Cited by: Attention-flow Alignment. J. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al. (2022) Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems 35, p. 23716–23736. Cited by: Introduction. S. Albaradei, F. Napolitano, M. A. Thafar, T. Gojobori, M. Essack, and X. Gao (2021) MetaCancer: a deep learning-based pan-cancer metastasis prediction model developed using multi-omics data. Computational and Structural Biotechnology Journal 19, p. 4404–4411. Cited by: Appendix A, Appendix A. R. Argelaguet, D. Arnol, D. Bredikhin, et al. (2020) MOFA+: a statistical framework for comprehensive integration of multi-modal single-cell data. Genome Biology 21, p. 1–17. Cited by: Appendix A. M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz (2019) Invariant risk minimization. arXiv preprint arXiv:1907.02893. Cited by: Introduction. T. Ashuach, M. I. Gabitto, R. V. Koodli, G. Saldi, M. I. Jordan, and N. Yosef (2023) MultiVI: deep generative model for the integration of multimodal data. Nature methods 20 (8), p. 1222–1231. Cited by: Introduction. D. Bahdanau, K. Cho, and Y. Bengio (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. Cited by: Introduction. M. H. Bailey, C. Tokheim, E. Porta-Pardo, S. Sengupta, D. Bertrand, A. Weerasinghe, A. Colaprico, M. C. Wendl, J. Kim, B. Reardon, et al. (2018) Comprehensive characterization of cancer driver genes and mutations. Cell 173 (2), p. 371–385. Cited by: Cross-omics Reconstruction and Reverse Molecular Inference. T. Baltrušaitis, C. Ahuja, and L. Morency (2018) Multimodal machine learning: a survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence 41 (2), p. 423–443. Cited by: Introduction. H. Bao, L. Dong, S. Piao, and F. Wei (2021) Beit: bert pre-training of image transformers. arXiv preprint arXiv:2106.08254. Cited by: Unified Latent Tokenization via Single-Omics VQ-VAE. P. W. Battaglia, J. B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner, et al. (2018) Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261. Cited by: Omics-Specific Residual Central-Dogma Attention. Z. Cai, R. C. Poulos, A. Aref, P. J. Robinson, R. R. Reddel, and Q. Zhong (2024) DeePathNet: a transformer-based deep learning model integrating multiomic data with cancer pathways. Cancer Research Communications 4 (12), p. 3151–3164. Cited by: Appendix A, Introduction, Cross-Cohort Transfer to METABRIC for Breast Cancer Subtyping. J. Cancer Genome Atlas Research Network et al. (2013) The cancer genome atlas pan-cancer analysis project. Nat. Genet 45 (10), p. 1113–1120. Cited by: Cross-Cohort Transfer to METABRIC for Breast Cancer Subtyping. Y. Chen, L. Li, L. Yu, A. El Kholy, F. Ahmed, Z. Gan, Y. Cheng, and J. Liu (2020) Uniter: universal image-text representation learning. In European conference on computer vision, p. 104–120. Cited by: Introduction. D. R. Cox (1972) Regression models and life-tables. Journal of the royal statistical society: Series B (methodological) 34 (2), p. 187–202. Cited by: Downstream Transfer and Adaptation. F. Crick (1970) Central dogma of molecular biology. Nature 227 (5258), p. 561–563. Cited by: Introduction. H. Cui, C. Wang, H. Maan, K. Pang, F. Luo, N. Duan, and B. Wang (2024) ScGPT: toward building a foundation model for single-cell multi-omics using generative ai. Nature methods 21 (8), p. 1470–1480. Cited by: Introduction. C. Curtis, S. P. Shah, S. Chin, G. Turashvili, O. M. Rueda, M. J. Dunning, D. Speed, A. G. Lynch, S. Samarajiwa, Y. Yuan, et al. (2012) The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature 486 (7403), p. 346–352. Cited by: Cross-Cohort Transfer to METABRIC for Breast Cancer Subtyping. W. Fedus, B. Zoph, and N. Shazeer (2022) Switch transformers: scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research 23 (120), p. 1–39. Cited by: Appendix A. Y. Ganin and V. Lempitsky (2015) Unsupervised domain adaptation by backpropagation. In International conference on machine learning, p. 1180–1189. Cited by: Introduction, Central Dogma-Guided Pretraining. H. A. Hama, B. S. Hasan, B. Barzngy, S. S. Abdulla, A. Y. Karim, R. A. Khailany, M. Miasko, J. M. Dabrowski, and B. Pucelik (2023) KRAS and mt-co1 genes in colorectal cancer: a molecular investigation. Cellular and Molecular Biology 69 (11), p. 30–35. Cited by: Table 5. K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick (2022) Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 16000–16009. Cited by: Central Dogma-Guided Pretraining. R. S. Joshi, S. S. Kanugula, S. Sudhir, M. P. Pereira, S. Jain, and M. K. Aghi (2021) The role of cancer-associated fibroblasts in tumor progression. Cancers 13 (6), p. 1399. Cited by: Table 5. E. L. Kaplan and P. Meier (1958) Nonparametric estimation from incomplete observations. Journal of the American statistical association 53 (282), p. 457–481. Cited by: Survival Risk Stratification in COAD and READ. P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan (2020) Supervised contrastive learning. Advances in neural information processing systems 33, p. 18661–18673. Cited by: Central Dogma-Guided Pretraining. W. Lan, H. Liao, Q. Chen, L. Zhu, Y. Pan, and Y. P. Chen (2024) DeepKEGG: a multi-omics data integration framework with biological insights for cancer recurrence prediction and biomarker discovery. Briefings in bioinformatics 25 (3), p. bbae185. Cited by: Introduction. Y. Li, E. Porta-Pardo, C. Tokheim, M. H. Bailey, T. M. Yaron, V. Stathias, Y. Geffen, K. J. Imbach, S. Cao, S. Anand, et al. (2023) Pan-cancer proteogenomics connects oncogenic drivers to functional states. Cell 186 (18), p. 3921–3944. Cited by: Introduction, Cross-omics Reconstruction and Reverse Molecular Inference. J. Linder, D. Srivastava, H. Yuan, V. Agarwal, and D. R. Kelley (2025) Predicting rna-seq coverage from dna sequence as a unifying model of gene regulation. Nature Genetics 57 (4), p. 949–961. Cited by: Cross-omics Reconstruction and Reverse Molecular Inference. L. Liu, C. Wang, S. Li, H. Bai, and J. Wang (2021) Tumor immune microenvironment in epidermal growth factor receptor-mutated non-small cell lung cancer before and after epidermal growth factor receptor tyrosine kinase inhibitor treatment: a narrative review. Translational Lung Cancer Research 10 (9), p. 3823. Cited by: Table 5. X. Liu, Y. Tao, Z. Cai, P. Bao, H. Ma, K. Li, M. Li, Y. Zhu, and Z. J. Lu (2024) Pathformer: a biological pathway informed transformer for disease diagnosis and prognosis using multi-omics data. Bioinformatics 40 (5), p. btae316. Cited by: Appendix A, Introduction, Cross-Cohort Transfer to METABRIC for Breast Cancer Subtyping. J. Lu, D. Batra, D. Parikh, and S. Lee (2019) Vilbert: pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems 32. Cited by: Introduction. R. Lupat, R. Perera, S. Loi, and J. Li (2023) Moanna: multi-omics autoencoder-based neural network algorithm for predicting breast cancer subtypes. Ieee Access 11, p. 10912–10924. Cited by: Appendix A, Appendix A. S. Moon and H. Lee (2022) MOMA: a multi-task attention learning algorithm for multi-omics data interpretation and classification. Bioinformatics 38 (8), p. 2287–2296. Cited by: Introduction. F. Peng, Q. Xu, X. Jing, X. Chi, Z. Zhang, X. Meng, X. Liu, J. Yan, X. Liu, and S. Shao (2023) GPX2 promotes emt and metastasis in non-small cell lung cancer by activating pi3k/akt/mtor/snail signaling axis. FASEB BioAdvances 5 (6), p. 233. Cited by: Table 5. B. Pereira, S. Chin, O. M. Rueda, H. M. Vollan, E. Provenzano, H. A. Bardwell, M. Pugh, L. Jones, R. Russell, S. Sammut, et al. (2016) The somatic mutation profiles of 2,433 breast cancers refine their genomic and transcriptomic landscapes. Nature communications 7 (1), p. 11479. Cited by: Cross-Cohort Transfer to METABRIC for Breast Cancer Subtyping. C. Riquelme, J. Puigcerver, B. Mustafa, M. Neumann, R. Jenatton, A. Susano Pinto, D. Keysers, and N. Houlsby (2021) Scaling vision with sparse mixture of experts. Advances in Neural Information Processing Systems 34, p. 8583–8595. Cited by: MoE Fusion: Conditional Computation for Heterogeneity, Central Dogma-Guided Interaction and Fusion. Y. Roohani, K. Huang, and J. Leskovec (2024) Predicting transcriptional outcomes of novel multigene perturbations with gears. Nature Biotechnology 42 (6), p. 927–935. Cited by: Cross-omics Reconstruction and Reverse Molecular Inference. C. Rudin (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence 1 (5), p. 206–215. Cited by: Introduction. B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio (2021) Toward causal representation learning. Proceedings of the IEEE 109 (5), p. 612–634. External Links: Document Cited by: Introduction. S. Serrano and N. A. Smith (2019) Is attention interpretable?. In Proceedings of the 57th annual meeting of the association for computational linguistics, p. 2931–2951. Cited by: Attention-flow Alignment. A. Singh, C. P. Shannon, B. Gautier, F. Rohart, M. Vacher, S. J. Tebbutt, and K. Lê Cao (2019) DIABLO: an integrative approach for identifying key molecular drivers from multi-omics assays. Bioinformatics 35 (17), p. 3055–3062. Cited by: Appendix A. T. Sutter, I. Daunhawer, and J. Vogt (2021) Generalized multimodal elbo. In International Conference on Learning Representations (ICLR), Cited by: Appendix A. Z. Tang, G. Chen, S. Chen, J. Yao, L. You, and C. Y. Chen (2024) Modal-nexus auto-encoder for multi-modality cellular data integration and imputation. Nature Communications 15 (1), p. 9021. Cited by: Introduction. C. V. Theodoris, L. Xiao, A. Chopra, M. D. Chaffin, Z. R. Al Sayed, M. C. Hill, H. Mantineo, E. M. Brydon, Z. Zeng, X. S. Liu, et al. (2023) Transfer learning enables predictions in network biology. Nature 618 (7965), p. 616–624. Cited by: Unified Latent Tokenization via Single-Omics VQ-VAE. D. A. Tsang, S. Y. Tam, and C. C. Oh (2023) Molecular alterations in cutaneous squamous cell carcinoma in immunocompetent and immunosuppressed hosts—a systematic review. Cancers 15 (6), p. 1832. Cited by: Table 5. A. Van Den Oord, O. Vinyals, et al. (2017) Neural discrete representation learning. Advances in neural information processing systems 30. Cited by: Unified Latent Tokenization via Single-Omics VQ-VAE. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. Advances in neural information processing systems 30. Cited by: Introduction, Omics-Specific Residual Central-Dogma Attention. F. Wang, Z. Zhuang, F. Gao, R. He, S. Zhang, L. Wang, J. Liu, and Y. Li (2024) TMO-net: an explainable pretrained multi-omics model for multi-task learning in oncology. Genome biology 25 (1), p. 149. Cited by: Appendix A, Appendix A, Appendix A, Cross-Cohort Transfer to METABRIC for Breast Cancer Subtyping. T. Wang, W. Shao, Z. Huang, H. Tang, J. Zhang, Z. Ding, and K. Huang (2021) MOGONET integrates multi-omics data using graph convolutional networks allowing patient classification and biomarker identification. Nature communications 12 (1), p. 3445. Cited by: Appendix A, Cross-Cohort Transfer to METABRIC for Breast Cancer Subtyping. X. Wang, M. Duan, P. Su, J. Li, J. Krull, J. Jin, H. Chen, Y. Sun, W. Wu, K. He, et al. (2025) Deep-learning-enabled multi-omics analyses for prediction of future metastasis in cancer. bioRxiv. Cited by: Metastasis Prediction on TCGA. Q. Xie, S. Liu, S. Zhang, L. Liao, Z. Xiao, S. Wang, and P. Zhang (2024) Research progress on the multi-omics and survival status of circulating tumor cells. Clinical and Experimental Medicine 24 (1), p. 49. Cited by: Metastasis Prediction on TCGA. J. Xin, S. Yun, J. Peng, I. Choi, J. L. Ballard, T. Chen, and Q. Long (2025) I2moe: interpretable multimodal interaction-aware mixture-of-experts. arXiv preprint arXiv:2505.19190. Cited by: Appendix A. Y. Zeng, J. Xie, Z. Wei, Y. Su, N. Shangguan, S. Yang, C. Zhang, W. Li, J. Zhang, N. Fang, et al. (2024) CellFM: a large-scale foundation model pre-trained on transcriptomics of 100 million human cells. bioRxiv, p. 2024–06. Cited by: Introduction. F. Zhang, M. Yang, X. Peng, Q. Qin, Y. Huang, R. Deng, D. Zhang, L. Zhang, H. Liang, and J. Li (2026) The tp53-lgals4 axis modulates the tumor immune microenvironment and synergizes with anti pd-l1 therapy in colorectal cancer. Journal of Translational Medicine 24 (1), p. 225. Cited by: Table 5. H. Zhang, D. Huang, E. Chen, D. Cao, T. Xu, B. Dizdar, G. Li, Y. Chen, P. Payne, M. Province, et al. (2024) MosGraphGPT: a foundation model for multi-omic signaling graphs using generative ai. bioRxiv. Cited by: Appendix A. J. Zhang, Y. Che, R. Liu, Z. Wang, and W. Liu (2025) Deep learning–driven multi-omics analysis: enhancing cancer diagnostics and therapeutics. Briefings in bioinformatics 26 (4), p. bbaf440. Cited by: Metastasis Prediction on TCGA. S. G. Zhao, W. S. Chen, R. Das, S. L. Chang, S. A. Tomlins, J. Chou, D. A. Quigley, H. X. Dang, T. J. Barnard, B. A. Mahal, et al. (2019) Clinical and genomic implications of luminal and basal subtypes across carcinomas. Clinical cancer research 25 (8), p. 2450–2457. Cited by: Table 5. Appendix A Technical Appendices and Supplementary Material Related Work Deep Multi-Omics Representation Learning. Multi-omics integration has evolved from similarity-based and latent-variable approaches, including similarity network fusion, MOFA, and supervised latent-component models (Argelaguet et al. 2020; Singh et al. 2019), toward deep architectures capable of modeling nonlinear dependencies across molecular measurements. MOGONET (Wang et al. 2021) constructs an omics-specific sample graph for each modality and integrates graph-based predictions through a view-correlation discovery network. More recent Transformer-based models further incorporate biological structure into representation learning. Pathformer (Liu et al. 2024) organizes multi-omics features around biological pathways and introduces pathway-aware attention biases, while DeePathNet (Cai et al. 2024) models interactions among cancer-specific pathway representations. These advances show that biologically informed architectures can substantially improve multi-omics modeling. DoGMA extends this line of research by introducing biological structure directly into communication among omics-level representations. Rather than treating molecular modalities as an unordered collection, DoGMA organizes their interactions according to the directional information flow of the central dogma. Its attention mechanism combines a fixed directionality prior with learnable omics-pair-specific residual biases, allowing the model to preserve a clear biological ordering while adapting individual cross-omics interactions to the data. This design provides an explicit bridge between molecular-layer organization and attention-based multi-omics fusion. TMO-Net (Wang et al. 2024) is a particularly relevant pan-cancer pretrained model. It combines self-modal and cross-modal variational autoencoders, a Cross Fusion Module, contrastive learning, and adversarial alignment to learn transferable representations and infer unavailable assays. DoGMA complements this direction with a structurally guided pretraining framework in which cross-modal communication is shaped by central-dogma-consistent attention. Masked self-reconstruction encourages robust modality-specific representations, while cross-omics reconstruction promotes predictive information exchange across molecular layers. Together, the directional attention prior and reconstruction objectives enable DoGMA to learn representations designed for transfer across cancer cohorts, downstream tasks, and incomplete assay configurations. mosGraphGPT (Zhang et al. 2024) provides another important precedent by mapping epigenomic, genomic, transcriptomic, and proteomic measurements onto multi-level signaling graphs. Its graph architecture propagates information through promoter-to-protein routes and learns molecular structure through masked interaction reconstruction. DoGMA captures a complementary level of biological organization. While mosGraphGPT represents detailed gene-, promoter-, and protein-level topology, DoGMA models the global direction of communication among entire omics layers. By encoding central-dogma-consistent flow as an explicit and learnable prior over omics-level attention, DoGMA provides a scalable mechanism for imposing biological directionality across heterogeneous molecular modalities. This prior is jointly optimized with self- and cross-omics reconstruction and modality-specific gated fusion, forming a unified framework for biologically structured pan-cancer representation learning. Learning with Incomplete Multi-Omics Data. Incomplete multi-omics measurements are common in clinical cohorts because assays differ in cost, availability, tissue requirements, and cohort coverage. Multimodal variational models address this problem by learning shared or aligned latent representations from different subsets of observed modalities (Sutter et al. 2021). TMO-Net further adapts cross-modal variational learning to pan-cancer data through self-modal and cross-modal encoders and decoders (Wang et al. 2024). These approaches establish cross-modal reconstruction as an effective strategy for learning from incomplete molecular profiles. DoGMA enriches cross-modal reconstruction with an explicit prior over molecular information flow. Its attention mechanism favors central-dogma-consistent routes when exchanging information among genomic, transcriptomic, and proteomic representations, while learnable residual biases capture dataset-specific interactions beyond the coarse prior. The reconstruction objectives then train the backbone to recover masked information from both within-modality context and complementary omics. This combination encourages the model to retain predictive upstream signals when downstream assays are unavailable and to exploit downstream molecular states when they are observed. DoGMA therefore treats incomplete multi-omics learning as structured representation recovery guided by the organization of molecular regulation. Mixture-of-Experts for Biological Heterogeneity. Cancer cohorts exhibit substantial heterogeneity across tumor types, molecular subtypes, assay combinations, and clinical tasks. Mixture-of-Experts architectures provide a natural mechanism for accommodating such heterogeneity through specialized transformations and learned routing. Sparse MoE models such as the Switch Transformer (Fedus et al. 2022) demonstrate the effectiveness of conditional expert computation at scale, while multimodal models such as I2MoE (Xin et al. 2025) use specialized experts to capture heterogeneous interactions among modalities. DoGMA introduces modality-specific expert specialization into multi-omics fusion. Following central-dogma-guided attention, each molecular representation is processed by an expert associated with its omics layer, and learned gates adaptively combine the resulting expert outputs. This organization preserves modality-specific processing while allowing the contribution of each molecular layer to vary across patients and available assay sets. The experts consequently capture the distinct statistical and biological characteristics of methylation, mutation, copy-number alteration, transcriptomic, and proteomic data, while the router coordinates their contributions to the fused representation. The resulting architecture unifies three complementary forms of biological inductive bias: directional communication among molecular layers, reconstruction-based learning across observed and masked omics, and modality-specific expert specialization. This combination enables DoGMA to learn transferable pan-cancer representations while retaining explicit correspondence between model structure and the organization of multi-omics data. Additional Analyses Forward Translation Heatmaps To verify whether DOGMA’s internal representations genuinely mirror the logic of the biological cascade, we visualize the results of the transcriptomic synthesis task. Figure 6 presents a side-by-side comparison between the ground-truth pan-cancer gene expression profiles and the synthetic outputs generated by DOGMA solely from genomic inputs (e.g., Mutation, CNV, and Methylation). Figure 6: The heatmap of forward prediction Feature Importance and Biomarker Identification We further investigate the relative importance of individual features contributing to DOGMA’s clinical decisions, particularly in the context of metastasis prediction. Figure 7 ranks the top-20 most influential molecular features identified by the model. Figure 7: Relative importance of the top-20 features contributing to the model’s decision. Features marked with a star (⋆ ) denote biomarkers with established clinical evidence in metastasis literature. Attention-Flow Visualization To qualitatively inspect whether the learned cross-omics communication follows the intended biological prior, we visualize the omics-level attention flow after downstream adaptation. Rows denote query/target omics and columns denote key/source omics; each entry reports the mean attention weight aggregated over samples, layers, and heads. Compared with conventional attention, which distributes mass more diffusely across bidirectional omics interactions, DoGMA concentrates attention on biologically plausible routes, including intra-DNA interactions and upstream-to-downstream dependencies from DNA-level omics to transcriptomic and proteomic states. The scratch variant shows a weaker form of this structure, suggesting that the architectural prior alone is insufficient and that reconstruction-based pretraining is needed to make the directed prior operational. This qualitative pattern is consistent with the DDI analysis in the main text, where stronger central-dogma-aligned flow correlates with improved downstream transfer. Figure 8: Omics-level attention-flow visualization. Rows indicate query/target omics and columns indicate key/source omics; values are mean attention weights aggregated over samples, layers, and heads. Conventional attention yields a more diffuse and bidirectional interaction pattern, whereas DoGMA places stronger attention on DoGMA-prior-consistent routes, including intra-DNA communication and upstream-to-downstream flow toward mRNA and protein. The clearer structure in pretrained DoGMA compared with DoGMA-scratch supports the role of reconstruction-based pretraining in turning the central-dogma prior into task-useful information flow. Biological Plausibility of In-Silico Interventions To complement the attention-flow and edge-intervention analyses in the main text, we evaluate whether DoGMA-sensitive directions correspond to biologically plausible molecular responses. For each selected upstream driver, we perform an in-silico intervention on the corresponding omics node and rank downstream RNA or protein features by the magnitude of their reconstructed change. This procedure asks whether the model’s most sensitive downstream responses recover known cancer-associated genes or pathway components. Table 5 summarizes representative interventions. Across mutation and copy-number drivers, the top-ranked downstream targets include genes and pathway components previously implicated in tumor subtype identity, immune signaling, epithelial programs, mitochondrial activity, or stromal remodeling. For example, TP53 and KRAS perturbations recover downstream epithelial and metabolic markers, whereas EGFR, CCND1, and FGFR1 copy-number interventions highlight immune, squamous-like, and fibroblast-associated programs. These results suggest that DoGMA’s learned cross-omics responses are not arbitrary feature changes, but often align with known molecular programs. Intervened Node Modality Top-10 Downstream Targets and Representative Literature Priors TP53 Mutation LGALS4 (Zhang et al. 2026), TFF1, SPINK1, KRT5 (Zhao et al. 2019), KRT14 (Zhao et al. 2019), CEACAM5, PRR15, GPX2, MUC13, EPS8L3 KRAS Mutation LGALS4, GPX2 (Peng et al. 2023), SPINK1, KRT5, MT-CO3 (Hama et al. 2023), KRT14, TFF1, MT-ATP8 (Hama et al. 2023), MT-CO1 (Hama et al. 2023), MT-CO2 (Hama et al. 2023) EGFR Copy Number IGHG1 (Liu et al. 2021), IGKC (Liu et al. 2021), IGLC2 (Liu et al. 2021), IGHG2 (Liu et al. 2021), MT-CO2, MT-ND2, KRT5, MT-CO3, MT-RNR2, MT-ND4 CCND1 Copy Number KRT5 (Tsang et al. 2023), KRT6A (Tsang et al. 2023), KRT14 (Tsang et al. 2023), SPRR1B, KRT17, KRT16, SPRR1A, ANXA8, LY6D, S100A8 (Tsang et al. 2023) FGFR1 Copy Number SPINK1, TNC (Joshi et al. 2021), RPS4Y1, COL1A2 (Joshi et al. 2021), KRT14, MT-CO2, KRT6A, GPX2, COL1A1 (Joshi et al. 2021), POSTN (Joshi et al. 2021) Table 5: Biological plausibility of DoGMA-guided in-silico interventions. For each upstream driver node, we perturb the corresponding omics feature and rank downstream RNA or protein targets by the magnitude of their reconstructed response. The table reports the top affected targets and representative literature priors supporting their association with cancer-relevant programs. These results are intended as hypothesis-generating evidence that DoGMA-sensitive directions recover biologically coherent downstream responses. Causal-Chain Hypothesis Generation We further aggregate high-frequency multi-step paths from upstream drivers to downstream molecular states and clinical endpoints. Figure 9 shows a representative PIK3CA-anchored chain. The recovered paths connect the driver alteration to intermediate methylation and expression nodes, including both literature-supported links and candidate regulatory intermediates. We therefore treat these chains as mechanistic hypotheses generated by the model. Figure 9: PIK3CA-anchored causal-chain hypothesis generated by DoGMA. We aggregate high-frequency paths connecting the upstream PIK3CA alteration to downstream molecular states and clinical endpoints. Dashed edges indicate links supported by prior literature, whereas solid edges denote model-inferred candidate paths. The graph suggests that DoGMA organizes upstream perturbations into structured downstream molecular programs, but the inferred chains should be interpreted as hypotheses rather than experimentally validated causal mechanisms. Datasets and Experimental Setup Dataset Overview We used four dataset families to evaluate DoGMA across complementary representation-learning and transfer settings. A TCGA pan-cancer cohort was used for multi-omics pretraining. Labeled TCGA-BRCA samples were used for breast-cancer model development, and the independent METABRIC cohort was reserved for external evaluation. A TCGA-derived MetaCancer cohort was used for distant-metastasis prediction. Finally, an independently collected colorectal-cancer cohort was used for overall-survival analysis under incomplete five-omics coverage. Table 6 summarizes the experimental role, modality composition, cohort size, and partitioning protocol of each dataset. Dataset Experimental role Modalities used Cohort size Partitioning protocol TCGA pan-cancer Multi-omics representation pretraining Gene expression, DNA methylation, somatic mutation, CNA, and clinical information 8,194 specimens from 32 cancer types 4,916 training, 1,639 validation, and 1,639 test specimens TCGA-BRCA Breast-cancer model development Gene expression, somatic mutation, and CNA 631 patients Labeled training and validation partitions used for fine-tuning and model selection METABRIC External breast-cancer evaluation Gene expression, somatic mutation, and CNA 1,689 patients Independent external test cohort TCGA-derived MetaCancer Primary/non-metastatic versus metastatic classification Gene expression and DNA methylation 399 specimens from 318 cases 287/33/79 specimens and 228/26/64 cases for training/validation/test Institutional COAD/READ Overall-survival analysis Gene expression, DNA methylation, somatic mutation, CNA, proteomics, and clinical information 1,005 patients; 997 outcome-eligible 598/200/199 outcome-eligible patients for training/validation/test Table 6: Overview of the datasets used in this study. For the MetaCancer cohort, both specimen and unique-case counts are reported because multiple specimens may originate from the same TCGA case. The institutional colorectal-cancer cohort contained 1,005 patients before endpoint filtering and 997 patients with analysable survival outcomes. TCGA Pan-Cancer Pretraining Cohort The pretraining cohort comprised 8,194 TCGA tumor specimens spanning 32 cancer types. Each specimen was represented by gene-expression profiles with 6,016 features, DNA-methylation profiles with 6,617 features, somatic-mutation profiles with 4,539 features, CNA profiles with 7,460 features, and associated clinical information. Cancer type was encoded as a 32-category semantic label during representation learning. This label encouraged samples from the same tumor type to form locally consistent latent representations; it was not treated as a downstream clinical endpoint during pretraining. The cohort was divided at the specimen level using a cancer-type-stratified 60/20/20 split, yielding 4,916 training, 1,639 validation, and 1,639 test specimens. The validation partition was used for model development and checkpoint selection, whereas the test partition was held out until final representation-level evaluation. The cohort covered adrenocortical carcinoma (ACC), bladder urothelial carcinoma (BLCA), breast invasive carcinoma (BRCA), cervical squamous-cell carcinoma and endocervical adenocarcinoma (CESC), cholangiocarcinoma (CHOL), colon adenocarcinoma (COAD), lymphoid neoplasm diffuse large B-cell lymphoma (DLBC), esophageal carcinoma (ESCA), glioblastoma multiforme (GBM), head and neck squamous-cell carcinoma (HNSC), kidney chromophobe (KICH), kidney renal clear-cell carcinoma (KIRC), kidney renal papillary-cell carcinoma (KIRP), lower-grade glioma (LGG), liver hepatocellular carcinoma (LIHC), lung adenocarcinoma (LUAD), lung squamous-cell carcinoma (LUSC), mesothelioma (MESO), ovarian serous cystadenocarcinoma (OV), pancreatic adenocarcinoma (PAAD), pheochromocytoma and paraganglioma (PCPG), prostate adenocarcinoma (PRAD), rectum adenocarcinoma (READ), sarcoma (SARC), skin cutaneous melanoma (SKCM), stomach adenocarcinoma (STAD), testicular germ-cell tumor (TGCT), thyroid carcinoma (THCA), thymoma (THYM), uterine corpus endometrial carcinoma (UCEC), uterine carcinosarcoma (UCS), and uveal melanoma (UVM). Cross-Cohort Breast-Cancer Evaluation The breast-cancer experiments used the processed TCGA-BRCA and METABRIC datasets released with Moanna (Lupat et al. 2023). Labeled TCGA-BRCA samples were divided into the training and validation partitions used in our experiments. The training partition was used for downstream fine-tuning, and the validation partition was used for model and checkpoint selection. The independent METABRIC cohort was reserved exclusively for external testing. This design evaluates whether a model developed on TCGA-BRCA transfers to a cohort collected under a different study design and molecular measurement platform. The processed TCGA-BRCA dataset contained 631 patients, while the METABRIC external test cohort contained 1,689 patients. Each patient was represented by 15,592 gene-expression features, 15,592 somatic-mutation features, and 15,592 CNA features. DNA methylation was not included in this processed breast-cancer release. All compared methods and ablation variants used the same TCGA-BRCA training and validation partitions and the same METABRIC external test cohort. METABRIC samples were not used for parameter estimation, hyperparameter tuning, checkpoint selection, or early stopping. We retained the label encoding supplied with the processed data. ER status was encoded as 0 for ER-positive and 1 for ER-negative. HER2 status was encoded as 0 for HER2-positive and 1 for HER2-negative. The four-class PAM50 endpoint used 0 for Basal-like, 1 for HER2-enriched, 2 for Luminal A, and 3 for Luminal B. Basal-status classification used 0 for Basal-like and 1 for non-Basal. No Normal-like class was included in the processed PAM50 labels. The four breast-cancer tasks reported in the main paper therefore correspond to ER-status prediction, HER2-status prediction, four-class PAM50 classification, and Basal versus non-Basal classification. All four tasks were optimized using cross-entropy loss. Primary/Non-Metastatic versus Metastatic Cohort The metastasis-classification cohort was derived from the MetaCancer resource (Albaradei et al. 2021) and contained 399 TCGA tumor specimens. Gene-expression profiles with 6,016 features and DNA-methylation profiles with 6,617 features were used as model inputs. The prediction target was defined from the American Joint Committee on Cancer distant-metastasis category. Class 0 corresponded to M0 disease, indicating no evidence of distant metastasis, and is referred to as the primary/non-metastatic class. Class 1 corresponded to M1 disease, indicating evidence of distant metastasis, and was treated as the positive class when computing precision, recall, and F1 score. The endpoint therefore represents distant-metastasis status rather than the anatomical site from which a specimen was collected. The 399 specimens represented 318 unique TCGA cases because multiple aliquots could originate from the same patient. To prevent case-level leakage, partitioning was performed using the 12-character TCGA case identifier. All specimens associated with the same case were retained in a single partition. Cases were stratified by M0/M1 status. Approximately 20% of cases were assigned to the test set, after which 10% of the remaining cases were assigned to validation. The resulting training, validation, and test sets were mutually exclusive at the case level. Measure Training Validation Test Total Specimens M0 144 17 39 200 M1 143 16 40 199 Total 287 33 79 399 Unique TCGA cases M0 85 10 24 119 M1 143 16 40 199 Total 228 26 64 318 Table 7: Partition and class distribution of the MetaCancer metastasis cohort. Splitting was performed at the TCGA-case level, whereas prediction was conducted at the specimen level. Institutional Colorectal-Cancer Survival Cohort The institutional colorectal-cancer cohort initially contained 1,005 patient-level records, including 590 patients with colon adenocarcinoma (COAD), 402 with rectum adenocarcinoma (READ), five with a non-standard cancer-type entry, and eight with missing cancer-type information. Molecular measurements included gene-expression profiles with 26,029 features, DNA-methylation profiles with 6,873 features, somatic-mutation profiles with 19,136 features, CNA profiles with 19,071 features, and proteomic profiles with 7,863 features. Overall survival was the study endpoint. The variable os.event was coded as 1 for death and 0 for right censoring. The variable os.delay recorded the number of days from diagnosis to death or last follow-up. Patients missing either os.event or os.delay were excluded from survival analysis. Eight patients were excluded under this criterion, leaving 997 analysis-eligible patients, including 275 deaths and 722 right-censored observations. Patients were partitioned before model fitting using a cancer-type-stratified 60/20/20 split. Twenty per cent of patients were first assigned to the test set, after which 25% of the remaining patients were assigned to validation. Following endpoint filtering, the training, validation, and test sets contained 598, 200, and 199 patients, respectively. Partitioning was performed at the patient level, and no patient appeared in more than one subset. Measure Training Validation Test Total Cohort construction Assigned patients 603 201 201 1,005 Eligible patients 598 200 199 997 Survival outcomes Deaths 171 48 56 275 Censored 427 152 143 722 Cancer type among eligible patients COAD 354 118 118 590 READ 241 81 80 402 Other 3 1 1 5 Table 8: Partition and outcome distribution of the institutional colorectal-cancer cohort. “Assigned patients” denotes the cohort before survival-endpoint filtering, whereas “eligible patients” denotes patients with both survival time and event indicator available. Cancer-type counts are reported among analysis-eligible patients. Multi-omics measurements were not available for every patient. Gene expression was available for 977 patients, DNA methylation for 1,005, somatic mutation for 993, CNA for 1,000, and proteomics for 447. A total of 436 patients had all five molecular modalities. We retained patients with partially observed modality sets rather than restricting the analysis to complete cases. This protocol allows the survival experiment to reflect the incomplete assay coverage encountered in the institutional cohort. Modality Training Validation Test Total Gene expression 591 194 192 977 DNA methylation 603 201 201 1,005 Somatic mutation 597 200 196 993 CNA 600 200 200 1,000 Proteomics 262 88 97 447 All five modalities 256 87 93 436 Table 9: Availability of individual molecular modalities in the institutional colorectal-cancer cohort. Counts follow the fixed patient-level partition manifests and are reported before survival-endpoint filtering. “All five modalities” denotes patients with complete gene-expression, DNA-methylation, somatic-mutation, CNA, and proteomic measurements. The accompanying clinical variables included sex, age, body mass index, smoking and drinking status, family history, tumor site and location, histological grade, TNM components, overall stage, mismatch-repair status, chemotherapy status, and consensus molecular subtype. Preprocessing and Pretraining Configuration Raw molecular features were standardized using statistics estimated from the corresponding training data. For each molecular modality, a separately pretrained VQ-VAE encoder mapped the original high-dimensional input into a 256-dimensional latent representation. These modality-specific representations were subsequently processed by the central-dogma-guided Transformer-MoE backbone. The backbone contained six Transformer-MoE blocks with eight attention heads. Pretraining used AdamW with a learning rate of 5×10−55× 10^-5, a weight decay of 1×10−51× 10^-5, a batch size of 128, and a dropout rate of 0.1. The model was pretrained for 200 epochs. The masking ratio for hierarchical omics reconstruction was set to 0.15. The adversarial alignment weight was set to λadv=0.05 _adv=0.05 and was linearly warmed up over the first 60 epochs. Checkpoint selection considered both reconstruction fidelity and representation geometry. Specifically, the checkpoints achieving the best validation reconstruction loss and the best validation silhouette score were linearly interpolated to obtain the initialization used in downstream experiments. Downstream Adaptation All downstream models were initialized from the selected pretrained checkpoint. We used hierarchical unfreezing to preserve lower-level modality representations while allowing upper Transformer-MoE blocks and task-specific prediction heads to adapt to the downstream endpoint. Breast-cancer classification. The labeled TCGA-BRCA training partition was used for parameter optimization, and the corresponding validation partition was used for model and checkpoint selection. The independent METABRIC cohort was reserved for external testing. ER status, HER2 status, four-class PAM50 subtype, and Basal status were optimized using cross-entropy loss. Metastasis prediction. The MetaCancer experiment was formulated as binary classification, with M1 disease treated as the positive class. Cross-entropy loss was used for optimization. Stratified epoch-balanced sampling maintained balanced exposure to the M0 and M1 classes during training. Because specimens from the same TCGA case were retained in a single subset, evaluation was not affected by aliquot-level overlap across partitions. Survival analysis. For the institutional colorectal-cancer cohort, the fused representation was mapped to a scalar risk score and optimized using the Cox proportional-hazards objective. A modality adapter projected the additional 7,863-dimensional proteomic input into the common 256-dimensional latent space. The learning rate was 5×10−55× 10^-5 for the pretrained backbone and task-specific head and was multiplied by 3 for newly initialized proteomic modules. The downstream batch size was 32, with a weight decay of 1×10−41× 10^-4 and a dropout rate of 0.2. Hyperparameter Development and Model Selection. During pretraining, we compared base learning rates in 5×10−5,1×10−4\5× 10^-5,1× 10^-4\, batch sizes in 64,128\64,128\, supervised contrastive-loss weights λcon∈0.5,1.0 _con∈\0.5,1.0\, and reverse-direction penalties A∈−1,−2A∈\-1,-2\. We also compared three attention parameterizations: standard self-attention, a shared global penalty with edge-specific residuals, and learnable edge- and head-specific penalties. Model development was guided primarily by the minimum composite reconstruction loss on the validation partition, while the checkpoint with the highest validation silhouette score was retained as a complementary representation-oriented solution. As described above, the initialization used for downstream adaptation was obtained by linearly interpolating these two checkpoints. For breast-cancer transfer, pretrained models were fine-tuned with a learning rate of 1×10−51× 10^-5, whereas models trained from scratch and the TMO-Net baseline used 1×10−41× 10^-4. The batch size was fixed to 128, and checkpoints were selected by macro-F1 on the TCGA-BRCA validation partition. METABRIC remained untouched during model development and was used only for final external evaluation. Metastasis classification used a learning rate of 1×10−41× 10^-4, a batch size of 64, and at most 300 training epochs, with checkpoint selection based on validation F1. The five random seeds used in these classification experiments were reserved for repeated evaluation and were not treated as additional hyperparameter trials. For the institutional survival task, the manually evaluated candidates were base learning rates in 1×10−5,3×10−5,5×10−5\1× 10^-5,3× 10^-5,5× 10^-5\, batch sizes in 32,64,128\32,64,128\, unfreezing depths corresponding to the last 1,2,4,6\1,2,4,6\ Transformer blocks, and reverse-direction penalties in −1,−2\-1,-2\. The final configuration used the edge- and head-specific residual parameterization initialized from a global penalty of −1-1, a base learning rate of 5×10−55× 10^-5, a batch size of 32, and all six Transformer blocks unfrozen. Training was run for at most 150 epochs with an early-stopping patience of 30 epochs, and the checkpoint with the highest validation C-index was retained for final evaluation. Reproducibility Protocol For the breast-cancer and metastasis experiments, each method was independently fine-tuned using five random seeds. The corresponding results are reported as mean ± standard deviation across the five downstream runs. The five-seed protocol was applied to downstream fine-tuning rather than to five independent pretraining runs. The same selected pretrained checkpoint was used to initialize each downstream run. All compared methods and ablation variants used identical cohort definitions and partition memberships. Training partitions were used for parameter estimation, validation partitions for model and checkpoint selection, and test cohorts only for final evaluation. In particular, METABRIC was not used for breast-cancer model selection, and MetaCancer test cases were not accessed during training or validation. Hyperparameter Pretraining Downstream adaptation Learning rate 5×10−55× 10^-5 1×10−51× 10^-5 Batch size 128 32 Latent dimension 256 256 Attention heads 8 8 Transformer-MoE blocks 6 6 Dropout 0.1 0.2 Weight decay 1×10−51× 10^-5 1×10−41× 10^-4 Table 10: Key architecture and optimization settings used for pretraining and downstream adaptation. Hardware and Software Environment Experiments were conducted on a high-performance computing cluster equipped with NVIDIA GeForce RTX 4090 GPUs with 24 GB of memory. The implementation used Python 3.9 and PyTorch 2.0.1. Hugging Face transformers was used for clinical-text encoding, lifelines for survival-analysis utilities, and Numba with the OpenMP threading layer for accelerated numerical computation. Data Availability The processed TCGA pan-cancer datasets used for pretraining are available through the data archive (Wang et al. 2024). The processed TCGA-BRCA and METABRIC datasets are available through the Moanna data release (Lupat et al. 2023). The distant-metastasis cohort was derived from the publicly available MetaCancer resource (Albaradei et al. 2021). The institutional colorectal-cancer survival dataset will be made publicly available upon acceptance, following de-identification and completion of the applicable institutional data-release procedures. The release will include the corresponding data dictionary and fixed partition manifests required to reproduce the reported experiments.