Paper deep dive
Morphology-aware deep learning predicts colorectal cancer survival from whole-slide images
Hikmat Khan, Metin N. Gurcan, Anil Parwani, Wendy L. Frankel, Wei Chen, Muhammad Khalid Khan Niazi
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Background: Five-year survival prediction from hematoxylin-and-eosin whole-slide images could support risk stratification, but the task is weakly supervised, and the tissue morphologies carrying prognostic information remain incompletely characterized. Methods: We conducted a retrospective secondary analysis using TCGA-COAD/READ (n=562) for morphology-specific analysis, Alliance/ CALGB 89803 (n=424) for model development, and SurGen (n=218) for external evaluation. We propose MorphContrast, a morphology-aware contrastive multiple-instance learning framework that uses tissue labels as structural supervision for five-year survival prediction. MorphContrast combines gated-attention pooling with three auxiliary objectives: patch-level morphology labels define within-patient contrasts, outcome concordance or discordance defines cross-patient contrasts within matched morphologies, and diversity regularization discourages representational collapse. Morphology-matched patches from patients with discordant outcomes serve as hard negatives, encouraging prognostic discrimination within comparable histologic contexts. Results: Morphology-specific models show weak but distributed prognostic discrimination: adipose tissue achieves an area under the receiver operating characteristic curve (AUC) of 0.622, stroma achieves 0.604, and both perform comparably to tumor epithelium (0.601-0.605). On Alliance/CALGB 89803, the single model achieves an AUC of 0.642, while the post hoc ensemble reaches a pooled out-of-fold AUC of 0.70 and a risk-group hazard ratio of 3.79 (95% confidence interval: 2.55-5.63). On the external SurGen colon cancer cohort (n=218), the single model and ensemble achieve AUCs of 0.669 and 0.678, respectively. Conclusions: These findings support tissue morphology as structural supervision for outcome-aware representation learning in colorectal cancer prognosis.
Tags
Links
- Source: https://arxiv.org/abs/2610.17603v1
- Canonical: https://arxiv.org/abs/2610.17603v1
Trouble viewing inline? Open PDF directly →
Full Text
78,161 characters extracted from source content.
Expand or collapse full text
Morphology-aware deep learning predicts colorectal cancer survival from whole-slide images Hikmat Khan Email: Hikmat.khan@osumc.edu Affiliation: Department of Pathology, The Ohio State University Wexner Medical Center, Columbus, OH, USA Metin N. Gurcan Affiliation: Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA Anil Parwani Affiliation: Department of Pathology, The Ohio State University Wexner Medical Center, Columbus, OH, USA Wendy L. Frankel Affiliation: Department of Pathology, The Ohio State University Wexner Medical Center, Columbus, OH, USA Wei Chen Affiliation: Department of Pathology, The Ohio State University Wexner Medical Center, Columbus, OH, USA Muhammad Khalid Khan Niazi Affiliation: Department of Pathology, The Ohio State University Wexner Medical Center, Columbus, OH, USA Abstract Background: Five-year survival prediction from hematoxylin-and-eosin whole-slide images could support risk stratification, but the task is weakly supervised, and the tissue morphologies carrying prognostic information remain incompletely characterized. Methods: We conducted a retrospective secondary analysis using TCGA-COAD/READ (n=562n=562) for morphology-specific analysis, Alliance/ CALGB 89803 (n=424n=424) for model development, and SurGen (n=218n=218) for external evaluation. We propose MorphContrast, a morphology-aware contrastive multiple-instance learning framework that uses tissue labels as structural supervision for five-year survival prediction. MorphContrast combines gated-attention pooling with three auxiliary objectives: patch-level morphology labels define within-patient contrasts, outcome concordance or discordance defines cross-patient contrasts within matched morphologies, and diversity regularization discourages representational collapse. Morphology-matched patches from patients with discordant outcomes serve as hard negatives, encouraging prognostic discrimination within comparable histologic contexts. Results: Morphology-specific models show weak but distributed prognostic discrimination: adipose tissue achieves an area under the receiver operating characteristic curve (AUC) of 0.6220.622, stroma achieves 0.6040.604, and both perform comparably to tumor epithelium (0.6010.601–0.6050.605). On Alliance/CALGB 89803, the single model achieves an AUC of 0.6420.642, while the post hoc ensemble reaches a pooled out-of-fold AUC of 0.700.70 and a risk-group hazard ratio of 3.793.79 (95%95\% confidence interval: 2.552.55–5.635.63). On the external SurGen colon cancer cohort (n=218n=218), the single model and ensemble achieve AUCs of 0.6690.669 and 0.6780.678, respectively. Conclusions: These findings support tissue morphology as structural supervision for outcome-aware representation learning in colorectal cancer prognosis. keywordsColorectal cancer, Survival prediction, Computational pathology, Multiple-instance learning, Contrastive learning, Tissue morphology Plain language summary Pathologists routinely examine stained tissue sections from colorectal cancers, and many hospitals now store these sections as digital whole-slide images. We developed MorphContrast, a computational method that learns from both the tissue patterns in these images and patients’ five-year outcomes. Different tissue patterns each carried modest prognostic information, suggesting that useful signals were distributed across the slide rather than confined to tumor cells. In an archival tissue cohort originating from a clinical trial, the method distinguished patients with different five-year outcomes, and it showed similar discrimination in an independent external cohort. The results support the use of tissue-pattern labels to guide the development of image-based prognostic models, but further prospective and independent validation is needed before clinical use. 1 Introduction Colorectal cancer (CRC) is the third most common malignancy and the second leading cause of cancer-related death worldwide. Its annual burden is projected to reach 2.22.2 million new cases and 1.11.1 million deaths by 2030 Rawla et al. (2019), and approximately 154,000154,000 new diagnoses were anticipated in the United States in 2025 Siegel et al. (2025). Hematoxylin-and-eosin (H&E) slides are routinely generated during histopathologic assessment of surgical resections. Although only an estimated 15%15\%–20%20\% of US pathology laboratories have adopted digital pathology, this share is steadily increasing Kather et al. (2026); Khan et al. (2026a); Datwani et al. (2025); Akbar et al. (2026c), and whole-slide images (WSIs) are already captured wherever slides are digitized. Predicting five-year survival directly from these WSIs is therefore an opportunistic use of existing data, offering additional risk stratification at minimal additional cost. Two properties make the task difficult. First, supervision is weak: a gigapixel slide containing thousands of patches carries a single patient-level outcome, so a model must locate prognostically relevant regions without patch-level annotation and aggregate them into one prediction Akbar et al. (2026b). Second, it remains unclear where prognostic information resides. Several established prognostic features extend beyond malignant epithelium of the main tumor bulk: tumor budding at the invasive front is associated with poor outcome Lugli et al. (2017), the tumor–stroma ratio carries prognostic information Gao et al. (2021), and a deep-learning-derived stroma score remains prognostic after adjustment for standard clinicopathologic variables Kather et al. (2019)—yet their relative prognostic value is insufficiently characterized. The distinction bears directly on model design: signal concentrated in one morphology would justify tissue-restricted analysis, whereas weak evidence spread across many morphologies would require aggregation over the complete tissue mixture with unequal weighting. Existing approaches do not resolve this question. Attention- and transformer-based methods for multiple-instance learning (MIL) Ilse et al. (2018); Lu et al. (2021); Shao et al. (2021); Khan et al. (2025a); Tang et al. (2024) learn unequal weights over patch representations but derive patch relevance primarily from the patient outcome. Pathology foundation models provide frozen features from large-scale, organ-agnostic pretraining Chen et al. (2024); Xu et al. (2024), and morphology-aware CRC models incorporate tissue information either as an input feature, as an intermediate representation, or as an auxiliary annotation Kather et al. (2019); Khan et al. (2026b); Sajjad et al. (2026). However, the approaches considered here do not jointly organize patch representations by tissue morphology and patient outcome. In particular, they do not explicitly compare matched morphologies across patients with concordant and discordant outcomes and therefore do not directly target prognostic variation within a morphology class. We therefore begin with an empirical question: how much prognostic information does each tissue morphology carry in isolation? The MorphDistill encoder Khan et al. (2026b) distills morphology-specific knowledge from several pathology foundation models and assigns CRC morphology labels to image patches. Using assignments for 1717 tissue morphologies, we train a separate attention-based MIL survival model for each morphology in The Cancer Genome Atlas colon and rectal adenocarcinoma cohorts (TCGA-COAD/READ). Across these morphology-specific models, the prognostic signal is weak and graded rather than concentrated in any single tissue class: several non-tumor morphologies perform comparably to tumor epithelium, yet no individual morphology is strongly predictive(Fig. 1, Table 1). We do not interpret this as a direct biological effect, since spatial context and correlated clinicopathologic factors are uncontrolled. The purpose is methodological: prognostic evidence is weak, graded, and spread across tumor and non-tumor morphologies, which motivates using tissue morphology annotations as auxiliary supervision for patient-level survival prediction rather than only as a means of distinguishing morphology classes. This observation motivates the design of MorphContrast, the central contribution of this work. Because prognostic information is distributed across morphologies, including non-tumor tissue, MorphContrast processes the complete tissue mixture rather than restricting analysis to tumor patches. Gated attention learns the relative contribution of each patch to the patient-level prediction. An intra-patient contrastive objective (ℒintraL_intra) organizes patch representations by morphology within each patient, whereas an inter-patient objective (ℒinterL_inter) incorporates outcome information: morphology-matched patches from patients with concordant outcomes form positive pairs, and those from patients with discordant outcomes form hard negatives. A diversity regularizer (ℒdivL_div) discourages projected patch representations from collapsing toward a common direction. These objectives are optimized jointly with the survival loss, while the MorphDistill patch encoder remains frozen. Together, these objectives distinguish MorphContrast from conventional attention-based MIL, which learns patch importance primarily from patient-level outcomes, and from morphology-aware pipelines in which tissue labels enter as static features or intermediate predictions Sajjad et al. (2026). MorphContrast instead uses morphology and outcome jointly to define contrastive relationships that target outcome-associated variation within matched histologic contexts. We evaluate MorphContrast on Alliance/CALGB 89803 as the primary cohort and on SurGen Myles et al. (2025) as an external cohort. On Alliance/CALGB 89803, the single model reaches an AUC of 0.6420.642, and an internally selected post hoc ensemble reaches a pooled OOF AUC of 0.700.70. On SurGen (n=218n=218), the single model and ensemble achieve AUCs of 0.6690.669 and 0.6780.678, respectively; the ensemble result is similar to that of FR-MIL, the strongest evaluated baseline (0.67660.6766). These results support morphology-informed representation learning within the evaluated cohorts but do not establish a clear external performance advantage. Our contributions can be summarized as follows: 1. A motivating morphology-specific analysis that estimates morphology-specific prognostic discrimination for 1717 CRC tissue morphologies using one survival model per class, characterizing the evidence as weak, graded, and distributed across tumor and non-tumor tissue. It is offered as methodological motivation rather than as a biological finding. 2. MorphContrast, a morphology-aware contrastive MIL framework in which tissue morphology labels supervise the structure of prognostic representations rather than serve only as post hoc annotations. The contrastive form follows supervised contrastive learning Khosla et al. (2020); the contribution lies in jointly optimizing morphology-aware contrastive objectives with the five-year survival objective. 3. Morphology-conditioned outcome contrast, which aligns morphology-matched patches from patients with concordant outcomes and treats morphology-matched patches from patients with discordant outcomes as hard negatives, thereby targeting prognostic variation within comparable histologic contexts. 4. Evaluation on a randomized trial cohort with independent external assessment, using Alliance/CALGB 89803 as the primary cohort and SurGen Myles et al. (2025) as the external cohort. The single model reaches an AUC of 0.6420.642, with sensitivity of 63.53%63.53\% and specificity of 55.37%55.37\%, on Alliance/CALGB 89803, while the post hoc ensemble reaches an AUC of 0.700.70, with sensitivity and specificity of 68.0%68.0\% each. On SurGen (n=218n=218), the single model and ensemble reach AUCs of 0.6690.669 and 0.6780.678, sensitivities of 57.58%57.58\% and 65.15%65.15\%, and specificities of 70.39%70.39\% and 66.45%66.45\%, respectively; the ensemble AUC is similar to that of FR-MIL, the strongest evaluated baseline (0.6760.676). Figure 1: Top-performing CRC tissue morphologies show modest prognostic discrimination. Each bar gives the five-fold cross-validated AUC of an attention-based multiple-instance learning (ABMIL) survival model trained on patches of a single morphology class in the TCGA-COAD/READ cohort (n=562n=562; Table 1); the dashed line marks chance (0.500.50). Only the top-performing morphologies with AUC values above 0.500.50 are displayed. Non-tumor morphologies—adipose tissue, stroma, normal epithelium, and muscle—score comparably to tumor-epithelial classes, while the displayed morphology-specific models remain modest predictors. Because patch abundance, spatial context, and clinicopathologic correlates are uncontrolled, this analysis motivates the design of MorphContrast rather than supporting a biological interpretation. 2 Related work MIL represents a WSI as a bag of patches and assigns a single label to the bag Ilse et al. (2018). Gated attention, CLAM, DSMIL, TransMIL, RRT-MIL, FR-MIL, and PANTHER improve aggregation through learned weighting, instance clustering, critical-instance selection, self-attention, regional re-embedding, feature recalibration, and prototype modeling, respectively Ilse et al. (2018); Lu et al. (2021); Li et al. (2021); Shao et al. (2021); Tang et al. (2024); Chikontwe et al. (2022); Song et al. (2024). Prognostic models such as those of Nakanishi et al. Nakanishi et al. (2023) and PROGPATH Yuan et al. (2025) apply related weakly supervised principles to recurrence and survival prediction, respectively. These methods can identify predictive patches but do not use explicit morphology labels to structure the prognostic representation Yuan et al. (2025); Nakanishi et al. (2023). MorphContrast retains gated-attention pooling and adds morphology- and outcome-aware auxiliary contrastive objectives together with diversity regularization. ACMIL reduces excessive attention concentration, whereas MHIM-MIL masks highly salient instances so that a student learns from harder instances Zhang et al. (2024); Tang et al. (2023). Both use attention-derived salience to determine which instances are masked during training. MorphContrast instead defines a hard negative as a patch with the same morphology as the anchor but from a patient with an outcome discordant with that of the anchor, so it changes what the representation encodes rather than which instances the aggregator receives. Contrastive learning attracts positive pairs and repels negatives defined by augmentation or class labels Chen et al. (2020); Khosla et al. (2020); Khan et al. (2025b). Large pathology foundation models use related objectives to produce transferable frozen features Chen et al. (2024); Lu et al. (2024); Xu et al. (2024); Zimmermann et al. (2024); Wang et al. (2022); Saillard et al. (2024); kaiko.ai et al. (2024); Kang et al. (2023); Filiot et al. (2024). Because these pretraining objectives do not explicitly use CRC morphology labels or patient outcomes, they do not constrain the resulting features to organize tissue identity and patient outcome jointly. MorphContrast adopts the supervised-contrastive formulation Khosla et al. (2020) but constructs its pairs from morphology and outcome and optimizes the corresponding objectives jointly with survival prediction. CRC studies have shown that non-tumor morphology can carry prognostic information Kather et al. (2019). MorphDistill Khan et al. (2026b) provides a patch embedding together with an 1818-class morphology label, and PRISM Sajjad et al. (2026) fuses morphology-informed and foundation-model features through cross-feature interaction and gated attention. These methods use morphology as an input, an intermediate representation, or auxiliary supervision. MorphContrast likewise starts from the patch embeddings and labels provided by the frozen MorphDistill encoder but uses morphology and outcome jointly to define representation-learning relationships, targeting prognostic variation within matched histologic contexts. Together, the analyses show that morphology labels can provide structural supervision for outcome-aware representations. The externally assessed discrimination is comparable to the strongest evaluated baseline, although prospective validation is required before clinical use. Figure 2: Overview of MorphContrast. (a) A frozen MorphDistill encoder Khan et al. (2026b) assigns every patch one of 1818 CRC morphology classes, shown as a color overlay on a whole-slide image; this step is not trained here. (b) Tissue is segmented and tiled, and the same frozen encoder yields a feature vector and a morphology label mbi∈0,…,17m_bi∈\0,…,17\ for each patch. A shared feature encoder then supplies two pathways. Along the predictive path (blue arrows), gated attention weights the patches and a fully connected network maps the pooled representation h¯b h_b (labeled “Slide Features”) to the five-year survival prediction under ℒsurvL_surv. Along the contrastive path (green arrows), a projection head is trained with intra-patient morphology contrast (ℒintraL_intra), outcome-aware inter-patient contrast (ℒinterL_inter), and diversity regularization (ℒdivL_div); the figure writes these as ℒintra-patientL_intra-patient, ℒinter-patientL_inter-patient, and ℒdiversityL_diversity. Contrastive pairs are drawn from the class-balanced patient pool (Section 3.3), from which one outcome-concordant and one outcome-discordant patient bag are sampled per anchor; losses from both pathways are accumulated across the three bags before a single backward pass. Snowflakes mark frozen components. Only the feature encoder, attention pooler, and survival classifier are retained at inference. 3 Methods 3.1 Overview Figure 2 summarizes MorphContrast. A frozen MorphDistill encoder Khan et al. (2026b) provides a 768768-dimensional feature vector and a morphology label for each retained patch. A shared feature encoder adapts these features for two pathways: gated-attention MIL produces a patient-level five-year survival prediction, while a projection head supports morphology- and outcome-aware contrastive learning. For each anchor patient, the training procedure samples one different patient with the same outcome and one with an outcome discordant with that of the anchor from a class-balanced pool constructed exclusively from the training partition of the current data split. Patches from the anchor and sampled patients define intra-patient morphology pairs, inter-patient outcome-aware pairs, and a diversity penalty. All losses are accumulated before a single backward pass. At inference, the patient pool and projection head are removed. 3.2 Architecture Let patient b be represented by NbN_b patches with precomputed MorphDistill features given by Xb=xbii=1NbX_b=\x_bi\_i=1^N_b, where xbi∈ℝ768x_bi ^768, with corresponding morphology labels mbi∈0,…,17m_bi∈\0,…,17\ and a patient-level outcome yb∈0,1y_b∈\0,1\, where yb=1y_b=1 denotes death within five years and yb=0y_b=0 denotes survival for at least five years. The MorphDistill encoder remains frozen throughout training. Shared feature encoder. A two-layer task-adaptation network maps each frozen patch feature to an encoded representation of the same dimensionality: hbi=gθ(xbi)∈ℝ768.h_bi=g_θ(x_bi) ^768. (1) The encoded patches hbih_bi are shared by the survival and contrastive pathways, allowing losses from both pathways to shape the representations used for patient-level prediction (Fig. 2). Projection head. During training, each encoded patch is mapped to a lower-dimensional contrastive space: zbi=fϕ(hbi)∥fϕ(hbi)∥2,zbi∈ℝdp,∥zbi∥2=1,dp∈128,256.z_bi= f_φ(h_bi) f_φ(h_bi) _2, z_bi ^d_p, z_bi _2=1, d_p∈\128,256\. (2) The projection head fϕf_φ contains two linear layers separated by a ReLU activation. For the pairwise objectives below, patches are indexed across the sampled patients by i, with ziz_i, bib_i, and mim_i denoting the projected representation, patient identity, and morphology label, respectively. Because each projected representation has unit L2L_2 norm, the inner product of two projected patches equals their cosine similarity. We define the temperature-scaled similarity as sij=zi⊤zjτ.s_ij= z_i z_jτ. (3) Here, τ>0τ>0 controls the concentration of the contrastive distribution. We restrict τ∈(0,1]τ∈(0,1] and set τ=0.1τ=0.1 in all experiments. The projection head is used only during training and is removed at inference. Gated-attention aggregation. The survival pathway applies gated attention to the encoded patches. Following ABMIL Ilse et al. (2018), the attention weight assigned to patch i is abi=softmaxi(w⊤[tanh(Wvhbi)⊙σ(Wuhbi)]).a_bi=softmax_i (w [ (W_vh_bi) σ(W_uh_bi) ] ). (4) Here, Wv,Wu∈ℝda×768W_v,W_u ^d_a× 768 and w∈ℝdaw ^d_a are trainable parameters, dad_a is the attention hidden dimension, ⊙ denotes elementwise multiplication, and σ is the logistic sigmoid. The softmax is computed over the NbN_b patches of patient b, such that ∑i=1Nbabi=1 _i=1^N_ba_bi=1. The resulting patient-level representation is h¯b=∑i=1Nbabihbi. h_b= _i=1^N_ba_bih_bi. (5) This aggregation allows individual patches and tissue morphologies to contribute unequally to the patient representation. Survival classifier. A classifier maps the patient representation h¯b h_b to a scalar survival logit: ηb=qψ(h¯b),p^b=σ(ηb), _b=q_ψ( h_b), p_b=σ( _b), (6) where p^b p_b is the predicted probability of death within five years. 3.3 Class-Balanced Patient Pool Inter-patient contrast requires patients from both outcome groups. For each data split, inter-patient contrast uses a class-balanced patient pool constructed exclusively from the training partition, preventing patient-level information leakage. The training pool contains equal numbers of patients who survived for at least five years and patients who died within five years. For each anchor patient, one different outcome-concordant patient and one outcome-discordant patient are sampled from this training-only pool and processed by the shared feature encoder. Up to 512512 morphology-balanced patches are retained per patient to limit memory use and reduce dominance by highly abundant tissue compartments. Each retained patch is associated with its projected representation, patient identity, morphology label, and patient outcome: (zi,bi,mi,ybi).(z_i,b_i,m_i,y_b_i). (7) These attributes define the patch-level positive and negative sets used by the contrastive objectives. 3.4 Patch-Level Contrastive Objectives For an anchor patch i with positive set P(i)P(i) and negative set N(i)N(i), we use the sum-over-positives supervised contrastive objective ℓ(i,P,N)=−log∑p∈P(i)exp(zi⊤zp/τ)∑k∈P(i)∪N(i)exp(zi⊤zk/τ). (i;P,N)=- _p∈ P(i) (z_i z_p/τ) _k∈ P(i)∪ N(i) (z_i z_k/τ). (8) The anchor itself is excluded from both sets. Anchors without at least one valid positive and one valid negative are omitted from the corresponding loss. Intra-patient morphology contrast. The intra-patient objective is designed to organize patch representations by tissue morphology within each patient. For anchor patch i, positives are patches from the same patient with the same morphology, Pa(i)=j≠i:bj=bi,mj=miP_a(i)=\j≠ i:b_j=b_i,\;m_j=m_i\, whereas negatives are patches from the same patient with a different morphology, Na(i)=j:bj=bi,mj≠miN_a(i)=\j:b_j=b_i,\;m_j≠ m_i\. The intra-patient loss is ℒintra=1|a|∑i∈aℓ(i,Pa,Na),L_intra= 1|A_a| _i _a (i;P_a,N_a), (9) where aA_a contains anchors with at least one same-morphology positive and one different-morphology negative. Inter-patient outcome-aware contrast. The inter-patient objective uses patient outcome to define contrastive relationships within each morphology class. For anchor patch i, positives are morphology-matched patches from a different patient with the same outcome, Pe(i)=j:bj≠bi,mj=mi,ybj=ybiP_e(i)=\j:b_j≠ b_i,\;m_j=m_i,\;y_b_j=y_b_i\. Morphology-matched patches from patients with an outcome discordant with that of the anchor patient form hard negatives, Nehard(i)=j:bj≠bi,mj=mi,ybj≠ybiN_e^hard(i)=\j:b_j≠ b_i,\;m_j=m_i,\;y_b_j≠ y_b_i\. Patches belonging to different morphologies provide standard negatives, Nestd(i)=j≠i:mj≠miN_e^std(i)=\j≠ i:m_j≠ m_i\. The complete negative set is Ne(i)=Nehard(i)∪Nestd(i)N_e(i)=N_e^hard(i)∪ N_e^std(i), and the inter-patient loss is ℒinter=1|e|∑i∈eℓ(i,Pe,Ne),L_inter= 1|A_e| _i _e (i;P_e,N_e), (10) where eA_e denotes the set of valid inter-patient anchors. Same-patient, same-morphology patches are excluded from this objective. The morphology-matched hard negatives encourage the representation to capture outcome-associated variation within comparable histologic contexts rather than tissue identity alone. Diversity regularization. A diversity term discourages the projected patches of a patient from collapsing toward a common direction Belagali et al. (2026). For patient b with KbK_b sampled patches, ℓbdiv=1Kb(Kb−1)∑i≠jzi⊤zj,ℒdiv=1B∑b=1Bℓbdiv. _b^div= 1K_b(K_b-1) _i≠ jz_i z_j, _div= 1B _b=1^B _b^div. (11) Here, the inner sum is over ordered pairs of distinct patches from patient b, and B is the number of patients contributing to the diversity term. Minimizing this term encourages dispersion of a patient’s projected patch representations rather than enforcing exact pairwise orthogonality. 3.5 Survival Objective and Joint Optimization The patient-level survival loss is computed after gated-attention pooling. For binary cross-entropy, ℒsurv=−1B∑b=1B[yblogp^b+(1−yb)log(1−p^b)].L_surv=- 1B _b=1^B [y_b p_b+(1-y_b) (1- p_b) ]. (12) Weighted cross-entropy and focal-loss variants are additionally considered during the ensemble search. The complete training objective is ℒ=λsℒsurv+λaℒintra+λeℒinter+λdℒdiv.L= _sL_surv+ _aL_intra+ _eL_inter+ _dL_div. (13) The survival-loss weight is fixed at λs=1.0 _s=1.0. The auxiliary-loss weights satisfy λa,λe,λd∈[0,1] _a, _e, _d∈[0,1]. For the model configuration, we use λa=0.25 _a=0.25, λe=0.1 _e=0.1, and λd=0.25 _d=0.25. For each update, ℒsurvL_surv is computed from the anchor prediction, while the auxiliary losses use patches from the anchor and the two sampled patients. A single backward pass updates the shared feature encoder through all four loss terms; ℒsurvL_surv also updates the attention network and classifier, whereas the auxiliary losses update the projection head. Thus, patch-level contrastive supervision reshapes the features later pooled for survival prediction. At inference, only the precomputed MorphDistill features, shared feature encoder, gated-attention pooler, and survival classifier are used. 3.6 Study design and cohorts This study is a retrospective secondary analysis of digitized archival histology and pre-existing outcome data. The three cohorts serve distinct roles. TCGA-COAD/READ is used for the morphology-specific analysis that empirically characterizes differences in prognostic value across tissue morphologies and motivates the proposed method (Section 4.1); Alliance/CALGB 89803 serves as the primary development cohort and provides the basis for the main comparisons in Sections 4.3–4.4; and SurGen serves as the independent external cohort. All cohorts use nominal 224×224224× 224 patches at 20×20×, with patches containing less than 25%25\% tissue discarded. Tissue regions are segmented using TRIDENT, and patches are extracted without overlap. The following paragraphs briefly describe each cohort and its role in the study. 3.6.1 Primary cohort and trial design The primary cohort derives from Alliance/CALGB 89803, a randomized phase I trial comparing adjuvant weekly bolus irinotecan with fluorouracil and leucovorin (IFL) and fluorouracil and leucovorin (FL) alone in 1,2641,264 patients with completely resected stage I colon cancer Saltz et al. (2007); National Cancer Institute (2004). CALGB subsequently became part of the Alliance for Clinical Trials in Oncology; we therefore refer to the cohort throughout as Alliance/CALGB 89803. Eligible patients had received no prior chemotherapy or radiotherapy, had performance status 00–22, and began treatment within 2121–5656 days of resection, with randomization stratified by nodal involvement (11–33 versus ≥4≥ 4 nodes), histological grade, and preoperative carcinoembryonic antigen. 3.6.2 Digitization and quality assurance Slides were digitized at 40×40× (0.25μm0.25\, per pixel) on an Aperio scanner (Leica Biosystems, Nussloch, Germany) at the participating academic medical center under an approved institutional review board protocol Sajjad et al. (2026). A trained pathologist, blinded to clinical information and outcome, reviewed each slide to confirm the presence of tumor; more than 98%98\% of cases contributed a single representative tumor slide. Quality control assessed tissue detection and coverage, color fidelity, focus, and scanning artifacts, and slides with inadequate staining were removed. This study was reviewed and approved by an institutional review board. 3.6.3 Cohort construction and endpoint Cases were excluded for absence of tumor, lymph-node-only specimens, non-disease-related death within five years, and loss to follow-up before five years. Patients censored before five years were excluded rather than censored, because the endpoint of interest is cancer-specific rather than all-cause mortality—a distinction that carries weight in a disease with a median age at diagnosis of 6666 years and substantial competing comorbidity Rogers and Johnson (2021). This exclusion produces a complete-case binary endpoint and restricts the analysis to patients with known five-year outcomes. Mucinous adenocarcinomas were also excluded, as in prior analyses of this cohort Sajjad et al. (2026), because their architecture precludes reliable assessment of tumor budding; following the World Health Organization definition of mucinous carcinoma as a tumor in which extracellular mucin constitutes at least 50%50\% of tumor volume, 5050 such cases were removed. The resulting cohort comprises 424424 patients and 424424 WSIs; among the patients, 103103 died within five years and 321321 survived for at least five years. All tumors arose in the cecum, ascending colon, hepatic flexure, transverse colon, splenic flexure, descending colon, or sigmoid colon. Supplementary Table 1 summarizes the clinicopathologic characteristics. 3.6.4 Discovery cohort: TCGA-COAD/READ The colon and rectal adenocarcinoma collections of The Cancer Genome Atlas The Cancer Genome Atlas Network (2012); National Cancer Institute (2026) provide 562562 patients with diagnostic formalin-fixed, paraffin-embedded (FFPE) slides and follow-up and are used exclusively for the morphology-specific empirical analysis described in Section 4.1. The morphology-specific models are evaluated using five-fold cross-validation with patient-level splits. The TCGA cohort differs from Alliance in scanning protocol and scanner vendor, patient population, and stage distribution; unlike the Alliance cohort, mucinous cases were retained. Slides are tiled using the shared protocol of 224×224224× 224 patches at 20×20×. Because the TCGA slides were scanned at a base resolution close to that of Alliance (≈0.24μm≈ 0.24\, per pixel versus 0.250.25), the resulting patches cover a comparable physical field. 3.6.5 External cohort: SurGen SurGen Myles et al. (2025) is a public colorectal digital pathology resource comprising 1,0201,020 H&E WSIs from 843843 cases collected through NHS Lothian. We use the survival-annotated SR386 subset, which contains 427427 primary colorectal cases with five-year outcomes available for 426426; the external evaluation is restricted to 218218 colon cancer cases from this subset to align the disease site with that of the colon-only Alliance cohort. The source subset includes both colon and rectal cancers across a broader stage distribution than Alliance. All slides were digitized on a ZEISS Axio Scan.Z1 at 40×40× and 0.1112μm0.1112\, per pixel, which differs substantially from the Alliance acquisition protocol. These differences in scanner resolution and case mix introduce potential distribution shift between the development and external cohorts. 3.7 Comparison Methods For the Alliance cohort, we compare MorphContrast with four groups of methods: (i) Attention- and transformer-based MIL: ABMIL Ilse et al. (2018), CLAM Lu et al. (2021), TransMIL Shao et al. (2021), and RRT-MIL Tang et al. (2024). (i) Task-specific CRC prognosis: the CRC recurrence-prediction model of Nakanishi et al. Nakanishi et al. (2023). (i) Morphology-aware CRC prognosis: MorphDistill with its standard survival head Khan et al. (2026b) and PRISM Sajjad et al. (2026). (iv) Frozen pathology foundation encoders: UNI v1/v2 Chen et al. (2024), Virchow2 Zimmermann et al. (2024), GigaPath Xu et al. (2024), CONCH v1.5 Lu et al. (2024), CTransPath Wang et al. (2022), H-Optimus-0 Saillard et al. (2024), Phikon v2 Filiot et al. (2024), Kaiko ViT-B/8 kaiko.ai et al. (2024), and Lunit ViT-S/8 Kang et al. (2023). Each frozen encoder is coupled with the same gated-attention survival head to hold the aggregation architecture constant when comparing patch representations. Methods in groups (i)–(i) use their published configurations and default decision thresholds. On the external SurGen cohort, we compare MorphContrast with three MIL baselines: TransMIL Shao et al. (2021) , FR-MIL Chikontwe et al. (2022), and MHIM-MIL Tang et al. (2023). These methods represent transformer aggregation, feature recalibration, and masked hard-instance mining, respectively. All models reported for SurGen are trained on Alliance/CALGB 89803 and applied in evaluation mode without additional training; preprocessing parameters, ensemble membership, and decision thresholds are fixed before external evaluation. Because the baseline suites differ between Alliance and SurGen, comparisons are made only within each cohort. 3.8 Statistical analyses We evaluate discrimination using the area under the receiver operating characteristic curve (AUC) and Harrell’s concordance index (C-index) Harrell et al. (1996). We additionally report balanced accuracy, sensitivity, and specificity at each model’s operating threshold, with death within five years defined as the positive class. Sensitivity, specificity, and balanced accuracy are computed from true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) as Sen =TPTP+FN, = TPTP+FN, (14) Spec =TNTN+FP, = TNTN+FP, BalAcc =12(Sen+Spec). = 12 (Sen+Spec ). Risk stratification is assessed using the log-rank test and a univariate Cox proportional-hazards model, with hazard ratios and 95%95\% confidence intervals Sajjad et al. (2026). For Alliance, baselines and the single MorphContrast model are reported as mean ± standard deviation across folds, whereas the ensemble uses pooled out-of-fold predictions with 1,0001,000-sample bootstrap 95%95\% confidence intervals. For SurGen, all methods are reported as point estimates with bootstrap 95%95\% confidence intervals for the independent cohort of 218218 cases. 3.9 Implementation and Training Details Alliance model development uses five-fold cross-validation with patient-level splits. For each evaluation split, patients are allocated in proportions of 70%70\%, 15%15\%, and 15%15\% to the training, validation, and test sets, respectively. The validation split is used for hyperparameter selection and early stopping, and the corresponding held-out test split is used only to report performance. Models are trained using the AdamW optimizer Loshchilov and Hutter (2019) for up to 150150 epochs with early stopping when the validation loss does not improve for 1515 consecutive epochs. The learning rate follows a cosine annealing schedule Loshchilov and Hutter (2017). The MorphDistill patch encoder remains frozen throughout training; only the task-specific feature encoder, projection head, gated-attention module, and survival classifier are optimized. Supplementary Table 2 summarizes the hyperparameter search space and fixed training settings. The initial search over the contrastive-loss weights selected λa=0.25 _a=0.25 and λe=0.1 _e=0.1, which are fixed in subsequent experiments. The MorphContrast uses binary cross-entropy with λs=1.0 _s=1.0 and the default objective weights described in Section 3.5. Weighted cross-entropy and focal loss are additionally evaluated among candidate configurations considered for the post hoc ensemble. 3.10 Post Hoc Ensemble Selection from Out-of-Fold Predictions Each hyperparameter configuration generates one out-of-fold (OOF) risk score per patient under five-fold cross-validation. Candidate configurations are retained only if they achieve AUC≥0.50AUC≥ 0.50, sensitivity ≥0.30≥ 0.30, and specificity ≥0.30≥ 0.30. Ensemble selection varies the number of members k∈3,5,7,10k∈\3,5,7,10\, survival-loss variant, member-selection strategy, score-aggregation method, and decision threshold, yielding 5050 candidate ensembles (Supplementary Table 2). Candidates are ranked using ens=AUC+BalAcc−12|Sen−Spec|,J_ens=AUC+BalAcc- 12 |Sen-Spec |, (15) which favors discrimination and balanced classification while penalizing large differences between sensitivity and specificity. The selected ensemble contains three Pareto-selected members. Their OOF risk scores are converted to percentile ranks and aggregated to obtain a single patient-level ensemble score, with a final decision threshold of t⋆=0.573t =0.573. We refer to the single model as the MorphContrast model and to the ensemble as the post hoc MorphContrast ensemble. 4 Results 4.1 Prognostic Discrimination Across Individual Tissue Morphologies To quantify the prognostic discrimination associated with individual tissue morphologies, we trained a separate ABMIL survival model for each morphology class in TCGA-COAD/READ. Among the 1717 non-background classes, four (low-grade adenoma dysplasia, mucus, necrosis, and sessile serrated lesion) produced constant predictions, while inflammation and hyperplastic polyp yielded AUCs of 0.500.50. Of the remaining 1111 morphologies, ten achieved AUCs above 0.500.50 and eight exceeded 0.550.55; the vessels class was the only class below chance (AUC 0.4660.466) (Table 1, Fig. 1). Adipose tissue achieved the highest AUC (0.6220.622), followed by high-grade adenocarcinoma (0.6050.605), high-grade adenoma dysplasia (0.6050.605), stroma (0.6040.604), low-grade adenocarcinoma (0.6010.601), and normal epithelium (0.5960.596). The highest-ranking classes therefore included both tumor-associated and non-tumor compartments, and no individual morphology exceeded an AUC of 0.6220.622. Table 1: Morphology-specific five-year survival prediction. Five-fold cross-validated ABMIL models trained independently using patches from a single morphology class in TCGA-COAD/READ. Morphology AUC Sen Spec Fat (adipose) 0.6220.622 0.5730.573 0.6090.609 Adenocarcinoma high-grade 0.605¯ 0.605 0.6980.698 0.4470.447 Adenoma high-grade dysp. 0.605¯ 0.605 0.6840.684 0.3870.387 Stroma 0.6040.604 0.5810.581 0.5280.528 Adenocarcinoma low-grade 0.6010.601 0.6370.637 0.4910.491 Normal epithelium 0.5960.596 0.6080.608 0.5410.541 Muscle 0.5870.587 0.5180.518 0.6190.619 Debris 0.5520.552 0.4560.456 0.5860.586 Fibrosis 0.5250.525 0.2460.246 0.8460.846 Lymphocyte 0.5250.525 0.4020.402 0.6000.600 Inflammation 0.5000.500 0.7200.720 0.2800.280 Hyperplastic polyp 0.5000.500 0.1600.160 0.8480.848 Adenoma low-grade dysp. 0.5000.500 0.0000.000 1.0001.000 Mucus 0.5000.500 0.0000.000 1.0001.000 Necrosis 0.5000.500 0.0000.000 1.0001.000 Sessile serrated lesion 0.5000.500 0.0000.000 1.0001.000 Vessels 0.4660.466 0.3700.370 0.5990.599 4.2 Single-Model and Ensemble Performance The MorphContrast model achieved an AUC of 0.6420.642, balanced accuracy of 0.5940.594, sensitivity of 0.6350.635, and specificity of 0.5540.554 (Table 2). The post hoc ensemble achieved a pooled OOF AUC of 0.700.70 (95%95\% CI: 0.650.65–0.760.76), with balanced accuracy, sensitivity, and specificity of 0.680.68, 0.680.68, and 0.680.68, respectively. Table 2: Performance of the MorphContrast model and post hoc ensemble. Single-model values are five-fold cross-validation means ± standard deviations. Ensemble values are computed from pooled OOF predictions, with 1,0001,000-sample bootstrap 95%95\% confidence intervals. Model AUC ↑ Bal-Acc ↑ Sen ↑ Spec ↑ MorphContrast 0.642¯±0.05 0.642± 0.05 0.594±0.0360.594± 0.036 0.635±0.0920.635± 0.092 0.554±0.0760.554± 0.076 MorphContrast (ensemble) 0.700.70 (0.65,0.76)(0.65,0.76) 0.680.68 (0.63,0.73)(0.63,0.73) 0.680.68 (0.59,0.73)(0.59,0.73) 0.680.68 (0.62,0.73)(0.62,0.73) 4.3 Comparison with MIL and Morphology-Aware Baselines Among the conventional MIL methods, TransMIL achieved the highest AUC (0.67±0.050.67± 0.05), followed by CLAM (0.66±0.060.66± 0.06) and ABMIL (0.63±0.060.63± 0.06) (Table 3). TransMIL yielded 5.25%5.25\% sensitivity and 99.74%99.74\% specificity, while CLAM yielded 25.61%25.61\% sensitivity and 84.68%84.68\% specificity. The task-specific model of Nakanishi et al. achieved an AUC of 0.61±0.060.61± 0.06, and RRT-MIL achieved 0.59±0.050.59± 0.05. The morphology-aware baselines achieved AUCs of 0.68±0.080.68± 0.08 for MorphDistill and 0.70±0.040.70± 0.04 for PRISM. The MorphContrast model achieved 0.642±0.050.642± 0.05, while the post hoc ensemble achieved a pooled OOF AUC of 0.700.70. For C-index, PRISM achieved 0.6720.672, followed by the MorphContrast ensemble (0.6650.665), MorphDistill (0.6610.661), and TransMIL (0.6470.647). Table 3: Comparison with MIL, morphology-aware, and task-specific CRC prognosis baselines. Baselines and the single MorphContrast model report mean ± standard deviation over folds; the ensemble reports pooled OOF performance. Model AUC ↑ Bal-Acc (%) ↑ Sen (%) ↑ Spec (%) ↑ C-index ↑ ABMIL Ilse et al. (2018) 0.63±0.060.63± 0.06 58.54±4.5058.54± 4.50 51.18±9.8351.18± 9.83 65.89±4.5065.89± 4.50 0.6070.607 CLAM Lu et al. (2021) 0.66±0.060.66± 0.06 55.15±3.3755.15± 3.37 25.61±8.1825.61± 8.18 84.68±8.8984.68± 8.89 0.6030.603 TransMIL Shao et al. (2021) 0.67±0.050.67± 0.05 52.50±4.6452.50± 4.64 5.25±9.115.25± 9.11 99.74±0.0599.74± 0.05 0.6470.647 Nakanishi Nakanishi et al. (2023) 0.61±0.060.61± 0.06 58.31±5.7758.31± 5.77 53.49±11.1153.49± 11.11 63.13±2.7563.13± 2.75 0.6040.604 RRT-MIL Tang et al. (2024) 0.59±0.050.59± 0.05 56.76±4.4656.76± 4.46 48.58±23.9248.58± 23.92 64.94±22.1964.94± 22.19 0.5750.575 MorphDistill Khan et al. (2026b) 0.68¯±0.08 0.68± 0.08 63.41±4.7063.41± 4.70 60.24±5.3860.24± 5.38 66.57±8.8166.57± 8.81 0.6610.661 PRISM Sajjad et al. (2026) 0.70±0.040.70± 0.04 68.00±1.7968.00± 1.79 67.14±8.2667.14± 8.26 68.86±8.6268.86± 8.62 0.6720.672 MorphContrast 0.642±0.050.642± 0.05 59.45±3.5659.45± 3.56 63.53±9.2363.53± 9.23 55.37±7.5955.37± 7.59 0.6160.616 MorphContrast (ensemble) 0.700.70 (0.65,0.76)(0.65,0.76) 0.680.68 (0.63,0.73)(0.63,0.73) 0.680.68 (0.59,0.73)(0.59,0.73) 0.680.68 (0.62,0.73)(0.62,0.73) 0.665¯ 0.665 4.4 Comparison with Frozen Pathology Foundation Encoders Using the same gated-attention survival head, AUCs for the ten frozen pathology foundation encoders ranged from 0.510.51 to 0.630.63 (Table 4). UNI v2 achieved the highest AUC (0.63±0.030.63± 0.03), followed by UNI v1 (0.62±0.050.62± 0.05), H-Optimus-0 (0.61±0.040.61± 0.04), and GigaPath (0.60±0.040.60± 0.04). CONCH v1.5 achieved an AUC of 0.51±0.070.51± 0.07. The MorphContrast model achieved 0.642±0.050.642± 0.05 under the same survival-head design. MorphDistill and PRISM, included as morphology-aware reference models, achieved AUCs of 0.68±0.080.68± 0.08 and 0.70±0.040.70± 0.04, respectively; the MorphContrast ensemble achieved a pooled OOF AUC of 0.700.70 (95%95\% CI: 0.650.65–0.760.76). Table 4: Frozen pathology foundation encoders under an identical survival head. Each encoder’s patch features are aggregated using the same gated-attention survival head. Values are mean ± standard deviation over five folds; the ensemble reports pooled OOF performance. Encoder AUC ↑ Bal-Acc (%) ↑ Sen (%) ↑ Spec (%) ↑ CONCH v1.5 Lu et al. (2024) 0.51±0.070.51± 0.07 50.81±3.8350.81± 3.83 41.80±15.7641.80± 15.76 59.81±16.6659.81± 16.66 CTransPath Wang et al. (2022) 0.58±0.020.58± 0.02 50.63±1.9750.63± 1.97 21.90±39.2221.90± 39.22 79.36±37.3879.36± 37.38 Lunit ViT-S/8 Kang et al. (2023) 0.58±0.040.58± 0.04 53.22±3.1453.22± 3.14 54.24±15.5254.24± 15.52 52.21±12.6952.21± 12.69 Phikon v2 Filiot et al. (2024) 0.58±0.040.58± 0.04 58.05±5.6758.05± 5.67 59.50±15.3859.50± 15.38 56.61±11.4256.61± 11.42 Virchow2 Zimmermann et al. (2024) 0.58±0.020.58± 0.02 55.73±3.8455.73± 3.84 52.71±12.9752.71± 12.97 58.75±19.8558.75± 19.85 Kaiko ViT-B/8 kaiko.ai et al. (2024) 0.59±0.110.59± 0.11 58.38±8.9158.38± 8.91 46.45±30.0246.45± 30.02 70.31±13.8570.31± 13.85 GigaPath Xu et al. (2024) 0.60±0.040.60± 0.04 56.37±2.2056.37± 2.20 35.86±6.1035.86± 6.10 76.89±4.6976.89± 4.69 H-Optimus-0 Saillard et al. (2024) 0.61±0.040.61± 0.04 56.59±4.1356.59± 4.13 36.71±19.2236.71± 19.22 76.46±18.3876.46± 18.38 UNI v1 Chen et al. (2024) 0.62±0.050.62± 0.05 53.58±2.4453.58± 2.44 38.14±20.9738.14± 20.97 69.02±21.7969.02± 21.79 UNI v2 Chen et al. (2024) 0.63±0.030.63± 0.03 60.58±3.4860.58± 3.48 44.62±8.8144.62± 8.81 76.53±3.8876.53± 3.88 MorphDistill Khan et al. (2026b) 0.68¯±0.08 0.68± 0.08 63.41±4.7063.41± 4.70 60.24±5.3860.24± 5.38 66.57±8.8166.57± 8.81 PRISM Sajjad et al. (2026) 0.70±0.040.70± 0.04 68.00±1.7968.00± 1.79 67.14±8.2667.14± 8.26 68.86±8.6268.86± 8.62 MorphContrast 0.64±0.050.64± 0.05 59.45±3.5659.45± 3.56 63.53±9.2363.53± 9.23 55.37±7.5955.37± 7.59 MorphContrast (ensemble) 0.700.70 (0.65,0.76)(0.65,0.76) 0.680.68 (0.63,0.73)(0.63,0.73) 0.680.68 (0.59,0.73)(0.59,0.73) 0.680.68 (0.62,0.73)(0.62,0.73) 4.5 Comparison with Clinical Variables Using the same five patient-disjoint folds (n=424n=424; 103103 events), we compared MorphContrast with ℓ2 _2-regularized clinical logistic-regression models (Supplementary Table 3). Models using all fourteen variables or nodal burden, T category, and grade only achieved pooled AUCs of 0.5410.541 and 0.5500.550, respectively (Table 5). The single MorphContrast model achieved a mean fold AUC of 0.642±0.050.642± 0.05, and the ensemble achieved a pooled OOF AUC of 0.700.70. Table 5: Clinical-variable models and MorphContrast on Alliance/CALGB 89803 (n=424n=424; 103103 events). Fold AUC reports the mean ± standard deviation across five test folds. Pooled AUC uses out-of-fold predictions, with DeLong 95%95\% CIs for the clinical models and a 1,0001,000-sample bootstrap 95%95\% CI for the ensemble. Variable coding and imputation are detailed in Supplementary Table 3. Model Fold AUC ↑ Pooled AUC [95% CI] ↑ C-index ↑ All fourteen variables 0.548±0.0660.548± 0.066 0.5410.541 (0.474,0.608)(0.474,0.608) 0.5560.556 Nodal burden, T category, grade 0.554±0.0520.554± 0.052 0.5500.550 (0.487,0.613)(0.487,0.613) 0.5530.553 MorphContrast 0.642±0.0500.642± 0.050 — 0.6160.616 MorphContrast (ensemble) — 0.700.70 (0.65,0.76)(0.65,0.76) 0.6650.665 4.6 Risk-Group Stratification The MorphContrast ensemble stratified the Alliance cohort into low- and high-risk groups, with an HR of 3.793.79 (95%95\% CI: 2.552.55–5.635.63, p<0.0001p<0.0001) (Figs. 3 and 4). PRISM and MorphDistill yielded HR estimates of 3.343.34 (95%95\% CI: 2.282.28–4.904.90) and 2.522.52 (95%95\% CI: 1.731.73–3.653.65), respectively. TransMIL yielded an HR of 6.306.30 (95%95\% CI: 3.053.05–13.0313.03) with nine patients assigned to the high-risk group; its corresponding operating point was 5.25%5.25\% sensitivity and 99.74%99.74\% specificity. Among the evaluated frozen foundation encoders, UNI v2 yielded the largest hazard-ratio point estimate (HR 2.082.08, 95%95\% CI: 1.431.43–3.023.02, p<0.0001p<0.0001), followed by Phikon v2 (HR 1.941.94, 95%95\% CI: 1.301.30–2.912.91) and GigaPath (HR 1.801.80, 95%95\% CI: 1.231.23–2.642.64) (Supplementary Fig. 1). Figure 3: Kaplan–Meier risk stratification for MorphContrast and comparison models. Patients are assigned to low- and high-risk groups at each model’s operating threshold; group sizes are shown in each panel. Hazard ratios and 95%95\% CIs are obtained from univariate Cox models, and p-values from log-rank tests. Figure 4: Discrimination and risk-group separation across MIL and morphology-aware models. Left: C-index. Right: univariate Cox hazard ratios with 95%95\% CIs for high- versus low-risk groups. 4.7 External Evaluation on the SurGen Cohort We trained all models reported for the SurGen colon cancer cohort exclusively on the Alliance/CALGB 89803 cohort and evaluated them on SurGen in inference mode, without any fine-tuning or adaptation to the external data. This inference-only evaluation stringently tests generalizability under realistic multi-site distribution shift, spanning differences in scanner vendor, resolution, staining, and case-mix composition. On this cohort, FR-MIL achieved the highest baseline AUC (0.67660.6766, 95%95\% CI: 0.5970.597–0.7540.754) and yielded 83.33%83.33\% sensitivity and 40.27%40.27\% specificity (Table 6). TransMIL and MHIM-MIL achieved AUCs of 0.55800.5580 (95%95\% CI: 0.4720.472–0.6380.638) and 0.55850.5585 (95%95\% CI: 0.4800.480–0.6450.645), respectively. The MorphContrast model achieved an AUC of 0.6690.669 (95%95\% CI: 0.580.58–0.750.75), with a balanced accuracy of 63.98%63.98\%, sensitivity of 57.58%57.58\%, and specificity of 70.39%70.39\%, while the MorphContrast post hoc ensemble achieved an AUC of 0.6780.678 (95%95\% CI: 0.600.60–0.760.76), with a balanced accuracy of 65.80%65.80\%, sensitivity of 65.15%65.15\%, and specificity of 66.45%66.45\%. Table 6: External validation performance for five-year overall survival prediction in the SurGen colon cancer cohort. Models were trained using the Alliance cohort and externally evaluated on the independent SurGen cohort. The best AUC point estimate is shown in bold, and the second-best AUC point estimate is underlined. Model AUC ↑ Bal-Acc (%) ↑ Sen. (%) ↑ Spec. (%) ↑ TransMIL Shao et al. (2021) 0.5580.558 (0.47,0.64)(0.47,0.64) 55.0655.06 (45.1,57.7)(45.1,57.7) 65.1565.15 (52.2,76.4)(52.2,76.4) 44.9744.97 (37.6,53.0)(37.6,53.0) FR-MIL Chikontwe et al. (2022) 0.676¯ 0.676 (0.59,0.75)(0.59,0.75) 58.8058.80 (46.5,60.0)(46.5,60.0) 83.3383.33 (73.3,92.0)(73.3,92.0) 40.2740.27 (32.6,48.6)(32.6,48.6) MHIM-MIL Tang et al. (2023) 0.5580.558 (0.48,0.64)(0.48,0.64) 53.6153.61 (47.4,61.4)(47.4,61.4) 51.5251.52 (40.0,63.2)(40.0,63.2) 55.7055.70 (47.9,64.5)(47.9,64.5) MorphContrast 0.6690.669 (0.58,0.75)(0.58,0.75) 63.9863.98 (60.1,72.3)(60.1,72.3) 57.5857.58 (30.3,85.1)(30.3,85.1) 70.3970.39 (44.0,96.2)(44.0,96.2) MorphContrast (ensemble) 0.6780.678 (0.60,0.76)(0.60,0.76) 65.8065.80 (60.7,73.4)(60.7,73.4) 65.1565.15 (43.9,85.5)(43.9,85.5) 66.4566.45 (45.4,87.4)(45.4,87.4) 4.8 Ablation of the Contrastive Objectives To assess the individual and joint contributions of the contrastive objectives to five-year survival prediction, we retrained MorphContrast using the same five-fold splits with ℒintraL_intra and ℒinterL_inter enabled individually and jointly, while holding ℒsurvL_surv and ℒdivL_div fixed (Table 7). The configuration without either contrastive objective reached an AUC of 0.60±0.070.60± 0.07 but favored predictions of death within five years, with a sensitivity of (82.12±7.06)%(82.12± 7.06)\% and a specificity of (22.71±7.01)%(22.71± 7.01)\%. Either term alone yielded only modest increases in AUC (0.62±0.090.62± 0.09 for ℒintraL_intra and 0.61±0.080.61± 0.08 for ℒinterL_inter). The full objective achieved the highest mean AUC (0.64±0.050.64± 0.05) and the most balanced operating point among the ablation configurations, with a sensitivity of (63.52±9.23)%(63.52± 9.23)\% and a specificity of (55.40±7.63)%(55.40± 7.63)\%. The sensitivity–specificity gap narrowed from 59.4159.41 to 8.128.12 percentage points. The performance gain thus emerges when both contrastive objectives are combined, rather than when either is used alone, and is driven primarily by improved specificity. Table 7: Ablation of the intra- and inter-patient contrastive losses (Alliance/CALGB 89803). Each configuration adds the indicated contrastive term to the shared survival (ℒsurvL_surv) and diversity (ℒdivL_div) objectives under identical five-fold splits; ✓/✗ mark inclusion. “Survival only” omits both contrastive terms; “Full” is complete MorphContrast. Weights are fixed at the established values (λsurv=1.0 _surv=1.0, λdiv=0.1 _div=0.1; λintra=0.25 _intra=0.25, λinter=0.1 _inter=0.1 when enabled). Values are mean ± standard deviation across folds; in the AUC column the best value is shown in bold, and the second-best is underlined. AUC is highest for the full objective, whereas either contrastive term alone is within noise of the survival-only baseline. Loss terms Five-year survival classification Configuration ℒintraL_intra ℒinterL_inter AUC Bal. Acc. (%) Sens. (%) Spec. (%) Survival only ✗ ✗ 0.60±0.070.60± 0.07 52.43±1.3252.43± 1.32 82.12±7.0682.12± 7.06 22.71±7.0122.71± 7.01 +ℒintra+\,L_intra ✓ ✗ 0.62¯±0.09 0.62± 0.09 53.62±2.4253.62± 2.42 72.01±1.7572.01± 1.75 35.04±1.0935.04± 1.09 +ℒinter+\,L_inter ✗ ✓ 0.61±0.080.61± 0.08 56.62±5.2056.62± 5.20 80.53±18.4180.53± 18.41 32.70±19.0232.70± 19.02 Full ✓ ✓ 0.64±0.050.64± 0.05 59.43±3.6159.43± 3.61 63.52±9.2363.52± 9.23 55.40±7.6355.40± 7.63 5 Discussion This study provides three main observations. First, morphology-restricted models showed modest but distributed prognostic discrimination across tumor and non-tumor tissue compartments, indicating that prognostic discrimination was not confined to malignant epithelium. Adipose tissue and stroma achieved AUCs of 0.6220.622 and 0.6040.604, respectively, compared with 0.6010.601–0.6050.605 for adenocarcinoma compartments. These results are consistent with established roles for the invasive front, stromal composition, tumor budding, and the broader tumor microenvironment in CRC outcome Lugli et al. (2017); Gao et al. (2021); Kather et al. (2019). However, these morphology-specific AUCs should not be interpreted as direct measures of biological importance. Patch abundance, spatial context, slide sampling, and correlated clinicopathologic factors were not controlled. The analysis therefore supports distributed prognostic discrimination across histologic compartments, not a causal ranking of tissue types. Second, MorphContrast structures frozen patch features through two contrastive objectives. Within patients, same-morphology patches form positive pairs and different-morphology patches form negative pairs. In the outcome-aware objective, morphology-matched patches from different patients with concordant outcomes form positive pairs, those with discordant outcomes form hard negatives, and patches with different morphologies form standard negatives. On Alliance/CALGB 89803, the model achieved an AUC of 0.6420.642, whereas the post hoc ensemble achieved a pooled OOF AUC of 0.700.70 and a risk-group HR of 3.793.79. The MorphContrast model achieved a higher mean AUC than all ten frozen pathology foundation encoders evaluated with the same gated-attention survival head (range, 0.510.51–0.630.63), but a lower mean AUC than MorphDistill (0.680.68) and PRISM (0.700.70); however, the post hoc ensemble achieved a pooled OOF AUC of 0.700.70, exceeding MorphDistill and matching PRISM. Third, evaluation on SurGen assessed external performance in a cohort that differed from Alliance in scanner resolution. Both the MorphContrast model and the post hoc ensemble had higher AUC point estimates than TransMIL and MHIM-MIL. The 95%95\% confidence interval for the post hoc ensemble overlapped the interval for FR-MIL, the strongest evaluated baseline. These external results support cross-cohort generalizability, with discrimination comparable to the strongest evaluated baseline. Despite the favorable performance of our method, several limitations warrant consideration. First, morphology labels are provided by a frozen encoder, so label noise may propagate directly into the contrastive pairs; ensuring consistently high-quality positive and negative sets may require further methodological refinement. Second, the endpoint is a complete-case binary label: patients censored before five years were excluded rather than censored, which discards time-to-event information and may introduce selection bias. Finally, the framework is unimodal; integrating genomic and transcriptomic data, including microsatellite instability status and consensus molecular subtypes, is a natural extension Akbar et al. (2026a). We intend to address these limitations in future work. Overall, MorphContrast organizes frozen patch features through within-patient contrasts defined by morphology and cross-patient contrasts defined jointly by morphology matching and outcome concordance or discordance. The prognostic performance observed on Alliance and independently assessed on SurGen supports the use of morphology labels as structural supervision for five-year survival prediction from WSIs. 6 Conclusion MorphContrast uses tissue morphology as structural supervision for five-year survival prediction from CRC H&E WSIs. The TCGA-COAD/READ analysis showed modest but distributed prognostic discrimination across tumor and non-tumor morphologies, motivating morphology- and outcome-aware contrastive learning. On Alliance/CALGB 89803, the model achieved an AUC of 0.6420.642, while the internally selected post hoc ensemble reached a pooled out-of-fold AUC of 0.700.70. On the external SurGen colon cancer cohort, the single model and ensemble achieved AUCs of 0.6690.669 and 0.6780.678, respectively, compared with 0.67660.6766 for FR-MIL, the strongest evaluated external baseline. The methodological contribution of MorphContrast is to use same-patient morphology matches and mismatches as positive and negative pairs, respectively, and cross-patient, morphology-matched patches with concordant and discordant outcomes as positive pairs and hard negatives, respectively. This design converts morphology labels into outcome-aware supervision for learning prognostic representations from the complete mixture of tumor and non-tumor tissue. Statements and Declarations Competing interests All authors declare no financial or non-financial competing interests. Ethics approval and consent to participate This study was reviewed and approved by the Institutional Review Board of The Ohio State University (IRB #2018C0098). All procedures involving human data were conducted in accordance with the ethical standards of the institutional research committee, applicable national regulations, and the 1964 Declaration of Helsinki and its later amendments. Given the retrospective nature of the study and because data were archival and de-identified, the requirement for informed consent to participate was formally waived by the Institutional Review Board. Funding The project described was supported in part by R01 CA276301 (PIs: Niazi and Chen) from the National Cancer Institute. The project was also supported in part by Pelotonia under IRP C13702 (PIs: Niazi, Vilgelm, and Roy), The Ohio State University Department of Pathology and Comprehensive Cancer Center. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute, National Institutes of Health, or The Ohio State University. Author contributions H.K. performed data preprocessing, experimental design, validation, and manuscript writing and editing. M.N.G., A.P., W.L.F., W.C., and M.K.K.N. assisted with manuscript review and editing. Acknowledgements The authors thank Dr. Fang-Shu Ou and the Alliance Statistics and Data Management Center for data acquisition, curation, and management. The authors also gratefully acknowledge the Ohio Supercomputer Center for providing high-performance computing resources under its contract with The Ohio State University College of Medicine. We thank the Department of Pathology and the Comprehensive Cancer Center at The Ohio State University for their valuable support. The authors also thank Usama Sajjad, a Ph.D. student at The Ohio State University Wexner Medical Center, for providing the baseline results for the Alliance and SurGen cohorts. Trial registration NCT00003835 (CALGB 89803); Registry: CTRP (Clinical Trial Reporting Program); Registry Identifier: NCI-2012-01844. Disclaimer The data from CALGB 89803 were obtained directly from the Alliance for Clinical Trials in Oncology, a National Clinical Trials Network cooperative group; however, all analyses and conclusions in this manuscript are the sole responsibility of the authors and do not necessarily reflect the opinions or views of the clinical trial investigators, the NCTN, the NCORP, or the NCI. Data availability The Alliance/CALGB 89803 whole-slide images and associated clinical outcomes are available to qualified investigators through the Alliance for Clinical Trials in Oncology via Datasharing@alliancenctn.org upon reasonable request. The remaining cohorts are publicly available: the TCGA-COAD and TCGA-READ data can be accessed through the GDC Data Portal (https://portal.gdc.cancer.gov/), and the SurGen data through the BioStudies repository (https://w.ebi.ac.uk/biostudies/bioimages/studies/S-BIAD1285). Code availability The source code will be publicly available upon acceptance. Code access may also be requested by emailing the corresponding author at Hikmat.khan@osumc.edu. References Rawla et al. (2019) Rawla P, Sunkara T, Barsouk A (2019) Epidemiology of colorectal cancer: incidence, mortality, survival, and risk factors. Gastroenterology Review 14(2):89–103 Siegel et al. (2025) Siegel RL, Kratzer TB, Giaquinto AN, et al (2025) Cancer statistics, 2025. CA: A Cancer Journal for Clinicians 75(1):10–45 Kather et al. (2026) Kather JN, Hanna MG, Garcia C, et al (2026) The cost of accessibility: Considerations for adopting microscope camera-derived images for digital and computational pathology. Mayo Clinic Proceedings: Digital Health 4(3):100379. 10.1016/j.mcpdig.2026.100379 Khan et al. (2026a) Khan H, Chen W, Niazi MKK (2026a) Weakly supervised teacher–student framework with progressive pseudo-mask refinement for gland segmentation. Journal of clinical and translational pathology 6(1):1 Datwani et al. (2025) Datwani S, Khan H, Niazi MKK, et al (2025) Artificial intelligence in breast pathology: Overview and recent updates. Human pathology p 105819 Akbar et al. (2026c) Akbar AR, Wales-McGrath S, Levya A, et al (2026c) Pathoscribe: Transforming pathology data into a living library with a unified llm-driven framework for semantic retrieval and clinical integration. arXiv preprint arXiv:260308935 Akbar et al. (2026b) Akbar AR, Sajjad U, Su Z, et al (2026b) A spatially aware deep learning framework for multiscale cellular ecology profiling to predict 5-year recurrence in invasive lung adenocarcinoma. npj Digital Medicine Lugli et al. (2017) Lugli A, Kirsch R, Ajioka Y, et al (2017) Recommendations for reporting tumor budding in colorectal cancer based on the international tumor budding consensus conference (ITBCC) 2016. Modern Pathology 30(9):1299–1311 Gao et al. (2021) Gao J, Wang L, Xu J, et al (2021) Impact of tumor–stroma ratio on the prognosis of colorectal cancer: a systematic review. Frontiers in Oncology 11:738080 Kather et al. (2019) Kather JN, Krisam J, Charoentong P, et al (2019) Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study. PLoS Medicine 16(1):e1002730. 10.1371/journal.pmed.1002730 Ilse et al. (2018) Ilse M, Tomczak JM, Welling M (2018) Attention-based deep multiple instance learning. In: International Conference on Machine Learning (ICML), p 2127–2136 Lu et al. (2021) Lu MY, Williamson DFK, Chen TY, et al (2021) Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering 5(6):555–570 Shao et al. (2021) Shao Z, Bian H, Chen Y, et al (2021) TransMIL: Transformer based correlated multiple instance learning for whole slide image classification. In: Advances in Neural Information Processing Systems (NeurIPS), p 2136–2147 Khan et al. (2025a) Khan H, Su Z, Zhang H, et al (2025a) Predicting neoadjuvant chemotherapy response in triple-negative breast cancer using pre-treatment histopathologic images. Cancers 17(15):2423 Tang et al. (2024) Tang W, Zhou F, Huang S, et al (2024) Feature re-embedding: Towards foundation model-level performance in computational pathology. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p 11343–11352 Chen et al. (2024) Chen RJ, Ding T, Lu MY, et al (2024) Towards a general-purpose foundation model for computational pathology. Nature Medicine 30(3):850–862 Xu et al. (2024) Xu H, Usuyama N, Bagga J, et al (2024) A whole-slide foundation model for digital pathology from real-world data. Nature 630(8015):181–188 Khan et al. (2026b) Khan H, Sajjad U, Gurcan MN, et al (2026b) MorphDistill: Distilling unified morphological knowledge from pathology foundation models for colorectal cancer survival prediction, manuscript, 2026 Sajjad et al. (2026) Sajjad U, Akbar AR, Su Z, et al (2026) Morphology-aware prognostic model for five-year survival prediction in colorectal cancer from H&E whole-slide images: A study using a multi-center clinical trial cohort. Cancers 18(7):1150. 10.3390/cancers18071150 Myles et al. (2025) Myles C, Um IH, Marshall C, et al (2025) SurGen: 1020 H&E-stained whole-slide images with survival and genetic markers. GigaScience 14:1–16. 10.1093/gigascience/giaf086, dataset: https://doi.org/10.6019/S-BIAD1285 Khosla et al. (2020) Khosla P, Teterwak P, Wang C, et al (2020) Supervised contrastive learning. In: Advances in Neural Information Processing Systems (NeurIPS) Li et al. (2021) Li B, Li Y, Eliceiri KW (2021) Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p 14318–14328 Chikontwe et al. (2022) Chikontwe P, Nam SJ, Go H, et al (2022) Feature re-calibration based multiple instance learning for whole slide image classification. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, p 420–430, 10.1007/978-3-031-16434-7_41 Song et al. (2024) Song AH, Chen RJ, Ding T, et al (2024) Morphological prototyping for unsupervised slide representation learning in computational pathology. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p 11566–11578 Nakanishi et al. (2023) Nakanishi R, Morooka K, Omori K, et al (2023) Artificial intelligence-based prediction of recurrence after curative resection for colorectal cancer from digital pathological images. Annals of Surgical Oncology 30(6):3506–3514 Yuan et al. (2025) Yuan W, Chen Y, Zhu B, et al (2025) Pancancer outcome prediction via a unified weakly supervised deep learning model. Signal Transduction and Targeted Therapy 10(1):285 Zhang et al. (2024) Zhang Y, Li H, Sun Y, et al (2024) Attention-challenging multiple instance learning for whole slide image classification. In: European Conference on Computer Vision (ECCV), Springer, p 125–143 Tang et al. (2023) Tang W, Huang S, Zhang X, et al (2023) Multiple instance learning framework with masked hard instance mining for whole slide image classification. In: IEEE/CVF International Conference on Computer Vision (ICCV), IEEE, p 4055–4064 Chen et al. (2020) Chen T, Kornblith S, Norouzi M, et al (2020) A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning (ICML) Khan et al. (2025b) Khan H, Zaidi SFA, Shah PM, et al (2025b) MorphGen: Morphology-guided representation learning for robust single-domain generalization in histopathological cancer classification. arXiv preprint 10.48550/arXiv.2509.00311, arXiv:2509.00311 Lu et al. (2024) Lu MY, Chen B, Williamson DFK, et al (2024) A visual-language foundation model for computational pathology. Nature Medicine 30(3):863–874 Zimmermann et al. (2024) Zimmermann E, Vorontsov E, Viret J, et al (2024) Virchow2: Scaling self-supervised mixed magnification models in pathology. arXiv preprint arXiv:240800738 Wang et al. (2022) Wang X, Yang S, Zhang J, et al (2022) Transformer-based unsupervised contrastive learning for histopathological image classification. Medical Image Analysis 81:102559 Saillard et al. (2024) Saillard C, Jenatton R, Llinares-López F, et al (2024) H-optimus-0. https://github.com/bioptimus/releases/tree/main/models/h-optimus/v0 kaiko.ai et al. (2024) kaiko.ai, Aben N, de Jong ED, et al (2024) Towards large-scale training of pathology foundation models. arXiv preprint arXiv:240415217 Kang et al. (2023) Kang M, Song H, Park S, et al (2023) Benchmarking self-supervised learning on diverse pathology datasets. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p 3344–3354 Filiot et al. (2024) Filiot A, Jacob P, Mac Kain A, et al (2024) Phikon-v2, a large and public feature extractor for biomarker prediction. arXiv preprint arXiv:240909173 Belagali et al. (2026) Belagali V, Kapse S, Marza P, et al (2026) Ticon: A slide-level tile contextualizer for histopathology representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p 5440–5451 Saltz et al. (2007) Saltz LB, Niedzwiecki D, Hollis D, et al (2007) Irinotecan fluorouracil plus leucovorin is not superior to fluorouracil plus leucovorin alone as adjuvant treatment for stage i colon cancer: results of calgb 89803. Journal of Clinical Oncology 25(23):3456–3461. 10.1200/JCO.2007.11.2144 National Cancer Institute (2004) National Cancer Institute (2004) Combination chemotherapy in treating patients with stage I colon cancer. URL https://clinicaltrials.gov/study/NCT00003835, clinicalTrials.gov identifier: NCT00003835; last updated October 8, 2013 Rogers and Johnson (2021) Rogers JE, Johnson B (2021) The reality of early-onset colorectal cancer: Highlighting the needs in a unique but emerging population. Digestive Medicine Research 4:63. 10.21037/dmr-21-77 The Cancer Genome Atlas Network (2012) The Cancer Genome Atlas Network (2012) Comprehensive molecular characterization of human colon and rectal cancer. Nature 487(7407):330–337. 10.1038/nature11252 National Cancer Institute (2026) National Cancer Institute (2026) Genomic Data Commons Data Portal. https://portal.gdc.cancer.gov/, accessed: 2026-08-03 Harrell et al. (1996) Harrell JFrank E., Lee KL, Mark DB (1996) Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine 15(4):361–387. 10.1002/(SICI)1097-0258(19960229)15:4¡361::AID-SIM168¿3.0.CO;2-4 Loshchilov and Hutter (2019) Loshchilov I, Hutter F (2019) Decoupled weight decay regularization. In: International Conference on Learning Representations Loshchilov and Hutter (2017) Loshchilov I, Hutter F (2017) Sgdr: Stochastic gradient descent with warm restarts. In: International Conference on Learning Representations Akbar et al. (2026a) Akbar A, Sajjad U, Leyva A, et al (2026a) Genoglyph: Pan-cancer genomic mutation inference and risk stratification from diagnostic histopathology slides. Research Square p rs–3 Supplementary Material Morphology-aware deep learning predicts colorectal cancer survival from whole-slide images Supplementary Fig. 1: Kaplan–Meier risk stratification using frozen pathology foundation encoders and MorphContrast. Each panel reports low- and high-risk group sizes, the hazard ratio with 95%95\% CI, and the log-rank p-value. The foundation-model features are aggregated using the same gated-attention survival head. Supplementary Table 1: Clinicopathologic characteristics of the Alliance/CALGB 89803 cohort. Alliance cohort characteristics Characteristic Category Value Number of slides 424 Number of patients 424 Mean age (years) 60.47 Mean household income median (USD) 43,194.59 Race White 391 Black 33 Sex Male 238 Female 186 Treatment 5FU/LV 212 CPT-11/5FU/LV 212 Zubrod performance scale 0 324 1 98 2 2 Tumor location Cecum 101 Ascending colon 64 Hepatic flexure 28 Transverse colon 46 Splenic flexure 19 Descending colon 19 Sigmoid colon 147 Grade I 20 I 296 I 108 IV 0 Stage I 424 Small blood/lymphatic vessel invasion No 286 Yes 138 Extramural vascular invasion No 395 Yes 29 Infiltrating border No 283 Yes 141 Five-year outcome (analysis endpoint) Died within 5 years 103 Survived ≥5≥ 5 years 321 Supplementary Table 2: Hyperparameter search space and fixed training settings for MorphContrast. Parameters with multiple candidate values were varied during hyperparameter search; single-valued parameters were fixed. Parameter Value(s) Setting Hyperparameter search space Feature-encoder hidden dimension 768,1024\768,1024\ Searched Projection dimension 128,256\128,256\ Searched Attention hidden dimension 512,1024,1536\512,1024,1536\ Searched Initial learning rate 10−4,10−5\10^-4,10^-5\ Searched Decision threshold 0.4,0.5\0.4,0.5\ Searched Intra-patient loss weight λa _a 0.1,0.25\0.1,0.25\ Initial search Inter-patient loss weight λe _e 0.1,0.25\0.1,0.25\ Initial search Diversity loss weight λd _d 0.1,0.25\0.1,0.25\ Searched Survival loss Weighted CE, focal loss Searched Fixed training settings Optimizer AdamW Fixed Learning-rate schedule Cosine annealing Fixed Weight decay 10−510^-5 Fixed Batch size 44 Fixed Dropout 0.10.1 Fixed Survival loss weight λs _s 1.01.0 Fixed Temperature τ 0.10.1 Fixed Maximum contrastive patches 512512 Fixed Maximum epochs 150150 Fixed Early-stopping patience 1515 epochs Fixed Supplementary Table 3: Clinicopathological variables used in the clinical models. The fourteen variables were obtained from Alliance/CALGB 89803 trial case-report forms (n=424n=424; 103103 events). Both full and reduced baselines use ℓ2 _2-regularized logistic regression to predict death within five years, evaluated using the same five patient-disjoint folds as MorphContrast. Missing values were imputed with medians estimated from the training patients in each fold. The reduced model includes positive lymph-node count, T category, and grade only. T category combines trial depth-of-invasion codes 44 and 55 as T4. FL denotes fluorouracil and leucovorin; IFL additionally includes irinotecan. No molecular markers were included. Variable Model coding Age at enrolment years, continuous Sex male vs female Histological grade I (poorly differentiated) vs I–I T category (depth of invasion) T4 vs T1–T3 Positive lymph nodes number, continuous Small blood/lymphatic vessel invasion present vs absent Extramural vascular invasion present vs absent Perineural invasion present vs absent Infiltrating border present vs absent Lymphocytic response present vs absent Obstruction present vs absent Perforation present vs absent Zubrod performance status ≥1≥ 1 vs 0 Treatment arm IFL vs FL