Paper deep dive
WILSON - a pathology foundation model framework for patient-level analysis and diagnostic text generation
Saghir Alfasly, Wataru Uegami, Sobhan Hemati, Wenchao Han, Xiaojia Tang, Kevin Thompson, Daniel Stone, Ghazal Alabtah, Saba Yasir, Michael R. Lucas, Eric W. Klee, Cheryl L. Willman, Judy C. Boughey, Matthew P. Goetz, Krishna R. Kalari, H. R. Tizhoosh
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/23/2026, 3:51:04 AM
Summary
The paper introduces WILSON, a vision-language foundation model for computational pathology that represents whole-slide images (WSIs) and multi-slide patient cases as single multi-magnification composite images. Trained on ~189k slides from Mayo Clinic, WILSON outperforms dedicated case-level models and matches larger slide-level models with significantly lower computational cost. It enables end-to-end fine-tuning for downstream tasks like histologic subtyping and generates diagnostic text with high recall.
Entities (16)
Relation Signals (8)
WILSON → uses → Composite Image
confidence 98% · WILSON... represents whole-slide images and multi-slide cases as single multi-magnification composite images
WILSON → finetunedon → MayoTNBC
confidence 95% · End-to-end fine-tuning on 508 triple-negative breast cancer cases [MayoTNBC] improved histologic subtyping
WILSON → outperforms → MOOZY
confidence 95% · WILSON exceeded a dedicated case-level model on all internal cohorts (macro-F1 0.52 versus 0.38)
WILSON → trainedon → Mayo Clinic
confidence 95% · trained on approximately 189k slides from Mayo Clinic
WILSON → evaluatedon → CPTAC-BRCA
confidence 90% · benchmark... on four case-classification tasks spanning an external cohort (CPTAC-BRCA...)
WILSON → evaluatedon → TCGA-BRCA
confidence 90% · four benchmarks spanning two public cohorts (TCGA-BRCA...)
WILSON → haslowercomputethan → PRISM2
confidence 90% · matched slide-level models up to 9.4 times larger at 272- to 2,155-fold lower compute... WILSON exceeds PRISM2 by 23.2 points with ≈9× fewer parameters
WILSON → matchesperformanceof → PRISM
confidence 90% · WILSON retrieved matching diagnostic text at 75.6% recall@1 (PRISM, 58.1%)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Pathologists integrate morphology across magnifications and across the slides of a patient case, whereas pathology foundation models encode thousands of tiles from single slides and aggregate their features. Here we present WILSON, a vision--language foundation model that represents whole-slide images and multi-slide cases as single multi-magnification composite images, trained on approximately 189k slides from Mayo Clinic spanning 42 organs and 829 diagnostic entities using pathology reports as supervision. Without task-specific training, WILSON exceeded a dedicated case-level model on all internal cohorts (macro-F1 0.52 versus 0.38) and matched slide-level models up to 9.4 times larger at 272- to 2,155-fold lower compute. End-to-end fine-tuning on 508 triple-negative breast cancer cases improved histologic subtyping and stromal tumor-infiltrating lymphocyte grading by 0.16 and 0.11 macro-F1. WILSON retrieved matching diagnostic text at 75.6% recall@1 (PRISM, 58.1%) and generated captions closer to report-derived references than PRISM and PRISM2 on the internal cohort and on most external comparisons. Composite images thus offer a compact, clinically aligned computational unit for pathology.
Tags
Links
- Source: https://arxiv.org/abs/2609.25123v1
- Canonical: https://arxiv.org/abs/2609.25123v1
Trouble viewing inline? Open PDF directly →
Full Text
174,767 characters extracted from source content.
Expand or collapse full text
WILSON – a pathology foundation model framework for patient-level analysis and diagnostic text generation Saghir Alfasly Affiliation: Affiliation: KIMIA Lab, Department of AI & Informatics, Mayo Clinic, Rochester, MN, USA Wataru Uegami Affiliation: Affiliation: KIMIA Lab, Department of AI & Informatics, Mayo Clinic, Rochester, MN, USA Sobhan Hemati Affiliation: Affiliation: KIMIA Lab, Department of AI & Informatics, Mayo Clinic, Rochester, MN, USA Wenchao Han Affiliation: Affiliation: Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA Xiaojia Tang Affiliation: Affiliation: Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, USA Kevin Thompson Affiliation: Affiliation: Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, USA Daniel Stone Affiliation: Affiliation: KIMIA Lab, Department of AI & Informatics, Mayo Clinic, Rochester, MN, USA Ghazal Alabtah Affiliation: Affiliation: KIMIA Lab, Department of AI & Informatics, Mayo Clinic, Rochester, MN, USA Saba Yasir Affiliation: Affiliation: Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA Michael R. Lucas Affiliation: Affiliation: Mayo Clinic Comprehensive Cancer Center, Mayo Clinic, Rochester, MN, USA Eric W. Klee Affiliation: Affiliation: Mayo Clinic Comprehensive Cancer Center, Mayo Clinic, Rochester, MN, USA Cheryl L. Willman Affiliation: Affiliation: Mayo Clinic Comprehensive Cancer Center, Mayo Clinic, Rochester, MN, USA Judy C. Boughey Affiliation: Affiliation: Division of Breast and Melanoma Surgical Oncology, Department of Surgery, Mayo Clinic, Rochester, MN, USA Matthew P. Goetz Affiliation: Affiliation: Department of Oncology, Mayo Clinic, Rochester, MN, USA Krishna R. Kalari Affiliation: Affiliation: Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, USA H.R. Tizhoosh Email: tizhoosh.hamid@mayo.edu Affiliation: Affiliation: KIMIA Lab, Department of AI & Informatics, Mayo Clinic, Rochester, MN, USA Affiliation: Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA Abstract Pathologists integrate morphology across magnifications and across the slides of a patient case, whereas pathology foundation models encode thousands of tiles from single slides and aggregate their features. Here we present WILSON, a vision–language foundation model that represents whole-slide images and multi-slide cases as single multi-magnification composite images, trained on approximately 189k slides from Mayo Clinic spanning 42 organs and 829 diagnostic entities using pathology reports as supervision. Without task-specific training, WILSON exceeded a dedicated case-level model on all internal cohorts (macro-F1 0.52 versus 0.38) and matched slide-level models up to 9.4 times larger at 272- to 2,155-fold lower compute. End-to-end fine-tuning on 508 triple-negative breast cancer cases improved histologic subtyping and stromal tumor-infiltrating lymphocyte grading by 0.16 and 0.11 macro-F1. WILSON retrieved matching diagnostic text at 75.6% recall@1 (PRISM, 58.1%) and generated captions closer to report-derived references than PRISM and PRISM2 on the internal cohort and on most external comparisons. Composite images thus offer a compact, clinically aligned computational unit for pathology. keywordsComputational pathology, Pathology foundation models, Whole-slide image representation learning, Composite images, Reports, Slide-level image retrieval †equal-contributors: The first and second authors contributed equally to this work†equal-contributors: The first and second authors contributed equally to this work Figure 1: Overview of WILSON. a, Slides and cases per organ (bars), with the cohort further grouped by organ and by diagnostic category (donuts). b, An LLM cleans each pathology report and generates three paraphrased caption variants per report. c, Composites are tiled from selected WSIs; 178,020 WSIs have both a rendered composite and a caption record, giving 1,759,787 composites (≈ 9.9/WSI) and 1,759,787×3=5,279,3611,759,787× 3=5,279,361 (≈ 5.28M) image–caption pairs. d, WILSON pretraining and training: tile-based SSL on 256×256256× 256 patches at 2.5×/10×/20×2.5×/10×/20× → composite-based SSL on 512×512512× 512 four-tile composites → vision–language alignment of 2048×20482048× 2048 WSI composites against report text (SigLIP distillation from Gemini embeddings, CLIP, and keyword regression) → CoCa-style report generator, a multimodal decoder that cross-attends over pooled WILSON visual tokens to generate a caption autoregressively. e, Vision-encoder size, pretraining data (WSIs) and cost (FLOPs per 10k-tile WSI) for WILSON (133.7M; ≈ 189k; 2.6 T) versus MOOZY (85.8M; ≈ 77k; 25.1 T), TITAN (354.6M; ≈ 335k; 703.2 T), PRISM (730.3M; ≈ 587k; 4.07 P), PRISM2 (1,251M; ≈ 2.3M; 4.13 P) and Prov-GigaPath (1,220M; ≈ 171k; 5.56 P). f, Report-generation performance averaged over three cohorts (Mayo, HistAI, TCGA), as a normalized score: PRISM 27.1, PRISM2 34.9, WILSON 58.1, WILSON exceeds PRISM2 by 23.2 points with ≈ 9× fewer parameters and ≈ 1,600× fewer FLOPs per WSI. 1 Main Histopathological diagnosis requires integration of morphological information across spatial scales and, frequently, across multiple whole-slide images (WSIs) from the same patient case. However, most computational pathology foundation models represent WSIs by independently encoding large numbers of small image patches with a tile-level encoder Chen et al. (2024); Xu et al. (2024) and subsequently aggregating their features into slide-level representations, typically through multiple-instance learning Ilse et al. (2018); Lu et al. (2021); Shao et al. (2021); Ding et al. (2025); Shaikovski et al. (2024); Kotp et al. (2026). Although this strategy has enabled substantial advances in WSI analysis, it introduces considerable computational overhead, separates local morphology from the broader tissue context during image encoding Chen et al. (2022); Wang et al. (2022), and typically treats the individual WSI rather than the multi-slide case as the primary unit of representation. These characteristics differ from clinical pathology practice, in which diagnostic interpretation integrates multiscale morphological findings across complementary tissue sections and slides, and they have prompted a reassessment of whether patch-and-aggregate pipelines are the most appropriate foundation for pathology Tizhoosh (2026); Mulliqi et al. (2025). Vision–language foundation models offer an additional opportunity to ground histopathological representations in the diagnostic semantics contained in pathology reports Radford et al. (2021); Huang et al. (2023); Lu et al. (2024a); Lu et al. (2024b); Sun et al. (2024). Yet establishing direct correspondence between diagnostic language and gigapixel WSIs remains challenging because conventional pipelines operate through patch-level representations and subsequent aggregation, which separates the visual encoder from the report text by an intermediate aggregation stage Ding et al. (2025); Shaikovski et al. (2024). A representation that captures diagnostically informative morphology across magnifications and multiple WSIs within a unified visual input could therefore enable image–language learning at the case level while reducing dependence on computationally intensive patch-wise processing. Here we introduce WILSON, named in honor of Louis B. Wilson, a pioneering Mayo Clinic pathologist whose work on rapid frozen-section diagnosis helped shape modern surgical pathology. WILSON (Fig. 1) is a pathology foundation model that represents WSIs and multi-WSI cases as multi-magnification composite images and processes each composite as a unified visual input. The composites integrate tissue regions sampled across spatial scales and, for case-level analysis, across multiple WSIs, enabling direct learning of WSI- and case-level representations without a separate patch-feature aggregation stage. WILSON is trained using diagnostic information derived from pathology reports, directly aligning visual representations with clinically meaningful text. This formulation also enables end-to-end adaptation of the visual encoder for downstream WSI tasks rather than restricting adaptation to an aggregator operating on frozen tile features. We trained WILSON using approximately 189k WSIs spanning 42 organs and 829 diagnostic entities and evaluated it across internal and external cohorts encompassing case- and WSI-level disease classification, molecular and treatment-related prediction, image–text retrieval and pathology-text generation. We show that composite images retain diagnostically relevant information compared with conventional patch-based representations, that WILSON is competitive with or outperforms specialized case- and WSI-level foundation models across multiple tasks, and that its unified representation substantially reduces computational requirements while permitting end-to-end fine-tuning. We further demonstrate bidirectional retrieval between histopathology and diagnostic text and generation of pathology descriptions from composite images. Together, these findings establish multi-magnification composite representation as an alternative approach to foundation modeling that brings the computational unit of pathology AI closer to the multiscale, multi-slide structure of clinical diagnostic reasoning. 2 Results 2.1 Multi-magnification composites preserve diagnostically relevant information from gigapixel whole-slide images Whole-slide images (WSIs) frequently contain billions of pixels, making direct end-to-end processing with contemporary vision encoders computationally challenging. We therefore asked whether diagnostically informative regions sampled across multiple magnifications could be assembled into a single, computationally manageable composite image while preserving the information required for downstream WSI analysis. Unlike conventional tile-based approaches, which independently encode hundreds to thousands of image tiles followed by feature aggregation on top of frozen embeddings, the composite representation presents tissue regions sampled across multiple spatial scales to an image encoder as a single visual input. To evaluate the composite representation independently of WILSON training, we used the pretrained UNI foundation model Chen et al. (2024), which had not been trained on composite images, to extract embeddings from two alternative WSI representations: multi-magnification composites and tiles selected using the Yottixel framework Kalra et al. (2020), a tile-based WSI representation method. We compared the resulting representations using top-1 retrieval across four benchmarks encompassing diagnostic classification (MayoSkin, MayoBreast, TCGA-Kidney) and treatment-response prediction (BCTherapy Sammut et al. (2022)), including two public datasets and two internal Mayo Clinic cohorts (Fig. 2a, Table 1). Figure 2: WILSON matches or exceeds specialized case- and WSI-level baselines and benefits from efficient end-to-end fine-tuning. a, Multi-magnification composites versus Yottixel tile selection as alternative WSI representations, both encoded with the pretrained UNI foundation model. Top-1 macro-averaged F1 retrieval is shown for four WSI-level benchmarks, split across two sub-axes with different y-ranges: BCTherapy and TCGA-Kidney (left, 00–11) and MayoSkin and MayoBreast (right, 00–0.50.5). Bars are single point estimates. b, Case-level zero-shot retrieval performance (macro-averaged F1 under top-5 majority voting, MV@5) of three WILSON variants (WILSON-distill, WILSON-KW, WILSON-CLIP) against MOOZY, a case-level pathology foundation model, on four case-level benchmarks: CPTAC-BRCA, MayoBreastSubtype, MayoCaseBreast and MayoCaseSkin. Error bars indicate the standard deviation across the 10 composite images generated per case; MOOZY operates directly on the WSI and therefore has no associated variance. Color-matched horizontal dashed lines mark each model’s mean across the four benchmarks. c, The same values as in b shown as a model × benchmark heatmap (rows, models; columns, benchmarks; shading, macro-averaged F1). d, WSI-level zero-shot MV@5 retrieval performance of the same three WILSON variants against three WSI-level foundation models (GigaPath, TITAN, PRISM) on four WSI-level benchmarks: TCGA-BRCA, BCTherapy, MayoBreast and MayoSkin. Error bars indicate the standard deviation across the 10 composite images generated per WSI; the three baselines operate on frozen tile-level features and therefore have no associated variance. Color-matched dashed lines mark each model’s mean across the four benchmarks. e, The same values as in d shown as a model × benchmark heatmap. f, The same six models and four WSI-level benchmarks shown as a radar plot (BCTherapy, top; TCGA-BRCA, right; MayoSkin, bottom; MayoBreast, left; radial axis, macro-averaged F1 from 0.00.0 to 0.80.8), with colors as in d. g, Effect of end-to-end fine-tuning of WILSON on MayoTNBC, a cohort of 508 triple-negative breast cancer cases, evaluated on histological subtyping (Histology) and stromal tumor-infiltrating lymphocyte (sTILs) grading. Accuracy (left) and macro-averaged F1 (right) are shown for three settings: the pretrained backbone evaluated zero-shot (Base Zero-shot), the same backbone after end-to-end fine-tuning and evaluated zero-shot with no task-specific head (FT-Zero-shot), and the fine-tuned backbone trained jointly with a task-specific classification head (FT-classification). Despite representing each WSI through a compact composite image, performance remained closely matched to patch-based WSI indexing across all four benchmarks, with absolute differences of ≤0.05≤ 0.05 in macro-F1. Composite representations performed better on three benchmarks (BCTherapy: 0.56 vs. 0.51; MayoSkin: 0.34 vs. 0.33; MayoBreast: 0.35 vs. 0.31) and slightly lower on one cohort (TCGA-Kidney: 0.88 vs. 0.89). Thus, across diagnostically and clinically distinct tasks, multi-magnification composites preserved task-relevant information at a level comparable to patch-based WSI representations, while being processed as a single visual input without requiring patch-level feature aggregation. 2.2 WILSON enables multi-slide case-level diagnostic retrieval A central motivation for the composite-image framework is its ability to integrate tissue regions sampled from multiple WSIs belonging to the same patient into a unified case-level representation (Section 1). This extends the computational unit from an individual slide to the patient case, reflecting the integration of information across multiple tissue sections and slides during pathological assessment. To evaluate whether WILSON can learn diagnostically informative representations at this level, we benchmarked the three WILSON variants (WILSON-distill, WILSON-KW and WILSON-CLIP) against MOOZY Kotp et al. (2026), a case-level pathology foundation model, on four case-classification tasks spanning an external cohort (CPTAC-BRCA, PAM50 Parker et al. (2009) molecular subtyping) and three internal Mayo Clinic cohorts (MayoCaseBreast, histological subtype; MayoBreastSubtype, breast tumor molecular subtype classification; and MayoCaseSkin, skin histological diagnosis). Performance was assessed using zero-shot top-5 majority-voting retrieval, with macro-averaged F1 computed across the 10 composite images generated per case to quantify variability (Figure 2b,c, 7). WILSON exceeded MOOZY on all three internal cohorts. The largest margin was observed on MayoCaseSkin, where WILSON-distill reached a macro-F1 of 0.66 compared with 0.30 for MOOZY (+0.36) and every WILSON variant scored at least 0.59, followed by MayoCaseBreast (0.52 vs. 0.40, +0.12) and MayoBreastSubtype (0.55 vs. 0.50, +0.05). Averaged across the four benchmarks, WILSON-distill attained 0.52 against 0.38 for MOOZY (+0.14), with WILSON-KW at 0.48 and WILSON-CLIP at 0.47. WILSON-distill, supervised using dense Gemini report embeddings, was the strongest variant on three of the four tasks, indicating that a richer text-side supervision signal transfers to case-level retrieval more effectively than keyword regression or contrastive alignment alone. Performance on the external CPTAC-BRCA cohort was both lower in absolute terms and more variable across composites for all models. WILSON-distill scored slightly above MOOZY (0.35 vs. 0.32), whereas WILSON-KW (0.30) and WILSON-CLIP (0.31) fell marginally below it; in each case the difference was within the composite-to-composite standard deviation and should not be read as a meaningful separation. This is the only task in the panel that requires predicting a molecular class from morphology alone in an external cohort, and the results indicate that such transfer is less consistent than transfer across the internal diagnostic cohorts. Collectively, these results show that histopathological information distributed across multiple WSIs can be integrated within a unified composite representation that supports case-level diagnostic retrieval across histologic and dermatopathologic classification tasks without task-specific fine-tuning. 2.3 WILSON matches dedicated WSI-level foundation models with substantially lower computational requirements Case-level retrieval evaluates WILSON at the representational level for which it was primarily designed. We next asked whether the same composite representation remains competitive at the conventional single-WSI level. We benchmarked the three WILSON variants (WILSON-distill, WILSON-KW and WILSON-CLIP) against three published WSI-level foundation models—Prov-GigaPath Xu et al. (2024), TITAN Ding et al. (2025) and PRISM Shaikovski et al. (2024)—on four benchmarks spanning two public cohorts (TCGA-BRCA, breast histological subtype; BCTherapy, breast treatment response prediction) and two internal Mayo Clinic cohorts (MayoBreast and MayoSkin, histological subtype). All models were evaluated using the same zero-shot top-5 majority-voting retrieval protocol. For WILSON, macro-averaged F1 was computed across the 10 composites generated per WSI to quantify variability; the baseline models operate on frozen tile-level representations and therefore have no corresponding composite-to-composite variance (Figure 2d–f). Across the three diagnostic benchmarks, WILSON-distill achieved the highest mean macro-F1 (0.60), compared with 0.58 for PRISM, 0.57 for WILSON-CLIP and 0.56 for both TITAN and WILSON-KW. On the external TCGA-BRCA cohort, WILSON-distill matched PRISM (0.46) and performed similarly to TITAN (0.45). On MayoBreast, WILSON-distill reached 0.65, compared with 0.66 for PRISM and 0.64 for TITAN and WILSON-CLIP. The largest separation was observed on MayoSkin, a 20-class dermatopathology task, where WILSON-distill achieved 0.69 compared with 0.63 for PRISM and 0.58 for TITAN, and all three WILSON variants outperformed the three WSI-level baselines. On BCTherapy, WILSON-CLIP achieved the highest macro-F1 (0.65), compared with 0.62 for TITAN and 0.50 for PRISM, although variability across composites was greater on this relatively small cohort (160 WSIs). Performance across datasets also differed among the published WSI-level models. Prov-GigaPath achieved lower performance on three of the four benchmarks, including macro-F1 values of 0.27 and 0.30 on MayoBreast and MayoSkin, respectively. TITAN and PRISM transferred more consistently to the internal cohorts, although neither uniformly outperformed WILSON. Collectively, these results indicate that WILSON’s composite representation remains competitive with dedicated WSI-level foundation models across both public and institutional cohorts, despite replacing thousands of independently encoded tiles and subsequent feature aggregation with a single composite input. This comparable performance is achieved with substantially lower computational requirements. WILSON represents each WSI as a single 2048×20482048× 2048 composite processed in one forward pass and uses 2.7–9.1× fewer parameters than the three WSI-level baselines, together with two to three orders of magnitude fewer FLOPs per WSI (Section 2.7). Thus, across the benchmarks evaluated here, explicit tile-wise encoding and feature aggregation were not necessary to achieve competitive slide-level retrieval performance. The compact composite representation therefore provides an efficient alternative for WSI-level representation while retaining the ability to adapt the complete visual encoder end-to-end, which we evaluate next. 2.4 WILSON enables end-to-end adaptation for downstream whole-slide image tasks Most WSI-level foundation models use a two-stage architecture in which a tile encoder generates representations for thousands of image patches and a separate slide-level aggregator operates on the resulting features. In practice, downstream adaptation commonly relies on frozen tile representations because propagating gradients through thousands of tiles per WSI makes end-to-end fine-tuning computationally challenging at WSI scale Mulliqi et al. (2025); Tizhoosh (2026). WILSON instead represents each WSI as a single composite image processed by a unified visual encoder, making it feasible to adapt the complete visual backbone end-to-end. We evaluated this capability on MayoTNBC, a cohort of 508 triple-negative breast cancer cases, using two downstream tasks: histologic subtyping and stromal tumor-infiltrating lymphocyte (sTIL) grading. We compared three settings: (i) pretrained retrieval, using the pretrained WILSON embedding without downstream adaptation; (i) fine-tuned retrieval, using embeddings from the backbone after end-to-end adaptation while retaining the retrieval-based evaluation without a task-specific classification head; and (i) fine-tuned classification, in which the complete backbone and a task-specific classification head were jointly optimized (Figure 2g, Table 2). End-to-end adaptation improved performance on both tasks. For histologic subtyping, fine-tuning the backbone increased accuracy from 0.617 to 0.637 and macro-F1 from 0.401 to 0.428. Joint optimization with a classification head further increased accuracy to 0.690 and macro-F1 to 0.557, a gain of +0.156 in macro-F1 over the pretrained model. For sTIL grading, backbone adaptation increased accuracy from 0.412 to 0.432 and macro-F1 from 0.425 to 0.456; joint optimization increased these values to 0.541 and 0.531, respectively, corresponding to a +0.106 gain in macro-F1 over the pretrained model. The ability to adapt the complete visual encoder is particularly relevant to WILSON because it is substantially smaller than current WSI-level foundation models and was pretrained on a comparatively modest cohort of approximately 189k WSIs. Rather than relying exclusively on increasing model capacity and pretraining-data scale to capture the diversity of downstream pathology tasks, WILSON provides a complementary strategy in which a compact pretrained representation can be specialized when task-specific data become available. The gains observed after end-to-end adaptation on MayoTNBC support this strategy: competitive zero-shot representations can serve as a starting point, while the computational tractability of the composite formulation allows the complete model to be further optimized for specific downstream applications. Together with the zero-shot comparisons above, these findings indicate that the composite representation provides two complementary properties: competitive WSI-level representations without explicit tile-feature aggregation and a practical route to end-to-end downstream adaptation. This combination of compact model size, comparatively modest pretraining scale and full-model adaptability may be particularly useful for specialized pathology applications in which sufficiently representative data for large-scale foundation-model pretraining are unavailable, but smaller task-specific cohorts can support targeted model adaptation. Figure 3: WILSON vs. PRISM across all six cohorts—MayoCaption, MayoDuodenum, MayoPancreas, MayoStomach, TCGA-Brain, TCGA-Kidney—at K∈1,3,5,10K∈\1,3,5,10\. (a) I2T recall as radar plots (one axis per cohort, R@1–R@10 left to right); (b) the same I2T values as grouped bars, with dashed lines marking each model’s across-cohort mean (WILSON ≈ 74.1%, PRISM ≈ 58.1% at R@1). (c,d) mirror (a,b) for T2I. The WILSON envelope contains the PRISM envelope almost everywhere and both narrow toward TCGA-Brain; the orange curve extends past blue only on the TCGA-Kidney axis (I2T, R@3–R@10) and the MayoPancreas axis (T2I, R@3/R@5), the two crossover cases detailed in Table 11. 2.5 WILSON enables bidirectional retrieval between whole-slide images and diagnostic text The preceding experiments evaluated the visual representation independently of WILSON’s text encoder. Because WILSON is trained by aligning composite images with pathology-report text, we next asked whether the resulting joint embedding space supports retrieval across modalities. We evaluated image-to-text (I2T) and text-to-image (T2I) retrieval using recall@K (K∈1,3,5,10K∈\1,3,5,10\), with a match defined by disease-group membership. WILSON-CoCa was compared with PRISM, a published slide-level histopathology vision–language foundation model, using the same evaluation protocol. Evaluation included an internal Mayo Clinic cohort (n=456n=456 WSIs), analyzed both collectively and by organ (duodenum, pancreas and stomach), and two external cohorts: TCGA-Brain (six glioma subtypes) and TCGA-Kidney (three renal-carcinoma subtypes) (Figure 3, Figure 13, Tables 3–8). Across the twelve cohort–direction combinations, WILSON showed its largest advantage over PRISM at the highest retrieval rank. Mean R@1 was 75.6% for WILSON compared with 58.1% for PRISM, an absolute difference of 17.9 percentage points. The advantage narrowed progressively with increasing K, to 13.5, 11.9 and 8.4 points at R@3, R@5 and R@10, respectively (Table 11). WILSON led PRISM at R@1 in every cohort and retrieval direction evaluated. At higher values of K, PRISM exceeded WILSON in a small number of comparisons, including TCGA-Kidney I2T and TCGA-Brain I2T, indicating that the principal advantage of WILSON was in ranking a relevant disease-group match first rather than simply making that group retrievable within a larger candidate set. On the internal Mayo Clinic cohort, WILSON achieved R@1 values of 78.29% for I2T and 81.14% for T2I retrieval, with a median rank of 1 in both directions. Performance remained high when evaluated separately by organ: I2T R@1 was 85.81% for stomach, 78.18% for pancreas and 77.78% for duodenum, while T2I R@1 ranged from 80.81% to 85.14%. These results indicate that the shared embedding space supports retrieval in both directions across anatomically distinct gastrointestinal tissues rather than being driven by a single organ or disease group. Performance on the external cohorts was more heterogeneous. On TCGA-Kidney, WILSON retained strong cross-modal retrieval, reaching 82.56% I2T and 77.91% T2I R@1, with a median rank of 1 in both directions. Performance was lower on TCGA-Brain, where I2T and T2I R@1 were 41.94% and 54.03%, respectively, although recall increased to 78.23% and 88.71% at R@10. PRISM showed a similar relative reduction on TCGA-Brain, suggesting that cross-modal discrimination among glioma subtypes is more challenging for both models than among the disease groups represented in the other cohorts. Together, these results demonstrate that aligning a single multi-magnification composite with diagnostic text produces a shared representation that supports bidirectional retrieval across internal and external cohorts. Despite its compact composite-based architecture, WILSON consistently ranked relevant disease-group matches above those retrieved by a substantially larger slide-level vision–language model at R@1, while differences between the models diminished as the retrieval set expanded. 2.6 WILSON generates high-quality pathology captions Cross-modal retrieval demonstrates that WILSON’s joint embedding space captures associations between histopathology and diagnostic language, but does not establish whether visual information can be translated into diagnostically meaningful text. We therefore evaluated WILSON-CoCa, a generative extension of WILSON that decodes free-text pathology captions directly from composite images. WILSON-CoCa was compared with two published histopathology vision–language foundation models capable of text generation: PRISM Shaikovski et al. (2024), which combines a Virchow Vorontsov et al. (2024) tile encoder with a BioGPT-style decoder, and PRISM2 Shaikovski and others (2025), which uses Virchow2 Zimmermann et al. (2024) with a Phi decoder Abdin and others (2024). Models were evaluated on the same held-out WSIs using a fixed random seed and the same caption-quality metrics against pathologist-authored reference text. WILSON-CoCa generated captions autoregressively from the attentional-pooler representation of a single composite image per WSI. Figure 4: WILSON-CoCa generates pathology captions that more closely match reference reports than PRISM and PRISM2, on an internal held-out set and on two external cohorts. WILSON-CoCa is a CoCa-style Yu et al. (2022) generative extension of WILSON that decodes a free-text diagnostic caption directly from a composite image, compared against two general-purpose pathology vision–language foundation models capable of report generation, PRISM Shaikovski et al. (2024) and PRISM2 Shaikovski and others (2025). a, Evaluation pipeline: a WSI is rendered as a multi-magnification composite and encoded by the model’s vision tower; the text decoder generates a caption autoregressively, which is scored against the report-derived reference using n-gram overlap (BLEU-4, CIDEr, ROUGE-1, ROUGE-L), embedding-based similarity to the reference (BERTScore precision/recall/F1) and CLIPScore. CLIPScore is computed between the slide’s image embedding and the generated caption’s text embedding in each model’s own image–text contrastive space, and therefore measures whether the caption is consistent with what the model encodes from the slide rather than whether it resembles the reference; because each contrastive space has its own scale, CLIPScore is interpretable within a model but not across models. It is not reported for PRISM2, which exposes no general-purpose text encoder for arbitrary strings; BERTScore provides an encoder-agnostic reference-similarity measure for all three models. b, BLEU-4, ROUGE-L and CLIPScore on three cohorts (rows): 1,000 image–caption pairs from a held-out subset of the Mayo Clinic cohort spanning multiple organs and diagnoses; HistAI (48 pairs); and TCGA (50), neither external cohort seen during training. Bars show the mean over pairs, error bars one standard deviation; y-axis ranges differ between panels, so heights are comparable within but not across panels. WILSON-CoCa leads both baselines on the overlap metrics, by roughly 60% in relative terms on the internal cohort and by smaller margins on the external cohorts, and achieves the highest BERTScore recall while BERTScore precision and F1 remain comparable across models. Per-metric values are given in Table 13 and metric definitions and interpretation caveats in Supplementary Note 9. c, Qualitative examples (placenta, gallbladder) showing the WSI, its clinical report, the composite input and the captions generated by each model; further cases in Fig. 14 On 1,000 held-out image–caption pairs from the Mayo Clinic cohort, WILSON-CoCa produced descriptions with greater overlap with the reference than either baseline, reaching ROUGE-1 of 0.371 versus 0.142 for PRISM and 0.231 for PRISM2, ROUGE-L of 0.256 versus 0.119 and 0.167, and CIDEr of 0.224 versus 0.002 and 0.055 (Table 13, Figure 4a). WILSON-CoCa also achieved the highest BERTScore recall (0.878 versus 0.842 and 0.856), indicating fuller coverage of the reference findings, whereas its BERTScore precision was the lowest of the three (0.863 versus 0.881 and 0.895), consistent with longer descriptions that add content beyond the reference; BERTScore F1 was highest for PRISM2 (0.875 versus 0.861 for PRISM and 0.870 for WILSON-CoCa). On an external TCGA cohort of 50 image–caption pairs not seen during training, WILSON-CoCa led on BLEU-4 and CIDEr (0.009 and 0.090 versus at most 0.002 and 0.027), while ROUGE-1 and ROUGE-L were within rounding of PRISM2 (0.214 versus 0.215 and 0.159 versus 0.160) and above PRISM (0.183 and 0.150) (Table 13, Figure 4b). Averaged across metrics after min–max normalization within metric and cohort, WILSON-CoCa ranked first on all three cohorts (Table 13; Figure 1f). These comparisons require one qualification. WILSON-CoCa’s decoder and text components were adapted using Mayo Clinic data, whereas PRISM and PRISM2 were evaluated using their released pretrained models. The internal caption-similarity comparison therefore reflects both model architecture and adaptation to the target reporting domain and should not be interpreted as a controlled comparison of architecture alone. The external TCGA and HistAI evaluations provide complementary evidence that the observed captioning performance is not restricted to the Mayo Clinic cohort. Together, these results show that diagnostic information captured within a single multi-magnification composite can be translated into free-text pathology descriptions, while the errors visible in Figure 4c and Figure 14 confirm that the output is a draft description requiring pathologist review. 2.7 WILSON substantially reduces WSI-level computational requirements We next quantified the computational advantage of replacing conventional tile-wise WSI processing with a single composite representation. WILSON processes each WSI as one 2048×20482048× 2048 composite assembled from 64 selected 256×256256× 256 image regions. Its visual encoder comprises a ConvNeXt-Base backbone (87.6M parameters), followed by three stride-2 downsampling blocks (46.2M parameters) that progressively reduce spatial resolution while expanding the feature dimension from 1,024 to 3,072, followed by global average pooling to produce a 3,072-dimensional WSI representation. The resulting model contains 133.7M parameters and processes the complete composite in a single forward pass, without independently encoding thousands of tiles or requiring a separate slide-level feature aggregator. This formulation substantially reduced the computational requirements for WSI inference. Evaluated at a common workload of 10,000 tiles per WSI, WILSON required 2.58T FLOPs, compared with 25.07T for MOOZY, 703.19T for TITAN, 4.07P for PRISM, 4.13P for PRISM2 and 5.56P for Prov-GigaPath (Table 12, Figure 1e), corresponding to reductions of roughly 10-, 272-, 1,577-, 1,601- and 2,155-fold, respectively. WILSON was also smaller than every WSI- and case-level baseline except MOOZY, with 133.7M parameters against 354.6M for TITAN, 730.3M for PRISM, 1,220M for Prov-GigaPath and 1,251M for PRISM2, corresponding to 2.7-, 5.5-, 9.1- and 9.4-fold reductions in model size; MOOZY is the one comparison in which WILSON is the larger model (85.8M), although it still requires an order of magnitude more compute per WSI. These efficiency gains were achieved alongside the strongest averaged report-generation performance of the three generative models compared (Figure 1f), so lower cost did not come at the expense of downstream capability. The difference in computational complexity arises primarily from the unit of WSI representation. Conventional tile-based pipelines independently encode thousands of image regions before aggregating their representations at the slide level, causing computational cost to increase with the number of tiles processed. In contrast, WILSON performs a single forward pass through a fixed-size composite, with 2.57T FLOPs attributable to the backbone and only 16.2G FLOPs, some 0.6% of the total, to the three additional downsampling blocks and the projection to the final WSI representation. Because those blocks operate on feature maps no larger than 32×3232× 32, they hold 34.5% of the encoder’s parameters while contributing almost none of its compute. This fixed-size formulation therefore avoids repeated tile-level encoding and the subsequent materialization and aggregation of thousands of tile embeddings. These reductions are particularly relevant in light of the results above: the substantially lower computational complexity and smaller model size were achieved while maintaining competitive WSI-level retrieval performance and enabling end-to-end downstream adaptation. Although FLOPs and parameter counts do not directly measure wall-clock inference time, memory consumption or energy use, they demonstrate that multi-magnification composite representation can substantially reduce the computational complexity of WSI-level foundation modeling without requiring a conventional tile-encoder–aggregator pipeline. 3 Discussion Recent pathology foundation models have demonstrated impressive performance across a wide range of tasks using tile-based representations of whole-slide images. At the same time, diagnostic pathology presents several distinctive characteristics, including multiscale interpretation, hierarchical tissue organization, marked spatial heterogeneity and the integration of findings across multiple slides. These properties motivate continued exploration of how pathology images may be represented for foundation-model learning. WILSON was developed as an initial attempt to design a foundation model around these pathological requirements. Specifically, we explored whether tissue morphology could be represented as a compact multi-magnification composite that captures diagnostically informative regions across scales within a single image while remaining computationally tractable. The principal finding of this study is therefore not simply that WILSON achieved competitive performance, but that such a pathology-informed representation was sufficient to support diagnostic retrieval, vision-language alignment, report generation and downstream adaptation while remaining competitive with substantially larger pathology foundation models. The composite formulation was inspired by how pathologists themselves summarize tissue morphology. In routine diagnosis, education and scientific communication, pathologists rarely examine or present tissue as thousands of independent fields. Instead, representative low- and high-power views are integrated into a coherent diagnostic impression. Composite images can be viewed as a structured and reproducible translation of this process into a machine-readable representation. Importantly, this representation is not tied to arbitrary slide boundaries or tissue placement on a glass slide. By operating on selected tissue regions rather than predefined slide units, the same framework can naturally represent individual regions, entire slides or multi-slide cases. One consequence of this reformulation is that it enables true end-to-end foundation-model training. In practice, the computational burden of processing thousands of tiles has led most pathology foundation models to rely on frozen tile encoders and optimize only downstream aggregators. By reducing the computational unit to a single composite image, WILSON makes full-model optimization tractable while preserving multiscale tissue context. Although each composite contains only a fraction of the pixels present in the original whole-slide image, WILSON remained competitive with, and occasionally outperformed, state-of-the-art models that process orders of magnitude more visual information. This observation suggests that diagnostically relevant information may be concentrated within a relatively small number of informative regions and that representation design may be at least as important as exhaustive information preservation for pathology foundation models. The ability to fine-tune the entire visual encoder represents a second practical implication of this design. Whereas foundation-model development is often evaluated through zero-shot performance, many pathology studies involve only a few hundred cases. In such settings, adaptation to institution-specific morphology, scanner characteristics or niche clinical tasks may be more important than maximizing zero-shot accuracy. The improvements observed on the MayoTNBC cohort suggest that compact, fully differentiable foundation models provide a complementary route to adaptation that is difficult to achieve with conventional tile-based architectures. These findings should not be interpreted as evidence that composite representations supersede tile-based foundation models. Large-scale tile aggregation remains a powerful strategy, particularly when computational resources are abundant and maximizing zero-shot performance is the primary objective. Rather, WILSON should be viewed as an alternative point within the pathology foundation-model design space. More broadly, the fact that competitive performance was achieved using fewer than 200,000 training WSIs suggests that progress in pathology foundation models may depend not only on scaling model parameters and training data, but also on exploring representations that are more closely aligned with the biological and diagnostic structure of pathology. Several limitations warrant consideration. First, the training corpus comprised fewer than 200,000 whole-slide images from a single institution, which remains modest relative to contemporary pathology foundation models. Although this scale was partly intentional, reflecting our goal of developing a compact and computationally tractable model, it remains unclear how the architecture will perform when trained on substantially larger and more diverse datasets. Second, the 8×8 multi-magnification composite used in this study should be regarded as one instantiation of a broader representational design space rather than as an optimal formulation. The most informative selection of regions, magnifications and layouts may vary across organs, specimen types and disease entities. Future work should therefore explore adaptive or learnable composite-generation strategies and determine whether complementary composites can be integrated without reintroducing the complexity of large-scale aggregation pipelines. Importantly, such optimization should occur at the representation-learning stage, thereby preserving the generality expected of a foundation model. Taken together, our findings establish multi-magnification composite representation as a viable alternative to exhaustive tile-based pathology modeling and demonstrate that a compact end-to-end trainable foundation model can support diagnostic retrieval, vision-language alignment, report generation and downstream adaptation within a unified framework. The broader implication is not that the current composite design is definitive, but that pathology foundation models need not be constrained by representational assumptions inherited from natural-image computer vision. The central contribution of WILSON is therefore not a specific architecture, but the demonstration that pathology-specific representations themselves constitute an important and largely unexplored design space for foundation models. Figure 5: Data curation strategy for WILSON. a, Mayo Clinic pathology reports were segmented using an LLM. b, To maximize diagnostic diversity, the number of whole-slide images (WSIs) selected from each disease category was capped, and representative specimens were chosen from WSIs identified as key slides. c, Natural institutional distribution of WSIs before sampling. Colors indicate distinct disease groups, illustrating the strong imbalance typically observed in routine clinical archives. d, Distribution of the WILSON training dataset after the curation procedures described in panels a–d. Diversity-aware sampling substantially reduced organ- and disease-level imbalance while preserving broad diagnostic coverage. e, Overview of the WILSON-DS training dataset. f, Number of patients per diagnostic category (log scale). 4 Method Study design and ethics This retrospective study developed and evaluated a vision–language foundation model for whole-slide histopathology using routinely archived clinical material from Mayo Clinic together with public external cohorts. The study was approved by the Mayo Clinic Institutional Review Board (protocol number 25-008701), which waived the requirement for informed consent for the use of de-identified archival slides and reports. All Mayo Clinic data were de-identified before model development, and no protected health information was used for training or evaluation. Public cohorts (TCGA, CPTAC, HistAI) were used under their respective data-use terms. Model development used only the training partition described below; all reported results are on held-out slides, cases or cohorts that were never seen during any training stage. Case selection from the clinical archive Clinical pathology archives are highly imbalanced, with a small number of organs and diagnoses accounting for a large fraction of all cases (Figure 5c). To construct a training corpus that maximized morphological, diagnostic, and linguistic diversity rather than reflecting clinical prevalence, we performed diversity-aware sampling from a clinical archive containing 1,454,318 pathology cases and 10,904,856 whole-slide images (WSIs). Pathology reports were first segmented into organ-site level diagnostic entries. Organ sites were identified from report metadata and grouped into 42 predefined organ categories by a board-certified pathologist using rule-based mappings. To characterize the highly heterogeneous diagnostic landscape within each organ, diagnoses were extracted from reports using a large language model and grouped into organ-specific disease categories using embedding-based clustering followed by pathologist curation. These disease categories were designed to capture diagnostically meaningful diversity and to consolidate common terminology variants rather than to generate case-level ground-truth labels. Representative WSIs were selected at the organ-site level using report-derived evidence. When multiple tissue blocks were available, a representative block was identified from explicit report references or ancillary testing patterns; organ sites without an identifiable representative block were excluded. Additional details of disease-category construction, representative-block selection, and scanner-specific rules are provided in Supplementary Note 7. To prevent common disease entities from dominating the dataset, a maximum of 350 WSIs was sampled from each organ–disease category. This strategy reduced more than 10 million archived WSIs to a balanced and diverse training corpus of 189,291 WSIs spanning 838 organ–disease combinations (Mayo189K dataset). The resulting dataset substantially increased representation of rare organs and diagnoses relative to the source archive while preserving broad morphological coverage (Figure 5c,d). Report-derived supervision For each selected organ site, organ-specific diagnostic text was extracted from the corresponding pathology report using Gemini 2.5 Pro Gemini Team (2025). The extracted text was converted into three semantically equivalent but lexically diverse histopathology captions, which were used as text targets for contrastive training and embedded with Gemini Embedding 2 Shanbhogue et al. (2026) to generate fixed text representations. (The details are described in Supplementary Note 7) A complementary keyword-based supervision signal was generated by assigning curated pathology terms supported by the report text using embedding-based retrieval and LLM verification. This produced a sparse multi-label keyword vector for each WSI. Details of report cleaning, caption generation, keyword curation, and prompts are provided in Supplementary Note 7. Multi-magnification composite generation Rather than encoding a slide as thousands of independent tiles, we represent each WSI as a fixed-size composite: an 8×88× 8 grid of 256×256256× 256-pixel tiles forming a single 2048×20482048× 2048 RGB image that presents selected tissue locations at several magnifications simultaneously. Composites are generated by an organ-aware, abnormality-guided procedure in three phases. Figure 6: COMPOSITE: Clustering-Oriented Composite Assembly Pipeline. a, Phase 1 builds a normal-tissue reference. Tissue regions in healthy whole-slide images (WSIs) are detected by Hue–Saturation–Density thresholding Bejnordi et al. (2016) and randomly tiled; a DINO pre-trained Vision Transformer (ViT) extracts 384-dimensional CLS-token embeddings in a single forward pass. Pooled embeddings are clustered with MiniBatch k-means (k=200k=200), and each cluster mean is replaced by its medoid so that centroids correspond to genuine tissue appearances. The 200 prototypes form a reusable organ-specific reference. b, Phase 2 scores tiles in disease slides. The same ViT emits both the CLS embedding and quality logits (valid/invalid) in one pass; sub-threshold tiles (marked x) are discarded as artifacts. Remaining tiles receive an abnormality score equal to their minimum Euclidean distance to the 200 normal centroids, higher values indicating greater deviation from normal tissue. c, Phase 3 selects tiles with spatial diversity. Naive top-k selection (red x) concentrates tiles in a single lesion, sacrificing representativeness. Instead, the tissue bounding box is divided into a spatial grid, the highest-scoring tile per cell is selected, and globally best unused tiles fill empty cells—guaranteeing coverage of all tissue zones while biasing toward diagnostically informative locations, yielding 60 positions per slide. Ranked locations are distributed round-robin across the 10 composites in order of abnormality score, so each composite spans the full abnormality spectrum and all spatial zones rather than concentrating the most atypical locations in one image. Locations per composite is configurable: the schematic shows a six-location layout (6×66× 6 grid), whereas all experiments used eight locations (8×88× 8 grid; panel d). Within each composite, rows are tissue locations ordered by abnormality score and columns encode multi-scale detail, from a 1.25× context view through 5× and 20× magnifications. d, Example 8×88× 8 composite. Eight low-power regions (A–H; 256×256256× 256 px, 8μ8~ /pixel) were drawn from four WSIs in the CaseLuminal dataset, each contributing three mid-power (green boxes; 256×256256× 256 px, 2μ2~ /pixel) and four high-power regions (blue boxes; 256×256256× 256 px, 0.5μ0.5~ /pixel), assembled into a single 2048×20482048× 2048 px composite. In the first phase, executed once per organ, a compact reference of normal tissue appearance is built from histologically unremarkable slides (Figure 6a). Tissue tiles are embedded using a Vision Transformer that simultaneously provides quality-control scores to remove artifactual regions. Embeddings from normal slides are clustered to generate an organ-specific set of representative normal tissue medoids. In the second phase (Figure 6b), tiles from each slide are embedded with the same network and assigned an abnormality score based on their distance from the organ-specific normal reference. To balance diagnostic focus with spatial coverage, high-scoring tiles are selected using a spatially aware sampling strategy. The NselectN_select highest-scoring candidates are retained and distributed across ten composites in round-robin order of abnormality score, with eight tissue locations per composite (the composite grid size is a configurable parameter; all experiments used an 8×8 layout) so that each composite spans the slide’s full abnormality spectrum rather than concentrating the most atypical regions in a single image. In the third phase (Figure 6d, 16), the selected tissue locations are assembled into multi-magnification composites. Each composite is an 8×88× 8 grid of 256×256256× 256-pixel tiles in which each row corresponds to one selected tissue location. The first column contains a 1.25×1.25× context tile covering the largest field of view. From this context region, three representative 5×5× tiles and four representative 20×20× tiles are selected, yielding seven higher-magnification views spatially linked to the same location. The 1.25×1.25× context tile is placed in the leftmost column, while the corresponding higher-magnification tiles occupy the remaining columns of the same row, preserving the relationship between low- and high-magnification observations. The resulting 8×88× 8 composite therefore encodes eight tissue locations, each represented by one contextual view and seven higher-resolution views. The composite serves as a compact visual summary of a whole-slide image, jointly encoding tissue heterogeneity, multi-scale morphology, and cross-scale spatial correspondence within a single image. Vision encoder All WILSON variants share the same convolutional encoder (ConvNeXtV3Extended; Supplementary Fig. 17 and Supplementary Table 27). A ConvNeXt-Base backbone Liu et al. (2022) (stage depths 3,3,27,33,3,27,3; widths 128,256,512,1024128,256,512,1024; overall stride 32; 87.57M parameters) is followed by three additional stride-2 downsampling blocks, each consisting of layer normalization, a 2×22× 2 stride-2 convolution and GELU, which expand the channel width from 1,024 to 3,072 (46.15M parameters) while reducing the 64×6464× 64 backbone feature map of a 2048×20482048× 2048 composite to 8×88× 8. The resulting 64 spatial tokens of dimension 3,072 are used by the report-generation decoder, and their global average yields a 3,072-dimensional slide embedding matched to the dimensionality of the Gemini text embeddings, which removes the need for a cross-modal adapter. The encoder has 133.7M parameters and processes a composite in a single forward pass of 2.6 TFLOPs; because the extended blocks operate on feature maps no larger than 32×3232× 32, they contribute 34.5% of the parameters but 0.6% of the compute. When the encoder is warm-started from a backbone-only self-supervised checkpoint, the extended-block convolutions are initialized by tiling the final backbone downsampling filters and rescaling them to He-initialization magnitude, preserving learned filter directions while keeping activations within half-precision range (Extended Data Fig. 17d). Self-supervised pretraining The WILSON vision encoder was obtained through a two-stage self-supervised learning (SSL) curriculum built on a ConvNeXt backbone (ConvNeXt-V3 with a global-average-pooling head), initialized from DINOv3 ConvNeXt-Base weights pretrained on the LVD-1689M natural-image corpus. Both stages used a DINO-style teacher–student self-distillation objective with multi-crop augmentation and histopathology-specific augmentations (random rotation and color/stain jitter), applied to hematoxylin-and-eosin whole-slide images spanning multiple organs and diagnoses. In the first, tile-based, stage the encoder was trained on individual 224×224224× 224-pixel tissue tiles for 20 epochs on 8 GPUs (H200; total batch size 1,024; learning rate 2×10−42× 10^-4 with a 1-epoch warm-up), using two global crops (224224 px) and eight local crops (9696 px) per tile. In the second, composite-based, stage four spatially co-registered 256×256256× 256-pixel patches captured at complementary magnifications (2.5×2.5×, 10×10×, and two 20×20× fields) were tiled into a single 512×512512× 512-pixel composite image (Fig. 1d), and training proceeded in three successive steps starting from the tile-based teacher checkpoint: (i) the full network was trained end-to-end at 512512-px resolution across two nodes (16 GPUs total; total batch size 512; learning rate 2×10−42× 10^-4); (i) additional SSL ”extension” blocks were appended to the pretrained backbone, which was frozen, and trained alone for 3 epochs (learning rate 1×10−41× 10^-4) on an expanded corpus of approximately 10 million composite tiles, allowing the new blocks to warm-start without perturbing the pretrained backbone features; and (i) the entire extended network was then unfrozen and fine-tuned end-to-end for a further 5 epochs (learning rate 1×10−41× 10^-4 with a 1-epoch warm-up) on the same corpus. The resulting composite-based teacher network constitutes the final WILSON vision encoder used in all downstream experiments. Vision–language training Starting from the composite-based SSL checkpoint (§4), four downstream training strategies were evaluated, each targeting a different downstream capability: (i) dense WILSON, obtained by SigLIP-style distillation from frozen Gemini caption embeddings, for retrieval against free-text queries without a trainable text tower; (i) sparse WILSON, obtained by multi-label regression against a controlled pathology-keyword vocabulary, for lightweight structured tagging and retrieval; (i) WILSON-CLIP, a jointly trained multi-caption contrastive alignment followed by supervised diagnostic refinement (§4), for retrieval and direct diagnosis/tissue classification; and (iv) a generative captioning model, obtained by fine-tuning a CoCa decoder on top of the distilled dense WILSON encoder, for free-text report generation. All four strategies share the same underlying corpus of pre-rendered 2048×20482048× 2048 multi-magnification WSI composites and LLM-generated captions, and all initialize their vision encoder from the same composite-based SSL checkpoint (WILSON-CLIP and CoCa are the two exceptions in the sense that CoCa initializes one step further downstream, from the already-distilled dense WILSON encoder — see §4). Shared training corpus The composite corpus contains 178,088 whole-slide images (WSIs) rendered into 1,760,461 composite images in total (≈ 9.9 composites per WSI on average). Of these, 178,020 WSIs (99.96%) have a matching LLM-generated caption record (three independent summary variants per report, §7) together with frozen Gemini text embeddings for the full report and for each of the three caption variants; all 178,088 composite-bearing WSIs have a matching multi-hot pathology-keyword annotation (a controlled vocabulary of 991 terms), regardless of whether a caption is available. Pairing every composite with its WSI’s three caption variants therefore yields 1,759,787×3=5,279,3611,759,787× 3=5,279,361 realized image–caption training pairs (≈ 5.28 million). Both caption-driven stages (dense WILSON and CoCa, §4, §4) sample this corpus with row-shuffle sampling: at each step, rather than iterating once over a fixed list of composites, two composites per WSI are pooled and freshly resampled, and this resampling is repeated 30 times per WSI over the course of an epoch (row_shuffle_ repeats =30=30). This oversamples the WSI-level diversity of the corpus — exposing the model to many different composite/caption-variant combinations per WSI — rather than treating each epoch as a single static pass over the 1.76M composites, and is the reason the realized number of image–text pairs seen during training substantially exceeds one pass over the static corpus size reported in Table 26. Dense WILSON: SigLIP-style distillation from Gemini caption embeddings The dense WILSON encoder was fine-tuned from the composite-based SSL checkpoint using a SigLIP-style objective Zhai et al. (2023) in which the target text representations are frozen, pre-computed Gemini embeddings of each WSI’s report or one of its three caption variants, rather than embeddings produced by a jointly trained text encoder. Unlike the symmetric softmax (InfoNCE) contrastive loss used by WILSON-CLIP (§4), a SigLIP-style loss treats every image–text pair as an independent binary classification (match/non-match) via a sigmoid, so it does not require the full in-batch normalization that a softmax loss does; this makes training less sensitive to the (comparatively small) global batch size available here. Distilling directly from Gemini’s frozen embedding space — rather than training a text tower from scratch on ∼ 178K captioned WSIs — lets the vision encoder inherit Gemini’s broader clinical-language representation, and removes the need to maintain a separate text encoder at inference time for text-conditioned retrieval. Architecturally, this works without an adapter layer because ConvNeXtV3Extended’s native GAP output is 3072-dimensional (§4), matching the Gemini embedding dimension exactly, so image and text embeddings are compared directly in the same 3072-d space rather than through a smaller learned shared projection (as used by WILSON-CLIP’s 256-d and CoCa’s 768-d joint spaces). Training used the row-shuffle sampling scheme described above (§4), drawing composite/caption pairs from the paired corpus (178,020 WSIs) for 20 epochs on 4 nodes × 8 H200 GPUs (32 workers), with a per-GPU batch size of 4 and 16 gradient accumulation steps, giving an effective batch of 32×4×16=2,04832× 4× 16=2,048 image–text pairs per optimizer step. 2% of WSIs were held out for validation, and checkpoints were saved every 5,000 steps. The resulting vision encoder is the initialization used for the CoCa captioning stage below (§4), making dense WILSON the common ancestor of both the retrieval-only and the generative branches of the pipeline. Sparse WILSON: multi-label pathology-keyword regression The sparse WILSON variant fine-tunes the same composite-based SSL checkpoint against a controlled vocabulary of pathology keywords instead of free-text captions. Each WSI’s report is reduced to a multi-hot target vector over a 991-term vocabulary, and the contrastive/captioning heads used by the dense and CLIP variants are disabled (keywords_only) so that the vision encoder is optimized purely for multi-label keyword regression against this sparse target (loss weight 1.0). Because keyword annotation does not depend on a WSI having a generated caption, this strategy is trained on the full composite-bearing corpus (178,088 WSIs, 1,760,461 composites; §4), 68 WSIs more than the caption-paired corpus used by the dense and CoCa stages — giving sparse WILSON the largest usable training set of the four strategies, at the cost of a coarser, non-generative supervisory signal. This variant provides a computationally light structured alternative to free-text captioning: the 991-dimensional multi-hot target is cheaper to predict against than a full caption or report embedding, and yields directly interpretable per-keyword tags that are useful for controlled-vocabulary retrieval and as an ablation of how much representational benefit the richer caption-level supervision (dense WILSON, WILSON-CLIP) provides over simple tag prediction. Training ran for 7 epochs (a shorter budget than the other three stages, consistent with its role as a lighter-weight variant) on a single node of 8 H200 GPUs, with a per-GPU batch size of 4 and 16 gradient accumulation steps (effective batch 8×4×16=5128× 4× 16=512), holding out 10% of WSIs for validation — a larger validation fraction than the other stages (2%), chosen to obtain a more stable validation estimate given the shorter, 7-epoch training budget. WILSON-CLIP: multi-caption contrastive alignment and supervised refinement As a third strategy, a collaborator trained WILSON-CLIP, which likewise initializes its visual encoder from the composite-based SSL checkpoint but — unlike dense WILSON’s frozen-embedding distillation — aligns it against all three caption variants per composite with a jointly trained text encoder (PathologyBERT) under a symmetric multi-positive InfoNCE (CLIP) objective (Phase 2). WILSON-CLIP is therefore the only one of the four strategies in which the text encoder itself is updated during vision–language training rather than held frozen (dense WILSON, sparse WILSON) or trained only downstream of a frozen retrieval space (CoCa’s contrastive branch reuses dense WILSON’s vision tower but trains its own CoCa text tower). WILSON-CLIP is also the only strategy with a second, purely supervised stage: Phase 3 fine-tunes the vision encoder alone against tissue and fine-grained diagnosis labels while the text encoder and CLIP projections remain frozen, giving WILSON-CLIP a direct classification capability that the other three variants do not have. Full architectural and optimization details, including the fixed 1,410,600/349,861 train/val composite manifest, are given in §4. CoCa: generative captioning The final stage fine-tunes a generative captioning model on top of the distilled dense WILSON encoder (§4), following the CoCa recipe Yu et al. (2022): a contrastive loss is trained jointly with an autoregressive next-token captioning loss over an open-clip CoCa (coca-ViT-L-14) text tower and multimodal decoder. This is the only one of the four strategies that produces free text at inference time — dense WILSON, sparse WILSON, and WILSON-CLIP all produce embeddings or class logits for retrieval/classification, whereas CoCa’s decoder is the “report generator” of Fig. 1d. The WILSON vision tower (ConvNeXtV3Extended) is fine-tuned in full (backbone and extension blocks; no frozen layers), and its 8×8=648× 8=64 spatial tokens for a 2048×20482048× 2048 composite are summarized by a 256-query, 8-head attentional pooler — trained from scratch, with no pretrained analog — before being cross-attended by the decoder. Concretely, the decoder generates a caption autoregressively: causal self-attention over previously generated text tokens is combined, at every decoder layer, with cross-attention over the 256 pooled visual tokens, and the next token is predicted at each step until an end-of-sequence token is produced. The CoCa text tower is trained jointly (no frozen layers); the decoder keeps its pretrained weights as initialization but is further trained rather than kept frozen. The training objective combines a SigLIP-style sigmoid contrastive loss — cross-GPU gathered embeddings supplemented by an 8-step negative bank (a rolling queue retaining embeddings from the previous 8 optimizer steps as additional negatives, initial logit scale 10.0 and logit bias −10.0-10.0, shared embedding dimension 768) — with the captioning cross-entropy loss at unit weight. The negative bank compensates for the comparatively modest number of in-batch negatives available at this scale (32 GPUs × 4 per-GPU =128=128 composites per step, versus the tens-of-thousands-scale batches typical of large CLIP training runs), giving the contrastive term a larger effective negative pool without increasing per-step memory cost. Training used the same paired corpus and row-shuffle sampling scheme as the dense WILSON stage (§4) for 20 epochs on 4 nodes × 8 H200 GPUs (32 workers), with a per-GPU batch size of 4 and 8 gradient accumulation steps, giving an effective batch of 32×4×8=1,02432× 4× 8=1,024 image–caption pairs per optimizer step (learning rates: 10−510^-5 for the vision tower and text tower, 10−310^-3 for the projection heads, 10% warm-up over 0.1 epochs, cosine decay to 1% of the peak rate, weight decay 10−410^-4, gradient clipping at 1.0, bfloat16 mixed precision). 2% of WSIs were held out for validation. Initializing from the dense WILSON encoder, rather than from the composite-based SSL checkpoint directly or from WILSON-CLIP, gives the CoCa decoder a vision tower whose representation is already aligned to a rich clinical-language embedding space (§4), rather than one that has been additionally specialized for closed-set diagnosis/tissue classification (WILSON-CLIP Phase 3) or left purely self-supervised (composite SSL) — a better-matched starting point for free-text generation. Composite versus patch-based slide representation To test whether a composite preserves the information available in a conventional patch-based representation independently of WILSON’s training, both representations were encoded with the same frozen tile foundation model, UNI Chen et al. (2024). For the patch-based arm, 2% tiles were selected with Yottixel Kalra et al. (2020) and their UNI embedding scores aggregated into a slide score; for the composite arm, the 2048×20482048× 2048 composite was encoded by UNI directly in a single pass. Slide-level retrieval was evaluated by top-1 macro-averaged F1 on BCTherapy, TCGA-Kidney, MayoSkin and MayoBreast. Zero-shot retrieval evaluation WILSON was evaluated without any task-specific training using embedding-space retrieval. A leave-one-patient-out protocol was employed: for each query slide, all slides originating from the same patient were excluded from the retrieval database to prevent patient-level information leakage, and retrieval was then performed over slides from all remaining patients. The k nearest neighbors by cosine similarity were retrieved and the query was assigned the majority label among them (majority voting, MV@k), evaluated at k=1,3,5k=1,3,5 and 77; k=1k=1 reduces to direct top-1 label transfer. Performance was quantified with three complementary metrics: accuracy, balanced accuracy (the mean per-class recall) and macro-averaged F1-score. Because several benchmarks are strongly class imbalanced — most markedly TCGA-BRCA and CPTAC-BRCA, where a single subtype dominates — raw accuracy can be inflated by majority-class prediction, so balanced accuracy and macro-averaged F1 are the informative measures on those cohorts. The main text reports macro-averaged F1 at MV@5 throughout; the complete grid of all three metrics at every k is given for case-level retrieval in Figs. 7–9 and for slide-level retrieval in Figs. 10–12. Because ten composites are generated per slide, retrieval was repeated once per composite and the mean and standard deviation across the ten runs are reported; the standard deviation therefore measures sensitivity to composite sampling rather than across-patient uncertainty. Case-level composites integrate tiles from all WSIs of a case as described above. Baselines were run under the identical protocol using their published checkpoints: MOOZY Kotp et al. (2026) for case-level retrieval, and TITAN Ding et al. (2025), PRISM Shaikovski et al. (2024) and Prov-GigaPath Xu et al. (2024) for slide-level retrieval, each operating on its own frozen tile-level features and slide aggregator and therefore yielding a single deterministic embedding per slide. Parameter counts were taken from the released models, and per-slide FLOPs were computed analytically at a 2048×20482048× 2048 input for WILSON and at each baseline’s native tile size assuming N=10,000N=10,000 tiles per WSI (Table 12). Cross-modal retrieval evaluation Image-to-text (I2T) and text-to-image (T2I) retrieval were evaluated in the shared image–text embedding space. For I2T, each slide embedding was used to query the set of reference caption embeddings; for T2I, each caption embedding was used to query the set of slide embeddings. In both directions the candidate set was ranked by cosine similarity, and a retrieved item counted as a match if it belonged to the same disease group as the query, so the task measures disease-group discrimination rather than exact caption recovery. Performance was reported as recall at K (K∈1,3,5,10K∈\1,3,5,10\), the fraction of queries with at least one match in the top K, together with the mean and median rank of the first match; lower ranks are better, and a median rank of 1 indicates that a same-disease-group item was retrieved first for the majority of queries. Three cohorts were used. The internal MayoCaption cohort comprises 456 WSIs from 397 patients across three organs and is reported both pooled (Table 3) and per organ: duodenum (n=198n=198; Table 4), pancreas (n=110n=110; Table 5) and stomach (n=148n=148; Table 6). The two external cohorts are TCGA-Brain (six glioma subtypes, n=124n=124; Table 7) and TCGA-Kidney (three renal-carcinoma subtypes, n=86n=86; Table 8). PRISM Shaikovski et al. (2024) was evaluated under the same queries, candidate sets and match criterion using its released image and text encoders. End-to-end fine-tuning Because WILSON’s encoder is a single differentiable network, it can be fine-tuned end-to-end on downstream labels, which tile-encoder pipelines with frozen feature extractors cannot do. We fine-tuned the dense (report-distilled) WILSON encoder on MayoTNBC, an internal cohort of 508 triple-negative breast carcinoma WSIs from 508 patients annotated with two four-class labels, histologic subtype and sTIL grade (Table 14). Each task was fine-tuned as a separate run. These slides were acquired on scanners different from those used for the Mayo189K pretraining corpus, so the cohort also probes robustness under scanner domain shift. Train and validation splits were fixed in advance and are disjoint at the patient level. A linear head was attached to the encoder’s 3072-dimensional pooled output and the network was optimized with a cross-entropy loss, with the backbone held frozen for the first two of five epochs before full end-to-end fine-tuning. Three settings were compared: the pretrained encoder with no adaptation (Base Zero-shot); the fine-tuned encoder with its head discarded, assessing the backbone representation alone (FT-Zero-shot); and the trained head evaluated directly as a classifier (FT-classification). The first two share a single nearest-neighbor retrieval metric and therefore isolate what fine-tuning did to the embedding space, whereas the third measures a learned decision boundary and is not expected to agree numerically with the second. Accuracy and macro-averaged F1 are reported for both tasks (Table 2); full optimization and evaluation details are given in the Supplementary Methods (§8). Caption evaluation Generated captions were compared with reference captions on three held-out cohorts: 1,000 Mayo Clinic WSIs, 48 HistAI WSIs and 50 TCGA WSIs, none of which were seen during any training stage. The three cohorts differ in how their references were derived. Mayo Clinic references are the caption variants generated from pathology reports with Gemini 2.5 Pro Gemini Team (2025) (§7). TCGA references were derived from the machine-readable pathology reports of Kefeli et al. Kefeli and Tatonetti (2024) and summarized into short captions with DeepSeek V4 Pro DeepSeek-AI (2026). HistAI references are the diagnostic conclusion field supplied with the dataset’s own metadata, used verbatim without LLM summarization; HistAI also distributes a longer full-report field, which was not used, so that all three cohorts are scored against short-caption-style references. Captions were scored with eight automated metrics spanning three families. (i) Surface n-gram overlap: BLEU-4, ROUGE-1 and ROUGE-L, which credit only literal token matches (exact 1–4-gram precision, unigram F-measure and longest-common-subsequence F-measure, respectively). (i) Weighted n-gram overlap: CIDEr, the cosine similarity of TF–IDF-weighted 1- to 4-gram vectors, which down-weights corpus-frequent n-grams so that distinctive, content-bearing ones dominate. (i) Embedding-based similarity: BERTScore precision, recall and F1, computed by greedy cosine matching of RoBERTa-large token embeddings, and CLIPScore, the cosine similarity between the slide’s image embedding and the generated caption’s text embedding in each model’s own image–text contrastive space. CLIPScore therefore measures agreement between the caption and what the model itself encodes from the slide rather than agreement with the reference, and is not comparable across models because each contrastive space has its own learned scale; it is not computable for PRISM2, which exposes no standalone text encoder. Full metric definitions and interpretation caveats are given in the Supplementary Methods (§9). PRISM Shaikovski et al. (2024) and PRISM2 Shaikovski and others (2025) were evaluated off the shelf on the same slides, matched by fixed random seed. Datasets All datasets used in this study are listed in Tables 14, 15. No evaluation slide or patient overlapped with the training corpus, and class counts are provided in Supplementary Tables 16–23. At the slide level, Mayo189K is an internal Mayo Clinic dataset used for training the WILSON model. In addition to whole-slide images (WSIs), associated slide captions and pathology-related keywords are available for each case. MayoCaption is a caption-based retrieval benchmark derived from a validation subset of Mayo189K and was developed for cross-modal retrieval experiments between pathology images and text descriptions. It consists of three organ-specific datasets: MayoDuodenum, MayoStomach, and MayoPancreas. MayoBreast (Table 18) comprises 1,339 WSIs from 1,289 patients across twelve breast diagnoses that span invasive carcinoma (no special type, lobular, mixed, mucinous and metaplastic), in situ disease (ductal and lobular carcinoma in situ), benign and fibroepithelial lesions (fibroadenoma, phyllodes tumor, benign proliferative lesion and fat necrosis), and lymphoproliferative disease. This cohort was sampled to be approximately balanced and has no patient-level overlap with Mayo189K. MayoSkin (Table 19) comprises 688 WSIs from 677 patients across twenty dermatopathologic diagnoses spanning melanocytic lesions (nevus, nevus with atypia, blue nevus, lentigo and malignant melanoma), keratinocytic neoplasia (actinic keratosis, seborrheic keratosis, basal cell carcinoma and squamous cell carcinoma), inflammatory dermatoses (psoriasiform and lichenoid dermatitis), cutaneous lymphoma, and a range of benign entities (dermatofibroma, neurofibroma, epidermal cyst, verruca, skin tag, scar, sebaceous gland hyperplasia and chondrodermatitis nodularis helicis). This cohort also has no patient-level overlap with Mayo189K. MayoTNBC is a WSI-level cohort comprising 508 patients with triple-negative breast cancer. WSIs are stored in Philips TIFF format and are associated with two labels: stromal tumor-infiltrating lymphocyte (sTIL) score and histologic subtype. TCGA-BRCA, TCGA-Brain, and TCGA-Kidney are derived from The Cancer Genome Atlas (TCGA) project Weinstein et al. (2013). For histologic classification tasks, we adopted the AI-ready diagnostic labels described by Uegami et al. Uegami et al. (2026). TCGA caption data were generated by summarizing pathology reports extracted by Kefeli et al. Kefeli and Tatonetti (2024) using a large language model (DeepSeek V4 ProDeepSeek-AI (2026)). TCGA-BRCA (Table 16) comprises 1,087 WSIs from 1,018 patients labeled with seven histological subtypes of invasive breast carcinoma. Its class distribution mirrors clinical prevalence and is therefore strongly imbalanced: invasive carcinoma of no special type accounts for 812 slides and invasive lobular carcinoma for 204, whereas the remaining five subtypes (mixed ductal and lobular, mucinous, metaplastic, medullary and invasive micropapillary carcinoma) together contribute 71 slides, with as few as four examples in the rarest class. BCTherapy Sammut et al. (2022) (Table 17) comprises 160 WSIs from 160 patients treated with neoadjuvant therapy, labeled as residual disease (n=119n=119) or pathologic complete response (n=41n=41); it is the smallest benchmark and the only binary one at the slide level. HistAI Nechaev et al. (2025) is a multi-organ dataset used for evaluating cross-modal retrieval and pathology caption generation. Fifty cases (five per organ across ten organs) were selected from the original HistAI-Mixed cohort to maximize histologic diversity. Two cases were excluded because of corrupted WSI files, resulting in a final cohort of 48 patients. From the publicly available metadata, the conclusion field was used as the caption and the specialization field was used as the organ label. All selected slides are listed in Table 24. At the case level, each unit of evaluation is a patient case comprising multiple WSIs, from which a single composite is assembled. CPTAC-BRCA National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC) (2020) (Table 21) is a publicly available dataset comprising 198 cases and 653 WSIs. A case is defined as the collection of WSIs belonging to a single patient, with an average of 3.5 WSIs per case. The dataset is labeled with the five PAM50 intrinsic molecular subtypes Parker et al. (2009) (luminal A, luminal B, HER2-enriched, basal-like, and normal-like) and is used for PAM50 genotype classification, making it the only benchmark that requires inferring a transcriptomic class from morphology. MayoCaseBreast (Table 22) comprises 494 cases and 4,361 WSIs across the same twelve breast diagnoses as MayoBreast. MayoCaseBreast is a patient-level subset of the MayoBreast dataset, with identical diagnostic categories. All available H&E slides from each case are included. Within a given case, all WSIs originate from either a Leica or a Pramana scanner, with no mixing of scanner types within the same case. MayoCaseSkin (Table 23) comprises 500 cases and 1,280 WSIs across the same twenty dermatopathologic diagnoses as MayoSkin. Similar to MayoCaseBreast, MayoCaseSkin is a case-level dataset derived from MayoSkin and includes all available H&E slides from each selected patient case. MayoBreastSubtype (Table 20) is an internal case-level cohort comprising 846 breast cancer cases treated with neoadjuvant chemotherapy at Mayo Clinic. A case is defined as the collection of WSIs belonging to a single patient. Cases are categorized into three clinically relevant subtypes: hormone receptor-positive/HER2-negative, HER2-positive, and triple-negative breast cancer. The cohort has no patient-level overlap with Mayo189K. Statistics and reproducibility No statistical method was used to predetermine sample size. Error bars on retrieval metrics denote the standard deviation across the ten composites generated per slide or case; baselines that operate on frozen tile features yield a single deterministic embedding per slide and are therefore shown as single values without variance. Training used a fixed random seed, and evaluation subsets were drawn with a single fixed seed shared across all models so that every model was scored on identical slides. Experiments were implemented in PyTorch with open_clip and run on an NVIDIA DGX cluster of H200 GPUs, using between 8 and 32 GPUs (1–4 nodes × 8) depending on the training stage; per-stage GPU counts are given in Table 26. Data availability All external data used in this study are publicly available. TCGA whole-slide images (TCGA-BRCA, TCGA-Brain and TCGA-Kidney) were obtained from the NCI Genomic Data Commons (https://portal.gdc.cancer.gov); the AI-ready diagnostic labels applied to them are available from Uegami et al. Uegami et al. (2026) (https://doi.org/10.5281/zenodo.19736866). The TCGA pathology-report text from which TCGA reference captions were derived is the machine-readable corpus of Kefeli and Tatonetti Kefeli and Tatonetti (2024), available from Mendeley Data (https://doi.org/10.17632/hyg5xkznpx.1) with the accompanying extraction pipeline at https://github.com/tatonetti-lab/tcga-path-reports. CPTAC-BRCA whole-slide images are available from The Cancer Imaging Archive National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC) (2020) (https://doi.org/10.7937/TCIA.CAEM-YS80). The HistAI dataset, including the whole-slide images and the per-case clinical metadata from which HistAI reference captions were taken, is available from https://github.com/HistAI/HISTAI and the associated Hugging Face repositories (https://huggingface.co/datasets/histai/HISTAI-metadata) Nechaev et al. (2025). The BCTherapy (TransNEO) hematoxylin-and-eosin slides used for treatment-response prediction are available from Zenodo (https://doi.org/10.5281/zenodo.6337925), with the accompanying study described by Sammut et al. Sammut et al. (2022). Mayo Clinic whole-slide images and pathology reports contain protected health information and cannot be shared. De-identified derived data, comprising composite tile coordinates, LLM-generated captions and keyword annotations for the Mayo189K corpus together with the evaluation slide identifiers for the MayoBreast, MayoSkin, MayoTNBC, MayoCaption and case-level benchmarks, are available from the corresponding author on reasonable request under a data-use agreement and subject to Mayo Clinic institutional approval. Acknowledgments The authors gratefully acknowledge the F. Craig and Patricia Jilk Fund for Data Science, Predictive Modeling & AI for Breast Cancer for supporting this study. The authors also acknowledge the Mayo Clinic Comprehensive Cancer Center, Rochester, MN, USA, for its ongoing support. Model Availability Upon publication and subject to clearance by the Mayo Clinic Ethics Office, the WILSON model will be made available for download to the research community to support further development, evaluation, and validation of computational pathology methods. References Abdin et al. (2024) M. Abdin et al. Phi-3 technical report: a highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219. Note: Microsoft Cited by: §2.6. Bejnordi et al. (2016) B. E. Bejnordi, G. Litjens, N. Timofeeva, I. Otte-Höller, A. Homeyer, N. Karssemeijer, and J. A. W. M. van der Laak Stain specific standardization of whole-slide histopathological images. IEEE Transactions on Medical Imaging 35 (2), p. 404–415. External Links: Document Cited by: Figure 6, Figure 6, §7. Cancer Genome Atlas Research Network et al. (2013) Cancer Genome Atlas Research Network, J. N. Weinstein, E. A. Collisson, G. B. Mills, K. R. M. Shaw, B. A. Ozenberger, K. Ellrott, I. Shmulevich, C. Sander, and J. M. Stuart The cancer genome atlas pan-cancer analysis project. Nature Genetics 45 (10), p. 1113–1120. External Links: Document Cited by: Table 16, Table 16. Caron et al. (2021) M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), IEEE/CVF (Ed.), p. 9650–9660. External Links: Document Cited by: §7, §7. Chen et al. (2022) R. J. Chen, C. Chen, Y. Li, T. Y. Chen, A. D. Trister, R. G. Krishnan, and F. Mahmood Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE/CVF (Ed.), p. 16144–16155. External Links: Document Cited by: §1. Chen et al. (2024) R. J. Chen, T. Ding, M. Y. Lu, D. F. K. Williamson, G. Jaume, A. H. Song, B. Chen, A. Zhang, D. Shao, M. Shaban, M. Williams, L. Oldenburg, L. L. Weishaupt, J. J. Wang, A. Vaidya, L. P. Le, G. K. Gerber, S. Sahai, W. Williams, and F. Mahmood Towards a general-purpose foundation model for computational pathology. Nature Medicine 30, p. 850–862. External Links: Document Cited by: §1, §2.1, §4. DeepSeek-AI (2026) DeepSeek-AI DeepSeek-V4: towards highly efficient million-token context intelligence. Note: Technical reportPreview release, 24 April 2026 External Links: Link Cited by: §4, §4. Ding et al. (2025) T. Ding, S. J. Wagner, A. H. Song, R. J. Chen, M. Y. Lu, A. Zhang, A. J. Vaidya, G. Jaume, M. Shaban, A. Kim, D. F. K. Williamson, H. Robertson, B. Chen, C. Almagro-Pérez, P. Doucet, S. Sahai, C. Chen, et al. A multimodal whole-slide foundation model for pathology. Nature Medicine 31 (11), p. 3749–3761. External Links: Document Cited by: §1, §2.3, §4. Gemini Team (2025) Gemini Team Gemini 2.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. External Links: 2507.06261, Link Cited by: §4, §4, §7, §7. Gemma Team (2025) Gemma Team Gemma 3 technical report. External Links: 2503.19786, Link Cited by: §7. Huang et al. (2023) Z. Huang, F. Bianchi, M. Yuksekgonul, T. J. Montine, and J. Zou A visual–language foundation model for pathology image analysis using medical twitter. Nature medicine 29 (9), p. 2307–2316. Cited by: §1. Ilse et al. (2018) M. Ilse, J. M. Tomczak, and M. Welling Attention-based deep multiple instance learning. In Proceedings of the 35th International Conference on Machine Learning (ICML), ICML (Ed.), Proceedings of Machine Learning Research, Vol. 80, p. 2132–2141. Cited by: §1. Kalra et al. (2020) S. Kalra, H. R. Tizhoosh, C. Choi, S. Shah, P. Diamandis, C. J. Campbell, and L. Pantanowitz Yottixel–an image search engine for large archives of histopathology whole slide images. Medical Image Analysis 65, p. 101757. Cited by: §2.1, §4. Kefeli and Tatonetti (2024) J. Kefeli and N. Tatonetti TCGA-reports: a machine-readable pathology report resource for benchmarking text-based ai models. Patterns 5 (3). Cited by: §4, §4, §4, 5th item. Kotp et al. (2026) Y. Kotp, V. Q. Trinh, C. Pal, and M. S. Hosseini MOOZY: a patient-first foundation model for computational pathology. arXiv preprint arXiv:2603.27048. Cited by: §1, §2.2, §4. Liu et al. (2022) Z. Liu, H. Mao, C. Wu, C. Feichtenhofer, T. Darrell, and S. Xie A ConvNet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE/CVPR (Ed.), p. 11976–11986. External Links: 2201.03545 Cited by: §4. Lu et al. (2024a) M. Y. Lu, B. Chen, D. F. K. Williamson, R. J. Chen, I. Liang, T. Ding, G. Jaume, I. Odintsov, L. P. Le, G. Gerber, A. V. Parwani, A. Zhang, and F. Mahmood A visual-language foundation model for computational pathology. Nature Medicine 30 (3), p. 863–874. External Links: Document Cited by: §1. Lu et al. (2024b) M. Y. Lu, B. Chen, D. F. K. Williamson, R. J. Chen, M. Zhao, A. K. Chow, K. Ikemura, A. Kim, D. Pouli, A. Patel, A. Soliman, C. Chen, T. Ding, J. J. Wang, G. Gerber, I. Liang, L. P. Le, A. V. Parwani, L. L. Weishaupt, and F. Mahmood A multimodal generative ai copilot for human pathology. Nature 634 (8033), p. 466–473. External Links: Document Cited by: §1. Lu et al. (2021) M. Y. Lu, D. F. K. Williamson, T. Y. Chen, R. J. Chen, M. Barbieri, and F. Mahmood Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering 5 (6), p. 555–570. External Links: Document Cited by: §1. Mulliqi et al. (2025) N. Mulliqi, A. Blilie, X. Ji, K. Szolnoky, H. Olsson, S. E. Boman, M. Titus, G. M. Gonzalez, J. A. Mielcarz, M. Valkonen, et al. Foundation models–a panacea for artificial intelligence in pathology?. arXiv preprint arXiv:2502.21264. Cited by: §1, §2.4. National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC) (2020) National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC) The clinical proteomic tumor analysis consortium breast invasive carcinoma collection (cptac-brca). External Links: Document, Link Cited by: §4, §4, Table 15, Table 21, Table 21. Nechaev et al. (2025) D. Nechaev, A. Pchelnikov, and E. Ivanova HISTAI: an open-source, large-scale whole slide image dataset for computational pathology. External Links: 2505.12120, Link Cited by: §4, §4, Table 14, Table 24, Table 24. Parker et al. (2009) J. S. Parker, M. Mullins, M. C. U. Cheang, S. Leung, D. Voduc, T. Vickery, S. Davies, C. Fauron, X. He, Z. Hu, J. F. Quackenbush, I. J. Stijleman, J. Palazzo, J. S. Marron, A. B. Nobel, E. Mardis, T. O. Nielsen, M. J. Ellis, C. M. Perou, and P. S. Bernard Supervised risk predictor of breast cancer based on intrinsic subtypes. Journal of Clinical Oncology 27 (8), p. 1160–1167. External Links: Document Cited by: §2.2, §4. Radford et al. (2021) A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, ICML (Ed.), Proceedings of Machine Learning Research, Vol. 139, p. 8748–8763. External Links: Link Cited by: §1. Sammut et al. (2022) S. Sammut, M. Crispin-Ortuzar, S. Chin, E. Provenzano, H. A. Bardwell, W. Ma, W. Cope, A. Dariush, S. Dawson, J. E. Abraham, J. Dunn, L. Hiller, J. Thomas, D. A. Cameron, J. M. S. Bartlett, L. Hayward, P. D. Pharoah, F. Markowetz, O. M. Rueda, H. M. Earl, and C. Caldas Multi-omic machine learning predictor of breast cancer therapy response. Nature 601 (7894), p. 623–629. External Links: Document Cited by: §2.1, §4, §4, Table 14, Table 17, Table 17. Shaikovski et al. (2024) G. Shaikovski, A. Casson, K. Severson, E. Zimmermann, Y. K. Wang, J. D. Kunz, J. A. Retamero, G. Oakley, D. Klimstra, C. Kanan, et al. PRISM: a multi-modal generative foundation model for slide-level histopathology. arXiv preprint arXiv:2405.10254. Cited by: §1, Figure 4, Figure 4, §2.3, §2.6, §4, §4, §4. Shaikovski et al. (2025) G. Shaikovski et al. PRISM2: unlocking multi-modal general pathology AI with clinical dialogue. arXiv preprint arXiv:2506.13063. Cited by: Figure 4, Figure 4, §2.6, §4. Shanbhogue et al. (2026) M. Shanbhogue, Z. Li, S. Zhang, G. H. Ábrego, S. Huang, A. Jain, D. Salz, S. Goenka, C. Hegde, J. Ma, et al. Gemini embedding 2: a native multimodal embedding model from gemini. arXiv preprint arXiv:2605.27295. Cited by: §4, §7. Shao et al. (2021) Z. Shao, H. Bian, Y. Chen, Y. Wang, J. Zhang, X. Ji, and Y. Zhang TransMIL: transformer based correlated multiple instance learning for whole slide image classification. In Advances in Neural Information Processing Systems (NeurIPS), NeurIPS (Ed.), Vol. 34, p. 2136–2147. Cited by: §1. Sun et al. (2024) Y. Sun, Y. Zhang, Y. Si, C. Zhu, Z. Shui, K. Zhang, J. Li, X. Lyu, T. Lin, and L. Yang PathGen-1.6M: 1.6 million pathology image-text pairs generation through multi-agent collaboration. arXiv preprint arXiv:2407.00203. External Links: Link Cited by: §1. Tizhoosh (2026) H. R. Tizhoosh Rethinking foundation models in pathology. Nature Biomedical Engineering, p. 1–6. External Links: Document Cited by: §1, §2.4. Uegami et al. (2026) W. Uegami, R. Kendziora, S. Al-Fasly, L. Pantanowitz, J. Garcia, and H. Tizhoosh Disease subtyping for computational pathology benchmarking with tcga dataset —white paper. Zenodo. External Links: Document, Link Cited by: §4, §4, Table 14, Table 14. Vera et al. (2025) H. S. Vera, S. Dua, B. Zhang, D. Salz, R. Mullins, S. R. Panyam, S. Smoot, I. Naim, J. Zou, F. Chen, et al. Embeddinggemma: powerful and lightweight text representations. arXiv preprint arXiv:2509.20354. Cited by: §7. Vorontsov et al. (2024) E. Vorontsov, A. Bozkurt, A. Casson, G. Shaikovski, M. Zelechowski, K. Severson, E. Zimmermann, J. Hall, N. Tenenholtz, N. Fusi, E. Yang, P. Mathieu, A. van Eck, D. Lee, J. Viret, E. Robert, Y. K. Wang, J. D. Kunz, M. C. H. Lee, J. H. Bernhard, R. A. Godrich, G. Oakley, E. Millar, M. Hanna, H. Wen, J. A. Retamero, W. A. Moye, R. Yousfi, C. Kanan, D. S. Klimstra, B. Rothrock, S. Liu, and T. J. Fuchs A foundation model for clinical-grade computational pathology and rare cancers detection. Nature Medicine 30 (10), p. 2924–2935. External Links: Document Cited by: §2.6. Wang et al. (2022) X. Wang, S. Yang, J. Zhang, M. Wang, J. Zhang, W. Yang, J. Huang, and X. Han Transformer-based unsupervised contrastive learning for histopathological image classification. Medical Image Analysis 81, p. 102559. External Links: Document Cited by: §1. Weinstein et al. (2013) J. N. Weinstein, E. A. Collisson, G. B. Mills, K. R. Shaw, B. A. Ozenberger, K. Ellrott, I. Shmulevich, C. Sander, and J. M. Stuart The cancer genome atlas pan-cancer analysis project. Nature genetics 45 (10), p. 1113–1120. Cited by: §4, Table 14, Table 14, Table 14, Table 25, Table 25. Xu et al. (2024) H. Xu, N. Usuyama, J. Bagga, S. Zhang, R. Rao, T. Naumann, C. Wong, Z. Gero, J. González, Y. Gu, Y. Xu, M. Wei, W. Wang, S. Ma, F. Wei, J. Yang, C. Li, J. Gao, J. Rosemon, T. Bower, S. Lee, R. Weerasinghe, B. J. Wright, A. Robicsek, B. Piening, C. Bifulco, S. Wang, and H. Poon A whole-slide foundation model for digital pathology from real-world data. Nature 630 (8015), p. 181–188. External Links: Document Cited by: §1, §2.3, §4. Yu et al. (2022) J. Yu, Z. Wang, V. Vasudevan, L. Yeung, M. Seyedhosseini, and Y. Wu CoCa: contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917. Cited by: Figure 4, Figure 4, §4. Zhai et al. (2023) X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), IEEE/CVF (Ed.), p. 11975–11986. External Links: Document Cited by: §4. Zimmermann et al. (2024) E. Zimmermann, E. Vorontsov, J. Viret, A. Casson, M. Zelechowski, G. Shaikovski, N. Tenenholtz, J. Hall, D. Klimstra, R. Yousfi, T. Fuchs, N. Fusi, S. Liu, and K. Severson Virchow2: scaling self-supervised mixed magnification models in pathology. arXiv preprint arXiv:2408.00738. Note: Microsoft Research and Paige collaboration Cited by: §2.6. 5 Additional Results Figure 7: Case-level image retrieval performance of the vision encoders. Retrieval uses each model’s vision encoder only (no text encoder at inference). All whole-slide images of a case were represented in a single case-level composite before getting its corresponding embedding, and each query case was matched against the remaining cases by cosine similarity; the label of the retrieved neighbour(s) was transferred to the query. Columns give the label-transfer rule (Top-1, or majority vote over the k=3,5,7k=3,5,7 nearest neighbours) and rows the resulting metric (accuracy, balanced accuracy, macro-averaged F1-score). Axes are the four evaluation cohorts; the radial scale spans 00–11 with gridlines every 0.20.2. All three WILSON variants — which differ only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive) — exceed MOOZY on every cohort, metric and k, with the largest margins on balanced accuracy and macro-averaged F1-score. Chance level differs per axis because class counts and prevalences differ between cohorts. Figure 8: Case-level image retrieval accuracy by cohort, macro-averaged F1-score. Per-cohort breakdown of the case-level retrieval results summarized in Fig. 7, using the vision encoder of each model only. Panels correspond to the label-transfer rule (majority vote over the k=1,3,5,7k=1,3,5,7 nearest neighbours); groups within each panel are the four evaluation cohorts (CPTAC-BRCA, Mayo Breast Subtyping, Mayo Breast Diagnosis, Mayo Skin). Bars show macro-averaged F1-score for MOOZY and three WILSON variants differing only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive); error bars denote [s.d. across n folds / seeds][s.d.\ across n folds / seeds] and horizontal dashed lines mark each model’s mean across the four cohorts. Note the truncated vertical axis. WILSON-distill achieves the highest cohort mean at every k and the largest single-cohort margin on Mayo Skin (0.62 vs. 0.28 for MOOZY at k=1k=1; 0.66 vs. 0.27 at k=7k=7). Gains are smaller on the breast cohorts, where MOOZY is comparable to WILSON-CLIP on Mayo Breast Subtyping and exceeds it on CPTAC-BRCA at every k, indicating that the advantage is cohort-dependent rather than uniform. Figure 9: Complete case-level image retrieval results. Numerical values underlying Figs. 7 and 8, using the vision encoder of each model only. Panels report accuracy (left), balanced accuracy (centre) and macro-averaged F1-score (right) on a common color scale; columns are the four evaluation cohorts and rows every combination of model (MOOZY, WILSON-distill, WILSON-KW, WILSON-CLIP) and label-transfer rule (majority vote over the k=1,3,5,7k=1,3,5,7 nearest neighbours), grouped by k. Three patterns are visible. First, the WILSON encoders’ advantage is largest on Mayo Skin, where all three exceed MOOZY by roughly 0.3 on every metric and at every k (e.g. 0.67 vs. 0.29 accuracy at k=7k=7). Second, on CPTAC-BRCA the accuracy of all models (0.42–0.58) substantially exceeds their balanced accuracy and macro-averaged F1-score (0.29–0.40), indicating that raw accuracy there is inflated by class imbalance; MOOZY shows the widest gap (0.57 vs. 0.35 at k=3k=3), consistent with a bias toward the majority class. Third, performance generally improves with increasing k for all models, with most of the gain realised by k=5k=5. Figure 10: Slide-level image retrieval performance of the vision encoders. Retrieval uses each model’s vision encoder only (no text encoder at inference). Each whole-slide image was encoded into a single slide embedding and matched against the remaining slides by cosine similarity; the label of the retrieved neighbour(s) was transferred to the query. Columns give the label-transfer rule (Top-1, or majority vote over the k=3,5,7k=3,5,7 nearest neighbours) and rows the resulting metric (accuracy, balanced accuracy, macro-averaged F1-score). Axes are the four evaluation cohorts (BCTherapy, TCGA-BRCA, Mayo Skin, Mayo Breast Diagnosis); the radial scale spans 00–11 with gridlines every 0.20.2. Six slide encoders are compared: GigaPath, TITAN, PRISM, and three WILSON variants differing only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive). The WILSON variants match or exceed the published foundation models across cohorts, metrics and values of k, with the clearest separation on Mayo Skin and Mayo Breast Diagnosis; GigaPath trails on every axis. Chance level differs per axis because class counts and prevalences differ between cohorts. Figure 11: Slide-level image retrieval accuracy by cohort, macro-averaged F1-score. Per-cohort breakdown of the slide-level retrieval results summarized in Fig. 10, using the vision encoder of each model only. Panels correspond to the label-transfer rule (majority vote over the k=1,3,5,7k=1,3,5,7 nearest neighbours); groups within each panel are the four evaluation cohorts (TCGA-BRCA, BCTherapy, Mayo Breast Diagnosis, Mayo Skin). Within each group, bars are ordered GigaPath, TITAN, PRISM, WILSON-distill, WILSON-KW, WILSON-CLIP, the last three differing only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive); error bars denote [s.d. across n folds / seeds][s.d.\ across n folds / seeds] and horizontal dashed lines mark each model’s mean across the four cohorts. Note the truncated vertical axis. The WILSON encoders lead on Mayo Skin at every k (up to 0.69 for WILSON-distill at k=7k=7 vs. 0.63 for PRISM and 0.57 for TITAN) and are comparable to TITAN and PRISM on BCTherapy and Mayo Breast Diagnosis, while TITAN and PRISM retain a small advantage on TCGA-BRCA at k=1k=1. GigaPath performs worst in every cohort and at every k, most markedly on Mayo Breast Diagnosis and Mayo Skin. Figure 12: Complete slide-level image retrieval results. Numerical values underlying Figs. 10 and 11, using the vision encoder of each model only. Panels report accuracy (left), balanced accuracy (centre) and macro-averaged F1-score (right); note that each panel has its own color scale, so shading is comparable within but not across panels. Columns are the four evaluation cohorts and rows every combination of model (GigaPath, TITAN, PRISM, WILSON-distill, WILSON-KW, WILSON-CLIP) and label-transfer rule (majority vote over the k=1,3,5,7k=1,3,5,7 nearest neighbours), grouped by k. Three patterns are visible. First, no single encoder leads everywhere: WILSON-distill is strongest on Mayo Skin at every k (0.69 accuracy and 0.69 macro F1 at k=7k=7, vs. 0.64 and 0.63 for PRISM), WILSON-CLIP on BCTherapy (0.77 accuracy at k=5k=5), PRISM on Mayo Breast Diagnosis (0.72 accuracy at k=7k=7) and TITAN on TCGA-BRCA (0.88 accuracy at k=7k=7). Second, the gap between accuracy and balanced accuracy is pronounced on TCGA-BRCA for all six encoders (e.g. 0.88 vs. 0.43 for TITAN at k=7k=7), indicating that accuracy in that cohort is dominated by class imbalance and that the balanced-accuracy and macro-F1 panels are the informative ones. Third, GigaPath is the weakest encoder throughout, and its deficit is largest on Mayo Breast Diagnosis and Mayo Skin (balanced accuracy 0.25–0.34) while remaining competitive on TCGA-BRCA accuracy (0.73–0.79), a pattern consistent with majority-class prediction rather than discriminative retrieval. Figure 13: Comprehensive Performance Comparison of WILSON and PRISM Models on Cross-Modal Retrieval Tasks. The figure presents recall metrics (R@1, R@3, R@5, R@10) evaluated across Mayo Clinic (Diagnoses, Duodenum, Pancreas, Stomach) and TCGA (Brain, Kidney) datasets. Top section (Bar charts): Quantitative comparison showing exact performance values for each metric and dataset. Bottom section (Radar charts): Multi-dimensional performance visualization enabling visual assessment of relative strengths across all metrics and datasets simultaneously. Each row represents a cross-modal retrieval direction: Image-to-Text (I2T) retrieves relevant diagnostic text reports given medical images; Text-to-Image (T2I) retrieves relevant medical images given text descriptions. Blue elements (bars and lines) represent WILSON model performance, purple elements represent PRISM model performance. All metrics are reported as percentages, with bar chart values displayed above each bar for precise quantitative reference. Figure 14: Additional qualitative comparison of generated slide-level reports. Seven further cases (colon, uterus, urinary bladder, liver, stomach, prostate, breast), showing the whole-slide image and its clinical report, the composite input, and the descriptions generated by WILSON, PRISM and PRISM2. Conventions as in Fig. 4. Figure 15: Caption-quality evaluation across cohorts, models, and metrics. Grouped bar charts of eight caption-quality metrics (columns) for PRISM, PRISM2 and WILSON (ours) on three cohorts (rows): held-out Mayo Clinic, HistAI and TCGA. Bars show the mean over caption pairs and error bars one standard deviation; annotations give the mean value. Metrics are BLEU-4 (n-gram precision), CIDEr (consensus-based TF–IDF-weighted n-gram similarity), ROUGE-1 (unigram overlap), ROUGE-L (longest common subsequence), BERTScore precision/recall/F1 (contextual similarity from BERT embeddings) and CLIPScore (image–text similarity in a contrastive embedding space). CLIPScore is computed within each model’s own embedding space and is therefore not comparable across models; PRISM2 exposes no general-purpose text encoder for arbitrary strings, so no PRISM2 bar is shown. The rightmost panel (“Avg”) summarizes each model as the mean of its metric values after min–max normalization within metric and cohort, rescaled to 0–100 [excluding CLIPScore]. Note that y-axis ranges differ between panels: bar heights are comparable within a panel but not across panels. Table 1: Composite images vs. Yottixel patches as WSI representations, both encoded with the pretrained UNI foundation model. Values are top-1 macro-F1 retrieval performance (higher is better) across two public and two internal Mayo Clinic cohorts. Dataset Task Yottixel + UNI Composite + UNI BCTherapy Treatment response 0.51 0.56 MayoSkin Diagnostic classification 0.33 0.34 MayoBreast Diagnostic classification 0.31 0.35 TCGA-Kidney Diagnostic classification 0.89 0.88 Table 2: End-to-end fine-tuning of WILSON (dense embedding) on MayoTNBC dataset.Accuracy (acc) and macro-averaged F1 (Macro avg) are reported for histologic subtyping and sTILs grading under three settings: the pretrained backbone evaluated zero-shot, the same backbone after end-to-end fine-tuning evaluated zero-shot, and the fine-tuned backbone with a jointly trained classification head. Base zero-shot Fine-tuned zero-shot Fine-tuned classification Task acc Macro avg acc Macro avg acc Macro avg Histology 0.617 0.401 0.637 0.428 0.690 0.557 sTILs 0.412 0.425 0.432 0.456 0.541 0.531 Table 3: Cross-modal retrieval on the MayoCaption dataset, horizontal evaluation across all diagnoses combined. Δ denotes WILSON minus PRISM. Direction Model R@1 (%) R@3 (%) R@5 (%) R@10 (%) Image-to-Text WILSON 78.29 85.75 88.82 92.11 PRISM 60.09 66.45 69.74 75.66 Δ (W−-P) +18.20 +19.30 +19.08 +16.45 Text-to-Image WILSON 81.14 94.08 96.49 98.03 PRISM 56.36 77.85 82.89 87.72 Δ (W−-P) +24.78 +16.23 +13.60 +10.31 WILSON: 456 samples; PRISM: 456 samples, 16 diagnosis labels. Table 4: Cross-modal retrieval on the MayoCaption dataset, per-organ vertical evaluation for Duodenum. Δ denotes WILSON minus PRISM. Direction Model R@1 (%) R@3 (%) R@5 (%) R@10 (%) Image-to-Text WILSON 77.78 83.33 86.36 89.90 PRISM 64.14 65.15 67.17 68.69 Δ (W−-P) +13.64 +18.18 +19.19 +21.21 Text-to-Image WILSON 80.81 94.95 97.98 98.48 PRISM 55.56 67.68 74.24 78.79 Δ (W−-P) +25.25 +27.27 +23.74 +19.70 WILSON: 198 samples; PRISM: 198 samples, 5 diagnosis labels. Table 5: Cross-modal retrieval on the MayoCaption dataset, per-organ vertical evaluation for Pancreas. Δ denotes WILSON minus PRISM. Direction Model R@1 (%) R@3 (%) R@5 (%) R@10 (%) Image-to-Text WILSON 78.18 89.09 90.91 93.64 PRISM 59.09 63.64 69.09 80.00 Δ (W−-P) +19.09 +25.45 +21.82 +13.64 Text-to-Image WILSON 83.64 92.73 95.45 99.09 PRISM 70.91 95.45 98.18 98.18 Δ (W−-P) +12.73 -2.73 -2.73 +0.91 WILSON: 110 samples; PRISM: 110 samples, 5 diagnosis labels. Table 6: Cross-modal retrieval on the MayoCaption dataset, per-organ vertical evaluation for Stomach. Δ denotes WILSON minus PRISM. Direction Model R@1 (%) R@3 (%) R@5 (%) R@10 (%) Image-to-Text WILSON 85.81 91.89 93.92 96.62 PRISM 81.76 87.84 90.54 95.95 Δ (W−-P) +4.05 +4.05 +3.38 +0.68 Text-to-Image WILSON 85.14 95.95 96.62 97.97 PRISM 65.54 92.57 95.27 96.62 Δ (W−-P) +19.59 +3.38 +1.35 +1.35 WILSON: 148 samples; PRISM: 148 samples, 6 diagnosis labels. Table 7: Cross-modal retrieval on the TCGA Brain dataset. Δ denotes WILSON minus PRISM. Direction Model R@1 (%) R@3 (%) R@5 (%) R@10 (%) Image-to-Text WILSON 41.94 58.87 65.32 78.23 PRISM 20.16 40.32 55.65 81.45 Δ (W−-P) +21.77 +18.55 +9.68 -3.23 Text-to-Image WILSON 54.03 70.97 82.26 88.71 PRISM 41.13 62.10 71.77 87.10 Δ (W−-P) +12.90 +8.87 +10.48 +1.61 WILSON: 124 WSIs; PRISM: 124 WSIs. Disease groups: Anaplastic astrocytoma, Anaplastic oligodendroglioma, Astrocytoma, Glioblastoma, Mixed glioma, Oligodendroglioma. Table 8: Cross-modal retrieval on the TCGA Kidney dataset. Δ denotes WILSON minus PRISM. Direction Model R@1 (%) R@3 (%) R@5 (%) R@10 (%) Image-to-Text WILSON 82.56 86.05 93.02 96.51 PRISM 63.22 93.10 96.55 100.00 Δ (W−-P) +19.34 -7.06 -3.53 -3.49 Text-to-Image WILSON 77.91 90.70 94.19 100.00 PRISM 54.02 59.77 67.82 78.16 Δ (W−-P) +23.88 +30.93 +26.37 +21.84 WILSON: 86 WSIs; PRISM: 87 WSIs. Disease groups: Chromophobe renal cell carcinoma, Clear cell renal cell carcinoma, Papillary renal cell carcinoma. Table 9: Summary of R@1 retrieval performance for WILSON and PRISM across all evaluated datasets, for both image-to-text (I2T) and text-to-image (T2I) retrieval directions. Image-to-Text R@1 (%) Text-to-Image R@1 (%) Dataset WILSON PRISM WILSON PRISM MayoCaption (all) 78.29 60.09 8 1.14 56.36 Mayo – Duodenum 77.78 64.14 80.81 55.56 Mayo – Pancreas 78.18 59.09 83.64 70.91 Mayo – Stomach 85.81 81.76 85.14 65.54 TCGA – Brain 41.94 20.16 54.03 41.13 TCGA – Kidney 82.56 63.22 77.91 54.02 Table 10: Comprehensive comparison of WILSON and PRISM cross-modal retrieval recall (R@1, R@3, R@5, R@10) across all used Mayo Clinic and TCGA dataset/organ splits, for both image-to-text (I2T) and text-to-image (T2I) retrieval. WILSON (%) PRISM (%) Dataset Direction R@1 R@3 R@5 R@10 R@1 R@3 R@5 R@10 MayoCaption I2T 78.29 85.75 88.82 92.11 60.09 66.45 69.74 75.66 T2I 81.14 94.08 96.49 98.03 56.36 77.85 82.89 87.72 Mayo – Duodenum I2T 77.78 83.33 86.36 89.90 64.14 65.15 67.17 68.69 T2I 80.81 94.95 97.98 98.48 55.56 67.68 74.24 78.79 Mayo – Pancreas I2T 78.18 89.09 90.91 93.64 59.09 63.64 69.09 80.00 T2I 83.64 92.73 95.45 99.09 70.91 95.45 98.18 98.18 Mayo – Stomach I2T 85.81 91.89 93.92 96.62 81.76 87.84 90.54 95.95 T2I 85.14 95.95 96.62 97.97 65.54 92.57 95.27 96.62 TCGA – Brain I2T 41.94 58.87 65.32 78.23 20.16 40.32 55.65 81.45 T2I 54.03 70.97 82.26 88.71 41.13 62.10 71.77 87.10 TCGA – Kidney I2T 82.56 86.05 93.02 96.51 63.22 93.10 96.55 100.00 T2I 77.91 90.70 94.19 100.00 54.02 59.77 67.82 78.16 Table 11: WILSON vs. PRISM at R@1 and R@10, all cohorts and directions. Δ = WILSON −- PRISM; bold negative values indicate PRISM leads. R@1 R@10 Dataset / Direction PRISM WILSON Δ PRISM WILSON Δ MayoCaption I2T 60.1 78.3 +18.2 75.7 92.1 +16.4 MayoCaption T2I 56.4 81.1 +24.7 87.7 98.0 +10.3 Mayo Duodenum I2T 64.1 77.8 +13.7 68.7 89.9 +21.2 Mayo Duodenum T2I 55.6 80.8 +25.2 78.8 98.5 +19.7 Mayo Pancreas I2T 59.1 78.2 +19.1 80.0 93.6 +13.6 Mayo Pancreas T2I 70.9 83.6 +12.7 98.2 99.1 +0.9 Mayo Stomach I2T 81.8 85.8 +4.0 95.9 96.6 +0.7 Mayo Stomach T2I 65.5 85.1 +19.6 96.6 98.0 +1.4 TCGA Brain I2T 20.2 41.9 +21.7 81.5 78.2 -3.3 TCGA Brain T2I 41.1 54.0 +12.9 87.1 88.7 +1.6 TCGA Kidney I2T 63.2 82.6 +19.4 100.0 96.5 -3.5 TCGA Kidney T2I 54.0 77.9 +23.9 78.2 100.0 +21.8 Mean 58.1 75.6 +17.9 85.7 94.1 +8.4 Table 12: Parameter counts and computational cost of pathology foundation models at a common workload of 10,000 tiles per whole-slide image (WSI). For tile-encoder plus slide-aggregator pipelines, cost per WSI is the tile encoder evaluated N=10,000N=10,000 times plus one pass of the slide aggregator. Counts cover the embedding path only; text decoders (PRISM’s BioGPT, PRISM2’s Phi-3 Mini, TITAN’s text tower) are excluded. A dash indicates that no per-tile cost is defined: WILSON encodes one 2048×20482048× 2048 composite of 8×8=648× 8=64 selected tissue regions in a single forward pass, and the MOOZY figure is reported for its frozen patch encoder together with its slide encoder. Prov-GigaPath’s slide encoder uses LongNet dilated attention, so its actual slide-encoder cost is lower than the dense-attention figure shown. Rows are ordered by cost per WSI. Model Architecture Total params FLOPs per tile FLOPs per WSI Prov-GigaPath ViT-G/14 (DINOv2) + LongNet 1,220M 555.4G 5.56P PRISM2 Virchow2 (ViT-H/14) + PerceiverAR 1,251M 413.2G 4.13P PRISM Virchow (ViT-H/14) + Perceiver 730.3M 406.7G 4.07P TITAN CONCH v1.5 (ViT-L/16) + slide encoder 354.6M 70.0G 703.19T MOOZY ViT-S/8 (Lunit DINO) + ViT 6L, 768-d 85.8M — 25.07T WILSON ConvNeXt-Base + extended head (single pass) 133.7M — 2.58T Table 13: Caption-quality evaluation across three models and three cohorts. BLEU-4 (n-gram overlap), CIDEr (consensus-based), ROUGE-1 (unigram overlap), ROUGE-L (longest common subsequence), BERTScore-P/R/F1 (contextual similarity from BERT embeddings) and CLIPScore (CLIP-based semantic similarity) are reported as raw scores. “Avg” is the mean performance across metrics after min–max normalization within each metric, rescaled to 0–100. Best model per cohort is bold. aCLIPScore is computed in each model’s own contrastive embedding space and is therefore not comparable across models; PRISM2 exposes no general-purpose text encoder for arbitrary strings. Excluded from bolding and from Avg. Mayo Clinic HistAI TCGA Metric PRISM PRISM2 WILSON PRISM PRISM2 WILSON PRISM PRISM2 WILSON BLEU-4 0.0004 0.0034 0.0155 0.0000 0.0004 0.0017 0.0019 0.0023 0.0085 CIDEr 0.0022 0.0550 0.2236 0.0944 0.0199 0.0236 0.0204 0.0272 0.0903 ROUGE-1 0.1424 0.2307 0.3711 0.1196 0.0948 0.1617 0.1825 0.2148 0.2136 ROUGE-L 0.1189 0.1670 0.2556 0.1023 0.0801 0.1262 0.1497 0.1596 0.1592 BERTScore-P 0.8812 0.8946 0.8627 0.8519 0.8290 0.8383 0.8808 0.8740 0.8609 BERTScore-R 0.8421 0.8560 0.8781 0.8450 0.8434 0.8579 0.8252 0.8461 0.8386 BERTScore-F1 0.8611 0.8745 0.8702 0.8480 0.8356 0.8476 0.8520 0.8592 0.8495 CLIPScorea 0.6077 N/A 0.8570 0.5232 N/A 0.8111 0.6112 N/A 0.8638 Avg 30.5 57.5 92.3 21.0 6.5 32.9 29.9 40.6 49.1 6 Supplementary Data Table 14: Summary of training and validation WSI datasets. The MayoTNBC dataset contains two tasks, histological subtype classification and stromal tumor-infiltrating lymphocyte (sTIL) prediction, each comprising four classes. The MayoCaption dataset was constructed by combining the MayoDuodenum, MayoStomach, and MayoPancreas datasets for horizontal cross-modal retrieval. Histological subtype annotations for the TCGA-derived datasets were assigned according to Uegami et al. Uegami et al. (2026). Dataset Organ Task Classes Patients Slides Format Mayo189K Various WILSON Training N/A 112,296 189,291 DICOM MayoBreast Breast (tumor, nontumor) Histological subtype classification 12 1,289 1,339 DICOM MayoSkin Skin (tumor, nontumor) Histological subtype classification 20 677 688 DICOM MayoTNBC Breast carcinoma (triple negative) Fine tuning 4/4 508 508 Philips Tiff MayoCaption Three organs Cross-modal retrieval 18 397 456 DICOM MayoDuodenum Duodenum Cross-modal retrieval 7 172 198 DICOM MayoStomach Stomach Cross-modal retrieval 6 127 148 DICOM MayoPancreas Pancreas Cross-modal retrieval 5 106 110 DICOM BCTherapy Sammut et al. (2022) Breast carcinoma Treatment response prediction 2 160 160 SVS TCGA-BRCA Weinstein et al. (2013) Breast carcinoma Histologic subtype classification 7 1,018 1,087 SVS TCGA-Brain Weinstein et al. (2013) Brain tumor Cross-modal retrieval 6 879 1,703 SVS TCGA-Kidney Weinstein et al. (2013) Kidney Cross-modal retrieval 3 885 927 SVS HistAI Nechaev et al. (2025) 10 organs Cross-modal retrieval N/A 48 48 generic Tiff Table 15: Summary of case-level validation WSI datasets. Dataset Organ Task Classes Cases WSIs Format MayoCaseBreast Breast (tumor, nontumor) Histological subtype classification 12 494 4,361 DICOM MayoCaseSkin Skin (tumor, nontumor) Histological subtype classification 20 500 1,280 DICOM MayoBreastSubtype Breast carcinoma Molecular subtype classification 3 846 2,254 DICOM CPTAC-BRCA National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC) (2020) Breast carcinoma PAM50 subtype classification 5 198 653 SVS Table 16: Class distribution of the TCGA-BRCA dataset. Cancer Genome Atlas Research Network et al. (2013) Class Count Invasive carcinoma of no special type 812 Invasive lobular carcinoma 204 Ductal and lobular carcinoma 27 Mucinous adenocarcinoma 20 Metaplastic carcinoma 13 Medullary carcinoma 7 Invasive micropapillary carcinoma 4 Total 1,087 Table 17: Class distribution of the BCTherapy dataset Sammut et al. (2022). Class Count RD 119 pCR 41 Table 18: Class distribution of the MayoBRCA dataset. Class Count Invasive breast carcinoma of no special type (ductal) 350 Invasive lobular carcinoma 70 Mixed ductal and lobular carcinoma 70 Mucinous carcinoma 70 Metaplastic carcinoma 70 Ductal carcinoma in situ 210 Lobular carcinoma in situ 70 Fibroadenoma 70 Phyllodes tumor 79 Lymphoma or lymphoproliferative lesion 70 Benign proliferative lesion 140 Fat necrosis 70 Table 19: Class distribution of the MayoSkin dataset. Class Count Lentigo 41 Actinic keratosis 40 Malignant melanoma 40 Blue nevus 39 Dermatofibroma 38 Verrucae 38 Epidermal cyst 38 Basal cell carcinoma 37 Neurofibroma 37 Psoriasiform dermatitis 36 Nevus 36 Nevus with atypia 36 Chondrodermatitis nodularis helicis 35 Squamous cell carcinoma 35 Seborrheic keratosis 33 Sebaceous gland hyperplasia 33 Scar 32 Skin tag 31 Lichenoid dermatitis 29 Lymphoma 20 Table 20: Class distribution of the MayoBreastSubtype dataset. Class Count Hormone receptor-positive/HER2-negative 269 HER2-positive 303 Triple-negative 274 Table 21: Class distribution of the CPTAC-BRCA dataset National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC) (2020). Class Count Luminal A 57 Basal-like 27 Luminal B 17 HER2-enriched 14 Normal-like 5 Table 22: Class distribution of the MayoCaseBreast dataset. Class Count Invasive carcinoma of no special type (ductal) 44 Invasive lobular carcinoma 42 Mixed ductal and lobular carcinoma 40 Mucinous carcinoma 41 Metaplastic carcinoma 38 Ductal carcinoma in situ 40 Lobular carcinoma in situ 45 Fibroadenoma 37 Phyllodes tumor 43 Lymphoma or lymphoproliferative disease 42 Benign proliferative lesion 40 Fat necrosis 48 Table 23: Class distribution of the MayoCaseSkin dataset. Class Count Sebaceous gland hyperplasia 31 Dermatofibroma 31 Malignant melanoma 31 Skin tag 29 Nevus with atypia 28 Seborrheic keratosis 27 Blue nevus 27 Scar 26 Squamous cell carcinoma 26 Lentigo 25 Basal cell carcinoma 25 Lichenoid dermatitis 24 Actinic keratosis 24 Chondrodermatitis nodularis helicis 23 Epidermal cyst 23 Neurofibroma 23 Nevus 22 Psoriasiform dermatitis 21 Verruca 20 Cutaneous lymphoma 14 Table 24: HistAI Nechaev et al. (2025) whole-slide images (WSIs) and corresponding organ categories. WSI ID Organ case_00760/slide_H&E_0.tiff Bone case_03070/slide_H&E_0.tiff Bone case_09511/slide_H&E_0.tiff Bone case_11368/slide_H&E_0.tiff Bone case_11593/slide_H&E_0.tiff Bone case_01871/slide_H&E_0.tiff Breast case_04775/slide_H&E_0.tiff Breast case_04858/slide_H&E_0.tiff Breast case_12828/slide_H&E_0.tiff Breast case_00107/slide_H&E_0.tiff Endocrine case_03889/slide_H&E_0.tiff Endocrine case_04347/slide_H&E_0.tiff Endocrine case_13008/slide_H&E_0.tiff Endocrine case_14497/slide_H&E_0.tiff Endocrine case_03357/slide_H&E_0.tiff Gastrointestinal case_04754/slide_H&E_0.tiff Gastrointestinal case_13674/slide_H&E_0.tiff Gastrointestinal case_14249/slide_H&E_0.tiff Gastrointestinal case_16527/slide_H&E_0.tiff Gastrointestinal case_05879/slide_H&E_0.tiff Genitourinary case_07247/slide_H&E_0.tiff Genitourinary case_10189/slide_H&E_0.tiff Genitourinary case_14971/slide_H&E_0.tiff Genitourinary case_01259/slide_H&E_0.tiff Gynecologic case_03934/slide_H&E_0.tiff Gynecologic case_05088/slide_H&E_0.tiff Gynecologic case_12091/slide_H&E_0.tiff Gynecologic case_15397/slide_H&E_0.tiff Gynecologic case_01585/slide_H&E_0.tiff Head and Neck case_02038/slide_H&E_0.tiff Head and Neck case_08881/slide_H&E_0.tiff Head and Neck case_13539/slide_H&E_0.tiff Head and Neck case_16658/slide_H&E_0.tiff Head and Neck case_01705/slide_H&E_0.tiff Skin case_12553/slide_H&E_0.tiff Skin case_17219/slide_H&E_0.tiff Skin case_17492/slide_H&E_0.tiff Skin case_17619/slide_H&E_0.tiff Skin case_00315/slide_H&E_0.tiff Soft tissue case_05077/slide_H&E_0.tiff Soft tissue case_06560/slide_H&E_1.tiff Soft tissue case_08261/slide_H&E_0.tiff Soft tissue case_11651/slide_H&E_0.tiff Soft tissue case_03457/slide_H&E_0.tiff Thorax case_06572/slide_H&E_0.tiff Thorax case_12689/slide_H&E_0.tiff Thorax case_15640/slide_H&E_0.tiff Thorax case_16730/slide_H&E_0.tiff Thorax Table 25: TCGA Weinstein et al. (2013) whole-slide images (WSIs) and corresponding organ categories. WSI ID Organ TCGA-06-0143-01Z-00-DX3.e9011249-11f6-454b-98f1-7f2bcfea228c.svs Brain TCGA-06-0154-01Z-00-DX1.e52d5443-e663-451f-8a1f-83ec5b9a401.svs Brain TCGA-DB-A75P-01Z-00-DX1.11655479-5A76-4689-B8F6-319F882B5EB7.svs Brain TCGA-DU-A6S3-01Z-00-DX1.08CE6930-D4E8-4084-A8C0-F2911B069E72.svs Brain TCGA-HT-7680-01Z-00-DX3.C157242E-F4D2-4246-8016-B0A44C4C4AAD.svs Brain TCGA-AC-A2QH-01Z-00-DX1.00B8BFFF-F1E2-4F99-A969-8D7E4F8E0B.svs Breast TCGA-AR-A24K-01Z-00-DX1.3A56BAEC-484E-4F9B-BCB4-360ABF6DDB4B.svs Breast TCGA-BH-A1F2-01Z-00-DX1.17E2FD6F-0DCF-425B-864B-21ADCDAE734B.svs Breast TCGA-BH-A1F8-01Z-00-DX1.8B026F7-35CB-483F-B665-4C3A3EF47E1B.svs Breast TCGA-MS-A51U-01Z-00-DX1.490DE85A-ECE5-4E2A-9657-841BE6FFCCA0.svs Breast TCGA-DS-A1OC-01Z-00-DX1.D3ECE6F5-90DC-4CF4-A203-AC9F28E89959.svs Cervix TCGA-IR-A3LC-01Z-00-DX1.D5E87530-B079-4374-9DFE-4100CE6A4CDA.svs Cervix TCGA-MY-A5BF-01Z-00-DX3.CF669DE8-217E-47A6-839F-ABE9657BA566.svs Cervix TCGA-VS-A8QF-01Z-00-DX1.D8DB41E0-8C0F-4394-A2A2-8721BE35644B.svs Cervix TCGA-ZJ-AB0H-01Z-00-DX1.1F0584E3-1E50-4755-BF3D-EB62502CB975.svs Cervix TCGA-A6-5665-01Z-00-DX1.3ad2c249-d138-4037-a59b-4747ce2b789a.svs Colorectum TCGA-A-3538-01Z-00-DX1.60d0b039-25d6-4b71-a36f-5b2764a983ef.svs Colorectum TCGA-A-A02E-01Z-00-DX1.04D47621-9DCF-437C-A4D6-44D17579FE6D.svs Colorectum TCGA-CK-6748-01Z-00-DX1.1d76660-7858-470c-a27b-36586b788125.svs Colorectum TCGA-G4-6628-01Z-00-DX1.d67973d1-9544-47e1-9ecb-e9d8d7f310e6.svs Colorectum TCGA-B9-4113-01Z-00-DX1.91e4a568-8f5b-4726-8352-94adf056f912.svs Kidney TCGA-B9-4617-01Z-00-DX1.d890e850-c29-49f8-b854-cbe9a9a73c95.svs Kidney TCGA-CZ-4861-01Z-00-DX1.c36faa05-5e3b-4711-897e-c91f84405870.svs Kidney TCGA-CZ-5468-01Z-00-DX1.e2bbe417-a24c-4511-934e-674221855695.svs Kidney TCGA-EU-5905-01Z-00-DX1.f83b5cb0-7295-4f67-acac-c1768b07c60.svs Kidney TCGA-33-4582-01Z-00-DX1.629AEDB6-E9A-4615-92E8-5DDAAFF6103E.svs Lung TCGA-44-6144-01Z-00-DX1.604b3c7c-92e8-474a-bae8-e48415ea6196.svs Lung TCGA-49-6744-01Z-00-DX4.a3d7995d-399f-4c53-aab8-adc4ea4dbfa8.svs Lung TCGA-66-2785-01Z-00-DX1.b9439e1-d22b-4ccd-b53b-ce7717a37a17.svs Lung TCGA-XC-A0X-01Z-00-DX1.61A34BE0-F16B-4EC1-8E7F-7BF94F6629F4.svs Lung TCGA-DX-A3LS-01Z-00-DX1.32E89F90-9C89-4EBE-9AD3-DF1D2457F6C8.svs Soft Tissue TCGA-Q-A8VF-01Z-00-DX1.708A1A71-F284-4FBE-A2A1-E4037788A9F8.svs Soft Tissue TCGA-SI-A8C-01Z-00-DX2.BD0020ED-4F05-477E-AC29-6C0DB4DF62C6.svs Soft Tissue TCGA-WK-A8XO-01Z-00-DX5.45E4CD45-C8B2-4686-861C-5E7264FBA983.svs Soft Tissue TCGA-X6-A7WB-01Z-00-DX3.F2662EFE-A72B-41D7-9E0F-55C40E01B06.svs Soft Tissue TCGA-BR-7715-01Z-00-DX1.30aff8b3-b1b-4aff-9f6b-2dde40122359.svs Stomach TCGA-D7-A4YT-01Z-00-DX1.645AF39D-1B70-4C46-AB80-7B532994E66.svs Stomach TCGA-FP-7735-01Z-00-DX1.97aac46a-898e-4c43-8c93-1496f40c43ed.svs Stomach TCGA-VQ-A8P5-01Z-00-DX1.F3FDCB47-3426-400C-82A9-750012F95B3B.svs Stomach TCGA-VQ-A91N-01Z-00-DX1.4F7CF3D-8A-43DE-98C1-E44B2B5FB019.svs Stomach TCGA-BJ-A45I-01Z-00-DX1.29C9DC34-228A-4712-A028-07A396B4D1BD.svs Thyroid TCGA-DJ-A2PW-01Z-00-DX1.57FC77A8-DDCC-40DC-9B08-79DE5EFD02F0.svs Thyroid TCGA-EM-A2OZ-01Z-00-DX1.3146F96A-8BFE-46D2-AFFA-8593A56B58CD.svs Thyroid TCGA-ET-A3DR-01Z-00-DX1.702323F9-77A9-4D26-8236-F009C6CFADE4.svs Thyroid TCGA-FE-A3PC-01Z-00-DX1.EBF30F16-EDC4-4324-B4E-2F5A9F9AF05E.svs Thyroid TCGA-AP-A1DR-01Z-00-DX1.59D56F74-930F-4B0D-91D4-B3C0B4A92073.svs Uterus TCGA-AX-A062-01Z-00-DX2.3B3151D7-83C9-4932-9A23-D6FE4D490F8B.svs Uterus TCGA-D1-A17S-01Z-00-DX1.8EBE62E2-B0D1-480D-8A1C-97E7359FAA79.svs Uterus TCGA-DF-A2KY-01Z-00-DX1.AE1DDE62-EA95-4DC0-9783-849591B6B656.svs Uterus TCGA-FI-A2EW-01Z-00-DX1.93D21AD2-076D-4685-A027-623C294974A5.svs Uterus Table 26: Training corpus and compute per vision–language stage. *WILSON-CLIP uses a fixed 1,410,600/349,861 train/val composite manifest (§4); all other rows are approximate WSI-level splits computed from each script’s val_frac argument applied to the paired corpus above, since exact per-sample assignment is determined at run time by the data loader. “Eff. batch” is GPUs × per-GPU batch size × gradient-accumulation steps. Stage Paired WSIs Composites Val. frac. Epochs GPUs Eff. batch WILSON-distill 178,020 1,759,787 0.02 20 32 (4×8) 2,048 WILSON-KW (keyword regr.) 178,088 1,760,461 0.10 7 8 (1×8) 512 WILSON-CLIP, Phase 2 1,760,461* 1,760,461* 0.199* 20 32 (4×8) 576 WILSON-CLIP, Phase 3 1,760,461* 1,760,461* 0.199* 15 32 (4×8) 768 WILSON-CoCa captioning 178,020 1,759,787 0.02 20 32 (4×8) 1,024 7 Supplementary Method Construction of the Mayo189K training corpus Organ-site extraction and organ classification Pathology reports were parsed into organ-site level diagnostic entries to enable linkage between report text and specific tissue specimens. Organ information was extracted from report metadata and mapped to predefined anatomical categories using rule-based assignment rules developed and reviewed by a board-certified pathologist. A total of 42 organ categories were defined to cover the spectrum of surgical pathology specimens encountered in the archive. Because pathology reports frequently contain multiple specimens from different organs within a single case, organ-site level decomposition was performed before downstream processing. This allowed disease categorization and sampling to be conducted independently for each organ site rather than at the case level. Construction of organ-specific disease categories Diagnostic terminology in routine pathology reports is highly heterogeneous, with substantial variation in nomenclature, reporting style, and diagnostic granularity. To organize this diagnostic landscape while preserving disease diversity, we constructed organ-specific disease categories using a combination of large language models, embedding-based clustering, and expert curation. For each organ site, the primary diagnosis was extracted from pathology reports using Gemma 3 27B Gemma Team (2025). Extracted diagnoses were embedded using EmbeddingGemma Vera et al. (2025) and clustered with HDBSCAN. This procedure grouped many lexical variants and closely related diagnostic expressions into common clusters while preserving distinct disease concepts. The resulting clusters were subsequently reviewed by a board-certified pathologist independently for each organ. During this curation process, diagnostically equivalent or closely related entities were merged into unified disease categories. For example, terminology such as pituitary adenoma and pituitary neuroendocrine tumor (PitNET) may appear in separate clusters despite referring to the same biological entity and were therefore consolidated into a single disease category. Conversely, clusters containing diagnostically distinct entities were separated when necessary. The resulting organ-specific disease categories were designed to facilitate diversity-aware sampling rather than to provide strict case-level diagnostic labels. Their primary purpose was to ensure broad representation of morphologic patterns and disease entities across the training corpus. To assign all organ-site entries to curated disease categories, the category list for each organ was provided to a large language model, which classified each extracted diagnosis into the most appropriate disease category. Identification of histologically unremarkable tissue Histologically unremarkable specimens are inconsistently described in pathology reports and are frequently not labeled explicitly as “normal”. To ensure representation of non-neoplastic and histologically unremarkable tissue within the dataset, a large language model classifier was used to identify specimens lacking significant pathological abnormalities. These specimens were subsequently included as distinct disease categories during sampling. Representative block selection Because lesion-level annotations were unavailable, a single representative tissue block was selected for each organ-site entry using report-derived evidence. When only a single tissue block was available, that block was selected directly. For specimens containing multiple tissue blocks, the pathology report was searched using a large language model to identify explicit references to a key block, target block, or diagnostically representative block. When such a designation was present, the referenced block was selected. If no explicit representative block could be identified, the block most frequently submitted for immunohistochemical (IHC) studies was selected as a surrogate representative block, based on the assumption that diagnostically important tissue regions are more likely to undergo ancillary testing. Organ sites for which no representative block could be confidently identified were excluded from dataset construction. Whole-slide image selection and scanner handling Only hematoxylin-and-eosin (H&E)-stained permanent-section slides were considered for inclusion. Frozen-section slides were excluded using slide-level metadata. Slides generated from the same representative tissue block were assumed to contain substantially equivalent tissue content. When multiple digitizations of the same specimen were available, scanner-specific selection rules were applied to ensure consistent representation while avoiding redundant inclusion of repeated scans. For specimens scanned on both Leica GT450 and Pramana SpectralHT platforms, the earliest scanned slide from the Leica GT450 series and the latest scanned slide from the Pramana SpectralHT series were retained. Dataset filtering Starting from 1,454,318 pathology cases and 10,904,856 WSIs, pathology reports were segmented into 2,782,862 organ-site diagnostic entries. We retained only H&E-stained permanent sections and excluded frozen sections using slide metadata. Organ categories represented by fewer than 4,000 tissue blocks were removed, leaving 36 of the 42 predefined organ categories. After filtering, the candidate pool comprised 817,205 cases, 1,543,499 organ sites, and 5,973,856 WSIs. Diversity-aware sampling Clinical archives are strongly enriched for a small number of common organs and diagnoses. Consequently, uniform random sampling would produce a training corpus dominated by a limited set of disease entities. To maximize morphological, diagnostic, and linguistic diversity, sampling was performed independently within each organ-specific disease category. For every organ-disease combination, a maximum of 350 WSIs was selected. When the number of eligible specimens exceeded this limit, reports belonging to that organ-disease category were embedded using EmbeddingGemma and partitioned into k=10k=10 clusters. Sampling was then performed approximately uniformly across clusters to preserve diversity in reporting language and diagnostic presentation. This strategy intentionally reduced the influence of highly prevalent diseases while increasing representation of less common entities. Starting from an archive containing more than 10 million WSIs, the final Mayo189K training corpus comprised 189,291 WSIs spanning 838 organ-disease combinations. The resulting dataset substantially increased representation of rare organs and diagnoses relative to the source archive while preserving broad morphological coverage. Report-derived supervision Overview Each selected organ site was paired with report-derived supervision signals generated from the corresponding pathology report. Because routine pathology reports frequently contain multiple specimens within a single accession, template text, procedural comments, references, disclaimers, and physician signatures, substantial preprocessing was required before report text could be used for vision-language training. Two complementary supervision signals were generated for each whole-slide image (WSI): (1) natural-language captions describing histopathologic findings and (2) a sparse pathology keyword vector. All report-processing and caption generation steps were performed using Gemini 2.5 Pro Gemini Team (2025). Extraction of organ-specific diagnostic text A single pathology report may contain diagnostic information for multiple specimens originating from different organs. Consequently, the complete report cannot be used directly as a textual description of an individual organ-site WSI. To isolate text relevant to the target organ site, Gemini 2.5 Gemini Team (2025) Pro was prompted to extract only the diagnostic content corresponding to the designated specimen part (Supplementary Prompt 7). The extraction procedure preserved original diagnostic wording while removing content unrelated to tissue morphology, including physician names, institutional information, dates, references, technical processing details, disclaimers, and other administrative text. When reports contained multiple labeled specimens (e.g., A, B, C), only the text corresponding to the target specimen was retained. If no specimen labels were present, the full report was returned. Additional organ-specific interpretive information, including immunohistochemistry and molecular results, was retained when explicitly linked to the target specimen. The resulting cleaned text served as the canonical report representation for all subsequent supervision-generation steps. Generation of caption supervision The cleaned organ-specific report text was converted into natural-language captions using Gemini 2.5 Pro (Supplementary Prompt 7). The objective of caption generation was to create textual descriptions that align closely with histomorphologic features observable on H&E-stained whole- slide images. Therefore, the model was instructed to describe only findings supported by the report and visually observable on H&E sections, while avoiding clinical interpretation, prognostic statements, etiologic speculation, or information not represented in tissue morphology. For each report, three semantically equivalent but lexically diverse captions were generated. The model was instructed to vary wording, sentence structure, and descriptive perspective while preserving the underlying diagnostic content. This multi-caption strategy was intended to reduce sensitivity to specific phrasing and improve linguistic diversity during contrastive vision-language training. Each caption explicitly included the organ and diagnosis and preserved all negative findings reported in the original text (e.g., “no dysplasia”, “absent mitotic activity”). Text embedding generation The three generated captions were embedded independently using the Gemini Embedding 2 model Shanbhogue et al. (2026), producing three fixed 3072-dimensional text embeddings for each WSI. These embeddings were computed offline and used as text supervision during model training. Construction and assignment of pathology keywords In addition to free-text captions, we generated a structured keyword-based supervision signal for each whole-slide image (WSI). A board-certified pathologist manually curated a vocabulary consisting of 991 pathology keywords. The vocabulary was designed to cover the most frequently encountered diagnostic entities, morphologic findings, immunophenotypic features, and diagnostic category terms present in routine pathology reports. The keyword list intentionally included concepts at multiple levels of granularity, ranging from specific diagnoses (e.g., serous carcinoma) to broader diagnostic categories (e.g., carcinoma and malignancy). Directly assigning keywords from pathology reports is challenging because reports may express the same concept using diverse terminology, abbreviations, or descriptive phrases. To address this issue, keyword assignment was performed using a two-stage retrieval and verification framework. First, both the cleaned pathology report and all 991 keywords were embedded using the Gemini Embedding 2 model. Cosine similarity was then calculated between the report embedding and each keyword embedding. The 50 highest-ranked keywords were retrieved as candidate concepts for the report. Second, the cleaned report together with the 50 candidate keywords was provided to Gemini 2.5 Pro. The model was instructed to select only those keywords that were explicitly supported by the report content while rejecting semantically related but unsupported candidates. Multiple overlapping keywords were allowed when clinically appropriate. For example, a report describing serous carcinoma could simultaneously activate broader concepts such as carcinoma and malignancy, reflecting hierarchical relationships that naturally exist within pathology terminology. The final selected keywords were encoded as a 991-dimensional multi-hot vector for each WSI. Across the Mayo189K dataset, keyword vectors contained 5–7 active terms on average. Compared with using diagnosis labels alone, keyword-based supervision provides additional semantic structure and allows the model to learn relationships between morphologic findings, immunophenotypic features, diagnostic entities, and higher-level disease concepts. Because multiple related concepts can be assigned simultaneously, the resulting sparse representation captures hierarchical and compositional pathology knowledge that is difficult to encode using mutually exclusive class labels. Composite Generation Rather than feeding a single tile to the histopathology model, we represent each whole-slide image (WSI) as a composite: a structured set of tiles that collectively cover the slide while prioritizing the regions most likely to carry diagnostically relevant information. The pipeline proceeds in two phases. Phase 1 is executed once per organ using healthy (normal) slides and produces a compact reference of normal tissue appearances. Phase 2 runs on every disease slide and uses that reference to select the most unusual tissue tiles for the composite. Phase 1: Building the Normal Reference Tissue detection. For each normal WSI the lowest-resolution pyramid level is retrieved and converted to the Hue–Saturation–Density (HSD) color space Bejnordi et al. (2016). HSD operates in optical-density space and is well suited to H&E slides because it explicitly separates hematoxylin and eosin staining from slide background, correctly retaining lightly stained structures (adipose tissue, loose stroma) that simpler thresholding methods tend to discard. A pixel is labeled tissue if the smoothed cy channel of the HSD representation exceeds a fixed threshold; the resulting binary mask is used to enumerate non-overlapping 256×256256× 256 px tile positions at the target magnification. Up to 2 000 tile positions are randomly sampled per slide to keep memory usage bounded. Both phases share a single small Vision Transformer (ViT) pre-trained with self-supervision Caron et al. (2021) and adapted for binary tile quality classification. At inference, a single forward pass yields both the 384-dimensional CLS embedding, used for abnormality scoring, and a valid-tile probability pvalidp_valid, used to discard artifacts (ink marks, air bubbles, out-of-focus regions, tissue-edge debris) at no additional computational cost. Embedding extraction. Each sampled tile is read from the WSI pyramid, resized to 64×6464× 64 px, and passed through a Vision Transformer (ViT) that was pre-trained with DINO Caron et al. (2021) self-supervised learning and subsequently fine-tuned as a two-class (valid/invalid) quality-control classifier. The 384-dimensional class token (CLS) produced by the ViT backbone is used as the tile embedding; the classification head is not used in this phase. Centroid estimation. All embeddings collected from the normal slides of one organ are pooled and clustered with MiniBatch k-Means (k=200k=200). To ensure that each centroid corresponds to a real tissue appearance rather than an uninterpretable average in embedding space, every mean centroid μc _c is replaced by its medoid—the embedding ∗e^* in cluster c that minimizes the Euclidean distance to the mean: c∗=argmini∈c‖i−c‖2,e^*_c= e_i _c \|e_i- μ_c \|_2, (1) where cC_c denotes the set of embeddings assigned to cluster c. The resulting 200×384200× 384 medoid matrix is saved as the organ’s normal reference and reused without modification in Phase 2. Phase 2: Abnormality-Guided Tile Selection Quality filtering. For each disease WSI, tissue tile coordinates are obtained using the same HSD mask as in Phase 1. Every tile is processed in a single forward pass through the quality-control ViT backbone, which yields the 384-dim CLS embedding while the ViT head simultaneously produces two-class logits from which the valid-tile probability pvalidp_valid is derived via softmax. Tiles with pvalid<0.5p_valid<0.5 are discarded, removing artifacts such as ink marks, air bubbles, out-of-focus regions, and tissue-edge debris before any scoring is performed. Abnormality scoring. For each remaining tile i with embedding ie_i, the abnormality score is defined as the minimum Euclidean distance to any of the K=200K=200 normal medoids kk=1K\m_k\_k=1^K: si=mink=1,…,K‖i−k‖2.s_i= _k=1,…,K \|e_i-m_k \|_2. (2) A high score indicates that the tile’s appearance deviates substantially from all known normal tissue patterns and is therefore a candidate for pathological content. The distance matrix is computed efficiently via the identity ‖−‖2=‖2+‖2−2⊤\|a-b\|^2=\|a\|^2+\|b\|^2-2\,a b, avoiding the construction of the full (N×K×D)(N× K× D) difference tensor. Spatial-grid selection. Selecting the globally top-scoring tiles would cluster the chosen patches within a single lesion region, sacrificing the spatial representativeness needed for slide-level modeling. To balance abnormality and coverage, the bounding box of all valid tissue tiles is divided into an ng×ngn_g× n_g spatial grid with ng=⌈Nselect⌉n_g= N_select (for Nselect=60N_select=60, ng=8n_g=8, giving 64 candidate cells). Within each cell the tile with the highest abnormality score sis_i is selected. Cells that contain no tissue tile are filled from the globally highest-scoring unused tiles. From the resulting up to ng2n_g^2 candidates, the final Nselect=60N_select=60 tiles with the largest scores are retained. This procedure guarantees that the composite samples from all spatial zones of the tissue while still biasing strongly towards the most diagnostically informative locations. Output. The selected tiles are stored as a JSON record per slide containing: the WSI filename; the OpenSlide pyramid level and magnification (2.5×); and, for each tile, the level-0 pixel coordinate (x,y)(x,y), the abnormality score sis_i, the valid-tile probability pvalid,ip_valid,i, and the index of the nearest normal centroid k∗=argmink‖i−k‖2k^*= _k\|e_i-m_k\|_2. Phase 3: Multi-Magnification Composite Assembly Phase 3 reads the JSON coordinate files produced by Phase 2 and assembles the final multi-magnification composite images. Each composite is an 8×88× 8 grid of 256×256256× 256 px tiles in which each row represents one selected tissue location across multiple magnifications. Composite generation. Phase 2 produces a pool of Nselect=80N_select=80 candidate tissue locations represented by 1.25×1.25× context tiles. To generate multiple valid realizations of the composite representation from the same WSI, a small random perturbation is added to the abnormality score of each candidate location, s~i=si+ϵi, s_i=s_i+ _i, (3) where sis_i is the abnormality score defined in Equation 2 and ϵi _i is a zero-mean random noise term. For each composite, candidate locations are ranked according to s~i s_i, and the eight highest-ranked locations are selected. This procedure is repeated independently to generate C=10C=10 composites per WSI. The perturbation serves two purposes. First, it enables assessment of the robustness of downstream predictions to modest variations in the tile-selection process, reducing the likelihood that performance depends on a single deterministic set of selected locations. Second, during WILSON pre-training and fine-tuning, the resulting diversity of composite realizations acts as a form of data augmentation, exposing the model to multiple valid multi-magnification summaries of the same slide and thereby reducing overfitting to specific tissue locations. Importantly, the perturbation is not intended to identify more informative regions than those selected by the original abnormality ranking. Rather, it is used to sample alternative yet diagnostically plausible composite representations from the same candidate pool. Representative multi-magnification region selection. For each selected tissue location, a 1.25×1.25× context tile is extracted, covering the largest field of view and providing architectural context. To identify representative higher-magnification regions within this context, the 1.25×1.25× tile is partitioned into a coarse 4×44× 4 grid and a finer 16×1616× 16 grid. Grid regions are characterized by their RGB color features and grouped using unsupervised clustering. Representative regions are then selected from the resulting clusters to capture the dominant visual and morphological diversity present within the context field while minimizing redundant sampling of similar tissue appearances. Using these representative locations, three regions are extracted at 5×5× magnification and four regions at 20×20× magnification. Together with the original 1.25×1.25× context tile, this yields eight tiles associated with the same tissue location. All tiles are resampled to 256×256256× 256 px. Composite layout. Each row of the composite corresponds to one tissue location. The leftmost column contains the 1.25×1.25× context tile. Columns 2–4 contain the three representative 5×5× tiles, and columns 5–8 contain the four representative 20×20× tiles. This arrangement preserves the relationship between broad tissue architecture and higher-magnification morphological detail while maintaining their spatial correspondence within the same context field. Each composite therefore contains eight tissue locations represented by eight tiles each, resulting in a total of 64 tiles arranged in an 8×88× 8 layout. Output. Phase 3 produces C=10C=10 composite PNG images per WSI, each of size 2048×20482048× 2048 px (8×2568× 256 per axis). The resulting representation encodes two complementary dimensions of information: rows correspond to distinct tissue locations selected from the abnormality-guided candidate pool, whereas columns correspond to contextual and representative higher-magnification observations derived from the same location. This structured representation provides a compact visual summary of the whole slide, jointly encoding tissue heterogeneity, multi-scale morphology, and the correspondence between low- and high-magnification observations within a single image. Figure 16: Example of composite image (single WSI). MayoBreast dataset. LLM Prompts 1 You are an expert system for extracting structured text from pathology reports. INPUT: - Pathology report: % [Pathology Report] - Target part label: B GOAL: Extract ONLY the text corresponding to the specified part, preserving the original wording and formatting exactly. INSTRUCTIONS: 1. IDENTIFY PART STRUCTURE - Reports may contain labeled sections such as: A., B., C., etc. - A part may appear: - As a single label (e.g., "B.") - Within a range (e.g., "A–C", "A-C", "A to C") 2. EXTRACTION RULES - If B appears as an explicit label: → Extract ALL text under that label until the next label begins. - If B falls within a label range: → Extract the ENTIRE text corresponding to that range. - Include: - Diagnosis text - Description - Immunohistochemistry (IHC) - Molecular findings - Any interpretation clearly linked to the target part 3. INCLUDE RELATED GLOBAL SECTIONS Include additional sections ONLY if they are clearly related to the target part, such as: - Receptor status (ER, PR, HER2, etc.) - IHC result blocks referencing the target part (e.g., "block B1") - Molecular findings tied to the part Exclude general sections if they are purely methodological or unrelated to interpretation. 4. NO LABELS CASE If NO part labels (A, B, etc.) exist in the report: → Return the ENTIRE report. 5. STRICT EXCLUSIONS Do NOT include: - Doctor names - Institutional or company names - Dates - Specimen processing details (fixation method, technical notes) - Scanner or equipment info - References to guidelines, journals, or protocols - Any ".DNR" or irrelevant trailing artifacts 6. OUTPUT FORMAT - Return ONLY the extracted text - Do NOT add explanations, comments, or formatting - Preserve line breaks, indentation, and wording as-is - Merge lines only if they are clearly part of the same sentence split by line breaks 7. EDGE CASE HANDLING - If boundaries are ambiguous, prefer including slightly MORE context rather than truncating relevant information - If multiple mentions of the same part exist, include all of them OUTPUT: Return only the extracted text. LLM Prompts 2 You are a board-certified pathologist generating training captions for H\&E-stained histopathology images. The goal is to align text with visual features for a vision-language model. Write exactly 3 captions, each under 50 words. Global rules: - Each caption MUST explicitly mention the organ and the diagnosis. - ONLY describe features that are visually observable on H\&E - Do NOT infer clinical history, causation, prognosis, or unseen findings. - Do NOT add any findings beyond the provided reports. - PRESERVE NEGATIONS strictly (e.g., ’no dysplasia’, ’absent mitoses’). These must remain negative and must not be rephrased as positive findings. Diversity requirements (CRITICAL): - Each caption must express a DISTINCT perspective. - Use different lexical choices (including medically valid synonyms where appropriate). - Avoid repeating the same phrase structure or keyword grouping. - Avoid trivial rewording — ensure meaningful semantic variation. Grounding rules: - Integrate all assigned keywords naturally into each caption. - Ensure the wording reflects how the feature appears microscopically. - Prefer concrete morphologic descriptors. Style constraints: - Keep sentences concise, formal, and descriptive. - Each caption must start differently. - Avoid meta language (no ’this image shows’ or ’there is’). Return ONLY valid JSON: "sentences": ["caption1", "caption2", "caption3"] % [Pathology Report] report Figure 17: WILSON vision encoder (ConvNeXtV3Extended). a, Stage-by-stage architecture for the 2048×20482048× 2048 reference input (an 8×88× 8 grid of 256-px tiles). Tensor glyphs encode each feature map (face ∝log2 _2 spatial size, depth ∝log2 _2 channels), making the progression from a broad, shallow 128×512×512128× 512× 512 map to a narrow, deep 3072×8×83072× 8× 8 map visible at a glance. A DINOv3-pretrained ConvNeXt-Base backbone (depths 3,3,27,33,3,27,3; widths 128,256,512,1024128,256,512,1024; stride 32; 87.57M params) is followed by three stride-2 downsampling blocks (extra_blocks ×3× 3; 46.15M params) that extend the width to 3072 (stride 256) and global average pooling, for 133.7M parameters. Column-aligned bars show where parameters and compute (conv/linear MACs) reside: Stage 3 dominates both (57.4M params, 939 GMACs), whereas the extended head holds 34.5% of the parameters but only 0.6% of the compute. b, Anatomy of the two block types with symbolic parameter counts; the first extended block omits its LayerNorm (its input is already normalized by the backbone), and a 1×11× 1 convolutional fallback replaces the stride-2 convolution only when a block’s input is already 1×11× 1 (e.g. 224-px inputs). c, The three operating resolutions. At 224–256 px (tile-based SSL) only the backbone is used and its 1024-d GAP output is taken directly (20 GMACs); at 512×512512× 512 (composite-based SSL pretraining, §4) the final map is 3072×2×23072× 2× 2 (81 GMACs); at 2048×20482048× 2048 (composite-based vision–language training, §4) the final 3072×8×83072× 8× 8 map provides 64 spatial tokens to the CoCa attentional pooler (§4) (1,291 GMACs). d, Warm-start of the extra-block convolutions when the backbone is loaded from a domain-specific SSL checkpoint: the 1024×512×2×21024× 512× 2× 2 filters of the last backbone downsampling layer are tiled to each extra-block shape and rescaled to He-initialization magnitude (std=2/fan_instd= 2/fan\_in), preserving filter directions while keeping activations within fp16 range; biases are tiled likewise and the extra-block LayerNorms start at (1,0)(1,0). Table 27: WILSON vision encoder (ConvNeXtV3Extended) — parameter and compute breakdown by component. Output shapes, cumulative stride and MACs are given for the 2048×20482048× 2048 reference input (an 8×88× 8 grid of 256-px tiles), the default resolution for composite-based vision–language training. Each “Block” is a standard ConvNeXt residual block (Fig. 17b). Parameter counts were verified by instantiating the model. The extended head holds 34.5% of the parameters but accounts for only 0.6% of the compute, because it operates on ≤32×32≤\!32× 32 maps; conversely Stage 3 alone accounts for 72.7% of the MACs. †Blocks 2–3 also instantiate a 1×11× 1 convolutional fallback (2,099,200 and 6,294,528 params, included in the totals) that is used only when the block’s input is already 1×11× 1; it is dormant for all inputs ≥256≥ 256 px, leaving 125,327,488 active parameters (≈ 125.3M). ‡Multiply–accumulates of convolution and linear layers only (LayerNorm, GELU, layer scale and pooling omitted); 1 MAC == 2 FLOPs. At 512×512512× 512 the total is 80.7 GMACs and at 256×256256× 256 (backbone only) 20.1 GMACs. Component Operation In→ ch. Stride Output (204822048^2) Params % GMACs‡ Backbone — DINOv3 ConvNeXt-Base (87,566,464 params, 65.5%; 1,283 GMACs, 99.4%) Stem Conv ×44\!×\!4 s4 + LN 3→1283→ 128 4 ×512128\!×\!512\!×\!512 6,528 0.0 1.6 Stage 1 3× Block 128128 4 ×512128\!×\!512\!×\!512 415,488 0.3 108 Downsample 1 LN + Conv ×22\!×\!2 s2 128→256128→ 256 8 ×256256\!×\!256\!×\!256 131,584 0.1 8.6 Stage 2 3× Block 256256 8 ×256256\!×\!256\!×\!256 1,617,408 1.2 106 Downsample 2 LN + Conv ×22\!×\!2 s2 256→512256→ 512 16 ×128512\!×\!128\!×\!128 525,312 0.4 8.6 Stage 3 27× Block 512512 16 ×128512\!×\!128\!×\!128 57,424,896 42.9 939 Downsample 3 LN + Conv ×22\!×\!2 s2 512→1024512→ 1024 32 ×641024\!×\!64\!×\!64 2,099,200 1.6 8.6 Stage 4 3× Block 10241024 32 ×641024\!×\!64\!×\!64 25,344,000 19.0 104 Final norm LayerNorm 10241024 32 ×641024\!×\!64\!×\!64 2,048 0.0 – Extended head — three stride-2 blocks (46,154,752 params, 34.5%; 8.1 GMACs, 0.6%) Extra block 1 Conv ×22\!×\!2 s2 + GELU (no LN) 1024→10241024→ 1024 64 ×321024\!×\!32\!×\!32 4,195,328 3.1 4.3 Extra block 2 LN + Conv ×22\!×\!2 s2 + GELU † 1024→20481024→ 2048 128 ×162048\!×\!16\!×\!16 10,491,904 7.8 2.1 Extra block 3 LN + Conv ×22\!×\!2 s2 + GELU † 2048→30722048→ 3072 256 ×83072\!×\!8\!×\!8 31,467,520 23.5 1.6 Global avg. pool AdaptiveAvgPool2d(1), Flatten 30723072 256 30723072 0 0.0 – Total 256 3072-d embedding 133,721,216 100 1,291 8 End-to-end fine-tuning on MayoTNBC Data and splits MayoTNBC comprises 508 triple-negative breast carcinoma WSIs from 508 patients, one slide per patient, stored in Philips TIFF format (Table 14). Each slide carries two independent four-class labels, histologic subtype and stromal tumor-infiltrating lymphocyte (sTIL) grade, and each label was fine-tuned as a separate run; slides with a missing value for the label in question were dropped from that run, so the two tasks do not use identical slide sets. Every WSI was rendered into 40 composites, giving a training split of 407 WSIs (16,280 composites) and a validation split of 101 WSIs (4,040 composites), an approximately 80/20 partition. The two splits were fixed in advance as separate manifest files and verified to share no composite directory, so they are disjoint at the slide level and, given the one-to-one slide-to-patient mapping, at the patient level. Because the pretraining corpus was scanned on Leica GT450 and Pramama SpectralHT systems, the Philips TIFF slides in this cohort were acquired on different hardware, so the experiment measures adaptation under scanner domain shift rather than in-domain adaptation alone. The two tasks differ substantially in class balance. Histologic subtype is severely imbalanced: the majority class accounts for 71.3% of training and 65.3% of validation composites, against 5.4% and 5.9% for the smallest class, a ratio of roughly 13:1, although the imbalance itself is consistent between splits. sTIL grade is close to uniform, with every class occupying between 20% and 31% of each split, but its train and validation distributions are less well matched, the validation split carrying about 8.5 percentage points more of the lowest grade and correspondingly less of the next. No class weighting, resampling or stratification was applied in either run; the loss is unweighted cross-entropy throughout. For histologic subtype this means accuracy is inflated relative to macro-averaged F1 by the dominant class, and the macro-averaged F1 column of Table 2 is the more informative measure of whether fine-tuning improved performance on the minority subtypes. Optimization A linear classification head was attached to the encoder’s 3072-dimensional global-average-pooled output and the network was optimized with a cross-entropy loss under a staged schedule: the backbone was held frozen for the first two epochs so that the randomly initialized head could settle, then unfrozen for full end-to-end fine-tuning over the remaining three, for five epochs in total. The head and backbone used separate learning rates of 10−310^-3 and 10−610^-6 respectively, with weight decay 10−410^-4, a per-GPU batch size of 4 and 8 gradient-accumulation steps across a single node of 8 H200 GPUs, giving an effective batch of 256, under mixed-precision training. No checkpoint selection was performed: the weights from the final epoch were used for all evaluations, so no information from the validation split influenced the reported model. The validation split therefore functions as a held-out test set for the fine-tuning experiment. Row-shuffle augmentation Training composites were augmented by online row-strip shuffling. For each training sample, two composites belonging to the same WSI were drawn (or one composite reused twice when only one was available), each sliced into its eight horizontal 256×2048256× 2048 row-strips, and eight of the resulting sixteen strips sampled without replacement and stacked vertically into a new synthetic 2048×20482048× 2048 composite. Only row order and row provenance are altered, so each row retains its intact multi-magnification column structure (one low-magnification context tile followed by its spatially linked higher-power views, §7). Each WSI contributed 40 augmented items per epoch, matching the number of composites rendered per WSI so that epoch length is unchanged relative to training on the unaugmented composites directly. Validation and all retrieval evaluation used the original, unaugmented composites. Evaluation settings The unit of evaluation is the individual composite rather than the slide: every composite of a training slide enters the retrieval gallery and every composite of a validation slide is issued as a separate query. Base Zero-shot and FT-Zero-shot both use top-1 cosine-similarity nearest-neighbor retrieval between L2-normalized embeddings, with the query taking the label of its single nearest gallery neighbor, and differ only in whether the encoder has been fine-tuned; FT-Zero-shot discards the classification head entirely, so the comparison isolates the effect of fine-tuning on the representation. FT-classification instead evaluates the trained linear head as an ordinary classifier. Because the head is a learned decision boundary and nearest-neighbor retrieval is memoryless, the two are not expected to agree numerically, and the intermediate setting is the one that speaks to representation quality. This protocol is deliberately simpler than the zero-shot retrieval protocol used elsewhere in this work (§9 , §4), which uses five nearest neighbors under majority voting, excludes all slides from the query’s own patient, and reports the mean and standard deviation across the ten composites generated per slide. Here a single fixed train gallery and validation query set are used with one neighbor, and all composites are pooled into one evaluation, so Table 2 reports a single value per cell without composite-to-composite variance. Table 28: WILSON + CoCa vision-language training hyperparameters. Hyperparameter Value Input resolution 2048×20482048× 2048 px Vision spatial tokens 8×8=648× 8=64, 3072-dim Vision-encoder trainable params 133.72M Attentional-pooler queries / heads 256 / 8 Joint embedding dim 768 Decoder width 768 Decoder loss weight 1.0 Initial logit scale / bias 10.0 / −10.0-10.0 Negative bank steps 8 LR (vision / text / proj.+pooler) 10−510^-5 / 10−510^-5 / 10−310^-3 Weight decay 10−410^-4 Warmup 10% of schedule Min. LR fraction 1% of peak Gradient clipping 1.0 Per-GPU batch size 4 Gradient accumulation 8 steps Effective batch size 1,024 pairs Epochs 20 Precision bfloat16 Hardware 4 nodes × 8 GPUs (32 total) Validation fraction (patients) 2% Row-shuffle repeats/epoch 30 Max caption length 76 tokens 9 Quantitative evaluation of report generation Evaluation protocol Report generation was evaluated by comparing each model’s generated caption with a reference caption derived from the slide’s clinical pathology report (Fig. 4a). For every held-out WSI, one multi-magnification composite was encoded by the model’s vision tower and decoded autoregressively into free text; for WILSON this is the CoCa multimodal decoder (§4), and for PRISM and PRISM2 the released report-generation heads, used off the shelf with their published checkpoints. Decoding was deterministic for all three models (no sampling). WILSON-CoCa decodes greedily to a maximum of 64 tokens; PRISM decodes from its BioGPT decoder with beam search (5 beams, maximum 100 tokens); and PRISM2 decodes from its Phi-3 decoder with beam search (5 beams, maximum 200 new tokens) in response to the prompt “Write a report describing this whole slide image.”. Each model’s published defaults were retained rather than harmonized, so the generated captions differ in typical length; because BLEU-4 applies a brevity penalty and both ROUGE variants are length-sensitive, absolute values partly reflect these differing generation budgets. Evaluation covered three cohorts: a held-out subset of the Mayo Clinic corpus (1,000 image–caption pairs sampled with a fixed random seed shared across models, spanning multiple organs and diagnoses), HistAI (48 pairs; Table 24) and TCGA (50 pairs; Table 25). No evaluation slide or patient overlapped the training corpus. All metrics were computed against a single reference caption per slide. Mayo Clinic slides carry three paraphrased caption variants per report (§7); the first variant was used, so that every cohort is scored under the same single-reference protocol as HistAI and TCGA, for which only one reference caption per slide is available. Metrics that natively support a reference set (BLEU-4, CIDEr, ROUGE) were therefore run in single-reference mode throughout, which lowers their absolute values — CIDEr especially, since it was designed for consensus against many references — but keeps the three cohorts and the three models directly comparable. All metrics are per-pair (caption, reference) scores averaged over pairs within a cohort; error bars in Fig. 15 denote one standard deviation across pairs. Metric families The eight metrics differ in what they treat as a match, and consequently in their usable dynamic range on long, freely worded diagnostic text. They fall into three families. Surface n-gram overlap. BLEU-4 takes the geometric mean of 1- to 4-gram precisions against the reference and multiplies it by a brevity penalty for captions shorter than the reference. It rewards only exact, contiguous token sequences and gives no credit for synonymy, paraphrase or reordering. Designed for machine translation, it approaches zero on long single-reference captions unless the wording closely mirrors the reference, so absolute BLEU-4 values here index lexical mimicry rather than diagnostic correctness. ROUGE-1 is the F-measure of unigram precision and recall after stemming; it credits shared vocabulary irrespective of order, making it the most forgiving of the overlap metrics but blind to grammar and phrase structure. ROUGE-L scores the longest common subsequence of the two captions — tokens shared in the same relative order but not necessarily adjacent — again as an F-measure after stemming, and therefore sits between ROUGE-1’s bag-of-words leniency and BLEU-4’s strict adjacency. Weighted n-gram overlap. CIDEr represents both captions as TF–IDF-weighted 1- to 4-gram vectors, so that corpus-frequent n-grams are down-weighted and distinctive, content-bearing n-grams dominate, and averages the cosine similarity across n-gram orders. CIDEr was designed as a corpus-level consensus measure against many references per image; the per-pair, single-reference computation used here is a substantially weaker signal and sits near zero even for clinically accurate captions. Embedding-based similarity. BERTScore embeds every token of both captions with a pretrained contextual language model (RoBERTa-large) and greedily matches each token to its most cosine-similar counterpart in the other caption: precision averages the best-match similarity over generated tokens, recall over reference tokens, and F1 is their harmonic mean. Because matching occurs in embedding space, reworded clinical phrasing and synonyms that BLEU, ROUGE and CIDEr treat as total mismatches are credited, which is why BERTScore values occupy a compressed high range (≈ 0.83–0.89 here) even when two captions share few literal tokens; differences of 0.01–0.02 in this range are therefore not negligible. Within BERTScore, recall is the component most sensitive to whether the generated caption covers the reference’s diagnostic content, and precision the component most sensitive to unsupported additions. CLIPScore projects the slide’s image embedding — PRISM’s tile-pooled slide representation, or WILSON’s own vision-tower encoding of the composite — and the generated caption’s text embedding into that model’s shared contrastive space and takes their cosine similarity. Unlike every other metric, it never inspects the reference: it measures whether the caption is consistent with what the model itself encodes from the slide, making it an image-grounding rather than a report-similarity measure. Because each model’s contrastive space has its own learned scale and temperature, a CLIPScore is interpretable only for ranking or comparing WSIs within one model; the raw cross-model difference is not evidence that one model is better grounded than another. CLIPScore is not reported for PRISM2, which exposes no general-purpose text encoder capable of embedding an arbitrary string independently of an image. Aggregate score Because the eight metrics have incompatible scales and effective ranges, the summary column of Table 13 (“Avg”) reports, for each model and cohort, the mean of its metric values after min–max normalization within each metric and cohort, rescaled to 0–100. Normalization is performed across the models compared in that cohort, so the aggregate is a relative ranking within a cohort and carries no absolute meaning and no cross-cohort comparability. CLIPScore is excluded from the aggregate, and from bolding, because it is not comparable across models and is undefined for PRISM2. Interpretation caveats • Floor effects are expected, not diagnostic of failure. BLEU-4 and CIDEr were designed for short, multi-reference captioning benchmarks; under the single-reference protocol used here all three models sit near zero on long diagnostic prose, and the ordering between them is more informative than the magnitudes. • Ceiling compression in BERTScore. All three models exceed 0.83 on every cohort because clinical text shares substantial contextual structure regardless of diagnostic agreement; BERTScore separates models weakly and should be read alongside the overlap metrics rather than in place of them. • No metric here credits paraphrase directly. BLEU-4, CIDEr and both ROUGE variants match surface tokens only, so a clinically equivalent caption worded differently from the reference is penalized; BERTScore is the only reference-based metric in this set that tolerates rewording, which is why it should not be read in isolation from the overlap metrics or from the qualitative comparisons. • CLIPScore is not a cross-model comparison. See above. • Reference provenance differs between cohorts. Mayo Clinic references were summarized from pathology reports with Gemini 2.5 Pro, TCGA references from the machine-readable reports of Kefeli et al. Kefeli and Tatonetti (2024) with DeepSeek V4 Pro, and HistAI references are the diagnostic conclusion field supplied with the dataset’s own metadata, used verbatim without LLM summarization. Because the overlap metrics reward literal token agreement, differences in reference style and length affect absolute values independently of caption quality, so metric values are comparable across models within a cohort but not across cohorts. • Domain adaptation is not controlled. WILSON-CoCa’s decoder and text tower were trained on Mayo Clinic captions generated by Gemini 2.5 Pro, whereas PRISM and PRISM2 were evaluated as released. The Mayo Clinic column is therefore the only one in which the reference style matches the style WILSON-CoCa learned to produce, and reflects both architecture and adaptation to the target reporting domain; the HistAI and TCGA columns, where no model was adapted to the reference style, are the more architecture-like comparisons. • Lexical similarity is not diagnostic correctness. A caption can score well by reproducing reporting boilerplate while naming the wrong entity, and can score poorly while being clinically correct, as the qualitative comparisons in Fig. 14 illustrate.