Paper deep dive
CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT
SΓ©lΓ©na Jean-Mactoux, Sophie Loizillon, Ghina Jardali, Amandine CrombΓ©, Nathalie Lassau, Alexandre BΓ΄ne
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 10/14/2026, 9:42:35 AM
Summary
The paper introduces CAPTOR, a vision-language foundation model designed for oncologic Chest-Abdomen-Pelvis (CAP) CT interpretation. Trained on a large multicenter dataset of 25,512 CT-report pairs using a region-specific contrastive pretraining strategy, CAPTOR outperforms state-of-the-art models (including Merlin, CT-CLIP, SPECTRE, COLIPRI, and CT-FM) in disease detection, survival prediction, and image retrieval tasks within oncologic contexts.
Entities (14)
Relation Signals (13)
CAPTOR β evaluatedon β TumorImagingBench
confidence 95% Β· survival prediction using TumorImagingBench (1)
CAPTOR β outperforms β Merlin
confidence 95% Β· CAPTOR outperformed all baseline models, on the classification task, on the CAP subset... and oncologic subset
CAPTOR β outperforms β CT-CLIP
confidence 95% Β· CAPTOR outperformed all baseline models... on the CAP subset
CAPTOR β trainedon β Internal Dataset
confidence 95% Β· CAPTOR was trained on the proprietary dataset... 25,512 CT-report pairs
CAPTOR β uses β ResNet152
confidence 95% Β· We used the Merlin [12] architecture... and an inflated ResNet152 [27]... as the 3D image encoder.
CAPTOR β uses β TotalSegmentator
confidence 95% Β· CT volumes are cropped into 12 anatomical sub-regions using TotalSegmentator28 masks
CAPTOR β uses β Clinical Longformer
confidence 95% Β· We used the Merlin [12] architecture... using Clinical Longformer [26] as text encoder
CAPTOR β evaluatedon β NSCLC-Radiomics
confidence 90% Β· CAPTOR outperformed all baseline models... on three of the four oncologic prognostic benchmarks... NSCLC-Radiomics
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Purpose : To develop and evaluate CAPTOR, a deep-learning vision-language foundation model for oncologic chest-abdomen-pelvis (CAP) CT interpretation. Materials & Methods : CAPTOR (Chest-Abdomen-Pelvis foundaTion model for Oncologic Radiology) is a contrastive vision-language model, trained using a pretraining strategy for oncologic CAP CT (in single- or multiple-series acquisitions), on a multicenter retrospective dataset from North America, Europe and Asia, predominantly from tumor lesion examinations. CAPTOR was compared with state-of-the-art CT foundation models on three downstream tasks: disease detection, survival prediction using TumorImagingBench and image retrieval. Classification and retrieval were both evaluated on held-out internal and public test sets, stratified by anatomical coverage and indication. Bootstrap resampling was used to estimate confidence intervals, and pairwise comparisons were performed using permutation tests. Results : CAPTOR was trained and tested on 25,512 and 10,794 CT-report pairs respectively. TumorImagingBench datasets comprised between 134 and 421 studies each. CAPTOR outperformed all baseline models, on the classification task, on the CAP subset (AUC 0.767, 95%CI: 0.734-0.793, vs. 0.695, 95%CI: 0.645-0.736 for the next-best, p<0.001) and oncologic subset (AUC 0.784, 95%CI: 0.759-0.805, vs. 0.734, 95%CI: 0.703-0.767 p<0.001), ranked first on three of the four oncologic prognostic benchmarks, and achieved superior retrieval performance on CAP and oncologic subsets. Conclusion : A foundation model trained on oncology-enriched CAP CT data outperformed relevant state-of-the-art models in its target clinical context, underscoring the importance of clinically aligned pretraining. We release our preprocessing, training and inference code : this https URL and model weights : this https URL
Tags
Links
- Source: https://arxiv.org/abs/2610.14226v1
- Canonical: https://arxiv.org/abs/2610.14226v1
Trouble viewing inline? Open PDF directly β
Full Text
77,264 characters extracted from source content.
Expand or collapse full text
CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT SΓ©lΓ©na Jean-Mactoux 1,2 Sophie Loizillon 2 Ghina Jardali 1,3 Amandine CrombΓ© 3,4,5 Nathalie Lassau 1,3 Alexandre BΓ΄ne 2 1 Biomaps, UMR1281, INSERM, CNRS, CEA, UniversitΓ© Paris-Saclay, Villejuif, France 2 Guerbet Research, Villepinte, France 3 Department of Diagnostic Oncologic Imaging, Gustave Roussy Institute, Villejuif, France 4 Department of Radiology, Pellegrin University Hospital, Bordeaux, France 5 Bordeaux Institute of Oncology, BRIC U1312, INSERM, Team SARCOTARGET, Bordeaux, France Abstract Purpose To develop and evaluate CAPTOR, a deep-learning vision-language foundation model for oncologic chest- abdomen-pelvis (CAP) CT interpretation. Materials & Methods CAPTOR (Chest-Abdomen-Pelvis foundaTion model for Oncologic Radiology) is a contrastive vision- language model, trained using a pretraining strategy for oncologic CAP CT (in single- or multiple-series acquisitions), on a multicenter retrospective dataset from North America, Europe and Asia, predomi- nantly from tumor lesion examinations. CAPTOR was compared with state-of-the-art CT foundation models on three downstream tasks: disease detec- tion, survival prediction using TumorImagingBench (1) and image retrieval. Classification and retrieval were both evaluated on held-out internal and public test sets, stratified by anatomical coverage and indi- cation. Bootstrap resampling was used to estimate confidence intervals, and pairwise comparisons were performed using permutation tests. Results CAPTOR was trained on 25,512 CT-report pairs (median age 66 years, 49% female) and evaluated on 10,794 studies (median age 60 years, 52% female). TumorImagingBench datasets comprised between 133 and 421 studies each (median ages between 58.5 and 69 years, between 25% and 41% female each). CAP- TOR outperformed all baseline models, on the classi- fication task, on the CAP subset (AUC 0.767, 95%CI: 0.734-0.793, vs. 0.695, 95%CI: 0.645-0.736 for the next-best, p<0.001) and oncologic subset (AUC 0.784, 95%CI: 0.759-0.805, vs. 0.734, 95%CI: 0.703-0.767 p<0.001), ranked first on three of the four onco- logic prognostic benchmarks, and achieved superior retrieval performance on CAP and oncologic subsets. Conclusion A foundation model trained on oncology-enriched CAP CT data outperformed relevant state-of-the-art models in its target clinical context, underscoring the importance of clinically aligned pretraining. We release our preprocessing, training and infer- ence code at this url : https://github.com/Guerbet- AI/CAPTOR and the model weights at this url : https://huggingface.co/Guerbet-AI/CAPTOR 1 Introduction With a projected increase of 63% in cancer incidence by 2045 [1], pressure intensifies on radiological prac- tice, especially for Chest-Abdomen-Pelvis (CAP) CT interpretation, a key modality for oncologic stag- ing, metastatic assessment, and treatment monitor- ing [2,3,4,5]. This further strains the radiology workforce, with a radiologist shortage documented across Europe and the United States [6,7], and el- evates the risk of interpretative errors, particularly under fatigue like during overnight shifts [8]. Scalable AI tools supporting multiple downstream tasks from a single examination are increasingly recognized by major radiology societies for their workflow augmen- tation potential [9]. Recent advances on foundation models, and specif- 1 arXiv:2610.14226v1 [cs.CV] 9 Oct 2026 ically vision-language models (VLMs) enable joint image-text representation learning from CT images and radiology reports through contrastive pretrain- ing, without manual annotations. CLIP [10] methods applied to medical context, such as CT-CLIP [11] or Merlin [12] have demonstrated the potential of this approach across various diagnostic and prognostic tasks. However, most CT foundation models are trained on anatomically specific datasets (e.g. Chest for CT-CLIP or Abdomen-Pelvis for Merlin), limiting their applicability to full CAP protocols. Others, such as SPECTRE [13] and COLIPRI [14], both transformer-based multimodal foundation models, achieve broader anatomical coverage through het- erogeneous training sets, but without oncologic-rich training data. While oncologic-specific foundation models exist, they primarily target lesion detection, characterization, or treatment response rather than whole-image analysis [15,16]. Yet, radiologists inter- preting CAP CT scans in oncologic patients must also identify a wide spectrum of non-cancerous findings, ranging from treatment-related complications to age- and comorbidity-related conditions, as well as inci- dental findings requiring further evaluation, which can be overlooked under high workload and when the clinical question is focused on cancer. A gap therefore exists between narrowly focused oncologic models, and generalist foundation models lacking exposure to multimorbidity-rich oncologic CAP imaging, with no evidence that oncology patients require dedicated foundation models. To bridge this gap, we developed CAPTOR (Chest- Abdomen-Pelvis foundaTion model for Oncologic Radiology), a contrastive vision-language founda- tion model trained using a preprocessing strategy to handle whole CAP CT examinations, in single or multiple-series acquisitions. The training set com- prised 25,512 predominantly oncologic CAP CT stud- ies paired with radiology reports, from diverse scan- ners and protocols, reflecting routine clinical prac- tice. CAPTOR was compared with five state-of-the- art CT foundation models, including region-specific (CT-CLIP [11], Merlin [12]) and generalist models (SPECTRE [13], COLIPRI [14], CT-FM [17]) across three downstream tasks : medical condition classifi- cation, 2-year survival prediction using TumorImag- ingBench [18], and image retrieval. To assess the influence of pretraining dataset size, we additionally evaluated a larger variant trained on approximately 100,000 studies (CAPTOR-100K). 2 Materials and Methods 2.1 Datasets This retrospective study comprised three dataset categories: (i) an internal multicenter image-report dataset for vision-language pretraining, collected under data use agreements with 11 institutions across the United States (66%), Spain (16%), France (14%), Japan (3%) and Vietnam (1%), (i) public image-report datasets for downstream experiments : Merlin [12], CT-RATE [11], INSPECT [19] and BIMCV-R [20], and (i) public oncologic benchmark datasets for two-year post-treatment survival pre- diction, using TumorImagingBench [18]: NSCLC- Radiomics [21] and NSCLC-Radiogenomics [22] on chest CT, and C4KC-KiTS [23] and Colorectal-Liver- Metastases [24] on abdominal CT. All datasets were used under their respective use terms. All data were de-identified; informed consent was waived by the local institutional review board or ethics committee and data handling complied with applicable regional data protection regulations, including HIPAA and GDPR where relevant. The internal dataset was curated specifically for oncologic CT interpretation, predominantly from tu- mor lesion examinations, plus data for incidental pancreatic cancer detection. Studiesβ indication (on- cologic vs. non-oncologic) was inferred from radiology reports using the LLM Mistral Large 3 [25]. A ran- dom patient-level 80%/20% split was used to create training and testing sets. Appendices A.1-A.3 provide details on all datasets 2.2 CAPTOR Foundation Model β Architecture We used the Merlin [12] architecture, a contrastive VLM using Clinical Longformer [26] as text encoder, and an inflated ResNet152 [27] initialized with 2D ImageNet pretrained weights as the 3D image en- coder. 2.3 CAPTOR Foundation Model β Preprocessing Figure 1 provides an overview of the CAPTOR pipeline, encompassing data preprocessing, model training and evaluation. 2 Figure 1: Overview of the CAPTOR pipeline. CT volumes are cropped into 12 anatomical sub-regions using TotalSegmentator28 masks (A); regional crops are paired with the Findings section of the corresponding report and CAPTOR is trained with a rotating-region contrastive objective (B). At inference, TotalSeg- mentator masks are used to generate regional crops, and each CT yields one embedding per region (C). The resulting frozen image and text encoders are evaluated via XGBoost classification, k-nearest neighbor (kNN) survival prediction, and cross-modal retrieval (D, E, F). MAP@K = mean average precision at K, with K the number of retrieved images, std = Study CAPTORβs main methodological innovation lies in its preprocessing strategy (Fig. 1.A) : rather than using fixed central crops like Merlin [12], we use TotalSegmentator [28] masks to center each crop on a specific organ or anatomical region, enabling a region-specific architecture to handle whole CAP CT examinations, including multiple-series studies (Appendix A.4). 2.4 CAPTOR Foundation Model β Training CAPTOR was trained on the proprietary dataset with the contrastive InfoNCE loss, pairing region- centered crops with the Findings part of the reports. Batches comprise studies, with the anatomical region rotating each batch (Fig. 1B). Shuffling between epochs ensures all crops of each study are seen dur- ing training. This strategy allows the model to learn representations from complete, including multi-series, CAP examinations, without modifying the underly- ing architecture (hyperparameters in Appendix A.5). 2.5Evaluation β Medical condition classi- fication We selected 26 medical conditions relevant to onco- logic CAP CT interpretation from an initial list of 178, by severity and utility as detection targets. Reference labels were built from radiology reports 3 using Mistral Large 3 [25], a method shown to achieve high agreement with expert annotations and provide useful supervision at scale [29,30]. To verify ex- traction quality, we compared performance against GPT-5.4 [31] and GPT-4o mini [32] on CT-RATEβs test set using the original labels and we additionally manually reviewed 25 randomly selected reports. For each condition, an XGBoost classifier was trained on embeddings from the relevant anatomical crop (Fig. 1.D). Performance was assessed on subsets stratified by anatomical coverage (Chest, AP or CAP) and indication (oncological vs. non-oncological), using the macro-AUC (unweighted mean of per- condition area under the receiver operating char- acteristic curve), restricted to conditions relevant to each subset. Generalizability to non-oncologic data was eval- uated on CT-RATE [11] and RAD-ChestCT [33], both chest CT datasets, using the original labels for condition classification. Additional details in Appendices A.7-A.9. 2.6 Evaluation β Survival prediction benchmark We also evaluated all models on four two-year sur- vival prediction tasks from TumorImagingBench [18], using each modelβs own preprocessing. Following TumorImagingBenchβs method, kNN classifiers (k op- timized in 1-50) were fit per model and dataset over 10 random splits (Fig. 1.E). 2.7 Evaluation β Retrieval Retrieval tasks evaluate embedding-space quality by measuring whether semantically similar images and reports are mapped close together. Zero-shot text- image retrieval was evaluated using Recall@K, the proportion of text queries whose matching CT exam- ination was retrieved among the top K images, with K the number of retrieved items considered. Zero- shot image-image retrieval was evaluated using Mean Average Precision at K (MAP@K) [34], rewarding early retrieval of images sharing at least one positive condition label (Fig. 1.F and full metrics definitions in Appendix A.10). Fold-increase over the random baseline was used for both metrics, better accounting for varying subset and K difficulty than raw scores. SPECTRE [13] could not be evaluated (unavailable projection-layer weights), and CT-FM [17], a vision- only model, was excluded from text-image retrieval. 2.8Evaluation - Impact of training data volume To assess the effect of larger, more heterogeneous, less clinically aligned training data, CAPTOR-100K was trained on the complete training set (internal and public combined) and compared with CAPTOR (in- ternal only) and baseline models. To study training data volumeβs impact on classification performance, we trained the classifiers with nested subsets of the whole training set (10%, 25%, 50% and 75%), gener- ated using iterative multilabel stratified sampling. 2.9 Statistical analysis We compared CAPTOR with five state-of-the art foundation models : Merlin [12], CT-CLIP [11], SPECTRE [13], COLIPRI [14] and CT-FM [17] (Ap- pendix A.6, Table 4). For all tasks, 95% confidence in- tervals were estimated with a patient-level bootstrap test (2,000 resamples; 1,000 for retrieval). For clas- sification, per-condition pairwise comparisons were performed using the DeLong test. Otherwise, pair- wise comparison significance was assessed with a patient-level two-sided permutation test (2,000 per- mutations). All p-values were corrected for each task separately, using the Benjamini-Hochberg false dis- covery rate (FDR) procedure, and differences were considered significant with a corrected p-value below 0.05. All analyses were performed using Python version 3.12.12 (Python Software Foundation); Torch version 2.7.0; NumPy version 1.26.4; Pandas version 2.2.3, and scikit-learn version 1.8.0. 2.10 Code availability We release our preprocessing, training and infer- ence code at this url : https://github.com/Guerbet- AI/CAPTOR and the model weights at this url : https://huggingface.co/Guerbet-AI/CAPTOR 3 Results 3.1 Dataset characteristics Table 1 summarizes dataset demographics and Figure 2 outlines how each dataset is used for pretraining and downstream tasks. The internal dataset comprised 29,893 studies (32,689 series) from 11 providers across the United States (66%), Spain (16%), France (14%), 4 DatasetSubsetStudiesAgeSexBody RegionIndicationContrastProvenance Report Section(s) Private Internal dataset Total 29,893 studies 32,689 series 66.0 [56.0-75.0] F : 49% M : 51% 44.4% 23.7% 13.5% 8.8% 6.9% 1.8% 0.9% AP CAP Chest Unknown Abdomen CA Pelvis 70% oncologic Injected : 70% Non-injected : 16% Unknown : 14% Multi-centric from 11 providers in : United States (66%) Spain (16%) France (14%) Japan (3%) Vietnam (1%) Full report Train 25,512 studies 27,698 series 66.0 [56.0-75.0] F : 49% M : 51% 45.6% 21.9% 13.5% 8.8% 7.5% 1.8% 0.9% AP CAP Chest Unknown Abdomen CA Pelvis 73% oncologic Injected : 67% Non-injected : 19% Unknown : 13% Test 4,381 studies 4,991 series 68.0 [57.0-77.0] F : 48% M : 52% 37.0% 22% 13.8% 8.8% 3.1% 1.8% 0.8% AP CAP Chest Unknown Abdomen CA Pelvis 69% oncologic Injected : 66% Non-injected : 17% Unknown : 17% Public CT-RATE Total 21,304 studies 25,692 series 50,188 volumes 46.0 [35.0-62.0] F : 42% M : 58% ChestUnknownNon-injected Mono-centric from Istanbul Medipol University Mega Hospital Findings & Impressions Train 20,000 studies 24,128 series 47,149 volumes 45.0 [34.0-61.0] F : 42% M : 58% Test 1,304 studies 1,304 series 3,039 volumes 45.0 [34.0-60.0] F : 44% M : 56% Merlin Total 25,414 studies 55.0 [39.0-68.0] F : 56% M : 44% Abdomen & PelvisUnknownInjected Multi-centric from Stanford Research Repository Findings & Impressions Train 20,305 studies 55.0 [39.0-68.0] F : 56% M : 44% Test 5,109 studies 55.0 [40.0-68.0] F : 57% M : 43% INSPECT (CTPA) Total 23,248 studies 18-39 13% 39-69 43% 69-89 25% F : 55% M : 45% Chest Pulmonary Embolism Injected Multi-centric from Stanford Research Repository Impressions only Train 20,034 studies 18-39 13% 39-69 43% 69-89 25% F 54% M : 46% Test 3,214 studies 18-39 14% 39-69 51% 69-89 32% F 56% M : 44% BIMCV-RTotal8,069UnknownUnknown Chest UnknownUnknown Multi-centric from Valencian Region Medical ImageBank Full report Public TumorImagingBench NSCLC - Radiomics Total 421 studies 68.7 [61.2-75.9] F : 31% M : 69% Chest NSCLC patients : - pre-radiation CT - GTV expert segmentation Unknown MAASTRO Clinic, The Netherlands No reports NSCLC - Radiogenomics Total 133 studies 69.0 [64.0-76.0] F : 25% M : 75% Chest NSCLC patients : - pre-operative CT - GTV expert segmentation Unknown Stanford University School of Medicine and Palo Alto Veterans Affairs Healthcare System No reports C4KC-KiTS Total 134 studies 58.5 [50.0-68.0] F : 41% M : 59% Abdomen Renal tumor patients : - partial/radical nephrectomy - pre-operative CT - tumor/kidney segmentations Injected University of Minnesota Medical Center No reports Colorectal-Liver Metastases Total 194 studies 61.0 [52.0-68.8] F : 40% M : 60% Abdomen CRLM patients : -pre-operative CT - liver/tumor /vessels segmentation - CRLM resection Injected Memorial Sloan Kettering Cancer Center No reports Table 1: Composition of the internal and public datasets used for training and testing. Percentages are reported at the study level. Train/test sets for TumorImagingBench1 are not included, as their original training/test splits were not used: survival prediction was instead assessed via k-nearest-neighbor classifiers across 10 random splits (Materials and Methods). The median age is shown, with inter-quartile range in brackets. CTPA = CT pulmonary angiogram; F=Female, M=Male, NSCLC = non-small cell lung cancer; GTV = gross tumor volume; CRLM = colorectal liver metastases. 5 Figure 2: Flow chart showing the datasets used for training CAPTOR and for training/testing down- stream tasks. CAP= Chest-Abdomen-Pelvis, AP= Abdomen-Pelvis Japan (3%) and Vietnam (1%), with a median age of 66.0 years, 49% female. Detailed characteristics by country of the proprietary dataset in Appendix B.1. The public image-text datasets included CT- RATE [11] (21,304 chest CT studies), Merlin [12] (25,414 abdomen-pelvis CT studies), INSPECT [19] (23,248 CT pulmonary angiograms), and BIMCV- R [20] (8,069 chest CT studies). The resulting full training and testing sets including internal and public datasets contain 92,302 studies and 10,794 studies respectively. The testing set has a median age of 60 years [IQR 45-73] and was 52% female. The four TumorImagingBench [18] datasets com- prised between 134 and 421 studies each, covering lung (NSCLC-Radiomics [21]: 421 studies, NSCLC- Radiogenomics [22]: 133 studies) and abdominal oncologic cohorts (C4KC-KiTS [23]: 134 studies; Colorectal-Liver Metastases [24]: 194 studies), all without associated radiology reports. 3.2 Medical condition classification Automatic performance assessment of label extrac- tion method on CT-RATE [11], showed that GPT- 5.4 [31], Mistral Large 3 [25], and GPT-4o mini [32] achieved F1-scores of 0.917, 0.915, and 0.891 respec- tively (Appendix B.2, Figure 7). Mistral Large 3 was selected for its comparable performance for approxi- mately 13% of the inference cost. Manual review of 25 randomly selected reports identified only two extraction errors; both times it was the coronary artery disease label set to True because the report mentioned atherosclerosis. As shown in Figure 3, CAPTOR achieved the high- est overall performance, reaching an AUC of 0.849 (95%CI: 0.828-0.866) on the whole test set, signifi- cantly (p<0.001) outperforming all models except for Merlin [12] (0.845, 95% CI: 0.817 β 0.859). Across anatomical regions, CAPTOR improved over CT-CLIP [11] on the Chest subset (0.838, 95%CI: 0.789-0.865 vs. 0.732, 95%CI: 0.674 β 0.777, p<0.001), while Merlin [12] outperformed CAPTOR on the AP subset (0.807, 95%CI: 0.797 - 0.816 vs. 0.821, 95%CI: 0.812-0.829, p<0.01). On the non- oncological subset, CAPTOR was only statistically significantly outperformed by SPECTRE [13](0.859, 95%CI: 0.833 - 0.879 vs. 0.874, 95%CI: 0.848 β 0.892, p<0.01) despite its oncology-focused training. Focusing specifically on our target use case, CAP images in an oncological context, CAPTOR statisti- cally significantly outperformed all baseline models, reaching an AUC of 0.767 (95%CI: 0.734 β 0.793) on the CAP subset and 0.784 (95%CI: 0.759 β 0.805) on the oncological subset. Notably, Merlin outper- formed the generalist models on both subsets despite being trained only on AP images. Figure 4 shows the details of per-condition classification performance on the oncological subset for CAPTOR and all baselines. Per-condition proportions and significance counts are available in Appendix B.3-B.4. CAPTOR was significantly outperformed by SPECTRE and COLIPRI [14] on both external non- oncologic chest CT datasets : CAPTOR reached an AUC of 0.796 (95%CI: 0.785-0.803) on CT-RATE vs. 0.847 (95%CI: 0.836-0.854) for COLIPRI and an AUC of 0.689 (95%CI: 0.669-0.706) on RAD-ChestCT [33] vs. 0.737 (95%CI: 0.719-0.752) for COLIPRI (Ap- pendix B.7). 3.3 Survival prediction CAPTOR outperformed all baseline models on three of the four prognostic tasks, achieving AUCs of 0.640 (95% CI: 0.567β0.676) on NSCLC-Radiogenomics [22], 0.637 (95% CI: 0.523β0.697) on CACK-KiTS [23], and 0.587 (95% CI: 0.546β0.609) on NSCLC- Radiomics [21]. On Colorectal-Liver-Metastases [24], CAPTOR ranked second with an AUC of 0.521 (95% CI: 0.427β0.604), behind COLIPRI [14] (0.549, 95% 6 Figure 3: Bar plot showing the classification scores of CAPTOR and five state-of-the art foundation models on our test set, stratified by visible anatomical region and clinical indication. The number of studies in each subset is in parentheses. One XGBoost was trained for each medical condition and model using the image embedding extracted with the modelβs image encoder. For each subset and model, the macro-AUC is the unweighted mean of the AUC scores for each med- ical condition relevant to the subset among the 26 conditions studied. The confidence intervals and the pairwise comparison were obtained with a patient- level bootstrap and a patient-level permutation test respectively, both based on 2,000 iterations. AP = Abdomen-Pelvis, CAP=Chest-Abdomen-Pelvis. *** = p < 0.001, ** = p < 0.01, * = p < 0.05 CI: 0.471β0.621) (Figure 5). Although few differences are statistically significant, the overall ranking pat- tern was consistent with the primary classification benchmark. Computation times for the Colorectal-Liver- Metastases are presented in Appendix B.5. 3.4 Retrieval CAPTOR achieved the highest text-image retrieval performance on CAP (Recall@10 of 47.8, 95% CI: 44.2β51.6), oncologic (83.4, 95% CI: 78.4β88.6), and chest (64.3 , 95% CI: 51.9β84.0) subsets, while Merlin performed best on the whole test set (235, 95% CI: 205β261), abdomen-pelvis (238, 95%CI : 230β245), and non-oncologic subsets (236, 95%CI: Figure 4: Per-condition classification performance of CAPTOR and five baselines on the oncological subset, for each of the 26 medical conditions studied. One XGBoost was trained for each medical condition and model, using the appropriate image embedding (i.e. the corresponding regional crop for CAPTOR and the corresponding series for the baselines) extracted with the modelβs image encoder. 199β272). Image-image retrieval showed the same ranking, with CAPTOR achieving a MAP@10 of 1.30 (95%CI: 1.25β1.35) on the CAP subset, 1.41 (95%CI: 1.37β1.46) on the oncological subset, and 1.23 (95%CI: 1.20β1.27) on the chest subset(Tables 2). 3.5 Impact of training data volume CAPTOR-100K, trained on the full pretraining dataset (internal and public image-report datasets), demonstrated comparable performance to CAPTOR on all subsets of the classification task, the biggest differences being for the CAP subset (0.744, 95%CI: 0.703-0.779 vs. 0.767, 95%CI: 0.734-0.793 for CAP- TOR), and the oncological subset (0.771, 95%CI: 0.743-0.796 vs. 0.784, 95%CI: 0.759-0.805 for CAP- TOR), although they were not significant. CAPTOR- 100K statistically significantly outperformed CAP- TOR on external non-oncologic chest CT datasets, reaching an AUC of 0.820 (95%CI: 0.813-0.827) on CT-RATE [11] and 0.719 (95%CI: 0.698-0.735) on RAD-ChestCT [33], while still outperformed by SPECTRE [13] and COLIPRI [14], however not sta- tistically significantly (Appendix B.7). 7 Retrieval text-imageRetrieval image-image DatasetK CAPTOR (OURS) Merlin CT-CLIP COLIPRI CAPTOR (OURS) Merlin CT-CLIP COLIPRI CTFM Whole test set 10,794 1 307 (266β348) 722 (622β817) 20.0 (8.88β31.1) 188 (104β281) 1.78 (1.72β1.82) 1.82 (1.75β1.85) 1.35 (1.27β1.40) 1.63 (1.54β1.68) 1.40 (1.33β1.46) 5 179 (159β199) 353 (307β394) 12.2 (6.60β18.7) 98.4 (57.8β144) 1.44 (1.39β1.45) 1.46 (1.41β1.46) 1.20 (1.12β1.20) 1.35 (1.28β1.36) 1.23 (1.16β1.24) 10 139 (125β153) 235 (205β261) 11.3 (6.71β16.6) 70.1 (42.6β101) 1.46 (1.44β1.50) 1.48 (1.45β1.51) 1.20 (1.16β1.24) 1.35 (1.31β1.40) 1.23 (1.20β1.28) 50 65.8 (60.9β70.4) 77.1 (67.8β84.8) 8.02 (4.97β11.8) 30.7 (21.7β40.5) 1.59 (1.57β1.64) 1.60 (1.58β1.65) 1.24 (1.21β1.31) 1.42 (1.39β1.49) 1.28 (1.26β1.35) 100 43.1 (40.2β45.8) 45.9 (40.9β50.2) 6.62 (4.24β9.54) 21.0 (16.0β26.5) 1.61 (1.59β1.66) 1.63 (1.60β1.68) 1.24 (1.21β1.31) 1.43 (1.40β1.50) 1.29 (1.26β1.36) Chest 3,944 1 145 (111β197) 23.0 (14.0β36.1) 22.0 (10.9β30.4) 184 (111β231) 1.33 (1.28β1.37) 1.27 (1.22β1.31) 1.17 (1.11β1.22) 1.29 (1.22β1.34) 1.18 (1.11β1.23) 5 85.2 (67.9β113) 19.0 (14.2β26.1) 12.8 (7.35β16.7) 92.0 (58.4β114) 1.22 (1.18β1.25) 1.18 (1.14β1.20) 1.11 (1.06β1.14) 1.19 (1.14β1.22) 1.12 (1.07β1.14) 10 64.3 (51.9β84.0) 15.3 (11.9β20.0) 11.6 (7.04β15.0) 63.3 (41.1β78.0) 1.23 (1.20β1.27) 1.19 (1.16β1.23) 1.11 (1.08β1.16) 1.19 (1.15β1.23) 1.12 (1.08β1.16) 50 28.6 (24.4β35.4) 9.10 (7.79β11.1) 8.08 (5.50β9.92) 25.7 (18.8β30.0) 1.26 (1.23β1.30) 1.23 (1.20β1.28) 1.12 (1.08β1.18) 1.18 (1.14β1.24) 1.13 (1.09β1.18) 100 18.8 (16.2β22.9) 6.71 (5.86β7.90) 6.58 (4.65β7.93) 16.8 (13.3β19.1) 1.25 (1.21β1.29) 1.22 (1.18β1.27) 1.10 (1.06β1.16) 1.16 (1.11β1.22) 1.11 (1.06β1.16) AP 8,731 1 270 (241β305) 733 (685β782) 6.00 (2.00β11.0) 29.0 (19.0β39.9) 1.72 (1.66β1.76) 1.78 (1.71β1.80) 1.25 (1.18β1.29) 1.53 (1.46β1.56) 1.28 (1.23β1.33) 5 157 (148β168) 357 (343β372) 4.40 (2.60β6.25) 22.0 (17.8β26.1) 1.41 (1.36β1.42) 1.43 (1.38β1.44) 1.14 (1.07β1.14) 1.29 (1.23β1.29) 1.17 (1.10β1.17) 10 122 (116β129) 238 (230β245) 4.00 (2.80β5.23) 18.2 (15.6β20.7) 1.43 (1.40β1.46) 1.45 (1.43β1.48) 1.14 (1.11β1.17) 1.30 (1.27β1.33) 1.17 (1.14β1.20) 50 56.6 (54.8β58.2) 78.0 (76.2β79.8) 2.92 (2.40β3.42) 12.8 (11.8β13.7) 1.54 (1.53β1.59) 1.56 (1.54β1.61) 1.16 (1.15β1.21) 1.35 (1.34β1.40) 1.20 (1.19β1.25) 100 36.9 (36.0β37.8) 46.2 (45.3β47.1) 2.74 (2.41β3.07) 9.94 (9.36β10.5) 1.56 (1.54β1.61) 1.59 (1.56β1.63) 1.17 (1.16β1.22) 1.36 (1.34β1.41) 1.20 (1.19β1.25) CAP 1,881 1 109 (90.3β129) 25.0 (15.9β34.9) 6.00 (1.99β11.2) 24.0 (14.6β34.3) 1.49 (1.40β1.56) 1.37 (1.30β1.46) 1.24 (1.15β1.32) 1.36 (1.27β1.43) 1.22 (1.13β1.31) 5 62.4 (56.1β68.9) 19.0 (15.3β22.6) 3.60 (2.02β5.44) 13.6 (10.5β17.0) 1.28 (1.22β1.32) 1.22 (1.16β1.26) 1.13 (1.06β1.16) 1.21 (1.14β1.24) 1.12 (1.05β1.16) 10 47.8 (44.2β51.6) 15.1 (12.8β17.4) 3.30 (2.24β4.54) 10.5 (8.55β12.5) 1.30 (1.25β1.35) 1.24 (1.19β1.29) 1.13 (1.09β1.19) 1.20 (1.16β1.26) 1.13 (1.09β1.19) 50 19.8 (18.9β20.6) 8.14 (7.42β8.86) 2.48 (2.04β2.94) 6.98 (6.32β7.62) 1.33 (1.29β1.40) 1.27 (1.23β1.34) 1.12 (1.08β1.20) 1.19 (1.15β1.26) 1.14 (1.10β1.21) 100 12.6 (12.2β13.0) 5.62 (5.20β5.99) 2.14 (1.85β2.44) 5.16 (4.79β5.56) 1.33 (1.28β1.39) 1.27 (1.23β1.34) 1.12 (1.07β1.19) 1.18 (1.13β1.25) 1.12 (1.08β1.20) Oncological 3,264 1 224 (195β253) 38.0 (26.1β51.2) 6.00 (1.99β11.0) 41.0 (29.1β53.9) 1.65 (1.56β1.70) 1.47 (1.40β1.55) 1.29 (1.20β1.36) 1.37 (1.29β1.44) 1.29 (1.21β1.36) 5 115 (106β125) 27.6 (23.1β32.2) 6.20 (4.19β8.43) 22.8 (18.7β27.1) 1.40 (1.34β1.42) 1.29 (1.23β1.32) 1.17 (1.10β1.19) 1.23 (1.16β1.25) 1.19 (1.11β1.21) 10 83.4 (78.4β88.6) 21.5 (18.8β24.2) 4.60 (3.32β5.90) 16.9 (14.4β19.5) 1.41 (1.37β1.46) 1.30 (1.26β1.35) 1.17 (1.12β1.22) 1.23 (1.18β1.27) 1.19 (1.14β1.24) 50 33.5 (32.4β34.6) 11.2 (10.3β12.0) 3.28 (2.79β3.78) 9.94 (9.16β10.7) 1.48 (1.45β1.55) 1.38 (1.34β1.44) 1.18 (1.15β1.25) 1.23 (1.20β1.30) 1.21 (1.18β1.27) 100 21.0 (20.5β21.5) 8.09 (7.61β8.60) 2.90 (2.56β3.23) 7.38 (6.93β7.88) 1.49 (1.45β1.55) 1.38 (1.34β1.44) 1.17 (1.13β1.24) 1.21 (1.17β1.28) 1.20 (1.16β1.26) Non-oncological 7,530 1 190 (160β224) 745 (619β880) 22.0 (7.78β36.1) 191 (93.7β288) 1.69 (1.61β1.74) 1.80 (1.72β1.83) 1.27 (1.17β1.35) 1.60 (1.51β1.65) 1.33 (1.23β1.40) 5 116 (102β132) 365 (308β423) 14.4 (6.30β22.3) 99.0 (53.6β142) 1.38 (1.31β1.39) 1.44 (1.38β1.45) 1.15 (1.06β1.17) 1.32 (1.25β1.34) 1.18 (1.10β1.20) 10 90.3 (80.5β101) 236 (199β272) 11.9 (5.55β18.2) 69.3 (38.8β98.1) 1.41 (1.37β1.45) 1.47 (1.44β1.50) 1.16 (1.11β1.21) 1.35 (1.30β1.39) 1.19 (1.14β1.25) 50 43.0 (39.7β46.7) 74.2 (64.0β84.8) 8.24 (4.10β12.4) 27.6 (18.7β36.5) 1.52 (1.48β1.58) 1.58 (1.55β1.63) 1.18 (1.14β1.27) 1.41 (1.36β1.48) 1.23 (1.18β1.31) 100 28.7 (27.0β30.6) 42.8 (37.2β48.5) 6.41 (3.27β9.48) 18.7 (13.6β23.5) 1.54 (1.49β1.60) 1.59 (1.55β1.64) 1.19 (1.14β1.27) 1.42 (1.36β1.49) 1.23 (1.18β1.31) Table 2: Zero-shot text-image (Recall@K) and image-image (MAP@K) retrieval performance of CAPTOR and baseline models, expressed as fold-increase over the random baseline (best in bold and second best underlined) with 95% confidence intervals in parentheses, by anatomical and oncologic subset. K is the number of retrieved images. SPECTRE [13] could not be evaluated (unavailable projection-layer weights) and CTFM [17], a vison model, was excluded from text-image retrieval. 8 Figure 5: Two-year survival-prediction performance (AUC) of CAPTOR and five baseline models on four oncologic TumorImagingBench1 cohorts (NSCLC- Radiomics21, NSCLC-Radiogenomics22, C4KC- KiTS23, Colorectal-Liver-Metastases24), using kNN classifiers on frozen image embeddings, averaged over 10 random splits. Statistical significance was assessed using a permutation test (2,000 permutations). The number of studies for each dataset is indicated be- tween parentheses. ** = p < 0.01, * = p < 0.05 CAPTOR outperformed CAPTOR-100K on the two abdomen datasets of the four oncological test cohorts while CAPTOR-100K demonstrated supe- rior performance on the two chest CT datasets (CAPTOR-100K AUC : 0.616, 95% CI: 0.581-0.642 on NSCLC-Radiomics [21] and 0.652, 95% CI: 0.583- 0.689 on NSCLC-Radiogenomics [22]), though none of the differences between the models were statisti- cally significant. For retrieval, CAPTOR-100K outperformed all baseline models on the whole test set, Chest, AP and non-oncological subsets, reaching a Recall@10 for the whole test set of 310 (95% CI:302β319) and MAP@10 of 1.50 (95% CI: 1.47-1.54), but was out- performed by CAPTOR on the CAP and oncologic subsets. Table 3 summarizes the performance differ- ences between CAPTOR and CAPTOR-100K across all downstream tasks. Detailed figures of CAPTOR- 100K performances compared to baselines are in Ap- pendix B.6 We further evaluated the effect of training data volume on classifier performance by varying the frac- tion of the training set used during XGBoost training (Figure 6). Increasing the amount of training data generally leads to improved performance, although the magnitude and consistency of these gains vary Datasets Task SubsetCAPTORCAPTOR-100K Whole 0.849 (0.828β0.866)0.846 (0.824β0.862) Chest 0.838 (0.789β0.865)0.828 (0.778β0.860) Global test set Medical condition classification AP 0.807 (0.797β0.816)0.808 (0.798β0.818) CAP 0.767 (0.734β0.793)0.744 (0.703β0.779) Oncological 0.784 (0.759β0.805)0.771 (0.743β0.796) Non-oncological 0.859 (0.833β0.879)0.856 (0.831β0.874) NSCLC-Radiomics 0.587 (0.546β0.609)0.616 (0.581β0.642) Oncological test sets Two-year survival prediction NSCLC- Radiogenomics 0.640 (0.567β0.676)0.652 (0.583β0.689) C4KC-KiTS 0.637 (0.523β0.697)0.556 (0.447β0.633) Colorectal-Liver Metastases 0.521 (0.427β0.604)0.482 (0.403β0.576) Whole 139 (125β153)310 (302β319) Chest 64.3 (51.9β84.0)95.2 (86.2β101) Global test set Text-Image retrieval AP 122 (116β129)254 (245β262) CAP 47.8 (44.2β51.6)38.6 (35.3β41.9) Oncological 83.4 (78.4β88.6)71.7 (67.3β76.6) Non-oncological 90.3 (80.5β101)265 (255β276) Whole 1.46 (1.44β1.50)1.50 (1.47β1.54) Chest 1.23 (1.20β1.27)1.24 (1.21β1.28) Global test set Image-Image retrieval AP 1.43 (1.40β1.46)1.46 (1.44β1.49) CAP 1.30 (1.25β1.35)1.28 (1.23β1.33) Oncological 1.41 (1.37β1.46)1.39 (1.35β1.44) Non-oncological 1.41 (1.37β1.45)1.47 (1.42β1.51) Table 3: Effect of training-data volume on CAP- TOR performance. Comparison of CAPTOR and CAPTOR-100K performances across all downstream tasks with 95%confidence intervals in parentheses. Retrieval results are shown for K=10, with K the number of retrieved images considered. across datasets. CAPTOR consistently achieves the highest mean AUC across all training fractions com- pared to baselines on the whole test set, the CAP and the oncological subsets, demonstrating strong robustness in low-data settings. 4 Discussion In this multicenter study comparing six CT vision- language foundation models across classification, prognostic benchmarks and retrieval, CAPTOR achieved the highest performance on CAP and on- cologic CT subsets. The principal finding of this study is that a foundation model trained on clini- cally representative oncologic CAP CT examinations achieved superior performance in the intended de- ployment setting compared with both region-specific and generalist CT foundation models, supporting the hypothesis that alignment between training data and clinical use case is a major determinant of foundation- model performance. A key architectural contribution enabling this per- formance is its region-centered preprocessing strat- egy, in which TotalSegmentator [28] masks were used to automatically crop anatomical sub-regions prior 9 Figure 6: Classification performance (macro-AUC) as a function of labeled training-set fraction (10-100%) for CAPTOR, CAPTOR-100K, and five baseline models, on (left) the whole test set, (middle) the CAP subset, and (right) the oncologic subset. to encoding, allowing CAPTOR to generate inde- pendent embeddings for each sub-region, preserv- ing region-specific representations while enabling full CAP coverage. This allowed CAPTOR to process full CAP vol- umes within the same framework as region-specific models such as Merlin [12], without sacrificing the whole-examination context that oncologic interpre- tation requires, likely contributing to CAPTORβs advantage on CAP and oncologic subsets. An important finding was that CAPTOR achieved performance comparable to, and in several analy- ses slightly exceeding, CAPTOR-100K despite being trained on substantially fewer examinations. CAP- TOR performed best on the intended oncologic CAP population, whereas CAPTOR-100K showed advan- tages on non-oncologic external datasets and retrieval. These findings indicate that distribution differences between pretraining and downstream populations (data drift) also matter for foundation models, high- lighting a trade-off between specialization and gener- alizability rather than a simple scaling effect. The retrieval experiments provide complementary evidence that the improvements observed in classifi- cation are attributable to stronger image represen- tations rather than solely to downstream classifier optimization. CAPTOR demonstrated consistently strong image-image and text-image retrieval perfor- mance, particularly on oncologic and CAP exami- nations, indicating improved cross-modal alignment between CT images and radiology reports. This study has several limitations. First, this is a retrospective study, with no prospective validation. Second, reference labels were automatically extracted from radiology reports using a large language model rather than manual expert annotation. Extraction performance was evaluated on the public dataset CT- RATE [11], and a random sample of 25 reports was manually reviewed to assess extraction quality, al- though, omissions in the original reports and LLM errors may have introduced some degree of label noise at scale. Third, CAPTOR is a case-level foundation model: it aggregates information across large anatom- ical regions into a single study-level embedding, en- abling efficient inference and strong examination-level performance, but is not designed for lesion-level lo- calization or characterization. Architectures such as SPECTRE [13], which preserve fine-grained spatial information, remain better suited to lesion-level tasks, and our results should be interpreted within the con- text of global examination understanding rather than lesion analysis. Fourth, several external oncologic datasets had small sample sizes, limiting statistical power for pairwise comparisons; although CAPTOR generally outperformed competing methods on these datasets, larger multicenter cohorts are needed to confirm these findings. Finally, CAPTORβs generaliz- ability beyond its intended oncologic CAP CT setting remains limited: on the non-oncologic CT-RATE and RAD-Chest CT [33] datasets, it was outperformed by generalist models such as SPECTRE and COL- IPRI [14], consistent with its oncology-enriched train- ing distribution. While CAPTOR remained compet- itive despite this domain shift, specialization for a targeted clinical use case may come at the expense of performance on substantially different imaging pop- ulations, and future work should explore strategies that better balance specialization and generalizabil- ity. In conclusion, CAPTOR demonstrates that a vision-language foundation model trained on a clini- cally targeted, oncology-enriched CAP CT dataset 10 can outperform both region-specific and generalist models on its intended use case, without requiring large-scale heterogeneous data. These results suggest that dataset composition and clinical alignment may be at least as important as scale in CT foundation model development. To support reproducibility and community adoption, the training code and model weights will be made publicly available. Future work should explore architectures and training strategies that preserve specialization gains while extending generalizability across broader clinical use cases. 5 Acknowledgments The authors used OpenAIβs ChatGPT (version 5) and Claude Sonnet 4.6 to assist with grammar and language when writing this article. This work was performed using HPC resources from GENCIβIDRIS (2026-103370). References [1] Global Cancer Observatory. Estimated number of new cases from 2022 to 2045, Both sexes, age [0-85+], February 2024. URLhttps://gco.ia rc.who.int/tomorrow/en/dataviz/isotype. [2]Kenneth D. Hopper, Kishor Singapuri, and Arkady Finkel. Body CT and Oncologic Imag- ing. Radiology, 2000. doi: 10.1148/radiology.21 5.1.r00ap1727. URLhttps://sci-hub.fr/10 .1148/radiology.215.1.r00ap1727. [3]ASCO GUIDELINES Bundle - Lung Cancer Surveillance After Definitive Curative-Intent Therapy, . URLhttps://eguideline.gui delinecentral.com/i/1475471-lung-cance r-surveillance-after-definitive-curat ive-intent-therapy/1? [4] A. Zer, M.-J. Ahn, F. Barlesi, L. Bubendorf, D. De Ruysscher, P. Garrido, O. Gautschi, L.E. Hendriks, P.A. JΓ€nne, K.M. Kerr, C. Mascaux, T. Mitsudomi, S. Peters, C. Rolfo, A. Sacher, S. Senan, P. Ugalde, and N.B. Leighl. Early and locally advanced non-small-cell lung cancer: ESMO Clinical Practice Guideline for diagnosis, treatment and follow-up. Annals of Oncology, 36 (11):1245β1262, November 2025. ISSN 09237534. doi: 10.1016/j.annonc.2025.08.003. URLhttps: //linkinghub.elsevier.com/retrieve/pii /S0923753425009238. [5] R. Glynne-Jones, L. Wyrwicz, E. Tiret, G. Brown, C. RΓΆdel, A. Cervantes, and D. Arnold. Rectal cancer: ESMO Clinical Prac- tice Guidelines for diagnosis, treatment and follow-upβ . Annals of Oncology, 28:iv22βiv40, July 2017. ISSN 0923-7534. doi: 10.1093/anno nc/mdx224. URLhttps://w.sciencedirec t.com/science/article/pii/S09237534194 21522. [6]R. J. M. Bruls and R. M. Kwee. Workload for radiologists during on-call hours: dramatic increase in the past 15 years. Insights into Imaging, 11:121, November 2020. ISSN 1869- 4101. doi: 10.1186/s13244-020-00925-z. URL https://pmc.ncbi.nlm.nih.gov/articles/ PMC7683675/. [7] Mina S. Makary, MD and Noah Takacs, BS. Are We Prepared for a Looming Radiologist Shortage? URLhttps://w.diagnosticim aging.com/view/are-we-prepared-for-a-l ooming-radiologist-shortage-. [8] Anika G. Patel, Victor J. Pizzitola, C. Daniel Johnson, Nan Zhang, and Maitray D. Patel. Ra- diologists Make More Errors Interpreting Off- Hours Body CT Studies during Overnight As- signments as Compared with Daytime Assign- ments. Radiology, 297(2):374β379, November 2020. ISSN 0033-8419. doi: 10.1148/radiol.202 0201558. URLhttps://pubs.rsna.org/doi/ 10.1148/radiol.2020201558. [9]Emanuele Neri, Nandita de Souza, Adrian Brady, Angel Alberich Bayarri, Christoph D. Becker, Francesca Coppola, Jacob Visser, and Euro- pean Society of Radiology (ESR). What the radiologist should know about artificial intel- ligence β an ESR white paper. Insights into Imaging, 10(1):44, April 2019. ISSN 1869-4101. doi: 10.1186/s13244-019-0738-2. URLhttps: //doi.org/10.1186/s13244-019-0738-2. [10] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision. In 11 Proceedings of the 38th International Conference on Machine Learning, pages 8748β8763. PMLR, July 2021. URLhttps://proceedings.mlr. press/v139/radford21a.html. [11]Ibrahim Ethem Hamamci, Sezgin Er, Chenyu Wang, Furkan Almas, Ayse Gulnihan Simsek, Sevval Nil Esirgun, Irem Dogan, Omer Faruk Durugol, Benjamin Hou, Suprosanna Shit, Weicheng Dai, Murong Xu, Hadrien Rey- naud, Muhammed Furkan Dasdelen, Bastian Wittmann, Tamaz Amiranashvili, Enis Simsar, Mehmet Simsar, Emine Bensu Erdemir, Ab- dullah Alanbay, Anjany Sekuboyina, Berkan Lafci, Ahmet Kaplan, Zhiyong Lu, Malgorzata Polacin, Bernhard Kainz, Christian Blueth- gen, Kayhan Batmanghelich, Mehmet Kemal Ozdemir, and Bjoern Menze. Generalist foun- dation models from a multimodal dataset for 3D computed tomography. Nature Biomedi- cal Engineering, February 2026. ISSN 2157- 846X. doi: 10.1038/s41551-025-01599-y. URL https://w.nature.com/articles/s41551 -025-01599-y. [12]Louis Blankemeier, Ashwin Kumar, Joseph Paul Cohen, Jiaming Liu, Longchao Liu, Dave Van Veen, Syed Jamal Safdar Gardezi, Hongkun Yu, Magdalini Paschali, Zhihong Chen, Jean- Benoit Delbrouck, Eduardo Reis, Robbie Hol- land, Cesar Truyts, Christian Bluethgen, Yufu Wu, Long Lian, Malte Engmann Kjeldskov Jensen, Sophie Ostmeier, Maya Varma, Jeya Maria Jose Valanarasu, Zhongnan Fang, Zepeng Huo, Zaid Nabulsi, Diego Ardila, Wei-Hung Weng, Edson Amaro Junior, Neera Ahuja, Jason Fries, Nigam H. Shah, Greg Zaharchuk, Marc Willis, Adam Yala, Andrew Johnston, Robert D. Boutin, Andrew Wentland, Curtis P. Langlotz, Jason Hom, Sergios Gatidis, and Akshay S. Chaudhari. Merlin: a computed tomography visionβlanguage foundation model and dataset. Nature, pages 1β11, March 2026. ISSN 1476- 4687. doi: 10.1038/s41586-026-10181-8. URL https://w.nature.com/articles/s41586 -026-10181-8. [13] Cris Claessens, Christiaan Viviers, Giacomo DβAmicantonio, Egor Bondarev, and Fons van der Sommen. Scaling Self-Supervised and Cross-Modal Pretraining for Volumetric CT Transformers, November 2025. URLhttp:// arxiv.org/abs/2511.17209. arXiv:2511.17209 [cs]. [14] Tassilo Wald, Ibrahim Ethem Hamamci, Yuan Gao, Sam Bond-Taylor, Harshita Sharma, Maxi- milian Ilse, Cynthia Lo, Olesya Melnichenko, Anton Schwaighofer, Noel C. F. Codella, Maria Teodora Wetscherek, Klaus H. Maier- Hein, Panagiotis Korfiatis, Valentina Salvatelli, Javier Alvarez-Valle, and Fernando PΓ©rez-GarcΓa. Comprehensive language-image pre-training for 3D medical image understanding, January 2026. URLhttp://arxiv.org/abs/2510.15042. arXiv:2510.15042 [cs]. [15] Jing Li, Jing Fu, Xin Geng, and Hongyang Wang. Foundation models in clinical oncology: Progresses and perspectives. Cancer Letters, 639:218220, February 2026. ISSN 0304-3835. doi: 10.1016/j.canlet.2025.218220. URL https://w.sciencedirect.com/scienc e/article/pii/S030438352500792X. [16]Jinxi Xiang, Xiyue Wang, Xiaoming Zhang, Yinghua Xi, Feyisope Eweje, Yijiang Chen, Yuchen Li, Colin Bergstrom, Matthew Gopaulchan, Ted Kim, Kun-Hsing Yu, Sierra Willens, Francesca Maria Olguin, Jeffrey J. Nirschl, Joel Neal, Maximilian Diehn, Sen Yang, and Ruijiang Li. A visionβlanguage foundation model for precision oncology. Nature, 638 (8051):769β778, February 2025. ISSN 1476-4687. doi: 10.1038/s41586-024-08378-w. URL https://w.nature.com/articles/s41586 -024-08378-w. [17] Suraj Pai, Ibrahim Hadzic, Dennis Bontempi, Keno Bressem, Benjamin H. Kann, Andriy Fe- dorov, Raymond H. Mak, and Hugo J. W. L. Aerts. Vision Foundation Models for Computed Tomography, 2025. URLhttps://arxiv.org/ abs/2501.09001. Version Number: 2. [18]Hugo Aerts, Suraj Pai, Ibrahim Hadzic, An- drey Fedorov, and Raymond Mak. Foundation model embeddings for quantitative tumor imag- ing biomarkers, May 2025. ISSN: 2693-5015. [19]Shih-Cheng Huang, Zepeng Huo, Ethan Stein- berg, Chia-Chun Chiang, Matthew P Lungren, Curtis P Langlotz, Serena Yeung, Nigam H Shah, and Jason A Fries. INSPECT: A Multimodal 12 Dataset for Pulmonary Embolism Diagnosis and Prognosis. [20]Yinda Chen, Che Liu, Xiaoyu Liu, Rossella Ar- cucci, and Zhiwei Xiong. BIMCV-R: A Land- mark Dataset for 3D CT Text-Image Retrieval, 2024. URLhttps://arxiv.org/abs/2403.1 5992. Version Number: 2. [21] NSCLC-RADIOMICS, . URLhttps://w.ca ncerimagingarchive.net/collection/nscl c-radiomics/. [22]NSCLC-RADIOGENOMICS, . URLhttps: //w.cancerimagingarchive.net/collect ion/nsclc-radiogenomics/. [23] C4KC-KITS, . URLhttps://w.cancerimag ingarchive.net/collection/c4kc-kits/. [24]COLORECTAL-LIVER-METASTASES, . URL https://w.cancerimagingarchive.net/c ollection/colorectal-liver-metastases/. [25]Mistral Large 2.1 - Mistral AI, . URLhttps: //docs.mistral.ai/models/mistral-large -2-1-24-11. [26]Yikuan Li, Ramsey M. Wehbe, Faraz S. Ah- mad, Hanyin Wang, and Yuan Luo. Clinical- Longformer and Clinical-BigBird: Transformers for long clinical sequences, January 2022. URL https://arxiv.org/abs/2201.11838v3. [27]JoΓ£o Carreira and Andrew Zisserman. Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. In 2017 IEEE Confer- ence on Computer Vision and Pattern Recog- nition (CVPR), pages 4724β4733, July 2017. doi: 10.1109/CVPR.2017.502. URLhttps: //ieeexplore.ieee.org/document/8099985. ISSN: 1063-6919. [28] Jakob Wasserthal, Hanns-Christian Breit, Man- fred T. Meyer, Maurice Pradella, Daniel Hinck, Alexander W. Sauter, Tobias Heye, Daniel T. Boll, Joshy Cyriac, Shan Yang, Michael Bach, and Martin Segeroth. TotalSegmentator: Ro- bust Segmentation of 104 Anatomic Structures in CT Images. Radiology: Artificial Intelligence, 5(5):e230024, September 2023. ISSN 2638-6100. doi: 10.1148/ryai.230024. URLhttp://pubs.r sna.org/doi/10.1148/ryai.230024. [29]Adam E. Flanders, Xindi Wang, Carol C. Wu, Felipe C. Kitamura, George Shih, John Mongan, and Yifan Peng. The Evolution of Radiology Image Annotation in the Era of Large Language Models. Radiology: Artificial Intelligence, 7(4): e240631, July 2025. doi: 10.1148/ryai.240631. URLhttps://pubs.rsna.org/doi/10.1148/ ryai.240631. [30]Felix J. Dorfner, Liv JΓΌrgensen, Leonhard Donle, Fares Al Mohamad, Tobias R. Boden- mann, Mason C. Cleveland, Felix Busch, Lisa C. Adams, James Sato, Thomas Schultz, Albert E. Kim, Jameson Merkow, Keno K. Bressem, and Christopher P. Bridge. Comparing Commercial and Open-Source Large Language Models for Labeling Chest Radiograph Reports. Radiology, 313(1):e241139, October 2024. ISSN 0033-8419. doi: 10.1148/radiol.241139. URLhttps://pu bs.rsna.org/doi/10.1148/radiol.241139. [31] Introducing GPT-5.4, July 2026. URLhttps:// openai.com/index/introducing-gpt-5-4/. [32]GPT-4o mini: advancing cost-efficient intelli- gence, . URLhttps://openai.com/index/gpt -4o-mini-advancing-cost-efficient-intel ligence/. [33]Rachel Lea Draelos, David Dov, Maciej A Mazurowski, Joseph Y. Lo, Ricardo Henao, Ge- offrey D. Rubin, and Lawrence Carin. RAD- ChestCT Dataset, October 2020. URLhttps: //zenodo.org/record/6406114. [34]Tetiana Manzhos, Tetiana Ianevych, and Olga Melnyk. Average precision at cutoff k under ran- dom rankings: expectation and variance. Mod- ern Stochastics: Theory and Applications, pages 1β18, April 2026. ISSN 2351-6046, 2351-6054. doi: 10.15559/26-VMSTA298. URLhttps://w w.vmsta.org/journal/VMSTA/article/353. [35]An Yan, Julian McAuley, Xing Lu, Jiang Du, Eric Y. Chang, Amilcare Gentili, and Chun- Nan Hsu. RadBERT: Adapting Transformer- based Language Models to Radiology. Radiology: Artificial Intelligence, 4(4):e210258, July 2022. doi: 10.1148/ryai.210258. URLhttps://pubs .rsna.org/doi/10.1148/ryai.210258. 13 A Supplementary methods A.1 Internal dataset For each study, series were selected via a priority rule favoring full-coverage and contrast-enhanced acquisi- tions. The series selection priority is the following: CAP > Chest + AP > CA + Pelvis > Chest + Ab- domen + Pelvis > Unknown > AP > CA > Chest or Abdomen or Pelvis. If several series correspond to a category (several CAP series for example), we select the series based on the phase and kernel type : Venous + Smooth > Unenhanced + Smooth > De- layed + Smooth > Arterial + Smooth > Unknown + Sharp > Unknown + Smooth > Venous + Sharp > First available series. A.2 External dataset series selection cri- teria INSPECT was excluded from the testing set because the reports only include the Impressions section which is not detailed enough to extract reliable labels, and BIMCVR was excluded from the test set due to its limited size and lack of descriptive information. A.3 More information on oncological datasets The four public oncological datasets we used among those in TumorImagingBench are the following : - NSCLC-Radiomics [21] : CT scans from 421 non-small cell lung cancer (NSCLC) patients, who re- ceived radiation therapy or chemo-radiation therapy. Pretreatment CT scans, expert segmentation of Gross Tumor Volume (GTV) and clinical outcome data are available for each patient. The median survival time of this study is 17.9 months. - NSCLC-Radiogenomics [22] : Cohort of 211 NSCLC patients with preoperative CT and PET/CT scans. We used images from the 133 patients with GTV annotated like TumorImagingBench. The me- dian survival time of these patients is 44.5 months. - C4KC-KiTS [23] : CT scans and tumor and kidney segmentations from renal tumor patients with partial or radical nephrectomy. Of the 210 patients in the dataset, 134 remain after filtering for survival data at 2 years. The median survival time of these patients is 15.5 months. - Colorectal-Liver-Metastases [24] : Preoperative CT scans and liver/tumor/vessels segmentation from 194 patients who underwent colorectal liver metas- tases (CRLM) resection. The median survival time of this study is 44.5 months. We did not use the two other datasets originally present in TumorImagingBench because they are lesion-level, which is not our aim with CAPTOR. A.4 More information on preprocessing CT volumes were reformatted to RAS orientation, resampled to isotropic in-plane resolution (1.5 m) and 3-m slice spacing, intensity-normalized, padded, and cropped to a fixed input size of 224x224x160 voxels. The full list of regions we used is the same as Merlin: chest, liver and biliary tree, gallbladder, spleen, pancreas, adrenal glands, kidneys and ureters, gastrointestinal tract, peritoneal cavity, pelvic organs, vasculature and lymph nodes, musculoskeletal Regarding report preprocessing, reports were auto- matically checked to make sure they did not contain unreadable characters. 94% of reports were acquired already in English (originally in English or already translated); a smaller subset (1,805 studies, 6% of the dataset) remained in Spanish and was used as-is for training, without separate handling. In the test set, Spanish reports represented a smaller share (2%) and were all excluded from all analyses. A.5 Training hyperparameters We used an Adam optimizer with 1.25e-6 learning rate,Ξ²= (0.9,0.99) and used gradient checkpointing for both the visual and text encoders. We used a batch size of 18 on a single H100 GPU. A.6 Baseline Models We compared our model with five state-of-the art foundation models. Two of them were trained on region-specific CT datasets: - MERLIN [12]: 3D contrastive vision-language model based on an inflated ResNet-152 image en- coder and a Clinical Longformer text encoder. It was trained on approximately 25,000 abdomen-pelvis CT-report pairs from the STARR repository (Merlin dataset), consisting predominantly of non-oncologic examinations. - CT-CLIP [11]: a 3D contrastive vision-language model using transformer vision and text encoders (adapted from CT-ViT for the vision encoder and 14 CXR-Bert for the text encoder), trained on the CT- RATE dataset comprising approximately 25,000 non- oncologic chest CT-report pairs. The others are trained on broader datasets: - SPECTRE [13]: a 3D Vision Transformer (ViT) pretrained using a combination of self-supervised and contrastive vision-language objectives. It was trained on 229,619 CT series from multiple public datasets spanning chest, abdomen, pelvis, and whole-body imaging (including CT-RATE, MERLIN, INSPECT, NLST, AMOS, AbdomenAtlas, PANORAMA, and AbdomenCT-1K). Although anatomically diverse, the training corpus was largely non-oncologic, with pancreatic cancer data (PANORAMA) representing approximately 1% of the total dataset. - COLIPRI [14]: a transformer-based multi- modal foundation model combining contrastive vision- language and vision-only pretraining. CT training used the same public CT-report datasets included in CAPTOR-100K, complemented by large image-only MRI datasets (brain, knee/thigh, and whole-body MRI), making it the broadest multimodal medical foundation model evaluated. - CT-FM [17]: a vision-only 3D foundation model trained using contrastive self-supervised learning on more than 148,000 CT examinations acquired across multiple cancer types and associated comorbidities. Unlike the other vision-language models, CT-FM does not use radiology reports during pretraining and learns image representations exclusively from CT volumes. Table 4 summarizes their characteristics. ModelVLMBody PartsTraining Dataset(s)Dataset size CAPTOR (OURS) βCAPInternal30K MerlinβAbdomen & PelvisMerlin25K CT-CLIPβChestCT-RATE 50K volumes 25K scans SPECTREβCAP Multiple open CT datasets incl. CT-RATE, INSPECT, Merlin 230K COLIPRIβChestCT-RATE, NLST100K CT-FMβCAP + othersImaging Data Commons148K Table 4: Characteristics description of CAPTOR and the five baseline models we used in our analyses. VLM = Vision-Language Model To compare the embedding quality of each model, we used the official documentation and code to prepro- cess the data and extract embeddings. For CAPTOR and the Merlin model we used the embeddings from the second to last layer, as it is the official image embedding method. A.7Label extraction - Prompt and con- ditions We used zero-shot prompting and intentionally avoided few-shot prompting because selecting rep- resentative demonstration examples introduces an additional source of variability and may reduce re- producibility across datasets. The prompt we used for the label extraction with Mistral is the following: System: "You are a radiologist reviewing radiology reports and classifying whether the patient has a condition or not." Prompt: βHere is a radiology report from what radiologists can see in CT scans: report_content and here is a list of medical conditions that can po- tentially be stated in these reports: list_conditions You are to determine if: - Each condition is stated or explained as detected in the report. In this case you will put the string βTrueβ in the field βdetectedβ of the right condition, the string βFalseβ otherwise. - Each condition is stated or explained as not de- tected in the report. If the report states that there is a history of the condition in the patient, that counts as not detected condition. If the condition is stated as not detected in the report you will put the string βTrueβ in the field βnot_detectedβ in the field of the right condition, the string βFalseβ otherwise - Each condition is not stated or explained at all in the report. In this case, you will put the string βTrueβ in the field βnot_mentionedβ of the field of the right condition, the string βFalseβ otherwise. You will give the results as a JSON with the following schema json_schema, with strings βTrueβ or βFalseβ in the fields βdetectedβ, βnot_detectedβ, and βnot_mentionedβ for each condition. If the condition is stated as detected but the report also states that there is a history of this condition in the patient, please fill βFalseβ in the βnot_detectedβ field because a condition cannot be present and not present at the same time. There can only be one βTrueβ for each condition. Please do not add any condition that is not in the list I gave you.β The full list of conditions is the following : Arterial dissection, Hemoperitoneum, Mesenteric ischemia, Active bleeding, Infarction of solid organs, Pseudo anevrysm, Visceral perforation, Pneumoperitoneum (free air), Peritonitis, Pyelocaliceal dilation, Arte- rial/venous occlusion, Fracture, Pancreatitis, Aortic aneurysm (unruptured), Hemothorax, Intestinal ob- struction, Liver fibrosis / Cirrhosis, Peritoneal car- 15 cinomatosis (omental caking), Pneumothorax, Pul- monary embolism, Pericardial effusion, Epiduritis, Pleural effusion, Peritoneal effusion, Intrahepatic bile duct dilatation, Common bile duct dilatation, Coro- nary artery disease calcification A.8XGBoost parameters and details on classification method XGBoost was chosen as a strong non-linear classi- fier while using identical hyperparameters across all conditions and foundation models, ensuring that per- formance differences primarily reflected embedding quality rather than downstream model optimization. To train the XGBoost classifiers the binary logistic objective function (objective = binary:logistic) was used for classification. To account for the unequal class distribution, the positive class was weighted using the scale_pos_weight parameter. A moder- ate learning rate (learning_rate = 0.1) was selected, and model complexity was constrained by limiting tree depth (max_depth = 3) and by applying both L1 (reg_alpha = 0.5) and L2 (reg_lambda = 5) regularization. Additional variance reduction was introduced through row subsampling (subsample = 0.9) and feature subsampling (colsample_bytree = 0.9). The number of boosting iterations was fixed at 100 trees (n_estimators = 100). Overall, these settings were chosen to favor model generalization and robustness while limiting overfitting, particularly in datasets containing a limited number of positive samples. For the baseline models we also used all available series, using each modelβs method to get the embed- dings from each series and using the embeddings from the right series for each condition (e.g. embeddings from the Chest series for chest conditions and from the AP series for AP conditions). A.9Further explanation on generaliza- tion evaluation We tested all models on broadly used chest CT datasets: CT-RATE [11] and RAD-ChestCT [33]. The original RAD-Chest CT dataset contains 35,747 non-contrast chest CT scans from 19,661 adult patients of the Duke University Health System be- tween January 2012 and April 2017. The publicly available version contains 3,630 chest CT scans, ap- proximately 10% of the dataset. Each CT scan is annotated with a matrix of 84 abnormality labels x 52 location labels. For our use case we only used the 84 abnormality labels: if the abnormality is present in at least one location, then we set it to present, absent otherwise. For both datasets we used the given labels, ex- tracted from the reports with a fine-tuned RadBERT- RoBERTa-4m [35] for CT-RATE and with a rule- based method for RAD-ChestCT, and the given train- ing and testing sets to train and test the classifiers. A.10Further explanation on retrieval metrics For text-image retrieval, the random baseline for Re- trieval@K is simply KN, where N is the total number of images, as each image has an equal probability of appearing in the top K retrieved images under random ranking. Consequently, both the random baseline and model scores increase with K, since the probability of retrieving the correct image within the top K candidates increases as K grows. For image-image retrieval, following standard prac- tice, the relevancy of an image with the reference image is computed with the Intersection over Union of labels, also known as the Jaccard index, thresh- olded at 0. The IoU consists in the number of positive labels in common (size of the intersection of labels) over the number of positive labels in total for both images (size of the union of labels). In short, in our computation, an image is considered relevant with another if they have the presence of at least one medical condition (positive label) in common. The MAP@K metric is then computed following the standard formulation [34]: AP@K = 1 min(R,K) K X i=1 P@iΒ· 1[rel(i)] (1) where R is the total number of relevant items among all the images for the query, P@i is the pre- cision at rank i (the proportion of relevant images in the top i retrieved images), and 1[rel(i)] indicates whether the item retrieved at rank i is relevant. The MAP@K is then the mean of AP@K over all queries. Importantly, only abnormal volumes (i.e., volumes presenting at least one positive label) are included as queries, as normal volumes have no relevant coun- terpart by construction under the Jaccard threshold of 0. The random image-image retrieval baseline is computed empirically via Monte Carlo simulation (50 16 trials, 500 queries per trial), where items are retrieved randomly. The random baseline increases from K=1 to K=5 before decreasing for larger K values, which is expected under the standard AP@K formulation. As K increases, the normalization by min(R,K) can initially inflate scores for queries with few relevant items (if the single relevant item is retrieved within K positions, AP@K equals 1 regardless of K) before the dilution of non-relevant items at higher ranks drives the score down. The baseline also varies across dataset subsets, reflecting differences in label density: subsets with more frequent or overlapping pathologies (e.g., Chest) exhibit higher random baselines than sparser subsets (e.g., AP), as a greater proportion of the gallery shares at least one label with any given query. Model performance is consequently expected to follow the same decreasing trend with K, since retrieving relevant items becomes relatively harder as the retrieved set grows. B Complementary results B.1Detailed internal dataset characteris- tics Table 5 presents the internal dataset detailed charac- teristics by country. B.2 Mistral performance Figure 7 displays the classification performance of Mistral Large 3 [25], GPT 5.4 [31] and GPT 4o mini [32] on the CT-RATE [11] dataset, using the datasetβs original labels extracted with a fine-tuned RadBERT-RoBERTa-4m [35] on 1000 reports. B.3 Label proportions Table 6 shows the proportions of each medical condi- tion in each subset of our whole testing set (internal dataset combined with public datasets) B.4Statistical significance results by con- dition Table 7 presents the results of the number of con- ditions for which the difference between our model CAPTOR and each baseline is statistically significant. Figure 7: Radar plot comparing the per-condition classification F1-score of Mistral Large 3, GPT-5.4, and GPT-4o mini on the CT-RATE dataset, evalu- ated against the datasetβs original labels (extracted with a fine-tuned RadBERT-RoBERTa-4m on 1,000 reports). B.5Computation times - oncologic datasets All the tests for this section were conducted on an Apple MacBook Pro equipped with an Apple M4 Pro chip (12-core CPU: 8 performance cores and 4 efficiency cores) and 24 GB of unified memory. Table 8 presents the computation time for the dataset Colorectal-Liver Metastases, which were consistent with the time on the other datasets. B.6Importance of data composition full results This section provides the complete CAPTOR-100K results referenced in the main text, extending the CAPTOR comparisons of Figures 3β5 and Table 2 to include the larger training-data variant. B.7External datasets with original labels CAPTOR, more specialized for CAP oncological imaging, is significantly outperformed (p<0.01) by our broader 100K model on both CT-RATE [11] and RADChest-CT [33] (Figure 11) Both our models are outperformed by COL- IPRI [14] and SPECTRE [13]but not significantly for CAPTOR-100K on RADChest-CT. 17 Country Studies Age Median [IQR] SexContrastDescription France4,25468 [56β77] F = 53% M = 47% Injected = 67.8% Non-injected = 13.4% Unknown = 18.8% Data collected for lesion detection throughout the CAP (incl. adrenal glands, bladder, kidneys, spleen, lungs, lymph nodes, ovaries, rectum, and colon) Japan94970 [35β117] F = 50% M = 47% Injected = 66.1% Non-injected = 21.1% Unknown = 12.9% Data collected for incidental findings of pancreatic cancer. Spain4,64566 [57β75] F = 47% M = 53% Injected = 47.8% Non-injected = 10.4% Unknown = 41.8% Data collected for lesion detection throughout the CAP (incl. the adrenal glands, bladder, kidneys, spleen, lungs, lymph nodes, ovaries, rectum, and colon) USA19,85266 [56β74] F = 49% M = 51% Injected = 71.7% Non-injected = 21.9% Unknown = 6.3% Data collected for disease detection on CAP examinations in an oncologic setting and for lesion detection throughout the CAP (incl. bone, adrenal glands, lungs, gastrointestinal tract, lymph nodes, and pancreas) Vietnam193NA F = 41% M = 59% Injected = 80.3% Non-injected = 13.0% Unknown = 6.7% Data collected for incidental findings of pancreatic cancer. Table 5: Detailed characteristics of the proprietary dataset by country. IQR = interquartile range, F=female, M=male Pathology Whole test set N=10,963 Chest N=4,008 AP N=8,887 CAP N=1,932 Oncological N=3,336 Non- oncological N=7,627 Active bleeding0.52%0.04%0.73%0.05%0.09%0.68% Aortic aneurysm (unruptured)5.13%9.22%3.13%7.92%6.29%4.72% Arterial dissection0.33%0.19%0.46%0.52%0.39%0.31% Arterial/venous occlusion4.39%1.99%5.99%4.66%4.71%4.28% Common bile duct dilatation4.48%-6.27%2.48%2.91%5.04% Coronary artery disease / calcification25.05%36.65%-40.06%35.82%21.19% Epiduritis0.07%0.09%-0.16%0.12%0.05% Fracture8.54%6.86%10.66%13.30%11.81%7.37% Hemoperitoneum1.07%-1.51%0.10%0.27%1.35% Infarction of solid organs1.59%0.42%2.23%1.09%0.87%1.85% Intestinal obstruction4.11%-5.82%0.52%0.99%5.24% Intrahepatic bile duct dilatation6.23%-8.73%2.85%3.66%7.15% Liver fibrosis / Cirrhosis3.43%-4.35%2.95%4.92%2.90% Mesenteric ischemia0.60%-0.86%0.21%0.15%0.76% Pancreatitis1.56%-2.22%0.78%1.35%1.63% Pericardial effusion5.17%7.13%-7.04%6.12%4.83% Peritoneal carcinomatosis (omental caking)3.81%-5.18%8.90%7.97%2.31% Peritoneal effusion16.69%-22.99%8.18%8.90%19.48% Peritonitis1.94%-2.76%0.21%0.24%2.55% Pleural effusion14.91%14.69%-18.22%16.70%14.27% Pneumoperitoneum (free air)2.48%-3.47%0.26%0.33%3.25% Pneumothorax0.53%0.30%-0.21%0.15%0.67% Pseudo anevrysm0.23%0.09%0.33%0.26%0.21%0.24% Pulmonary embolism0.51%0.55%-0.98%0.99%0.34% Pyelocaliceal dilation4.80%-6.34%4.50%4.38%4.96% Visceral perforation1.54%-2.18%0.21%0.18%2.03% Table 6: Medical condition proportions in each subset of our whole testing set (internal dataset combined with public datasets) 18 Dataset Number of relevant conditions Merlin CT-CLIP SPECTRE COLIPRI CTFM Whole test set26+8 / -7 +24 / -0+9 / -4+20 / -0 +25 / -0 CAP26+11 / -0 +16 / -0+15 / -0+13 / -0 +16 / -0 Chest13+9 / -1+8 / -0+7 / -0+3 / -0+8 / -0 AP20+3 / -6 +19 / -0+6 / -6+15 / -0 +19 / -0 Oncological26+13 / -0 +15 / -0+17 / -0+15 / -0 +16 / -0 Non-oncological26+2 / -9 +23 / -0+4 / -12+14 / -1 +25 / -0 Table 7: Number of conditions for which the difference between our model CAPTOR and each baseline is statistically significant. β+β shows the number of conditions where CAPTOR is statistically better, β-β shows the number of conditions where the baseline is statistically better. ModelTotal computation time Computation time per image CAPTOR (OURS)7min38s2.4s Merlin7min49s2.4s CT-CLIP6min34s2.0s SPECTRE1h34min12s29.1s COLIPRI55min28s17.2 CT-FM1h09min13s21.4s Table 8: Computation times for the dataset Colorectal-Liver Metastases. All computations were conducted on an Apple MacBook Pro equipped with an Apple M4 Pro chip (12-core CPU: 8 performance cores and 4 efficiency cores) and 24 GB of unified memory 19 Retrieval text-imageRetrieval image-image DatasetK CAPTOR (OURS) CAPTOR-100K (OURS) Merlin CT-CLIP COLIPRI CAPTOR (OURS) CAPTOR-100K (OURS) Merlin CT-CLIP COLIPRI CTFM Whole test set 10,794 1 307 (266β348) 746 (688β797) 722 (622β817) 20.0 (8.88β31.1) 188 (104β281) 1.78 (1.72β1.82) 1.86 (1.79β1.90) 1.82 (1.75β1.85) 1.35 (1.27β1.40) 1.63 (1.54β1.68) 1.40 (1.33β1.46) 5 179 (159β199) 437 (421β452) 353 (307β394) 12.2 (6.60β18.7) 98.4 (57.8β144) 1.44 (1.39β1.45) 1.48 (1.43β1.49) 1.46 (1.41β1.46) 1.20 (1.12β1.20) 1.35 (1.28β1.36) 1.23 (1.16β1.24) 10 139 (125β153) 310 (302β319) 235 (205β261) 11.3 (6.71β16.6) 70.1 (42.6β101) 1.46 (1.44β1.50) 1.50 (1.47β1.54) 1.48 (1.45β1.51) 1.20 (1.16β1.24) 1.35 (1.31β1.40) 1.23 (1.20β1.28) 50 65.8 (60.9β70.4) 119 (117β121) 77.1 (67.8β84.8) 8.02 (4.97β11.8) 30.7 (21.7β40.5) 1.59 (1.57β1.64) 1.63 (1.60β1.68) 1.60 (1.58β1.65) 1.24 (1.21β1.31) 1.42 (1.39β1.49) 1.28 (1.26β1.35) 100 43.1 (40.2β45.8) 72.1 (71.1β73.3) 45.9 (40.9β50.2) 6.62 (4.24β9.54) 21.0 (16.0β26.5) 1.61 (1.59β1.66) 1.65 (1.62β1.71) 1.63 (1.60β1.68) 1.24 (1.21β1.31) 1.43 (1.40β1.50) 1.29 (1.26β1.36) Chest 3,944 1 145 (111β197) 219 (175β257) 23.0 (14.0β36.1) 22.0 (10.9β30.4) 184 (111β231) 1.33 (1.28β1.37) 1.37 (1.30β1.40) 1.27 (1.22β1.31) 1.17 (1.11β1.22) 1.29 (1.22β1.34) 1.18 (1.11β1.23) 5 85.2 (67.9β113) 134 (118β145) 19.0 (14.2β26.1) 12.8 (7.35β16.7) 92.0 (58.4β114) 1.22 (1.18β1.25) 1.23 (1.19β1.26) 1.18 (1.14β1.20) 1.11 (1.06β1.14) 1.19 (1.14β1.22) 1.12 (1.07β1.14) 10 64.3 (51.9β84.0) 95.2 (86.2β101) 15.3 (11.9β20.0) 11.6 (7.04β15.0) 63.3 (41.1β78.0) 1.23 (1.20β1.27) 1.24 (1.21β1.28) 1.19 (1.16β1.23) 1.11 (1.08β1.16) 1.19 (1.15β1.23) 1.12 (1.08β1.16) 50 28.6 (24.4β35.4) 38.2 (36.2β39.6) 9.10 (7.79β11.1) 8.08 (5.50β9.92) 25.7 (18.8β30.0) 1.26 (1.23β1.30) 1.28 (1.24β1.33) 1.23 (1.20β1.28) 1.12 (1.08β1.18) 1.18 (1.14β1.24) 1.13 (1.09β1.18) 100 18.8 (16.2β22.9) 24.4 (23.6β25.0) 6.71 (5.86β7.90) 6.58 (4.65β7.93) 16.8 (13.3β19.1) 1.25 (1.21β1.29) 1.26 (1.22β1.31) 1.22 (1.18β1.27) 1.10 (1.06β1.16) 1.16 (1.11β1.22) 1.11 (1.06β1.16) AP 8,731 1 270 (241β305) 604 (560β651) 733 (685β782) 6.00 (2.00β11.0) 29.0 (19.0β39.9) 1.72 (1.66β1.76) 1.80 (1.73β1.82) 1.78 (1.71β1.80) 1.25 (1.18β1.29) 1.53 (1.46β1.56) 1.28 (1.23β1.33) 5 157 (148β168) 357 (342β371) 357 (343β372) 4.40 (2.60β6.25) 22.0 (17.8β26.1) 1.41 (1.36β1.42) 1.45 (1.39β1.45) 1.43 (1.38β1.44) 1.14 (1.07β1.14) 1.29 (1.23β1.29) 1.17 (1.10β1.17) 10 122 (116β129) 254 (245β262) 238 (230β245) 4.00 (2.80β5.23) 18.2 (15.6β20.7) 1.43 (1.40β1.46) 1.46 (1.44β1.49) 1.45 (1.43β1.48) 1.14 (1.11β1.17) 1.30 (1.27β1.33) 1.17 (1.14β1.20) 50 56.6 (54.8β58.2) 96.6 (94.7β98.4) 78.0 (76.2β79.8) 2.92 (2.40β3.42) 12.8 (11.8β13.7) 1.54 (1.53β1.59) 1.57 (1.56β1.62) 1.56 (1.54β1.61) 1.16 (1.15β1.21) 1.35 (1.34β1.40) 1.20 (1.19β1.25) 100 36.9 (36.0β37.8) 58.5 (57.7β59.3) 46.2 (45.3β47.1) 2.74 (2.41β3.07) 9.94 (9.36β10.5) 1.56 (1.54β1.61) 1.60 (1.58β1.64) 1.59 (1.56β1.63) 1.17 (1.16β1.22) 1.36 (1.34β1.41) 1.20 (1.19β1.25) CAP 1,881 1 109 (90.3β129) 78.0 (61.6β95.7) 25.0 (15.9β34.9) 6.00 (1.99β11.2) 24.0 (14.6β34.3) 1.49 (1.40β1.56) 1.48 (1.37β1.53) 1.37 (1.30β1.46) 1.24 (1.15β1.32) 1.36 (1.27β1.43) 1.22 (1.13β1.31) 5 62.4 (56.1β68.9) 53.6 (47.7β60.3) 19.0 (15.3β22.6) 3.60 (2.02β5.44) 13.6 (10.5β17.0) 1.28 (1.22β1.32) 1.27 (1.20β1.30) 1.22 (1.16β1.26) 1.13 (1.06β1.16) 1.21 (1.14β1.24) 1.12 (1.05β1.16) 10 47.8 (44.2β51.6) 38.6 (35.3β41.9) 15.1 (12.8β17.4) 3.30 (2.24β4.54) 10.5 (8.55β12.5) 1.30 (1.25β1.35) 1.28 (1.23β1.33) 1.24 (1.19β1.29) 1.13 (1.09β1.19) 1.20 (1.16β1.26) 1.13 (1.09β1.19) 50 19.8 (18.9β20.6) 15.9 (15.1β16.8) 8.14 (7.42β8.86) 2.48 (2.04β2.94) 6.98 (6.32β7.62) 1.33 (1.29β1.40) 1.31 (1.27β1.37) 1.27 (1.23β1.34) 1.12 (1.08β1.20) 1.19 (1.15β1.26) 1.14 (1.10β1.21) 100 12.6 (12.2β13.0) 10.8 (10.4β11.2) 5.62 (5.20β5.99) 2.14 (1.85β2.44) 5.16 (4.79β5.56) 1.33 (1.28β1.39) 1.31 (1.26β1.37) 1.27 (1.23β1.34) 1.12 (1.07β1.19) 1.18 (1.13β1.25) 1.12 (1.08β1.20) Oncological 3,264 1 224 (195β253) 151 (128β174) 38.0 (26.1β51.2) 6.00 (1.99β11.0) 41.0 (29.1β53.9) 1.65 (1.56β1.70) 1.63 (1.54β1.68) 1.47 (1.40β1.55) 1.29 (1.20β1.36) 1.37 (1.29β1.44) 1.29 (1.21β1.36) 5 115 (106β125) 93.8 (86.0β102) 27.6 (23.1β32.2) 6.20 (4.19β8.43) 22.8 (18.7β27.1) 1.40 (1.34β1.42) 1.37 (1.32β1.41) 1.29 (1.23β1.32) 1.17 (1.10β1.19) 1.23 (1.16β1.25) 1.19 (1.11β1.21) 10 83.4 (78.4β88.6) 71.7 (67.3β76.6) 21.5 (18.8β24.2) 4.60 (3.32β5.90) 16.9 (14.4β19.5) 1.41 (1.37β1.46) 1.39 (1.35β1.44) 1.30 (1.26β1.35) 1.17 (1.12β1.22) 1.23 (1.18β1.27) 1.19 (1.14β1.24) 50 33.5 (32.4β34.6) 28.7 (27.6β29.8) 11.2 (10.3β12.0) 3.28 (2.79β3.78) 9.94 (9.16β10.7) 1.48 (1.45β1.55) 1.46 (1.43β1.52) 1.38 (1.34β1.44) 1.18 (1.15β1.25) 1.23 (1.20β1.30) 1.21 (1.18β1.27) 100 21.0 (20.5β21.5) 18.2 (17.7β18.7) 8.09 (7.61β8.60) 2.90 (2.56β3.23) 7.38 (6.93β7.88) 1.49 (1.45β1.55) 1.46 (1.42β1.52) 1.38 (1.34β1.44) 1.17 (1.13β1.24) 1.21 (1.17β1.28) 1.20 (1.16β1.26) Non-oncological 7,530 1 190 (160β224) 659 (609β711) 745 (619β880) 22.0 (7.78β36.1) 191 (93.7β288) 1.69 (1.61β1.74) 1.78 (1.69β1.83) 1.80 (1.72β1.83) 1.27 (1.17β1.35) 1.60 (1.51β1.65) 1.33 (1.23β1.40) 5 116 (102β132) 377 (361β395) 365 (308β423) 14.4 (6.30β22.3) 99.0 (53.6β142) 1.38 (1.31β1.39) 1.43 (1.36β1.45) 1.44 (1.38β1.45) 1.15 (1.06β1.17) 1.32 (1.25β1.34) 1.18 (1.10β1.20) 10 90.3 (80.5β101) 265 (255β276) 236 (199β272) 11.9 (5.55β18.2) 69.3 (38.8β98.1) 1.41 (1.37β1.45) 1.47 (1.42β1.51) 1.47 (1.44β1.50) 1.16 (1.11β1.21) 1.35 (1.30β1.39) 1.19 (1.14β1.25) 50 43.0 (39.7β46.7) 95.8 (93.0β98.5) 74.2 (64.0β84.8) 8.24 (4.10β12.4) 27.6 (18.7β36.5) 1.52 (1.48β1.58) 1.57 (1.53β1.64) 1.58 (1.55β1.63) 1.18 (1.14β1.27) 1.41 (1.36β1.48) 1.23 (1.18β1.31) 100 28.7 (27.0β30.6) 56.6 (55.3β58.0) 42.8 (37.2β48.5) 6.41 (3.27β9.48) 18.7 (13.6β23.5) 1.54 (1.49β1.60) 1.58 (1.53β1.66) 1.59 (1.55β1.64) 1.19 (1.14β1.27) 1.42 (1.36β1.49) 1.23 (1.18β1.31) Table 9: Zero-shot text-image (Recall@K) and image-image (MAP@K) retrieval performance of CAPTOR, CAPTOR-100K and baseline models, expressed as fold-increase over the random baseline (best in bold and second best underlined), by anatomical and oncologic subset. K is the number of retrieved images. SPECTRE [13] could not be evaluated (unavailable projection-layer weights), and CT-FM [17], a vision-only model, was excluded from text-image retrieval. 20 Figure 8: Bar plot showing the classification scores of CAPTOR, CAPTOR-100K and five state-of-the art foundation models, on our test set stratified by anatomical region visible and indication. One XGBoost was trained for each medical condition and model using the image embedding extracted with the modelβs image encoder. For each subset and model, the macro-AUC is the unweighted mean of the AUC scores for each medical condition relevant to the subset among the 26 conditions we studied. The confidence intervals and the pairwise comparison were obtained with a patient-level bootstrap and a patient-level permutation test respectively, both of size 2,000. ***= p < 0.001, ** = p < 0.01, * = p < 0.05, the stars are the color of the comparison model (orange for CAPTOR-100K and blue for CAPTOR) 21 Figure 9: Per-condition classification performance of CAPTOR, CAPTOR-100K, and baselines for the 26 medical conditions, shown separately for the CAP subset (left) and the oncological subset (right). One XGBoost was trained for each medical condition and model using the appropriate image embedding (i.e. the right regional crop for CAPTOR and the right series for the baselines) extracted with the modelβs image encoder 22 Figure 10: Two-year survival-prediction performance (AUC) of CAPTOR, CAPTOR-100K and five baseline models on four oncologic TumorImagingBench cohorts (NSCLC-Radiomics, NSCLC-Radiogenomics, C4KC- KiTS, Colorectal-Liver-Metastases), using kNN classifiers on frozen image embeddings averaged over 10 random splits. Statistical significance was assessed using a permutation test (2,000 permutations). 23 Figure 11: Classification macro-AUC of CAPTOR, CAPTOR-100K, and five baseline models for each abnormality label on the external, non-oncologic CT-RATE [11] and RAD-Chest CT [33] datasets, using each datasetβs original labels. One XGBoost was trained for each medical condition and model using the image embedding extracted with the modelβs image encoder. For each subset and model, the macro-AUC is the unweighted mean of the AUC scores for each medical condition relevant to the subset among the 26 conditions we studied. The confidence intervals and the pairwise comparison were obtained with a patient-level bootstrap and a patient-level permutation test respectively, both of size 2,000. *** = p < 0.001, ** = p < 0.01, * = p < 0.05 24