Paper deep dive
How Far from Clinical Deployment? Evaluating the Complete Unsupervised Domain Adaptation Pipeline in Medical Imaging
Yiheng Xiong, Luisa Gallée, Daniel Santak Wolf, Heiko Hillenhagen, Michael Götz
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Deploying unsupervised domain adaptation (UDA) in clinical practice requires choosing which algorithm to use and which of its trained models to ship. However, the deployment (target) domain is unlabeled, so models cannot be evaluated directly on it, leaving it unclear which to select. We address this by evaluating the complete UDA pipeline, considering both adaptation and label-free selection together. Our study covers eleven clinically relevant cross-domain scenarios from nine medical imaging datasets, with ten UDA algorithms and 13 label-free selection methods (validators), evaluating over 80,000 trained models in total. By this, we find that a capable adapted model usually exists, but identifying it without target labels is difficult: the validator-selected models leave a large and structural target performance gap to the best available one, with no evaluated validator consistently reliable. Towards closing it, we explore two strategies, ensembling and a small target-labeling budget; both narrow this gap but do not close it entirely. Overall, deployable UDA depends on the complete pipeline; addressing the less explored selection step could bring much of current UDA closer to clinical use.
Tags
Links
- Source: https://arxiv.org/abs/2608.12035v1
- Canonical: https://arxiv.org/abs/2608.12035v1
Trouble viewing inline? Open PDF directly →
Full Text
121,753 characters extracted from source content.
Expand or collapse full text
How Far from Clinical Deployment? Evaluating the Complete Unsupervised Domain Adaptation Pipeline in Medical Imaging Yiheng Xiong Affiliation: Section of Experimental Radiology, Ulm University Medical Center Luisa Gallée Affiliation: Section of Experimental Radiology, Ulm University Medical Center Daniel Santak Wolf Affiliation: Section of Experimental Radiology, Ulm University Medical Center Affiliation: Visual Computing Group, Ulm University Heiko Hillenhagen Affiliation: Section of Experimental Radiology, Ulm University Medical Center Michael Götz Affiliation: Section of Experimental Radiology, Ulm University Medical Center Abstract Deploying unsupervised domain adaptation (UDA) in clinical practice requires choosing which algorithm to use and which of its trained models to ship. However, the deployment (target) domain is unlabeled, so models cannot be evaluated directly on it, leaving it unclear which to select. We address this by evaluating the complete UDA pipeline, considering both adaptation and label-free selection together. Our study covers eleven clinically relevant cross-domain scenarios from nine medical imaging datasets, with ten UDA algorithms and 13 label-free selection methods (validators), evaluating over 80,000 trained models in total. By this, we find that a capable adapted model usually exists, but identifying it without target labels is difficult: the validator-selected models leave a large and structural target performance gap to the best available one, with no evaluated validator consistently reliable. Towards closing it, we explore two strategies, ensembling and a small target-labeling budget; both narrow this gap but do not close it entirely. Overall, deployable UDA depends on the complete pipeline; addressing the less explored selection step could bring much of current UDA closer to clinical use. Our code is open sourced at Complete UDA Pipeline. 1 Introduction Figure 1: The complete UDA pipeline at deployment. Adaptation trains models on a labeled source domain (here SLO) and an unlabeled target domain (here OCT), producing many candidate models that vary in UDA algorithm and hyperparameter configuration. At deployment, one model needs to be selected, yet without target labels they cannot be evaluated directly to decide which. Deep learning models excel at medical image-based diagnosis, but their performance often degrades substantially under domain shifts arising from differences in imaging modalities, acquisition protocols, and patient populations [22]. Annotating training data from a new domain is costly, requiring scarce medical expertise. Unsupervised domain adaptation (UDA) addresses both challenges by transferring knowledge from a labeled source domain to an unlabeled target domain, with recent algorithms adapting robustly across clinically relevant shifts, such as cross-modality [50], cross-site [26], and cross-age [10]. However, deploying UDA in clinical routine requires a practitioner to decide which algorithm to use and which of its checkpoints (a model snapshot at a training iteration) to ship, as shown in Figure 1. Thus, model selection is an unavoidable part of the complete UDA pipeline. Furthermore, without target labels, candidate models cannot be evaluated directly on the target domain, leaving it unclear which one to select for deployment: the standard approach of comparing them on a held-out labeled validation set is unavailable [34]. Yet, existing medical UDA surveys [3, 13, 39, 24, 22, 42, 51] and evaluations [7, 36, 1, 41, 44] primarily focus on adaptation in isolation, leaving the selection step largely unexamined. As a result, it remains unclear how the complete UDA pipeline behaves under clinical deployment conditions, and thus how far current UDA truly is from clinical deployment. To address this, we evaluate the complete UDA pipeline in medical imaging, considering both adaptation and label-free selection that clinical deployment requires. For adaptation, we include ten UDA algorithms spanning multiple paradigms, with some tailored to medical imaging [14, 54]. For selection, we adopt 13 established label-free selection approaches (a.k.a. validators), such as IWCV [43] and DEV [53]. Each validator assigns a scalar validation score to each checkpoint without using target labels, and the checkpoint with the best score is selected. Our evaluation covers brain MRI, chest X-ray (CXR), and retinal imaging, with eleven clinically relevant cross-domain scenarios in total. Brain MRI and CXR use four datasets each, with within-modality shifts arising from differences in hospitals, scanners, or patient populations, while retinal imaging involves across-modality shifts between SLO and OCT. Altogether, this amounts to roughly 16,500 checkpoint configurations (each set by the algorithm, its hyperparameters, and training iteration), or over 80,000 trained checkpoints once repeated across folds or random seeds. Our study points to a consistent picture across different cross-domain scenarios. Adaptation itself is often effective: a capable adapted model usually exists among the candidates. What proves difficult is identifying it without target labels: the models selected by the evaluated validators leave a large target performance gap to the best available model. We further trace the origin of this gap and find it largely structural: no single evaluated validator works consistently well across all scenarios; one that succeeds in some fails in others, so no default choice can be recommended in advance. In the absence of a consistently reliable validator across all scenarios, towards closing the gap, we turn to two strategies. Instead of relying on one validator, the first ensembles predictions across validator-selected checkpoints from every algorithm; instead of remaining fully label-free, the second spends a small target-labeling budget. Both narrow the gap, though neither closes it entirely and each carries its own limitations. Our contributions are summarized as follows: • We present, to our knowledge, the first study to evaluate the complete adaptation and label-free selection UDA pipeline in medical imaging under clinical deployment conditions, spanning ten algorithms, 13 validators, and nine datasets across eleven cross-domain scenarios. • By considering adaptation and selection as a whole, we find that a capable adapted model usually exists, yet the remaining distance to clinical deployment lies largely in label-free selection: the selection gap is large and structural, with no evaluated validator consistently reliable. • Towards closing the gap, we point to two strategies, ensembling and a small target-labeling budget; both narrow this gap and open directions for more reliable selection. 2 Related Work 2.1 Surveys and Benchmarks on Medical UDA UDA in medical imaging has been reviewed from several angles. Choudhary et al. [3] give an early overview of deep domain adaptation, Guan and Liu [13] provide a widely used taxonomy across modalities, and Kumari and Singh [22] survey more recent deep UDA methods and datasets. More recent surveys cover emerging diffusion- and foundation-model approaches [51], broader reviews of distribution shift also include UDA [42], and others focus on specific settings such as functional brain data [39] and segmentation [24]. Alongside these surveys, several benchmarks evaluate adaptation under realistic shifts: M3DA [41] for 3D MRI/CT segmentation, CrossMoDA [7] for cross-modality segmentation, M3-UDA [36] for multi-hospital fetal ultrasound detection, and Chamarthi et al. [1] and Sultana et al. [44] for skin lesion classification. These efforts have established rigorous comparisons of adaptation methods on clinically relevant data, but the selection step remains largely underexplored. Our study complements them by evaluating the complete pipeline that clinical deployment requires, adaptation and label-free selection together. 2.2 Label-Free Model Selection in UDA Selecting a model without target labels has been studied mainly in general-vision UDA, where a range of validators have been proposed [43, 11, 53, 37, 35, 48, 52, 16] and evaluated. Musgrave et al. [34] provide one of the first UDA model selection benchmarks covering many algorithms, though over a relatively small set of three label-free validators. Subsequent evaluations [9, 17, 23] strengthen evaluation practice further with more validators, with SKADA-Bench [23] additionally covering broader modalities, though with an emphasis on shallow adaptation methods. These works, however, primarily focus on selecting checkpoints within individual algorithms. In practice, deployment also requires choosing which algorithm to use, so a more realistic setting also calls for selecting across algorithms. Beyond this, medical images pose distinct challenges, exhibiting more pronounced and varied shifts than natural images [22], under which the behavior of these validators remains unclear. To address these, our study examines selection also across algorithms, over a broad set of 13 validators and ten deep UDA algorithms, including medical-specific ones, and, in particular, under the shifts that clinical routine faces, such as cross-modality, cross-institution, and cross-cohort, assessing how the complete UDA pipeline behaves under deployment conditions. 3 Datasets and Methods Table 1: Overview of the datasets, UDA algorithms, and validators used in our study. (a) Datasets across brain MRI, CXR, and retinal imaging (SLO / OCT), with the number of positive (Pos.) and negative (Neg.) samples per dataset. Positive and negative denote, respectively, Alzheimer’s disease and cognitively normal for brain MRI, pneumonia and non-pneumonia for CXR, and glaucoma and non-glaucoma for retinal imaging. (b) UDA algorithms spanning multiple paradigms. (c) Validators grouped into source-guided and target-based criteria. (a) Datasets Organ Modality Dataset Pos. Neg. Brain MRI ADNI-1 [18] 200 221 ADNI-2 [18] 159 232 ADNI-3 [18] 85 431 AIBL [8] 78 477 Chest X-Ray RSNA [49] 6,012 20,672 Child CXR [21] 4,273 1,583 LDD [31] 5,776 3,919 CRD [4] 9,237 10,319 Eye SLO FairDomain [45] 4,453 5,547 OCT 4,453 5,547 (b) UDA Algorithms Feat. Dist.: MMD [29] Pseudo Lab.: ATDOC [25] Info. Max: MCC [19] Cls. Disc.: MCD [38] SVD Loss: BNM [5] Adv. Align: DANN [12], CDAN [30], DALN [2] Medical-Spec.: AD2A (Brain MRI) [14], CoUDA (CXR) [54] (c) Validators Source-Guided: Source-Risk [11], IWCV [43], DEV [53], DEV-N [35] Target-Based: Entropy [33], InfoMax [34], Corr-C [48], BNM (V) [35], MCC (V) [19], SND [37], ClassAMI [35], MixVal [16], TransScore [52] We study the complete UDA pipeline in medical imaging under the conditions a clinical deployment would face, focusing on classification tasks such as disease diagnosis. The pipeline comprises two stages. In the adaptation stage, UDA algorithms share a common structure, a backbone, a classification head, and an adaptation module, and optimize a joint objective ℒ=ℒcls+λℒadapt,L=L_cls+λ\,L_adapt, (1) where ℒclsL_cls supervises the model with source labels, ℒadaptL_adapt aligns feature distributions across domains, and λ controls the adaptation strength. Running an algorithm with a given λ produces a training run, during which we save a sequence of checkpoints. Each checkpoint θ is thus uniquely defined by its algorithm, its adaptation strength λ, and its training iteration, and every checkpoint is a candidate for deployment. In the selection stage, we replicate the constraint of clinical deployment: target labels are unavailable, so the model to deploy must be chosen without them. The role of a validator is to provide a label-free validation score for the target performance. Formally, a validator is a scoring function V:θ↦s∈ℝ,V:θ s , (2) that maps a checkpoint θ to a scalar score s=V(θ)s=V(θ) without using target labels. The selected model for deployment is the checkpoint with the best validation score, θ⋆=argmaxθ∈ΘV(θ),θ = _θ∈ V(θ), (3) where Θ is the candidate checkpoint pool and scores are oriented so that higher is better. Datasets. To construct medical UDA scenarios, four widely adopted brain MRI datasets are used: ADNI-1, ADNI-2, ADNI-3 [18], and AIBL [8]. Subjects appearing in multiple ADNI datasets are kept in only one to prevent data leakage. In addition, four publicly available CXR datasets are used: RSNA [49], Child CXR [21], LDD [31], and CRD [4]. For retinal imaging, we use FairDomain dataset [45], which provides paired SLO and OCT acquisitions of the same cohort. For brain MRI and CXR, each dataset is treated as a separate domain, and transfer is performed across datasets within each modality; for retinal data, transfer is performed between SLO and OCT modalities. Brain MRI datasets contain Alzheimer’s disease and cognitively normal subjects, CXR datasets contain pneumonia and non-pneumonia subjects, and retinal datasets contain glaucoma and non-glaucoma subjects. All modalities undergo standard preprocessing following prior work [14, 54, 45]. Dataset statistics are summarized in Table 1 (a). Methods. Since we study the complete pipeline as a whole, we include a representative set of UDA algorithms and, crucially, a diverse set of label-free validators. As shown in Table 1 (b) and (c), the ten algorithms span multiple paradigms, including feature-distance minimization [29], adversarial alignment [12, 30, 2], information maximization [19], SVD loss [5], pseudo-labeling [25], classifier discrepancy [38], and medical-specific techniques [14, 54]. The 13 validators cover both source-guided criteria [11, 35, 43, 53] and target-based ones [33, 34, 48, 19, 37, 52, 16]. Experimental Setup. Following Guan et al. [14], five UDA scenarios (source→ ) are constructed for brain MRI: ADNI-1→ -2, ADNI-1→ -3, ADNI-2→ -1, ADNI-2→ -3, and ADNI-1+2→ . Following Feng et al. [10] and Liu et al. [27], two CXR transfers are used, each in both directions: RSNA↔ CXR and LDD↔ . For retinal imaging, the two cross-modality directions SLO→ and OCT→ are evaluated. Both source and target data are split into training and validation sets, and target performance is measured on the target validation set using balanced accuracy. For datasets without predefined splits, stratified five-fold cross-validation is performed; for datasets with explicit train/test splits, results are averaged over three random seeds. In both cases the mean ± standard deviation is reported; median values with 95% confidence intervals are additionally provided in the supplementary material. Table 2: Complete UDA results on ADNI-1→ -2 (target accuracy, %; eight best validators by Across-Algo accuracy shown). Each cell reports the target accuracy of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD AD2A Avg. Across-Algo TargetOnly Oracle 88.2± 2.0 89.3± 4.0 90.0± 4.0 89.9± 3.0 91.0± 3.5 91.0± 1.7 89.9± 4.4 85.7± 9.8 87.4± 4.3 92.0± 0.93 89.5± 2.0 93.2± 2.0 90.9± 2.4 DEV-N 80.9± 5.8 81.7± 2.8 84.9± 2.0 81.8± 8.4 80.8± 9.1 83.5± 3.3 82.9± 3.4 76.0± 12 81.0± 6.2 87.6± 1.9 - 85.1± 1.3 - Source-Risk 80.9± 5.8 82.1± 3.1 81.5± 3.7 81.1± 8.0 80.8± 9.1 83.3± 3.2 84.1± 4.1 73.2± 12 81.5± 6.2 87.5± 2.3 - 83.5± 2.9 - InfoMax 84.2± 2.1 84.8± 5.7 87.3± 4.4 86.0± 1.5 88.2± 3.8 81.8± 9.6 83.5± 6.9 78.9± 10 83.3± 5.6 86.3± 2.2 - 83.2± 7.2 - BNM (V) 84.7± 2.1 84.8± 5.7 87.5± 4.0 84.9± 3.8 87.2± 2.2 81.8± 9.6 83.5± 6.9 78.9± 10 83.3± 5.6 86.3± 2.2 - 81.8± 9.6 - ClassAMI 81.3± 5.8 71.5± 13 74.8± 8.7 83.0± 4.0 84.2± 1.6 81.7± 7.5 79.6± 4.4 78.3± 10 63.7± 19 72.7± 16 - 80.8± 3.6 - SND 73.9± 12 83.4± 4.1 80.0± 2.6 79.2± 5.8 85.5± 4.9 78.2± 5.9 78.5± 8.6 75.8± 12 82.2± 5.6 80.7± 8.7 - 80.7± 8.7 - MCC (V) 84.2± 2.7 84.4± 6.4 84.9± 3.3 84.5± 4.0 86.4± 3.7 80.4± 8.3 85.9± 3.6 80.7± 13 83.1± 3.7 87.1± 3.7 - 80.4± 2.3 - TransScore 81.3± 1.0 77.5± 11 85.1± 4.5 86.7± 2.3 85.2± 5.9 80.4± 9.9 76.5± 11 72.7± 10 82.4± 3.7 86.3± 2.8 - 79.9± 10 - Implementation Details. For all experiments, the classification head is a three-layer MLP with a dropout rate of 0.5. For brain MRI, a 3D ResNet-50 [15] trained from scratch is used as the backbone, with a batch size of eight per domain. For CXR and retinal imaging, a 2D ResNet-50 pretrained on ImageNet [6] is used, with a batch size of 48 per domain. All algorithms are trained with AdamW (weight decay 1e-4) and a one-cycle learning rate schedule with warm-up and a peak learning rate of 1e-3, for 10k iterations on brain MRI and retinal imaging and 30k iterations on CXR. The adaptation strength is varied over λ∈0.1,0.5,1.0λ∈\0.1,0.5,1.0\, giving three runs per algorithm. After warm-up, checkpoints are saved at uniform intervals, yielding 50 checkpoints per run and 150 checkpoints per algorithm. ℒclsL_cls is binary cross-entropy with class-balanced weighting derived from the source labels. All runs are conducted on an A6000 GPU. 4 Results and Discussion We use ADNI-1→ -2 as an illustrative example for the complete UDA pipeline in Table 2, which reports the target accuracy of the checkpoint each validator selects. This selection is compared against the Oracle, which selects directly with target labels. We define the selection gap as the difference in target accuracy between the Oracle-selected model and the model selected by a label-free validator. The per-algorithm columns report selection within a single algorithm, and Avg. is their mean, while Across-Algo pools all algorithms’ checkpoints and selects across them. Due to space constraints, we display the eight best validators by their Across-Algo accuracy. Full results for all scenarios can be found in the supplementary material. We report SourceOnly (no adaptation) and TargetOnly (trained on labeled target data) as references. 4.1 Adaptation Works in Principle Figure 2: Target accuracy under Oracle selection across UDA scenarios. Bars show SourceOnly (no adaptation), the per-algorithm average (Avg.), Across-Algo (selection over all algorithms’ pooled checkpoints), and TargetOnly (labeled-target). Across-Algo exceeds SourceOnly in all scenarios and approaches TargetOnly, indicating a capable adapted model usually exists. Error bars: standard deviation. As shown in Table 2, under Oracle selection, ADNI-1→ -2 scenario reaches 93.2% accuracy when pooling all checkpoints (Across-Algo), exceeding both SourceOnly (88.2%) and TargetOnly (90.9%). The gain is broad: the Oracle-selected checkpoint beats SourceOnly for seven of the nine algorithms, with a per-algorithm average of 89.5% (Avg.), and surpasses TargetOnly for three. Figure 2 extends Table 2 to all eleven UDA scenarios, reporting the same four quantities under Oracle selection. At the per-algorithm level, the average (Avg.) exceeds SourceOnly in eight of the eleven scenarios, and in two it even surpasses TargetOnly. Pooling checkpoints across algorithms strengthens this further: Across-Algo lies above SourceOnly in every scenario, and exceeds TargetOnly in five. Averaged over all scenarios (Mean), the ordering is consistent: SourceOnly << Avg. << Across-Algo, with Across-Algo reaching 84.8% and approaching TargetOnly (87.5%). Adaptation is therefore often effective across these clinical UDA scenarios; in particular, a capable adapted model exists in most scenarios (evidenced by Across-Algo). Nevertheless, this model is only useful if it can be identified without target labels, as deployment requires. 4.2 Large Selection Gap from Validators Figure 3: Target accuracy across UDA scenarios under three selections (%). Bars show the accuracy of the model selected by the best validator over the across-algorithm pool (Best Validator), by the best algorithm–validator combination (Best Pair), and by the across-algorithm Oracle. The first two fall below the Oracle in every scenario, with the gap up to 10.5 points and averaging 6.1 (Mean). Note that Best Validator and Best Pair differ from scenario to scenario and are identified using target labels, so they cannot be chosen in advance; the gap realized at deployment can only be larger. Error bars: standard deviation. The capable model established above can only be deployed if a validator selects it. Return to ADNI-1→ -2 scenario as shown in Table 2, under Across-Algo, the Oracle attains 93.2%. Over the same across-algorithm pool, the model selected by the best validator (DEV-N) reaches 85.1%, and that selected by the best algorithm–validator pair (DALN with InfoMax) reaches 88.2%. Even these best cases fall 5.0 to 8.1 points short of the Oracle. Figure 3 extends Table 2 to all eleven UDA scenarios, reporting the target accuracy of the model selected by the best validator over the across-algorithm pool (Best Validator), by the best algorithm–validator pair (Best Pair), and by the across-algorithm Oracle. In every scenario, both Best Validator and Best Pair fall below the Oracle: the gap reaches up to 10.5 points (RSNA→ CXR) and averages 6.1 points across scenarios (Mean). Crucially, Best Validator and Best Pair differ from scenario to scenario and are themselves identified using target labels; at deployment, where no labels are available to choose them in advance, the realized gap can only be larger. A capable model exists in the pool, but selecting it without labels leaves a substantial portion of the achievable performance unrealized. The Selection Gap Persists Across Architectures. To test whether our findings depend on the backbone, we repeat RSNA→ CXR with three additional architectures spanning distinct families: ConvNeXt [28] (a modern CNN), ResMLP [46] (an MLP-based model), and DeiT [47] (a vision transformer). As shown in Table 3, the model selected by the best validator leaves a noticeable gap to the Oracle for every architecture. This suggests the selection gap is not specific to a particular backbone. (a) OCT→ , DALN (b) OCT→ , ATDOC (c) MMD, ADNI-2→ -1 (d) MMD, SLO→ Figure 4: Within-algorithm Spearman correlation (ρ) between validation score and target accuracy. Each point is a checkpoint. The same validator can flip from reliable to reversed with a change of algorithm (a, b) or scenario (c, d) alone. Table 3: Target accuracy (%) on RSNA→ CXR for four backbones. Best Val. is the accuracy of the model selected by the best validator under the across-algorithm pool, and Oracle that of the actual best model. The gap persists across all four backbones. Backbone Best Val. Oracle Δ ResNet-50 79.0 89.5 10.5 ResMLP 70.9 79.2 8.3 ConvNeXt 82.6 90.2 7.6 DeiT 80.4 86.5 6.1 4.3 Structural Selection Gap from Validators Having established the magnitude of the selection gap, we now investigate its origin. The gap may arise either from a suboptimal choice of validator, which a more suitable one would remedy, or from a more structural limitation that no evaluated validator overcomes. Since selection is fundamentally a ranking problem (a validator succeeds by ranking checkpoints so that the best one scores highest), we trace the selection gap to validator reliability, measured by the Spearman rank correlation ρ between validation scores and true target accuracy. For validators where a lower score indicates a better checkpoint, the sign is flipped so that a positive correlation always denotes the intended direction. Within-Algorithm Selection. Figure 4 (a) and (b) fix the scenario OCT→ and validator BNM (V) and vary the algorithm: BNM (V) ranks DALN’s checkpoints well and in the designed direction (ρ=0.88,p<0.001ρ=0.88,p<0.001), yet ranks ATDOC’s checkpoints in reverse (ρ=−0.42,p<0.001ρ=-0.42,p<0.001). Figure 4 (c) and (d) fix the algorithm (MMD) and validator (ClassAMI) and vary the scenario: the correlation is positive as intended for ADNI-2→ -1 (ρ=0.59,p<0.001ρ=0.59,p<0.001) but reversed for SLO→ (ρ=−0.51,p<0.001ρ=-0.51,p<0.001). Further individual analyses can be found in the supplementary material. Figure 5 aggregates the within-algorithm Spearman correlation between validation scores and target accuracy two ways: (a) averaged over algorithms for a per-scenario view, and (b) averaged over scenarios for a per-algorithm view; in both, blue denotes correlation in the intended direction (darker is stronger) and warm red the opposite. On average, validators are moderately informative, but their reliability is uneven: no validator attains consistent correlation across all scenarios in (a) or all algorithms in (b), and the strongest validator differs from column to column. DEV-N, for example, is the most reliable validator for OCT→ (ρ=0.88ρ=0.88) yet drops to near zero for RSNA→ CXR; likewise, InfoMax is strongest for AD2A (ρ=0.79ρ=0.79) but falls to ρ=0.15ρ=0.15 for ATDOC. (a) Per-scenario view (averaged over algorithms) (b) Per-algorithm view (averaged over scenarios) Figure 5: Within-algorithm Spearman correlation (ρ) between validation scores and target accuracy. Blue denotes correlation in the intended direction (darker is stronger), warm red the opposite. No validator is consistently reliable across all scenarios or algorithms, and the most reliable one differs from column to column. (a) Within BNM algorithm (b) Across Algorithms Figure 6: Spearman correlation (ρ) between Source-Risk scores and target accuracy on ADNI-1→ -2. Each point is a checkpoint. (a) Within BNM algorithm, the correlation is moderate. (b) Across all algorithms, it collapses to near zero. Across-Algorithm Selection. Figure 6 (a) shows Source-Risk on ADNI-1→ -2 for a single algorithm (BNM): within this algorithm, it ranks checkpoints reasonably well (ρ=0.57,p<0.001ρ=0.57,p<0.001). Figure 6 (b) instead pools checkpoints across all algorithms for the same scenario and validator. To form this pool while keeping the comparison meaningful, we take, for each algorithm, the ten checkpoints that the validator scores highest: this retains the competitive checkpoints worth selecting among, while ten per algorithm provides a sufficient sample to estimate the rank correlation. On this pooled set, the correlation collapses to near zero (ρ=−0.04,p=0.0.678ρ=-0.04,p=0.0.678). Figure 7 reports the same across-algorithm correlation across all validators and scenarios: most validators are unreliable or contradict their intended direction, with correlations weak or negative. Figure 7: Across-algorithm Spearman correlation (ρ) between validation scores and target accuracy. Blue denotes correlation in the intended direction (darker is stronger), warm red the opposite. Most cells are weak or negative, showing that most validators are unreliable in this across-algorithm setting. For within-algorithm selection, a suitable validator may exist among evaluated ones, but which one differs by algorithm and scenario, so it cannot be chosen in advance. For across-algorithm selection, even this fails: most evaluated validators correlate weakly or in reverse. The gap therefore appears largely structural rather than a matter of choosing a better validator among those evaluated ones. 4.4 Strategies to Narrow the Selection Gap The structural selection gap leaves a large distance between what adaptation achieves and what label-free selection realizes at deployment. In the absence of a validator consistently reliable across scenarios, we explore two practical strategies and show that each recovers part of this gap. Ensembling. Since no single evaluated validator is consistently reliable, we avoid committing to one and instead ensemble the checkpoints selected by different validators, similar to Hu et al. [17] but additionally ensembling across algorithms. In the across-algorithm setting, where checkpoints from all algorithms are pooled, we select one checkpoint per algorithm with every validator and ensemble these selections by averaging their predictions. As shown in Table 4, ensembling matches or exceeds Best Val. in most scenarios. Again Best Val. is the model selected by the best validator for each scenario, which differs across scenarios and cannot be known in advance. The remaining gap to the Oracle is small in several cases (e.g., 0.6 for OCT→ and 2.6 for LDD→ ), showing that it recovers much of the achievable performance. It nonetheless remains far from the Oracle in other cases such as RSNA→ CXR. Small Target Labeling Budget. Since fully label-free selection cannot be trusted under the evaluated validators, we explore whether spending a small budget on target labeling can identify a better checkpoint. We label a small subset of the target validation set and select the checkpoint with the highest accuracy on it. The budget is varied over five levels (B1 to B5): 5, 10, …, 25 labeled samples for brain MRI, whose target validation set is around 100 samples, and 1% to 5% for the others. Each budget is repeated over ten random trials, and we report the mean. As shown in Table 4, a small budget improves selection steadily, surpassing both Best Val. (78.7) and ensembling (78.4) from B4 (20 labeled samples for brain MRI, 4% for the other scenarios) on average, though a gap to the Oracle still remains. Table 4: Two strategies for narrowing the selection gap (target accuracy, %). Ens. ensembles the checkpoints selected by all validators across all algorithms. Small Labeling Budget selects the best checkpoint on a small labeled target subset, at five levels (B1 to B5; 5–25 samples for brain MRI, 1–5% otherwise). On average, ensembling matches Best Val. (the best validator under the across-algorithm pool), and the labeling budget surpasses both from around B4, though a gap to the Oracle remains for both. Ens. Small Labeling Budget Best Val. Oracle Scenario B1 B2 B3 B4 B5 ADNI-1→ -2 87.6 78.6 83.0 84.0 86.5 87.5 85.1 93.2 ADNI-1→ -3 87.6 78.7 80.8 81.1 81.2 83.3 84.5 94.2 ADNI-2→ -1 89.0 82.8 84.1 84.6 87.4 87.7 86.4 92.5 ADNI-2→ -3 87.0 79.5 79.9 83.5 81.3 83.6 85.0 92.1 ADNI-1+2→ 88.8 81.6 82.9 82.9 85.1 86.0 89.0 93.8 RSNA→ CXR 80.7 76.6 78.4 81.5 83.1 85.0 79.0 89.5 Child CXR→ 74.2 74.3 75.8 76.3 76.9 76.9 74.9 78.3 LDD→ 81.0 80.2 81.6 82.0 82.2 82.1 81.9 83.6 CRD→ 57.1 75.2 77.9 80.2 79.8 80.4 72.5 82.7 SLO→ 62.9 58.9 61.2 60.8 61.3 62.6 60.1 65.6 OCT→ 66.8 60.0 60.9 63.5 64.5 64.4 67.3 67.4 Mean 78.4 75.1 76.9 78.2 79.0 80.0 78.7 84.8 4.5 Discussion Our results show that the main barrier between current UDA and clinical deployment lies less in producing a capable adapted model, which usually exists, than in selecting it without target labels. Under Oracle selection, such a model exists in the Across-Algo pool in most scenarios, often recovering much of the gap between SourceOnly and TargetOnly and at times exceeding both, confirming that adaptation is usually effective. The evaluated validators, however, do not reliably identify this model, leaving a large and structural gap to the best available one. For within-algorithm selection, a reliable validator may exist, but which one is best shifts from algorithm to algorithm and from scenario to scenario, so no fixed choice can be made in advance. For across-algorithm selection, most validators correlate only weakly with target performance, and many rank checkpoints opposite to their intended direction. Note that we do not optimize TargetOnly extensively; in several scenarios it falls below the per-algorithm Oracle average (Avg.) or even SourceOnly, which may stem from training on the target domain alone, with less labeled data than the combined source and target signal exploited by adaptation, and from target class imbalance (e.g., AIBL). The unreliability of evaluated validators also explains an observation that might otherwise question the value of adaptation: under many validators, the selected adapted model performs worse than the SourceOnly baseline (e.g., MCC with InfoMax in Table 2). We interpret this as a selection problem, not adaptation alone: a capable adapted model often exists (the Oracle confirms it), but an unreliable validator may fail to find it and sometimes picks one worse than not adapting. On the other hand, in Table 2, under Oracle selection ATDOC and MCD fall below SourceOnly, so the limitation here lies also in adaptation. This reflects either our relatively small hyperparameter set (λ and the training iteration) or the unsuitability of these algorithms for this transfer. It nonetheless reinforces the need for reliable validators: at the across-algorithm selection scale, a good validator should recognize and avoid such negative transfer. We also notice that validator reliability, measured by Spearman correlation ρ, and the target accuracy of the selected model do not always align. As shown in Figure 7 and the Across-Algo column of Table 2, for ADNI-1→ -2, ClassAMI has a higher correlation than InfoMax (ρ=0.24ρ=0.24 vs. 0.160.16) yet selects a less accurate model (80.8% vs. 83.2%). This is because the two measure different things: ρ captures how well a validator orders all checkpoints, whereas selection depends on its single top-scored checkpoint, so a validator with lower ρ can still place a good checkpoint at the top. This suggests two complementary needs: a reliable validator to produce a good ranking, and potentially a further step to identify the truly best one. How Far from Clinical Deployment? The answer is two-sided. Under Oracle selection, a capable model exists in the checkpoint pool in most scenarios, so adaptation itself is often effective. What is missing is the ability to identify that model without target labels. Deployment therefore depends on the complete pipeline, not adaptation alone: selection remains a largely unsolved step, and closing it would bring much of current UDA closer to clinical use. In some scenarios the achievable accuracy itself remains low, and for some algorithms even the Oracle-selected model falls below SourceOnly, so better adaptation algorithms are still needed. Yet even there, and for any future algorithm, a capable model still has to be identified. Improving algorithms raises what is achievable, while reliable selection determines what is actually reached at deployment. The two strategies in Section 4.4 recover part of this selection gap but do not close it, and both carry practical costs. Ensembling requires training all algorithms across their hyperparameters, since which suits a given transfer is not known in advance, and multiple forward passes per image at inference; it is computationally costly and can even harm performance under naive aggregation when poor checkpoints are included (e.g., CRD→ ). The labeling budget, though small in relative terms (5 to 25 volumes for brain MRI, 1% to 5% otherwise), carries a real annotation cost: even a single brain MRI volume is time-consuming, and 1% of a large dataset can amount to hundreds of images (e.g., over 260 for RSNA), more so for richer tasks such as segmentation or detection. Active selection [40, 32, 20] may match these gains with fewer labels. Overall, reliable label-free selection remains the central open problem for clinically deployable UDA. Limitations and Future Work. Our study covers binary classification; extending it to multi-class, multi-label, segmentation, or detection tasks, where the prediction structure differs and selection criteria may need adapting, is a valuable future direction. As classification lies at the core of these more complex tasks, label-free selection may not become easier there, and the gap could even widen. We evaluate by balanced accuracy, while other metrics such as sensitivity and specificity also matter clinically, and whether validators rank checkpoints well under them remains open. 5 Conclusion In this work, we evaluated the complete UDA pipeline in medical imaging, adaptation and label-free selection together, under clinical deployment conditions. Spanning eleven clinically relevant cross-domain scenarios from nine datasets across brain MRI, CXR, and retinal imaging, ten UDA algorithms, and 13 validators, our evaluation covers over 80,000 trained checkpoints. By considering adaptation and selection as a whole, we find that a capable adapted model often exists, but selecting it without target labels is difficult: the validator-selected models leave a large and structural gap to the best available one, as no evaluated validator is consistently reliable, for either within-algorithm or across-algorithm selection. Two strategies, ensembling and a small target-labeling budget, narrow this gap but do not close it entirely, and both have limitations. Overall, deployable UDA depends on both adaptation and selection working together; addressing the less explored selection step could bring much of current UDA closer to clinical use. Acknowledgments This study was funded by the German Research Foundation DFG (Project: KEMAI, GRK 3012 – 520750254) and by the German Federal Ministry of Research, Technology and Space BMFTR as part of the University Medicine Network 3.0 (Project: RACOON, 01KX2524). References [1] S. Chamarthi, K. Fogelberg, T. J. Brinker, et al. (2024) Mitigating the influence of domain shift in skin lesion classification: a benchmark study of unsupervised domain adaptation methods. Informatics in Medicine Unlocked 44, p. 101430. Cited by: §1, §2.1. [2] L. Chen, H. Chen, Z. Wei, et al. (2022) Reusing the task-specific classifier as a discriminator: discriminator-free adversarial domain adaptation. In CVPR, p. 7181–7190. Cited by: Table 1, §3. [3] A. Choudhary, L. Tong, Y. Zhu, et al. (2020) Advancing medical imaging informatics by deep learning-based domain adaptation. Yearbook of medical informatics 29 (01), p. 129–138. Cited by: §1, §2.1. [4] COVID-19 radiography database. Note: https://w.kaggle.com/datasets/tawsifurrahman/covid19-radiography-database Cited by: Table 1, §3. [5] S. Cui, S. Wang, J. Zhuo, et al. (2020) Towards discriminability and diversity: batch nuclear-norm maximization under label insufficient situations. In CVPR, p. 3941–3950. Cited by: Table 1, §3. [6] J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei (2009) Imagenet: a large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, p. 248–255. Cited by: §3. [7] R. Dorent, A. Kujawa, M. Ivory, S. Bakas, N. Rieke, S. Joutard, B. Glocker, J. Cardoso, M. Modat, K. Batmanghelich, et al. (2023) CrossMoDA 2021 challenge: benchmark of cross-modality domain adaptation techniques for vestibular schwannoma and cochlea segmentation. Medical Image Analysis 83, p. 102628. Cited by: §1, §2.1. [8] K. A. Ellis, A. I. Bush, D. Darby, et al. (2009) The australian imaging, biomarkers and lifestyle (aibl) study of aging: methodology and baseline characteristics of 1112 individuals recruited for a longitudinal study of alzheimer’s disease. International psychogeriatrics 21 (4), p. 672–687. Cited by: Table 1, §3. [9] L. Ericsson, D. Li, and T. Hospedales (2023) Better practices for domain adaptation. In International Conference on Automated Machine Learning, p. 4–1. Cited by: §2.2. [10] Y. Feng, Z. Wang, X. Xu, Y. Wang, H. Fu, S. Li, L. Zhen, X. Lei, Y. Cui, J. S. Z. Ting, et al. (2023) Contrastive domain adaptation with consistency match for automated pneumonia diagnosis. Medical Image Analysis 83, p. 102664. Cited by: §1, §3. [11] Y. Ganin and V. Lempitsky (2015) Unsupervised domain adaptation by backpropagation. In ICML, p. 1180–1189. Cited by: §2.2, Table 1, §3. [12] Y. Ganin, E. Ustinova, H. Ajakan, et al. (2016) Domain-adversarial training of neural networks. Journal of machine learning research 17 (59), p. 1–35. Cited by: Table 1, §3. [13] H. Guan and M. Liu (2021) Domain adaptation for medical image analysis: a survey. IEEE Transactions on Biomedical Engineering 69 (3), p. 1173–1185. Cited by: §1, §2.1. [14] H. Guan, Y. Liu, E. Yang, et al. (2021) Multi-site mri harmonization via attention-guided deep domain adaptation for brain disorder identification. Medical image analysis 71, p. 102076. Cited by: §1, Table 1, §3, §3, §3. [15] K. He, X. Zhang, S. Ren, et al. (2016) Deep residual learning for image recognition. In CVPR, p. 770–778. Cited by: §3. [16] D. Hu, J. Liang, J. H. Liew, C. Xue, S. Bai, and X. Wang (2023) Mixed samples as probes for unsupervised model selection in domain adaptation. Advances in Neural Information Processing Systems 36, p. 37923–37941. Cited by: §2.2, Table 1, §3. [17] D. Hu, R. Luo, J. Liang, et al. (2024) Towards reliable model selection for unsupervised domain adaptation: an empirical study and a certified baseline. NeurIPS 37, p. 135883–135903. Cited by: §2.2, §4.4. [18] C. R. Jack Jr, M. A. Bernstein, N. C. Fox, et al. (2008) The alzheimer’s disease neuroimaging initiative (adni): mri methods. Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine 27 (4), p. 685–691. Cited by: Table 1, Table 1, Table 1, §3. [19] Y. Jin, X. Wang, M. Long, et al. (2020) Minimum class confusion for versatile domain adaptation. In ECCV, p. 464–480. Cited by: Table 1, Table 1, §3. [20] J. Kay, G. Van Horn, S. Maji, et al. (2025) Consensus-driven active model selection. In ICCV, p. 4594–4604. Cited by: §4.5. [21] D. S. Kermany, M. Goldbaum, W. Cai, et al. (2018) Identifying medical diagnoses and treatable diseases by image-based deep learning. cell 172 (5), p. 1122–1131. Cited by: Table 1, §3. [22] S. Kumari and P. Singh (2024) Deep learning for unsupervised domain adaptation in medical imaging: recent advancements and future perspectives. Computers in Biology and Medicine 170, p. 107912. Cited by: §1, §2.1, §2.2. [23] Y. Lalou, T. Gnassounou, A. Collas, et al. (2024) SKADA-bench: benchmarking unsupervised domain adaptation methods with realistic validation on diverse modalities. arXiv preprint arXiv:2407.11676. Cited by: §2.2. [24] Y. Li and Y. Fan (2023) Medical image segmentation with domain adaptation: a survey. arXiv preprint arXiv:2311.01702. Cited by: §1, §2.1. [25] J. Liang, D. Hu, and J. Feng (2021) Domain adaptation with auxiliary target domain-oriented classifier. In CVPR, p. 16632–16642. Cited by: Table 1, §3. [26] J. Liu, H. Li, C. Yang, M. Deutges, A. Sadafi, X. You, K. Breininger, N. Navab, and P. J. Schüffler (2025) Hasd: hierarchical adaption for pathology slide-level domain-shift. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 332–342. Cited by: §1. [27] W. Liu, Z. Ni, Q. Chen, and L. Ni (2023) Attention-guided partial domain adaptation for automated pneumonia diagnosis from chest x-ray images. IEEE Journal of Biomedical and Health Informatics 27 (12), p. 5848–5859. Cited by: §3. [28] Z. Liu, H. Mao, C. Wu, et al. (2022) A convnet for the 2020s. CVPR. Cited by: §4.2. [29] M. Long, Y. Cao, J. Wang, et al. (2015) Learning transferable features with deep adaptation networks. In ICML, p. 97–105. Cited by: Table 1, §3. [30] M. Long, Z. Cao, J. Wang, et al. (2018) Conditional adversarial domain adaptation. NeurIPS 31. Cited by: Table 1, §3. [31] Lungs disease dataset (4 types). Note: https://w.kaggle.com/datasets/omkarmanohardalvi/lungs-disease-dataset-4-types Cited by: Table 1, §3. [32] M. Matsuura and S. Hara (2023) Active model selection: a variance minimization approach. In NeurIPS 2023 Workshop on Adaptive Experimental Design and Active Learning in the Real World, Cited by: §4.5. [33] P. Morerio, J. Cavazza, and V. Murino (2017) Minimal-entropy correlation alignment for unsupervised deep domain adaptation. arXiv preprint arXiv:1711.10288. Cited by: Table 1, §3. [34] K. Musgrave, S. Belongie, and S. Lim (2021) Unsupervised domain adaptation: a reality check. arXiv preprint arXiv:2111.15672. Cited by: §1, §2.2, Table 1, §3. [35] K. Musgrave, S. Belongie, and S. Lim (2022) Three new validators and a large-scale benchmark ranking for unsupervised domain adaptation. arXiv preprint arXiv:2208.07360. Cited by: §2.2, Table 1, Table 1, §3. [36] B. Pu, L. Wang, J. Yang, et al. (2024) M3-uda: a new benchmark for unsupervised domain adaptive fetal cardiac structure detection. In CVPR, p. 11621–11630. Cited by: §1, §2.1. [37] K. Saito, D. Kim, P. Teterwak, et al. (2021) Tune it the right way: unsupervised validation of domain adaptation via soft neighborhood density. In ICCV, p. 9184–9193. Cited by: §2.2, Table 1, §3. [38] K. Saito, K. Watanabe, Y. Ushiku, et al. (2018) Maximum classifier discrepancy for unsupervised domain adaptation. In CVPR, p. 3723–3732. Cited by: Table 1, §3. [39] G. Sarafraz, A. Behnamnia, M. Hosseinzadeh, et al. (2022) Domain adaptation and generalization on functional medical images: a systematic survey. arXiv preprint arXiv:2212.03176. Cited by: §1, §2.1. [40] C. Sawade, N. Landwehr, and T. Scheffer (2012) Active comparison of prediction models. Advances in neural information processing systems 25. Cited by: §4.5. [41] B. Shirokikh, A. Kurmukov, M. Donskova, et al. (2025) M3DA: benchmark for unsupervised domain adaptation in 3d medical image segmentation. arXiv preprint arXiv:2502.17029. Cited by: §1, §2.1. [42] Z. Su, J. Guo, X. Yang, Q. Wang, F. Coenen, A. Hussain, and K. Huang (2024) Navigating distribution shifts in medical image analysis: a survey. arXiv preprint arXiv:2411.05824. Cited by: §1, §2.1. [43] M. Sugiyama, M. Krauledat, and K. Müller (2007) Covariate shift adaptation by importance weighted cross validation.. JMLR 8 (5). Cited by: §1, §2.2, Table 1, §3. [44] N. Sultana, W. Lu, X. Fan, et al. (2025) Domain adaptation for skin lesion: evaluating real-world generalisation. In CVPRW, p. 3433–3443. Cited by: §1, §2.1. [45] Y. Tian, C. Wen, M. Shi, M. M. Afzal, H. Huang, M. O. Khan, Y. Luo, Y. Fang, and M. Wang (2024) Fairdomain: achieving fairness in cross-domain medical image segmentation and classification. In European Conference on Computer Vision, p. 251–271. Cited by: Table 1, §3. [46] H. Touvron, P. Bojanowski, M. Caron, et al. (2022) Resmlp: feedforward networks for image classification with data-efficient training. IEEE TPAMI 45 (4), p. 5314–5321. Cited by: §4.2. [47] H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou (2021) Training data-efficient image transformers & distillation through attention. In International conference on machine learning, p. 10347–10357. Cited by: §4.2. [48] W. Tu, W. Deng, T. Gedeon, et al. Assessing model out-of-distribution generalization with softmax prediction probability baselines and a correlation method. Cited by: §2.2, Table 1, §3. [49] X. Wang, Y. Peng, L. Lu, et al. (2017) Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In CVPR, p. 2097–2106. Cited by: Table 1, §3. [50] Q. Xie, Y. Li, N. He, M. Ning, K. Ma, G. Wang, Y. Lian, and Y. Zheng (2022) Unsupervised domain adaptation for medical image segmentation by disentanglement learning and self-training. IEEE Transactions on Medical Imaging 43 (1), p. 4–14. Cited by: §1. [51] H. Yang, H. Guo, G. Liu, L. Zhou, J. Xu, L. Wei, R. Su, and Q. Jin (2026) A survey on unsupervised domain adaptation in medical imaging: methods, dataset, and future outlook. Applied Soft Computing, p. 115314. Cited by: §1, §2.1. [52] J. Yang, H. Qian, Y. Xu, K. Wang, and L. Xie (2024) Can we evaluate domain adaptation models without target-domain labels?. In International Conference on Learning Representations, Vol. 2024, p. 35061–35081. Cited by: §2.2, Table 1, §3. [53] K. You, X. Wang, M. Long, et al. (2019) Towards accurate model selection in deep unsupervised domain adaptation. In ICML, p. 7124–7133. Cited by: §1, §2.2, Table 1, §3. [54] Y. Zhang, Y. Wei, Q. Wu, et al. (2020) Collaborative unsupervised domain adaptation for medical image diagnosis. IEEE TIP 29, p. 7834–7844. Cited by: §1, Table 1, §3, §3. 6 Supplementary Material In this supplementary material, we provide the full results of our complete unsupervised domain adaptation (UDA) pipeline for each evaluated clinically relevant cross-domain scenario in Section 6.1. In addition, we provide the full within-algorithm validator reliability analyses, reported per algorithm and per scenario, in Section 6.2. Figure 8: Target accuracy under Oracle selection across UDA scenarios, reported as the median with 95% confidence intervals. Bars show SourceOnly (no adaptation), the per-algorithm average (Avg.), Across-Algo (selection over all algorithms’ pooled checkpoints), and TargetOnly (labeled-target). Across-Algo exceeds SourceOnly in all scenarios and approaches TargetOnly, indicating a capable adapted model usually exists. Error bars: 95% confidence interval. Figure 9: Target accuracy across UDA scenarios under three selections (%), reported as the median with 95% confidence intervals. Bars show the accuracy of the model selected by the best validator over the across-algorithm pool (Best Validator), by the best algorithm–validator combination (Best Pair), and by the across-algorithm Oracle. The first two fall below the Oracle in every scenario, with the gap up to 9.9 points and averaging 5.1 (Mean). Best Validator and Best Pair differ from scenario to scenario and are identified using target labels, so they cannot be chosen in advance; the gap realized at deployment can only be larger. Error bars: 95% confidence interval. 6.1 Full Results on the Complete UDA Pipeline This section provides the full per-scenario complete UDA results. Each table reports, for one cross-domain scenario, the target accuracy of the checkpoint selected by every validator (rows) for every algorithm (columns), along with Oracle, SourceOnly, and TargetOnly references. For each scenario, the first table reports mean± and the immediately following table reports median with 95% CI (e.g., Table 5 is mean± and Table 6 is median with 95% CI, and analogously for the remaining ten scenarios). Tables 5–25 cover the eleven clinically relevant transfer scenarios. Brain MRI: ADNI-1→ -2 (Table 5), ADNI-1→ -3 (Table 7), ADNI-2→ -1 (Table 9), ADNI-2→ -3 (Table 11), ADNI-1+2→ (Table 13). Chest X-ray: RSNA→ CXR (Table 15), Child CXR→ (Table 17), LDD→ (Table 19), CRD→ (Table 21). Retinal: OCT→ (Table 23), SLO→ (Table 25). We also assess whether adaptation works in principle and the size of the selection gap using the median with 95% confidence intervals, shown in Figures 8 and 9. The conclusions match those drawn from the mean ± standard deviation in the main paper. Table 5: Complete UDA results on ADNI-1→ -2 (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD AD2A Avg. Across-Algo TargetOnly Oracle 88.2± 2.0 89.3± 4.0 90.0± 4.0 89.9± 3.0 91.0± 3.5 91.0± 1.7 89.9± 4.4 85.7± 9.8 87.4± 4.3 92.0± 0.93 89.5± 2.0 93.2± 2.0 90.9± 2.4 Source-Risk 80.9± 5.8 82.1± 3.1 81.5± 3.7 81.1± 8.0 80.8± 9.1 83.3± 3.2 84.1± 4.1 73.2± 12 81.5± 6.2 87.5± 2.3 - 83.5± 2.9 - IWCV 76.8± 9.7 80.3± 5.3 74.6± 12 81.1± 6.4 80.1± 8.4 76.0± 10 76.4± 3.8 72.3± 11 80.1± 11 84.3± 2.8 - 76.7± 4.4 - DEV 78.1± 11 83.1± 2.0 83.9± 5.2 82.0± 6.7 85.2± 6.2 76.9± 11 75.9± 7.3 68.0± 12 74.7± 14 82.5± 8.6 - 79.4± 13 - DEV-N 80.9± 5.8 81.7± 2.8 84.9± 2.0 81.8± 8.4 80.8± 9.1 83.5± 3.3 82.9± 3.4 76.0± 12 81.0± 6.2 87.6± 1.9 - 85.1± 1.3 - Entropy 60.9± 15 58.8± 20 56.7± 15 57.0± 16 50.0± 15 60.2± 15 57.5± 16 57.9± 13 70.1± 19 56.9± 15 - 50.0± 10 - InfoMax 84.2± 2.1 84.8± 5.7 87.3± 4.4 86.0± 1.5 88.2± 3.8 81.8± 9.6 83.5± 6.9 78.9± 10 83.3± 5.6 86.3± 2.2 - 83.2± 7.2 - Corr-C 84.6± 4.1 80.4± 4.3 83.4± 3.6 83.1± 3.6 84.2± 6.7 77.8± 5.3 80.4± 3.4 78.3± 8.8 84.1± 4.4 85.6± 5.1 - 79.6± 3.0 - MCC (V) 84.2± 2.7 84.4± 6.4 84.9± 3.3 84.5± 4.0 86.4± 3.7 80.4± 8.3 85.9± 3.6 80.7± 13 83.1± 3.7 87.1± 3.7 - 80.4± 2.3 - BNM (V) 84.7± 2.1 84.8± 5.7 87.5± 4.0 84.9± 3.8 87.2± 2.2 81.8± 9.6 83.5± 6.9 78.9± 10 83.3± 5.6 86.3± 2.2 - 81.8± 9.6 - ClassAMI 81.3± 5.8 71.5± 13 74.8± 8.7 83.0± 4.0 84.2± 1.6 81.7± 7.5 79.6± 4.4 78.3± 10 63.7± 19 72.7± 16 - 80.8± 3.6 - SND 73.9± 12 83.4± 4.1 80.0± 2.6 79.2± 5.8 85.5± 4.9 78.2± 5.9 78.5± 8.6 75.8± 12 82.2± 5.6 80.7± 8.7 - 80.7± 8.7 - MixVal 75.9± 12 57.4± 6.4 77.9± 8.7 69.8± 13 66.8± 16 82.1± 8.7 71.3± 16 70.2± 15 71.8± 16 60.0± 16 - 61.1± 18 - TransScore 81.3± 1.0 77.5± 11 85.1± 4.5 86.7± 2.3 85.2± 5.9 80.4± 9.9 76.5± 11 72.7± 10 82.4± 3.7 86.3± 2.8 - 79.9± 10 - Table 6: Complete UDA results on ADNI-1→ -2 (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean± results in Table 5. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM AD2A ATDOC MCD Avg. Across-Algo TargetOnly Oracle 89.3± 2.4 88.6± 5.4 89.1± 5.2 89.8± 3.7 91.7± 4.8 91.3± 1.9 90.3± 4.8 92.3± 1.2 89.3± 12 87.7± 5.4 90.0± 1.5 92.8± 2.6 91.7± 3.0 Source-Risk 80.7± 7.9 82.2± 3.8 82.0± 4.5 82.7± 10.0 82.0± 12 83.8± 4.1 81.4± 4.1 87.3± 3.0 74.6± 14 81.1± 7.3 - 83.8± 4.0 - IWCV 75.8± 13 80.7± 7.0 70.7± 14 80.7± 8.4 83.7± 10 79.2± 12 76.5± 4.8 86.9± 7.5 74.6± 14 85.8± 13 - 79.2± 5.2 - DEV 83.3± 13 82.2± 3.8 83.7± 6.7 83.7± 8.4 87.5± 6.4 80.8± 13 79.4± 8.9 86.9± 9.2 66.0± 15 77.7± 18 - 81.6± 17 - Entropy 53.2± 17 50.0± 0.85 50.0± 17 50.0± 18 50.0± 0.0 50.0± 17 50.0± 18 50.0± 17 50.0± 19 77.3± 19 - 50.0± 0.0 - InfoMax 85.0± 2.5 82.3± 7.2 86.5± 6.0 86.8± 1.9 86.7± 4.7 83.3± 12 84.3± 8.5 85.7± 2.7 84.5± 12 81.1± 7.4 - 83.3± 9.0 - Corr-C 85.0± 4.5 82.5± 4.9 82.3± 4.3 82.7± 4.7 87.3± 7.7 79.3± 6.5 82.0± 4.2 87.9± 6.1 81.7± 11 81.4± 5.0 - 79.3± 3.8 - MCC (V) 84.8± 3.7 84.2± 8.6 84.5± 4.3 85.6± 4.9 86.7± 4.6 83.3± 11 86.7± 4.5 87.5± 4.7 85.2± 17 81.4± 4.7 - 83.3± 11 - BNM (V) 85.7± 2.5 82.3± 7.2 86.5± 5.2 86.8± 4.6 86.7± 2.4 83.3± 12 84.3± 8.5 85.7± 2.7 84.5± 12 81.1± 7.4 - 83.3± 12 - SND 79.0± 14 83.3± 4.8 79.7± 3.6 75.9± 6.8 84.5± 6.3 78.8± 8.0 79.6± 11 82.8± 5.4 81.7± 14 83.4± 7.6 - 79.7± 10 - ClassAMI 83.3± 7.2 75.7± 18 77.7± 10 81.3± 4.7 84.2± 2.2 84.1± 9.5 79.6± 5.0 73.8± 20 84.4± 10 51.0± 19 - 81.3± 4.9 - DEV-N 80.7± 7.9 82.2± 3.4 85.2± 2.8 82.7± 10.0 82.0± 12 84.4± 4.1 81.4± 4.1 86.9± 2.4 74.6± 14 78.8± 7.1 - 85.0± 1.7 - MixVal 80.3± 15 55.7± 8.5 78.0± 11 71.3± 16 65.2± 18 83.2± 10 77.7± 18 68.8± 20 67.6± 19 79.8± 20 - 53.0± 19 - TransScore 81.3± 0.93 77.3± 14 86.2± 5.5 86.4± 3.2 83.8± 6.9 83.3± 11 81.8± 13 86.0± 3.2 74.5± 13 81.1± 5.0 - 83.3± 11 - Table 7: Complete UDA results on ADNI-1→ -3 (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD AD2A Avg. Across-Algo TargetOnly Oracle 90.2± 3.7 87.3± 4.5 90.7± 3.1 93.5± 3.5 90.8± 4.1 90.8± 3.3 89.0± 5.6 86.9± 5.8 88.6± 3.5 91.1± 4.3 89.9± 2.0 94.2± 3.1 90.2± 4.6 Source-Risk 82.0± 9.5 79.0± 8.9 83.8± 5.3 82.7± 8.8 82.8± 7.6 76.8± 5.1 78.0± 7.1 77.9± 10 82.9± 5.1 82.4± 7.6 - 82.9± 5.2 - IWCV 70.0± 10 76.7± 11 79.6± 4.9 84.1± 5.9 74.7± 15 74.0± 12 56.5± 9.1 69.3± 11 79.2± 6.7 75.3± 12 - 64.9± 15 - DEV 73.9± 12 69.2± 15 84.2± 3.3 85.0± 6.1 81.3± 7.6 80.4± 6.9 72.7± 15 74.5± 12 71.5± 12 77.4± 14 - 84.2± 7.8 - DEV-N 83.3± 7.1 79.2± 9.7 83.8± 5.3 82.7± 8.8 82.4± 5.0 77.3± 5.7 76.1± 8.5 76.6± 12 75.2± 9.8 83.2± 4.4 - 84.5± 5.0 - Entropy 67.8± 13 66.8± 15 57.2± 15 57.3± 14 63.2± 18 66.1± 17 64.0± 11 64.1± 18 62.5± 17 50.3± 0.72 - 50.0± 0.0 - InfoMax 79.0± 5.9 72.0± 14 79.7± 5.0 78.2± 4.1 79.4± 6.8 70.6± 7.9 72.7± 6.5 72.4± 12 80.2± 5.8 80.2± 5.7 - 70.6± 7.9 - Corr-C 79.7± 6.7 70.8± 5.1 76.8± 1.5 75.9± 3.7 74.5± 4.3 71.3± 8.3 69.9± 5.0 67.1± 13 76.1± 4.8 72.1± 11 - 68.7± 8.5 - MCC (V) 79.1± 7.9 74.9± 14 84.6± 2.3 84.5± 4.0 79.9± 6.3 75.9± 9.8 72.4± 4.2 73.8± 18 85.1± 3.9 83.5± 7.6 - 75.9± 9.8 - BNM (V) 78.2± 7.3 75.0± 15 79.7± 5.0 78.2± 4.1 79.4± 6.8 70.6± 7.9 72.7± 6.5 74.3± 13 80.9± 4.9 80.2± 5.7 - 70.6± 7.9 - ClassAMI 79.5± 6.2 55.1± 19 80.2± 3.6 76.0± 6.5 82.8± 6.9 84.6± 5.1 79.6± 8.9 68.6± 19 59.9± 15 75.7± 19 - 79.2± 9.7 - SND 73.8± 6.0 71.6± 5.0 77.5± 4.5 75.4± 6.0 76.8± 4.3 73.2± 5.3 71.2± 9.0 69.4± 8.8 77.4± 3.2 74.5± 2.0 - 76.2± 3.9 - MixVal 78.2± 11 64.9± 18 73.3± 9.1 78.1± 12 66.7± 14 82.3± 4.0 72.6± 14 72.7± 11 53.4± 6.0 59.8± 12 - 64.7± 13 - TransScore 81.9± 6.1 78.5± 8.2 83.1± 3.1 84.6± 7.2 79.8± 2.6 69.6± 7.1 58.3± 19 79.2± 4.0 78.8± 3.9 77.0± 12 - 71.1± 8.0 - Table 8: Complete UDA results on ADNI-1→ -3 (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean± results in Table 7. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM AD2A ATDOC MCD Avg. Across-Algo TargetOnly Oracle 88.5± 4.5 86.2± 5.1 90.0± 3.5 93.8± 4.7 91.6± 5.5 90.7± 4.4 87.8± 6.9 90.8± 5.0 86.0± 7.4 87.6± 4.6 89.4± 2.6 94.8± 3.8 90.7± 6.5 Source-Risk 82.8± 12 79.2± 10 85.4± 6.7 80.1± 11 86.1± 9.8 78.5± 6.2 79.1± 8.7 84.7± 9.6 80.5± 13 84.9± 6.3 - 81.4± 6.4 - IWCV 64.3± 11 78.9± 15 81.5± 5.3 86.1± 7.6 77.8± 19 79.4± 15 56.0± 11 75.5± 17 69.8± 14 76.1± 7.6 - 60.5± 16 - DEV 73.1± 14 73.6± 17 83.8± 4.2 83.7± 7.1 79.5± 10 84.1± 8.2 74.9± 20 81.0± 18 80.4± 14 73.3± 16 - 84.1± 10 - Entropy 68.4± 17 64.3± 19 50.0± 17 50.0± 16 51.2± 20 64.5± 19 69.6± 13 50.0± 0.81 66.7± 23 51.0± 17 - 50.0± 0.0 - InfoMax 77.0± 7.3 73.3± 18 80.8± 6.6 79.7± 5.0 77.4± 8.7 72.4± 11 75.0± 8.2 79.3± 6.8 76.5± 14 78.4± 6.9 - 72.4± 11 - Corr-C 79.0± 8.0 72.6± 6.3 76.7± 1.8 76.7± 4.0 76.0± 5.2 72.4± 11 70.7± 6.3 74.6± 14 72.6± 14 77.9± 6.1 - 71.6± 11 - MCC (V) 77.0± 10 80.7± 18 84.2± 3.1 86.0± 4.5 76.8± 7.4 72.2± 12 72.8± 5.3 79.3± 7.9 78.1± 23 86.6± 4.9 - 72.2± 12 - BNM (V) 75.5± 8.9 82.4± 18 80.8± 6.6 79.7± 5.0 77.4± 8.7 72.4± 11 75.0± 8.2 79.3± 6.8 78.1± 17 78.4± 5.2 - 72.4± 11 - SND 74.2± 7.9 72.6± 6.2 78.7± 5.5 74.3± 7.5 77.1± 5.4 74.9± 6.7 69.3± 11 74.0± 2.3 72.8± 11 77.9± 4.1 - 77.1± 5.2 - ClassAMI 78.0± 7.6 53.2± 24 81.5± 4.3 74.9± 8.8 83.8± 8.8 84.1± 5.9 78.2± 10 80.0± 24 73.7± 24 55.6± 18 - 80.0± 11 - DEV-N 82.8± 7.8 77.6± 11 85.4± 6.7 80.1± 11 80.8± 6.4 78.5± 7.6 78.0± 11 84.7± 4.5 80.5± 16 77.9± 11 - 84.9± 6.4 - MixVal 80.9± 14 57.7± 18 70.8± 11 80.7± 17 69.4± 16 81.6± 4.9 75.4± 17 54.8± 15 74.8± 15 56.1± 7.2 - 65.3± 16 - TransScore 83.4± 7.8 76.5± 10 82.6± 4.1 88.4± 8.6 80.1± 3.2 71.9± 9.2 63.1± 20 79.3± 15 78.7± 5.1 80.1± 4.3 - 72.4± 10 - Table 9: Complete UDA results on ADNI-2→ -1 (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD AD2A Avg. Across-Algo TargetOnly Oracle 89.3± 4.0 89.8± 4.2 91.3± 2.2 90.8± 2.7 91.3± 2.9 91.3± 2.8 91.1± 2.3 85.0± 4.1 85.2± 5.3 91.0± 1.9 89.6± 2.5 92.5± 1.8 91.0± 4.2 Source-Risk 79.5± 8.1 82.6± 5.1 85.7± 3.9 84.8± 5.9 84.2± 3.6 86.3± 5.6 83.7± 4.5 77.0± 4.5 77.7± 8.2 88.9± 3.6 - 79.5± 6.3 - IWCV 81.2± 6.3 80.6± 2.2 80.6± 3.6 81.8± 6.0 76.8± 9.1 67.3± 15 79.7± 3.0 75.3± 7.7 70.0± 9.8 77.7± 15 - 75.3± 7.7 - DEV 82.8± 2.9 79.3± 2.0 80.5± 6.9 85.3± 3.3 86.7± 3.0 82.5± 4.6 85.2± 1.9 74.8± 5.2 68.0± 6.6 70.8± 19 - 84.9± 3.3 - DEV-N 80.7± 8.3 81.4± 3.6 84.7± 3.7 84.8± 5.7 83.3± 2.9 85.8± 3.7 82.5± 2.5 77.0± 4.5 76.8± 9.0 87.9± 5.3 - 79.5± 6.3 - Entropy 63.2± 19 60.8± 15 62.5± 17 63.8± 19 57.9± 14 64.5± 13 69.8± 18 66.0± 15 70.0± 14 65.0± 20 - 55.8± 13 - InfoMax 86.0± 5.1 83.1± 6.4 87.2± 3.3 85.9± 3.7 86.0± 2.9 85.4± 3.0 86.4± 5.1 64.9± 5.4 63.6± 12 87.6± 3.8 - 85.7± 2.9 - Corr-C 84.5± 5.6 81.5± 6.5 84.9± 3.9 85.5± 5.5 82.2± 6.0 86.0± 5.1 81.7± 3.0 80.4± 5.2 77.0± 8.4 86.6± 2.9 - 82.4± 5.5 - MCC (V) 85.5± 5.3 85.8± 3.1 85.2± 4.7 84.7± 3.0 85.9± 3.7 75.3± 11 80.9± 7.4 80.1± 6.3 77.4± 7.2 85.5± 3.9 - 80.1± 6.3 - BNM (V) 86.4± 4.9 85.2± 4.4 87.2± 3.3 86.2± 3.8 85.8± 2.7 85.4± 3.0 85.5± 4.3 65.7± 7.2 63.7± 6.2 86.5± 3.5 - 65.1± 7.0 - ClassAMI 84.4± 5.1 77.5± 8.3 86.2± 5.0 85.4± 3.9 82.9± 3.8 82.6± 2.1 82.3± 4.4 79.5± 0.83 63.7± 15 80.3± 6.2 - 81.8± 3.0 - SND 78.5± 8.0 80.1± 3.5 81.4± 4.6 82.5± 3.1 81.5± 5.0 82.4± 4.1 76.8± 6.3 75.0± 3.1 74.6± 3.6 80.6± 2.6 - 78.8± 6.6 - MixVal 83.2± 6.5 73.9± 12 71.0± 13 82.3± 4.0 72.7± 17 73.4± 14 78.9± 7.8 75.0± 13 60.8± 16 68.8± 14 - 63.3± 11 - TransScore 82.4± 5.7 87.3± 3.3 85.1± 6.2 83.7± 3.9 84.3± 4.0 86.4± 3.5 85.8± 3.5 79.5± 3.2 76.3± 7.9 83.9± 1.8 - 86.4± 3.5 - Table 10: Complete UDA results on ADNI-2→ -1 (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean± results in Table 9. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM AD2A ATDOC MCD Avg. Across-Algo TargetOnly Oracle 91.3± 4.8 91.6± 5.4 91.7± 2.9 90.5± 3.2 92.0± 3.5 92.4± 3.4 91.6± 3.0 90.9± 2.6 84.1± 5.2 82.0± 5.7 89.7± 3.8 93.1± 2.3 92.8± 4.6 Source-Risk 80.8± 10 81.1± 6.2 85.3± 5.1 84.8± 7.3 85.0± 4.4 87.5± 6.7 83.7± 6.0 89.8± 4.3 78.8± 5.3 72.6± 9.2 - 79.8± 8.1 - IWCV 83.3± 7.9 80.8± 2.5 80.6± 4.6 80.3± 7.1 79.2± 12 65.0± 20 80.2± 4.2 82.0± 20 74.3± 8.9 66.9± 13 - 74.3± 8.9 - DEV 82.6± 3.8 79.5± 2.7 78.1± 9.1 86.1± 3.9 87.5± 3.0 84.3± 5.1 85.3± 2.6 82.4± 18 74.9± 6.4 66.5± 8.8 - 86.0± 4.2 - Entropy 50.0± 21 50.0± 15 50.0± 18 50.0± 20 53.1± 16 60.2± 17 79.2± 19 50.0± 19 73.8± 14 70.6± 19 - 50.0± 14 - InfoMax 87.8± 6.4 85.2± 7.7 88.7± 3.9 86.2± 5.1 85.0± 3.4 85.5± 3.6 87.4± 6.5 88.9± 4.4 64.7± 6.9 60.4± 13 - 85.7± 3.6 - Corr-C 84.7± 7.1 85.2± 6.8 85.0± 5.2 87.0± 7.2 83.0± 7.4 85.8± 6.2 82.0± 3.5 86.6± 3.4 78.6± 6.3 76.2± 12 - 79.8± 6.2 - MCC (V) 86.9± 6.4 87.7± 3.3 85.7± 5.9 85.2± 3.8 86.0± 4.7 78.6± 13 83.7± 9.5 86.4± 4.6 78.8± 8.3 75.3± 9.3 - 78.8± 8.3 - BNM (V) 87.8± 6.4 85.2± 5.1 88.7± 3.9 87.2± 5.1 85.0± 3.2 85.5± 3.6 86.7± 5.7 84.8± 4.2 66.5± 8.3 65.0± 7.7 - 65.0± 8.3 - SND 81.3± 10 80.7± 4.6 82.3± 5.8 81.4± 3.8 83.7± 6.1 81.1± 5.2 78.5± 7.6 79.7± 3.1 75.2± 4.0 75.1± 4.7 - 81.2± 8.3 - ClassAMI 86.9± 6.4 79.2± 11 87.2± 6.4 85.5± 4.9 82.7± 4.7 83.0± 2.6 80.5± 5.0 81.4± 8.0 79.6± 1.0 59.5± 19 - 79.8± 3.2 - DEV-N 82.6± 10 81.1± 4.6 84.1± 5.1 85.7± 7.3 84.1± 3.8 84.3± 4.6 83.7± 3.0 89.8± 6.9 78.8± 5.3 70.8± 9.5 - 79.8± 8.1 - MixVal 85.3± 8.1 71.3± 15 75.0± 13 81.1± 5.0 82.2± 17 79.2± 15 81.1± 10 70.0± 14 78.8± 17 53.7± 20 - 58.1± 13 - TransScore 81.0± 7.5 88.3± 4.2 83.2± 6.6 85.0± 4.8 85.8± 4.7 86.8± 4.8 84.0± 3.8 84.0± 2.5 80.9± 3.9 73.8± 9.7 - 86.8± 4.8 - Table 11: Complete UDA results on ADNI-2→ -3 (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD AD2A Avg. Across-Algo TargetOnly Oracle 88.1± 3.8 89.3± 3.8 91.1± 3.3 91.2± 3.0 90.7± 2.9 90.1± 4.1 90.1± 2.9 89.2± 3.1 90.0± 3.0 91.2± 2.8 90.3± 0.74 92.1± 2.4 90.2± 4.6 Source-Risk 82.0± 7.2 81.2± 5.7 80.8± 6.8 84.9± 5.9 83.0± 2.6 82.4± 6.5 79.2± 7.4 81.3± 5.4 84.0± 6.5 85.2± 6.4 - 82.3± 7.7 - IWCV 81.0± 2.3 79.6± 2.2 82.0± 5.8 84.4± 6.2 84.0± 7.0 81.2± 2.9 78.8± 8.0 77.4± 7.6 82.0± 5.8 85.2± 5.5 - 84.2± 6.8 - DEV 81.6± 7.7 80.5± 5.4 84.3± 3.2 81.0± 6.8 82.4± 8.2 80.6± 6.7 85.3± 6.2 77.6± 3.5 83.6± 6.5 83.0± 8.9 - 80.3± 7.8 - DEV-N 82.0± 7.2 80.7± 3.1 81.1± 7.3 84.1± 6.5 83.7± 3.4 83.2± 5.8 80.7± 5.7 80.2± 6.8 82.8± 5.7 85.5± 5.9 - 78.3± 9.7 - Entropy 57.0± 16 56.6± 15 65.8± 22 58.4± 19 49.9± 0.26 50.0± 0.0 64.2± 20 69.0± 18 67.7± 16 51.9± 4.3 - 50.0± 0.0 - InfoMax 81.5± 4.8 83.2± 5.4 83.5± 4.9 82.7± 4.7 78.9± 6.3 81.7± 3.8 80.1± 3.0 82.6± 7.0 78.5± 8.7 84.8± 6.5 - 80.6± 3.2 - Corr-C 74.8± 10 77.5± 2.8 74.0± 4.7 76.7± 1.5 72.3± 6.6 76.0± 4.9 76.9± 2.7 72.4± 2.6 79.2± 7.0 79.7± 5.4 - 76.9± 2.7 - MCC (V) 83.8± 6.7 82.7± 5.3 86.7± 6.4 85.6± 5.8 84.6± 6.8 84.7± 6.3 82.4± 6.4 78.6± 7.2 79.9± 7.6 74.4± 19 - 82.8± 3.9 - BNM (V) 81.2± 5.2 83.2± 5.4 85.3± 1.3 82.7± 4.7 78.9± 6.3 81.7± 3.8 80.1± 3.0 54.4± 7.5 60.9± 17 83.9± 6.0 - 54.4± 7.5 - ClassAMI 79.5± 9.9 82.2± 4.8 79.8± 4.7 81.9± 1.9 83.7± 7.8 82.0± 6.2 84.4± 6.2 82.5± 7.0 69.0± 13 86.5± 3.8 - 85.0± 3.7 - SND 76.0± 5.1 77.7± 2.8 74.2± 4.4 71.7± 3.9 74.9± 8.2 74.8± 2.0 75.4± 2.2 76.5± 4.5 78.5± 5.5 75.3± 9.9 - 71.2± 5.7 - MixVal 83.8± 6.1 63.9± 13 66.5± 16 75.2± 13 70.6± 16 75.6± 13 65.9± 18 75.8± 15 65.9± 17 71.0± 18 - 59.0± 12 - TransScore 82.1± 6.5 80.3± 7.2 83.0± 3.4 80.8± 4.7 82.2± 3.8 79.7± 5.4 79.2± 1.7 73.9± 7.8 81.8± 7.0 85.9± 4.5 - 79.7± 2.2 - Table 12: Complete UDA results on ADNI-2→ -3 (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean± results in Table 11. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM AD2A ATDOC MCD Avg. Across-Algo TargetOnly Oracle 85.9± 4.5 89.3± 5.0 92.0± 3.9 90.8± 4.1 90.3± 3.7 90.3± 5.6 88.9± 3.7 90.7± 3.4 87.5± 3.6 89.1± 3.5 89.9± 1.3 92.1± 3.1 90.7± 6.5 Source-Risk 84.7± 9.5 80.8± 7.0 77.8± 7.7 84.2± 8.3 83.4± 3.0 79.6± 8.0 79.5± 10 88.0± 7.4 83.9± 6.5 87.5± 7.5 - 80.8± 11 - IWCV 81.1± 2.9 80.8± 2.7 79.6± 6.1 84.3± 7.7 80.1± 8.1 81.9± 3.8 77.2± 10 88.0± 6.0 80.7± 9.0 80.6± 6.9 - 81.9± 8.9 - DEV 80.9± 11 77.5± 6.6 84.2± 4.3 79.3± 9.0 78.4± 10 80.7± 7.8 82.6± 7.4 85.3± 12 77.5± 4.8 82.7± 8.7 - 78.3± 10 - Entropy 50.0± 17 50.0± 17 50.0± 22 50.0± 21 50.0± 0.29 50.0± 0.0 50.0± 20 50.0± 4.8 63.4± 20 71.3± 19 - 50.0± 0.0 - InfoMax 80.2± 6.1 84.8± 6.7 85.6± 6.0 81.9± 5.9 77.8± 7.2 82.2± 4.2 79.5± 3.7 84.2± 7.5 83.4± 9.3 74.7± 8.8 - 81.4± 3.7 - Corr-C 77.6± 13 79.0± 3.4 75.6± 5.7 76.5± 1.8 73.5± 9.0 75.5± 6.1 77.3± 3.0 81.4± 6.8 72.4± 3.5 81.0± 8.5 - 77.3± 3.0 - MCC (V) 81.8± 7.9 82.6± 6.8 88.7± 8.2 83.0± 7.0 84.4± 9.2 84.2± 8.3 81.4± 7.9 80.2± 25 79.8± 9.7 76.8± 8.4 - 81.4± 4.3 - BNM (V) 80.2± 6.9 84.8± 6.7 85.6± 1.5 81.9± 5.9 77.8± 7.2 82.2± 4.2 79.5± 3.7 84.2± 7.5 53.6± 10 65.7± 22 - 53.6± 10 - SND 76.1± 7.0 77.5± 3.8 76.7± 5.2 71.1± 5.1 76.9± 11 75.0± 2.3 76.6± 2.6 80.0± 13 77.3± 5.9 80.0± 7.3 - 72.8± 7.5 - ClassAMI 80.4± 13 81.0± 6.1 77.1± 4.7 82.0± 2.5 83.5± 11 79.5± 7.5 83.7± 8.2 87.5± 4.7 84.1± 9.4 69.5± 18 - 85.2± 4.6 - DEV-N 84.7± 9.5 80.8± 4.3 77.8± 8.7 84.1± 8.3 84.8± 3.9 81.9± 7.1 79.5± 7.3 88.0± 6.8 82.8± 8.4 83.8± 6.9 - 77.2± 13 - MixVal 82.1± 7.7 56.9± 15 59.5± 19 77.4± 17 73.1± 19 79.5± 16 58.8± 18 69.8± 20 78.9± 19 56.7± 20 - 56.7± 15 - TransScore 79.0± 7.9 78.8± 9.1 83.5± 4.7 79.7± 6.5 82.2± 5.4 81.0± 7.1 79.5± 2.0 84.2± 5.5 73.0± 9.1 84.7± 7.4 - 80.0± 2.4 - Table 13: Complete UDA results on ADNI-1+2→ (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD AD2A Avg. Across-Algo TargetOnly Oracle 92.2± 2.3 91.6± 2.0 92.8± 1.1 93.4± 1.7 93.1± 2.0 91.8± 1.6 91.4± 1.9 88.0± 6.0 91.9± 1.7 93.1± 2.6 91.9± 1.5 93.8± 2.1 81.9± 3.3 Source-Risk 88.4± 3.0 85.0± 3.5 86.6± 4.5 87.8± 3.9 85.6± 8.2 87.6± 2.7 84.0± 3.2 79.9± 4.9 86.2± 4.6 86.5± 2.1 - 84.0± 5.1 - IWCV 86.8± 3.3 83.2± 4.6 83.8± 4.7 84.3± 6.0 84.2± 8.0 86.4± 4.5 83.3± 3.2 82.3± 4.2 83.2± 3.9 87.8± 1.9 - 84.4± 3.3 - DEV 86.3± 5.3 84.1± 5.0 85.3± 2.2 86.9± 7.8 89.2± 1.7 82.3± 5.0 83.3± 3.2 82.0± 3.6 84.1± 4.7 83.3± 2.5 - 89.0± 1.7 - DEV-N 87.6± 4.2 85.0± 3.5 85.0± 5.0 88.9± 3.5 85.6± 8.2 88.2± 2.3 84.0± 3.2 79.9± 4.9 84.6± 4.8 88.8± 3.7 - 82.7± 5.0 - Entropy 88.3± 3.3 84.5± 5.4 88.6± 4.8 86.9± 8.0 66.0± 20 66.7± 15 79.0± 16 80.7± 7.8 79.1± 16 80.5± 18 - 64.8± 16 - InfoMax 84.7± 6.7 82.4± 4.9 78.9± 3.2 84.2± 5.0 79.5± 7.2 79.4± 5.5 81.6± 5.0 82.2± 6.4 85.8± 4.2 82.9± 2.9 - 81.6± 5.0 - Corr-C 80.4± 6.8 77.3± 4.7 75.4± 3.0 77.5± 3.6 77.6± 5.8 74.3± 4.7 75.0± 3.4 77.3± 3.2 77.4± 3.6 75.9± 3.9 - 75.1± 3.2 - MCC (V) 88.3± 3.8 86.0± 3.0 84.7± 5.5 89.7± 3.4 86.3± 4.6 80.6± 11 83.8± 2.9 81.3± 6.2 84.8± 3.1 88.8± 3.8 - 84.1± 4.8 - BNM (V) 88.2± 4.1 84.7± 3.0 80.2± 1.1 83.9± 5.2 82.1± 8.4 79.4± 5.5 81.6± 5.0 65.7± 8.2 74.6± 13 86.7± 6.3 - 65.7± 8.2 - ClassAMI 86.6± 7.0 68.5± 19 83.9± 6.1 84.6± 4.6 85.3± 6.1 85.6± 2.8 85.9± 2.0 84.5± 6.2 80.5± 7.2 88.0± 3.0 - 88.0± 3.0 - SND 74.0± 1.8 75.2± 3.8 76.0± 5.6 77.9± 3.6 76.6± 6.8 75.2± 4.7 74.1± 5.7 74.8± 2.0 78.3± 4.4 73.4± 4.4 - 76.9± 5.2 - MixVal 88.8± 4.0 59.4± 12 82.0± 7.5 84.1± 8.7 77.8± 12 76.5± 16 81.2± 6.1 79.4± 5.5 76.9± 14 67.9± 13 - 70.9± 16 - TransScore 87.8± 2.0 89.5± 2.4 85.9± 5.3 86.6± 3.8 85.4± 3.3 80.7± 6.0 83.7± 2.9 79.9± 5.5 82.8± 4.1 86.5± 2.7 - 81.3± 6.4 - Table 14: Complete UDA results on ADNI-1+2→ (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean± results in Table 13. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM AD2A ATDOC MCD Avg. Across-Algo TargetOnly Oracle 91.5± 2.9 91.8± 2.1 92.5± 1.4 92.5± 1.9 92.0± 1.9 91.5± 1.9 91.3± 2.5 92.0± 3.2 89.6± 7.9 92.3± 2.1 91.7± 0.91 92.5± 2.4 80.3± 4.7 Source-Risk 86.9± 3.6 86.5± 3.5 87.2± 5.5 88.6± 5.3 86.6± 9.7 87.4± 3.0 85.3± 3.9 86.5± 2.4 78.7± 6.0 87.1± 5.5 - 85.1± 6.6 - IWCV 86.9± 4.1 84.0± 5.9 85.1± 6.3 85.2± 7.6 85.1± 11 84.9± 6.0 84.2± 3.8 88.5± 2.4 83.3± 5.5 82.8± 5.5 - 84.9± 4.2 - DEV 88.9± 6.4 85.3± 6.7 85.1± 2.7 87.7± 11 89.1± 2.3 84.8± 6.2 84.2± 3.8 81.8± 2.8 83.3± 4.1 83.4± 5.5 - 88.5± 2.3 - Entropy 89.9± 3.4 86.4± 6.7 90.0± 5.5 88.7± 10 53.9± 22 61.5± 18 85.3± 20 85.1± 22 83.7± 10 86.8± 19 - 61.5± 18 - InfoMax 85.8± 9.3 84.4± 6.2 79.7± 4.2 83.8± 6.6 80.5± 9.0 81.6± 6.7 83.5± 5.7 82.3± 3.7 84.3± 7.2 86.8± 5.2 - 83.5± 5.7 - Corr-C 76.6± 7.4 78.9± 6.2 76.6± 3.6 77.1± 4.0 79.1± 7.4 74.0± 6.0 75.5± 4.7 75.3± 5.3 77.8± 3.2 78.7± 4.5 - 76.0± 4.1 - MCC (V) 87.9± 4.3 86.8± 3.5 84.3± 6.9 88.4± 4.2 87.7± 5.8 85.6± 13 84.8± 3.8 88.2± 4.9 77.8± 6.4 85.4± 3.8 - 85.6± 6.2 - BNM (V) 86.3± 4.6 83.5± 3.4 79.9± 1.4 83.8± 6.6 84.9± 11 81.6± 6.7 83.5± 5.7 86.9± 7.0 62.1± 9.4 79.7± 16 - 62.1± 9.4 - SND 74.4± 2.1 76.0± 5.1 76.3± 7.1 77.8± 4.5 73.0± 8.0 74.0± 6.3 73.0± 7.1 73.2± 5.8 75.0± 2.1 78.8± 5.8 - 75.5± 6.4 - ClassAMI 87.2± 9.1 78.9± 20 82.8± 7.4 85.6± 5.5 81.8± 6.7 85.1± 3.1 86.1± 2.5 87.2± 3.9 87.5± 7.8 82.0± 9.7 - 87.2± 3.9 - DEV-N 86.9± 5.6 86.5± 3.5 82.8± 6.1 88.7± 4.3 86.6± 9.7 87.7± 3.0 85.3± 3.9 87.7± 4.8 78.7± 6.0 83.4± 5.7 - 81.8± 6.6 - MixVal 90.6± 4.9 57.7± 14 79.7± 8.2 86.6± 11 82.5± 14 82.8± 19 79.9± 7.6 61.5± 15 77.2± 6.6 81.1± 18 - 75.5± 19 - TransScore 87.4± 2.7 89.1± 3.3 85.8± 6.5 86.5± 4.7 86.6± 4.1 83.5± 7.1 84.8± 3.8 86.9± 3.4 78.6± 7.0 84.2± 4.8 - 84.0± 7.7 - Table 15: Complete UDA results on RSNA→ CXR (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD CoUDA Avg. Across-Algo TargetOnly Oracle 82.3± 2.1 82.1± 1.9 82.8± 2.0 84.9± 1.8 83.5± 1.4 86.5± 1.3 84.0± 1.1 81.5± 2.0 84.0± 1.2 89.5± 0.56 84.3± 2.3 89.5± 0.56 92.7± 1.1 Source-Risk 73.7± 4.2 72.2± 1.2 73.4± 2.2 72.4± 4.2 72.6± 2.6 83.5± 2.4 75.4± 5.9 51.5± 14 78.0± 4.5 84.8± 3.0 - 73.2± 4.3 - IWCV 73.3± 4.3 72.3± 4.6 74.1± 0.93 74.0± 3.0 76.6± 4.5 76.4± 6.3 74.5± 4.3 54.3± 12 76.9± 3.7 86.2± 1.5 - 76.4± 6.3 - DEV 73.3± 4.9 73.1± 9.2 72.1± 4.4 72.1± 3.4 71.7± 7.0 79.3± 6.3 78.4± 3.6 61.9± 17 69.5± 9.3 76.9± 6.0 - 72.6± 12 - DEV-N 71.8± 6.0 70.8± 3.5 71.7± 2.2 70.9± 2.7 74.1± 6.1 82.9± 2.3 74.8± 2.5 66.5± 9.7 73.6± 1.6 85.1± 3.5 - 75.8± 3.3 - Entropy 65.6± 12 58.0± 9.9 56.9± 6.1 63.2± 14 51.7± 3.4 56.2± 12 52.1± 3.0 57.2± 7.7 68.9± 11 71.0± 17 - 51.0± 0.68 - InfoMax 71.8± 3.6 74.5± 3.5 78.6± 2.1 80.2± 3.5 72.0± 5.3 81.6± 1.2 72.4± 6.7 57.2± 12 76.2± 0.48 86.8± 1.8 - 78.8± 5.9 - Corr-C 72.0± 3.3 76.2± 5.2 77.4± 7.0 81.0± 3.7 74.2± 2.5 81.7± 1.2 71.2± 14 63.0± 17 69.5± 22 87.1± 1.6 - 79.0± 6.2 - MCC (V) 70.0± 7.1 71.9± 1.1 74.5± 1.2 78.0± 4.9 69.5± 5.1 73.8± 14 64.5± 16 54.9± 11 74.6± 2.6 85.0± 3.3 - 73.8± 14 - BNM (V) 71.9± 3.4 74.5± 3.5 76.8± 2.8 75.6± 13 72.0± 5.3 82.0± 1.6 72.3± 6.5 57.2± 12 75.3± 1.8 86.6± 1.6 - 78.8± 5.9 - ClassAMI 55.5± 2.9 65.7± 10 78.3± 2.6 77.8± 1.1 58.9± 11 82.8± 3.2 74.4± 8.5 68.5± 6.2 75.5± 5.7 82.1± 7.8 - 74.2± 8.7 - SND 73.3± 6.3 75.4± 4.2 75.5± 6.5 75.8± 6.2 71.8± 5.5 73.1± 5.5 70.4± 9.8 60.8± 10 78.6± 5.0 85.1± 1.4 - 70.5± 9.7 - MixVal 72.7± 12 75.4± 9.4 74.7± 11 79.7± 2.8 71.7± 12 70.2± 12 68.8± 9.8 51.5± 9.8 73.5± 1.4 87.9± 1.7 - 48.5± 6.0 - TransScore 68.9± 8.7 72.2± 4.7 72.9± 2.4 76.4± 3.0 70.6± 9.0 72.1± 13 62.9± 13 57.2± 11 74.9± 2.8 85.1± 2.7 - 70.5± 12 - Table 16: Complete UDA results on RSNA→ CXR (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean± results in Table 15. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM CoUDA ATDOC MCD Avg. Across-Algo TargetOnly Oracle 81.9± 2.6 81.2± 2.2 82.1± 2.4 83.7± 2.0 83.8± 1.9 86.6± 1.7 83.6± 1.5 89.4± 0.68 80.8± 2.5 83.5± 1.5 83.9± 2.7 89.4± 0.68 93.0± 1.2 Source-Risk 73.4± 5.6 72.3± 1.7 74.4± 2.8 71.0± 5.4 72.4± 3.7 84.4± 2.9 75.0± 8.0 84.9± 4.0 52.3± 15 77.5± 6.0 - 72.4± 6.0 - IWCV 74.0± 5.2 74.1± 6.0 73.7± 1.1 75.6± 3.5 77.8± 5.4 75.1± 7.8 73.6± 5.8 85.6± 1.7 55.0± 15 76.7± 4.6 - 75.1± 7.8 - DEV 73.0± 5.9 75.8± 11 74.2± 5.3 73.6± 3.7 73.0± 9.4 79.9± 8.0 76.9± 4.4 77.9± 7.0 64.6± 20 72.1± 12 - 74.4± 16 - Entropy 68.8± 12 51.1± 9.9 56.7± 6.9 63.6± 17 50.0± 3.9 51.2± 14 51.1± 3.6 79.7± 18 55.6± 7.7 72.1± 13 - 51.1± 0.92 - InfoMax 71.4± 4.9 74.7± 4.3 78.5± 2.8 80.8± 4.9 75.2± 5.8 81.7± 1.5 70.5± 8.6 85.9± 2.0 62.6± 15 76.1± 0.51 - 81.2± 7.2 - Corr-C 71.8± 4.5 79.5± 5.2 79.5± 8.3 80.6± 4.7 75.2± 3.0 81.8± 1.5 72.8± 17 86.3± 1.6 64.6± 23 80.0± 26 - 81.7± 7.3 - MCC (V) 71.4± 8.0 71.7± 1.2 74.8± 1.6 78.3± 6.8 69.5± 6.1 80.4± 17 71.7± 20 85.8± 4.5 53.9± 14 75.8± 3.0 - 80.4± 17 - BNM (V) 71.4± 4.6 74.7± 4.3 77.6± 3.5 80.8± 17 75.2± 5.8 82.0± 2.1 70.5± 8.3 85.9± 1.8 62.6± 15 75.8± 2.4 - 81.2± 7.2 - SND 71.5± 7.7 76.1± 5.2 77.8± 8.2 78.3± 6.7 73.2± 6.5 71.9± 7.2 72.2± 13 85.0± 1.7 63.0± 13 79.1± 6.2 - 70.1± 13 - ClassAMI 56.8± 3.4 62.6± 13 77.8± 3.5 77.8± 1.5 53.6± 14 84.2± 3.9 76.9± 9.6 83.9± 10 65.4± 7.6 75.8± 7.6 - 78.2± 9.8 - DEV-N 73.4± 6.4 72.2± 4.2 72.9± 2.5 70.4± 3.5 72.0± 7.2 83.1± 3.1 74.9± 3.1 87.0± 4.0 68.4± 13 73.0± 1.8 - 75.4± 4.0 - MixVal 76.3± 14 78.9± 11 78.0± 14 80.3± 3.1 75.9± 16 72.7± 15 71.0± 12 88.0± 2.0 50.6± 14 73.1± 1.6 - 50.3± 7.8 - TransScore 72.1± 9.5 70.7± 6.0 74.4± 2.5 74.9± 3.5 71.0± 11 76.0± 15 67.5± 15 85.6± 3.5 62.6± 14 75.8± 3.4 - 72.3± 15 - Table 17: Complete UDA results on Child CXR→ (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD CoUDA Avg. Across-Algo TargetOnly Oracle 75.1± 0.36 74.6± 0.99 77.2± 1.1 76.2± 0.32 78.1± 0.45 77.3± 0.66 77.2± 0.47 76.3± 0.89 77.0± 0.39 73.2± 0.44 76.4± 1.5 78.3± 0.26 82.4± 0.46 Source-Risk 72.4± 2.8 69.3± 4.7 68.8± 2.7 71.5± 1.2 74.6± 2.1 73.5± 1.8 71.8± 3.1 61.9± 17 67.5± 2.9 71.5± 0.84 - 70.9± 3.1 - IWCV 73.5± 1.5 71.7± 3.8 70.5± 3.0 70.2± 1.2 76.1± 1.4 70.1± 7.5 69.3± 4.5 59.2± 12 70.9± 3.5 71.2± 1.3 - 59.2± 12 - DEV 71.3± 2.0 66.6± 5.8 68.8± 3.4 64.6± 7.0 75.9± 1.3 71.3± 7.4 68.7± 9.4 53.5± 18 72.9± 4.4 71.4± 0.17 - 71.3± 0.32 - DEV-N 71.7± 2.8 68.4± 4.4 68.8± 2.7 71.5± 1.2 74.9± 2.1 73.5± 1.8 71.8± 3.1 59.4± 19 67.5± 2.9 72.4± 0.56 - 70.6± 3.6 - Entropy 72.4± 1.1 66.9± 1.4 68.3± 1.8 67.2± 4.6 76.2± 1.6 74.2± 1.6 72.8± 3.4 68.0± 7.0 75.5± 0.70 71.2± 0.32 - 74.2± 1.6 - InfoMax 72.5± 1.0 72.1± 2.9 73.2± 1.0 74.5± 0.94 75.7± 1.2 74.2± 2.0 74.1± 0.98 65.7± 16 74.2± 1.7 71.2± 0.32 - 74.2± 2.0 - Corr-C 72.3± 0.96 74.0± 2.0 76.3± 1.1 75.8± 0.36 76.0± 0.51 75.5± 1.1 73.9± 0.92 71.6± 2.9 74.0± 1.9 71.7± 0.49 - 74.6± 1.4 - MCC (V) 72.6± 1.2 67.5± 2.1 68.8± 1.8 69.1± 3.4 74.3± 4.3 74.8± 2.1 74.4± 1.4 72.5± 2.8 75.7± 0.99 71.2± 0.32 - 72.0± 1.3 - BNM (V) 72.5± 1.0 72.1± 2.9 73.2± 1.0 74.5± 0.94 75.7± 1.2 74.2± 2.0 74.0± 0.98 65.7± 16 73.8± 1.1 71.2± 0.32 - 74.2± 2.0 - ClassAMI 71.4± 1.2 69.2± 1.8 69.5± 1.7 70.9± 2.5 75.3± 0.84 73.8± 1.4 72.9± 1.5 70.4± 3.2 70.0± 3.7 71.3± 0.36 - 72.9± 2.3 - SND 73.0± 1.4 73.9± 0.93 74.7± 5.6 76.2± 0.32 74.7± 1.2 73.1± 2.1 71.6± 4.3 63.4± 13 71.2± 2.6 72.4± 0.73 - 74.7± 1.2 - MixVal 72.6± 1.1 68.0± 2.3 70.3± 2.6 71.7± 3.3 76.7± 0.72 73.7± 0.94 73.2± 1.1 61.6± 16 73.3± 1.1 71.2± 0.28 - 73.3± 1.1 - TransScore 70.9± 2.2 69.8± 1.4 70.7± 1.3 72.6± 2.8 76.8± 0.90 73.6± 1.3 72.4± 2.9 56.0± 20 74.9± 0.98 71.3± 0.29 - 74.9± 0.98 - Table 18: Complete UDA results on Child CXR→ (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean± results in Table 17. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM CoUDA ATDOC MCD Avg. Across-Algo TargetOnly Oracle 75.2± 0.46 74.7± 1.1 77.4± 1.2 76.2± 0.40 78.2± 0.58 77.4± 0.87 77.0± 0.62 72.9± 0.47 76.6± 1.0 77.2± 0.46 76.4± 1.6 78.3± 0.37 82.4± 0.64 Source-Risk 73.5± 3.4 72.2± 5.1 70.1± 3.0 70.6± 1.3 73.7± 2.5 73.5± 2.5 73.3± 3.8 71.9± 1.0 71.4± 20 67.4± 3.5 - 72.2± 3.9 - IWCV 73.5± 2.0 73.5± 4.6 71.7± 3.9 70.4± 1.6 76.3± 1.9 72.2± 8.5 71.0± 5.7 71.6± 1.6 63.1± 15 71.4± 4.3 - 63.1± 15 - DEV 70.5± 2.4 68.3± 7.1 67.8± 4.2 68.7± 8.2 76.4± 1.4 75.7± 8.5 73.1± 12 71.5± 0.19 63.1± 21 74.1± 5.5 - 71.5± 0.39 - Entropy 72.5± 1.3 66.2± 1.7 69.2± 2.0 65.6± 5.5 76.8± 1.8 74.1± 2.1 73.2± 4.2 71.2± 0.42 69.2± 8.5 75.5± 0.82 - 74.1± 2.1 - InfoMax 72.6± 1.3 71.1± 3.2 73.2± 1.3 74.8± 1.2 74.9± 1.3 74.0± 2.5 74.2± 1.3 71.2± 0.42 73.2± 18 74.1± 2.3 - 74.0± 2.5 - Corr-C 71.7± 1.1 74.7± 2.4 76.7± 1.3 75.6± 0.36 75.9± 0.61 76.0± 1.3 74.1± 1.1 72.0± 0.57 71.8± 4.0 73.4± 2.4 - 74.9± 1.9 - MCC (V) 73.1± 1.4 66.5± 2.6 69.2± 2.0 69.8± 4.4 76.1± 5.2 74.2± 2.5 75.2± 1.6 71.2± 0.42 73.2± 3.4 76.1± 1.2 - 71.7± 1.7 - BNM (V) 72.6± 1.3 71.1± 3.2 73.2± 1.3 74.8± 1.2 74.9± 1.3 74.0± 2.5 74.1± 1.3 71.2± 0.42 73.2± 18 74.1± 1.5 - 74.0± 2.5 - SND 73.2± 1.9 73.7± 1.2 77.4± 6.7 76.2± 0.40 75.0± 1.5 74.0± 2.5 69.6± 4.6 72.2± 0.95 68.9± 17 72.0± 3.5 - 75.0± 1.5 - ClassAMI 71.3± 1.4 69.2± 2.4 70.3± 2.1 71.5± 3.1 75.2± 1.1 73.5± 1.8 72.1± 1.8 71.2± 0.43 71.8± 3.5 70.6± 4.4 - 73.0± 2.4 - DEV-N 70.5± 3.4 67.9± 5.1 70.1± 3.0 70.6± 1.3 75.9± 2.2 73.5± 2.5 73.3± 3.8 72.1± 0.65 71.4± 21 67.4± 3.5 - 72.4± 4.0 - MixVal 72.9± 1.3 68.1± 3.3 71.6± 3.1 72.3± 4.1 76.7± 0.88 74.0± 1.1 72.7± 1.3 71.2± 0.35 68.0± 20 73.4± 1.3 - 73.4± 1.3 - Table 19: Complete UDA results on LDD→ (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD CoUDA Avg. Across-Algo TargetOnly Oracle 79.0± 0.82 82.1± 0.52 82.4± 0.54 82.6± 0.42 83.0± 0.52 82.4± 0.90 83.2± 0.50 83.0± 0.50 83.0± 0.63 81.7± 0.80 82.6± 0.47 83.6± 0.13 95.7± 0.34 Source-Risk 67.6± 1.8 79.7± 0.96 78.7± 2.2 69.2± 5.3 79.9± 2.4 74.1± 2.5 81.0± 1.8 78.8± 3.9 79.5± 0.96 75.3± 1.4 - 74.0± 6.8 - IWCV 69.5± 4.2 79.5± 0.43 77.7± 3.4 71.8± 3.6 79.2± 2.1 74.7± 2.2 79.7± 3.1 75.4± 6.5 73.8± 5.4 76.1± 1.1 - 74.7± 2.2 - DEV 74.5± 5.1 73.5± 9.4 79.8± 0.45 78.4± 1.5 79.8± 3.8 72.9± 8.8 76.2± 4.7 57.6± 12 79.8± 3.2 79.3± 1.6 - 74.8± 10 - DEV-N 68.2± 2.0 79.9± 0.72 78.2± 1.3 69.8± 5.2 80.1± 2.5 74.1± 2.6 81.4± 1.1 78.8± 3.9 79.2± 1.3 75.5± 1.3 - 73.9± 6.8 - Entropy 62.4± 7.0 73.8± 13 68.6± 17 68.7± 11 75.2± 14 70.5± 12 62.7± 17 73.5± 14 80.8± 1.8 75.2± 0.08 - 54.3± 9.7 - InfoMax 74.0± 4.3 81.0± 1.3 81.5± 0.67 81.4± 0.95 82.2± 0.88 81.5± 1.4 80.6± 2.6 80.6± 1.5 81.9± 0.90 78.8± 0.85 - 81.9± 0.59 - Corr-C 76.7± 2.9 79.6± 1.4 81.5± 0.76 79.5± 2.4 82.3± 1.0 79.2± 2.6 80.8± 1.4 78.8± 6.2 81.9± 1.4 79.2± 2.1 - 81.1± 1.6 - MCC (V) 67.4± 0.93 79.8± 0.76 79.9± 1.6 74.4± 1.3 81.3± 1.6 75.7± 5.2 81.7± 0.49 79.9± 3.2 81.1± 1.9 75.2± 0.08 - 75.7± 5.2 - BNM (V) 72.8± 3.5 80.7± 0.93 81.5± 0.67 81.4± 0.95 82.2± 0.88 81.5± 1.4 80.6± 2.6 80.6± 1.4 81.8± 1.1 78.3± 0.93 - 81.9± 0.58 - ClassAMI 69.5± 7.8 76.8± 2.7 80.8± 1.2 72.5± 6.2 79.3± 2.6 78.5± 3.9 79.2± 1.9 80.7± 1.2 76.6± 4.2 77.2± 1.6 - 78.6± 3.2 - SND 68.5± 4.8 72.6± 4.4 74.6± 4.1 71.4± 2.3 76.9± 1.3 68.5± 4.6 72.2± 3.3 72.8± 4.5 72.3± 3.4 77.9± 3.0 - 75.8± 2.6 - MixVal 69.3± 8.8 80.7± 0.95 80.9± 0.85 74.6± 5.2 80.7± 1.6 75.9± 6.5 78.5± 1.6 71.2± 22 79.0± 4.2 76.6± 0.63 - 79.0± 4.2 - TransScore 69.9± 4.1 80.1± 1.3 81.7± 0.38 79.6± 1.0 81.8± 1.4 81.5± 1.4 81.9± 0.88 64.4± 21 81.7± 1.1 77.0± 0.75 - 81.7± 1.1 - Table 20: Complete UDA results on LDD→ (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean± results in Table 19. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM CoUDA ATDOC MCD Avg. Across-Algo TargetOnly Oracle 79.1± 1.1 82.2± 0.62 82.3± 0.66 82.5± 0.58 83.3± 0.55 82.5± 1.2 83.4± 0.64 82.1± 0.87 82.8± 0.56 82.8± 0.73 82.7± 0.46 83.6± 0.15 96.0± 0.36 Source-Risk 68.8± 2.1 80.1± 1.1 77.8± 2.7 72.2± 5.8 80.8± 3.1 74.7± 2.9 81.9± 2.2 75.0± 1.9 80.6± 4.7 79.5± 1.2 - 77.4± 7.8 - IWCV 70.6± 4.7 79.7± 0.56 77.8± 4.5 72.5± 4.8 80.0± 2.6 76.0± 2.7 81.1± 3.8 75.4± 1.2 77.7± 8.2 74.7± 7.3 - 76.0± 2.7 - DEV 73.9± 5.9 78.5± 11 79.8± 0.61 79.0± 1.9 81.3± 4.5 75.9± 11 77.6± 6.3 79.8± 2.0 63.3± 12 80.8± 3.8 - 80.0± 12 - Entropy 64.8± 8.8 79.6± 15 80.4± 16 72.3± 13 80.2± 16 74.8± 15 50.0± 16 75.2± 0.11 80.4± 16 81.2± 2.2 - 50.0± 11 - InfoMax 73.8± 5.6 81.2± 1.5 81.4± 0.88 81.5± 1.2 82.2± 1.0 81.4± 1.9 81.7± 3.2 78.5± 1.0 80.8± 1.9 81.7± 1.1 - 81.8± 0.71 - Corr-C 76.9± 3.7 79.6± 1.9 81.5± 1.1 80.5± 2.7 81.8± 1.1 80.0± 2.9 81.0± 1.7 79.5± 2.6 81.3± 7.4 82.4± 1.7 - 81.7± 1.7 - MCC (V) 67.8± 1.1 79.6± 0.98 80.4± 2.0 74.7± 1.7 80.7± 1.7 77.8± 6.4 81.6± 0.63 75.2± 0.11 81.0± 3.9 82.0± 2.2 - 77.8± 6.4 - BNM (V) 73.8± 4.8 81.1± 1.1 81.4± 0.88 81.5± 1.2 82.2± 1.0 81.4± 1.9 81.7± 3.2 77.9± 1.1 80.8± 1.9 81.7± 1.5 - 81.8± 0.71 - SND 68.5± 6.1 74.7± 4.6 76.4± 5.0 70.9± 3.1 76.6± 1.8 69.5± 5.7 71.5± 4.0 78.1± 3.9 72.8± 5.5 70.8± 4.2 - 76.3± 3.6 - ClassAMI 73.3± 9.5 76.5± 3.2 80.5± 1.4 74.0± 7.9 80.2± 3.4 80.5± 4.6 78.9± 2.3 77.3± 2.1 80.3± 1.4 76.7± 5.7 - 78.3± 4.3 - DEV-N 68.8± 2.5 80.1± 0.91 77.8± 1.5 72.2± 6.4 80.8± 3.1 74.7± 2.9 81.9± 1.3 75.9± 1.6 80.9± 4.7 79.5± 1.5 - 76.5± 7.8 - MixVal 68.2± 11 80.7± 1.3 81.4± 0.91 74.3± 6.5 80.9± 2.0 77.8± 7.5 79.1± 2.1 77.0± 0.73 81.0± 25 80.6± 5.1 - 80.6± 5.1 - TransScore 69.0± 5.2 79.6± 1.5 81.6± 0.50 79.8± 1.3 82.2± 1.7 81.4± 1.9 81.8± 1.0 77.1± 0.93 77.0± 25 82.3± 1.3 - 81.8± 1.4 - Table 21: Complete UDA results on CRD→ (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD CoUDA Avg. Across-Algo TargetOnly Oracle 77.2± 3.0 78.6± 1.6 77.9± 5.8 81.0± 1.5 77.7± 1.9 78.5± 3.6 80.0± 3.6 74.8± 3.4 76.3± 5.0 63.9± 1.4 76.5± 4.8 82.7± 1.1 98.7± 0.17 Source-Risk 68.2± 1.6 44.9± 1.3 64.7± 11 66.2± 14 58.4± 12 57.1± 9.7 60.0± 8.2 49.8± 7.5 56.0± 8.4 56.6± 0.80 - 54.8± 3.8 - IWCV 67.7± 7.3 46.1± 3.6 61.7± 15 70.0± 10 56.5± 6.4 59.2± 11 58.3± 11 62.0± 17 57.8± 8.0 56.3± 3.1 - 63.7± 6.1 - DEV 68.2± 7.2 58.1± 9.9 67.1± 11 66.6± 11 62.0± 11 60.5± 11 58.0± 11 48.3± 15 60.1± 5.7 54.1± 4.2 - 49.6± 16 - DEV-N 68.2± 1.6 51.5± 13 67.7± 13 63.5± 15 56.4± 12 57.1± 9.7 59.3± 9.2 74.1± 4.3 56.0± 8.4 63.2± 2.4 - 64.6± 14 - Entropy 65.4± 8.9 44.8± 1.6 59.2± 14 73.9± 13 58.8± 11 68.1± 7.9 52.2± 9.6 47.6± 3.0 61.5± 9.9 56.2± 1.3 - 58.4± 7.7 - InfoMax 67.1± 2.5 50.5± 11 73.1± 9.8 69.9± 10 66.4± 8.3 55.0± 11 54.4± 11 51.5± 10 61.6± 9.6 56.2± 1.3 - 55.0± 11 - Corr-C 64.1± 1.6 63.1± 11 73.1± 9.2 63.5± 10 67.0± 8.1 53.3± 2.6 55.4± 2.3 51.2± 14 55.8± 7.9 55.3± 1.3 - 68.1± 12 - MCC (V) 68.0± 2.1 44.8± 1.6 59.8± 15 70.0± 12 62.6± 11 63.0± 14 55.1± 12 48.6± 4.6 49.4± 3.7 56.2± 1.3 - 59.3± 15 - BNM (V) 67.4± 1.9 50.4± 11 73.1± 9.8 69.3± 11 66.4± 8.3 55.0± 11 54.4± 11 45.2± 16 61.6± 9.6 56.2± 1.3 - 55.0± 11 - ClassAMI 68.2± 3.9 50.7± 11 66.2± 16 66.2± 8.4 62.4± 11 52.7± 9.3 58.9± 6.6 47.1± 14 64.2± 1.8 55.1± 2.3 - 72.5± 9.3 - SND 60.6± 2.9 61.5± 11 71.2± 4.6 67.8± 8.8 53.3± 8.4 51.0± 2.0 57.9± 9.2 62.8± 8.2 64.9± 8.3 52.8± 2.3 - 51.8± 7.2 - MixVal 67.7± 1.8 47.7± 9.4 71.7± 11 64.6± 11 61.8± 13 63.0± 13 54.2± 15 37.7± 6.5 67.2± 2.4 56.0± 2.2 - 67.2± 2.4 - TransScore 67.9± 1.8 60.9± 15 67.8± 12 68.2± 14 52.1± 5.5 61.2± 17 57.1± 8.3 51.8± 10 54.5± 6.2 55.6± 1.7 - 56.7± 5.6 - Table 22: Complete UDA results on CRD→ (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean± results in Table 21. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM CoUDA ATDOC MCD Avg. Across-Algo TargetOnly Oracle 76.7± 4.1 77.7± 1.9 80.1± 7.0 81.4± 1.9 78.2± 2.3 76.5± 3.8 80.0± 4.6 64.0± 1.6 75.5± 4.0 76.1± 6.6 76.6± 5.1 82.7± 1.4 98.8± 0.20 Source-Risk 68.1± 2.0 45.0± 1.7 64.6± 12 67.0± 17 59.0± 12 60.4± 13 62.9± 8.9 56.7± 1.0 44.9± 8.2 58.5± 9.3 - 56.3± 4.7 - IWCV 65.4± 9.5 45.7± 4.5 52.9± 15 70.9± 13 59.5± 7.9 62.4± 14 64.7± 12 55.7± 4.0 70.8± 18 58.0± 11 - 64.7± 8.3 - DEV 68.2± 9.5 51.8± 10 71.5± 12 68.3± 13 58.6± 12 62.4± 14 64.7± 12 54.6± 5.9 51.8± 20 62.9± 6.6 - 51.8± 20 - Entropy 70.0± 11 44.6± 2.1 50.2± 14 80.0± 15 57.6± 15 65.8± 9.5 50.1± 13 56.4± 1.6 47.6± 3.1 66.6± 10 - 62.8± 7.9 - InfoMax 66.9± 3.2 45.4± 13 77.0± 11 75.7± 11 68.5± 11 53.0± 14 50.4± 14 56.4± 1.6 55.4± 12 64.5± 11 - 53.0± 14 - Corr-C 64.9± 1.9 69.8± 11 76.5± 11 58.4± 12 68.5± 11 54.7± 2.8 55.0± 3.1 55.2± 1.8 51.0± 18 53.7± 10 - 76.3± 12 - MCC (V) 68.1± 2.3 44.6± 2.1 50.2± 15 73.8± 13 62.4± 14 62.8± 19 51.9± 13 56.4± 1.6 47.6± 5.5 47.6± 4.4 - 62.6± 19 - BNM (V) 66.9± 2.4 45.4± 13 77.0± 11 75.7± 11 68.5± 11 53.0± 14 50.4± 14 56.4± 1.6 55.4± 18 64.5± 11 - 53.0± 14 - SND 60.0± 3.4 59.8± 14 72.2± 5.6 69.2± 11 52.6± 10 51.0± 2.8 53.0± 11 53.1± 3.0 65.3± 10 64.7± 11 - 52.6± 10 - ClassAMI 68.6± 4.4 46.0± 14 75.6± 17 63.6± 11 61.5± 13 58.8± 9.5 56.1± 7.5 56.2± 2.7 45.5± 19 64.2± 2.1 - 77.3± 11 - DEV-N 68.1± 2.0 47.0± 15 75.1± 13 67.0± 17 49.0± 12 60.4± 13 62.9± 11 63.7± 2.9 75.5± 5.2 58.5± 9.3 - 69.8± 17 - MixVal 67.4± 2.1 43.4± 11 75.9± 13 61.8± 13 68.3± 13 65.4± 18 50.1± 20 56.3± 2.9 36.8± 8.0 66.3± 2.6 - 66.3± 2.6 - Table 23: Complete UDA results on OCT→ (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD Avg. Across-Algo TargetOnly Oracle 61.9± 1.3 66.7± 0.52 67.0± 0.39 67.2± 0.53 66.1± 0.79 64.2± 1.4 65.0± 0.66 60.4± 2.5 64.2± 2.4 65.1± 2.1 67.4± 0.15 75.0± 0.58 Source-Risk 59.5± 1.1 63.3± 1.6 66.3± 0.59 64.0± 0.65 63.8± 1.2 57.6± 3.9 55.8± 5.0 38.1± 4.1 54.2± 3.5 - 57.6± 13 - IWCV 58.4± 2.3 63.9± 1.4 65.9± 1.5 64.0± 1.1 64.6± 0.67 49.7± 5.6 49.1± 6.5 47.6± 3.2 55.2± 2.9 - 63.9± 1.4 - DEV 55.3± 3.4 63.8± 1.5 60.8± 4.7 62.4± 3.2 55.4± 5.0 61.2± 2.7 55.9± 5.8 51.1± 1.6 51.4± 3.8 - 51.6± 3.7 - DEV-N 61.9± 1.3 66.6± 0.64 66.4± 0.07 66.8± 1.0 66.1± 0.79 63.8± 1.5 64.7± 0.79 60.4± 2.5 63.1± 3.3 - 67.3± 0.22 - Entropy 58.0± 3.3 63.2± 1.5 66.1± 0.53 63.6± 1.6 64.0± 0.60 50.4± 3.9 50.1± 0.71 45.4± 4.6 48.8± 4.8 - 50.2± 3.9 - InfoMax 59.2± 1.2 63.7± 1.9 65.8± 0.23 66.4± 0.58 64.0± 0.60 54.8± 7.9 56.4± 6.4 44.6± 3.9 49.5± 4.2 - 49.7± 4.0 - Corr-C 58.9± 1.9 63.8± 2.0 65.8± 1.1 65.9± 0.74 64.5± 0.21 59.7± 3.0 54.5± 5.1 41.5± 4.7 53.5± 4.3 - 57.3± 7.0 - MCC (V) 59.1± 1.6 63.3± 1.4 66.1± 0.53 65.0± 0.97 64.0± 0.60 54.0± 7.4 54.8± 8.8 45.4± 4.6 48.8± 4.8 - 54.0± 7.4 - BNM (V) 59.2± 1.2 63.4± 1.6 66.2± 0.43 66.4± 0.58 64.0± 0.60 52.2± 5.3 56.4± 6.4 44.6± 3.9 49.5± 4.2 - 49.7± 4.0 - ClassAMI 55.5± 6.7 64.8± 1.1 65.3± 0.98 63.7± 1.0 64.8± 0.38 54.5± 7.1 60.4± 2.8 41.6± 8.0 51.1± 7.9 - 50.5± 6.8 - SND 58.9± 1.8 62.7± 1.0 62.5± 0.94 61.6± 2.9 59.4± 0.87 60.8± 3.8 57.0± 10 55.3± 8.5 60.3± 8.0 - 59.4± 0.87 - MixVal 56.4± 6.5 62.8± 1.3 64.8± 0.91 65.6± 0.70 64.0± 0.25 58.7± 4.5 51.3± 2.9 44.8± 6.6 51.5± 3.7 - 51.5± 3.7 - TransScore 59.0± 1.8 64.0± 1.4 65.1± 0.75 65.2± 0.94 64.4± 0.40 56.8± 4.3 48.9± 2.7 45.9± 2.8 53.8± 3.6 - 53.8± 3.6 - Table 24: Complete UDA results on OCT→ (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean± results in Table 23. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD Avg. Across-Algo TargetOnly Oracle 61.5± 1.3 66.5± 0.49 66.9± 0.39 67.4± 0.50 66.5± 0.73 64.1± 1.4 65.3± 0.63 61.3± 2.4 63.8± 2.4 65.2± 2.0 67.4± 0.15 75.7± 0.57 Source-Risk 59.1± 1.0 62.9± 1.6 66.5± 0.54 63.6± 0.56 63.7± 1.2 55.5± 3.4 58.4± 4.5 37.2± 4.0 55.4± 3.4 - 63.6± 12 - IWCV 58.1± 2.3 64.4± 1.3 66.4± 1.4 64.4± 1.1 64.6± 0.67 50.4± 5.5 50.6± 6.4 47.7± 3.2 54.6± 2.8 - 64.4± 1.3 - DEV 55.4± 3.4 63.0± 1.3 60.7± 4.7 64.2± 2.8 53.7± 4.8 61.1± 2.7 57.2± 5.7 50.3± 1.4 49.5± 3.4 - 50.1± 3.4 - Entropy 59.7± 3.0 63.9± 1.4 65.9± 0.49 64.1± 1.5 64.0± 0.60 50.6± 3.9 50.0± 0.70 42.9± 4.1 46.7± 4.4 - 50.0± 3.9 - InfoMax 59.4± 1.2 64.3± 1.8 65.9± 0.23 66.1± 0.52 64.0± 0.60 56.2± 7.8 58.6± 6.1 43.5± 3.8 48.8± 4.2 - 48.8± 3.9 - Corr-C 59.8± 1.8 64.8± 1.8 66.2± 1.0 66.0± 0.73 64.4± 0.20 61.3± 2.6 54.8± 5.1 41.5± 4.7 54.4± 4.2 - 61.3± 6.1 - MCC (V) 59.7± 1.5 63.9± 1.2 65.9± 0.49 65.0± 0.97 64.0± 0.60 54.2± 7.4 50.0± 7.7 42.9± 4.1 46.7± 4.4 - 54.2± 7.4 - BNM (V) 59.4± 1.2 64.2± 1.5 66.1± 0.41 66.1± 0.52 64.0± 0.60 54.1± 5.0 58.6± 6.1 43.5± 3.8 48.8± 4.2 - 48.8± 3.9 - SND 59.8± 1.6 63.0± 0.99 62.7± 0.91 63.0± 2.6 59.1± 0.82 62.4± 3.6 62.9± 9.2 59.0± 7.8 62.5± 7.7 - 59.1± 0.82 - ClassAMI 58.7± 6.1 65.0± 1.1 65.1± 0.97 63.4± 0.97 64.7± 0.37 58.2± 6.3 58.8± 2.5 37.2± 7.1 47.9± 7.4 - 47.9± 6.4 - DEV-N 61.5± 1.3 66.5± 0.64 66.5± 0.07 67.1± 0.95 66.5± 0.73 63.4± 1.4 64.3± 0.73 61.3± 2.4 62.1± 3.2 - 67.3± 0.21 - MixVal 60.0± 5.7 63.1± 1.3 64.9± 0.91 65.7± 0.69 64.0± 0.25 60.5± 4.2 50.4± 2.8 47.9± 6.0 51.4± 3.7 - 51.4± 3.7 - TransScore 59.6± 1.7 64.6± 1.3 64.8± 0.71 65.5± 0.90 64.4± 0.40 55.9± 4.2 50.0± 2.5 45.1± 2.7 53.7± 3.6 - 53.7± 3.6 - Table 25: Complete UDA results on SLO→ (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD Avg. Across-Algo TargetOnly Oracle 60.7± 1.2 63.0± 1.0 64.6± 1.5 65.4± 0.34 63.6± 1.1 62.3± 1.4 61.3± 3.3 58.4± 5.2 62.0± 1.6 62.6± 2.0 65.6± 0.62 74.4± 0.38 Source-Risk 57.9± 1.4 60.7± 0.83 61.1± 1.6 63.3± 1.4 59.6± 1.3 53.5± 4.6 49.7± 4.2 45.1± 0.56 54.6± 6.7 - 55.2± 6.7 - IWCV 56.6± 0.78 56.2± 0.90 60.5± 2.5 60.8± 0.25 58.7± 0.39 54.1± 3.6 48.6± 1.2 51.2± 3.5 55.7± 4.7 - 52.0± 5.4 - DEV 58.4± 1.7 56.8± 3.4 61.4± 2.3 60.3± 1.8 56.1± 1.9 55.9± 3.4 51.8± 4.9 46.2± 1.5 57.5± 3.0 - 54.2± 7.8 - DEV-N 57.0± 3.0 61.6± 2.3 62.4± 0.69 62.2± 1.6 60.3± 1.7 52.6± 6.2 49.4± 4.5 43.7± 2.5 49.7± 3.8 - 60.1± 4.9 - Entropy 57.1± 1.3 49.6± 4.8 60.5± 1.7 61.9± 1.4 61.4± 1.1 45.6± 3.2 50.3± 0.90 44.9± 4.0 51.1± 6.0 - 45.9± 3.6 - InfoMax 56.9± 1.6 49.7± 4.8 60.1± 2.3 61.4± 2.6 61.6± 1.1 45.3± 4.9 48.3± 1.6 45.2± 3.9 52.1± 5.8 - 45.3± 4.9 - Corr-C 57.9± 1.9 50.1± 4.4 61.4± 2.5 62.6± 2.1 61.7± 1.4 55.0± 5.4 49.3± 1.9 47.0± 4.6 51.8± 5.3 - 49.3± 1.9 - MCC (V) 56.9± 1.5 49.6± 4.8 60.5± 1.7 61.0± 1.2 61.4± 1.1 45.5± 3.1 49.7± 1.6 44.9± 4.0 50.8± 5.6 - 45.5± 3.1 - BNM (V) 56.9± 1.5 49.7± 4.8 60.5± 1.7 61.4± 2.6 61.6± 1.1 45.3± 4.9 48.5± 1.9 45.2± 3.9 52.2± 5.9 - 45.3± 4.9 - ClassAMI 57.8± 2.1 59.8± 1.5 59.5± 2.5 60.4± 1.1 61.8± 1.6 52.6± 7.5 47.0± 1.4 43.6± 3.8 52.7± 6.0 - 46.0± 5.9 - SND 59.0± 2.4 58.5± 3.2 60.6± 2.3 60.1± 2.3 57.0± 1.4 57.6± 4.4 54.2± 6.7 54.4± 5.9 58.7± 2.1 - 57.0± 1.4 - MixVal 58.1± 1.9 49.8± 4.4 59.7± 2.1 62.3± 2.2 60.0± 1.4 46.2± 4.6 50.7± 2.7 44.8± 3.5 53.6± 5.6 - 52.2± 6.5 - TransScore 57.8± 1.8 60.9± 2.2 60.2± 1.8 62.3± 0.26 59.7± 0.90 47.2± 3.5 50.2± 3.8 48.0± 6.2 57.3± 3.2 - 57.3± 3.2 - Table 26: Complete UDA results on SLO→ (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean± results in Table 25. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. SourceOnly MMD DANN CDAN DALN MCC BNM ATDOC MCD Avg. Across-Algo TargetOnly Oracle 61.2± 1.1 62.4± 0.89 64.0± 1.4 65.5± 0.33 63.3± 1.1 63.0± 1.2 62.8± 3.0 60.2± 4.9 61.2± 1.4 62.8± 1.6 65.5± 0.62 75.8± 0.30 Source-Risk 58.1± 1.4 61.1± 0.76 61.6± 1.5 62.7± 1.3 59.3± 1.3 53.1± 4.6 48.8± 4.1 45.2± 0.54 53.2± 6.6 - 54.2± 6.7 - IWCV 56.2± 0.73 56.6± 0.81 60.6± 2.5 60.9± 0.23 58.8± 0.37 53.1± 3.5 48.2± 1.1 50.3± 3.4 57.4± 4.5 - 50.3± 5.2 - DEV 58.3± 1.7 56.9± 3.4 60.1± 2.0 60.7± 1.8 56.2± 1.9 56.8± 3.3 49.9± 4.6 45.3± 1.3 59.2± 2.6 - 58.2± 6.9 - Entropy 56.4± 1.2 48.4± 4.7 60.6± 1.7 62.4± 1.4 60.9± 1.1 44.2± 3.0 50.0± 0.85 46.6± 3.7 49.8± 5.9 - 44.2± 3.3 - InfoMax 56.2± 1.5 48.5± 4.7 60.6± 2.3 60.4± 2.4 61.5± 1.1 44.5± 4.8 47.5± 1.4 47.1± 3.6 54.5± 5.4 - 44.5± 4.8 - Corr-C 58.6± 1.8 48.5± 4.2 60.5± 2.4 62.4± 2.1 60.9± 1.3 53.5± 5.2 48.5± 1.8 44.5± 4.0 53.2± 5.2 - 48.5± 1.8 - MCC (V) 56.2± 1.4 48.4± 4.7 60.6± 1.7 60.3± 1.0 60.9± 1.1 44.2± 2.9 49.6± 1.6 46.6± 3.7 49.8± 5.6 - 44.2± 2.9 - BNM (V) 56.2± 1.4 48.5± 4.7 60.6± 1.7 60.4± 2.4 61.5± 1.1 44.5± 4.8 47.5± 1.7 47.1± 3.6 54.5± 5.5 - 44.5± 4.8 - SND 59.3± 2.4 58.9± 3.2 60.0± 2.3 60.5± 2.3 56.8± 1.4 58.5± 4.3 56.5± 6.5 56.0± 5.7 59.8± 1.9 - 56.8± 1.4 - ClassAMI 58.7± 2.0 60.4± 1.4 58.3± 2.2 59.8± 0.94 61.0± 1.4 52.2± 7.5 46.9± 1.3 42.5± 3.6 50.4± 5.6 - 45.2± 5.9 - DEV-N 58.1± 2.8 61.1± 2.2 62.4± 0.68 62.9± 1.5 61.1± 1.5 53.7± 6.2 47.7± 4.3 44.5± 2.5 50.1± 3.7 - 62.4± 4.4 - MixVal 58.6± 1.8 47.9± 4.1 59.7± 2.1 61.7± 2.2 60.7± 1.3 48.2± 4.3 50.9± 2.7 45.3± 3.5 52.4± 5.5 - 48.8± 5.8 - TransScore 58.6± 1.6 59.6± 2.0 60.6± 1.8 62.4± 0.25 59.2± 0.81 48.3± 3.3 48.4± 3.5 46.9± 6.2 56.6± 3.2 - 56.6± 3.2 - 6.2 Full Validator Reliability Analyses for Within-Algorithm Selection This section provides the full per-scenario validator reliability analyses. Each table reports, for one cross-domain scenario, the within-algorithm Spearman correlation (ρ) between each validation score and target accuracy, computed over the checkpoints of each algorithm. Positive values always indicate ranking in the intended direction. The ρ values here are means across folds or random seeds, whereas the example scatter plots of the main paper show a single fold or random seed for readability, so individual cells may not match exactly. Tables 27–37 cover the eleven clinically relevant transfer scenarios. Brain MRI: ADNI-1→ -2 (Table 27), ADNI-1→ -3 (Table 28), ADNI-2→ -1 (Table 29), ADNI-2→ -3 (Table 30), ADNI-1+2→ (Table 31). Chest X-ray: RSNA→ CXR (Table 32), Child CXR→ (Table 33), LDD→ (Table 34), CRD→ (Table 35). Retinal: OCT→ (Table 36), SLO→ (Table 37). Table 27: Within-algorithm Spearman’s ρ on ADNI-1→ -2. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction. MMD DANN CDAN DALN MCC BNM ATDOC MCD AD2A Source-Risk 0.80 0.76 0.74 0.74 0.74 0.70 0.53 0.79 0.80 IWCV 0.49 0.29 0.31 -0.28 0.29 0.04 0.05 0.48 0.38 DEV 0.52 0.47 0.45 0.02 0.23 0.20 0.18 0.53 0.53 DEV-N 0.77 0.74 0.72 0.71 0.73 0.69 0.53 0.78 0.79 Entropy 0.45 0.61 0.60 0.57 0.55 0.51 0.33 0.72 0.67 InfoMax 0.78 0.87 0.87 0.87 0.83 0.79 0.61 0.81 0.89 Corr-C 0.82 0.79 0.81 0.83 0.78 0.74 0.62 0.82 0.87 MCC (V) 0.70 0.83 0.82 0.81 0.76 0.73 0.55 0.78 0.86 BNM (V) 0.75 0.86 0.86 0.85 0.82 0.79 0.60 0.80 0.88 SND 0.75 0.58 0.57 0.65 0.50 0.45 0.38 0.78 0.66 ClassAMI 0.29 0.70 0.70 0.73 0.69 0.67 0.53 0.47 0.71 MixVal 0.22 0.33 0.37 0.33 0.35 0.35 0.25 0.31 0.29 TransScore 0.30 0.43 0.52 0.54 0.75 0.70 0.47 0.53 0.50 Table 28: Within-algorithm Spearman’s ρ on ADNI-1→ -3. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction. MMD DANN CDAN DALN MCC BNM ATDOC MCD AD2A Source-Risk 0.52 0.52 0.61 0.40 0.47 0.40 0.48 0.72 0.75 IWCV 0.16 0.14 0.22 -0.12 0.15 0.11 0.13 0.50 0.36 DEV 0.24 0.28 0.33 0.09 0.19 0.21 0.07 0.47 0.53 DEV-N 0.51 0.52 0.61 0.39 0.48 0.40 0.28 0.70 0.75 Entropy 0.35 0.53 0.44 0.22 0.20 0.24 0.05 0.55 0.56 InfoMax 0.46 0.61 0.58 0.39 0.39 0.18 0.43 0.66 0.73 Corr-C 0.40 0.35 0.37 0.21 0.29 0.03 0.20 0.64 0.64 MCC (V) 0.45 0.66 0.64 0.41 0.40 0.30 0.42 0.64 0.70 BNM (V) 0.46 0.63 0.61 0.40 0.39 0.20 0.42 0.64 0.72 SND 0.34 0.14 0.24 0.18 0.24 -0.01 0.24 0.62 0.52 ClassAMI 0.02 0.56 0.58 0.37 0.46 0.41 0.44 0.41 0.61 MixVal 0.13 0.20 0.27 0.13 0.20 0.19 0.26 0.27 0.29 TransScore 0.22 0.39 0.43 0.18 0.35 0.15 0.14 0.33 0.21 Table 29: Within-algorithm Spearman’s ρ on ADNI-2→ -1. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction. MMD DANN CDAN DALN MCC BNM ATDOC MCD AD2A Source-Risk 0.73 0.58 0.53 0.59 0.58 0.56 0.68 0.76 0.79 IWCV 0.43 -0.08 -0.24 -0.37 -0.28 -0.12 0.03 0.45 0.05 DEV 0.17 -0.02 -0.03 -0.03 -0.01 -0.07 0.07 0.51 -0.02 DEV-N 0.71 0.57 0.54 0.57 0.56 0.55 0.68 0.76 0.77 Entropy 0.43 0.39 0.50 0.49 0.46 0.44 0.38 0.67 0.57 InfoMax 0.80 0.72 0.74 0.77 0.76 0.77 0.04 0.54 0.84 Corr-C 0.75 0.72 0.69 0.67 0.68 0.73 0.72 0.82 0.84 MCC (V) 0.73 0.59 0.64 0.69 0.63 0.65 0.76 0.77 0.77 BNM (V) 0.78 0.70 0.73 0.76 0.74 0.76 0.40 0.66 0.82 SND 0.53 0.35 0.22 0.32 0.25 0.37 0.38 0.71 0.62 ClassAMI 0.16 0.53 0.55 0.56 0.60 0.67 0.62 0.58 0.72 MixVal 0.24 0.25 0.25 0.36 0.29 0.26 0.55 0.30 0.35 TransScore 0.52 0.34 0.33 0.55 0.74 0.64 0.57 0.56 0.42 Table 30: Within-algorithm Spearman’s ρ on ADNI-2→ -3. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction. MMD DANN CDAN DALN MCC BNM ATDOC MCD AD2A Source-Risk 0.72 0.68 0.66 0.69 0.67 0.57 0.57 0.74 0.82 IWCV 0.38 0.30 0.23 -0.23 0.20 0.01 0.24 0.54 0.60 DEV 0.49 0.46 0.41 0.21 0.39 0.33 0.25 0.58 0.68 DEV-N 0.64 0.61 0.57 0.58 0.58 0.49 0.52 0.71 0.78 Entropy 0.30 0.20 0.15 0.17 0.19 0.19 0.19 0.63 0.49 InfoMax 0.76 0.71 0.68 0.68 0.64 0.51 0.36 0.63 0.80 Corr-C 0.63 0.51 0.55 0.56 0.48 0.33 0.19 0.61 0.58 MCC (V) 0.70 0.64 0.60 0.62 0.60 0.57 0.41 0.68 0.73 BNM (V) 0.75 0.72 0.68 0.69 0.65 0.54 -0.04 0.55 0.78 SND 0.53 0.39 0.45 0.49 0.35 0.24 0.17 0.52 0.44 ClassAMI -0.01 0.66 0.57 0.56 0.65 0.60 0.55 0.23 0.55 MixVal 0.37 0.29 0.27 0.32 0.30 0.23 0.24 0.28 0.31 TransScore 0.42 0.36 0.36 0.33 0.59 0.46 0.24 0.53 0.41 Table 31: Within-algorithm Spearman’s ρ on ADNI-1+2→ . Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction. MMD DANN CDAN DALN MCC BNM ATDOC MCD AD2A Source-Risk 0.71 0.56 0.65 0.56 0.62 0.54 0.58 0.66 0.77 IWCV 0.52 0.15 0.18 -0.28 0.25 0.08 0.22 0.41 0.53 DEV 0.32 0.12 0.05 0.03 0.29 0.20 0.31 0.29 0.30 DEV-N 0.71 0.56 0.62 0.56 0.62 0.56 0.57 0.67 0.76 Entropy 0.55 0.46 0.46 0.41 0.50 0.41 0.43 0.72 0.69 InfoMax 0.65 0.40 0.58 0.46 0.45 0.31 0.54 0.75 0.71 Corr-C 0.42 -0.04 0.24 0.06 0.04 -0.12 -0.09 0.41 0.35 MCC (V) 0.65 0.54 0.62 0.54 0.58 0.44 0.48 0.72 0.76 BNM (V) 0.66 0.47 0.61 0.50 0.50 0.35 -0.13 0.51 0.73 SND 0.34 -0.14 0.09 0.01 -0.11 -0.20 -0.18 0.24 0.16 ClassAMI -0.25 0.37 0.38 0.33 0.46 0.50 0.39 0.29 0.60 MixVal 0.31 0.27 0.21 0.13 0.16 0.13 0.18 0.24 0.27 TransScore 0.51 0.36 0.54 0.36 0.49 0.32 0.33 0.63 0.58 Table 32: Within-algorithm Spearman’s ρ on RSNA→ CXR. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction. MMD DANN CDAN DALN MCC BNM ATDOC MCD CoUDA Source-Risk 0.11 0.20 -0.08 0.21 0.49 0.22 -0.10 0.47 0.57 IWCV -0.05 0.14 -0.06 -0.02 0.41 0.17 0.03 0.30 0.42 DEV -0.02 0.02 -0.08 -0.02 0.23 0.07 0.10 -0.04 0.21 DEV-N -0.20 -0.17 -0.30 0.23 0.39 0.22 -0.08 0.17 0.46 Entropy -0.16 -0.15 -0.39 -0.19 0.33 0.07 -0.14 0.26 0.34 InfoMax 0.29 0.32 0.01 0.09 0.65 0.58 0.01 0.52 0.56 Corr-C 0.75 0.75 0.63 0.18 0.58 0.59 0.07 0.70 0.65 MCC (V) 0.10 0.11 -0.22 -0.08 0.45 0.23 -0.14 0.33 0.47 BNM (V) 0.24 0.27 -0.04 0.08 0.65 0.57 0.01 0.49 0.55 SND 0.76 0.76 0.68 0.24 0.39 0.50 0.16 0.62 0.59 ClassAMI -0.18 0.64 0.44 -0.28 0.57 0.44 0.04 0.07 -0.25 MixVal 0.43 0.37 0.46 0.36 0.30 0.12 -0.10 0.37 0.15 TransScore 0.13 0.06 -0.17 0.09 0.63 0.57 -0.02 0.31 0.54 Table 33: Within-algorithm Spearman’s ρ on Child CXR→ . Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction. MMD DANN CDAN DALN MCC BNM ATDOC MCD CoUDA Source-Risk 0.19 0.11 -0.08 0.22 0.12 0.17 -0.08 0.02 -0.11 IWCV 0.20 0.16 0.12 0.14 0.23 0.09 -0.02 0.14 0.04 DEV 0.12 0.09 0.03 0.17 0.22 0.08 -0.02 0.15 -0.05 DEV-N 0.19 0.10 -0.07 0.22 0.12 0.18 -0.08 0.03 0.27 Entropy -0.12 -0.29 -0.25 0.44 0.07 0.32 -0.01 0.60 -0.43 InfoMax 0.81 0.79 0.87 0.47 0.58 0.44 0.09 0.73 -0.36 Corr-C 0.96 0.97 0.99 0.43 0.61 0.38 0.13 0.69 -0.16 MCC (V) -0.01 -0.15 0.04 0.42 0.15 0.34 -0.01 0.61 -0.42 BNM (V) 0.76 0.72 0.84 0.47 0.57 0.44 0.09 0.73 -0.37 SND 0.90 0.92 0.96 -0.27 0.58 0.29 0.15 0.46 0.58 ClassAMI 0.35 -0.08 0.16 0.21 0.05 0.12 0.12 0.45 0.06 MixVal 0.07 0.24 0.44 0.39 -0.19 -0.16 0.11 0.18 -0.14 TransScore 0.22 -0.06 -0.11 0.19 0.40 0.39 0.06 0.65 -0.37 Table 34: Within-algorithm Spearman’s ρ on LDD→ . Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction. MMD DANN CDAN DALN MCC BNM ATDOC MCD CoUDA Source-Risk 0.57 0.58 0.06 0.68 0.20 0.74 0.30 0.63 -0.08 IWCV 0.43 0.22 -0.19 -0.03 -0.18 -0.25 -0.02 -0.16 -0.16 DEV 0.30 0.21 -0.07 0.09 -0.01 -0.14 -0.03 0.10 -0.04 DEV-N 0.55 0.55 0.04 0.66 0.18 0.72 0.28 0.57 -0.10 Entropy 0.53 0.57 -0.06 0.68 0.07 0.67 0.26 0.69 -0.07 InfoMax 0.78 0.85 0.85 0.87 0.89 0.84 0.39 0.93 0.40 Corr-C 0.65 0.75 0.87 0.81 0.87 0.76 0.39 0.83 0.84 MCC (V) 0.63 0.66 0.08 0.78 0.20 0.77 0.34 0.76 0.05 BNM (V) 0.77 0.84 0.81 0.86 0.87 0.84 0.39 0.93 0.35 SND 0.16 0.34 0.70 0.09 0.74 0.34 0.11 0.44 0.67 ClassAMI 0.30 0.15 0.23 0.36 0.26 0.68 0.45 0.48 0.38 MixVal 0.66 0.58 0.43 0.68 0.23 -0.08 0.24 0.42 0.42 TransScore 0.50 0.58 0.20 0.65 0.73 0.80 0.27 0.65 0.01 Table 35: Within-algorithm Spearman’s ρ on CRD→ . Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction. MMD DANN CDAN DALN MCC BNM ATDOC MCD CoUDA Source-Risk -0.01 0.44 0.42 0.11 0.46 0.15 -0.04 -0.18 0.31 IWCV 0.28 0.13 0.19 0.09 0.10 0.13 0.15 0.17 -0.02 DEV 0.25 0.15 0.30 0.13 0.14 0.13 -0.15 0.15 0.02 DEV-N 0.01 0.45 0.43 0.12 0.47 0.16 0.96 -0.18 0.88 Entropy -0.21 0.34 0.28 -0.02 0.46 0.06 -0.07 -0.23 0.27 InfoMax -0.02 0.49 0.47 0.15 0.03 0.31 0.01 -0.34 0.24 Corr-C 0.39 0.58 0.43 0.38 -0.13 0.35 0.05 -0.42 0.09 MCC (V) -0.14 0.39 0.34 0.05 0.47 0.10 -0.05 -0.24 0.26 BNM (V) -0.05 0.48 0.45 0.14 0.06 0.29 -0.02 -0.33 0.24 SND 0.50 0.21 0.15 0.37 -0.25 0.32 0.12 -0.14 -0.34 ClassAMI -0.10 0.44 0.34 0.19 0.32 0.17 -0.18 -0.15 -0.10 MixVal -0.12 0.32 0.29 0.06 0.38 -0.02 -0.18 0.01 0.18 TransScore -0.12 0.29 0.27 0.04 0.14 0.28 0.02 -0.28 0.26 Table 36: Within-algorithm Spearman’s ρ on OCT→ . Each cell is the mean ρ over three random seeds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction. MMD DANN CDAN DALN MCC BNM ATDOC MCD Source-Risk -0.08 0.40 0.27 0.59 -0.15 -0.12 -0.29 0.15 IWCV -0.19 0.22 0.16 0.75 0.11 0.16 -0.17 0.07 DEV 0.07 0.04 0.02 -0.15 -0.04 -0.12 -0.11 -0.01 DEV-N 0.82 0.84 0.74 0.98 0.91 0.91 0.91 0.90 Entropy -0.15 0.44 0.22 0.76 -0.11 -0.16 -0.34 0.15 InfoMax -0.11 0.48 0.60 0.78 0.45 -0.04 -0.37 0.32 Corr-C 0.15 0.46 0.70 0.78 0.52 0.05 -0.26 0.37 MCC (V) -0.14 0.45 0.26 0.77 -0.02 -0.14 -0.35 0.18 BNM (V) -0.11 0.48 0.56 0.78 0.44 -0.05 -0.37 0.32 SND 0.26 -0.12 0.14 0.13 0.51 0.22 -0.02 0.34 ClassAMI 0.12 0.44 0.29 0.26 0.18 0.07 -0.06 -0.10 MixVal -0.13 0.48 0.26 0.72 0.27 -0.07 -0.28 0.22 TransScore -0.16 0.29 0.15 0.56 0.24 -0.16 -0.37 0.34 Table 37: Within-algorithm Spearman’s ρ on SLO→ . Each cell is the mean ρ over three random seeds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction. MMD DANN CDAN DALN MCC BNM ATDOC MCD Source-Risk 0.16 0.36 0.49 0.53 -0.35 -0.09 -0.26 0.19 IWCV 0.01 -0.06 -0.03 -0.40 0.15 0.02 0.08 0.04 DEV -0.11 0.19 0.10 -0.02 0.01 0.07 0.05 -0.04 DEV-N 0.12 0.33 0.37 0.54 -0.34 -0.07 -0.26 0.18 Entropy 0.04 0.35 0.51 0.65 -0.39 -0.18 -0.44 0.13 InfoMax 0.09 0.36 0.59 0.68 -0.18 -0.07 -0.43 0.18 Corr-C 0.07 0.31 0.41 0.69 -0.06 -0.05 -0.33 0.26 MCC (V) 0.08 0.36 0.53 0.67 -0.40 -0.15 -0.44 0.14 BNM (V) 0.08 0.36 0.59 0.68 -0.19 -0.07 -0.43 0.18 SND 0.07 -0.02 -0.07 0.12 0.08 0.06 0.01 0.18 ClassAMI -0.36 0.41 0.47 0.60 -0.23 -0.18 -0.26 -0.03 MixVal 0.08 0.35 0.50 0.61 -0.19 -0.13 -0.19 0.34 TransScore 0.09 0.31 0.46 0.05 -0.32 -0.07 -0.38 0.24