Paper deep dive
Picking the Right Image to Classify: Reliable-Input Selection in Teledermatology
Fabian Gröger, Marco Weishaupt, Philippe Gottfrois, Simone Lionetti, Linda Wermelinger, Nipun Ranasekara, Ludovic Amruthalingam, Alexander A. Navarini, Marc Pouly
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/23/2026, 1:40:12 AM
Summary
This paper introduces and benchmarks 'reliable-input selection' for teledermatology, a task where multiple images of the same case are available, and the goal is to select the image most likely to be classified correctly by a frozen model. The authors demonstrate that an oracle selector can improve weighted F1 by approximately 20 percentage points across six datasets and nine backbones. However, they find that training-data-free selectors (embedding norm, neighborhood consensus, perturbation stability, classifier confidence) and reference-set selectors (Mahalanobis distance, fusion) fail to close this gap significantly, with the best methods recovering only about a quarter of the potential gain. The study highlights that while there is ample 'room' for improvement in mixed cases, the signal for reliability in current frozen encoders is too weak.
Entities (15)
Relation Signals (14)
Oracle → achievesimprovementon → Weighted-F1
confidence 95% · An oracle... raises weighted F1 by about 20 percentage points
Reliable-Input Selection → appliesto → Teledermatology
confidence 95% · Dermatology models face distribution shifts in teledermatology settings... We call this task reliable-input selection.
Classifier Confidence → isbenchmarkedas → Reliable-Input Selection
confidence 95% · We benchmark four such training-data-free selectors... classifier confidence
Mahalanobis Distance → isbenchmarkedas → Reliable-Input Selection
confidence 95% · The best selector overall, a fusion of confidence and Mahalanobis distance
Reliable-Input Selection → evaluatedon → Derm7pt
confidence 92% · derm7pt [13]... clinical and a dermoscopic image
Reliable-Input Selection → evaluatedon → HAM10000
confidence 92% · HAM10000 [24]... repeated dermoscopy
Reliable-Input Selection → evaluatedon → PAD-UFES-20
confidence 92% · PAD-UFES-20 [22]... repeated smartphone photos
Reliable-Input Selection → evaluatedon → DermaCon-IN
confidence 92% · DermaCon-IN [19]... selection is per-patient image triage
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Dermatology models face distribution shifts in teledermatology settings, where submitted images differ from the training data in lighting, angle, distance, focus, and framing. These test-time images are ordinary clinical photographs, but some fall outside the model's training conditions, leading the model to often misclassify them due to shifts in acquisition between training and deployment. When multiple images of the same case exist (several photos of one patient or lesion), a natural way to improve accuracy is therefore to select the image the model is most likely to classify correctly. We call this task reliable-input selection. An oracle that, for each case, selects a correctly classified image when one exists raises weighted F1 by about 20 percentage points on average across six dermatology datasets and nine frozen backbones. This oracle is an upper bound that sees the labels, whereas a selector must choose blindly. Capturing this gain in practice is hard. A selector that needs no pretraining data applies to any frozen model, including those whose data is not public. It must judge reliability from quantities the model exposes at inference: its embeddings, their norms, and its confidence. We benchmark four such training-data-free selectors: the embedding norm, the neighborhood consensus among a case's images, the stability of the prediction under small perturbations, and the model's own confidence. No training-data-free selector substantially narrows this oracle gap. The best of them is the model's own confidence, but it recovers only a small part of the gap on the clinical datasets. A small labeled reference set does not help either: the best selector overall, a fusion of confidence and Mahalanobis distance, still leaves most of the gap. To our knowledge, this is the first study to introduce and benchmark reliable input selection, a clinically important, unsolved task.
Tags
Links
- Source: https://arxiv.org/abs/2608.16198v1
- Canonical: https://arxiv.org/abs/2608.16198v1
Trouble viewing inline? Open PDF directly →
Full Text
29,677 characters extracted from source content.
Expand or collapse full text
Picking the Right Image to Classify: Reliable-Input Selection in Teledermatology Fabian Gröger Affiliation: University of Basel Affiliation: Lucerne University of Applied Sciences and Arts Marco Weishaupt Affiliation: Lucerne University of Applied Sciences and Arts Philippe Gottfrois Affiliation: University of Basel Affiliation: University Hospital Basel ∗ equal contribution † equal advising Simone Lionetti Affiliation: Lucerne University of Applied Sciences and Arts Linda Wermelinger Affiliation: University of Basel Affiliation: Lucerne University of Applied Sciences and Arts Nipun Ranasekara Affiliation: University of Basel Affiliation: Lucerne University of Applied Sciences and Arts Ludovic Amruthalingam Affiliation: Lucerne University of Applied Sciences and Arts Alexander A. Navarini Affiliation: University of Basel Affiliation: University Hospital Basel ∗ equal contribution † equal advising Marc Pouly Affiliation: Lucerne University of Applied Sciences and Arts Abstract Dermatology models face distribution shifts in teledermatology settings, where submitted images differ from the training data in lighting, angle, distance, focus, and framing. These test-time images are ordinary clinical photographs, but some fall outside the model’s training conditions, leading the model to often misclassify them due to shifts in acquisition between training and deployment. When multiple images of the same case exist (several photos of one patient or lesion), a natural way to improve accuracy is therefore to select the image the model is most likely to classify correctly. We call this task reliable-input selection. An oracle that, for each case, selects a correctly classified image when one exists raises weighted F1 by about 2020 percentage points on average across six dermatology datasets and nine frozen backbones. This oracle is an upper bound that sees the labels, whereas a selector must choose blindly. Capturing this gain in practice is hard. A selector that needs no pretraining data applies to any frozen model, including those whose data is not public. It must judge reliability from quantities the model exposes at inference: its embeddings, their norms, and its confidence. We benchmark four such training-data-free selectors: the embedding norm, the neighborhood consensus among a case’s images, the stability of the prediction under small perturbations, and the model’s own confidence. No training-data-free selector substantially narrows this oracle gap. The best of them is the model’s own confidence, but it recovers only a small part of the gap on the clinical datasets. A small labeled reference set does not help either: the best selector overall, a fusion of confidence and Mahalanobis distance, still leaves most of the gap. To our knowledge, this is the first study to introduce and benchmark reliable input selection, a clinically important, unsolved task. Keywords: Dermatology Reliable-input selection Acquisition shift. 1 Introduction one patient, several imageswhat we do: choose which image to classifyx1x_1 matches training ✓x2x_2 off-angle / distant ×x3x_3 unusual lighting ×frozenmodelz1z_1z2z_2z3z_3score each image,select argmaxvs(zv) _vs(z_v)classify theselected imagey y ✓ Figure 1: Reliable-input selection. In teledermatology a patient submits several images of their skin condition, varying in body site and acquisition. Here one matches the model’s training conditions (standardized) and two differ in angle, distance, or lighting (off-condition). A frozen model embeds each image as zvz_v and classifies the standardized image correctly (✓) but the off-condition images incorrectly (×), silent failures that selection avoids. The task is to score the images and classify the selected one. Dermatology models are often trained on controlled, standardized clinical photographs and applied afterwards to images captured at test time under different angles, distances, or lighting conditions. Such acquisition shifts are a well-documented cause of accuracy loss [7, 23]. These images are ordinary clinical photographs, not anomalies, so an out-of-distribution detector would not flag them and the model classifies them as usual. But because it was not trained for these conditions, it often misclassifies them, apparently confident [11]. When multiple images of the same case are available, they can arise in several ways, three of which we study: repeated captures of one lesion, several viewpoints or modalities of one lesion, or several photographs of one patient covering different body sites, as in teledermatology. Rather than classifying an arbitrary image, one can select the image the model is most likely to classify correctly, a task we call reliable-input selection (Fig. 1). Here we show that an oracle that, for each case, selects a correctly classified image when one exists raises weighted F1 by about 2020 percentage points on average without a stronger model or further training, a gain that current systems rarely exploit. A selector that needs no pretraining data applies to any frozen model, and works from the frozen embeddings alone, with at most a small set of labeled images to calibrate a probe. We benchmark four such training-data-free selectors, together with reference-set selectors that may also use a small labeled set, across six datasets and nine backbones, and the results are largely negative. No selector recovers much of the gap. The model’s own confidence helps on the multi-image clinical datasets but does not beat a fixed best view, the simple strategy of always using the single strongest acquisition type, where such a type is identifiable. Even a small labeled reference set leaves most of the gap. Our contributions are: (1) To our knowledge, we are the first to introduce and benchmark reliable-input selection: choosing, among a case’s images, the one a deployed dermatology model classifies most reliably, without its pretraining data. (2) We quantify a large oracle gap (about 2020 percentage points of weighted F1 on average over six datasets) and show that four training-data-free selectors leave most of it. The strongest, classifier confidence, recovers at most about a quarter of the gap and does not beat a fixed best view where one is available. (3) We trace the difficulty to the task itself rather than to a specific backbone or a simple methodological fix, and even a small labeled reference set helps little: the best such selector, a fusion of confidence and a class-conditional Mahalanobis distance, also recovers at most about a quarter of the gap. 2 Related Work Generalization under acquisition shift. A large body of literature documents that accuracy drops when test inputs deviate from the training distribution. 7 showed that networks matching humans on clean images are far less robust to image distortions. 23 showed that robustness to synthetic perturbations does not transfer to natural shifts and that the main known remedy is training on larger, more diverse data. These studies establish that shift degrades accuracy, but none select among several available images for a given case at inference time. To our knowledge, no prior work studies this choice, and we are the first to introduce and benchmark it. Test-time adaptation [17] updates the model to fit each shifted input, and multi-modal priors [28] add side information. Both change the model or its training, whereas we leave the model fixed and only choose which of a case’s images to classify. Confidence, OOD detection, and sample rejection. Selective prediction equips models with a reject option, abstaining on inputs deemed unreliable [6, 11], often via confidence or out-of-distribution scores such as maximum softmax probability [12], ODIN [18], and Mahalanobis distance [16]. Our task differs in two ways. We do not abstain but select among several valid images of one case, so the case is always processed. And the failure is not an anomaly: an unfamiliar but valid acquisition produces an ordinary, in-distribution embedding that outlier and density scores miss, yet the model misclassifies it. Foundation models and video in dermatology. Domain-specific foundation models provide strong dermatology embeddings with little labeled data [27, 15, 26], while audits of the field’s data basis reveal wide coverage gaps across skin tones and conditions [9], which makes deployment-time reliability important. Video pipelines exploit redundancy across frames for more resilient detection [1], and selecting reliable frames is a direct companion to such systems. 3 Methods 3.1 The Reliable-Input Selection Task Each case has V images x1,…,xV\x_1,…,x_V\, where a case is a single lesion or, in the per-patient datasets below, one patient. A frozen backbone maps them to embeddings z1,…,zV\z_1,…,z_V\, a selection rule picks one index v, and the prediction on xvx_v is reported. The metric is the final downstream performance of the selected images, which we report as weighted F1 because the diagnostic classes are imbalanced. We use six publicly available dermatology datasets, two in each of three multi-image regimes, and our primary focus is the per-patient regime, the closest match to real-world teledermatology. Multiple images per patient (different body sites): PASSION [8], patient-submitted images from sub-Saharan Africa across Fitzpatrick skin types I–VI (1,0221,022 cases, 44 conditions, 22–1818 images per case, median 33), and DermaCon-IN [19], a clinical collection from India (1,4571,457 cases, 88 classes, 22–1313 images), where a case is a patient and selection is per-patient image triage rather than a choice among aligned views. Distinct viewpoints or modalities of one lesion, the aligned datasets and the only ones for which a fixed best view is defined: SCIN [25], whose cases give three images of one lesion from distinct viewpoints (close-up, at an angle, at a distance), from which we take the highest-weighted expert label, keep the 2020 most frequent conditions, and drop records lacking all three views or a label, leaving 1,0451,045 lesions, and derm7pt [13], a clinical and a dermoscopic image of each lesion. Repeated captures of one lesion: HAM10000 [24] (repeated dermoscopy, 1,9561,956 lesions, 22–66 captures) and PAD-UFES-20 [22] (repeated smartphone photos, 512512 lesions, 22–88 captures). 3.2 Backbones We evaluate nine backbones spanning multiple pretraining paradigms, with no fine-tuning. These include PanDerm [27] (dermatology, masked image modeling), MONET [14] (contrastive CLIP), supervised [5] and masked-autoencoding [10] ImageNet ViTs, as well as DINO [3] and DINOv2 [21] (self-distillation). For DINOv2, we test four variants: plain, register [4], patch-mean, and register-plus-patch-mean. Spanning these paradigms tests whether the difficulty is a property of one model or of the task. None of the backbones is trained on our evaluation datasets: the ImageNet backbones use no dermatology data, PanDerm is pretrained on a corpus that excludes public benchmarks [27], and MONET on dermatology image-text pairs drawn from the medical literature [14]. 3.3 Selectors A selector assigns each image a score and takes the per-case argmax . We group selectors by what they are allowed to use. Baseline and references. Our baseline is random (classify any of the case’s images): the realistic default, and the only choice defined for every dataset. We also report two ways to use a case’s images without selecting one: majority vote (the most frequent prediction across the images, random tie-breaks) and soft vote (the argmax of their mean predicted probability). Two non-deployable references bound what selection could achieve. Best fixed always classifies the same acquisition type, the one with the highest average F1. It is defined only when every case shares the same set of acquisition types, such as SCIN’s three viewpoints or derm7pt’s two modalities, and is undefined when a case’s images are not aligned this way. The per-case oracle picks a correctly classified image when one exists, which marks the upper bound any selector could reach. Training-data-free. The realistic deployment setting, in which selectors use only the frozen embeddings and the given probe. The embedding norm selector scores each image by LevyScore [20], logpdfχK(∥z∥) \, _K( z ): the typicality of the norm ∥z∥ z under an isotropic Gaussian latent space, in which it follows a χK _K distribution that recent self-supervised objectives encourage [2]. Neighborhood consensus scores each image by its mean cosine similarity to the case’s other images and prefers the most central one, on the assumption that the more an image’s embedding deviates from the rest, the less reliable its prediction. Perturbation stability prefers the image whose prediction changes least under small Gaussian perturbations of its embedding, on the assumption that a reliable prediction is locally robust. Classifier confidence is the probe’s maximum softmax probability. Reference-set selectors (small labeled set). To establish an upper reference for the benefit of a small labeled set, we also include two selectors fitted on the practitioner’s own labeled set (still without the model’s pretraining data). The Mahalanobis selector favors the image that looks most typical of its predicted class, measured by the Mahalanobis distance from its embedding to the class mean and covariance. The fusion selector sums this typicality score with the classifier’s confidence, both standardized. The class-conditional mean and covariance are estimated on the same split that fits the probe (Sec. 3.4), not a separate hold-out, so they use no labeled data beyond the probe’s. 3.4 Evaluation Protocol On the frozen features, we evaluate two standard readout probes, a linear (logistic) probe and a kkNN probe (k=5k=5, cosine distance), which is the established protocol for assessing frozen representations [3, 21], and both yield the same trends here. Per seed we draw a stratified 6060/4040 train/test split of cases. Each probe is fit on the training split for one acquisition type and applied to every image of each held-out case. For each backbone we average each metric over the 1010 seeds and training-view choices. Because the classes are imbalanced, we report weighted F1. Error bars show ±1± 1 standard deviation across the nine backbones. Accuracy shows the same trends. Figure 2: A large oracle gap that current selectors barely narrow. Weighted F1 on the six datasets, grouped by multi-image regime (mean over nine frozen backbones and 1010 seeds, error bars show ±1± 1 standard deviation across backbones). An oracle that picks a correct image per case (green) reaches F1 well above random selection (gray), whereas the best training-data-free selector (blue) recovers only a small part of that gap. We baseline against random throughout. A fixed best view is omitted, as it is defined only for the aligned datasets and is not deployable (see Sec. 4.1). 4 Experiments and Results 4.1 The Oracle Gap We first measure the gain a perfect selection could recover. An oracle that, for each case, picks a correct image when one exists reaches F1 far above random selection (Fig. 2), adding about 2020 percentage points on average. A fixed best acquisition, defined only on the aligned datasets (SCIN viewpoints, derm7pt modalities), sits just 22 to 88 F1 points above random. As it is not deployable, we use it as a realistic target rather than the baseline we measure gains against. 4.2 Selectors We next ask how much of the gap any selector recovers, and find that none recovers much. Table 1 gives each method’s weighted-F1 gain over random. The embedding norm adds essentially nothing and neighborhood consensus barely helps. The strongest training-data-free selector is the model’s own confidence (+3.7+3.7 on average, up to +5+5 on the clinical datasets). A small labeled reference set helps a little more: a fusion of confidence with a class-conditional Mahalanobis score is the best selector (+4.4+4.4). Both recover at most about a quarter of the oracle gap, so reliable-input selection remains unsolved. Aggregating all of a case’s images rather than selecting one leaves most of the gap too: soft voting reaches +4.4+4.4, as much as the best selector, and majority voting only +1.6+1.6 (it collapses to near-random where V=2V=2 forces tie-breaks). Table 1: No selector recovers much of the oracle gap (weighted-F1 gain over random selection, percentage points, mean over nine backbones, 1010 seeds). The strongest training-data-free selector is the model’s own confidence. A fusion with a class-conditional Mahalanobis score (a small labeled reference set) is best overall, yet both leave the bulk of the oracle gap (last row) unexploited. Majority vote and soft vote aggregate a case’s images instead of selecting one. Datasets are grouped by multi-image regime. The standard deviation across backbones is at most 33 points. per-patient viewpoint / modality repeated capture Method PASSION DermaCon-IN SCIN derm7pt PAD-UFES HAM10000 mean majority vote +3.9+3.9 +2.6+2.6 +1.3+1.3 +0.2+0.2 +0.7+0.7 +1.0+1.0 +1.6+1.6 soft vote +6.3+6.3 +6.0+6.0 +2.4+2.4 +4.8+4.8 +3.6+3.6 +3.6+3.6 +4.4+4.4 training-data-free embedding norm +0.1+0.1 +0.4+0.4 −0.2-0.2 +0.6+0.6 +0.4+0.4 +0.2+0.2 +0.3+0.3 neighborhood consensus +1.1+1.1 +0.7+0.7 +1.0+1.0 +1.0+1.0 +0.5+0.5 +0.5+0.5 +0.8+0.8 perturbation stability +3.5+3.5 +3.3+3.3 +1.1+1.1 +3.0+3.0 +2.1+2.1 +2.0+2.0 +2.5+2.5 classifier confidence +5.2+5.2 +4.8+4.8 +1.6+1.6 +4.5+4.5 +2.8+2.8 +3.3+3.3 +3.7+3.7 with reference set Mahalanobis +1.8+1.8 +1.4+1.4 +1.6+1.6 +7.1+7.1 +2.4+2.4 +0.4+0.4 +2.4+2.4 fusion (conf. ++ geom.) +4.8+4.8 +4.4+4.4 +2.3+2.3 +7.7+7.7 +4.0+4.0 +2.9+2.9 +4.4+4.4 oracle +19.8+19.8 +24.0+24.0 +18.7+18.7 +19.0+19.0 +21.7+21.7 +14.5+14.5 +19.6+19.6 Figure 3: Why selection is hard: ample room, weak signal. Left: every case is all-correct (selection irrelevant), no-image-correct (unrescuable), or mixed (rescuable). The mixed group is large, so there is room to rescue. Right: AUROC of each training-data-free score for per-image correctness on the mixed cases. The embedding norm and neighborhood consensus sit at chance. Only the classifier’s confidence is clearly above it, and even then only moderately, too weak to exploit that room. All panels use all six datasets, in the same order as Fig. 2. The dermoscopy datasets (derm7pt, HAM10000) show the same pattern as the clinical ones. Means over nine backbones and 1010 seeds. Error bars on the right panel show ±1± 1 standard deviation across backbones. 4.3 Sources of Difficulty Selection can help only in the mixed cases shown in Fig. 3 (left), where a case has both correctly and incorrectly classified images. The panel splits every case into all-correct (selection irrelevant), no-image-correct (no selector can help), or mixed. This room is not the bottleneck: the mixed group is substantial across datasets, from 31%31\% (HAM10000) to 49%49\% (DermaCon-IN) of cases, so there is ample opportunity to rescue. (On SCIN a further 59%59\% of cases have no correct image at all and are beyond any selector’s reach.) The bottleneck is signal: whether any training-data-free score can identify the correct image among the mixed cases, which we measure as the AUROC of each score for per-image correctness (Fig. 3, right). The embedding norm sits at chance and neighborhood consensus barely above it. Only the model’s own confidence is clearly above chance (AUROC 0.580.58 to 0.700.70, highest on PASSION), with perturbation stability, which derives from it, in between. A usable per-image reliability signal exists, but lives only in the classifier’s confidence, is at best moderate, and is too weak to exploit the room that exists. This explains the small gains directly: a selector’s accuracy is the all-correct fraction plus the mixed fraction times its hit rate on the mixed cases, and even the confidence selector raises that hit rate only modestly above random, far short of the oracle. Ample room with a weak signal leaves little to capture (Table 1). Selection is also not replaceable by a single fixed choice. On SCIN, the oracle’s correct picks spread almost evenly across the three viewpoints (36%36\%, 33%33\%, 31%31\%). The best image changes from case to case, beyond any fixed viewpoint. 4.4 Generality of the Result The failure to recover the gap is tied to the dataset, not the backbone: the oracle gap’s standard deviation across the nine backbones is under 22 points, far smaller than its variation across datasets (Table 1). It is also not removed by a simple methodological change. It also recurs across all three regimes: the oracle gap is large in every one (+14+14 to +24+24 points), and the confidence selector recovers most in the per-patient regime (+5+5) and less elsewhere (+3+3). The embedding norm is at chance across all nine backbones, because the images of a case are nearly indistinguishable in norm and position. The gap is also not an artifact of keeping 2020 classes: restricting SCIN to the K most frequent conditions (K∈5,10,15,20K∈\5,10,15,20\) leaves the oracle 1616 (at the full 2020 conditions) to 2323 (at 55) weighted-F1 points above the best training-data-free selector. 5 Discussion and Conclusion Reliable input selection is clinically relevant, since the choice of input alone adds about 2020 percentage points of weighted F1 on average across six datasets, led by the clinical teledermatology collections. Yet the task remains unsolved without the model’s pretraining data: across nine backbones, no training-data-free selector recovers more than a small fraction of the gap, and even a small labeled reference set leaves most of it. A selector can only succeed if the model exposes a reliable signal of its own competence, which current frozen encoders do not provide. A likely obstacle is the heavy augmentation used in self-supervised pretraining. While it pushes encoders toward invariance to the acquisition changes at issue here, the resulting embeddings remain far from truly invariant. Two images of the same case receive different predictions 34%34\% to 69%69\% of the time (50%50\% on average). Their embeddings move, on average, about 0.62×0.62× as far as embeddings of entirely different cases, indicating a within- to between-case cosine-distance ratio of 0.360.36 to 1.01.0 across datasets. A promising direction is therefore to train or adapt encoders whose embeddings track input reliability rather than discarding it. The absolute scores reflect benchmark difficulty (a hard 2020-class SCIN task, small usable subset, noisy labels), but our claim is relative: the oracle gap and the failure of every selector to recover it hold across datasets, backbones, and seeds. Acknowledgements S.L., L.A., L.W., N.R., and M.P. are supported by the Swiss National Science Foundation (SNSF) under grant 20HW-1_228541. Disclosure of Interests. The authors have no competing interests to declare that are relevant to the content of this article. References [1] S. T. Ahmed, A. S. Guthur, P. K. Rai, and P. S. N. (2025) Advanced video-based deep learning framework for comprehensive detection, diagnosis, and classification of dermatological conditions in real-time datasets. Procedia Computer Science 259, p. 424–432. External Links: Document Cited by: §2. [2] R. Balestriero and Y. LeCun (2025) LeJEPA: provable and scalable self-supervised learning without the heuristics. Note: arXiv:2511.08544 Cited by: §3.3. [3] M. Caron, H. Touvron, I. Misra, et al. (2021) Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: §3.2, §3.4. [4] T. Darcet, M. Oquab, J. Mairal, and P. Bojanowski (2024) Vision transformers need registers. In International Conference on Learning Representations (ICLR), Cited by: §3.2. [5] A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al. (2021) An image is worth 16x16 words: transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), Cited by: §3.2. [6] Y. Geifman and R. El-Yaniv (2017) Selective classification for deep neural networks. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §2. [7] R. Geirhos, C. R. Medina Temme, J. Rauber, et al. (2018) Generalisation in humans and deep neural networks. In Advances in Neural Information Processing Systems, Vol. 31. Cited by: §1, §2. [8] P. Gottfrois, F. Gröger, F. H. Andriambololoniaina, L. Amruthalingam, A. Gonzalez-Jimenez, C. Hsu, A. Kessy, S. Lionetti, D. Mavura, W. Ng’ambi, et al. (2024) Passion for dermatology: bridging the diversity gap with pigmented skin images from sub-saharan africa. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 703–712. Cited by: §3.1. [9] F. Gröger, S. Lionetti, P. Gottfrois, et al. (2025) A global atlas of digital dermatology to map innovation and disparities. Note: arXiv:2601.00840 Cited by: §2. [10] K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick (2022) Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 15979–15988. External Links: Document Cited by: §3.2. [11] K. Hendrickx, L. Perini, D. Van der Plas, W. Meert, and J. Davis (2024) Machine learning with a reject option: a survey. Machine Learning 113 (5), p. 3073–3110. External Links: Document Cited by: §1, §2. [12] D. Hendrycks and K. Gimpel (2017) A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations (ICLR), Cited by: §2. [13] J. Kawahara, S. Daneshvar, G. Argenziano, and G. Hamarneh (2019) Seven-point checklist and skin lesion classification using multitask multimodal neural nets. IEEE Journal of Biomedical and Health Informatics 23 (2), p. 538–546. External Links: Document Cited by: §3.1. [14] C. Kim, S. U. Gadgil, A. J. DeGrave, et al. (2024) Transparent medical image AI via an image-text foundation model grounded in medical literature. Nature Medicine 30 (4), p. 1154–1165. External Links: Document Cited by: §3.2. [15] A. P. Kiraly, S. Baur, K. Philbrick, F. Mahvar, L. Yatziv, T. Chen, B. Sterling, N. George, F. Jamil, J. Tang, et al. (2024) Health AI developer foundations. Note: arXiv:2411.15128 Cited by: §2. [16] K. Lee, K. Lee, H. Lee, and J. Shin (2018) A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §2. [17] J. Liang, R. He, and T. Tan (2025) A comprehensive survey on test-time adaptation under distribution shifts. International Journal of Computer Vision 133 (1), p. 31–64. External Links: Document Cited by: §2. [18] S. Liang, Y. Li, and R. Srikant (2018) Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference on Learning Representations (ICLR), Cited by: §2. [19] S. S. Madarkar, M. Madarkar, M. Venkatesh, T. Prakash, K. R. Mopuri, V. MV, K. Sathwika, A. Kasturi, G. Raj, P. Supranitha, et al. (2025) DermaCon-in: a multiconcept-annotated dermatological image dataset of indian skin disorders for clinical ai research. Advances in Neural Information Processing Systems 38. Cited by: §3.1. [20] L. Maes, D. Scieur, and R. Balestriero (2025) LevyScore: a fast sample-wise confidence score of pretrained joint embedding model. In Workshop on Unifying Representations in Neural Models (UniReps), External Links: Link Cited by: §3.3. [21] M. Oquab, T. Darcet, T. Moutakanni, et al. (2024) DINOv2: learning robust visual features without supervision. Transactions on Machine Learning Research. External Links: Link Cited by: §3.2, §3.4. [22] A. G. C. Pacheco, G. R. Lima, A. S. Salomão, et al. (2020) PAD-UFES-20: a skin lesion dataset composed of patient data and clinical images collected from smartphones. Data in Brief 32, p. 106221. External Links: Document Cited by: §3.1. [23] R. Taori, A. Dave, V. Shankar, et al. (2020) Measuring robustness to natural distribution shifts in image classification. In Advances in Neural Information Processing Systems, Vol. 33, p. 18583–18599. Cited by: §1, §2. [24] P. Tschandl, C. Rosendahl, and H. Kittler (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 5, p. 180161. External Links: Document Cited by: §3.1. [25] A. Ward, J. Li, J. Wang, et al. (2024) Creating an empirical dermatology dataset through crowdsourcing with web search advertisements. JAMA Network Open 7 (11), p. e2446615. External Links: Document Cited by: §3.1. [26] S. Yan, M. Hu, Y. Jiang, X. Li, H. Fei, P. Tschandl, H. Kittler, and Z. Ge (2025) Derm1M: a million-scale vision-language dataset aligned with clinical ontology knowledge for dermatology. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), External Links: Link Cited by: §2. [27] S. Yan, Z. Yu, C. Primiero, et al. (2025) A multimodal vision foundation model for clinical dermatology. Nature Medicine 31 (8), p. 2691–2702. External Links: Document Cited by: §2, §3.2. [28] H. Zhou, L. Halilaj, S. Monka, S. Schmid, Y. Zhu, B. Xiong, and S. Staab (2025) Robust visual representation learning with multi-modal prior knowledge for image classification under distribution shift. Note: arXiv:2410.15981 Cited by: §2.