Paper deep dive
FRAME: separating sampling variation from representational cause in medical imaging fairness
Mahshad Lotfinia, Daniel Truhn, Andreas Maier, Soroosh Tayebi Arasteh
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/29/2026, 3:45:16 AM
Summary
The paper introduces FRAME (Fair-model Reference And Mechanism Evaluation), a two-step framework to audit subgroup performance differences in medical imaging AI. Step 1 generates a 'fair-model reference' by simulating performance differences expected under exact fairness given observed subgroup sizes, effectively quantifying sampling variation. Step 2 tests the remaining 'remainder' difference by injecting candidate causes (demographic decodability, disease direction entanglement) into model representations. Results across 702,206 images and 36 encoders show the reference accounts for a median 41% of race differences and 22% of age differences. Interventions like injecting demographic decodability did not change the remainder, while entangling group with disease direction increased differences. The framework suggests many reported fairness issues are compatible with sampling variation rather than mechanistic bias.
Entities (12)
Relation Signals (8)
FRAME → distinguishes → Sampling Variation
confidence 95% · Applying FRAME before choosing an intervention could distinguish differences that need a mechanistic explanation from differences compatible with sampling variation
FRAME → uses → Fair-model Reference
confidence 95% · The first step derives a fair-model reference... In the second step, we test the remainder
Fair-model Reference → accountsfor → Race Difference
confidence 90% · the reference accounts for a median 41% of the reported race difference
Fair-model Reference → accountsfor → Age Difference
confidence 90% · and 22% of the age difference
Demographic Decodability → doesnotchange → Remainder Difference
confidence 90% · Injecting demographic decodability leaves the remainder unchanged
Dermatology → hassubgroupdifference → Skin Type
confidence 90% · Dermatology classifiers lose accuracy on dark skin
Chest Radiography → hassubgroupdifference → race
confidence 90% · Chest radiograph classifiers underdiagnose Black patients... race difference
Entangling Group with Disease → increases → Race Difference
confidence 90% · entangling the group with the disease direction raises the race difference from 0.077 to 0.118
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Subgroup performance differences are the standard evidence for fairness bias in medical imaging, and the usual response removes the demographic information that a model encodes. Here we introduce Fair-model Reference And Mechanism Evaluation (FRAME), a two-step framework for auditing such a claim. The first step derives a fair-model reference, the distribution of the difference under exact fairness at the observed subgroup sizes. In the second step, we test the remainder with two operators in representation space. One operator cannot change a within-group ranking by construction. Across 702,206 images and 36 encoders, the reference accounts for a median 41% of the reported race difference and 22% of the age difference. Injecting demographic decodability leaves the remainder unchanged, while entangling the group with the disease direction raises the race difference from 0.077 to 0.118. No intervention we tested changes the remainder more than a change of random seed does. Those interventions reduce a difference at the operating point and leave the within-group ranking difference at a median of 0.000. Applied to 89 differences in 9 published studies across 6 medical imaging modalities, the reference accounts for a median 25% of a rate difference and 70% of a difference in the area under the receiver operating characteristic curve. Image-text pretraining instead raises worst-group performance by about 0.05. Applying FRAME before choosing an intervention could distinguish differences that need a mechanistic explanation from differences compatible with sampling variation at the current cohort sizes.
Tags
Links
- Source: https://arxiv.org/abs/2608.25981v1
- Canonical: https://arxiv.org/abs/2608.25981v1
Trouble viewing inline? Open PDF directly →
Full Text
158,146 characters extracted from source content.
Expand or collapse full text
FRAME: separating sampling variation from representational cause in medical imaging fairness Mahshad Lotfinia 1 , Daniel Truhn 2,3 , Andreas Maier 1 , and Soroosh Tayebi Arasteh 2,3∗ 1 Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany 2 Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany 3 Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany ∗ Correspondence to: Soroosh Tayebi Arasteh (soroosh.arasteh@rwth-aachen.de) Abstract. Subgroup performance differences are the standard evidence for fairness bias in medical imaging, and the usual response removes the demographic information that a model encodes. Here we introduce Fair-model Reference And Mechanism Evaluation (FRAME), a two-step framework for auditing such a claim. The first step derives a fair-model reference, the distribution of the difference under exact fairness at the observed subgroup sizes. In the second step, we test the remainder with two operators in representation space. One oper- ator cannot change a within-group ranking by construction. Across 702,206 images and 36 encoders, the reference accounts for a median 41% of the reported race difference and 22% of the age difference. Injecting demographic decodability leaves the remainder unchanged, while entangling the group with the disease direction raises the race difference from 0.077 to 0.118. No intervention we tested changes the remainder more than a change of random seed does. Those interventions reduce a difference at the operating point and leave the within- group ranking difference at a median of 0.000. Applied to 89 differences in 9 published studies across 6 medical imaging modalities, the reference accounts for a median 25% of a rate difference and 70% of a difference in the area under the receiver operating character- istic curve. Image-text pretraining instead raises worst-group performance by about 0.05. Applying FRAME before choosing an intervention could distinguish differences that need a mechanistic explanation from differences compatible with sampling variation at the current cohort sizes. Introduction Artificial intelligence systems for medical imaging perform differently for different groups of pa- tients. Chest radiograph classifiers underdiagnose Black patients, women, and patients on Medicaid, and most severely patients in more than one of those groups [1]. Training on a sex-imbalanced cor- pus lowers performance for the underrepresented sex [2]. Dermatology classifiers lose accuracy on dark skin [3]. Computational pathology models separate White from Black patients by 3 to 16 percentage points of the area under the receiver operating characteristic curve (AUROC) across three diagnostic tasks [4]. These findings have motivated fairness interventions that modify the model [5]. That response depends on a chain of four beliefs. (i) Models show demographic performance differences [1], measured almost everywhere as the maximum minus the minimum subgroup value of a performance measure [6]. (i) A model can recover a patient’s self-reported race from the image alone, at high accuracy and from images degraded far past what a radiologist can read [7]. The features of disease classifiers also encode these characteristics [8]. (i) The difference therefore arises because the model encodes demographics. A model can use race where no radiologist would detect it [7]. Correcting a demographic shortcut also removes the difference inside the training arXiv:2608.25981v1 [cs.CV] 26 Aug 2026 2 distribution [9]. (iv) Removing that encoding will therefore equalize performance. The mitigation literature is built on belief (iv), from a per-group threshold to a constrained loss [10], a worst-group objective [11], an adversarial classifier, and a concept-erasure step [12,13]. Beliefs (i) and (iv) have not been tested by intervention, and (i) has been questioned on grounds that were not quantified. Two commentaries on the underdiagnosis report argued that dataset bias and unmeasured confounders complicate any reported difference [14,15]. Differences in task difficulty can also produce unequal subgroup performance where the model itself is not the source [16]. A causal account showed that current mitigation addresses a narrow set of the mechanisms involved [17]. A multi-site study found that correcting a demographic shortcut is optimal only inside the distribution that it was corrected on [9]. Aggregate metrics hide the risk borne by individual patients and by the smallest groups [18]. Outside medicine, the statistic itself has been shown to be inflated at small samples, since a maximum minus a minimum cannot fall below zero and rises with sampling variation alone [19,20,21]. A subgroup AUROC is a within-group rank statistic, so its value changes only if the ordering of disease scores within that group changes. In medical imaging, we are not aware of a study that has quantified the difference expected under exact fairness at the reported subgroup sizes, or injected a demographic effect into a representation to test whether the subgroup difference changes. Here we introduce Fair-model Reference And Mechanism Evaluation (FRAME), a framework that audits a reported subgroup difference in two steps (Fig. 1). In the first step, we assign every subgroup the model’s own overall AUROC and simulate the statistic at the observed per-subgroup counts. Those simulations define the fair-model reference, which then replaces zero as the com- parison value for a reported difference. In the second step, we inject each candidate cause into the cached features of a released encoder at graded strengths and record whether the remainder changes. One of the two operators cannot change a within-group ranking by construction. We then use FRAME to test beliefs (i) and (iv). We also train 22 encoders under a matrix that varies only the objective, the backbone, the pretraining data composition, and the seed at a fixed budget, since the objective and the pretraining domain are otherwise confounded [22]. Because its first step needs only a reported difference and the counts behind it, we also apply it to 89 subgroup differences in 9 published studies across 6 imaging modalities. The evidence base is 702,206 images from 3 modalities and 10 sites, 36 encoders, 9 mitigation methods, and 4 levels of unfreezing. Every model is open-weight and run locally. The fair-model reference accounts for a median 41% of the reported race difference and 22% of the age difference, and 51 of 130 combinations of encoder and finding do not exceed it for race. In the published record, it accounts for a median 25% of a rate difference and 70% of an AUROC difference. Across encoders, demographic decodability spans 0.659 to 0.872 while the differences themselves span 0.069 to 0.088. Injecting decodability raises decodability to 0.942 while the difference stays at 0.077. Entangling the group with the disease direction raises the difference to 0.118. The median change in the remainder under mitigation is 0.005 for race and 0.008 for age, against 0.012 under a change of pretraining seed. Beliefs (i) and (iv) concern only the remainder, the part of a reported difference that exceeds the fair-model reference. Injecting demographic decodability does not change the remainder. Image-text pretraining raises worst-group AUROC by about 0.05. No fairness intervention we tested produces a comparable gain. Separating a reported difference from what exact fairness produces at those sizes could change which findings in this field need explaining, and which need only a larger cohort. Results We evaluate subgroup performance on four measures: AUROC, sensitivity, the false positive rate (FPR), and the expected calibration error. Each attribute’s subgroup performance difference is the 3 Models perform differently across groups Models encode demographics from images The encoding causes the difference Removing the encoding equalizes performance One test radiograph Cardiomegaly Per-subgroup counts White7,479 Other5,716 Black2,012 Asian747 Hispanic or Latino392 Positive cases in the pooled test split One value per subgroup Reported difference the maximum minus the minimum subgroup value 125,992 radiographs, 41,621 patients, 6 sites 13 findings, 10 frozen encoders 130 combinations Overall performance A ChancePerfect The same counts All subgroups have the same true performance B= 2,000 simulations Reference Reported difference Exceedance level Subgroup difference under exact fairness Above the reference, more than exact fairness produces At or below the reference, consistent with exact fairness Needs only a reported difference and the counts behind it Applied to 89 differences in 9 published studies Cached featuresDecodability injection Adds a demographic direction with no disease signal Cannot change a within-group ranking, by construction Strengths s = 0, 0.25, 0.5, 1.0 Entanglement injection Removes disease signal along the shared direction, in one group Changes the signal that determines the ranking Strengths k = 0, 1, 4, 16, 64, 256 iiiiiiiv Step 1 measures thisStep 2 measures this link a The reported difference b Step 1, the fair-model reference c Step 2, the candidate causes Disease direction Demographic direction One point is one image One simulated fair model Fig. 1: FRAME and the audit of a reported subgroup difference. The top row lists the four beliefs that motivate a fairness intervention, and labels mark which FRAME step evaluates each belief. a, The object being audited. The example image is a cardiomegaly-positive test radiograph from NIH ChestX-ray14 at the 224× 224 preprocessed size used throughout. Classifier performance is evaluated separately in each subgroup. The bars show the number of positive cases in each race subgroup of the pooled test split. The reported difference is the maximum minus the minimum of the per-subgroup values. b, The first step. It uses the model’s own overall performance, marked as the symbol A on a scale from chance to perfect, together with those same counts, and simulates models in which every subgroup has the same true performance. Each simulation still produces a nonzero maximum minus minimum, because a subgroup with a few hundred cases has more sampling variation than a subgroup with several thousand. Their distribution defines the fair-model reference, which replaces zero as the comparison value for the reported difference. A difference above the reference exceeds the variation expected under exact fairness. The shaded tail is the exceedance level. c, The second step, which tests the part above the reference. Each candidate cause is injected into a released encoder’s cached features at graded strengths. One operator adds a demographic direction with no disease signal, and the other removes disease signal along the shared direction. The first operator cannot change a within-group ranking by construction. The comparison therefore distinguishes demographic information that a classifier can decode from representation changes that alter the disease ranking. A solid arrow is a flow and a dashed arrow is a comparison. FRAME, Fair-model Reference And Mechanism Evaluation. 4 maximum minus the minimum of a measure across its groups, where a smaller value is better. Per- formance values are written as the bootstrap mean ± standard deviation with the 95% confidence interval (CI) in brackets [23]. Observed and achievable differences, demographic decodability, and the geometry measures are point estimates without resampling intervals. Tests are two-sided at 0.05 under Benjamini-Hochberg false discovery rate (FDR) control within each test family [24], and a corrected value is written p FDR . Step one of FRAME: the reference against the reported difference We fit one linear disease head per finding on the frozen features of 10 image encoders and evaluated every head on the pooled test split of n = 125,992 chest radiographs from 41,621 patients across 6 sites (Table 1 and Table 2), and computed the subgroup performance difference for self-reported race and for age (Eq. 1). Each head is evaluated only on the images with an available label for that finding. Sex and insurance were evaluated on the same runs and are reported in Supplementary Note 1 and Supplementary Table 1. For each of the 130 resulting combinations of encoder and finding, we derived the reference per attribute at the observed per-subgroup counts (Eq. 2) and tested every observed difference against it. Supplementary Table 2 reports the same quantities per finding. Race is recorded at two of the six sites. The pooled race attribute therefore includes an Other category that is partly a site indicator (Supplementary Table 3 and Supplementary Note 2). A perfectly fair model already produces much of the reported difference (Fig. 2a,b). Across the panel, the observed race difference has a median of 0.082 (range 0.029–0.218) against a fair- model reference of 0.034 (0.013–0.087). The reference accounts for a median 41% of the reported value (interquartile range 26–55). For age, the observed difference is 0.066 (0.014–0.160) against a reference of 0.014 (0.006–0.038), a median share of 22% (11–34) (Fig. 2c). We call the difference that remains after the reference is subtracted the remainder. It still exceeds the reference in most combinations, more often for age than for race. Against its own reference, the age difference is significantly larger in 116 of the 130 combinations and the race difference in 79 (Fig. 2e). Over every combination in the study, which adds the finetuned models and the two other modalities, the counts are 226 of 252 for age and 129 of 243 for race. The race difference is computed across five evaluable subgroups and the age difference across four. A subgroup is evaluable when it contains at least 20 labeled images. The median number of positive cases in the smallest evaluable subgroup is 136 for race and 949 for age (Fig. 2g). Outside dermatology age, no attribute in dermatology or funduscopy exceeds its reference in more than 2 of 9 combinations at these cohort sizes (Fig. 3). The per-combination values are in Supplementary Table 4 and Supplementary Note 3. The reference is largest where the smallest evaluable subgroup is smallest (Fig. 2d,f). For pleural other, at 20 positive cases, it is 0.083 against an observed 0.077, and for support devices, at 396, it is 0.015 against 0.058 (Supplementary Table 2). Across the 130 frozen combinations, the Spearman correlation between the reference and the observed race difference is 0.486 (p FDR = 0.005). For age, the same correlation is −0.140 (p FDR = 0.140). Step two of FRAME: candidate causes of the remainder We measured how well a linear classifier recovers race from each frozen encoder’s features and compared it against the unmitigated difference and against the achievable difference of Eq. 6, then injected each candidate cause into the cached features of two released chest radiograph encoders that we did not train, RAD-DINO [34] and the TorchXRayVision DenseNet-121 [36], at graded strengths over three findings spanning the prevalence range (Fig. 4). Decodability varies far more than the difference does (Fig. 4a). Across the ten encoders, the median race decodability spans 0.659 to 0.872, while the median unmitigated race difference spans 5 Table 1: Imaging cohorts and their subgroup composition. Counts are images at the final curated state of each manifest, with the train, validation, and test columns following the released or assigned split and the last column reporting the number of patients in the test split. Subgroup rows list the harmonized groups that the fairness analysis evaluates, and rows with a missing value for an attribute are listed as not recorded and are excluded from that attribute’s analysis. The chest radiograph race category Other combines patients recorded as another race at MIMIC-CXR and CheXpert with every radiograph from the four sites that record no race field. Age bands are in years. Dermatology sources are split disjointly over cases for DDI and Fitzpatrick17k and over lesions for ISIC 2019, which is why its test patients are fewer than its test images. Each fundus record is one image from one patient. DDI, Diverse Dermatology Images. Source or subgroupImages Train ValidationTest Test patients Chest radiography, six sites MIMIC-CXR [25]243,345 180,30018,409 44,63611,179 CheXpert [26]157,865 115,44913,098 29,3189,810 NIH ChestX-ray14 [27] 112,120 77,8708,654 25,5962,797 PadChest [28]110,525 79,6978,783 22,04513,438 VinDr-CXR [29]18,000 15,00003,0003,000 VinDr-PCXR [30]8,3526,95501,3971,397 All chest radiographs650,207 475,27148,944 125,99241,621 Chest radiograph subgroups Sex, female278,816 203,74722,366 52,70318,289 Sex, male341,000 248,10525,545 67,35019,552 Sex, not recorded30,391 23,4191,0335,9393,782 Age, 0–40118,888 86,7038,170 24,01510,608 Age, 40–60194,383 139,96715,026 39,39012,080 Age, 60–80231,499 169,84417,826 43,82913,082 Age, 80 and over84,456 63,0016,889 14,5664,997 Age, not recorded20,981 15,7561,0334,1922,035 Race, White234,624 174,12218,025 42,47710,972 Race, Black43,419 31,4203,5538,4462,082 Race, Asian23,794 17,5061,9204,3681,407 Race, Hispanic or Latino 11,6878,6218862,180552 Race, Other336,683 243,60224,560 68,52126,621 Dermatology, three sources DDI [3]65639466196196 Fitzpatrick17k [31]16,0129,6071,6014,8044,804 ISIC 2019 [32]25,331 15,0232,6727,6364,179 All dermatology41,999 25,0244,339 12,6369,179 Skin type I–I7,9634,7847852,3942,394 Skin type I–IV6,3303,7416721,9171,917 Skin type V–VI2,3751,476210689689 Retinal fundus, one source Harvard-FairVision [33]10,0006,0001,0003,0003,000 Race, White7,7254,6107812,3342,334 Race, Black1,448881145422422 Race, Asian82750974244244 6 Table 2: The encoder panel and the controlled pretraining matrix. Every model is open-weight and is used locally as a frozen feature extractor at 224× 224 pixels, and the dimension column lists the width of the global embedding for each model. The panel spans domain-specific medical encoders, general-purpose vision encoders at three widths, and an untrained transformer of the same architecture as the smallest controlled backbone, so a difference between encoders is not confounded with model size. The controlled block lists the 22 encoders trained in house from a size-matched DINOv3 initialization on chest radiographs under a fixed budget, where the run count is the product of the varying factors named in the objective and model columns, and the six seed runs repeat the three objectives at ViT-S under two further seeds. Random ViT-S is a randomly initialized vision transformer of the same architecture as ViT-S/16. CXR, chest radiograph; SSL, self-supervised learning; ViT, vision transformer. ModelPretraining objectiveDomainDimensionRole Publicly released encoders RAD-DINO [34]Self-supervisedCXR768Panel, injections BiomedCLIP [35]Image-textCXR512Panel TorchXRayVision [36]Label-supervisedCXR1,024Panel, injections DINOv3 ViT-L [37]Self-supervisedGeneral1,024Panel DINOv3 ViT-B [37]Self-supervisedGeneral768Panel DINOv3 ViT-S [37]Self-supervisedGeneral384Panel DINOv2 ViT-L [38]Self-supervisedGeneral1,024Panel CLIP ViT-L/14 [39]Image-textGeneral768Panel SigLIP2-L [40]Image-textGeneral1,152Panel Random ViT-S [41]NoneGeneral384Untrained reference MONET [42]Image-textDermatology 768Dermatology only DermLIP ViT-B/16 [43] Image-textDermatology 512Dermatology only RETFound [44]Self-supervisedFundus1,024Fundus only FLAIR [45]Image-textFundus512Fundus only Controlled pretraining on chest radiographs, 22 runs ViT-S/16 and ViT-B/16 SSL, label-supervised, image-text CXR384, 76812 core runs ViT-S/16 and ViT-B/16 Image-text, scrubbed re- ports CXR384, 7682 runs ViT-S/16 and ViT-B/16 Image-text, amplified re- ports CXR384, 7682 runs ViT-S/16SSL, label-supervised, image-text CXR3846 seed runs 7 RaceAgeExceeds its referenceDoes not exceedIdentity or 100% line 0.000.050.100.150.20 Fair-model reference 0.05 0.10 0.15 0.20 Observed difference Median 0.082 vs 0.034 79 of 130 exceed Spearman ρ = 0.486 0.000.050.100.150.20 Fair-model reference 0.05 0.10 0.15 0.20 Observed difference Median 0.066 vs 0.014 116 of 130 exceed Spearman ρ = -0.140 RaceAge 0 20 40 60 80 100 120 140 Reference as a share of the observed difference (%) 41% 22% 10 2 10 3 Positive cases in the smallest evaluable subgroup 0.00 0.02 0.04 0.06 0.08 Fair-model reference Pleural other Support devices 1050510 Encoders exceeding the reference Pleural other Lung lesion Fracture Enlarged cardiomediastinum Consolidation Pneumothorax Pneumonia Edema Atelectasis Pleural effusion Lung opacity Cardiomegaly Support devices 110 22 510 5 910 610 39 310 1010 1010 1010 1010 1010 Race, 79 of 130Age, 116 of 130 0.000.020.040.060.080.100.120.14 Race difference Edema Support devices Pneumonia Pleural other Cardiomegaly Lung opacity Enlarged cardiomediastinum Pneumothorax Pleural effusion Atelectasis Consolidation Lung lesion Fracture Race (5 subgroups) Age (4 subgroups) 10 2 10 3 Positive cases in the smallest evaluable subgroup 136 949 a Race b Age c Share accounted for d Size dependence e Exceedance by finding f Per-finding medians, race g Subgroup sizes Fig. 2: The reported subgroup performance difference against the difference that a perfectly fair model produces. Every panel uses the 130 combinations of 10 frozen encoders and 13 findings on the pooled chest radiograph test split of 125,992 images from 41,621 patients, race and age evaluated from the same scores. Blue denotes race and red denotes age. A filled marker exceeds its reference at an FDR of 0.05 and an open marker does not. Gray dashed lines mark the identity and the 100% line. a,b, The observed difference against its fair-model reference, one point per combination, race and age on the same axes. c, The reference as a share of the observed difference. The box shows the median and the interquartile range, with whiskers at 1.5 times the interquartile range. d, The reference against positive cases in the smallest evaluable subgroup, on a logarithmic axis. e, The number of the 10 encoders whose difference exceeds the reference, per finding. Race extends to the left and age extends to the right. f, The per-finding race medians over the 10 encoders. g, Positive cases in the smallest evaluable subgroup, per finding. Exceedance is the one-sided share of 2,000 simulated fair models matching or exceeding the observed difference, under FDR control within each attribute. Spearman ρ is the rank correlation over those combinations. FDR, false discovery rate. 8 0.000.020.040.060.080.10 Subgroup difference Dermatology, Age Dermatology, Skin type Dermatology, Sex Funduscopy, Age Funduscopy, Race Funduscopy, Ethnicity Funduscopy, Sex 9 of 9 0 of 9 0 of 9 2 of 9 1 of 9 0 of 9 0 of 9 Fair-model reference Observed DermatologyFunduscopy Age Race Ethnicity Skin type Sex 9/92/9 1/9 0/9 0/9 0/90/9 0.020.040.06 Fair-model reference 0.00 0.03 0.06 0.09 0.12 Observed difference Dermatology Funduscopy Exceeds its reference Does not exceed 0369 Combinations exceeding, of 9 Age Skin type Sex Age Race Ethnicity Sex 0.000.020.040.060.080.10 Subgroup difference Age Skin type Sex Age Race Ethnicity Sex Unmitigated Lowest of the nine 10 2 10 3 Positive cases in the smallest evaluable subgroup 0.00 0.01 0.02 0.03 0.04 0.05 0.06 Fair-model reference Age Skin type Sex Age Race Ethnicity Sex a Reference and observed b Difference and exceedance c Every combination against its reference d Exceedance count e Lowest difference under mitigation f Reference against subgroup size Fig. 3: Dermatology and funduscopy against their own fair-model references. Blue is dermatology and red is funduscopy throughout. Every panel uses the nine combinations of encoder and finding evaluated in each modality, on the test splits of 12,636 dermatology images from 9,179 subjects and 3,000 fundus images from 3,000 patients, giving 63 combinations of encoder, finding, and attribute in total. a, The median reference and the median observed difference for each attribute in each modality. An arrow connects the median reference to the median observed difference, and the number of combinations exceeding their reference is printed beside each group. b, The same seven groups of modality and attribute. Marker area increases with the observed difference, and a solid fill marks a group in which at least one combination exceeds its reference. c, Every combination against its own reference, with the dashed identity line. A filled marker exceeds its reference at an FDR of 0.05. d, Combinations exceeding their reference, per group. e, The unmitigated difference and the lowest difference that any of the nine mitigation methods produces at matched disease performance, as medians over the nine combinations. f, The reference against the positive cases in the smallest evaluable subgroup of that group, on a logarithmic axis. Exceedance is the one-sided share of 2,000 simulated fair models matching or exceeding the observed difference, under FDR control within each attribute. FDR, false discovery rate. 9 −0.08 −0.06 −0.04 −0.02 0.00 0.02 0.04 0.06 0.08 Change from the uninjected value Decodability injectionEntanglement injectionNot significant at an FDR of 0.05Span of the observed difference 0.650.700.750.800.850.90 Linear race decodability 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.10 Race difference Untrained ViT-S Span 0.020 Observed difference Achievable difference −0.20.00.2 Spearman ρ with the achievable difference Linear decodability Erasure cost Geometric overlap −0.02−0.010.00 Cross-fitted out-of-sample R 2 Erasure cost, overlap Linear dec. Nonlinear dec. -0.007 -0.017 -0.017 0.000.250.500.751.00 Value Linear decodability Nonlinear decodability Race difference Disease AUROC Paired change, difference / AUROC LEACE -0.012 / 0.030 INLP -0.010 / 0.049 Unmitigated LEACE INLP 0.860.880.900.920.94 Linear race decodability 0.07 0.08 0.09 0.10 0.11 0.12 0.13 Race difference s = 1 k = 256 No injection 00.250.501.00 Injected strength s Linear decodability Nonlinear decodability Race difference Achievable difference Disease AUROC 0.8660.8820.9100.942 0.8570.8840.9210.941 0.0770.0770.0770.077 0.0750.0750.0750.075 0.8370.8370.8380.838 0141664256 Deflated dimensions k Linear decodability Nonlinear decodability Race difference Achievable difference Disease AUROC 0.8660.8800.9290.9490.9480.937 0.8570.9220.9390.9440.9410.932 0.0770.0780.0770.0870.1010.118 0.0750.0760.0760.0820.0940.112 0.8370.8380.8360.8200.8120.802 No injection Decodability Entanglement 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 Race difference 0.077 0.077 0.118 −1.0−0.50.00.51.0 Spearman ρ with strength Decodability Nonlinear dec. Difference Achievable Overlap a Decodability across encoders b Predictors of the difference c Erasure d Race difference against decodability e Decodability injection f Entanglement injection g Difference at full strength h Response to strength Fig. 4: Decodability, entanglement, and the two injections. Every panel reports race on chest ra- diographs. Panels a–c use the frozen panel of 10 encoders and 13 findings. Panels d–h show the injection experiment on the cached features of RAD-DINO over three findings. Mid blue circles show the decodability injection and dark blue diamonds show the entanglement injection. An open marker is not significant at an FDR of 0.05, and dark gray is the overall disease AUROC. a, Per-encoder medians against linear decodability. The band marks the full span of the observed difference. b, Spearman correlation of each property with the achievable difference over the 130 combinations, above the cross-fitted out-of-sample R 2 of the three feature sets, which are the two geometry measures together and each decodability measure alone. c, The two erasure methods against the unmitigated model, as medians over the panel, with the paired change beside them. d, The race difference against linear race decodability across injection strengths, starting from the shared uninjected model. Marker size increases with strength. e,f, Every quantity at every strength. The value is printed and the color encodes the change from the uninjected model, on one scale for both operators. g, The race difference at the largest strength of each grid, as the mean over the three findings. Each finding is drawn on the bar. h, Spearman correlation of each quantity with strength, for both operators. Correlations are permutation tested and corrected within each op- erator under FDR control. AUROC, area under the receiver operating characteristic curve; FDR, false discovery rate. 10 0.069 to 0.088 and the achievable difference spans 0.063 to 0.081. Neither decodability nor the two geometry measures predict the achievable difference out of sample (Fig. 4b). Over the 130 combinations, the Spearman correlation is 0.036 for linear decodability, −0.142 for the cost of erasure in disease AUROC, and−0.027 for the principal-angle overlap, none of them significant (all p FDR ≥ 0.329). Cross-fitted out-of-sample R 2 is negative for the two geometry measures together and for either decodability measure alone. LEACE [13] drives linear race decodability from a median 0.812 to the chance level of 0.502 while nonlinear decodability remains at 0.755 (Fig. 4c). The race difference falls by a median 0.012. Values below are means over the three injected findings. On RAD-DINO, the decodability injection (Eq. 3, Fig. 4d,e) raises linear race decodability from 0.866 to 0.942 (Spearman 0.842, p FDR = 0.014) and nonlinear decodability from 0.857 to 0.941 (0.885, p FDR = 0.014) while the race difference stays at 0.077 (−0.022, p FDR > 0.999). The entanglement injection at 256 dimensions (Fig. 4d,f–h) raises the race difference from 0.077 to 0.118 (0.724, p FDR = 0.014) and the achievable difference with it (0.574, p FDR = 0.040), while the overall disease AUROC falls by 0.035. On cardiomegaly, the difference is unchanged at 0.074 ± 0.009 [0.068, 0.103] under the decodability injection and rises to 0.132 ± 0.012 [0.121, 0.167] under the entanglement injection. Decodability rises under both injections. On the TorchXRayVision DenseNet-121, linear decodability rises from 0.800 to 0.901 (0.842, p FDR = 0.014) while the race difference stays at 0.106 against 0.102 (−0.194, p FDR = 0.862). The invariance of Eq. 4 therefore holds on a second architecture and a second embedding width. The entanglement injection removes a 256-dimensional disease subspace. On this encoder, removing that subspace leaves the overall disease AUROC unchanged on all three findings, at 0.852 against 0.857 on cardiomegaly, 0.673 against 0.670 on fracture, and 0.905 against 0.906 on pleural effu- sion. The race difference stays at 0.106 against 0.103 (−0.066, p FDR = 0.942). Because removing that subspace does not reduce the disease AUROC, this experiment does not test whether entan- glement produces the difference on that encoder. The operator requires the removed subspace to contain disease-predictive signal. In these DenseNet features, predictive signal remains outside that subspace (Supplementary Note 1). In a synthetic model with collinearity as a free parameter, raising the collinearity between the disease and the group direction from 0 to 1 increases the difference from 0.010 to 0.121 while decodability falls (Fig. 5). The remainder under every intervention We applied 9 mitigation methods, spanning resampling and reweighing, loss constraints [11,10], per-group operating point and calibration adjustment [46], and erasure, to the frozen features of every encoder and finding, and recorded the achievable difference of every combination (Fig. 6). A tenth method is reported in Supplementary Note 1. The achievable race difference has a median of 0.071, a reduction of 0.005 (range 0.000–0.111) that is significant in 10 of 128 combinations (Fig. 6a–c). The achievable age difference is 0.057, a reduction of 0.008 (0.000–0.042) significant in 45 of 130. Among the pretraining factors of the controlled matrix, the objective has the largest median effect on the subgroup difference (Fig. 6d). Image-text pretraining lowers the race difference against self-supervised pretraining by a median 0.013, significant in 9 of 52 comparisons, and the age difference by 0.015, significant in 22 of 52. Balancing the pretraining data on race changes the race difference by a median of −0.002 to +0.008 across the three objectives and is significant in 0 of 26 comparisons for every one of them. Neutralizing demographic terms in the training reports is significant in 0 of 26 comparisons for race on every measure. Prepending an explicit demographic sentence to every report, which is the mirror-image control, is significant in 0 of 26 comparisons on 11 0.00.20.40.60.81.0 Collinearity 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 Difference Collinearity vs achievable difference ρ s = 0.732, p FDR = 0.004, n= 30 Removable by mitigation Achievable difference Observed difference 0.000.250.500.751.00 Collinearity 0.00 0.05 0.10 0.15 0.20 Overlap and erasure cost Geometric overlap Erasure cost (AUROC) ρ s = −0.495, p FDR = 0.006, n= 30 0.56 0.57 0.58 0.59 0.60 0.61 0.62 0.63 Linear decodability Linear decodability 0.000.100.250.500.751.00 Collinearity 0 25 50 75 100 Share of the difference (%) 81%83%95%87%99%99% Achievable, so not removable Removable by mitigation By encoder (10 groups) 0.000.050.100.150.20 Achievable race difference By finding (13 groups) Linear decodability Geometric overlap Erasure cost −2 −1 0 1 2 Standardized value Largest achievable Smallest achievable The other 8 encoders Entanglement Linear decodability Nonlinear decodability Entanglement, encoder held out −0.03 −0.02 −0.01 0.00 0.01 0.02 0.03 0.04 Cross-fitted out-of-sample R 2 -0.007 p FDR = 0.318 -0.017 p FDR = 0.338 -0.017 p FDR = 0.338 0.024 p FDR = 0.021 −3−2−10 Out-of-sample R 2 of the transferred predictor -3.40 Dermatology (n= 9, p FDR = 0.787) -1.92 Funduscopy (n= 9, p FDR = 0.787) 0 is the mean of that modality 0.0000.0250.0500.0750.1000.1250.1500.1750.200 Achievable difference Chest radiograph (n= 130, median 0.071) Dermatology (n= 9, median 0.052) Funduscopy (n= 9, median 0.061) a The synthetic model across collinearity b Geometry and decodability c The removable share d Achievable difference e Geometry by encoder f Predicting the achievable difference g The predictor in another modality h Achievable difference by modality Fig. 5: The synthetic collinearity model and the geometry predictor. a–c, The model of Eq. 5 at the six values of the collinearity ρ between the disease and the demographic direction, from orthogonal at 0 to collinear at 1, each level over five seeds and 4,000 samples. a, The observed difference (dark blue circles) and the achievable difference of Eq. 6 (mid blue diamonds). The lower band shows the achievable difference and the upper band shows the further part that every mitigation method could remove. b, The geometric overlap and the AUROC cost of erasure on the left axis and linear demographic decodability on the right axis, each curve labeled at its right end. c, The achievable difference as a share of the observed difference. The value is printed inside the bar. d–f, The 130 combinations of encoder and finding of the frozen chest radiograph panel, evaluated for race. d, The achievable difference of every combination, grouped by encoder in the upper track and by finding in the lower track on one scale. The bar marks each group’s median. e, Each encoder’s median of linear decodability, the geometric overlap, and the erasure cost, standardized across the 10 encoders. The encoders with the largest and the smallest median achievable difference are drawn in color. f, Out-of-sample R 2 of a predictor of the achievable difference by five-fold cross-fitting, one bar per feature set. A filled bar is significant at an FDR of 0.05 against 1,000 target shuffles. Entanglement is the erasure cost and the overlap together. The rightmost bar reports leave-one-encoder-out performance. g, The chest radiograph predictor applied to dermatology (blue) and funduscopy (red). h, The achievable difference per modality for race, skin type, and race respectively, every combination with its median. Correlations in a and b are permutation tested over the 30 level and seed combinations under FDR control. AUROC, area under the receiver operating characteristic curve; FDR, false discovery rate. 12 −0.1 0.0 0.1 Paired change RaceAgeNot significant at an FDR of 0.05Seed-to-seed spread −0.015−0.010−0.0050.000 Change in the difference LEACE INLP Adversarial removal Group DRO Exponentiated gradient Group-balanced resampling Reweighing Operating point shift Platt recalibration Seed-to-seed spread 0.000.040.080.12 Reduction in disease AUROC −0.0025 0.0000 0.0025 0.0050 0.0075 0.0100 0.0125 0.0150 0.0175 Reduction in race difference Five methods below 0.01 disease AUROC Group DRO Adversarial removal LEACE INLP 0.000.020.040.06 Reduction in the difference 0.2 0.4 0.6 0.8 1.0 Share of combinations Race median 0.005 Age median 0.008 −0.04−0.020.000.02 Paired change in the difference Image-text vs self-supervised Image-text vs label supervision Self-supervised vs label supervision Race-balanced pretraining data Demographic terms removed Demographic sentence prepended 9/52 22/52 5/52 22/52 1/52 5/52 0/78 6/78 0/26 0/26 0/26 0/26 Linear head Last block LoRA Full finetune 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 Subgroup difference 0.062 0.049 Race Age 0.000.050.10 Change in the difference Race seeds Race mitigation Age seeds Age mitigation 0.012 0.005 0.006 0.008 Group-balanced resampling Reweighing Group DRO Adversarial removal Exponentiated gradient Operating point shift Platt recalibration LEACE INLP AUROC difference Sensitivity difference FPR difference Calibration difference -0.003-0.002-0.004-0.005-0.003-0.000+0.000-0.012-0.010 -0.059-0.013+0.095+0.019-0.047-0.075+0.091-0.066-0.089 -0.049-0.015+0.106+0.027-0.035-0.069+0.102-0.147-0.167 -0.017-0.070+0.022-0.021-0.071+0.016-0.077-0.000-0.036 −0.15−0.10−0.050.00 Change in the race AUROC difference Preprocessing In-processing Postprocessing Erasure a Median change by method b Difference against disease AUROC c Achievable reduction d Changing how the encoder is pretrained e Unfreezing the backbone f Retraining vs mitigating g Every method on all four measures h By family Fig. 6: Disparity reduction and disease AUROC across every intervention. Blue is race and red is age. Every value is a paired change from the unmitigated model over the 130 combinations of 10 encoders and 13 findings. a, The median paired change for each of the nine methods. The gray band shows the median spread across three pretraining seeds. b, The reduction in the race difference against the reduction in the disease AUROC. Marker area increases with the change in the FPR difference. c, The share of combinations at each reduction, using the largest reduction among the nine at matched disease performance. d, The controlled pretraining contrasts, as the median and interquartile range of the paired change. The number of significant comparisons is printed beside each contrast, and a filled marker indicates a nonzero count. e, The four unfreezing levels, with one faint line per combination. The thick line is the median. f, The change under retraining with two further seeds, against the largest reduction that any of the nine methods produces. g, Every method on all four measures. The median paired change for race is printed and encoded by color. h, The same change grouped into the four method families. Significance is a paired cluster bootstrap over patients at 1,000 resamples under FDR control at 0.05 within each family. AUROC, area under the receiver operating characteristic curve; DRO, distributionally robust optimization; FDR, false discovery rate; FPR, false positive rate; INLP, iterative nullspace projection. 13 the difference. So the difference is unlikely to come from the demographic content of the training reports, at least as those terms can be removed or amplified here. Across four levels of unfreezing, from a linear head to the last block, LoRA adapters, and a full finetune, the median race difference decreases from 0.079 to 0.062 and the median age difference from 0.071 to 0.049 (Fig. 6e). No trend across levels is significant for any attribute or encoder (Spearman −0.007 to −0.233, all p FDR ≥ 0.169). Retraining the same configuration under two further seeds changes the race difference by a median 0.012 (range 0.001–0.093), against a median reduction of 0.005 under the nine mitigation methods. Every intervention we ran therefore changes the difference by less than a change of pretraining seed does (Fig. 6f). None of them optimizes the within-group ranking directly. Operating point and ranking disparities under mitigation We then evaluated every method on all four measures from the identical test scores, with the two rates taken at a threshold fixed on validation at 0.80 sensitivity. For the unmitigated models, the median race differences are 0.177 for sensitivity, 0.245 for FPR, and 0.093 for calibration. Each method reduces the measure that it directly targets (Fig. 6g). The per-group threshold shift and per-group Platt calibration cannot change a within-group ranking while their score maps are strictly increasing (Eq. 4). A per-group operating point shift keeps the race AUROC difference at a median 0.000 while cutting the sensitivity difference by a median 0.075 and the FPR difference by 0.069. The calibration difference increases by 0.016. Per-group Platt calibration also leaves the AUROC difference at 0.000 and reduces the calibration difference by 0.077. The sensitivity and FPR differences increase by 0.091 and 0.102. The AUROC difference is unchanged in 252 of the 260 combinations and changes by less than 1×10 −6 in three more. In the remaining five combinations, the change ranges from 0.130 to 0.440. All five are race comparisons on the three rarest findings, where a map fitted on 20 to 34 positive cases is no longer strictly increasing (Supplementary Note 1). The two erasure methods produce the largest reductions in the race AUROC difference (Fig. 6h). LEACE lowers that difference by a median 0.012 and lowers the disease AUROC by 0.030. Iterative nullspace projection lowers the difference by 0.010 and lowers the disease AUROC by 0.049. Group distributionally robust optimization lowers the difference by 0.004 and lowers the disease AUROC by 0.091. Adversarial removal lowers the difference by 0.005 and the disease AUROC by 0.067. Both also increase the sensitivity and FPR differences (Supplementary Table 5). A threshold shift or a calibration map can change a difference at the operating point but not a within-group ranking. None of the nine methods optimizes the subgroup AUROC difference directly. Acquisition view and site as competing explanations A published analysis attributes chest radiograph performance differences to technical acquisition factors and not to demographics [47]. We evaluated the same test scores in four analyses: the pooled demographic difference, the demographic difference within each view, view as an attribute in its own right, and site as an attribute in its own right. Each analysis is compared against its own reference computed from the rows that were evaluated. Holding the view fixed does not remove the race difference (Fig. 7). The pooled value of 0.081 becomes 0.066 in anteroposterior and 0.090 in posteroanterior radiographs, and neither differs from it (Wilcoxon signed-rank [48], both p FDR ≥ 0.271 over 26 combinations). Part of the age difference is view composition. The pooled age difference of 0.074 falls to 0.044 within anteroposterior radiographs (p FDR = 8.8×10 −5 ), while within posteroanterior radiographs it is unchanged at 0.074 (p FDR = 0.437). Every demographic difference still exceeds its own reference inside every view (all p FDR ≤ 9.5× 10 −5 ). 14 Acquisition viewData sourceRaceAgeFair-model reference 0.000.050.10 Subgroup performance difference Race, Pooled Race, AP only Race, PA only Age, Pooled Age, AP only Age, PA only 0.081 0.066 0.090 0.074 0.044 0.074 0.050.100.15 Pooled difference −0.05 0.00 0.05 0.10 0.15 0.20 Change inside one view Race AP: p FDR = 0.784 Race PA: p FDR = 0.272 Age AP: p FDR = 8.8 × 10 −5 Age PA: p FDR = 0.437 AP PA −0.08 −0.06 −0.04 −0.02 0.00 0.02 0.04 0.06 0.08 Change from pooled Race AP only Race PA only Age AP only Age PA only Pleural effusion Lung opacity Atelectasis Consolidation Pneumothorax Cardiomegaly Edema Pneumonia Fracture Lung lesion Enlarged cardiomediastinum Pleural other Support devices 0.0000.0250.0500.0750.1000.1250.1500.175 Subgroup performance difference Age Race Data source Acquisition view 0.074 (p FDR = 8.9 × 10 −8 ) 0.081 (p FDR = 8.9 × 10 −8 ) 0.095 (p FDR = 6.0 × 10 −8 ) 0.100 (p FDR = 3.0 × 10 −8 ) 0.000.050.100.150.200.250.30 Subgroup performance difference 0.25 0.50 0.75 1.00 Share of the 26 combinations 0.000.050.100.150.200.250.300.35 Subgroup performance difference AUROC Sensitivity FPR Calibration error PooledAP onlyPA only 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 Fair-model reference 0.031 0.038 0.058 Race 0.013 0.016 0.027 Age a Holding the acquisition view fixed b The paired change c The change per finding d View and site as attributes e The full distribution per attribute f Attribute order on each measure g The reference inside a view Fig. 7: Subgroup performance differences with the acquisition view held fixed, and with acquisition view and site treated as attributes. Every panel uses the 26 combinations of 2 encoders and 13 findings on the pooled chest radiograph test split of 125,992 images from 41,621 patients, with anteroposterior and posteroanterior as the two views. Blue marks the acquisition view, pale blue marks the data source, dark blue marks race, and red marks age. The change in b and c is the within-view difference minus the pooled difference of the same combination. a, The median race and age difference pooled and inside each view, each with its own fair-model reference. b, That change against the pooled difference, one point per combination. Circles mark AP and triangles mark PA, with the p value for each pairing. c, The same change per finding, as the median over the 2 encoders. d, Each attribute from its own reference to the median observed difference, with the p value of the paired test against that reference. e, The cumulative distribution of the difference over the 26 combinations, per attribute. f, The median difference on the four measures, AUROC, sensitivity, FPR, and the expected calibration error. The horizontal bar connects the smallest and the largest attribute-specific difference on each measure. g, The fair-model reference in each condition. Every p value comes from a two-sided Wilcoxon signed-rank test over the paired combinations at 0.05, corrected within its family under FDR control. AP, anteroposterior; AUROC, area under the receiver operating characteristic curve; FDR, false discovery rate; FPR, false positive rate; PA, posteroanterior. 15 Treated as attributes in their own right, acquisition view and data source give median differences of 0.100 and 0.095, against references of 0.023 and 0.025. The race difference is 0.081 against a reference of 0.031, and the age difference is 0.074 against a reference of 0.013. All four exceed their references (all p FDR ≤ 9.0× 10 −8 ). On this pool, the median difference is larger across acquisition view and data source than across patient race. Worst-group performance across pretraining objectives and unfreezing levels We evaluated the worst-group AUROC and the equity-scaled AUROC [49], the overall value dis- counted by the spread of the subgroup values around it, on the controlled encoders and at each of the four unfreezing levels (Fig. 8 and Supplementary Table 6). We computed the subgroup performance difference from the same scores. Image-text pretraining raises both by about 0.05 over self-supervised pretraining at fixed data, architecture, and budget (Fig. 8a,b,h). Across the 52 comparisons on natural data, the median gain in worst-group AUROC is 0.049 for race, sig- nificant in 35 of them, 0.054 for age, significant in 50, and 0.044 for sex, significant in 49. The equity-scaled AUROC gains 0.049, 0.047, and 0.042 on the same comparisons, significant in 32, 44, and 49 (Fig. 8c). In both contrasts, the median change in the subgroup difference is at most 0.015. On cardiomegaly at the ViT-B backbone (Fig. 8g), the equity-scaled value is 0.751 ± 0.013 [0.725, 0.775] under image-text pretraining and 0.711 ± 0.013 [0.683, 0.733] under self-supervision, while the race difference is 0.069 ± 0.006 [0.061, 0.085] and 0.082 ± 0.009 [0.069, 0.103]. From a linear head to a full finetune, the median overall AUROC rises from 0.785 to 0.830 and the equity-scaled value from 0.680 to 0.732 for race and from 0.708 to 0.784 for age, against the 0.017 of race difference that the same four levels remove (Fig. 8d–f). Image-text pretraining and deeper finetuning each raise the worst-group AUROC by about 0.05. The median change in the subgroup difference is at most 0.022. FRAME applied to published subgroup differences Everything above is measured on our own experiments. We therefore applied step one of FRAME to subgroup differences already published, using each study’s own reported values and none of our models. Nine studies report everything the reference needs, giving 89 claims over ten datasets, ten tasks, six imaging modalities, and the race, sex, race-by-sex, age, and skin-tone attributes (Fig. 9 and Supplementary Table 7). Of the 89 claims, 41 exceed their reference. The 53 rate differences, meaning a sensitivity or an FPR difference at a fixed threshold, exceed in 40 cases at a median reference share of 25%. The 36 differences in AUROC exceed in 1 case at a median share of 70%. In 22 of the 89, the reference is larger than the difference that the study reported. Within each modality that reports both kinds of statistic, rate differences exceed the reference more often than AUROC differences do. In chest radiography, the most cited claim [1], the under- diagnosis difference across race and sex on MIMIC-CXR [25], is 0.122 against a reference of 0.017 ± 0.008 [0.005, 0.035] at a smallest subgroup of 1,152 cases. On CheXpert [26], the benchmark that standardized this statistic [6] reports a race difference of 0.002 and a sex difference of 0.000, both against a reference of 0.008. In dermatology, across four labelers on the same 656 clinical pho- tographs [50], 11 of 16 rate differences exceed at a median 25% share while none of the 8 AUROC differences does. In the smallest cohorts, the reference accounts for most of the reported difference, though a sufficiently large difference still exceeds it. Across 6 race-stratified recall differences in computational pathology [51], the reference accounts for a median 68% and 3 of the 6 exceed, at smallest subgroups of 9 to 100 slides. In two of the three that do not, the reference is larger than the difference reported. In a referable diabetic retinopathy system evaluated by skin tone on 16 RaceAgeSexGain from the level beforeOverall AUROC at that level 0.000.050.100.150.20 Gain in worst-group AUROC, image-text over self-supervised Race Age Sex 0.049 0.054 0.044 RaceAgeSex 0.00 0.02 0.04 0.06 0.08 0.10 Median gain Over self-supervised Over label supervision 013263952 Comparisons of 52 Race, Worst-group Age, Worst-group Sex, Worst-group Race, Equity-scaled Age, Equity-scaled Sex, Equity-scaled 35 50 49 32 44 49 0.60.70.80.9 Equity-scaled AUROC Linear head Last block LoRA Full finetune Linear head Last block LoRAFull finetune 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 Subgroup difference Linear head Last block LoRAFull finetune Total 0.74 0.76 0.78 0.80 0.82 0.84 Overall disease AUROC 0.785 +0.031 +0.008 +0.0050.830 0.660.680.700.720.740.760.780.80 Equity-scaled AUROC Self-supervised Image-text 0.711 ± 0.013 0.751 ± 0.013 0.050.060.070.080.090.100.11 Race difference Self-supervised Image-text 0.082 ± 0.009 0.069 ± 0.006 0.60.70.80.9 Self-supervised 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 Image-text Worst-group AUROC a Image-text vs self-supervised b Against either baseline c Significant comparisons d Equity-scaled AUROC by level e The difference across levels f AUROC change per level g One worked comparison, cardiomegaly at ViT-B h Every comparison Fig. 8: Worst-group performance and the subgroup performance difference across pretraining ob- jectives and unfreezing levels. Blue is race, red is age, and dark gray is sex. Panels a–c and h use the 52 paired comparisons of the controlled matrix on natural pretraining data. Panels d–f use the four unfreezing levels on both finetuned encoders. a, The distribution of the paired gain in worst-group AUROC from image-text over self-supervised pretraining, with every comparison and the median drawn. b, The median gain against each baseline objective, with the interquartile range. The paler bar marks the contrast against label supervision. c, The number of significant comparisons among the 52, on the worst-group AUROC and the equity-scaled AUROC. d, The equity-scaled AUROC at each unfreezing level, as a density per attribute with its median marked. e, The subgroup difference on the identical runs, every combination drawn with its median. f, The change in overall disease AUROC between successive unfreezing levels. g, One worked comparison. The point marks the bootstrap mean, the thick band shows one standard deviation, and the thin line shows the 95% confidence interval. h, Every comparison of image-text against self-supervised pretraining on the same axes, with the identity line. Significance is a paired cluster bootstrap over patients at 1,000 resamples under FDR control at 0.05 within each family. AUROC, area under the receiver operating characteristic curve; FDR, false discovery rate. 17 Chest radiograph Dermatology Pathology Funduscopy Mammography Abdominal CT Does not exceed its referenceFair-model reference 0.050.100.15 Fair-model reference 0.0 0.1 0.2 0.3 0.4 0.5 Difference the study reported 0255075100125150 Reference as a share of the reported value (%) Chest radiograph 20 of 21 exceed Dermatology 11 of 16 exceed Pathology 3 of 6 exceed Funduscopy 5 of 6 exceed Mammography 1 of 4 exceed Chest radiograph 1 of 8 exceed Dermatology 0 of 8 exceed Abdominal CT 0 of 20 exceed Rate differences AUROC differences 0102030 Claims Seyyed-Kalantari 2021 MEDFAIR Yala 2021 Vaidya 2024 Burlina 2021 Glocker 2023 Lotter 2024 Tayebi Arasteh 2024 Daneshjou 2022 1 of 1 1 of 4 1 of 4 3 of 6 5 of 6 7 of 12 12 of 12 0 of 20 11 of 24 10 1 10 2 10 3 10 4 Cases in the smallest subgroup 10 −2 10 −1 Fair-model reference 10 2 10 3 Median smallest subgroup (cases) 0 20 40 60 80 100 Median reference share (%) Pathology 68% at 40 cases Dermatology 25% at 94 cases Funduscopy 26% at 100 cases Chest radiograph 18% at 1152 cases Mammography 47% at 1862 cases 0.00.10.20.30.40.5 Difference PDAC, AUROC Tayebi, n= 75 Emory, TPR Yala, n= 301 CheXpert, TPR Glocker, n= 897 MGB lung, TPR Vaidya, n= 100 MIMIC-CXR, FPR Seyyed-Kalantari, n= 1152 MGB breast, TPR Vaidya, n= 55 TCGA glioma, TPR Vaidya, n= 15 DDI common, TPR Daneshjou, n= 16 DDI, TPR Daneshjou, n= 48 Burlina DR, TPR Burlina, n= 100 020406080 Claims Printed in the article Derived by arithmetic Reconstructed from composition 73 12 4 a Claims against their references b The reference as a share c By study d Reference against subgroup size e Across the modalities f The largest claim in each cohort g The provenance of the counts Fig. 9: Published subgroup performance differences compared with the fair-model reference. Every panel uses the 89 audited claims from nine studies over ten cohorts and six imaging modalities, with one color and marker per modality. A filled marker exceeds its reference and an open marker does not. a, Each claim’s reported difference against the reference derived at that claim’s own subgroup counts, with the identity line. b, The reference as a share of the reported value, per statistic and modality. Every claim is shown with the group median and the number exceeding. Two dermatology claims report a difference of 0.000, so their share is undefined. Neither the violin nor the points include them. c, The number of claims exceeding their reference in each study, of the claims included from that study. d, The reference against the number of cases in the smallest subgroup that the difference ranges over, on logarithmic axes. e, The median reference share of a rate difference against the median smallest subgroup, per modality; the audit includes AUROC claims only for abdominal computed tomography, which has no point here. f, The largest reported difference in each cohort. The point marks the reported difference, the diamond marks the reference, and the line shows its 95% interval. g, The provenance of each claim’s per-subgroup counts. Exceedance is the one-sided share of 2,000 simulated fair models matching or exceeding the reported difference, corrected across the claims sharing a metric under FDR control at 0.05. Per-claim values are in Supplementary Table 7. AUROC, area under the receiver operating characteristic curve; FDR, false discovery rate; FPR, false positive rate; TPR, true positive rate. 18 100 referable and 100 healthy images per group [52], 5 of 6 rate differences exceed. The untreated system’s sensitivity difference is 0.500 against a reference of 0.055 ± 0.042 [0.000, 0.160]. The meaning of non-exceedance depends on what the study claimed. Across 20 AUROC differ- ences in abdominal computed tomography, evaluated by sex and by age across 10 privacy budgets from one model family [53], not one exceeds its reference. Those differences range from 0.010 to 0.056. The corresponding references range from 0.004 to 0.086, and in 9 of the 20 the reference is larger than the difference reported. Those subgroups contain 75 to 127 evaluable cases. That study [53] concluded that private training did not amplify discrimination against age or sex. The reference supports that conclusion and quantifies the sampling component that the original anal- ysis did not report. Where a study instead reports a difference as its finding, non-exceedance is weaker evidence for a disparity. The audit therefore records the direction of each study’s own claim alongside the exceedance level. The share that the reference accounts for is not ordered by the subgroup size alone. Among the rate differences, it is a median 68% in pathology at a median smallest subgroup of 41 cases, 26% in funduscopy at 100, 25% in dermatology at 94, and 18% in chest radiography at 1,152. Screening mammography has the largest denominators in the set and a reference share of 47% [54]. Its reported differences are also the smallest, at a median of 0.020 against 0.077 in chest radiography. In the two modalities that report both kinds of statistic, the published studies show the same distinction as our own panel. Rate differences exceed the reference and AUROC differences generally do not. The reference needs only the counts that an evaluation already has, so any of these claims can be compared against it without new data and without retraining. The second step needs the feature representation, which a reported number does not provide. We therefore applied the second step to two released encoders whose features we could compute. Discussion Medical imaging fairness studies commonly treat the difference between the best-performing and the worst-performing subgroup as a disparity, and then apply an intervention to the model. We built FRAME to estimate the subgroup difference expected under exact fairness at the observed subgroup sizes and to measure what remains once that reference is subtracted. Its second step tests which manipulation of the representation changes the remainder. On a pooled test set of 125,992 chest radiographs, the fair-model reference accounts for a median 41% of the reported race difference and 22% of the age difference. For race, 51 of the 130 combinations of encoder and finding do not exceed it. The remaining 79 race combinations and 116 of the 130 age combinations do exceed it. Injecting demographic decodability does not change the remainder. No mitigation method changes it by more than a change of random seed. The entanglement injection raises the difference from 0.077 to 0.118. Image-text pretraining, by contrast, raises worst-group performance by about 0.05 and changes the subgroup difference by at most a median 0.015. The inflation of the reported statistic at small subgroup sizes is not itself new. FRAME quan- tifies the sampling contribution separately for each combination of encoder and finding. Dataset bias and unmeasured confounders were already named as complications for any reported differ- ence [14,15]. Petersen and colleagues reached the same conclusion from first principles, that unequal performance can arise from unequal task difficulty even where the model is not the source [16]. Work outside medicine showed the maximum-minus-minimum statistic to be biased away from zero at small samples [19,20]. Because the reference is derived at each unit’s own subgroup counts, it estimates the part of that unit’s difference expected from sampling variation. A recent study of privacy risk in medical artificial intelligence observed that its subgroup effects correlate with group size and concluded that they may be driven in part by that [18]. The same reasoning applies to fairness evaluation. The reference quantifies the part of a reported difference attributable to 19 subgroup size. In a domain transfer study across five institutions and more than 590,000 chest radiographs, subgroup differences remained under one percentage point almost everywhere [55], consistent with the smaller fair-model reference expected at larger subgroup sizes. We then applied the reference to differences already in the published record. Across 89 such differences spanning six imaging modalities, each on its own cohort, model, and label policy, 41 exceed what exact fairness produces at the sizes that they were measured on. Differences in a thresholded rate, the measure that the underdiagnosis literature is built on, exceed their reference in 40 of 53 cases and keep a median 75% of their reported size, though that share falls to 32% in pathology, where the subgroups are smallest. Differences in AUROC exceed in 1 of 36, and the reference is a median 70% of the reported value. In 22 of the 89 claims, it is larger than the difference that the study reported. A claim that does not exceed its reference is not thereby refuted. The interpretation of non-exceedance depends on the direction of the study’s own claim. Twenty of the 48 that do not exceed come from a study whose conclusion was that private training did not amplify discrimination. For that study, the reference supports the conclusion instead of weakening it. The published studies therefore show the same distinction between rate and AUROC differences that our injections produce. The measured associations do not support demographic decodability as the cause of the re- maining subgroup difference. Models can recover self-reported race from images with no cue that a radiologist can use [7]. The features of disease classifiers also encode protected characteristics [8]. We reproduce both findings, since race is decodable at up to 0.872 across our panel. High de- mographic decodability alone does not establish that demographic encoding causes the subgroup difference. Decodability varies across encoders by more than 0.20 while the achievable difference varies by less than 0.02. An untrained transformer that has learned nothing already decodes race at 0.659. Neither decodability nor the two geometry measures we tested predict the achievable difference out of sample. Adding decodability along a direction with no disease signal leaves the difference unchanged at every strength, while removing disease signal from the minority rows raises it. Demographic decodability is therefore distinct from the representation change that alters the within-group ranking. The studies cited above measured decodability and not that change. Glocker and colleagues had already argued that statistical differences between subgroups must be accounted for before a subgroup difference is interpreted as a disparity [8]. Our results are consistent with that argument. Jones and colleagues make the compatible point that current mitigation addresses a narrow subset of the mechanisms at work [17]. Across nine mitigation methods, the controlled pretraining matrix, and four levels of unfreezing, the subgroup AUROC difference changes by less than a change of pretraining seed produces. A per- group threshold and a per-group calibration map cannot alter a within-group rank statistic while the map they apply is strictly increasing. They reduce the sensitivity and calibration differences while the median change in the AUROC difference remains 0.000. The calibration map leaves the AUROC difference unchanged in 252 of 260 combinations. In the five combinations whose smallest race subgroup has 20 to 34 positive cases, the fitted map is no longer strictly increasing. LEACE produces the largest reduction in the race AUROC difference, at a median 0.012, and lowers the disease AUROC by 0.030. The threshold and calibration methods reduce the operating point measures that they optimize. None of the nine optimizes the subgroup AUROC difference directly, and those nine span the families that the field uses. A failure to reduce that difference therefore does not establish that the difference is intractable. Where the reported difference is a difference at the operating point, the postprocessing methods do reduce it and remain the right tool. Image-text pretraining and deeper finetuning each raised the worst-group AUROC by about 0.05. No fairness intervention we tested matched that gain. Studies of privacy-preserving training report that stricter privacy widens subgroup differences, and that the widening is amplified in 20 smaller and more heterogeneous datasets [56]. Part of that widening may reflect the larger fair- model reference expected at smaller subgroup sizes. An increase measured against zero is larger than the same increase measured against the fair-model reference. The same caution applies to our own earlier work, where private training reduced performance most on the harder diagnoses and the harder subgroups [53], and to a speech study in which gender differences under privacy were minimal at reasonable budgets [57]. We did not test whether privacy changes fairness after correcting for sampling variation. Such interactions should be estimated against a reference instead of against zero. The same argument has been made for the privacy-utility trade-off itself, where an assumed cost turned out to be far smaller at a realistic operating point [58]. This study has several limitations. First, the mechanism behind the remainder is not identified. Injecting alignment between the group and the disease direction raises the subgroup difference, but our interventions are applied to cached features and not during pretraining. So we cannot say what produces the entanglement in a trained encoder. The entanglement injection also removed disease signal on RAD-DINO and not on the DenseNet-121, where removing a 256-dimensional disease subspace from 1,024 dimensions left the overall disease AUROC unchanged. Second, the pooled race attribute assigns every record from the four sites that record no race field to an Other category, which contains 52,038 of its 68,521 test radiographs. The pooled race contrast is therefore partly a contrast between sites. We analyze site as an attribute in its own right for that reason. A cohort in which every site records race would separate the race effect from site. Third, the evidence for a remainder above the reference is not uniform across the study. For race, 129 of 243 combinations exceed their reference, against 226 of 252 for age. Race is evaluated across more and smaller subgroups, which raises its fair-model reference. Outside dermatology age, no attribute in dermatology or funduscopy exceeds its reference in more than 2 of 9 combinations at these cohort sizes. The causal claims depend on chest radiographs for that reason. Fourth, the reference models sampling at the image level. It does not model within-patient correlation, which makes the correction conservative and the remaining share an upper bound. Fifth, the sensitive labels are coarse self-reported categories and the disease labels at the two largest sites are derived from reports by an automated labeler, so both are subject to measurement error that we did not model. Sixth, the audit of published claims depends on what each article prints, so a study reporting a difference without the counts behind it cannot be checked. For one of the nine, MEDFAIR [6], the per-subgroup counts behind its four claims had to be reconstructed from the composition and split fraction that it states. Much of what is currently reported as demographic unfairness in medical imaging is a property of the statistic and the subgroup sizes that it is computed on. The difference that remains does not change when the demographic information that a model encodes is injected. Applying the fair- model reference distinguishes the findings that exceed the variation expected under exact fairness from the findings that do not. Its first step can be run from counts that a study already has, without new data and without retraining. On the differences that remain, the interventions we tested reduce a difference at the operating point and leave the within-group ranking difference at a median of 0.000, while image-text pretraining raises the worst-group AUROC. Larger cohorts in other imaging modalities could test whether the operating point and the ranking separate the same way beyond chest radiographs, and whether entanglement between the group and the disease direction arises during pretraining. Methods Ethics statement This study was conducted in accordance with the relevant guidelines and regulations. It reuses pre- viously collected, de-identified imaging data released for research by the institutions that maintain 21 them, publicly or under credentialed access. It generated no new patient data and recruited no participants. Institutional review board approval and individual informed consent were therefore not required for this study. Every source was used in accordance with its license and data use agreement. No re-identification was attempted, and no source data are redistributed apart from the single openly released chest radiograph reproduced in Fig. 1. All models are open-weight and were run locally, so no image, report, or record was sent to a third-party service. Datasets All experiments operate on a curated three-modality imaging corpus of n = 702,206 images as- sembled for this work: chest radiography (n = 650,207), dermatology (n = 41,999), and retinal funduscopy (n = 10,000). Counts are reported at the final curated state of the manifests, and per-source composition is given in Table 1 and Supplementary Table 3. Chest radiography. The pool contains n = 650,207 frontal radiographs from six sites: MIMIC-CXR (n = 243,345 images from 63,945 patients) [25], CheXpert (n = 157,865 from 57,872) [26], NIH ChestX-ray14 (n = 112,120 from 30,805) [27], PadChest (n = 110,525 from 67,205) [28], VinDr- CXR (n = 18,000) [29], and the pediatric VinDr-PCXR (n = 8,352) [30]. The two VinDr sources provide no patient identifier. Lateral views were excluded and every image file was confirmed present before a record entered the pool. Each site’s released partition was preserved, giving n = 475,271 training, n = 48,944 validation, and n = 125,992 test radiographs over 41,621 test patients, with per-site partitions in Table 1. Recorded sex is male for n = 341,000, female for n = 278,816, and other or absent for n = 30,391; view is anteroposterior for n = 344,253, posteroanterior for n = 279,602, and unrecorded for n = 26,352; age is available for n = 629,227 radiographs (97%) at a median of 60.0 years (interquartile range 45.0–72.2). Labels are the 13 canonical findings, which are the 14 CheXpert categories with No Finding removed. MIMIC-CXR and CheXpert encode each finding as positive, negative, uncertain, or blank, resolved throughout as positive for the positive code, negative for the negative and uncertain codes, and excluded for blank. The other four sites release binary labels. Exclusion is per finding, so evaluable counts differ across findings. Prevalence over the pool ranges from 1% for pleural other to 41% for support devices, with the per-finding evaluable and positive counts in Supplementary Table 8. The image-text objective uses n = 401,077 image-report pairs drawn from the pool’s own radiographs, n = 243,345 from MIMIC-CXR and n = 157,732 from CheXpert Plus, partitioned into n = 295,655 training, n = 31,493 validation, and n = 73,929 test pairs. The report text was taken from the findings and impression sections. Dermatology. The pool contains n = 41,999 images from three sources: ISIC 2019 (n = 25,331) [32,59], Fitzpatrick17k (n = 16,012) [31], and the Diverse Dermatology Images set (n = 656) [3]. Its 30,599 subjects are partitioned at fractions of 0.6, 0.1, and 0.3 (Table 1), disjointly over cases for the Diverse Dermatology Images set and Fitzpatrick17k and over lesions for ISIC 2019. The task is malignant against benign, with n = 11,671 malignant and n = 18,636 benign images and n = 11,692 unlabeled, taken from the released malignancy annotation for the Diverse Dermatology Images set, the three-partition label for Fitzpatrick17k, and the diagnostic classes for ISIC 2019, where melanoma, basal cell carcinoma, actinic keratosis, and squamous cell carcinoma are malignant and nevus, benign keratosis, dermatofibroma, and vascular lesion are benign. Fitz- patrick skin type is recorded for n = 16,668 images, n = 7,963 as types I and I, n = 6,330 as I and IV, and n = 2,375 as V and VI. 22 Retinal funduscopy. The pool is the scanning laser ophthalmoscopy images of Harvard-FairVision (n = 10,000 images from 10,000 patients) [33], partitioned by the release’s own folders (Table 1), with glaucoma as the task at n = 5,006 positive images. Race is White for n = 7,725, Black for n = 1,448, and Asian for n = 827; ethnicity is non-Hispanic for n = 9,622 and Hispanic for n = 378; sex is female for n = 5,824 and male for n = 4,176. Sensitive attributes. Attributes were harmonized across sites before analysis. Age is banded at under 40, 40 to under 60, 60 to under 80, and 80 or over. Race is mapped by case-insensitive substring matching to White, Black, Asian, Hispanic or Latino, and Other, and insurance to Medicare, Medicaid, Private, and Other, with any unmatched string assigned to Other. Over the chest radiograph pool, the counts are n = 234,624 White, n = 43,419 Black, n = 23,794 Asian, n = 11,687 Hispanic or Latino, and n = 336,683 Other for race, and n = 194,075 Medicare, n = 90,142 Private, n = 51,734 Medicaid, and n = 314,256 Other for insurance. Fitzpatrick skin type is grouped from six levels into three, and the fundus source records race as White, Black, and Asian with a separate ethnicity field. Race and insurance are recorded at two of the six chest radiograph sites, CheXpert directly and MIMIC-CXR through the MIMIC-IV version 3.1 demographics linkage [60]. A record from a site without the field is assigned to Other, which Supplementary Note 2 quantifies. A subgroup with fewer than 20 evaluable images is dropped from a comparison, and an attribute needs at least two evaluable subgroups. We prespecified both bounds before analyzing any result. We analyze the acquisition view and the site of origin both as variables to hold fixed and as attributes in their own right, with view recorded as unknown dropped. FRAME FRAME audits one reported subgroup difference in two steps. Neither step is specific to a measure or to an imaging modality. Step one needs the reported difference, the per-subgroup counts that it was computed on, and the common performance value assigned to every subgroup under exact fairness. Step two additionally needs the feature representation that produced the scores. Let G = 1,...,J index the evaluable subgroups of one attribute on one evaluation unit, a combination of encoder, finding, and attribute, and let m j be a measure evaluated on subgroup j. The reported quantity throughout is the range over subgroups, ∆ m = max j∈G m j − min j∈G m j .(1) Eq. 1 is an order statistic over subgroups, so its expectation is positive under exact fairness and grows as subgroups shrink [19,20]. For every evaluation unit, we therefore construct the distribution that the statistic takes when no disparity is present. Let ˆ A be the unit’s observed overall AUROC, (n + j ,n − j ) the positive and negative counts of subgroup j on the test split, and F(· ; A,n + ,n − ) the sampling distribution of an empirical AUROC at true value A and those counts, taken in closed form from Hanley and McNeil [61] and truncated to [0, 1]. The construction needs only a sampling distribution for the measure at a given true value and count, so it extends to any such measure. For a rate, every subgroup shares one true rate under exact fairness and its observed value is a binomial proportion at that subgroup’s own denominator, which is exact and needs no closed-form approximation. Assigning every subgroup the same true value ˆ A makes sampling variation the only source of spread, and the reference distribution of the difference is the law of G = max j∈G A j − min j∈G A j , A j ind ∼ F · ; ˆ A, n + j , n − j .(2) 23 The reference r = E[G] and its interval are estimated from B = 2,000 draws at seed 0, with the 2.5th and 97.5th percentiles as the interval. The remainder is the excess of the observed difference over the reference, ˆ ∆−r, with the one-sided exceedance level 1 +|b : G b ≥ ˆ ∆| /(B + 1). Supplementary Algorithm 1 states the procedure and Supplementary Note 4 the sampling distribution it draws from. Subgroups below 20 evaluable images are dropped and an attribute needs at least two, matching the panel. The closed-form draw was checked against an exact simulation on randomly chosen units in every run (Supplementary Note 4). References were built for the frozen panel, the two smaller modalities, and the four unfreezing levels. The frozen chest radiograph panel of ten encoders and 13 findings has 130 evaluation units per attribute. Each of the two smaller modalities has nine per attribute over a single task. Two operators inject a candidate cause into the cached embedding matrix X ∈ R N×D of one encoder, between feature extraction and head fitting. Let a∈−1, 0, +1 N equal +1 on the largest subgroup, −1 on every other named subgroup, and 0 where the attribute is missing; let σ be the mean per-dimension standard deviation of X; let w be the unit disease direction of a linear head fitted on the training rows; and let U k ∈ R D×k have orthonormal columns spanning a k-dimensional disease subspace obtained by fitting a disease direction, deflating the training features along it, and refitting k times. With u a random unit vector satisfying u ⊤ w = 0, the two operators are T dec s (X) = X + sσ au ⊤ , T ent k (X) = X − D a X U k U ⊤ k , D a = diag 1[a < 0] , (3) at strengths s ∈ 0, 0.25, 0.5, 1.0 and k ∈ 0, 1, 4, 16, 64, 256, with T 0 = id in both grids. Both grids were fixed from a preliminary calibration of the strengths on one encoder. The strength of Eq. 3 is applied to the cached matrix in row blocks of 50,000, which makes the result bit-identical across runs. Because a subgroup AUROC depends on the scores of that subgroup only through their ranks, and T dec s adds the constant vector sσu to every row of a group, a linear head θ evaluated after the decodability injection satisfies A j θ; T dec s (X) = A j θ; X for every j and every s,(4) since within group j every score is displaced by the same sσa j θ ⊤ u. Both operators act before the head is fitted, so that head is refitted on the injected features at every strength. Eq. 4 holds for any fixed linear head, so only a refit could reorder scores within a group under T dec s . No such invariance holds for T ent k , which projects out disease signal for the minority rows alone and so reorders scores within that group. The injections were run on two released chest radiograph encoders, RAD-DINO at 768 dimensions and the TorchXRayVision DenseNet-121 at 1,024, for race over pleural effusion, cardiomegaly, and fracture, with every measure and every geometry measure recomputed at each strength. T ent k removes a fixed number of directions, so it changes disease information only where the removed k-dimensional subspace contains disease-predictive signal. We verify that condition with the overall disease AUROC at the largest strength, which falls where disease signal was removed and does not fall where predictive signal remains outside the removed subspace. We check it before interpreting the correlation with strength. The two steps run in order. Step one derives the reference at the observed counts. Where the observed difference does not exceed it, we classify the difference as consistent with exact fairness and do not run step two. Otherwise we apply both operators to the remainder ˆ ∆− r over their strength grids and compare their rank correlations with strength. Supplementary Algorithm 1 states the whole procedure and Supplementary Algorithm 2 the operator internals. Experimental design We apply FRAME to beliefs (i) and (iv) under three controls used in every experiment. Every difference is compared against its own reference at that unit’s subgroup counts, so no claim de- 24 pends on a comparison against zero. Every candidate cause is tested by intervention as well as by observation, because a cross-model association alone cannot show whether changing the candidate changes the difference. And every intervention is compared at matched disease performance, so a method that lowers a difference by lowering performance is not counted as a success. We run the full design on chest radiography, the only modality with cohorts large enough to leave a testable remainder across a panel of encoders. We evaluate dermatology and funduscopy on the identical protocol as a scope check. A final experiment applies step one to 89 subgroup differences in nine published studies across six imaging modalities, on their cohorts, their models, and their label policies. Representation geometry and the synthetic model Four measures describe how much a frozen representation encodes the sensitive attribute. Each is fitted on the training split and evaluated on the test split. Linear decodability is the macro one-vs-rest AUROC of a logistic regression at C = 1.0 predicting the attribute from the features, with chance at 0.5; nonlinear decodability is the same quantity from a one-hidden-layer network of 256 units with an L 2 penalty of 1×10 −3 , at most 300 iterations, and early stopping; the geometric overlap is the mean cosine of the principal angles between the fitted sensitive and disease subspaces in the standardized feature space; and the erasure cost is the disease AUROC lost when the sensitive direction is removed by LEACE. A synthetic model varies the collinearity between the group and disease directions while holding both group effects fixed. Each sample draws y ∈0, 1 and a∈0, 1 independently and uniformly and forms x∈ R D with D = 160, with signal present in d = 32 dimensions, x = ε + α(2y− 1)w d | z disease + β(2a− 1)w a |z rank-neutral shift + 1[a = 1]γ η w a |z group-specific variance ,(5) with ε ∼ N(0,I D ), η ∼ N(0, 1), unit vectors w d ,w a in the signal subspace at prescribed inner product w ⊤ d w a = ρ, and α = 1.2, β = 0.5, γ = 2.0. The second term satisfies the invariance of Eq. 4 and the third does not. The projection of the third term onto w d scales with ρ. In Eq. 5, ρ takes 0, 0.1, 0.25, 0.5, 0.75, 1.0 at five seeds and 4,000 samples per setting, split 0.6, 0.15, and 0.25, with the identical measures and geometry measures recomputed at each level. Encoder panel and controlled pretraining Every publicly released model is treated as a frozen feature extractor mapping an image to one global embedding vector. The panel is ten encoders for chest radiography (Table 2): the chest radio- graph encoders RAD-DINO [34] (768 dimensions), BiomedCLIP [35] (512), and the TorchXRayVi- sion DenseNet-121 [36] (1,024); the general-purpose self-distilled vision transformers DINOv3 ViT- L, ViT-B, and ViT-S [37] (1,024, 768, 384) and DINOv2 ViT-L [38] (1,024); the general-purpose language-supervised encoders CLIP ViT-L/14 [39] (768) and SigLIP2-L [40] (1,152); and a ran- domly initialized ViT-S/16 [41] (384) as the untrained reference. For dermatology and funduscopy, we replace the three chest radiograph encoders with two modality-specific encoders, giving nine encoders in each modality. Those encoders are MONET [42] (768) and DermLIP ViT-B/16 [43] (512) for dermatology, and RETFound [44] (1,024) and FLAIR [45] (512) for funduscopy. We use each encoder’s native global vector: the class token for the self-distilled transformers, the pooled projection output for the language-supervised encoders, and the pooled classifier feature for the DenseNet. Images are resized and center-cropped to 224× 224 pixels and normalized with the ImageNet channel statistics [62]. Extraction runs once per encoder at a batch size of 64 with 25 four workers in half precision, and the features are cached. Each model’s downloaded weights are deleted afterwards. Twenty-two encoders were pretrained on the chest radiograph training split under a matrix that fixes data, architecture, and budget while varying the training recipe. Each run is initialized from the DINOv3 checkpoint matched to its backbone: DINOv3-S for ViT-S/16 and DINOv3-B for ViT-B/16 [37]. These checkpoints transfer to chest radiograph classification across pediatric and adult cohorts [63]. All runs share one configuration: 224× 224 inputs; 50 epochs; a batch size of 256 with gradient accumulation to an effective 512; AdamW [64] at a learning rate of 1× 10 −4 and weight decay 0.05; a linear warmup over the first 10% of steps followed by cosine decay to zero; bfloat16 precision with gradient checkpointing; and augmentation by random resized crop, random horizontal flip, and color jitter of strength 0.4 applied with probability 0.5, the crop covering 20% to 100% of the image for the self-supervised objective and 50% to 100% otherwise. The three objectives are an image-image contrastive loss over two augmented views in the manner of SimCLR [65] at temperature 0.1; a masked mean binary cross-entropy over the 13 findings; and an image-text contrastive loss [39] at temperature 0.07 against a Bio_ClinicalBERT text encoder [66], with the report text truncated at 256 tokens. The image-image objective uses the backbone output through a two-layer projection head of width 2,048 mapping to a 256-dimensional space. The image-text objective uses one linear projection to 256 dimensions on each encoder. Both are trained jointly with the encoder and discarded at feature extraction. The matrix has 22 runs: 12 core runs over three objectives, two backbones, and two pretraining data compositions, the natural composition and one resampled to equalize the race composition of the pretraining set; 4 runs varying the report text of the image-text objective across the two backbones, one condition neutralizing demographic tokens by case-insensitive whole-word removal against a fixed term list and one prepending an explicit sentence naming race, sex, and age band; and 6 seed runs repeating the three objectives at ViT-S under seeds 1 and 2, seed 0 being the core run. Subgroup measures and disease heads Four measures enter Eq. 1: AUROC, which needs no threshold; sensitivity and the FPR, both evaluated at one threshold shared by every subgroup, fixed on the validation split at the 20th percentile of the positive scores, which sets overall sensitivity to 0.80; and the expected calibration error over 15 equal-width probability bins. The false negative rate difference equals the sensitivity difference and is not reported separately. A secondary operating point at the Youden index [67] was computed alongside and is not reported. Differences are computed for race, age, sex, and insurance on the chest radiograph pool, for skin type, sex, and age in dermatology, and for race, ethnicity, sex, and age in funduscopy. Two summaries of the same subgroup AUROC values are also recorded, the worst-group value min j A j and the equity-scaled AUROC of Luo et al. [49], the overall AUROC divided by one plus the total absolute deviation of the subgroup values from it. Outside the finetuning experiment, the encoder is frozen and one head is fitted per finding on cached training features and evaluated on the pooled test split. The linear head is a logistic regression at C = 1.0 with at most 1,000 iterations and class weights inverse to class frequency, on features standardized with training-split statistics; the nonlinear head is a one-hidden-layer network of 256 units with an L 2 penalty of 1× 10 −4 , at most 200 iterations, and early stopping; both use seed 0 and the scikit-learn implementations [68]. We use at most 100,000 training rows to fit each head, and the test split is uncapped. The main text reports the linear head and Supplementary Note 1 the nonlinear head. 26 Mitigation methods, the achievable difference, and its prediction Nine methods were applied to the frozen features of every encoder and finding for race and for age, spanning four families: methods acting on the training data, methods acting on the objective, meth- ods acting on the scores, and methods erasing the demographic direction from the features. The data methods are group-balanced resampling, which resamples each group to a common size, and Kamiran-Calders reweighing [69], which weights a case in group g with label y by P(g)P(y)/P(g,y). The objective methods are group distributionally robust optimization [11] at 80 epochs, a learning rate of 1× 10 −2 , a step of 0.01 on the group weights applied once per epoch, and Adam with a weight decay of 1×10 −4 ; adversarial removal through a gradient reversal layer [70,71] whose 64-unit adversary receives the predictor’s logit as input, at 80 epochs, a learning rate of 1× 10 −3 , a fixed adversarial weight of 1.0, and the same optimizer and weight decay; and the exponentiated-gradient reduction of Agarwal et al. [10] under an equalized-odds constraint, which returns a randomized ensemble. The score methods are a per-group threshold shift that re-centers each group on its own 0.80-sensitivity validation quantile [72] and per-group Platt recalibration [46], which fits a one-dimensional logistic map per group on the validation split. The erasure methods are LEACE in closed form [13] and iterative nullspace projection at ten iterations [12]. Every method is fitted on the training split and evaluated on the test split at seed 0. The two data methods and the three objective methods each fit a disease classifier and return its scores. The two score methods apply their per-group map to the unmitigated model’s scores. After erasure, we refit the linear head on the transformed training features. Both in-processing predictors are linear and are trained full batch on the standardized training features. A tenth method, a Gaussian optimal transport map between subgroup feature distributions with covariance shrinkage 0.1 in the manner of Fair- CLIP [49], applies to the image-text encoder only and is reported in Supplementary Note 1. LetM denote the nine methods, let ∆ m and A m denote the difference and the disease AUROC produced by method m, and let A 0 and ∆ 0 denote the unmitigated values. The quantity that we report for a unit is the difference reachable at matched disease performance, ∆ ⋆ = min ∆ m : m∈M∪0, A m ≥ A 0 − τ , τ = 0.010.(6) The unmitigated model is always feasible in Eq. 6, so ∆ ⋆ ≤ ∆ 0 . The nine methods were run for race and for age on the chest radiograph pool and for every attribute in the two smaller modalities, with both head types, and with LoRA adapters added for the controlled encoders. Supplementary Table 5 reports the paired change for each method. Three feature sets were regressed on ∆ ⋆ over the frozen panel by ridge regression at a penalty of 1.0, on features that are standardized within each training fold. The sets are the erasure cost and the geometric overlap together, linear decodability alone, and nonlinear decodability alone. Out- of-sample R 2 comes from five-fold cross-fitting at seed 0 and its reference distribution from 1,000 target shuffles. A negative value means that the predictor does worse than the held-out mean. We also fit the two geometry measures with leave-one-encoder-out cross-validation. That predictor was then fitted on the chest radiograph panel and applied to dermatology and to funduscopy (Fig. 5). Finetuning and the acquisition controls RAD-DINO and DINOv2 ViT-L were finetuned at four ordinal levels of unfreezing: a linear head, the last transformer block, LoRA adapters of rank 8 and scaling 16 [73], and the full backbone. All levels share one configuration: at most 10 epochs; batch size 64; AdamW with weight decay 0.05 and a linear warmup over the first 10% of steps; learning rate 1× 10 −3 for the head and 1× 10 −5 for unfrozen backbone parameters; bfloat16 precision with gradient checkpointing; and early stopping on mean validation AUROC across findings with patience 3, minimum improvement 27 0.0005, at least one epoch evaluated, and the best epoch restored before inference. Race, sex, and age were evaluated at every level, each with its own fair-model reference. Without early stopping the full level memorized the training split, its training loss falling from 0.244 to 0.0012 over ten epochs while its test AUROC fell below the LoRA level. On RAD-DINO and DINOv2 ViT-L, the race and age differences were recomputed within each acquisition view and compared with their pooled values by a paired test over evaluation units. Each view has its own reference from Eq. 2 at that view’s subgroup counts. We prespecified a minimum of 500 evaluable images per view before analyzing any result. We also analyzed view and site as attributes in their own right, entering Eq. 1 and Eq. 2 as a demographic attribute does. Auditing published subgroup differences Step one of FRAME needs the reported difference, the per-subgroup counts that it was computed on, and the common performance value assigned to every subgroup under exact fairness. None of these has to come from our own models, so it applies to any difference that a study has already published. The set of studies was assembled to cover the reports that this field treats as its principal empirical evidence of subgroup performance differences, with at least one study for each imaging modality in which such differences have been reported. It is not a systematic search. Twenty-two studies were screened against four criteria: the reported statistic is an AUROC, a true positive rate (TPR), or an FPR, which are the statistics that the reference is derived for; the article states the difference numerically instead of plotting it alone; it names the subgroups that the difference ranges over and states the number of cases in each on the denominator that the statistic uses; and it reports an overall performance value or enough per-subgroup values to recover an overall value. Nine studies meet all four [1,8,47,6,50,51,52,54,53], giving 89 claims over six imaging modalities, ten datasets, ten tasks, and the race, sex, race-by-sex, age, and skin-tone attributes. Of the 13 excluded studies, 5 fail the first criterion by reporting accuracy, an F1 score, or a parity ratio in place of a rate or an AUROC, 5 fail the third by reporting a difference without the per-subgroup counts it was computed on, and 3 fail the second by plotting every result. Two of those exclusions print a group-wise value and still cannot be audited. (i) One reports the demographic composition but not the label distribution that the AUROC denominators need. (i) The other reports the average of AUROC differences from a baseline, averaged over five classifications, so no per-subgroup value exists. Where a study reports one auditable block and one that is not auditable, only the auditable block is taken. One study reporting an abdominal computed tomography AUROC per subgroup is therefore audited on that block and not on its chest radiograph block, whose per-subgroup value is averaged over eight diagnoses. Each reported value and each count was transcribed from the article or its supplement into a versioned table, together with the exact source table. No value was estimated from a figure. Each claim records where its denominators come from. We obtain the denominators in three ways: printed directly in the article, derived by arithmetic on totals and shares that the article prints, or reconstructed from the article’s own pool composition and its stated split fraction where no per-subgroup count is printed. Subgroups that a study excluded from its own comparison are excluded here. Where fewer than two subgroups have both a value and a denominator, we report the claim as not auditable and estimate no value for it. The reference is then built at those counts as in Eq. 2. Where the article reports an overall value, we assign that value to every subgroup. Otherwise we use the denominator-weighted mean of the reported subgroup values, which recovers the overall value without approximation whenever the statistic is a rate. For a difference in AUROC, the per-subgroup draw is F as above. For a difference in a rate, meaning a TPR or an FPR at whatever threshold the study fixed, every subgroup shares one true rate under exact fairness. Its observed value is a binomial proportion at the subgroup’s own denominator, which is the positive count for a TPR and the negative count 28 for an FPR. That draw is exact and needs no closed-form approximation. Each reference uses the same number of draws at seed 0 and the same one-sided exceedance level as above, corrected across the claims sharing a metric under FDR control at α = 0.05. An AUROC difference is therefore not corrected against a family of rate differences. Supplementary Table 7 reports every claim with its provenance. The audit uses no embedding, no model, and no image from this study. Statistical analysis Seeds are fixed throughout: 0 for head fitting, the bootstrap, the reference simulation, and the erasure and injection fits, and 1 and 2 for the two additional pretraining runs per objective. Every resampling and simulation count in the study is fixed once and used everywhere. Uncer- tainty on a performance quantity comes from a cluster bootstrap resampled at the patient level, so all images of one patient are drawn or omitted together, with n boot = 1,000 resamples at seed 0 [23]. The fair-model reference of Eq. 2 uses a separate and larger count, B = 2,000 simulated fair models per evaluation unit at seed 0, because that distribution is the object being estimated and its upper tail sets the resolution of the exceedance level. Permutation tests use n perm = 1,000 draws, and the cross-fitted predictor uses five folds with 1,000 target shuffles for its reference dis- tribution. A permutation or simulation level is therefore reported no finer than 1 in 1,000 or 1 in 2,000, respectively. A reported performance value is the bootstrap mean with its standard deviation (SD) and the 2.5th and 97.5th percentiles of the resampled distribution as a 95% CI, written mean±SD [95% CI]. The same three quantities describe the reference of Eq. 2, taken over the 2,000 simulated models. Metrics are given on the 0 to 1 scale to three decimals, a share of one such value by another in percent, and correlations and p-values to three decimals. The observed difference, the achievable difference of Eq. 6, the two decodability measures, the geometric overlap, and the erasure cost are reported as point estimates without resampling intervals. Every test on those quantities uses either a paired change or the reference of Eq. 2, and each of those has its own interval. A contrast between two conditions evaluated on the same patients uses the paired cluster bootstrap, resampling the patient clusters once and recomputing both conditions on every resample. The interval is computed for the paired difference and not for either value. Its p-value is obtained by shifting the resampled difference distribution to zero and reflecting it. A comparison across evaluation units, where the units share no patients, uses the Wilcoxon signed-rank test over the paired units [48]. A trend along an injection grid, a collinearity grid, or the four unfreezing levels uses the Spearman rank correlation over strengths. The exceedance of an observed difference over its own reference uses the simulated distribution directly, as the share of the 2,000 fair models matching or exceeding the observed value. All tests are two-sided at a significance level of 0.05, with one exception: the exceedance test is one-sided by construction, since only a difference larger than the reference is evidence of a disparity. Multiplicity is controlled within each family by the Benjamini-Hochberg FDR procedure [24] at 0.05, and corrected values are written p FDR throughout while an uncorrected value is written p. A family is one experiment and one attribute for the reference, acquisition, worst-group performance, and injection tests, so the encoders and findings inside one such experiment are corrected together. Four of the 41 families are broader. The two mitigation families cover the paired change in the difference against disease performance, and the reduction to the achievable difference. Each is corrected across every attribute and across the frozen panel, the two smaller modalities, and the injection experiment. The cross-modality transfer family and the family comparing the reference with the observed difference are corrected across attributes. Counts of evaluation units, such as how many exceed their reference, are reported as counts out of the evaluable total and are based on the FDR-corrected per-unit tests. A unit enters its family 29 whenever it meets the minimum subgroup size of 20 evaluable images in at least two subgroups. No unit was excluded after its result was computed. Data availability All data used in this study come from existing, publicly released sources. No images or derived records are redistributed here. The one exception is the single chest radiograph reproduced in Fig. 1, which comes from the openly released NIH ChestX-ray14 collection. The chest radiographs come from six sources. MIMIC-CXR [25] is available from PhysioNet under credentialed access at https://physionet.org/content/mimic-cxr-jpg/, with a signed data use agreement and completion of the required human-subjects training. The race, ethnicity, and insurance fields are taken from MIMIC-IV [60] (https://physionet.org/content/mimiciv/) under the same terms. CheXpert [26], which supplies the report text used for the image-text pretraining runs, is available from the Stanford AIMI portal (https://aimi.stanford.edu/) after registration and acceptance of its research use agreement. NIH ChestX-ray14 [27] is openly available from the US National Institutes of Health Clinical Center at https://nihcc.app.box.com/v/ChestXray-NIHCC. PadChest [28] is available from the BIMCV repository at https://bimcv.cipf.es/bimcv-p rojects/padchest/ under its academic license on request. VinDr-CXR [29] (https://physio net.org/content/vindr-cxr/) and the pediatric VinDr-PCXR [30] (https://physionet. org/content/vindr-pcxr/) are available from PhysioNet under credentialed access. The two smaller modalities are likewise public. The dermatology pool combines ISIC 2019 [59,32], openly available from the ISIC Archive at https://challenge.isic-archive.com/, Fitzpatrick17k [31], available at https://github.com/mattgroh/fitzpatrick17k under its stated terms, and the Diverse Dermatology Images set [3], available from the Stanford AIMI portal after registration and acceptance of a research use agreement. The funduscopy pool is the scanning laser ophthalmoscopy images of Harvard-FairVision [33], available at https://ophai.hms.harvard.edu/datasets/ on request under its license. The audit of published subgroup differences uses no data beyond what the audited articles print. Those articles are Seyyed-Kalantari et al., Nature Medicine 2021 [1]; Glocker et al., EBioMedicine 2023 [8]; Lotter, Nature Communications 2024 [47]; Zong et al., ICLR 2023 [6]; Daneshjou et al., Science Advances 2022 [50]; Burlina et al., Translational Vision Science & Technology 2021 [52]; Vaidya et al., Nature Medicine 2024 [51]; Yala et al., Journal of Clinical Oncology 2022 [54]; and Tayebi Arasteh et al., Communications Medicine 2024 [53]. Code availability The analysis code is publicly available at https://github.com/mahshadlotfinia/bias_origin. The repository provides the data build, controlled pretraining, analysis, and figure code. It also records the fixed bootstrap, permutation, and simulation seeds. It does not redistribute the model weights or the underlying datasets. All evaluated encoders are local, open-weight models run entirely on-site as frozen feature extractors, without any cloud service or third-party application programming interface. No closed model and no model served over a network was used. Model access differs by checkpoint. Most are ungated downloads under permissive licenses, whereas the DINOv3 checkpoints are distributed under a custom license that must be accepted before use. The controlled encoders were trained in- house from the DINOv3 ViT-S and ViT-B checkpoints. The untrained reference is a ViT-S/16 with random weights and has no released checkpoint. The models were accessed and all experiments run between June and August 2026. The URLs of the evaluated checkpoints are: 30 Chest radiograph encoders: – RAD-DINO: https://huggingface.co/microsoft/rad-dino – BiomedCLIP: https://huggingface.co/microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch1 6_224 – TorchXRayVision DenseNet-121 (densenet121-res224-all): https://github.com/mlmed/torchxrayvi sion General-purpose image encoders: – DINOv3 ViT-L, ViT-B, and ViT-S: https://huggingface.co/facebook/dinov3-vitl16-pre train-lvd1689m, https://huggingface.co/facebook/dinov3-vitb16-pretrain-lvd1689m, https://huggingface.co/facebook/dinov3-vits16-pretrain-lvd1689m – DINOv2 ViT-L: https://huggingface.co/facebook/dinov2-large – CLIP ViT-L/14: https://huggingface.co/openai/clip-vit-large-patch14 – SigLIP2-L: https://huggingface.co/google/siglip2-large-patch16-512 Dermatology and fundus encoders: – MONET: https://huggingface.co/chanwkim/monet – DermLIP ViT-B/16: https://huggingface.co/redlessone/DermLIP_ViT-B-16 – RETFound: https://huggingface.co/iszt/RETFound_mae_meh – FLAIR: https://github.com/jusiro/FLAIR Text encoder for the image-text pretraining runs: – Bio_ClinicalBERT: https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT Analyses used Python 3.11, PyTorch 2.2.2 with torchvision 0.17.2, Hugging Face Transform- ers 4.40.2, timm 0.9.16, OpenCLIP 2.24.0, TorchXRayVision 1.2.1, NumPy 1.26.4, SciPy 1.11.4, pandas 2.1.4, scikit-learn 1.4.2, fairlearn 0.10.0, pingouin 0.5.4, and concept-erasure 0.2.4, with the full pinned environment in the repository. Encoder pretraining, finetuning, and feature extraction ran on NVIDIA L40S GPUs (48 GB each) with Intel Xeon Silver 4310 CPUs. Every other stage ran on CPU. Acknowledgements DT is supported by the German Ministry of Research, Technology and Space (TRANSFORM LIVER - 031L0312C, DECIPHER-M - 01KD2420B), DFG (515639690), and the European Union (Horizon Europe, ODELIA - GA 101057091, ERC Starting Grant SAGMA – GA 101222556). STA is supported by the Excellence Strategy of the German Federal Government, the Länder, and RWTH ERS (START_526-26). Author contributions The formal analysis was conducted by ML and STA. The original draft was written by ML and STA and edited by STA. ML developed the code. The experiments were performed by ML. The statistical analyses were performed by ML and STA. DT provided clinical expertise. ML, DT, AM, and STA provided technical expertise. The study was defined by STA. All authors read the manuscript and agreed to the submission of this paper. 31 Competing interests ML is employed by Generali Deutschland Services GmbH, Germany, and is on the editorial board of European Radiology Experimental. DT received honoraria for lectures from Bayer, GE, Roche, AstraZeneca, and Philips and holds shares in StratifAI GmbH, Germany, and in Synagen GmbH, Germany. AM is an associate editor at IEEE Transactions on Medical Imaging. STA is on the editorial board of Communications Medicine and of European Radiology Experimental, and on the trainee editorial board of Radiology: Artificial Intelligence. The authors declare no other competing financial or non-financial interests. References 1. Seyyed-Kalantari, L., Zhang, H., McDermott, M.B., Chen, I.Y., Ghassemi, M.: Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature medicine 27(12), 2176–2182 (2021) 2. Larrazabal, A.J., Nieto, N., Peterson, V., Milone, D.H., Ferrante, E.: Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. Proceedings of the National Academy of Sciences 117(23), 12592–12594 (2020) 3. Daneshjou, R., Vodrahalli, K., Novoa, R.A., et al.: Disparities in dermatology ai performance on a diverse, curated clinical image set. Science Advances 8(32), eabq6147 (2022). https://doi.org/10.1 126/sciadv.abq6147 4. Vaidya, A., Chen, R.J., Williamson, D.F., Song, A.H., Jaume, G., Yang, Y., Hartvigsen, T., Dyer, E.C., Lu, M.Y., Lipkova, J., et al.: Demographic bias in misdiagnosis by computational pathology models. Nature Medicine 30(4), 1174–1190 (2024) 5. Chen, R.J., Wang, J.J., Williamson, D.F., Chen, T.Y., Lipkova, J., Lu, M.Y., Sahai, S., Mahmood, F.: Algorithmic fairness in artificial intelligence for medicine and healthcare. Nature biomedical engineering 7(6), 719–742 (2023) 6. Zong, Y., Yang, Y., Hospedales, T.: MEDFAIR: Benchmarking fairness for medical imaging. In: The Eleventh International Conference on Learning Representations (2023), https://openreview.net/f orum?id=6ve2CkeQe5S 7. Gichoya, J.W., Banerjee, I., Bhimireddy, A.R., Burns, J.L., Celi, L.A., Chen, L.C., Correa, R., Dullerud, N., Ghassemi, M., Huang, S.C., et al.: Ai recognition of patient race in medical imaging: a modelling study. The Lancet Digital Health 4(6), e406–e414 (2022) 8. Glocker, B., Jones, C., Bernhardt, M., Winzeck, S.: Algorithmic encoding of protected characteristics in chest x-ray disease detection models. EBioMedicine 89 (2023) 9. Yang, Y., Zhang, H., Gichoya, J.W., Katabi, D., Ghassemi, M.: The limits of fair medical imaging ai in real-world generalization. Nature medicine 30(10), 2838–2848 (2024) 10. Agarwal, A., Beygelzimer, A., Dudík, M., Langford, J., Wallach, H.: A reductions approach to fair classification. In: International conference on machine learning. p. 60–69. PMLR (2018) 11. Sagawa*, S., Koh*, P.W., Hashimoto, T.B., Liang, P.: Distributionally robust neural networks. In: International Conference on Learning Representations (2020), https://openreview.net/forum?id= ryxGuJrFvS 12. Ravfogel, S., Elazar, Y., Gonen, H., Twiton, M., Goldberg, Y.: Null it out: Guarding protected at- tributes by iterative nullspace projection. In: Proceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics. p. 7237–7256. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.647 13. Belrose, N., Schneider-Joseph, D., Ravfogel, S., Cotterell, R., Raff, E., Biderman, S.: Leace: Perfect linear concept erasure in closed form. In: Advances in Neural Information Processing Systems. vol. 36, p. 66044–66063 (2023). https://doi.org/10.52202/075280-2884, https://proceedings.neurips. c/paper_files/paper/2023/file/d066d21c619d0a78c5b557fa3291a8f4-Paper-Conference.pdf 14. Bernhardt, M., Jones, C., Glocker, B.: Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms. Nature Medicine 28(6), 1157–1158 (2022) 32 15. Mukherjee, P., Shen, T.C., Liu, J., Mathai, T., Shafaat, O., Summers, R.M.: Confounding factors need to be accounted for in assessing bias by machine learning algorithms. Nature Medicine 28(6), 1159–1160 (2022) 16. Petersen, E., Holm, S., Ganz, M., Feragen, A.: The path toward equal performance in medical machine learning. Patterns 4(7) (2023) 17. Jones, C., Castro, D.C., De Sousa Ribeiro, F., Oktay, O., McCradden, M., Glocker, B.: A causal perspective on dataset bias in machine learning for medical imaging. Nature Machine Intelligence 6(2), 138–146 (2024) 18. Knolle, M.A., Menten, M.J., Jungmann, F., Meissen, F., Glocker, B., Rueckert, D., Kaissis, G.: Dis- parate privacy risks from medical ai. Nature p. 1–7 (2026) 19. Borchers, C., Baker, R.S.: Abroca distributions for algorithmic bias assessment: Considerations around interpretation. In: Proceedings of the 15th International Learning Analytics and Knowledge Confer- ence. p. 837–843 (2025) 20. Briscoe, J., Kepler, G., DeFord, D.R., Gebremedhin, A.: Sample-size-induced bias in confusion-matrix metrics. In: The 28th International Conference on Artificial Intelligence and Statistics (2025), https: //openreview.net/forum?id=5bm7TH6tWb 21. Paes, L.M., Suresh, A.T., Beutel, A., Calmon, F.P., Beirami, A.: Multi-group fairness evaluation via conditional value-at-risk testing. IEEE Journal on Selected Areas in Information Theory 5, 659–674 (2024) 22. Arasteh, S.T., Farajiamiri, M., Lotfinia, M., Hinrichs-Puladi, B., Bienzeisler, J., Alhaskir, M., Rusu, M., Kuhl, C., Nebelung, S., Truhn, D.: The pretraining domain outweighs the training objective in setting the privacy-utility trade-off of differentially private medical image analysis (2026), https: //arxiv.org/abs/2601.19618 23. Efron, B.: Bootstrap methods: another look at the jackknife. In: Breakthroughs in statistics: Method- ology and distribution, p. 569–593. Springer (1992) 24. Benjamini, Y., Hochberg, Y.: Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological) 57(1), 289–300 (1995) 25. Johnson, A.E., Pollard, T.J., Berkowitz, S.J., Greenbaum, N.R., Lungren, M.P., Deng, C.y., Mark, R.G., Horng, S.: Mimic-cxr, a de-identified publicly available database of chest radiographs with free- text reports. Scientific data 6(1), 317 (2019) 26. Irvin, J., Rajpurkar, P., Ko, M., et al.: Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on artificial intelligence. vol. 33, p. 590–597 (2019) 27. Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases . In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). p. 3462–3471. IEEE Computer Society (Jul 2017). https://doi.org/10.1109/CVPR.2017.369 28. Bustos, A., Pertusa, A., Salinas, J.M., de la Iglesia-Vayá, M.: Padchest: A large chest x-ray image dataset with multi-label annotated reports. Medical Image Analysis 66, 101797 (2020). https://doi. org/https://doi.org/10.1016/j.media.2020.101797 29. Nguyen, H.Q., Lam, K., Le, L.T., et al.: Vindr-cxr: An open dataset of chest x-rays with radiologist’s annotations. Scientific Data 9(1), 429 (2022) 30. Pham, H.H., Nguyen, N.H., Tran, T.T., Nguyen, T.N., Nguyen, H.Q.: Pedicxr: an open, large-scale chest radiograph dataset for interpretation of common thoracic diseases in children. Scientific data 10(1), 240 (2023) 31. Groh, M., Harris, C., Soenksen, L., Lau, F., Han, R., Kim, A., Koochek, A., Badri, O.: Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). p. 1820–1828. IEEE (2021) 32. Hernández-Pérez, C., Combalia, M., et al.: Bcn20000: Dermoscopic lesions in the wild. Scientific data 11(1), 641 (2024) 33. Luo, Y., Khan, M.O., Tian, Y., Shi, M., Dou, Z., Elze, T., Fang, Y., Wang, M.: Fairvision: Equitable deep learning for eye disease screening via fair identity scaling (2024), https://arxiv.org/abs/2310 .02492 33 34. Pérez-García, F., Sharma, H., Bond-Taylor, S., Bouzid, K., Salvatelli, V., Ilse, M., Bannur, S., Castro, D.C., Schwaighofer, A., Lungren, M.P., et al.: Exploring scalable medical image encoders beyond text supervision. Nature Machine Intelligence 7(1), 119–130 (2025) 35. Zhang, S., Xu, Y., Usuyama, N., Xu, H., Bagga, J., Tinn, R., Preston, S., Rao, R., Wei, M., Valluri, N., et al.: A multimodal biomedical foundation model trained from fifteen million image–text pairs. Nejm Ai 2(1), AIoa2400640 (2025) 36. Cohen, J.P., Viviano, J.D., Bertin, P., Morrison, P., Torabian, P., Guarrera, M., Lungren, M.P., Chaud- hari, A., Brooks, R., Hashir, M., et al.: Torchxrayvision: A library of chest x-ray datasets and models. In: International Conference on Medical Imaging with Deep Learning. p. 231–249. PMLR (2022) 37. Siméoni, O., Vo, H.V., Seitzer, M., et al.: Dinov3 (2025), https://arxiv.org/abs/2508.10104 38. Oquab, M., Darcet, T., Moutakanni, T., et al.: DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research (2024), https://openreview.net/forum?i d=a68SUt6zFt 39. Radford, A., et al.: Learning transferable visual models from natural language supervision. In: ICML. p. 8748–8763. PmLR (2021) 40. Tschannen, M., Gritsenko, A., Wang, X., Naeem, M.F., Alabdulmohsin, I., Parthasarathy, N., Evans, T., Beyer, L., Xia, Y., Mustafa, B., Hénaff, O., Harmsen, J., Steiner, A., Zhai, X.: Siglip 2: Multilin- gual vision-language encoders with improved semantic understanding, localization, and dense features (2025), https://arxiv.org/abs/2502.14786 41. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: ICLR 2021 (2021), https://openreview.net/forum ?id=YicbFdNTTy 42. Kim, C., Gadgil, S.U., DeGrave, A.J., Omiye, J.A., Cai, Z.R., Daneshjou, R., Lee, S.I.: Transparent medical image ai via an image–text foundation model grounded in medical literature. Nature medicine 30(4), 1154–1165 (2024) 43. Yan, S., Hu, M., Jiang, Y., Li, X., Fei, H., Tschandl, P., Kittler, H., Ge, Z.: Derm1m: A million-scale vision-language dataset aligned with clinical ontology knowledge for dermatology. In: 2025 IEEE/CVF International Conference on Computer Vision (ICCV). p. 12681–12690 (2025). https://doi.org/10 .1109/ICCV51701.2025.01178 44. Zhou, Y., Chia, M.A., Wagner, S.K., et al.: A foundation model for generalizable disease detection from retinal images. Nature 622(7981), 156–163 (2023) 45. Silva-Rodriguez, J., Chakor, H., Kobbi, R., Dolz, J., Ayed, I.B.: A foundation language-image model of the retina (flair): Encoding expert knowledge in text supervision. Medical Image Analysis 99, 103357 (2025) 46. Platt, J., et al.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10(3), 61–74 (1999) 47. Lotter, W.: Acquisition parameters influence ai recognition of race in chest x-rays and mitigating these factors reduces underdiagnosis bias. Nature Communications 15(1), 7465 (2024) 48. Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics bulletin 1(6), 80–83 (1945) 49. Luo, Y., Shi, M., Khan, M.O., Afzal, M.M., Huang, H., Yuan, S., Tian, Y., Song, L., Kouhana, A., Elze, T., et al.: Fairclip: Harnessing fairness in vision-language learning. In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 12289–12301. IEEE (2024) 50. Daneshjou, R., Vodrahalli, K., Novoa, R.A., Jenkins, M., Liang, W., Rotemberg, V., Ko, J., Swetter, S.M., Bailey, E.E., Gevaert, O., et al.: Disparities in dermatology ai performance on a diverse, curated clinical image set. Science advances 8(31), eabq6147 (2022) 51. Vaidya, A., Chen, R.J., Williamson, D.F., Song, A.H., Jaume, G., Yang, Y., Hartvigsen, T., Dyer, E.C., Lu, M.Y., Lipkova, J., et al.: Demographic bias in misdiagnosis by computational pathology models. Nature Medicine 30(4), 1174–1190 (2024) 52. Burlina, P., Joshi, N., Paul, W., Pacheco, K.D., Bressler, N.M.: Addressing artificial intelligence bias in retinal diagnostics. Translational Vision Science & Technology 10(2), 13–13 (2021) 53. Tayebi Arasteh, S., Ziller, A., Kuhl, C., Makowski, M., Nebelung, S., Braren, R., Rueckert, D., Truhn, D., Kaissis, G.: Preserving fairness and diagnostic accuracy in private large-scale ai models for medical imaging. Communications medicine 4(1), 46 (2024) 34 54. Yala, A., Mikhael, P.G., Strand, F., Lin, G., Satuluru, S., Kim, T., Banerjee, I., Gichoya, J., Trivedi, H., Lehman, C.D., et al.: Multi-institutional validation of a mammography-based breast cancer risk model. Journal of Clinical Oncology 40(16), 1732–1740 (2022) 55. Tayebi Arasteh, S., Lotfinia, M., Nolte, T., Sähn, M.J., Isfort, P., Kuhl, C., Nebelung, S., Kaissis, G., Truhn, D.: Securing collaborative medical ai by using differential privacy: domain transfer for classification of chest radiographs. Radiology: Artificial Intelligence 6(1), e230212 (2023) 56. Mohammadi, M., Vejdanihemmat, M., Lotfinia, M., Rusu, M., Truhn, D., Maier, A., Tayebi Arasteh, S.: Differential privacy for medical deep learning: methods, tradeoffs, and deployment implications. npj Digital Medicine 9(1), 93 (2026) 57. Tayebi Arasteh, S., Lotfinia, M., Perez-Toro, P.A., Arias-Vergara, T., Ranji, M., Orozco-Arroyave, J.R., Schuster, M., Maier, A., Yang, S.H.: Differential privacy enables fair and accurate ai-based analysis of speech disorders while protecting patient data. npj Artificial Intelligence 1(1), 37 (2025) 58. Ziller, A., Mueller, T.T., Stieger, S., Feiner, L.F., Brandt, J., Braren, R., Rueckert, D., Kaissis, G.: Reconciling privacy and accuracy in ai for medical imaging. Nature Machine Intelligence 6(7), 764–774 (2024) 59. Tschandl, P., Rosendahl, C., Kittler, H.: The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data 5(1), 180161 (2018) 60. Johnson, A.E., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T.J., Hao, S., Moody, B., Gow, B., et al.: Mimic-iv, a freely accessible electronic health record dataset. Scientific data 10(1), 1 (2023) 61. Hanley, J.A., McNeil, B.J.: The meaning and use of the area under a receiver operating characteristic (roc) curve. Radiology 143(1), 29–36 (1982) 62. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. p. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848 63. Arasteh, S.T., Shaigan, M., Kuhl, C., Kather, J.N., Nebelung, S., Truhn, D.: Resolution scaling governs dinov3 transfer performance in chest radiograph classification (2026), https://arxiv.org/abs/2510 .07191 64. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (2019), https://openreview.net/forum?id=Bkg6RiCqY7 65. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International conference on machine learning. p. 1597–1607. PmLR (2020) 66. Alsentzer, E., Murphy, J., Boag, W., Weng, W.H., Jindi, D., Naumann, T., McDermott, M.: Publicly available clinical bert embeddings. In: Proceedings of the 2nd clinical natural language processing workshop. p. 72–78 (2019) 67. Youden, W.J.: Index for rating diagnostic tests. Cancer 3(1), 32–35 (1950) 68. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Pret- tenhofer, P., Weiss, R., Dubourg, V., et al.: Scikit-learn: Machine learning in python. the Journal of machine Learning research 12, 2825–2830 (2011) 69. Kamiran, F., Calders, T.: Data preprocessing techniques for classification without discrimination. Knowledge and information systems 33(1), 1–33 (2012) 70. Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., March, M., Lempitsky, V.: Domain-adversarial training of neural networks. Journal of machine learning research 17(59), 1–35 (2016) 71. Zhang, B.H., Lemoine, B., Mitchell, M.: Mitigating unwanted biases with adversarial learning. In: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. p. 335–340 (2018) 72. Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. Advances in neural information processing systems 29 (2016) 73. Hu, E.J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Conference on Learning Representations (2022), https://openreview.net/forum?id=nZeVKeeFYf9 35 Supplementary information Supplementary Note 1: Additional checks on the chest radiograph protocol Sex and insurance as sensitive attributes We focus on race and age because the field reports its differences on those attributes, and because a fair-model reference could be derived for them at every combination. Two further attributes were evaluated on the same runs. For sex, across the 104 combinations of encoder and finding for which a reference is available, the observed difference has a median of 0.009 against a reference of 0.004. It exceeds the reference in 35 of the 104. A sex difference is therefore roughly one order of magnitude smaller than a race or age difference measured on the identical scores. Exact fairness produces most of the sex difference at those subgroup sizes. Insurance was harmonized into four categories, Medicare, Medicaid, Private, and Other, and evaluated on the controlled encoders. Across the 78 combinations of encoder, backbone, and finding, the difference has a median of 0.062 (interquartile range 0.047–0.076), between the corresponding race and age medians (Supplementary Table 1). No fair-model reference was computed for insur- ance, so the observed value cannot be separated into the part that exact fairness produces and the part that remains. No significance test was run on it either. Its Other category also contains 64,749 test radiographs, 52,038 of them from the four sites that record no insurance field, so the contrast is even more strongly a site contrast than the race contrast is (Supplementary Note 2). We therefore do not interpret these values as evidence of a disparity. Non-monotone per-group calibration maps A per-group threshold or a per-group calibration map cannot change a within-group rank statistic while the map is strictly increasing. Neither can therefore change a difference in the area under the receiver operating characteristic curve (AUROC), except where the fitted map is not monotone. Across the 260 combinations of encoder, finding, and attribute on which per-group Platt calibration was run, the race or age AUROC difference is unchanged in 252 and changes by less than 1× 10 −6 in three more. In the remaining five combinations, the change is between 0.130 and 0.440. Every one of the five is a race comparison on pleural other, lung lesion, or fracture, the three findings whose smallest race subgroups contain 20, 34, and 34 positive cases. A logistic map fitted on that many cases can return a constant, which destroys the within-group ordering it was supposed to preserve. For the per-group operating point shift, the change in the AUROC difference is nonzero in 238 of 260 combinations and is at most 0.016. That is the size of a tie at the threshold and not a change of ranking. Neither method changes the reported medians. The five calibration failures occur only in the smallest race subgroups, at 20 to 34 positive cases, which are also the subgroups with the largest fair-model references. The nonlinear disease head Every controlled analysis was run twice, once with a linear disease head and once with a two- layer head on the same frozen features. Pairing the two heads within each combination of encoder, backbone, and finding, the median change from the linear to the nonlinear head is −0.002 for race, +0.004 for age, +0.002 for insurance, and +0.001 for sex, over 78 paired combinations per attribute (Supplementary Table 1). The largest single change is 0.088, on one race combination. Head capacity therefore does not account for the subgroup performance difference, and the main text reports the linear head. 36 The second injected encoder The two injection operators were run on a second released encoder, the TorchXRayVision DenseNet- 121, on the same three findings and the same strength grids as on RAD-DINO. On both encoders, injecting decodability raises decodability and leaves the race difference unchanged. The invariance of Eq. 4 therefore holds at a second embedding width and on a convolutional backbone instead of a transformer. The entanglement operator removes a k-dimensional disease subspace. At k = 256 of the 1,024 available dimensions, the overall disease AUROC is unchanged on every finding, while the same grid lowers RAD-DINO’s overall disease AUROC from 0.837 to 0.802. The race difference changes little for each finding, at 0.075 to 0.079 on cardiomegaly, 0.177 to 0.166 on fracture, and 0.067 to 0.064 on pleural effusion. Its rank correlation with strength is −0.066 under false discovery rate (FDR) control (p FDR = 0.942), against 0.724 (p FDR = 0.014) on RAD-DINO. Decodability and the principal-angle overlap do change under the intervention, decodability rising from 0.800 to 0.912 (Spearman 0.593, p FDR = 0.028) and the overlap falling (Spearman −0.574, p FDR = 0.040). The operator therefore changed the representation without removing disease signal. Because the overall disease AUROC is unchanged, this intervention does not test the effect of entanglement between the group and the disease direction on the DenseNet-121. The second step requires the removed subspace to contain disease-predictive signal, and we verify that condition with the overall disease AUROC at the largest strength. The causal claim of the main text therefore depends on RAD-DINO alone. An optimal-transport fairness method A tenth mitigation method was run separately from the nine, matching subgroup feature distri- butions by optimal transport in the manner of FairCLIP [49]. Across the 13 findings, the race difference has a median of 0.078 under the method against 0.079 unmitigated. The reduction is significant for one finding, pleural effusion, where the difference falls by 0.006 (p FDR = 0.026). For the other 12 findings, the reduction is not significant. It is reported here because it belongs to a different family from the nine and was run on one encoder only. Supplementary Note 2: Data and reporting caveats Pooled attributes whose residual category is largely a site indicator Race is recorded at two of the six chest radiograph sites and insurance at the same two. The harmonization assigns every record from a site without the field to the Other category. That category mixes patients recorded as another race or another insurance type with patients from a site that does not record the field (Supplementary Table 3). In the test split, the race Other category contains 68,521 radiographs, 52,038 of them (76%) from the four sites with no race field. The insurance Other category contains 64,749, the same 52,038 of them (80%). Any pooled contrast on either attribute is therefore partly a contrast between sites. We analyze site as an attribute in its own right, with its own reference, and we also report the demographic differences within a fixed acquisition view. The limits of the sampling model behind the reference The reference draws each subgroup’s AUROC at that subgroup’s own positive and negative counts, and those counts are images. Radiographs from one patient are not independent, which is why every 37 bootstrap interval in the study resamples patient clusters. The reference is built at image counts instead. Under positive within-patient correlation, the effective sample behind a subgroup AUROC is smaller than its image count. The reference therefore understates the true spread that a perfectly fair model produces. Our correction is therefore conservative. It attributes less of the reported difference to sampling than a patient-level model would. The share of each reported difference that remains after the correction is an upper bound. We did not estimate the within-patient correlation, so the direction of that bound is an assumption and not a measurement. Supplementary Note 3: Dermatology and funduscopy Dermatology and funduscopy against their references Extended Data Fig. 3 summarizes dermatology and funduscopy, and this Note reports the corre- sponding per-combination values. The fair-model reference and the nine mitigation methods were run on dermatology and on funduscopy with the same protocol as on chest radiographs, over nine combinations of encoder and finding in each modality per attribute. Of the seven combinations of modality and attribute, one exceeds its reference in most comparisons (Supplementary Table 4). For dermatology, the age difference exceeds its reference in 9 of 9 combinations, at an observed median of 0.068 against a reference of 0.023. No other attribute in either modality exceeds its reference in more than 2 of 9. For fundus sex, the observed median of 0.008 is below the reference of 0.014. None of the 126 tests of the mitigation methods across both modalities, attributes, and metrics is significant. At these cohort sizes, the analysis cannot distinguish most observed differences from sampling variation under exact fairness. In dermatology and fundus cohorts of a few thousand images, subgroup counts are small enough that the difference that a perfectly fair model produces is already the size of the reported difference. The chest radiograph analysis quantifies the same mechanism at larger sizes. The causal analyses in the main text are based on chest radiographs for that reason. Transfer of the chest radiograph predictor We tested directly whether the geometry measured on chest radiographs predicts the achievable difference in another modality, by fitting the predictor on the chest radiograph panel and applying it to dermatology and to funduscopy. Out-of-sample R 2 is −3.40 for dermatology and −1.92 for funduscopy, neither significant (p FDR = 0.787 for both modalities). A negative value means that the fitted predictor does worse than predicting the mean of that modality, so no transfer was detected. Within chest radiography, the leave-one-encoder-out predictor has an out-of-sample R 2 of 0.024 (p FDR = 0.021), against −0.007 for the same features pooled across encoders (Extended Data Fig. 5). The entanglement geometry therefore contains a small amount of encoder-specific information. Supplementary Note 4: The steps of FRAME FRAME end to end Supplementary Algorithm 1 specifies the full procedure of Fair-model Reference And Mechanism Evaluation (FRAME) for one reported subgroup difference. In Supplementary Algorithm 1, SE denotes the Hanley-McNeil standard error of an empirical AUROC at a given true value and a given pair of positive and negative counts [61]. Step 1 needs nothing beyond the model’s overall 38 AUROC and the per-subgroup positive and negative counts. So it can be run on a published result from counts alone. Step 2 needs the cached features, so it applies where the representation is available. The closed-form draw is an approximation. On 12 randomly chosen units per run, we re- estimated the same reference by ranking binormal draws at the observed counts, which is correct by construction and far slower. The two estimates agreed. Adding 1 to the numerator and denominator keeps the exceedance level strictly positive at the resolution of the simulation. Where the observed difference exceeds every simulated fair-model difference, the exceedance level is reported at 1/(B + 1) and not at zero. Supplementary Algorithm 1 FRAME, end to end, for one reported subgroup difference Require: scores z, labels y, subgroup assignment g on the test split; cached features X and train index T of the encoder that produced z; draws B; level α; strength grids S,K 1: ˆ A← AUROC(y,z); m j ← AUROC(y,z|g = j); ˆ ∆← max j m j − min j m j Eq. 1 2: (n + j ,n − j )← positive and negative counts of subgroup j, keeping j with n + j + n − j ≥ 20 3: Step 1, reference. 4: for b = 1 to B do 5: A jb ← min 1, max 0, N( ˆ A, SE( ˆ A,n + j ,n − j ) 2 ) for each j same true AUROC for every subgroup 6: G b ← max j A jb − min j A jb 7: end for 8: r ← 1 B P b G b ; p← 1 +|b : G b ≥ ˆ ∆| /(B + 1) 9: if p > α then 10: return r, p, classification = consistent with exact fairness no remainder to explain 11: end if 12: e← ˆ ∆− r the remainder 13: Step 2, mechanism. 14: for s∈S do 15: ∆ dec s ← recompute ˆ ∆ on T dec s (X) Alg. 2; invariant by Eq. 4 16: end for 17: for k ∈K do 18: ∆ ent k ← recompute ˆ ∆ on T ent k (X) Alg. 2 19: end for 20: ρ dec ← Spearman(S,∆ dec ); ρ ent ← Spearman(K,∆ ent ) 21: return r, e, p, and the classification from (ρ dec ,ρ ent ): entanglement if ρ ent is significant and ρ dec is not Injection operators Supplementary Algorithm 2 states both operators step by step, and four details of the construction are recorded only here. Group membership enters as a sign vector equal to +1 on the largest subgroup, −1 on every other named subgroup, and 0 where the attribute is missing. Rows with a missing attribute value are unchanged, because a = 0 for those rows. The random direction of the decodability operator is orthogonalized against the fitted disease direction. The absence of disease signal is exact and does not rely on the near-orthogonality of two random directions in a high- dimensional space. The disease subspaces of the entanglement operator are nested by construction, since each direction is recorded before the training features are deflated along it. We reuse the basis fitted at the largest strength for every smaller strength, so no strength is fitted twice. At strength zero, the matrix is unchanged. The zero-strength point on each response curve therefore uses the unmodified experimental values and not a separately computed baseline. 39 Supplementary Algorithm 2 Injection operators on a cached embedding matrix Require: X ∈ R N×D ; signs a∈−1, 0, +1 N ; labels y; train index T; mode; strength 1: if strength = 0 then 2: return X 3: end if 4: σ ← 1 D P D d=1 sd(X ·d ) 5: w ← θ/∥θ∥, θ ← logistic disease head fitted on X T , rescaled to raw features 6: if mode = decodability, strength s then 7: u∼N(0,I D ); u← u− (u ⊤ w)w; u← u/∥u∥ u ⊤ w = 0 8: return X + sσ au ⊤ 9: else 10: U ← [w ] 11: for i = 2 to k do 12:X T ← X T − X T v ⊤ with v the direction just recorded; refit; append its unit direction to U 13: end for 14: U ← qr(U); D a ← diag(1[a < 0]) 15: return X− D a X U ⊤ 16: end if Supplementary Table 1: Subgroup performance differences for four sensitive attributes, and the linear against the nonlinear disease head. Values are areas under the receiver operating character- istic curve on the 0 to 1 scale, measured on the controlled encoders at natural data composition and one seed, over 78 combinations of encoder, backbone, and finding per attribute. The two head columns report the median and the interquartile range of the difference under each head. The paired columns compare the two heads within each combination, and the largest change column reports the largest absolute change over the 78 pairs. Race and age are the attributes that the main text reports. Sex and insurance are reported here only. Insurance has no fair-model reference and no significance test. AttributeLinear headNonlinear headMedian paired change Largest change Race0.075 (0.057–0.103) 0.072 (0.056–0.102)−0.0020.088 Age0.067 (0.047–0.094) 0.074 (0.056–0.097)+0.0040.061 Insurance 0.062 (0.047–0.076) 0.064 (0.049–0.083)+0.0020.028 Sex0.012 (0.007–0.018) 0.012 (0.007–0.024)+0.0010.055 40 Supplementary Table 2: The frozen chest radiograph panel by finding, over which the subgroup counts vary. Findings are ordered by the number of positive cases in the smallest evaluable race subgroup, given in the n column, and each value is the median over the ten encoders on the pooled test split of 125,992 radiographs from 41,621 patients. The reference is the mean of the sampling distribution of the same statistic under exact fairness over 2,000 simulated fair models at that finding’s own per-subgroup counts. The share column reports the median of the ten per-encoder ratios of the reference to the observed difference. That ratio is computed for each encoder before the median is taken across encoders, so it is not the ratio of the two medians beside it. AUROC, area under the receiver operating characteristic curve. FindingRaceAge n Observed Reference Share n Observed Reference Share Support devices3960.0580.01526 32580.0480.00715 Cardiomegaly3920.0790.01923 16010.1150.0108 Lung opacity3840.0800.01924 22900.1540.0096 Pleural effusion3390.0840.01518 18900.0690.00710 Atelectasis2910.0930.02324 19530.0930.01011 Edema2430.0370.01952 11690.0580.01017 Pneumonia1360.0690.03854 9490.0500.01734 Pneumothorax830.0820.03441 6240.0650.01422 Consolidation690.1000.04240 6720.0620.01728 Enlarged cardiomediastinum 480.0820.05567 3430.0540.02750 Fracture340.1470.06645 3190.0860.02732 Lung lesion340.1030.06059 3230.0400.02870 Pleural other200.0770.083102 2020.1010.03736 Supplementary Table 3: Composition of the two pooled chest radiograph attributes whose residual category contains every image from the sites that do not record the field. Counts are radiographs in the pooled test split of 125,992 images. The recorded column counts images from MIMIC-CXR and CheXpert, the two sites that record the field, and the not recorded column counts images from NIH ChestX-ray14, PadChest, VinDr-CXR, and VinDr-PCXR, which do not. Every image from a site that does not record the field is assigned to the Other category by the harmonization, so that category alone mixes the two provenances. Attribute CategoryRecorded Not recordedTotal RaceWhite42,4770 42,477 RaceBlack8,44608,446 RaceAsian4,36804,368 RaceHispanic or Latino2,18002,180 RaceOther16,48352,038 68,521 Insurance Medicare35,0140 35,014 Insurance Private16,5230 16,523 Insurance Medicaid9,70609,706 Insurance Other12,71152,038 64,749 41 Supplementary Table 4: The fair-model reference and the mitigation methods in dermatology and funduscopy. Each row is one attribute in one modality over nine combinations of encoder and finding. The observed and reference columns report the median across those nine combinations of the subgroup performance difference and of the difference that a perfectly fair model produces at the same subgroup sizes, both as areas under the receiver operating characteristic curve on the 0 to 1 scale. The exceeding column counts the combinations whose observed difference is significantly larger than its reference. Exceedance is tested as the one-sided share of 2,000 simulated fair-model differences that match or exceed the observed difference, under FDR control at 0.05. The achievable column reports the median minimum difference across the nine mitigation methods, holding disease performance within 0.010 of the unmitigated model. Skin type is the Fitzpatrick scale grouped from six levels into three. FDR, false discovery rate. ModalityAttribute Observed Reference Exceeding Achievable Dermatology Age0.0680.0239 of 90.055 Dermatology Skin type0.0720.0370 of 90.052 Dermatology Sex0.0060.0070 of 90.004 FundusAge0.0870.0502 of 90.069 FundusRace0.0670.0381 of 90.061 FundusEthnicity0.0630.0380 of 90.041 FundusSex0.0080.0140 of 90.004 Supplementary Table 5: Every mitigation method, over all combinations on the chest radiograph pool. n is the number of combinations of encoder and finding that the method was run on for that attribute, and the change is the unmitigated AUROC difference minus the mitigated value on the identical combination, so a positive value is a reduction. Values are the median and the interquartile range of that paired change. AUROC, area under the receiver operating characteristic curve; DRO, distributionally robust optimization; IQR, interquartile range; INLP, iterative nullspace projection; LEACE, least-squares concept erasure. MethodRaceAge n Median IQRn Median IQR Group-balanced resampling 1300.003 -0.004 to 0.009 1300.001 -0.002 to 0.005 Reweighing1300.002 -0.002 to 0.005 1300.002 -0.001 to 0.005 Group DRO1300.004 -0.010 to 0.017 1300.000 -0.010 to 0.010 Adversarial removal1300.005 -0.013 to 0.023 1300.005 -0.007 to 0.026 Exponentiated gradient1300.003 -0.005 to 0.010 1300.007 0.001 to 0.013 Operating point shift1300.000 -0.000 to 0.000 1300.000 -0.000 to 0.000 Platt recalibration1300.000 0.000 to 0.000 1300.000 0.000 to 0.000 LEACE1300.012 0.000 to 0.023 1300.007 0.001 to 0.016 INLP1300.010 -0.002 to 0.028 1300.007 0.000 to 0.020 42 Supplementary Table 6: The controlled pretraining matrix, one row per contrast and attribute. n is the number of paired comparisons, each pairing two runs that differ only in the pretraining objective at a fixed backbone, pretraining data composition, seed, and budget, where supervised means label- supervised pretraining. The gain is the median paired difference in the worst-group AUROC and in the equity-scaled AUROC, and the count beside each is the number of significant comparisons under a paired cluster bootstrap over patients under FDR control at α = 0.05. AUROC, area under the receiver operating characteristic curve; FDR, false discovery rate. ContrastAttribute n Worst-group AUROC Equity-scaled AUROC GainSignificantGainSignificant Image-text vs self-supervised Race52 0.04935 0.04932 Image-text vs self-supervised Age52 0.05450 0.04744 Image-text vs self-supervised Sex52 0.04449 0.04249 Image-text vs supervisedRace52 0.04634 0.04634 Image-text vs supervisedAge52 0.05051 0.04747 Image-text vs supervisedSex52 0.04652 0.04551 Self-supervised vs supervised Race52 0.00215 -0.00115 Self-supervised vs supervised Age52 -0.00125 0.00523 Self-supervised vs supervised Sex52 0.00431 0.00531 Supplementary Table 7: Published subgroup performance differences, measured against the fair- model reference. Each row is one difference reported in a published study, audited with step one of FRAME using that study’s own reported values and subgroup counts and none of our models. The studies listed are those meeting all four inclusion criteria, and the modality that each study evaluates is given beside its name. The condition column names the study’s own experimental condition, with the training dataset in parentheses where the study varied it, and N/A where the study reports a single condition. The groups column reports the number of subgroups that the difference ranges over, and the value in parentheses is the number of cases in the smallest of those subgroups on the denominator that the metric uses, meaning positives for TPR, negatives for FPR, and both for AUROC. The counts column records where the denominators come from: R, printed in the article; D, derived by arithmetic on totals and shares printed in the article; P, reconstructed from the article’s pool composition and its stated split fraction, because no per-subgroup count is printed. The reference is the mean of the sampling distribution of the same statistic under exact fairness at those counts, over 2,000 simulated fair models, and the interval reports its 2.5th and 97.5th percentiles, drawn from the Hanley-McNeil distribution for AUROC and the binomial for a rate. p FDR is the one-sided exceedance level, the share of simulated fair models matching or exceeding the reported difference, corrected across claims sharing a metric under FDR control at α = 0.05;† marks a reported difference larger than exact fairness produces at those subgroup sizes. AUROC, area under the receiver operating characteristic curve; CT, computed tomography; DR, diabetic retinopathy; ERM, empirical risk minimization; FDR, false discovery rate; FPR, false positive rate; FRAME, Fair-model Reference And Mechanism Evaluation; N/A, not applicable; PDAC, pancreatic ductal adenocarcinoma; TC, Tyrer-Cuzick; TPR, true positive rate. DatasetTaskAttribute ConditionMetric Groups Counts Reported Referencep FDR Seyyed-Kalantari et al. [1] (chest radiograph) MIMIC-CXRNo finding Race×sex N/AFPR4 (1,152) R0.122 0.017 [0.005,0.035] <0.001 † Glocker et al. [8] (chest radiograph) 43 Supplementary Table 7, continued DatasetTaskAttribute ConditionMetric Groups Counts Reported Referencep FDR CheXpertNo finding RaceN/AAUROC 3 (2,746) R0.010 0.017 [0.003,0.038] 0.864 CheXpertNo finding RaceN/AFPR3 (2,434) R0.030 0.010 [0.002,0.023] 0.006 † CheXpertNo finding RaceN/ATPR3 (312)R0.050 0.029 [0.005,0.065] 0.149 CheXpertEffusionRaceN/AAUROC 3 (2,746) R0.020 0.009 [0.001,0.022] 0.157 CheXpertEffusionRaceN/AFPR3 (1,849) R0.050 0.012 [0.002,0.027] <0.001 † CheXpertEffusionRaceN/ATPR3 (897)R0.070 0.016 [0.003,0.037] 0.002 † MIMIC-CXRNo finding RaceN/AAUROC 3 (2,082) R0.010 0.010 [0.002,0.025] 0.707 MIMIC-CXRNo finding RaceN/AFPR3 (1,423) R0.080 0.011 [0.002,0.027] <0.001 † MIMIC-CXRNo finding RaceN/ATPR3 (659)R0.060 0.017 [0.003,0.041] 0.003 † MIMIC-CXREffusionRaceN/AAUROC 3 (2,082) R0.020 0.010 [0.002,0.023] 0.157 MIMIC-CXREffusionRaceN/AFPR3 (1,504) R0.070 0.011 [0.002,0.026] <0.001 † MIMIC-CXREffusionRaceN/ATPR3 (578)R0.050 0.017 [0.002,0.041] 0.010 † Lotter [47] (chest radiograph) MIMIC-CXRFindingsRaceBaseline (CXP)FPR3 (689)D0.109 0.018 [0.003,0.043] <0.001 † MIMIC-CXRFindingsRaceAugmentation (CXP) FPR3 (689)D0.107 0.017 [0.002,0.044] <0.001 † MIMIC-CXRFindingsRacePer-view (CXP)FPR3 (689)D0.105 0.019 [0.003,0.046] <0.001 † MIMIC-CXRFindingsRaceBaseline (MXR)FPR3 (689)D0.092 0.017 [0.002,0.042] <0.001 † MIMIC-CXRFindingsRaceAugmentation (MXR) FPR3 (689)D0.099 0.017 [0.003,0.042] <0.001 † MIMIC-CXRFindingsRacePer-view (MXR)FPR3 (689)D0.079 0.018 [0.003,0.046] <0.001 † MIMIC-CXRFindingsRaceBaseline (CXP)TPR3 (1,303) D0.075 0.012 [0.002,0.029] 0.002 † MIMIC-CXRFindingsRaceAugmentation (CXP) TPR3 (1,303) D0.095 0.012 [0.002,0.028] 0.002 † MIMIC-CXRFindingsRacePer-view (CXP)TPR3 (1,303) D0.054 0.012 [0.002,0.029] 0.002 † MIMIC-CXRFindingsRaceBaseline (MXR)TPR3 (1,303) D0.077 0.011 [0.002,0.028] 0.002 † MIMIC-CXRFindingsRaceAugmentation (MXR) TPR3 (1,303) D0.097 0.011 [0.002,0.027] 0.002 † MIMIC-CXRFindingsRacePer-view (MXR)TPR3 (1,303) D0.042 0.011 [0.002,0.027] 0.003 † Zong et al. [6] (chest radiograph) CheXpertNo finding RaceERMAUROC 2 (9,716) P0.002 0.008 [0.000,0.022] 0.924 CheXpertNo finding SexERMAUROC 2 (9,057) P0.000 0.008 [0.000,0.022] 0.982 MIMIC-CXRNo finding RaceERMAUROC 2 (14,630) P0.008 0.003 [0.000,0.010] 0.157 MIMIC-CXRNo finding SexERMAUROC 2 (17,743) P0.014 0.003 [0.000,0.009] 0.036 † Daneshjou et al. [50] (dermatology) DDIMalignancy Skin tone DeepDermAUROC 2 (207)R0.110 0.054 [0.002,0.156] 0.246 DDIMalignancy Skin tone DermatologistsAUROC 2 (207)R0.130 0.050 [0.002,0.146] 0.157 DDIMalignancy Skin tone HAM10000AUROC 2 (207)R0.150 0.052 [0.002,0.151] 0.157 DDIMalignancy Skin tone ModelDermAUROC 2 (207)R0.090 0.053 [0.002,0.153] 0.319 DDIMalignancy Skin tone DeepDermFPR2 (159)R0.300 0.045 [0.000,0.126] <0.001 † DDIMalignancy Skin tone DermatologistsFPR2 (159)R0.190 0.042 [0.000,0.113] <0.001 † DDIMalignancy Skin tone HAM10000FPR2 (159)R0.000 0.009 [0.000,0.025] >0.999 DDIMalignancy Skin tone ModelDermFPR2 (159)R0.140 0.035 [0.000,0.094] 0.002 † DDIMalignancy Skin tone DeepDermTPR2 (48)R0.460 0.084 [0.009,0.235] 0.002 † DDIMalignancy Skin tone DermatologistsTPR2 (48)R0.440 0.076 [0.006,0.211] 0.002 † DDIMalignancy Skin tone HAM10000TPR2 (48)R0.040 0.039 [0.001,0.105] 0.510 DDIMalignancy Skin tone ModelDermTPR2 (48)R0.290 0.081 [0.007,0.219] 0.007 † DDI commonMalignancy Skin tone DeepDermAUROC 2 (156)R0.090 0.073 [0.003,0.208] 0.609 DDI commonMalignancy Skin tone DermatologistsAUROC 2 (156)R0.010 0.061 [0.003,0.172] 0.925 DDI commonMalignancy Skin tone HAM10000AUROC 2 (156)R0.130 0.071 [0.003,0.201] 0.307 DDI commonMalignancy Skin tone ModelDermAUROC 2 (156)R0.020 0.069 [0.003,0.196] 0.923 DDI commonMalignancy Skin tone DeepDermFPR2 (140)R0.290 0.047 [0.002,0.131] <0.001 † DDI commonMalignancy Skin tone DermatologistsFPR2 (140)R0.180 0.044 [0.002,0.121] 0.002 † DDI commonMalignancy Skin tone HAM10000FPR2 (140)R0.000 0.009 [0.000,0.027] >0.999 DDI commonMalignancy Skin tone ModelDermFPR2 (140)R0.110 0.037 [0.002,0.101] 0.018 † DDI commonMalignancy Skin tone DeepDermTPR2 (16)R0.400 0.115 [0.003,0.315] 0.010 † DDI commonMalignancy Skin tone DermatologistsTPR2 (16)R0.310 0.078 [0.006,0.214] 0.005 † DDI commonMalignancy Skin tone HAM10000TPR2 (16)R0.040 0.061 [0.006,0.164] 0.593 DDI commonMalignancy Skin tone ModelDermTPR2 (16)R0.200 0.120 [0.006,0.330] 0.228 Burlina et al. [52] (funduscopy) Burlina DR set Referable DR Skin tone N/AFPR2 (100)R0.250 0.052 [0.000,0.140] <0.001 † Burlina DR set Referable DR Skin tone Debiased (DR)FPR2 (100)R0.190 0.050 [0.000,0.140] 0.004 † Burlina DR set Referable DR Skin tone Debiased (retina)FPR2 (100)R0.030 0.042 [0.000,0.120] 0.698 Burlina DR set Referable DR Skin tone N/ATPR2 (100)R0.500 0.055 [0.000,0.160] 0.002 † Burlina DR set Referable DR Skin tone Debiased (DR)TPR2 (100)R0.200 0.052 [0.000,0.150] 0.004 † Burlina DR set Referable DR Skin tone Debiased (retina)TPR2 (100)R0.180 0.055 [0.000,0.160] 0.018 † Vaidya et al. [51] (computational pathology) MGB-breastBreastRaceUni (idc)TPR3 (69)R0.057 0.053 [0.008,0.126] 0.459 MGB-breastBreastRaceUni (ilc)TPR3 (55)R0.175 0.058 [0.009,0.131] 0.005 † MGB-lungLungRaceUni (luad)TPR3 (100)R0.079 0.024 [0.005,0.058] 0.007 † MGB-lungLungRaceUni (lusc)TPR3 (26)R0.073 0.087 [0.011,0.209] 0.577 TCGA-GBM/LGG IDH1RaceUNI (mutant)TPR3 (15)R0.295 0.128 [0.021,0.295] 0.038 † TCGA-GBM/LGG IDH1RaceUNI (wild-type)TPR3 (9)R0.110 0.125 [0.039,0.298] 0.510 Yala et al. [54] (mammography) Emory5-year risk RaceAt TC sensitivityFPR2 (3,422) R0.012 0.005 [0.000,0.015] 0.076 Emory5-year risk RaceAt TC specificityFPR2 (3,422) R0.018 0.007 [0.000,0.019] 0.040 † Emory5-year risk RaceAt TC sensitivityTPR2 (301)R0.023 0.026 [0.000,0.076] 0.510 44 Supplementary Table 7, continued DatasetTaskAttribute ConditionMetric Groups Counts Reported Referencep FDR Emory5-year risk RaceAt TC specificityTPR2 (301)R0.061 0.031 [0.001,0.086] 0.157 Tayebi Arasteh et al. [53] (abdominal CT) PDACPDACSexε = 0.29AUROC 2 (127)R0.026 0.032 [0.001,0.092] 0.781 PDACPDACAgeε = 0.29AUROC 4 (75)R0.047 0.086 [0.024,0.167] 0.924 PDACPDACSexε = 0.54AUROC 2 (127)R0.026 0.024 [0.001,0.069] 0.690 PDACPDACAgeε = 0.54AUROC 4 (75)R0.056 0.064 [0.018,0.123] 0.827 PDACPDACSexε = 1.06AUROC 2 (127)R0.032 0.018 [0.001,0.053] 0.319 PDACPDACAgeε = 1.06AUROC 4 (75)R0.046 0.048 [0.014,0.092] 0.781 PDACPDACSexε = 2.04AUROC 2 (127)R0.031 0.014 [0.001,0.039] 0.185 PDACPDACAgeε = 2.04AUROC 4 (75)R0.025 0.034 [0.010,0.065] 0.864 PDACPDACAgeε = 4.71AUROC 4 (75)R0.021 0.027 [0.008,0.051] 0.827 PDACPDACSexε = 4.71AUROC 2 (127)R0.028 0.011 [0.000,0.031] 0.157 PDACPDACSexε = 5.0AUROC 2 (127)R0.028 0.011 [0.000,0.031] 0.157 PDACPDACAgeε = 5.0AUROC 4 (75)R0.021 0.027 [0.008,0.050] 0.827 PDACPDACSexε = 6.0AUROC 2 (127)R0.027 0.011 [0.000,0.030] 0.157 PDACPDACAgeε = 6.0AUROC 4 (75)R0.022 0.026 [0.008,0.049] 0.827 PDACPDACSexε = 7.0AUROC 2 (127)R0.026 0.011 [0.000,0.030] 0.157 PDACPDACAgeε = 7.0AUROC 4 (75)R0.021 0.026 [0.008,0.049] 0.827 PDACPDACSexε = 8.0AUROC 2 (127)R0.016 0.007 [0.000,0.018] 0.157 PDACPDACAgeε = 8.0AUROC 4 (75)R0.016 0.016 [0.004,0.030] 0.707 PDACPDACSexNon-privateAUROC 2 (127)R0.010 0.004 [0.000,0.011] 0.157 PDACPDACAgeNon-privateAUROC 4 (75)R0.015 0.009 [0.002,0.018] 0.207 Supplementary Table 8: Per-finding evaluable and positive counts over the chest radiograph pool. The evaluable column counts the images whose label for that finding is not blank, so it differs across findings because a blank label excludes an image from that finding alone. The positive column counts the images with the positive code, and prevalence is the positive count as a share of the evaluable count. Counts are over all 650,207 pooled radiographs and not over the test split. FindingEvaluable Positive Prevalence (%) Atelectasis610,32392,97715 Cardiomegaly635,43782,31113 Consolidation616,83727,6174 Edema499,08574,84615 Enlarged cardiomediastinum382,76815,4444 Fracture418,17011,2803 Lung lesion418,01813,0633 Lung opacity415,187 129,36131 Pleural effusion516,408 130,58925 Pleural other417,0055,1681 Pneumonia622,41428,5555 Pneumothorax638,40733,3085 Support devices400,228 164,26241