Paper deep dive
Diagnosing Low-Confidence Patient Profiles via Localized Harmful Covariate Shift Detection
Olivier Lefebvre, Hakima Laribi, Dan Poenaru, Martin Vallières
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Before deploying clinical prediction models, their reliability must be assessed to ensure safe use. Errors may arise from distribution shifts between training and deployment or from heterogeneous performance where the model systematically underperforms for certain patient subgroups. Yet most post-hoc diagnostic methods evaluate reliability only at the population level, checking whether the deployment data resemble the training data and whether the model performs well on average, without accounting for heterogeneity that may cause some patients to be disproportionately affected. We propose a unified diagnostic pipeline that combines both perspectives. MED3pa first discovers interpretable low-confidence patient profiles, and Detectron is then applied within each profile to test whether its degradation is associated with a detectable harmful covariate shift. Across three clinical prediction tasks and three injected subgroup shifts in MIMIC-IV, we find that shift detectability itself is informative. An injected age shift, undetectable at the population level, was nonetheless localized to specific degraded profiles, whereas sex- and race-related shifts were detected broadly and therefore offered little guidance for identifying the most affected patients.
Tags
Links
- Source: https://arxiv.org/abs/2610.19178v1
- Canonical: https://arxiv.org/abs/2610.19178v1
Trouble viewing inline? Open PDF directly →
Full Text
21,973 characters extracted from source content.
Expand or collapse full text
DIAGNOSING LOW-CONFIDENCE PATIENT PROFILES VIA LOCALIZED HARMFUL COVARIATE SHIFT DETEC- TION Accepted to SMASH 2026 Olivier Lefebvre 1 , Hakima Laribi 1 , Dan Poenaru 2,3 , and Martin Valli ` eres 4 1 Department of Computer Science, Universit ́ e de Sherbrooke, Sherbrooke, Canada 2 Department of Pediatric Surgery, Montreal Children’s Hospital, Montreal, Canada 3 Centre for Outcomes Research and Evaluation (CORE), Research Institute of the McGill University Health Centre, Montreal, Canada 4 Medical Physics Unit, Department of Oncology, McGill University, Montreal, Canada ABSTRACT Before deploying clinical prediction models, their reliability must be assessed to ensure safe use. Errors may arise from distribution shifts between training and deployment or from heterogeneous performance where the model systematically underperforms for certain patient subgroups. Yet most post-hoc diagnostic meth- ods evaluate reliability only at the population level, checking whether the deploy- ment data resemble the training data and whether the model performs well on average, without accounting for heterogeneity that may cause some patients to be disproportionately affected. We propose a unified diagnostic pipeline that com- bines both perspectives. MED3pa first discovers interpretable low-confidence pa- tient profiles, and Detectron is then applied within each profile to test whether its degradation is associated with a detectable harmful covariate shift. Across three clinical prediction tasks and three injected subgroup shifts in MIMIC-IV, we find that shift detectability itself is informative. An injected age shift, undetectable at the population level, was nonetheless localized to specific degraded profiles, whereas sex- and race-related shifts were detected broadly and therefore offered little guidance for identifying the most affected patients. 1INTRODUCTION Machine learning (ML) models developed for clinical decision support are typically validated using aggregate performance metrics such as the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity, computed over an entire evaluation cohort. These global met- rics can mask substantial heterogeneity in model reliability across individual patients and across subgroups of patients (Oakden-Rayner et al., 2020; Suresh et al., 2018). MED3pa (Lefebvre et al., 2026) addresses this by estimating, for a fixed and already-trained classifier (BaseModel), a con- fidence score for each individual patient and for profiles of patients, which are groups of patients defined by shared characteristics and with similar confidence levels. This identifies at-risk-of-error profiles with an interpretable tree structure, where the BaseModel exhibits reduced reliability and its predictions should be interpreted with caution. This approach, however, does not indicate why a given profile is affected. Potential causes include irreducible aleatoric variability, epistemic un- certainty due to underrepresentation during training, or a distribution shift between the training and deployment populations (Kendall & Gal, 2017; Lahoti et al., 2023). Another line of work targets the detection of distributional mismatch between the BaseModel’s train- ing and deployment contexts, helping ensure the model operates only within a validated deployment distribution (Silva et al., 2025; Qui ̃ nonero-Candela et al., 2008). The Detectron (Ginsberg et al., 2023) focuses on detecting harmful covariate shifts (HCS), defined as any change in the covariate distribution between the training and deployment contexts that may weaken the generalization of a 1 arXiv:2610.19178v1 [cs.LG] 13 Oct 2026 predictive model. Not every change in the input distribution degrades performance, as models may generalize beyond the exact support of their training distribution. The Detectron therefore ties HCS detection directly to the BaseModel’s behavior, detecting only those shifts in the covariate distribu- tion that affect its generalizability. Furthermore, despite that it is sample-efficient — requiring as few as ten test examples — the Detectron is designed and validated as a whole-population test and cannot determine whether a detected shift is confined only to a specific subgroup of patients. These two methods are complementary as MED3pa identifies where a model underperforms and groups patients with similar predictive confidence using interpretable rules, while the Detectron can test if these localized failures are associated with a harmful distributional shift. A closely related work is SHIFT (Singh et al., 2025), which detects heterogeneous performance drift between train- ing and deployment contexts and infers subgroup structure from the discovered shift patterns, which can lead to discrepancies between the subgroups flagged as shifted and the interpretable rule-defined subgroups (Lakkaraju et al., 2016; Singh et al., 2025). In contrast, our framework characterizes het- erogeneous model reliability across patients, and performs harmful covariate shift detection within the actual rule-defined subgroups, thereby ensuring that the diagnostic test is applied precisely within interpretable regions of the population. In this work, we therefore propose to use MED3pa’s interpretable reliability profiles as localized regions in which Detectron can test for harmful covariate shift. This reveals shifts that are unde- tectable at the population level while distinguishing localized from population-wide degradation. We evaluate this combined framework in a controlled setting where the ground-truth shift is known by design, across three clinical prediction tasks and three injected subgroup shifts. 2METHODS Data and tasks. We used data extracted from the MIMIC-IV (Johnson et al., 2023) database, combining chart and laboratory events, and patient demographics. From this dataset, three prediction tasks were defined: (i) 30-day readmission, whether a patient is readmitted to the hospital within 30 days of a prior discharge; (i) 48-Hour live discharge, whether a patient admitted to the intensive care unit (ICU) will be discharged alive within the following 48 hours; and (i) 48-Hour Mortality, whether a patient admitted to the ICU will die within the following 48 hours. Each task-specific cohort was split in half into separate training and testing sets for the BaseModel using a patient-level split, ensuring that no patient’s data contributed to both sets. The training set was used to develop the BaseModel, and the testing set was used to simulate the detection of covariate shift. For each task and shift condition, a separate BaseModel was trained after removing the corresponding subgroup from the training set, while the testing set remained unchanged. Each BaseModel is a multi-layer perceptron (MLP) trained on its task-specific cohort, the pipeline is however model-agnostic and compatible with any classifier. Injected covariate shift. For each task, we construct three shifted training cohorts, each obtained by completely removing a single subgroup from training only: (i) patients under 50 (age<50); (i) female patients (sex̸=male); and (i) non-White patients (race̸=white). The testing set is identi- cal and representative across conditions, so any performance change is attributable to the induced training-time shift. Profile discovery (MED3pa). For each combination of task and shift-condition, we first split the testing set in half. MED3pa is applied to the BaseModel using the first half, where it learns to estimate a per-patient confidence score that reflects the reliability of the BaseModel for each sample. Using these confidence estimates on that same subset, a regression tree then partitions patients into interpretable, rule-based profiles grouping those with similar confidence scores. The other half of the testing set is finally used to evaluate the BaseModel’s performance within each profile. MED3pa therefore identifies patient profiles defined by interpretable, feature-based rules in which the BaseModel exhibits systematically higher or lower performance. Profile-level HCS detection (Detectron). For each patient profile S identified by MED3pa, we use Detectron to determine whether it is affected by a harmful covariate shift between the Base- Model’s training distribution P and the testing distribution Q, so that a detected shift can be inter- preted as a potential explanation for the performance gap observed in that profile. 2 Detectron identifies harmful shifts by learning an ensemble of constrained disagreement classifiers (CDCs), which are initialized from the BaseModel and trained to preserve the same predictions on most of the training data (P t ), while also maximizing their disagreement with the BaseModel on both the testing distribution and a small reference subset (P r ) drawn from the training distribution (P t ∪ P r = P , P t ∩ P r = ∅). The three subsets P t , P r , and Q allow Detectron to compare the disagreement that CDCs can induce on a controlled reference subset of the training distribution against disagreement on the testing distribution. Here, disagreement refers to the fraction of samples on which a CDC’s prediction differs from that of the BaseModel. If the testing distribution reflects a shift that alters the model’s generalization behavior, the CDCs can more easily disagree with the BaseModel on the testing samples than on the reference subset. Detectron then determines whether this difference in disagreement is statistically significant, thereby detecting harmful covariate shift between the training and testing distributions. For each profile S, we extract P S = x ∈ P : x ∈ S and Q S = x ∈ Q : x ∈ S. P S is split in half: one half, P r S , is held out as a profiled reference set, while the other half is com- bined with all out-of-profile training samples to form the profiled-training set P t S = P \ P r S , giv- ing the CDCs enough data to preserve the BaseModel’s behavior while retaining an in-profile ref- erence for evaluating disagreement. The CDCs’ disagreement on Q S is then compared against their disagreement on P r S : if Q S reflects a shift that alters the BaseModel’s generalization be- havior, the CDCs can more easily disagree with it on Q S than on P r S . A statistically signifi- cant difference indicates HCS within the profile, which is assessed using a one-sided test with a false-positive rate of α = 0.05 for declaring harmful covariate shift. We restrict this analysis to profiles containing at least 5% of the overall population to ensure reliable statistical testing and to limit the number of profile-level tests, as runtime scales directly with the number of profiles. The full pipeline was run on approximately 10,000 samples using roughly 140 CPU cores over one week. Figure 1 summarizes the full pipeline. Code to reproduce all experiments is available at https://github.com/MEDomicsLab/study_MED3pa_SHIFT2026. Figure 1: Overview of the profile-level Detectron workflow. The BaseModel is trained on the full training distributionP and evaluated on the testing distributionQ. MED3pa partitions the population into confidence-based profiles. For each profile S, Detectron is applied by training constrained disagreement classifiers (CDCs) to match the BaseModel on the profiled training subset P t S while inducing disagreement on the profiled testing subset Q S and reference subset P r S . A statistically significant difference in disagreement indicates harmful covariate shift (HCS) within that profile. 3RESULTS Age shift. The injected age shift was not detected globally for any of the three tasks. Profile-level Detectron nevertheless identified HCSs within specific profiles, showing that a shift too weak to be detected at the population level can still be revealed within a localized region of the population. For 48-hour live discharge, no HCS was detected in profiles covering 39.2% of the population. The strongest localized effect occurred in patients with a Glasgow Coma Scale (GCS) Verbal ≤ 3, for whom sensitivity decreased to zero. A separate profile (GCS-Verbal > 3, white blood cell count > 14.88×10 9 /L, hemoglobin > 10.3g/dL) also exhibited an HCS. In this profile, the outcome preva- lence increased from 9.47% in training to 16.35% in testing, a plausible explanation of the detected HCS despite the absence of performance degradation. By contrast, a closely related profile with lower hemoglobin (≤ 10.3g/dL) showed no HCS, no change in prevalence, and no degradation in performance. For 48-hour mortality, only one profile covering 8.76% of the population was identi- fied as affected by an age-related HCS: patients with GCS-Motor ≤ 5 and age ≤ 66. This profile 3 showed a substantial decrease in sensitivity and specificity. No HCS was detected in the remaining profiles, which generally exhibited improved specificity in testing. A similar localized effect was found for 30-day readmission. Although no population-level HCS was detected, Detectron flagged a profile covering 17.68% of the population, characterized by hemoglobin≤ 10.91g/dL, length of stay≤ 5 days, and no specified malignant neoplasm. The mean age in this profile decreased from 73.3 to 67.8 years in between training and testing, consistent with the direction of the injected age shift. This profile also exhibited decreases in AUC (0.80 to 0.64), balanced accuracy (0.72 to 0.58), and sensitivity (0.67 to 0.42), while specificity remained relatively stable (0.77 to 0.75). Thus, al- though the age shift was not harmful at the population level, it was associated with a substantial performance degradation within this specific low-confidence profile. Sex shift. The sex-based injected shift was detected across all profiles for both the 48-hour live dis- charge and 48-hour mortality tasks. Performance degradation between training and testing was consistent across all profiles, with AUC and sensitivity decreasing by approximately 0.05-0.07, while specificity remained stable, indicating that this shift primarily affected the model’s ability to identify positive outcomes. For 30-day readmission, an HCS was detected in all but two profiles (23.8% of the population), both corresponding to patients with prolonged stays (>7 days), but one with low hemoglobin (≤11.6g/dL) and hematocrit (≤30%) while the other with higher hemoglobin (>11.6g/dL). Despite not triggering Detectron, these profiles exhibited the same severe performance collapse as the rest of the population: AUC and sensitivity dropped by roughly 0.25, and specificity by roughly 0.10. This indicates that the BaseModel performs uniformly poorly across all patient profiles under this shift, with no subgroup showing resilience. Race shift. Race-related HCS was detected in every profile across all three tasks, accompanied by consistent decreases in AUC and sensitivity across profiles, while specificity remained compara- tively stable. This shows that this shift impacts the population similarly to the sex-related shift. 4DISCUSSION & CONCLUSIONS Our results demonstrate the complementarity of MED3pa and Detectron for diagnosing heteroge- neous model reliability. Applying Detectron within MED3pa profiles revealed harmful shifts not apparent at the population level. In particular, the age shift (patients under 50, ∼18% of the co- hort) was undetected globally yet identified within specific profiles showing substantial performance degradation. In contrast, sex- and race-related shifts affect larger subgroups (∼46% female,∼33% non-White), producing broader effects across most or all profiles that were less informative for lo- calizing the most affected patients. Importantly, a detected HCS does not by itself explain the observed performance degrada- tion. Some profiles showed performance decreases without a corresponding HCS (e.g., the two hemoglobin/hematocrit profiles under sex shift for 30-day readmission), while other detected HCS that did not necessarily translate into performance differences (GCS-Verbal > 3, white blood cell count > 14.88×10 9 /L, hemoglobin > 10.3g/dL under age shift for 48-hour live discharge). Fur- thermore, because Detectron is applied independently within each profile at α = 0.05, no correction for multiple comparisons is applied, and results should be interpreted as exploratory signals rather than confirmatory findings. Profiles showing a detected HCS and performance degradation represent particularly strong candidates for further investigation or targeted model adaptation. This study is a proof-of-concept with several limitations. The experiments rely on controlled single- axis subgroup shifts injected into a single dataset, which simplifies the evaluation but does not capture the diversity or complexity of naturally occurring deployment shifts. In addition, we only evaluate shifts that affect relatively large portions of the population; future work should examine whether profile-level HCS detection remains effective when shifts impact much smaller subgroups. Broader validation on additional clinical datasets and under more realistic temporal, institutional, and demographic shifts is also needed. Several pipeline design choices, including the reference- set ratio and the minimum profile size, were fixed a priori and could be optimized in future work. The pipeline’s diagnostic accuracy, its profile-level false-positive rate, and how it compares with subgroup-monitoring methods like SHIFT (Singh et al., 2025) remain to be evaluated. Overall, combining MED3pa with profile-level Detectron extends reliability analysis from identify- ing where a model is less reliable to investigating whether a harmful distribution shift may explain why it is less reliable in those patient profiles. 4 REFERENCES Tom Ginsberg, Zhongyuan Liang, and Rahul G Krishnan. A learning based hypothesis test for harmful covariate shift. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=rdfgqiwz7lZ. Alistair E. W. Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J. Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, Li-wei H. Lehman, Leo A. Celi, and Roger G. Mark. MIMIC-IV, a freely accessible electronic health record dataset. Scientific Data, 10(1):1, January 2023. ISSN 2052-4463. doi: 10.1038/s41597-022-01899-x. URL https: //w.nature.com/articles/s41597-022-01899-x. Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Com- puter Vision? In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Cur- ran Associates, Inc., 2017. URL https://proceedings.neurips.c/paper_files/ paper/2017/file/2650d6089a6d640c5e85b2b88265dc2b-Paper.pdf. Preethi Lahoti, Krishna Gummadi, and Gerhard Weikum. Responsible model deployment via model-agnostic uncertainty learning. Machine Learning, 112(3):939–970, March 2023. ISSN 1573-0565.doi: 10.1007/s10994-022-06248-y.URL https://doi.org/10.1007/ s10994-022-06248-y. Himabindu Lakkaraju, Stephen H. Bach, and Jure Leskovec. Interpretable Decision Sets: A Joint Framework for Description and Prediction. In Proceedings of the 22nd ACM SIGKDD Interna- tional Conference on Knowledge Discovery and Data Mining, KDD ’16, p. 1675–1684, New York, NY, USA, August 2016. Association for Computing Machinery. ISBN 978-1-4503-4232-2. doi: 10.1145/2939672.2939874. URL https://dl.acm.org/doi/10.1145/2939672. 2939874. Olivier Lefebvre, F ́ elix Camirand Lemyre, Jean-Franc ̧ois Ethier, Lyna Hiba Chikouche, Ludmila Amriou, Dan Poenaru, and Martin Valli ` eres.Predictive performance precision analysis in medicine: identification of low-confidence predictions at patient and profile levels (MED3pa I). Journal of the American Medical Informatics Association, 33(6):1100–1111, June 2026. ISSN 1527-974X. doi: 10.1093/jamia/ocag034. URL https://doi.org/10.1093/jamia/ ocag034. Luke Oakden-Rayner, Jared Dunnmon, Gustavo Carneiro, and Christopher Re. Hidden stratification causes clinically meaningful failures in machine learning for medical imaging. In Proceedings of the ACM Conference on Health, Inference, and Learning, CHIL ’20, p. 151–159, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450370462. doi: 10.1145/ 3368555.3384468. URL https://doi.org/10.1145/3368555.3384468. Joaquin Qui ̃ nonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D. Lawrence. Dataset Shift in Machine Learning. The MIT Press, December 2008. ISBN 978-0-262-25510- 3. doi: 10.7551/mitpress/9780262170055.001.0001. URL https://doi.org/10.7551/ mitpress/9780262170055.001.0001. Gabriel Ferreira dos Santos Silva, Fabiano Novaes Barcellos Filho, Roberta Moreira Wichmann, Francisco Costa da Silva Junior, and Alexandre Dias Porto Chiavegatto Filho. Strategies for detecting and mitigating dataset shift in machine learning for health predictions: A system- atic review. Journal of Biomedical Informatics, 170:104902, October 2025. ISSN 1532-0464. doi: 10.1016/j.jbi.2025.104902. URL https://w.sciencedirect.com/science/ article/pii/S1532046425001315. Harvineet Singh, Fan Xia, Alexej Gossmann, Andrew Chuang, Julian C. Hong, and Jean Feng. “Who experiences large model decay and why?” A Hierarchical Framework for Diagnosing Het- erogeneous Performance Drift. Proceedings of machine learning research, 267:55757–55787, July 2025. ISSN 2640-3498. URL https://pmc.ncbi.nlm.nih.gov/articles/ PMC12747154/. 5 Harini Suresh, Jen J. Gong, and John V. Guttag. Learning tasks for multitask learning: Heterogenous patient populations in the icu. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’18, p. 802–810, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781450355520. doi: 10.1145/3219819.3219930. URL https://doi.org/10.1145/3219819.3219930. 6