Paper deep dive
Conformal Prediction for Risk-Controlled Medical Entity Extraction Across Clinical Domains
Manil Shrestha, Edward Kim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 7/20/2026, 3:45:40 AM
Summary
This paper introduces a conformal prediction framework to control the false discovery rate (FDR) in Large Language Model (LLM)-based medical entity extraction. The study evaluates two clinical domains: structured FDA drug labels and free-text radiology reports (MIMIC-CXR). Key findings reveal a reversal in model miscalibration: LLMs are underconfident on structured FDA labels but overconfident on free-text radiology reports. The framework applies risk-controlling prediction sets to set dynamic confidence thresholds, achieving target coverage with manageable rejection rates, demonstrating that calibration is domain-dependent rather than a global model property.
Entities (8)
Relation Signals (6)
GPT-4.1 → usedfor → FDA Drug Label Extraction
confidence 98% · First, we extract structured entities from 1,000 FDA drug labels across eight sections using GPT-4.1
llama-4-maverick → usedfor → Radiology Report Extraction
confidence 97% · Second, we extract radiological entities from MIMIC-CXR reports using the RadGraph schema with GPT-4.1 and Llama-4-Maverick
Conformal Prediction → controls → False Discovery Rate
confidence 96% · provides finite-sample false discovery rate (FDR) guarantees for LLM-based extraction
Conformal Prediction → appliedto → Medical Entity Extraction
confidence 95% · We present a conformal prediction framework that provides finite-sample coverage guarantees for LLM-based extraction across two clinical domains.
FDA Drug Labels → exhibits → Underconfidence
confidence 93% · on well-structured FDA labels, models are underconfident
MIMIC-CXR → exhibits → Overconfidence
confidence 93% · on free-text radiology reports, models are overconfident
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large Language Models (LLMs) are increasingly used for medical entity extraction, yet their confidence scores are often miscalibrated, limiting safe deployment in clinical settings. We present a conformal prediction framework that provides finite-sample coverage guarantees for LLM-based extraction across two clinical domains. First, we extract structured entities from 1,000 FDA drug labels across eight sections using GPT-4.1, verified via FactScore-based atomic statement evaluation (97.7\% accuracy over 128,906 entities). Second, we extract radiological entities from MIMIC-CXR reports using the RadGraph schema with GPT-4.1 and Llama-4-Maverick, evaluated against physician annotations (entity F1: 0.81 to 0.84). Our central finding is that miscalibration direction reverses across domains: on well-structured FDA labels, models are underconfident, requiring modest conformal thresholds ($\tau \approx 0.06$), while on free-text radiology reports, models are overconfident, demanding strict thresholds ($\tau$ up to 0.99). Despite this heterogeneity, conformal prediction achieves target coverage ($\geq 90\%$) in both settings with manageable rejection rates (9--13\%). These results demonstrate that calibration is not a global model property but depends on document structure, extraction category, and model architecture, motivating domain-specific conformal calibration for safe clinical deployment.
Tags
Links
- Source: https://arxiv.org/abs/2603.00924v2
- Canonical: https://arxiv.org/abs/2603.00924v2
Trouble viewing inline? Open PDF directly →
Full Text
31,272 characters extracted from source content.
Expand or collapse full text
11institutetext: Department of Computer Science, Drexel University, Philadelphia, USA 11email: ms5267, ek826@drexel.edu Conformal Prediction for Risk-Controlled Medical Entity Extraction Across Clinical Domains Manil Shrestha Edward Kim Abstract Large Language Models (LLMs) are increasingly used for medical entity extraction, yet their confidence scores are often miscalibrated, limiting safe deployment in clinical settings. We present a conformal prediction framework based on risk-controlling prediction sets [3] that provides finite-sample false discovery rate (FDR) guarantees for LLM-based extraction across two clinical domains. First, we extract structured entities from 1,000 FDA drug labels across eight sections using GPT-4.1, verified via FactScore-based atomic statement evaluation (97.7% accuracy over 128,906 entities). Second, we extract radiological entities from MIMIC-CXR reports using the RadGraph schema with GPT-4.1 and Llama-4-Maverick, evaluated against physician annotations (entity F1: 0.83–0.84). Our central finding is that miscalibration direction reverses across domains: on well-structured FDA labels, models are underconfident, and the global baseline FDR of 2.3% trivially satisfies α=0.05α=0.05, though per-section analysis reveals that three sections require 41–100% rejection. On free-text radiology reports, models are overconfident, and FDR control at α=0.10α=0.10 produces sharply different outcomes across models: Llama-4-Maverick rejects 19.6% of extractions while GPT-4.1 rejects 59.3%, with both models rejecting all uncertain observations. Sweep analysis across α values reveals sharp transitions in acceptance behavior that expose the baseline error structure of each domain. These results demonstrate that calibration is not a global model property but depends on document structure, extraction category, and model architecture, motivating domain-specific conformal calibration for safe clinical deployment. 1 Introduction Extracting structured entities from unstructured medical text is critical for drug safety surveillance, clinical coding, and biomedical research. Large Language Models (LLMs) have shown strong performance on these tasks [13, 20], achieving near-expert accuracy on entity extraction from electronic health records (EHRs), drug labels, and radiology reports. However, deploying LLMs in clinical settings requires not only high accuracy but also reliable uncertainty quantification: a model that assigns high confidence to incorrect extractions can lead to silent errors propagating through clinical decision pipelines. LLM softmax probabilities are poorly calibrated in general [8, 9]. Models can be systematically overconfident, assigning near-certain probabilities to incorrect predictions, or underconfident, assigning low probabilities to correct ones. Standard post-hoc calibration techniques such as temperature scaling [9] require held-out validation data and provide no formal coverage guarantees. Moreover, calibration quality varies significantly across tasks, domains, and entity types, making it difficult to establish a single reliability threshold. Conformal prediction (CP) addresses these challenges by converting raw model scores into accept/reject decisions with finite-sample guarantees under only an exchangeability assumption [2]. Bates et al. [3] extended this framework to risk-controlling prediction sets, which provide guarantees on the expected loss under arbitrary monotone loss functions, including false discovery rate (FDR) control. Rather than requiring well-calibrated probabilities, CP adapts its acceptance threshold to the empirical distribution of scores, automatically becoming more conservative when models are poorly calibrated and more permissive when they are well-calibrated. This makes CP particularly appealing for clinical deployment, where regulatory requirements demand provable reliability bounds. In this work, we apply FDR-controlling conformal prediction to LLM-based medical entity extraction across two fundamentally different clinical domains: (1) structured FDA drug labels, where we extract entities from eight standardized sections, and (2) free-text radiology reports, where we extract entities and relations following the RadGraph schema [10]. These domains differ in document structure, vocabulary, and annotation methodology, allowing us to study how calibration behavior varies across clinical contexts. Our contributions are: (1) a conformal prediction framework for medical entity extraction that provides finite-sample FDR guarantees across heterogeneous clinical domains, ensuring that the proportion of accepted but incorrect extractions is bounded; (2) the empirical finding that LLM miscalibration direction reverses across domains, with models underconfident on structured FDA labels and overconfident on free-text radiology reports; (3) sweep analysis across miscoverage rates revealing sharp transitions in acceptance behavior that expose the baseline error structure of each domain and the critical difference between global and per-category thresholds; (4) cross-model comparison showing that FDR-controlling thresholds depend not only on extraction accuracy but on how well a model’s confidence scores discriminate between correct and incorrect extractions. 2 Related Work LLM-Based Medical Entity Extraction. LLMs have been applied to a range of clinical extraction tasks. Kim et al. [13] demonstrated structured extraction from medical knowledge bases using GPT-4, while Wu et al. [20, 19] developed frameworks for extracting adverse events from FDA drug labeling documents using LLMs with regulatory-grade evaluation. For radiology reports, the RadGraph dataset [10] provides physician-annotated entities and relations from MIMIC-CXR chest X-rays [11], enabling systematic evaluation of extraction systems. Confidence and Calibration in LLMs. Geng et al. [8] survey confidence estimation methods for LLMs, highlighting that token-level log-probabilities, while informative, do not reliably indicate factual correctness. Guo et al. [9] showed that modern neural networks are systematically miscalibrated, a finding that extends to LLMs. Laban et al. [14] demonstrated that LLM confidence can be fragile under adversarial probing. These findings motivate distribution-free approaches like conformal prediction. Conformal Prediction in Medicine and NLP. Conformal prediction provides distribution-free coverage guarantees under exchangeability [2]. Bates et al. [3] extended this to risk-controlling prediction sets, enabling control of arbitrary monotone loss functions including FDR. In medicine, CP has been applied to skin lesion classification [6], genomic analysis [16], medical coding [17], and EHR mining [7]. Vazquez and Facelli [18] survey CP applications across clinical domains. Campos et al. [4] provide a comprehensive survey of CP for NLP tasks. In our prior work [12], we applied CP to LLM-based EHR extraction across 10,000 visits, validating thresholds against expert consensus using the VeriFact framework [5]. The present work extends this approach to FDA labels and radiology reports, revealing domain-dependent calibration behavior not observed in the EHR setting. 3 Methods We apply FDR-controlling conformal prediction to two extraction tasks using GPT-4.1 [1] and Llama 4 Maverick (17B parameters, 128 experts) [15]. Both models return per-token log-probabilities, enabling span-level confidence estimation. The pipeline has four steps. 3.1 Step 1: Extraction with Token Confidence FDA Drug Labels. We extract entities from 1,000 FDA drug labels across eight standardized sections: indications and usage, adverse reactions, drug interactions, contraindications, warnings, pregnancy, pediatric use, and geriatric use. Each drug label is processed by GPT-4.1, which extracts entities (drug names, conditions, adverse events, anatomical structures, etc.) along with per-token log-probabilities. The source dataset comprises 2,437 asthma-indicated drugs from the FDA’s openFDA API. Radiology Reports. We extract entities and relations from 100 MIMIC-CXR chest X-ray reports [11] following the RadGraph schema [10]. The schema defines four entity categories: anatomy definitely present (anat-dp), observation definitely present (obs-dp), observation definitely absent (obs-da), and observation uncertain (obs-u). Relations include located_at, modify, and suggestive_of. We evaluate both GPT-4.1 and Llama-4-Maverick with 5-shot prompting (examples drawn from the RadGraph dev set) and zero-shot prompting. Span Confidence. For each entity span of mem_e tokens with softmax probabilities pe,tp_e,t, we compute a span-level confidence via the geometric mean: p^e=exp(1me∑t=1melogpe,t) p_e= \! ( 1m_e _t=1^m_e p_e,t ) (1) This aggregation is motivated by the observation that span correctness depends on all constituent tokens: a single low-probability token (e.g., a misspelled drug name or wrong anatomical modifier) typically indicates an extraction error. The geometric mean is more sensitive to such outlier tokens than the arithmetic mean, providing a more informative confidence signal. 3.2 Step 2: Verification FDA Labels. Verification uses an LLM-as-a-judge approach following the VeriFact framework [5]. GPT-5-mini evaluates each extracted entity against the source FDA label text, producing a fact-score on a 0–3 scale: 0 (hallucinated/unsupported), 1 (partially supported), 2 (mostly supported), and 3 (fully verified). Only score 3 counts as correct (ye=1y_e=1). This strict threshold ensures that conformal calibration is performed against a high-quality ground truth. Over 128,906 entities, 97.7% received a fact-score of 3. Radiology Reports. For RadGraph, we evaluate against physician-annotated gold entities from the RadGraph test set (100 reports, 2,763 gold entities). An extracted entity is marked correct (ye=1y_e=1) if it exactly matches a gold entity in both text span and label. This exact-match criterion is stricter than token-overlap matching but ensures clinically meaningful evaluation. 3.3 Step 3: Nonconformity Score We define the nonconformity score as the logit of span confidence: se=logit(p^e)=log(p^e1−p^e)s_e=logit( p_e)= \! ( p_e1- p_e ) (2) This transformation maps confidence from (0,1)(0,1) to (−∞,+∞)(-∞,+∞), spreading out the high-confidence region [0.9,1.0)[0.9,1.0) where most entities cluster. Higher scores indicate higher confidence. The logit transformation is standard in conformal prediction for probability-valued scores [2]. 3.4 Step 4: FDR-Controlling Conformal Calibration We partition verified entities into calibration (50%) and test (50%) sets using a fixed random seed. Rather than targeting marginal coverage, we calibrate thresholds to control the false discovery rate among accepted extractions, following the risk-controlling prediction sets framework of Bates et al. [3]. For each domain d and extraction category c, we select the threshold τd,c _d,c as the smallest value such that the empirical FDR on the calibration set does not exceed α: τd,c=inft:∑e∈d,ccal(ye=0)⋅(se≥t)max(1,∑e∈d,ccal(se≥t))≤α _d,c= \t: _e ^cal_d,c1(y_e=0)·1(s_e≥ t) \! (1,\; _e ^cal_d,c1(s_e≥ t) )≤α \ (3) Entities with se≥τd,cs_e≥ _d,c are accepted; the rest are flagged for human review. This guarantees that the expected proportion of accepted but incorrect extractions is bounded by α: [1|Ad,c|∑e∈Ad,c(ye=0)]≤αE [ 1|A_d,c| _e∈ A_d,c1(y_e=0) ]≤α (4) where Ad,c=e:se≥τd,cA_d,c=\e:s_e≥ _d,c\ is the accepted set. We evaluate at α=0.05α=0.05 for FDA labels and α=0.10α=0.10 for RadGraph. For both domains, we additionally conduct sweep analysis across multiple α values to characterize the precision-rejection tradeoff. For RadGraph, we compute both global thresholds and per-category thresholds τc _c, since calibration behavior varies substantially across entity types. 4 Results and Discussion 4.1 FDA Label Extraction We extracted 128,906 entities from 1,000 FDA drug labels across eight sections using GPT-4.1, of which 110,664 have both confidence scores and fact-score verification. Overall fact-score accuracy is 97.7% (108,107/110,664 entities scored as fully verified). Calibration Analysis. Calibration curves (Figure 1) reveal that GPT-4.1 is underconfident on FDA labels: predicted confidence p p consistently falls below empirical accuracy. Expected Calibration Error (ECE) ranges from 0.012 (Adverse Reactions, n=63,442n=63,442) to 0.214 (Pediatric Use, n=1,954n=1,954). The underconfidence pattern is consistent across most sections, with the model assigning conservative probabilities to entities it extracts correctly. Pediatric Use is a notable exception: its calibration curve falls below the diagonal, indicating overconfidence, with the highest ECE among all sections. This contrasts with the overconfidence typically observed in free-text generation tasks [8] and suggests that the structured, formulaic nature of FDA labels makes extraction easier than the model’s confidence suggests for most sections. FDR Sweep Analysis. Table 1 presents a sweep across α values using a single global threshold. The global baseline FDR across all 110,664 entities is approximately 2.3%. As a result, any α≥0.03α≥ 0.03 accepts all extractions, since the baseline error rate already satisfies the FDR target. At α=0.01α=0.01, no threshold can bring FDR below the target, resulting in full rejection. Table 1: FDR-controlled conformal prediction sweep across α for FDA drug label extraction (GPT-4.1, n=110,664n=110,664 entities, global threshold). At α≥0.03α≥ 0.03 the global baseline FDR (2.3%) is already below target, so nearly all entities are accepted; per-section thresholds (Table 2) reveal the heterogeneity masked by global pooling. α τc _c p^min p_ Rej.% Cov. Prec. 0.01 ∞ – 100.0 0.000 – 0.03 −0.49-0.49 .380 0.0 1.000 0.977 0.05 −0.49-0.49 .380 0.0 1.000 0.977 0.10 −0.49-0.49 .380 0.0 1.000 0.977 0.15 −0.49-0.49 .380 0.0 1.000 0.977 0.20 −0.49-0.49 .380 0.0 1.000 0.977 Per-Section FDR Results. The global sweep masks substantial per-section heterogeneity (Table 2). At α=0.05α=0.05, high-accuracy sections such as Adverse Reactions and Indications & Usage accept all entities with 0% rejection, while sections with higher baseline error rates require aggressive filtering: Drug Interactions rejects 59.8% of extractions, Contraindications rejects 41.5%, and Pediatric Use rejects 100%, meaning that no threshold can bring its FDR below 0.05. Pediatric Use is the clear outlier with baseline accuracy of only 74.3% and the highest ECE (0.214); unlike most other sections where the model is underconfident, Pediatric Use exhibits overconfidence, and FDR control correctly identifies it as unsuitable for automated acceptance at this error tolerance. Table 2: Calibration metrics and FDR-controlled conformal prediction for FDA drug label extraction with GPT-4.1 (α=0.05α=0.05, n=1,000n=1,000 drugs). τc _c is the per-section FDR threshold in log-odds; p^min=σ(τc) p_ =σ( _c) is the equivalent confidence cutoff. FDR control guarantees precision ≥1−α≥ 1-α among accepted entities. Warnings section omitted (no valid entities). Section # Entities Brier↓ ECE↓ FactS. τc _c p^min p_ Rej.% Cov. Prec. Adverse Reactions 63,442 0.013 0.012 2.98 −0.09-0.09 .479 0.0 1.000 0.987 Indications & Usage 34,895 0.022 0.018 2.94 0.850.85 .701 0.0 1.000 0.979 Drug Interactions 7,251 0.051 0.055 2.92 10.5210.52 >.999>.999 59.8 0.400 0.948 Contraindications 2,109 0.048 0.041 2.90 7.657.65 >.999>.999 41.5 0.590 0.966 Pregnancy 545 0.005 0.004 2.99 2.712.71 .938 0.4 0.996 0.996 Pediatric Use 1,954 0.236 0.214 2.52 ∞ – 100.0 0.000 – Geriatric Use 468 0.000 0.007 3.00 3.063.06 .955 0.0 1.000 1.000 All sections 110,664 0.023 0.018 2.95 −0.49-0.49 .380 0.0 1.000 0.977 Figure 1: Calibration curves for four sections from FDA label extraction with GPT-4.1 (n=1,000n=1,000 drugs). Predicted span confidence p p (geometric mean of token probabilities) versus empirical accuracy. The model is systematically underconfident across most sections: curves lie above the diagonal. Pediatric Use is an exception, exhibiting overconfidence. ECE ranges from 0.012 (Adverse Reactions) to 0.214 (Pediatric Use). 4.2 RadGraph Entity Extraction We evaluated extraction on 100 MIMIC-CXR test reports against RadGraph gold annotations using both GPT-4.1 and Llama-4-Maverick. Extraction Quality. With 5-shot prompting, GPT-4.1 achieves entity F1 of 0.826 (precision 0.798, recall 0.856) and relation F1 of 0.502. Llama-4-Maverick achieves entity F1 of 0.841 (precision 0.865, recall 0.818) and relation F1 of 0.490 (Table 3). The two models exhibit complementary strengths: GPT-4.1 has higher recall while Llama-4-Maverick has higher precision. Relation extraction remains challenging for both models at around F1 = 0.50. Zero-shot performance degrades substantially for both models, with entity F1 dropping by 8.0 percentage points for GPT-4.1 and 11.7 points for Llama-4-Maverick, and relation F1 dropping by over 22 points for both, indicating that the RadGraph schema’s labeling conventions require in-context demonstration. Table 3: RadGraph entity and relation extraction on 100 test reports. Few-shot prompting (5 examples from dev set) substantially improves both entity and relation extraction compared to zero-shot. Entity Relation Model Setting Prec. Rec. F1 Prec. Rec. F1 GPT-4.1 5-shot .798 .856 .826 .484 .521 .502 GPT-4.1 0-shot .787 .710 .746 .301 .252 .274 Llama-4-Maverick 5-shot .865 .818 .841 .526 .458 .490 Llama-4-Maverick 0-shot .783 .674 .724 .228 .227 .228 Calibration Analysis. In stark contrast to FDA labels, both models are overconfident on radiology reports (Figure 2): they assign near-certain probabilities to incorrect extractions. Llama-4-Maverick achieves lower ECE than GPT-4.1 across most categories (overall 0.085 vs. 0.147), indicating better-calibrated confidence despite similar extraction quality. OBS-U (uncertain observations) remains the most challenging category for both models, with ECE exceeding 0.40, reflecting the inherent ambiguity of hedging language in radiology reports (e.g., “cannot exclude,” “possibly representing”). Figure 2: Per-label calibration curves for GPT-4.1 and Llama-4-Maverick (5-shot, 100 CXR reports [11]). Predicted span confidence p p versus empirical accuracy across four entity categories. Both models are overconfident (curves below diagonal). Llama-4-Maverick achieves lower ECE overall (0.085 vs. 0.147), while OBS-U remains poorly calibrated for both models. FDR Sweep Analysis. Table 4 presents a sweep across α values using global thresholds. At α=0.05α=0.05, both models reject all extractions: the baseline error rate of 15–20% makes it impossible to guarantee that fewer than 5% of accepted extractions are incorrect. A sharp transition occurs at α=0.10α=0.10, where Llama-4-Maverick accepts 80.4% of extractions (19.6% rejection) while GPT-4.1 accepts only 40.7% (59.3% rejection). This difference reflects Llama-4-Maverick’s better calibration: its confidence scores separate correct from incorrect extractions more effectively, allowing the FDR threshold to be satisfied with less aggressive filtering. At α≥0.15α≥ 0.15, Llama-4-Maverick accepts nearly all extractions while GPT-4.1 continues to require substantial rejection until α=0.25α=0.25. Table 4: FDR-controlled conformal prediction sweep across α for RadGraph entity extraction (global threshold, 5-shot). At α<0.10α<0.10, neither model can guarantee the target precision, resulting in full rejection. The sharp transition at α=0.10α=0.10 reflects the baseline error rate (∼15−20% 15-20\%); Llama-4-Maverick’s better calibration yields a lower threshold and less rejection at the same α. GPT-4.1 Llama-4-Maverick α Rej.% Cov. Prec. Rej.% Cov. Prec. 0.05 100.0 0.000 – 100.0 0.000 – 0.10 59.3 0.445 0.876 19.6 0.824 0.870 0.15 41.7 0.617 0.849 0.2 0.998 0.849 0.20 6.8 0.936 0.805 0.2 0.998 0.849 0.25 0.1 0.999 0.802 0.2 0.998 0.849 Per-Category FDR Results. Per-category results at α=0.10α=0.10 (Table 5) reveal further heterogeneity. Both models reject 100% of OBS-U extractions, confirming that uncertain observations cannot be reliably extracted by either model at this error tolerance. For the remaining categories, the two models exhibit strikingly different patterns. Llama-4-Maverick accepts nearly all OBS-DP and OBS-DA extractions (0.2% and 0.0% rejection), while GPT-4.1 rejects 52.1% and 62.6% of those same categories. For ANAT-DP, both models require strict thresholds (p^min>0.999 p_ >0.999), but GPT-4.1 rejects a larger fraction (57.3% vs. 41.4%). These cross-model differences arise despite similar extraction quality and demonstrate that FDR-controlling thresholds depend not only on accuracy but on how well a model’s confidence scores discriminate between correct and incorrect extractions. Table 5: Per-category entity extraction and FDR-controlled conformal prediction on 100 RadGraph test reports (α=0.10α=0.10, 5-shot). τc _c is the per-category FDR threshold in log-odds; p^min=σ(τc) p_ =σ( _c) is the equivalent confidence cutoff. FDR control guarantees precision ≥1−α≥ 1-α among accepted entities. GPT-4.1 Llama-4-Maverick Category Prec. Rec. F1 ECE↓ τc _c p^min p_ Rej.% Prec. Rec. F1 ECE↓ τc _c p^min p_ Rej.% anat-dp .834 .933 .880 .128 9.929.92 >.999>.999 57.3 .865 .893 .879 .085 11.1111.11 >.999>.999 41.4 obs-dp .827 .769 .797 .140 6.456.45 .998 52.1 .905 .724 .804 .064 0.000.00 .500 0.2 obs-da .833 .914 .871 .102 13.1213.12 >.999>.999 62.6 .902 .900 .901 .051 0.000.00 .500 0.0 obs-u .305 .648 .415 .525 ∞ – 100.0 .444 .580 .502 .396 ∞ – 100.0 4.3 Cross-Domain Calibration Reversal Our most striking finding is the reversal of miscalibration direction across domains. On FDA labels, GPT-4.1 is systematically underconfident across most sections (ECE 0.004–0.055, curves above the diagonal in Figure 1), with Pediatric Use as the exception showing overconfidence (ECE 0.214). On radiology reports, the same model family is overconfident (ECE 0.102–0.525, curves below the diagonal in Figure 2). We attribute this reversal to differences in document structure. FDA labels follow standardized formatting with regulatory language, making entity boundaries clear and extraction straightforward. The model assigns conservative probabilities, perhaps because the structured format resembles training data where careful hedging is rewarded. Radiology reports, by contrast, use terse, variable-format clinical shorthand with implicit negation and hedging, making extraction genuinely harder. The model assigns high confidence to entities that appear linguistically plausible but are semantically incorrect (e.g., labeling a finding as “definitely present” when the report says “cannot be excluded”). The consequences for FDR control are dramatic. On FDA labels, the global baseline FDR of 2.3% trivially satisfies α=0.05α=0.05, though per-section analysis reveals three sections requiring 41–100% rejection. On RadGraph, neither model can satisfy α=0.05α=0.05 at all, and even at α=0.10α=0.10 the two models require very different levels of filtering (19.6% vs. 59.3% global rejection). This finding has important implications for clinical deployment: a single calibration strategy cannot work across document types, and FDR control exposes heterogeneity that coverage-based approaches and global thresholds obscure. 5 Conclusion We have presented a conformal prediction framework for LLM-based medical entity extraction that provides finite-sample FDR guarantees across two clinical domains with fundamentally different calibration characteristics. Our experiments demonstrate that: (1) LLM calibration direction reverses between structured FDA labels (underconfident) and free-text radiology reports (overconfident); (2) FDR-controlling conformal prediction adapts automatically to both regimes, with per-category thresholds revealing heterogeneity that global pooling masks; (3) sweep analysis across α values exposes sharp transitions in acceptance behavior tied to the baseline error structure of each domain; (4) cross-model comparison shows that FDR-controlling thresholds depend on confidence discriminability, not just extraction accuracy, with Llama-4-Maverick requiring far less rejection than GPT-4.1 at the same α on RadGraph despite similar F1. Limitations. Our framework requires access to token-level log-probabilities, which some frontier models do not currently expose. The FDA label verification relies on an LLM-as-a-judge (GPT-5-mini), which may introduce systematic biases in the ground truth. The RadGraph evaluation is limited to 100 test reports, which constrains the statistical power of per-category conformal analysis, particularly for rare categories like OBS-U (n=88n=88 gold entities). Additionally, our approach assumes exchangeability within each domain, which may be violated if certain drug types or report formats are systematically harder. Future Work. Several directions remain open. First, extending conformal guarantees to black-box models without log-probabilities via surrogate confidence estimation or verbalized uncertainty. Second, comparison with post-hoc calibration baselines (temperature scaling, Platt scaling) to quantify the advantage of conformal prediction’s formal guarantees. Third, deploying the framework in a clinical workflow study to measure the impact of conformal filtering on downstream clinical decision-making. Fourth, exploring adaptive conformal prediction methods that can handle distribution shift across time as drug labels and reporting practices evolve. 00footnotetext: Declaration of AI Assistance: The authors used GitHub Copilot for writing or drafting manuscript content, and refinement or formatting of code reported in the submitted manuscript. After using this technology, the authors reviewed the results and take full responsibility for the contents of the manuscript. References [1] J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. (2023) Gpt-4 technical report. arXiv preprint arXiv:2303.08774. Cited by: §3. [2] A. N. Angelopoulos and S. Bates (2023) Conformal prediction: a gentle introduction. Foundations and Trends in Machine Learning 16 (4), p. 494–591. Cited by: §1, §2, §3.3. [3] S. Bates, A. Angelopoulos, L. Lei, J. Malik, and M. Jordan (2021) Distribution-free, risk-controlling prediction sets. Journal of the ACM (JACM) 68 (6), p. 1–34. Cited by: §1, §2, §3.4. [4] M. Campos, A. Farinhas, C. Zerva, M. A. Figueiredo, and A. F. Martins (2024) Conformal prediction for natural language processing: a survey. Transactions of the Association for Computational Linguistics 12, p. 1497–1516. Cited by: §2. [5] P. Chung, A. Swaminathan, A. J. Goodell, Y. Kim, S. M. Reincke, L. Han, B. Deverett, M. A. Sadeghi, A. Ariss, M. Ghanem, et al. (2025) Verifact: verifying facts in llm-generated clinical text with electronic health records. arXiv preprint arXiv:2501.16672. Cited by: §2, §3.2. [6] J. Fayyad, S. Alijani, and H. Najjaran (2024) Empirical validation of conformal prediction for trustworthy skin lesions classification. Computer Methods and Programs in Biomedicine 253, p. 108231. Cited by: §2. [7] J. Genari and G. T. Goedert (2025) Mining unstructured medical texts with conformal active learning. arXiv preprint arXiv:2502.04372. Cited by: §2. [8] J. Geng, F. Cai, Y. Wang, H. Koeppl, P. Nakov, and I. Gurevych (2024) A survey of confidence estimation and calibration in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 6577–6595. Cited by: §1, §2, §4.1. [9] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger (2017) On calibration of modern neural networks. In International conference on machine learning, p. 1321–1330. Cited by: §1, §2. [10] S. Jain, A. Agrawal, A. Saporta, S. Q. Truong, D. Nguyen Duong, T. Bui, P. Chambon, M. Lungren, A. Ng, C. Langlotz, and P. Rajpurkar (2021-06) RadGraph: Extracting Clinical Entities and Relations from Radiology Reports. PhysioNet. Note: Version 1.0.0 External Links: Document Cited by: §1, §2, §3.1. [11] A. E. Johnson, T. J. Pollard, L. Shen, L. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. Anthony Celi, and R. G. Mark (2016) MIMIC-i, a freely accessible critical care database. Scientific data 3 (1), p. 1–9. Cited by: §2, §3.1, Figure 2. [12] E. Kim, R. Foty, M. Shrestha, and V. Seyfert-Margolis (2025) Conformal prediction and verification of large language model extractions in ehr data. In Proceedings of the AAAI Symposium Series, Vol. 7, p. 539–546. Cited by: §2. [13] E. Kim, M. Shrestha, R. Foty, T. DeLay, and V. Seyfert-Margolis (2024) Structured extraction of real world medical knowledge using llms for summarization and search. In 2024 IEEE International Conference on Big Data (BigData), p. 3421–3430. Cited by: §1, §2. [14] P. Laban, L. Murakhovs’ ka, C. Xiong, and C. Wu (2023) Are you sure? challenging llms leads to performance drops in the flipflop experiment. arXiv preprint arXiv:2311.08596. Cited by: §2. [15] A. Meta (2025) The llama 4 herd: the beginning of a new era of natively multimodal ai innovation. Cited by: §3. [16] C. Papangelou, K. Kyriakidis, P. Natsiavas, I. Chouvarda, and A. Malousi (2025) Reliable machine learning models in genomic medicine using conformal prediction. Frontiers in Bioinformatics 5, p. 1507448. Cited by: §2. [17] C. Snyder and V. Brodsky (2024) Conformal prediction and large language models for medical coding. American Journal of Clinical Pathology 162, p. S171–S172. Cited by: §2. [18] J. Vazquez and J. C. Facelli (2022) Conformal prediction in clinical medical sciences. Journal of Healthcare Informatics Research 6 (3), p. 241–252. Cited by: §2. [19] L. Wu, H. Fang, Y. Qu, J. Xu, and W. Tong (2025) Leveraging fda labeling documents and large language model to enhance annotation, profiling, and classification of drug adverse events with askfdalabel: l. wu et al.. Drug Safety 48 (6), p. 655–665. Cited by: §2. [20] L. Wu, J. Xu, S. Thakkar, M. Gray, Y. Qu, D. Li, and W. Tong (2024) A framework enabling llms into regulatory environment for transparency and trustworthiness and its application to drug labeling document. Regulatory Toxicology and Pharmacology 149, p. 105613. Cited by: §1, §2.