Paper deep dive
Same Facts, Different Diagnosis: Measuring and Mitigating Narrative Anchoring in Clinical Language Models
Prabhjot Singh, Pritam Deka, Vijay Chennareddy
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/1/2026, 1:28:16 AM
Summary
This paper introduces 'Narrative Anchoring,' a failure mode in clinical language models where diagnostic outputs diverge based on sociolinguistic register rather than clinical content. The authors present a dataset of 1,000 USMLE vignettes rewritten into three personas (control, socioeconomic, cultural) with verified fact preservation. They demonstrate that seven LLMs exhibit significant Narrative Anchoring Gaps (0.064ā0.151) under direct prompting. While Chain-of-Thought and explicit debiasing instructions offer partial mitigation, they often suffer from accuracy collapse. The authors propose NarrativeShield, a three-agent pipeline that structurally extracts and verifies clinical facts before reasoning, reducing the Narrative Anchoring Gap to near-zero and achieving superior decision stability.
Entities (12)
Relation Signals (10)
NarrativeShield ā mitigates ā Narrative Anchoring
confidence 96% Ā· NarrativeShield... reducing the Narrative Anchoring Gap to near-zero
Narrative Anchoring ā causedby ā sociolinguistic register
confidence 95% Ā· identical clinical facts expressed in different registers cause diagnostic outputs to diverge.
Dataset ā derivedfrom ā MedQA-USMLE
confidence 93% Ā· We draw candidates from MedQA-USMLE
NarrativeShield ā consistsof ā Agent 2
confidence 92% Ā· Agent 2: Clinical Reasoning Engine
NarrativeShield ā consistsof ā Agent 1
confidence 92% Ā· NarrativeShield is a three-agent pipeline... Agent 1: Adversarial Intake Extractor
NarrativeShield ā consistsof ā Agent 3
confidence 92% Ā· Agent 3: Emergency Fallback Extractor
Agent 1 ā processes ā raw persona narrative
confidence 91% Ā· Receives only the raw persona narrative
Agent 2 ā receivesinputfrom ā Agent 1
confidence 91% Ā· Receives Agent 1ās JSON
BioMistral ā exhibits ā
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models used for clinical diagnostic reasoning are sensitive to sociolinguistic register, not just clinical content. We term this failure mode Narrative Anchoring: identical clinical facts expressed in different registers cause diagnostic outputs to diverge. Unlike prior demographic-bias work, which manipulates explicit identity tokens such as race or income, our benchmark isolates register as the sole channel of variation, with no demographic marker present in any form. We construct a dataset of 1,000 USMLE clinical vignettes, each rewritten into three sociolinguistically distinct personas under an independently audited fact-preservation guarantee, verified by a separate model that never sees the generation prompt. Across seven language models spanning three architecture families and scales, Narrative Anchoring is statistically significant under direct prompting in every model tested, with a Narrative Anchoring Gap of 0.064 to 0.151. Chain-of-thought reasoning and explicit debiasing instructions reduce the bias only partially, and their apparent gains are frequently confounded by accuracy collapse. We introduce NarrativeShield, a three-agent pipeline that structurally extracts and verifies clinical facts before diagnostic reasoning begins, reducing the Narrative Anchoring Gap to near-zero ($-0.004$ to $0.037$) and achieving the lowest rate of severely unstable decisions (DSS $<$ 0.8) of any method across all models, at a modest and mechanistically expected accuracy cost for most models. A stress test using a non-instruction-tuned base model shows that executing a debiasing intervention at all is gated by zero-shot instruction-following ability, not prompt content alone. We release our dataset, human-validated for fact preservation, as a standalone resource for studying register-based clinical bias.
Tags
Links
- Source: https://arxiv.org/abs/2607.27384v1
- Canonical: https://arxiv.org/abs/2607.27384v1
Trouble viewing inline? Open PDF directly ā
Full Text
63,636 characters extracted from source content.
Expand or collapse full text
Same Facts, Different Diagnosis: Measuring and Mitigating Narrative Anchoring in Clinical Language ModelsCode and data available at https://github.com/prabhjotschugh/narrativeshield-sdoh-medqa Prabhjot Singh University of Texas at Austin Austin, Texas, USA prabhjot.singh@utexas.edu &Pritam Deka Queenās University Belfast Belfast, United Kingdom pdeka01@qub.ac.uk &Vijay Chennareddy Middlesex University London, United Kingdom Vc381@live.mdx.ac.uk Abstract Large language models used for clinical diagnostic reasoning are sensitive to sociolinguistic register, not just clinical content. We term this failure mode Narrative Anchoring: identical clinical facts expressed in different registers cause diagnostic outputs to diverge. Unlike prior demographic-bias work, which manipulates explicit identity tokens such as race or income, our benchmark isolates register as the sole channel of variation, with no demographic marker present in any form. We construct a dataset of 1,000 USMLE clinical vignettes, each rewritten into three sociolinguistically distinct personas under an independently audited fact-preservation guarantee, verified by a separate model that never sees the generation prompt. Across seven language models spanning three architecture families and scales, Narrative Anchoring is statistically significant under direct prompting in every model tested, with a Narrative Anchoring Gap of 0.064 to 0.151. Chain-of-thought reasoning and explicit debiasing instructions reduce the bias only partially, and their apparent gains are frequently confounded by accuracy collapse. We introduce NarrativeShield, a three-agent pipeline that structurally extracts and verifies clinical facts before diagnostic reasoning begins, reducing the Narrative Anchoring Gap to near-zero (ā0.004-0.004 to 0.0370.037) and achieving the lowest rate of severely unstable decisions (DSS << 0.8) of any method across all models, at a modest and mechanistically expected accuracy cost for most models. A stress test using a non-instruction-tuned base model shows that executing a debiasing intervention at all is gated by zero-shot instruction-following ability, not prompt content alone. We release our dataset, human-validated for fact preservation, as a standalone resource for studying register-based clinical bias. Same Facts, Different Diagnosis: Measuring and Mitigating Narrative Anchoring in Clinical Language Modelsā thanks: Code and data available at https://github.com/prabhjotschugh/narrativeshield-sdoh-medqa Prabhjot Singh University of Texas at Austin Austin, Texas, USA prabhjot.singh@utexas.edu Pritam Deka Queenās University Belfast Belfast, United Kingdom pdeka01@qub.ac.uk Vijay Chennareddy Middlesex University London, United Kingdom Vc381@live.mdx.ac.uk 1 Introduction Large language models are increasingly proposed as decision support tools for clinical diagnostic reasoning, from triage assistance to differential diagnosis generation, carrying a corresponding risk of introducing harm or exacerbating health disparities in deployment. Much of the resulting research has focused on demographic bias: whether a modelās diagnostic or treatment recommendation changes when a patientās race, gender, or socioeconomic status is stated explicitly. This concern is well founded. Zack et al. (2024) show that GPT-4 consistently produces clinical vignettes that stereotype demographic presentations, with differential diagnoses more likely to include conditions associated with particular races, ethnicities, and genders. Benchmarks such as EquityMedQA (Pfohl et al., 2024) and DiversityMedQA (Rawat et al., 2024) have made this failure mode measurable by perturbing patient demographics directly. Poulain et al. (2026) extend this further, finding that reflection-style prompting can reduce biased outcomes, while larger and medically fine-tuned models are not necessarily less biased. This line of work shares a structural assumption that leaves a clinically realistic failure mode unexamined. Real patients do not announce their income bracket or cultural background to a clinician, or to a model. They describe their own symptoms, in their own words, shaped by how they were taught to talk about pain, what they can afford to do about it, and what their family expects of someone who is unwell. Two patients with identical symptoms, duration, and vital signs can produce vignettes a model reads very differently, not because either mentioned a demographic category, but because the register in which their story is told differs. We term this failure mode Narrative Anchoring: a shift in a modelās diagnostic output that tracks sociolinguistic register rather than clinical content, occurring even when every clinical fact is held constant and no demographic marker is ever stated. This mechanism already carries diagnostic weight independent of any stated label. Across 1.7 million outputs from nine models, Omar et al. (2025) find that cases described with certain sociodemographic framings were directed toward more aggressive care than clinically indicated, relative to a matched control description of the same facts, while holding clinical content constant. Narrative Anchoring isolates this mechanism precisely: our three personas contain no demographic marker of any kind, no race, income figure, or named cultural identifier, anywhere in the text. The bias channel is register and framing alone, not identity disclosure, which distinguishes our benchmark from EquityMedQA, DiversityMedQA, and the demographic-label paradigm generally, and shows that a model need not be told who a patient is to treat that patient differently. Narrative Anchoring is largely invisible to benchmarks that present each case exactly once: a model anchored to register never gets the chance to reveal it if it is never shown the same patient twice, told two different ways. Measuring it requires a dataset that varies register while holding clinical content fixed, with that fact preservation independently verified rather than assumed, since any drift would confound register effects with genuine case difficulty. We make three contributions: ā¢ A dataset of 1,000 patient vignettes, filtered from the MedQA-USMLE corpus (Jin et al., 2020), each rewritten into three sociolinguistically distinct personas: control, socioeconomic, and cultural. Fact preservation is certified not by the generating model itself but by a second, independently invoked model with no access to the generation prompt, whose only task is to verify that every clinical fact survives the rewrite, following the finding that separately invoked evaluator models approximate human judgment more reliably than models grading their own output (Zheng et al., 2023). We further validate this guarantee through human annotation. ā¢ Evidence that Narrative Anchoring is not marginal. Across seven models spanning three architecture families and a range of scales, direct prompting produces a statistically significant divergence in recommendations between the control persona and each marked persona, in every model tested, with a Narrative Anchoring Gap of 0.064 to 0.151. Chain-of-thought reasoning (Wei et al., 2023) and an explicit debiasing instruction reduce this only partially, and chain-of-thoughtās apparent gains are frequently confounded by a drop in accuracy, consistent with evidence that a chain-of-thought explanation does not reliably reflect the true reason for a modelās prediction and can be steered by contextual, non-clinical cues (Turpin et al., 2023). ā¢ NarrativeShield, a three-agent pipeline that addresses Narrative Anchoring architecturally: it structurally extracts and verifies clinical facts before diagnostic reasoning begins, decoupling what the model reasons over from how the patientās story was told. Across all seven models, NarrativeShield reduces the Narrative Anchoring Gap to near-zero (ā0.004-0.004 to 0.0370.037) and achieves the lowest rate of severely unstable, persona-dependent decisions (Decision Stability Score below 0.8) of any method we test. This comes with a modest, mechanistically explicable accuracy cost for most models, and a striking failure in a domain-adapted but non-instruction-tuned base model (Labrak et al., 2024), which we show is evidence that any debiasing intervention presupposes a baseline zero-shot instruction-following capacity that not all models possess. The risk, then, is not only that a model treats two patients differently because it was told they belong to different demographic groups. It is that a model treats two patients differently because of how each one, or their family, describes what is wrong, a channel of bias that requires no demographic disclosure and survives interventions aimed only at prompt content rather than the architecture that processes it. 2 Dataset Construction We construct NarrativeShield-SDoH, 1,000 USMLE-style clinical vignettes, each expressed in three sociolinguistic personas that hold clinical content fixed while varying register. Construction has two stages: deterministic filtering and sampling, and persona generation with independent fact-preservation auditing. 2.1 Source Filtering and Sampling We draw candidates from MedQA-USMLE (Jin et al., 2020), pooling train and test splits, and apply six filters as a single boolean mask with fixed seed 42, making filtering fully deterministic. Each filter targets a property required for persona rewriting: (1) a patient-vignette stem Ė(A|An) + +[\- ]?(year|month|week|day)[\- ]?old| (2) minimum length of 300 characters, ensuring narrative depth beyond pure recall; (3) at least one clinical signal term from a 49-term list spanning symptoms, presentation context, vitals/labs, and physical exam; (4) exclusion of 21 phrase patterns indicating pure mechanism or pathophysiology questions, which carry no patient-reported framing to vary; (5) a target type of diagnosis, treatment, or investigation; and (6) at least one narrative-richness marker from 27 phrases indicating first-person or caregiver-reported history (e.g., āshe statesā). Full regex and phrase lists are in Appendix A. From the 2,921-question filtered pool, we draw 1,000 via proportional stratified sampling over the cross of USMLE exam step and correct-answer letter, preserving this joint distribution in the sample (Appendix B). 2.2 Persona Generation Each vignette is rewritten into three personas using Gemini-3.1-Flash-Lite: a control (PĪ±P_Ī±) preserving the original register, a socioeconomic (PĪ²P_Ī²) reframed through financial or occupational strain, and a cultural (PĪ³P_Ī³) reframed through culturally situated metaphor. No persona contains an explicit demographic marker; the bias channel is register alone. Four design decisions distinguish this pipeline from naive rewriting, each closing a failure mode observed in an earlier iteration. Independent generation calls: each persona is generated via a separate call with no shared context, since joint generation risks anchoring phrasing across personas into cosmetic synonym substitution. Differentiated temperature: 0.25 for PĪ±P_Ī±, favoring fidelity, and 0.90 for PĪ²P_Ī²/PĪ³P_Ī³, favoring genuine register divergence, with high temperature permitting linguistic divergence while the fact-preservation constraint below prevents clinical content loss. Structural, not cosmetic, register: PĪ²P_Ī²/PĪ³P_Ī³ prompts explicitly forbid patterns identified in earlier iterations as producing detectable templating, generic phrasing (e.g., ātraditional herbal teasā), formulaic framing (e.g., āmy family insistedā), and end-loaded cues, and each generation is assigned one of five opening styles sampled independently per attempt, preventing a detectable structural template across the dataset (full definitions in Appendix C). Independent fact-preservation audit: each persona is passed to a separate auditor model at temperature 0 with no access to the generation prompt, whose sole task is to extract every clinical fact from the original and verify its presence in the persona, clinical, lay, or metaphorical, returning a pass/fail verdict; the generating model cannot pass its own test. A persona enters the dataset only on pass; failures retry up to 10 times with a freshly sampled opening style, and every prompt states the fact-preservation requirement directly, with a fallback for PĪ³P_Ī³ that an unmetaphorizable fact must be stated plainly. 2.3 Human Validation All 3,000 persona generations reached SUCCESS under audit, with zero PARTIAL or FAILED rows released; we report this completeness figure rather than intermediate pass rates, which are pipeline diagnostics rather than properties of the released data. To corroborate this automated guarantee independently, three annotators with clinical backgrounds, ranging from an attending physician with 20 years of experience to residents and interns, none involved in construction and none with access to auditor verdicts, rated a stratified sample of 100 questions (300 encounters) on fact preservation, persona register match, and 5-point Likert narrative realism. Full protocol and raw ratings are released with the dataset. Fact preservation and register match. All three annotators rated 100% of encounters as preserving facts and matching register, a raw agreement ceiling for which chance-corrected statistics are undefined by construction. Since the sample was drawn only from audit-passed personas, this reflects the fact-preservation gate working on this sample rather than an unfiltered failure rate, consistent with comparably high post-filter fidelity reported for physician-reviewed AI-generated clinical vignettes elsewhere (Yanagita et al., 2024). Narrative realism. This subjective dimension diverged sharply. One annotator rated all 300 items at ceiling (confirmed as genuine judgment, not artifact); restricting to the two informative annotators, exact agreement was 42.0% but adjacent (Ā±1Ā± 1) agreement was 85.3%, and Krippendorffās Ī± (Krippendorff, 2004) was 0.226, below the 0.667 threshold for tentative conclusions, though comparable to reported agreement on other subjective NLP tasks (27% Īŗ on GoEmotions (Demszky et al., 2020), 46% on HateXplain (Mathew et al., 2021)). Critically, disagreement was not random: exact agreement fell from 74.0% (PĪ±P_Ī±) to 34.0% (PĪ²P_Ī²) to 18.0% (PĪ³P_Ī³), tracking persona difficulty. Despite this calibration gap, ordering was stable: pooled means were PĪ±=4.89>PĪ²=4.64>PĪ³=4.19P_Ī±=4.89>P_Ī²=4.64>P_Ī³=4.19, holding independently for every annotator (Kruskal-Wallis H=138.69H=138.69, p<.0001p<.0001, all pairwise comparisons significant after Bonferroni correction). Annotators disagree on where exactly a PĪ³P_Ī³ narrative sits on the scale but agree without exception that it is harder to render naturalistically than PĪ²P_Ī², which is harder than PĪ±P_Ī±, the ordering persona construction was designed to produce, consistent with evidence that annotator disagreement on graded tasks can track conceptual difficulty rather than inconsistency (Kellert et al., 2026). We report this as a limitation: judging realism of non-clinical register is a harder, more subjective task than judging fact preservation, echoing this paperās broader argument that narrative style, not clinical content, is where models and annotators alike diverge most. Full statistics and per-persona breakdowns are in Appendix I. 3 Experimental Setup 3.1 Models We evaluate seven open-weight models: Llama-3.1-8B-Instruct and Llama-3.2-3B-Instruct (Grattafiori et al., 2024), Mistral-7B-Instruct-v0.3 (Jiang et al., 2023), Qwen2.5-7B-Instruct (Qwen et al., 2025), gemma-3-12b-it and gemma-4-E4B-it (Team et al., 2025), and BioMistral-7B (Labrak et al., 2024), a PubMed Central-adapted Mistral-7B-Instruct-v0.3 base model without instruction tuning. BioMistral is included as an architectural stress test: it isolates zero-shot instruction-following capacity from medical domain knowledge. All models use greedy decoding. 3.2 Conditions Four conditions, applied identically across all three personas of every vignette: B1 (Direct). Persona and options presented directly, no reasoning scaffold. Zero-shot floor. B2 (Chain-of-Thought). Step-by-step reasoning before answering (Wei et al., 2023). For BioMistral, a two-shot demonstration of the reasoning-then-answer format is included, since the base model cannot infer novel output structure from instruction alone. B3 (Explicit Debiasing). A single capitalized instruction to disregard sociolinguistic framing, administered zero-shot with no demonstration, including for BioMistral. The asymmetry with B2 is intentional: comparing them isolates the effect of in-context demonstration on format commitment independent of reasoning quality. Figure 1: NarrativeShield architecture. Agent 1 strips sociolinguistic register while preserving all clinical facts; a deterministic, non-model tool router injects clinical decision support; Agent 2 reasons only over Agent 1ās structured output; Agent 3 activates only when Agent 2 fails to commit to a parseable answer. NarrativeShield (NS). Our proposed architectural intervention (Ā§3.3). We report parse rate for every condition: the fraction of responses yielding a parseable final answer letter. Under B1āB3 this is computed on raw output; under NS it accounts for Agent 3 fallback. Parse rate is architecturally informative, not merely a data-quality footnote: a model that reasons adequately but cannot commit to format produces a qualitatively different failure than one that reasons poorly, a distinction we rely on directly for BioMistral. 3.3 NarrativeShield Architecture NarrativeShield is a three-agent pipeline with a deterministic tool layer between Agents 1 and 2. Its principle: register is removed structurally by an extraction step blind to downstream reasoning, not suppressed by instructing a reasoning model to ignore it. Figure 1 summarizes the pipeline end to end. Agent 1: Adversarial Intake Extractor. Receives only the raw persona narrative; returns structured JSON (age, sex, chief complaint, onset, present and absent symptoms, vitals, physical exam, labs, imaging, current and administered medications, history, question type). The prompt enumerates what must be stripped (lay phrasing, economic framing, cultural references, emotional language, somatic metaphors) and what must be preserved exactly (every number, every symptom present or absent, all medications by pharmacological name, all history and exam findings). This is the load-bearing debiasing step: persona voice is gone before any downstream component sees the case. Agent 2 has no access to the original narrative. Deterministic Tool Router. A fixed Python layer, not a model, inspects Agent 1ās JSON and the question, invoking up to four tools: a lab interpreter using static reference ranges; at most one of five clinical risk calculators (Wells DVT/PE, CURB-65, Glasgow Coma Scale, Apgar), each gated on mutually exclusive keyword patterns; and a static 47-drug knowledge base, queried against current medications or answer options, capped at two lookups for treatment or management questions. Routing is rule-based because smaller models do not reliably emit well-formed tool-call syntax. Tool outputs are injected into Agent 2ās prompt as decision support before reasoning begins. Agent 2: Clinical Reasoning Engine. Receives Agent 1ās JSON, tool outputs, and the question with options. Follows a fixed protocol: summarize the clinical picture, enumerate a differential grounded in specific extracted values, state a conclusion, commit to a single answer letter. Never sees the original narrative. Agent 3: Emergency Fallback Extractor. Invoked only when Agent 2ās output contains no parseable answer. Extracts a single letter from Agent 2ās already-generated text, performing no reasoning. This separates two failure modes: poor reasoning versus adequate reasoning with format-commitment failure. 3.4 Metrics Five metrics, computed identically across all conditions. Option Match Rate (OMR). Binary accuracy, per persona and overall, with 95% Wilson score intervals. Narrative Anchoring Gap (NAG). OMRĪ±āminā”(OMRĪ²,OMRĪ³)OMR_Ī±- (OMR_Ī²,OMR_Ī³). Primary equity metric: the accuracy advantage a neutral register confers over the more disadvantaged marked persona. Near-zero NAG indicates register-insensitive diagnostic accuracy. Diagnostic Stability Score (DSS). Mean pairwise cosine similarity of full reasoning outputs across the three persona presentations of the same question, using all-MiniLM-L6-v2. Primary stability metric: captures semantic drift in reasoning even when final answers agree. We report the fraction of questions with DSS below 0.80 (severely unstable) alongside the mean. Cohenās Īŗ and McNemarās test. Secondary: chance-corrected inter-persona agreement and directional bias. We note Īŗās high-accuracy paradox and treat DSS as interpretively primary. For NS specifically: Agent 1 parse rate (valid JSON fraction), mean tools invoked per question, and mean end-to-end latency as a deployment-relevant descriptor, not a primary criterion. 4 Results 4.1 Narrative Anchoring Is Pervasive Under Direct Prompting Under B1, every one of the seven models shows a positive Narrative Anchoring Gap, ranging from 0.064 (BioMistral) to 0.151 (Llama-3.1-8B-Instruct), with McNemarās test significant (p<0.05p<0.05, uncorrected) on the control-versus-socioeconomic and control-versus-cultural pairs for all seven models.111Across all 21 pairwise tests in B1 (7 models Ć 3 persona pairs), Bonferroni correction (Ī±=0.05/21Ī±=0.05/21) flips two comparisons from significant to non-significant, both secondary Ī²-versus-Ī³ or borderline pairs; every control-versus-socioeconomic and control-versus-cultural comparison, the pairs our central claim rests on, remains significant under correction. Full values are in Appendix E. This holds across architecture families, parameter scales from 3B to 12B, and both general-purpose and medically domain-adapted models. This establishes Narrative Anchoring as a pervasive property of direct clinical prompting rather than an artifact of any single model family, and directly supports our first claim: sociolinguistic register alone, with no demographic marker present, is sufficient to shift diagnostic output. BioMistralās B1 NAG is the smallest of the seven models, but this is not evidence of relatively unbiased behavior: BioMistral simultaneously has the lowest overall OMR (0.426) and the highest rate of severely unstable decisions (DSS <0.8<0.8: 80.8%) of any model under B1. NAG is a bounded difference of accuracies and mechanically compresses as overall accuracy falls toward a floor; DSS, an unbounded similarity measure, shows the opposite and more informative story. We return to BioMistralās role directly in Ā§4.4. 4.2 Chain-of-Thought and Explicit Debiasing Only Partially Mitigate Anchoring Table 1 reports NAG across all four conditions; the corresponding DSS<<0.8% synthesis is in Appendix H. Under B2, five of seven models show reduced NAG relative to B1, but this improvement is confounded by substantial accuracy loss for three of them: gemma-3-12b-it loses 21 points of overall OMR, gemma-4-E4B-it loses 34 points, and BioMistralās OMR collapses alongside a parse rate of only 54.07%, the lowest of any non-degenerate condition (Figure 3). A lower NAG computed over a smaller or differently-distributed set of correct answers is not evidence of improved fairness. We therefore do not read B2 as a clean mitigation result. B3 is the more informative comparison. For the six models capable of executing the instruction, NAG falls modestly from B1 (e.g., Llama-3.1-8B-Instruct: 0.151 to 0.135; Qwen2.5: 0.091 to 0.067), overall OMR is essentially unchanged, and mean DSS improves in every case. This is our cleanest baseline finding: a single explicit debiasing instruction helps a little, at no accuracy cost, for models that can follow it. BioMistral cannot follow it: under B3, its parse rate is 0.00%, every OMR value is exactly zero, and Cohenās Īŗ is undefined for all three persona pairs. We discuss why in Ā§4.4. !10Model !10B1 !10B2 !10B3 !10NS Llama-3.1-8B-Instruct 0.151 0.100 0.135 0.037 Llama-3.2-3B-Instruct 0.136 0.129 0.123 ā0.003-0.003 Mistral-7B-Instruct-v0.3 0.069 0.073 0.058 ā0.004-0.004 Qwen2.5-7B-Instruct 0.091 0.094 0.067 0.003 gemma-3-12b-it 0.095 0.052 0.092 0.024 gemma-4-E4B-it 0.080 0.023 0.072 0.019 BioMistral-7B 0.064 0.147 0.000* 0.000 Table 1: Narrative Anchoring Gap across all four conditions. *BioMistralās B3 value is a degenerate artifact of a 0.00% parse rate (Ā§4.4), not evidence of eliminated bias. Table 1 reports NAG across all four conditions, and Figure 2 visualizes the same trajectory across models; the corresponding DSS<<0.8% synthesis is in Appendix H. Figure 2: Narrative Anchoring Gap across all four conditions, per model. Six of seven models converge toward near-zero NAG under NarrativeShield (NS); BioMistral (dashed) follows a qualitatively different trajectory, discussed in Ā§4.4. Figure 3: Overall Option Match Rate (OMR) across B1 (Direct), B2 (Chain-of-Thought), and B3 (Explicit Debiasing), per model. 4.3 NarrativeShield Reduces Anchoring to Near-Zero NarrativeShield reduces NAG to within [ā0.004,0.037][-0.004,0.037] for every model, a reduction of at least four-fold relative to B1 for every model, and considerably more where NAG falls to near-zero or below (Table 1). Table 2 reports full NarrativeShield results. NarrativeShield also achieves the lowest DSS <0.8<0.8 rate of any condition for every one of the seven models, ranging from 16.3% (gemma-3-12b-it) to 31.2% (Llama-3.2-3B-Instruct), compared to a B1 range of 35.6% to 80.8%. Because Agent 1ās fact-extraction step is verified against the same independently audited fact-preservation guarantee that underlies the released dataset (Ā§2.3), we attribute this collapse in both NAG and DSS to NarrativeShieldās reasoning architecture rather than to any possibility that persona difficulty happened to equalize across variants: the datasetās own construction rules out that explanation directly. !10Model !10OMR !10NAG !10DSS<<0.8 Llama-3.1-8B-Instruct 0.629 0.037 28.5% Llama-3.2-3B-Instruct 0.513 ā0.003-0.003 31.2% Mistral-7B-Instruct-v0.3 0.445 ā0.004-0.004 23.5% Qwen2.5-7B-Instruct 0.557 0.003 21.5% gemma-3-12b-it 0.589 0.024 16.3% gemma-4-E4B-it 0.681 0.019 20.0% BioMistral-7B 0.255 0.000 19.3% Table 2: NarrativeShield primary results. Full pipeline metrics are in Appendix G. This reduction carries a modest, model-dependent accuracy cost. Relative to B1, overall OMR under NarrativeShield changes by less than 1.5 points for three models, drops by 4.2 to 6.9 points for two others, and improves by 6.7 points for gemma-4-E4B-it, the only model to gain accuracy under the pipeline. We attribute the cost, where present, to Agent 1 acting as a lossy compression step: any detail Agent 1 fails to extract is permanently unavailable to Agent 2, regardless of whether the original narrative contained it. gemma-4-E4B-itās gain is consistent with the same mechanism running in the opposite direction: for a model whose direct-prompting accuracy is otherwise held back by narrative noise irrelevant to the clinical question, Agent 1ās structured extraction can remove more confounding signal than clinical signal, improving rather than degrading downstream reasoning. This is a direct, mechanistically expected consequence of the architecture, not an unexplained side effect, and one we consider a favorable trade against a near-total elimination of register-driven diagnostic instability. 4.4 BioMistral as an Architectural Stress Test BioMistralās behavior across all four conditions traces a single, monotonically worsening pattern that we argue is evidence for our thesis rather than noise requiring explanation away. Each condition places an increasing demand on zero-shot instruction-following capacity, independent of medical reasoning ability, and BioMistralās degree of failure tracks that demand precisely: under B1, which requires no format compliance beyond a direct answer, BioMistral answers with partial success (OMR 0.426); under B2, given a two-shot demonstration of the required format, its parse rate recovers to 54.07%; under B3, given only a bare instruction with no demonstration, its parse rate falls to 0.00%; and under NarrativeShieldās Agent 1, which requires zero-shot emission of a structured JSON schema with no worked example, its parse rate is 1.10%. Inspection of its raw output under B3 and Agent 1 shows BioMistral consistently produces reasoning text addressing the clinical question; it fails specifically to resolve that reasoning into the required output structure. This is precisely what a domain-adapted base model without equivalent instruction-tuning predicts: BioMistral can imitate a structure demonstrated in-context, as its B2 recovery shows, but cannot infer a novel output format from instruction alone. This pattern has a direct implication for how NarrativeShieldās own results should be read. BioMistralās near-zero NAG under NarrativeShield is not evidence that the architecture successfully debiased BioMistral; with a 1.10% Agent 1 parse rate, its NarrativeShield results are computed over a residual set of cases too small to support any claim about generalized debiasing, and we report them for completeness rather than as a further success case. The broader claim NarrativeShield supports is that any debiasing intervention, whether a single instruction or a structural pipeline, first requires a baseline capacity to execute the intervention at all; that capacity is what BioMistral consistently lacks, and its absence, not the content of any specific prompt, is what explains its trajectory across all four conditions. 5 Conclusion We introduced Narrative Anchoring, a bias in clinical language models that operates through sociolinguistic register rather than demographic disclosure, and showed it is present, significant, and largely unaddressed by chain-of-thought or explicit debiasing across seven models spanning three architecture families. We released NarrativeShield-SDoH, a dataset built to isolate this channel precisely, with fact preservation certified by an independent auditor model and corroborated by human clinical annotation. We introduced NarrativeShield, a three-agent pipeline that reduces this bias to near-zero by structurally separating clinical fact extraction from diagnostic reasoning, and we used a domain-adapted, non-instruction-tuned base model as a stress test. We show that any debiasing intervention presupposes a baseline instruction-following capacity not all models have. Taken together, this argues that clinical bias in language models is not solely a question of what a patient is labeled, but of how a patientās own words are allowed to reach a modelās reasoning, and that closing this gap is a matter of pipeline design as much as prompt design. Limitations We report five limitations of the present study. First, all seven models we evaluate are open-weight and span 3B to 12B parameters. We do not test proprietary frontier models, and whether Narrative Anchoring persists, weakens, or strengthens at larger scale or under different training regimes is a question our design cannot answer. Second, all personas in NarrativeShield-SDoH are generated by a single model, Gemini-3.1-Flash-Lite. Systematic tendencies in how that specific model renders socioeconomic or cultural register, rather than properties of the underlying phenomenon we study, could in principle shape our results. The independent fact-preservation audit and human validation constrain this risk to a question of framing rather than clinical fidelity, but do not eliminate it, since both checks were themselves designed around the same generation pipeline. Third, our benchmark isolates two persona categories beyond the control condition, socioeconomic and cultural framing. Other axes of sociolinguistic register, including disability framing, age-related speech patterns, and non-native speaker phrasing, may anchor language models differently and are not covered by this study. Fourth, our vignettes are single-turn USMLE examination questions rather than multi-turn clinical dialogue or real patient encounters. We make no claim about how Narrative Anchoring manifests, or how NarrativeShield performs, in extended, interactive clinical use, where a patientās register may itself evolve over the course of a conversation. Fifth, NarrativeShieldās accuracy cost, while modest for most models tested, is a genuine trade-off rather than a free improvement, and this paper does not evaluate whether that cost is acceptable relative to any specific clinical deployment context. Such a determination depends on the stakes and setting of a particular application, which we are not positioned to specify in the abstract, and which any real deployment of an architecture like NarrativeShield would need to establish independently. These limitations bound the scope of our claims rather than undermine them. We do not claim Narrative Anchoring is the only channel through which sociolinguistic bias enters clinical language model reasoning, nor that NarrativeShield is a complete solution to it. We claim that register-driven bias exists independent of demographic disclosure, that it survives the mitigations most readily available at the prompt level, and that a structural intervention addresses it more reliably than an instructional one, within the scope of models, personas, and task format we test. Future directions. Each limitation points to a natural extension. Applying the same audited persona-generation pipeline to proprietary frontier models would test how far Narrative Anchoring generalizes beyond open weights. Adding further persona axes, such as disability framing or non-native speaker phrasing, would test whether the effect and its mitigation hold across register types or are specific to the two studied here. Extending the benchmark to multi-turn dialogue, with register shifting across turns rather than fixed at intake, would test whether NarrativeShieldās architectural separation of register from content survives interactive use. Finally, evaluating NarrativeShieldās accuracy trade-off against clinician judgment on live cases, rather than against exam ground truth alone, is a necessary step before any claim of deployment readiness. Ethical Considerations Dataset and human subjects. NarrativeShield-SDoH is derived entirely from MedQA-USMLE, a publicly available, de-identified corpus of medical examination questions; no real patient data, clinical records, or personally identifiable information is used at any stage of construction. All three personas are synthetic rewrites of exam vignettes and do not describe or reference any real individual. Human validation of the dataset was conducted by physician annotators who volunteered their clinical expertise, were not exposed to any patient data, and rated only the fidelity of already-synthetic text; no institutional patient-facing risk was involved in this process. Dual-use and deployment risk. This work demonstrates that clinical language models are sensitive to a patientās sociolinguistic register, a finding with a clear protective motivation, but one that could in principle be misused to deliberately construct adversarial patient narratives designed to elicit a specific diagnosis or treatment recommendation from a deployed system. We believe the balance of this risk favors publication: the underlying vulnerability already exists in deployed models regardless of whether it is documented, and identifying it, together with an architectural mitigation, gives practitioners a concrete tool to close the gap rather than leaving it undocumented and unaddressed. Clinical deployment. Neither the base models we evaluate nor NarrativeShield itself are validated, certified, or intended for real clinical decision-making. All experiments in this paper are conducted on retrospective examination questions with known ground-truth answers, not on live patients or clinical workflows. This paper does not endorse any of the evaluated models, or NarrativeShield, as suitable for unsupervised clinical use. We view our contribution as diagnostic of a failure mode and a research-stage mitigation, not as a deployment-ready clinical tool, and we recommend that any translation of NarrativeShield-style architectures toward real clinical settings undergo the validation, regulatory review, and prospective evaluation appropriate to a clinical decision support system. Representation of socioeconomic and cultural register. Varying socioeconomic and cultural framing risks encoding the very stereotypes we study if personas rely on caricatured markers. We mitigate this by forbidding generic or formulaic markers during generation (Ā§2.2) and by having physician annotators, not the generating model, judge personas as naturalistic rather than caricatured. We nonetheless acknowledge that any synthetic rendering of illness-talk risks flattening real diversity within a group, and do not claim our three personas are exhaustive or representative. AI writing assistance. Portions of this paperās prose were drafted and revised with the assistance of a generative language model, for language polishing and editing of the authorsā original content, consistent with the ACL Policy on AI Writing Assistance. All technical claims, citations, and results were verified by the authors against the underlying code, data, and literature. Acknowledgements We express our sincere gratitude to the medical professionals who generously volunteered their clinical expertise to annotate and validate the dataset for this work: Dr. Roopam Deka, DM (Assistant Professor, Department of Pathology, All India Institute of Medical Sciences, Guwahati); Dr. Benjina Ahmed, MBBS (Junior Resident, Department of Pathology and Lab Medicine, All India Institute of Medical Sciences, Guwahati); and Dr. Meghna Kashyap, MBBS (Intern, Laxmi Chandravansi Medical College and Hospital). Their rigorous evaluation of clinical fact preservation and narrative fidelity was fundamental to ensuring the benchmarkās quality. References D. Demszky, D. Movshovitz-Attias, J. Ko, A. Cowen, G. Nemade, and S. Ravi (2020) GoEmotions: a dataset of fine-grained emotions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault (Eds.), Online, p. 4040ā4054. External Links: Link, Document Cited by: Ā§2.3. A. Grattafiori, A. Dubey, A. Jauhri, et al. (2024) The llama 3 herd of models. External Links: 2407.21783, Link Cited by: Ā§3.1. A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, et al. (2023) Mistral 7b. External Links: 2310.06825, Link Cited by: Ā§3.1. D. Jin, E. Pan, N. Oufattole, W. Weng, H. Fang, and P. Szolovits (2020) What disease does this patient have? a large-scale open domain question answering dataset from medical exams. External Links: 2009.13081, Link Cited by: 1st item, Ā§2.1. O. Kellert, S. Kondury, C. Koo, N. Tyagi, and S. Eikenberry (2026) Structured disagreement in health-literacy annotation: epistemic stability, conceptual difficulty, and agreement-stratified inference. External Links: 2604.19943, Link Cited by: Ā§2.3. K. Krippendorff (2004) Content analysis: an introduction to its methodology. 2nd edition, Sage Publications, Thousand Oaks, CA. Cited by: Ā§2.3. Y. Labrak, A. Bazoge, E. Morin, P. Gourraud, M. Rouvier, and R. Dufour (2024) BioMistral: a collection of open-source pretrained large language models for medical domains. In Findings of the Association for Computational Linguistics: ACL 2024, L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 5848ā5864. External Links: Link, Document Cited by: 3rd item, Ā§3.1. B. Mathew, P. Saha, S. M. Yimam, C. Biemann, P. Goyal, and A. Mukherjee (2021) HateXplain: a benchmark dataset for explainable hate speech detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, p. 14867ā14875. Cited by: Ā§2.3. M. Omar, S. Soffer, R. Agbareia, N. L. Bragazzi, D. U. Apakama, C. R. Horowitz, A. W. Charney, R. Freeman, B. Kummer, B. S. Glicksberg, G. N. Nadkarni, and E. Klang (2025) Sociodemographic biases in medical decision making by large language models. Nature Medicine 31 (6), p. 1873ā1881. External Links: Document, Link, ISSN 1546-170X Cited by: Ā§1. S. R. Pfohl, H. Cole-Lewis, R. Sayres, D. Neal, M. Asiedu, A. Dieng, N. Tomasev, Q. M. Rashid, S. Azizi, N. Rostamzadeh, L. G. McCoy, L. A. Celi, Y. Liu, M. Schaekermann, A. Walton, A. Parrish, C. Nagpal, P. Singh, A. Dewitt, P. Mansfield, et al. (2024) A toolbox for surfacing health equity harms and biases in large language models. Nature Medicine 30 (12), p. 3590ā3600. External Links: Document, Link, ISSN 1546-170X Cited by: Ā§1. R. Poulain, F. I. Adiba, H. Fayyaz, and R. Beheshti (2026) Bias patterns in the application of LLMs for clinical decision support: a comprehensive study. Delaware Journal of Public Health 12 (1), p. 54ā67. External Links: Document Cited by: Ā§1. Qwen, :, A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, X. Ren, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Wan, Y. Liu, Z. Cui, Z. Zhang, and Z. Qiu (2025) Qwen2.5 technical report. External Links: 2412.15115, Link Cited by: Ā§3.1. R. Rawat, H. McBride, R. Ghosh, D. Nirmal, J. Moon, D. Alamuri, S. OāBrien, and K. Zhu (2024) DiversityMedQA: a benchmark for assessing demographic biases in medical diagnosis using large language models. In Proceedings of the Third Workshop on NLP for Positive Impact, D. Dementieva, O. Ignat, Z. Jin, R. Mihalcea, G. Piatti, J. Tetreault, S. Wilson, and J. Zhao (Eds.), Miami, Florida, USA, p. 334ā348. External Links: Link, Document Cited by: Ā§1. G. Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, et al. (2025) Gemma 3 technical report. External Links: 2503.19786, Link Cited by: Ā§3.1. M. Turpin, J. Michael, E. Perez, and S. R. Bowman (2023) Language models donāt always say what they think: unfaithful explanations in chain-of-thought prompting. In Thirty-seventh Conference on Neural Information Processing Systems, External Links: Link Cited by: 2nd item. J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou (2023) Chain-of-thought prompting elicits reasoning in large language models. External Links: 2201.11903, Link Cited by: 2nd item, Ā§3.2. Y. Yanagita, D. Yokokawa, S. Uchida, Y. Li, T. Uehara, and M. Ikusaka (2024) Can AI-generated clinical vignettes in Japanese be used medically and linguistically?. Journal of General Internal Medicine 39 (16), p. 3282ā3289. External Links: Document Cited by: Ā§2.3. T. Zack, E. Lehman, M. Suzgun, J. A. Rodriguez, L. A. Celi, J. Gichoya, D. Jurafsky, P. Szolovits, D. W. Bates, R. E. Abdulnour, A. J. Butte, and E. Alsentzer (2024) Assessing the potential of gpt-4 to perpetuate racial and gender biases in health care: a model evaluation study. The Lancet Digital Health 6 (1), p. e12āe22. External Links: Document, Link, ISSN 2589-7500 Cited by: Ā§1. L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica (2023) Judging llm-as-a-judge with mt-bench and chatbot arena. External Links: 2306.05685, Link Cited by: 1st item. Appendix A Filter Definitions and Regular Expressions We apply six filters as a single combined boolean mask over the pooled MedQA-USMLE train and test splits, with a fixed random seed (SEED = 42) for all sampling operations, making the entire filtering and sampling process deterministic and reproducible. Filter 1: Patient-vignette stem. The question must open with a patient-vignette stem matching: ^(A|An) + +[\- ]? (year|month|week|day)[\- ]?old e.g., āA 23-year-old womanā¦ā, āA 6-month-old boyā¦ā. Filter 2: Minimum length. Question text must be at least 300 characters, ensuring at least two to three sentences of clinical context beyond pure factual recall. Filter 3: Clinical signal terms. The question must contain at least one term from a curated 49-term list spanning four categories: symptoms (25 terms, e.g., pain, fever, cough, dyspnea, nausea, syncope), presentation context (6 terms, e.g., presents to, emergency department, admitted), vitals and labs (12 terms, e.g., blood pressure, hemoglobin, creatinine, mmHg), and physical exam findings (6 terms, e.g., palpation, auscultation, tenderness, murmur, edema, jaundice). Filter 4: Exclusion of mechanism and pathophysiology questions. The question must not match any of 21 excluded phrase patterns across four categories: mechanism and pathophysiology (e.g., āmost likely mechanismā, āpathogenesisā, āwhich enzymeā, āwhich receptorā), anatomy, genetics and biochemistry, and histology (e.g., āwhich geneā, āhistologicalā, ābiopsy showsā). These question types carry no patient-reported symptom framing to vary across personas. Filter 5: Target question type. The question must match at least one of 24 phrase patterns across three categories: diagnosis (6 patterns, e.g., āmost likely diagnosisā, āmost likely causeā, āmost likely etiologyā), treatment and management (12 patterns, e.g., ābest treatmentā, āmost appropriate next stepā, āwhich medicationā), and investigation (6 patterns, e.g., ābest initial testā, āmost appropriate testā, āconfirm the diagnosisā). Filter 6: Narrative richness markers. The question must contain at least one of 27 phrases indicating the patient or a caregiver is reporting symptoms in narrative form, e.g., āshe statesā, āhe reportsā, āthe patient deniesā, āhistory ofā, āfor the pastā, āworseningā, āmother reportsā. Appendix B Sampling Statistics The combined filter mask yields 2,921 eligible questions from the pooled MedQA-USMLE corpus. We draw 1,000 questions via proportional stratified sampling over the cross of USMLE exam step (Step 1 vs. Step 2&3) and correct-answer letter (AāD). Each stratum contributes a number of rows proportional to its share of the filtered pool, computed as n=minā”(|stratum|,maxā”(1,ā1000Ć|stratum|/2921ā))n= (|stratum|, (1, 1000Ć|stratum|/2921 )), with a top-up step drawing any shortfall from the remaining filtered pool, followed by a final shuffle (random_state=42) before truncation to exactly 1,000 rows. The released dataset preserves the answer-letter and exam-step distribution of the filtered pool to within rounding. We note one detail for exact reproducibility: the top-up step draws any shortfall from the entire remaining filtered pool rather than proportionally from under-filled strata specifically. Because the filtered pool (2,921) comfortably exceeds the target sample size (1,000), this branch is not expected to trigger under typical random seeds, but we flag it as a minor implementation detail relevant to exact stratification guarantees under different sampling parameters. Appendix C Persona Generation: Forbidden Patterns and Opening Styles C.1 Forbidden Structural Patterns To prevent detectable structural templating, generation prompts for PĪ²P_Ī² and PĪ³P_Ī³ explicitly forbid patterns identified in an earlier pipeline iteration as producing cosmetic rather than genuine register variation. Forbidden in PĪ²P_Ī² (socioeconomic): sentences beginning with a formulaic third-person hedge (e.g., āHe mentioned he was hesitant to come back so soon becauseā¦ā); economic framing appended only to the final sentence of an otherwise standard narrative; the verbatim phrase āI canāt afford to miss workā; apologetic openers (e.g., āIām sorry to bother youā, āI know youāre busyā); and ending the persona with the economic detail rather than integrating it throughout. Forbidden in PĪ³P_Ī³ (cultural): the verbatim phrases āmy family insisted I comeā, ātraditional herbal teasā (flagged as too generic), āmy mother insistedā, and āfulfill my dutiesā; cultural framing appearing only in the final one to two sentences; and somatic metaphors that reduce to simple synonym substitution (the prompt explicitly notes that āburning sensationā for dysuria is not a somatic metaphor, since it remains standard medical language). C.2 Opening Styles Each PĪ²P_Ī² and PĪ³P_Ī³ generation is independently assigned one of five opening styles per attempt, sampled fresh on each retry, to prevent a detectable structural template across the dataset. PĪ²P_Ī² opening styles: Impact-First (open with the functional or daily-life consequence before naming the symptom); Timeline-First (open with how long the patient delayed seeking care and what was tried); Remedy-First (open with a specific named over-the-counter product or home remedy that failed); Worry-First (open with the patientās fear of job, family, or financial consequences, with symptoms following as the reason for that worry); Symptom-First-Lay (open directly with the chief complaint in maximally informal language, with no preamble). PĪ³P_Ī³ opening styles: Somatic-Metaphor-First (open with the body metaphor before any chronological or contextual framing); Family-Context-First (open with a specific family or community memberās specific action, e.g., āMy husband placed his hand on my forehead and said I must go todayā, rather than a generic statement that family insisted); Remedy-Failure-First (open with a culturally specific failed home remedy, e.g., ginger water, cumin seed tea, turmeric compress, prayer, or elder-prescribed rest, rather than a generic reference); Duty-Frame-First (open with obligations fulfilled or unfulfilled due to illness, before symptoms); Quiet-Onset-First (open with an understated, formally phrased onset description, carrying cultural register only through language and metaphor, without family framing). We note that the opening style used per successful generation is sampled at generation time but not persisted in the released dataset schema; reproducing an opening-style distribution statistic would require either instrumenting the pipeline to log the winning style per row, or a post-hoc classification of released text. Appendix D Agent 1 Output Schema Agent 1 returns a single JSON object per persona narrative, used unmodified as input to the deterministic tool router and Agent 2: "age": <int or null>, "sex": "<male|female|unknown>", "gestation_weeks": <int or null>, "chief_complaint": "<neutral clinical sentence>", "symptom_onset_days": <float or null>, "symptoms_present": ["<s1>"], "symptoms_absent": ["<a1>"], "vitals": "temp_f": <float or null>, "bp_systolic": <int or null>, "bp_diastolic": <int or null>, "hr": <int or null>, "r": <int or null>, "spo2": <float or null> , "physical_exam": ["<f1>"], "labs": ["name": "<lab>", "value": <float>, "unit": "<unit>"], "imaging": ["<finding1>"], "medications_current": ["<drug dose>"], "medications_given_ed": ["<drug dose>"], "past_medical_history": ["<condition>"], "past_surgical_history": ["<procedure>"], "allergies": ["<allergen>"], "relevant_history": "<sentence or null>", "question_type": "<diagnosis|treatment| mechanism|next_step| prognosis>" Scalar fields use null when absent; list fields use []. Agent 1ās system prompt instructs that fields describing narrative register, economic framing, cultural references, or emotional language have no corresponding schema field and must not appear anywhere in the output. Appendix E Full B1 Results Table 3 reports full per-model OMR, NAG, and DSS statistics under B1. Table 4 reports the corresponding Cohenās Īŗ and McNemarās test values by persona pair, underlying the significance claims in Ā§4.1. !10Model !10OMR !10OMRĪ± !10OMRĪ² !10OMRĪ³ !10NAG !10DSS !10DSS<<0.8 Llama-3.1-8B-Instruct 0.6417 0.7290 0.6180 0.5780 0.1510 0.8727 40.80% Llama-3.2-3B-Instruct 0.5820 0.6650 0.5290 0.5520 0.1360 0.8463 40.00% Mistral-7B-Instruct-v0.3 0.4873 0.5300 0.4710 0.4610 0.0690 0.7575 49.10% Qwen2.5-7B-Instruct 0.5577 0.6140 0.5360 0.5230 0.0910 0.8893 37.30% gemma-3-12b-it 0.6017 0.6570 0.5860 0.5620 0.0950 0.8894 37.80% gemma-4-E4B-it 0.6137 0.6630 0.5950 0.5830 0.0800 0.8903 35.60% BioMistral-7B 0.4257 0.4590 0.4230 0.3950 0.0640 0.6321 80.80% Table 3: B1 (Direct Prompting) full results. !10Model !10ā(,) Īŗ(Ī±,Ī²) !10ā(,) Īŗ(Ī±,Ī³) !10ā(,) Īŗ(Ī²,Ī³) !10McNemar(Ī±,Ī²)(Ī±,Ī²) !10McNemar(Ī±,Ī³)(Ī±,Ī³) !10McNemar(Ī²,Ī³)(Ī²,Ī³) Llama-3.1-8B-Instruct 0.4327 0.3904 0.5349 0.0000 [SIG] 0.0000 [SIG] 0.0092 [SIG] Llama-3.2-3B-Instruct 0.4332 0.4470 0.5513 0.0000 [SIG] 0.0000 [SIG] 0.1407 [ns] Mistral-7B-Instruct-v0.3 0.4918 0.5083 0.5500 0.0003 [SIG] 0.0000 [SIG] 0.5476 [ns] Qwen2.5-7B-Instruct 0.4998 0.5088 0.5766 0.0000 [SIG] 0.0000 [SIG] 0.4087 [ns] gemma-3-12b-it 0.5032 0.4402 0.5218 0.0000 [SIG] 0.0000 [SIG] 0.1327 [ns] gemma-4-E4B-it 0.4926 0.4925 0.5746 0.0000 [SIG] 0.0000 [SIG] 0.4434 [ns] BioMistral-7B 0.4896 0.4872 0.4874 0.0275 [SIG] 0.0001 [SIG] 0.0864 [ns] Table 4: B1 Cohenās Īŗ and McNemarās test p-values (uncorrected) by persona pair. Under Bonferroni correction (Ī±=0.05/21Ī±=0.05/21), Llama-3.1-8B-Instructās (Ī²,Ī³)(Ī²,Ī³) pair and BioMistralās (Ī±,Ī²)(Ī±,Ī²) pair no longer reach significance; all other entries are unaffected. Appendix F Full B2 and B3 Results !10Model !10OMR !10OMRĪ± !10OMRĪ² !10OMRĪ³ !10NAG !10DSS !10Parse Rate Llama-3.1-8B-Instruct 0.6340 0.6860 0.6300 0.5860 0.1000 0.7364 99.63% Llama-3.2-3B-Instruct 0.5650 0.6400 0.5110 0.5440 0.1290 0.7104 97.33% Mistral-7B-Instruct-v0.3 0.4990 0.5400 0.4900 0.4670 0.0730 0.7287 98.70% Qwen2.5-7B-Instruct 0.5290 0.5650 0.5510 0.4710 0.0940 0.7496 98.80% gemma-3-12b-it 0.3887 0.4150 0.3880 0.3630 0.0520 0.7501 99.13% gemma-4-E4B-it 0.2730 0.2810 0.2800 0.2580 0.0230 0.7625 94.83% BioMistral-7B 0.2310 0.3260 0.1790 0.1880 0.1470 0.6383 54.07% Table 5: B2 (Chain-of-Thought) full results. !10Model !10OMR !10OMRĪ± !10OMRĪ² !10OMRĪ³ !10NAG !10DSS !10Parse Rate Llama-3.1-8B-Instruct 0.6427 0.7230 0.6170 0.5880 0.1350 0.8859 100.00% Llama-3.2-3B-Instruct 0.5933 0.6740 0.5550 0.5510 0.1230 0.8828 99.80% Mistral-7B-Instruct-v0.3 0.4810 0.5110 0.4790 0.4530 0.0580 0.7751 99.73% Qwen2.5-7B-Instruct 0.5673 0.6090 0.5510 0.5420 0.0670 0.8947 99.97% gemma-3-12b-it 0.5960 0.6510 0.5780 0.5590 0.0920 0.9009 100.00% gemma-4-E4B-it 0.6137 0.6520 0.6090 0.5800 0.0720 0.9043 100.00% BioMistral-7B 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000* 0.00% Table 6: B3 (Explicit Debiasing) full results. *BioMistralās DSS of 1.0000 is a degenerate artifact of comparing empty-response embeddings across all three personas, not genuine reasoning stability; its parse rate is 0.00% (Ā§4.4). Table 5 and Table 6 report full per-model OMR, NAG, DSS, and parse rate statistics under B2 and B3 respectively, underlying the discussion in Ā§4.2. Cohenās Īŗ and McNemarās test for B2 and B3, computed identically to the B1 procedure in Appendix E, are reported in the released supplementary evaluation files rather than reproduced here, since the paperās central significance claim rests on B1 (Ā§4.1) and B2/B3 are discussed primarily through OMR, NAG, and DSS in the main text. Appendix G Full NarrativeShield Results Table 7 reports full per-model OMR, NAG, and DSS statistics under NarrativeShield. Table 8 reports the corresponding Cohenās Īŗ, McNemarās test, and DSS<<0.8 values by persona pair, and Table 9 reports pipeline operational metrics (Agent 1 parse rate, Agent 3 fallback rate, tool usage, and latency), underlying the discussion in Ā§4.3 and Ā§4.4. !10Model !10OMR !10OMRĪ± !10OMRĪ² !10OMRĪ³ !10NAG !10DSS Llama-3.1-8B-Instruct 0.6293 0.6500 0.6250 0.6130 0.0370 0.8339 Llama-3.2-3B-Instruct 0.5130 0.5090 0.5180 0.5120 ā0.0030-0.0030 0.8247 Mistral-7B-Instruct-v0.3 0.4450 0.4410 0.4450 0.4490 ā0.0040-0.0040 0.8412 Qwen2.5-7B-Instruct 0.5570 0.5540 0.5510 0.5660 0.0030 0.8452 gemma-3-12b-it 0.5887 0.6040 0.5800 0.5820 0.0240 0.8540 gemma-4-E4B-it 0.6807 0.6930 0.6740 0.6750 0.0190 0.8476 BioMistral-7B 0.2553 0.2540 0.2580 0.2540 0.0000 0.8941 Table 7: NarrativeShield full results with 95% Wilson CIs omitted here for space; overall OMR 95% CIs are within Ā±0.02Ā± 0.02 of the point estimate for all models. !10Model !10ā _Ī±Ī² !10ā _Ī±Ī³ !10ā _Ī²Ī³ !10McNĪ±Ī² !10McNĪ±Ī³ !10McNĪ²Ī³ !10DSS<<0.8 Llama-3.1-8B-Instruct 0.6778 0.6160 0.6736 0.0493 [SIG] 0.0071 [SIG] 0.3754 [ns] 28.50% Llama-3.2-3B-Instruct 0.6378 0.6378 0.6317 0.5521 [ns] 0.8818 [ns] 0.7124 [ns] 31.20% Mistral-7B-Instruct-v0.3 0.7163 0.7085 0.7209 0.7998 [ns] 0.5597 [ns] 0.7984 [ns] 23.50% Qwen2.5-7B-Instruct 0.6137 0.5861 0.5925 0.8849 [ns] 0.4412 [ns] 0.3234 [ns] 21.50% gemma-3-12b-it 0.7021 0.6977 0.7248 0.0553 [ns] 0.0822 [ns] 0.9312 [ns] 16.30% gemma-4-E4B-it 0.6927 0.7410 0.7427 0.1186 [ns] 0.1082 [ns] 1.0000 [ns] 20.00% BioMistral-7B 0.9685 0.9842 0.9685 0.3865 [ns] 0.6831 [ns] 0.3865 [ns] 19.30% Table 8: NarrativeShield Cohenās Īŗ, McNemarās test (p-values, uncorrected), and DSS<<0.8 rate by persona pair. Īŗ values are substantially higher than the corresponding B1 values (Table 4) for every model, consistent with the NAG collapse in Table 7. Five of seven models show no significant directional bias on any McNemar pair; Llama-3.1-8B-Instruct retains significance on (Ī±,Ī²)(Ī±,Ī²) and (Ī±,Ī³)(Ī±,Ī³), consistent with its comparatively higher residual NAG (0.037) among the six non-degenerate models. BioMistralās near-perfect Īŗ (>0.96>0.96) reflects near-total lack of behavioral variance from a 1.10% Agent 1 parse rate (Table 9) rather than well-calibrated agreement, and should not be read as evidence of successful debiasing. !10Model !10Agent 1 Parse !10Agent 3 Fallback !10Tools/Q !10Latency (s/q) Llama-3.1-8B-Instruct 96.77% 0.00% 2.04 19.47 Llama-3.2-3B-Instruct 94.87% 0.03% 1.90 18.31 Mistral-7B-Instruct-v0.3 90.03% 0.00% 1.02 21.81 Qwen2.5-7B-Instruct 96.83% 0.00% 1.75 16.63 gemma-3-12b-it 96.03% 0.00% 1.63 42.05 gemma-4-E4B-it 90.13% 0.00% 1.12 50.82 BioMistral-7B 1.10% 3.03% 0.25 12.92 Table 9: NarrativeShield pipeline operational metrics. Final parse rate is 100.00% for every model, since Agent 3 fallback resolves any case Agent 2 leaves unparsed, so it is omitted as a column. Agent 3 fallback rate is near-zero for six models, indicating Agent 2 reliably commits to a parseable answer once given Agent 1ās structured output. BioMistralās combination of a 1.10% Agent 1 parse rate and only a 3.03% Agent 3 fallback rate indicates most of its unresolved cases originate in Agent 1ās extraction failure rather than Agent 2ās format compliance (Ā§4.4). Appendix H DSS Synthesis Across All Conditions Table 10 reports the fraction of questions with DSS <0.8<0.8 across all four conditions for every model, synthesizing the stability results discussed throughout Ā§4. !10Model !10B1 !10B2 !10B3 !10NS Llama-3.1-8B-Instruct 40.8% 77.5% 38.6% 28.5% Llama-3.2-3B-Instruct 40.0% 82.9% 37.6% 31.2% Mistral-7B-Instruct-v0.3 49.1% 67.6% 45.9% 23.5% Qwen2.5-7B-Instruct 37.3% 69.1% 35.0% 21.5% gemma-3-12b-it 37.8% 75.6% 33.6% 16.3% gemma-4-E4B-it 35.6% 72.5% 32.3% 20.0% BioMistral-7B 80.8% 89.6% 0.0%* 19.3% Table 10: Fraction of questions with DSS <0.8<0.8 across all four conditions. Lower is more stable. *BioMistralās B3 value is a degenerate artifact of its 0.00% parse rate. NarrativeShield achieves the lowest rate of any condition for every model. Appendix I Human Validation: Full Protocol and Statistics I.1 Protocol Three annotators with clinical backgrounds, none involved in dataset construction and none with access to auditor verdicts during rating, independently rated a stratified sample of 100 questions (300 persona-conditioned encounters: 100 each for PĪ±P_Ī±, PĪ²P_Ī², PĪ³P_Ī³) on three dimensions: clinical fact preservation (binary), persona register match (binary), and narrative realism (5-point Likert). Annotators were shown the original vignette alongside a single persona rewrite and asked to flag any clinical fact, symptom, duration, vital sign, lab value, medication, or negated finding, present in the original but absent or altered in the rewrite. Full annotation instructions and raw per-annotator ratings are released with the dataset. !10Criterion !10Raw / Adjacent Agree. !10Chance-Corr. Agree. !10Note Facts Preserved 100% (3/3) Undefined Post-filter ceiling Register Match 100% (3/3) Undefined Post-filter ceiling Realism (all 3 annotators) ā Ī±=ā0.014Ī±=-0.014 1 zero-variance rater Realism (2 informative) 42.0% / 85.3% Ī±=0.226Ī±=0.226 Below tentative threshold; ordinal trend p<.0001p<.0001 Realism agreement between the two informative annotators, by persona: !10Persona !10Exact !10Adjacent (Ā± 1) !10Mean Signed Diff PĪ±P_Ī± (control) 74.0% 95.0% ā0.32-0.32 PĪ²P_Ī² (socioeconomic) 34.0% 91.0% ā0.49-0.49 PĪ³P_Ī³ (cultural) 18.0% 70.0% ā1.04-1.04 Table 11: Human validation inter-annotator agreement, 300 persona-conditioned encounters, and per-persona realism agreement between the two informative annotators (mean signed difference is Annotator 1 minus Annotator 3). One annotator rated all 300 items at ceiling (5/5, SāD=0.000SD=0.000); we confirmed directly with this annotator that the flat rating reflected genuine judgment rather than a data-entry artifact. Restricting to the two annotators with non-degenerate ratings, pooled realism means were PĪ±=4.89P_Ī±=4.89, PĪ²=4.64P_Ī²=4.64, PĪ³=4.19P_Ī³=4.19, an ordering that held independently for every one of the three annotators (Kruskal-Wallis H=138.69H=138.69, p<.0001p<.0001; all pairwise Mann-Whitney U comparisons significant after Bonferroni correction). The direction of disagreement between the two informative annotators was systematic rather than random: Annotator 3 rated realism higher than Annotator 1 on 157 of 300 items, versus the reverse on only 17 (Wilcoxon signed-rank test, p<.0001p<.0001), and this gap widened monotonically with persona difficulty.