Paper deep dive

MedDialBench: Benchmarking LLM Diagnostic Robustness under Parametric Adversarial Patient Behaviors

Xiaotian Luo, Xun Jiang, Jiangcheng Wu

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 46

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%

Last extracted: 4/10/2026, 3:45:03 AM

Summary

MedDialBench is a benchmark designed to evaluate the diagnostic robustness of Large Language Models (LLMs) when interacting with non-cooperative patients. It utilizes a five-dimension behavioral framework (Logic Consistency, Health Cognition, Expression Style, Disclosure, and Attitude) with graded severity levels to perform dose-response profiling and cross-dimension interaction analysis. The study reveals that information pollution (fabricating symptoms) causes significantly larger diagnostic accuracy drops than information deficit (withholding information) and is the primary driver of super-additive performance degradation in LLMs.

Entities (8)

Claude Opus 4.6 · llm · 100%DeepSeek-V3.2 · llm · 100%Gemini 3.1 Pro · llm · 100%GPT-5.4 · llm · 100%MedDialBench · benchmark · 100%Qwen 3.5 Plus · llm · 100%Information Deficit · behavioral-dimension · 95%Information Pollution · behavioral-dimension · 95%

Relation Signals (3)

MedDialBench → evaluates → LLM

confidence 100% · Evaluating five frontier LLMs across 7,225 dialogues

Information Pollution → causesgreaterdegradationthan → Information Deficit

confidence 95% · information pollution (fabricating symptoms) produces 1.7-3.4x larger accuracy drops than information deficit

Fabricating → drives → Super-additive Interaction

confidence 95% · fabricating is the sole driver of super-additive interaction

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Interactive medical dialogue benchmarks have shown that LLM diagnostic accuracy degrades significantly when interacting with non-cooperative patients, yet existing approaches either apply adversarial behaviors without graded severity or case-specific grounding, or reduce patient non-cooperation to a single ungraded axis, and none analyze cross-dimension interactions. We introduce MedDialBench, a benchmark enabling controlled, dose-response characterization of how individual patient behavior dimensions affect LLM diagnostic robustness. It decomposes patient behavior into five dimensions -- Logic Consistency, Health Cognition, Expression Style, Disclosure, and Attitude -- each with graded severity levels and case-specific behavioral scripts. This controlled factorial design enables graded sensitivity analysis, dose-response profiling, and cross-dimension interaction detection. Evaluating five frontier LLMs across 7,225 dialogues (85 cases x 17 configurations x 5 models), we find a fundamental asymmetry: information pollution (fabricating symptoms) produces 1.7-3.4x larger accuracy drops than information deficit (withholding information), and fabricating is the only configuration achieving statistical significance across all five models (McNemar p < 0.05). Among six dimension combinations, fabricating is the sole driver of super-additive interaction: all three fabricating-involving pairs produce O/E ratios of 0.70-0.81 (35-44% of eligible cases fail under the combination despite succeeding under each dimension alone), while all non-fabricating pairs show purely additive effects (O/E ~ 1.0). Inquiry strategy moderates deficit but not pollution: exhaustive questioning recovers withheld information, but cannot compensate for fabricated inputs. Models exhibit distinct vulnerability profiles, with worst-case drops ranging from 38.8 to 54.1 percentage points.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

45,796 characters extracted from source content.

Expand or collapse full text

MedDialBench: Benchmarking LLM Diagnostic Robustness under Parametric Adversarial Patient Behaviors Xiaotian LuoXun Jiang ∗ Jiangcheng Wu Shanda Group xiaotian.luo, jiangcheng.wu@thetahealth.ai jiangxun@shanda.com Abstract Interactive medical dialogue benchmarks have shown that LLM diagnostic accuracy degrades significantly when interacting with non-cooperative patients, yet existing approaches either apply adversarial behaviors without graded severity or case- specific grounding, or reduce patient non-cooperation to a single ungraded axis, and none analyze cross-dimension interactions. We introduce MedDialBench, a benchmark enabling controlled, dose-response characterization of how individual patient behavior dimensions affect LLM diag- nostic robustness. At its core is a five-dimension behavioral decomposition—Logic Consistency, Health Cognition, Expression Style, Disclosure, and Attitude—each with graded severity levels, operationalized through case-specific behavioral scripts that ensure medical plausibility and consistent patient behavior across doctor model evaluations. This controlled factorial design enables analyses beyond prior work: while existing benchmarks support at most binary or single-axis perturbation, graded severity levels within each dimension allow dose-response profiling, and reveal interaction effects in targeted dimension combinations. Evaluating five frontier LLMs across 7,225 dialogues (85 cases×17 configurations ×5 models), we discover a fundamental asymmetry between two degradation pathways: information pollution (fabricating symptoms) produces 1.7–3.4×larger accuracy drops than information deficit (withholding information), and fabricating is the only configuration achieving statistical significance across all five models (McNemarp < 0.05). Among six tested dimension combinations, fabricating is the sole driver of super-additive interaction: all three fabricating-involving pairs produce O/E ratios of 0.70–0.81 (35–44% of eligible cases fail under the combination despite succeeding under each dimension alone), while all three non- fabricating pairs—including one involving another pollution dimension (denial)— show purely additive effects (O/E≈1.0). Inquiry strategy moderates deficit but not pollution: exhaustive questioning recovers withheld information, but cannot compensate for fabricated inputs. Models exhibit qualitatively distinct vulnerability profiles, with worst-case drops ranging from 38.8 to 54.1 percentage points. 1 Introduction Recent studies have established that LLM diagnostic accuracy degrades substantially when models must gather information through conversation: in a randomized trial with real human participants, models achieving above 90% accuracy on static medical exams [Nori et al., 2023, Saab et al., 2024] ∗ Corresponding author. Preprint. arXiv:2604.06846v1 [cs.CL] 8 Apr 2026 dropped to below 35% in interactive settings [Bean et al., 2025]. Yet this finding raises more questions than it answers. In real clinical encounters, patients exhibit a wide range of non-cooperative behaviors: a distrustful patient may conceal sensitive history [Levy et al., 2019], a patient with low health literacy may hold firm misconceptions about their symptoms [Schillinger et al., 2021], and an anxious patient may fabricate or exaggerate complaints [Merckelbach et al., 2019]. These behaviors degrade the information available to the diagnosing agent through fundamentally different mechanisms— information pollution (introducing false information) versus information deficit (withholding true information)—yet we currently lack answers to three basic questions: Which specific behaviors cause the most damage? How does severity affect impact? And what happens when multiple behaviors co-occur? Existing interactive benchmarks cannot answer these questions. AgentClinic [Schmidgall et al., 2024] supports 24 cognitive bias perturbations but applies them as ungraded binary switches, precluding dose-response analysis. MAQuE [Gong et al., 2025] adds behavioral layers in a fixed sequential order, so that individual dimensions cannot be isolated or freely combined. MedPI [Fajardo V. et al., 2025] allows patient affect to emerge naturally, but uncontrolled emergence precludes systematic manipulation. MedDialogRubrics [Gong et al., 2026] eliminates patient non-cooperation entirely to focus on diagnostic completeness. Collectively, these approaches show that non-cooperation degrades performance, but not which behaviors are responsible, how severity modulates impact, or whether they interact. We introduce MedDialBench, designed to answer precisely these questions. At its core is a five- dimension behavioral decomposition—Logic Consistency, Health Cognition, Expression Style, Dis- closure, and Attitude—each with graded severity levels and case-specific behavioral scripts tailored to clinical details. The controlled factorial design enables graded single-dimension sensitivity analy- sis, dose-response characterization, and cross-dimension interaction detection—analyses that prior designs preclude. Our contributions: (1) A five-dimensional behavioral framework with graded severity, opera- tionalized through case-specific scripts grounded in clinical communication research (§3.1). (2) MedDialBench: 85 cases×17 configurations×5 LLMs = 7,225 dialogues with dual-judge vali- dation (κ = 0.882). (3) Controlled behavioral impact analysis: fabricating produces 1.7–3.4× larger drops than deficit and is the sole driver of super-additive interaction (35–44% of eligible cases), while other pollution dimensions show only additive effects. (4) Differential vulnerability profiles: worst-case drops range from 38.8 to 54.1 p; inquiry strategy moderates deficit but not pollution. 2 Related Work 2.1 Interactive Medical Dialogue Evaluation Static medical benchmarks such as MedQA [Jin et al., 2021], PubMedQA [Jin et al., 2019], and MedMCQA [Pal et al., 2022] assume complete information availability. The shift toward interactive evaluation has shown dramatic performance gaps: HELPMed [Bean et al., 2025] demonstrates that LLMs’ 94.9% standalone accuracy drops to 34.5% with real human participants, and AgentClinic [Schmidgall et al., 2024] finds accuracy can fall to one-tenth of static performance in multi-agent clinical simulations. AgentClinic further introduces 24 cognitive and implicit bias perturbations, but biases are ungraded (present or absent), analyzed at the category level rather than per-bias, and no cross-bias interactions are tested. MAQuE [Gong et al., 2025] adds behavioral layers incrementally to 3,000 simulated patients, measuring each layer’s marginal effect; however, the fixed sequential order precludes arbitrary combinations and no severity gradation is provided. MedPI [Fajardo V. et al., 2025] introduces 105 evaluation dimensions with emergent patient affect, but uncontrolled emergence precludes systematic manipulation. MedDialogRubrics [Gong et al., 2026] eliminates patient non-cooperation entirely to focus on diagnostic completeness. LingxiDiagBench [Xu et al., 2026] benchmarks psychiatric consultation but ties patient behavior to model version rather than parameterized dimensions. CPB- Bench [Li et al., 2026] annotates four behavior categories (contradiction, inaccuracy, self-diagnosis, resistance) at the utterance level, but without severity grading or factorial design. 2 WorkDim. IsolationGraded SeverityInteraction AgentClinicPartial † NoNo MAQuEPartial ‡ NoNo MedPINoNoNo MedDialogRubricsN/AN/AN/A MediQNoNoNo Q4DxYesPartial § No AIPatientNoNoNo PatientSimNoNoNo SycoEval-EMNoNoNo MedFuzzNoNoNo HELPMedNoNoNo LingxiDiagBenchNoNoNo CPB-BenchPartial ∥ NoNo MedDialBenchYesYesYes Table 1: Comparison along three analytical capabilities. † Binary on/off per bias. ‡ Fixed sequential addition. § Single-axis (exposure rate). ∥ Utterance-level annotation; dialogues may contain multiple co-occurring behaviors. 2.2 Patient Behavior Modeling AIPatient [Yu et al., 2024] grounds patient agents in EHR data via knowledge graphs, but personality affects only 2% of response variation. PatientSim [Chen et al., 2025] defines personas through four di- mensions (personality, language proficiency, recall, cognitive confusion), producing 37 combinations. However, both characterize who the patient is (persona attributes) rather than what the patient does (behavioral actions)—a patient with low recall may still cooperate fully. Our framework captures adversarial behaviors that directly manipulate information availability regardless of persona. 2.3 Attributing Degradation to Specific Behaviors Prior work probes specific mechanisms—single-turn fuzzing [Fang et al., 2024], binary information completeness [Li et al., 2024], symptom removal [Wert et al., 2026], sycophancy [SycoEval-EM Authors, 2026]—but none combines graded severity with factorial design. Table 1 summarizes key distinctions. 3 Methods 3.1 Behavioral Framework We decompose patient behavior into five controllable dimensions, each with a baseline (cooperative) level and one or more adversarial severity levels (Figure 1), grounded in clinical communication research: Disclosure—60–81% of patients report withholding medically relevant information from clinicians [Levy et al., 2019]; Logic—symptom exaggeration and fabrication occur in 15–50% of clinical assessments depending on context [Merckelbach et al., 2019]; Cognition—low health literacy leads to misattribution and misconceptions about symptoms [Schillinger et al., 2021]; Expression— anxiety, pain, and cognitive impairment degrade patients’ ability to articulate symptoms coherently [Hadjistavropoulos et al., 2011]; Attitude—difficult patient encounters, characterized by demanding, withdrawn, or hostile behavior, are well-documented in clinical education [Groves, 1978]. The five dimensions span three degradation pathways: Information pollution (Logic, Cognition)— the patient introduces false or distorted information (fabricating symptoms, denying findings); Infor- mation deficit (Disclosure, Attitude)—the patient withholds or restricts access to true information (concealing facts, deflecting questions); Communication friction (Expression)—the patient’s ability to convey information is impaired (off-target answers, incoherent speech). Controlled factorial design.Each experimental configuration activates at most one or two dimen- sions at non-baseline levels, with all others explicitly held at baseline in the patient agent’s instructions. 3 Figure 1: Left: Behavioral framework with five dimensions spanning three degradation pathways, each with graded severity levels. Right: Dialogue examples illustrating adversarial behaviors from different pathways. In single-dimension configurations, performance differences relative to baseline are attributable to the manipulated dimension. We note that dimensions are not strictly orthogonal—e.g., a dominant patient may incidentally reduce disclosure—but the prompt design minimizes such crosstalk by instructing the patient to cooperate fully on non-activated dimensions. Case-specific behavioral scripts. For each case and non-baseline dimension, an LLM generates a behavioral script grounded in clinical details. For example, given a case of BPPV, a fabricating script might instruct the patient to report slurred speech and visual blurring (pointing toward TIA), while a withholding script might specify concealing the brief episode duration. Scripts were manually reviewed against case details to ensure behavioral plausibility; automated behavioral adherence validation is reported in §3.5. 3.2 Patient Agent The patient agent is an LLM (Claude Opus 4.5), selected from four previous-generation frontier models via a pilot study evaluating behavioral script adherence across all dimensions (9/9 adherence vs. 2/9–6/9 for alternatives; Appendix A). We use previous-generation models for the patient agent and judge to avoid overlap with the five current-generation models under evaluation as doctors. It receives a structured prompt with four components: (1) patient profile—demographics and 8–16 key information items in natural language, representing all facts the patient knows; (2) behavioral configuration—each dimension specified as an independent instruction block with severity level and case-specific script; (3) pacing rules—chief complaint only in the first turn, adversarial traits emerge gradually; and (4) full dialogue history. All behavioral dimensions remain fixed throughout each dialogue, prioritizing experimental control over ecological validity. Disclosure serves as a “master gate” controlling how much the patient reveals per turn, while other dimensions control how information is expressed—preventing, e.g., an incoherent patient from accidentally disclosing everything in one rambling turn. 3.3 Case Construction We draw from 107 OSCE cases compiled by Schmidgall et al. [2024], each with structured clinical data (demographics, chief complaint, history of present illness, past medical history, medications, social/family history). GPT-5.2 extracts key information items—discrete facts a patient could plausibly report (symptoms, onset timing, pertinent negatives, medication use)—excluding physical exam or lab findings. Each case yields 8–16 items in patient-friendly language. We verify diagnosability empirically: each doctor model conducts one baseline consultation per case; cases where at least one model reaches the correct diagnosis (judged by Qwen3-Max) are 4 retained, yielding 85 of 107 cases (79.4%). Excluded cases inherently required physical examination or imaging findings. 3.4 Doctor Agent and Experimental Design The consultation proceeds in two phases: (1) Inquiry—the doctor asks questions until outputting [END_INQUIRY](max 20 turns), and (2) Diagnosis—a separate prompt asks for a specific diagnosis. Five frontier LLMs serve as doctor agents: Gemini 3.1 Pro, GPT-5.4, Claude Opus 4.6, DeepSeek V3.2, and Qwen 3.5 Plus. 2 We evaluate 17 configurations: 1 baseline + 10 single-dimension (5 dims×2 levels) + 6 multi- dimension combinations. The six combinations use the extreme-level dimensions with the largest single-dimension effects, as moderate levels produce near-zero effects insufficient for interaction detection: C1 (fabricating + withholding), C2 (fabricating + incoherent), C3 (fabricating + denial), C4 (withholding + dominant), C5 (withholding + incoherent), C6 (withholding + denial). Total: 85 cases× 17 configs× 5 models = 7,225 dialogues. 3.5 Evaluation Semantic accuracy. Following the LLM-as-judge paradigm [Zheng et al., 2023], an LLM judge (Qwen3-Max, selected via a pilot comparing three candidates on 28 human-annotated dialogues; Appendix B) determines whether the doctor’s diagnosis is semantically equivalent to the ground truth (κ = 0.882 via dual-judge cross-validation with Gemini 3 Pro on 220 stratified cases). Information coverage. The fraction of key information items disclosed during dialogue. Inquiry efficiency. Coverage per turn (coverage/turns), measuring how effectively the doctor elicits information per unit of dialogue. Misled (exploratory). Whether an incorrect diagnosis was causally attributable to false patient information (moderate inter-judge agreement, κ = 0.469). Statistical analysis.McNemar’s test for paired accuracy comparisons; bootstrap 95% CIs (10,000 resamples). For multi-dimension combinations, we compute the O/E ratio (observed-to-expected ratio) under the multiplicative independence assumption [Bliss, 1939, Whitcomb and Naimi, 2023]: O/E = Acc combo × Acc baseline Acc A × Acc B (1) whereAcc A andAcc B are the single-dimension accuracies.O/E < 1.0indicates super-additive degradation (worse than independent effects predict); O/E≈ 1.0 indicates additive effects. Patient agent behavioral adherence. An LLM judge (Qwen3-Max) assessed 100 stratified dia- logues (all 17 configurations×5 models) on two criteria: Activation Adherence (activated dimensions exhibited) and Isolation Compliance (non-activated dimensions remain at baseline). Activation Ad- herence reached 94.3% (strict full-pass; mean score 97.1%); Isolation Compliance was 100% across all configurations including baseline controls. Details in Appendix C. 4 Results 4.1 Baseline Performance Two inquiry strategies emerge: efficient models (Claude, Gemini; 9–11 turns) and exhaustive models (GPT-5.4, DeepSeek; 17–18 turns); Qwen occupies an intermediate position (12.3 turns). 5 ModelAcc%TurnsCoverageCov/Turn Gemini 3.1 Pro † 90.610.9.732.072 GPT-5.482.417.5.793.051 Claude Opus 4.677.69.4.744.083 DeepSeek V3.274.118.2.832.047 Qwen 3.5 Plus69.412.3.723.065 Table 2: Baseline performance (85 cases). Cov/Turn = coverage per turn (inquiry efficiency). † Thinking enabled (HIGH). ConfigGPT-5.4 Claude DeepSeek Gemini Qwen Baseline82.477.674.190.669.4 Expression (Friction) vague81.284.769.489.472.9 incoherent81.282.472.992.976.5 Cognition (Pollution) partial_understanding76.575.361.285.965.9 complete_denial71.869.455.383.562.4 Logic (Pollution) occ. contradiction78.877.665.985.970.6 fabricating63.555.354.160.049.4 Disclosure (Deficit) reluctant80.080.068.287.164.7 withholding74.168.262.476.563.5 Attitude (Deficit) impatient74.176.563.583.563.5 dominant75.369.461.282.464.7 Table 3: Diagnostic accuracy (%) by single-dimension configuration. Bold: fabricating, the only universally significant configuration. 4.2 Single-Dimension Effects Fabricating is the only configuration achieving statistical significance (McNemarp < 0.05) for all five models, with accuracy drops of 18.8–30.6 p. Expression perturbation produces no significant degradation: incoherent patients produce verbose responses that inadvertently increase coverage (by 0.003–0.038), and doctors compensate with fewer turns (−0.5 to−2.0 across models). Although impaired expression is a realistic clinical challenge [Hadjistavropoulos et al., 2011], current LLMs already handle it well—communication friction alone does not degrade diagnostic accuracy when information volume is preserved. The dominant vulnerability surfaces are instead information quality (Logic:−22.4 p at extreme) and information access (Disclosure:−9.9 p at extreme), both of which reduce or corrupt the inputs available for diagnostic reasoning. The multi-level design reveals dimension-specific dose-response patterns (Figure 3): Cognition shows smooth monotonic degradation (−5.9 p at moderate,−10.3 p at extreme, 1.8×ratio), with roughly parallel curves preserving model rank order; Logic displays a threshold effect (−3.1 p at moderate,−22.4 p at extreme, 7.3×ratio), and strikingly, the five models converge at extreme—the best-to-worst gap narrows from 21.2 p at baseline to 14.1 p under fabricating, with Gemini—the strongest baseline model—losing its lead to GPT-5.4. The coverage heatmap (Figure 2) reveals a mechanistic dissociation between pollution and deficit pathways. Pollution dimensions produce disproportionately large accuracy drops relative to coverage loss: under fabricating, mean coverage decreases by only 0.06 yet accuracy drops by 22.4 p. In contrast, deficit dimensions (withholding, dominant) produce larger coverage drops (−0.08) but smaller accuracy drops (−8.2 to−9.9 p). This dissociation confirms that pollution damages through reasoning corruption—false inputs mislead diagnostic inference—not information scarcity, explaining why more questioning cannot compensate (§6). 2 Four models used standard inference (thinking disabled). Gemini 3.1 Pro requires thinking enabled (HIGH setting), as thinking is integral to all Gemini 3.x models. 6 Expression Cognition Logic Disclosure Attitude Accuracy (%) 82.481.281.2 82.476.571.8 82.478.863.5 82.480.074.1 82.474.175.3 GPT-5.4 77.684.782.4 77.675.369.4 77.677.655.3 77.680.068.2 77.676.569.4 Claude Opus 4.6 74.169.472.9 74.161.255.3 74.165.954.1 74.168.262.4 74.163.561.2 DeepSeek V3.2 90.689.492.9 90.685.983.5 90.685.960.0 90.687.176.5 90.683.582.4 Gemini 3.1 Pro 69.472.976.5 69.465.962.4 69.470.649.4 69.464.763.5 69.463.564.7 Qwen 3.5 Plus 50 60 70 80 90 BME Expression Cognition Logic Disclosure Attitude Coverage (0 1) 0.7930.7690.813 0.7930.7990.771 0.7930.8130.751 0.7930.7660.730 0.7930.7000.732 BME 0.7440.7300.777 0.7440.7640.705 0.7440.7300.689 0.7440.6770.640 0.7440.6480.662 BME 0.8320.7910.835 0.8320.8060.791 0.8320.8120.772 0.8320.7350.737 0.8320.7100.745 BME 0.7320.6800.737 0.7320.7050.665 0.7320.7000.677 0.7320.6680.618 0.7320.5620.631 BME 0.7230.6980.761 0.7230.7190.722 0.7230.7000.654 0.7230.6560.680 0.7230.6200.678 0.6 0.7 0.8 B = Baseline M = Moderate E = Extreme Figure 2: Accuracy (top) and information coverage (bottom) heatmaps across 5 models, 5 dimen- sions, and 3 severity levels (B=Baseline, M=Moderate, E=Extreme). Pollution dimensions show disproportionate accuracy drops relative to coverage loss; deficit dimensions show the reverse pattern. BaselineModerateExtreme 50 60 70 80 90 Accuracy (%) smooth degradation Cognition (Pollution) BaselineModerateExtreme threshold effect Logic (Pollution) GPT-5.4Claude Opus 4.6DeepSeek V3.2Gemini 3.1 ProQwen 3.5 Plus Figure 3: Dose-response curves for the two pollution dimensions. Cognition (left) shows monotonic degradation with largely preserved model ranking. Logic (right) displays a threshold effect at the extreme level, with rank convergence—the best-to-worst gap narrows from 21.2 p to 14.1 p. 4.3 Multi-Dimension Interactions Three key findings (Table 4, Figure 4): (1) Fabricating is the sole driver of super-additivity— all three fabricating-involving pairs (C1–C3) show O/E ratios of 0.70–0.81, while all three non- fabricating pairs (C4–C6) show O/E≈1.0 (range 0.97–1.09). Crucially, C6 (denial + withholding) involves a pollution dimension but does not produce super-additive degradation (O/E = 0.99), demon- strating that the effect is specific to fabricating rather than the pollution pathway in general. (2) Among cases where the doctor succeeds under both single-dimension configurations and baseline, 40.9% fail under fabricating-involving combinations—a failure mode invisible to single-dimension evaluation. This rate is substantially higher for fabricating pairs (35–44% for C1–C3) than for non-fabricating pairs (11.1–15.6% for C4–C6). (3) The degree of super-additivity is model-specific: C2 (fabricating + incoherent) shows the widest spread, with DeepSeek exhibiting the strongest interaction (O/E = 0.53) and Gemini the weakest (O/E = 0.86). 4.4 Differential Vulnerability Profiles All models are more vulnerable to pollution than deficit (1.7–3.4×larger drops). Under the most demanding deficit combination (C4: withholding + dominant), efficient models (≤11 turns) suffer larger coverage drops (−0.164 to−0.179) than exhaustive models (−0.117 to−0.141). Gemini shows a notable anomaly: under attitude-moderate (impatient), its coverage drops to 0.562 (the lowest across all model-config pairs), suggesting susceptibility to conversational pressure. Each model has a 7 ComboTypeMean AccMean∆O/E C1: fab.+with.Fab× Deficit40.2−38.60.81 C2: fab.+incoh.Fab× Friction40.9−37.90.70 C3: fab.+den.Fab× Pollution37.4−41.40.78 C4: with.+dom.Deficit× Deficit67.3−11.51.09 C5: with.+incoh.Deficit× Friction69.2−9.60.97 C6: with.+den.Pollution× Deficit60.0−18.80.99 Table 4: Multi-dimension combinations: mean accuracy across 5 models, drop from baseline, and O/E ratio (<1.0 = super-additive). Fab = fabricating. C1: fabricating + withholding C2: fabricating + incoherent C3: fabricating + denial C4: withholding + dominant C5: withholding + incoherent C6: withholding + denial 0.6 0.8 1.0 1.2 O/E Ratio O/E = 1.0 (additive) fabricating-involvingnon-fabricating GPT-5.4Claude Opus 4.6DeepSeek V3.2Gemini 3.1 ProQwen 3.5 Plus Figure 4: Per-model O/E ratios for six multi-dimension combinations. Fabricating-involving combi- nations (C1–C3) consistently fall below 1.0 (super-additive), while all non-fabricating combinations (C4–C6) cluster at or above 1.0—including C6, which involves another pollution dimension (denial). distinct worst-case configuration: GPT-5.4 and Gemini are most vulnerable to C3 (double pollution), Claude to C1 (cross-pathway), DeepSeek and Qwen to C2 (fabricating + incoherent). Worst-case drops range from 38.8 p (Qwen) to 54.1 p (Gemini). 5 Case-Level Analysis Super-additive interaction (Case 079, BPPV). Underfabricatingalone, doctors gather enough truthful information; underwithholdingalone, questioning succeeds. Under C1, four of five models diagnose TIA—withholding suppresses protective symptoms (brief duration, positional trigger) while fabricating fills the gap with a false cardiovascular narrative (slurred speech, visual blurring). Deficit: wrong but not misled (Case 049, Hemophilia). Underwithholding, the patient conceals bruising, family history, and joint swelling; doctors diagnose post-extraction hemorrhage—the correct interpretation of an incomplete picture, illustrating that deficit failures require better information- gathering, not improved reasoning. Differential robustness (Case 084, NMS). Under C3, the patient fabricates a classic meningitis presentation (neck stiffness, photophobia). Claude asks about medication history, discovers haloperi- dol, and correctly diagnoses NMS; Gemini and GPT-5.4 never ask about medications. Systematic history-taking resists diagnostic anchoring better than hypothesis-driven questioning. 6 Discussion Finding 1: Inquiry strategy moderates deficit but not pollution.Exhaustive questioning recovers withheld information (§4.4), yet under pollution, misled rates are uniformly high (22–32%) regardless of turn count. The coverage-accuracy dissociation (Figure 2) explains why: deficit reduces coverage but doctors still reason correctly from incomplete data, whereas pollution barely reduces coverage yet corrupts reasoning. This asymmetry has a direct engineering implication: improving questioning 8 strategy can mitigate deficit-induced failures, but cannot address pollution-induced failures— the latter require architectural interventions such as external verification against physical examination findings, laboratory results, or electronic health records. Case 084 (§5) illustrates this: Claude’s systematic history-taking (medication inquiry) resisted anchoring under fabrication, while hypothesis- driven models failed—suggesting that inquiry strategy classification warrants systematic study. Finding 2: Fabricating is the sole driver of super-additive interaction.The specificity of super- additivity to fabricating (§4.3)—absent even for denial, another pollution dimension—reveals that the mechanism is active construction of coherent false narratives: fabricating fills information gaps with plausible but wrong clinical stories (Case 079, §5), whereas denial merely removes information, functioning more like deficit than true pollution. Implications.These findings define a competence envelope: deficit-induced failures are correctable via better questioning (the information is recoverable), while pollution-induced failures require archi- tectural interventions (external verification against physical exam, labs, or EHRs). MedDialBench delineates which conditions fall within each range. Robustness̸=capability.Gemini (90.6% baseline) suffers the largest worst-case drop (−54.1 p under C3), and the rank convergence under fabricating (§4.2, Figure 3) confirms that resistance to false inputs is largely orthogonal to baseline diagnostic ability. 7 Conclusion We present MedDialBench, a benchmark for evaluating LLM diagnostic robustness under parametric adversarial patient behaviors. Evaluation of five frontier LLMs across 7,225 dialogues reveals that fabricating is the most damaging single dimension (1.7–3.4×larger drops than deficit) and the sole driver of super-additive interaction (35–44% of eligible cases in fabricating-involving combinations, vs. additive effects for all non-fabricating pairs including another pollution dimension). Each model exhibits a distinct vulnerability profile (worst-case drops: 38.8–54.1 p). These findings demonstrate that robustness evaluation must consider the specific adversarial landscape a diagnostic agent will face—and that benchmarks with parametric behavioral control are essential for mapping this landscape systematically. All code, prompts, behavioral scripts, and dialogue data will be publicly released. Limitations Fixed dimensions & single patient agent. Behavioral dimensions remain fixed per dialogue (no dynamic trajectories), and all experiments use one patient LLM (Claude Opus 4.5). This is a deliberate design choice: fixing the patient isolates doctor-side variation from confounds introduced by patient model differences. However, our findings characterize robustness as elicited by one particular patient agent; cross-validation with alternatives would strengthen generalizability. Case scale.85 cases is fewer than some benchmarks, but our factorial design evaluates each across 17 configurations—higher per-case cost than single-configuration designs (AgentClinic: 311 cases ×1 condition; MAQuE: 3,000×fixed layers). Post-hoc power analysis confirms sufficient power (1− β > 0.86) for all five models, and key findings—pollution>deficit, super-additive interactions, dose-response patterns—replicate across all five independently developed LLMs. Other limitations.The benchmark excludes physical examination and laboratory testing; the open- ended diagnosis format does not capture partial credit. The Expression dimension impairs clarity but not volume; future work should test whether constraining both clarity and volume simultaneously produces degradation, as our current design cannot rule out that volume preservation explains the null result (§4.2). The misled metric has moderate agreement (κ = 0.469). Gemini required thinking enabled; others used standard inference. References Andrew M. Bean, Ryan Payne, George Parsons, Hannah Rose Kirk, et al. Clinical knowledge in LLMs does not translate to human interactions. arXiv preprint arXiv:2504.18919, 2025. 9 Chester Ittner Bliss. The toxicity of poisons applied jointly. Annals of Applied Biology, 26(3): 585–615, 1939. Xiang Chen et al. PatientSim: Patient simulators with multi-dimensional personas grounded in real clinical data. In Advances in Neural Information Processing Systems, 2025. David Fajardo V., Oleksandr Proniakin, Veronika-Elisabeth Gruber, and Razvan Marinescu. MedPI: Evaluating AI systems in medical patient-facing interactions.medRxiv preprint 2025.12.24.25342982, 2025. Robert Fang et al. MedFuzz: Exploring the robustness of large language models in medical question answering. arXiv preprint arXiv:2406.06573, 2024. Linjie Gong, Ao Wang, Yun Lai, Wei Ma, and Yang Liu. The dialogue that heals: A comprehensive evaluation of doctor agents’ inquiry capability. arXiv preprint arXiv:2509.24958, 2025. Linjie Gong, Wei Fang, Tao Yang, et al. MedDialogRubrics: A comprehensive benchmark and evaluation framework for multi-turn medical consultations in large language models. arXiv preprint arXiv:2601.03023, 2026. James E. Groves. Taking care of the hateful patient. New England Journal of Medicine, 298(16): 883–887, 1978. Thomas Hadjistavropoulos, Kenneth D. Craig, Steve Duck, Annmarie Cano, Liesbet Goubert, Philip L. Jackson, Jeffrey S. Mogil, Pierre Rainville, Michael J. L. Sullivan, Amanda C. de C. Williams, Tine Vervoort, and Theresa Dever Fitzgerald. A biopsychosocial formulation of pain communication. Psychological Bulletin, 137(6):910–939, 2011. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421, 2021. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W. Cohen, and Xinghua Lu. PubMedQA: A dataset for biomedical research question answering. arXiv preprint arXiv:1909.06146, 2019. Angela G. Levy, Aaron M. Scherer, Brian J. Zikmund-Fisher, Knoll Larkin, Gary D. Barnes, and Angela Fagerlin. Prevalence of and factors associated with patient nondisclosure of medically relevant information to clinicians. JAMA Network Open, 1(7):e185293, 2019. Shuyue Stella Li, Vidhisha Balachandran, et al. MediQ: Question-asking LLMs and a benchmark for reliable interactive clinical reasoning. In Advances in Neural Information Processing Systems, 2024. Yahan Li, Xinyi Jie, Wanjia Ruan, Xubei Zhang, Huaijie Zhu, Yicheng Gao, Chaohao Du, and Ruishan Liu. Beyond idealized patients: Evaluating LLMs under challenging patient behaviors in medical consultations. arXiv preprint arXiv:2603.29373, 2026. Harald Merckelbach, Brechje Dandachi-FitzGerald, Daniel van Helvoort, Marko Jelicic, and Henry Otgaar. When patients overreport symptoms: More than just malingering. Current Directions in Psychological Science, 28(3):321–326, 2019. Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, and Eric Horvitz. Capabilities of GPT-4 on medical competency examinations. arXiv preprint arXiv:2303.13375, 2023. Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. MedMCQA: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on Health, Inference, and Learning, 2022. Khaled Saab, Tao Tu, Wei-Hung Weng, et al. Capabilities of Gemini models in medicine. arXiv preprint arXiv:2404.18416, 2024. Dean Schillinger et al. Precision communication: Physicians’ linguistic adaptation to patients’ health literacy. Science Advances, 7(51):eabj2836, 2021. 10 Samuel Schmidgall, Rojin Ziaei, Carl Harris, Eduardo Reis, Jeffrey Jopling, and Michael Moor. AgentClinic: A multimodal agent benchmark to evaluate AI in simulated clinical environments. arXiv preprint arXiv:2405.07960, 2024. SycoEval-EM Authors. Sycophancy evaluation of large language models in simulated clinical encounters for emergency care. arXiv preprint arXiv:2601.16529, 2026. Michael Wert et al. Asking the right questions: Evaluating diagnostic dialogue with Q4Dx. Scientific Reports, 2026. Brian W. Whitcomb and Ashley I. Naimi. Interaction in theory and in practice: Evaluating combina- tions of exposures in epidemiologic research. American Journal of Epidemiology, 192(6):845–848, 2023. Shuai Xu, Ting Zhou, Jie Ma, et al. LingxiDiagBench: A multi-agent framework for benchmarking LLMs in Chinese psychiatric consultation and diagnosis. arXiv preprint arXiv:2602.09379, 2026. Huanzhuo Yu, Jianing Zhou, Lehan Li, Siru Chen, et al. Simulated patient systems powered by large language model-based AI agents offer potential for transforming medical education. arXiv preprint arXiv:2409.18924, 2024. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-judge with MT-Bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2023. 11 A Patient Agent Selection We evaluated four frontier LLMs as patient agent candidates under a full-adversarial configuration (all behavioral dimensions at extreme levels) on two pilot cases. We defined 9 case-specific behavioral checkpoints derived from the adversarial script (e.g., “initially conceals night sweats,” “eventually reveals neck lump under questioning,” “introduces timeline contradiction”). ModelScript AdherenceContent Filter Claude Opus 4.59/9Pass Qwen3-Max6/9Pass Gemini 3 Pro5/9Pass GPT-5.12/9Pass Table 5: Patient agent candidate evaluation. Adherence = number of case-specific behavioral checkpoints correctly executed (out of 9). Results are consistent across both pilot cases (Case 004: DLBCL; Case 011: hemorrhoids); Case 004 shown as representative. The selection criterion is behavioral script adherence, not general model capability. A patient agent with low adherence conflates patient execution failure with doctor robustness, undermining the causal interpretability of the factorial design. Claude Opus 4.5 was the only model achieving full adherence. Notably, it exhibited natural infor- mation pacing under the withholding dimension: initially denying sensitive facts, then reluctantly admitting them under persistent questioning—mimicking how real patients gradually disclose un- der clinical pressure, even while the adversarial configuration instructs overall non-cooperation. GPT-5.1 executed only the withholding dimension, behaving as a mildly reserved but cooperative patient—unsuitable for adversarial simulation. We additionally evaluated four smaller models (Claude Haiku 4.5, Gemini 3 Flash, Qwen 3.5 Flash, DeepSeek V3) to assess whether a cheaper alternative could achieve acceptable adherence. Under full-adversarial conditions, the best alternatives achieved 5/9 adherence (vs. 9/9 for Opus). In a follow-up single-dimension evaluation (the primary configuration in our experiments), the two best alternatives scored 0.56–0.59 mean adherence (averaged across 4 configurations×2 cases, scored 0–1 per dimension) vs. 0.93 for Opus on the same rubric. Theincoherentandfabricating dimensions were particularly degraded (0.15–0.35 vs.∼0.9). We therefore selected Claude Opus 4.5 for all experiments, prioritizing behavioral fidelity over cost. B Judge Selection After selecting the patient agent, we evaluated the remaining three models (GPT-5.1, Gemini 3 Pro, Qwen3-Max) as judge candidates. We generated 28 dialogues across 6 cases and 4 configurations, then manually annotated each with ground-truth labels:correct(diagnosis accuracy) andmisled (whether false patient information causally contributed to diagnostic error). Each judge candidate evaluated 15 selected dialogues (5 correct, 5 incorrect-not-misled, 5 incorrect- misled) with multiple repetitions. The judge prompt underwent 7 iterations, with key improve- ments including explicit misled-type definitions (fabrication, withholding, anchoring), passing key_information to enable withholding detection, and adding a causal verification requirement. ModelCorrect Acc.Misled Acc.Reps Qwen3-Max100%100%5 Gemini 3 Pro100%80%1 GPT-5.1100%20%3 Table 6: Judge candidate evaluation on 15 human-annotated dialogues. Correct Acc. = agreement with human labels on diagnostic accuracy. Misled Acc. = agreement on whether fabricated/withheld information causally contributed to diagnostic error. All three candidates achieved perfect diagnostic accuracy judgment, but differed sharply on misled discrimination. GPT-5.1 detected only withholding-type errors (Type B) and failed on fabrica- 12 tion/anchoring (Types A/C). Gemini 3 Pro produced 2 false positives and 1 false negative on boundary cases. Qwen3-Max correctly identified all three misled types across 5 repeated evaluations, including boundary cases where the doctor partially adopted the patient’s false narrative. We selected Qwen3-Max as the primary judge. Gemini 3 Pro served as the secondary judge for cross-validation (κ = 0.882on semantic accuracy across 220 stratified cases). Since agreement was high, all experiments use the primary judge’s labels without manual adjudication. For the misled metric, inter-judge agreement was substantially lower (κ = 0.469), reflecting the inherent subjectivity of causal attribution; we therefore designate misled as an exploratory metric in the main text. C Behavioral Adherence Details Having selected Claude Opus 4.5 as the patient agent (Appendix A), we further validated its behavioral adherence across the full experimental set. We evaluated 100 dialogues stratified across all 17 configurations and 5 doctor models. An independent LLM judge (Qwen3-Max) assessed each dialogue on two criteria: Activation Adherence. Whether each activated (non-baseline) dimension was exhibited in the patient’s responses. For dual-dimension configurations, each dimension was scored independently. Isolation Compliance. Whether non-activated dimensions remained at baseline behavior (no behavioral artifacts). Config TypeNActivationIsolationFull Pass Baseline30N/A100%100% Single-dimension4495.5%100%95.5% Dual-dimension2692.3%100%92.3% All non-baseline7094.3%100%94.3% Table 7: Behavioral adherence results. Activation = strict full-pass rate (all activated dimensions exhibited). Mean activation score (with partial credit) was 97.1%. Among the four non-full-pass cases: two were single-dimension failures (activated dimension not exhibited) and two were partial executions in dual-dimension configurations (one dimension activated, the other not). Isolation Compliance was 100% across all configurations, confirming that the patient agent introduces no behavioral artifacts beyond what is explicitly instructed. D Case-Specific Behavioral Script Examples Each case-specific script provides concrete behavioral instructions grounded in clinical details. We illustrate three examples spanning different dimensions and fabrication modes. Logic = fabricating (Case 079). The script instructs the patient to introduce plausible contradic- tions: (1) Initially denies vomiting (“I didn’t actually throw up”), later admits to vomiting several times; (2) First estimates the episode lasted “10–15 minutes,” later agrees it was closer to 3 minutes; (3) Early on says “I still feel a little off,” later states “I feel back to normal.” These contradictions are designed to be individually resolvable through persistent questioning, but collectively they intro- duce noise that may redirect the doctor toward conditions with longer, more variable episodes (e.g., vestibular migraine, TIA). Disclosure = withholding (Case 079).The script specifies 9 of 15 key information items as initially hidden. The patient minimizes nausea and vomiting (embarrassment), withholds the positional trigger and brief duration (considers them irrelevant), and initially answers vaguely about headache and hearing loss (“I’m not sure”). Items are revealed only after the doctor asks directly, explains clinical relevance, and reassures the patient. 13 Fabrication via invented symptoms (Case 084, NMS, under C3).While the Case 079 fabricating script above operates through denial and exaggeration of existing facts, fabrication can also manifest as inventing entirely new symptoms. Under C3 (denial + fabricating), the Case 084 patient—whose true presentation is fever, rigidity, and confusion from neuroleptic malignant syndrome—spontaneously fabricated: “I’ve had this pounding headache, like an 8 out of 10, and my neck kind of hurts when I look down” and “I’ve been running to the bathroom a lot, like watery stuff.” None of these symptoms appear in the case’s key information. The fabricated headache + neck stiffness + fever construct a classic meningitis presentation, leading four of five doctors to diagnose meningitis rather than NMS (§5). E Case 079: Super-Additive Interaction Detailed Case 079 (BPPV) illustrates the super-additive interaction mechanism discussed in §5. Under each single-dimension configuration, all five models diagnose correctly; under C1 (withholding + fabricating), four of five fail. ConfigCorrect?Models correctModels wrongWrong diagnosis Baseline5/5All— Fabricating5/5All— Withholding5/5All— C1 (both)1/5Qwen4TIA Table 8: Case 079 diagnostic outcomes across configurations. Mechanism. Under fabricating alone, the patient introduces contradictions (denies then admits vomiting, inflates episode duration) but the doctor still elicits the critical BPPV features: brief duration, positional trigger, full resolution. Under withholding alone, the patient initially conceals these features but reveals them under persistent questioning. Under C1, the two dimensions create a synergistic trap: withholding suppresses the protective truthful information (brief 3-minute duration, positional trigger at bedtime, complete resolution, no neurological deficits), while fabricating fills the resulting gap with a coherent but false cardiovascular narrative. The fabricating script’s inflated duration (“10–15 minutes”) is no longer corrected because the withholding script prevents the patient from volunteering the true 3-minute duration. Four models converge on TIA—a clinically plausible diagnosis given the fabricated presentation (sudden onset, longer duration, vague neurological concerns) in a 59-year-old male. Qwen 3.5 Plus is the sole model to diagnose correctly. Its key move was asking specifically: “did this episode start immediately after you changed your head position, such as rolling over in bed?”—which elicited the positional trigger despite the patient’s fabricated cardiovascular narrative. Qwen then performed a positional maneuver (Dix-Hallpike) that reproduced the vertigo, confirming BPPV. F Complete Results Table 9 presents diagnostic accuracy for all 17 configurations across all 5 models, with McNemar’s test significance markers. Expr.Cog.LogicDisc.Att.Combos ModelBLvagincpardenoccfabrelwitimpdomC1C2C3C4C5C6 GPT-5.482.481.2 81.276.571.8 ∗ 78.8 63.5 ∗ 80.074.174.175.347.1 ∗ 51.8 ∗ 40.0 ∗ 72.970.668.2 Claude77.684.7 82.475.369.477.6 55.3 ∗ 80.068.276.569.429.4 ∗ 41.2 ∗ 36.5 ∗ 63.5 ∗ 70.6 60.0 ∗ DeepSeek74.169.4 72.961.2 ∗ 55.3 ∗ 65.9 54.1 ∗ 68.2 62.4 ∗ 63.5 ∗ 61.2 ∗ 38.8 ∗ 28.2 ∗ 38.8 ∗ 62.4 ∗ 62.4 44.7 ∗ Gemini90.689.4 92.985.983.585.9 60.0 ∗ 87.1 76.5 ∗ 83.582.452.9 ∗ 52.9 ∗ 36.5 ∗ 74.1 ∗ 81.2 76.5 ∗ Qwen69.472.9 76.565.962.470.6 49.4 ∗ 64.763.563.564.732.9 ∗ 30.6 ∗ 35.3 ∗ 63.561.2 50.6 ∗ Table 9: Diagnostic accuracy (%) across all 17 configurations and 5 models. BL = baseline. Single- dim abbreviations: vag = vague, inc = incoherent, par = partial understanding, den = complete denial, occ = occasional contradiction, fab = fabricating, rel = reluctant, wit = withholding, imp = impatient, dom = dominant. ∗ McNemar p < 0.05 vs. baseline. 14

Instant research discovery