Paper deep dive

Dual- versus Single-Suggestion AI Support for Radiographic Interpretation in Residents: Randomized Multireader Study

Lin Wu, Zhe Xu, Hongyi Wang, Feifei Zhou, Wei Deng, Chunlong Zhang, Yuting Zhu, Kaixiao Chen, Xiao Liang, Chen Yang, Yeyuan Chen, Hao Chen, Fuqing Zhou

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 83

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%

Last extracted: 10/11/2026, 2:49:36 AM

Summary

This randomized multicenter study compared the efficacy of single versus dual AI suggestion support for radiographic interpretation by residents. The study involved 123 residents (radiology and non-radiology) interpreting 60 radiographs. Results indicated that dual-suggestion support (combining GPT-5.4 with either Kimi-K2.6 or Gemini-3.6 Flash) significantly improved diagnostic accuracy compared to single-suggestion support (GPT-5.4 alone), particularly when the primary AI suggestion was incorrect. This benefit was statistically significant for radiology residents but not for non-radiology residents, suggesting that dual-suggestion support mitigates the impact of erroneous AI suggestions, especially among specialists.

Entities (8)

Gemini 3.6 Flash · ai-model · 99%GPT-5.4 · ai-model · 99%Kimi-K2.6 · ai-model · 99%Non-Radiology Residents · user-group · 98%Radiology Residents · user-group · 98%Dual-Suggestion Support · intervention · 97%Single-Suggestion Support · intervention · 97%Diagnostic Accuracy · metric · 95%

Relation Signals (6)

Dual-Suggestion Support → uses → GPT-5.4

confidence 99% · GPT-5.4 was the shared model; both AI suggestions were presented simultaneously

Dual-Suggestion Support → uses → Kimi-K2.6

confidence 99% · groups B and C received GPT-5.4 plus Kimi-K2.6

Dual-Suggestion Support → uses → Gemini 3.6 Flash

confidence 99% · groups B and C received GPT-5.4 plus Gemini-3.6 Flash

Dual-Suggestion Support → benefits → Radiology Residents

confidence 93% · Radiology residents showed greater accuracy improvement with dual- than single-suggestion support... whereas accuracy change did not differ in non-radiology residents

Dual-Suggestion Support → improves → Diagnostic Accuracy

confidence 92% · Radiology residents showed greater accuracy improvement with dual- than single-suggestion support

Dual-Suggestion Support → mitigates → Erroneous AI Suggestions

confidence 90% · When GPT-5.4 was incorrect, AI-assisted accuracy was higher with dual- than single-suggestion support

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Purpose: To compare dual- and single-suggestion AI support for radiographic interpretation by residents, particularly when the shared AI suggestion was incorrect. Materials and Methods: This prospective, multicenter, randomized three-arm reader study was conducted at three hospitals in China from July to September 2026 (ChiCTR2600129243). After specialty stratification, 132 residents with fewer than 3 years of clinical experience were randomized 1:1:1 to GPT-5.4 alone (group A), GPT-5.4 plus Kimi-K2.6 (group B), or GPT-5.4 plus Gemini-3.6 Flash (group C); 123 were analyzed. Participants interpreted 60 radiographs before and after AI support. The primary outcome was accuracy change. Welch ANOVA and Holm-adjusted t tests compared support conditions; HC3 linear models assessed specialty interaction. Results: Among 123 residents (mean age, 24.1 years +/- 1.4; 65 women), radiology residents showed greater accuracy improvement with dual- than single-suggestion support (B-A, 6.69 percentage points [95% CI, 0.97-12.40]; C-A, 7.87 percentage points [95% CI, 1.64-14.11]; Holm-adjusted P = .030 for both), whereas accuracy change did not differ in non-radiology residents (P = .20). When GPT-5.4 was incorrect, AI-assisted accuracy was higher with dual- than single-suggestion support in radiology residents (40.1% and 40.4% vs 20.0%) and non-radiology residents (31.3% and 31.0% vs 12.1%) (all Holm-adjusted P < .001). The dual-suggestion effect differed by specialty (interaction difference, 10.44 percentage points; 95% CI, 4.36-16.52; P < .001). Conclusion: Dual-suggestion support may mitigate the influence of erroneous AI suggestions, with greater accuracy improvement observed in radiology but not non-radiology residents.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

79,517 characters extracted from source content.

Expand or collapse full text

Dual- versus Single-Suggestion AI Support for Radiographic Interpretation in Residents: Randomized Multireader Study Lin Wu Affiliation: Department of Radiology, The First Affiliated Hospital, Jiangxi Medical College, Nanchang University Affiliation: Jiangxi Province Medical Imaging Research Institute Affiliation: Lin Wu and Zhe Xu contributed equally to this work. Zhe Xu Affiliation: Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, China Affiliation: Lin Wu and Zhe Xu contributed equally to this work. Hongyi Wang Affiliation: Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, China Feifei Zhou Affiliation: Department of Radiology, Shangrao City People’s Hospital, Shangrao, China Wei Deng Affiliation: Department of Radiology, Nanchang People’s Hospital, Nanchang, China Chunlong Zhang Affiliation: Department of Radiology, The First Affiliated Hospital, Jiangxi Medical College, Nanchang University Affiliation: Jiangxi Province Medical Imaging Research Institute Yuting Zhu Affiliation: Department of Radiology, The First Affiliated Hospital, Jiangxi Medical College, Nanchang University Affiliation: Jiangxi Province Medical Imaging Research Institute Kaixiao Chen Affiliation: Department of Radiology, The First Affiliated Hospital, Jiangxi Medical College, Nanchang University Affiliation: Jiangxi Province Medical Imaging Research Institute Xiao Liang Affiliation: Department of Radiology, The First Affiliated Hospital, Jiangxi Medical College, Nanchang University Affiliation: Jiangxi Province Medical Imaging Research Institute Chen Yang Affiliation: Department of Radiology, The First Affiliated Hospital, Jiangxi Medical College, Nanchang University Affiliation: Jiangxi Province Medical Imaging Research Institute Yeyuan Chen Affiliation: Department of Radiology, The First Affiliated Hospital, Jiangxi Medical College, Nanchang University Affiliation: Jiangxi Province Medical Imaging Research Institute Affiliation: Corresponding Authors Hao Chen Affiliation: Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, China Affiliation: Department of Chemical and Biological Engineering, Hong Kong University of Science and Technology, Hong Kong SAR, China Affiliation: Division of Life Science, Hong Kong University of Science and Technology, Hong Kong SAR, China Affiliation: State Key Laboratory of Nervous System Disorders, The Hong Kong University of Science and Technology, Hong Kong SAR, China Affiliation: HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute, The Hong Kong University of Science and Technology, Futian, Shenzhen, China Affiliation: Corresponding Authors Fuqing Zhou Affiliation: Contact: Fuqing Zhou (ndyfy02301@ncu.edu.cn) Affiliation: Department of Radiology, The First Affiliated Hospital, Jiangxi Medical College, Nanchang University Affiliation: Jiangxi Province Medical Imaging Research Institute Affiliation: Corresponding Authors Abstract Purpose: To compare dual- and single-suggestion AI support for radiographic interpretation by residents, particularly when the shared AI suggestion was incorrect. Materials and Methods: This prospective, multicenter, randomized three-arm reader study was conducted at three hospitals in China from July to September 2026 (ChiCTR2600129243). After specialty stratification, 132 residents with fewer than 3 years of clinical experience were randomized 1:1:1 to GPT-5.4 alone (group A), GPT-5.4 plus Kimi-K2.6 (group B), or GPT-5.4 plus Gemini-3.6 Flash (group C); 123 were analyzed. Participants interpreted 60 radiographs before and after AI support. The primary outcome was accuracy change. Welch ANOVA and Holm-adjusted t tests compared support conditions; HC3 linear models assessed specialty interaction. Results: Among 123 residents (mean age, 24.1 years ± 1.4; 65 women), radiology residents showed greater accuracy improvement with dual- than single-suggestion support (B–A, 6.69 percentage points [95% CI, 0.97–12.40]; C–A, 7.87 percentage points [95% CI, 1.64–14.11]; Holm-adjusted P = .030 for both), whereas accuracy change did not differ in non-radiology residents (P = .20). When GPT-5.4 was incorrect, AI-assisted accuracy was higher with dual- than single-suggestion support in radiology residents (40.1% and 40.4% vs 20.0%) and non-radiology residents (31.3% and 31.0% vs 12.1%) (all Holm-adjusted P < .001). The dual-suggestion effect differed by specialty (interaction difference, 10.44 percentage points; 95% CI, 4.36–16.52; P < .001). Conclusion: Dual-suggestion support may mitigate the influence of erroneous AI suggestions, with greater accuracy improvement observed in radiology but not non-radiology residents. 1 Introduction Artificial intelligence (AI) has substantial potential to improve the accuracy and efficiency of radiographic interpretation[1, 2, 3]. By combining imaging findings with clinical information, AI can provide diagnostic suggestions and supporting rationales[4, 5]. However, the effectiveness of AI assistance depends not only on model performance but also on how clinicians respond to AI suggestions, particularly when those suggestions are erroneous[6, 7]. Previous studies have examined how information presentation within a single-AI framework influences clinicians’ responses to AI suggestions, including diagnostic conclusions alone or with confidence scores[8, 9], content-based image retrieval[10], visual heatmaps[10], and text-based explanations[2, 11]. Evidence suggests that less experienced physicians may derive greater benefit from AI assistance, yet they are also more susceptible to erroneous AI suggestions[12, 8, 13]. Failure to recognize and reject erroneous AI suggestions can lead to diagnostic errors[14, 15]. Limiting the diagnostic influence of erroneous AI suggestions among less experienced physicians is an important objective in human–AI interaction[7, 16]. Drawing on multiple independent opinions is an established approach to reducing reliance on a single judgment, consistent with the principle of collective intelligence[17, 18]. In medical diagnosis, combining multiple independent judgments has been shown to outperform individual judgments across a range of medical cases[19]. Similarly, structured information sharing among clinicians has been shown to reduce diagnostic errors and improve individual clinical decisions compared with isolated reflection[20]. Applied to AI-assisted diagnosis, presenting two independent AI suggestions may provide an additional diagnostic reference when one suggestion is erroneous[21]. However, whether dual-suggestion support can mitigate the diagnostic impact of erroneous AI suggestions among residents during radiographic interpretation remains unclear. Building on the principle of collective intelligence[21, 22], we evaluated dual-suggestion support as a human–AI interaction strategy. We hypothesized that dual-suggestion support would show greater improvement in diagnostic accuracy than single-suggestion support, particularly when the shared AI suggestion was incorrect. We conducted a three-arm randomized study in which participants interpreted each case first without AI assistance and then with the assigned single- or dual-suggestion support. The primary outcome was change in diagnostic accuracy after AI assistance. Diagnostic accuracy was also assessed according to whether the shared AI suggestion was correct or incorrect. Secondary outcomes included interpretation time and confidence change, and effects were also evaluated by specialty. The purpose of this study was to compare dual- and single-suggestion AI support for radiographic interpretation by residents, particularly when the shared AI suggestion was incorrect. 2 Materials and Methods 2.1 Study Design and Participants This multicenter, prospective, randomized, three-arm, parallel-group multireader study was conducted at three hospitals in China from July to September 2026. The study was approved by the institutional ethics committee (2026-756), which waived informed consent, and was registered with the Chinese Clinical Trial Registry (ChiCTR2600129243). Residents were recruited through advertisements. Eligibility required a valid Chinese physician license, prior training in radiographic interpretation, and fewer than 3 years of clinical experience. Participants were classified a priori as radiology residents or non-radiology residents from internal medicine, general surgery, or emergency medicine. Participants with incomplete assessments due to network interruption or physical discomfort were excluded from the final analysis (Figure 1). Figure 1: Participant Flow Diagram. Participants were stratified by specialty and allocated 1:1:1 to the three AI support conditions. Testing was performed using a smartphone or tablet. Network failure was defined as connectivity or webpage failure preventing image display or response submission; physical discomfort was defined as feeling unwell enough to interfere with concentration and require termination of testing. Final analytic samples are shown by specialty. 2.2 Case Selection and AI Output Generation We retrospectively assembled 218 chest and abdominal radiography examinations performed at Center 1 from January 2023 to June 2024 according to prespecified diagnostic categories. Examinations represented four chest and three abdominal conditions, were of sufficient image quality, and had diagnoses confirmed pathologically or clinically. GPT-5.4 (OpenAI), Kimi-K2.6 (Moonshot AI), and Gemini-3.6 Flash (Google) independently generated a primary diagnosis and supporting rationale from the same deidentified radiograph and clinical history[10, 11]. Outputs were generated once, fixed before case selection, and used unchanged throughout the study. To evaluate suggestion number rather than differences in overall model performance, 60 cases were purposively selected to balance disease composition and match overall model accuracy. The final set comprised 30 chest and 30 abdominal radiographs; each model correctly diagnosed 39 of 60 cases (65.0%), while Kimi-K2.6 and Gemini-3.6 Flash differed in case-level concordance with GPT-5.4. Before the reader study, five radiologists with 1–5 years of experience independently interpreted the 60 cases without AI assistance; mean accuracy was 54.3%. Additional details are provided in Supplementary Appendix 1. 2.3 Study Procedure Participants were stratified by specialty and randomly assigned in a 1:1:1 ratio to groups A, B, or C using a prespecified computer-assisted allocation procedure. The allocation list was fixed before assessment and concealed from enrollment personnel until assignment. Details of the allocation procedure are provided in the Supplementary Methods. Group A received GPT-5.4 alone; groups B and C received GPT-5.4 plus Kimi-K2.6 and GPT-5.4 plus Gemini-3.6 Flash, respectively. The two dual-suggestion conditions assessed consistency across model pairings. GPT-5.4 was the shared model; both AI suggestions were presented simultaneously and consisted of a primary diagnosis and supporting rationale. Model identities were not disclosed to participants. All groups interpreted the same 60 cases in independently randomized order. Testing occurred in supervised group sessions using a smartphone or tablet and was completed in one session. Participants completed the assessment independently using deidentified radiographs and clinical histories. For each case, participants recorded an unaided diagnosis and confidence on a 5-point scale; responses were then locked. The assigned AI suggestion or suggestions were displayed, after which participants recorded a final diagnosis and confidence. The platform recorded responses and interpretation times (Figure 2). Figure 2: Reader Workflow and AI Support Conditions. a, Initial interpretation without AI assistance, including diagnosis and confidence. b, Final interpretation after AI assistance with single-suggestion support (group A, GPT-5.4) or one of two parallel dual-suggestion conditions (group B, GPT-5.4 plus Kimi-K2.6; group C, GPT-5.4 plus Gemini-3.6 Flash). In both dual-suggestion conditions, the two AI suggestions were presented simultaneously. 2.4 Outcomes and Response Adjudication The primary outcome was change in diagnostic accuracy, defined as AI-assisted minus AI-unassisted accuracy. Accuracy was also evaluated according to whether the shared GPT-5.4 suggestion was correct (39 cases) or incorrect (21 cases). Secondary outcomes were interpretation time, defined as the total time across the unaided and AI-assisted interpretation stages, and confidence change, defined as final minus initial mean confidence. Diagnostic revisions were classified as beneficial (incorrect-to-correct) or harmful (correct-to-incorrect). Responses were scored using a prespecified case-specific diagnostic lexicon (Supplementary Appendix 2) established by two senior radiologists (>10 years’ experience). Free-text responses were classified by an automated matching pipeline and independently reviewed by both radiologists while masked to participant identity and support condition; disagreements were resolved by consensus. 2.5 Statistical Analysis Sample size was based on feasible recruitment; no formal a priori calculation was performed. Data are summarized as mean ± SD or counts and percentages. Paired t tests assessed changes in diagnostic accuracy before and after AI assistance. Between-specialty comparisons used Welch t tests. Prespecified primary comparisons were accuracy changes in groups B and C versus group A within each specialty. Support conditions were compared using Welch ANOVA, with B–A and C–A contrasts assessed using Welch t tests with Holm adjustment. Accuracy before and after AI assistance was also compared across support conditions for shared GPT-5.4-correct and -incorrect cases using the same approach. Linear models with heteroskedasticity-robust standard errors (HC3 estimator) assessed specialty interactions for accuracy change and beneficial and harmful revisions[23]. The dual-suggestion effect was defined as [(B+C)/2]−A[(B+C)/2]-A, and interactions were assessed using Wald tests. Effect estimates are reported with 95% CIs. No important changes to the study methods or prespecified outcomes were made after trial commencement. All tests were two-sided, with P < .05 indicating statistical significance. Analyses were performed using Python, version 3.12.14. 3 Results 3.1 Participant Flow and Characteristics Of 154 residents assessed for eligibility, 132 were allocated to the three support conditions; nine had incomplete assessments because of network failure (n = 7) or physical discomfort (n = 2), leaving 123 residents for analysis. The 123 residents had a mean age of 24.1 years ± 1.4; 65 were women, including 60 radiology and 63 non-radiology residents. Mean radiographic interpretation experience was 13.9 months among radiology residents and 14.7 months among non-radiology residents (mean difference, −0.8-0.8 months; 95% CI, −2.9-2.9 to 1.2; P=.43P=.43). Of the 123 residents, 87 (70.7%) were recruited from Center 1, 20 (16.3%) from Center 2, and 16 (13.0%) from Center 3 (Table 1). Participant characteristics by support condition are shown in Supplementary Table S1. Table 1: Participant Characteristics Overall and by Specialty Characteristic Total residents (n=123n=123) Radiology (n=60n=60) Non-radiology (n=63n=63) Age (y) 24.1±1.424.1± 1.4 24.1±1.524.1± 1.5 24.0±1.324.0± 1.3 Sex (male) 58 (47.2) 22 (36.7) 36 (57.1) Experience (m) 14.3±5.714.3± 5.7 13.9±5.913.9± 5.9 14.7±5.514.7± 5.5 Center Center 1 87 (70.7) 34 (56.7) 53 (84.1) Center 2 20 (16.3) 18 (30.0) 2 (3.2) Center 3 16 (13.0) 8 (13.3) 8 (12.7) Note.—Data are mean ± SD or n (%). Experience denotes months of clinical work involving radiographic interpretation. Center indicates the recruiting center. 3.2 Diagnostic Accuracy before and after AI Assistance The three AI models had identical overall diagnostic accuracy (65.0%), while Kimi-K2.6 and Gemini-3.6 Flash differed in case-level concordance with GPT-5.4 (Figure 3a–c). Across all 123 residents, mean diagnostic accuracy increased from 46.3% ± 11.3 to 61.5% ± 8.0 after AI assistance (mean change, 15.2 percentage points; 95% CI, 13.7–16.8; P<.001P<.001). Before AI assistance, radiology residents had higher accuracy than non-radiology residents (48.8% vs 43.9%; mean difference, 4.9 percentage points; 95% CI, 0.9–8.9; P=.017P=.017). After AI assistance, accuracy remained higher among radiology residents (63.2% vs 59.9%; mean difference, 3.3 percentage points; 95% CI, 0.4–6.1; P=.024P=.024), whereas accuracy change did not differ between specialties (14.4 vs 16.0 percentage points; mean difference, –1.6 percentage points; 95% CI, –4.8 to 1.6; P=.32P=.32) (Supplementary Table S2). Diagnostic accuracy increased after AI assistance in all three support conditions: by 13.9 percentage points in group A (95% CI, 11.3–16.5), 15.0 percentage points in group B (95% CI, 12.0–18.0), and 16.8 percentage points in group C (95% CI, 14.1–19.6) (all P<.001P<.001) (Figure 3d–f; Supplementary Table S3). Figure 3: AI Model Concordance and Diagnostic Accuracy With and Without AI Assistance. a–c, AI diagnostic performance and case-level concordance in groups A–C. In the dual-suggestion groups, concordant and discordant categories indicate whether the second model agreed with GPT-5.4 among GPT-5.4-correct and GPT-5.4-incorrect cases. d–f, Mean AI-unassisted and AI-assisted diagnostic accuracy among all residents, radiology residents, and non-radiology residents in groups A–C. Error bars represent 95% CIs. ***P<.001P<.001 for paired comparisons between AI-unassisted and AI-assisted accuracy. 3.3 Accuracy Change, Interpretation Time, and Confidence Change Among radiology residents, accuracy change differed across support conditions (group A, 9.6 percentage points ± 7.1; group B, 16.3 ± 10.6; group C, 17.5 ± 11.4; P=.016P=.016). Compared with group A, improvement was greater in group B by 6.69 percentage points (95% CI, 0.97–12.40; Holm-adjusted P=.030P=.030) and in group C by 7.87 percentage points (95% CI, 1.64–14.11; Holm-adjusted P=.030P=.030) (Figure 4a). Among non-radiology residents, accuracy change did not differ across support conditions (group A, 18.2 percentage points ± 6.9; group B, 13.7 ± 8.5; group C, 16.3 ± 5.6; P=.20P=.20) (Figure 4d). Interpretation time and confidence change did not differ across support conditions within either specialty stratum (all P>.05P>.05) (Figure 4; Supplementary Table S4). Figure 4: Diagnostic Outcomes by Support Condition. a–c, Accuracy change, interpretation time, and confidence change, respectively, among radiology residents. d–f, Corresponding outcomes among non-radiology residents. Bars represent mean values, and error bars represent 95% CIs. *Holm-adjusted P<.05P<.05 versus group A. ns indicates a nonsignificant overall comparison across the three support conditions. 3.4 Accuracy According to GPT-5.4 Suggestion Correctness Before AI assistance, accuracy did not differ across support conditions for GPT-5.4 correct or GPT-5.4 incorrect cases in either specialty (all P>.05P>.05). Among radiology residents, when GPT-5.4 was correct, AI-assisted accuracy did not differ across groups (78.5% ± 12.5, 80.1% ± 9.8, and 79.1% ± 8.6, respectively; P=.89P=.89) (Figure 5a). When GPT-5.4 was incorrect, AI-assisted accuracy was higher in groups B and C than in group A (20.0% ± 13.5, 40.1% ± 17.9, and 40.4% ± 9.4 for groups A, B, and C, respectively; both Holm-adjusted P<.001P<.001) (Figure 5b). Among non-radiology residents, when GPT-5.4 was correct, AI-assisted accuracy was lower in groups B and C than in group A (87.4% ± 7.2, 74.8% ± 10.0, and 74.5% ± 9.3, respectively; both Holm-adjusted P<.001P<.001) (Figure 5c). In contrast, when GPT-5.4 was incorrect, AI-assisted accuracy was higher in groups B and C than in group A (12.1% ± 12.9, 31.3% ± 11.7, and 31.0% ± 11.1, respectively; both Holm-adjusted P<.001P<.001) (Figure 5d) (Supplementary Tables S7–S9). Figure 5: Diagnostic Accuracy According to GPT-5.4 Suggestion Correctness. a–b, AI-unassisted and AI-assisted diagnostic accuracy among radiology residents in cases with correct (n=39n=39) and incorrect (n=21n=21) GPT-5.4 suggestions, respectively. c–d, Corresponding results among non-radiology residents. Lines represent groups A–C, and error bars represent 95% CIs. ***Holm-adjusted P<.001P<.001 versus group A. 3.5 Revision Components and Specialty Interaction Accuracy change was decomposed into beneficial revisions (incorrect-to-correct) and harmful revisions (correct-to-incorrect). Among radiology residents, beneficial revisions differed across support conditions (group A, 14.8% ± 7.8; group B, 19.9% ± 11.0; group C, 22.8% ± 10.5; P=.027P=.027), whereas harmful revisions did not (group A, 5.2% ± 3.9; group B, 3.7% ± 2.4; group C, 5.4% ± 3.3; P=.131P=.131) (Figure 6a; Supplementary Table S13). Among non-radiology residents, neither beneficial revisions (P=.204P=.204) nor harmful revisions (P=.796P=.796) differed across support conditions. The dual-suggestion effect on accuracy change differed by specialty (interaction difference, 10.44 percentage points; 95% CI, 4.36–16.52; P<.001P<.001), with effects of 7.28 percentage points (95% CI, 2.53–12.03; P=.003P=.003) in radiology residents and −3.16-3.16 percentage points (95% CI, −6.95-6.95 to 0.64; P=.103P=.103) in non-radiology residents. A specialty interaction was also observed for beneficial revisions (9.37 percentage points; 95% CI, 3.17–15.57; P=.003P=.003), but not for harmful revisions (−1.07-1.07 percentage points; 95% CI, −3.95-3.95 to 1.81; P=.47P=.47) (Figure 6b; Supplementary Table S16). For GPT-5.4 incorrect cases, groups B and C showed higher beneficial revision rates than group A in both specialties (both P<.001P<.001; Supplementary Table S19). Figure 6: Revision Components and Dual-Suggestion Effects by Specialty. a, Definition of revision components. Beneficial and harmful revisions represent incorrect-to-correct and correct-to-incorrect transitions, respectively; gray flows indicate no change. b, Revision components of accuracy change by support condition and specialty. Bars show means with 95% CIs; harmful revisions are plotted below zero for visualization. c, Dual-suggestion effects by specialty, defined as [(B+C)/2]–A. Points show mean differences with 95% CIs; P for interaction indicates differences between specialties. 4 Discussion In this prospective multicenter multireader study, we compared dual- with single-suggestion AI support for radiographic interpretation by radiology and non-radiology residents. When the shared GPT-5.4 suggestion was incorrect, AI-assisted accuracy was higher with dual-suggestion support in both groups. Radiology residents showed greater overall improvement in diagnostic accuracy with dual- than with single-suggestion support, whereas accuracy change did not differ across support conditions among non-radiology residents. Interpretation time did not differ across support conditions in either specialty. These findings suggest that dual-suggestion support may help limit the diagnostic influence of erroneous AI suggestions. Previous studies have shown that AI assistance can improve diagnostic performance, while erroneous AI suggestions can also adversely influence diagnostic decisions[24, 25], particularly among less experienced physicians[12, 13]. Existing human–AI interaction strategies have largely focused on optimizing the presentation of information from a single AI. Our study extends this work by evaluating dual-suggestion support, in which an additional independent AI opinion is presented alongside the shared AI suggestion. The main contribution was observed when the shared AI suggestion was incorrect: diagnostic accuracy was higher with dual- than single-suggestion support in residents, suggesting that an additional independent AI opinion may help limit the diagnostic influence of erroneous AI suggestions. Importantly, this improvement was not accompanied by a detectable increase in interpretation time. These findings extend human–AI interaction research beyond single-AI presentation and suggest that dual-suggestion support may help limit the influence of erroneous AI suggestions without increasing interpretation time. The higher diagnostic accuracy observed with dual-suggestion support when the shared AI suggestion was incorrect is consistent with evidence that access to an additional opinion can reduce inappropriate reliance on erroneous AI advice[26]. Outside medicine, actively seeking a second opinion has been shown to reduce overreliance on AI suggestions, including erroneous ones[27]. In medicine, structured information sharing among physicians has been associated with fewer diagnostic errors[20]. These findings provide a conceptual basis for presenting an additional independent AI opinion when evaluating an AI-supported diagnosis. Notably, in an exploratory analysis of cases in which both AI suggestions were incorrect, dual- and single-suggestion support did not differ significantly in AI-assisted accuracy or increased confidence in incorrect final diagnoses (Supplementary Table S20). Taken together, these findings support the potential value of dual-suggestion support when an AI suggestion may be erroneous. The effect of dual-suggestion support differed by specialty. Although both groups had fewer than 3 years of clinical experience, radiology and non-radiology residents differed in specialty background and baseline diagnostic performance. Greater overall accuracy improvement with dual-suggestion support was observed in radiology residents but not in non-radiology residents. Among non-radiology residents, however, the effect varied with shared-AI correctness: accuracy was higher with dual-suggestion support when the shared AI suggestion was incorrect but lower when it was correct, while overall beneficial and harmful revision rates did not differ across support conditions. Previous studies have shown that the effect of AI assistance varies with task-specific expertise[10, 28]. Differences in domain-specific knowledge may influence how clinicians evaluate and accept AI suggestions[2, 8, 29]. In our study, specialty interactions were observed for accuracy change and beneficial revisions. The incremental effect of dual-suggestion support may vary across specialties. Several limitations should be considered. First, process-level mechanisms were not directly assessed. Eye tracking can assess visual attention[30, 15], and suggestion-weighting measures can quantify the relative influence of dual AI suggestions on diagnostic decisions[31]. Future studies incorporating these measures may help clarify the mechanisms underlying the diagnostic value of dual-suggestion support. Second, the paired models differed in case-level error patterns, and dual-suggestion support therefore sometimes provided a correct additional suggestion when the shared GPT-5.4 suggestion was incorrect. The frequency and direction of such disagreement may vary across model pairings, and whether the effect of dual-suggestion support depends on these disagreement patterns warrants further investigation. Third, the 60-case set was purposively selected to balance disease composition and overall AI diagnostic accuracy and may not reflect the case mix of routine clinical practice. Further evaluation in unselected clinical populations is warranted. Nevertheless, the controlled multireader design enabled direct comparison of dual- and single-suggestion support under standardized conditions. In conclusion, dual-suggestion support may help limit the diagnostic influence of erroneous AI suggestions during radiographic interpretation. Compared with single-suggestion support, it was associated with greater overall improvement in diagnostic accuracy among radiology residents but not among non-radiology residents, without increased interpretation time. These findings highlight the potential of human–AI interaction design to reduce the diagnostic consequences of erroneous AI suggestions. Data Availability The deidentified 60-case radiographs, groud-truth diagnoses, and the diagnostic lexicon used for free-text scoring are available at GitHub. Code Availability The inference code is available at GitHub. Model outputs were generated through platform APIs: GPT-5.4 and Kimi-K2.6 via Microsoft Foundry (formerly Azure AI Foundry), and Gemini-3.6 Flash via Google Vertex AI. Funding This study was supported by the Jiangxi Province Natural Science Foundation (No. 20232BAB216099), the Jiangxi Clinical Research Center for Medical Imaging (No. 20223BCG74001), and the Jiangxi Province Key Laboratory for Precision Pathology and Intelligent Diagnosis (No. 2024SSY06281). References [1] Gommers J, Verboom SD, Duvivier KM, et al. Influence of AI decision support on radiologists’ performance and visual search in screening mammography. Radiology 2025;316(1):e243688. doi: 10.1148/radiol.243688. [2] Schramm S, Le Guellec B, Topka M, et al. Performing best when needed least: reader experience shapes accuracy gains in large language model–assisted brain MRI differential diagnosis. Radiology 2026;319(2):e253477. doi: 10.1148/radiol.253477. [3] Hanneman K, Patlas MN. From promise to practice: implementing artificial intelligence in radiology. Diagn Interv Imaging 2026;107(4):127–128. doi: 10.1016/j.diii.2026.01.010. [4] Duron L, Lecler A. Multimodal artificial intelligence in radiology: text-dominant reasoning limits image understanding. Diagn Interv Imaging 2025;106(10):333–334. doi: 10.1016/j.diii.2025.05.008. [5] Tanno R, Barrett DGT, Sellergren A, et al. Collaboration between clinicians and vision–language models in radiology report generation. Nat Med 2025;31(2):599–608. doi: 10.1038/s41591-024-03302-1. [6] Thrall JH. Challenges of implementing artificial intelligence–enabled programs in the clinical practice of radiology. Radiol Artif Intell 2024;6(5):e240411. doi: 10.1148/ryai.240411. [7] Angus DC, Khera R, Lieu T, et al. AI, health, and health care today and tomorrow: the JAMA summit report on artificial intelligence. JAMA 2025;334(18):1650–1664. doi: 10.1001/jama.2025.18490. [8] Prinster D, Mahmood A, Saria S, et al. Care to explain? AI explanation types differentially impact chest radiograph diagnostic performance and physician trust in AI. Radiology 2024;313(2):e233261. doi: 10.1148/radiol.233261. [9] Tang JSN, Lai JKC, Bui J, et al. Impact of different artificial intelligence user interfaces on lung nodule and mass detection on chest radiographs. Radiol Artif Intell 2023;5(3):e220079. doi: 10.1148/ryai.220079. [10] Xu X, Hu H, Zhang H, et al. Divergent impacts of explainable AI for dermatological diagnosis on clinicians versus lay people. Nat Med 2026;32(8):3000–3009. doi: 10.1038/s41591-026-04553-w. [11] Song J, Jeong WG, Han DH, et al. Determinants of reader-LLM interaction in thoracic radiology: impact of model confidence and reader expertise. Radiology 2026;320(2):e253397. doi: 10.1148/radiol.253397. [12] Dratsch T, Chen X, Rezazade Mehrizi M, et al. Automation bias in mammography: the impact of artificial intelligence BI-RADS suggestions on reader performance. Radiology 2023;307(4):e222176. doi: 10.1148/radiol.222176. [13] Teng D, Tan L, Cao Q, Xia Y, Zhang N, Li J, Zhao D. Impact of AI misinformation on diagnostic accuracy and confidence calibration in novice medical students. NPJ Digit Med 2026;9(1):356. doi: 10.1038/s41746-026-02547-z. [14] Park C, Lee YI, Seo J, et al. Beyond expertise: exploring behavioral personas in AI-assisted rare renal cancer diagnosis. NPJ Digit Med. Published online August 13, 2026. doi: 10.1038/s41746-026-03134-y. [15] Taib AG, Partridge GJW, Phillips P, et al. Automation bias in action: eye tracking of humans reading screening mammograms with and without AI prompts. Radiology 2026;320(1):e252590. doi: 10.1148/radiol.252590. [16] Vaccaro M, Almaatouq A, Malone T. When combinations of humans and AI are useful: a systematic review and meta-analysis. Nat Hum Behav 2024;8(12):2293–2303. doi: 10.1038/s41562-024-02024-1. [17] Riedl C, Kim YJ, Gupta P, Malone TW, Woolley AW. Quantifying collective intelligence in human groups. Proc Natl Acad Sci U S A 2021;118(21):e2005737118. doi: 10.1073/pnas.2005737118. [18] Woolley AW, Gupta P. Understanding collective intelligence: investigating the role of collective memory, attention, and reasoning processes. Perspect Psychol Sci 2024;19(2):344–354. doi: 10.1177/17456916231191534. [19] Barnett ML, Boddupalli D, Nundy S, Bates DW. Comparative accuracy of diagnosis by collective intelligence of multiple physicians vs individual physicians. JAMA Netw Open 2019;2(3):e190096. doi: 10.1001/jamanetworkopen.2019.0096. [20] Centola D, Becker J, Zhang J, Aysola J, Guilbeault D, Khoong E. Experimental evidence for structured information–sharing networks reducing medical errors. Proc Natl Acad Sci U S A 2023;120(31):e2108290120. doi: 10.1073/pnas.2108290120. [21] Zöller N, Berger J, Lin I, et al. Human–AI collectives most accurately diagnose clinical vignettes. Proc Natl Acad Sci U S A 2025;122(24):e2426153122. doi: 10.1073/pnas.2426153122. [22] Kameda T, Toyokawa W, Tindale RS. Information aggregation and collective intelligence beyond the wisdom of crowds. Nat Rev Psychol 2022;1(6):345–357. doi: 10.1038/s44159-022-00054-y. [23] Mansournia MA, Nazemipour M, Naimi AI, Collins GS, Campbell MJ. Reflection on modern methods: demystifying robust standard errors for epidemiologists. Int J Epidemiol 2021;50(1):346–351. doi: 10.1093/ije/dyaa260. [24] Jabbour S, Fouhey D, Shepard S, et al. Measuring the impact of AI in the diagnosis of hospitalized patients: a randomized clinical vignette survey study. JAMA 2023;330(23):2275–2284. doi: 10.1001/jama.2023.22295. [25] Chen M, Wang Y, Wang Q, et al. Impact of human and artificial intelligence collaboration on workload reduction in medical image interpretation. NPJ Digit Med 2024;7(1):349. doi: 10.1038/s41746-024-01328-w. [26] Sokol K, Fackler J, Vogt JE. Artificial intelligence should genuinely support clinical reasoning and decision making to bridge the translational gap. NPJ Digit Med 2025;8(1):345. doi: 10.1038/s41746-025-01725-9. [27] Lu Z, Wang D, Yin M. Does more advice help? The effects of second opinions in AI-assisted decision making. Proc ACM Hum Comput Interact 2024;8(CSCW1):1–31. doi: 10.1145/3653708. [28] Yu F, Moehring A, Banerjee O, Salz T, Agarwal N, Rajpurkar P. Heterogeneity and predictors of the effects of AI assistance on radiologists. Nat Med 2024;30(3):837–849. doi: 10.1038/s41591-024-02850-w. [29] Jackson NJ, Brown KE, Miller R, et al. Factors influencing the effectiveness of artificial intelligence-assisted decision-making in medicine: a scoping review. J Am Med Inform Assoc 2026;33(5):1054–1064. doi: 10.1093/jamia/ocag002. [30] Belde D, Kapaj A, Fabrikant SI, et al. The impact of AI on eye gaze patterns in chest X-ray interpretation: an eye tracking study of novice and expert radiologists. Invest Radiol 2026;61(11):790–796. doi: 10.1097/RLI.0000000000001289. [31] Glickman M, Sharot T. How human–AI feedback loops alter human perceptual, emotional and social judgements. Nat Hum Behav 2025;9(2):345–359. doi: 10.1038/s41562-024-02077-2. Supplementary Materials These tables provide participant-level descriptive results, prespecified support-condition comparisons, and additional analyses for the 123 participants included in the study. Group A: GPT; group B: GPT plus Kimi; group C: GPT plus Gemini. Radiology residents: n=60n=60 (A/B/C, 20/21/19); non-radiology residents: n=63n=63 (20/21/22). All groups evaluated the same 60 cases. Conditional analyses used the 39 cases in which the common GPT suggestion was correct and the 21 cases in which it was incorrect. This classification was identical across groups. Accuracy change is AI-assisted minus AI-unassisted accuracy. Beneficial and harmful revisions are incorrect-to-correct and correct-to-incorrect transitions, respectively. Transition percentages use all 60 cases as the denominator. Interpretation time sums both interpretation stages; confidence change is final minus initial mean confidence. Contents • Supplementary Statistical Methods • Supplementary Appendix 1. Case Composition, Model Concordance, and Representative Fixed AI Suggestions • Figure S1. Composition of the 60-case set • Figure S2. Correctness-status maps for the study conditions • Supplementary Appendix 2. Prespecified Diagnostic Lexicon for Free-Text Response Scoring • Appendix Table A1. Accepted Diagnostic Expressions and Examples Not Accepted as Standalone Answers • Supplementary Table S1. Participant Characteristics by Specialty and Support Condition • Supplementary Table S2. Participant-Level Outcomes by Specialty • Supplementary Table S3. Diagnostic Accuracy Before and After AI Assistance by Specialty and Support Condition • Supplementary Table S4. Accuracy Change, Interpretation Time, and Confidence Change by Specialty and Support Condition • Supplementary Table S5. Diagnostic Accuracy in Cases With a Correct Common GPT Suggestion • Supplementary Table S6. Diagnostic Accuracy in Cases With an Incorrect Common GPT Suggestion • Supplementary Table S7. Pairwise Accuracy Contrasts by Specialty and GPT Suggestion Correctness • Supplementary Table S8. Diagnostic Revision Patterns by Specialty and Support Condition • Supplementary Table S9. Dual-Suggestion Effects and Specialty Interactions • Supplementary Table S10. Revision Components by Shared GPT Suggestion Correctness, Specialty, and Support Condition • Supplementary Table S11. Outcomes in Cases for Which Both AI Models Were Incorrect Supplementary Methods: Allocation Procedure Participant allocation was performed within specialty strata by an investigator not involved in participant assessment. Deidentified participant information, including specialty classification, was submitted to DeepSeek using a prespecified prompt instructing random and balanced allocation in a 1:1:1 ratio to groups A, B, and C. The resulting allocation list was accepted without manual modification, fixed before participant assessment, and concealed from enrollment personnel until assignment. DeepSeek was used solely for participant allocation and was not one of the diagnostic AI models evaluated in the study. Supplementary Statistical Methods All analyses used participant-level outcomes. Paired t tests assessed AI-unassisted versus AI-assisted accuracy within each group. Welch t tests compared specialties, and Welch ANOVA assessed differences across support conditions within each specialty. Prespecified B–A and C–A comparisons used Welch t tests with Holm adjustment within each outcome and specialty stratum; for conditional analyses, adjustment was additionally performed within each GPT-correctness subset. B–C comparisons were exploratory and unadjusted. Additional analyses compared AI-unassisted accuracy, AI-assisted accuracy, and accuracy change across support conditions separately within the shared GPT-correct and GPT-incorrect subsets (Supplementary Tables S7–S9). Separate ordinary least-squares models with heteroskedasticity-robust standard errors using the HC3 estimator evaluated accuracy change, beneficial revisions, and harmful revisions. Dual-suggestion effect was the equally weighted contrast [(B+C)/2]−A[(B+C)/2]-A. B–A and C–A effects were also estimated separately within each specialty. Specialty interaction contrasts were the radiology effect minus the corresponding non-radiology effect. Individual contrasts used two-sided Wald z tests. The overall three-group-by-specialty interaction used a two-degree-of-freedom Wald chi-square test. Model-specific B–A and C–A contrasts were Holm-adjusted in pairs within each outcome and analysis block. An additional exploratory analysis evaluated cases in which both AI suggestions were incorrect (Supplementary Table S20). For the GPT plus Kimi condition, group B was compared with group A using the 11 cases in which both GPT-5.4 and Kimi-K2.6 were incorrect; for the GPT plus Gemini condition, group C was compared with group A using the 10 cases in which both GPT-5.4 and Gemini-3.6 Flash were incorrect. Participant-level outcomes included AI-assisted accuracy, accuracy change, confidence change, and the proportion of incorrect final diagnoses with increased confidence. Comparisons used two-sided Welch t tests, with Holm adjustment applied to the two dual-versus-single comparisons within each outcome. All tests were two-sided, with P<.05P<.05 indicating statistical significance. Analyses were performed using Python, version 3.12.14. Supplementary Appendix 1. Case Composition, Model Concordance, and Representative Fixed AI Suggestions This appendix describes the 60-case set, the correctness patterns of the AI suggestions, and four representative fixed outputs. Correctness refers to the primary diagnosis under the diagnostic lexicon. Disease composition. Thirty examinations were chest radiographs and 30 were abdominal radiographs. Urinary tract calculi include renal and ureteral calculi. Normal abdominal examinations are shown separately from disease categories. Figure S1: Composition of the 60-case set. Thirty examinations were chest radiographs and 30 were abdominal radiographs. Urinary tract calculi include renal and ureteral calculi. Normal abdominal examinations are shown separately from disease categories. Case selection and control of model performance. Model outputs for all 218 candidate examinations were generated and fixed before selection of the final study cases. The final 60-case set was selected to balance disease composition and to match overall diagnostic accuracy across GPT-5.4, Kimi-K2.6, and Gemini-3.6 Flash. Thirty chest and 30 abdominal examinations were included, and each model correctly diagnosed 39 of 60 cases (65.0%). Matching overall accuracy did not require the models to be correct on the same cases, thereby preserving case-level differences in model correctness and concordance. Case-level correctness concordance. Each square represents one case, numbered 1–60 in the same order in both panels. Example case numbers correspond to these square labels. Concordance means agreement in correctness status, not agreement in diagnostic wording. Group A (GPT). GPT correct: 39; GPT incorrect: 21. Group A has no second-model status. Group B (GPT plus Kimi). Both correct: 29; GPT correct / second model incorrect: 10; GPT incorrect / second model correct: 10; both incorrect: 11. Group C (GPT plus Gemini). Both correct: 28; GPT correct / second model incorrect: 11; GPT incorrect / second model correct: 11; both incorrect: 10. Group A (GPT) GPT correct (n=39n=39) GPT incorrect (n=21n=21) Group B (GPT plus Kimi) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Group C (GPT plus Gemini) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Both correct GPT correct / second model incorrect GPT incorrect / second model correct Both incorrect Figure S2: Correctness-status maps for the single- and dual-suggestion conditions. Each square represents one case, numbered 1–60 in the same order in both dual-suggestion panels. Group A has no second-model status and uses the same two-color key as the examples below (teal, correct; rust, incorrect). The four colors below apply only to groups B and C. Both-incorrect cases count as correctness-concordant even when the diagnostic labels differ. The four examples below use the same case-level correctness categories as Figure S2. Teal panels mark a correct primary diagnosis; rust panels mark an incorrect primary diagnosis. Example 1. Case 8 — GPT correct / Kimi correct Case ID: G0385195 Reference diagnosis: Pneumonia Clinical history: Cough and chest tightness. Concordance: both models correct GPT suggestion (correct) Primary diagnosis: Bronchopneumonia The chest radiograph shows patchy and reticular ill-defined opacities around both hila and in both lower lungs, with increased lung markings, more pronounced in the left middle and lower lung. The heart is not enlarged, and there is no pleural effusion. With cough and chest tightness, bronchopneumonia is the first consideration, followed by acute bronchitis. Kimi suggestion (correct) Primary diagnosis: Bronchopneumonia The chest radiograph shows increased and ill-defined lung markings bilaterally, scattered patchy ill-defined opacities, and increased hilar opacity. With cough and chest tightness, the findings are consistent with bronchopneumonia and airway inflammatory changes. Example 2. Case 13 — GPT correct / Kimi incorrect Case ID: G0404250 Reference diagnosis: Pneumonia Clinical history: Cough and chest tightness. Concordance: GPT correct / second model incorrect GPT suggestion (correct) Primary diagnosis: Left lower-lobe pneumonia The chest radiograph shows patchy consolidation in the left lower lung and retrocardiac region, with mildly increased lung markings bilaterally. The heart is not enlarged, and there is no pleural effusion. With cough and chest tightness, left lower-lung infection with associated bronchitic changes is the first consideration. Kimi suggestion (incorrect) Primary diagnosis: Acute bronchitis The chest radiograph shows increased and thickened lung markings with ill-defined margins, without evident consolidation or pleural effusion. With cough and chest tightness, the imaging appearance is consistent with bronchitis. The cardiac silhouette and diaphragm are unremarkable. Example 3. Case 7 — GPT incorrect / Kimi correct Case ID: P4216711 Reference diagnosis: Pneumothorax Clinical history: Cough and chest tightness. Concordance: GPT incorrect / second model correct GPT suggestion (incorrect) Primary diagnosis: Bronchitis / small-airway inflammation The chest radiograph shows mildly increased bilateral lung lucency and mildly increased and thickened markings in the hilar and lower-lung regions. There is no focal consolidation, pleural effusion, or pneumothorax. The heart is not enlarged. With cough and chest tightness, bronchitis or small-airway inflammation is favored. Kimi suggestion (correct) Primary diagnosis: Left pneumothorax The chest radiograph shows abnormal lucency without lung markings in the peripheral left upper lung, with a visible visceral pleural line and the margin of the compressed lung. With cough and chest tightness, the findings are consistent with left pneumothorax. Example 4. Case 1 — GPT incorrect / Kimi incorrect Case ID: P537136 Reference diagnosis: Pneumothorax Clinical history: Cough and chest tightness. Concordance: both models incorrect GPT suggestion (incorrect) Primary diagnosis: Bronchitis The chest radiograph shows relatively preserved bilateral lung lucency and no cardiac enlargement. No focal consolidation, pleural effusion, or pneumothorax is seen. Markings in both lower lungs and around the hila are mildly thickened. With cough and chest tightness, bronchitis or airway inflammation is favored. Kimi suggestion (incorrect) Primary diagnosis: Bronchitis The chest radiograph shows thickened and disorganized lung markings bilaterally and a small amount of patchy ill-defined opacity in the right lower lung. The cardiac and diaphragmatic contours are unremarkable. With cough and chest tightness, the findings are consistent with bronchitis, possibly accompanied by mild right lower-lung infection. Supplementary Appendix 2. Prespecified Diagnostic Lexicon for Free-Text Response Scoring The purpose of this appendix defines the diagnostic categories, acceptable diagnostic expressions, and exclusions used to score participants’ free-text primary diagnoses before and after AI presentation. Each case was linked to its expert-established reference diagnosis before automated scoring. Scoring Rules. 1. Responses were considered correct when the primary diagnosis matched the reference diagnosis or an accepted diagnostic expression. Qualifiers such as “likely” or “suggestive of” did not invalidate an otherwise correct primary diagnosis. 2. Negated diagnoses were not accepted. Ambiguous statements, contradictory diagnoses, and unclear abbreviations required expert adjudication. 3. Scoring focused on the reference diagnostic category. Omitted laterality, site, or severity did not invalidate a correct category-level diagnosis. Explicitly incorrect modifiers required expert adjudication. 4. Automated matching used the locked lexicon. Two senior radiologists independently reviewed all classifications while masked to participant identity and study group. Disagreements were resolved by consensus. Diagnostic Category Lexicon. Abbreviations: AI = artificial intelligence; GPT = generative pretrained transformer. Appendix Table A1. Accepted Diagnostic Expressions and Examples Not Accepted as Standalone Answers Reference diagnostic category Accepted diagnostic expressions in Chinese English equivalents and diagnostic scope Examples of descriptive findings or alternative diagnoses not accepted alone Pneumothorax / pleural air 气胸/胸腔积气气胸；左侧/右侧/双侧气胸；胸腔积气；张力性气胸（存在气胸时） Pneumothorax; unilateral or bilateral pneumothorax; pleural air/gas; tension pneumothorax when pneumothorax is present 肺大疱；胸膜增厚；胸腔积液；肺气肿 Pneumonia 肺炎肺炎；支气管肺炎；大叶性肺炎；肺部感染；感染性肺实变；明确诊断为炎性浸润/炎症 Pneumonia; bronchopneumonia; lobar pneumonia; pulmonary infection; infectious consolidation; inflammatory infiltrate stated as a diagnosis 肺纹理增多；斑片影；肺实变或浸润影但未提示感染；支气管炎 Lung cancer 肺癌肺癌；原发性肺癌；支气管肺癌；中央型/周围型肺癌；肺恶性肿瘤；肺恶性占位；明确提示恶性的肺部肿块；肺癌病理亚型 Primary lung cancer; bronchogenic carcinoma; central/peripheral lung cancer; malignant pulmonary neoplasm or mass; a specified lung-cancer histologic subtype 肺结节；肺肿块；肺占位（未提示恶性）；肺转移瘤；肺炎 Bronchitis 支气管炎支气管炎；急性/慢性支气管炎；气管支气管炎；支气管感染或炎症 Bronchitis; acute or chronic bronchitis; tracheobronchitis; bronchial infection/inflammation 肺纹理增粗或增多；肺门影增浓；肺炎；慢性阻塞性肺疾病 Urinary tract calculi 泌尿系结石泌尿系结石；尿路结石；肾结石；输尿管结石；肾输尿管结石；肾区/输尿管区结石影（按病例参考部位复核） Urolithiasis/urinary tract stone; nephrolithiasis/renal calculus; ureterolithiasis/ureteral calculus; renal and ureteral calculi 盆腔钙化；静脉石；肾钙化；腹部钙化影（未明确泌尿系来源） Bowel obstruction 肠梗阻肠梗阻；机械性肠梗阻；小肠/结肠梗阻；高位/低位肠梗阻；完全性/不完全性肠梗阻；绞窄性肠梗阻 Bowel/intestinal obstruction; mechanical obstruction; small- or large-bowel obstruction; complete, partial, high, low, or strangulated obstruction 肠胀气；肠管扩张；气液平面；肠麻痹/麻痹性肠梗阻（除非参考诊断为该类别） Hollow-viscus perforation / pneumoperitoneum 空腔脏器穿孔/气腹空腔脏器穿孔；消化道/胃肠道穿孔；胃/十二指肠/肠穿孔；气腹；腹腔游离气体；膈下游离气体；腹腔积气 Hollow-viscus or gastrointestinal perforation; gastric, duodenal, or intestinal perforation; pneumoperitoneum; free intraperitoneal or subdiaphragmatic air 肠梗阻；腹胀；腹膜炎；腹腔积液；皮下气肿（未提示腹腔游离气体） Supplementary Tables Table S1: Participant Characteristics by Specialty and Support Condition Characteristic Group A Group B Group C Statistic P value Radiology residents Number 20 21 19 — — Age (y) 24.0±1.824.0± 1.8 24.0±1.624.0± 1.6 24.2±1.224.2± 1.2 F=0.053F=0.053 .949 Sex, male 5 (25.0%) 9 (42.9%) 8 (42.1%) χ2=1.761χ^2=1.761 .415 Experience (mo) 9 (9–20) 9 (9–20) 20 (9–21) H=0.919H=0.919 .632 Non-radiology residents Number 20 21 22 — — Age (y) 24.4±1.524.4± 1.5 24.0±1.224.0± 1.2 23.8±1.323.8± 1.3 F=1.095F=1.095 .341 Sex, male 11 (55.0%) 13 (61.9%) 12 (54.5%) χ2=0.293χ^2=0.293 .864 Experience (mo) 9 (9–19) 19 (9–20) 19 (9–20) H=2.064H=2.064 .356 Note.—Data are mean ± SD, median (IQR), or n (%). Age was compared using one-way analysis of variance, experience using the Kruskal–Wallis test, and sex using the Pearson χ2χ^2 test. F = analysis-of-variance statistic; H = Kruskal–Wallis statistic; SD = standard deviation. Table S2: Participant-Level Outcomes by Specialty Outcome Radiology (n=60n=60) Non-radiology (n=63n=63) Mean difference (95% CI) Welch t (df) P value AI-unassisted accuracy (%) 48.8±13.048.8± 13.0 43.9±9.043.9± 9.0 4.89 (0.88 to 8.90) 2.42 (104.7) .017 AI-assisted accuracy (%) 63.2±9.363.2± 9.3 59.9±6.259.9± 6.2 3.28 (0.44 to 6.11) 2.29 (102.2) .024 Accuracy change (p) 14.4±10.314.4± 10.3 16.0±7.216.0± 7.2 −1.62-1.62 (−4.81-4.81 to 1.58) −1.00-1.00 (104.9) .318 Total interpretation time (min) 52.0±23.752.0± 23.7 55.1±27.355.1± 27.3 −3.07-3.07 (−12.18-12.18 to 6.04) −0.67-0.67 (120.0) .505 AI-assisted interpretation time (min) 21.2±8.321.2± 8.3 21.5±8.921.5± 8.9 −0.25-0.25 (−3.31-3.31 to 2.81) −0.16-0.16 (120.9) .871 Confidence change (points) 0.13±0.190.13± 0.19 0.24±0.270.24± 0.27 −0.12-0.12 (−0.20-0.20 to −0.04-0.04) −2.85-2.85 (113.3) .005 Note.—Group A: GPT; group B: GPT plus Kimi; group C: GPT plus Gemini. Radiology residents: n=60n=60 (A/B/C, 20/21/19); non-radiology residents: n=63n=63 (20/21/22). Values are mean ± SD. Differences are radiology minus non-radiology. Between-specialty P values are unadjusted. CI = confidence interval; SD = standard deviation; df = degrees of freedom; p = percentage points. Table S3: Diagnostic Accuracy Before and After AI Assistance by Specialty and Support Condition Population Group n AI-unassisted (%) AI-assisted (%) Change, p (95% CI) Paired t (df) P value All participants A 40 45.7±10.145.7± 10.1 59.5±7.459.5± 7.4 13.87 (11.27 to 16.48) 10.77 (39) <.001<.001 All participants B 42 47.9±12.847.9± 12.8 62.9±9.262.9± 9.2 15.00 (12.01 to 17.99) 10.13 (41) <.001<.001 All participants C 41 45.3±11.045.3± 11.0 62.2±6.962.2± 6.9 16.83 (14.10 to 19.56) 12.45 (40) <.001<.001 Radiology residents A 20 48.4±9.348.4± 9.3 58.0±8.458.0± 8.4 9.58 (6.26 to 12.91) 6.03 (19) <.001<.001 Radiology residents B 21 49.8±15.649.8± 15.6 66.1±10.866.1± 10.8 16.27 (11.42 to 21.12) 7.00 (20) <.001<.001 Radiology residents C 19 48.1±13.748.1± 13.7 65.5±5.565.5± 5.5 17.45 (11.98 to 22.93) 6.70 (18) <.001<.001 Non-radiology residents A 20 42.9±10.342.9± 10.3 61.1±6.061.1± 6.0 18.16 (14.95 to 21.37) 11.84 (19) <.001<.001 Non-radiology residents B 21 45.9±9.245.9± 9.2 59.6±6.059.6± 6.0 13.73 (9.86 to 17.60) 7.41 (20) <.001<.001 Non-radiology residents C 22 43.0±7.543.0± 7.5 59.2±6.759.2± 6.7 16.29 (13.80 to 18.77) 13.62 (21) <.001<.001 Note.—Values are mean ± SD. Change is final minus initial accuracy. Paired tests are two-sided and unadjusted. CI = confidence interval; SD = standard deviation; df = degrees of freedom; p = percentage points. Table S4: Accuracy Change, Interpretation Time, and Confidence Change by Specialty and Support Condition Population Outcome Group A Group B Group C Welch F (df1, df2) P value Radiology residents Accuracy change (p) 9.6±7.19.6± 7.1 16.3±10.616.3± 10.6 17.5±11.417.5± 11.4 4.67 (2, 36.0) .016 Radiology residents Interpretation time (min) 59.0±30.159.0± 30.1 52.6±19.352.6± 19.3 44.0±18.744.0± 18.7 2.01 (2, 36.8) .148 Radiology residents Confidence change (points) 0.16±0.290.16± 0.29 0.09±0.100.09± 0.10 0.13±0.140.13± 0.14 1.00 (2, 33.2) .380 Non-radiology residents Accuracy change (p) 18.2±6.918.2± 6.9 13.7±8.513.7± 8.5 16.3±5.616.3± 5.6 1.67 (2, 38.4) .201 Non-radiology residents Interpretation time (min) 57.6±27.457.6± 27.4 62.7±35.062.7± 35.0 45.5±13.845.5± 13.8 3.21 (2, 33.3) .053 Non-radiology residents Confidence change (points) 0.28±0.190.28± 0.19 0.22±0.200.22± 0.20 0.24±0.370.24± 0.37 0.41 (2, 38.8) .666 Note.—Values are mean ± SD. Omnibus P values are unadjusted. CI = confidence interval; SD = standard deviation; df = degrees of freedom; p = percentage points. Table S5: Accuracy Change, Interpretation Time, and Confidence Change by Specialty and Support Condition (continued). Radiology residents; pairwise contrasts Outcome Contrast Mean difference (95% CI) Welch t (df) P value Holm P Accuracy change (p) B–A 6.69 (0.97 to 12.40) 2.38 (35.0) .023 .030 Accuracy change (p) C–A 7.87 (1.64 to 14.11) 2.58 (30.0) .015 .030 Accuracy change (p) B–C −1.19-1.19 (−8.26-8.26 to 5.89) −0.34-0.34 (37.0) .736 NA Interpretation time (min) B–A −6.38-6.38 (−22.57-22.57 to 9.80) −0.80-0.80 (32.1) .427 .427 Interpretation time (min) C–A −14.96-14.96 (−31.23-31.23 to 1.31) −1.87-1.87 (32.0) .070 .140 Interpretation time (min) B–C 8.58 (−3.58-3.58 to 20.73) 1.43 (37.8) .161 NA Confidence change (points) B–A −0.08-0.08 (−0.22-0.22 to 0.07) −1.10-1.10 (23.2) .282 .565 Confidence change (points) C–A −0.03-0.03 (−0.18-0.18 to 0.12) −0.45-0.45 (28.0) .656 .656 Confidence change (points) B–C −0.04-0.04 (−0.12-0.12 to 0.04) −1.08-1.08 (31.7) .287 NA Note.—Two-sided Welch t tests. Differences are first-named minus second-named group. Holm adjustment was applied to the B–A and C–A comparisons within each outcome and specialty stratum. B–C is exploratory; its adjusted P value is not applicable. CIs are pointwise. CI = confidence interval; df = degrees of freedom; p = percentage points; NA = not applicable. Table S6: Accuracy Change, Interpretation Time, and Confidence Change by Specialty and Support Condition (continued). Non-radiology residents; pairwise contrasts Outcome Contrast Mean difference (95% CI) Welch t (df) P value Holm P Accuracy change (p) B–A −4.44-4.44 (−9.31-9.31 to 0.44) −1.84-1.84 (38.0) .073 .146 Accuracy change (p) C–A −1.88-1.88 (−5.82-5.82 to 2.06) −0.97-0.97 (36.8) .340 .340 Accuracy change (p) B–C −2.56-2.56 (−7.04-7.04 to 1.92) −1.16-1.16 (34.4) .254 NA Interpretation time (min) B–A 5.17 (−14.66-14.66 to 25.00) 0.53 (37.6) .601 .601 Interpretation time (min) C–A −12.02-12.02 (−25.96-25.96 to 1.92) −1.77-1.77 (27.4) .088 .176 Interpretation time (min) B–C 17.19 (0.37 to 34.01) 2.10 (25.8) .046 NA Confidence change (points) B–A −0.06-0.06 (−0.18-0.18 to 0.07) −0.90-0.90 (39.0) .373 .747 Confidence change (points) C–A −0.04-0.04 (−0.22-0.22 to 0.14) −0.45-0.45 (32.2) .656 .747 Confidence change (points) B–C −0.01-0.01 (−0.20-0.20 to 0.17) −0.17-0.17 (32.8) .869 NA Note.—Two-sided Welch t tests. Differences are first-named minus second-named group. Holm adjustment was applied to the B–A and C–A comparisons within each outcome and specialty stratum. B–C is exploratory; its adjusted P value is not applicable. CIs are pointwise. CI = confidence interval; df = degrees of freedom; p = percentage points; NA = not applicable. Table S7: Diagnostic Accuracy in Cases With a Correct Common GPT Suggestion Population Outcome Group A Group B Group C Welch F (df1, df2) P Radiology residents AI-unassisted accuracy (%) 57.6±9.157.6± 9.1 57.6±16.557.6± 16.5 53.3±15.853.3± 15.8 0.55 (2, 34.8) .580 Radiology residents AI-assisted accuracy (%) 78.5±12.578.5± 12.5 80.1±9.880.1± 9.8 79.1±8.679.1± 8.6 0.12 (2, 37.4) .889 Radiology residents Accuracy change (p) 20.9±11.720.9± 11.7 22.5±15.122.5± 15.1 25.8±14.325.8± 14.3 0.67 (2, 37.4) .517 Non-radiology residents AI-unassisted accuracy (%) 52.3±10.752.3± 10.7 54.1±11.454.1± 11.4 48.5±8.848.5± 8.8 1.79 (2, 38.9) .180 Non-radiology residents AI-assisted accuracy (%) 87.4±7.287.4± 7.2 74.8±10.074.8± 10.0 74.5±9.374.5± 9.3 16.92 (2, 39.5) <.001<.001 Non-radiology residents Accuracy change (p) 35.1±10.235.1± 10.2 20.8±10.820.8± 10.8 26.0±8.826.0± 8.8 9.88 (2, 39.2) <.001<.001 Note.—Values are mean ± SD. Omnibus P values are unadjusted. Conditional accuracy uses the common 39 GPT-correct cases as the denominator. CI = confidence interval; SD = standard deviation; df = degrees of freedom; p = percentage points. Table S8: Diagnostic Accuracy in Cases With an Incorrect Common GPT Suggestion Population Outcome Group A Group B Group C Welch F (df1, df2) P Radiology residents AI-unassisted accuracy (%) 31.4±14.931.4± 14.9 35.4±17.635.4± 17.6 38.3±14.338.3± 14.3 1.08 (2, 38.0) .349 Radiology residents AI-assisted accuracy (%) 20.0±13.520.0± 13.5 40.1±17.940.1± 17.9 40.4±9.440.4± 9.4 15.90 (2, 36.6) <.001<.001 Radiology residents Accuracy change (p) −11.4±10.6-11.4± 10.6 4.8±10.14.8± 10.1 2.0±9.72.0± 9.7 13.70 (2, 37.9) <.001<.001 Non-radiology residents AI-unassisted accuracy (%) 25.5±13.525.5± 13.5 30.6±12.630.6± 12.6 32.7±9.832.7± 9.8 1.90 (2, 38.4) .163 Non-radiology residents AI-assisted accuracy (%) 12.1±12.912.1± 12.9 31.3±11.731.3± 11.7 31.0±11.131.0± 11.1 15.66 (2, 39.4) <.001<.001 Non-radiology residents Accuracy change (p) −13.3±9.1-13.3± 9.1 0.7±9.30.7± 9.3 −1.7±9.6-1.7± 9.6 13.35 (2, 39.9) <.001<.001 Note.—Values are mean ± SD. Omnibus P values are unadjusted. Conditional accuracy uses the common 21 GPT-incorrect cases as the denominator. CI = confidence interval; SD = standard deviation; df = degrees of freedom; p = percentage points. Table S9: Pairwise Accuracy Contrasts by Specialty and GPT Suggestion Correctness. Radiology residents; GPT correct (39 cases); pairwise contrasts Outcome Contrast Mean difference (95% CI) Welch t (df) P value Holm P AI-unassisted accuracy (p) B–A 0.07 (−8.33-8.33 to 8.47) 0.02 (31.4) .987 .987 AI-unassisted accuracy (p) C–A −4.26-4.26 (−12.75-12.75 to 4.23) −1.03-1.03 (28.4) .313 .626 AI-unassisted accuracy (p) B–C 4.32 (−6.00-6.00 to 14.64) 0.85 (37.9) .401 NA AI-assisted accuracy (p) B–A 1.64 (−5.50-5.50 to 8.78) 0.46 (36.1) .645 1.000 AI-assisted accuracy (p) C–A 0.62 (−6.33-6.33 to 7.57) 0.18 (33.8) .857 1.000 AI-assisted accuracy (p) B–C 1.02 (−4.88-4.88 to 6.92) 0.35 (38.0) .729 NA Accuracy change (p) B–A 1.57 (−6.95-6.95 to 10.09) 0.37 (37.4) .711 .711 Accuracy change (p) C–A 4.88 (−3.62-3.62 to 13.37) 1.17 (34.8) .251 .503 Accuracy change (p) B–C −3.31-3.31 (−12.72-12.72 to 6.10) −0.71-0.71 (37.9) .481 NA Note.—Two-sided Welch t tests. Differences are first-named minus second-named group. Holm adjustment was applied to the B–A and C–A comparisons within each outcome, specialty stratum, and GPT-correctness subset. B–C is exploratory; its adjusted P value is not applicable. CIs are pointwise. CI = confidence interval; df = degrees of freedom; p = percentage points; NA = not applicable. Table S10: Pairwise Accuracy Contrasts by Specialty and GPT Suggestion Correctness (continued). Non-radiology residents; GPT correct (39 cases); pairwise contrasts Outcome Contrast Mean difference (95% CI) Welch t (df) P value Holm P AI-unassisted accuracy (p) B–A 1.78 (−5.21-5.21 to 8.78) 0.52 (39.0) .609 .609 AI-unassisted accuracy (p) C–A −3.82-3.82 (−9.99-9.99 to 2.34) −1.26-1.26 (36.8) .217 .434 AI-unassisted accuracy (p) B–C 5.61 (−0.70-0.70 to 11.91) 1.80 (37.6) .080 NA AI-assisted accuracy (p) B–A −12.59-12.59 (−18.10-18.10 to −7.07-7.07) −4.63-4.63 (36.3) <.001<.001 <.001<.001 AI-assisted accuracy (p) C–A −12.96-12.96 (−18.13-18.13 to −7.79-7.79) −5.07-5.07 (39.0) <.001<.001 <.001<.001 AI-assisted accuracy (p) B–C 0.37 (−5.61-5.61 to 6.35) 0.13 (40.4) .901 NA Accuracy change (p) B–A −14.37-14.37 (−20.99-20.99 to −7.76-7.76) −4.39-4.39 (39.0) <.001<.001 <.001<.001 Accuracy change (p) C–A −9.14-9.14 (−15.09-15.09 to −3.19-3.19) −3.11-3.11 (37.7) .004 .004 Accuracy change (p) B–C −5.23-5.23 (−11.31-11.31 to 0.84) −1.74-1.74 (38.5) .089 NA Note.—Two-sided Welch t tests. Differences are first-named minus second-named group. Holm adjustment was applied to the B–A and C–A comparisons within each outcome, specialty stratum, and GPT-correctness subset. B–C is exploratory; its adjusted P value is not applicable. CIs are pointwise. CI = confidence interval; df = degrees of freedom; p = percentage points; NA = not applicable. Table S11: Pairwise Accuracy Contrasts by Specialty and GPT Suggestion Correctness (continued). Radiology residents; GPT incorrect (21 cases); pairwise contrasts Outcome Contrast Mean difference (95% CI) Welch t (df) P value Holm P AI-unassisted accuracy (p) B–A 3.95 (−6.33-6.33 to 14.22) 0.78 (38.5) .442 .442 AI-unassisted accuracy (p) C–A 6.92 (−2.55-2.55 to 16.38) 1.48 (37.0) .147 .294 AI-unassisted accuracy (p) B–C −2.97-2.97 (−13.21-13.21 to 7.26) −0.59-0.59 (37.6) .560 NA AI-assisted accuracy (p) B–A 20.14 (10.15 to 30.13) 4.08 (37.2) <.001<.001 <.001<.001 AI-assisted accuracy (p) C–A 20.35 (12.79 to 27.91) 5.47 (34.0) <.001<.001 <.001<.001 AI-assisted accuracy (p) B–C −0.21-0.21 (−9.31-9.31 to 8.88) −0.05-0.05 (31.0) .962 NA Accuracy change (p) B–A 16.19 (9.63 to 22.75) 4.99 (38.6) <.001<.001 <.001<.001 Accuracy change (p) C–A 13.43 (6.83 to 20.03) 4.13 (36.9) <.001<.001 <.001<.001 Accuracy change (p) B–C 2.76 (−3.58-3.58 to 9.10) 0.88 (37.9) .384 NA Note.—Two-sided Welch t tests. Differences are first-named minus second-named group. Holm adjustment was applied to the B–A and C–A comparisons within each outcome, specialty stratum, and GPT-correctness subset. B–C is exploratory; its adjusted P value is not applicable. CIs are pointwise. CI = confidence interval; df = degrees of freedom; p = percentage points; NA = not applicable. Table S12: Pairwise Accuracy Contrasts by Specialty and GPT Suggestion Correctness (continued). Non-radiology residents; GPT incorrect (21 cases); pairwise contrasts Outcome Contrast Mean difference (95% CI) Welch t (df) P value Holm P AI-unassisted accuracy (p) B–A 5.14 (−3.11-3.11 to 13.38) 1.26 (38.4) .215 .215 AI-unassisted accuracy (p) C–A 7.21 (−0.24-0.24 to 14.66) 1.97 (34.4) .058 .115 AI-unassisted accuracy (p) B–C −2.07-2.07 (−9.04-9.04 to 4.90) −0.60-0.60 (37.8) .551 NA AI-assisted accuracy (p) B–A 19.15 (11.35 to 26.94) 4.97 (38.2) <.001<.001 <.001<.001 AI-assisted accuracy (p) C–A 18.81 (11.27 to 26.35) 5.05 (37.7) <.001<.001 <.001<.001 AI-assisted accuracy (p) B–C 0.34 (−6.69-6.69 to 7.37) 0.10 (40.6) .923 NA Accuracy change (p) B–A 14.01 (8.20 to 19.83) 4.87 (39.0) <.001<.001 <.001<.001 Accuracy change (p) C–A 11.60 (5.78 to 17.43) 4.03 (39.9) <.001<.001 <.001<.001 Accuracy change (p) B–C 2.41 (−3.41-3.41 to 8.23) 0.84 (41.0) .408 NA Note.—Two-sided Welch t tests. Differences are first-named minus second-named group. Holm adjustment was applied to the B–A and C–A comparisons within each outcome, specialty stratum, and GPT-correctness subset. B–C is exploratory; its adjusted P value is not applicable. CIs are pointwise. CI = confidence interval; df = degrees of freedom; p = percentage points; NA = not applicable. Table S13: Diagnostic Revision Patterns by Specialty and Support Condition Population Outcome Group A Group B Group C Welch F (df1, df2) P Radiology residents Incorrect → correct 14.8±7.814.8± 7.8 19.9±11.019.9± 11.0 22.8±10.522.8± 10.5 3.98 (2, 36.9) .027 Radiology residents Correct → incorrect 5.2±3.95.2± 3.9 3.7±2.43.7± 2.4 5.4±3.35.4± 3.3 2.15 (2, 35.6) .131 Radiology residents Incorrect → incorrect 36.8±11.136.8± 11.1 30.2±11.230.2± 11.2 29.1±6.529.1± 6.5 3.55 (2, 36.1) .039 Radiology residents Correct → correct 43.3±7.843.3± 7.8 46.2±15.846.2± 15.8 42.7±12.942.7± 12.9 0.34 (2, 34.6) .713 Non-radiology residents Incorrect → correct 24.0±6.824.0± 6.8 19.7±8.419.7± 8.4 22.8±6.122.8± 6.1 1.66 (2, 38.9) .204 Non-radiology residents Correct → incorrect 5.8±3.95.8± 3.9 6.0±3.96.0± 3.9 6.5±3.26.5± 3.2 0.23 (2, 39.1) .796 Non-radiology residents Incorrect → incorrect 33.1±7.833.1± 7.8 34.4±6.134.4± 6.1 34.2±8.334.2± 8.3 0.20 (2, 39.2) .821 Non-radiology residents Correct → correct 37.1±9.837.1± 9.8 39.9±9.939.9± 9.9 36.4±6.936.4± 6.9 0.89 (2, 37.9) .418 Note.—Values are mean ± SD. Omnibus P values are unadjusted. Transition values are percentages using all 60 cases as the denominator. Beneficial revision: incorrect → correct; harmful revision: correct → incorrect. CI = confidence interval; SD = standard deviation; df = degrees of freedom; p = percentage points. Table S14: Diagnostic Revision Patterns by Specialty and Support Condition (continued). Radiology residents; pairwise contrasts Outcome Contrast Mean difference (95% CI) Welch t (df) P value Holm P Incorrect → correct B–A 5.17 (−0.84-0.84 to 11.18) 1.75 (36.1) .089 .089 Incorrect → correct C–A 8.06 (2.01 to 14.10) 2.71 (33.1) .011 .021 Incorrect → correct B–C −2.89-2.89 (−9.77-9.77 to 3.99) −0.85-0.85 (37.9) .401 NA Correct → incorrect B–A −1.52-1.52 (−3.60-3.60 to 0.57) −1.48-1.48 (31.1) .148 .297 Correct → incorrect C–A 0.18 (−2.17-2.17 to 2.54) 0.16 (36.5) .875 .875 Correct → incorrect B–C −1.70-1.70 (−3.58-3.58 to 0.18) −1.84-1.84 (32.5) .074 NA Incorrect → incorrect B–A −6.60-6.60 (−13.64-13.64 to 0.45) −1.89-1.89 (38.9) .066 .066 Incorrect → incorrect C–A −7.71-7.71 (−13.61-13.61 to −1.82-1.82) −2.67-2.67 (30.9) .012 .024 Incorrect → incorrect B–C 1.12 (−4.72-4.72 to 6.95) 0.39 (32.6) .700 NA Correct → correct B–A 2.94 (−4.93-4.93 to 10.81) 0.76 (29.5) .451 .902 Correct → correct C–A −0.53-0.53 (−7.56-7.56 to 6.50) −0.15-0.15 (29.1) .878 .902 Correct → correct B–C 3.47 (−5.73-5.73 to 12.67) 0.76 (37.7) .450 NA Note.—Two-sided Welch t tests. Differences are first-named minus second-named group. Holm adjustment was applied to the B–A and C–A comparisons within each revision outcome and specialty stratum. B–C is exploratory; its adjusted P value is not applicable. CIs are pointwise. Beneficial revision: incorrect → correct; harmful revision: correct → incorrect. CI = confidence interval; df = degrees of freedom; p = percentage points; NA = not applicable. Table S15: Diagnostic Revision Patterns by Specialty and Support Condition (continued). Non-radiology residents; pairwise contrasts Outcome Contrast Mean difference (95% CI) Welch t (df) P value Holm P Incorrect → correct B–A −4.32-4.32 (−9.16-9.16 to 0.52) −1.81-1.81 (38.0) .079 .157 Incorrect → correct C–A −1.20-1.20 (−5.24-5.24 to 2.85) −0.60-0.60 (38.2) .553 .553 Incorrect → correct B–C −3.12-3.12 (−7.68-7.68 to 1.43) −1.39-1.39 (36.2) .173 NA Correct → incorrect B–A 0.12 (−2.35-2.35 to 2.59) 0.10 (38.9) .923 1.000 Correct → incorrect C–A 0.68 (−1.57-1.57 to 2.93) 0.61 (36.8) .543 1.000 Correct → incorrect B–C −0.56-0.56 (−2.77-2.77 to 1.64) −0.52-0.52 (38.8) .609 NA Incorrect → incorrect B–A 1.36 (−3.10-3.10 to 5.82) 0.62 (36.0) .540 1.000 Incorrect → incorrect C–A 1.16 (−3.87-3.87 to 6.18) 0.47 (39.9) .644 1.000 Incorrect → incorrect B–C 0.20 (−4.28-4.28 to 4.68) 0.09 (38.6) .928 NA Correct → correct B–A 2.84 (−3.38-3.38 to 9.06) 0.92 (38.9) .362 .724 Correct → correct C–A −0.64-0.64 (−6.03-6.03 to 4.74) −0.24-0.24 (33.8) .809 .809 Correct → correct B–C 3.48 (−1.81-1.81 to 8.77) 1.33 (35.7) .190 NA Note.—Two-sided Welch t tests. Differences are first-named minus second-named group. Holm adjustment was applied to the B–A and C–A comparisons within each revision outcome and specialty stratum. B–C is exploratory; its adjusted P value is not applicable. CIs are pointwise. Beneficial revision: incorrect → correct; harmful revision: correct → incorrect. CI = confidence interval; df = degrees of freedom; p = percentage points; NA = not applicable. Table S16: Dual-Suggestion Effects and Specialty Interactions. Accuracy change Analysis Contrast Effect estimate, p (95% CI) Wald statistic P value Holm P Radiology residents Dual vs single 7.28 (2.53 to 12.03) z=3.00z=3.00 .003 NA Radiology residents B–A 6.69 (1.03 to 12.34) z=2.32z=2.32 .021 .024 Radiology residents C–A 7.87 (1.73 to 14.02) z=2.51z=2.51 .012 .024 Non-radiology residents Dual vs single −3.16-3.16 (−6.95-6.95 to 0.64) z=−1.63z=-1.63 .103 NA Non-radiology residents B–A −4.44-4.44 (−9.27-9.27 to 0.40) z=−1.80z=-1.80 .072 .144 Non-radiology residents C–A −1.88-1.88 (−5.79-5.79 to 2.03) z=−0.94z=-0.94 .346 .346 Specialty interaction Dual vs single 10.44 (4.36 to 16.52) z=3.36z=3.36 <.001<.001 NA Specialty interaction B–A 11.12 (3.68 to 18.56) z=2.93z=2.93 .003 .007 Specialty interaction C–A 9.75 (2.47 to 17.03) z=2.62z=2.62 .009 .009 Overall interaction Group × specialty NA χ2(2)=11.38χ^2(2)=11.38 .003 NA Note.—HC3 linear models. Specialty interaction is the radiology contrast minus the non-radiology contrast. Holm adjustment covers B–A/C–A within each outcome and analysis block. The dual-suggestion effect was defined as the equally weighted contrast [(B+C)/2]−A[(B+C)/2]-A. Averaged contrasts and omnibus tests are unadjusted; NA = not applicable. CI = confidence interval; p = percentage points. Table S17: Dual-Suggestion Effects and Specialty Interactions (continued). Beneficial revisions Analysis Contrast Effect estimate, p (95% CI) Wald statistic P value Holm P Radiology residents Dual vs single 6.61 (1.72 to 11.51) z=2.65z=2.65 .008 NA Radiology residents B–A 5.17 (−0.78-0.78 to 11.12) z=1.70z=1.70 .089 .089 Radiology residents C–A 8.06 (2.08 to 14.04) z=2.64z=2.64 .008 .017 Non-radiology residents Dual vs single −2.76-2.76 (−6.57-6.57 to 1.05) z=−1.42z=-1.42 .156 NA Non-radiology residents B–A −4.32-4.32 (−9.12-9.12 to 0.48) z=−1.76z=-1.76 .078 .156 Non-radiology residents C–A −1.20-1.20 (−5.21-5.21 to 2.82) z=−0.58z=-0.58 .559 .559 Specialty interaction Dual vs single 9.37 (3.17 to 15.57) z=2.96z=2.96 .003 NA Specialty interaction B–A 9.49 (1.84 to 17.14) z=2.43z=2.43 .015 .024 Specialty interaction C–A 9.25 (2.05 to 16.46) z=2.52z=2.52 .012 .024 Overall interaction Group × specialty NA χ2(2)=8.80χ^2(2)=8.80 .012 NA Note.—HC3 linear models. Specialty interaction is the radiology contrast minus the non-radiology contrast. Holm adjustment covers B–A/C–A within each outcome and analysis block. The dual-suggestion effect was defined as the equally weighted contrast [(B+C)/2]−A[(B+C)/2]-A. Averaged contrasts and omnibus tests are unadjusted; NA = not applicable. CI = confidence interval; p = percentage points. Table S18: Dual-Suggestion Effects and Specialty Interactions (continued). Harmful revisions Analysis Contrast Effect estimate, p (95% CI) Wald statistic P value Holm P Radiology residents Dual vs single −0.67-0.67 (−2.66-2.66 to 1.33) z=−0.65z=-0.65 .513 NA Radiology residents B–A −1.52-1.52 (−3.57-3.57 to 0.54) z=−1.45z=-1.45 .148 .297 Radiology residents C–A 0.18 (−2.15-2.15 to 2.52) z=0.15z=0.15 .877 .877 Non-radiology residents Dual vs single 0.40 (−1.67-1.67 to 2.47) z=0.38z=0.38 .705 NA Non-radiology residents B–A 0.12 (−2.33-2.33 to 2.57) z=0.10z=0.10 .924 1.000 Non-radiology residents C–A 0.68 (−1.55-1.55 to 2.91) z=0.60z=0.60 .550 1.000 Specialty interaction Dual vs single −1.07-1.07 (−3.95-3.95 to 1.81) z=−0.73z=-0.73 .468 NA Specialty interaction B–A −1.63-1.63 (−4.84-4.84 to 1.57) z=−1.00z=-1.00 .317 .633 Specialty interaction C–A −0.50-0.50 (−3.73-3.73 to 2.74) z=−0.30z=-0.30 .763 .763 Overall interaction Group × specialty NA χ2(2)=1.14χ^2(2)=1.14 .564 NA Note.—HC3 linear models. Specialty interaction is the radiology contrast minus the non-radiology contrast. Holm adjustment covers B–A/C–A within each outcome and analysis block. The dual-suggestion effect was defined as the equally weighted contrast [(B+C)/2]−A[(B+C)/2]-A. Averaged contrasts and omnibus tests are unadjusted; NA = not applicable. CI = confidence interval; p = percentage points. Table S19: Revision Components by Shared GPT Suggestion Correctness, Specialty, and Support Condition Shared GPT suggestion Revision component Group A Group B Group C Overall P B–A, Holm P C–A, Holm P Radiology residents Correct (n=39n=39) Incorrect → correct 22.1±11.622.1± 11.6 24.3±15.424.3± 15.4 29.0±14.029.0± 14.0 .260 .600 .203 Correct → incorrect 1.2±1.91.2± 1.9 1.8±2.01.8± 2.0 3.2±3.23.2± 3.2 .063 .279 .040 Incorrect (n=21n=21) Incorrect → correct 1.2±2.11.2± 2.1 11.8±7.811.8± 7.8 11.3±5.511.3± 5.5 <.001<.001 <.001<.001 <.001<.001 Correct → incorrect 12.6±10.512.6± 10.5 7.0±5.87.0± 5.8 9.3±6.89.3± 6.8 .116 .089 .245 Non-radiology residents Correct (n=39n=39) Incorrect → correct 36.0±10.536.0± 10.5 25.3±11.825.3± 11.8 29.1±8.329.1± 8.3 .012 .007 .025 Correct → incorrect 0.9±1.70.9± 1.7 4.5±3.54.5± 3.5 3.1±3.43.1± 3.4 <.001<.001 <.001<.001 .009 Incorrect (n=21n=21) Incorrect → correct 1.7±2.31.7± 2.3 9.3±6.59.3± 6.5 11.0±6.611.0± 6.6 <.001<.001 <.001<.001 <.001<.001 Correct → incorrect 15.0±9.315.0± 9.3 8.6±7.08.6± 7.0 12.8±7.512.8± 7.5 .042 .037 .402 Note.—Data are mean ± SD and represent the percentage of cases within each correctness stratum. Beneficial revisions are incorrect-to-correct transitions; harmful revisions are correct-to-incorrect transitions. Overall P values were calculated using Welch analysis of variance. Pairwise P values compare groups B and C separately with group A and were Holm-adjusted across the two prespecified comparisons within each specialty and revision component. B–C comparisons are not shown. The correctness strata were defined by the GPT suggestion shared across all three support conditions; correctness of the second suggestion in groups B and C could differ within each stratum. Table S20: Outcomes in Cases for Which Both AI Models Were Incorrect Outcome Single-suggestion support Dual-suggestion support Mean difference (95% CI) P value Holm P GPT + Kimi: Group B vs Group A (11 cases per participant) AI-assisted accuracy (%) 10.0±9.410.0± 9.4 8.4±10.78.4± 10.7 −1.56-1.56 (−5.98-5.98 to 2.86) .485 .970 Accuracy change (p) −10.7±11.6-10.7± 11.6 −12.1±10.2-12.1± 10.2 −1.44-1.44 (−6.25-6.25 to 3.37) .553 .553 Confidence change (points) 0.11±0.340.11± 0.34 0.11±0.240.11± 0.24 0.00 (−0.13-0.13 to 0.13) .956 1.000 Incorrect final diagnoses with increased confidence (%) 20.5±22.120.5± 22.1 17.3±17.517.3± 17.5 −3.14-3.14 (−11.92-11.92 to 5.64) .479 .957 GPT + Gemini: Group C vs Group A (10 cases per participant) AI-assisted accuracy (%) 5.3±9.15.3± 9.1 4.6±10.74.6± 10.7 −0.62-0.62 (−5.01-5.01 to 3.78) .781 .970 Accuracy change (p) −15.3±13.6-15.3± 13.6 −19.8±14.1-19.8± 14.1 −4.51-4.51 (−10.62-10.62 to 1.61) .146 .292 Confidence change (points) 0.09±0.320.09± 0.32 0.09±0.330.09± 0.33 −0.01-0.01 (−0.15-0.15 to 0.14) .921 1.000 Incorrect final diagnoses with increased confidence (%) 21.8±21.021.8± 21.0 19.3±22.419.3± 22.4 −2.48-2.48 (−12.08-12.08 to 7.12) .608 .957 Note.—Data are participant-level mean ± SD. Mean differences are dual-suggestion minus single-suggestion support. Accuracy change was calculated as AI-assisted minus AI-unassisted accuracy. Negative values indicate a smaller accuracy gain or a larger decrease with dual-suggestion support. Incorrect final diagnoses with increased confidence were final incorrect decisions for which confidence increased after AI assistance, expressed as a percentage of cases in the corresponding stratum. Comparisons used two-sided Welch t tests. Holm adjustment covered the two prespecified dual-versus-single contrasts within each outcome. CI = confidence interval; p = percentage points.

Instant research discovery