Paper deep dive
Measuring and Improving Complex-Atomic Answer Consistency in Endoscopic VQA
Yuhao Liu, Cheng Zhao, Guanghui Yue
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/21/2026, 5:50:27 AM
Summary
This paper introduces EndoCA, a benchmark for evaluating complex-atomic answer consistency in endoscopic Visual Question Answering (VQA), and proposes Atomic-Support Reconciliation (ASR), a training-free mechanism to improve consistency by using atomic answers as premises for revising complex answers.
Entities (10)
Relation Signals (9)
EndoCA → containssuite → EndoCA-Core
confidence 95% · EndoCA contains two suites: EndoCA-Core... and EndoCA-Diagnostic
EndoCA → containssuite → EndoCA-Diagnostic
confidence 95% · EndoCA contains two suites: ... and EndoCA-Diagnostic
Atomic-Support Reconciliation → includesvariant → ASR-Revise
confidence 95% · ASR-Revise improves paired complex-atomic correctness... while ASR-Selective improves accuracy
Atomic-Support Reconciliation → includesvariant → ASR-Selective
confidence 95% · ASR-Revise improves paired complex-atomic correctness... while ASR-Selective improves accuracy
EndoCA → measures → Complex-Atomic Inconsistency
confidence 93% · EndoCA reports four metrics... Complex-Atomic Inconsistency is computed among correct complex-answer cases
EndoCA → derivedfrom → Kvasir-VQA
confidence 92% · We construct EndoCA from the two-level QA structure formed by Kvasir-VQA and Kvasir-VQA-x1.
EndoCA → derivedfrom → Kvasir-VQA-x1
confidence 92% · We construct EndoCA from the two-level QA structure formed by Kvasir-VQA and Kvasir-VQA-x1.
ASR-Revise → usesinput → atomic answers
confidence 90% · ASR-Revise revises complex answers with atomic answers through a training-free answer reconciliation procedure.
ASR-Selective → usesinput → atomic answers
confidence 90% · ASR-Selective performs selective answering under complex-atomic consistency... consistent with the atomic premise set
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Endoscopic visual question answering (VQA) increasingly asks complex questions that combine several endoscopic answer components rather than isolated factual queries. Such complex answers may be scored as correct even when the same model fails on associated atomic questions. We introduce EndoCA, a paired complex-atomic answer consistency benchmark for evaluating whether complex answers remain consistent with same-image atomic answers. EndoCA contains two suites: EndoCA-Core evaluates compact question-complexity patterns commonly seen in practical endoscopic VQA, and EndoCA-Diagnostic supports controlled analysis across increasing question complexity. We evaluate 11 VLMs spanning open, medical, endoscopy-adapted, and closed-source models on EndoCA. Some VLMs achieve high complex-answer accuracy, yet their atomic-answer accuracy and complex-atomic answer consistency remain substantially lower. To reduce this complex-atomic inconsistency, we introduce Atomic-Support Reconciliation (ASR), a training-free mechanism that uses model-generated atomic answers as contextual premises for answer revision and consistency-guided selective answering. On four selected publicly available models, ASR-Revise improves paired complex-atomic correctness with modest changes in complex-answer accuracy, while ASR-Selective improves accuracy on answered cases by allowing the model to abstain from less reliable cases. Together, EndoCA and ASR provide a consistency-aware benchmark and a training-free mechanism for answer reconciliation and selective answering in endoscopic VQA.
Tags
Links
- Source: https://arxiv.org/abs/2607.17834v1
- Canonical: https://arxiv.org/abs/2607.17834v1
Trouble viewing inline? Open PDF directly →
Full Text
38,961 characters extracted from source content.
Expand or collapse full text
Measuring and Improving Complex-Atomic Answer Consistency in Endoscopic VQA Yuhao Liu, Cheng Zhao, Guanghui Yue Abstract Endoscopic visual question answering (VQA) increasingly asks complex questions that combine several endoscopic answer components rather than isolated factual queries. Such complex answers may be scored as correct even when the same model fails on associated atomic questions. We introduce EndoCA, a paired complex-atomic answer consistency benchmark for evaluating whether complex answers remain consistent with same-image atomic answers. EndoCA contains two suites: EndoCA-Core evaluates compact question-complexity patterns commonly seen in practical endoscopic VQA, and EndoCA-Diagnostic supports controlled analysis across increasing question complexity. We evaluate 11 VLMs spanning open, medical, endoscopy-adapted, and closed-source models on EndoCA. Some VLMs achieve high complex-answer accuracy, yet their atomic-answer accuracy and complex-atomic answer consistency remain substantially lower. To reduce this complex-atomic inconsistency, we introduce Atomic-Support Reconciliation (ASR), a training-free mechanism that uses model-generated atomic answers as contextual premises for answer revision and consistency-guided selective answering. On four selected publicly available models, ASR-Revise improves paired complex-atomic correctness with modest changes in complex-answer accuracy, while ASR-Selective improves accuracy on answered cases by allowing the model to abstain from less reliable cases. Together, EndoCA and ASR provide a consistency-aware benchmark and a training-free mechanism for answer reconciliation and selective answering in endoscopic VQA. I Introduction Visual question answering (VQA) provides a task-directed way to evaluate visual understanding: a model must interpret an image under a natural-language query and return an answer grounded in the relevant visual evidence [2]. Medical VQA extends this paradigm to clinically oriented image interpretation [19]. With stronger vision-language models (VLMs), endoscopic VQA is moving from isolated recognition prompts, such as whether a polyp is present, toward complex questions that combine multiple visual components. We use the term atomic question to denote a same-image component check for one visual commitment, such as an anatomical site, a polyp count, or an instrument type. A complex question can combine several such checks into one natural answer. Figure 1: Paired complex-atomic evaluation. A correct complex answer can still disagree with associated atomic answers on the same image. Recent endoscopic VLM benchmarks reflect this broader evaluation trend. For example, EndoBench evaluates models across multiple endoscopic scenarios, clinical tasks, and visual prompting granularities [21]. Such benchmarks provide important final-answer and task-performance signals. At the same time, when a complex question involves multiple component judgments, each component can be checked through a simpler atomic question. If the same model gives a correct complex answer but fails corresponding atomic questions under the same image and related question context, its image understanding and evidence use remain unstable in a way that complex-answer accuracy can obscure. The motivating example in Fig. 1 illustrates this gap: the complex answer is accepted while one corresponding atomic answer is wrong. In medical settings, this hidden inconsistency matters because apparently correct answers may mask unreliable interpretation of the underlying endoscopic evidence. We construct EndoCA from the two-level QA structure formed by Kvasir-VQA [10] and Kvasir-VQA-x1 [9]. Kvasir-VQA provides the component-level atomic QA pairs, while Kvasir-VQA-x1 combines corresponding same-image pairs into complex questions and naturalized complex answers. EndoCA keeps this link: each complex question is evaluated together with its associated atomic questions. EndoCA-Core covers common compact cases with one or two associated atomic questions and serves as the main comparison setting. EndoCA-Diagnostic follows the same protocol but adds a controlled three-atomic-question stratum to test how paired correctness changes as more endoscopic visual judgments are combined. Across both suites, EndoCA reports complex-answer accuracy, atomic-answer accuracy, joint accuracy, and complex-atomic inconsistency to separate final-answer correctness from same-image paired consistency. Our study evaluates 11 VLMs from general open, medical open, endoscopy-adapted, and closed-source groups. Across both EndoCA suites, complex-answer rankings change once every associated atomic prediction must also be correct, and higher question complexity places stronger pressure on all-atomic accuracy. These results support reporting complex-answer accuracy together with atomic-answer accuracy, joint accuracy, and complex-atomic inconsistency whenever associated atomic questions are available. Building on this paired evaluation signal, we introduce Atomic-Support Reconciliation (ASR), a training-free mechanism that uses model-generated atomic answers as contextual premises for answer revision and consistency-guided selective answering. This paper makes three contributions: • We introduce EndoCA, a paired complex-atomic answer consistency benchmark for endoscopic VQA, with EndoCA-Core for common compact question patterns and EndoCA-Diagnostic for controlled higher-complexity analysis. • We evaluate 11 VLMs across general open, medical open, endoscopy-adapted, and closed-source groups, showing that complex-answer accuracy alone can hide substantial same-image complex-atomic answer inconsistency. • We propose ASR, a training-free mechanism that uses model-generated atomic answers as contextual premises for answer revision and consistency-guided selective answering. TABLE I: Representative EndoCA complex-atomic QA examples. Question- complexity Image Complex QA Atomic QA 1 Q: How many polyps are present in the gastrointestinal tract shown? A: one polyp observed Polyp Count q_1: How many polyps are in the image? Atomic Answer a_1: 1 2 Q: Are there any green or black box artifacts and is there visible text on the image? A: Evidence of green and black box artifacts along with visible text is present. Box Artifact Presence q_1: Is there a green/black box artifact? Atomic Answer a_1: yes Text Presence q_2: Is there text? Atomic Answer a_2: yes 3 Q: Where in the image are the instruments located, is there a green/black box artifact, and what instruments are visible? A: A tube is visible in the center-right and upper-right regions, with a green/black box artifact present. Instrument Location q_1: Where in the image is the instrument? Atomic Answer a_1: center-right; upper-right Box Artifact Presence q_2: Is there a green/black box artifact? Atomic Answer a_2: yes Instrument Presence q_3: Are there any instruments in the image? Check all that are present. Atomic Answer a_3: tube I Related Work I-A Medical and Endoscopic VQA Medical VQA has progressed from modality-specific QA resources to broader clinical VLM benchmarks. VQA-RAD [16], SLAKE [20], and PathVQA [11] provide foundations for radiology, semantically labeled medical QA, and pathology, respectively. In gastrointestinal endoscopy, Kvasir-VQA [10] and Kvasir-VQA-x1 [9] provide concise endoscopic QA and complex reasoning-oriented QA. Recent efforts further broaden the evaluation context: EndoBench evaluates VLMs across multiple endoscopic scenarios, clinical tasks, and visual prompting granularities [21]; Gastric-X studies multimodal gastric cancer analysis across VQA, report generation, retrieval, classification, and localization [22]; and MEDVQA-GI explores challenge systems for gastrointestinal endoscopy VQA [8]. These studies expand the task scope and clinical context of medical and endoscopic VQA; their primary evaluation signals still center on final-answer accuracy and task performance, leaving model-internal consistency and stability less directly assessed. I-B Consistency Evaluation and Answer Reconciliation Consistency-aware VQA and atomic evaluation study whether model answers remain reliable across related questions, structured reasoning steps, or smaller factual units. SQuINTing probes VQA models with lower-level perception sub-questions [27], and GQA provides structured real-world visual reasoning questions that support compositional evaluation [13]. In generation evaluation, FActScore decomposes long-form text into atomic facts [24], while FaithScore and POPE assess VLM hallucination and image-grounded faithfulness at finer granularity [14, 18]. Reliability-oriented work has also appeared in surgical and endoscopic settings, such as SurgViVQA for temporally grounded surgical VideoQA robustness [7]. EndoCA brings this consistency-aware atomic checking perspective to endoscopic VQA by focusing on whether accepted complex answers remain supported by their associated atomic answers. Answer correction and self-checking methods use related predictions or multiple generations to improve reliability after an initial response. SOrT uses sub-question relevance to improve VQA consistency [6]; self-consistency decoding samples multiple reasoning paths and selects an answer supported by agreement across paths [29]; and SelfCheckGPT detects hallucination by comparing consistency across sampled passages [23]. In endoscopic diagnosis, EndoCogniAgent uses closed-loop self-consistency validation for agentic reasoning [28]. ASR differs from prior self-checking methods by using model-generated atomic answers as explicit premises for complex-answer revision and as consistency evidence for selective answering. I EndoCA Benchmark I-A Source Dataset EndoCA uses the paired QA structure of Kvasir-VQA and Kvasir-VQA-x1. Kvasir-VQA contains concise QA pairs over gastrointestinal endoscopic images, covering common endoscopic findings, landmarks, instruments, and artifacts. Kvasir-VQA-x1 records complex questions and naturalized complex answers formed from corresponding same-image QA pairs. In EndoCA, the associated atomic questions are these same-image component questions linked to a complex question. This recorded association links each complex question to its atomic answer annotations, providing the paired units used for complex-atomic evaluation. I-B Benchmark Construction The construction process starts from samples that contain an endoscopic image, a non-empty complex question, a complex answer annotation, and at least one associated atomic question-answer pair. Fixed parsing, answer normalization, consistency checks, and validity filtering standardize QA fields, canonicalize answer forms, require complete image/question/answer fields, well-formed atomic-question sets, and supported question types, and verify agreement between the complex annotation and its associated atomic annotations under the same scoring rules. The resulting paired units are illustrated in Table I. EndoCA is organized into two complementary suites. Question complexity is the number of associated atomic questions in a complex answer. EndoCA-Core is the main evaluation suite, with 12,000 complex samples, 15,736 atomic QA items, and 27,736 total QA items per model; it focuses on complexity-1 and complexity-2 cases for stable cross-model comparison. EndoCA-Diagnostic contains 6,000 complex samples, 9,300 atomic QA items, and 15,300 total QA items per model; it keeps compact questions as the majority and adds 900 complexity-3 cases as a controlled higher-complexity stratum. The benchmark atomic questions cover high-confidence endoscopic question types over common findings, landmarks, instruments, and artifacts. The suites use distinct sample identities: Core provides the main benchmark distribution, and Diagnostic provides a separate setting for question-complexity analysis. I-C Answer Scoring and Metrics For each sample i, EndoCA stores an endoscopic image ViV_i, a complex question QiQ_i, a complex answer annotation AiA_i, and an associated atomic-question list UiU_i: xi x_i =(Vi,Qi,Ai,Ui), =(V_i,Q_i,A_i,U_i), (1) Ui U_i =((qi,1,ai,1),…,(qi,mi,ai,mi)). =((q_i,1,a_i,1),…,(q_i,m_i,a_i,m_i)). where qijq_ij and aija_ij are the j-th associated atomic question and its answer annotation. The quantity mi=|Ui|m_i=|U_i| is the question complexity, defined as the number of atomic questions associated with QiQ_i. EndoCA evaluates answers with a fixed question-type-aware scorer. The same concise-answer prompt template is used for complex and atomic probes, with the question field replaced by QiQ_i or qijq_ij. Model outputs may be short phrases or complete sentences, so the scorer normalizes common surface variation, including case, spacing, punctuation, yes/no forms, count words, and canonical labels. For separate atomic predictions, the expected question type determines the matching rule: binary questions are mapped to yes/no values, count questions to integers, categorical questions to canonical labels, and multi-label questions to normalized label sets. For complex-answer scoring, EndoCA uses the verified associated atomic annotations as component-level ground truth for the original complex question. A complex prediction is counted as correct only when the normalized answer matches every associated atomic answer annotation under the corresponding question-type rule. Empty outputs, execution errors, ambiguous answers, and non-committal responses are counted as incorrect. This fixed protocol enables reproducible comparison across open-source and closed-source VLMs and keeps the evaluation independent of an additional LLM judge. TABLE I: Example scoring of an EndoCA paired unit. Step Scoring process and result Paired unit xi=(Vi,Qi,Ai,Ui)x_i=(V_i,Q_i,A_i,U_i), where UiU_i contains two associated atomic QA annotations with ai,1=a_i,1= yes and ai,2=a_i,2= yes. Prompt template Complex and atomic probes use the same template, replacing only ⟨Q⟩ Q : Answer the question based on the endoscopic image. Respond with a concise short answer only. Do not explain. Question: ⟨Q⟩ Q Complex QA ⟨Q⟩=Qi Q =Q_i: Are there any green/black box artifacts and is there visible text in the image? Raw output A^i A_i: A green/black box artifact is present, and visible text is present. Complex score The scorer extracts component labels: box artifact = yes and visible text = yes. Both match (ai,1,ai,2)(a_i,1,a_i,2), so i=1C_i=1. Atomic QA ⟨Q⟩=qi,1 Q =q_i,1: Is there a green/black box artifact? Raw output a^i,1= a_i,1= yes. ⟨Q⟩=qi,2 Q =q_i,2: Is there visible text? a^i,2= a_i,2= no. Atomic score a^i,1 a_i,1 matches ai,1a_i,1, so i,1=1A_i,1=1. a^i,2 a_i,2 does not match ai,2a_i,2, so i,2=0A_i,2=0. Indicators After normalization, i=1C_i=1, i,1=1A_i,1=1, i,2=0A_i,2=0, and i⋆=0A _i=0. For a model output, let A^i A_i be the complex answer and a^ij a_ij be the answer to atomic question qijq_ij. Let SC(A^i,xi)∈0,1S_C( A_i,x_i)∈\0,1\ denote complex-answer correctness, and SA(a^ij,qij,aij)∈0,1S_A( a_ij,q_ij,a_ij)∈\0,1\ denote atomic-answer correctness. We define i _i =SC(A^i,xi), =S_C( A_i,x_i), ij _ij =SA(a^ij,qij,aij), =S_A( a_ij,q_ij,a_ij), (2) i⋆ _i =∏j=1miij. = _j=1^m_iA_ij. Here, i=1C_i=1 when the complex prediction matches all associated atomic answer annotations for sample i, ij=1A_ij=1 when the model’s answer to qijq_ij matches its associated annotation, and i⋆=1A _i=1 only when all atomic questions for the sample are answered correctly. In question-complexity analysis, all-atomic accuracy averages i⋆A _i within each complexity group. EndoCA reports four metrics. Complex-answer accuracy measures the accuracy of model answers to complex questions. Atomic-answer accuracy averages correctness across the model’s separate answers to the associated atomic questions. Joint accuracy requires both the complex prediction and every separately generated atomic prediction to match their associated answer annotations for the same sample. Complex-atomic inconsistency is computed among correct complex-answer cases and measures the share with at least one wrong atomic prediction: Complex-answer Accuracy =1N∑i=1Ni, = 1N _i=1^NC_i, (3) Atomic-answer Accuracy =∑i=1N∑j=1miij∑i=1Nmi, = _i=1^N _j=1^m_iA_ij _i=1^Nm_i, (4) Joint Accuracy =1N∑i=1Nii⋆, = 1N _i=1^NC_iA _i, (5) Complex-Atomic Inconsistency =∑i=1Ni(1−i⋆)∑i=1Ni. = _i=1^NC_i(1-A _i) _i=1^NC_i. (6) If a model has no correct complex-answer cases, this conditional metric is undefined and should be reported as N/A; in our evaluated rows, every model has at least one correct complex answer, so the denominator is nonzero. Result tables abbreviate the three accuracy metrics as Complex Acc., Atomic Acc., and Joint Acc.; the inconsistency metric is written out. IV Method IV-A Motivation EndoCA exposes complex-atomic inconsistency in endoscopic VQA: a model may give an acceptable complex answer while failing one supporting same-image atomic question. This observation suggests a direct inference-time mitigation strategy. ASR uses only the image, the complex question, the direct model answer, and model-generated atomic answers; the associated atomic annotations are used only for EndoCA scoring. Atomic-Support Reconciliation (ASR) treats the model-generated atomic answers as explicit component-level evidence for answer revision and selective answering. Workflow 1: Atomic-Support Reconciliation Input: image ViV_i, complex question QiQ_i, associated atomic questions qij\q_ij\, VLM M. 1. Direct answer: A^i←M(Vi,Qi) A_i← M(V_i,Q_i). 2. Atomic answers: for j=1,…,mij=1,…,m_i do a^ij←M(Vi,qij) a_ij← M(V_i,q_ij). end for 3. Atomic premises: Pi←(qij,a^ij)j=1miP_i←\(q_ij, a_ij)\_j=1^m_i. 4. Revise: A~i←M(Vi,Qi,A^i,Pi) A_i← M(V_i,Q_i, A_i,P_i). 5. Select: di←M(Vi,Qi,A~i,Pi)d_i← M(V_i,Q_i, A_i,P_i), checking consistency between A~i A_i and PiP_i, where di∈answer,abstaind_i∈\answer,abstain\. Return: ASR-Revise: A~i A_i. ASR-Selective: A~i A_i if di=answerd_i=answer, otherwise abstain. Figure 2: ASR workflow for revision and selective answering. IV-B ASR-Revise ASR-Revise revises complex answers with atomic answers through a training-free answer reconciliation procedure. For each sample, the model first produces a direct complex answer A^i A_i to the complex question QiQ_i. The same model then answers every associated atomic question qijq_ij, yielding a^ij a_ij. ASR organizes the atomic question-answer pairs into an atomic premise set PiP_i, which records the model’s own same-image component-level evidence: Pi=(qij,a^ij)j=1mi.P_i=\(q_ij, a_ij)\_j=1^m_i. (7) The revise step asks the model to produce a reconciled complex answer A~i A_i using the image, the complex question, the model’s direct complex answer, and the model-generated atomic answers as contextual premises. This preserves the original complex-answer task while exposing the model to a structured summary of its atomic responses. All ASR calls use a fixed prompt template per variant: ASR-Revise requests a concise revised answer, and ASR-Selective requests an explicit answer/abstain decision together with the revised answer and a coarse consistency label. Selective-answering coverage counts only samples with an explicit answer decision; revised text is scored by the same EndoCA protocol as direct complex answers. By conditioning on the model-generated atomic answers and applying a consistency-based answer decision, ASR can improve joint accuracy and reduce complex-atomic inconsistency even when full-coverage complex-answer accuracy changes only modestly. IV-C ASR-Selective ASR-Selective performs selective answering under complex-atomic consistency. After ASR-Revise produces A~i A_i, the model is asked to judge whether the revised complex answer is consistent with the atomic premise set PiP_i. The output is a compact decision object containing an “answer”/“abstain” decision, the revised answer, and a coarse consistency label. If the decision is “answer”, ASR-Selective returns A~i A_i; otherwise, it abstains for that sample. This selective-answering step provides a controlled way to withhold low-consistency responses. We evaluate it with coverage, the percentage of samples for which an answer is returned, and selective accuracy, the complex-answer accuracy on the answered subset. In endoscopic VQA, this reliability–coverage view is useful because the model returns more answers when its complex answer and model-generated atomic answers agree, and fewer answers when the paired evidence is unstable. V Experiments TABLE I: EndoCA-Core paired evaluation. Group Model Params QA Count Complex Acc. (%) Atomic Acc. (%) Joint Acc. (%) Complex-Atomic Inconsistency (%) General open InternVL2.5-8B 8B 27,736 51.4 44.0 31.5 38.7 Qwen2.5-VL-7B 7B 27,736 54.1 55.5 38.0 29.8 Qwen3-VL-8B 8B 27,736 55.1 67.3 47.6 13.7 Medical open MedGemma-4B 4B 27,736 35.4 29.1 17.1 51.8 Lingshu-7B 7B 27,736 51.6 53.7 36.9 28.6 LLaVA-Med 7B 27,736 28.7 17.0 4.9 83.0 Endoscopy -adapted MedGemma-FT 4B 27,736 74.5 73.0 55.8 25.1 Qwen2.5-VL-FT 7B 27,736 72.1 59.3 41.6 42.4 Closed source GPT-5.5 – 27,736 60.8 63.0 47.4 22.1 Grok-4.20 – 27,736 34.8 36.9 19.7 43.3 Claude Opus 4.7 – 27,736 63.9 76.1 56.0 12.5 TABLE IV: EndoCA-Diagnostic paired evaluation. Group Model Params QA Count Complex Acc. (%) Atomic Acc. (%) Joint Acc. (%) Complex-Atomic Inconsistency (%) General open InternVL2.5-8B 8B 15,300 46.1 43.6 26.7 42.0 Qwen2.5-VL-7B 7B 15,300 48.1 55.0 33.0 31.4 Qwen3-VL-8B 8B 15,300 50.3 66.0 42.1 16.3 Medical open MedGemma-4B 4B 15,300 32.1 28.4 14.8 54.0 Lingshu-7B 7B 15,300 46.3 54.1 32.9 28.9 LLaVA-Med 7B 15,300 22.6 15.3 3.8 83.1 Endoscopy -adapted MedGemma-FT 4B 15,300 70.4 71.2 50.7 28.0 Qwen2.5-VL-FT 7B 15,300 70.5 58.0 36.8 47.8 Closed source GPT-5.5 – 15,300 55.2 61.0 41.7 24.4 Grok-4.20 – 15,300 30.1 35.8 16.7 44.6 Claude Opus 4.7 – 15,300 54.2 56.6 36.0 33.5 V-A Experimental Setup EndoCA evaluates whether a model’s complex prediction matches the associated atomic answer annotations and remains consistent with the model’s separate answers to the associated atomic questions on the same image. We evaluate 11 VLMs from four model groups. The General open group includes InternVL2.5-8B [5], Qwen2.5-VL-7B [3], and Qwen3-VL-8B [4]. The Medical open group includes MedGemma-4B [26], Lingshu-7B [15], and LLaVA-Med [17]. The endoscopy-adapted group includes MedGemma-FT and Qwen2.5-VL-FT [9], which are LoRA-adapted on the endoscopic VQA domain [12]. We use them as in-domain adaptation references: they are expected to be strong on complex-answer accuracy, and the analysis asks whether that strength also carries over to paired atomic correctness. The Closed-source group includes GPT-5.5 [25], Grok-4.20 [30], and Claude Opus 4.7 [1]. Each model answers every complex question and every associated atomic question on the same image, and all outputs are scored with the fixed question-type-aware scorer defined in Section I. All local open-source models are evaluated with PyTorch on a server with 8 NVIDIA RTX 4090 GPUs. All prompts are zero-shot and contain one endoscopic image, one user question, and the fixed concise-answer instruction described in Section I, with no system prompt or in-context examples. Open-source models use deterministic decoding with do_sample=False and a maximum generation length of 80 tokens. Closed-source models are evaluated with temperature=0 and max_tokens=80; their reasoning settings are set to xhigh for GPT-5.5 and Grok-4.20, and max for Claude Opus 4.7. Images are loaded in RGB format and processed by each model’s default visual processor, without manual cropping or global resizing; closed-source API calls use base64-encoded original images. All reported runs cover the expected QA Count, and empty or invalid outputs are scored as incorrect under the fixed EndoCA scoring protocol. V-B EndoCA Benchmark Results On EndoCA-Core, complex-answer accuracy and paired atomic correctness diverge substantially (Table IV). The in-domain adapted references achieve the highest complex-answer accuracy, with MedGemma-FT and Qwen2.5-VL-FT reaching 74.5% and 72.1%, respectively. Their joint accuracy differs by 14.2 percentage points, indicating that similar complex-answer performance can correspond to different levels of atomic-answer correctness. Across open and closed-source models, rankings based on complex-answer accuracy change when every separately generated atomic prediction must also be correct. This supports the motivating concern that an accepted endoscopic complex answer may still hide unsupported component judgments. Figure 3: Accuracy–consistency radar chart for six representative VLMs on EndoCA-Core and EndoCA-Diagnostic. Complex-atomic consistency denotes the complement of inconsistency. The diagnostic suite confirms the same reliability gap under a controlled complexity axis. MedGemma-FT and Qwen2.5-VL-FT obtain nearly identical complex-answer accuracy, 70.4% and 70.5%, yet their joint accuracy differs substantially, 50.7% versus 36.8% (Table IV). Their complex-atomic inconsistency also differs by 19.8 percentage points. This contrast is central to EndoCA: paired metrics expose whether a correct complex response remains aligned with the model’s same-image atomic predictions. The six-model radar summary reinforces this separation between accepted complex answers and paired atomic correctness. Several models remain competitive on complex-answer accuracy but drop on joint accuracy or complex-atomic consistency. For visualization, Fig. 3 plots the complement of complex-atomic inconsistency so that all axes follow a higher-is-better direction. Question complexity makes the same effect visible at the sample level. All-atomic accuracy uses i⋆A _i from Eq. (2); a single wrong atomic prediction sets the sample-level indicator to zero. At complexity 3, several models retain non-trivial complex-answer accuracy while all-atomic accuracy remains much lower (Fig. 4). As a complex question combines more atomic judgments, all-atomic accuracy becomes a paired correctness stress test. Figure 4: Question-complexity analysis on EndoCA-Diagnostic. All-atomic accuracy requires every separate atomic prediction to match its associated answer annotation. V-C Evaluation of ASR We next evaluate whether ASR can reduce complex-atomic inconsistency by using model-generated atomic answers as contextual premises. We apply ASR to four stronger publicly available models selected from the general open, medical open, and endoscopy-adapted groups: Qwen3-VL-8B, Lingshu-7B, MedGemma-FT, and Qwen2.5-VL-FT. Relative to the direct EndoCA-Core results, ASR-Revise improves joint accuracy for all four models, with gains from +3.1 to +8.1 percentage points, and reduces complex-atomic inconsistency by 10.4 points on average (Table V). Its effect on full-coverage complex-answer accuracy is mixed, with the clearest benefit appearing in paired reconciliation. TABLE V: ASR-Revise and ASR-Selective results on EndoCA-Core. Model Direct Complex Acc. (%) Revise Complex Acc. (%) Joint Acc. Δ Complex-Atomic Inconsistency Δ Coverage (%) Selective Complex Acc. (%) Qwen3-VL-8B 55.1 54.7 +5.3 -10.3 73.0 68.3 MedGemma-FT 74.5 76.0 +8.1 -9.2 95.7 76.8 Qwen2.5-VL-FT 72.1 69.6 +3.1 -6.6 76.1 74.9 Lingshu-7B 51.6 48.4 +5.2 -15.5 58.9 63.9 Average 63.3 62.2 +5.4 -10.4 75.9 71.0 ASR-Selective characterizes a reliability–coverage trade-off for the answered subset. On average, it returns answers for 75.9% of samples and reaches 71.0% selective complex-answer accuracy. Models with less stable paired behavior improve answered-subset accuracy by abstaining more often. Fig. 5 compares Direct, ASR-Revise, and ASR-Selective complex-answer accuracy together with the coverage reported in Table V. Figure 5: Direct, ASR-Revise, and ASR-Selective complex-answer accuracy on EndoCA-Core. Direct and ASR-Revise accuracies use all samples; ASR-Selective accuracy is computed over answered samples only. V-D Ablation Study We isolate the role of atomic premises using no-atomic self-check controls on the same four models and EndoCA-Core samples. Direct is the original complex answer. For Complex Acc., Joint Acc., and Complex-Atomic Inconsistency, w/o atomic is Self-Revise, which revises using the image, complex question, and direct answer without model-generated atomic premises. For Coverage and Selective Complex Acc., w/o atomic is Self-Selective, which makes the answer/abstain decision without atomic consistency evidence. ASR adds atomic premises to the corresponding revise and selective variants. TABLE VI: Atomic-premise ablation on EndoCA-Core. Method Complex Acc. (%) Joint Acc. (%) Complex-Atomic Inconsistency (%) Coverage (%) Selective Complex Acc. (%) Direct 63.3 45.5 27.4 100.0 63.3 w/o atomic 60.6 43.0 28.2 84.5 66.8 ASR 62.2 50.9 17.1 75.9 71.0 The ablation shows that generic self-checking alone does not recover the paired-correctness gains of ASR. Self-Revise without atomic premises reduces full-coverage complex-answer accuracy from 63.3% to 60.6% and does not improve joint accuracy or inconsistency, whereas ASR keeps complex-answer accuracy close to Direct at 62.2%, improves joint accuracy to 50.9%, and reduces complex-atomic inconsistency to 17.1% (Table VI). The selective columns show a related pattern: Self-Selective filters some unreliable answers, reaching 66.8% answered-subset accuracy at 84.5% coverage, while ASR-Selective achieves 71.0% at 75.9% coverage by using atomic premises as a stronger consistency signal. These results indicate that conditioning revision and selection on model-generated atomic answers is the key source of the paired-correctness gains. VI Conclusion This paper presented EndoCA, a paired complex-atomic evaluation protocol for endoscopic VQA that scores complex predictions together with same-image atomic predictions. Across 11 VLMs, complex-answer accuracy gives an incomplete view of paired correctness: models with strong complex-answer performance can still show lower atomic-answer accuracy, lower joint accuracy, and non-negligible complex-atomic inconsistency. ASR further shows that model-generated atomic answers can serve as contextual premises for answer revision and consistency-guided selective answering, exposing a reliability–coverage trade-off. The fixed EndoCA scorer enables reproducible comparison across model groups, while future work should extend paired evaluation to free-form clinical questions through reliable automatic decomposition and careful comparison of decomposition strategies. References [1] Anthropic (2026-04) Introducing Claude Opus 4.7. Note: Anthropic News. https://w.anthropic.com/news/claude-opus-4-7 Cited by: §V-A. [2] S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh (2015) VQA: visual question answering. In Proceedings of the IEEE International Conference on Computer Vision, p. 2425–2433. Cited by: §I. [3] S. Bai et al. (2025) Qwen2.5-VL technical report. Note: arXiv:2502.13923 External Links: 2502.13923 Cited by: §V-A. [4] S. Bai et al. (2025) Qwen3-VL Technical Report. Note: arXiv:2511.21631 External Links: 2511.21631 Cited by: §V-A. [5] Z. Chen et al. (2024) Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. Note: arXiv:2412.05271 External Links: 2412.05271 Cited by: §V-A. [6] S. Dharur, P. Tendulkar, D. Batra, D. Parikh, and R. R. Selvaraju (2021) SOrT-ing VQA Models: contrastive gradient learning for improved consistency. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, p. 3103–3111. Cited by: §I-B. [7] M. O. Drago, L. Carlini, P. C. Balyemez, D. Pierantozzi, C. Lena, C. Hassan, D. Stoyanov, E. De Momi, S. Bano, and M. I. Hoque (2026) SurgViVQA: temporally grounded video question answering for surgical scene understanding. International Journal of Computer Assisted Radiology and Surgery. External Links: Document Cited by: §I-B. [8] P. K. Gaihre, A. More, R. Kusumba, M. Bhalerao, S. Sinha, C. Bhatt, G. Thakur, D. K. Yadav, and A. K. Jaiswal (2025) Multimodal AI for gastrointestinal diagnostics: tackling VQA in MEDVQA-GI 2025. Note: arXiv:2507.14544 External Links: 2507.14544 Cited by: §I-A. [9] S. Gautam, M. A. Riegler, and P. Halvorsen (2025) Kvasir-VQA-x1: a multimodal dataset for medical reasoning and robust MedVQA in gastrointestinal endoscopy. Note: arXiv:2506.09958 External Links: 2506.09958 Cited by: §I, §I-A, §V-A. [10] S. Gautam, A. Storås, C. Midoglu, S. A. Hicks, V. Thambawita, P. Halvorsen, and M. A. Riegler (2024) Kvasir-VQA: a text-image pair gi tract dataset. In Proceedings of the First International Workshop on Vision-Language Models for Biomedical Applications, External Links: Document Cited by: §I, §I-A. [11] X. He, Y. Zhang, L. Mou, E. P. Xing, and P. Xie (2020) PathVQA: 30000+ questions for medical visual question answering. Note: arXiv:2003.10286 External Links: 2003.10286 Cited by: §I-A. [12] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, External Links: Link Cited by: §V-A. [13] D. A. Hudson and C. D. Manning (2019) GQA: a new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §I-B. [14] L. Jing, R. Li, Y. Chen, and X. Du (2024) FaithScore: fine-grained evaluations of hallucinations in large vision-language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, p. 5042–5063. Cited by: §I-B. [15] LASA Team, W. Xu, et al. (2025) Lingshu: a generalist foundation model for unified multimodal medical understanding and reasoning. Note: arXiv:2506.07044 External Links: 2506.07044 Cited by: §V-A. [16] J. J. Lau, S. Gayen, A. Ben Abacha, and D. Demner-Fushman (2018) A dataset of clinically generated visual questions and answers about radiology images. Scientific Data 5, p. 180251. External Links: Document Cited by: §I-A. [17] C. Li, C. Wong, S. Zhang, N. Usuyama, H. Liu, J. Yang, T. Naumann, H. Poon, and J. Gao (2023) LLaVA-Med: training a large language-and-vision assistant for biomedicine in one day. In Advances in Neural Information Processing Systems, Vol. 36, p. 28541–28564. Note: Datasets and Benchmarks Track Cited by: §V-A. [18] Y. Li, Y. Du, K. Zhou, J. Wang, X. Zhao, and J. Wen (2023) Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 292–305. External Links: Document Cited by: §I-B. [19] Z. Lin, D. Zhang, Q. Tao, D. Shi, G. Haffari, Q. Wu, M. He, and Z. Ge (2023) Medical visual question answering: a survey. Artificial Intelligence in Medicine 143, p. 102611. External Links: Document Cited by: §I. [20] B. Liu, L. Zhan, L. Xu, L. Ma, Y. Yang, and X. Wu (2021) SLAKE: a semantically-labeled knowledge-enhanced dataset for medical visual question answering. In 2021 IEEE 18th International Symposium on Biomedical Imaging, External Links: Document Cited by: §I-A. [21] R. Liu et al. (2025) EndoBench: a comprehensive evaluation of multi-modal large language models for endoscopy analysis. Note: arXiv:2505.23601 External Links: 2505.23601 Cited by: §I, §I-A. [22] S. Lu et al. (2026) Gastric-X: a multimodal multi-phase benchmark dataset for advancing vision-language models in gastric cancer analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §I-A. [23] P. Manakul, A. Liusie, and M. J. F. Gales (2023) SelfCheckGPT: zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 9004–9017. External Links: Document Cited by: §I-B. [24] S. Min, K. Krishna, X. Lyu, M. Lewis, W. Yih, P. W. Koh, M. Iyyer, L. Zettlemoyer, and H. Hajishirzi (2023) FActScore: fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 12076–12100. External Links: Document Cited by: §I-B. [25] OpenAI (2026-04) Introducing GPT-5.5. Note: https://openai.com/index/introducing-gpt-5-5/ Cited by: §V-A. [26] A. Sellergren et al. (2025) MedGemma technical report. Note: arXiv:2507.05201 External Links: 2507.05201 Cited by: §V-A. [27] R. R. Selvaraju, P. Tendulkar, D. Parikh, E. Horvitz, M. T. Ribeiro, B. Nushi, and E. Kamar (2020) SQuINTing at VQA models: introspecting VQA models with sub-questions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 10003–10011. Cited by: §I-B. [28] S. Tang et al. (2025) EndoCogniAgent: closed-loop agentic reasoning with self-consistency validation for endoscopic diagnosis. Note: arXiv:2508.07292 External Links: 2508.07292 Cited by: §I-B. [29] X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, and D. Zhou (2023) Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations, Cited by: §I-B. [30] xAI (2026) Grok 4.20. Note: xAI Docs. https://docs.x.ai/developers/models/grok-4.20 Cited by: §V-A.