Paper deep dive

VisExMEM: Visual Evidence-Based Explainable Multimodal Evaluation Metric

Subhadeep Roy, Christoph Leiter, Steffen Eger

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 144

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Current image-text evaluation often relies on a global score, which gives limited insight into which semantic details are supported by the image. Explainable metrics provide rationales, but the visual evidence supporting those rationales can still be difficult to verify. We introduce VisExMEM, a reference-free evaluator that decomposes captions into atomic claims, localizes visual evidence, verifies each claim, and averages claim-level support into a global score. Explanations use the same evidence and decisions. We train Qwen3.5-based 4B and 9B models on 100K samples with multi-task SFT, counterfactual refinement, and evidence-aware GDPO. Across eight benchmarks, VisExMEM-9B achieves the highest descriptive composite among open-source evaluators in our comparison, exceeding VQAScore by 11.7 points while remaining competitive with GPT-5.6 Sol. We also introduce AGITA-Bench, containing 600 human-annotated image-text pairs and 8.7K atomic claims for global and claim-level evaluation. For untuned Qwen3.5 and GPT-5.6 Sol controls, decomposed prompting raises the same composite by 3.6-5.1 points relative to holistic prompting. Human evaluation shows that VisExMEM explanations outperform prior explainable metrics and approach GPT-5.6 Sol.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

141,087 characters extracted from source content.

Expand or collapse full text

Preprint VISEXMEM: VISUAL EVIDENCE-BASED EXPLAINABLE MULTIMODAL EVALUATION METRIC Subhadeep Roy, Christoph Leiter, Steffen Eger Natural Language Learning & Generation Lab, University of Technology Nuremberg subhadeep.roy, christoph.leiter, steffen.eger@utn.de ABSTRACT Current image–text evaluation often relies on a global score, which gives limited insight into which semantic details are supported by the image. Explainable met- rics provide rationales, but the visual evidence supporting those rationales can still be difficult to verify. We introduce VISEXMEM, a reference-free evalua- tor that decomposes captions into atomic claims, localizes visual evidence, ver- ifies each claim, and averages claim-level support into a global score. Expla- nations use the same evidence and decisions. We train Qwen3.5-based 4B and 9B models on 100K samples with multi-task SFT, counterfactual refinement, and evidence-aware GDPO. Across eight benchmarks, VISEXMEM-9B achieves the highest descriptive composite among open-source evaluators in our comparison, exceeding VQAScore by 11.7 points while remaining competitive with GPT-5.6 Sol. We also introduce AGITA-BENCH, containing 600 human-annotated image– text pairs and 8.7K atomic claims for global and claim-level evaluation. For un- tuned Qwen3.5 and GPT-5.6 Sol controls, decomposed prompting raises the same composite by 3.6–5.1 points relative to holistic prompting. Human evaluation shows that VISEXMEM explanations outperform prior explainable metrics and approach GPT-5.6 Sol. 1 1INTRODUCTION The rapid progress of multimodal generative models has made reliable image–text evaluation in- creasingly important for benchmarking and model development (Hartwig et al., 2025; Berger et al., 2025; Sarto et al., 2025). Automatic evaluation has progressed from global cross-modal similarity (Hessel et al., 2021) to learned preference models (Kirstain et al., 2023; Xu et al., 2023; Wu et al., 2023), structured VQA-based evaluation (Hu et al., 2023; Cho et al., 2024; Lin et al., 2024), VLM- as-a-judge approaches (Chen et al., 2025; Pu et al., 2025; Li et al., 2026), and structured alignment functions (Chen et al., 2026a; Koyama et al., 2026). However, a high global score can still hide a local semantic error. An image may look broadly correct while getting a count, attribute binding, ac- tion, spatial relation, or subject–object role wrong. Benchmarks such as Winoground, TIFA, GenAI- Bench, MMIG-Bench, and ImagenWorld are designed to test these fine-grained and compositional errors (Thrush et al., 2022; Hu et al., 2023; Li et al., 2024a; Hua et al., 2025; Mahdizadeh Sani et al., 2026). Strong evaluation performance also does not make a metric easy to inspect. VIEScore and FLEUR provide natural-language rationales (Ku et al., 2024; Lee et al., 2024), while recent work adds criterion-level explanations (Kim et al., 2025), multi-criteria aggregation (Oi et al., 2026), and element-level grounding (Shi et al., 2026). A remaining issue is whether the explanation reflects the visual evidence and the decisions that produced the score. A fluent explanation can still sound reasonable even when it is based on the wrong evidence or decision. Similarly, global score corre- lation does not show whether a metric responds to the specific visual or linguistic constraint that is wrong (Ross et al., 2024). We introduce VISEXMEM (Visual Evidence-Based Explainable Multimodal Evaluation Metric), a reference-free metric that decomposes text into atomic claims and localizes visual evidence (Fig- 1 VisExMEM and AGITA-Bench will be released at: https://github.com/NL2G/VisExMEM. 1 arXiv:2610.16441v1 [cs.CV] 12 Oct 2026 Preprint Input Image + Text 1 "A woman and a man stand in a kitchen holding a red mug and a green mug, while a red mug, a blue mug, and a striped mug sit on the counter beside a stainless-steel kettle." 2 Decomposition into Atomic Claims woman man kitchen mug counter kettle red mug green mug red mug stainless-steel kettle blue mug striped mug Object Attribute Action holdingsitstand Relation woman and man holding red and green mugs three mugs on the counter counter mugs beside the kettle woman and man in the kitchen Count three mugs on the counter two mugs being held 3 Visual Evidence Extraction GroundingDINO localizes claim- relevant visual evidence, producing candidate bounding box(es) with confidence scores. Atomic-Level Evaluation and Explanation 4 q i Each atomic claim is verified against localized visual evidence to produce a grounded support score and explanation. Arithmetic Aggregation 5 Atomic Support Scores Arithmetic Mean Final Score q 1, q 2,..., q M Final Output 6 VisExMEM Score 0.89 / 1 High Alignment with Grounded Diagnosis TypeAtomic claimSupport qiEvidenceGrounded explanation Objectwoman0.99C1A woman is clearly visible. Objectman0.99C2A man is clearly visible. ............... Attributered mug0.99C3+C5Red mugs are clearly visible. Attributegreen mug0.04C4The held mug is blue, not green. ............... Actionstand0.98C1+C2Both people are standing. Actionholding0.99C1+C2+C3+C4Both people are holding mugs. Actionsit0.96C5+C6+C7+C9Three mugs sit on the counter. Relationwoman and man holding red and green mugs 0.06C1+C2+C3+C4The second held mug is blue, not green. Relationthree mugs on the counter 0.99C5+C6+C7+C9Three mugs are on the counter. ............... Counttwo mugs being held 0.99C3+C4Exactly two mugs are being held. Countthree mugs on the counter 0.99C5+C6+C7Exactly three mugs are on the counter. Figure 1: Overview of VisExMEM. VisExMEM decomposes captions into atomic claims, localizes visual evidence for each claim, computes claim support scores, and averages them into a global score. Explanations are generated afterward from the same evidence and fixed claim decisions. ure 1). For each claim, the localized evidence is used to compute a Yes/No log-probability margin, which is mapped to a claim support score. The arithmetic mean of these scores gives the global score. Explanations are generated afterward from the same evidence and fixed claim decisions, so they do not determine the score. We train Qwen3.5-based (Qwen Team, 2026) 4B and 9B models in three stages (Figure 2). Multi-task SFT establishes the evaluation pipeline and its structured out- puts. Counterfactual refinement uses minimal semantic changes so the model responds to a single changed constraint while keeping the remaining content fixed. Finally, evidence-aware GDPO (Liu et al., 2026) improves semantic verification and encourages predictions to rely on localized visual evidence. To evaluate both global and claim-level agreement with humans, we introduce AGITA-BENCH (Atomic and Global Image–Text Alignment Benchmark), a 600-case benchmark with human annota- tions covering image-to-text (I2T) and text-to-image (T2I) evaluation. In I2T, captions are evaluated against real images. In T2I, generation prompts are evaluated against generated images. Each case belongs to an easy, medium, or hard compositional setting. Annotators provide an overall image– text consistency rating and claim-level support judgments. This allows us to evaluate both global and claim-level scores. Our contributions are: • Evidence-Grounded Explainable Evaluation: We introduce VISEXMEM, which links atomic claim scores to localized visual evidence and generates explanations from the same evidence and claim decisions. • Progressive Multimodal Verification Training: We develop a three-stage training strategy for learning structured evaluation, sensitivity to minimal semantic changes, and dependence on visual evidence. • AGITA-Bench: We provide a benchmark that evaluates global agreement and individual claim decisions in both I2T and T2I settings across different levels of compositional diffi- culty. 2RELATED WORK Image–text alignment evaluation. Reference-free metrics include embedding-based methods that score an image–text pair as a whole, without decomposing it into smaller semantic units (Hes- sel et al., 2021; Li et al., 2023) and learned human-preference models (Kirstain et al., 2023; Xu et al., 2023; Wu et al., 2023; Zhang et al., 2024). More recent methods break the evaluation into smaller semantic units. TIFA (Hu et al., 2023) and DSG (Cho et al., 2024) decompose prompts into verifiable questions, VQAScore (Lin et al., 2024) uses generative VQA likelihood, and T2I- FineEval (Hosseini et al., 2025) decomposes both text and image. Recent approaches also explore hyperbolic alignment (Chen et al., 2026a) and task-adapted caption scoring (Koyama et al., 2026). VISEXMEM follows this fine-grained setting by decomposing the caption into atomic claims for evaluation. Explainable and evidence-grounded evaluation. FLEUR (Lee et al., 2024) and VIEScore (Ku et al., 2024) pair scores with natural-language rationales, while EXPERT (Kim et al., 2025) and HarmonicEval (Oi et al., 2026) expose structured criteria. Grounding-oriented work includes 2 Preprint GLaMM (Rasheed et al., 2024), Grounding-IQA (Chen et al., 2026b), and multimodal-rationale VQA (Li et al., 2025). Most closely, REVEALER (Shi et al., 2026) performs element-level align- ment evaluation through a grounding–reasoning–scoring trajectory. VISEXMEM instead uses lo- calized visual evidence to score each atomic claim. These claim scores determine the global score, while the explanation is generated afterward from the same evidence and fixed claim decisions. Diagnostic benchmarks and metric meta-evaluation. Benchmarks test different aspects of image–text evaluation.Winoground (Thrush et al., 2022) and EqBen (Wang et al., 2023a) probe compositional sensitivity; TIFA (Hu et al., 2023), GenAI-Bench (Li et al., 2024a), T2I- CompBench++ (Huang et al., 2025), and CountBench (Paiss et al., 2023) assess fine-grained faithfulness, compositionality, and counting, while Pick-a-Pic (Kirstain et al., 2023) provides human preference comparisons.Broader diagnostic resources include EvalMuse-40K (Han et al., 2026), MMIG-Bench (Hua et al., 2025), FineGRAIN (Hayes et al., 2025), and Imagen- World (Mahdizadeh Sani et al., 2026). For metric meta-evaluation, PROTOBIAS (Roy et al., 2026) tests prototypicality bias, T2IScoreScore (Saxon et al., 2024) evaluates ordering and separation over semantic error graphs, and CROC (Leiter et al., 2026) uses pseudo- and human-labeled contrastive robustness checks. AGITA-BENCH complements these benchmarks by pairing a global rating with claim-level support judgments for the same image–text case, allowing overall agreement and indi- vidual claim decisions to be evaluated together. 3VISEXMEM VISEXMEM is a reference-free image–caption evaluation metric that grounds claim-level verifi- cation in localized visual evidence. In this section, we describe its scoring formulation, training procedure, and grounded explanation mechanism. 3.1EVIDENCE-GROUNDED VERIFICATION AND SCORING Given image I and caption c, VISEXMEM decomposes c into M typed atomic claims A(c) = (a i ,t i ) M i=1 , where a i is the claim and t i its type (object, attribute, action, relation, count or nega- tion). GroundingDINO (Liu et al., 2024) localizes claim-relevant regions as visual evidence. As shown in Figure 1, the model evaluates each claim using this evidence with the full image to retain scene context. For each claim, the model assigns mean token log-probabilities s Y i and s N i to yes and no. We define m i = s Y i − s N i and q i = σ(m i ), where σ(x) = 1/(1 + e −x ). Applying sigmoid after subtraction maps the Yes/No margin to [0, 1]; applying it first would compare compressed scores. The global score is S(I,c) = M −1 P M i=1 q i . We retain each claim’s type, support score, localized box(es), grounding confidence, and decision. Explanations are generated afterward from these fixed records and cannot alter the score. 3.2TRAINING We train Qwen3.5-4B and Qwen3.5-9B (Qwen Team, 2026) VISEXMEM models with our novel three-stage training procedure (Figure 2): (1) multi-task SFT directly supervises decomposition, claim verification, evidence selection, and explanation, (2) counterfactual refinement teaches the model to distinguish minimally incorrect semantic edits from supported claims, and (3) evidence- aware GDPO (Liu et al., 2026) optimizes claim verification to depend on localized visual evidence. Training data. Training data come from MS COCO (Lin et al., 2014), Visual Genome (Kr- ishna et al., 2017), VSR (Liu et al., 2023), Flickr30k Entities (Plummer et al., 2015), Open Im- ages (Kuznetsova et al., 2020) with Localized Narratives (Pont-Tuset et al., 2020), DOCCI (Onoe et al., 2024), PixMo-Count (Deitke et al., 2025), CROC syn (Leiter et al., 2026), and the training split of EvalMuse-40K (Han et al., 2026). We convert the text into typed atomic claims, using native annotations and human judgments when available and an offline Qwen3.6-27B teacher for miss- ing decomposition or structure; uncertain outputs are filtered out. GroundingDINO then localizes candidate boxes as visual evidence cues, which are provided as evidence rather than SFT targets. We use boxes instead of pixel masks because they localize the relevant region without revealing its 3 Preprint 1 Multi-task SFT Learn the structured evaluation interface Input "Two dogs sit on a blue sofa with a red ball." Structured supervision Caption decomposition "dogs" (object) "blue sofa" (attribute) "dogs sit" (action) "red ball" (object) ... Visual Verification ClaimYes / No Structured Output verification + explanation (JSON) Next-token supervision Teaches decomposition, verification, and structured output format 2 Counterfactual refinement Learn to reject minimally false edits True Claim Counterfactual edit "Two dogs sit on a blue sofa" m i + > 0m i - < 0 "Three dogs sit on a blue sofa" One semantic change Pairwise + pointwise loss Separates one-constraint edits while preserving margin signs. 3 Evidence-aware GDPO Optimize correctness & causal dependence (a) Semantic-verification (rollout-based) Image + Claim (e.g., "dogs on sofa") Sample G = 8 completions Completion 1YesR = +1 Completion 1Yes R = -1 Completion 1Yes R = +1 Group-normalized advantages + clipped policy loss (b) Causal-evidence (multi-view) Claim: "dogs on sofa" Full image (I) Evidence removed (I \ E i ) Evidence only (I E i ) Causal evidence loss Evidence relevance (푑 푖 rel ) , removal sensitivity (푑 푖 drop ) , sufficiency (푑 푖 suff ) , full-image anchor (푑 푖 anchor ) . SFT replay during counterfactual refinement Random-control removed (I \ R i ) Random only (I R i ) SFT + counterfactual replay during GDPO Figure 2: VisExMEM training pipeline. Multi-task SFT learns the structured evaluation tasks, counterfactual refinement targets semantic changes, and evidence-aware GDPO improves verifica- tion and evidence dependence. Earlier supervision is replayed in later stages. exact silhouette, reducing mask-shape shortcuts. We construct 40K SFT examples, a 20K coun- terfactual source pool, and 40K Stage 3 examples equally split between semantic-verification and causal-evidence supervision (see Appendix Figure 7). Counterfactual data combine CROC syn im- age pairs with text-side minimal pairs that change one verified semantic constraint while keeping the image fixed. Explanations are constructed only after claims, decisions, and evidence are fixed. Full construction, filtering, source composition, and replay schedules are in Appendix A.1. Stage 1: Multi-task SFT. Stage 1 teaches the structured evaluation tasks used by VISEXMEM. We jointly supervise caption decomposition, claim-level verification, evidence selection, and ex- planation generation through next-token supervision. For image I , caption c, and structured target y = (y 1 ,...,y T ), we minimize L SFT =−E T X t=1 logπ θ (y t | y <t ,I,c).(1) where π θ is the trainable model. Candidate boxes are supplied as visual evidence; evidence selection predicts candidate IDs rather than box coordinates. Representative processed records are shown in Appendix Figure 7; exact target formats, task mixtures, and optimization settings are given in Appendix A.2.3. Stage 2: Counterfactual refinement. Stage 2 makes claim verification sensitive to controlled semantic changes. Each supported image–claim pair is paired with a contradicted counterpart. Text- side pairs change one constraint while keeping the image fixed, e.g., “two dogs” to “three dogs,” while image-side pairs provide the corresponding visual contrast. Let m θ (I,a) = ℓ θ (yes| I,a)− ℓ θ (no| I,a),(2) where ℓ θ denotes the decision-token logit under model parameters θ. Training requires a positive margin for the supported example, a negative margin for the counterfactual, and their difference to exceed a target margin. This is supervised margin training, not reinforcement learning. Stage 1 decomposition and verification examples are replayed to retain earlier supervision (Luo et al., 2026); the full loss, filtering rules, and replay settings are given in Appendix A.2.4. Stage 3: Evidence-aware GDPO. Stage 3 improves semantic verification while making predic- tions depend on localized visual evidence. Building on GRPO-style group-relative optimization (Shao et al., 2024), we use an evidence-aware GDPO variant inspired by GDPO (Liu et al., 2026): semantic correctness provides the group-relative policy signal, while evidence dependence is op- timized through a separate causal objective. For semantic input x i , we sample G completions o i,g ∼ π θ old (· | x i ), where g indexes rollouts and π θ old is the rollout policy. For semantic re- ward r i,g , let μ i and σ i be the group mean and standard deviation, and define h i,g,t = (x i ,o i,g,<t ). The normalized advantage and policy ratio are A i,g = r i,g − μ i σ i , ρ i,g,t = π θ (o i,g,t | h i,g,t ) π θ old (o i,g,t | h i,g,t ) .(3) 4 Preprint EasyMed.HardEasyMed.Hard 0 25 50 75 100 % of ratings 3.863.973.793.723.483.00 I2T2I 12345 EasyMed.HardEasyMed.Hard 3.353.673.603.213.062.61 I2T2I 01234 050100 % of atomic claims Reasonable Needs merge Needs split Redundant Not entailed 93.35% 6.24% 0.29% 0.02% 0.10% (a)Global alignment (1–5)(b)Atomic support (0–4)(c)Decomposition quality Figure 3: AGITA-Bench annotation summary. (a) Global-alignment and (b) atomic-support dis- tributions by direction and difficulty; means are shown above bars. (c) Decomposition quality. Here, A i,g compares each rollout with its group, while ρ i,g,t measures the token-level change from the rollout policy to the updated policy. Numerical stabilization and zero-variance handling are given in Appendix A.2.5. Following the clipped GRPO objective, we optimize J sem =E " 1 G G X g=1 1 |o i,g | X t minρ i,g,t A i,g , clip(ρ i,g,t , 1− ε, 1 + ε)A i,g − βd KL i,g,t # ,(4) where ε is the clipping radius, β weights KL regularization, and d KL i,g,t penalizes deviation from the frozen Stage 2 reference. Semantic reward alone does not ensure evidence dependence. For each eligible supported claim, selected evidence E i is paired with a matched random region R i , and s i (V ) denotes the verification margin under image view V . We encourage s i (I)−s i (I i ) > s i (I)−s i (I i ) and s i (I⊙E i ) > s i (I ⊙ R i ), so selected evidence is more necessary and sufficient than its control; I \ E masks region E, while I ⊙ E retains only it. Stage 3 minimizesL Stage3 =−J sem + λ c L causal , where λ c weights the causal loss, using a 7:2:1 GDPO:counterfactual:SFT update mixture. Further details are in Appendix A.2.5; the Verification-GRPO and Causal-GRPO ablations branch from Stage 2 and are described in Appendix A.3. 4AGITA-BENCH To evaluate both global and claim-level agreement with humans, we introduce AGITA-BENCH (Atomic and Global Image–Text Alignment), a 600-pair benchmark with human judgments at both levels. It contains 300 I2T real photographs, with 100 each from ADE20K (Zhou et al., 2017), HICO-DET (Chao et al., 2018), and Objects365 (Shao et al., 2019), and 300 T2I pairs from Dif- fusionDB (181) (Wang et al., 2023b) and MJHQ-30K (119) (Li et al., 2024b). I2T captions cover high, partial, and low alignment, while T2I uses the original generation prompts. We balance 200 easy, 200 medium, and 200 hard cases by semantic complexity, not alignment severity. GPT-5.6 Sol decomposes the text into atomic claims, and fixed rules assign difficulty from claim number and composition, easy cases contain few simple claims, medium cases add relations or attribute/count assignments, and hard cases require more constraints or correct binding of attributes, counts, roles, or spatial relations. This produces 8,735 claims spanning objects, attributes, actions, relations (in- cluding spatial relations), counts, and negation. Representative examples are shown in Figure 4; AGITA-Bench is excluded from training, calibration, and model selection. Annotators rate atomic support from 0 (contradicted/not supported) to 4 (fully supported) and global alignment from 1 (not aligned) to 5 (fully aligned), with cannot judge excluded from atomic statis- tics. Decomposition quality is rated separately; reasonable as written means a clear, non-redundant atomic claim requiring neither splitting nor merging, independent of image support. Mean global alignment is 3.64/5 and atomic support 3.35/4, while 93.35% of claims are judged reasonable and 2.07% receive cannot judge. Figure 3 shows that I2T ratings remain relatively high across com- plexity levels, whereas T2I shifts toward lower global and atomic ratings as complexity increases. All seven annotators independently rate a shared set of 30 pairs (460 unique claims), giving mean pairwise quadratic-weighted Cohen’s κ of .602 for global alignment and .564 for atomic support; the remaining 570 pairs are rated once. Full annotation instructions, agreement results, annotator details, and the decomposition prompt are given in Appendix A.6. 5 Preprint Easy· I2T InputEvidence Caption. A baseball player swings a bat while wearing neither a helmet nor a shin guard. Global H/V: 0.75 / 79 Atomic H/V + Explanation C1 baseball player (1.00/0.98): person in baseball uniform and batting gear. C2 bat (1.00/0.96): bat clearly visible in the player’s hands. C3 swings (1.00/0.97): body and bat position depict a swing. C4 baseball player swings a bat (1.00/0.95): player is visibly holding and swinging the bat. C5 baseball player wearing neither a helmet nor a shin guard (0.00/0.10): both a helmet and shin guard are visible. Medium· T2I InputEvidence Prompt. two swedish girls, 9 and 11 years old, holding their gigantic pet male polar bear in tight leash while walking the streets of stockholm, summertime, sunny and warm life style stock image popular no text prompt trend. pinterest contest winner Global H/V: 0.75 / 0.72 Atomic H/V + Explanation C1 girls (1.00/0.89): a foreground girl and another female- presenting pedestrian are visible. C2 two girls (1.00/0.63): two female-presenting people are visible, but only one is clearly a girl. C3 polar bear (1.00/0.98): a large white bear is clearly visi- ble. C4 gigantic polar bear (1.00/0.91): its scale relative to the girl supports “gigantic.” C5 leash (0.00/0.67): straps are visible, but no attachment to the bear is shown. C6 tight leash (0.00/0.07): the visible strap is slack rather than tight. C7 streets (1.00/0.95): the figures are on a paved city street. C8 holding (1.00/0.93): the girl’s hand visibly holds the strap-like object. C9 walking (1.00/0.92): leg positions depict forward walk- ing. C10 girls holding their polar bear in a tight leash (0.46/0.10): only one girl holds slack straps, with no tight attachment to the bear. C11 girls walking the streets (1.00/0.62): two female- presenting figures walk in the street, though age is unclear. C12 sunny (1.00/0.91): direct light and pronounced shadows indicate sunny conditions. Hard· I2T InputEvidence Caption. A team in predominantly white jerseys with black and red trim stands in a diagonal line stretching from the upper left toward the lower right, with mostly dark sticks angled to the right; the padded goaltender occupies the middle of the line as spectators watch from behind the advertising boards. Global H/V: 0.75 / 0.81 Atomic H/V + Explanation C1 team (1.00/0.98): uniforms and formation indicate a team. C2 jerseys (1.00/0.97): numbered jerseys are visible. C3 predominantly white jerseys (1.00/0.95): jerseys are pre- dominantly white. C4 jerseys with black and red trim (1.00/0.89): black and red trim is visible. C5 sticks (1.00/0.93): multiple hockey sticks are visible. C6 mostly dark sticks (1.00/0.83): most visible sticks are dark. C7 goaltender (1.00/0.97): mask and large pads identify a goaltender. C8 padded goaltender (1.00/0.94): large protective pads are visible. C9 spectators (1.00/0.90): spectators are visible above the boards. C10 advertising boards (1.00/0.92): text and logos appear on the boards. C11 stands (1.00/0.96): players are standing upright. C12 watch (1.00/0.62): spectators face the ice, though indi- vidual gaze is unclear. C13 team stands in a diagonal line from upper left to lower right (0.50/0.18): the visible diagonal has the opposite ori- entation. C14 sticks angled to the right (1.00/0.95): most visible sticks project to the right. C15 goaltender in the middle of the line (0.75/0.11): the goaltender is near an end rather than the middle. C16 spectators behind the advertising boards (1.00/0.92): spectators are visibly behind the boards. Figure 4: Qualitative AGITA-Bench examples with VISEXMEM-9B. Easy–Hard cases show the input, localized evidence, caption/prompt, and Human/VISEXMEM-9B global and atomic scores (H/V); human ratings are normalized to [0, 1], with short claim-level explanations. 5EXPERIMENTS Benchmarks and Protocols. We evaluate ten benchmarks, comprising eight established bench- marks, T2IScoreScore (TS2), and AGITA-Bench.We report Group scores on Winoground (WG) (Thrush et al., 2022) and EqBen (Wang et al., 2023a), pairwise accuracy (PAcc) on TIFA160 (Hu et al., 2023) and GenAI-Bench (Li et al., 2024a), and preference accuracy (Pref. Acc.) on Pick-a-Pic (Kirstain et al., 2023). For CROC hum (Leiter et al., 2026), we use the mean over its forward and inverse text-based and image-based evaluations. We report SRCC on the T2I- CompBench++ (T2I-CB++) human subset (Huang et al., 2025) and top-1 counting accuracy on CountBench (Paiss et al., 2023). TS2 (Saxon et al., 2024) organizes images into semantic error graphs (SEGs). We report ordering rank m and separation sep m over all SEGs and the Synth, Nat, and Real subsets. Synth uses generated images for predefined error graphs, Nat uses natural images in constructed graphs, and Real uses naturally occurring T2I errors. AGITA-Bench (Sec. 4) reports Spearman’s ρ for global and atomic human agreement across Easy, Medium, and Hard. Full proto- cols are in Appendix A.5. We also evaluate OOD generalization on VisPlotBench (Ni et al., 2026), CulturalFrames (Nayak et al., 2025), and ProtoBias (Roy et al., 2026) in Appendix A.4.1. Evaluation Metrics. We compare VISEXMEM-4B and VISEXMEM-9B with embedding- and matching-based CLIPScore (Hessel et al., 2021), BLIPv2Score (Li et al., 2023), and ALIGN- Score (Saxon et al., 2024), preference-trained PickScore (Kirstain et al., 2023), ImageReward (Xu et al., 2023), HPSv2 (Wu et al., 2023), and MPS (Zhang et al., 2024), general-purpose or explain- able evaluators GPT4V-Eval (Zhang et al., 2023), VIEScore (Ku et al., 2024), FLEUR, and EX- PERT, and structured and VQA-based VQ 2 (Yarom et al., 2023), Davidsonian Scene Graph (Cho et al., 2024), our DSG + GPT-5.6 Sol setup, and VQAScore (Lin et al., 2024). We also evaluate GroundingDINO (Liu et al., 2024) directly as a grounding-based metric. Controlled VLM-as-a- Judge experiments use GPT-5.6 Sol and untuned Qwen3.5 (Qwen Team, 2026) with Holistic (Hol.) prompting on the full image-caption pair or Decomposed (Dec.) prompting with a decompose, ver- ify, and aggregate procedure, without VISEXMEM-specific training, learned evidence conditioning, or external grounding cues. On AGITA-Bench, global agreement uses native scores, while atomic agreement is reported only for methods directly scoring the fixed benchmark claims. Prompts are 6 Preprint Table 1: Main results. Comparison of prior image–text alignment metrics, controlled VLM-as-a- Judge baselines, and VISEXMEM across eight established benchmarks. Higher is better. Bold, underlined , and italicized values mark the best, second-best, and third-best results. MethodWGEqBen TIFA160 Pick-a-Pic GenAI CROC hum T2I-CB++ CountBenchAvg. Group GroupPAccPref. Acc.PAccScoreSRCCAcc. CLIPScore7.825.054.176.052.269.828.530.743.0 BLIPv2Score17.535.057.580.055.176.137.047.350.7 ALIGNScore8.328.659.680.053.471.836.134.546.5 PickScore6.823.659.470.057.774.730.356.547.4 ImageReward12.826.467.375.058.775.450.633.850.0 HPSv24.017.155.269.049.872.342.466.647.0 MPS1.810.059.069.056.167.024.233.840.1 GPT4V-Eval36.335.064.074.061.4–54.1 † VIEScore34.532.963.978.060.386.554.947.357.3 EXPERT29.327.168.083.064.683.054.243.956.6 FLEUR28.535.767.282.063.581.752.241.056.5 VQ 2 10.020.048.773.051.978.148.651.147.7 Davidsonian15.520.054.370.054.677.760.637.948.8 DSG + GPT-5.6 Sol49.040.666.881.061.472.149.851.359.0 VQAScore46.047.971.284.063.386.153.231.760.4 GroundingDINO9.312.958.257.051.156.818.520.635.6 Qwen3.5-4B (Hol.)28.244.064.372.562.278.156.736.655.3 Qwen3.5-4B (Dec.)49.047.166.080.062.576.557.244.260.3 ∆ Dec.–Hol.(+20.8)(+3.1)(+1.7)(+7.5)(+0.3)(-1.6)(+0.5)(+7.6)(+5.0) Qwen3.5-9B (Hol.)40.435.750.174.250.376.255.825.551.0 Qwen3.5-9B (Dec.)41.245.762.281.054.674.454.835.056.1 ∆ Dec.–Hol.(+0.8)(+10.0)(+12.1)(+6.8)(+4.3)(-1.8)(-1.0)(+9.5)(+5.1) GPT-5.6 Sol (Hol.)58.570.171.882.466.991.063.887.674.0 GPT-5.6 Sol (Dec.)80.070.074.386.070.594.061.185.177.6 ∆ Dec.–Hol.(+21.5)(-0.1)(+2.5)(+3.6)(+3.6)(+3.0)(-2.7)(-2.5)(+3.6) VisExMEM-4B (Ours)65.659.170.280.266.387.057.851.967.3 vs. VQAScore(+19.6)(+11.2)(-1.0)(-3.8)(+3.0)(+0.9)(+4.6)(+20.2)(+6.8) VisExMEM-9B (Ours)68.970.173.185.669.491.757.760.772.2 vs. VQAScore(+22.9)(+22.2)(+1.9)(+1.6)(+6.1)(+5.6)(+4.5)(+29.0)(+11.7) † GPT4V-Eval’s Avg. uses its five available benchmark results; all other Avg. values use all eight. T2I-CB++ reports 100ρ (theoretical range [−100, 100]). Avg. is a descriptive composite of different benchmark statistics. Blue rows show signed differences relative to VQAScore in the displayed units; T2I-CB++ differences are 100∆ρ, not percentage points. given in Appendix A.5.1. The omission of REVEALER (Shi et al., 2026) and T2I-FineEval (Hos- seini et al., 2025) is discussed in Appendix A.5.2. Explanation Evaluation. We compare EXPERT, VIEScore, GPT-5.6 Sol (Dec.), and VI- SEXMEM-9B. Annotators rate each explanation from 1 to 5 for diagnostic correctness, visual grounding, clarity, and diagnostic usefulness. Correctness measures whether the explanation identi- fies the relevant matches or errors, grounding whether its statements are supported by the image and caption, clarity how easy it is to understand, and usefulness how well it explains why the image and caption agree or disagree. Annotators also flag unsupported factual content. For systems with lo- calized evidence, they rate localization accuracy and sufficiency from 1 to 5. Localization measures whether the highlighted region identifies the relevant visual evidence, while sufficiency measures whether that evidence is enough to judge the claim. Full instructions and agreement statistics are given in Appendix A.7. 6RESULTS & ANALYSIS Comparison with Existing Metrics. Table 1 compares performance across the eight established benchmarks. VISEXMEM-9B has the highest composite among open-source evaluators in our comparison. It exceeds VQAScore on all eight, with an 11.7-point composite difference. VI- SEXMEM-4B improves the composite by 6.8 points over VQAScore and exceeds all prior open- source baselines. For 9B, the largest gains are on Winoground (+22.9), EqBen (+22.2), and Count- Bench (+29.0), with a further +6.1 points on GenAI-Bench. VISEXMEM-9B also exceeds GPT-5.6 Sol (Hol.) on five benchmarks and matches it on EqBen. Its eight-benchmark composite is 72.2, compared with 74.0 for GPT-5.6 Sol (Hol.) and 77.6 for GPT-5.6 Sol (Dec.). 7 Preprint 0.000.250.500.751.00 Separation Score (sep m ) 0.0 0.2 0.4 0.6 0.8 Ordering Score (rank m ) EXPERT VisExMEM-4B VisExMEM-9B FLEUR ALIGNScore-TS2 CLIPScore TIFA + InstructBLIP VQAScore DSG + InstructBLIP VIEScore (a) All SEGs 0.000.250.500.751.00 Separation Score (sep m ) EXPERT VisExMEM-4B FLEUR ALIGNScore-TS2 VisExMEM-9B CLIPScore VQAScore TIFA + InstructBLIP DSG + InstructBLIP VIEScore (b) Synth 0.000.250.500.751.00 Separation Score (sep m ) EXPERT VisExMEM-4B VisExMEM-9B FLEUR ALIGNScore-TS2 CLIPScore TIFA + InstructBLIP VQAScore DSG + InstructBLIP VIEScore (c) Nat 0.000.250.500.751.00 Separation Score (sep m ) VisExMEM-9B CLIPScore EXPERT VisExMEM-4B FLEUR ALIGNScore-TS2 TIFA + InstructBLIP VQAScore DSG + InstructBLIP VIEScore (d) Real Figure 5: T2IScoreScore performance. Ordering (rank m ) versus separation (sep m ) for (a) all SEGs, (b) Synth, (c) Nat, and (d) Real. Higher is better on both axes. Table 2:Human Agreement on AGITA- Bench.Global and atomic agreement across Easy, Medium, and Hard difficulty tiers. Bold, underline , and italics denote first, second, and third best, respectively. Table 3: Training-Stage Ablation.Per- formance of VISEXMEM across successive training stages at 4B and 9B scales. Blue rows show gains relative to the indicated reference stage; bold denotes the best result within each model scale. GlobalAtomic MethodEasy Med. Hard Avg. Easy Med. Hard Avg. CLIPScore19.6 -3.6-8.62.55.4-4.8-0.20.1 BLIPv2Score21.13.1-0.87.83.32.14.53.3 AlignScore24.53.26.2 11.3 5.8-0.50.51.9 PickScore33.5 29.1 34.7 32.4 11.1 -3.2-2.02.0 ImageReward38.8 21.9 16.8 25.8– HPSv217.5 14.6 13.5 15.2 14.85.88.09.5 EXPERT50.6 33.8 45.5 43.3– FLEUR52.2 31.5 43.7 42.5– VIEScore51.2 31.2 42.6 41.7– VQAScore54.6 31.7 43.0 43.1 52.7 32.3 41.1 42.0 Qwen3.5-4B (Hol.) 48.5 39.3 40.3 42.7– Qwen3.5-4B (Dec.) 50.5 37.1 44.1 43.9 41.2 31.7 39.1 37.3 Qwen3.5-9B (Hol.) 53.7 42.2 40.4 45.4– Qwen3.5-9B (Dec.) 59.0 39.5 41.2 46.6 40.6 30.8 38.2 36.5 GPT-5.6 Sol (Hol.) 72.1 54.660.662.4– GPT-5.6 Sol (Dec.) 76.9 60.7 62.5 66.7 70.1 55.6 68.3 64.7 VisExMEM-4B56.843.150.650.246.634.143.241.3 VisExMEM-9B63.345.255.654.756.233.045.544.9 Model VariantWGEqBTIFAPickGenAI 4B Base (Hol.)28.244.064.372.562.2 SFT50.848.956.870.054.1 vs. Base(+22.6)(+4.9)(-7.5)(-2.5)(-8.1) + CF61.656.264.977.264.9 vs. SFT(+10.8)(+7.3)(+8.1)(+7.2)(+10.8) + Verif.-GRPO62.557.966.473.964.3 + Causal-GRPO64.158.667.672.865.2 + GDPO65.659.170.280.266.3 vs. CF(+4.0)(+2.9)(+5.3)(+3.0)(+1.4) 9B Base (Hol.)40.435.750.174.250.3 SFT61.958.959.861.857.2 vs. Base(+21.5)(+23.2)(+9.7)(-12.4)(+6.9) + CF66.364.968.977.567.2 vs. SFT(+4.4)(+6.0)(+9.1)(+15.7)(+10.0) + Verif.-GRPO67.467.271.578.067.9 + Causal-GRPO68.566.771.079.167.1 + GDPO68.970.173.185.669.4 vs. CF(+2.6)(+5.2)(+4.2)(+8.1)(+2.2) Structured Evaluation Helps Across Backbones. Replacing holistic judgment with a VI- SEXMEM-style procedure that decomposes, verifies, and aggregates improves the average perfor- mance of Qwen3.5-4B, Qwen3.5-9B, and GPT-5.6 Sol by +5.0, +5.1, and +3.6 points, respectively, with GPT-5.6 Sol increasing from 74.0 (Hol.) to 77.6 (Dec.). These controlled comparisons show that structured inference improves average performance across all three VLM backbones, although gains vary by benchmark. Together with dedicated VISEXMEM training, the 4B and 9B models achieve the strongest open-source results in our comparison. T2IScoreScore Performance. Figure 5 shows that both VISEXMEM variants remain in the high-ordering, high-separation region across All, Synth, Nat, and Real. VISEXMEM-4B performs strongly on All, Synth, and Nat, while VISEXMEM-9B shows its clearest advantage on Real, where it achieves the highest ordering score while maintaining strong separation. This indicates strong per- formance on naturally occurring T2I errors as well as the constructed TS2 settings. Fine-Grained Human Alignment on AGITA-Bench. Table 2 reports agreement with human judgments at both the global and atomic levels. VISEXMEM-9B achieves the strongest open-source global average at 54.7 and improves over the corresponding untuned Qwen3.5 controls across both evaluation levels. Among methods that directly score the fixed benchmark claims, it reaches an atomic average of 44.9, compared with 64.7 for GPT-5.6 Sol (Dec.). The higher correlations on Hard than Medium should not be read as Hard being easier. In T2I, difficulty reflects semantic com- plexity, and Hard prompts contain more claims (15.72 vs. 8.58) and lower human alignment (2.99 vs. 3.49), which changes the within-tier score distribution. This pattern is not observed in I2T, where Medium and Hard have comparable claim counts. 8 Preprint 12345 Human Rating (1–5) Correctness Grounding Clarity Usefulness (a) Explanation Quality 0255075100 Unsupported Content (%) ↓ VIEScore-official EXPERT GPT-5.6 Sol (Dec.) VisExMEM-9B 83.7% 27.5% 8.1% 10.3% (b) Unsupported Content 4.04.24.44.64.85.0 Human Rating (1–5) Localization Sufficiency (c) Evidence Quality VIEScore-officialEXPERTGPT-5.6 Sol (Dec.)VisExMEM-9B Figure 6: Human evaluation of explanation quality. (a) Correctness, visual grounding, clarity, and diagnostic usefulness. (b) Unsupported-content rate. (c) Localization accuracy and evidence sufficiency. Error bars show 95% confidence intervals. Explanation Quality Analysis. Figure 6 shows that VISEXMEM-9B is close to GPT-5.6 Sol on explanation quality, reaching 4.75 diagnostic usefulness with 10.3% unsupported content, compared with 4.85 and 8.1% for GPT-5.6 Sol. It exceeds EXPERT and VIEScore in usefulness (3.65 and 2.13) while producing much less unsupported content (27.5% and 83.7%). Its localized evidence also receives high ratings for localization accuracy and sufficiency, showing that the highlighted regions are relevant and provide useful evidence for judging the associated claims. Training-Stage Ablation. Table 3 shows that SFT improves the five-benchmark average over Base by +1.9/+9.8 points for 4B/9B, although the gains vary across benchmarks. Counterfactual refine- ment then gives a larger and more consistent gain over SFT of +8.8/+9.0 points. From the coun- terfactual checkpoint, Verification-GRPO and Causal-GRPO provide smaller, less consistent gains, while joint GDPO performs best on all five benchmarks at both scales and adds a further +3.3/+4.5 points on average. This supports the progression from structured supervision to counterfactual re- finement and joint semantic and evidence-aware optimization. Efficiency, Cost, and Reproducibility. Although GPT-5.6 Sol benefits from the decomposed evaluation procedure, evaluating the eight benchmarks and AGITA-Bench cost approximately $800 (∼$150 holistic and ∼$650 decomposed). With vLLM-based (Kwon et al., 2023) inference, VI- SEXMEM-9B requires 8.9 s per image-text pair including explanation generation, compared with about 30 s for GPT-5.6 Sol. At 9B parameters, VISEXMEM-9B provides a compact, locally deploy- able alternative to proprietary frontier judges. Its fixed checkpoint, deterministic aggregation, and explicit atomic decisions and visual evidence support interpretability and reproducibility without reliance on a proprietary API. 7CONCLUSION, LIMITATIONS, AND FUTURE WORK We introduced VISEXMEM, an evidence-grounded, reference-free image–text evaluator that de- rives a global score from localized atomic verification, together with AGITA-BENCH for evaluating global and fine-grained human agreement. Across established benchmarks, controlled VLM-as- a-Judge experiments, and human evaluation, structured decomposed prompting improves average performance across the tested backbones, while task-specific training further strengthens sensitiv- ity to semantic errors and produces grounded explanations approaching those of proprietary judges. Overall, our results show that compact, open evaluators can combine strong alignment judgment with interpretable evidence and reproducible local inference. VISEXMEM remains limited by errors in fine-grained verification and evidence localization, while its multi-stage inference is still more expensive than lightweight embedding-based metrics. Our prompting experiments also do not exhaust alternatives such as self- or other-feedback, and AGITA- Bench focuses on compositional semantic alignment rather than perceptual, aesthetic, or domain- specific quality. Future work should improve binding-sensitive verification and grounding, reduce inference cost, explore stronger verification strategies, and extend atomic evaluation to broader do- mains and modalities. More broadly, these directions could support increasingly capable, open, and auditable evaluators that reduce reliance on proprietary VLM judges. 9 Preprint AI USE STATEMENT We used large language models for literature retrieval and discovery, improving the clarity and pre- sentation of the manuscript, and providing feedback during research development. We also used Claude Code for implementation, debugging, and routine engineering tasks. Model-based compo- nents used in the research, including the offline teacher for training-data construction and model- assisted AGITA-Bench decomposition, are described in the paper. All research questions, method- ological choices, experimental design, training objectives, evaluation protocols, analyses, and con- clusions were determined by the authors. AI-generated suggestions, code, and text were reviewed, tested, and revised before use. We take full responsibility for the work presented in this paper. ETHICS STATEMENT VISEXMEM is designed to evaluate semantic alignment between images and text. Its scope does not extend to image quality, social acceptability, or the correctness of culturally or demographically sensitive representations. Like other learned multimodal evaluators, its behavior can reflect biases and representation gaps in the backbone models, grounding model, teacher models, and training data. This is especially relevant for content involving people, demographic attributes, or cultur- ally specific concepts. Our out-of-distribution experiments on CulturalFrames and ProtoBias there- fore characterize metric behavior under distribution shifts, with conclusions limited to the evaluated benchmarks. Localized visual evidence and natural-language explanations make predictions easier to inspect, but they do not guarantee correctness. Errors can arise during decomposition, grounding, or claim verification, so consequential uses should include independent human review and other sources of evidence. Human annotation is used in AGITA-Bench and in the explanation and evidence evaluation. These tasks focus on image-text alignment, decomposition quality, explanation quality, and visual evi- dence. In the explanation study, system identities are hidden and outputs are shown under random- ized labels to reduce presentation bias. Annotator demographics are reported only in aggregate. Our experiments rely mainly on existing public research datasets, whose collection procedures, licensing conditions, geographic and cultural coverage, and representation of people and concepts can affect the resulting evaluation. We therefore interpret the reported results within the scope of the datasets and benchmarks used in this study. REFERENCES Uri Berger, Gabriel Stanovsky, Omri Abend, and Lea Frermann. Surveying the landscape of image captioning evaluation: A comprehensive taxonomy, trends, and metrics analysis. Transactions of the Association for Computational Linguistics, 13:1597–1644, 2025. doi: 10.1162/tacl.a.52. URL https://aclanthology.org/2025.tacl-1.73/. Yu-Wei Chao, Yunfan Liu, Xieyang Liu, Huayi Zeng, and Jia Deng. Learning to detect human-object interactions. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), p. 381–389. IEEE, 2018. doi: 10.1109/WACV.2018.00048. Wenzhi Chen, Bo Hu, Leida Li, Lihuo He, Wen Lu, and Xinbo Gao.Hyperalign: Hyper- bolic entailment cones for adaptive text-to-image alignment assessment, 2026a. URL https: //arxiv.org/abs/2601.04614. Zhaorun Chen, Zichen Wen, Yichao Du, Yiyang Zhou, Chenhang Cui, Siwei Han, Jen Weng, Chaoqi Wang, Zhengwei Tong, Leria Huang, Canyu Chen, Haoqin Tu, Qinghao Ye, Zhi- hong Zhu, Yuqing Zhang, Jiawei Zhou, Zhuokai Zhao, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao.Mj-bench: Is your multimodal reward model really a good judge for text-to-image generation?In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, volume 38, p. 69174–69228. Curran Associates, Inc., 2025. doi: 10.52202/085713-2081. URL https://proceedings.neurips.c/paper_files/paper/2025/ hash/59d2eaa5842fa641f9b8e4c7f0f6e-Abstract-Datasets_and_ Benchmarks_Track.html. 10 Preprint Zheng Chen, Xun Zhang, Wenbo Li, Renjing Pei, Fenglong Song, Xiongkuo Min, Xiaohong Liu, Xin Yuan, Yong Guo, and Yulun Zhang. Grounding-iqa: Grounding multimodal language model for image quality assessment. In C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust (eds.), International Conference on Learning Representations, volume 2026, p. 39839– 39851, 2026b. URL https://proceedings.iclr.c/paper_files/paper/2026/ file/42f23b6ed414b846a46c8db43509cb92-Paper-Conference.pdf. Jaemin Cho, Yushi Hu, Jason Baldridge, Roopal Garg, Peter Anderson, Ranjay Krishna, Mohit Bansal, Jordi Pont-Tuset, and Su Wang.Davidsonian scene graph:Improv- ing reliability in fine-grained evaluation for text-to-image generation.In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (eds.), International Conference on Learning Representations,volume 2024,p. 15625–15645,2024. URL https://proceedings.iclr.c/paper_files/paper/2024/file/ 446905abfec9db8856f7f7465bbd5552-Paper-Conference.pdf. Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Moham- madreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison- Burch, Andrew Head, Rose Hendrix, Favyen Bastani, Eli VanderBilt, Nathan Lambert, Yvonne Chou, Arnavi Chheda, Jenna Sparks, Sam Skjonsberg, Michael Schmitz, Aaron Sarnat, Byron Bischoff, Pete Walsh, Chris Newell, Piper Wolters, Tanmay Gupta, Kuo-Hao Zeng, Jon Bor- chardt, Dirk Groeneveld, Crystal Nam, Sophie Lebrecht, Caitlin Wittlif, Carissa Schoenick, Oscar Michel, Ranjay Krishna, Luca Weihs, Noah A. Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, and Aniruddha Kembhavi. Molmo and pixmo: Open weights and open data for state-of- the-art vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 91–104, June 2025. Shuhao Han, Haotian Fan, Jiachen Fu, Liang Li, Tao Li, Junhui Cui, Yunqiu Wang, Yang Tai, Jing- wei Sun, Chun-Le Guo, and Chongyi Li. Evalmuse-40k: A fine-grained benchmark with com- prehensive human annotations for text-to-image generation model alignment evaluation. Pro- ceedings of the AAAI Conference on Artificial Intelligence, 40(6):4583–4591, 2026. doi: 10. 1609/aaai.v40i6.42458. URL https://ojs.aaai.org/index.php/AAAI/article/ view/42458. Sebastian Hartwig, Dominik Engel, Leon Sick, Hannah Kniesel, Tristan Payer, Poonam Poonam, Michael Glöckler, Alex Bäuerle, and Timo Ropinski. A survey on quality metrics for text-to- image generation. IEEE Transactions on Visualization and Computer Graphics, 31(10):9464– 9483, 2025. doi: 10.1109/TVCG.2025.3585077. Kevin Hayes, Micah Goldblum, Vikash Sehwag, Gowthami Somepalli, Ashwinee Panda, and Tom Goldstein.Finegrain: Evaluating failure modes of text-to-image models with vi- sion language model judges.In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, vol- ume 38, p. 132119–132143. Curran Associates, Inc., 2025.doi: 10.52202/085713-3977. URL https://proceedings.neurips.c/paper_files/paper/2025/ hash/accaab47784b91d9d381868eb638be0a-Abstract-Datasets_and_ Benchmarks_Track.html. Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi.CLIPScore: A reference-free evaluation metric for image captioning.In Marie-Francine Moens, Xuan- jing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Processing, p. 7514–7528, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguis- tics. doi: 10.18653/v1/2021.emnlp-main.595. URL https://aclanthology.org/2021. emnlp-main.595/. Seyed Mohammad Hadi Hosseini, Amir Mohammad Izadi, Ali Abdollahi, Armin Saghafian, and Mahdieh Soleymani Baghshah. T2i-fineeval: Fine-grained compositional metric for text-to-image evaluation, 2025. URL https://arxiv.org/abs/2503.11481. 11 Preprint Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022. URL https://openreview.net/forum? id=nZeVKeeFYf9. Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A. Smith.Tifa:Accurate and interpretable text-to-image faithfulness eval- uation with question answering.In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV), p. 20406–20417, October 2023.URL https: //openaccess.thecvf.com/content/ICCV2023/html/Hu_TIFA_Accurate_ and_Interpretable_Text-to-Image_Faithfulness_Evaluation_with_ Question_Answering_ICCV_2023_paper.html. Hang Hua, Ziyun Zeng, Yizhi Song, Yunlong Tang, Liu He, Daniel G. Aliaga, Wei Xiong, and Jiebo Luo. Mmig-bench: Towards comprehensive and explainable evaluation of multi-modal image generation models.In Advances in Neural Information Processing Systems, vol- ume 38, p. 159706–159743. Curran Associates, Inc., 2025.doi: 10.52202/085713-4818. URL https://proceedings.neurips.c/paper_files/paper/2025/ hash/d3f8714b9f9727b43d64019f2f4043d2-Abstract-Datasets_and_ Benchmarks_Track.html. Kaiyi Huang, Chengqi Duan, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu.T2i- compbench++: An enhanced and comprehensive benchmark for compositional text-to-image generation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5):3563–3579, 2025. doi: 10.1109/TPAMI.2025.3531907. URL https://doi.org/10.1109/TPAMI. 2025.3531907. Kung-Hsiang Huang, Mingyang Zhou, Hou Pong Chan, Yi Fung, Zhenhailong Wang, Lingyu Zhang, Shih-Fu Chang, and Heng Ji. Do LVLMs understand charts? analyzing and correct- ing factual errors in chart captioning. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, p. 730–749, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024. findings-acl.41. URL https://aclanthology.org/2024.findings-acl.41/. Hyunjong Kim, Sangyeop Kim, Jongheon Jeong, Yeongjae Cho, and Sungzoon Cho. EXPERT: An explainable image captioning evaluation metric with structured explanations.In Wanx- iang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Find- ings of the Association for Computational Linguistics: ACL 2025, p. 26642–26657, Vi- enna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-256- 5. doi: 10.18653/v1/2025.findings-acl.1367. URL https://aclanthology.org/2025. findings-acl.1367/. Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to-image generation. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Pro- cessing Systems, volume 36, p. 36652–36663. Curran Associates, Inc., 2023. doi: 10.52202/ 075280-1594.URL https://proceedings.neurips.c/paper_files/paper/ 2023/file/73aacd8b3b05b4b503d58310b523553c-Paper-Conference.pdf. Shuitsu Koyama, Kazuki Matsuda, Yuiga Wada, Shinnosuke Hirano, Daichi Yashima, and Komei Sugiura. Rigel: Self-distilled score adaptation for image and video captioning evaluation, 2026. URL https://arxiv.org/abs/2606.29997. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. Vi- sual genome: Connecting language and vision using crowdsourced dense image annotations. In- ternational Journal of Computer Vision, 123(1):32–73, 2017. doi: 10.1007/s11263-016-0981-7. URL https://doi.org/10.1007/s11263-016-0981-7. Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen. VIEScore: Towards explain- able metrics for conditional image synthesis evaluation. In Lun-Wei Ku, Andre Martins, and 12 Preprint Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), p. 12268–12290, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.663. URL https://aclanthology.org/2024.acl-long.663/. Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, Tom Duerig, and Vittorio Ferrari. The open images dataset v4: Unified image classification, object detection, and visual relation- ship detection at scale. International Journal of Computer Vision, 128(7):1956–1981, Mar 2020. ISSN 1573-1405. doi: 10.1007/s11263-020-01316-z. URL https://doi.org/10.1007/ s11263-020-01316-z. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica.Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Sys- tems Principles, SOSP ’23, p. 611–626, New York, NY, USA, 2023. Association for Com- puting Machinery. ISBN 9798400702297. doi: 10.1145/3600006.3613165. URL https: //doi.org/10.1145/3600006.3613165. Yebin Lee, Imseong Park, and Myungjoo Kang. FLEUR: An explainable reference-free evaluation metric for image captioning using a large multimodal model. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), p. 3732–3746, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.205. URL https://aclanthology.org/2024.acl-long.205/. Christoph Leiter, Yuki M. Asano, Margret Keuper, and Steffen Eger. CROC: Evaluating and training T2I metrics with pseudo- and human-labeled contrastive robustness checks. Transactions of the Association for Computational Linguistics, 14:1119–1139, 2026. doi: 10.1162/tacl.a.699. URL https://aclanthology.org/2026.tacl-1.50/. Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Xide Xia, Pengchuan Zhang, Graham Neubig, and Deva Ramanan.Evaluating and improving composi- tional text-to-visual generation.In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) Workshops, p. 5290–5301, June 2024a. URL https://openaccess.thecvf.com/content/CVPR2024W/EvGenFM/ html/Li_Evaluating_and_Improving_Compositional_Text-to-Visual_ Generation_CVPRW_2024_paper.html. Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi. Playground v2.5: Three insights towards enhancing aesthetic quality in text-to-image generation, 2024b. URL https://arxiv.org/abs/2402.17245. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Pro- ceedings of Machine Learning Research, p. 19730–19742. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/li23q.html. Kun Li, George Vosselman, and Michael Ying Yang.Multimodal rationales for explain- able visual question answering.In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) Workshops, p. 191–201, June 2025. URL https://openaccess.thecvf.com/content/CVPR2025W/MULA2025/ html/Li_Multimodal_Rationales_for_Explainable_Visual_Question_ Answering_CVPRW_2025_paper.html. Zhikai Li, Jiatong Li, Xuewen Liu, Wangbo Zhao, Pan Du, Kaicheng Zhou, Qingyi Gu, Yang You, Zhen Dong, and Kurt Keutzer. K-sort eval: Efficient preference evaluation for visual generation via corrected vlm-as-a-judge. In C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust (eds.), International Conference on Learning Representations, volume 2026, p. 34622– 34637, 2026. URL https://proceedings.iclr.c/paper_files/paper/2026/ file/3a81bc33b77662ed235a3328c00aebce-Paper-Conference.pdf. 13 Preprint Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars (eds.), Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V, vol- ume 8693 of Lecture Notes in Computer Science, p. 740–755. Springer, 2014. doi: 10.1007/ 978-3-319-10602-1_48.URL https://doi.org/10.1007/978-3-319-10602-1_ 48. Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation, 2024. URL https://arxiv.org/abs/2404.01291. Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the As- sociation for Computational Linguistics, 11:635–651, 2023. doi: 10.1162/tacl_a_00566. URL https://aclanthology.org/2023.tacl-1.37/. Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, Peter Belcak, Mingjie Liu, Min-Hung Chen, Hongxu Yin, Yu-Chiang Frank Wang, Kwang-Ting Cheng, Yejin Choi, Jan Kautz, and Pavlo Molchanov. Gdpo: Group reward-decoupled normalization policy optimization for multi-reward rl optimization. In Proceedings of the 43rd International Conference on Machine Learning, 2026. URL https://icml.c/virtual/2026/poster/63333. Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. Grounding DINO: Marrying DINO with grounded pre-training for open-set object detection. In Computer Vision – ECCV 2024, Lecture Notes in Computer Science, p. 38–55. Springer, 2024. doi: 10.1007/978-3-031-72970-6_3. URL https://doi.org/10.1007/978-3-031-72970-6_3. Mao-Lin Luo, Zhe-Xu Wang, Zi-Hao Zhou, Bo Ye, Jian Zhao, Min-Ling Zhang, and Tong Wei. Rl forgets! towards continual policy optimization, 2026. URL https://arxiv.org/abs/ 2607.04364. Samin Mahdizadeh Sani, Max Ku, Nima Jamali, Matina Sani, Paria Khoshtab, Wei-Chieh Sun, Parnian Fazel, Zhi Rui Tam, Thomas Chong, Edisy Kin Wai Chan, Donald Tsang, Chiao-Wei Hsu, Ting Lam, Ho Yin Sam Ng, Chiafeng Chu, Chak-Wing Mak, Kem- ing Wu, Hiu Tung Wong, Yik Chun Ho, Chi Ruan, Zhuofeng Li, I-Sheng Fang, Shih- Ying Yeh, Ho Kei Cheng, Ping Nie, and Wenhu Chen.Imagenworld:Stress-testing image generation models with explainable human evaluation on open-ended real-world tasks.In C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust (eds.), International Conference on Learning Representations, volume 2026, p. 41886–41916, 2026. URL https://proceedings.iclr.c/paper_files/paper/2026/file/ 458fa8e331566383d8e74bdb647f829-Paper-Conference.pdf. Sina Malakouti, Boqing Gong, and Adriana Kovashka.Culture in action:Eval- uating text-to-image models through social activities.In C. Vondrick, B. Hari- haran, C. Raffel, L. Pinto, D. Yang, and A. Faust (eds.), International Confer- ence on Learning Representations, volume 2026, p. 136235–136262, 2026.URL https://proceedings.iclr.c/paper_files/paper/2026/file/ dca63f2650fe9e88956c1b68440b8e9-Paper-Conference.pdf. Shravan Nayak, Mehar Bhatia, Xiaofeng Zhang, Verena Rieser, Lisa Anne Hendricks, Sjoerd van Steenkiste, Yash Goyal, Karolina Stanczak, and Aishwarya Agrawal. CulturalFrames: Assess- ing cultural expectation alignment in text-to-image models and evaluation metrics. In Chris- tos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Computational Linguistics: EMNLP 2025, p. 20918–20953, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176-335-7. doi: 10.18653/v1/2025.findings-emnlp.1141. URL https://aclanthology.org/2025. findings-emnlp.1141/. Yuansheng Ni, Songcheng Cai, Xiangchao Chen, Jiarong Liang, Zhiheng LYU, Jiaqi Deng, Kai Zou, PING NIE, Fei Yuan, Xiang Yue, and Wenhu Chen. Viscoder2: Building multi-language visual- ization coding agents. In C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust 14 Preprint (eds.), International Conference on Learning Representations, volume 2026, p. 104353–104422, 2026. URL https://proceedings.iclr.c/paper_files/paper/2026/file/ aac47a65339d7353562f6365526e81b-Paper-Conference.pdf. Masanari Oi, Masahiro Kaneko, Naoaki Okazaki, and Nakamasa Inoue. Multi-modal, multi-task, multi-criteria automatic evaluation with vision language models. In Stelios Piperidis, Núria Bel, Henk van den Heuvel, Nancy Ide, Simon Krek, and Antonio Toral (eds.), Proceedings of the Fifteenth Language Resources and Evaluation Conference, p. 9396–9408, Palma de Mallorca, Spain, May 2026. ELRA Language Resource Association. doi: 10.63317/575aknbvd9hs. URL https://aclanthology.org/2026.lrec-1.738/. Yasumasa Onoe, Sunayana Rane, Zachary Berger, Yonatan Bitton, Jaemin Cho, Roopal Garg, Alexander Ku, Zarana Parekh, Jordi Pont-Tuset, Garrett Tanzer, Su Wang, and Jason Baldridge. Docci: Descriptions of connected and contrasting images, 2024. URL https://arxiv.org/ abs/2404.19753. Roni Paiss, Ariel Ephrat, Omer Tov, Shiran Zada, Inbar Mosseri, Michal Irani, and Tali Dekel.Teaching clip to count to ten.In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), p. 3170–3180, October 2023.URL https://openaccess.thecvf.com/content/ICCV2023/html/Paiss_ Teaching_CLIP_to_Count_to_Ten_ICCV_2023_paper.html. Bryan A. Plummer, Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, and Svetlana Lazebnik.Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models.In Proceedings of the IEEE International Confer- ence on Computer Vision (ICCV), p. 2641–2649, December 2015.doi: 10.1109/ICCV. 2015.303. URL https://openaccess.thecvf.com/content_iccv_2015/html/ Plummer_Flickr30k_Entities_Collecting_ICCV_2015_paper.html. Jordi Pont-Tuset, Jasper Uijlings, Soravit Changpinyo, Radu Soricut, and Vittorio Ferrari. Connect- ing vision and language with localized narratives. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.), Computer Vision – ECCV 2020, volume 12350 of Lecture Notes in Computer Science, p. 647–664, Cham, 2020. Springer International Publishing. ISBN 978-3-030-58558-7. doi: 10.1007/978-3-030-58558-7_38. URL https://doi.org/10. 1007/978-3-030-58558-7_38. Shu Pu, Yaochen Wang, Dongping Chen, Yuhang Chen, Guohao Wang, Qi Qin, Zhongyi Zhang, Zhiyuan Zhang, Zetong Zhou, Shuang Gong, Yi Gui, Yao Wan, and Philip S. Yu. Judge anything: Mllm as a judge across any modality. In KDD ’25: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, p. 5742–5753, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400714542. doi: 10.1145/3711896. 3737409. URL https://doi.org/10.1145/3711896.3737409. Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URL https://qwen. ai/blog?id=qwen3.5. Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M. Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S. Khan.Glamm: Pixel grounding large multimodal model.In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 13009–13018, June 2024. URL https: //openaccess.thecvf.com/content/CVPR2024/html/Rasheed_GLaMM_ Pixel_Grounding_Large_Multimodal_Model_CVPR_2024_paper.html. Candace Ross, Melissa Hall, Adriana Romero Soriano, and Adina Williams. What makes a good metric? evaluating automatic metrics for text-to-image consistency. In First Conference on Lan- guage Modeling, 2024. URL https://openreview.net/forum?id=LFfktMPAci. Subhadeep Roy, Gagan Bhatia, and Steffen Eger. Prototypicality bias reveals blindspots in multi- modal evaluation metrics, 2026. URL https://arxiv.org/abs/2601.04946. 15 Preprint Sara Sarto, Marcella Cornia, and Rita Cucchiara. Image captioning evaluation in the age of multi- modal llms: Challenges and future perspectives. In James Kwok (ed.), Proceedings of the Thirty- Fourth International Joint Conference on Artificial Intelligence, IJCAI-25, p. 10632–10640. In- ternational Joint Conferences on Artificial Intelligence Organization, 8 2025. doi: 10.24963/ijcai. 2025/1180. URL https://doi.org/10.24963/ijcai.2025/1180. Survey Track. Michael Saxon, Fatima Jahara, Mahsa Khoshnoodi, Yujie Lu, Aditya Sharma, and William Yang Wang.Who evaluates the evaluations?objectively scoring text-to-image prompt coher- ence metrics with t2iscorescore (ts2).In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, p. 85630–85657. Curran Associates, Inc., 2024.doi: 10.52202/ 079017-2719.URL https://proceedings.neurips.c/paper_files/paper/ 2024/file/9b9cfd5428153ccfbd4ba34b7e007305-Paper-Conference.pdf. Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 8430–8439, October 2019.URL https://openaccess.thecvf.com/content_ICCV_2019/html/ Shao_Objects365_A_Large-Scale_High-Quality_Dataset_for_Object_ Detection_ICCV_2019_paper.html. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402. 03300. FuLin Shi, Wenyi Xiao, Leilei Gan, Liang Ding, and Binchen. REVEALER: Reinforcement-guided visual reasoning for element-level text-image alignment evaluation. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens (eds.), Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 47630–47649, San Diego, California, United States, July 2026. Association for Computational Linguistics. ISBN 979-8-89176-390-6. doi: 10.18653/v1/2026.acl-long.2200. URL https://aclanthology. org/2026.acl-long.2200/. Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross.Winoground: Probing vision and language models for visio-linguistic compositionality. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 5238–5248, June 2022. URL https://openaccess.thecvf. com/content/CVPR2022/html/Thrush_Winoground_Probing_Vision_and_ Language_Models_for_Visio-Linguistic_Compositionality_CVPR_2022_ paper.html. Tan Wang, Kevin Lin, Linjie Li, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Equivariant similarity for vision-language foundation models. In Proceed- ings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 11998–12008, October 2023a.URL https://openaccess.thecvf.com/content/ICCV2023/ html/Wang_Equivariant_Similarity_for_Vision-Language_Foundation_ Models_ICCV_2023_paper.html. Zijie J. Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. DiffusionDB: A large-scale prompt gallery dataset for text-to-image gen- erative models. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers), p. 893–911, Toronto, Canada, July 2023b. Association for Computational Linguistics. doi: 10.18653/v1/2023.acl-long.51. URL https://aclanthology.org/2023.acl-long. 51/. Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to- image synthesis, 2023. URL https://arxiv.org/abs/2306.09341. 16 Preprint Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong.Imagereward:Learning and evaluating human preferences for text- to-image generation.In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, p. 15903–15935. Curran Associates, Inc., 2023.doi:10.52202/075280-0700.URL https://proceedings.neurips.c/paper_files/paper/2023/file/ 33646ef0ed554145eab65f6250fab0c9-Paper-Conference.pdf. Michal Yarom, Yonatan Bitton, Soravit Changpinyo, Roee Aharoni, Jonathan Herzig, Oran Lang, Eran Ofek, and Idan Szpektor.What you see is what you read?im- proving text-image alignment evaluation.In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Process- ing Systems, volume 36, p. 1601–1619. Curran Associates, Inc., 2023.doi: 10.52202/ 075280-0079.URL https://proceedings.neurips.c/paper_files/paper/ 2023/file/056e8e9c8ca9929cb6cf198952bf1dbb-Paper-Conference.pdf. Gyeongtae Yoo, Sanghyeok Park, Soohyuk Jang, Ik hwan Kim, and Sungroh Yoon. Rollout-level advantage-prioritized experience replay for grpo, 2026. URL https://arxiv.org/abs/ 2606.04560. Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingting Gao, Di Zhang, and Zhongyuan Wang. Learning multi-dimensional human preference for text-to-image generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 8018–8027, June 2024.URL https://openaccess.thecvf.com/content/ CVPR2024/html/Zhang_Learning_Multi-Dimensional_Human_Preference_ for_Text-to-Image_Generation_CVPR_2024_paper.html. Xinlu Zhang, Yujie Lu, Weizhi Wang, An Yan, Jun Yan, Lianke Qin, Heng Wang, Xifeng Yan, William Yang Wang, and Linda Ruth Petzold. Gpt-4v(ision) as a generalist evaluator for vision- language tasks, 2023. URL https://arxiv.org/abs/2311.01361. Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Tor- ralba.Scene parsing through ade20k dataset.In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), p. 633–641,July 2017.URL https://openaccess.thecvf.com/content_cvpr_2017/html/ Zhou_Scene_Parsing_Through_CVPR_2017_paper.html. AAPPENDIX A.1TRAINING DATA CONSTRUCTION Data sources. Table 4 summarizes the public datasets used to construct the training corpus and the supervision extracted from each source. We combine dense region annotations, grounded cap- tions, spatial-relation labels, count annotations, natural long-form descriptions, human judgments, and contrastive image pairs to obtain complementary coverage of the semantic skills required by VISEXMEM. Atomic claim construction. We represent each caption as a set of typed, visually verifiable claims drawn from six categories: object, attribute, action, relation, count, and negation. When the source dataset provides explicit structure–e.g., Visual Genome objects and attributes, Flickr30k phrase– region correspondences, VSR spatial labels, or PixMo-Count annotations–we convert this supervi- sion programmatically. For sources dominated by free-form descriptions, including DOCCI, CROC, and Localized Narratives, we use Qwen3.6-27B 2 as an offline decomposition teacher. The teacher is not used to overwrite reliable source annotations. 2 GPT-5.6 Sol, Qwen3.6-27B 17 Preprint Atomic verification. Each claim receives a supported or contradicted decision. Labels are derived preferentially from native dataset annotations or human judgments. When a reliable decision cannot be obtained directly from the source, the offline teacher may provide a missing decision or structural link; low-confidence or unresolved outputs are discarded. Thus, teacher supervision supplements rather than replaces source-grounded labels. Counterfactual supervision. We construct counterfactual examples through two complementary mechanisms. First, CROC provides image-side contrastive pairs in which the same textual descrip- tion is evaluated against independently generated images with known match labels. Second, for verified positive claims on natural images, we programmatically create text-side counterfactuals by modifying a single semantic constraint—object, attribute, action, relation, count, or negation— while preserving the image. This produces minimal pairs in which the intended semantic change is explicitly known. Visual evidence. Evidence is taken directly from native annotations whenever available. For claims without native bounding boxes, we use GroundingDINO to localize the already-labeled se- mantic content. GroundingDINO therefore serves only as an evidence-localization mechanism and never provides the Yes/No correctness label. A failed detection is treated as a localization failure rather than evidence that the claim is absent. Evidence is represented according to claim type: object and attribute claims use the relevant regions, relations and actions may involve multiple participat- ing regions, and counting claims retain the set of candidate instances. Pure absence claims are not assigned fabricated evidence regions. Causal-evidence construction. To distinguish genuine visual evidence from arbitrary image re- gions, each eligible evidence region E i is paired with a random control region R i from the same image. These selected-versus-random pairs provide the controlled interventions used by the Stage 3 causal-evidence objective. Exact random-region matching and image-view construction are de- scribed in Appendix A.2.5. Explanation supervision. Explanation supervision is constructed only after the atomic claims, decisions, and evidence have been fixed. Explanation targets are generated from these structured records rather than used to determine them. In particular, the explanation teachers do not infer new bounding boxes or redefine correctness labels. This separation ensures that explanation learning cannot modify the supervision from which the numerical metric score is derived. Processed training examples. Figure 7 shows representative trainer-facing records after prepro- cessing. Stage 1 uses separate SFT views for decomposition, verification, evidence selection, and explanation. GroundingDINO boxes are supplied as candidate evidence regions; the evidence- selection task predicts candidate IDs rather than box coordinates. Stage 2 uses paired supported and counterfactual claims and optimizes their decision margins without a generated target sequence. Stage 3 combines semantic examples, where groups of sampled completions are scored by Yes/No correctness, with causal-evidence examples that compare selected evidence against a matched ran- dom region using multiple image views. Stage 3 also replays earlier SFT and counterfactual exam- ples. The examples below are taken from the processed training files; unused bookkeeping fields are omitted and long coordinates are rounded only for readability. Data isolation. All final evaluation benchmarks are excluded from the training corpus and model- selection data. Training and development groups are separated before optimization to prevent the same underlying example from appearing across splits. Only the training split of EvalMuse is used for training. A.2TRAINING AND IMPLEMENTATION DETAILS A.2.1MODELS AND PARAMETER-EFFICIENT TRAINING We train Qwen3.5-4B and Qwen3.5-9B using LoRA (Hu et al., 2022) with rank 16, scaling factor α = 32, dropout 0.05, and no bias. LoRA is applied to the q, k, v, o, gate, up, and down pro- jections. All training uses bfloat16 precision and disables the Qwen3.5 thinking mode. The main 18 Preprint Source: coco_train2017 Sample: coco_3333 DECOMPOSITION VIEW Caption: "A fire truck near a yellow fire hydrant." Target: "parts":[ "text":"fire truck", "type":"object", "text":"yellow fire hydrant", "type":"attribute", "text":"fire truck near a yellow fire hydrant", "type":"relation" ] EVIDENCE-SELECTION VIEW Claim (attribute): yellow fire hydrant Candidate E1: box=[93,279,224,482] Target: "selected_evidence_ids":["E1"] Training signal: supervised structured target Stage 1: Structured SFT Source: coco_train2017 Claim type: attribute SUPPORTED SIDE Caption: "A fire truck near a yellow fire hydrant." Claim: yellow fire hydrant Decision: YES COUNTERFACTUAL SIDE Caption: "A fire truck near a green fire hydrant." Claim: green fire hydrant Decision: NO Change: yellow -> green Image pixels: identical for this example Training signal: Yes/No decision-margin loss Generated target text: none Stage 2: Counterfactual Refinement SEMANTIC EXAMPLE Source: coco_train2017 Sample: cf_coco_9065_count Caption: "A dog and three people walking on sand." Claim (count): three Decision: NO Rollouts per prompt: 8 Reward: +1.00 correct decision -1.00 incorrect decision -1.25 invalid JSON CAUSAL-EVIDENCE EXAMPLE Source: coco_train2017 Sample: coco_3333 Claim (attribute): yellow fire hydrant Decision: YES Selected region: [92.85,278.53,224.34,482.07] Matched random region: [141.57,62.35,270.76,272.01] Image views: full mask_selected mask_random evidence_only random_only Stage 3: Evidence-Aware GDPO Figure 7: Representative processed training examples. Real trainer-facing examples from the three VisExMEM training stages. Unused metadata are omitted and coordinates are rounded for readability. 19 Preprint SourceSupervision usedPrimary skills MS COCOCaptions and object boxesObject, count Visual GenomeRegion descriptions, object boxes, attributesObject, attribute, relation VSRImage–caption pairs with spatial-relation la- bels Spatial relation Flickr30k EntitiesPhrase–entity–region correspondencesObject, grounding Open Images + Localized Narratives Grounded free-form narrativesObject, attribute, action, rela- tion DOCCIDetailed connected and contrasting descrip- tions Attribute, relation, negation PixMo-CountExact counts with point-level evidenceCount CROCsynContrastive image pairs with match labelsCounterfactual verification EvalMuse-40KElement-level human judgments from the training split Atomic verification Table 4: Public data sources used to construct the VISEXMEM training corpus. TaskTarget format Decomposition "parts": [...] ; each part contains part_id, text, type, parent_part_id, head_part_id, and target_part_id. Verification "decision": "yes" or "decision": "no". Evidence selection "selected_evidence_ids": ["E1", ...]. Explanation "explanation": "...". Table 5: Task-specific target formats used during Stage 1 SFT. Candidate bounding boxes are inputs to evidence selection and are not prediction targets. training hyperparameters are summarized in Table 6; construction of the underlying training corpus is described separately in Appendix A.1. A.2.2VERIFICATION PROMPT AND DECISION SCORING Counterfactual refinement and Stage 3 use the same verification prompt template to avoid prompt drift between supervised and policy optimization. Given an image, caption, claim type, and atomic claim, the model is instructed to return exactly one JSON decision, "decision":"yes" or "decision":"no". Outputs that do not produce either valid decision are treated as invalid during Stage 3 policy optimization. During counterfactual refinement, no free-form generation is performed. For each supported/coun- terfactual pair, the same image is evaluated with the corresponding claim, and the verification margin in Equation 2 is obtained directly from the yes and no decision logits after their shared output pre- fix. In Stage 3 semantic optimization, the same prompt is instead used to sample G = 8 decision completions. For causal-evidence training, the textual prompt is held fixed while only the image view changes. Thus, differences between causal views arise from the image intervention rather than from changes to the textual instruction. A.2.3MULTI-TASK SFT Stage 1 uses a four-part curriculum over decomposition, verification, evidence selection, and joint supervision. The task-specific views contain 13.5K decomposition, 15K verification, and 7K evidence-selection examples. The final joint stage contains 40K examples, including 4.5K explana- tion examples, and samples decomposition, verification, evidence-selection, and explanation tasks using an 8:9:4:3 schedule. Each curriculum stage is traversed once. SFT target formats. Each SFT example has a single task-specific JSON target y, and loss is computed only over the target tokens using the next-token objective in Equation 1. Table 5 summa- rizes the exact target formats. Candidate bounding boxes are provided only as inputs for evidence selection and are never prediction targets. 20 Preprint Optimization. We use AdamW with cosine learning-rate decay, 3% warmup, and gradient clip- ping at 1.0. The learning rate is 10 −5 for the first three task-specific curriculum stages and 7× 10 −6 for final joint training. A.2.4COUNTERFACTUAL REFINEMENT Stage 2 starts from the promoted Stage 1 checkpoint. The source pool contains 20K supported/coun- terfactual pairs. Training interleaves counterfactual updates, verification replay, and decomposition replay in an 8:1:1 update ratio. For pair i, let m + i = m θ (I i ,a + i ) and m − i = m θ (I i ,a − i ) denote the verification margins from Equation 2 for the supported and counterfactual claims, respectively. Let m + i,ref denote the positive margin produced by the frozen pre-Stage-2 checkpoint. We define the retained positive target as ̄m + i = max τ + ,m + i,ref − δ keep ,(5) and optimize L CF =E i h λ pair γ cf − (m + i − m − i ) + + λ + ̄m + i − m + i + + λ − m − i + τ − + i ,(6) where [z] + = max(0,z). The three terms respectively enforce separation between the supported and counterfactual claims, preserve sufficient support for the positive claim, and push the counterfactual below the negative target. We use γ cf = 3.0, τ + = 3.0, τ − = 2.0, δ keep = 1.0, λ pair = 1.0, and λ + = λ − = 0.5. Thus, the positive target is max(3.0,m + i,ref − 1.0) and the negative target is−2.0. Verification replay uses its original binary verification objective together with the retention con- straint, whereas decomposition replay retains the Stage 1 token-level sequence objective. We train for one counterfactual epoch (450 optimizer steps) with AdamW, learning rate 5×10 −7 , 4% warmup, cosine decay to a 10% learning-rate floor, and gradient clipping at 1.0. A.2.5EVIDENCE-AWARE GDPO Stage 3 starts from the promoted counterfactual checkpoint and uses 20K semantic-verification ex- amples together with 20K supported causal-evidence examples. Our final Stage 3 procedure is an evidence-aware GDPO variant: semantic correctness supplies the group-relative rollout signal, while causal evidence is optimized directly as an auxiliary objective. In particular, the causal signal is not treated as a second independently group-normalized rollout reward; this distinction is important be- cause the image-intervention signal is shared across rollouts of the same input and would otherwise be attenuated by within-group centering. Semantic policy optimization. For each semantic prompt, we sample G = 8 completions with temperature 0.7, top-p = 1.0, and a maximum of 16 generated tokens. Correct, incorrect, and invalid decisions receive rewards +1,−1, and−1.25, respectively. The group-relative quantities in Equation 3 are implemented with A i,g = r i,g − μ i σ i + ε A , ε A = 10 −4 ,(7) whereμ i andσ i are the mean and population standard deviation of theG rollout rewards for semantic example i. If all rewards in a group are identical, every centered reward is zero and the group therefore provides no relative policy signal. For the clipped semantic objective in Equation 4, we use ε = 0.2. KL regularization uses β = 0.02 with the frozen Stage 2 checkpoint as the reference policy. For a generated token, the token-level KL penalty is d KL = exp(κ)− κ− 1, κ = logπ ref − logπ θ .(8) Replay schedule. Stage 3 interleaves semantic GDPO updates, counterfactual replay, and SFT replay in a 7:2:1 update ratio. Replay draws from the earlier training pools and introduces no ad- ditional training examples. Counterfactual replay therefore retains the Stage 2 margin objective in Equation 6, while SFT replay retains the original sequence-supervision objective in Equation 1. 21 Preprint Five-view causal construction. For causal example i, let E i denote the selected evidence region and R i its matched random control. We construct five image views: the intact image I , selected- evidence removal I i , random-control removal I i , selected-evidence-only I⊙E i , and random- control-only I⊙R i . Here\ masks the specified region while retaining the rest of the image, whereas ⊙ masks the remaining canvas and retains only the specified region. The masking implementation uses either Gaussian blur or local-mean filling. Let s i (V ) denote the yes–no verification margin for claim i under image view V . We define ∆ sel i = s i (I)− s i (I \ E i ),∆ rnd i = s i (I)− s i (I \ R i ), ∆ suff i = s i (I ⊙ E i )− s i (I ⊙ R i ), ∆ anchor i = s i (I)− s i (I ⊙ E i ). (9) The first two quantities measure the effect of removing selected versus random regions, ∆ suff i com- pares their evidence-only sufficiency, and ∆ anchor i measures the drop from the intact image to the selected-evidence-only view. Causal-evidence objective. Using the quantities in Equation 9, we optimize L causal (u) = λ ramp (u) w comp softplus γ comp − (∆ sel i − ∆ rnd i ) + w drop softplus γ drop − ∆ sel i + w suff softplus γ suff − ∆ suff i + w suffdrop softplus ∆ anchor i − γ max . (10) The four terms respectively encourage (i) a larger removal effect for selected than random evi- dence, (i) sensitivity to removing selected evidence, (i) greater support from selected than random evidence-only views, and (iv) a bounded degradation when verification is restricted to selected evi- dence alone. We use γ comp = 0.25, γ drop = 0.25, γ suff = 0.75, and γ max = 1.0, with (w comp ,w drop ,w suff ,w suffdrop ) = (1.0, 0.5, 1.0, 0.25). The outer causal coefficient in the Stage 3 objective is λ c = 0.3. The internal coefficient λ ramp (u) increases linearly from 0 to 0.5 over the first 100 completed causal updates and remains at 0.5 thereafter. The maximum resulting multiplier on the weighted causal auxiliary is therefore 0.3× 0.5 = 0.15. Eligibility and random-control matching. A causal example contributes to L causal only when the source image and all five required views can be constructed. Examples that fail this requirement are excluded from the causal auxiliary; if no eligible causal examples remain in an update, the causal term is omitted for that update. Each selected evidence region E i is paired with a random control region R i from the same im- age. The control is matched to E i within±15% in area and±20% in aspect ratio while satisfying IoU(R i ,E) < 0.10 for every evidence region E in that image. We allow at most 200 sampling attempts. Joint Stage 3 objective and optimization. The final Stage 3 update combines the semantic ob- jective from Equation 4 with the direct causal auxiliary from Equation 10: L Stage3 =−J sem + λ c L causal .(11) The causal term is optimized directly rather than incorporated into the group-normalized rollout reward, preserving the image-intervention signal while semantic correctness is learned through the rollout-based policy objective. Stage 3 uses AdamW with learning rate 10 −6 , cosine decay, 3% warmup, zero weight decay, and gradient clipping at 1.0. The sequential Verification-GRPO and Causal-GRPO alternatives used in the training ablations are described separately in Appendix A.3; they branch from the Stage 2 checkpoint and are not additional stages of the final VISEXMEM model. 22 Preprint Table 6: Training hyperparameters for the 4B and 9B VISEXMEM models. For SFT, 4/2 denotes per-device batch size 4 for the decomposition curriculum stage and 2 for the remaining stages; the two learning rates correspond to the first three task-specific curriculum stages and the final joint stage, respectively. Replay/update mixtures denote counterfactual:verification:decomposition for Stage 2 and semantic-GDPO:counterfactual:SFT for Stage 3. SFTCounterfactualEvidence-aware GDPO Hyperparameter4B9B4B9B4B9B Training duration (hours)4.44.62.73.440.943.3 GPUs4× H1004× H100 4× H100 4× H100 2× H100 2× H100 Per-device batch size4/24/21122 Gradient accumulation888822 Learning rate10 −5 /7×10 −6 10 −5 /7×10 −6 5×10 −7 5×10 −7 10 −6 10 −6 Learning-rate schedulercosinecosinecosinecosinecosinecosine Warmup3%3%4%4%3%3% Weight decay0.00.00.010.010.00.0 Gradient clipping1.01.01.01.01.01.0 Replay/update mixture–8:1:18:1:17:2:17:2:1 Generations per prompt–88 A.3GRPO ABLATIONS Ablation strategy. We evaluate a sequential GRPO alternative to the final VISEXMEM training strategy. The ablation branch starts from the counterfactually refined Stage 2 checkpoint: Counterfactual→ Verification-GRPO→ Causal-GRPO. In contrast, the final evidence-aware GDPO stage branches directly from the counterfactual check- point. Thus, Verification-GRPO and Causal-GRPO are controlled ablations rather than additional stages of the deployed model. Verification-GRPO. Verification-GRPO isolates reinforcement learning for semantic correctness. For each image–claim input, we sample G = 8 completions and assign reward +1 to a correct yes/no decision, −1 to an incorrect valid decision, and −1.25 to an invalid output. Rewards are normalized within each rollout group as A i,g = r i,g − μ i σ i + ε A , ε A = 10 −4 ,(12) where μ i and σ i are the mean and population standard deviation of the G rollout rewards. Groups with identical rewards therefore provide no relative policy signal. We then apply the same clipped policy objective and KL regularization described in Equation 4. Unlike Stage 2, which directly optimizes verification margins, Verification-GRPO learns from relative differences between sampled decisions. Causal-GRPO. Causal-GRPO continues from Verification-GRPO and tests whether visual evi- dence dependence can be incorporated through the rollout reward itself. It uses the same GRPO formulation, but for a correct supported decision adds a causal bonus derived from the difference between masking selected evidence and a matched random region: r causal i,g = r sem i,g + α clip(e i , 0, 1), α = 0.3,(13) where e i denotes the selected-versus-random evidence advantage. This ablation therefore tests a reward-based treatment of causal evidence, whereas the final model optimizes evidence dependence directly. Replay. Both GRPO ablations interleave decomposition examples from the earlier SFT stage to preserve the structured evaluation interface during policy optimization. This differs from GRPO experience replay (Yoo et al., 2026): we do not store or reuse previous policy rollouts. Instead, earlier supervised examples are revisited with their original training objective. 23 Preprint The final GDPO stage uses the same retention principle more broadly, interleaving semantic policy updates, counterfactual replay, and SFT replay. The contribution of this replay strategy is evaluated separately in the training ablations. Relation to the final GDPO objective. The sequential GRPO branch separates semantic verifi- cation and causal evidence into two successive policy-optimization stages. Our final model instead combines them in a single Stage 3 optimization. Semantic verification provides the rollout-based policy signal, while the five-view causal objective from Equation 10 is optimized directly: L Stage3 =−J sem + λ c L causal .(14) This distinction is important because a causal signal shared across all rollouts of the same input is largely removed by within-group centering and therefore provides little relative GRPO signal. Direct optimization preserves the image-intervention signal while allowing semantic correctness and evidence dependence to be learned jointly. A.4ADDITIONAL RESULTS A.4.1OUT-OF-DISTRIBUTION GENERALIZATION We further evaluate VISEXMEM on three external benchmarks that are not used in training, covering structured visualizations, culturally grounded image–text alignment, and prototypicality- sensitive semantic evaluation. VisPlotBench (Ni et al., 2026), introduced with VisCoder2, provides a structured-visual distribution based on rendered visualization tasks. We evaluate 886 examples and report pairwise accuracy (PAcc) and the stricter Group score. We additionally include CHARTVE (Huang et al., 2024), a visual-entailment model developed for factual-consistency evaluation of chart captions. CulturalFrames (Nayak et al., 2025) evaluates alignment with explicit and implicit cultural expec- tations across countries and socio-cultural domains. We evaluate 3,577 images and report Spear- man’s ρ with human alignment judgments and within-prompt pairwise accuracy (PAcc wp ). We also include the culturally targeted AHEaD metrics (Malakouti et al., 2026): ALIGN measures cover- age of expected cultural descriptors, while HAL measures unsupported cultural content. We report 1− HAL so that higher is better throughout. ProtoBias (Roy et al., 2026) tests whether evaluation metrics prefer semantically correct images over more prototypical alternatives containing controlled semantic errors. We evaluate its 600 human-annotated examples, with 200 each from Animals, Demography, and Objects, and report category-level accuracy and Cohen’s κ. The GPT-5 (Judge) and PROTOSCORE results are taken directly from the original ProtoBias study (Roy et al., 2026) and are marked with † in the table; all remaining values are obtained under our evaluation setup. Results. The OOD results show that VISEXMEM-specific training provides substantial gains over the corresponding untuned Qwen3.5 judges. On VisPlotBench, VISEXMEM-4B improves over both 4B controls and reaches 91.3 PAcc and 74.4 Group, while VISEXMEM-9B reaches 91.3 PAcc and the highest Group score of 78.6, compared with 74.5 for the strongest 9B Qwen control. On CulturalFrames, the gain is clearest at 9B: VISEXMEM-9B reaches ρ = .311 and 65.8 PAcc wp , exceeding both corresponding Qwen3.5 controls and all conventional metrics; the strongest conven- tional baseline, VQAScore, obtains ρ = .250 and 64.4 PAcc wp . The same pattern appears on ProtoBias. VISEXMEM-9B substantially improves over both 9B Qwen controls across all three categories, reaching 73.5/71.5/81.0 accuracy and .254/.205/.286 κ on Animals/Demography/Objects. VISEXMEM-4B likewise improves strongly over its Qwen base- lines in category accuracy. Among the conventional general-purpose metrics, the VISEXMEM variants obtain the strongest ProtoBias results. PROTOSCORE achieves higher category accuracy, as expected for a model trained specifically for the prototypicality-bias setting, while VISEXMEM-9B remains stronger in κ on Animals and Objects despite receiving no ProtoBias-specific training. 24 Preprint Table 7: Out-of-distribution generalization. Results on VisPlotBench, CulturalFrames, and the 600 human-annotated ProtoBias examples. VisPlotBench reports pairwise accuracy (PAcc) and Group score; CulturalFrames reports Spearman’s ρ and within-prompt pairwise accuracy (PAcc wp ); ProtoBias reports category-level accuracy (Acc.) and Cohen’s κ. Accuracy-based metrics are re- ported on a 0–100 scale; correlations and κ are reported on their native scale. Higher is better. Bold, underlined, and italicized values denote the best, second-best, and third-best results in each column, respectively. VisPlotBenchCulturalFramesProtoBias MethodPAcc GroupρPAcc wp AnimalDemographyObject Acc.κAcc.κAcc.κ CLIPScore82.552.6.14560.525.0 .00128.0-.00532.0.058 ALIGNScore79.145.3.14957.851.0 .03334.0-.06432.0.011 PickScore77.139.5 .20662.232.0 .00134.0-.01431.0.126 HPSv279.737.4.21961.525.0 .02438.0-.00638.0.125 VQAScore85.161.4.25064.451.0 .02847.0-.00755.0.065 ChartVE69.231.3 – GPT-5 (Judge)–56.0 .10851.0.17654.0.265 ProtoScore– –95.0 .18291.0.23298.0.178 AHEaD-ALIGN–.11455.635.5 .06449.5.08355.5 -.030 AHEaD-HAL (1− HAL)–.13155.0– Qwen3.5-4B (Hol.)84.470.2.24256.739.8 .13145.8.07544.5.161 Qwen3.5-4B (Dec.)85.873.8.29649.443.5 .12139.0.10260.0.269 Qwen3.5-9B (Hol.)81.572.4.25215.232.0 .04926.0.14240.5.102 Qwen3.5-9B (Dec.)83.474.5.21051.442.9 .05654.0.16755.6.195 VisExMEM-4B91.374.4.19860.576.0.18967.0.10673.5.256 VisExMEM-9B91.378.6.31165.873.5.25471.5.20581.0.286 Table 8: Replay ablation for VISEXMEM-9B. Both paths start from the same SFT checkpoint and either retain or disable replay throughout CF and GDPO. ∆ denotes Replay− No replay. Higher is better. VariantReplayWGEqB TIFAPickGenAI Base (Hol.)–40.435.750.174.250.3 + SFT–61.958.959.861.857.2 + CFYes66.364.968.977.567.2 + CFNo67.966.271.181.167.3 ∆Yes− No(-1.6)(-1.3)(-2.2)(-3.6)(-0.1) + GDPOYes68.970.173.185.669.4 + GDPONo66.164.870.774.565.1 ∆Yes− No(+2.4)(+5.3)(+2.4)(+11.1)(+4.3) A.4.2REPLAY ABLATION We examine the role of replay during progressive training using VISEXMEM-9B. Both variants start from the same base model and SFT checkpoint. We then train two complete trajectories: the standard trajectory replays earlier-stage supervision during both counterfactual refinement (CF) and evidence- aware GDPO, whereas the no-replay trajectory disables replay in both stages. Table 8 reports the resulting checkpoints on the same five benchmarks used in the main training-stage ablation. At the CF checkpoint, disabling replay is consistently beneficial. The no-replay variant improves over the replay variant on all five benchmarks, raising the average from 69.0 to 70.7. The largest difference occurs on Pick-a-Pic (81.1 vs. 77.5), while GenAI-Bench changes only marginally (67.3 vs. 67.2). Thus, replay is not required for effective counterfactual adaptation and slightly constrains the immediate CF checkpoint. The behavior changes markedly after evidence-aware GDPO. Starting from the stronger no-replay CF checkpoint, continuing training without replay reduces the average from 70.7 to 68.2. In contrast, 25 Preprint Table 9: Aggregation ablation. Performance of VISEXMEM-9B under alternative fixed aggrega- tion rules. Higher is better. AggregationWG EqBen TIFA160 Pick-a-Pic GenAI Arithmetic mean 68.970.173.185.669.4 Geometric68.469.671.684.468.7 Harmonic67.360.369.681.564.4 Minimum62.955.468.980.463.7 the replay trajectory improves from 69.0 after CF to 73.3 after GDPO and outperforms the no-replay trajectory on all five benchmarks. The largest final differences are observed on Pick-a-Pic (+11.1) and EqBen (+5.3). This reversal suggests that replay plays different roles across the two training stages: it is less im- portant for short-term counterfactual adaptation, but becomes beneficial during policy optimization, where retaining earlier structured and counterfactual supervision helps avoid degradation while opti- mizing the new semantic-policy and causal-evidence objectives. Importantly, the two GDPO check- points continue from their respective replay and no-replay CF branches; therefore, this ablation es- tablishes the effect of the complete replay versus no-replay training trajectories, rather than isolating replay within Stage 3 alone. A.4.3AGGREGATION ABLATION We evaluate the sensitivity of VISEXMEM-9B to the rule used to aggregate claim-level support scores. Using the same claim predictions, we compare the arithmetic mean with geometric, har- monic, and minimum aggregation. As shown in Table 9, the arithmetic mean performs best on all five evaluated benchmarks, with the geometric mean as the closest alternative. This result supports the arithmetic mean used in the main method, which gives equal weight to all visually verifiable claims and requires no learned or benchmark-specific aggregation parameters. A.5BENCHMARK EVALUATION PROTOCOLS We use the same fixed VISEXMEM inference procedure across all benchmarks. The evaluation set and meta-evaluation statistic for each benchmark are fixed in advance; no VISEXMEM prompt, decision threshold, or aggregation rule is tuned on benchmark test data. When an evaluation statistic itself includes a calibration step, as in pairwise accuracy with tie calibration, this is applied only at meta-evaluation time to the already-computed metric scores and does not modify VISEXMEM inference. Winoground and EqBen. Winoground (Thrush et al., 2022) and EqBen (Wang et al., 2023a) eval- uate fine-grained image–text matching using two matched image–caption pairs per example. Follow- ing the evaluation setting used in our comparisons, we use the complete 400-example Winoground set and the 280-example EqBen-Mini subset. Let (I 0 ,c 0 ) and (I 1 ,c 1 ) denote the two matched pairs and S(I,c) the metric score. The Text condition is satisfied when each image prefers its matched caption, S(I 0 ,c 0 ) > S(I 0 ,c 1 ) and S(I 1 ,c 1 ) > S(I 1 ,c 0 ), whereas the Image condition is satisfied when each caption prefers its matched image, S(I 0 ,c 0 ) > S(I 1 ,c 0 ) and S(I 1 ,c 1 ) > S(I 0 ,c 1 ). The Group condition requires both the Text and Image conditions to hold. We report all three scores where space permits and use Group for the main comparison because it requires all four matching inequalities to be correct. TIFA160 and GenAI-Bench. For TIFA160 (Hu et al., 2023), we evaluate the 800 image–text pairs formed from 160 prompts and outputs from five text-to-image models, together with their 1–5 human alignment ratings. For GenAI-Bench (Li et al., 2024a), we use the 527-prompt image coreset used for human and automatic meta-evaluation, rather than the full 1,600-prompt collection. 26 Preprint For both benchmarks, we report pairwise accuracy (PAcc) with tie calibration. PAcc is computed from the vectors of human ratings and metric scores by comparing their induced pairwise high- er/lower/tie decisions; the metric’s tie threshold is calibrated according to the pairwise-accuracy protocol. Thus, these benchmarks contain scalar human ratings rather than predefined binary pref- erence pairs, and PAcc measures agreement between the pairwise orderings induced by the human and metric scores. Results are reported on the 0–100 scale. Pick-a-Pic. For Pick-a-Pic (Kirstain et al., 2023), we follow the filtered evaluation subset used in the VQAScore protocol: 100 prompts, each paired with two generated images, for a total of 200 images. Human-tied examples are excluded from this subset. We report binary preference accuracy, where a comparison is correct when the human-preferred image receives the larger metric score for the corresponding prompt. CROC hum . CROC hum (Leiter et al., 2026) evaluates contrastive robustness in four directions ob- tained by crossing text-based versus image-based evaluation with forward versus inverse evaluation. In text-based evaluation, the image is held fixed while the original and contrastive texts are com- pared; in image-based evaluation, the text is held fixed while original and contrastive images are compared. Forward evaluation treats the original text–image pair as the matching pair, whereas in- verse evaluation treats the contrastive pair as the matching pair. We compute all four direction scores using the benchmark protocol and report their unweighted arithmetic mean as the single CROC hum score in the main comparison. T2I-CompBench++. For T2I-CompBench++ (Huang et al., 2025), we use its released human- evaluation image–text examples and report Spearman’s rank correlation coefficient (SRCC) between automatic metric scores and the corresponding human alignment judgments on the evaluation subset used in our experiments. Spearman correlation evaluates agreement in rank ordering and is invariant to monotonic rescaling of either score. The identical sample set and correlation computation are used for every method in our comparison. CountBench. CountBench (Paiss et al., 2023) evaluates zero-shot visual counting for counts from two through ten. For each image, the benchmark constructs nine candidate captions by replacing the count in its caption with each number intwo,..., ten. We score all nine image–caption combina- tions using the same metric inference procedure and take the count appearing in the highest-scoring caption as the predicted count. We report exact top-1 counting accuracy. T2IScoreScore. T2IScoreScore (TS2) (Saxon et al., 2024) meta-evaluates image–text metrics us- ing semantic error graphs (SEGs). Each SEG associates a prompt with groups of images containing controlled semantic-error configurations. We use the two principal TS2 statistics reported in the main paper. The ordering score rank m is the benchmark’s Spearman-based measure of whether met- ric scores correctly order images along walks through an SEG according to semantic-error severity. The separation score sep m measures how well score distributions for neighboring error nodes are distinguished using the two-sample Kolmogorov–Smirnov statistic. Higher values indicate better performance for both measures. We report rank m and sep m over all evaluated SEGs (All) and separately for the Synth, Nat, and Real partitions. Synth uses generated images to populate predefined semantic error graphs, Nat constructs error graphs from natural images, and Real contains naturally occurring semantic errors from text-to-image generations. Here, “Real” refers to the errors being naturally occurring rather than to the images being real photographs. Reporting both ordering and separation distinguishes metrics that recover the intended semantic ordering from those that also assign clearly separated scores to different error populations. AGITA-Bench. For AGITA-Bench (Sec. 4), evaluation is performed separately at the global and atomic levels. Global agreement is Spearman’s ρ between each method’s image–text score and the human global-alignment rating. Atomic agreement is computed only for methods that can directly score the fixed benchmark claims and is Spearman’s ρ between claim-level metric scores and human atomic-support judgments; claims marked cannot judge are excluded. Correlations are reported separately for the Easy, Medium, and Hard semantic-complexity subsets, with the reported average 27 Preprint equal to the unweighted mean of the three subset correlations. The benchmark construction and full human-annotation protocol are provided in Appendix A.6. GroundingDINO baseline. To test how much image-text alignment can be recovered from grounding alone, we replace the VISEXMEM verifier with GroundingDINO-base (Liu et al., 2024). Captions are decomposed into atomic claims by the same frozen text-only VISEXMEM v2 decom- poser, which receives neither images nor benchmark labels. Each claim is converted to a lowercase, period-terminated query, and GroundingDINO is the only component that receives image pixels. We use a box threshold of 0.30, text threshold of 0.25, NMS IoU of 0.5, and at most 32 candidate boxes per query. Claim support is based on the strongest phrase-aligned confidence before thresholding, with the scoring rule determined by claim type. Object claims use the maximum confidence and negated claims use its complement. Count claims combine the strongest detections with agreement between the detected and expected counts. At- tribute claims combine entity and attribute confidence with box overlap, while spatial claims also use the relative positions of subject and object boxes. Verb relations combine subject, object, and phrase confidence, and unary actions combine subject and phrase confidence. Depth relations and abstract claims are marked ungroundable because no reliable 2D box rule is defined. If M of the N claims have a defined score, the final metric is S GDINO = M N exp 1 M M X i=1 log max(s i , 10 −6 ) ! . The geometric mean makes low-support claims reduce the overall score, while the coverage term M/N penalizes captions for which only part of the content can be grounded. All thresholds and constants are fixed before evaluation. The baseline uses no training, calibration, or benchmark- specific tuning, and GroundingDINO confidence is treated as an uncalibrated grounding support score rather than a probability. A.5.1VLM-AS-A-JUDGE EVALUATION We evaluate untuned Qwen3.5-4B, Qwen3.5-9B, and GPT-5.6 Sol as VLM-as-a-Judge controls un- der two fixed inference settings. Results on the eight established benchmarks and AGITA-Bench are reported in Tables 1 and 2, respectively. Holistic (Hol.). The VLM receives the image and complete caption and directly assigns a single alignment score using a fixed rubric from 0.05 to 0.95. No decomposition, claim-level verification, or localized evidence is used. The complete prompt is shown in Figure 8. Decomposed (Dec.). The same backbone is evaluated with a structured end-to-end decompose– verify–aggregate procedure. The VLM receives the image and complete caption and, in a single response, decomposes the caption into visually verifiable atomic claims, localizes claim-level visual evidence, assigns each claim a support score in [0, 1], and generates a short evidence-grounded explanation. Possible claim types include object, attribute, count, action, relation, spatial, negation, and other. After evaluating all claims, the VLM produces an overall score in [0, 1] based on the completed part-level judgments. The overall score is generated by the VLM rather than computed using an external aggregation formula. The complete prompt is shown in Figure 9. These controls use no VISEXMEM-specific fine-tuning and do not use VISEXMEM’s learned evidence-conditioned verifier or external grounding cues. Thus, within each backbone, Hol. ver- sus Dec. changes the inference structure while keeping the underlying VLM fixed. A.5.2BASELINE COVERAGE AND REPRODUCIBILITY REVEALER (Shi et al., 2026) and T2I-FineEval (Hosseini et al., 2025) are particularly relevant to VISEXMEM because both perform fine-grained image–text evaluation rather than relying only on a global similarity score. We therefore examined whether they could be included in the controlled benchmark comparison. 28 Preprint REVEALER. REVEALER performs element-level alignment evaluation through a structured grounding–reasoning–conclusion trajectory and trains its evaluator with reinforcement learning over grounding and alignment objectives. At the time of our evaluation, we were unable to obtain a us- able public implementation and released checkpoint sufficient to reproduce the published method on our benchmark suite. Reimplementing the method from the paper would introduce uncontrolled choices in model initialization, training data, reward construction, and inference, and would there- fore not constitute a faithful baseline reproduction. We consequently do not report reconstructed REVEALER numbers. T2I-FineEval. T2I-FineEval decomposes both modalities: its reported configuration uses GPT-4 with two-shot prompting to decompose the input text and generate entity- and relation-level ques- tions, YOLOv9 to obtain image regions, and BLIP-VQA for question–region scoring. Although evaluation code is publicly available, faithfully reproducing the published configuration therefore requires the proprietary GPT-4 decomposition stage and its corresponding prompting configuration. Replacing this stage with a different language model would change the evaluated method and con- found the comparison with backbone and prompting differences. Because we could not reproduce the published configuration exactly under our controlled evaluation setup, we do not report a modi- fied T2I-FineEval result. We also avoid copying numbers reported in the original papers into our main tables, since their evaluated subsets, scoring statistics, and/or underlying models do not match our fixed protocols. All quantitative comparisons in Tables 1 and 2 therefore use scores obtained under the evaluation settings specified in this paper. 29 Preprint ----- HOLISTIC system ----- You are a strict visual alignment evaluator. You judge how well a caption matches an image, using only visible image evidence. Do not assume something is true because it sounds plausible. Do not use world knowledge when the image does not show it. Do not reward fluency or aesthetic quality. Return only the required JSON schema. ----- HOLISTIC user (image + full caption; no decomposition) ----- Judge how well the caption below matches the supplied image. Consider every visually verifiable element the caption expresses: objects, attributes, actions, relations, counts, and negation. A meaningful contradiction must lower the score even when most of the caption is correct. alignment_score rubric (from 0.05 to 0.95): 0.90-0.95 clearly and directly supported by visible evidence 0.75-0.85 strongly supported, with minor visual ambiguity 0.60-0.70 more supported than not, but incomplete/weak evidence 0.45-0.55 genuinely uncertain / ambiguous / not enough visible evidence 0.30-0.40 more unsupported than supported 0.15-0.25 strongly unsupported / likely contradicted 0.05-0.10 clearly contradicted, or required visual evidence is clearly absent Return valid JSON only, in exactly this shape: "alignment_score":0.95 Caption: caption VLM-as-a-Judge (Holistic) Figure 8: Prompt used for the holistic VLM-as-a-Judge setting. The model directly evaluates the complete image–caption pair and returns a single alignment score without atomic decomposition. 30 Preprint You are an expert visual evaluator. Your task is to evaluate how well the provided image supports the provided caption. You must perform the COMPLETE evaluation in this single response. CAPTION: caption Follow these steps internally in order: 1. Decompose the caption into visually verifiable atomic claims. 2. For every atomic claim, locate the visual evidence needed to judge it. 3. Assign an evidence-grounded support score to every atomic claim. 4. Give one short explanation for each score based on the visible evidence. 5. Only after evaluating all atomic claims, produce one overall image-caption support score based on those part-level judgments. Do not skip any visually meaningful component of the caption. -------------------------------- ATOMIC DECOMPOSITION -------------------------------- Split the caption into the smallest useful independently verifiable visual claims. Possible claim types include: - object - attribute - count - action - relation - spatial - negation - other Examples of information that should normally become separate claims when present: - an object exists; - an object has an attribute; - a specific number of objects exists; - an entity performs an action; - one entity acts on another; - two entities have a relation; - an entity is left/right/above/below/i ⌋ n-front-of/behind another; - a stated object is absent or a condition is negated. Preserve binding. For example, do not reduce: "the man in a red shirt is holding a blue umbrella" to unrelated statements such as: "red exists" "blue exists" "man exists" "umbrella exists" The atomic claims must preserve which property/action/relation belongs to which entity. Do not invent claims not contained in the caption. Do not evaluate non-visual linguistic details that cannot reasonably be verified from the image. -------------------------------- BOUNDING BOXES / VISUAL EVIDENCE -------------------------------- For each atomic claim identify only the visual entities or regions needed to verify that claim. Represent every bounding box using normalized coordinates: [x_min, y_min, x_max, y_max] with each coordinate an integer from 0 to 1000. Coordinate system: - (0,0) = top-left of the image - (1000,1000) = bottom-right of the image Each evidence item must contain: - label - role - box Example: "label": "dog", "role": "subject", "box": [120, 310, 430, 820] Use multiple boxes when necessary. Examples: COUNT: box all clearly visible relevant instances needed to judge the count. RELATION / SPATIAL: box all relevant participating entities. ACTION: box the actor and other important participant/object where applicable. ATTRIBUTE: box the entity to which the attribute is bound. INCORRECT CLAIM: show the visible counter-evidence when possible. For example, if the caption claims three dogs but only two are visible, return the boxes of those two visible dogs. NEGATION / ABSENCE: if the claim is supported specifically because something is absent and there is no meaningful region to box, `evidence_boxes` may be an empty list. Do not invent bounding boxes merely to satisfy the schema. -------------------------------- PART-LEVEL SUPPORT SCORE -------------------------------- For every atomic claim output a support score in: [0.0, 1.0] Interpretation: 1.00 = clearly and fully supported by the image 0.75 = mostly supported, with minor ambiguity or uncertainty 0.50 = genuinely ambiguous, partially supported, or visually uncertain 0.25 = mostly unsupported 0.00 = clearly contradicted or absent Use the full continuous range when appropriate. The score must measure SUPPORT FOR THE SPECIFIC ATOMIC CLAIM, not generic image-caption similarity. Pay particular attention to: - entity identity; - attribute binding; - subject/object roles; - action participants; - spatial ordering; - counts; - negation; - same-class entity binding. Do not give a high score merely because all relevant words or objects appear somewhere in the image. -------------------------------- SHORT EXPLANATION -------------------------------- For each atomic claim provide ONE concise evidence-grounded sentence. The explanation should state the visually relevant fact that justifies the score. Good examples: "Two dogs are visible, while the claim requires three." "The red shirt is worn by the man, matching the claim." "The dog is to the right of the cat, not to its left." "The woman is holding the umbrella rather than the man." Avoid: - generic descriptions; - unnecessary prose; - hidden chain-of-thought; - repeating the numerical score; - mentioning coordinate values in the explanation. The explanation must agree with both the score and the returned evidence boxes. -------------------------------- OVERALL SCORE -------------------------------- After ALL atomic parts have been evaluated, return: overall_score in: [0.0, 1.0] The overall score must be derived from your completed part-level judgments. IMPORTANT: Do NOT independently ignore the part scores and make an unrelated holistic judgment. The overall score should reflect how well the caption as a whole is supported by the image given the atomic-part scores. A strongly contradicted essential claim --- especially an incorrect count, binding, action, relation, spatial relation, or negation --- should meaningfully lower the overall score. At the same time, do not invent an external formula or expose chain-of-thought. The final overall score is YOUR end-to-end assessment based on the part-level results you produced above. -------------------------------- OUTPUT FORMAT -------------------------------- Return ONLY valid JSON. No Markdown. No code fences. No text before or after the JSON. Use exactly this top-level structure: "atomic_parts": [ "id": "p1", "claim": "atomic visual claim", "type": "object", "evidence_boxes": [ "label": "entity name", "role": "target", "box": [0, 0, 1000, 1000] ], "score": 0.95, "explanation": "One short evidence-grounded sentence." ], "overall_score": 0.90 Requirements: -`atomic_parts` must contain at least one item. - IDs must be p1, p2, p3, ... in order. - Every claim must have exactly one score. - Every claim must have exactly one explanation. - Every score must be within [0,1]. - Every box coordinate must be within [0,1000]. -`evidence_boxes` may be [] only when no meaningful localized visual evidence exists. -`overall_score` must be within [0,1]. -`overall_score` must be produced after considering all atomic-part judgments. Return only the JSON object. VLM-as-a-Judge (Decomposed, End-to-End) Figure 9: Prompt used for the decomposed end-to-end VLM-as-a-Judge setting. The model per- forms atomic decomposition, evidence localization, claim-level scoring, short explanation genera- tion, and final overall scoring in a single response. 31 Preprint A.6AGITA-BENCH A.6.1BENCHMARK CONSTRUCTION AGITA-Bench contains 600 image-text pairs, split equally between I2T and T2I evaluation. The 300 I2T cases use 100 real images each from ADE20K (Zhou et al., 2017), HICO-DET (Chao et al., 2018), and Objects365 (Shao et al., 2019). Their captions cover high, partial, and low alignment. The 300 T2I cases use original generation prompts paired with 181 images from DiffusionDB (Wang et al., 2023b) and 119 from MJHQ-30K (Li et al., 2024b). We construct 200 easy, 200 medium, and 200 hard cases based on semantic complexity rather than alignment severity. Fixed rules assign difficulty using the number and composition of atomic claims. Easy cases contain few simple claims. Medium cases add relations or attribute and count assign- ments. Hard cases contain more constraints or require correct binding of attributes, counts, roles, or spatial relations. The resulting benchmark contains 8,735 atomic claims. AGITA-Bench is excluded from training, calibration, and model selection. Atomic decomposition prompt. AGITA-Bench captions are decomposed into typed visual se- mantic units using GPT-5.6 Sol with a fixed caption-only prompt. The model does not receive the image during decomposition. It is instructed to preserve entity and attribute bindings, counts, ac- tions, relations, spatial relations, and negation while avoiding unobservable or stylistic content. The complete prompt is shown in Figure 10. A.6.2HUMAN ANNOTATION PROTOCOL AGITA-Bench is annotated at two complementary levels. For each atomic claim, annotators as- sign an image-support score from 0, meaning contradicted or not supported, to 4, meaning fully supported. A separate cannot judge option is available when the visual evidence is insufficient for a reliable decision. Each image-text pair is also rated for global alignment from 1, meaning not aligned, to 5, meaning fully aligned. Decomposition quality is judged independently of image sup- port. A claim is reasonable as written when it is a clear and non-redundant atomic proposition that requires neither splitting nor merging. This separates decomposition errors from genuine image-text mismatches. AGITA-Bench provides human score annotations rather than human bounding-box annotations. Any localized evidence boxes reported by VISEXMEM are produced by the metric and are not part of the benchmark’s human labels. All seven annotators independently annotate the same 30 image-text pairs, comprising 460 unique atomic claims, for inter-annotator agreement analysis. The remaining 570 pairs are each assigned to one annotator. Pairwise quadratic-weighted Cohen’s κ is reported in Tables 10 and 11. Aggregate annotator demographics are reported in Table 12. A.6.3ANNOTATION INTERFACE, GUIDELINES AND EXAMPLES Annotations are collected through a dedicated interface that presents the same information and rating controls to every annotator. Figure 11 shows the dashboard and sample-level interface. Before start- ing, annotators can review the annotation guide and worked examples. For each assigned sample, they record decomposition quality and atomic-support judgments for individual claims alongside the image and input text. Global image-text alignment is rated once for the complete sample. The full annotator instructions are shown in Figure 12. We also provide four worked examples covering high, partial, and weak image-text alignment, in- cluding one example that shows the difference between decomposition quality and visual support. Each example includes the input, global alignment judgment, and claim-level decomposition and support annotations. 32 Preprint Table 10:Inter-annotator agreement for global alignment. Pairwise quadratic-weighted Cohen’s κ on the 30 shared image–text pairs; mean pairwise κ = .602. Only unique annotator pairs are shown. Ann2 Ann3 Ann4 Ann5 Ann6 Ann7 Ann1.57.78.54.76.37.62 Ann2.62.70.64.54.48 Ann3.73.81.39.62 Ann4.63.47.61 Ann5.51.65 Ann6.61 Table 11:Inter-annotator agreement for atomic support. Pairwise quadratic-weighted Cohen’s κ on shared claims; cannot judge re- sponses are excluded pairwise (N = 418–453). Mean pairwise κ = .564. Ann2 Ann3 Ann4 Ann5 Ann6 Ann7 Ann1.52.72.61.60.47.39 Ann2.59.60.58.59.55 Ann3.65.58.67.47 Ann4.55.55.49 Ann5.56.45 Ann6.66 Table 12: AGITA-Bench annotator demographics. De- mographic summary of the seven human annotators (N = 7). AttributeCategoryCount NationalityGerman3 Indian1 Chinese1 UAE2 EducationPostdoctoral researcher1 PhD student2 Master’s student4 Age18–254 26–302 > 301 33 Preprint You split a caption into VISUAL SEMANTIC UNITS for a compositional benchmark. You receive ONLY the caption text. You have not seen any image. Never add anything the caption does not state, and never judge whether it is true. THE UNITS ARE PHRASES, NOT SENTENCES. Stay close to the caption's own wording. Do not rewrite meaning into full propositions. This is the single most important rule. caption: "A person is moving green hay towards an elephant that is inside the back of a white truck." CORRECT type person object hay object green hay attribute elephant object truck object white truck attribute moving action green hay towards an elephant relation elephant inside the back of a white truck spatial WRONG -- do not produce this A person exists. Hay exists. The hay is green. An elephant exists. A truck exists. The truck is white. caption: "Dozens of mostly red apples hang from the branches of a leafy green fruit tree." CORRECT apples object dozens of apples count mostly red apples attribute branches object fruit tree object leafy fruit tree attribute green fruit tree attribute hang action apples hang from the branches of a fruit tree relation WRONG There are dozens of apples. Most of the apples are red. The fruit tree is leafy. The fruit tree is green. BINDING IS THE POINT. An attribute, count or role must stay attached to the entity the caption gives it. "The woman wears blue and the man wears red." woman / man / woman wears blue / man wears red NOT: woman / man / blue / red "The dog chases the cat." -> dog chases cat (dog is the agent; never "cat chases dog") "one black dog and one white dog" -> dogs / black dog / white dog "A red cup is to the left of a blue bowl." -> red cup / blue bowl / red cup left of blue bowl "A man without a hat" -> man / man without a hat (keep the negation) TYPES -- use exactly these: object a visible entity, as a compact phrase: person, dog, fruit tree, branches attribute a property bound to its entity: green hay, white truck, mostly red apples count a quantity bound to its entity: three dogs, dozens of apples action what is happening, compactly: moving, hang, chases, holding relation who/what relates to whom/what: apples hang from the branches spatial an arrangement, direction preserved: cup left of bowl, cat under table negation an absence or denial: no dog, man without a hat other visually checkable, none of the above GRANULARITY -- the smallest USEFUL unit, not every lexical combination. "leafy green fruit tree" -> fruit tree, leafy fruit tree, green fruit tree NOT also: tree, leafy tree, green tree, leafy green tree, leafy green fruit tree But "dog" AND "black dog" are BOTH wanted: presence and attribute are different checks. Likewise "dogs" AND "three dogs". OMIT entirely: style, medium, quality and render words (8k, cinematic, unreal engine, masterpiece, trending on artstation, photorealistic, octane render, artist names), camera and lens jargon, aspect ratios, mood words with no visual test (epic, dreamy), and unobservable facts about a subject (their nationality, their profession when not visible). Return STRICT JSON, no prose: "parts": ["text": "...", "type": "object", "text": "...", "type": "attribute"] Do not number the parts and do not invent ids. AGITA-Bench Atomic Decomposition Figure 10: Prompt used for AGITA-Bench atomic decomposition. GPT-5.6 Sol receives only the caption and decomposes it into typed visual semantic units without access to the image or making any image-based correctness judgment. 34 Preprint Figure 11: AGITA-Bench annotation interface. Dashboard and sample-level interface used for human annotation. 35 Preprint Figure 12: AGITA-Bench annotation guidelines. Complete annotator-facing guidelines. 36 Preprint Example 1: High alignment High alignment Input text A man works on a red bicycle in a workshop with tools hanging on a green pegboard. Global alignment score 5 / 5 #TypeAtomic claimDecomp. qualityAtomic support 1objectmanReasonable4 2objectbicycleReasonable4 3attributered bicycleReasonable4 4objectworkshopReasonable4 5objecttools Reasonable4 6objectpegboard Reasonable4 7attributegreen pegboardReasonable4 8actionman works on a red bicycle Reasonable4 9spatialman in a workshop Reasonable4 10relationtools hanging on a green pegboard Reasonable4 • Overall rationale The image clearly supports the man, red bicycle, workshop, green pegboard, tools, and the described actions and relations. 37 Preprint Example 2: Partial alignment Partial alignment Input text A black dog lies on a patterned sofa beside several pillows, with a red ball near its front paws. Global alignment score 3 / 5 #TypeAtomic claimDecomp. qualityAtomic support 1objectdogReasonable4 2attributeblack dogReasonable0 3objectsofaReasonable4 4attributepatterned sofaReasonable4 5objectpillowsReasonable4 6countseveral pillowsReasonable4 7objectballReasonable0 8attributered ballReasonable0 9objectfront pawsReasonableCannot judge 10actiondog lies on a patterned sofaReasonable4 11spatialdog beside several pillowsReasonable4 12spatialred ball near the dog’s front pawsReasonable0 Annotation note. p9 (front paws): The dog’s front paws are sufficiently obscured that this standalone part cannot be confidently judged from visible evidence. Overall rationale The dog, patterned sofa, pillows, and lying position are well supported, but the dog is brown rather than black and no red ball is visible. 38 Preprint Example 3: Weak alignment Weak alignment Input text A Shiba Inu wearing a red beret and a blue turtle- neck sits in front of a green wooden wall. Global alignment score 2 / 5 #TypeAtomic claimDecomp. qualityAtomic support 1objectShiba InuReasonable4 2objectberetReasonable4 3attributered beretReasonable0 4objectturtleneckReasonable4 5attributeblue turtleneckReasonable0 6objectwallReasonable4 7attributegreen wallReasonable0 8attributewooden wallReasonable0 9relationShiba Inu wearing a red beretReasonable2 10relationShiba Inu wearing a blue turtleneck Reasonable2 11actionShiba Inu sitsReasonableCannot judge 12spatialShiba Inu in front of a wallReasonable4 Annotation notes. p9 (Shiba Inu wearing a red beret): Partially supported: the dog IS wearing a beret, but the required red attribute is wrong. A composite relation that is partly correct but contains an important mismatch scores 2, not 0. p10 (Shiba Inu wearing a blue turtleneck): Partially supported for the same reason: the turtleneck is present but black rather than blue. p11 (Shiba Inu sits): The lower body is outside the frame, so the sitting posture cannot be judged confidently. Overall rationale The Shiba Inu, beret, turtleneck, and wall are present, but the beret is black rather than red, the turtle- neck is black rather than blue, and the wall is light-colored brick rather than green wood. The lower body is outside the frame, so the sitting posture cannot be judged confidently. 39 Preprint Example 4: Decomposition quality Decomposition quality Teaching point. Decomposition quality and image support are independent: this example pairs every decomposition label with a high support score, and a Reasonable decomposition with a support score of 0. Input text A woman in a red shirt is holding a blue umbrella beside a brown dog, while standing on a wet sidewalk in front of a yellow bus. Global alignment score 4 / 5 #TypeAtomic claimDecomp. qualityAtomic support 1objectwomanReasonable4 2attributered shirtReasonable4 3attributeblue umbrella Reasonable4 4attributebrown dogReasonable4 5attributewet sidewalk Reasonable4 6objectbusReasonable4 7attributeyellow bus Reasonable0 8relationwoman holding a blue umbrellaReasonable4 9spatialwoman beside a brown dog Reasonable4 10relationwoman standing on a wet sidewalkReasonable4 11spatialwoman in front of a yellow busReasonable2 12relationwoman in a red shirt holding a blue umbrella beside a brown dog Needs split4 13actionholdingNeeds merge4 14attributeshirt that is redRedundant4 15actionsmiling woman Not entailed4 Annotation notes. p7 (yellow bus): The text correctly decomposes to ‘yellow bus’, but the visible bus is red. p11 (woman in front of a yellow bus): The woman is in front of a bus, but the required yellow attribute is incorrect. p12 (woman in a red shirt holding a blue umbrella beside a brown dog): It combines several independently checkable claims, although the complete statement is visually supported. p13 (holding): The action is visually present, but as a decomposition it loses who is holding what. p14 (shirt that is red): It repeats the visual claim already represented by ‘red shirt’. p15 (smiling woman): The original text never says the woman is smiling, but the image clearly shows her smiling. Overall rationale The woman, red shirt, blue umbrella, brown dog, wet sidewalk, and described actions and spatial relations are clearly supported. The meaningful mismatch is the bus colour: the text specifies a yellow bus, while the image shows a red bus. 40 Preprint Table 13: Inter-annotator agreement for explanation and evidence evaluation. Values show mean pairwise agreement. Ordinal ratings use quadratic-weighted Cohen’s κ, while Unsupported Content uses unweighted κ. EvaluationMean κWithin±1 (%) Task A explanation quality.5989.5 Unsupported Content.42– Task B localized evidence.5886.9 A.7EXPLANATION AND EVIDENCE ANNOTATION Protocol. The study contains 300 image–text pairs. Each pair presents four explanations, one each from EXPERT, VIEScore, GPT-5.6 Sol, and VISEXMEM, for 1,200 sample–system explanation instances. System identities are hidden and the explanations are shown under randomized labels A/B/C/D. Annotation has two stages. In Task A, annotators rate all four explanations without seeing localized evidence. They assess diagnostic correctness, visual grounding, clarity, and diagnostic usefulness on a 1 to 5 scale, and flag unsupported factual content. For systems that provide localized evidence, Task B is shown only after Task A is complete. Annotators then rate up to three claim- region pairs for localization accuracy and evidence sufficiency, again on a 1 to 5 scale. Task A responses are locked before Task B is shown so the highlighted regions cannot affect the earlier explanation ratings. The complete questions, scales, and instructions are reproduced in Figure 13. Annotators and agreement. Three annotators took part in the study. Their reported countries of nationality were Germany, India, and the UAE. Two were aged 18 to 25 and one was aged 26 to 30, and all three were Master’s students. All three annotated the same 30 image–text pairs. Agree- ment is computed pairwise on items completed by both annotators in each pair after resolving the randomized labels to their underlying systems. For the ordinal ratings, we use quadratic-weighted Cohen’s κ, while Unsupported Content uses unweighted Cohen’s κ. Across the four explanation- quality criteria, mean pairwise weighted κ is .59, with 89.5% of paired ratings within one scale point. Unsupported Content gives κ = .42. For localized evidence, mean weighted κ is .58, with 86.9% within-one-point agreement. Because the evidence ratings are strongly concentrated near the top of the scale, we report near-agreement together with κ. 41 Preprint A· WHAT YOU ARE DOING For every sample you see an image and the caption being evaluated. Four different automatic systems each wrote a short explanation of whether the image supports the caption. The explanations are shown only as Explanation A / B / C / D; the system identity is hidden and the order varies across samples. 1. Task A — rate each of the four explanations, one at a time. No highlighted regions are shown yet. 2. Task B — after all four explanations are rated, a second panel opens only for explanation(s) whose system provides a highlighted image re- gion for a claim. Depending on the sample, this may apply to none, one, or multiple explana- tions. Up to three highlighted regions are rated. Once Task B is revealed, the corresponding Task A answers are locked and can no longer be changed, keeping the two judgments indepen- dent. Not every explanation has a Task B. Some systems do not produce localized image regions; this is expected and should not be flagged. B· TASK A — RATING ONE EXPLANATION Diagnostic Correctness “How accurately does the explanation identify the relevant matches or mis- matches between the image and cap- tion?” 1 Completely incorrect · 2 Mostly in- correct · 3 Partially correct · 4 Mostly correct · 5 Completely correct Visual Grounding “How well are the statements in the explanation supported by what is visibly present in the image and stated in the caption?” 1 Not supported · 2 Mostly unsupported · 3 Partially supported · 4 Mostly sup- ported · 5 Fully supported Clarity“How easy is the explanation to under- stand, without unnecessary or confusing information?” 1 Very unclear · 2 Somewhat unclear · 3 Neither clear nor unclear · 4 Clear · 5 Very clear Diagnostic Usefulness Primary question. “How useful is the explanation for understanding why the image and caption align or fail to align?” 1 Not useful · 2 Slightly useful · 3 Moderately useful · 4 Very useful · 5 Extremely useful Unsupported Content “Does this explanation contain any spe- cific factual statement that is contradicted by, or unsupported by, the image/cap- tion?” Yes / No / Unsure Judge every explanation against the image yourself. C· TASK B — RATING HIGHLIGHTED EVIDENCE For each qualifying explanation, annotators evaluate up to three claims, each accompanied by its own highlighted image region and short claim text. Evidence Localization Accuracy “How accurately do the highlighted region(s) identify the visual evidence relevant to this claim?” 1 Completely incorrect · 2 Mostly in- correct · 3 Partially correct · 4 Mostly correct · 5 Completely correct Evidence Sufficiency “How sufficient are the highlighted re- gion(s) for deciding whether this claim is supported by the image?” 1 Completely insufficient · 2 Mostly insufficient · 3 Partially sufficient · 4 Mostly sufficient · 5 Fully sufficient D· SKIP FOR NOW / TECHNICAL ISSUE Skip for nowPostpone the whole sample. It remains in the annotator’s assignment and is marked as unfinished on the dashboard until com- pleted. Technical issueUse only when something is actually bro- ken, such as an image failing to load, garbled text, or the annotation interface malfunctioning. E· NAVIGATING, SAVING AND EXPORTING • Work saves automatically; annotators may close the tab and return later. • Completed samples can be reopened, but after Task B has been revealed for a sample, its Task A answers remain locked. Task B ratings may still be revised. • Annotators are asked to use the same browser and device where practical and not clear site data before exporting. • A backup can be downloaded from the menu at any time. After completing all assigned samples, annotators use Export final annotations to pro- duce the final annotation file. Annotation order Image + Caption −→ Task A: Explanation −→ Lock Task A −→ Task B: Evidence HUMAN EXPLANATION & EVIDENCE ANNOTATION GUIDELINES Figure 13: Human explanation and evidence annotation guidelines. Complete instructions pro- vided to annotators. 42

Instant research discovery