Paper deep dive

FuzzingRL: Reinforcement Fuzz-Testing for Revealing VLM Failures

Jiajun Xu, Jiageng Mao, Ang Qi, Weiduo Yuan, Alexander Romanus, Helen Xia, Vitor Campagnolo Guizilini, Yue Wang

Year: 2026Venue: arXiv preprintArea: cs.LGType: PreprintEmbeddings: 50

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 7/21/2026, 3:47:28 AM

Summary

The paper introduces FuzzingRL, a framework for automatically revealing vulnerabilities in Vision-Language Models (VLMs) by combining vision-language fuzzing with adversarial reinforcement fine-tuning. The method generates diverse, challenging queries to induce incorrect responses, effectively lowering model accuracy. The approach demonstrates transferability across different VLM architectures and scales.

Entities (8)

FuzzingRL · methodframework · 98%Vision-Language Models · model-class · 95%Adversarial Reinforcement Fine-Tuning · technique · 93%Vision-Language Fuzzing · technique · 93%Qwen2.5-VL-32B · model · 92%Qwen2.5-VL-7B · model · 90%GPT-4o · model · 88%Direct Preference Optimization · algorithm · 85%

Relation Signals (6)

FuzzingRL → targets → Vision-Language Models

confidence 98% · FuzzingRL, a framework that aims to automatically expose and amplify the weaknesses of vision-language models.

FuzzingRL → uses → Vision-Language Fuzzing

confidence 95% · The framework consists of two synergistic components: vision-language fuzzing for systematic input diversification and adversarial reinforcement finetuning...

FuzzingRL → uses → Adversarial Reinforcement Fine-Tuning

confidence 95% · The framework consists of two synergistic components: ... and adversarial reinforcement finetuning for adaptive vulnerability discovery.

Qwen2.5-VL-32B → testedby → FuzzingRL

confidence 92% · the accuracy of Qwen2.5-VL-32B on our generated questions drops from 86.58% to 65.53% in four RL iterations.

Qwen2.5-VL-7B → usedas → Fuzzing Model

confidence 90% · We selected the Qwen2.5-VL-7B model as the fuzzing model...

FuzzingRL → employs → Direct Preference Optimization

confidence 85% · preference pairs are used for DPO training to update the fuzzing model.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Vision Language Models (VLMs) are prone to errors, and identifying where these errors occur is critical for ensuring the reliability and safety of AI systems. In this paper, we propose an approach that automatically generates questions designed to deliberately induce incorrect responses from VLMs, thereby revealing their vulnerabilities. The core of this approach lies in fuzz testing and reinforcement finetuning: we transform a single input query into a large set of diverse variants through vision and language fuzzing. Based on the fuzzing outcomes, the question generator is further instructed by adversarial reinforcement fine-tuning to produce increasingly challenging queries that trigger model failures. With this approach, we can consistently drive down a target VLM's answer accuracy -- for example, the accuracy of Qwen2.5-VL-32B on our generated questions drops from 86.58\% to 65.53\% in four RL iterations. Moreover, a fuzzing policy trained against a single target VLM transfers to multiple other VLMs, producing challenging queries that degrade their performance as well.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

48,819 characters extracted from source content.

Expand or collapse full text

FuzzingRL: Reinforcement Fuzz-Testing for Revealing VLM Failures Jiajun Xu 1 , Jiageng Mao 1† , Ang Qi 1 , Weiduo Yuan 1 , Alexander Romanus 1 , Helen Xia 1 , Vitor Campagnolo Guizilini 2 , and Yue Wang 1† 1 University of Southern California 2 Toyota Research Institute † Equal advising. Abstract.Vision-Language Models (VLMs) are prone to errors, and identifying where these errors occur is critical for ensuring the reliability and safety of AI systems. In this paper, we propose an approach that au- tomatically generates questions designed to deliberately induce incorrect responses from VLMs, thereby revealing their vulnerabilities. The core of this approach lies in fuzz-testing and reinforcement fine-tuning: we transform a single input query into a large set of diverse variants through vision and language fuzzing. Based on the fuzzing outcomes, the question generator is further instructed by adversarial reinforcement fine-tuning to produce increasingly challenging queries that trigger model failures. With this approach, we can consistently drive down a target VLM’s answer accuracy—for example, the accuracy of Qwen2.5-VL-32B on our generated questions drop from 86.58% to 65.53% in four RL iterations. Moreover, a fuzzing policy trained against a single target VLM transfers to multiple other VLMs, producing challenging queries that degrade their performance as well. 1 Introduction Recently, Vision–Language Models (VLMs) have been advancing rapidly and are now widely adopted as the visual backbone in domains such as Vision–Language–Action (VLA) systems, world models, and AI agents. However, issues such as uneven token weight distribution in attention mechanisms, textual bias in the LLM component, and misalignment between the vision encoder and the language model have led to various forms of hallucination in VLMs. As VLMs become the core component of multimodal systems and autonomous agents, their errors can directly cause decision failures, bias propagation, or even safety risks. Therefore, ensuring the robustness and controllability of VLMs is crucial for the development of trustworthy multimodal intelligence. Therefore, discovering failures of VLMs becomes increasingly important. Prior work has primarily designed static benchmarks that assess model capability with specific items, including broad coverage suites [42,43] , multi-dimensional objective evaluations [18, 22, 46], assessments closer to open domain QA and complex scenes [20,41], as well as specialized datasets [23,32] and defect-diagnostic lines [17, 25, 47, 48]. While these efforts substantially improve coverage and arXiv:2603.06600v1 [cs.LG] 17 Feb 2026 2Authors Suppressed Due to Excessive Length diagnosability, mainstream evaluations often require humans to first identify specific shortcomings of the VLM, and then construct a benchmark targeting those weaknesses. They are largely static and rely heavily on humans to manually reveal VLM failures, making it difficult to adaptively focus on the truly high- failure regions within the vast vision–language combinatorial space. This raises a fundamental question: can we design a framework that au- tonomously discovers failures in VLMs? To explore this, we decompose the problem into two key challenges. First, we must generate inputs to the VLM that are sufficiently diverse to ensure broad coverage of the vast vision–language space. Second, given the large set of generated question answer pairs, we need a mecha- nism to adaptively guide the input generation process toward the model’s most vulnerable regions, and continuously refine it to produce increasingly challenging queries that can reliably expose VLM failures. To this end, we propose FuzzingRL, a framework that aims to automatically expose and amplify the weaknesses of vision-language models. The framework consists of two synergistic components: vision-language fuzzing for systematic input diversification and adversarial reinforcement finetuning for adaptive vul- nerability discovery. Our framework is inspired by the concept of fuzzing (or fuzz testing) in software engineering, which expands a small set of seed inputs into many systematically varied test cases using automated templates. In a similar spirit, we propose vision-language fuzzing to generate diverse variants of a single input query in both the vision and language domains. For example, given a seed image of a “red apple", visual perturbations may include transformations such as flipping or color adjustments, while linguistic perturbations convert the query “What color is the apple?” into variants like “What color is the apple not?” or “Is the apple red or green?”. By generating such systematic perturbations, vision- language fuzzing constructs a large and diverse family of test cases, enabling the discovery of latent vulnerabilities that would remain hidden under isolated inputs. However, since VLM vulnerabilities are highly heterogeneous and the query space is vast, relying solely on fixed transformation templates provides limited coverage. To address this, we introduce a fuzzing model trained via adversarial reinforcement learning to adaptively explore the most failure-prone regions of the target model. During training, incorrect predictions are assigned higher rewards, driving the fuzzing model to generate increasingly challenging queries that effectively probe the weaknesses of the VLM. Through iterative refinement, FuzzingRL progressively sharpens its probing capability, leading to a targeted and incisive discovery of model failures. The framework outputs reproducible, auditable failure cases with full meta- data and aggregates them into an attributable error profile, thereby achieving automatic, reproducible, and scalable discovery and localization of systematic VLM failures with minimal human involvement under fixed budgets. We selected the Qwen2.5-VL-7B model as the fuzzing model and the Qwen2.5-VL-32B model as the target model. After four iterations of training, the target model’s accuracy drops from 86.58% to 65.53%. FuzzingRL3 Dimension / Subdimension: State / Action Visual Perturbation Question: Is the elephant moving towards the water? Test Model’s Answer: original: No Original imageFuzzed image flip Image: fuzzed: Yes Dimension / Subdimension: Space & Structure Understanding / Relative Relations Compositional Reasoning Original question: Is the butterfly closer to the lemon or to the banana? Image: Fuzzed question: Is the butterfly closer to the lemon or to the banana in the bowl? Test Model’s Answer: original: Banana fuzzed: Lemon Dimension / Subdimension: Attribute Understanding / Material Linguistic Paraphrasing Original question: What is the material of the refrigerator’s surface? Image: Fuzzed question: What is the material of the surface that h- as numerous magnets and photos attached? Test Model’s Answer: original: Metal fuzzed: Refrigerator Dimension / Subdimension: Existence & Recognition / Scene Recognition Counterfactual Reasoning Fuzzed question: How many people are flying kites in the image? Test Model’s Answer: One Explain: There are actually three people flying kites in the image, but the model tends to focus on the main subject, which is a boy with a kite, so it makes mistakes. Image: Dimension / Subdimension: Relations & Interactions / Human- Object Interaction Discourse Logic Original question: Is it true that the person is using a phone to take a mirror selfie? Image: Fuzzed question: Isn't it true that the per- son is not using a phone to take a mirror selfie? Test Model’s Answer: original: No fuzzed: Yes Dimension / Subdimension: Space & Structure Understanding / Position/Orientation Spatial Reasoning Image: Fuzzed question: Which is closer to the camera, the clock or the bus? Test Model’s Answer: The clock Explain: The bus is closer to camera than the clock. Model usually make mistakes when it focus on depth. Dimension / Subdimension: Scene Semantics & Abstract Reasoning / Causal/Temporal Reasoning Contextual Bias Image:Fuzzed question: If the batter hits the ball and it lands outside the field, is it a foul ball? Test Model’s Answer: Yes Explain: In the baseball game, a ball landing out may not be a foul ball. The model must have additional to answer it. Dimension / Subdimension: Quantity & Comparison / Count Hypothetical Reasoning Original question: Are there two donuts in total? Image: Fuzzed question: If another donut like the one on the plate is placed next to the cup, are there three donuts in total? Test Model’s Answer: original: No fuzzed: Yes Fig. 1: FuzzingRL probe examples. For a given capability subdimensiond, each panel shows an answerable probe generated by a specific fuzzing role and the answer from the target model (Qwen2.5-VL-32B). We further study transferability: after training on a single target VLM, the resulting fuzzing model can be reused to test other VLMs. Concretely, we fix the trained fuzzing model and use it to produce a held out set of image grounded probes, then query multiple test VLMs with the same questions and images and report their accuracies under a unified human annotation rubric. Across diverse architectures and scales, the generated probes consistently reduce test model accuracy. We also track this cross model effect through training iterations on a fixed panel of held-out test VLMs, and use it to select a checkpoint that avoids overfitting to the target model. Additionally, our probes reveal recurring failure patterns, including spatial reasoning, counting, and sensitivity to instruction phrasing. 2 Related Work Static Benchmarks. Most existing VLM evaluations rely on static test suites that span general capabilities and multiple task domains: general multimodal suites [22,23,42,46] assess overall ability; classic vision datasets [7,11,16,19,49] ground detection and scence recognition; QA datasets [10,12] anchor answers in image evidence; composition and relations [36,40] probe structured reasoning and alignment; counting and comparison [1,39] capture number sensitivity; interaction and events [6,14,37] evaluate actions and roles; test and chart understanding [4,27,28,34,35] test reading and layout. Robustness lines of work typically apply 4Authors Suppressed Due to Excessive Length functional perturbations (flip/rotation, crop/occlusion, compression/noise) and linguistic edits (paraphrase, negation, conditional augmentation) to test answer stability [2,13,26,31,47]. However, such static evaluations share two limitations: (i) costly labeling, one-shot test banks, and different designs that weaken out of distribution comparability; (i) little use of observed failure signals to generate new test items, making it hard to continuously expose model weaknesses. In contrast, we actively explore VLM failure-prone regions, and our test set updates dynamically as new weaknesses are discovered. Generative Benchmark. To break the static bottleneck, recent work pursues programmatic dynamic test generation: Task Me Anything [44] expands the task space via templates and asset libraries; ProVision [45] scales up visual task; Open- ended VQA [9] automatically convert classification taxonomies into open-ended VQA; earlier CLEVR [15] provides a controllable synthetic pipeline. However, common gaps remains: (i) heavily relying on templates and synthetic assets, which limits transfer to real images; (i) most method rely on a single signal, so they miss many skills from low-level perception to high-level relational reasoning; (i) lets test composition dominate outcomes, so scores reflect question allocation rather than true model ability. In contrast, our fuzz-testing approach searches a much broader space for failures and is model-targeted, adapting tests to the specific model’s errors. Unlike coverage-oriented programmatic generators and classical coverage guided fuzzing that rely on predefined heuristics, FuzzingRL closes the loop by using failure feedback to learn a transferable fuzzing model that progressively produces harder queries across models. Fuzz Testing. Classical fuzz testing originated in systems reliability, where randomized inputs were injected into UNIX utilities to expose crashes and anomalous behavior [29,30]. Coverage-guided greybox fuzzing later scaled this paradigm by instrumenting targets and using branch-coverage feedback to steer input mutation and exploration (e.g., AFL and libFuzzer), markedly improving efficiency under tight budgets [5,33]. This establishs a general “mutate–measure– explore” loop for efficiently searching vast input spaces. Large Model Safety. Recent work adapts fuzzing style, failure seeking evaluation to large models. On the text side, adversarial suffixes and automated optimization repeatedly elicit unsafe or policy violating outputs from aligned LLMs, revealing prompt fragility and transferability [8,38,50]. On the multimodal side, studies demonstrate that images can carry hidden instructions artifacts that trigger unsafe behaviors, motivating systematic safety testbeds and attack taxonomies [3,21,24]. Together, these results underscore the limits of one-shot static suites and the need for continually generated, judgeable tests with explicit accounting of what capabilities are probed. 3 Method 3.1 Overview Our method aims to actively find VLM failures across multiple dimensions, rather than static benchmark targeting one specific area. At a high level, our goal is FuzzingRL5 to learn the question generatorπ θ to maximize the expected failure rate of the target VLMfunder diverse visual and linguistic conditions. Formally, for images x∈X, we define the global objective as max θ E x∼X,q∼π θ (·|x,d,r) h J x,q,f(x,q) | z proxy for failure rate i (1) Here,J(x,q, ˆa)∈ −1,0,1is the judgment signal assigning higher values to incorrect answers of target model. For the question generatorπ θ (· | x,d,r), d ∼ p(d) andr ∼ p(r) denote the sampling priors over fuzzing subdimensions and roles in vision-language fuzzing (Section 3.2). This objective encouragesπ θ to generate diverse, valid questions that maximize the likelihood of VLM failure, while remaining controllable across multiple fuzzing dimensions. To optimize Eq. 1, we instantiate judgeability and coverage via a vision-language fuzzing module, and then train the policyπ θ with adversarial reinforcement finetuning to prefer higher-utility questions. We will introduce the vision-language fuzzing module in Section 3.2 and the adversarial reinforcement finetuning ofπ θ in Section 3.3. 3.2 Vision-Language Fuzzing Fuzz testing, or fuzzing, is a method for discovering system vulnerabilities. It works by automatically generating a large number of diverse cases to test a system, expecting to trigger and catch the failures that can indicate the weaknesses of the system. This method expands the probing space and find the failure cases automatically. In this paper, we propose a novel approach to adapt the concept of fuzzing to the realm of vision and language, termed vision–language fuzzing. Our goal is to systematically generate semantically valid and diverse variants of input queries that can reveal inconsistencies or brittleness in vision–language models. To apply fuzz testing for VLMs, we ground our vision-language fuzzing in 24 subdimensionsd, which map to the key abilities VLMs are commonly tested on. Across these subdimensions, we apply 8 fuzzing rolesrto form simple and structured variations in how questions are formed. The concrete examples that the vision-language fuzzing system generates are shown in Figure 1. We organize 24 subdimensions into 7 capability groups (Table 1). Difficulty styles are instantiated by eight fuzzing roles (Table 2); each probe is labeled by (d,r) for controllable difficulty and attribution. We introduce the fuzzing roles as follows: Visual Perturbation. We apply semantics-preserving visual transforms (e.g., flips, mild noise) while keeping the question unchanged. A failure is an answer change that should be invariant to such transforms (e.g., a count differs after a horizontal flip). Linguistic Paraphrasing. We rewrite the question via synonym substitution and syntactic alternations while keeping the visual evidence and answer type fixed. A failure is sensitivity to surface form across meaning-equivalent paraphrases. 6Authors Suppressed Due to Excessive Length Table 1: Capability dimensions d for stress-testing VLMs Capability groupSubdimensions Existence & Recognition• Object Presence • Scene Recognition • Person/Animal Presence Attributes• Color • Material • Size (Relative) • State/Action Spatial & Structural• Position/Orientation • Relative Relations • Part–Whole Hierarchy • Occlusion/Perspective Quantity & Comparison• Count (1,2,3. . . ) • Fuzzy Quantity (few/many/some) • Comparison/Ranking Relations & Interactions• Human–Object Interaction • Human–Human Interaction • Event Recognition Scene-semantic Composition & Abstract Reasoning • Multi-object Reasoning • Task/Scene Identification • Causal/Temporal Reasoning • Intention/Goal Recognition Symbols & Pragmatics• Text (OCR) Recognition & Understanding • Symbol/Sign Meaning • Pragmatic/Social Cues Discourse Logic. We wrap a base query with discourse operators (negation, entailment, concessives) designed to preserve the intended truth conditions. Failures include polarity flips or logically inconsistent answers under equivalent rewrites. Contextual Bias. We add plausible but unsupported commonsense distrac- tors to test whether answers remain grounded in the image rather than priors. A failure is an overconfident fabricated claim when the image does not provide sufficient evidence. Compositional Reasoning. We form multi-constraint queries that jointly require multiple grounded attributes and/or relations (e.g., color + position + size). A failure is partial grounding where the model satisfies one constraint but ignores or misbinds another. Counterfactual Reasoning. We probe prior-defying yet visually explicit configurations to test whether the model overrides strong priors with direct perception. A failure is defaulting to commonsense against clear visual evidence (e.g., insisting on “five” for a visible six-finger hand). Spatial Reasoning. We ask depth/occlusion/perspective questions that require 3D reasoning beyond 2D layout cues (front–back ordering, relative depth). FuzzingRL7 Table 2: Fuzzing roles r for generating VLM queries Role (r)What it stresses Visual PerturbationRobustness to light visual transforms; avoid superficial shortcuts. Linguistic Paraphrasing Linguistic invariance without changing semantics (paraphrase/reorder robustness). Discourse LogicLogical consistency under discourse cues (negation/entailment; polarity robustness). Contextual BiasSeparate visual grounding from world knowledge; resist prior-led guessing. Compositional Reasoning Compositionality across multiple attributes/relations within one query. Counterfactual Reasoning Overcoming strong priors with visual evidence; rare or prior-violating patterns. Spatial Reasoning3D/relative depth ordering, occlusion and perspective cues. Hypothetical ReasoningOne–two-step numeric/logic reasoning over visible entities. Failures include systematic depth confusions or treating apparent size as true distance under perspective distortion. Hypothetical Reasoning. We add lightweight conditional modifications (add/remove/recolor) that require grounded mental simulation and simple infer- ence. A failure is treating the condition as irrelevant or producing inconsistent grounded updates (e.g., incorrect remaining count after removal). Equipping the question generatorπwith our 8 fuzzing rolesrmeasurably increases the diagnostic pressure on the test VLM, even before the fine-tuning stage. For instance, applying Qwen2.5-VL-72B as the in-context question gener- ator and Qwen2.5-VL-32B as the test VLM, the overall answer accuracy drops from 93.29% to 85.33% before RFT. 3.3 Adversarial Reinforcement Finetuning While the structured vision-language fuzzing system ensures broad coverage of VLM capabilities, it lacks dynamic adaptability to focus on high failure regions. Given an input imagex, a specified subdimensiondand roler, the static sampling ofd,rcombinations leads to inefficient resource allocation, as most queries may lead to low failure regions. To address this, we propose a novel adversarial reinforcement finetuning paradigm to train a fuzzing model that converges the query space to high failure regions. The core idea is to assign a high reward to the questions if the target VLM produces incorrect answers, thereby guiding the question generator to ask increasingly difficult questions. The whole training process is shown in Figure 2. Letx∈Xbe an image,d∈1,...,24a subdimension, andr ∈1,...,8a fuzzing role. The generator policyπ θ (q | x,d,r)produces a questionq. The target 8Authors Suppressed Due to Excessive Length Fuzzing Model (e.g. Qwen2.5- VL-7B) Image Base Generate Error-Prone Question Target Model Answer Reward Calculation Create Preference Pairs existence attribute relations spatial 24 subdimensions8 fuzzing roles linguistic visual Spatial Compositional DPO Training Iteration 0 (Finetuned) How close is the cat to the camera compared to the toilet? Image Subdemension: Relative Relations Reward: -1 Iteration 1 Which is closer to you, the cat or the sink? Reward: 0 Iteration 2 Is the cat closer to the camera than the sink? Yes Reward: 0 Iteration 3 If a towel is placed on the toilet, is there a towel on the toilet? No Reward: 1 High confidence: GPT-4o judger Low confidence: Human judger Yes Cat Fig. 2: Overview of FuzzingRL. A fuzzing model (e.g., Qwen2.5-VL-7B) samples from an image base organized by 24 subdimensions and 8 fuzzing roles to generate diverse, error-prone questions for a target VLM. The target’s responses are scored via reward calculation, using a GPT-4o judge when confidence is high and a human judge otherwise, and the resulting preference pairs are used for DPO training to update the fuzzing model. The bottom panel illustrates how iterative training progressively sharpens the generated queries from ordinary perception questions to more failure- inducing, compositional prompts, thereby improving the fuzzing model’s ability to surface failure cases and making it more likely to elicit incorrect answers from VLMs. model outputsˆa=f(x,q). A judgeJassigns a ternary labely ∈−1,0,1toˆa, where−1denotes unanswerable,0denotes correct, and1denotes incorrect. In our setup,Jis a committee composed of GPT-4o and human judges: we query GPT-4o five times and collect its self reported confidence; if the majority label has agreement≥80% and all majority runs have confidence≥0.90, we accept the GPT-4o majority label asy, otherwise we defer to a human judge. We validate this gating by measuring agreement with human annotations, achieving 88.12% agreement for the high-confidence subset and 61.14% for the low-confidence subset. To more robustly train the question generatorπ θ , we propose a novel three-step learning paradigm: SFT bootstrapping. We superviseπ θ on two complementary synthetic batches to obtain a format-ready, role-controllable initializerπ 0 (frozen asπ ref ). Batch A (coverage) exhaustively instantiates all24×8(d,r)on a seed image set to learn strict formats and role control; Batch B (preference hints) picks one plausible role perdon a larger image set to provide weak role-selection cues without overfitting. The SFT loss is L SFT =−E (x,d,r,q ⋆ ) logπ θ (q ⋆ | x,d,r) (2) We set π 0 = arg min θ L SFT andπ ref = π 0 . In-context preference construction. To neutralize content difficulty, we fix the context(x,d)and vary the fuzzing rolerand wording. Herer ∈1,...,8 denotes the fuzzing role. We sampleNcandidates q i ∼ π θ (·| x,d,r i ), r i ∈1,..., 8, i = 1,...,N,(3) FuzzingRL9 and query the target VLM for answers ˆa i = f(x,q i ).(4) Then, we let the committee judges assign ternary labels y i ∈−1, 0, 1 ( −1=unanswerable,0=correct,1=incorrect). We score each candidate by s i = y i ∈ −1, 0, 1,(5) and form a in-context preference pair simply as q + ∈ arg max i s i , q − ∈ arg min i s i ,(6) which constructs preferences that primarily reflect role selection and phrasing under the same(x,d). Direct preference optimization. Fixingπ ref , we optimizeπ θ with direct preference optimization (DPO) such that under the same(x,d),π θ prefers questions with higher failure scores (including unanswerable). Let(q + ,q − )be the contrastive pairs in Eq. 6. Our DPO objective is L DPO =−E (x,d,q + ,q − ) h logσ β ∆ θ,ref (x,d,q + ,q − ) i + λ KL E (x,d) h KL π θ (·|x,d)∥π ref (·|x,d) i , (7) where ∆ θ,ref is ∆ θ,ref = logπ θ (q + |x,d)− logπ θ (q − |x,d) − logπ ref (q + |x,d)− logπ ref (q − |x,d) . (8) We repeat the preference construction and optimization stages multiple times. After several iterations, the updates stabilize and yield a trained fuzzing model that concentrates generation on the target VLM’s most failure-prone regions. 4 Experiments In this section, we investigate the efficacy of FuzzingRL for automatically detecting VLM failures, and analyze key design choices and limitations. To this end, we organize our study to answer the following empirical questions, in order: (Q1) Overall Performance: How effective and efficient is FuzzingRL in revealing VLM failure modes? (Q2) Vision-Language Fuzzing: To what extent does our vision–language fuzzing strategy broaden the test space and improve the discovery of diverse failures? 10Authors Suppressed Due to Excessive Length (Q3) Adversarial Reinforcement Fine-Tuning: Can adversarial RFT successfully steer the question generator toward increasingly challenging and failure-inducing queries? (Q4) Data Analysis: What insights can we obtain from analyzing the collected failure cases, and what do these imply for future VLM robustness research? 4.1 Implementation Details For image queries to VLMs, we sample them from the COCO dataset [19]. Evaluation follows our 24-subdimension taxonomy. Unless noted, each method produces one annotators-pass probe per (image x, subdimension d). Evaluating a question generator. We evaluate a question generator by letting it sample images from the same validation image pool and generate one probe per (imagex, subdimensiond) under identical decoding conditions (temperature, top-p, and max tokens). We then query a fixed test model (GPT-4o) with each (image, question) pair, and human judges annotate the test model’s outputs for correctness and answerability under a unified rubric. Metrics. We report three diagnosis-oriented metrics on annotators-pass items: Fooling rateFR(= 1− Accof the test model), where a higher value indicates that the question generator more effectively hits the test model’s weak spots. Unanswerable rateUR: fraction of generated probes that are unanswerable from the image (as judged by the human annotators); lowerURindicates cleaner, image-grounded probing. Distinct RatioDR: measuring per-image uniqueness and diversity of generated questions. For each imagei, letQ i be the set of generated questions anduniq(Q i ) the de-duplicated set. We define DR = 1 |I| X i∈I uniq(Q i ) |Q i | , whereIis the set of images. A higherDRindicates fewer templated duplicates and broader variation. 4.2 Overall Performance of FuzzingRL Following the above evaluation protocol, we compare our trained FuzzingRL generator against Qwen2.5-VL, Llama-3.2-Vision and GPT-4o as question gen- erators, each without an explicit fuzzing role prompt. In all cases, we keep the evaluation test model fixed (GPT-4o) and computeFR/UR/DRfrom human annotations. Table 3 summarizes results. FuzzingRL11 Table 3: Overall performance of FuzzingRL. Applying FuzzingRL to the small generator Qwen2.5-VL-7B enhances fooling rate from 0% to 34.47% while keeping the unanswerable rate low and diversity competitive. Remarkably, a small base model Qwen2.5-VL-7B + FuzzingRL outperforms the Llama-3.2-11B, the much larger models Qwen2.5-VL-72B and the closed-source GPT-4o, effectively turning a weak base gener- ator into a strong vulnerability finder. Question GeneratorFR ↑UR ↓DR ↑ Qwen2.5-VL-7B0.00%100.00%– Qwen2.5-VL-72B6.71%9.33%95.83% Llama-3.2-11B-Vision1.00%12.84%93.98% GPT-4o7.59%4.50%91.37% Qwen2.5-VL-7B + fuzzingRL 34.47%7.75%91.50% 4.3 Vision-Language Fuzzing We ablate the contribution of the vision-language fuzzing design by comparing generation with versus without fuzzing roles used as few-shot in-context exemplars, while keeping the test model and decoding budget fixed. Table 4 reports both the absolute numbers and the deltas(∆ = with− without). Across all generators in Table 4, adding vision-language fuzzing roles to the few-shot context consistently increases FR, indicating that role prompts impose a structured prior over question space and steer generation toward systematic, error-revealing probes rather than benign queries. This shift is not driven by more unanswerable outputs: UR stays stable or decreases, suggesting that roles constrain generation toward answerable questions that test models attempt rather than reject. DR also increases with roles, because roles provide distinct templates that diversify how the same subdimension is probed and reduce near-duplicate questions per image. Overall, roles act as a structured exploration prior, expanding surface realizations while keeping questions answerable and on-topic. Combining vision-language fuzzing with reinforcement learning (FuzzingRL) produces a clear synergy. Vision-language fuzzing diversifies the probing space in a controlled, answerable manner, and RL further concentrates probability mass on hard yet valid queries. This combination preserves the higher FR and typically increases DR by reducing template reuse and encouraging distinct, role-consistent realizations. Overall, FuzzingRL strengthens failure discovery while keeping UR low, improving both diagnostic power and diversity at the same time. 4.4 Adversarial Reinforcement Fine-Tuning We fine-tune the fuzzing model with multiple adversarial RL rounds. In each round the policy proposes questions, the fixed target model answers, a verifier judges success and validity, and the policy is updated from the online payoff. Iteration 0 is the SFT initializer. Figure 3 reports the fooling rate (FR = 1−Acc) of the generated probes across training iterations, evaluated on the training target model and several held-out test VLMs. 12Authors Suppressed Due to Excessive Length Table 4: Effect of Vision-Language Fuzzing. We evaluate whether vision-language fuzzing itself, used as few-shot in-context exemplars without RFT, can improve the quality of generated test queries. We report deltas between with and without fuzzing (∆=With fuzzing− Without fuzzing) forFR,UR, andDR. Across different models, vision-language fuzzing consistently increases the fooling rate and the diversity of generated questions while reducing unrealistic queries. For a fair comparison with the without fuzzing setting, we keep the generation budget identical and generate exactly one question per image in the with fuzzing setting as well. ModelSettingFR ↑UR ↓DR ↑ Qwen2.5-VL-72Bw/o fuzzing6.71%9.33%95.83% Qwen2.5-VL-72Bw/ fuzzing14.67%2.33%98.75% ∆+7.96-7.00+2.92 Llama-3.2-11B-Visionw/o fuzzing1.00%12.84%93.98% Llama-3.2-11B-Visionw/ fuzzing3.61%5.23%94.39% ∆+2.61-7.61+0.41 GPT-4ow/o fuzzing7.59%4.50%91.37% GPT-4ow/ fuzzing9.62%4.75%93.08% ∆+2.03+0.25+1.71 FuzzingRL (Ours)w/o fuzzing31.26%7.67%91.13% FuzzingRL (Ours)w/ fuzzing34.47%7.75%91.50% ∆+3.21+0.08+0.37 We observe that FR on the target model increases monotonically as training proceeds, indicating that the fuzzing generator becomes progressively better at eliciting failures on the target. However, transfer performance on held-out test VLMs saturates and can even degrade after roughly four iterations, suggesting over-specialization to the target. Therefore, we stop at iteration 4 as our final checkpoint to balance target effectiveness and cross-model generalization. 4.5 Generalization: Stress Testing Diverse VLMs with a Trained Fuzzing Generator Beyond degrading a single target model during training, we study whether a trained fuzzing generator can serve as a reusable stress-testing tool for other VLMs. Concretely, we fix a trained fuzzing model (our final FuzzingRL checkpoint) and use it to generate one probe per (imagex, subdimensiond) on a held-out set ofNCOCO images, following the same 24-subdimension taxonomy. We then evaluate a set of test VLMs by querying each model with the same generated questions and images, and compute their accuracy under identical evaluation rules. To attribute the gain to FuzzingRL rather than to question formatting artifacts, we compare three generators under the same budget: (i) a base generator without adversarial fine-tuning, (i) the same base generator augmented with our vision-language fuzzing roles as in-context exemplars (§3.2) but without RFT, FuzzingRL13 sftv1v2v3v4v6 iteration 0.15 0.20 0.25 0.30 0.35 Fooling Rate (FR = 1 - accuracy) 13.42% 19.25% 27.75% 32.76% 34.47% 35.65% Fooling Rate by Iteration qwen3vl llama3_2 onevision qwen2.5vl_32b (target model) Fig. 3: FR by iteration with transfer evaluation. FR (= 1− Acc) over training iterations on the target model (Qwen2.5-VL-32B) and three held-out test VLMs. FR on the target increases steadily, whereas transfer FR peaks around iteration 4 and may drop with further training, so we stop at iteration 4. and (i) our trained FuzzingRL generator. Importantly, questions are generated once by each generator and are not selected or filtered based on any test model’s responses, avoiding cherry-picking effects. Table 5: Generalization across test VLMs. We use the trained FuzzingRL gen- erator to produce one probe per (imagex, subdimensiond) onNheld-out COCO im- ages, and evaluate multiple test VLMs on the resulting question set. Answer correct- ness is annotated by humans. Lower accu- racy indicates that the generated questions are harder for the test model. Test ModelAcc (↑) Human100.00% GPT-4o83.86% Gemini-1.5-Flash (V) 78.71% Qwen3-VL-32B73.78% Llama-3.2-Vision-11B 69.58% LLaVA-OneVision-1.5-8B 67.45% Table 5 shows that questions gen- erated by FuzzingRL consistently re- duce the accuracy of diverse test VLMs compared to both baselines, indicating strong cross-model generalization of failure-seeking behaviors. Meanwhile, the unanswerable rate remains low, suggesting that the difficulty increase is not driven by invalid or image- ungrounded questions, but by system- atically probing challenging visual rea- soning patterns that VLMs tend to fail on. Overall, these results support the use of our trained fuzzing generator as a transferable stress-testing mech- anism: once trained, it can be directly applied to other VLMs to elicit harder, failure-inducing queries under a fixed evaluation budget. 14Authors Suppressed Due to Excessive Length Question: Is the giraffe closer to you, closer to the palm tree or to the fence? Answer: you Closer to the palm tree Fence Fence Closer to the palm tree The giraffe is closer to the fence than the palm tree Image: Subdimension: Part–Whole Hierarchy Fuzzing role: Compositional Reasoning Question: How many chefs are working in the kitchen? Answer: 7 6 6 6 6 5 Image: Subdimension: Multi-object Reasoning Fuzzing role: Visual Perturbation Question: Isn't it true that the person is not preparing food in the kitchen? Answer: no Yes Yes Yes Yes The person in the image is not preparing to cook. Image: Subdimension: Task/Scenario Recognition Fuzzing role: Discourse Logic Question: If another motorcycle of the same size is parked next to this one, will there be three motorcycles visible in total? Answer: no Yes Yes Yes Yes Yes Image: Subdimension: Object Presence Fuzzing role: Hypothetical Reasoning Fig. 4: Cross-model results of the trained fuzzing model. Example probes and test-model outputs showing four recurrent failure families: (a) part–whole/compositional proximity, (b) counting under clutter, (c) discourse/negation in scenario recognition, and (d) hypothetical object-presence reasoning. 4.6 Findings Through our trained fuzzing model, we identify several recurring failure patterns: Different subjects. Even when the intended semantics remain the same, changing the subject or reference frame of a question can lead to different answers. For example, rewriting “Which one is closer to you?” as “Which one is closer to the camera?” may flip the prediction, despite the two being semantically equivalent in context. This suggests sensitivity to superficial phrasing cues rather than stable grounding in the visual evidence. Yes/No Questions. For the same underlying query, rephrasing an alterna- tive question into a yes/no form can change the model’s response. We observe a tendency to over-predict “Yes” on some binary questions, making such formula- tions more likely to induce incorrect answers. This suggests a systematic yes-bias that can outweigh visual evidence. Add Additions. The model can be disrupted by additional constraints or conditions, even when the resulting question remains simple for humans. For instance, changing “How many cars are there here?” to “If 20 cars identical to the one on the left in the image were added, how many cars would there be here?” often yields an incorrect count. This points to brittleness under compositional conditionals and grounded arithmetic. High Count Answer. For counting queries, performance is generally reliable when the number of target objects is small; however, once the count exceeds five, accuracy drops sharply. 5 Conclusion Drawing inspiration from software fuzz-testing, we introduced FuzzingRL, a reinforcement-driven framework for the automatic discovery of modern VLM FuzzingRL15 failures. Through our study, we demonstrate that VLM vulnerabilities can be progressively surfaced by refining a question generator that actively seeks out high-failure regions of the model’s input space. By combining structured vi- sion–language fuzzing with adversarial reinforcement finetuning, FuzzingRL consistently amplifies failure-inducing behaviors, driving down model accuracy over subsequent iterations and revealing weaknesses that static benchmarks cannot capture. To demonstrate our method’s transferability to other VLMs, we apply the trained fuzzing generator to multiple held-out test VLMs and observe consistent accuracy drops on the same image-grounded probes, revealing systematic weaknesses in spatial reasoning, compositionality, discourse logic, and multi-object understanding. References 1. Acharya, M., Kafle, K., Kanan, C.: Tallyqa: Answering complex counting questions (2018), https://arxiv.org/abs/1810.12440 2. Alhamoud, K., Alshammari, S., Tian, Y., Li, G., Torr, P., Kim, Y., Ghassemi, M.: Vision-language models do not understand negation (2025),https://arxiv.org/ abs/2501.09425 3.Beutel, A., Xiao, K., Heidecke, J., Weng, L.: Diverse and effective red teaming with auto-generated rewards and multi-step reinforcement learning. arXiv preprint arXiv:2412.18693 (2024) 4.Biten, A.F., Tito, R., Mafla, A., Gomez, L., Rusiñol, M., Valveny, E., Jawahar, C.V., Karatzas, D.: Scene text visual question answering (2019),https://arxiv. org/abs/1905.13648 5.Böhme, M., Pham, V.T., Nguyen, M.D., Roychoudhury, A.: Directed greybox fuzzing. In: Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. p. 2329–2344 (2017) 6.Chao, Y.W., Liu, Y., Liu, X., Zeng, H., Deng, J.: Learning to detect human-object interactions (2018), https://arxiv.org/abs/1702.05448 7.Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. p. 248–255 (2009).https://doi.org/10.1109/CVPR.2009. 5206848 8.Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kadavath, S., Mann, B., Perez, E., Schiefer, N., Ndousse, K., et al.: Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858 (2022) 9.Ging, S., Bravo, M.A., Brox, T.: Open-ended vqa benchmarking of vision-language models by exploiting classification datasets and their semantic hierarchy. arXiv preprint arXiv:2402.07270 (2024) 10.Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., Parikh, D.: Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2017) 11.Gupta, A., Dollár, P., Girshick, R.: Lvis: A dataset for large vocabulary instance segmentation (2019), https://arxiv.org/abs/1908.03195 12.Hudson, D.A., Manning, C.D.: Gqa: A new dataset for real-world visual reasoning and compositional question answering (2019),https://arxiv.org/abs/1902.095 06 16Authors Suppressed Due to Excessive Length 13.Ishmam, M.F., Tashdeed, I., Saadat, T.A., Ashmafee, M.H., Kamal, A.R.M., Hos- sain, M.A.: Visual robustness benchmark for visual question answering (vqa) (2024), https://arxiv.org/abs/2407.03386 14.Ji, J., Krishna, R., Fei-Fei, L., Niebles, J.C.: Action genome: Actions as composition of spatio-temporal scene graphs (2019), https://arxiv.org/abs/1912.06992 15.Johnson, J., Hariharan, B., Van Der Maaten, L., Fei-Fei, L., Lawrence Zitnick, C., Girshick, R.: Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 2901–2910 (2017) 16. Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., Duerig, T., Ferrari, V.: The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. International Journal of Computer Vision 128(7), 1956–1981 (Mar 2020).https://doi.org/10.1007/s11263-020-01316-z,http: //dx.doi.org/10.1007/s11263-020-01316-z 17.Li, B., Lin, Z., Peng, W., Nyandwi, J.d.D., Jiang, D., Ma, Z., Khanuja, S., Krishna, R., Neubig, G., Ramanan, D.: Naturalbench: Evaluating vision-language models on natural adversarial samples. Advances in Neural Information Processing Systems 37, 17044–17068 (2024) 18.Li, B., Ge, Y., Ge, Y., Wang, G., Wang, R., Zhang, R., Shan, Y.: Seed-bench: Benchmarking multimodal large language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 13299–13308 (2024) 19.Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference on computer vision. p. 740–755. Springer (2014) 20.Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. Advances in neural information processing systems 36, 34892–34916 (2023) 21.Liu, X., Zhu, Y., Gu, J., Lan, Y., Yang, C., Qiao, Y.: Mm-safetybench: A benchmark for safety evaluation of multimodal large language models (2024),https://arxiv. org/abs/2311.17600 22.Liu, Y., Duan, H., Zhang, Y., Li, B., Zhang, S., Zhao, W., Yuan, Y., Wang, J., He, C., Liu, Z., et al.: Mmbench: Is your multi-modal model an all-around player? In: European conference on computer vision. p. 216–233. Springer (2024) 23. Lu, P., Bansal, H., Xia, T., Liu, J., Li, C., Hajishirzi, H., Cheng, H., Chang, K.W., Galley, M., Gao, J.: Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255 (2023) 24.Lu, Y., Jiang, D., Chen, W., Wang, W.Y., Choi, Y., Lin, B.Y.: Wildvision: Eval- uating vision-language models in the wild with human preferences. Advances in Neural Information Processing Systems 37, 48224–48255 (2024) 25.Mai, Z., Chowdhury, A., Wang, Z., Jeon, S., Wang, L., Hou, J., Chao, W.L.: Ava-bench: Atomic visual ability benchmark for vision foundation models. arXiv preprint arXiv:2506.09082 (2025) 26. Mashrur, A., Luo, W., Zaidi, N.A., Robles-Kelly, A.: Robust visual question an- swering via semantic cross modal augmentation. Computer Vision and Image Understanding 238, 103862 (2024) 27. Masry, A., Long, D.X., Tan, J.Q., Joty, S., Hoque, E.: Chartqa: A benchmark for question answering about charts with visual and logical reasoning (2022), https://arxiv.org/abs/2203.10244 28. Mathew, M., Karatzas, D., Jawahar, C.V.: Docvqa: A dataset for vqa on document images (2021), https://arxiv.org/abs/2007.00398 FuzzingRL17 29.Miller, B.P., Fredriksen, L., So, B.: An empirical study of the reliability of unix utilities. Communications of the ACM 33(12), 32–44 (1990) 30.Miller, B.P., Koski, D., Lee, C.P., Maganty, V., Murthy, R., Natarajan, A., Steidl, J.: Fuzz revisited: A re-examination of the reliability of unix utilities and services. Tech. rep., University of Wisconsin-Madison Department of Computer Sciences (1995) 31.Nguyen, V.N., Groueix, T., Hu, Y., Salzmann, M., Lepetit, V.: Nope: Novel object pose estimation from a single image (2024),https://arxiv.org/abs/2303.13612 32.Saikh, T., Ghosal, T., Mittal, A., Ekbal, A., Bhattacharyya, P.: Scienceqa: A novel resource for question answering on scholarly articles. International Journal on Digital Libraries 23(3), 289–301 (2022) 33.Serebryany, K.: Continuous fuzzing with libfuzzer and addresssanitizer. In: 2016 IEEE Cybersecurity Development (SecDev). p. 157–157. IEEE (2016) 34. Sidorov, O., Hu, R., Rohrbach, M., Singh, A.: Textcaps: a dataset for image captioning with reading comprehension (2020),https://arxiv.org/abs/2003.1 2462 35.Singh, A., Natarajan, V., Shah, M., Jiang, Y., Chen, X., Batra, D., Parikh, D., Rohrbach, M.: Towards vqa models that can read (2019),https://arxiv.org/ab s/1904.08920 36. Suhr, A., Artzi, Y.: Nlvr2 visual bias analysis (2019),https://arxiv.org/abs/19 09.10411 37.Takemoto, K., Yamada, M., Sasaki, T., Akima, H.: Hico-det-sg and v-coco-sg: New data splits for evaluating the systematic generalization performance of human-object interaction detection models (2024), https://arxiv.org/abs/2305.09948 38.Wei, A., Haghtalab, N., Steinhardt, J.: Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems 36, 80079–80110 (2023) 39.Welde, T.M., Liao, L.: Counting in visual question answering: Methods, datasets, and future work. International Journal of Image and Graphics 25(05), 2550044 (2025) 40.Yu, L., Poirson, P., Yang, S., Berg, A.C., Berg, T.L.: Modeling context in referring expressions (2016), https://arxiv.org/abs/1608.00272 41. Yu, W., Yang, Z., Li, L., Wang, J., Lin, K., Liu, Z., Wang, X., Wang, L.: Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490 (2023) 42.Yue, X., Ni, Y., Zhang, K., Zheng, T., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., et al.: Mmmu: A massive multi-discipline multimodal understand- ing and reasoning benchmark for expert agi. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 9556–9567 (2024) 43.Yue, X., Zheng, T., Ni, Y., Wang, Y., Zhang, K., Tong, S., Sun, Y., Yu, B., Zhang, G., Sun, H., et al.: Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. arXiv preprint arXiv:2409.02813 (2024) 44. Zhang, J., Huang, W., Ma, Z., Michel, O., He, D., Gupta, T., Ma, W.C., Farhadi, A., Kembhavi, A., Krishna, R.: Task me anything (2025),https://arxiv.org/ab s/2406.11775 45.Zhang, J., Xue, L., Song, L., Wang, J., Huang, W., Shu, M., Yan, A., Ma, Z., Niebles, J.C., Savarese, S., et al.: Provision: Programmatically scaling vision-centric instruction data for multimodal language models. arXiv preprint arXiv:2412.07012 (2024) 46.Zhang, Y.F., Zhang, H., Tian, H., Fu, C., Zhang, S., Wu, J., Li, F., Wang, K., Wen, Q., Zhang, Z., et al.: Mme-realworld: Could your multimodal llm challenge 18Authors Suppressed Due to Excessive Length high-resolution real-world scenarios that are difficult for humans? arXiv preprint arXiv:2408.13257 (2024) 47. Zhang, Y., Su, Y., Liu, Y., Yeung-Levy, S.: Negvqa: Can vision language models understand negation? arXiv preprint arXiv:2505.22946 (2025) 48. Zhao, T., Zhang, T., Zhu, M., Shen, H., Lee, K., Lu, X., Yin, J.: Vl-checklist: Evaluating pre-trained vision-language models with objects, attributes and relations. arXiv preprint arXiv:2207.00221 (2022) 49.Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 40(6), 1452–1464 (2018).https://doi.org/10.1109/TPAMI. 2017.2723009 50.Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J.Z., Fredrikson, M.: Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043 (2023)

Instant research discovery