Paper deep dive
Silent Failures in Multimodal Agentic Search:A Diagnostic Taxonomy and Cross-Judge Evaluation
Zhengxian Wu, Junjie Gao, Kai Yang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/23/2026, 2:39:36 AM
Summary
This paper introduces a diagnostic taxonomy of six 'silent failure' types in multimodal agentic search systems, including modality shortcuts, phantom grounding, and provenance hallucination. The authors develop a trajectory-level evaluation pipeline using LLM judges to assess both answer correctness and evidence grounding. Experiments on four frontier models (Claude, Gemini, GPT-4o) reveal that surface accuracy often overestimates true reliability, as silent failures persist or shift across models rather than disappearing.
Entities (14)
Relation Signals (13)
Silent Failures → includes → Over-Retrieval Laundering
confidence 95% · We introduce a six-category taxonomy covering... over-retrieval laundering...
Silent Failures → includes → Phantom Grounding
confidence 95% · We introduce a six-category taxonomy covering... phantom grounding...
Silent Failures → includes → Wrong-Evidence-Right-Answer
confidence 95% · We introduce a six-category taxonomy covering... wrong-evidence-right-answer cases...
Silent Failures → includes → Modality Shortcut
confidence 95% · We introduce a six-category taxonomy covering modality shortcuts...
Silent Failures → includes → Cross-Modal Contradiction
confidence 95% · We introduce a six-category taxonomy covering... cross-modal contradiction...
Silent Failures → includes → Provenance Hallucination
confidence 95% · We introduce a six-category taxonomy covering... provenance hallucination.
True Correctness Rate → measures → Silent Failures
confidence 92% · True Correctness Rate (TCR) further requires the trajectory to be answer-correct and free of any silent-failure flag.
MMSearch-Plus → usedforevaluation → Silent Failures
confidence 90% · Experiments on MMSearch-Plus trajectories... show that surface accuracy consistently overestimates true trajectory-level correctness.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multimodal agentic search systems increasingly rely on external tools to answer knowledge-intensive visual questions. However, existing evaluations mainly focus on final-answer accuracy and may miss failures in the search trajectory. In this work, we study such hidden reliability issues as silent failures. We introduce a six-category taxonomy covering modality shortcuts, phantom grounding, wrong-evidence-right-answer cases, over-retrieval laundering, cross-modal contradiction, and provenance hallucination. Based on this taxonomy, we build a trajectory-level diagnostic pipeline that evaluates both answer correctness and evidence-grounding quality under a unified ReAct-style scaffold. Experiments on MMSearch-Plus trajectories across four frontier multimodal models show that surface accuracy consistently overestimates true trajectory-level correctness. We further use cross-judge validation, blank-image stress tests, and tool ablations to show that silent failures are capability-dependent and often shift rather than disappear. Home-page: this https URL
Tags
Links
- Source: https://arxiv.org/abs/2607.19793v1
- Canonical: https://arxiv.org/abs/2607.19793v1
Trouble viewing inline? Open PDF directly →
Full Text
27,578 characters extracted from source content.
Expand or collapse full text
Silent Failures in Multimodal Agentic Search: A Diagnostic Taxonomy and Cross-Judge Evaluation Zhengxian Wu Ant Group Hangzhou, China sean.wzx@antgroup.com Junjie Gao Ant Group Hangzhou, China jungao.gjj@antgroup.com Kai Yang Ant Group Hangzhou, China chengyue.yk@antgroup.com Abstract Multimodal agentic search systems increasingly rely on external tools to answer knowledge-intensive visual questions. However, existing evaluations mainly focus on final-answer accuracy and may miss failures in the search trajectory. In this work, we study such hidden reliability issues as silent failures. We introduce a six- category taxonomy covering modality shortcuts, phantom ground- ing, wrong-evidence-right-answer cases, over-retrieval laundering, cross-modal contradiction, and provenance hallucination. Based on this taxonomy, we build a trajectory-level diagnostic pipeline that evaluates both answer correctness and evidence-grounding quality under a unified ReAct-style scaffold. Experiments on MMSearch- Plus trajectories across four frontier multimodal models show that surface accuracy consistently overestimates true trajectory-level correctness. We further use cross-judge validation, blank-image stress tests, and tool ablations to show that silent failures are capability-dependent and often shift rather than disappear. Home-page: https://github.com/DingWu1021/ silent-failures-multimodal-agentic-search Keywords multimodal evaluation, agentic search, faithfulness, silent failure, LLM-as-judge 1 Introduction Multimodal agentic search has recently become an important par- adigm for knowledge-intensive visual question answering[2,9]. Unlike models that answer mainly from parametric knowledge, these systems interact with external search and browsing tools to gradually collect evidence, so that the final answer can be supported by traceable external information [4,7,8]. This traceability is in- creasingly critical as fabricated and synthetic content proliferates across information retrieval ecosystems [5]. Despite this progress, existing evaluations of multimodal agentic search remain largely answer-level[3]. However, in multimodal agentic search[6], the final answer is often produced through a sequence of intermediate decisions, including visual inspection, tool selection, evidence retrieval, and evidence integration. Errors may occur at any stage. Recent studies on agent systems also show that many failures are silent [1]. Motivated by this gap, we ask a central question: what does it mean for a multimodal search agent to be truly correct? To answer this question, we need to examine the full search trajectory, systematically characterize the forms of silent failures in multimodal search agents, measure how often they occur, and assess whether they can be automatically diagnosed. In this work, we study multimodal agentic search from a diag- nostic perspective. Our goal is to understand how current frontier agents behave when their full trajectories are evaluated for ground- ing and faithfulness. We first introduce a taxonomy of silent failures for multimodal agentic search. The taxonomy includes six recur- ring failure types: modality shortcut, phantom grounding, wrong- evidence-right-answer, over-retrieval laundering, cross-modal con- tradiction, and provenance hallucination. These categories cover failures before retrieval, during retrieval, and after retrieval. They are designed to identify trajectories whose final answers may ap- pear reasonable, but whose reasoning processes are not sufficiently supported by the image or retrieved evidence. Based on this taxonomy, we build a lightweight trajectory-level diagnostic pipeline. Each model is run under a unified ReAct-style framework with the same tool interfaces and logging format. Then, an LLM judge guided by a structured evaluation rubric evaluates both answer correctness and the presence of each silent-failure cat- egory. To assess the reliability of judge-based diagnosis, we further introduce same-family and cross-family LLM cross-validators. We apply this pipeline to 800 trajectories from a stratified MMSearch- Plus sample, covering multiple frontier multimodal models. Our re- sults show that surface accuracy consistently overestimates trajectory- grounded correctness. We further find that silent failures do not sim- ply disappear as models become stronger. Instead, their distribution shifts across models with different capabilities. Our contributions are summarized as follows: •We formulate silent failures in multimodal agentic search as a trajectory-level reliability problem, showing that answer-level accuracy alone cannot distinguish faithful grounded reasoning from superficially correct but unsupported trajectories. •We introduce a six-category taxonomy of silent failures and build a unified diagnostic pipeline that applies an LLM-judge rubric to evaluate both answer correctness and trajectory grounding. •We assess judge reliability with same-family and cross-family validators, showing which findings are stable and which fine- grained labels remain judge-dependent. •We conduct a study on MMSearch-Plus trajectories across fron- tier multimodal agents, revealing a gap between surface accuracy and true correctness rate, and showing that silent failures are capability-dependent and often shift rather than disappear. 2 Method 2.1 Pipeline Given a multimodal search task푥= (퐼,푞), where퐼denotes the input image and푞denotes the user question, an agent produces a final answer through a sequence of tool calls and intermediate 1 arXiv:2607.19793v1 [cs.AI] 22 Jul 2026 Wu et al. Taxonomy of silent failures in multimodal agentic search Modality Shortcut (mod-sc) Over-Retrieval Laundering(or-ld) Phantom Grounding (pht-gr) Cross-Modal Contra- diction(cm-ct) Wrong-Evidence-Right- Answer (we-ra) Provenance Halluci- nation(prv-hl) In what year was the company that produces the pigment in the image founded? The agent guesses “cadm ium yellow” from the tex t query alone and never i nspects the brand label vi sible in the image.(No m eaningful use of visual e vidence.) This picture was taken inside a simulator, which organization and location do the facility in the image belong to? The agent claims that a CNBC article identifies a specific CAE simulator in Montreal, but the fetched article is about an unrelated activist investor in another city.(The cited source does not support the claim.) What is the name of the film? The agent claims that image search returned “Children’s Hospital” and uses it to justify the answer, but the answer appears to come from parametric memory triggered by the visible Adult Swim logo. (Correct answer with an invalid reasoning step.) What is the title of this paper? The image shows a figure from “Consistent Sampling and Simulation,” where “Two For One” is only a baseline, but the agent misidentifies it as the “Two For One” paper itself. (Answer contradicts the visual evidence.) What is the YouTube account of the barista who designed the latte art in the image? The agent invents a YouTube URL, youtube.com/@umpaul, and attributes a “6-time World Latte Art Champion” title to it, neither of which appears in any retrieved observation. (Fabricated provenance.) The agent makes multiple image and web searches, but none identify the song. It still guesses “Bohemian Rhapsody” from the movie context and presents it as retrieval-grounded. (Redundant retrieval masks an answer from parametric knowledge.) What song is the band in the picture practicing? Figure 1: Taxonomy of silent failures in multimodal agentic search. Each panel presents one failure category with a representa- tive task example and the observed behavior that triggers the diagnostic label. reasoning steps. We denote one complete run as a trajectory: 휏=푧 1 ,푎 1 ,표 1 , . . .,푧 푇 ,푎 푇 ,표 푇 ,푦, where푧 푡 is the intermediate reasoning at step푡,푎 푡 is the action or tool call,표 푡 is the tool observation, and푦is the final answer. Standard evaluation usually checks only whether푦matches the reference answer, while ignoring the intermediate reasoning and evidence use in 휏 . We argue that answer correctness alone is not sufficient to mea- sure reliability in multimodal agentic search. To this end, for each trajectory, we produce a structured diag- nostic record: 푑(휏)=푐, 푓 1 , 푓 2 , . . ., 푓 6 , where푐 ∈ 0,1indicates whether the final answer is correct, and each푓 푖 ∈ 0,1indicates whether the trajectory triggers the푖-th silent-failure category. The failure labels are not mutually exclusive and a single trajectory may contain multiple failure modes. This formulation extends answer-level evaluation into trajectory-level diagnosis. 2.2 Taxonomy of Silent Failures To characterize failures in multimodal agentic search, we introduce a taxonomy of silent failures tailored to this setting. The taxonomy contains six failure types and covers three stages of the search process: before retrieval, during retrieval, and after retrieval. (1)Modality Shortcut. A modality shortcut occurs when the agent produces an answer without sufficiently using the input im- age. The model may make no image-related tool calls, or it may only describe the image at a superficial level before relying mainly on the text question or its own parametric knowledge. This failure usually occurs before retrieval. It indicates that the model bypasses the mul- timodal nature of the task instead of using the visual information. (2)Phantom Grounding. Phantom grounding occurs when the agent cites a retrieved source, but the source content does not actu- ally support the claimed fact. This failure is more subtle than the absence of retrieval, because the trajectory does contain external evidence. However, there is no valid support relation between the evidence and the conclusion. For example, the model may cite a webpage as evidence even though the page only contains informa- tion unrelated to the answer. (3)Wrong-Evidence-Right-Answer. This failure refers to cases where the final answer is correct, but some explicit evidence or reasoning step in the trajectory is factually wrong. This failure shows that a correct answer does not neces- sarily imply a reliable reasoning process. The model may arrive at the correct answer through parametric knowledge or accidental cues, while still providing an incorrect retrieval-based explanation or an invalid reasoning chain. (4)Over-Retrieval Laundering. Over-retrieval laundering occurs when the agent issues multiple redundant retrieval calls, but the final answer is not supported by the retrieved results and instead comes from the model’s prior para- metric knowledge. In this case, retrieval mainly serves as a form of presentation, making the answer appear externally grounded. This failure often appears during the retrieval process, especially when the model gives a detailed and confident answer after unhelpful retrieval results. (5)Cross-Modal Contradiction. Cross-modal contradiction occurs when the final answer agrees with some noisy text retrieval results but conflicts with what is clearly shown in the input image. This failure reflects a central challenge in multimodal search: the model must not only retrieve evidence, but also check consistency between visual and textual evidence. When the model over-relies on noisy text evidence and underuses the image, this type of failure can arise. (6)Provenance Hallucination. Prove- nance hallucination occurs when the agent generates URLs, dates, source names, or citations that look plausible but do not appear in any tool observation. This failure directly harms the traceability of the answer. Even if the final answer appears credible, invented provenance can mislead users into believing that the answer is supported by real external evidence. These six categories are not mutually exclusive. Instead, they capture different aspects of trajectory reliability. Modality shortcut 2 Silent Failures in Multimodal Agentic Search: A Diagnostic Taxonomy and Cross-Judge Evaluation Table 1: Trajectory terminal states per model (푁=200 each). Headline metrics are computed on the committed subset. ModelCommitted Refused Exhausted Crashed Claude Sonnet 4.6136 (68%)0 (0%)46 (23%)18 (9%) Gemini 2.5 Pro133 (66%)1 (0%)58 (29%)8 (4%) Gemini 3.1 Pro Preview124 (62%)0 (0%)76 (38%)0 (0%) GPT-4o175 (88%)3 (2%)14 (7%)8 (4%) mainly corresponds to pre-retrieval failure, over-retrieval laun- dering mainly corresponds to retrieval-process failure, and phan- tom grounding, wrong-evidence-right-answer, cross-modal con- tradiction, and provenance hallucination mainly correspond to post-retrieval failures. With this taxonomy, we can identify hidden reliability issues that are difficult to detect from the final answer alone. 2.3 Trajectory-Level Diagnostic Pipeline Based on this taxonomy, we build a lightweight diagnostic pipeline. The pipeline consists of three steps: unified trajectory collection, rubric-guided LLM diagnosis, and cross-judge validation. Unified trajectory collection. To ensure fair comparison across models, we run all models under the same ReAct-style agent frame- work. Each model uses the same system prompt, the same tool interfaces, and the same trajectory logging format. The tool set includes web search, image search, webpage reading, and image cropping. For each task–model pair, we record the full reasoning, tool calls, tool inputs, tool observations, and final answer. Each trajectory is allowed a fixed maximum number of tool calls, which prevents unbounded search and keeps the budget consistent across models. Rubric-guided LLM diagnosis. After collecting the trajectory, we use an LLM judge guided by a structured evaluation rubric to diagnose it. The judge receives the task question, the input image, the reference answer, the full trajectory, and the final answer. It then produces a structured output that includes whether the final answer is correct and whether each of the six silent-failure categories is present. For each triggered failure category, the judge also provides a short justification and points to the relevant trajectory steps. This improves the interpretability of the diagnosis and supports later manual inspection and agreement analysis. Based on the diagnostic output, we define the true correctness rate as requiring both a correct final answer and no silent-failure flags in the trajectory. Cross-judge validation. Since LLM judges may have their own biases, we introduce a cross-validation mechanism to assess the stability of the diagnostic results. Specifically, we use both a same- family judge and a cross-family judge to re-evaluate the same strat- ified set of trajectories, and compute Cohen’s휅between each val- idator and the primary judge. This design serves two purposes. First, it tests whether answer- correctness judgments remain stable across different judges. Second, it measures whether fine-grained failure categories depend on the choice of judge. Table 2: Accuracy versus true correctness rate (TCR) on the committed subset, with 95% Wilson CIs. ModelRaw 푁 Acc (raw) Acc (committed) TCR (committed) Claude Sonnet 4.620025.0%36.0% [28.4, 44.4]34.6% [27.1, 42.9] Gemini 2.5 Pro20021.5%31.6% [24.3, 39.9]28.6% [21.6, 36.8] Gemini 3.1 Pro Preview20039.0% 61.3% [52.5, 69.4] 54.0% [45.3, 62.6] GPT-4o20020.0%22.9% [17.3, 29.6]22.3% [16.8, 29.0] 3 Experiments 3.1 Experimental Setup Data and models. We sample 200 tasks from MMSearch-Plus us- ing stratified sampling, and run a unified search-agent framework on four frontier multimodal models, resulting in 800 trajectories in total. The sampling covers different task categories and difficulty levels, so that the results are not dominated by a single task type. The evaluated models include Claude Sonnet 4.6, Gemini 2.5 Pro, Gemini 3.1 Pro Preview, and GPT-4o. Trajectory collection. All models are run under the same ReAct-style scaffold, with identical tool interfaces, system prompts, and trajectory logging format. Each trajectory is allowed at most 10 tool calls. We record the full inter- mediate reasoning, tool calls, tool observations, and final answer. Trajectory states. Not every trajectory produces a substantive answer that can be evaluated. We therefore divide terminal states into four types: committed, where the model gives a substantive final answer; refused, where the model refuses to answer; exhausted, where the model reaches the tool-call limit without producing a final answer; and crashed, where the model run fails due to an unrecoverable error. Since our goal is to study reliability when the model actually attempts to answer, we report the main results on the committed subset, while also reporting raw-푁results as additional context as shown in Table 1. Metrics. We report two main metrics. Accuracy measures whether the final answer is cor- rect. True Correctness Rate (TCR) further requires the trajectory to be answer-correct and free of any silent-failure flag. Thus, Accu- racy measures surface answer correctness, while TCR measures trajectory-level correctness. 3.2 Main Results Table 2 reports Accuracy and TCR for the four models on the com- mitted subset. Across all models, TCR is lower than surface Accu- racy. After removing answer-correct trajectories with silent-failure flags, the measured correctness drops by 0.6 to 7.3 percentage points: Claude drops from 36.0% to 34.6%, Gemini 2.5 from 31.6% to 28.6%, Gemini 3.1 from 61.3% to 54.0%, and GPT-4o from 22.9% to 22.3%. These results show that answer-level accuracy overestimates the reliability of multimodal search agents. Even when the final answer is judged correct, the full trajectory may still contain problems that answer-level evaluation cannot capture, such as wrong evidence, hallucinated provenance, or cross-modal contradiction. Notably, Gemini 3.1 obtains the highest committed accuracy, but also has the largest drop from Accuracy to TCR. This suggests that stronger models do not necessarily eliminate silent failures. As models pro- duce correct answers more often, cases where the answer is correct but the process is unreliable can also become more visible. 3 Wu et al. 0 20 40 60 80 Silent-failure rate(%) Silent-failure category distribution per model MOD-SCPHT-GRWE-RA OR-LDCM-CTPRV-HL Gemini-2.5-pro Gemini-3.1- pro-preview Claude-sonnet-4.6 gpt-4o Figure 2: Silent-failure rates by category on the committed subset, revealing capability-dependent failure shifts. Table 3: Blank-image stress test on originally correct com- mitted trajectories, reported as survival cases. ModelClaudeGemini 2.5Gemini 3.1GPT-4o Survival rate0/50 (0.0%)0/43 (0.0%)2/75 (2.7%)0/40 (0.0%) 3.3 Diagnostic Analyses Failure profiles shift with model capability. To understand how different models fail silently, we analyze the distribution of failure categories among committed trajectories, as shown in Figure 2. The results show that silent failures do not simply decrease as model capability improves; instead, the failure profile shifts. For Claude, Gemini 2.5, and GPT-4o, failures mainly occur after retrieval, es- pecially pht-gr and cm-ct. This suggests that these models often perform retrieval, but still fail to use the retrieved evidence reliably. For example, they may cite webpages that do not support the final answer, or follow textual evidence that conflicts with the image. Gemini 3.1 shows a different pattern. It has the lowest rates of pht-gr and cm-ct, indicating fewer post-retrieval evidence-use failures, but it has the highest rate of mod-sc. Our trajectory in- spection shows that Gemini 3.1 identifies the image with its vision encoder and answers without calling reverse image search. When visual recognition is incorrect, this behavior becomes a pre-retrieval modality shortcut. Thus, stronger models do not eliminate silent failures; instead, they shift where failures occur. Blank-image stress test. One possible concern is that some MMSearch-Plus questions may not truly require the image. If cor- rect answers can be obtained without the image, then the trajectory issues we observe may not necessarily reflect failures in multimodal grounding. To test this, we perform a blank-image stress test on all trajectories in the committed subset that are judged answer-correct by the primary judge. Specifically, we replace the input image with a same-size blank white image, rerun the model under the same scaffold, and judge the resulting trajectory again. This test covers 208 originally answer-correct model–task pairs, including 50 for Claude, 43 for Gemini 2.5, 75 for Gemini 3.1, and 40 for GPT-4o. As shown in Table 3, the survival rate under blank images is zero for three models, and only 2/75 for Gemini 3.1. Overall, 206 out of 208 correct trajectories no longer remain after removing the image. We further inspect the two surviving cases and find that they are not cases where the model successfully bypasses the image. Instead, the question text itself already contains enough information. Table 4: Cross-judge agreement on푛=100 paired verdicts, reporting Cohen’s휅for same- and cross-family validators and cross-family observed agreement 푃 표 . MetricSame 휅Cross 휅Cross 푃 표 answer_correct0.9160.81798% mod-sc0.2600.03125% pht-gr0.6200.26075% or-ld0.5650.30166% cm-ct0.5130.46779% prv-hl0.3220.32296% we-ra–99% Judge reliability. Table 4 reports cross-judge agreement on the same set of푛=100 paired verdicts. Answer correctness is highly stable across judges, with휅=0.916 for the same-family validator and휅=0.817 for the cross-family validator. The cross- family observed agreement 푃 표 is also high at 98%. For fine-grained failure categories, the pattern is more mixed. mod-sc has the lowest cross-family agreement, with휅=0.031 and 푃 표 =25%, indicating a strong difference in how judges apply this label. or-ld shows a similar but weaker pattern, with cross-family 휅=0.301 and푃 표 =66%. These results suggest that some categories are sensitive to the judge’s implicit decision boundary, especially when deciding whether the model truly used the image or whether retrieval mainly served to support parametric knowledge. Other categories show different behavior. pht-gr and cm-ct have moderate cross-family observed agreement, with푃 표 =75% and 79%, respectively, suggesting more balanced judgment differences. In contrast, prv-hl and we-ra have very high푃 표 but few positive cases, so their 휅 are unstable or undefined under low prevalence. Tool ablation: better tools reshape rather than eliminate failures. Finally, we study how tool design affects failure patterns by comparing a full setting withreverse_image_searchagainst an ablated setting where this tool is removed, while keeping all other tools, prompts, and execution settings unchanged. As shown in Ta- ble 5,reverse_image_searchimproves committed accuracy by 13.9 to 18.9 percentage points for the three more retrieval-dependent models: Claude, Gemini 2.5, and GPT-4o. It also shifts their failure distribution: mod-sc and or-ld decrease, suggesting less image bypassing and less use of redundant retrieval to support paramet- ric knowledge, while pht-gr and cm-ct increase, suggesting that models obtain more image-related evidence but may still misuse it. Gemini 3.1 shows a different pattern: the tool improves committed accuracy by only 4.8 points and only slightly reduces all silent- failure categories. Trajectory inspection suggests that Gemini 3.1 often identifies the image with its own vision encoder before any tool call, so reverse image search largely duplicates an existing capability while consuming tool-call budget. Overall, better tools do not always reduce failures monotonically; their effect depends on model capability and often changes where failures occur. 4 Conclusion We presented a trajectory-level study of silent failures in multi- modal agentic search. Our taxonomy and diagnostic pipeline show that final-answer accuracy is insufficient for measuring reliability, since correct answers may still rely on unsupported evidence, hallu- cinated provenance, or cross-modal contradictions. Experiments on MMSearch-Plus reveal a consistent gap between surface accuracy 4 Silent Failures in Multimodal Agentic Search: A Diagnostic Taxonomy and Cross-Judge Evaluation Table 5: Controlled tool ablation with versus without reverse_image_search on the committed subset. Claude Gem 2.5 Gem 3.1 GPT-4o Acc. v119.612.756.58.6 Acc. v236.031.661.322.5 ΔAcc. +16.5 +18.9 +4.8 +13.9 Δmod-sc −10.1 −6.5 −0.3 −23.2 Δor-ld −35.4 −15.6 −1.6 −1.9 Δpht-gr +1.8 +16.6 −2.0 +18.9 Δcm-ct +9.1 +8.3 −2.0 +13.5 and true correctness rate. Cross-judge validation shows that an- swer correctness is stable, while fine-grained failure labels are more judge-dependent. Blank-image and tool-ablation analyses further suggest that stronger models and better tools often shift failures rather than remove them. References [1] Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya G. Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. 2025. Why Do Multi-Agent LLM Systems Fail? ArXiv abs/2503.13657 (2025).https: //api.semanticscholar.org/CorpusID:277103715 [2] Yong Xien Chng, Tianxi Hu, Wenwen Tong, Xueheng Li, Jiandong Chen, Hao Yu, Jiefan Lu, Hewei Guo, Hanming Deng, Xie Chen, Gao Huang, Dahua Lin, and Lewei Lu. 2025. SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning. ArXiv abs/2512.24330 (2025). https: //api.semanticscholar.org/CorpusID:284351490 [3]Xinyu Geng, Peng Xia, Zhen Zhang, Xinyu Wang, Qiuchen Wang, Ruixue Ding, Chenxi Wang, Jialong Wu, Yida Zhao, Kuan Li, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. 2025. WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent. ArXiv abs/2508.05748 (2025). https://api.semanticscholar. org/CorpusID:280561766 [4]Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanmin Wu, Jiayi Lei, Pengshuo Qiu, Pan Lu, Zehui Chen, Guanglu Song, Peng Gao, Yu Liu, Chunyuan Li, and Hongsheng Li. 2024. MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Engines. ArXiv abs/2409.12959 (2024). https://api.semanticscholar.org/ CorpusID:272753572 [5] Ping Liu, Zhedong Zheng, J. Shane Culpepper, and Xin Yu. 2026. SynthIR: Work- shop on Synthetic Content in Information Retrieval Ecosystems. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2026) Workshops. [6]Kartik Narayan, Yang Xu, Tian Cao, Kavya Nerella, Vishal M. Patel, Navid Shiee, Peter Grasch, Chao Jia, Yinfei Yang, and Zhe Gan. 2025. DeepMMSearch-R1: Empowering Multimodal LLMs in Multimodal Web Search. ArXiv abs/2510.12801 (2025). https://api.semanticscholar.org/CorpusID:282064928 [7]Xuying Ning, Dongqi Fu, Tianxin Wei, Mengting Ai, Jiaru Zou, Ting-Wei Li, Hanghang Tong, Yada Zhu, Hendrik F. Hamann, and Jingrui He. 2026. MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains. ArXiv abs/2603.00873 (2026). https://api.semanticscholar.org/ CorpusID:286224515 [8] Xijia Tao, Yihua Teng, Xinxing Su, Xinyu Fu, Jihao Wu, Chaofan Tao, Ziru Liu, Haoli Bai, Rui Liu, and Lingpeng Kong. 2025. MMSearch-Plus: Bench- marking Provenance-Aware Search for Multimodal Browsing Agents. https: //api.semanticscholar.org/CorpusID:281658177 [9] Jinming Wu, Zihao Deng, Wei Li, Yiding Liu, Bo You, Bo Li, Zejun Ma, and Ziwei Liu. 2025. Mmsearch-r1: Incentivizing lmms to search. arXiv preprint arXiv:2506.20670 (2025). 5