Paper deep dive

Neos: Learning to Interleave Search and Visual Reasoning

Ekko He, Jack Hong, Shilin Yan, Xintong Zhang, Yujie Zhang, Wenjie Li, Haoran Sun, Kaixun Jiang, Jiyuan Fu, Guannan Zhang, Hongwei Xue

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 131

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Recent multimodal deep search agents have advanced tool use and visual content retrieval, yet they still struggle to convert intermediate visual evidence into actionable guidance for subsequent search, limiting their effectiveness in long-horizon settings. To address this gap, we present Neos, a multimodal deep search framework that learns both how to acquire evidence and how to use it to guide subsequent decisions. To make this coupling learnable, Neos constructs training tasks with intermediate visual dependencies while filtering out textual shortcuts, and introduces a lightweight, trainable Multimodal Evidence Interface (MEI) that transforms retrieved content into goal-conditioned observations. At its core, an agent-environment co-evolution procedure treats MEI as a co-adaptive component of this loop. Specifically, reference-guided credit assignment uses verified successful trajectories to localize critical policy errors while preserving credit for useful decisions in failed searches, whereas rollout feedback selects informative local evidence-processing interactions for MEI supervised fine-tuning (SFT). By alternating policy reinforcement learning (RL) with MEI SFT, Neos allows the policy's evolving evidence requests to shape MEI training and the adapted MEI to reshape subsequent observations, closing the loop between evidence acquisition and evidence-conditioned action. Across ten benchmarks, Neos-9B and Neos-35B demonstrate leading performance among specialized multimodal deep-search agents.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

127,367 characters extracted from source content.

Expand or collapse full text

Neos: Learning to Interleave Search and Visual Reasoning Ekko He 1* , Jack Hong 3,*,‡ , Shilin Yan 1,*,‡ , Xintong Zhang 1 , Yujie Zhang 2 , Wenjie Li 2 , Haoran Sun 3 , Kaixun Jiang 3 , Jiyuan Fu 3 , Guannan Zhang 1 , Hongwei Xue 1,† 1 Accio Team, Alibaba Group 2 Shanghai Innovation Institute, 3 Fudan University Abstract Recent multimodal deep search agents have advanced tool use and visual content retrieval, yet they still struggle to convert intermediate visual evidence into actionable guidance for subsequent search, limiting their effectiveness in long-horizon settings. To address this gap, we present Neos, a multimodal deep search framework that learns both how to acquire evidence and how to use it to guide subsequent decisions. To make this coupling learnable, Neos constructs training tasks with intermediate visual dependencies while filtering out textual shortcuts, and introduces a lightweight, trainable Multimodal Evidence Interface (MEI) that transforms retrieved content into goal-conditioned observations. At its core, an agent–environment co-evolution procedure treats MEI as a co-adaptive component of this loop. Specifically, reference-guided credit assignment uses verified successful trajectories to localize critical policy errors while preserving credit for useful decisions in failed searches, whereas rollout feedback selects informative local evidence-processing interactions for MEI supervised fine-tuning (SFT). By alternating policy reinforcement learning (RL) with MEI SFT, Neos allows the policy’s evolving evi- dence requests to shape MEI training and the adapted MEI to reshape subsequent observations, closing the loop between evidence acquisition and evidence-conditioned action. Across ten benchmarks, Neos- 9B and Neos-35B demonstrate leading performance among specialized multimodal deep-search agents. Project Page: https://Accio-Lab.github.io/Neos GitHub Repo: https://github.com/Accio-Lab/Neos MMSearch 90 85 80 75 70 81.3 74.0 72.9 83.6 85.7 85.0 88.7 MMSearch+ 70 60 50 40 30 20 47.6 40.6 26.6 34.4 63.3 61.1 65.9 BC-VL 70 65 60 55 50 45 63.2 64.2 57.2 55.9 66.4 61.7 66.4 M-BC 60 50 40 30 20 10 39.5 30.5 23.5 28.2 54.5 55.3 59.5 DeepVoyagerVL30B REDSearcherMM30B SimpleSearchVL30B Qwen3.8 - Max Gemini - 3.1 - Pro GPT - 5.6 - Sol Neos - 9B 83.7 Neos - 35B 52.7 66.4 48.3 DeepVoyagerVL30BREDSearcherMM30B SimpleSearchVL30B Qwen3.8 - Max Gemini - 3.1 - Pro GPT - 5.6 - Sol Neos - 9B Neos - 35B DeepVoyagerVL30BREDSearcherMM30B SimpleSearchVL30B Qwen3.8 - Max Gemini - 3.1 - Pro GPT - 5.6 - Sol Neos - 9B Neos - 35B DeepVoyagerVL30BREDSearcherMM30B UniBrowse35 B Qwen3.8 - Max Gemini - 3.1 - Pro GPT - 5.6 - Sol Neos - 9B Neos - 35B Figure 1: Benchmark performance of Neos-9B and Neos-35B. ∗ Equal contribution. ‡ Project leaders. † Corresponding author. This work was done during Ekko He’s internship at the Accio Team, Alibaba Group. arXiv:2610.17043v1 [cs.AI] 12 Oct 2026 1 Introduction Real-world information-seeking problems are often open-ended and grounded in both textual and visual evidence. Deep search agents have made substantial progress in text-centric settings by iteratively searching the web, browsing sources, and synthesizing grounded answers [Jin et al., 2025, Zheng et al., 2025, Wu et al., 2026, Li et al., 2025a, Team et al., 2025b,a, Li et al., 2026]. Recent multimodal systems extend these agents with multimodal tool use, multi-turn visual interaction, scalable data synthesis, and visual context management [Hong et al., 2026, Chng et al., 2025, Zhang et al., 2026c,d, Yao et al., 2026, Huang et al., 2026a, Liu et al., 2026b, Du et al., 2026]. Despite these advances, many systems fail to consistently leverage visual evidence during search. Yet such evidence is pivotal for guiding subsequent queries, interpreting retrievals, and determining the need for further inspection [Li et al., 2025b, Tao et al., 2026, Zeng et al., 2026]. The central challenge is therefore to sustain an evidence–decision loop, in which search decisions determine which evidence is acquired, and newly acquired evidence guides subsequent decisions. Existing methods still exhibit three intertwined limitations: I) Weak intermediate visual depen- dencies. Training and evaluation tasks often do not require newly acquired visual evidence to influence later search, while synthesized tasks may confine vision to initial recognition or final attribute extraction [Wu et al., 2025, Zhang et al., 2026d], while captions, metadata, or indexed text can bypass intermediate visual reasoning [Zeng et al., 2026]. Consequently, agents can succeed without internalizing the paradigm of using newly acquired visual evidence to guide subsequent decisions. I) Ambiguous process credit. Prior work provides fine-grained learning signals through advantage estimation, reward shaping, and process-aware optimization [Feng et al., 2026, Xia et al., 2026, Lu et al., 2026, Liu et al., 2026a, Yan et al., 2026a]. However, these signals often lack an evidence-grounded reference for attributing trajectory-level outcomes to individual decisions. Final- answer supervision therefore remains too global to identify critical errors, distinguish them from downstream consequences, or preserve credit for useful decisions in failed searches. I) Decoupled policy and evidence interface. Retrieved content is commonly converted into observations by fixed, often proprietary auxiliary models [Zhang et al., 2026b]. Beyond their inference cost, these interfaces cannot adapt to the policy’s evolving evidence needs, preventing evidence presentation and search behavior from improving together. To address these limitations, we present Neos, a multimodal deep search framework that learns both how to acquire evidence and how to use it to guide subsequent decisions. Neos closes this evidence– decision loop through three complementary components: I) Evidence-dependent data synthesis. We construct tasks with intermediate visual dependencies and screen for textual shortcuts, producing questions and trajectories in which newly acquired visual evidence is necessary for later search. I) Reference-guided credit assignment. Using verified successful trajectories and the evidence available at each turn, our credit-assignment method localizes critical errors in evidence seeking and use, attenuates penalties for their downstream consequences, and preserves positive credit for useful decisions in failed searches. This provides targeted reinforcement-learning signals for both evidence acquisition and evidence-conditioned decision making. I) Agent–environment co-evolution. We embed a lightweight, trainable Multimodal Evidence Interface (MEI) into the search tools to transform retrieved content into goal-conditioned observations, while leaving search and on-demand visual inspection under policy control. After supervised initialization, Neos alternates policy reinforcement learning (RL) with MEI supervised fine-tuning (SFT). Rollout outcomes and shared process feedback select local evidence-processing interactions for MEI training, including useful interactions from failed searches. The policy’s evolving evidence needs thereby shape MEI training, while the adapted MEI reshapes observations for policy learning. Our contributions are fourfold: •We introduce an evidence-dependent synthesis pipeline that produces multimodal search tasks in which newly acquired visual evidence is necessary for subsequent search decisions. •We propose reference-guided credit assignment, which localizes critical policy errors and their downstream consequences, while preserving positive credit for productive and corrective decisions in failed searches. •We develop a lightweight Multimodal Evidence Interface (MEI) and an agent–environment co-evolution procedure, allowing joint improvement of evidence presentation and the search policy. •Across ten benchmarks, Neos-9B and Neos-35B outperform their respective base workflows by 17.4 and 15.5 points on average, achieving leading open-source multimodal deep-search performance. 2 2 Related Work From multimodal understanding to active information seeking. MLLMs have advanced visual understanding and reasoning but remain limited by parametric knowledge and inference-time ev- idence. AVIS and Vision Search Assistant extend visual question answering with tools and web search [Hu et al., 2023, Zhang et al., 2024]. Recent “thinking with images” methods integrate visual interaction into reasoning: DeepEyes learns image manipulation via RL [Zheng et al., 2026]; Adaptive-CoF performs question-guided region selection and zooming, and MGPO learns multi-turn visual grounding from answer-level rewards [Zhang et al., 2025a, Huang et al., 2026c]; RSAgent uses visual feedback for segmentation [He et al., 2025]; PyVision and Thyme use generated Python tools and executable code for visual computation [Zhao et al., 2025a, Zhang et al., 2025b]; Mini-o3 extends exploratory visual reasoning to longer interactions [Lai et al., 2026]; and MAPO aligns tool outputs with visual observations in policy optimization [Yang et al., 2026]. These methods make visual inspection and manipulation part of reasoning, but multimodal deep search must iteratively acquire and verify external evidence to guide subsequent search. Multimodal deep search agents. Multimodal agents learn multi-turn evidence acquisition through RL, e.g., MMSearch-R1 and WebWatcher [Wu et al., 2025, Geng et al., 2026b]. ProMMSearchAgent uses process-oriented rewards, and OpenSearch-VL preserves useful prefixes via fatal-aware masking and advantage clamping [Yan et al., 2026b, Chen et al., 2026]. Scalable supervision includes graph-based task construction and trajectory synthesis [Chu et al., 2026, Yao et al., 2026]; VSearcher combines iterative-injection data synthesis with SFT-then-RL [Zhang et al., 2026c], while ODE adapts data generation from rollout feedback [Huang et al., 2026a]. Evidence handling also differs: Vision- DeepResearch supports multi-turn, multi-entity, and multi-scale search [Huang et al., 2026b]; LMM- Searcher separates image references from on-demand acquisition [Du et al., 2026]; POINTS-Seeker renders older observations as images while keeping recent text [Liu et al., 2026b]; DeepVoyager-VL constructs intermediate visual dependencies with helper-mediated observations and reusable visual state [Zhang et al., 2026b]; and SimpleSearch-VL places webpage summarization and thumbnail verification within the policy [Dai et al., 2026]. 3 Method 3.1 Problem Setting and Agent Workflow We formulate multimodal deep search as an evidence–decision loop. Given a questionqand initial imagesI 0 , the policyπ θ interacts with a tool environmentE φ to produce an answer. At turnt, history h t contains the inputs and preceding action–observation pairs, a t ∼ π θ (·| h t ), o t ∼E φ (·| h t ,a t ),(1) with h t+1 = h t ⊕ (a t ,o t ). The interaction ends when the policy produces a final answer. NeosdiscoverscandidateevidencethroughTextSearch,ImageSearch,and ReverseImageSearch, and accesses source pages viaWebVisit.These tools return text and image URLs, whileFetchImageandCropImageexpose selected images and regions on demand [Du et al., 2026]. Loaded images and crops remain accessible through reusable identifiers in an episode-local visual register [Zhang et al., 2026b]. To present retrieved multimodal content as policy observations, Neos incorporates a trainable Multimodal Evidence Interface (MEI) withinWebVisit,ImageSearch, andReverseImageSearchto convert search results into goal-conditioned observations. The parametersφgovern this interface withinE φ , while the underlying web and search services remain fixed (Section 3.2). 3.2 Multimodal Evidence Interface Online content becomes useful for decision-making only after the agent identifies evidence relevant to its current search goal. Existing search pipelines often delegate this processing to large or proprietary auxiliary models [Wu et al., 2025, Zhang et al., 2026b], incurring repeated inference latency and serving cost. When these models remain fixed, evidence processing cannot adapt to the evolving policy’s needs. MEI addresses these limitations through a lightweight, trainable vision–language model, g φ , that adapts evidence presentation to the policy’s current search objective. 3 (a) Structured Question Construction 1 Reasoning Structures 2 Blind Visual Grounding 3 Question Rewriting (b) Shortcut Filtering (c) Expert Trajectory Synthesis and MEI Supervision Wikipedia-linked Wiki-data seeds Backward expansion via distinguishing sets Entry image e₀ e₁ u r₁ r₂ e α Optional convergence Online images and source pages Full image Crop image Target names and answers hidden Independent observation Global DescriptionsLocal Descriptions A B C D Verified visual bindings Attribute answers require support from both views Use verified images and observations Visual rewriting Remove identity clues and procedural wording Answer preserved Relations and directions covered Visual dependencies retained Check identity- revealing search shortcuts Candidate questions Screening under fixed models and budgets 1 Text-only direct answering 2 Text-only search 3 Entry-image VQA 4 Short multimodal search <answer>:object/attribute Multi Judge Answer Correct Reason Chain Correct Process Effectiveness Retrieved image Target view · The final answer must match GT · No multi answer e ᵦ e* Answer seed · With reason chain as input (no GT), judge can infer GT. DAG and Reasoning Structure Construction · Candidates · Fetch & search · Not long-tailed · Max length < 128k · objects · scene attributes · visual Details · fine grained details · attributes · spatial relationships 1. Candidate matching with distractors 2. Reverse search/ source corroboration · Base question Final question Step-to-span alignment 1 Text span 1 2 Text span 2 3 Verified Complete Trajectories Text span 3 Relation direction Long horizon multimodal deep search trajectories and MEI I/O Any Solved: →Reject All Unsolved: →Retain Goal: discard if · textual shortcut · no need of image · no need of search · easy to solve Trajectory Generation Text Search Image Search Reverse Image SearchWeb Visit Fetch Image Crop Image Tool Environment Expert Model Multimodal Evidence Interface RL reference Observation Interact with Online Web Env Raw Rollout Policy SFT image search Reasoning Content 1 Turn 1 <tool>: · Turn 2 Reasoning Content 2 <tool>:fetch image Preserve local goal + supplied evidence Verifylocal targets MEI SFT Turn N Reasoning Content n Figure 2: Data synthesis pipeline. Reasoning structures with visual checkpoints undergo blind visual grounding, question rewriting, and shortcut screening. Verified successful searches provide policy demonstrations, success references, and local MEI training interactions. Embedded inWebVisit,ImageSearch, andReverseImageSearch, MEI is implemented by a lightweight vision–language model. Conditioned on the policy’s local goal and the returned material, it extracts relevant page evidence, filters and ranks image candidates, and assesses visual matches. Thus, evidence is selected and summarized for the current search objective rather than presented as an undifferentiated collection of retrieved material. It inspects candidate images internally, but returns structured text and image references as ordinary tool observations. The agent does not invoke MEI as a separate model and retains control over query formulation and on-demand visual inspection. This separates learning what evidence to present from learning how to act on it. 3.3 Evidence-Dependent Task Synthesis Our data pipeline is designed to instantiate the evidence–decision loop at the task level: an agent must acquire intermediate visual evidence, and that evidence must constrain subsequent search and reasoning. The presence of images alone does not guarantee this property. Tasks may restrict vision to initial recognition or final attribute extraction, while textual clues can reveal identities intended to be inferred visually. We therefore construct tasks in which intermediate visual evidence supplies entity bindings for later reasoning steps, and combine structural checks, blind visual grounding, and shortcut screening before collecting verified searches (Figure 2). Reasoning structures and identifiability. To make intermediate evidence decision-relevant, we construct reasoning structures in which visual checkpoints are required for later relational inference. Starting from Wikipedia-linked Wikidata entities, we recursively expand distinguishing sets: neigh- boring relational constraints that jointly identify an entity within a knowledge-graph (KG) snapshot K. The resulting DAG connects prerequisites to identified entities while preserving the direction of each KG fact. Reasoning structures comprise paths or convergent subgraphs with a visual entry anchor and at least one intermediate visual checkpoint; converging branches have disjoint entity sets before their shared entity. These checkpoints connect visual entity identification to subsequent relational reasoning, making newly acquired visual evidence a prerequisite for later steps. To check that the specified evidence bindings identify an unambiguous target, we evaluate U K (C,B) =    u ∃z, z out = u, (v,r,w)∈C (z v ,r,z w )∈K, v∈B z v = b v    .(2) 4 (a) Agent–Environment Co-evolution Update θ Update φ Outcome + process feedbackLocal MEI input–response pairs Search policy π θ Neos-9B MEI stays inside retrieval tools Tool call + local goal Tool observations Repeated evidence acquisition Policy RL Terminal-answer rewards + Reference-guided credit (b) Group-relative policy update MEI SFT Self-generated response targets Every K policy updates Outcome / process selection (b) · Success: all valid calls · Failure: calls at productive or corrective turns Online rollouts (b) Reference-Guided Credit Assignment Failed search trajectory Successful reference Multimodal credit judge Localize critical errors Critical errors and downstream credit P t1 E t2 D t3 D t4 C t5 E t6 D t7 D t8 + − correctivenew error decaydecay P Productive E Critical error D Downstream C Corrective Bounded turn-level advantages Positive credit for useful turns Sum of token advantages < 0 for assessed failures Neutral turns receive zero credit. Text Search Image Search Reverse Image Search Web Visit Fetch ImageCrop Image Tool Environment Multimodal Evidence Interface (MEI) g φ error <answer>: object/attribute Raw Rollout image search Reasoning Content 1 Turn 1 <tool>: · Turn 2 Reasoning Content 2 <tool>:fetch image Turn N Reasoning Content n Rollout trajectories step-by-step. Turn 3 Reasoning Content 3 <tool>: Crop image Group Rollout · · · · · · Successfulsearch trajectory Trajectory- level reward Encourage trial and error Figure 3: Agent–environment co-evolution. Policy RL alternates with MEI SFT on outcome- and process-filtered online interactions. Reference-guided credit supports policy optimization and identifies useful local interactions for MEI updates. Here,Cdenotes relational constraints,Bdenotes bindings at the entry and checkpoints,zassigns entities to graph variables, andz out denotes the final target entity. We retain structures satisfying U K (C,B) = u ⋆ , ensuring that the seedu ⋆ is uniquely identified withinKconditional on the specified bindings. This check establishes structural identifiability; the following stages assess whether the required evidence is visually grounded and whether textual bypasses remain. Blind visual grounding. To ensure that the required bindings are genuinely obtainable from visual evidence, an observer model describes full images and crops independently, without target identities or expected answers. These descriptions are matched against candidates containing distractors, with the target unmarked. Bindings require distinctive visual evidence or compatible observations corroborated by reverse search or source pages; visual attribute answers require support from both full-image and crop observations. Question rewriting and filtering. Even with evidence-dependent structures, a question can still be solvable through leaked names or text-only shortcuts. We verbalize each reasoning structure, then use a multimodal rewriter conditioned on verified images, crops, and observations to remove revealing names and procedural wording while preserving the answer and dependencies. An independent check aligns each step with a span in the final question to verify coverage and relation direction, while targeted text searches screen for wording that exposes hidden identities. We then apply four sequential probes—text-only answering, text-only search, entry-image VQA, and multimodal search—and reject questions solved by any probe, with limited budgets for the search probes. The text-only and entry- image probes screen for routes that bypass intermediate visual acquisition, while the budget-limited multimodal probe additionally filters task difficulty. Trajectory curation. For retained questions, an expert model conducts long-horizon searches, producing trajectories that follow the desired evidence–decision loop. Successful trajectories serve as demonstrations and success references for credit assignment, while local tool interactions provide MEI training inputs. The resulting supervision connects task construction to both search-policy learning and evidence-interface training. Further details appear in Appendix A. 3.4 Reference-Guided Credit Assignment An incorrect final answer does not invalidate all preceding decisions; a failed search may still acquire relevant evidence or correct earlier mistakes. Although process-oriented rewards and prefix-preserving updates provide finer-grained supervision [Yan et al., 2026b, Chen et al., 2026], uniformly propagating trajectory-level penalties still obscures turn-level contributions. Neos instead uses verified successful trajectories as references to identify the role of each turn. 5 Supervised initialization. The agent and MEI are initialized on their respective curated examples using a masked autoregressive objective normalized over target tokens. Policy targets comprise reasoning, tool calls, and final answers, while MEI targets comprise structured evidence responses. Conditioning content, including tool observations, and padding are excluded from the loss. Reference-guided assessment. For each question, we sampleGrollouts underπ θ old andE φ , retaining complete groups containing both successes and failures. Binary answer-correctness rewardsr i define group-relative advantagesA i = (r i − ̄r)/(std(r) + ε std ), where ̄r = G −1 P j r j . For failed trajectories, a credit judge assesses recorded actions and observations against a verified successful reference. The reference grounds assessment in a demonstrated successful investigation, rather than prescribing a required sequence of actions. The judge identifies critical policy errors, their downstream consequences, and productive or corrective decisions while respecting the evidence available at each turn. For successful trajectories, alternative valid search routes are permitted. Turn-level credit. We focus negative credit on critical errors and attenuate penalties on their downstream consequences. Productive and corrective decisions receive positive credit, so useful evidence acquisition and recovery can be reinforced even within failed searches, rather than merely exempted from penalties. For an assessed failure, we set b i =|A i | and assign w i,t =              − max(κ e ,s i,t c i,t ),critical error, −λ d max(κ d ,s i,t c i,t )γ d i,t −1 , downstream consequence, λ c c i,t ,corrective, λ p c i,t ,productive, 0,neutral. (3) Here,s i,t andc i,t denote severity and confidence. Unannotated turns following an active error inherit its severity and confidence, withd i,t ≥ 1measuring elapsed turns; otherwise, they are neutral. Corrective turns stop propagation, and subsequent critical errors restart it. Each policy tokenk receives e A i,k = b i w i,t(k) . Additional engineering techniques of RL are provided in Appendix B. Policy optimization. We rescale positive and negative credit into bounded advantages b A i,k , keeping their sum negative for each assessed failure while preserving the uniform advantageA i for successful trajectories. This reinforces useful local decisions while keeping failed trajectories negative in aggregate. LetM i index policy-generated tokens andρ i,k (θ)denote the likelihood ratio betweenπ θ and π θ old on the recorded prefix. Using the dual-clipped surrogate ℓ dc , we optimize L RL (θ) =− P i P k∈M i ℓ dc ρ i,k (θ), b A i,k P i |M i | .(4) Outcome rewards remain unchanged, and credit estimates are held fixed during optimization. The SFT objective, credit rescaling, and dual-clipping definitions appear in Appendix D. The same process labels also select useful local MEI interactions, connecting decision-level credit with evidence- interface adaptation in Section 3.5. 3.5 Agent–Environment Co-evolution Optimizing either component while holding the other fixed limits adaptation: a fixed policy exposes MEI to a static distribution of evidence requests, whereas a fixed MEI cannot adjust its observations to the policy’s evolving needs. Neos instead co-evolves the two components, allowing evidence presentation and search behavior to adapt to each other and improve jointly (Figure 3). Outcome- and process-filtered supervision. Since the evidence environment must adapt to the policy’s evolving evidence requests, its supervision is drawn from online agent–MEI interactions rather than static demonstrations alone. After supervised initialization, we record each agent–MEI interaction, including inputs to MEI, its generated response, the corresponding policy turn and the outcome. Correct trajectories contribute all valid MEI calls; failed trajectories selected for agent optimization contribute only calls labeled productive or corrective. This filtering allows MEI to learn from evidence-processing interactions that remain useful even when the final search fails. Alternating adaptation. We collect agent rollouts asynchronously under a fixed MEI version and update the agent after each. After everyKagent updates, MEI is fine-tuned from its latest weights 6 Table 1: Comparison of multimodal search methods across 10 benchmarks. The best and second-best results are bolded andunderlined. Green∆rows report absolute score improvements over Qwen3.5- 9B and Qwen3.5-35B-A3B under our workflow for the SFT and RL models. ModelMMSearch MMSearch+ BC-VL M-BC VisBrowse BC-V 3 VDR SimpleVQA LiveVQA FVQAOverall Direct Answer GPT-5.6-Sol68.321.547.416.836.722.719.269.373.063.743.9 Gemini-3.1-Pro 62.324.141.611.824.318.716.064.764.357.738.5 Kimi-K361.317.040.111.830.815.311.470.072.761.039.1 Qwen3.8-Max53.015.841.114.028.411.78.872.768.759.037.3 Qwen3.5-35B-A3B26.74.525.38.517.86.32.655.356.334.323.8 Qwen3.5-9B22.31.924.86.817.25.02.252.754.728.321.6 Agentic Workflow GPT-5.6-Sol88.765.966.459.570.455.744.687.094.088.772.1 Gemini-3.1-Pro85.061.161.755.363.351.340.283.088.781.367.1 Kimi-K3 82.365.064.756.568.053.333.885.091.082.068.1 Qwen3.8-Max85.763.366.454.566.956.339.085.793.086.769.7 Qwen3.5-35B-A3B71.028.051.429.042.030.719.471.783.765.049.2 Qwen3.5-9B66.324.843.622.833.722.317.066.774.761.343.3 Multimodal Deep Search Agents MMSearch-R1-7B53.8–4.0–57.448.458.4– DeepEyesV2-7B 63.7–59.4–60.6– Vision-DeepResearch-8B69.620.442.6–11.729.2–76.764.7– M-DeepResearch-8B67.8–37.9–65.965.069.2– POINTS-Seeker-8B70.825.244.4–68.877.771.2– OpenSearch-VL-8B64.5–37.6–20.871.659.671.5– Visual-Seeker-8B72.227.347.616.134.7– SearchEyes-9B69.2–42.3–28.375.466.174.2– SimpleSearch-VL-8B77.132.552.1–76.675.276.8– DeepVoyager-VL-8B72.737.158.424.047.332.335.076.382.782.754.9 Neos-9B (Ours, SFT)79.744.760.437.849.135.330.282.387.779.758.7 ∆ SFT vs. 9B workflow+13.4+19.9+16.8+15.0+15.4+13.0+13.2+15.6+13.0+18.4+15.4 Neos-9B (Ours, RL)81.347.663.239.553.337.031.481.388.783.360.7 ∆ RL vs. 9B workflow+15.0+22.8+19.6+16.7+19.6+14.7+14.4+14.6+14.0+22.0+17.4 WebWatcher-32B55.3–27.0–8.7–59.058.7– SenseNova-MARS-32B74.3–74.160.872.6– Vision-DeepResearch-30B-A3B 69.628.553.7–37.8–77.674.2– REDSearcher-M-30B-A3B 72.926.657.223.5–79.3– M-DeepResearch-32B 69.0–43.0–67.668.070.1– LMM-Searcher-30B-A3B72.334.8–30.148.3– OpenSearch-VL-32B72.3–43.8–33.876.270.574.7– SearchEyes-27B82.4–49.3–39.480.977.379.1– SimpleSearch-VL-30B-A3B83.634.455.9–79.681.179.0– UniBrowse-35B-A3B75.234.157.828.2–76.7– DeepVoyager-VL-30B-A3B74.040.664.230.553.835.039.481.082.784.758.6 Neos-35B (Ours, SFT)81.048.661.444.852.745.331.081.389.379.761.5 ∆ SFT vs. 35B workflow+10.0+20.6+10.0+15.8+10.7+14.6+11.6+9.6+5.6+14.7+12.3 Neos-35B (Ours, RL)83.752.766.448.357.447.033.682.791.383.764.7 ∆ RL vs. 35B workflow+12.7+24.7+15.0+19.3+15.4+16.3+14.2+11.0+7.6+18.7+15.5 on newly retained calls collected since its previous update. The evolving agent determines which evidence-processing interactions become available for MEI training, and the updated MEI changes the observations used for subsequent agent decisions. This alternating procedure creates a reciprocal adaptation loop: policy updates reshape MEI’s training signal, and MEI updates reshape the policy’s decision context, while the underlying web and tool APIs remain unchanged. 4 Experiments 4.1 Experimental Setup Benchmarks. We evaluate Neos on ten benchmarks spanning factual visual question answering, multi-step information search, and reasoning over online visual evidence. The evaluation suite comprises MMSearch [Jiang et al., 2024], SimpleVQA [Cheng et al., 2025], LiveVQA [Fu et al., 2025], FVQA [Wu et al., 2025], BrowseComp-VL [Geng et al., 2026b], M-BrowseComp [Li et al., 2025b], MMSearch-Plus [Tao et al., 2026], VDR-Bench [Zeng et al., 2026], BrowseComp-V 3 [Zhang et al., 2026a], and VisBrowse-Bench [Zhang et al., 2026e]. 7 Table 2: Training-data ablation. The VQA mixture draws from FVQA, LiveVQA, InfoSeek, and Visu- alProbe, with the same expert model for trajectory generation and rejection sampling. Table 3: Co-evolution ablation. Initial and adapted refer to MEI before and after co- evolution. The RL settings use the same SFT policy initialization. SFT dataBC-VL MMSearch+ FVQA Base Qwen3.5-9B43.624.861.3 VQA mixture (0.7K)53.432.972.7 REDSearcher-M (1.1K)56.938.977.3 Our synthetic data (1K)58.443.278.3 Full mixture (2.9K)60.444.779.7 Policy MEIMMSearch+ BC-V 3 VDR SFTInitial42.833.728.0 SFTAdapted44.735.330.2 RLFrozen initial45.835.728.2 RLFrozen adapted46.936.330.4 Alternating47.637.031.4 Baselines. We evaluate three categories of baselines, covering proprietary and open-source MLLMs as well as specialized multimodal search agents (Table 1). Direct Answer measures model capability using only the input question and images, without external tools. Agentic Workflow allows the same models to gather textual evidence and inspect images through our tool framework, assessing the benefit of iterative search without additional model training. Multimodal deep search Agents compares against existing systems specifically trained for multimodal search and browsing, providing a comparison with learned search policies. Implementation Details. We perform cold-start SFT of Qwen3.5-9B and Qwen3.5-35B-A3B [Team et al., 2026b] using MS-Swift [Zhao et al., 2025b] on 8 NVIDIA H100 GPUs for two epochs. Agentic RL is implemented in verl [Sheng et al., 2025] with Megatron [Shoeybi et al., 2019] as the training backend. The co-evolution stage uses 40 H100 GPUs, with 32 allocated to agentic RL and 8 to MEI serving and fine-tuning. Policy RL is performed for one epoch with 8 rollouts per question, using a batch size of 64 for Neos-9B and 32 for Neos-35B. The vision encoder and multimodal merger remain frozen throughout training, and evaluation allows up to 50 interaction turns per question. Detailed experimental settings are provided in Appendix C. 4.2 Main Results As shown in Table 1, Neos achieves leading performance among specialized multimodal deep-search agents at both model scales. The advantage is particularly pronounced for Neos-35B on MMSearch- Plus, M-BrowseComp, and BrowseComp-V 3 , where it achieves 52.7, 48.3, and 47.0, surpassing the strongest prior specialized-agent results by 12.1, 17.8, and 12.0 points. It also reaches 57.4 on VisBrowse-Bench and 91.3 on LiveVQA, demonstrating strong capabilities across both multimodal browsing and visual question answering. The training progression further highlights the value of specialized training beyond merely providing tool access. With our workflow, the Qwen3.5-9B and Qwen3.5-35B-A3B backbones improve from 21.6 and 23.8 to 43.3 and 49.2. The proposed training pipeline effectively strengthens the models’ interactive search, tool-use, and reasoning capabilities, yielding final scores of 60.7 and 64.7. These correspond to gains of 17.4 and 15.5 points over the corresponding base workflows. Neos-9B already outperforms several 30B-scale search agents, while the stronger results of Neos-35B show that effective agent training and larger model capacity are complementary. Overall, Neos approaches frontier-model performance, demonstrating that specialized multimodal search agents can be made highly competitive with frontier workflows. 4.3 Ablation Studies Unless otherwise specified, ablations follow the settings of the corresponding stage in our main experiments. Except for the MEI-related ablations, all experiments use our final co-evolved MEI. Additional ablations and analyses are provided in Appendix E. Training data. Table 2 compares SFT with different training sources. All trajectories, including those based on REDSearcher-M and the VQA mixture, are generated with the same expert model. Our 1K synthetic examples yield 58.4, 43.2, and 78.3 on BrowseComp-VL, MMSearch-Plus, and FVQA, exceeding the REDSearcher-M variant by 1.5, 4.3, and 1.0 points. Combining all sources further improves performance to 60.4, 44.7, and 79.7, suggesting complementary benefits from general visual question answering and structured multimodal search supervision. 8 Table 4: MEI backbone ablation.Table 5: Credit-assignment ablation. MEI modelMM-BC BC-VL VisBrowse Gemini-3.7-Flash41.364.254.4 Qwen-3.7-Flash39.362.451.5 GPT-5.6-Luna38.761.252.7 GPT-5-nano36.557.947.9 Qwen3.5-9B35.058.146.7 Co-evolved MEI (9B)39.563.253.3 Training variantBC-VL M-BC FVQA Cold-start SFT60.437.879.7 Outcome-only GRPO59.635.378.3 Productive credit only60.436.377.7 + First-error penalty61.237.880.7 + Downstream attribution61.938.381.3 + Corrective credit62.739.082.0 Full credit assignment63.239.583.3 Agent–environment co-evolution. Table 3 separates the benefit of a stronger MEI from that of adapting it alongside the policy. Beyond evaluating the SFT policy with different MEI checkpoints, we compare three RL settings from the same SFT policy initialization with matched policy-update budgets: freezing the initial MEI, freezing the co-evolved MEI, and alternating policy RL with MEI SFT. Using the co-evolved MEI as a fixed interface improves over freezing the initial MEI by 1.1, 0.6, and 2.2 points on MMSearch-Plus, BrowseComp-V 3 , and VDR-Bench. Alternating updates further improve performance by 0.7, 0.7, and 1.0 points over this stronger fixed-interface control. These results show that a better MEI accounts for part of the improvement, while continued adaptation during policy optimization provides additional gains across all three benchmarks. 81.7% 12.2% 0.6% 1.6% 2.3% 1.6% MMSearch 49.2% 27.8% 8.3% 7.7% 5.1% 2.0% MMBC 78.0% 16.6% 0.5% 1.9% 1.2% 1.8% BCVL 68.8% 15.9% 3.3% 3.4% 3.9% 4.6% MMSearchPlus 52.6% 26.1% 8.1% 3.8% 3.5% 5.9% BC-V3 52.8% 18.3% 10.1% 10.5% 5.9% 2.4% VisBrowse 67.0% 19.9% 1.6% 3.3% 3.7% 4.5% VDR 73.2% 9.6% 6.4% 1.6% 4.1% 5.0% SimpleVQA 74.5% 12.3% 1.3% 4.2% 3.5% 4.3% LiveVQA 77.3% 10.6% 0.8% 4.3% 3.7% 3.3% FVQA TextSearchWebVisitImageSearchReverseImageSearchFetchImageCropImage Figure 4: Tool call distribution of ten benchmarks. MEI backbone. We fix the final Neos-9B policy and vary the model serving as MEI in Ta- ble 4. Relative to the Qwen3.5-9B MEI baseline, our co-evolved MEI improves M-BrowseComp, BrowseComp-VL, and VisBrowse-Bench by 4.5, 5.1, and 6.6 points, respectively. It outperforms Qwen-3.7-Flash, GPT-5.6-Luna, and GPT-5-nano on all three benchmarks, while remaining within 1.8 points of Gemini-3.7-Flash on each. These results show that a task-adapted 9B evidence interface can support search performance competitive with strong off-the-shelf models. Reference-guided credit assignment. Table 5 progressively introduces the credit components. Outcome-only GRPO falls below the cold-start SFT baseline on all three benchmarks, while productive-only credit provides mixed gains. Adding first-error penalties improves over productive- only credit by 0.7, 1.5, and 3.0 points on BrowseComp-VL, M-BrowseComp, and FVQA, respec- tively. Downstream attribution and corrective credit then yield further improvements across all three benchmarks. The full method reaches 63.2, 39.5, and 83.3, exceeding outcome-only GRPO by 3.6, 4.2, and 5.0 points. This progression highlights the value of distinguishing critical errors, downstream consequences, and useful actions rather than rewarding productive steps alone. Tool-use analysis. Figure 4 shows thatTextSearchaccounts for 81.7 and 78.0 of calls on M- Search and BrowseComp-VL. This prevalence may partly reflect the relatively low overhead of text-based exploration, together with online textual shortcuts that allow some benchmark questions to bypass visual search [Zeng et al., 2026]. In contrast, its share falls to 49.2, 52.6, and 52.8 on M- BrowseComp, BrowseComp-V 3 , and VisBrowse-Bench, where visual tools contribute approximately 23, 21, and 29. These patterns suggest that more demanding visual investigations shift the tool mix toward image acquisition and inspection, beyond comparatively inexpensive textual exploration. 9 1–5 6–10 11–1516–2021–2526–3031–3536–4041–4546–49 50 0 20 40 60 Trajectories (%) MMSearch 1–5 6–10 11–1516–2021–2526–3031–3536–4041–4546–49 50 MMBC 1–5 6–10 11–1516–2021–2526–3031–3536–4041–4546–49 50 BCVL 1–5 6–10 11–1516–2021–2526–3031–3536–4041–4546–49 50 MMSearchPlus 1–5 6–10 11–1516–2021–2526–3031–3536–4041–4546–49 50 BC-V3 1–5 6–10 11–1516–2021–2526–3031–3536–4041–4546–49 50 0 20 40 60 Trajectories (%) VisBrowse 1–5 6–10 11–1516–2021–2526–3031–3536–4041–4546–49 50 VDR 1–5 6–10 11–1516–2021–2526–3031–3536–4041–4546–49 50 SimpleVQA 1–5 6–10 11–1516–2021–2526–3031–3536–4041–4546–49 50 LiveVQA 1–5 6–10 11–1516–2021–2526–3031–3536–4041–4546–49 50 FVQA Assistant turns CorrectNot correct Figure 5: Neos-9B’s turn distribution across ten benchmarks. 0 20 40 60 80 100 Share of tool calls (%) 62.3 59.7 55.1 51.9 48.6 46.1 42.0 42.9 44.5 19.8 21.2 25.5 25.9 28.4 30.3 32.8 33.8 32.1 4.3 4.3 4.5 5.2 5.4 5.9 6.6 6.3 5.7 9.0 9.7 9.1 10.7 11.2 10.7 10.8 9.1 10.2 SFT cold start TextSearch WebVisit ImageSearch ReverseImageSearch FetchImage CropImage 1 (SFT) 1–56–1011–1516–2021–2526–3031–3536–40 RL update window 20 30 Calls / trajectory 30.5 29.7 27.2 25.2 23.4 22.1 20.920.9 20.2 Figure 6: Tool usage distribution during Neos-9B’s co-evolution training. Search horizons across benchmarks. Figure 5 shows Neos-9B’s interaction lengths, separated by answer correctness. Before the 50-turn limit, MMSearch and the VQA benchmarks concentrate in the first ten turns, whereas others span broader horizons, consistent with more sustained mul- timodal investigation. All benchmarks nevertheless accumulate at the limit; these trajectories are predominantly correct on LiveVQA and FVQA but predominantly incorrect on M-BrowseComp, BrowseComp-V 3 , and VDR-Bench. Thus, search lengths are heterogeneous, and reaching the budget alone indicates neither success nor failure. Tool-use dynamics during RL. Figure 6 tracks the composition and frequency of tool calls across RL update windows. From the first to the final window, the share ofTextSearchdecreases from 59.7 to 44.5, whileWebVisitincreases from 21.2 to 32.1. The four visual tools—ImageSearch, ReverseImageSearch,FetchImage, andCropImage—collectively increase their share from ap- proximately 19.1 to 23.4. Meanwhile, the average number of calls per trajectory falls from 29.7 to 20.2, a reduction of 32.0%. Thus, the changing tool mix accompanies fewer overall calls rather than an expansion of interaction volume. These trends are consistent with a shift from text-search-heavy exploration toward more selective information gathering, with greater relative emphasis on examining source pages and visual evidence. The resulting trajectories are more compact in tool-call count while retaining a diverse set of evidence-processing operations. Tool ablation. Table 6 evaluates individual tool contributions using Qwen3.5-35B-A3B without additional search-specific policy training; removing any tool hurts all three benchmarks, with task- dependent effects.ReverseImageSearchis most critical on SimpleVQA (71.7 to 58.0), where removingImageSearchalso costs 4.7 points. By contrast,FetchImagecauses the largest drops on BrowseComp-V 3 and M-BrowseComp (3.7 points each), whileCropImageandWebVisitalso contribute consistently. Together, these results support combining evidence discovery, page reading, and visual inspection in the Neos workflow. 10 Table 6: Tool ablation using Qwen3.5-35B- A3B base model. Table 7: Workflow comparison with our reproduc- tion of REDSearcher-M. Tool configurationSimpleVQA BC-V 3 M-BC Full Neos workflow71.730.729.0 w/o FetchImage68.727.025.3 w/o CropImage69.729.028.3 w/o WebVisit70.329.027.0 w/o ImageSearch67.028.727.7 w/o ReverseImageSearch58.027.328.3 PolicyWorkflowBC-VL M-BC MMSearch+ GPT-5.6 REDSearcher-M64.235.553.5 Neos66.459.565.9 Qwen3.5- 35B-A3B REDSearcher-M45.414.521.6 Neos51.429.028.0 Qwen3.5- 9B REDSearcher-M38.69.819.3 Neos43.622.824.8 010203040 Policy update 58 59 60 61 62 63 64 BrowseComp-VL score K= 1 K= 2 K= 5 K= 10K= 20 Figure 7: Effect of the MEI update interval on BrowseComp-VL with the 9B model.Kis the number of policy-training batches between MEI updates. Workflow comparison. Table 7 compares Neos with our reproduction of the REDSearcher-M workflow [Chu et al., 2026], holding the policy model fixed within each comparison. Neos improves performance across all nine model–benchmark pairs, with average gains of 12.9, 9.0, and 7.8 points across the three benchmarks for GPT-5.6, Qwen3.5-35B-A3B, and Qwen3.5-9B. The largest improvements occur on M-BrowseComp, reaching 24.0, 14.5, and 13.0 points. Notably, Qwen3.5- 9B with Neos surpasses Qwen3.5-35B-A3B with REDSearcher-M on both M-BrowseComp and MMSearch-Plus. These results indicate that the benefits of Neos extend across policy backbones and that workflow design can partially compensate for differences in model scale. MEI update interval. Figure 7 shows that frequent MEI updates with smallKare unstable, producing oscillations and occasional regressions across policy-training checkpoints. AsKincreases, performance becomes progressively more stable, but the gains are neither monotonic nor linear in the update interval. This suggests that batching more policy updates before refreshing MEI primarily stabilizes policy–interface co-evolution rather than yielding proportional performance improvements. 5 Conclusions We presented Neos, a multimodal deep search framework that sustains an evidence–decision loop by learning both evidence acquisition and evidence-conditioned decision making. Its evidence- dependent pipeline constructs tasks with intermediate visual dependencies while screening textual shortcuts. A lightweight Multimodal Evidence Interface (MEI) co-adapts with the policy through agent–environment co-evolution, while reference-guided credit assignment localizes critical errors and preserves credit for useful decisions in failed searches. Alternating policy RL with MEI SFT turns rollout outcomes into supervision for both policy and MEI. Across ten benchmarks, Neos-9B and Neos-35B achieve leading performance among specialized multimodal deep search agents and substantially narrow the gap to frontier proprietary models. 11 References Shuang Chen, Kaituo Feng, Hangting Chen, Wenxuan Huang, Dasen Dai, Quanxin Shou, Yunlong Lin, Xiangyu Yue, Shenghua Gao, and Tianyu Pang. Opensearch-vl: An open recipe for frontier multimodal search agents. arXiv preprint arXiv:2605.05185, 2026. Yang Chen, Hexiang Hu, Yi Luan, Haitian Sun, Soravit Changpinyo, Alan Ritter, and Ming-Wei Chang. Can pre-trained vision and language models answer visual information-seeking questions? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14948–14968, 2023. Xianfu Cheng, Wei Zhang, Shiwei Zhang, Jian Yang, Xiangyuan Guan, Xianjie Wu, Xiang Li, Ge Zhang, Jiaheng Liu, Yuying Mai, et al. Simplevqa: Multimodal factuality evaluation for multimodal large language models. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4637–4646. IEEE, 2025. Yong Xien Chng, Tao Hu, Wenwen Tong, Xueheng Li, Jiandong Chen, Haojia Yu, Jiefan Lu, Hewei Guo, Hanming Deng, Chengjun Xie, et al. Sensenova-mars: Empowering multimodal agentic reasoning and search via reinforcement learning. arXiv preprint arXiv:2512.24330, 2025. Zheng Chu, Xiao Wang, Jack Hong, Huiming Fan, Yuqi Huang, Yue Yang, Guohai Xu, Chenxiao Zhao, Cheng Xiang, Shengchao Hu, et al. Redsearcher: A scalable and cost-efficient framework for long-horizon search agents. arXiv preprint arXiv:2602.14234, 2026. Ming Dai, Zhihong Lu, Jinjie Gu, Jiedong Zhuang, Yefeng Liu, Wankou Yang, Jian Wang, and Chunhua Shen. Simplesearch-vl: A simple recipe for multimodal agentic deep search. arXiv preprint arXiv:2606.31504, 2026. Yifan Du, Zikang Liu, Jinbiao Peng, Jie Wu, Junyi Li, Jinyang Li, Wayne Xin Zhao, and Ji-Rong Wen. Towards long-horizon agentic multimodal search. arXiv preprint arXiv:2604.12890, 2026. Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. Group-in-group policy optimization for llm agent training. Advances in Neural Information Processing Systems, 38:46375–46408, 2026. Mingyang Fu, Yuyang Peng, Benlin Liu, Yao Wan, and Dongping Chen. Livevqa: Live visual knowledge seeking. arXiv e-prints, pages arXiv–2504, 2025. Xinyu Geng, Xuanhua He, Sixiang Chen, Yanjing Xiao, Fan Zhang, Shijue Huang, Haitao Mi, Zhenwen Liang, Tianqing Fang, and Yi R Fung. Deepsearch-world: Self-distillation for deep search agents in a verifiable environment. arXiv preprint arXiv:2607.07820, 2026a. Xinyu Geng, Peng Xia, Zhen Zhang, Xinyu Wang, Qiuchen Wang, Ruixue Ding, Chenxi Wang, Jialong Wu, Kuan Li, Yida Zhao, et al. Webwatcher: Breaking new frontiers of vision-language deep research agent. In International Conference on Learning Representations, volume 2026, pages 61240–61271, 2026b. Google. Gemini 3.1 Pro Preview. Gemini API documentation, 2026. URLhttps://ai.google. dev/gemini-api/docs/models/gemini-3.1-pro-preview. Accessed: 2026-09-21. Nikita Gupta, Riju Chatterjee, Lukas Haas, Connie Tao, Andrew Wang, Chang Liu, Hidekazu Oiwa, Elena Gribovskaya, Jan Ackermann, John Blitzer, et al. Deepsearchqa: Bridging the comprehensiveness gap for deep research agents. arXiv preprint arXiv:2601.20975, 2026. Xingqi He, Yujie Zhang, Shuyong Gao, Wenjie Li, Lingyi Hong, Mingxi Chen, Kaixun Jiang, Jiyuan Fu, and Wenqiang Zhang. Rsagent: Learning to reason and act for text-guided segmentation via multi-turn tool invocations. arXiv preprint arXiv:2512.24023, 2025. Jack Hong, Chenxiao Zhao, ChengLin Zhu, Weiheng Lu, and Guohai Xu. Deepeyesv2: Toward agentic multimodal model. In International Conference on Learning Representations, volume 2026, pages 114851–114872, 2026. Ziniu Hu, Ahmet Iscen, Chen Sun, Kai-Wei Chang, Yizhou Sun, David Ross, Cordelia Schmid, and Alireza Fathi. Avis: Autonomous visual information seeking with large language model agent. Advances in Neural Information Processing Systems, 36:867–878, 2023. 12 Shijue Huang, Hangyu Guo, Guanting Dong, Chenxin Li, Junting Lu, Xinyu Geng, Zhaochen Su, Zhenyu Li, Shuang Chen, Hongru Wang, et al. Towards on-policy data evolution for visual-native multimodal deep search agents. arXiv preprint arXiv:2605.10832, 2026a. Wenxuan Huang, Yu Zeng, Qiuchen Wang, Zhen Fang, Shaosheng Cao, Zheng Chu, Qingyu Yin, Shuang Chen, Zhenfei Yin, Lin Chen, et al. Vision-deepresearch: Incentivizing deepresearch capability in multimodal large language models. arXiv preprint arXiv:2601.22060, 2026b. Xinyu Huang, Yuhao Dong, Weiwei Tian, Bo Li, Rui Feng, and Ziwei Liu. Mgpo: Thinking with images via multi-turn grounding-based reinforcement learning. In Findings of the Association for Computational Linguistics: ACL 2026, pages 383–399, 2026c. Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanmin Wu, Jiayi Lei, Pengshuo Qiu, Pan Lu, Zehui Chen, Chaoyou Fu, Guanglu Song, et al. Mmsearch: Benchmarking the potential of large models as multi-modal search engines. arXiv preprint arXiv:2409.12959, 2024. Zhengbo Jiao, Yiming Cheng, Yilei Jiang, Kaituo Feng, Rui Huang, Tianyi Jiang, Juanxi Tian, Qunzhong Wang, Tailai Chen, Qianshan Wei, et al. Searcheyes: Towards frontier multimodal deep search intelligence via search world simulation. arXiv preprint arXiv:2607.05943, 2026. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516, 2025. Xin Lai, Junyi Li, Wei Li, Tao Liu, Tianjian Li, and Hengshuang Zhao. Mini-o3: Scaling up reasoning patterns and interaction turns for visual search. In International Conference on Learning Representations, volume 2026, pages 76722–76746, 2026. Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, et al. Websailor: Navigating super-human reasoning for web agent. arXiv preprint arXiv:2507.02592, 2025a. Shilong Li, Xingyuan Bu, Wenjie Wang, Jiaheng Liu, Jun Dong, Haoyang He, Hao Lu, Haozhe Zhang, Chenchen Jing, Zhen Li, et al. Mm-browsecomp: A comprehensive benchmark for multimodal browsing agents. arXiv preprint arXiv:2508.13186, 2025b. Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yongkang Wu, Ji-Rong Wen, Yutao Zhu, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. Advances in Neural Information Processing Systems, 38:120091–120131, 2026. Boyang Liu, Senjie Jin, Peixin Wang, Zhangyue Yin, Yibo Wang, Yuhao Zhou, Xinbing Liang, Shizheng Zhu, Yuhui Wang, Jingqi Tong, et al. Cafe: Self-improving search agents need co- evolving feedback. arXiv preprint arXiv:2608.24794, 2026a. Yikun Liu, Yuan Liu, Le Tian, Xiao Zhou, Jiangchao Yao, Yanfeng Wang, and Weidi Xie. Points- seeker: Towards training a multimodal agentic search model from scratch. arXiv preprint arXiv:2604.14029, 2026b. Yijun Lu, Rui Ye, Jiajun Wang, Yuwen Du, Tian Jin, Songhua Liu, and Siheng Chen. Abseeker: Training long-horizon search agents via answer-backtracked credit assignment. arXiv preprint arXiv:2608.05102, 2026. OpenAI. GPT-5.6: Frontier intelligence that scales with your ambition, July 2026. URLhttps: //openai.com/index/gpt-5-6/. Accessed: 2026-09-21. Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. Humanity’s last exam. arXiv preprint arXiv:2501.14249, 2025. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297, 2025. 13 Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catan- zaro. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053, 2019. Xijia Tao, Teng Yihua, Xinxing Su, Xinyu Fu, Jihao Wu, Chaofan Tao, Ziru Liu, Haoli Bai, Rui Liu, and Lingpeng Kong. Mmsearch-plus: Benchmarking provenance-aware search for multimodal browsing agents. In International Conference on Learning Representations, volume 2026, pages 70385–70422, 2026. Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, Jianfeng Cai, Xinyuan Cai, Peizhou Cao, Yux- uan Cao, Ziwei Chai, Y Charles, et al. Kimi k3: Open frontier intelligence. arXiv preprint arXiv:2607.24653, 2026a. MiroMind Team, Song Bai, Lidong Bing, Carson Chen, Guanzheng Chen, Yuntao Chen, Zhe Chen, Ziyi Chen, Jifeng Dai, Xuan Dong, et al. Mirothinker: Pushing the performance boundaries of open- source research agents via model, context, and interactive scaling. arXiv preprint arXiv:2511.11793, 2025a. Qwen Team. Qwen3.8-max: A new bar for coding and cowork, 2026. Qwen Team et al.Qwen3.5: Towards native multimodal agents, february 2026.URL https://qwen.ai/blog, 2026b. Tongyi DeepResearch Team, Baixuan Li, Bo Zhang, Dingchu Zhang, Fei Huang, Guangyu Li, Guoxin Chen, Huifeng Yin, Jialong Wu, Jingren Zhou, et al. Tongyi deepresearch technical report. arXiv preprint arXiv:2510.24701, 2025b. Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: A simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504.12516, 2025. Xiyu Wei, Qingwei Zong, Zhuocheng Yu, and Sujian Li. Unibrowse: A data-to-agent framework for multimodal browsecomp. arXiv preprint arXiv:2607.10557, 2026. Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhenglin Wang, Zhengwei Tao, Ding-Chu Zhang, Zekun Xi, Robert Tang, et al. Webdancer: Towards autonomous information seeking agency. Advances in Neural Information Processing Systems, 38:120957–120985, 2026. Jinming Wu, Zihao Deng, Wei Li, Yiding Liu, Bo You, Bo Li, Zejun Ma, and Ziwei Liu. Mmsearch-r1: Incentivizing lmms to search. arXiv preprint arXiv:2506.20670, 2(4), 2025. Tianle Xia, Ming Xu, Lingxiang Hu, Yiding Sun, Wenwei Li, Linfang Shang, Liqun Liu, Peng Shu, Huan Yu, and Jie Jiang. Search-p1: Path-centric reward shaping for stable and efficient agentic rag training. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), pages 517–534, 2026. Shilin Yan, Jintao Tong, Hongwei Xue, Xiaojun Tang, Yangyang Wang, Kunyu Shi, Guannan Zhang, Ruixuan Li, and Yixiong Zou. Act wisely: Cultivating meta-cognitive tool use in agentic multimodal models. arXiv preprint arXiv:2604.08545, 2026a. Wentao Yan, Shengqin Wang, Huichi Zhou, Yihang Chen, Kun Shao, Yuan Xie, and Zhizhong Zhang. Prommsearchagent: A generalizable multimodal search agent trained with process-oriented rewards. arXiv preprint arXiv:2604.20486, 2026b. Wenhao Yang, Yu Xia, Jinlong Huang, Shiyin Lu, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, Yuchen Zhou, Xiaobo Xia, et al. Walk the talk: Bridging the reasoning-action gap for thinking with images via multimodal agentic policy optimization. arXiv preprint arXiv:2604.06777, 2026. Huanjin Yao, Qixiang Yin, Min Yang, Ziwang Zhao, Yibo Wang, Haotian Luo, Jingyi Zhang, and Jiaxing Huang. Mm-deepresearch: A simple and effective multimodal agentic search baseline. arXiv preprint arXiv:2603.01050, 2026. 14 Yu Zeng, Wenxuan Huang, Zhen Fang, Shuang Chen, Yufan Shen, Yishuo Cai, Xiaoman Wang, Zhenfei Yin, Lin Chen, Zehui Chen, et al. Vision-deepresearch benchmark: Rethinking visual and textual search for multimodal large language models. arXiv preprint arXiv:2602.02185, 2026. Huanyao Zhang, Jiepeng Zhou, Bo Li, Bowen Zhou, Yanzhe Shan, Haishan Lu, Zhiyong Cao, Jiaoyang Chen, Yuqian Han, Zinan Sheng, et al. Browsecomp-v 3 : A visual, vertical, and verifiable benchmark for multimodal browsing agents. arXiv preprint arXiv:2602.12876, 2026a. Huanyao Zhang, Jiepeng Zhou, Runhao Zhao, Yanzhe Shan, Jiaoyang Chen, Bowen Zhou, Bo Li, Fang Wang, Jialong Wu, Zhengwei Tao, et al. Deepvoyager-vl: Incentivizing vision-in-the-loop search for long-horizon multimodal agents. arXiv preprint arXiv:2608.01827, 2026b. Ruiyang Zhang, Qianguo Sun, Chao Song, Yiyan Qi, and Zhedong Zheng. Vsearcher: Long-horizon multimodal search agent via reinforcement learning. arXiv preprint arXiv:2603.02795, 2026c. Xintong Zhang, Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaowen Zhang, Yang Liu, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, et al. Chain-of-focus: Adaptive visual search and zooming for multimodal reasoning via rl. arXiv e-prints, pages arXiv–2505, 2025a. Yi-Fan Zhang, Xingyu Lu, Shukang Yin, Chaoyou Fu, Wei Chen, Xiao Hu, Bin Wen, Kaiyu Jiang, Changyi Liu, Tianke Zhang, et al. Thyme: Think beyond images. arXiv preprint arXiv:2508.11630, 2025b. Zhengbo Zhang, Changtao Miao, Jinbo Su, Zhaowen Zhou, Chunxia Zhang, Xukai Wang, Ruiqi Liu, Kaiyuan Zheng, Jiansheng Cai, Bo Zhang, et al. Visual-seeker: Towards visual-native multimodal agentic search via active visual reasoning. arXiv preprint arXiv:2606.15231, 2026d. Zhengbo Zhang, Jinbo Su, Zhaowen Zhou, Changtao Miao, Yuhan Hong, Qimeng Wu, Yumeng Liu, Feier Wu, Yihe Tian, Yuhao Liang, et al. Visbrowse-bench: Benchmarking visual-native search for multimodal browsing agents. arXiv preprint arXiv:2603.16289, 2026e. Zhixin Zhang, Yiyuan Zhang, Xiaohan Ding, and Xiangyu Yue. Vision search assistant: Empower vision-language models as multimodal search engines. arXiv preprint arXiv:2410.21220, 2024. Shitian Zhao, Haoquan Zhang, Shaoheng Lin, Ming Li, Qilong Wu, Kaipeng Zhang, and Chen Wei. Pyvision: Agentic vision with dynamic tooling. arXiv preprint arXiv:2507.07998, 2025a. Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, et al. Swift: a scalable lightweight infrastructure for fine-tuning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 29733–29735, 2025b. Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deepresearcher: Scaling deep research via reinforcement learning in real-world environments. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 414–431, 2025. Ziwei Zheng, Minghao Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, and Chao Shen. Deepeyes: Incentivizing" thinking with images" via reinforcement learning. In International Conference on Learning Representations, volume 2026, pages 126775–126798, 2026. Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, et al. Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese. arXiv preprint arXiv:2504.19314, 2025. Bin Zhu, Qianghuai Jia, Tian Lan, Junyang Ren, Feng Gu, Feihu Jiang, Longyue Wang, Zhao Xu, and Weihua Luo. Marco deepresearch: Unlocking efficient deep research agents via verification-centric design. arXiv preprint arXiv:2603.28376, 2026. 15 A Details of Our Data Pipeline and Training Data A.1 Details of Our Data Pipeline We construct questions designed to require newly acquired visual evidence during multimodal deep search. Their intended solutions interleave relation search with visual grounding and, when needed, visual answer extraction. Intermediate images resolve entities needed for subsequent search, while images of the resolved target provide visual attributes requested by the question. These roles can coexist within a question and recur along its solution path. Our pipeline first selects seed entities and constructs relational reasoning graphs, then derives and verifies programs with explicit visual dependencies, grounds them in online image evidence, and realizes natural-language questions before filtering candidates and curating successful search trajectories. Selecting seeds for compositional visual search. Seed selection considers both the relational structure around an entity and the opportunities it offers for visual grounding. We draw from Wikipedia-linked Wikidata entities with substantive encyclopedic coverage, excluding disambiguation pages and non-entity entries. Within semantic domains, we prioritize diverse incoming and outgoing relations and the presence of image-associated neighbors, while downweighting highly connected hubs and neighborhoods dominated by generic attributes or taxonomic links. This favors seeds whose neighborhoods offer several potential visual checkpoints along a reasoning structure, including entities other than the seed itself. These signals guide construction; successful visual grounding and resistance to textual shortcuts are assessed downstream. Expanding seeds into relational DAGs. We treat each seed as a candidate terminal entity and expand its relational prerequisites backward. For each expanded entity, we select a small distin- guishing set of neighboring entities and relations whose conjunction uniquely identifies it in the KG snapshot. The neighbors become prerequisites for resolving that entity, and recursively expanding them produces a directed acyclic graph (DAG) of identification dependencies. Edges are oriented from prerequisites to the entity they identify, with the direction of each underlying Wikidata fact preserved. Expansion explores complementary relations across branches and recursively extends their dependencies, subject to depth and branching constraints. We favor concrete semantic relations, such as creation, membership, and location, and exclude encyclopedic maintenance and bibliographic- source links. The resulting DAG provides a structured search space for extracting reasoning structures, with entity identities and relation constraints established before question realization. Constructing visual dependencies.From each DAG, we enumerate paths leading to the seed and subgraphs with two branches that converge on a shared entity, which either is the terminal entity or leads to it through further relations. Convergent branches have disjoint entity sets before the merge and constrain the shared entity through distinct relations. We select an upstream entity as the visual entry anchor and identify feasible visual checkpoints within the program. At an intermediate checkpoint, the visually grounded entity supplies the input to a subsequent relation; at an answer checkpoint, the retrieved image supplies the requested property of the resolved target. Programs contain one or multiple intermediate checkpoints, optionally followed by visual answer extraction. We apply structural filters before acquiring images and compile each program into a conjunctive query over the complete local KG snapshot (Equation 2), whereCspecifies relation constraints expressed in their original KG direction,Bspecifies entry and planned visual-entity bindings, and z out denotes the terminal entity. For a seed entityu ⋆ , we retain programs withU K (C,B) =u ⋆ ; multiple endpoints or unresolved query evaluations are rejected. This verifies endpoint identity conditional on the planned bindings. The target is eitheru ⋆ or a visual property subsequently verified from images of that target. Grounding dependencies in online evidence. For each verified program, a planner discovers images and source pages for the required checkpoints without introducing new graph relations. A blind observer describes full images using sanitized scene cues, with target names, search titles, graph relations, and expected answers withheld. A separate observer examines actual crops without receiving the first observer’s conclusions. For entity grounding, a binding step matches the visual 16 evidence to candidate entities including distractors, without marking the intended target. We accept identities supported by distinctive pixels or by compatible visual observations corroborated through reverse search or source pages; incorrect and unresolved matches are rejected. Each intermediate binding is explicitly connected to its subsequent relation. For visual answer extraction, both full- image and crop observations must support the requested attribute. When a program spans converging branches, we additionally check distinct images, different source domains, and consistency with the verified graph. We verbalize the relations, compose a base question, and rewrite it to remove revealing names and mechanical structural cues while preserving its facts and answer. A semantic-coverage check verifies that each required relation remains expressed. The solver receives only the question and entry image; images needed at later checkpoints, including final attribute reading, must be acquired during search. Filtering and curating search trajectories. We sequentially test candidates using text-only an- swering, text-only search withTextSearch, direct visual question answering with the entry image, and short multimodal search. Both search settings allow at most four tool-use turns followed by a mandatory final-answer turn. Any correct answer rejects the candidate; unresolved infrastructure or judging failures are excluded. These filters screen for shortcuts and insufficient search depth relative to the chosen models and budgets. For surviving questions, a stronger teacher performs long-horizon search, and an independent answer judge selects successful trajectories. We retain traces with complete observations, valid action order, and compliant sequence lengths for policy SFT and training-side success references. Local tool tasks additionally provide MEI training inputs, with targets separately regenerated and checked against the supplied material. A.2 Training Data We construct the policy training sets from our synthetic search tasks, REDSearcher-M data, and a mixture of visual question answering datasets. Qwen3.8-Max serves as the expert model for trajectory generation and rejection sampling within the Neos workflow. Table 8 summarizes the resulting SFT and RL datasets. Data sources and expert trajectories. Our data pipeline produces approximately 2K synthetic search instances with verified successful trajectories. To incorporate external search questions, we rank the REDSearcher-M SFT data [Chu et al., 2026] by decreasing trajectory length, measured in interaction turns, and retain the top 60%. We additionally sample approximately 3.7K candidate questions from FVQA [Wu et al., 2025], LiveVQA [Fu et al., 2025], InfoSeek [Chen et al., 2023], and VisualProbe [Lai et al., 2026]. For FVQA and LiveVQA, all sampled instances are selected from their corresponding official training splits, and after deduplication against the benchmarks used in our evaluation, the overlap rate with the evaluation benchmarks is 0%. For these external questions, we generate new trajectories using Qwen3.8-Max in Neos and retain successful trajectories through rejection sampling, rather than directly adopting demonstrations produced under another workflow. SFT composition. We allocate 989 examples from our synthetic data to policy SFT and reserve the remaining 973 instances for RL. From the expert-generated external trajectories, we select 1,177 based on REDSearcher-M questions and 732 based on the VQA mixture. The resulting SFT dataset contains 2,898 trajectories, combining demonstrations of interleaved multimodal search with broader visual question answering. RL question selection. After cold-start SFT, we generate three rollouts for each question in the available data pool using the initialized policy, excluding the 973 synthetic instances reserved for RL. We retain questions answered correctly in zero or one of the three rollouts. The selected pool contains 1,599 questions: 599 from our synthetic data, 500 from REDSearcher-M, and 500 from the VQA mixture. Combining these questions with the 973 reserved synthetic instances yields an RL dataset of 2,572 questions. Table 8 summarizes the composition of both training sets. 17 Table 8: Composition of the policy training datasets. SFT counts refer to expert-generated trajecto- ries, whereas RL counts refer to questions. Difficulty filtering retains questions answered correctly in zero or one of three cold-start policy rollouts. Data sourceSelectionCount SFT total2,898 Our synthetic dataExpert trajectories989 REDSearcher-MMExpert trajectories1,177 VQA mixtureExpert trajectories732 RL total2,572 Reserved synthetic dataReserved before difficulty filtering973 Our synthetic dataDifficulty-filtered599 REDSearcher-MMDifficulty-filtered500 VQA mixtureDifficulty-filtered500 Table 9: Tools exposed to Neos. MEI is embedded in tools. Search and page-reading return text and image references; pixels enter the policy context through FetchImage or CropImage. ToolRoleInputOutput TextSearchDiscover candidate webpages Text query or query batchPage titles, URLs, and snippets WebVisitExtract page evidence with MEI Page URL(s) and local goal Goal-relevant evidence and page-image URLs ImageSearchRank image candidates with MEI Text query(s) and visual goal Ranked image/source URLs and visual analysis ReverseImageSearchMatch visual evidence with MEI Image IDs/URLs and matching goal Match analysis and candidate/source URLs FetchImageLoad selected visual evidence Image URLs; optional captions Image pixels and reusable image IDs CropImageInspect fine-grained regions Image IDs and normalized bounding boxes Crop pixels and reusable image IDs B Engineering Techniques in Agentic RL Turn-level asynchronous execution. We decouple policy generation from tool execution and answer judging. Each trajectory maintains its own history; while its tools execute, other trajectories can submit ready turns to available SGLang replicas. Requests are scheduled at turn boundaries without permanently binding a trajectory to one replica. Thus, asynchronous execution overlaps model and environment work, while each rollout round still uses fixed policy and MEI versions. Adaptive mixed-group sampling. For an update withBquestion groups andGtrajectories per group, we admit up to2BGtrajectories over an active window of2Bquestions. Completed trajectories are answer-judged immediately, and released slots replenish groups with the fewest completed samples. Groups withGhomogeneous outcomes receive priority for additional samples, up to 1.5G; groups still homogeneous at that limit are deferred. Once B mixed-outcome groups are available, we select eligible groups in queue order. If a selected group has more thanGcompleted trajectories, we retain the lowest-session-index success and failure, then fill the remaining positions by session index, without enforcing a balanced success/failure ratio. This yields exactlyBGtrajectories with a nondegenerate group-relative learning signal. Overlapped credit assessment and policy boundaries. Credit assessment of failures starts once a group becomes eligible and overlaps the remaining rollout work. Only assessments attached to the ultimately selected trajectories affect optimization; successful trajectories retain ordinary GRPO 18 advantages. Structured-output validation and bounded repair retries isolate malformed or timed-out assessments, with ordinary GRPO as the fallback. Before optimization, surplus old-policy rollouts are cancelled or discarded; unused questions are carried forward for fresh sampling after the weight update. Asynchrony therefore operates within a rollout round, not as unbounded cross-version training on stale trajectories. Training and runtime overview. Figure 8 summarizes the training and runtime diagnostics of Neos-9B. Rollout time remains stable around 80–90 minutes per update, confirming that asyn- chronous scheduling maintains consistent throughput. Policy optimization time drops sharply after the first update and stabilizes near 6 minutes, indicating negligible overhead from overlapped credit assessment. The selected-batch reward, surrogate objective, gradient norm, and PPO likelihood-ratio proxy all exhibit typical noisy-but-bounded dynamics without divergence or instability. 1510152025303540 Policy update 0.0 0.2 0.4 0.6 0.8 1.0 Mean reward Selected-batch reward 1510152025303540 Policy update −0.30 −0.28 −0.26 −0.24 −0.22 Signed GRPO loss Policy surrogate objective 1510152025303540 Policy update 0.055 0.060 0.065 0.070 0.075 Logged gradient norm Gradient norm 1510152025303540 Policy update 75 80 85 90 95 Minutes / logged update Rollout time 1510152025303540 Policy update 5 10 15 20 25 30 Minutes / logged update Policy optimization time 1510152025303540 Policy update 0.0006 0.0007 0.0008 0.0009 Logged PPO KL proxy PPO likelihood-ratio diagnostic Figure 8: Training and runtime diagnostics. Top row: selected-batch reward, signed GRPO surrogate objective, and logged gradient norm. Bottom row: wall-clock rollout time (including environment waits), policy optimization time, and PPO likelihood-ratio diagnostic. C Experimental Details C.1 Benchmarks and Evaluation Splits We evaluate Neos on ten benchmarks covering factual visual question answering, multi-step multi- modal search, and reasoning over newly acquired visual evidence. Table 9 introduces the tools in Neos framework, and Table 10 specifies the evaluation set used for each benchmark. Factual visual question answering.SimpleVQA [Cheng et al., 2025], LiveVQA [Fu et al., 2025], and FVQA [Wu et al., 2025] assess factual question answering grounded in visual inputs. LiveVQA emphasizes up-to-date visual knowledge, while FVQA includes both search-required and search- free questions. For these three benchmarks, we use the same 300-example subsets released with Vision-DeepResearch [Huang et al., 2026b]. Multimodal search and browsing.MMSearch [Jiang et al., 2024] evaluates information seeking using visual and textual web evidence, while MMSearch-Plus [Tao et al., 2026] emphasizes fine- grained visual clues and provenance-aware search. We evaluate all 300 MMSearch queries and all 311 MMSearch-Plus questions. BrowseComp-VL [Geng et al., 2026b] tests multi-step vision–language search; we include both difficulty levels, totaling 399 questions. For M-BrowseComp [Li et al., 2025b], which requires integrating evidence encountered during multimodal browsing, we use the expanded 400-question release. 19 Table 10: Evaluation sets used in our experiments. BenchmarkExamplesEvaluation set MMSearch300Complete benchmark MMSearch-Plus311Complete benchmark BrowseComp-VL399Both difficulty levels M-BrowseComp400Expanded 400-question release VisBrowse-Bench169Complete benchmark BrowseComp-V 3 300Complete benchmark VDR-Bench500Official testmini subset SimpleVQA300Vision-DeepResearch subset LiveVQA300Vision-DeepResearch subset FVQA300Vision-DeepResearch subset Visual-native and verifiable search.BrowseComp-V 3 [Zhang et al., 2026a] evaluates cross-modal, multi-hop browsing with verifiable reasoning requirements, and VisBrowse-Bench [Zhang et al., 2026e] emphasizes inspecting newly discovered web images to advance the search. We use their complete evaluation sets. For VDR-Bench [Zeng et al., 2026], which targets complementary visual and textual search, we use the official 500-example testmini subset. C.2 Baselines and Evaluation Protocol Baseline categories. We organize the baselines in Table 1 into three groups, covering general- purpose MLLMs with and without tool access, as well as specialized multimodal search agents. Direct Answer. We consider GPT-5.6-Sol [OpenAI, 2026], Gemini-3.1-Pro [Google, 2026], Kimi- K3 [Team et al., 2026a], Qwen3.8-Max [Team, 2026], Qwen3.5-35B-A3B, and Qwen3.5-9B. Each model receives the benchmark question and available input images and produces an answer without external tools. This setting assesses visual understanding and factual reasoning using only the supplied inputs and parametric knowledge. Agentic Workflow. The same general-purpose models operate within our framework. The policy models are used without additional search-specific training and are allowed up to 50 interaction turns per question, which is the same as our Neos-9B ’s evaluation protocol. Comparing this setting with Direct Answer measures the benefit of external evidence acquisition, while comparisons with Neos-9B and Neos-35B assess the additional gains from specialized training. Multimodal deep search Agents. We compare against MMSearch-R1 [Wu et al., 2025], Web- Watcher [Geng et al., 2026b], DeepEyesV2 [Hong et al., 2026], SenseNova-MARS [Chng et al., 2025], Vision-DeepResearch [Huang et al., 2026b], REDSearcher-M [Chu et al., 2026], M- DeepResearch [Yao et al., 2026], POINTS-Seeker [Liu et al., 2026b], LMM-Searcher [Du et al., 2026], OpenSearch-VL [Chen et al., 2026], Visual-Seeker [Zhang et al., 2026d], SearchEyes [Jiao et al., 2026], SimpleSearch-VL [Dai et al., 2026], UniBrowse [Wei et al., 2026], and DeepVoyager- VL [Zhang et al., 2026b]. These baselines span different model scales and approaches to multimodal search training. For results taken from prior publications, we retain their reported benchmark coverage and evaluation settings. Unreported benchmark scores are marked with “–”. Evaluation metrics.We assess answer correctness using Qwen3.7-Flash with the official evaluation prompt released by Vision-DeepResearch [Huang et al., 2026b]. The prompt supplies the question, reference answer, and model-produced answer or report. The judge determines whether the response contains an answer consistent with the reference, without requiring a particular answer format or location within the response. We apply this prompt and judge model across all ten benchmarks for predictions generated in our experiments. 20 Table 11: Hyperparameters for policy supervised fine-tuning. Differing settings are reported in the order 9B / 35B; otherwise, a single setting is shared. Parentheses group compound settings. HyperparameterValue Base modelQwen3.5-9B / Qwen3.5-35B-A3B Training epochs2 Maximum sequence length131,072 tokens Global / micro batch size8 / 1 Training GPUs8 TP / CP / P / DP(4, 2, 1, 1) / (2, 4, 1, 1) EP / ETP(1, 1) / (8, 1) MoE routerNot applicable / frozen OptimizerAdamW Adam (β 1 ,β 2 )(0.9, 0.95) Adam ε10 −8 Peak learning rate10 −5 Minimum learning rate10 −6 Learning-rate scheduleCosine Warmup fraction5% Weight decay0.1 Gradient clipping1.0 PrecisionBF16 Packing / padding-freeDisabled / enabled Random seed42 C.3 Training and Implementation Search infrastructure. Neos integrates Google for text-based web search, Google Images for image retrieval, and Google Lens for reverse image search, with Jina Reader supporting webpage access.FetchImageandCropImagematerialize selected images and image regions as actual pixels, retain them in an episode-local visual register, and encode them as multimodal inputs to the policy. Alibaba Cloud OSS primarily provides externally accessible URLs for query images required by reverse image search. To reduce API costs and improve efficiency, we employ a hierarchical cache spanning in-memory storage, node-local disk, and shared persistent storage. Reusing search results, webpage content, and image assets across repeated requests avoids redundant external calls and data transfers, accelerating both rollout collection and evaluation. Models and supervised initialization.We trained Qwen3.5-9B and Qwen3.5-35B-A3B as policy models, while MEI remains Qwen3.5-9B. Their language backbones are trainable, while the vision encoders and multimodal mergers remain frozen throughout training. Cold-start policy SFT uses MS- Swift [Zhao et al., 2025b] on 8 NVIDIA H100 GPUs, with 2.9K examples, two epochs, and a learning rate of10 −5 . The detailed hyperparameters of our SFT training are provided in Table 11 Policy supervision covers assistant reasoning, serialized tool calls, and final answers; tool observations serve as conditioning context and are excluded from the loss. MEI is supervised on local goal-conditioned evidence-processing responses. Training-data composition is described in Section A. Agentic RL and MEI adaptation. Policy RL uses verl [Sheng et al., 2025] with the Megatron training backend [Shoeybi et al., 2019]. The co-evolution stage runs on 40 H100 GPUs: 32 support policy RL, and 8 support MEI serving and supervised fine-tuning. Policy RL is performed for one epoch with 8 rollouts per question, using a batch size of 64 for Neos-9B and 32 for Neos-35B, with Qwen3.7-Plus as the credit judge model. The detailed hyperparameters are shown in Table 12. Online MEI updates use outcome- and process-filtered local interactions, retaining valid calls from correct trajectories and productive or corrective turns from selected failures. The MEI version remains fixed within each trajectory, and updated checkpoints are activated before subsequent rollout batches. MEI adaptation therefore uses SFT with a learning rate of 10 −6 . 21 Table 12: Hyperparameters for agentic RL and online MEI supervised fine-tuning. Differing settings are reported in the order 9B / 35B; otherwise, a single setting is shared. Parentheses group compound settings. Policy optimizationOnline MEI SFT HyperparameterValueHyperparameterValue OptimizerAdamWEpochs per update1 Learning rate10 −6 Peak learning rate10 −6 Adam (β 1 ,β 2 )(0.9, 0.999)Minimum learning rate10 −7 Learning-rate scheduleConstantLearning-rate scheduleCosine WarmupNoneWarmup fraction5% Weight decay0.01Weight decay0.1 Gradient clipping1.0Gradient clipping1.0 PPO epochs per batch1Global / micro batch size(16, 1) / (32, 2) PPO clip, ε0.2Maximum sequence length32,768 tokens Dual clip, c dc 3.0Adam (β 1 ,β 2 )(0.9, 0.95) KL / entropy coefficients0 / 0MEI update interval, K10 Training GPUs32Training GPUs8 TP / CP / P / DP(4, 2, 1, 4) / (4, 4, 1, 2)TP / CP / P / DP4 / 1 / 1 / 2 EP / ETP(1, 1) / (16, 1)EP / ETP1 / 1 MoE routerNot applicable / frozenVisual encoder and projectorFrozen PrecisionBF16 Rollout samplingCredit assignment HyperparameterValueHyperparameterValue Groups per update, B64 / 32Critical-error floor, κ e 0.5 Trajectories per group, G8Downstream floor, κ d 0.25 Maximum samples per group12Downstream strength, λ d 0.5 Active question window128 / 64Downstream decay, γ0.65 Trajectory admission limit1,024 / 512Corrective weight, λ c 0.3 Maximum context / turns131,072 tokens / 50Productive weight, λ p 0.1 Temperature / top-p / top-k1.0 / 1.0 / disabledNegative amplification cap,M 4 Correct / incorrect reward1.0 / 0.0Negative margin, η0.02 D Supplementary Training Details Supervised objective.For either the policy or MEI, letf ψ denote the model andSa minibatch of serialized examples. We use L SFT (ψ;S) =− P x∈S P k m k (x) logf ψ (x k |x <k ) P x∈S P k m k (x) ,(5) wherem k selects target tokens and excludes conditioning content and padding. Policy targets include reasoning, tool calls, and final answers; tool observations provide context only. MEI targets are structured evidence responses conditioned on local goals and retrieved material. The same objective is used for MEI updates during co-evolution. Credit parameters and rescaling.In Equation 3, severity and confidence satisfys i,t ,c i,t ∈ [0, 1]. We useγ ∈ (0, 1),κ e ,κ d ∈ (0, 1],λ d ∈ [0, 1], and0 < λ p ≤ λ c . LetL i =|M i |and define positive and negative credit masses P i = X k∈M i [ e A i,k ] + , N i = X k∈M i [− e A i,k ] + ,(6) 22 where[u] + = max(u, 0). For an assessed failure withN i > 0, a target marginμ i = ηb i L i (η > 0), and an amplification cap M ≥ 1, we set α i = min M, max 1, P i + μ i N i , β i = min 1, [α i N i − μ i ] + P i , b A i,k = β i [ e A i,k ] + − α i [− e A i,k ] + , (7) withβ i = 1whenP i = 0. This keeps the sum of token advantages negative while bounding each advantage in[−Mb i ,λ c b i ]; the target margin is enforced only when feasible. Successful trajectories and failures without valid assessments identifying a critical error retain b A i,k = A i ; the same fallback applies when N i = 0. Dual-clipped surrogate. For a token likelihood ratio ρ and advantage A, define ℓ ε (ρ,A) = minρA, clip(ρ, 1− ε, 1 + ε)A.(8) With c dc > 1, the surrogate in Equation 4 is ℓ dc (ρ,A) = maxℓ ε (ρ,A),c dc A, A < 0, ℓ ε (ρ,A),A≥ 0. (9) All judge assessments and adjusted advantages are detached from policy optimization. E Additional Ablation Studies. Learning the evidence–decision loop.Table 13 suggests that the benefit of two-stage training lies in the complementary roles of supervised initialization and reinforcement learning. Neither SFT-only nor RL-only training on the deduplicated union of both training sets matches the final Neos-9B model. We interpret this pattern as supporting a curriculum for learning the evidence–decision loop: cold-start SFT provides a behavioral prior for acquiring evidence, interpreting visual observations, and selecting the next action, while RL refines these transitions through interaction and outcome feedback. In particular, RL can improve how the agent decides whether the available evidence is sufficient, requires further inspection, or calls for a different search direction. Expanding SFT alone does not provide this on-policy refinement, whereas RL directly from the base model must acquire useful interaction patterns while also learning which decisions lead to successful outcomes. Cold-start SFT therefore offers a structured starting point that allows subsequent RL to focus on improving an already functional loop, rather than discovering the entire behavior through trial and error. This provides a plausible mechanism for reducing the additional rollout exploration, and consequently the training time and compute, that RL-only learning may require. Table 13: Training-stage ablation.Sis the question pool underlying the 2,898 SFT trajectories, Rcontains 2,572 RL questions, andU = S ∪Rdenotes their question-level deduplicated union. RL-only variants start directly from the base Qwen3.5-9B policy. SFT-only variants use the final co-evolved MEI; RL-based variants use the MEI obtained in their respective RL runs. The best and second-best results are bolded and underlined. Training strategySFT poolRL poolBC-VLMMSearch+FVQA Qwen3.5-9B–43.624.861.3 SFT onlyS–60.444.779.7 SFT onlyU–61.445.881.3 RL only–S59.644.180.3 RL only–U62.4 46.380.3 Neos-9B (SFT→ RL)SR63.247.683.3 23 Generalization to text-only tasks.We further evaluate Neos-9B on BrowseComp [Wei et al., 2025], BrowseComp-ZH [Zhou et al., 2025], HLE-Text [Phan et al., 2025], and DeepSearchQA [Gupta et al., 2026] to examine its applicability beyond multimodal tasks. As shown in Table 14, Neos-9B improves over the base Qwen3.5-9B under the same workflow by 12.2, 9.7, 11.9, and 10.6 points on these benchmarks, respectively, yielding an average gain of 11.1 points. Compared with specialized search agents, it exceeds Marco-DR-8B [Zhu et al., 2026] by 5.0 points on BrowseComp and 25.1 points on DeepSearchQA. On HLE-Text, Neos-9B reaches 33.5, outperforming DeepSearch-World-9B [Geng et al., 2026a] and REDSearcher-M-30B [Chu et al., 2026] by 7.8 and 8.2 points, respectively. It also remains competitive on BrowseComp-ZH, scoring 45.3 compared with Marco-DR-8B’s 47.1 and exceeding the other baselines. Together, these results show that Neos’s improvements extend beyond multimodal inputs to text-based search and reasoning, with gains over the base workflow highlighting the contribution of training beyond visual ability alone. Table 14: Generalization to text-only benchmarks. The best and second-best scores in each column are bolded and underlined, respectively. ModelBrowseComp BrowseComp-ZH HLE-Text DeepSearchQA Qwen3.5-9B-base(our workflow)24.235.621.643.7 Marco-DR-8B31.447.1–29.2 REDSearcher-M-30B31.244.525.3– DeepSearch-World-9B31.236.425.7– Neos-9B36.445.333.554.3 0510152025 Median assistant turn Productive Corrective Neutral Causal error Downstream Re-derailment 4 (no change) 12 → 9 13 → 11 3 → 4 20 → 19 13 → 12 Early: policy steps 6–10 Late: policy steps 31–35 (a) Shift in credit position. 1–56–1011–1516–2021–2526–3031–3536–40 Policy updates (five-step windows) 0 20 40 60 80 100 Turns in credit-assessed failures (%) 11.1 9.4 10.0 11.111.1 11.211.2 10.6 4.2 4.3 4.6 4.8 14.3 14.1 11.8 14.2 21.2 21.3 23.5 21.9 4.1 4.1 4.1 4.1 4.1 4.1 4.1 4.1 65.6 68.4 70.1 66.5 59.2 59.0 56.5 58.5 n=1,142n=1,057n=1,066n=1,039n=1,170n=1,149n=1,239n=398 Productive Corrective Neutral Causal error Downstream consequence Re-derailment (b) Credit type composition across training. Figure 9: Evolution of reference-guided credit assignment over 40 policy updates. (a) Median assistant turn at which each credit type occurs, comparing early (steps 6–10) and late (steps 31–35) training. (b) Composition of six credit types aggregated in five-step windows. Credit Dynamics During RL Training. Figure 9 shows that RL training changes both where credit types occur and how frequently they appear. As shown in Figure 9a, corrective turns move earlier in the trajectory, from a median position of turn 12 to turn 9, while neutral turns shift from turn 13 to turn 11 and re-derailment from turn 13 to turn 12. Productive turns remain anchored at turn 4, causal error shifts slightly later from turn 3 to turn 4, and downstream consequence remains concentrated near the end of the trajectory, with its median position changing only from turn 20 to turn 19. Figure 9b further shows that downstream consequence dominates throughout training, accounting for 66–70% of credited failure turns in the first half before declining to 56–59% later, while the share of neutral turns rises from around 14% to 22–24%. In contrast, productive and causal error shares remain relatively stable at approximately 10–11% and 4%, respectively. The share of corrective turns increases from 3.2% to 4.8%, indicating more frequent recovery behavior within failed searches, whereas re-derailment appears only in the earliest window (1.7%) and is rarely observed thereafter. Taken together, these results suggest that, as training progresses, corrective actions occur earlier and fewer assessed turns are attributed to downstream consequences, while the overall positional structure of productive decisions and late-stage error propagation remains stable. 24 F Prompts We provide the system prompt, native tool schemas and definitions, user-message template, and final-turn instruction used by the search policy, followed by the task-specific prompts of MEI and the reference-guided credit judge. The policy operates within 50 assistant turns and addresses loaded images through reusable img_N identifiers. System prompt You are a long-horizon multimodal web-search agent. Solve the user’s question by interleaving careful reasoning with live-web retrieval and selective visual observation. Work for as many steps as needed, but return the best-supported final answer within 50 assistant turns. EVIDENCE DISCIPLINE 1. Track the question’s constraints and unresolved evidence gaps. Use each action to reduce a specific uncertainty, revise hypotheses when evidence conflicts, and verify decisive facts before answering. 2. Distinguish visual evidence from metadata. Titles, snippets, alt text, captions, URLs, and helper summaries are discovery signals, not proof of what an image’s pixels contain. 3. Search results deliberately contain only lightweight references. A candidate image becomes visible evidence only after FetchImage loads it into the visual register. CropImage creates another reusable visual observation from pixels already in that register. 4. Inspect fine-grained visual details when relevant: text, logos, faces, object parts, colors, counts, spatial relations, and background clues. Do not claim a pixel-level fact about an image that has not been loaded. 5. Prefer focused, complementary searches and source pages. Cross-check ambiguous identities and final claims; do not repeatedly issue an identical cached call expecting new information. TOOL WORKFLOW - TextSearch discovers webpages as title-URL-snippet records. - WebVisit extracts goal-relevant evidence and up to fifteen lightweight page-image references. - ImageSearch discovers and goal-ranks image candidates without exposing their pixels to you. - ReverseImageSearch searches from up to three loaded image tokens or image URLs and returns textual match evidence plus lightweight candidate references; it does not load candidate pixels. - FetchImage loads up to five selected URLs in parallel and appends successful images to the visual register in request order. - CropImage crops one or more loaded img_N observations with normalized 0..1000 [left, top, right, bottom] boxes. It processes requested crops in parallel and appends them in deterministic request order. NATIVE ACTION CONTRACT - Tool definitions are supplied through the API tools field. Use native function calls only. Never print XML tags, JSON tool-call wrappers, or a textual imitation of a function call. - On each assistant turn, take exactly one action: either issue exactly one native function call or return the final answer as ordinary text. Batch independent queries, URLs, images, or boxes inside that one function call when its schema permits. - Think carefully before choosing the action. Do not duplicate a narration of the intended function call in answer text. - Do not fabricate tool results. The environment returns a role=tool message whose tool_call_id matches the assistant tool call and reports any newly loaded img_N references. - Tool-returned pixels are registered as img_N only when the environment explicitly reports them. Only use existing visual-register tokens. - When the complete answer is supported, stop calling tools and return only the concise exact answer requested by the user as ordinary text. Do not wrap it in XML, Markdown, an answer label, or an explanation unless the question explicitly asks for one. Before answering, synthesize the visual and web evidence and check that the visual identification and all answer constraints are supported. Figure 10: System prompt. The policy interleaves retrieval and selective visual observation, issuing one native function call or a final answer per turn. 25 Tool schema TextSearch "type": "function", "function": "name": "TextSearch", "description": "Search the live web. Each query returns up to 10 lightweight title-URL-snippet results; it does not load page or image pixels.", "parameters": "type": "object", "properties": "query": "type": "array", "minItems": 1, "items": "type": "string", "minLength": 1, "description": "A non-empty array of focused web queries." , "required": ["query"], "additionalProperties": false WebVisit "type": "function", "function": "name": "WebVisit", "description": "Visit up to five webpages and return goal-conditioned textual evidence plus lightweight page-image references. Referenced images are not visually observed until FetchImage loads them.", "parameters": "type": "object", "properties": "url": "type": "array", "minItems": 1, "items": "type": "string", "minLength": 1, "maxItems": 5, "description": "A non-empty array of at most 5 complete http(s) URLs." , "goal": "type": "string", "description": "The non-empty evidence goal for these pages." , "required": ["url", "goal"], "additionalProperties": false Figure 11: Tool schema and definitions for TextSearch and WebVisit. Each block is a native function object, including its description, parameter types, and required fields. 26 Tool schema ImageSearch "type": "function", "function": "name": "ImageSearch", "description": "Search for online images with up to five text queries. Return at most five goal-conditioned candidate references per query; candidate pixels are not placed in the policy context. Use FetchImage on a selected direct image URL before making a visual claim.", "parameters": "type": "object", "properties": "query": "type": "array", "minItems": 1, "items": "type": "string", "minLength": 1, "maxItems": 5, "description": "A non-empty array of at most 5 image queries." , "goal": "type": "string", "description": "The visual evidence goal used to rank candidates." , "required": ["query", "goal"], "additionalProperties": false ReverseImageSearch "type": "function", "function": "name": "ReverseImageSearch", "description": "Run reverse-image search on up to three loaded image tokens and/or public image URLs. Return goal-conditioned textual matches and lightweight candidate references, never implicit candidate pixels.", "parameters": "type": "object", "properties": "image_token": "type": "array", "minItems": 1, "items": "type": "string", "minLength": 1, "maxItems": 3, "description": "A non-empty array of at most 3 loaded img_N tokens." , "image_url": "type": "array", "minItems": 1, "items": "type": "string", "minLength": 1, "maxItems": 3, "description": "A non-empty array of at most 3 public http(s) image URLs." , "goal": "type": "string", "description": "The identity or visual-match goal." , "required": ["goal"], "anyOf": ["required": ["image_token"], "required": ["image_url"]], "additionalProperties": false Figure 12: Tool schema and definitions for ImageSearch and ReverseImageSearch. Candidate discovery returns lightweight references; it does not implicitly expose candidate pixels to the policy. 27 Tool schema FetchImage "type": "function", "function": "name": "FetchImage", "description": "Load up to five selected image URLs in parallel. Successfully decoded pixels are appended to the visual register as new img_N tokens in request order. Every successful result record directly includes its new output_ref.", "parameters": "type": "object", "properties": "urls": "type": "array", "minItems": 1, "items": "type": "string", "minLength": 1, "maxItems": 5, "description": "A non-empty ordered array of at most 5 direct image URL strings." , "required": ["urls"], "additionalProperties": false CropImage "type": "function", "function": "name": "CropImage", "description": "Crop loaded visual observations in parallel. crop_config maps each img_N token to one normalized [left, top, right, bottom] box in the 0..1000 coordinate space or a list of boxes. At most five boxes per image and ten total are materialized as reusable new img_N observations in deterministic request order. Every successful crop record directly includes its new output_ref.", "parameters": "type": "object", "properties": "crop_config": "type": "object", "minProperties": 1, "additionalProperties": "anyOf": [ "type": "array", "items": "type": "number", "minimum": 0, "maximum": 1000, "minItems": 4, "maxItems": 4 , "type": "array", "minItems": 1, "maxItems": 5, "items": "type": "array", "items": "type": "number", "minimum": 0, "maximum": 1000, "minItems": 4, "maxItems": 4 ] , "description": "Mapping from loaded img_N tokens to normalized 0..1000 boxes." , "required": ["crop_config"], "additionalProperties": false Figure 13: Tool schema and definitions for FetchImage and CropImage. Selected images and image regions become addressable visual observations. 28 User-message template Question: question Use the available live-web and visual-observation tools to investigate the question. Return only the concise exact answer as ordinary text once the evidence chain is supported. The question image is already loaded as img_1. FetchImage and CropImage outputs are appended as img_2, img_3, ... in the order reported by the environment. Runtime instruction at turn 50 FINAL TURN (50/50): tools are disabled. Using only the evidence already collected, return the best-supported concise exact answer now as ordinary text, with no answer label or surrounding markup. Figure 14: User-message template, shown for one initial image, and runtime instruction at turn 50. Tools are disabled on the final turn and the policy answers from the accumulated evidence. MEI prompt for WebVisit System prompt You are a precise multimodal-aware web content analyst. Return a valid JSON object only, with no Markdown or commentary. Never include navigation menus, sidebars, cookie banners, footers, or advertisements. User-message template Given the webpage content, the user goal, and the list of images found on the page: (1) locate the sections, tables, names, dates, numbers, and claims directly related to the goal, and explain briefly why the page is relevant; (2) extract the strongest supporting content from the article body while preserving qualifications and source claims; (3) summarize how that evidence answers or contributes to the goal without introducing unsupported facts; and (4) select only page images that may help verify or answer the goal, retaining their alt, url, and a short caption. If the page contains no direct evidence, return empty evidence and an empty image list. Output exactly the keys rational, evidence, summary, and relevant_images, with no additional keys. User goal: goal Webpage content: page_text Page image references: image_refs Figure 15: MEI prompts for WebVisit. Extracted webpage text, a local goal, and page-image references are transformed into structured evidence, a summary, and relevant image references. 29 MEI prompt for ImageSearch System prompt You are a precise visual-search candidate analyst. Inspect pixels, not only titles or captions. Return one valid JSON object and no Markdown. User-message template Rank the candidate images for the stated visual goal. For each candidate, report a relevance level (high, medium, low, or none) and a concise pixel-grounded caption. Metadata can guide inspection but is not visual proof. Return exactly: "rational": string, "candidates": ["index": integer, "relevance": "high|medium|low|none", "caption": string] Search query: query Visual goal: goal Candidate metadata: candidates Figure 16: MEI prompts for ImageSearch. Candidate metadata and attached image pixels support goal-conditioned relevance judgments and visual descriptions. MEI prompt for ReverseImageSearch System prompt You are a precise reverse-image comparison analyst. Compare visible pixels and return one valid JSON object with no Markdown. User-message template The first image is the query observation. Remaining images are reverse-search candidates in metadata order. Compare their pixels for the stated goal. Use match_state visual_match, partial_match, not_match, or insufficient_evidence. Return exactly: "rational": string, "candidates": ["index": integer, "match_state": "visual_match|partial_match|not_match|insufficient_evidence", "caption": string] Goal: goal Candidate metadata: candidates Figure 17: MEI prompts for ReverseImageSearch. The query observation and candidate images are compared in a fixed order to produce structured visual-match judgments. 30 Credit judge System prompt You are a causal credit-assignment judge for a multimodal web-research policy. The candidate trajectory has already been judged to have a wrong final answer. Compare its complete trajectory with one independently verified successful reference trajectory for the same question. Identify policy-authored assistant turns that causally pushed the candidate onto a wrong branch. This is causal process supervision, not imitation learning and not another final-answer grading task. Rules: 1. The reference is one valid solution, not the only valid solution. Never penalize a different search query, evidence order, tool choice, or reasoning path merely for differing from the reference when it remains valid and useful. 2. Grade assistant decisions only. A failed URL, empty search result, provider timeout, or misleading webpage is an environment observation, not itself a policy error. The policy may be at fault only when it ignores contrary evidence, overclaims what pixels/text establish, repeatedly pursues a disproven hypothesis, or makes an unjustified decision from that observation. 3. Distinguish a causal error from its downstream consequences. Mark the decision turn that makes the wrong branch materially more likely as causal_error; later turns that merely continue the same unrecovered branch are downstream_consequence, not additional root errors. 4. Detect recovery. If the candidate later rejects or repairs the earlier mistake and again approaches a valid solution, mark that turn recovery. If it subsequently makes a new independent error, mark that later turn re_derailment. There may be multiple error events only when genuine recovery or independent re-branching occurred between them. 5. Use visual evidence only when the corresponding image is available in the candidate's history up to that turn. Titles, URLs, captions, snippets, and helper summaries are not pixel evidence. img_N numbering is local to each trajectory. 6. A wrong final-answer turn is causal only if it introduces/selects an unsupported answer after viable evidence remained. If it merely states the inevitable result of an earlier unrecovered mistake, mark it downstream_consequence. 7. Return only significant causal/recovery events and genuinely productive turns. Omitted turns are neutral only outside an active error branch. During an active error branch, unannotated turns receive downstream-consequence credit through the post-processing rule. Use only candidate span IDs shown in the trajectory; do not invent or duplicate IDs. Event meanings: - causal_error: first currently unrecovered decision that diverts the solution; - recovery: explicitly repairs an earlier error and restores a viable route; - re_derailment: a new causal error after recovery, or an independent later error. List a turn under productive_turns only when it establishes valid evidence or materially advances a viable solution. An explicitly productive turn receives positive credit even if an earlier error has not yet been repaired; it must not be relabeled as a downstream consequence. Productive is not automatically recovery: only an explicit recovery event clears the earlier error, and other intervening turns retain their chronological downstream decay. Outside an active error branch, omitted turns are neutral. Some terminal tool events are marked observation_delivered_to_policy=false: the tool ran, but its result did not fit the remaining policy context and was not presented to the policy. Their audit-only response/pixels are intentionally absent here. Do not infer those unavailable results or blame the policy for not using them. Images may be uniformly enlarged and/or padded for API compatibility; the border and img_N label are not evidence and no new scene detail is created. Return strict JSON only, matching the requested schema. Severity and confidence are numbers in [0,1]. Keep rationales concise and evidence-specific. Figure 18: Credit judge system prompt. 31 Credit judge User-message template ===== VERIFIED_REFERENCE TRAJECTORY START ===== verified_reference_trajectory_with_inline_images ===== VERIFIED_REFERENCE TRAJECTORY END ===== ===== FAILED_CANDIDATE TRAJECTORY START ===== failed_candidate_trajectory_with_inline_images ===== FAILED_CANDIDATE TRAJECTORY END ===== Candidate span IDs, in chronological order: candidate_span_ids Return strict JSON only using this compact schema once: output_schema Figure 19: User-message template. Credit judge Output format "schema_version": "m-detective-failed-turn-credit-v2", "candidate_overall_summary": "brief causal account", "events": [ "span_id": "candidate Txx.Ay ID", "type": "causal_error | recovery | re_derailment", "severity": "0..1; zero for recovery", "confidence": "0..1", "error_category": "short snake_case label or empty string", "rationale": "concise evidence-specific reason", "reference_support": ["reference assistant-turn numbers, if useful"] ], "productive_turns": [ "span_id": "candidate Txx.Ay ID", "confidence": "0..1", "rationale": "valid evidence or useful progress" ], "confidence": "0..1" Figure 20: Credit judge output format. 32 G Visualization of Interleaved Multimodal Search We visualize constructed questions and representative interactions that illustrate the Neos workflow. The examples show how visual observations guide subsequent retrieval and how newly acquired images are inspected to resolve fine-grained questions. Evidence descriptions are abridged, while turn indices and the action sequence are preserved. In the action strips, TS, WV, IS, RS, FI, CI, and A denote TextSearch, WebVisit, ImageSearch, ReverseImageSearch, FetchImage, CropImage, and Answer, respectively. A new visual artifact resolves the answer Data construction 6-turn reference trajectory Question.The supplied portrait shows a Baroque-era Hessian landgrave in a gray curled wig, dark breastplate, buff-colored sleeve, and blue mantle; he founded Kassel’s Baroque orangery. The lettering on a photographed entrance canopy identifies the eleventh edition of an international contemporary-art exhibition held at that orangery. An Ivorian artist who participated in that edition devised the writing system represented on a reference card whose blue frame encloses a green leaf and a reclining figure. What visible object—specifically, which handwritten numeral—is centered beneath that blue-framed illustration? Input portraitimg_1 Newly retrieved card T4img_2 Number crop T5 img_4 Construction reasoning structure Charles I of Hesse-Kassel Orangerie Kassel documenta11 Frédéric Bruly Bouabré Bété syllabary reference card Handwritten numeral 249 foundedvenue of participant devised read pixels Reference search trajectory TurnActionEvidence and subsequent decision 1TextSearchConnect the landgrave, Kassel exhibition, participating artist, and the Bété syllabary. These relations identify what to search for, not the card’s handwritten number. 2WebVisitLook for the relevant reference card on pages about the writing system and the artist. The requested card is not yet obtained. 3ImageSearchSearch for the blue-framed leaf-and-reclining-figure card. The visual-search observation proposes a matching image and transcribes its numeral. 4FetchImageLoad the candidate card from the art-documentation site. Its subject and inscription are new evidence, distinct from the initial portrait. 5CropImageCrop beneath the blue border and verify the handwritten numeral “249” in the returned pixels. T6 answer:249. Complete action sequence 1 TS2 WV3 IS4 FI5 CI6 A Figure 21: A new visual artifact resolves the answer. 33 Two visual branches converge on one answer Data construction 7-turn reference trajectory Question.The labeled satellite view shows one continuous metropolitan area spanning the densely urbanized shores of the narrow strait between Honshu and Kyushu, while a planning map of that metropolitan extent highlights the labeled northern Kyushu municipality in pale yellow and divides it into seven wards. In Japan’s national semiprofessional football league, the circular red-and-orange crest featuring a stylized white winged bird and “SINCE 1971” belongs to a football club. Which person was both born in the mapped municipality and a member of the club represented by that crest? Branch A: geographic evidence Input satellite viewimg_1 Branch B: club evidence Retrieved crest T3img_2 Convergent reasoning structure Kanmon metropolitan regionJapan Football League Kitakyushu Municipality with seven wards Honda FC Swallow crest; “SINCE 1971” has part; map cue club in league; crest cue People born in this municipalityPeople who played for this club birthplace constraint team-membership constraint Yasuto Honda Same person satisfies both branches INTERSECTION Reference trajectory: ground, intersect, verify TurnActionEvidence and subsequent decision 1TextSearchRead the input’s geographic labels to identifyKitakyushu. Search the seven-ward clue and the independent red-and-orange, bird-and-1971 crest clue. 2–3WebVisit; FetchImage Open the emblem article, retrieve its image URL, and load the crest as img_2. The policy confirmsHonda FCfrom the returned pixels. 4TextSearchCombine both branches in the query:“Honda FC” soccer player “born in Kitakyushu”. The retrieved biography identifiesYasuto Honda. 5WebVisitVerify both relations on the biography:born in Kitakyushuand played for Honda from 1988 to 1992. Check the club’s league on its own page. 6TextSearchTest alternative club members. Retrieved biographies place Masaki Tsukano’s birth in Yonago and Nobuhiro Takeda’s in Osaka, not Kitakyushu. 7AnswerReturnYasuto Honda, supported by the birthplace and team-membership constraints together. Complete action sequence 1 TS2 WV3 FI4 TS5 WV6 TS7 A Figure 22: Two visual branches converge on one answer. The geographic branch identifies Ki- takyushu and constrains the birthplace, while the emblem branch identifies Honda FC and constrains team membership. Their intersection yields Yasuto Honda. The reference trajectory grounds both branches, combines them in a joint search, and verifies both biographical relations before answering. 34 A visual checkpoint redirects the search Data construction 20-turn reference trajectory Question.The supplied tan title page, framed by a leaf-like ornamental border, displays the three-word masthead of an American literary journal and states that it was founded in 1815. The journal was published in the Midwestern college town named in oversized gold lettering on a municipal mural mounted on a red-brick wall, with sepia images of a church, a historic commercial street, and a house representing three local historic districts. That college town is in the same U.S. state as Cedar Rapids, where a red-brick acute-care hospital is identified by large silver lettering beside a cross logo on its entrance wall. Which person was born in the mural-named college town and died at that hospital? Input journalimg_1 New mural: fetched at T13, cropped at T14img_6 Town-name crop T14img_4 Visual binding selects the next relational query The journal has multiple publication places. The mural selects the town used in the birthplace constraint. Text-led hypothesis Cedar Falls Unresolved birth- place search T13–T14 mural pixels Mount Vernon Intermediate town resolved T15–T19 relation search Born inMount Vernon Died atMercy Hospital inspect and revise use town as query key TurnActionEvidence and subsequent decision 1–6TextSearch; WebVisit IdentifyThe North American Reviewand pursue people born in its modern publication town, Cedar Falls. These searches do not resolve the person. 7–12ImageSearch; WebVisit; TextSearch A mural candidate suggests Mount Vernon, while hospital images suggest Mercy Medical Center. The policy remains uncertain about the town and continues the Cedar Falls line. 13–14FetchImage; CropImage Load a new mural photograph and crop its municipal emblem and lettering. The visible place name resolves the town asMount Vernon, rather than merely confirming the earlier text-led choice. 15–16TextSearch; WebVisit Switch to Mount Vernon: verify the journal’s Cornell College publication period (1964–1968), then retrieve people associated with that town. 17–19TextSearch; WebVisit Check candidate biographies. The usable biography confirms George E. Petrick’sbirth in Mount Vernonanddeath at Mercy Hospital in Cedar Rapids. T20 final answer:George E. Petrick. Visual-to-search dependency.The image determines an intermediateplace; the answer is aperson obtained through subsequent birth- and death-place retrieval. Complete action sequence 1 TS2 TS3 WV4 TS5 WV6 TS7 IS8 WV9 TS10 WV11 WV12 WV13 FI14 CI15 TS16 WV17 TS18 TS19 WV20 A Figure 23: Intermediate visual disambiguation in a constructed question. The policy initially follows Cedar Falls, a publication place of the journal. A newly retrieved mural and its town-name crop resolve Mount Vernon instead, redirecting the subsequent birthplace and hospital-death searches that identify George E. Petrick. 35 From a concert photograph to a recorder’s controls Neos / MMBC 10-turn search trajectory Question.The band in the picture released a studio album in 1967. During the recording process, how many knobs were there on the front panel of the tape recorder they used? Input concert photograph / img_1 First recorder image / T5 / img_2 Actual controls crop / T6 / img_4 Additional verification view / T9 /img_5 TurnActionEvidence and subsequent decision 1ReverseImageSearchThe returned candidates identify the concert photograph with the Beatles, supplying an entity for subsequent text search. 2–4TextSearch; WebVisit Investigate the 1967 recordings. TheSgt.Pepperpage returns a Studer J37 image reference; its text does not state the requested knob count. 5–6FetchImage; CropImage Load the recorder and crop its controls. One crop exposes the central cluster of five rotary knobs, but it does not settle the full-panel count by itself. 7WebVisitAbbey Road and equipment pages confirm the recorder model without giving an explicit count. The policy returns to image acquisition. 8–9ImageSearch; FetchImage Retrieve additional front-panel photographs. The policy counts five central knobs plus one separate right-side knob. Answer:6.Reference:6 Interleaved action sequence 1 RS2 TS3 WV4 TS5 FI6 CI7 WV8 IS9 FI10 A Case insight.A search result identifies an object, a crop narrows the visual question, and missing textual detail triggers additional image retrieval. The answer is grounded in newly loaded images rather than the concert photograph alone. Figure 24: MMBC example: a concert photograph leads to a recorder model. A partial crop is followed by additional image acquisition to verify the complete front-panel knob count. 36 Cross-checking an interior match with a local visual cue Neos / MMSearchPlus 8-turn search trajectory Question.What is the name of this hotel in Los Angeles? Input kitchenette /img_1Input room /img_2 Retrieved property photograph / T5 /img_6 Actual TV crop / T6 / img_7 TurnActionEvidence and subsequent decision 1ReverseImageSearchSearch from both input images. The returned candidates suggest Residence Inn properties, including Marina del Rey. 2WebVisitA hotel listing returns the name “Residence Inn by Marriott Marina del Rey” and its location; this is a textual candidate, not yet a pixel-level match. 3–4ImageSearch; FetchImage Retrieve property-specific kitchen photographs and compare floating shelves, cabinetry, the dishwasher, and the wall pattern withimg_1. 5–6FetchImage; CropImage Load another property image, then revisit the original room image. The crop ofimg_2exposes “BONVOY,” supporting the Marriott affiliation. 7TextSearchCross-check the property name and Los Angeles-area location before returning the final answer. Answer:Residence Inn by Marriott Marina del Rey. Reference:Residence Inn Interleaved action sequence 1 RS2 WV3 IS4 FI5 FI6 CI7 TS8 A Case insight.The policy alternates candidate discovery, page verification, image comparison, and a late crop of aninitialimage. The visual register makes earlier evidence addressable after several retrieval turns. Figure 25: MMSearchPlus example: candidate hotel interiors are checked against retrieved photographs and source pages. A late crop of an initial image exposes the Bonvoy cue, allowing earlier visual evidence to be revisited during search. 37 Comparing two regions in a newly retrieved campaign image Neos / VisBrowse 9-turn search trajectory Question.The girl in the picture shot a poster for a brand’s outerwear series in 2024. In a poster with a rainy background, what item has the same pattern as the umbrella she is holding? Input photograph /img_1 Retrieved campaign image / T5 / img_3 Actual comparison crop / T6 /img_6 TurnActionEvidence and subsequent decision 1–2ReverseImageSearch; TextSearch The returned identity candidates lead to a search for the 2024 outerwear campaign and its rainy-background imagery. 3WebVisitThe official Burberry campaign page supplies campaign evidence and image references. The requested shared pattern still requires inspection. 4–5ImageSearch; FetchImage Locate and load campaign photographs, including the rainy scene registered asimg_3. 6CropImageCrop umbrella and scarf regions fromimg_3. Both exhibit the same red check pattern in the recorded visual observation. 7–8TextSearchAn additional search cross-checks the campaign; the last search returns a gateway timeout. The final answer uses evidence already collected. Answer:The scarf. Reference:scarf Interleaved action sequence 1 RS2 TS3 WV4 IS5 FI6 CI7 TS8 TS9 A Case insight.The target relation is between two regions of aretrievedimage. Entity-level retrieval identifies the relevant campaign, while selective visual inspection grounds the comparison needed for the answer. Figure 26: VisBrowse example: campaign retrieval is followed by fetching and cropping a new image to compare the umbrella and scarf, connecting entity-level search with region-level visual reasoning. 38 Connecting two visual clues to one historical painting NeosBC-V326-turn trajectory Question.The city where the tram in Figure 1 operates is the birthplace of a Prussian Field Marshal. This marshal turned the tide of a war. Figure 2 shows a factory of a certain vehicle. The artist who painted this picture also created a work depicting the meeting between this marshal and another marshal after the war. In this meeting painting, which direction are the horses of the two men standing in the center of the picture facing? Initial tram photographimg_1 Initial shipyard paintingimg_2 Visual clue 1: the tram Reverse search identifies Rostock. Text search links the city to the birthplace of Gebhard Leberecht von Blücher. Visual clue 2: the shipyard Reverse search identifies Daniel Maclise’s Peter the Great at Deptford Dockyard, sup- plying the artist. T1–3: the constraints converge The marshal and artist point toThe Meeting of Wellington and Blücher after the Battle of Waterloo. ImageSearch now targets this painting, rather than the two initial images. Retrieved panoramaT4img_3 T4: acquire pixels before deciding Candidate descriptions disagree about the riders and their horses. FetchImage retrieves the panorama and several reproductions. Identifying the painting does not yet answer the spatial question. Figure 27: Long-horizon BC-V3 example, part one. Two initial images provide different entity constraints: a tram leads to a city and a marshal, while a shipyard painting identifies an artist. Their intersection determines a new painting to retrieve and inspect. 39 Verifying the spatial relation across reproductions BC-V3 turns 5–26 Retrieved central detail T6img_9 Recorded crop T7img_10 TurnActionEvidence and subsequent decision 5–8CropImage; FetchImage Inspect the central group, retrieve a clearer reproduction, and crop the mounted figures. The search moves from scene identity to the local relation between the horses. 9–13WebVisit; TextSearch; CropImage Check descriptions of the mural and revisit its central region. Pages help identify Wellington and Blücher, but do not settle the requested direction. 14–18FetchImage; CropImage; WebVisit One full-image download fails. An alternative Wikimedia reproduction loads at T15, is cropped at T16, and is compared with another print acquired at T18. 19–25ImageSearch; TextSearch; WebVisit; FetchImage Additional descriptions remain inconsistent or omit the spatial detail; some further downloads fail. The policy returns to the already acquired visual evidence. Alternative reproduction, cropped T16img_18Central riders T16img_19 T26 answer:Left—bothcentralhorsesfaceleft. Complete action sequence 1 RS2 TS3 IS4 FI5 CI6 FI7 CI8 CI9 WV10 CI11 TS12 CI13 WV14 FI 15 FI16 CI17 WV18 FI19 IS20 TS21 WV22 TS23 TS24 WV25 FI26 A Figure 28: Long-horizon BC-V3 example, part two. The 26-turn trajectory alternates between image acquisition, local crops, and source-page checks. Conflicting descriptions and unsuccessful downloads lead the policy back to previously acquired pixels to answer the spatial question. 40 Disambiguating visually similar species Neos VDR 28-turn trajectory Question.What is the official conservation ranking of the yellow flower in the foreground, and in which counties is it primarily found? Initial photographimg_1 Foreground flower T1 cropimg_2 TurnActionEvidence and subsequent decision 1–3CropImage; ReverseImageSearch; ImageSearch Isolate the yellow foreground flower. Reverse search does not identify the exact source, so the policy gathers several species-level alternatives. 4–6WebVisit; TextSearch Early taxonomy and status lookups are inconclusive. These tentative names are not yet used as a final identification. 7–12ReverseImageSearch; WebVisit; ImageSearch; CropImage; TextSearch Reuse the foreground crop as a new reverse-search query, inspect further candidates, and revisit flower morphology. A later, overly tight crop does not resolve the ambiguity. 13–16FetchImage; ImageSearch; WebVisit Acquire comparison photographs and taxon pages. The candidates differ in flower shape, distribution, and conservation status, making the species choice consequential. Earlier comparison T13img_5 C. pulchellusT17img_7 C. amabilisT18img_9 T17–18: images change the next query FetchImage brings in the lead photographs ofC. pulchellusandC. amabilis. After comparing the flower forms with the foreground crop, the policy focuses subsequent conservation searches onC. pulchellus. Figure 29: Long-horizon VDR example, part one. A foreground crop is reused for reverse search, and comparison photographs help distinguish visually similar species before the policy commits to species-specific factual retrieval. 41 From visual identification to authoritative evidence VDR turns 19–28 Observed foregroundimg_2 Selected species comparisonimg_7 Alternative comparison T17 img_8 T19: an uninformative page does not end the search The NatureServe search page returns no species-specific rank or county information. The policy switches to a targeted TextSearch for the California Rare Plant Rank rather than treating the empty page as evidence. T20–21: locate and read the conservation source TextSearch locates the California Native Plant Society pageRare Plant of the Month: Mt. Diablo fairy-lantern. WebVisit returns two answer-bearing facts forCalochortus pulchellus: California Rare Plant Rank1B.2, and endemism inAlameda and Contra Costa counties. T22–23: unsuccessful additional image acquisition The full CNPS photograph and its thumbnail both fail to download. Neither is treated as a new visual observation; the previously fetched comparison photographs remain available. T24–27: corroborate the assembled answer Further text queries revisit the county range, species description, and NatureServe status. The identification and requested facts now come from different, linked stages of the search. T28 answer:California Rare Plant Rank1B.2; primarilyAlameda and Contra Costa counties. The response also reports NatureServe G2 (Imperiled). Complete action sequence 1 CI2 RS3 IS4 WV5 TS6 TS7 RS8 WV9 IS10 WV11 CI12 TS13 FI14 IS 15 WV16 WV17 FI18 FI19 WV20 TS21 WV22 FI23 FI24 TS25 TS26 TS27 TS28 A Figure 30: Long-horizon VDR example, part two. The 28-turn trajectory connects visual identifi- cation with conservation evidence. An uninformative database page triggers another search, after which a conservation organization supplies the rank and county range. 42

Instant research discovery